feat(compaction): support universal & force level0 compaction strategy for pk table

[lxy-9602]

Purpose

Linked issue: #93

Introduces compact strategy for PK tables, which selects input files and determines output based on configuration. This PR introduces:

• Add CompactStrategy interface.
• Implement UniversalCompaction strategy: targeting the use cases requiring lower write amplification, trading-off read amplification and space amplification.
• Implement ForceUpLevel0Compaction strategy: force compacting level 0 files.

TODO:

• Integration with RecordLevelExpire and BucketedDvMaintainer.

Tests

EarlyFullCompactionTest
OffPeakHoursTest
UniversalCompactionTest
ForceUpLevel0CompactionTest

API and Format

Compaction options in defs.h.

Documentation

[Copilot]

Pull request overview

Adds a compaction strategy framework for MergeTree PK tables, centered around a RocksDB-style “universal compaction” picker with configurable triggers and off-peak behavior, plus supporting options and utilities.

Changes:

• Introduce CompactStrategy + CompactUnit, LevelSortedRun, and implement UniversalCompaction picker logic.
• Add “early full compaction” triggers and “off-peak hours” ratio adjustment, wired via new CoreOptions / Options keys.
• Extend DateTimeUtils with local-hour helper and add/adjust unit tests around these components.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/paimon/core/mergetree/sorted_run_test.cpp	Adds coverage for SortedRun / LevelSortedRun stringification.
src/paimon/core/mergetree/sorted_run.h	Adds SortedRun::ToString().
src/paimon/core/mergetree/level_sorted_run.h	Introduces LevelSortedRun wrapper with ToString().
src/paimon/core/mergetree/compact/universal_compaction_test.cpp	Adds unit tests for universal compaction picking behavior.
src/paimon/core/mergetree/compact/universal_compaction.h	Declares UniversalCompaction strategy.
src/paimon/core/mergetree/compact/universal_compaction.cpp	Implements universal compaction selection logic.
src/paimon/core/mergetree/compact/off_peak_hours_test.cpp	Adds tests for off-peak hour ratio behavior and option creation.
src/paimon/core/mergetree/compact/off_peak_hours.h	Adds OffPeakHours helper (option-driven).
src/paimon/core/mergetree/compact/early_full_compaction_test.cpp	Adds tests for early full-compaction triggers.
src/paimon/core/mergetree/compact/early_full_compaction.h	Declares EarlyFullCompaction trigger helper.
src/paimon/core/mergetree/compact/early_full_compaction.cpp	Implements early full-compaction trigger logic.
src/paimon/core/mergetree/compact/compact_strategy.h	Introduces strategy interface + static “full compaction” helper.
src/paimon/core/core_options_test.cpp	Extends options tests for new compaction/off-peak settings.
src/paimon/core/core_options.h	Adds getters for new compaction/off-peak settings.
src/paimon/core/core_options.cpp	Parses and stores new compaction/off-peak options.
src/paimon/core/compact/compact_unit.h	Introduces CompactUnit container for selected files/output level.
src/paimon/common/utils/date_time_utils_test.cpp	Renames a test and adds coverage for local-hour helper.
src/paimon/common/utils/date_time_utils.h	Adds DateTimeUtils::GetCurrentLocalHour().
src/paimon/common/defs.cpp	Defines new option key strings.
src/paimon/CMakeLists.txt	Registers new compaction sources and tests in the build.
include/paimon/defs.h	Documents new compaction/off-peak option keys.

Comments suppressed due to low confidence (2)

include/paimon/defs.h:307

• The wording "expressed as an integer between 0 and 23, exclusive" is ambiguous (it can be read as excluding 0 and 23). Consider rephrasing to something like "an integer in [0, 23]; the end hour is exclusive" to match the intended semantics and avoid misconfiguration.

    /// "compaction.offpeak.end.hour" - The end of off-peak hours, expressed as an integer between 0
    /// and 23, exclusive. Set to -1 to disable off-peak. Default is -1.
    static const char COMPACT_OFFPEAK_END_HOUR[];

src/paimon/core/mergetree/level_sorted_run.h:21

• level_sorted_run.h includes fields_comparator.h but doesn't use anything from it. Removing the unused include will reduce compile time and avoid unnecessary header coupling.

#include "paimon/core/mergetree/sorted_run.h"
#include "paimon/core/utils/fields_comparator.h"

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[lucasfang]

+1