feat: implement RangeBitmapGlobalIndex for global range-bitmap index support#199
Open
lxy-9602 wants to merge 4 commits intoalibaba:mainfrom
Open
feat: implement RangeBitmapGlobalIndex for global range-bitmap index support#199lxy-9602 wants to merge 4 commits intoalibaba:mainfrom
lxy-9602 wants to merge 4 commits intoalibaba:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Implements RangeBitmapGlobalIndex (wrapping RangeBitmapFileIndex) to add global range-bitmap index support for efficient range predicate evaluation and integration into global-index scan planning.
Changes:
- Added
RangeBitmapGlobalIndex+ factory registration for a new global indexer type. - Refactored file-index → global-index result conversion into
FileIndexReaderWrapper. - Added unit/integration tests covering range-bitmap global index queries and composition with bitmap indexes.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| test/inte/global_index_test.cpp | Adds integration tests validating batch scan behavior with range-bitmap (alone and combined with bitmap). |
| src/paimon/common/global_index/wrap/file_index_reader_wrapper.h | Adds shared ToGlobalIndexResult conversion utility used by multiple global indexers. |
| src/paimon/common/global_index/wrap/file_index_reader_wrapper_test.cpp | Adds unit tests for ToGlobalIndexResult. |
| src/paimon/common/global_index/rangebitmap/range_bitmap_global_index.h | Introduces RangeBitmapGlobalIndex / reader wrapper types. |
| src/paimon/common/global_index/rangebitmap/range_bitmap_global_index.cpp | Implements reader/writer creation for range-bitmap global index. |
| src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_factory.h | Declares factory for range-bitmap global indexer creation. |
| src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_factory.cpp | Registers range-bitmap global indexer factory and identifier. |
| src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_test.cpp | Adds unit tests for range-bitmap global index across types and predicates. |
| src/paimon/common/global_index/bitmap/bitmap_global_index.cpp | Switches bitmap global index to use shared wrapper conversion utility. |
| src/paimon/common/global_index/bitmap/bitmap_global_index.h | Removes now-duplicated conversion helper declaration. |
| src/paimon/common/global_index/bitmap/bitmap_global_index_test.cpp | Uses shared MockIndexPathFactory and removes duplicate conversion tests. |
| src/paimon/common/global_index/CMakeLists.txt | Adds range-bitmap global index sources + links file-index shared dependency. |
| src/paimon/common/file_index/rangebitmap/range_bitmap_file_index.h | Exports RangeBitmapFileIndex for shared-library visibility. |
| src/paimon/CMakeLists.txt | Registers new unit tests in the test build. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_factory.cpp
Show resolved
Hide resolved
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index.cpp
Show resolved
Hide resolved
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_test.cpp
Show resolved
Hide resolved
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_test.cpp
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #146
Implement
RangeBitmapGlobalIndexand its factory class to support range-bitmap as a global index type. This is achieved by wrappingRangeBitmapFileIndex, building on the foundational work contributed by @fafacao86 . The index enables efficient evaluation of range predicates (e.g., GreaterThan, LessThan, GreaterOrEqual, LessOrEqual) at the global index level.Furthermore, the global range-bitmap index can be integrated with the Lumina DiskANN vector index by using the range-bitmap as a pre-filter before vector retrieval. This hybrid indexes improves recall accuracy by filtering out irrelevant candidates early, while also reducing vector search latency through a smaller candidate set.
Tests
RangeBitmapGlobalIndexTest
GlobalIndexTest.TestDataEvolutionBatchScanWithRangeBitmap
GlobalIndexTest.TestDataEvolutionBatchScanWithRangeBitmapAndBitmap
API and Format
Documentation
Generative AI tooling
Generated-by: Claude-4.6-Opus