Skip to content

feat: implement RangeBitmapGlobalIndex for global range-bitmap index support#199

Open
lxy-9602 wants to merge 4 commits intoalibaba:mainfrom
lxy-9602:rangebitmap-global-index
Open

feat: implement RangeBitmapGlobalIndex for global range-bitmap index support#199
lxy-9602 wants to merge 4 commits intoalibaba:mainfrom
lxy-9602:rangebitmap-global-index

Conversation

@lxy-9602
Copy link
Copy Markdown
Collaborator

Purpose

Linked issue: close #146

Implement RangeBitmapGlobalIndex and its factory class to support range-bitmap as a global index type. This is achieved by wrapping RangeBitmapFileIndex, building on the foundational work contributed by @fafacao86 . The index enables efficient evaluation of range predicates (e.g., GreaterThan, LessThan, GreaterOrEqual, LessOrEqual) at the global index level.

Furthermore, the global range-bitmap index can be integrated with the Lumina DiskANN vector index by using the range-bitmap as a pre-filter before vector retrieval. This hybrid indexes improves recall accuracy by filtering out irrelevant candidates early, while also reducing vector search latency through a smaller candidate set.

Tests

RangeBitmapGlobalIndexTest
GlobalIndexTest.TestDataEvolutionBatchScanWithRangeBitmap
GlobalIndexTest.TestDataEvolutionBatchScanWithRangeBitmapAndBitmap

API and Format

Documentation

Generative AI tooling

Generated-by: Claude-4.6-Opus

@zjw1111 zjw1111 requested a review from Copilot March 26, 2026 09:58
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements RangeBitmapGlobalIndex (wrapping RangeBitmapFileIndex) to add global range-bitmap index support for efficient range predicate evaluation and integration into global-index scan planning.

Changes:

  • Added RangeBitmapGlobalIndex + factory registration for a new global indexer type.
  • Refactored file-index → global-index result conversion into FileIndexReaderWrapper.
  • Added unit/integration tests covering range-bitmap global index queries and composition with bitmap indexes.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
test/inte/global_index_test.cpp Adds integration tests validating batch scan behavior with range-bitmap (alone and combined with bitmap).
src/paimon/common/global_index/wrap/file_index_reader_wrapper.h Adds shared ToGlobalIndexResult conversion utility used by multiple global indexers.
src/paimon/common/global_index/wrap/file_index_reader_wrapper_test.cpp Adds unit tests for ToGlobalIndexResult.
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index.h Introduces RangeBitmapGlobalIndex / reader wrapper types.
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index.cpp Implements reader/writer creation for range-bitmap global index.
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_factory.h Declares factory for range-bitmap global indexer creation.
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_factory.cpp Registers range-bitmap global indexer factory and identifier.
src/paimon/common/global_index/rangebitmap/range_bitmap_global_index_test.cpp Adds unit tests for range-bitmap global index across types and predicates.
src/paimon/common/global_index/bitmap/bitmap_global_index.cpp Switches bitmap global index to use shared wrapper conversion utility.
src/paimon/common/global_index/bitmap/bitmap_global_index.h Removes now-duplicated conversion helper declaration.
src/paimon/common/global_index/bitmap/bitmap_global_index_test.cpp Uses shared MockIndexPathFactory and removes duplicate conversion tests.
src/paimon/common/global_index/CMakeLists.txt Adds range-bitmap global index sources + links file-index shared dependency.
src/paimon/common/file_index/rangebitmap/range_bitmap_file_index.h Exports RangeBitmapFileIndex for shared-library visibility.
src/paimon/CMakeLists.txt Registers new unit tests in the test build.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support RangeBitmap File Index

2 participants