Skip to content

Conversation

@KKould
Copy link
Member

@KKould KKould commented Dec 5, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

small_union and small_symmetric_difference functions were rewritten for in-place calculations to avoid frequent creation of HybridBitmap.

main

Benchmark Size Fastest Slowest Median Mean Samples Iters
bitmap_or_agg 100000 8.182 ms 8.462 ms 8.311 ms 8.316 ms 61 61
1000000 82.87 ms 83.85 ms 83.49 ms 83.38 ms 6 6
bitmap_union 100000 7.907 ms 8.47 ms 7.995 ms 8.066 ms 62 62
1000000 79.89 ms 80.81 ms 80.38 ms 80.34 ms 7 7
bitmap_union_disjoint 100000 7.094 ms 7.483 ms 7.209 ms 7.222 ms 70 70
1000000 84.33 ms 85.63 ms 84.88 ms 84.92 ms 6 6
bitmap_xor_agg 100000 38.36 ms 39.99 ms 38.59 ms 38.65 ms 13 13
1000000 474.3 ms 475.3 ms 474.8 ms 474.8 ms 2 2
bitmap_xor_agg_overlap 100000 6.687 ms 6.982 ms 6.813 ms 6.826 ms 74 74
1000000 68.16 ms 69.17 ms 68.61 ms 68.64 ms 8 8

pr

Benchmark Size Fastest Slowest Median Mean Samples Iters
bitmap_or_agg 100000 5.816 ms 6.177 ms 5.946 ms 5.954 ms 84 84
1000000 59.36 ms 59.97 ms 59.67 ms 59.69 ms 9 9
bitmap_union 100000 5.805 ms 6.079 ms 5.923 ms 5.926 ms 85 85
1000000 59.21 ms 59.98 ms 59.35 ms 59.44 ms 9 9
bitmap_union_disjoint 100000 6.932 ms 7.303 ms 7.093 ms 7.095 ms 71 71
1000000 81.9 ms 82.79 ms 82.4 ms 82.35 ms 7 7
bitmap_xor_agg 100000 37.38 ms 37.93 ms 37.64 ms 37.65 ms 14 14
1000000 469.9 ms 475 ms 472.5 ms 472.5 ms 2 2
bitmap_xor_agg_overlap 100000 5.163 ms 5.407 ms 5.261 ms 5.267 ms 95 95
1000000 52.85 ms 53.88 ms 53.35 ms 53.34 ms 10 10

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

…tten for in-place calculations to avoid frequent creation of HybridBitmap.
@KKould KKould changed the title perf: small_union and small_symmetric_difference feat: perf small_union and small_symmetric_difference Dec 5, 2025
@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Dec 5, 2025
@KKould KKould requested a review from Copilot December 5, 2025 08:54
@KKould KKould self-assigned this Dec 5, 2025
@KKould
Copy link
Member Author

KKould commented Dec 5, 2025

@codex review

Copilot finished reviewing on behalf of KKould December 5, 2025 08:57
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves performance of bitmap operations by optimizing small_union and small_symmetric_difference functions to perform in-place calculations instead of creating new HybridBitmap instances. Benchmark results show performance improvements of approximately 25-30% for bitmap union operations and 20-25% for XOR operations on overlapping data.

Key changes:

  • Refactored small_union and small_symmetric_difference to use in-place algorithms that work backwards through the data
  • Optimized decode_small_bitmap to use ptr::read_unaligned for better deserialization performance
  • Added comprehensive benchmark tests for various bitmap aggregate functions

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/common/io/src/bitmap.rs Refactored small_union and small_symmetric_difference to in-place operations; optimized decode_small_bitmap; updated tests; added performance comments
src/query/functions/benches/bench.rs Reorganized imports into bitmap module; added helper functions and comprehensive benchmark tests for bitmap operations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. Hooray!

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant