-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increment 2.12.0-SNAPSHOT to 2.13.0-SNAPSHOT in BWC workflow #1505
Conversation
Signed-off-by: Varun Jain <[email protected]>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1505 +/- ##
============================================
+ Coverage 85.09% 85.11% +0.01%
- Complexity 1280 1281 +1
============================================
Files 168 168
Lines 5232 5232
Branches 495 495
============================================
+ Hits 4452 4453 +1
Misses 572 572
+ Partials 208 207 -1 ☔ View full report in Codecov by Sentry. |
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-1505-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e1a7cb7aa1a523cf75beba3446985a87682f4d2c
# Push it to GitHub
git push --set-upstream origin backport/backport-1505-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x Then, create a pull request where the |
We don't need backport PR because the version update is already done by #1470 |
* Optimize Faiss Query With Filters: Reduce iteration and memory for id filter (#1402) * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator Signed-off-by: luyuncheng <[email protected]> * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator. Use Bitmap And Batch to do id filter. and you sparse or fixed bitset do exact ANN search Signed-off-by: luyuncheng <[email protected]> * Using int64_t instead of long type for GetLongArrayElements Signed-off-by: luyuncheng <[email protected]> * Add IDSelectorJlongBitmap Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class 3. Spotless apply Signed-off-by: luyuncheng <[email protected]> * Rebase remote-tracking branch 'origin/main' into Filter Signed-off-by: luyuncheng <[email protected]> * tidy Signed-off-by: luyuncheng <[email protected]> * Add Changelog Signed-off-by: luyuncheng <[email protected]> * fix javadoc tasks Signed-off-by: luyuncheng <[email protected]> * fix bwc javadoc Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * Rebase faiss_wrapper.cpp Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector as Byte.SIZE Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For comments Signed-off-by: luyuncheng <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> * Increment 2.12.0-SNAPSHOT to 2.13.0-SNAPSHOT in BWC workflow (#1505) Signed-off-by: Varun Jain <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> Signed-off-by: Varun Jain <[email protected]> Co-authored-by: luyuncheng <[email protected]> Co-authored-by: Varun Jain <[email protected]>
* Optimize Faiss Query With Filters: Reduce iteration and memory for id filter (#1402) * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator Signed-off-by: luyuncheng <[email protected]> * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator. Use Bitmap And Batch to do id filter. and you sparse or fixed bitset do exact ANN search Signed-off-by: luyuncheng <[email protected]> * Using int64_t instead of long type for GetLongArrayElements Signed-off-by: luyuncheng <[email protected]> * Add IDSelectorJlongBitmap Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class 3. Spotless apply Signed-off-by: luyuncheng <[email protected]> * Rebase remote-tracking branch 'origin/main' into Filter Signed-off-by: luyuncheng <[email protected]> * tidy Signed-off-by: luyuncheng <[email protected]> * Add Changelog Signed-off-by: luyuncheng <[email protected]> * fix javadoc tasks Signed-off-by: luyuncheng <[email protected]> * fix bwc javadoc Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * Rebase faiss_wrapper.cpp Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector as Byte.SIZE Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For comments Signed-off-by: luyuncheng <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> * Increment 2.12.0-SNAPSHOT to 2.13.0-SNAPSHOT in BWC workflow (#1505) Signed-off-by: Varun Jain <[email protected]> * Manually install zlib for win CI (#1513) Signed-off-by: John Mazanec <[email protected]> * Upgrade faiss to 12b92e9 (#1509) Upgrades faiss to facebookresearch/faiss@12b92e9. Cleanup outdated patches. Signed-off-by: John Mazanec <[email protected]> * Disable sdc table for HNSWPQ read-only indices (#1518) Passes flag to disable sdc table for the HNSWPQ indices. This table is only used by HNSWPQ during graph creation to compare nodes already present in graph. When we call load index, the graph is read only. Hence, we wont be doing any ingestion and so the table can be disabled to save some memory. Along with this, added a unit test and a couple test helper methods for generating random data. Signed-off-by: John Mazanec <[email protected]> * Support distance type radius search for Lucene engine Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve comments Signed-off-by: Junqiu Lei <[email protected]> * Resolve comments Signed-off-by: Junqiu Lei <[email protected]> * Add RNNQueryFactory class Signed-off-by: Junqiu Lei <[email protected]> * Add javadoc Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> Signed-off-by: Varun Jain <[email protected]> Signed-off-by: John Mazanec <[email protected]> Signed-off-by: Junqiu Lei <[email protected]> Co-authored-by: luyuncheng <[email protected]> Co-authored-by: Varun Jain <[email protected]> Co-authored-by: John Mazanec <[email protected]>
…ject#1498) * Optimize Faiss Query With Filters: Reduce iteration and memory for id filter (opensearch-project#1402) * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator Signed-off-by: luyuncheng <[email protected]> * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator. Use Bitmap And Batch to do id filter. and you sparse or fixed bitset do exact ANN search Signed-off-by: luyuncheng <[email protected]> * Using int64_t instead of long type for GetLongArrayElements Signed-off-by: luyuncheng <[email protected]> * Add IDSelectorJlongBitmap Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class 3. Spotless apply Signed-off-by: luyuncheng <[email protected]> * Rebase remote-tracking branch 'origin/main' into Filter Signed-off-by: luyuncheng <[email protected]> * tidy Signed-off-by: luyuncheng <[email protected]> * Add Changelog Signed-off-by: luyuncheng <[email protected]> * fix javadoc tasks Signed-off-by: luyuncheng <[email protected]> * fix bwc javadoc Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * Rebase faiss_wrapper.cpp Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector as Byte.SIZE Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For comments Signed-off-by: luyuncheng <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> * Increment 2.12.0-SNAPSHOT to 2.13.0-SNAPSHOT in BWC workflow (opensearch-project#1505) Signed-off-by: Varun Jain <[email protected]> * Manually install zlib for win CI (opensearch-project#1513) Signed-off-by: John Mazanec <[email protected]> * Upgrade faiss to 12b92e9 (opensearch-project#1509) Upgrades faiss to facebookresearch/faiss@12b92e9. Cleanup outdated patches. Signed-off-by: John Mazanec <[email protected]> * Disable sdc table for HNSWPQ read-only indices (opensearch-project#1518) Passes flag to disable sdc table for the HNSWPQ indices. This table is only used by HNSWPQ during graph creation to compare nodes already present in graph. When we call load index, the graph is read only. Hence, we wont be doing any ingestion and so the table can be disabled to save some memory. Along with this, added a unit test and a couple test helper methods for generating random data. Signed-off-by: John Mazanec <[email protected]> * Support distance type radius search for Lucene engine Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve comments Signed-off-by: Junqiu Lei <[email protected]> * Resolve comments Signed-off-by: Junqiu Lei <[email protected]> * Add RNNQueryFactory class Signed-off-by: Junqiu Lei <[email protected]> * Add javadoc Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> Signed-off-by: Varun Jain <[email protected]> Signed-off-by: John Mazanec <[email protected]> Signed-off-by: Junqiu Lei <[email protected]> Co-authored-by: luyuncheng <[email protected]> Co-authored-by: Varun Jain <[email protected]> Co-authored-by: John Mazanec <[email protected]>
* Optimize Faiss Query With Filters: Reduce iteration and memory for id filter (opensearch-project#1402) * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator Signed-off-by: luyuncheng <[email protected]> * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator. Use Bitmap And Batch to do id filter. and you sparse or fixed bitset do exact ANN search Signed-off-by: luyuncheng <[email protected]> * Using int64_t instead of long type for GetLongArrayElements Signed-off-by: luyuncheng <[email protected]> * Add IDSelectorJlongBitmap Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class 3. Spotless apply Signed-off-by: luyuncheng <[email protected]> * Rebase remote-tracking branch 'origin/main' into Filter Signed-off-by: luyuncheng <[email protected]> * tidy Signed-off-by: luyuncheng <[email protected]> * Add Changelog Signed-off-by: luyuncheng <[email protected]> * fix javadoc tasks Signed-off-by: luyuncheng <[email protected]> * fix bwc javadoc Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * Rebase faiss_wrapper.cpp Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector as Byte.SIZE Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For comments Signed-off-by: luyuncheng <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> * Increment 2.12.0-SNAPSHOT to 2.13.0-SNAPSHOT in BWC workflow (opensearch-project#1505) Signed-off-by: Varun Jain <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> Signed-off-by: Varun Jain <[email protected]> Co-authored-by: luyuncheng <[email protected]> Co-authored-by: Varun Jain <[email protected]>
…ject#1498) * Optimize Faiss Query With Filters: Reduce iteration and memory for id filter (opensearch-project#1402) * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator Signed-off-by: luyuncheng <[email protected]> * Optimize Faiss Query With Filters. Reduce iteration copy for docid set iterator. Use Bitmap And Batch to do id filter. and you sparse or fixed bitset do exact ANN search Signed-off-by: luyuncheng <[email protected]> * Using int64_t instead of long type for GetLongArrayElements Signed-off-by: luyuncheng <[email protected]> * Add IDSelectorJlongBitmap Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class Signed-off-by: luyuncheng <[email protected]> * 1. Add IDSelectorJlongBitmap and UT for it 2. Move FilterIdsSelectorType to a util class 3. Spotless apply Signed-off-by: luyuncheng <[email protected]> * Rebase remote-tracking branch 'origin/main' into Filter Signed-off-by: luyuncheng <[email protected]> * tidy Signed-off-by: luyuncheng <[email protected]> * Add Changelog Signed-off-by: luyuncheng <[email protected]> * fix javadoc tasks Signed-off-by: luyuncheng <[email protected]> * fix bwc javadoc Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector Signed-off-by: luyuncheng <[email protected]> * Rebase faiss_wrapper.cpp Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For description Select different FilterIdsSelectorType Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector as Byte.SIZE Signed-off-by: luyuncheng <[email protected]> * UpdatedFilterIdsSelector For comments Signed-off-by: luyuncheng <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> * Increment 2.12.0-SNAPSHOT to 2.13.0-SNAPSHOT in BWC workflow (opensearch-project#1505) Signed-off-by: Varun Jain <[email protected]> * Manually install zlib for win CI (opensearch-project#1513) Signed-off-by: John Mazanec <[email protected]> * Upgrade faiss to 12b92e9 (opensearch-project#1509) Upgrades faiss to facebookresearch/faiss@12b92e9. Cleanup outdated patches. Signed-off-by: John Mazanec <[email protected]> * Disable sdc table for HNSWPQ read-only indices (opensearch-project#1518) Passes flag to disable sdc table for the HNSWPQ indices. This table is only used by HNSWPQ during graph creation to compare nodes already present in graph. When we call load index, the graph is read only. Hence, we wont be doing any ingestion and so the table can be disabled to save some memory. Along with this, added a unit test and a couple test helper methods for generating random data. Signed-off-by: John Mazanec <[email protected]> * Support distance type radius search for Lucene engine Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve comments Signed-off-by: Junqiu Lei <[email protected]> * Resolve comments Signed-off-by: Junqiu Lei <[email protected]> * Add RNNQueryFactory class Signed-off-by: Junqiu Lei <[email protected]> * Add javadoc Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> * Resolve feedback Signed-off-by: Junqiu Lei <[email protected]> --------- Signed-off-by: luyuncheng <[email protected]> Signed-off-by: Varun Jain <[email protected]> Signed-off-by: John Mazanec <[email protected]> Signed-off-by: Junqiu Lei <[email protected]> Co-authored-by: luyuncheng <[email protected]> Co-authored-by: Varun Jain <[email protected]> Co-authored-by: John Mazanec <[email protected]>
Description
Increase bwc version from 2.12.0-SNAPSHOT to 2.13.0-SNAPSHOT