[Backport 2.x] Add expand_nested_docs Parameter support to NMSLIB engine#2340

Closed

heemin32 wants to merge 416 commits intomainfrom backport/backport-2331-to-2.x

+78,505-16,213

This pull request is big! We're only showing the most recent 250 commits

Commits on Nov 27, 2023

Fix Spotless Transitive Dependency and bump aiohttp to fix CVE (#1323 ) (#1324 )

opensearch-trigger-bot[bot]
and
naveentatikonda
authored

Commits on Dec 1, 2023

Add more description about running OpenSearch on MAC M1 to developer guide (#1334 )
gaobinlong
authored

Commits on Dec 7, 2023

Allow nested knn field mapping when train model (#1318 ) (#1339 )
junqiu-lei
authored

Commits on Dec 13, 2023

[Backport 2.x] Properly designate model state for actively training models when nodes crash or leave cluster (#1348 )
ryanbogan
authored

Commits on Dec 14, 2023

Increase Lucene max dimension limit to 16,000 (#1346 ) (#1352 )

opensearch-trigger-bot[bot]
and
junqiu-lei
authored

Commits on Dec 22, 2023

Tuned default values for ef_search and ef_construction for better indexing and search performance (#1353 ) (#1362 )
navneet1v
authored

Commits on Jan 3, 2024

Commits on Jan 4, 2024

Fix script score queries not getting cached (#1373 )
junqiu-lei
authored

Commits on Jan 5, 2024

Commits on Jan 9, 2024

Throw proper exception to invalid k-NN query (#1380 ) (#1381 )
junqiu-lei
authored

Commits on Jan 12, 2024

Bump lucene codec to 99 (#1383 ) (#1386 )
naveentatikonda
authored

Commits on Jan 19, 2024

Commits on Jan 22, 2024

Few changes in integration test (#1395 ) (#1404 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on Jan 23, 2024

Commits on Jan 30, 2024

Fix flaky model tests (#1429 ) (#1430 )

opensearch-trigger-bot[bot]
and
ryanbogan
authored

Commits on Jan 31, 2024

Bump aiohttp from 3.8.6 to 3.9.2 in /benchmarks/osb (#1436 )

opensearch-trigger-bot[bot]
and
dependabot[bot]
authored

Commits on Feb 2, 2024

Pass correct value on IDSelectorBitmap initialization (#1444 ) (#1446 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on Feb 7, 2024

Add release notes for 2.12.0 release (#1463 ) (#1464 )

opensearch-trigger-bot[bot]
and
ryanbogan
authored

Commits on Feb 8, 2024

Disable warning checks in k-NN test case (#1442 ) (#1471 )

opensearch-trigger-bot[bot]
and
junqiu-lei
authored

Commits on Feb 9, 2024

Use valid data for sqfp16 test (#1474 ) (#1476 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on Feb 10, 2024

Fix a bug on sqfp16 test (#1477 ) (#1478 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on Feb 13, 2024

Update faiss commit and resolve conflict (#1443 ) (#1482 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on Feb 20, 2024

Remove unnecessary dataset (#1485 ) (#1486 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on Feb 23, 2024

Sync benchmark folder from main (#1497 )
heemin32
authored

Commits on Feb 27, 2024

Commits on Mar 4, 2024

Optimize Faiss Query With Filters: Reduce iteration and memory for id filter (#1402 ) (#1504 )

opensearch-trigger-bot[bot]
and
luyuncheng
authored

Commits on Mar 6, 2024

Commits on Mar 7, 2024

Commits on Mar 15, 2024

Commits on Mar 18, 2024

Commits on Mar 19, 2024

Commits on Mar 27, 2024

Add no new development notice to perf tool (#1578 ) (#1579 )

opensearch-trigger-bot[bot]
and
VijayanB
authored

Commits on Apr 8, 2024

Fix script to pick up correct zip (#1596 )
opensearch-trigger-bot[bot]
authored

Commits on Apr 9, 2024

Support script score when doc value is disabled (#1573 ) (#1587 )
bugmakerrrrrr
authored

Commits on Apr 10, 2024

Implemented the Streaming Feature to stream vectors from Java to JNI layer to enable creation of larger segments for vector indices (#1604 ) (#1608 )
navneet1v
authored

Commits on Apr 12, 2024

Remove unnecessary toString conversion of vector field and added some minor optimization in KNNCodec (#1613 ) (#1615 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Apr 13, 2024

[Backport 2.x] [k-NN] Add Clear Cache API (#1055 )

opensearch-trigger-bot[bot]
and
naveentatikonda
authored

Commits on Apr 22, 2024

Commits on Apr 24, 2024

Commits on Apr 25, 2024

Updates Developer guide (#1642 ) (#1657 )

opensearch-trigger-bot[bot]
and
shatejas
authored

Commits on Apr 30, 2024

Commits on May 1, 2024

Update 2.14 release notes (#1679 ) (#1680 )

opensearch-trigger-bot[bot]
and
naveentatikonda
authored

Commits on May 2, 2024

Update DEVELOPER_GUIDE.md for M Chip Mac (#1682 ) (#1683 )

opensearch-trigger-bot[bot]
and
junqiu-lei
authored

Commits on May 6, 2024

Fix build CI failure on MacOS (#1685 ) (#1689 )
opensearch-trigger-bot[bot]
authored

Commits on May 9, 2024

Block commas in model description (#1695 )
ryanbogan
authored

Commits on May 13, 2024

Add KnnCircuitBreakerException and modify exception message (#1688 ) (#1697 )

opensearch-trigger-bot[bot]
and
ryanbogan
authored

Commits on May 14, 2024

Add stats for radial search (#1684 ) (#1701 )
junqiu-lei
authored

Commits on May 15, 2024

Commits on May 16, 2024

Add luyuncheng as maintainer (#1628 ) (#1704 )
VijayanB
authored

Commits on May 17, 2024

Fix flaky test in Faiss JNI range search (#1705 ) (#1706 )

opensearch-trigger-bot[bot]
and
junqiu-lei
authored

Commits on May 21, 2024

Support script score when doc value is disabled and fix misusing DISI (#1711 )

opensearch-trigger-bot[bot]
and
bugmakerrrrrr
authored

Commits on May 23, 2024

Update threshold value after new result is added (#1715 ) (#1716 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on May 24, 2024

Use the Lucene Distance Calculation Function in Script Scoring for doing exact search (#1699 ) (#1717 )

opensearch-trigger-bot[bot]
and
ryanbogan
authored

Commits on May 30, 2024

Add validation for pq m parameter before training starts (#1713 ) (#1721 )

opensearch-trigger-bot[bot]
and
ryanbogan
authored

Commits on Jun 7, 2024

Refractor createParseField function in mappers for code reusability (#1726 ) (#1727 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Jun 11, 2024

Commits on Jun 12, 2024

Bump jinja2 from 2.11.3 to 3.1.3 (#1750 ) (#1751 )

opensearch-trigger-bot[bot]
and
naveentatikonda
authored

Commits on Jun 17, 2024

Optimize logic and tests for blocking model deletion when in use (#1757 ) (#1759 )

opensearch-trigger-bot[bot]
and
ryanbogan
authored

Commits on Jun 28, 2024

Increment version to 2.16.0-SNAPSHOT (#1749 )

opensearch-trigger-bot[bot]
and
opensearch-ci-bot
authored

Commits on Jul 4, 2024

Adds ef_search as a query parameter for Lucene, FAISS and NMSLIB (#1783 ) (#1791 )
shatejas
authored

Commits on Jul 5, 2024

Support ef_search parameter in radial search faiss engine (#1790 ) (#1794 )
junqiu-lei
authored

Commits on Jul 8, 2024

Fix linux build CI error due to action runner env upgrade node 20 (#1795 ) (#1798 )

opensearch-trigger-bot[bot]
and
junqiu-lei
authored

Commits on Jul 9, 2024

Fixing the arithmetic to find the number of vectors to stream from java to jni layer (#1804 ) (#1805 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Jul 10, 2024

Setting locale to Locale.ROOT while using String.format to ensure that UTs doesn't fail when validating the exception messages (#1812 ) (#1813 )
navneet1v
authored

Commits on Jul 11, 2024

Add jni interface to use a binary hnsw index with faiss (#1778 ) (#1817 )
heemin32
authored

Commits on Jul 12, 2024

Release memory properly for an array type (#1820 ) (#1821 )

opensearch-trigger-bot[bot]
and
heemin32
authored

Commits on Jul 16, 2024

Commits on Jul 17, 2024

Commits on Jul 18, 2024

Commits on Jul 19, 2024

Commits on Jul 22, 2024

Add binary format support with IVF method in Faiss Engine (#1784 ) (#1863 )
junqiu-lei
authored

Commits on Jul 24, 2024

Removes benchmark folder from 2.x branch to resolve CVEs from the folder (#1875 )
shatejas
authored

Commits on Jul 27, 2024

Introduce NativeEngineKNNVectorsFormat as a KNNVectorsFormat for Native engines (#1855 ) (#1887 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Jul 29, 2024

Refactor parsing of KNNQueryBuilder (#1895 )
jmazanec15
authored

Commits on Jul 31, 2024

[Backport 2.x] Refactor engine package structure (#1915 )

opensearch-trigger-bot[bot]
and
jmazanec15
authored

Commits on Aug 1, 2024

Introduced KNNVectorValues interface to iterate on different types of Vector values during indexing and search (#1897 ) (#1919 )
navneet1v
authored

Commits on Aug 5, 2024

Refactor KNNVectorFieldType from KNNVectorFieldMapper to a separate class for better readability. (#1931 ) (#1935 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Aug 8, 2024

Increment version to 2.17.0-SNAPSHOT (#1922 )

opensearch-trigger-bot[bot]
and
opensearch-ci-bot
authored

Commits on Aug 9, 2024

Commits on Aug 10, 2024

Refactor Around Mapper and Mapping (#1947 )
opensearch-trigger-bot[bot]
authored

Commits on Aug 12, 2024

Integrate Lucene Vector field with native engines to use KNNVectorFormat during segment creation (#1945 ) (#1948 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Aug 13, 2024

[Backport 2.x] Quantization Framework Implementation with 1bit and MultiBit Binary Quantizer (#1955 )
Vikasht34
authored

Commits on Aug 14, 2024

Integrate KNNVectorValues with vector ANN Search flow (#1952 ) (#1958 )
navneet1v
authored

Commits on Aug 16, 2024

Commits on Aug 19, 2024

Fixing the dimension for the vector when using Lucene field in ModelFieldMapper (#1980 ) (#1986 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Aug 21, 2024

Add support for qframework from interface (#1998 )
jmazanec15
authored

Commits on Aug 26, 2024

Commits on Aug 29, 2024

Commits on Aug 30, 2024

Commits on Sep 3, 2024

Makes NativeEngines990KnnVectorFormat as default for index version (#2020 ) (#2021 )
shatejas
authored

Commits on Sep 5, 2024

Commits on Sep 9, 2024

Commits on Sep 10, 2024

Commits on Sep 12, 2024

Adds index and query BWC tests for missing engines (#2035 ) (#2096 )
shatejas
authored

Commits on Sep 13, 2024

Add short circuit if no live docs are in segments (#2059 ) (#2104 )

opensearch-trigger-bot[bot]
and
VijayanB
authored

Commits on Sep 19, 2024

Remove benchmarks folder from k-NN repo (#2129 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Sep 20, 2024

Integrates KNN plugin with ConcurrentSearchRequestDecider interface (#2111 ) (#2126 )
shatejas
authored

Commits on Sep 23, 2024

Commits on Sep 25, 2024

Add space_type as acceptable field in train API (#2141 ) (#2143 )

opensearch-trigger-bot[bot]
and
ryanbogan
authored

Commits on Sep 26, 2024

Commits on Sep 27, 2024

Add Release Notes for 2.17.1.0 (#2154 ) (#2156 )

opensearch-trigger-bot[bot]
and
naveentatikonda
authored

Commits on Sep 28, 2024

KNNIterators should support with and without filters (#2155 ) (#2166 )

opensearch-trigger-bot[bot]
and
VijayanB
authored

Commits on Sep 30, 2024

Optimize reduceToTopK in ResultUtil by removing pre-filling and reducing peek calls (#2146 ) (#2164 )

opensearch-trigger-bot[bot]
and
junqiu-lei
authored

Commits on Oct 2, 2024

Commits on Oct 3, 2024

Commits on Oct 7, 2024

[Backport 2.x] Score Fix for Binary Quantized Vector and Setting Default value in case of shard level rescoring is disabled for oversampling factor (#2187 )

opensearch-trigger-bot[bot]
and
Vikasht34
authored

Commits on Oct 9, 2024

Commits on Oct 10, 2024

[Backport 2.x] Add support to build vector data structures greedily and perform exact search when there are no engine files (#2201 )

opensearch-trigger-bot[bot]
and
VijayanB
authored

Commits on Oct 11, 2024

Bump byte-buddy to 1.15.4 (#2204 ) (#2205 )

opensearch-trigger-bot[bot]
and
naveentatikonda
authored

Commits on Oct 15, 2024

Bump Faiss commit from 33c0ba5 to 4eecd91 (#2194 ) (#2203 )

opensearch-trigger-bot[bot]
and
naveentatikonda
authored

Commits on Oct 16, 2024

Add CompressionLevel Calculation for PQ (#2216 )

opensearch-trigger-bot[bot]
and
jmazanec15
authored

Commits on Oct 21, 2024

[Backport 2.x] Introduce a loading layer in NMSLIB. (#2185 ) (#2220 )

0ctopus13prime
and
Dooyong Kim
authored

Commits on Oct 24, 2024

[Backport 2.x] Update approximate_threshold to 15K documents (#2229 ) (#2231 )
VijayanB
authored

Commits on Oct 25, 2024

Update bwc test to update index setting (#2236 ) (#2238 )
VijayanB
authored

Commits on Nov 5, 2024

Added 'j' option to use multiple cpus to build JNI library. (#2244 ) (#2245 )

opensearch-trigger-bot[bot]
and
0ctopus13prime
authored

Commits on Nov 8, 2024

Commits on Nov 18, 2024

Upgrade bytebuddy and objenesis version to match OpenSearch core and, update github ci runner for macos (#2279 ) (#2280 )

opensearch-trigger-bot[bot]
and
VijayanB
authored

Commits on Nov 20, 2024

Fixing the bug when a segment has no vector field present for disk based vector search (#2281 ) (#2282 )

opensearch-trigger-bot[bot]
and
navneet1v
authored

Commits on Nov 22, 2024

Added null checks for fieldInfo in ExactSearcher to avoid NPE while running exact search for segments with no vector field (#2278 ) (#2285 )

opensearch-trigger-bot[bot]
and
navneet1v
authored