[BUG] Incorrect faiss::IDSelectorBitmap size in faiss_wrapper.cpp

[ErikML]

What is the bug?
The faiss::IDSelectorBitmap is given the incorrect size. It is created as new faiss::IDSelectorBitmap(filterIdsLength, bitmap.data()). It should be created as new faiss::IDSelectorBitmap(bitsetArraySize, bitmap.data()).

In particular, this can cause faiss to incorrectly filter valid candidates when filterIdsLength < bitsetArraySize, as faiss rejects all candidates with ids larger than the IDSelectorBitmap size.

How can one reproduce the bug?
Steps to reproduce the behavior:

Create a filter where the number of elements and maxIdValue satisfies maxIdValue / (8 * sizeof(faiss::idx_t)) < filterIdsLength < maxIdValue / 8. Also make the element with the largest id the element with the highest score

This should trigger the bitmap filter and also make the bitmap size smaller than the number of entries. We don't return the highest scoring element even if faiss considers it because faiss filters it out.

What is the expected behavior?
Faiss should return the highest scoring element if it is considered.

What is your host/environment?
OS 2.11.0 with k-NN plugin 2.11.0.0.

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
No.

[heemin32]

If merged, this PR will resolve the issue. #1402

[ErikML]

It is also possible to over-read the buffer when filterIdsLength is larger than bitsetArraySize. This can allow returning datapoints that do not satisfy the filter.

[heemin32]

@ErikML, Thanks for reporting the bug. This is a great finding.

[heemin32]

Raised a PR to push the fix in 2.12 release.