Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Add more details for post filters support to existing Hybrid Query page #9176

Open
1 of 4 tasks
martin-gaievski opened this issue Feb 11, 2025 · 0 comments
Open
1 of 4 tasks
Assignees
Labels
1 - Backlog - DOC Doc writer assigned to issue responsible for creating PR. search-and-query Issue: Issues related to search and query features

Comments

@martin-gaievski
Copy link
Member

What do you want to do?

  • Request a change to existing documentation
  • Add new documentation
  • Report a technical problem with the documentation
  • Other

Tell us about your request. Provide a summary of the request.
Hybrid query today supports post filter with syntax format that is similar to other queries. Unlike with other queries, filters in hybrid query may also affect final scores. We need to state that to minimize customer's confusion, preferably with some example.

This is the section in Hybrid Query page that describes how to use post filters:
https://opensearch.org/docs/latest/search-plugins/hybrid-search/#hybrid-search-with-post-filtering

This is the example we can use:

example 1
assume without filter following documents matches your queries in hybrid query.
query: [ d2: 5.0, d4: 3.0, d1: 2.0 ]
normalized scores [d2: 1.0, d4: 0.33, d1: 0.0]

example 2, with a filter
same query: [ d2: 5.0, d4: 3.0, d1: 2.0 ]
filter matches only [ d2, d4 ]
after the post filter applied this became
[ d2: 1.0, d4: 0.0 ]
you can see that for d4 score changes from 0.33 in example 1 to 0.0

example 3, two sub-queries
query1: [ d2: 5.0, d4: 3.0, d1: 2.0 ], query2: [ d1: 1.0, d5: 0.5, d4: 0.25 ]
normalized scores: [d2: 1.0, d4: 0.33, d1: 0.0], [d1: 1.0, d5: 0.33, d4: 0.0]
combined scores: [d2: 1.0, d1: 0.5, d5: 0.33, d4: 0.165]

apply same filter to that results: filter matches [ d2, d4 ]
score became query1: [ d2: 5.0, d4: 3.0], query2: [ d4: 0.25 ]
normalized scores: [ d2: 1.0, d4: 0.0], query2: [ d4: 1.0 ]
combined scores: [d2: 1.0, d4: 0.5]

for d2 score remains the same with and without the filter, but for d4 score has changed

Version: List the OpenSearch version to which this issue applies, e.g. 2.14, 2.12--2.14, or all.
This applies to all versions where hybrid query supports post filters, starting from 2.13

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

@Naarcha-AWS Naarcha-AWS added 1 - Backlog - DOC Doc writer assigned to issue responsible for creating PR. search-and-query Issue: Issues related to search and query features and removed untriaged labels Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1 - Backlog - DOC Doc writer assigned to issue responsible for creating PR. search-and-query Issue: Issues related to search and query features
Projects
None yet
Development

No branches or pull requests

3 participants