Keyword matching behavior

[vprelovac]

Hi team Meili and thanks for the wonderful product!

I just started using Meili and currently use it at default settings.

I noticed that when I search for gpt3 it will return results including GPT-3 which is what I want

But I also noticed that when I search for 'greece' I get 'greener' and when I search for 'conan' I get 'connection' and 'conal' which is what I dont want.

What do you recommend doing?

[curquiza]

Hello @vprelovac, thanks for your feedback

But I also noticed that when I search for 'greece' I get 'greener' and when I search for 'conan' I get 'connection' and 'conal' which is what I dont want.

Aren't the documents with greece inside considered more relevant than the documents with greener? (more relevant = at the top of the hits). The documents without typo should be at the top.
But maybe you just want to remove completely the greener document from the hits. In this case, it means you don't want the typo tolerance to be taken into account. At the moment, you cannot disable this feature from MeiliSearch, but we consider adding this possibility in the future.

[vprelovac]

documents with 'greener' should not be retrieved for 'greece' as these are false positives and completely irrelevant. in search context it is better to retrieve nothing than be irrelevant. but gpt3 and gpt-3 are the same thing. this is why getting search right is hard :) thanks for your effort.

[curquiza]

documents with 'greener' should not be retrieved for 'greece' as these are false positives and completely irrelevant.

It seems irrelevant for you, but there are not irrelevant in terms of searching 🙂 greener matches greece with 2 typos. The typo tolerance of MeiliSearch is applied.

but gpt3 and gpt-3 are the same thing.

Nope, word split is applied, not the typo tolerance, so that's not the same situation 🙂 MeiliSearch tries to split gpt3 into multiple ways (g and pt3, gp and t3...etc) and noticed that gpt and 3 is the best separation concerning your documents. Also GPT-3 contained an soft separator (-) which allows MeiliSearch to match GPT-3 and the separation gpt 3.

[vprelovac]

I am surprised by the comment that 'greener' would be somehow relevant to 'greece'? And both greece and greener are vocabulary words so no typo correction should apply.

but if you want to always do best effort retrieval, then you should also return scores so that the client can decide if the result is above certain threshold.

[curquiza]

And both greece and greener are vocabulary words so no typo correction should apply.

What do you mean? The typo tolerance is here to retrieve your document if you type greeec instead of greece (for example), it's not related to the category of the word. greener is just greece with 2 typos, that's why MeiliSearch retrieves it, but after all the documents that are more relevant 🙂

but if you want to always do best effort retrieval, then you should also return scores so that the client can decide if the result is above certain threshold.

Returning the score is indeed something we have in consideration! Thanks for this feedback!

[vprelovac]

I mean that typo tollerance should not apply to vocabularly words. In your example 'greeec' is not a vocabularly word and its ok to apply typo correction to it.

[curquiza]

We are not sure to exactly understand what you mean by vocabularly words. Are they the words in the dictionary?

Applying typo depending on whether the word is in the dictionary or not would involve MeiliSearch has a database containing all the words of all the languages. Or a database provided by the user. It's clearly an ambitious feature that would imply complexity with multilanguage handling, the conjugation of verbs (not in the dictionary), and several corner cases we don't have in mind yet because the current MeiliSearch simply does not work like this at the moment. But of course, it's not impossible.
However, it's not what we want for MeiliSearch currently. We might consider it one day, but not in a near future 🙂

[Boscop]

Just because a mistyped word is also in the dictionary doesn't mean it's not mistyped.
E.g. if you type take card when you meant to type take care, even though card is in the dictionary, you still want to see documents in search results containing take care.
The engine can't know the intention. So I think it should not take into account whether a word is in a dictionary or not.
Conversely, dictionaries are often missing words that are not typos but technical terms, I notice this every day with SwiftKey on my phone which doesn't know certain words (or names) that I then have to manually type (instead of swipe).
So if Meili took into account whether a word is in a dictionary or not, it would judge words outside the dictionary more likely as typos, which is also bad.
So please don't do this :)

[gmourier]

Locked in favor of #255