Ignore HTML tags at search

[mishushakov]

Is your feature request related to a problem? Please describe.

The search should ignore HTML tags

Describe the solution you'd like

MeiliSearch should strip HTML tags when processing search requests as Algolia does

Describe alternatives you've considered

Manually stripping HTML tags and storing the resulting text in the index

[curquiza]

Hello @mishushakov! Thanks for your report and your suggestion!

[curquiza]

Might be related to meilisearch/meilisearch#955

[mishushakov]

those problems are not completely related, but i'd assume, that the solution would solve both

in my case, i'd like to search a query on HTML pages, but MeiliSearch doesn't recognise XML/HTML, which not only contains text content, but also tags, that should have been stripped

[NicolaiSchmid]

I'm still in need of this particular feature, any progress?

[gmourier]

Hello everyone! 👋

Quick update: We plan to work on a way to customize the tokenizer separators in the near term (ideally v1.2).

Would you find it valuable to list html/xml tags to be considered as separators in an index settings and thus not being indexed by the engine as searchable terms?

Is it a critical need since you can already strip the html version in a dedicated field (being listed as a searchableAttribute) and keep the html version for display (not being listed as a searchableAttribute) as a work-around? Do you have difficulties with this?

Thanks for your help!

cc @ManyTheFish

[NicolaiSchmid]

Hey Guillaume,
thanks for the reply!
If the HTML Tags (including custom-tags like <deep-link query="don't analyse this">analyse this</deep-link>) would be included as a token separator, it would be plenty for me.
Currently, though, I really need the highlighting feature for HTML strings without breaking my HTML and Meili doesn't offer that. Even if I strip out HTML before indexing the highlighting is of course off, so your suggestion, unfortunately, does not work in my case.

Best,
Nicolai

[gmourier]

Hello @NicolaiSchmid 👋 Thanks for your answer. I'm not sure to understand!

Could you give me an example showing the raw document field value containing HTML, a query string, and what you want to be highlighted on the front end? It would be superuseful!

Thanks!

[NicolaiSchmid]

Basic example:

query: random sample
indexed:

<div>random sample text</div>

highlighted:

<div><em>random</em><em>sample</em></div>

Here everything works as expected, since there are no collisions

HTML Example:

query: div
indexed:

<div>random sample text</div>

highlighted:

<div>random sample</div>

This currently doesn't work, since it would also start highlighting the div tag, resulting in something like: <<em>div</em>>... which is obviously broken HTML

Custom HTML-tags:

query: random sample
indexed:

<deep-link query="random sample">random sample text</deep-link>

highlighted:

<deep-link query="random sample"><em>random</em><em>sample</em>text</deep-link>

And in this last example, I wanted to showcase the custom elements that I want to strip. Basically the analyser should strip all tags and their attributes but preserve the children and the go recursively

[stephane-klein]

@NicolaiSchmid I have the same need as you 👍.

[RodoAgusSM]

Hello, any news about this feature? I'm in the need to search text wrapped with HTML tags. It's weird for the user to have to type <u>underline to find the world underline.
Thanks.

[macraig]

Hi @RodoAgusSM , thank you for sharing your use case. Unfortunately we don't have any news on this feature for now.

[conradfr]

If I have a text:

<a href="https://www.google.com">pizza</a>

If I search "google" it works
If I search ">pizza" it works
if I search "pizza" it doesn't work

If this expected or is there currently a setting to make it work?

[macraig]

Hi @conradfr , you can make it work by adding < and > to the list of separator tokens

[conradfr]

@macraig Thanks a lot for the tip.

[SorteKanin]

Just ran into this as well - how come < and > are not separators by default? I can't really imagine why you wouldn't want them to separate tokens.