Customize setting for hard limit

[jordyw-exsited]

The hard limit on queries is now set to 1000 results, and not configurable. For e-commerce websites containing a lot of products this limit can be insufficient. For example I have product category containing 1500k products, I can only show a max of 1000 products. I know 1000 is still a lot, but I need the ability to show them all. I recon there will be other, better use cases where a hard limit of 1000 doesn't fit the needs.

So what I would like to see is the possibility to override the hard limit of 1000 by a custom setting per index. When 0 is entered unlimited results will be given.

See: meilisearch/meilisearch#2436

[unknownhasher]

It is also important for us that this is solved. We are now having problems with results when using page numbering.

[curquiza]

Hello @unknownhasher can you detail what you mean by "We are now having problems with results when using page numbering."? 😄

[ManyTheFish]

Hello @unknownhasher!
We are working on improving pagination in Meilisearch and we just released a first prototype that makes creating numbered pagination way easier! 🔥
It would be super helpful if you could let us know if this helps with your issue.
To know more about the prototype and how to use it, take a look at the dedicated discussion.

[jordyw-exsited]

I think paginaton and this hard limit are 2 separate things that are very much related to each other. AFAIK the only method to provide a full pagination functionality is by using the nbHits property, which has become a very unreliable way since the hard limit got built in.

The way I "fixed" my pagination is by manually clipping the nbHits to 1000 en calculating the pagination by so. Ofcourse this is not an ideal solution.

Personally I think the setting described above would be a first step to fix what's been broken by the hard non-customizable limit. After that we could maybe think about a pagination feature on Meilisearch side.

[gmourier]

Hi @jordyw-exsited 👋

This was one of our assumptions when we introduced this hard limit, and it is clearly proving to be true with the feedback we've been getting.

We also imagine users that may want to reduce it to something less than 1000.

Although we will work on the pagination to make it more reliable in the next versions, I think we will also work on a setting to adjust this hard limit.

[ManyTheFish]

Hello @jordyw-exsited!
We are working on improving pagination in Meilisearch and we just released a first prototype that makes creating numbered pagination way easier! 🔥
It would be super helpful if you could let us know if this helps with your issue.
To know more about the prototype and how to use it, take a look at the dedicated discussion.

[Arsapol]

I'm working on an e-commerce. After hard limit was introduced in 0.27, I have to downgrade my meilisearch version to 0.26.1 in order to use this ability.

It's great that 0.27.x has nested object support, but unlimited query results is more important to me in my use case.

[gmourier]

Thanks @Arsapol!

[unknownhasher]

Sorry for my late response. We use Meili in 2 different projects with the same problem.

First of all we have a e-commerce website and we use Meili to show products for filters like categories and search queries. We want Google to index all product detail pages for everey page. In some categories we have 20 to 25 pages with 60 products on it. So 1200 to 1500 products. With this update Google cannot find items above 1000 so they won't index or lost there linkjuice.

In another product we have database of companies in different categories. We also want Google to index alll the companies within a categorie and there are many more than 1000 companies in each category.

To have all companies and products indexable it is necessary for us to have a setting to adjust the limit.

[curquiza]

Hello everyone here!
The first RC (release candidate) of Meilisearch v0.28.0 is out, meaning a setting has been introduced to custom the searched document limit! 🔥 🔥

This default limit is now customizable using the pagination setting, accessible via the /indexes/{uid}/settings/pagination route.

For instance, to increase this limit to 20000:

curl \
  -X PATCH 'http://localhost:7700/indexes/books/settings/pagination' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "limitedTo": 2000
  }'

You can also get the value using the HTTP GET verb, or reset the value using the DELETE verb.

You can use the v0.28.0rc0 docker image or download the available binaries in the release assets.
The official v0.28.0 release is planned for 11th July!

Any feedback is more than welcome and would definitely help us improve our product 🙏 ❤️

[unknownhasher]

thanks for the good work!

[gmourier]

Following the v0.28 release, this discussion is now locked.

We encourage you to create a new thread if needed.

Thank you ✨