Zero Downtime Ugrade

[dtruffaut]

An instance of MeiliSearch can index up to 2 Million docs.
Let's assume we have 10 Billion documents (like 10 Billion users).
We need a middleware to route requests on several nodes.
Let's assume the middleware tracks via counters the number of docs in each node (example: Cloudflare Durable Object), so it knows when one shard is full or not, and dynamically creates new nodes (1 node = 1 machine with an instance of MeiliSearch) on the fly.

But then comes the process of upgrading all the instances (Ex: MeiliSearch v1 -> MeiliSearch v2).

Currently, on every node, the update process of Meilisearch requires to :

• Log in to the node
• Stop the current instance
• Start the new instance
• Load the dump (several seconds)

Developers have 2 ways to handle this downtime:

• Suffer 404 queries, because the instance is down or has no data
• Mark the instance as being down (in an upper middleware), so queries are routed on another node

It could be improved.

Please provide a solution for a nearly zero downtime upgrade, whatever the number of nodes are.

Ideally we should not login to the shard, but simply send an API command, as the current MeiliSearch instance is up and listening.

[gmourier]

Hello @dtruffaut

Thanks for sharing that.

Today the 0 downtime upgrade is available on our cloud solution; The engine does not do this natively yet.

We are exploring/working on distributing Meilisearch, this is something we could possibly implement in a future iteration.

Thank you