Finalize the high availability and sharding feature asap

[ThomasLohmann]

Hi everyone,

it would be great if you can finalize the high availability and sharding feature e.g. via the Raft algorithm asap. Since it's a must have for big projects. Otherwise, big projects would just reject Meilisearch in favor of e.g. Typesense which already supports it.

See links:

https://roadmap.meilisearch.com/c/24-high-availibility

https://roadmap.meilisearch.com/c/26-sharding

Best regards,
Thomas

[gmourier]

Hello @ThomasLohmann 👋

We are aware that this is a very requested feature, we want to provide the first iteration of high availability to guarantee the availability of the search in case a node is not accessible anymore.

We aim to make progress on this during Q4 2021.

[ThomasLohmann]

@gmourier thanks for your response! Actually It would be great, if you can finalize/release it in Q4 instead of just make some "progress" as you mentioned.

Regards,
Thomas

[gmourier]

We gladly accept any help on the subject of implementing a distributed consensus protocol to deliver it on time.

By progress, I mean to release the first iteration for the high availability feature. The one I mentioned above.

The team does its best to improve MeiliSearch within short release cycles. 🤓

Would a first iteration offering a fault-tolerant search solve some of your needs? I'm curious to have more details.

[ThomasLohmann]

Our use case is the following: We provide persons and businesses a Social Commerce Platform where they can post photos, videos, texts, products and services. And we want to index all this content, so that our users are able to find the content of each other via the search (Meilisearch). As you can imagine, our dataset will be quite huge and that's why we need sharding + high availability.

[liesislukas]

Any update on the progress?

[huanyang712]

Any update on the progress?

[gmourier]

Hello everyone,

We just rebooted that subject on #617 now that we have finally released Meilisearch v1.0

I'm sorry for the delay in my answer; I had nothing to share for a while. 🙇‍♂️

We have no ETA for that subject, but we hope to explore it more actively with the community's help anytime now.

First, we must define the first iteration scope, and we defined some questions hoping to get a clearer view on prior need to make the subject move.

Feel free to jump into the discussion!