🧪 Multi-search API endpoint Prototype

[gmourier]

Hello everyone, 👋

EDIT: It's now available in the v1.1 release candidate

We just released a 🧪 prototype that allows sending multiple search requests into one HTTP call, and we need your feedback.

This new endpoint aims to make two things easier as it removes the need to parallelize multiple HTTP requests on the client side:

• Search in multiple indexes (results are returned separately per index); if you need to merge results from multiple sources into one list, this is not possible yet; we will look into that in a future iteration
• Compute facets in a disjunctive way

How to get the prototype?

Using docker, use the following command:

docker run -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-multi-search-3

From source, compile Meilisearch on the prototype-multi-search-3 tag

How to use the prototype?

⚠️ Do not use the prototype in production ⚠️

POST /multi-search

Request Body

Describing a query object present in the queries array

Parameter name	Description
indexUid	The indexUid on which the query will be performed
…	All properties of a standard search request. 📚 https://docs.meilisearch.com/reference/api/search.html#body

Response Body

Describing a search response object in the results array

Parameter name	Description
indexUid	The indexUid on which the query has been performed.
…	All response properties of a standard search request. 📚 https://docs.meilisearch.com/reference/api/search.html#response

Usage Examples

curl \
  -X POST 'http://localhost:7700/multi-search' \
  -H 'Content-Type: application/json'

Multi-index Search

Request Payload

{
  "queries": [
    {
      "indexUid": "products",
      "q": "Nike",
      "limit": 1
    },
    {
      "indexUid": "brands",
      "q": "Nike",
      "limit": 1
    }
  ]
}

Response

{
	"results": [
	  {
	    "indexUid": "products",
	    "hits": [
	      {
	        "id": 20480,
	        "label": "Nike Air Max 90",
	        "description": "La Nike Air Max 90 reste fidèle au modèle de running OG avec sa semelle extérieure à motif gaufré emblématique, ses renforts cousus et ses plaques en TPU coloré classiques.",
	        "price": 149.99
	        "thumbnail": "https://image.tmdb.org/t/p/w500/7SCbUPwoB8Z7VUIA1Rn1WWwjNiT.jpg",
	      }
	    ],
	    "query": "Nike",
	    "processingTimeMs": 1,
	    "limit": 1,
	    "offset": 0,
	    "estimatedTotalHits": 17
	  },
	  {
	    "indexUid": "brands",
	    "hits": [
	      {
	        "id": 41212,
	        "name": "Nike",
	        "overview": "Né en 1972, il n'aura pas fallu plus d'une année à Nike pour s'imposer comme un acteur incontournable de la chaussure de sport.",
	        "logo": "https://image.tmdb.org/t/p/w500/2Ju5hn1ofOPeP1eRJtQWakiHuhW.jpg",
	      }
	    ],
	    "query": "Nike",
	    "processingTimeMs": 0,
	    "limit": 1,
	    "offset": 0,
	    "estimatedTotalHits": 7
	  }
	]
}

Disjunctive Facets Distribution

e.g Typing "Screen" and having selected facets

Request Payload

{
  "queries": [
    {
      "indexUid": "products",
      "q": "Screen",
      "filter": "(size = 16 OR size = 24) AND resolution = 1080 AND freq = 75",
      "limit": 20
    },
    {
      "indexUid": "products",
      "q": "Screen",
      "filter": "resolution = 1080 AND freq = 75",
      "facets": [
        "size"
      ],
      "limit": 0
    },
    {
      "indexUid": "products",
      "q": "Screen",
      "filter": "(size = 16 OR size = 24) AND freq = 75",
      "facets": [
        "resolution"
      ],
      "limit": 0
    },
    {
      "indexUid": "products",
      "q": "Screen",
      "filter": "(size = 16 OR size = 24) AND resolution= 1080",
      "facets": [
        "freq"
      ],
      "limit": 0
    }
  ]
}

Response

{
  "results": [
    {
      "indexUid": "products",
      "hits": [
        {
          "id": 20480,
          "label": "..."
        }
      ],
      "query": "Screen",
      "processingTimeMs": 1,
      "limit": 20,
      "offset": 0,
      "estimatedTotalHits": 17
    },
    {
      "indexUid": "products",
      "hits": [],
      "facetsDistribution": {
        "size": {
          "16": 18,
          "24": 22,
          "32": 3
        }
      },
      "query": "Screen",
      "processingTimeMs": 1,
      "limit": 0,
      "offset": 0,
      "estimatedTotalHits": 0
    },
    {
      "indexUid": "products",
      "hits": [],
      "facetsDistribution": {
        "resolution": {
          "720p": 1,
          "1080p": 30,
          "4k": 8
        }
      },
      "query": "Screen",
      "processingTimeMs": 1,
      "limit": 0,
      "offset": 0,
      "estimatedTotalHits": 0
    },
    {
      "indexUid": "products",
      "hits": [],
      "facetsDistribution": {
        "freq": {
          "60hz": 1,
          "74hz": 21,
          "144hz": 8
        }
      },
      "query": "Screen",
      "processingTimeMs": 1,
      "limit": 0,
      "offset": 0,
      "estimatedTotalHits": 0
    }
  ]
}

Thanks to anyone being involved in making Meilisearch better.

If you want to have a direct chat with us about the prototype, we have a dedicated discord discussion. Feel free to join! ✨

[flexchar]

This is amazing! Very excited for meilisearch-instant to become multi index compatible :)

[bidoubiwa]

Hey @flexchar, it is compatible with a work-around on instant-meilisearch since yesterday 😊 See release

[flexchar]

You guys are the top stars! So excited to finally be able to sit at the discussion table and bring a strong alternative to Algolia. Ever since discovering Meilisearch in late 2021, it's been a pleasure.
If you happen to see, ability to deploy a Search CDN (aka, support for running multiple slave read only instances) is also very highly desired feature. :)

[dureuill]

Hello @flexchar 👋

We're having discussions about having a distributed Meilisearch at the moment. Read-only workers with a read-write coordinator is a solution we're discussing right now. Discussions take place on #617 and on the Discord forum dedicated to the topic. No ETA at the time though :-)

[gmourier]

Hello everyone 👋

Quick update

We released a new prototype version 🧪 prototype-multi-search-3

• /search is renamed /multi-search to avoid confusion between /search and /indexes/:indexUid/search.
• Now handles tenant tokens on the multi-search endpoint

Installation guide and Usage examples have been updated accordingly.

cc @bidoubiwa, @dureuill, @maryamsulemani97, @oluademola

[baderth]

great stuff, thank you.
I'd like to note that its no perfect for my usecase yet, but i dont know if its an edge-case or others are in need too:

I'm indexing everything I want to search in, which are instances with their attributes of interest, lets say:
Tickets (extId, subject, body)
Orders (extId, customer, picklist)
Workpackages (extId, subject, body)

Every class goes into their own index now (before multisearch i put them all in one)

Now the point/problem:
I want to fire a search against all indexes and dont want to specify their names/uids as the query would always need an update if I'm adding another class/index. (Or I'd have to do it in a way to query the available indexes and build my query against all of them for every search or cache them, not ideal in either way)

So:
POST /multi-search
{ "q": "find this in all indexes" }

should return results for every index just multi-search is doing now.

Hope my requirement is clear and I'm not the only one with it :)

[gmourier]

Hello @baderth,

Thank you for raising that point!

Or I'd have to do it in a way to query the available indexes and build my query against all of them for every search or cache them, not ideal in either way

While not ideal, is it something that you would note as critical?

One question that could help us to better envision it. Do you need to get the indexes' results separately for each searched index or aggregated within the same list?

One solution could be to allow something like that (non aggregated indexes results):

• It expands a list of search results per encountered indexes automatically

Search Query

{
    "queries": [
         {
              "indexUid": "*",
              "q": "find all this in all indexes",
              ...,
         }
     ]
}

Search Response

{
     "results": [
          {
               "indexUid": "tickets",
               ...,
               "hits": [...]
          },
          {
               "inexUid": "orders",
               ...,
               "hits": [...]
          },
          ... //other indexes
     ]
}

WDYT?

Note: This will probably not be included in this first iteration.

[baderth]

Hi @gmourier,
Yep -as I stated, the results by index like in the multi-index-search implementation would just be fine.

Critical, I dont know, I'd like to use the multi-index-search for my described use-case.
If i have to dynamically build my query, I could have also queried every index separately and deal with/combine the results, so in terms of "1 query to fulfill my usecase" its critical as all other options require more than one.

[gmourier]

Hello, everyone following this discussion 👋

We have just released the first RC (release candidate) of Meilisearch containing this new implementation!
You can test it by using

• the release assets
• the Meilisearch Docker image

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:v1.1.0-rc.0

You are welcome to communicate any feedback about this new implementation in this discussion.

If you encounter any bugs, please report them here.
Thanks in advance for your help and your involvement in Meilisearch ❤️

🎉 Official and stable release containing this change will be available on 3rd April 2023

⚠️ RC (release candidates) are not recommended for production

[gmourier]

Hey folks ✨

v1.1 has been released and features the /multi-search endpoint. It allows you to search for multiple indexes in a single HTTP request. ✨

Note: it's not possible to aggregate results coming from different indexes on a single list natively yet.

📚 https://docs.meilisearch.com/learn/what_is_meilisearch/features.html#multi-index-search
📚 https://docs.meilisearch.com/reference/api/multi_search.html#post-route