Confusing nbHits

[curquiza]

Sorry if there is already an open discussion regarding this, I have not found it.

nbHits is really confusing for multiple users who expect to get the number exact of hits (with no limit applied on it) which is not the case.
Even the documentation is not explicit regarding this: https://docs.meilisearch.com/reference/api/search.html#response
nbHits is not exhaustive by default, but only an approximation. If exhaustiveNbHits is true, then, the nbHits is exact. However the current search engine (since v0.21.0) always returns false as exhaustiveNbHits value, so nbHits is always an approximation.

One of the most problematic consequences is the users based on the nbHits value to build finished pagination. We had to create an issue to avoid this, cf meilisearch/documentation#561

Here is the old issue regarding this problem on Meilisearch's repo side: meilisearch/meilisearch#1120

---

The goal of this discussion here is to let the users explain their usage with nbHits: As a user, what is the goal you attempt to reach by using nbHits? What is your usecase?

This would definitely help the product team redesign this feature 🙏

[gmourier]

From previous discussions i've read so far, I liked the approximativeNbHits proposal for nbHits and the removal of always falsy exhaustive* fields in the search responses.

Keeping even an approximate number of hits can be useful to display this kind of information on a page.

@bidoubiwa also mentioned that matches and _matchesInfo may be unclear regarding the search results. We may consider renaming those fields though renaming nbHits may be sufficient to solve the case of users being confused when trying to implement a finite pagination.

[gmourier]

I just reread this discussion in detail and I agree with most of the points made by @bidoubiwa here.

FYI, I'm writing a document explaining our choice for v0.28 regarding the pagination which is mainly to have an API naming to better guide users by indicating in a natural way that finite pagination is not supported by Meilisearch for various reasons.

In terms of naming, it's clear that nbHits is hyper confusing and suggests that it is exhaustive. On the other hand, we have fields like exhaustiveNbHits and exhaustiveFacetsCount which are always false and therefore useless.

My proposal would be:

• Rename nbHits with the emphasis on the fact that it is approximate. Why not approximativeCandidates / approximativeHits / approximativeMatches.
• Delete exhaustiveNbHits and exhaustiveFacetsCount.
• Why not add a response field to say there are more documents to be seen. e.g. isMore as a boolean. Would it be technically possible to implement something like that @meilisearch/core-team? Mostly to avoid fetching the n+1 document via the limit @bidoubiwa mentioned as a workaround. Could also help to eliminate useless network calls in infinite scroll pagination.

We often talk about matches when we relate to matched documents where matches bring the position of the matched query terms in the given search results, why not rename matches in the API as well, I'd like to see something more precise that leaves no room for doubt. e.g. showMatchedQueryTermsPositions. matches is not very clear as it is.

[bidoubiwa]

I like approximativeCandidates as a way of differentiating nbHits and hits ! Also the naming feels intuitive, @dichotommy ?

Could also help to eliminate useless network calls in infinite scroll pagination.

This is now implemented in instant-meilisearch.

isMore as a boolean

✅ The only downside would be that by adding a new field in the search response the response object might become too big?

showMatchedQueryTermsPositions

If we decide to update matches, I like clear naming and that one is clear to me!

[gmourier]

@meilisearch/docs-team We have several proposed names to replace nbHits, the idea is to make it obvious that the number returned is not exhaustive.

Not being sure of the best term to use, we list here different proposals:

• estimatedNbHits / estimatedTotalHits
• approximatedNbHits / approximatedTotalHits

estimated seems to be easier to understand and shorter than approximated but we would like to have your opinion, also what do you think about the ed form?

Do you have a preference between Nb and Total?

[curquiza]

For me: estimatedTotalHits 😇

[bidoubiwa]

There is also totalHitsEstimation (but I like estimatedTotalHits as well)

[ManyTheFish]

For me, the estimation word is the most important information in this field and needs to be read first.
So I would prefer estimatedTotalHits over totalHitsEstimation. 🤔

[guimachiavelli]

Among the ideas so far I lean towards estimatedTotalHits. It's more natural than totalHitsEstimation and, as @ManyTheFish noted, puts a crucial bit of information first.

Do we need to use hits? I understand it's shorter, but I wonder if it wouldn't be clearer and more immediate if we used estimatedTotalMatches or estimatedTotalResults.

[bidoubiwa]

I have a soft spot for estimatedTotalMatches but currently we also have matches and _matchesInfo in our search request/return. match in this case represents the matched query words inside the fields of the returned documents: Hello <em>world</em>. World in this case is a match. So if we want to go with that one we would have reconsider in the same PR the rename/suppression of the match feature.

About estimatedTotalResults this one is tricky. results is used on all (almost) our routes returning a list of resources. Example /indexes returns { results: [{ uid: "movies"}] }. Thus we are using results for another reason.

A second issue is in the long run where we will have multiple index search. If we stay coherent with our other routes the multiple search request are going to be returned inside a results object.

Example in algolia on then Search Multiple Indices feature.

So for me, in this scenario it might become confusing: searchResponse.results[0].estimatedTotalResults where both results mean a different thing.

Quick preview:

{
  "results": [
    {
      "hits": [
        {
          "firstname": "Jimmie",
          "lastname": "Barninger",
          "objectID": "433",
        }
        ],
        // ...
        "index": "people"
    },
    {
      "hits": [
        {
          "firstname": "Jimmie",
          "lastname": "Barninger",
          "objectID": "433",
        }
        ],
        // ...
        "index": "famous_people"
    }
  ]
}

[curquiza]

I personally prefer using hits since it's a word already used in the hits array in the search response. Using matches or results would imply the users to know another new word definition in additionally to hits.

You could say we could consider changing every single hits by result or matches but

• this is a huge breaking, so the change should be really worth it, like "hit is definitely vague, the users are confused and this new word XXX is way better by definitely fixing the confusion the users previously had"
• algolia uses hits which helps the migration from them to us

[guimachiavelli]

Thanks for your comment, @curquiza. I must admit I have no particular love for hits and do find it a bit confusing, but not confusing enough to warrant a major breaking change. And, as @bidoubiwa mentioned, neither of the proposed ideas are without their own problems.

So I guess estimatedTotalHits it is?

[curquiza]

Final survey

Knowing the hits word should be kept, and the estimated/approximated word should be put first (see this conversation) here are the possibilities

😁 approximatedNbHits
🎉 approximatedTotalHits
♥️ estimatedNbHits
🚀 estimatedTotalHits

Let's vote!

[curquiza]

estimatedTotalHits it is! thanks to all the participants!

[gmourier]

Following the v0.28 release, this discussion is now locked.

We encourage you to create a new thread if needed.

Thank you ✨