Replace all objects

[carlosbaraza]

Algolia's replaceAllObjects equivalent. Algolia creates a temporary index and when it is ready, it swaps both indexes and deletes the old one.

This is pretty handy as a cronjob to re-export everything from your source of truth and ensure that deletions and additions are synced properly.

[dureuill]

Hey @carlosbaraza 👋, would your use case be covered by index swapping? Here's a tutorial about it: https://blog.meilisearch.com/zero-downtime-index-deployment/

[carlosbaraza]

That can indeed what I need. I guess that with that information I can create my own replaceAllObjects function. However, it'd be more ideal if Meilisearch provided the sugar to do that automatically. Just call an endpoint with the data and all of those steps happened automatically in the background.

[carlosbaraza]

For anyone trying to do this in NodeJS, this is our implementation of replaceAllObjects:

const waitForAllTasks = async () => {
  const tasks = (await meiliClient.tasks.getTasks()).results.filter((t) =>
    ['enqueued', 'processing'].includes(t.status)
  );
  console.log('Meilisearch: Waiting for the following tasks', tasks);
  await meiliClient.waitForTasks(
    tasks.map((t) => t.uid),
    {
      timeOutMs: 1000 * 60 * 5,
    }
  );
};

export const replaceAllObjects = async (documents: SearchResultDocument[]) => {
  const newIndex = meiliClient.index('main_new');
  await newIndex.deleteAllDocuments();
  const settings = await meiliClient.index('main').getSettings()
  await newIndex.addDocuments(documents, { primaryKey: 'id' });
  await newIndex.updateSettings(settings);
  await waitForAllTasks();

  await meiliClient.swapIndexes([
    { indexes: ['main', 'main_new'] },
  ]);
  await waitForAllTasks();

  await meiliClient.deleteIndex('main_new');
  await waitForAllTasks();
};

[gmourier]

Thanks @carlosbaraza

cc @brunoocasali