EventStoreDB: ReadModel Subscription and Checkpoint

[martinsimonovski]

First of all, awesome repository, very complete with examples and all.

The question that I want to ask is: How to store the checkpoint in the Read Model and why would we go with an internal EventStoreDB checkpoint. In the current example you are storing the checkpoint in the MainSubscription which is an internal one. Right now if I delete everything that is in MartenDB, it won't be "rehydrated" if I restart the application. Or what if you add multiple Read Models, how would I force them to read from the beginning if there is already an existing checkpoint?

In the EventStoreDB documentation it says:

Storing checkpoints
The best practice for subscriptions that project events to another storage, is to store checkpoints in the same storage. Projecting an event and storing the checkpoint in one transaction allows you t achieve the exactly once event processing.

My guess is that we would use something like this, but I am not sure:

var eventStoreOptions = new EventStoreDBOptions()
    {
        UseInternalCheckpointing = false
    };
  
services.AddEventStoreDB(config, eventStoreOptions);
services.AddMarten(config, configKey: "ReadModel");
services.AddTransient<ISubscriptionCheckpointRepository, MartenDBCheckpointRepository>(); 

Maybe the answer is obvious but can you point me in the right direction or give me some example for it? 🙇

[oskardudycz]

@martinsimonovski, I apologise for the delay in answering, but it seems GH notifications suck...

Regarding your question, it's recommended to store it in the same storage as the read model if we want to make it transactional (if end storage supports that). That may help in reducing idempotency, as there won't be possible to have read model applied, but checkpoint not being stored.

My sample shows the most straightforward approach, so storing checkpoints inside the EventStoreDB. It's still possible to reset the checkpoint by soft deleting or setting truncate before on the checkpoint stream. Then no checkpoint will be available there, and the subscription after restarting will start from the beginning.

The proposed solution by you, of course, will work fine. Then instead of clearing the checkpoint stream, you may remove the Marten document that contains checkpoint data. I created an issue to add such a sample: #87.