Queued URLs are lost when browser closes

[tjhorner]

Since the URL queue (which gets flushed automatically every 15 minutes) is stored in memory, if the user closes their browser, the queue will be lost and the URLs will never be sent to ArchiveBox. Originally, I decided that flushing the URL queue as a user closes the browser would be a good solution, but that doesn't seem to be possible.

The next best thing would be to store the URL queue in persistent storage, so the next time the user opens their browser, the URL queue can be flushed immediately and the URLs will reach ArchiveBox. However, this still depends on the user opening their browser after it closes.

I think a good solution would be to hook into other events that indicate the user may be idle, such as unfocusing the browser window, and flushing the URL queue then.

We could also make the interval at which it gets flushed configurable.

Open to ideas here!

[musabgultekin]

I'm thinking to directly sending visited urls without using queue.

Why we're using queue? As archivebox already has its own queue.