Bug in drop availability calculation (maybe?)

[jordyamc]

For some reason the program decided to prioritize a long not started drop ending on the 31 (SMITE), before a nearly finished drop ending in less than 24 hours (Diablo 4).

I'm using the low availability mode

[DevilXD]

Hello,

Please provide the dump.dat file, that can be generated by running the miner with the --dump flag. I'll need it to determine the potential cause for this behavior.

[jordyamc]

I will need to wait for another case like this, the Diablo 4 drop had only like 15 min left, it was incomplete like that for 2 days aprox, I didn't said anything because it was days until the drop end, but yesterday I saw that behavior, the program was farming another drop from 0 instead of farming the 15 minutes left of that other drop.

I reloaded the inventory from settings so it changed to Diablo 4 after a few minutes

[jordyamc]

Edit: I opened the app and I found a similar case

This is the dump file (I had to change the extension because dat is not supported by github)
dump.dat.dmp

After restarting the app I got the same behaviour, now farming another drop for 11/5 instead of the 10/30 one

[JourneyOver]

Honestly I can at least somewhat say that the drop availability calculation does feel somewhat kind of buggy a bit with how low availability behaves, as I've experienced this exact scenario in the past myself and ended up just switching back to just ending soonest instead of low availability as I didn't fully know if it was an actual bug or not really and I kind of just forgot about it until this issue cropped up.

[DevilXD]

Hmm, well, the behavior might be unexpected and "buggy" at first, yes. However, the logic for availability is quite simple, it's just "campaign end remaining time" divided by "total remaining drops time", for ex. a campaign ends in 60 minutes, the drops within have 30 minutes remaining total, so the availability is: 60/30 = 2. However, as the drops in the campaign are mined, both of these times will decrease together: 59/29 => 58/28 => ... => 33/3 => 32/2 => 31/1. This makes the availability raise: 60/30 is 2, but 32/2 is 16, and then 31/1 is 31. On the other hand, all other campaigns and their drops that aren't getting mined, will keep losing their availability as time goes on: 60/30 => 59/30 => 58/30 => ... => 33/30 => 32/30 => 31/30 => 30/30.

All of this means that, as the drops are mined, the currently mined drop will keep raising their lowest availability, while all other availabilities will decrease, until the two values "meet" at some point in the middle, at which point they'll all be mined "together", with the lowest availability switching rapidly between all campaigns/drops, sort of like this:

https://github.com/user-attachments/assets/c2d944d7-21cc-4a84-8f16-e69706eefb2f
https://github.com/user-attachments/assets/d5dba5ab-d75a-4337-97bd-790f65eea230

In reality though, the miner is not able to switch back and forth between campaigns/games/channels this fast, as the availability-based sorting order is only updated on a reload (GAMES_UPDATE state specifically). That means that the miner will stick to a single campaign (single order of campaigns, switching once the current is finished), until an hour later the availability is recalculated, and a different campaign receives more priority instead. If the miner switched away from a drop that's going to end soon, that means there's another drop in a different campaign, that'll also "end soon", relative to how much time it has remaining, and the campaign's remaining time.

I'm actually starting to doubt this mode being "better" than the simple "ending soonest", even though it's premise of giving more priority to campaigns requiring more time to finish sounds better on paper. I'm sure there's a way to "fix" the availability calculation, for ex. by giving more weight to the remaining campaign end time instead of the remaining drop time, but I'd need to do some tests to determine how to improve it.

Thoughts?

[JourneyOver]

Tbh, given that the miner has to perform the entire "reload" sequence to update the sorting order and all that (and by default, this only happens every 60 minutes), I don't think the low availability option really is a good choice currently. We could potentially increase the weighting of the remaining campaign end time, as you mentioned above, which might improve it slightly. However, ultimately, I believe low availability would really only be advantageous for campaigns with the same end date/time, which is where I think this priority mode could possibly shine a bit more.

If we had a more efficient way to doing the game switching and all that other than solely during reload, low availability mode would likely perform much better overall. As it stands right now though, it feels like ending soonest simply seems like the stronger option.

[DevilXD]

Right, I agree with pretty much everything you've said. The question remains: Where to go from here?

I personally only use the Priority list, and these extra priority modes are, well, "extra". It's a little bit quirky, but it works somewhat. Should I leave it as is, try to fix it / rework the formula a little, or just remove/disable it temporarily, until for ex. #220 could bring it more to life?

[JourneyOver]

I'd say for now, the best approach would probably be to temporarily remove or disable it until we get #220 done. Then, we can revisit how it would affect low availability mode and determine if it would actually need any improvement or not as well, as imo anything we try to fix or rework to do with low availability mode right now wouldn’t really provide much advantage and with ending soonest being the stronger option of the two right now I'd say it's probably safe enough to just temp remove/disable low availability mode.

[jordyamc]

I don't know, maybe doing something like (availability + days remaining) so even if some drops have the same availability the drops sooner to expire will be prioritized?

[DevilXD]

I can see the voices are split. One part of me wants to just rework the mode to fix it, but the other part of me sees now that the mode itself has a fatal flaw - the value it sorts with changes with time. As long as it changes, the miner will try to optimize the value across all of the campaigns first, and then keep switching back-n-forth between all campaigns, leading to the erratic behavior I've described above.

If I were to fix it, the value would have to be changed to remain constant, regardless of the passing time. That kinda goes against the idea of the mode itself though, so constant cannot be used. Instead of constant, it could be rounded to some kind of a timeframe, so that the value changes for ex. every 30 minutes or an hour - this would end up having the same effect as the system functions right now though, because of how the priorities are updated only on reload. This implies having to change the timeframe to at least something longer than one hour, like 6-12-24 hours. That means it'd only really make sense for longer campaigns, and campaigns shorter than 24 hours could use the equivalent of "ending soonest" as fallback.

I'll try to simulate some example cases over the weekend, if I can find some time for this, to see how could a rework like this work. If all else fails, the mode will probably end up being removed/disabled in the future.