[Bug] WARN fetch_envelopes_by_sequence{client=1}: imap_client::tasks: skipping invalid fetch (2028 occurrences)

[ajanvrin]

As I was sifting through thousands of mails with himalaya (great tool by the way!), I came across a weird instance where himalaya would report that my inbox only contained a single mail, whereas the ProtonMail web client reported a lot more.
Turning --debug on indicated that there were 2828 WARN fetch_envelopes_by_sequence{client=1}: imap_client::tasks: skipping invalid fetch warnings.

Context (Environment)

I'm using himalaya on a Windows laptop, with ProtonBridge running.
Other than the reported issue, and the occasional error/panic, it mostly works, I was able to create folders, move many mails to those folders, delete folders, etc.

Steps to Reproduce

I want to help pinpoint and troubleshoot this issue, but the logs contain tons of PII that I cannot share.
I was sadly unable to send myself a mail that would reproduce the issue.

I've taken some troubleshooting steps and have been able to eliminate some possible causes:

• Envelope format produced by ProtonMail/ProtonBridge does not seem to be the cause, as all mails load fine in ThunderBird through ProtonBridge
• Although non-ascii chars were present in the fields of a whooping 2824 out of 2828 envelopes, I was able to rule out this possible cause, as a mail containing "àéèç" in the subject were not affected
• The emails come from multiple mail providers, there is no common pattern there, they come from Proton, Gmail, O365, etc.
• Attachments are not the cause, as there are many envelopes with and without attachments within the invalid fetches.
• Various combinations of --page-size and --page did not produce much difference. To get the 2828 warnings, I used the rather aggressive --page-size 10000

I am struggling to find more useful debugging steps, short of reading the code (which I may do this weekend, but being unfamiliar with IMAP and this codebase, I will probably struggle there too)

Here is one example of an invalid fetch:

2024-12-12T07:22:26.354595Z  WARN fetch_envelopes_by_sequence{client=1}: imap_client::tasks: skipping invalid fetch fetch="* 2829 FETCH (UID 2829 FLAGS (\\Flagged \\Seen) ENVELOPE (\"Sun, 29 May 2022 11:19:52 +0200\" \"[redacted]\" ((\"[redacted]\" NIL \"[redacted]\" \"gmail.com\")) ((\"[redacted]\" NIL \"[redacted]\" \"gmail.com\")) ((\"[redacted]\" NIL \"[redacted]\" \"gmail.com\")) ((NIL NIL \"[redacted]\" \"[redacted]\")) NIL NIL NIL \"<[redacted]@mail.gmail.com>\") BODYSTRUCTURE ((\"text\" \"html\" (\"charset\" \"utf-8\") NIL NIL \"quoted-printable\" 177 3 NIL NIL NIL NIL)(\"image\" \"jpeg\" (\"filename\" \"20220529_111616.jpg\" \"name\" \"20220529_111616.jpg\") \"<1810f1c27ce83dd33723>\" NIL \"base64\" 4877872 NIL (\"attachment\" (\"filename\" \"20220529_111616.jpg\")) NIL NIL)(\"image\" \"jpeg\" (\"filename\" \"20220529_111537.jpg\" \"name\" \"20220529_111537.jpg\") \"<1810f1c27fb526ec0f04>\" NIL \"base64\" 5090040 NIL (\"attachment\" (\"filename\" \"20220529_111537.jpg\")) NIL NIL)(\"image\" \"jpeg\" (\"filename\" \"20220529_111710.jpg\" \"name\" \"20220529_111710.jpg\") \"<1810f1c27bab86ee1ad2>\" NIL \"base64\" 5711388 NIL (\"attachment\" (\"filename\" \"20220529_111710.jpg\")) NIL NIL)(\"image\" \"jpeg\" (\"filename\" \"20220529_111721.jpg\" \"name\" \"20220529_111721.jpg\") \"<1810f1c27a5ba31d1cd1>\" NIL \"base64\" 5172832 NIL (\"attachment\" (\"filename\" \"20220529_111721.jpg\")) NIL NIL) \"mixed\" (\"boundary\" \"6d7f72d5d329be9d2b25a01e202c148edc6338fbe6d7be0ff33f5b3eff336e7a\") NIL NIL NIL))\r\n"

And here is a shorter one:

2024-12-12T07:45:43.592444Z  WARN fetch_envelopes_by_sequence{client=1}: imap_client::tasks: skipping invalid fetch fetch="* 1941 FETCH (UID 1941 FLAGS (\\Seen) ENVELOPE (\"Mon, 01 Jun 2015 07:46:41 +0000\" \"2.zip\" ((\"[redacted]\" NIL \"[redacted]\" \"gmail.com\")) ((\"[redacted]\" NIL \"[redacted]\" \"gmail.com\")) ((\"[redacted]\" NIL \"[redacted]\" \"gmail.com\")) ((NIL NIL \"[redacted]\" \"gmail.com\")) NIL NIL NIL \"<[redacted]@google.com>\") BODYSTRUCTURE (\"text\" \"html\" (\"charset\" \"utf-8\") NIL NIL \"quoted-printable\" 4781 62 NIL NIL NIL NIL))\r\n"

Is there any step I can make to help pinpoint this?

Expected Behavior

I'd expect himalaya to be able to handle these 2828 mails just fine, as ThunderBird is able to.

Current Behavior

Many WARN fetch_envelopes_by_sequence{client=1}: imap_client::tasks: skipping invalid fetch warnings

[soywod]

I cannot investigate right now, but it may be related to modern-email/defects#38.