Support for custom keywords

[louis-lau]

This has been a long time coming, finally found the motivation recently :)

This PR adds support for custom keywords (which are just flags). You can use these to represent labels in a webmail (#763), and sync those same labels with for example Thunderbird over IMAP.

Overview:

• Added keywords array to message CRUD
• Added endpoint to compute counters for a keyword, using redis and deltas in the same manner as mailbox counters
• Once that endpoint has been called and the value is in redis, keyword counters will be emitted over sse (KEYWORD_COUNTERS) for that keyword. Implemented it this way to avoid computing counters for irrelevant keywords IMAP clients might add
• Keywords also added to sse events related to message creates/updates. keywords on EXISTS, and addedKeywords and removedKeywords on FETCH
• Added seperate endpoint to compute counters for flagged messages. Given \flagged is a system flag it shouldn't be part of custom keywords, but knowing the total/unread for flagged messages is still very helpful
• Also added SSE for that (FLAGGED_COUNTER)
• No changes to IMAP or indexes, this was already fully supported

Some things:

• The current flag update code didn't allow for both deleting and adding flags in the same call, it would for example error if you set/unset both seen and deleted in the same api call. This is a MongoDB limitation, you can't use $addToSet and $pull in the same query. I switched it over to a simple aggregation pipeline that does allow for this.
• Setting draft set flagged in the db for some reason? Assuming that's wrong and fixed
• There were no tests for filter actions, so while I was there I added tests for all existing actions as well

I normally only work on modern TS codebases, so JS plus the old callback style isn't too familiar for me. Let me know if you see any obvious mistakes :)

[NickOvt]

Hello Louis!

Thank you for your PR!

I have a quick question, does this PR implement just keywords (IMAP standard) or labels similar
to those used by gmail and other email clients (not standard IMAP)?

Best regards
Nick

[louis-lau]

Just keywords, but that's enough to quite easily implement labels in your webmail. I've implemented labels in our webmail using this :)

Since labels aren't an IMAP server feature, I didn't feel like it was in scope for wildduck, but keywords are.

[louis-lau]

Ah something I forgot to say is that I of course added a keyword filter to the message search endpoint. So you can actually list messages for a keyword. That together with counters makes it usable as labels within a webmail

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 20 days.

[louis-lau]

Bumpiebump

[NickOvt]

Might merge next week when I finally get to it :D

[NickOvt]

Hello @louis-lau. It is showing me that this branch has conflicts and I cannot seem to fix them. Can you please review what is conflicting exactly and fix that?
Thank you!

[louis-lau]

Just rebased to fix conflicts. Haven't tested the result though, I'll do so later, or you can if you'd like in the meantime :)

[NickOvt]

@louis-lau Hello!
This branch again has conflicts :D. Please fix when you have time and after that I'll merge it

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 20 days.