mailer - multiple notifications for same policy execution

[TylerMiller-TomTom]

I have a policy that is matching 77 resources, in which I want to send a notification about. The results however are chunked into one queue message with 50 resources, and one with 27. Looking through the code I saw that there was some chunking utils that have a default of 50. I was wondering if they're a config option for this, and if not then why? getting multiple emails like this every time the policy runs is something I'd like to be able to resolve, as it breaks my template into harder to manage groups. To find all storage accounts that match the policy for example, I need to now look through multiple emails. I know it's not the mailer's fault, the chunks are being pushed to the queue like this. Is there a way to overwrite or disable this chunking, either globally or on a per-policy / per action level?

[kapilt]

the issue is the underlying transports that notify uses to send to the mailer, those transports have per message size limits (some variance by cloud provider 256kb aws sqs & sns, 64kb with azure storage queues, 5+mb w/ gcp pub/sub). the notify action currently just utilizes a static default that works well across all resources per provider transport and can send multiple messages per policy execution if the resource count exceeds that message size limit.

there's a couple of options on improvement

• if this azure (?) then we could investigate using service bus which increases the size (256kb or 1mb)
• we could try dynamically sizing the notify message buffer to get best fit based on the actual resource representation as opposed to static across all resource types.
• we could introduce some sort of aggregation into mailer, while this is likely the right thing to do long term, its probably something we would want to coincide with a larger refactoring of mailer, which has been something we've kicked down the road for a bit.

[TylerMiller-TomTom]

This is indeed on Azure. Option 3 sounds more plausible, perhaps some form of code at the beginning or end of the message to signal that it's not complete, or some form of message ID that's used to merge multiple messages together once the messages have been pulled from the queue. So it seems that at the moment at least there isn't any way to affect this short of modifying cloud custodian for our purposes? We are creating our own docker image from the repository for reasons I won't get into, so there's no reason I couldn't make the build process apply a patch to the code after cloning. Thanks for the help anyways.

[kapilt]

yes this requires a code change, option 3 is a change to the separate mailer tool in the same repo which is intended as a reference. Contributing to open source rather than applying patches (forking) in your environment is generally recommended.

[TylerMiller-TomTom]

Long term contributing would be fine, but we need this change very short term, and it usually takes time for these changes to get tested. If we do go down this route I'll make sure to submit my changes for review. The company I'm for has recently started embracing Open Source so this is something we're interested in.