Feature request: Missing parser type for IoT Core events

[basvandriel]

Use case

On AWS IoT, you can configure registry events, which send messages to MQTT topics on for example creation, updating or deletion of things.

This generates the underneath format.

{
    "eventType" : "THING_EVENT",
    "eventId" : "f5ae9b94-8b8e-4d8e-8c8f-b3266dd89853",
    "timestamp" : 1234567890123,
    "operation" : "CREATED|UPDATED|DELETED",
    "accountId" : "123456789012",
    "thingId" : "b604f69c-aa9a-4d4a-829e-c480e958a0b5",
    "thingName" : "MyThing",
    "versionNumber" : 1,
    "thingTypeName" : null,
    "attributes": {
                "attribute3": "value3",
                "attribute1": "value1",
                "attribute2": "value2"
    }
}

This can be useful when developing lambda functions getting triggered by these events.

@event_parser
def handle(event: IoTCRUDEvent, _: LambdaContext):
    ...

Solution/User Experience

Implement the AWSIoTCRUDEvent so it's typed.

Alternative solutions

Acknowledgment

• This feature request meets Powertools for AWS Lambda (Python) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[dreamorosi]

Hi @basvandriel, thank you for opening this issue.

As a rule of thumb, we have been adding models only for events that come from direct triggers of AWS Lambda functions.

If I understand the documentation correctly, these are events that are sent by AWS IoT Core to specific IoT Topics instead.

While I understand that you can create rules to make it so that events for these topics can end up triggering a Lambda function, I wonder if this type of event is more of an application-specific event rather than one that belongs to the library.

Before moving forward with the review of the implementation, I'd like us maintainers to decide a criteria for deciding when to include this kind of events. I'm saying this mainly because there are many service integrations with IoT Core, many of which generate specific kinds of event, and I don't know if we should add them all.

[leandrodamascena]

Hi @basvandriel! Thanks for opening this issue! I worked on a project using IoT Core a while ago and integrated it with SQS and DynamoDB as a topic target. I also added some Lambda as a target, but they were receiving generic events that have nothing in common.

I see some potential in adding support for IoT as Event Source and Parser because I see a lot of customers using IoT jobs and events in their workloads. I also see that events have several fields in common, which makes sense to have a generalist class for those fields and specialized subclasses for each event type, like: THING_EVENT, THING_TYPE_EVENT and others.

Could you tell me in your architecture how the message from the MQTT topic is being sent to AWS Lambda? I want to confirm some architectural patterns before we proceed with this.

[basvandriel]

Hi @leandrodamascena , in my case, I'm using an IOT Rule to trigger the Lambda. There I'm selecting the correct data from the MQTT topic.

[leandrodamascena]

Hi @basvandriel! Thanks for sending me more information. We're going to move forward with this implementation, but we need to make some changes to this PR to add support for all event types. I see you added a generic way to capture all events on this page, but I think we can be more intentional and create classes for specific events like:

class IOTCoreEventsBase(BaseModel): # all the common fields here
  ...

class IOCoreEventsThingEvent(IOTCoreEventsBase): # Specific fields for `THING_EVENT`
  ...

class IOCoreEventsThingTypeEvent(IOTCoreEventsBase): # Specific fields for `THING_TYPE_EVENT`
  ...

We also need to add tests to make sure we can parse those events.

Please reach out if you have any questions and we can work together to merge this PR.

[basvandriel]

I'll pick this up ASAP

[leandrodamascena]

I'll pick this up ASAP

Thanks a lot! Please let me know if you have any questions that I can help you!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.