-
Notifications
You must be signed in to change notification settings - Fork 100
docs(examples): add SAM example for large-messages module #2319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # Powertools for AWS Lambda (Java) - Large Messages Example | ||
|
|
||
| This project contains an example of a Lambda function using the **Large Messages** module of Powertools for AWS Lambda (Java). For more information on this module, please refer to the [documentation](https://docs.powertools.aws.dev/lambda-java/utilities/large_messages/). | ||
|
|
||
| The example demonstrates an SQS listener that processes messages using the `LargeMessages` functional utility. It handles the retrieval of large payloads offloaded to S3 automatically. | ||
|
|
||
| ## Deploy the sample application | ||
|
|
||
| This sample is based on Serverless Application Model (SAM). To deploy it, check out the instructions for getting | ||
| started with SAM in [the examples directory](../README.md). | ||
|
|
||
| ## Test the application | ||
|
|
||
| Since this function is triggered by an SQS Queue, you can test it by sending a message to the queue created by the SAM template. | ||
|
|
||
| 1. **Find your Queue URL:** | ||
| Run the following command (replacing `LargeMessageExample` with the name of your deployed stack): | ||
| ```bash | ||
| aws cloudformation describe-stacks --stack-name LargeMessageExample --query "Stacks[0].Outputs[?OutputKey=='QueueURL'].OutputValue" --output text | ||
|
Comment on lines
+18
to
+19
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's not forgot to close the markdown block again. |
||
|
|
||
| 2. **Send a Test Message:** | ||
| Note: To test the actual "Large Message" functionality (payload offloading), you would typically use the SQS Extended Client in a producer application. However, you can verify the Lambda trigger with a standard message: | ||
| ```bash | ||
| aws sqs send-message --queue-url [YOUR_QUEUE_URL] --message-body '{"message": "Hello from CLI"}' | ||
|
|
||
| 3. **Verify Logs:** | ||
| Go to AWS CloudWatch Logs and check the Log Group for your function. You should see the processed message logged by the application. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <groupId>software.amazon.lambda.examples</groupId> | ||
| <artifactId>powertools-examples-large-messages</artifactId> | ||
| <version>2.8.0</version> | ||
| <name>Powertools for AWS Lambda (Java) - Examples - Large Messages</name> | ||
|
|
||
| <properties> | ||
| <maven.compiler.source>11</maven.compiler.source> | ||
| <maven.compiler.target>11</maven.compiler.target> | ||
| <log4j.version>2.21.1</log4j.version> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This maven property is not used anywhere. |
||
| <aspectj.version>1.9.21</aspectj.version> | ||
| <amazon-sqs-java-extended-client-lib.version>2.1.1</amazon-sqs-java-extended-client-lib.version> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not used anywhere. |
||
| </properties> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>software.amazon.lambda</groupId> | ||
| <artifactId>powertools-large-messages</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>com.amazonaws</groupId> | ||
| <artifactId>amazon-sqs-java-extended-client-lib</artifactId> | ||
| <version>${amazon-sqs-java-extended-client-lib.version}</version> | ||
| </dependency> | ||
|
Comment on lines
+25
to
+29
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't use this dependency in the consuming Lambda. |
||
|
|
||
| <dependency> | ||
| <groupId>com.amazonaws</groupId> | ||
| <artifactId>aws-lambda-java-events</artifactId> | ||
| <version>3.11.4</version> | ||
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>software.amazon.lambda</groupId> | ||
| <artifactId>powertools-logging-log4j</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.aspectj</groupId> | ||
| <artifactId>aspectjrt</artifactId> | ||
| <version>${aspectj.version}</version> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-deploy-plugin</artifactId> | ||
| <version>3.1.4</version> | ||
| <configuration> | ||
| <skip>true</skip> | ||
| </configuration> | ||
| </plugin> | ||
|
|
||
| <plugin> | ||
| <groupId>dev.aspectj</groupId> | ||
| <artifactId>aspectj-maven-plugin</artifactId> | ||
| <version>1.14</version> | ||
| <configuration> | ||
| <source>${maven.compiler.source}</source> | ||
| <target>${maven.compiler.target}</target> | ||
| <complianceLevel>${maven.compiler.target}</complianceLevel> | ||
| <aspectLibraries> | ||
| <aspectLibrary> | ||
| <groupId>software.amazon.lambda</groupId> | ||
| <artifactId>powertools-logging</artifactId> | ||
| </aspectLibrary> | ||
| </aspectLibraries> | ||
| </configuration> | ||
| <executions> | ||
| <execution> | ||
| <goals> | ||
| <goal>compile</goal> | ||
| </goals> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
|
|
||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-shade-plugin</artifactId> | ||
| <version>3.6.1</version> | ||
| <executions> | ||
| <execution> | ||
| <phase>package</phase> | ||
| <goals> | ||
| <goal>shade</goal> | ||
| </goals> | ||
| <configuration> | ||
| <createDependencyReducedPom>false</createDependencyReducedPom> | ||
| <transformers> | ||
| <transformer | ||
| implementation="org.apache.logging.log4j.maven.plugins.shade.transformer.Log4j2PluginCacheFileTransformer" /> | ||
| </transformers> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| <dependencies> | ||
| <dependency> | ||
| <groupId>org.apache.logging.log4j</groupId> | ||
| <artifactId>log4j-transform-maven-shade-plugin-extensions</artifactId> | ||
| <version>0.1.0</version> | ||
| </dependency> | ||
| </dependencies> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </project> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| /* | ||
| * Copyright 2023 Amazon.com, Inc. or its affiliates. | ||
| * Licensed under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| * | ||
| */ | ||
|
|
||
| package helloworld; | ||
|
|
||
| import com.amazonaws.services.lambda.runtime.Context; | ||
| import com.amazonaws.services.lambda.runtime.RequestHandler; | ||
| import com.amazonaws.services.lambda.runtime.events.SQSEvent; | ||
| import com.amazonaws.services.lambda.runtime.events.SQSEvent.SQSMessage; | ||
| import org.apache.logging.log4j.LogManager; | ||
| import org.apache.logging.log4j.Logger; | ||
| import software.amazon.lambda.powertools.largemessages.LargeMessages; | ||
|
|
||
| /** | ||
| * Example handler showing how to use LargeMessageProcessor functionally. | ||
| * This approach gives you more control than the @LargeMessage annotation. | ||
| */ | ||
| public final class App implements RequestHandler<SQSEvent, String> { | ||
|
|
||
| private static final Logger LOG = LogManager.getLogger(App.class); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Following our documentation, let's use SLF4J here instead of direct calls to Log4j2. Also, we forgot to add the https://docs.aws.amazon.com/powertools/java/latest/core/logging/ |
||
|
|
||
| @Override | ||
| public String handleRequest(final SQSEvent event, final Context context) { | ||
| LOG.info("Received event with {} records", event.getRecords().size()); | ||
|
|
||
| for (SQSMessage message : event.getRecords()) { | ||
| LargeMessages.processLargeMessage(message, (processedMessage) -> { | ||
| LOG.info("Processing message ID: {}", processedMessage.getMessageId()); | ||
| LOG.info("Processing body content: {}", processedMessage.getBody()); | ||
| return "Processed"; | ||
| }); | ||
| } | ||
|
|
||
| return "SUCCESS"; | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <Configuration> | ||
| <Appenders> | ||
| <Console name="JsonAppender" target="SYSTEM_OUT"> | ||
| <JsonTemplateLayout eventTemplateUri="classpath:LambdaJsonLayout.json" /> | ||
| </Console> | ||
| </Appenders> | ||
| <Loggers> | ||
| <Logger name="JsonLogger" level="INFO" additivity="false"> | ||
| <AppenderRef ref="JsonAppender" /> | ||
| </Logger> | ||
| <Root level="info"> | ||
| <AppenderRef ref="JsonAppender" /> | ||
| </Root> | ||
| </Loggers> | ||
| </Configuration> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| AWSTemplateFormatVersion: "2010-09-09" | ||
| Transform: AWS::Serverless-2016-10-31 | ||
| Description: > | ||
| Large Message Handling Example using SQS and S3 offloading | ||
|
|
||
| Globals: | ||
| Function: | ||
| Timeout: 30 | ||
| Runtime: java11 | ||
| MemorySize: 512 | ||
| Tracing: Active | ||
| Environment: | ||
| Variables: | ||
| JAVA_TOOL_OPTIONS: "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" | ||
| POWERTOOLS_LOG_LEVEL: INFO | ||
| POWERTOOLS_SERVICE_NAME: LargeMessageExample | ||
|
|
||
| Resources: | ||
| LargeMessageBucket: | ||
| Type: AWS::S3::Bucket | ||
| Properties: | ||
| LifecycleConfiguration: | ||
| Rules: | ||
| - Id: DeleteOldMessages | ||
| Status: Enabled | ||
| ExpirationInDays: 1 | ||
| MyLargeMessageQueue: | ||
| Type: AWS::SQS::Queue | ||
| Properties: | ||
| VisibilityTimeout: 30 | ||
| LargeMessageProcessingFunction: | ||
| Type: AWS::Serverless::Function | ||
| Properties: | ||
| CodeUri: . | ||
| Handler: helloworld.App::handleRequest | ||
|
|
||
| Policies: | ||
| - S3CrudPolicy: | ||
| BucketName: !Ref LargeMessageBucket | ||
| - SQSPollerPolicy: | ||
| QueueName: !GetAtt MyLargeMessageQueue.QueueName | ||
|
|
||
| Environment: | ||
| Variables: | ||
| POWERTOOLS_LARGE_MESSAGES_BUCKET: !Ref LargeMessageBucket | ||
|
|
||
| Events: | ||
| SQSEvent: | ||
| Type: SQS | ||
| Properties: | ||
| Queue: !GetAtt MyLargeMessageQueue.Arn | ||
| BatchSize: 1 | ||
| Outputs: | ||
| LargeMessageBucketName: | ||
| Description: "S3 Bucket for large payloads" | ||
| Value: !Ref LargeMessageBucket | ||
| QueueURL: | ||
| Description: "SQS Queue URL" | ||
| Value: !Ref MyLargeMessageQueue | ||
| FunctionArn: | ||
| Description: "Lambda Function ARN" | ||
| Value: !GetAtt LargeMessageProcessingFunction.Arn |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.