Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update emit metric function to differentiate tests #921

Merged
merged 1 commit into from
Jan 3, 2025

Conversation

tejas2008
Copy link
Contributor

@tejas2008 tejas2008 commented Jan 3, 2025

Background

  • The Codegen Canary workflow comprises three distinct test suites:
    1. build-app-ts
    2. build-app-android
    3. build-app-swift
  • To improve error detection speed and efficiency, updates to the Canary frequency and alarms were necessary.
  • Separating metrics for each test suite facilitates easier troubleshooting during ALARM states and helps have fewer data points to consider during each Evaluation period for the workflow.

Changes Implemented

  1. Enhanced _emitCodegenCanaryMetric function:
    • Added a new test dimension to differentiate between test suites.
    • The test dimension now includes the following values:
      • ts for TypeScript tests
      • swift for Swift tests
      • android for Android tests
    • These values correspond directly to their respective TEST_SUITE in the canary_workflow.yml.

Testing

  1. Deployed local changes to a personal CodeBuild Testing account.
  2. Executed the Codegen canary workflow to validate:
    • Correct metric naming conventions
    • Proper functionality of the new dimension

Metrics Visualization

Amazon Drive Link If image not visible

Checklist

  • PR description included
  • Relevant documentation is changed or added (and PR referenced)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@tejas2008 tejas2008 requested a review from a team as a code owner January 3, 2025 00:27
@@ -500,6 +500,6 @@ function _emitCodegenCanaryMetric {
--namespace amplify-codegen-canary-e2e-tests \
--unit Count \
--value $CODEBUILD_BUILD_SUCCEEDING \
--dimensions branch=release \
--dimensions branch=release,test=$(basename "$TEST_SUITE" .test.ts | sed "s/build-app-//") \
--region us-west-2

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dummy curiosity question, do we know why the region is tight to us-west-2? Does that mean we only run codegen canary in this region?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The region us-west-2 just signifies the metrics will be visible in the us-west-2 region in AWS CloudWatch dashboard. It is not associated or related to canary running in this region, to be precise the codegen canary runs in us-east-1 and thus has no relation to the above region.

@tejas2008 tejas2008 merged commit 50dfd7e into main Jan 3, 2025
4 checks passed
@palpatim palpatim deleted the update-canary-metrics branch January 8, 2025 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants