fix: total deploy time is not well-accounted for#1596
Merged
Conversation
We used to have the following spans: - COMMAND/INVOKE: a single span for the entire CLI invocation (synth + the entire deploy phase). - DEPLOY: a span *per stack* that gets deployed. This does not allow easy accounting for total deploy time of all stacks (may be with or without parallelism), nor attribute time to assets which are built and deployed separately from stacks. In this PR, making the following changes: - First, as a refactor, extract out the anonymous functions that do the actual work from the huge `deploy()` function to a separate class. Unfortunately this leads to the need to copy a whole bunch of property values around, but the code becomes more clear. - This made it clear we were misprinting the "total time taken" statement, because that was printed *per stack* instead of *for the entire operation*. - Introduce the notion of "marker nodes" in the work graph, which get added for assets, before we start building the asset and after we have published it everywhere. - Marker nodes are linked to starting and ending spans. - Spans are linked to IoHost messages, which link to telemetry. We record a bunch of additional counters and timers on the global INVOKE span as well, just because they will be easier to extract and graph that way when we look at total impact. Adding to COMMAND span: - `load`: time spent loading CLI library, before calling first function - `init`: time spent running CLI initialization code, before starting user operation - `totalDeployTime`: wall clock time spent waiting for assets plus deploys (everything after synth). - `fileAsset/dockerAsset`: number of assets encountered - `buildAssetST/publishAssetST`: time spent on asset building/publishing, parallelism may end up with this higher than total CLI duration, indicated via `ST` suffix. - `totalDeployedStacks`: how many stacks deployed in this invocation - `totalDeployedResources`: how many resources deployed in this invocation (mind parallelism). New ASSET span: - `duration` represents asset duration from build to last publish finishing. - `fileAsset/dockerAsset`: `1` on one or the other to indicate asset type. ALSO in this PR: - Make timers `Disposable` so we can `using` them.
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1596 +/- ##
==========================================
+ Coverage 88.19% 88.25% +0.06%
==========================================
Files 76 76
Lines 10864 11038 +174
Branches 1503 1528 +25
==========================================
+ Hits 9581 9742 +161
- Misses 1255 1267 +12
- Partials 28 29 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
kaizencc
approved these changes
Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We used to have the following spans:
This does not allow easy accounting for total deploy time of all stacks (may be with or without parallelism), nor attribute time to assets which are built and deployed separately from stacks.
In this PR, making the following changes:
deploy()function in the CLI to a separate class. Unfortunately this leads to the need to copy a whole bunch of property values around, but the code becomes more clear.toolkit-liband I haven't carried out the same refactoring there (because no need for telemetry there...) so there is now more divergence between these code paths.We record a bunch of additional counters and timers on the global INVOKE span as well, just because they will be easier to extract and graph that way when we look at total impact.
Adding to COMMAND span:
load: time spent loading CLI library, before calling first functioninit: time spent running CLI initialization code, before starting user operationtotalDeployTime: wall clock time spent waiting for assets plus deploys (everything after synth).fileAsset/dockerAsset: number of assets encounteredbuildAssetST/publishAssetST: time spent on asset building/publishing, parallelism may end up with this higher than total CLI duration, indicated viaSTsuffix.totalDeployedStacks: how many stacks deployed in this invocationtotalDeployedResources: how many resources deployed in this invocation (mind parallelism).New ASSET span:
durationrepresents asset duration from build to last publish finishing.fileAsset/dockerAsset:1on one or the other to indicate asset type.ALSO in this PR:
Disposableso we canusingthem.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license