-
Notifications
You must be signed in to change notification settings - Fork 383
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Address sucker_punch flaky test #4344
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 22011 Passed, 1476 Skipped, 6m 23s Total Time |
BenchmarksBenchmark execution time: 2025-02-07 16:32:19 Comparing candidate commit e02ad49 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 31 metrics, 2 unstable metrics. |
f5df59f
to
7d4066c
Compare
90a3346
to
9aa6492
Compare
6e9867e
to
d1767ee
Compare
d1767ee
to
98e5299
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4344 +/- ##
==========================================
- Coverage 97.73% 97.72% -0.02%
==========================================
Files 1347 1347
Lines 82349 82351 +2
Branches 4187 4187
==========================================
- Hits 80486 80478 -8
- Misses 1863 1873 +10 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable! Any reason for this to still be marked as draft? :)
1dff58c
to
e02ad49
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 LGTM. Ahaha one of those PRs that looks deceptively simple after all the work we poured into making it simple :D
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 LGTM. Ahaha one of those PRs that looks deceptively simple after all the work we poured into making it simple :D
What does this PR do?
This PR avoids a concurrency issue in
sucker_punch
tests by retrieving the length of the span arrays.Motivation:
Minimize our flaky tests! Per Ivo's comment below, the issue was caused by
traces.collect(&:spans)
in the originalfetch_spans
call. The rspec thread was reading the span arrays inTraceSegment#spans
at the same time as thesucker_punch
thread was mutating them. This created flakiness with occasional concurrency issues. Since thesucker_punch
tests only require span counts, we can instead specifically retrieve the array lengths, which avoids reading (and iterating) through the span arrays like before.This PR addresses: https://github.com/DataDog/ruby-guild/issues/186.
Change log entry
No.
Additional Notes:
N/A
How to test the change?
Hope to never see this bug again in CI... I tested the flakiness by running the test 1000 times in a row without failure.