chore: set the default of _should_skip to false #11274
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
Datadog ReportBranch report: ✅ 0 Failed, 1048 Passed, 550 Skipped, 17m 53.28s Total duration (23m 8.72s time saved) |
quinna-h
reviewed
Nov 1, 2024
wantsui
changed the title
This was referenced Nov 12, 2024
wantsui
added
the
changelog/no-changelog
avara1986
pushed a commit
that referenced
this pull request
Nov 15, 2024
As part of #11274, the previous `until` timestamp for grpc was Oct 2024 which has already passed. Unless anyone knows about ipv6 support, this PR extends the timestamp until Jan 2025 to buy some more time. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
![datadog-datadog-prod-us1[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
wantsui
added a commit
that referenced
this pull request
Nov 20, 2024
The pyramid test flaky marker was set to Jan 2024 so it is expired by now. This PR updates the test. While looking at the failed pipelines, example: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/703862626 One of the reasons it fails is because there's a new base service name and it is no longer blank: ``` meta mismatch on '_dd.base_service': got 'tests.contrib.pyramid.app' which does not match expected ''. ``` Related to #11274 Also, this PR sets a new flaky timestamp because the pyramid sample app doesn't spin up correctly in the subprocesses, leading to failed tests. Relates to AIDM-483 ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
2 tasks
emmettbutler
pushed a commit
that referenced
this pull request
Dec 19, 2024
The `until` timestamp in the flaky decorator on Django snapshots has been expired since Jan 2024, which was uncovered by #11274 As seen in this failed run: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/742978251, if we remove it, the current failure is on: > meta mismatch on '_dd.base_service': got 'tests.contrib.django' which does not match expected ''. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
…st after the flaky time is up.
Co-authored-by: Quinna Halim <[email protected]>
wantsui
force-pushed
the
wantsui/fix-should-skip
branch
from
697b076 to
8c5208a
Compare
Datadog ReportBranch report: ✅ 0 Failed, 130 Passed, 1184 Skipped, 3m 24.99s Total duration (25m 54.8s time saved) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While reviewing the flaky marker in gunicorn's
test_span_schematizationindd-trace-py/tests/contrib/gunicorn/test_gunicorn.py
Line 188 in 223c261
ie:
This translates to Jan 31, 2024.
However, if you run the test for
test_span_schematizationtoday, it passes fine because it is getting skipped:The bug fix here is to set the default to false, but keep the spirit of the original
conditionif statement.If you rerun the test_span_schematization with this branch, it now doesn't get skip and the test correctly fails:
Leaving this PR in a draft state for now to see what other tests this affects.
Checklist
Reviewer Checklist