Skip to content

test(debugger): ensure span error budgets are reasonable#6224

Open
tylfin wants to merge 1 commit intomainfrom
tyler.finethy/DEBUG-3797
Open

test(debugger): ensure span error budgets are reasonable#6224
tylfin wants to merge 1 commit intomainfrom
tyler.finethy/DEBUG-3797

Conversation

@tylfin
Copy link
Member

@tylfin tylfin commented Feb 5, 2026

Motivation

Add a test to verify that span decoration probes with improper expressions (e.g., "Cannot dereference field") have their error logs rate-limited to at most once per second. Currently marked as a bug for Java since span decoration probes lack probe budgets, producing 150 error snapshots for 150 requests instead of the expected ~1 per second.

Changes

  • Added a new test method test_span_probe_expression_budgets in tests/debugger/test_debugger_probe_budgets.py to verify that error logs from span probe expression failures are rate-limited to at most 1 per second, with appropriate assertions and handling for language-specific limitations.
  • Introduced a helper method setup_span_probe_expression_budgets to configure and trigger the span probe expression budget scenario in the same test file.
  • Added a new probe configuration file probe_span_method_budgets_expression.json under tests/debugger/utils/probes/ to define a span probe with an intentionally failing expression, used for testing error logging and rate limiting.
  • Imported the bug decorator from utils in tests/debugger/test_debugger_probe_budgets.py to allow marking tests with known bugs, specifically for Java.

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@tylfin tylfin requested review from a team as code owners February 5, 2026 19:35
@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2026

CODEOWNERS have been resolved as:

tests/debugger/utils/probes/probe_span_method_budgets_expression.json   @DataDog/debugger @DataDog/system-tests-core
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
tests/debugger/test_debugger_probe_budgets.py                           @DataDog/debugger @DataDog/system-tests-core

@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from d3870b0 to fe883a9 Compare February 5, 2026 19:40
@tylfin tylfin requested review from a team as code owners February 5, 2026 19:40
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from fe883a9 to a87e67b Compare February 5, 2026 19:43
@tylfin tylfin changed the title test(debugger): span probe budgets (#DEBUG-3797) test(debugger): ensure span error budgets are reasonable Feb 5, 2026
@tylfin tylfin requested a review from jpbempel February 5, 2026 20:30
@tylfin tylfin enabled auto-merge (squash) February 5, 2026 21:56
Copy link
Collaborator

@nccatoni nccatoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for @DataDog/system-tests-core, you should also get a review from someone familiar with the feature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants