Skip to content

[skipci] Make parametric tests run easier#6242

Open
bm1549 wants to merge 1 commit intomainfrom
brian.marks/update-fine-tuning
Open

[skipci] Make parametric tests run easier#6242
bm1549 wants to merge 1 commit intomainfrom
brian.marks/update-fine-tuning

Conversation

@bm1549
Copy link
Contributor

@bm1549 bm1549 commented Feb 6, 2026

Motivation

Avoiding the sad loop when cursor fails to run parametric tests 3x each time I start a new agent

Checking how to run parametric tests for a specific library:
Running the test for node.js using the parametric scenario:
Build runner
❌ Attempt 1 failed with exit code 1
Build step failed after 1 attempts
./utils/build/build.sh: line 162: venv/bin/activate: Operation not permitted
Retrying with network access to build Docker images:
test_startup_logs_enab
diff: venv/requirements.txt: Operation not permitted
== Run build script (attempt 1 on 1) with timeout 600s ==
==================================
build images for system tests

TEST_LIBRARY:      nodejs
WEBLOG_VARIANT:    express4
BUILD_IMAGES:      runner
EXTRA_DOCKER_ARGS: 

-----------------------
Build runner
❌ Attempt 1 failed with exit code 1
Build step failed after 1 attempts
./utils/build/build.sh: line 162: venv/bin/activate: Operation not permitted
Checking the build script and venv setup:
-rw-r--r-- 1 brian.marks 2112 Feb 12  2025 venv/bin/activate
Trying with full permissions to bypass sandbox restrictions:

.                                                                        [100%]
- generated xml file: /Users/brian.marks/go/src/github.com/DataDog/system-tests/logs_parametric/reportJunit.xml -
============================== 1 passed in 10.72s ==============================
Test passed.

Changes

Added a one-liner as suggested by Cursor to fix the problem

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@bm1549 bm1549 requested a review from a team as a code owner February 6, 2026 23:45
@bm1549 bm1549 added the ai-generated The pull request includes a significant amount of AI-generated code label Feb 6, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

CODEOWNERS have been resolved as:

.cursor/rules/fine-tuning-guidance.mdc                                  @DataDog/system-tests-core

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-generated The pull request includes a significant amount of AI-generated code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant