Skip to content

test(node): Enable additionalDependencies in integration runner #17361

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 11, 2025

Conversation

RulaKhaled
Copy link
Member

This update enhances the Node integration test runner to support per-scenario dependency overrides via a temporary folder that contains package.json.

When additionalDependencies are provided, the runner now:

  1. Creates a unique temp directory with a package.json containing the requested dependencies.
  2. Copies the ESM and CJS versions of the scenario and instrument files into the temp directory.
  3. Installs the specified dependency versions.
  4. Runs tests

ESM and CJS test modes continue to run normally using the files from the temp workspace.

Also adds:

  • Minimal test scenario for vercel AI test using ai@^5.0.0. (Adjusted expectations to match the current v5 output format)

const cjsInstrumentPath = join(cwd, `tmp_${instrumentPath.replace('.mjs', '.cjs')}`);
// If additionalDependencies are provided, we create a dedicated tmp directory that includes
// copied ESM & CJS scenario/instrument files and a nested package.json with those dependencies installed.
const useTmpDir = Boolean(options?.additionalDependencies && Object.keys(options.additionalDependencies).length > 0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, what about simply always creating a tmp dir? Would unify this a bit, I suppose...?

@RulaKhaled RulaKhaled changed the title test(node): Enable additionalDependencies in integration runner [WIP] test(node): Enable additionalDependencies in integration runner Aug 8, 2025
@RulaKhaled RulaKhaled force-pushed the support-versioning-in-runner branch from 4da02bd to 8c6f644 Compare August 8, 2025 13:55
@RulaKhaled RulaKhaled marked this pull request as ready for review August 8, 2025 14:03
@RulaKhaled RulaKhaled changed the title [WIP] test(node): Enable additionalDependencies in integration runner test(node): Enable additionalDependencies in integration runner Aug 8, 2025
if (deps.length > 0) {
// Prefer npm for temp installs to avoid Yarn engine strictness; see https://github.com/vercel/ai/issues/7777
// We rely on the generated package.json dependencies and run a plain install.
const result = spawnSync('npm', ['install', '--silent', '--no-audit', '--no-fund'], {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m: Should we run yarn add here instead? 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure, I guess maybe npm is easier actually, just wondering...

Copy link
Member

@mydea mydea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is sweet! so much better than before ❤️

@mydea
Copy link
Member

mydea commented Aug 11, 2025

Side note, maybe for a follow up: we seem to duplicate this in node-core-integration-tests as well 🤔 we should probably extract the runner into e.g. test-utils and re-use it everywhere we need it.

throw e;
}
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Test Runner File Overwrite and Cleanup Failures

The test runner has two distinct issues: First, if the instrumentPath does not end with .mjs, the CJS conversion overwrites the ESM instrument file. This occurs because the CJS filename generation logic fails to create a distinct path when the original file lacks the .mjs extension, causing both ESM and CJS versions to point to the same temporary file. Second, temporary directories created for tests are not reliably cleaned up. If npm install fails during setup, the afterAll cleanup hook, defined within a describe block, will not execute, leaving the temporary directory behind.

Fix in Cursor Fix in Web

@RulaKhaled RulaKhaled merged commit f307a22 into develop Aug 11, 2025
36 checks passed
@RulaKhaled RulaKhaled deleted the support-versioning-in-runner branch August 11, 2025 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants