Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v20.19.0 proposal #57349

Open
wants to merge 140 commits into
base: v20.x
Choose a base branch
from
Open

v20.19.0 proposal #57349

wants to merge 140 commits into from

Conversation

marco-ippolito
Copy link
Member

@marco-ippolito marco-ippolito commented Mar 6, 2025

2025-03-13, Version 20.19.0 'Iron' (LTS), @marco-ippolito

Notable Changes

require(esm) is now enabled by default

Support for loading native ES modules using require() had been available on v20.x under the command line flag --experimental-require-module, and available by default on v22.x and v23.x. In this release, it is now no longer behind a flag on v20.x.

This feature has been tested on v23.x and v22.x, and we are looking for user feedback from v20.x to make more final tweaks before fully stabilizing it. When the Node.js instance encounters a native ES module in require() somewhere outside node_modules for the first time, it will emit an experimental warning unless require() comes from a path that contains node_modules. If there happens to be any regressions caused by this feature, users can report it to the Node.js issue tracker. Meanwhile this feature can also be disabled using --no-experimental-require-module as a workaround.

With this feature enabled, Node.js will no longer throw ERR_REQUIRE_ESM if require() is used to load a ES module. It can, however, throw ERR_REQUIRE_ASYNC_MODULE if the ES module being loaded or its dependencies contain top-level await. When the ES module is loaded successfully by require(), the returned object will either be a ES module namespace object similar to what's returned by import(), or what gets exported as "module.exports" in the ES module.

Users can check process.features.require_module to see whether require(esm) is enabled in the current Node.js instance. For packages, the "module-sync" exports condition can be used as a way to detect require(esm) support in the current Node.js instance and allow both require() and import to load the same native ES module. See the documentation for more details about this feature.

Contributed by Joyee Cheung in #55085

Module syntax detection is now enabled by default

Module syntax detection (the --experimental-detect-module flag) is now
enabled by default. Use --no-experimental-detect-module to disable it if
needed.

Syntax detection attempts to run ambiguous files as CommonJS, and if the module
fails to parse as CommonJS due to ES module syntax, Node.js tries again and runs
the file as an ES module.
Ambiguous files are those with a .js or no extension, where the nearest parent
package.json has no "type" field (either "type": "module" or
"type": "commonjs").
Syntax detection should have no performance impact on CommonJS modules, but it
incurs a slight performance penalty for ES modules; add "type": "module" to
the nearest parent package.json file to eliminate the performance cost.
A use case unlocked by this feature is the ability to use ES module syntax in
extensionless scripts with no nearby package.json.

Thanks to Geoffrey Booth for making this work on #53619.

Other Notable Changes

  • [285bb4ee14] - crypto: update root certificates to NSS 3.107 (Node.js GitHub Bot) #56566
  • [73b5c16684] - (SEMVER-MINOR) worker: add postMessageToThread (Paolo Insogna) #53682
  • [de313b2336] - (SEMVER-MINOR) module: only emit require(esm) warning under --trace-require-module (Joyee Cheung) #56194
  • [4fba01911d] - (SEMVER-MINOR) process: add process.features.require_module (Joyee Cheung) #55241
  • [df8a045afe] - (SEMVER-MINOR) module: implement the "module-sync" exports condition (Joyee Cheung) #54648
  • [f9dc1eaef5] - (SEMVER-MINOR) module: add __esModule to require()'d ESM (Joyee Cheung) #52166
Commits
  • [d84be843e3] - benchmark: add validateStream to styleText bench (Rafael Gonzaga) #56556
  • [1ee4bf9690] - build: test macos-13 on GitHub actions (Michaël Zasso) #56307
  • [1cc8d69882] - build: build v8 with -fvisibility=hidden on macOS (Joyee Cheung) #56275
  • [52f1f7e22b] - child_process: fix parsing messages with splitted length field (Maksim Gorkov) #56106
  • [5ef3c3c996] - crypto: add missing return value check (Michael Dawson) #56615
  • [285bb4ee14] - crypto: update root certificates to NSS 3.107 (Node.js GitHub Bot) #56566
  • [d4ca38fe8e] - deps: macro ENODATA is deprecated in libc++ (Cheng) #56698
  • [15214e6508] - deps: update simdutf to 6.0.3 (Node.js GitHub Bot) #56567
  • [1e44f5d84b] - deps: update simdutf to 5.7.2 (Node.js GitHub Bot) #56388
  • [b92ff7be38] - deps: update googletest to 7d76a23 (Node.js GitHub Bot) #56387
  • [e1b71a81a9] - deps: update googletest to e54519b (Node.js GitHub Bot) #56370
  • [c0d45e7f38] - deps: update simdutf to 5.7.0 (Node.js GitHub Bot) #56332
  • [d69107f5a8] - deps: update icu to 76.1 (Node.js GitHub Bot) #55551
  • [5c9a397699] - deps: V8: backport 9ab40592f697 (Lu Yahan) #56781
  • [8342233f6d] - deps: update corepack to 0.31.0 (Node.js GitHub Bot) #56795
  • [561493d35e] - deps,src: simplify base64 encoding (Daniel Lemire) #52714
  • [6207b2936c] - doc: move anatoli to emeritus (Michael Dawson) #56592
  • [b0ab483400] - doc: fix styles of the expandable TOC (Antoine du Hamel) #56755
  • [53e4dc2a82] - doc: add "Skip to content" button (Antoine du Hamel) #56750
  • [33ee4645c3] - doc: improve accessibility of expandable lists (Antoine du Hamel) #56749
  • [b514438418] - doc: add note regarding commit message trailers (Dario Piotrowicz) #56736
  • [627f2997e3] - doc: fix typo in example code for util.styleText (Robin Mehner) #56720
  • [68548dcb48] - doc: fix inconsistencies in WeakSet and WeakMap comparison details (Shreyans Pathak) #56683
  • [337cfb2549] - doc: add RafaelGSS as latest sec release stewards (Rafael Gonzaga) #56682
  • [e890c86d7b] - doc: clarify cjs/esm diff in queueMicrotask() vs process.nextTick() (Dario Piotrowicz) #56659
  • [978263923f] - doc: WeakSet and WeakMap comparison details (Shreyans Pathak) #56648
  • [aba280ccd8] - doc: mention prepare --security (Rafael Gonzaga) #56617
  • [0a009a527b] - doc: tweak info on reposts in ambassador program (Michael Dawson) #56589
  • [d2f09e2ab3] - doc: add type stripping to ambassadors program (Marco Ippolito) #56598
  • [b0b77d7fbe] - doc: improve internal documentation on built-in snapshot (Joyee Cheung) #56505
  • [4b3e7fee94] - doc: document CLI way to open the nodejs/bluesky PR (Antoine du Hamel) #56506
  • [03878b0384] - doc: update gcc-version for ubuntu-lts (Kunal Kumar) #56553
  • [acbbd7c1a6] - doc: fix parentheses in options (Tobias Nießen) #56563
  • [3fe80c30b8] - doc: include CVE to EOL lines as sec release process (Rafael Gonzaga) #56520
  • [ff8af58046] - doc: add esm examples to node:trace_events (Alfredo González) #56514
  • [27b9cfd135] - doc: add message for Ambassadors to promote (Michael Dawson) #56235
  • [020c939da1] - doc: allow request for TSC reviews via the GitHub UI (Antoine du Hamel) #56493
  • [1ef9c9a354] - doc: add example for piping ReadableStream (Gabriel Schulhof) #56415
  • [e675c3a7fc] - doc: expand description of parseArg's default (Kevin Gibbons) #54431
  • [bc756da876] - doc: use <ul> instead of <ol> in SECURITY.md (Antoine du Hamel) #56346
  • [ad59c82a49] - doc: clarify that WASM is trusted (Matteo Collina) #56345
  • [8e76cc69e5] - doc: move dual package shipping docs to separate repo (Joyee Cheung) #55444
  • [9fda8e29cd] - doc: mark --env-file-if-exists flag as experimental (Juan José) #56893
  • [9e975f1a7d] - doc: fix link and history of SourceMap sections (Antoine du Hamel) #57098
  • [64ce95b8fc] - doc: update require(ESM) history and stability status (Antoine du Hamel) #55199
  • [697a39248b] - doc: fix history of process.features (Antoine du Hamel) #54897
  • [7c38e503a3] - doc: add documentation for process.features (Marco Ippolito) #54897
  • [c85b386a39] - esm: fix jsdoc type refs to ModuleJobBase in esm/loader (Jacob Smith) #56499
  • [4813a6a66c] - esm: throw ERR_REQUIRE_ESM instead of ERR_INTERNAL_ASSERTION (Antoine du Hamel) #54868
  • [0d327c8e47] - esm: refactor get_format (Antoine du Hamel) #53872
  • [e87db6c9bc] - events: add hasEventListener util for validate (Sunghoon) #55230
  • [674b932f33] - http: don't emit error after destroy (Robert Nagy) #55457
  • [4c24ef8f71] - http2: omit server name when HTTP2 host is IP address (islandryu) #56530
  • [533afe8124] - lib: reduce amount of caught URL errors (Yagiz Nizipli) #52658
  • [34221a1d6e] - lib: allow CJS source map cache to be reclaimed (Chengzhong Wu) #51711
  • [f13589f1f9] - lib,src: iterate module requests of a module wrap in JS (Chengzhong Wu) #52058
  • [6afee9ea43] - meta: move one or more collaborators to emeritus (Node.js GitHub Bot) #56580
  • [85bb738739] - meta: add codeowners of security release document (Rafael Gonzaga) #56521
  • [48f9ca0992] - meta: move one or more collaborators to emeritus (Node.js GitHub Bot) #56342
  • [4d724121b4] - meta: move MoLow to TSC regular member (Moshe Atlow) #56276
  • [5e2dab7868] - module: fix bad require.resolve with option paths for . and .. (Dario Piotrowicz) #56735
  • [f507c05060] - module: simplify --inspect-brk handling (Joyee Cheung) #55679
  • [ed2d373e5a] - module: disable require(esm) for policy and network import (Joyee Cheung) #56927
  • [de313b2336] - (SEMVER-MINOR) module: only emit require(esm) warning under --trace-require-module (Joyee Cheung) #56194
  • [3d89e6b6fa] - module: mark evaluation rejection in require(esm) as handled (Joyee Cheung) #56122
  • [e01dd4bd4f] - module: do not warn when require(esm) comes from node_modules (Joyee Cheung) #55960
  • [011e6e0032] - module: fix error thrown from require(esm) hitting TLA repeatedly (Joyee Cheung) #55520
  • [fdf50289c6] - module: trim off internal stack frames for require(esm) warnings (Joyee Cheung) #55496
  • [8d33f78ca5] - module: allow ESM that failed to be required to be re-imported (Joyee Cheung) #55502
  • [8192dd6cf3] - module: include module information in require(esm) warning (Joyee Cheung) #55397
  • [1db210a0ec] - module: check --experimental-require-module separately from detection (Joyee Cheung) #55250
  • [cf8701c866] - module: use kNodeModulesRE to detect node_modules (Joyee Cheung) #55243
  • [dc66632261] - module: support 'module.exports' interop export in require(esm) (Guy Bedford) #54563
  • [1ac1dda9a4] - (SEMVER-MINOR) module: unflag --experimental-require-module (Joyee Cheung) #55085
  • [683c93f45f] - module: refator ESM loader for adding future synchronous hooks (Joyee Cheung) #54769
  • [df8a045afe] - (SEMVER-MINOR) module: implement the "module-sync" exports condition (Joyee Cheung) #54648
  • [249d82b686] - module: report unfinished TLA in ambiguous modules (Antoine du Hamel) #54980
  • [1925d729f9] - module: remove bogus assertion in CJS entrypoint handling with --import (Joyee Cheung) #54592
  • [d1331fccb2] - module: do not warn for typeless package.json when there isn't one (Joyee Cheung) #54045
  • [9916458b44] - (SEMVER-MINOR) module: unflag detect-module (Geoffrey Booth) #53619
  • [f9dc1eaef5] - (SEMVER-MINOR) module: add __esModule to require()'d ESM (Joyee Cheung) #52166
  • [b86f575504] - module: do not set CJS variables for Worker eval (Antoine du Hamel) #53050
  • [30ed93db12] - module: cache synchronous module jobs before linking (Joyee Cheung) #52868
  • [a03faf289d] - module: support ESM detection in the CJS loader (Joyee Cheung) #52047
  • [b07ad39bda] - module: detect ESM syntax by trying to recompile as SourceTextModule (Joyee Cheung) #52413
  • [132a5c190f] - module: eliminate performance cost of detection for cjs entry (Geoffrey Booth) #52093
  • [55a57a189f] - node-api: remove deprecated attribute from napi_module_register (Vladimir Morozov) #56162
  • [4fba01911d] - (SEMVER-MINOR) process: add process.features.require_module (Joyee Cheung) #55241
  • [c0fad18ac0] - src: add nullptr handling from X509_STORE_new() (Burkov Egor) #56700
  • [5b88d48cbb] - src: add default value for RSACipherConfig mode field (Burkov Egor) #56701
  • [e3b69e57a6] - src: fix build with GCC 15 (tjuhaszrh) #56740
  • [a7c1d8c0e8] - src: initialize FSReqWrapSync in path that uses it (Michaël Zasso) #56613
  • [c06ac66356] - src: fix undefined script name in error source (Chengzhong Wu) #56502
  • [500f3ccc66] - src: lock the thread properly in snapshot builder (Joyee Cheung) #56327
  • [cf25a5edeb] - src: drain platform tasks before creating startup snapshot (Chengzhong Wu) #56403
  • [8af1b53bb8] - src: safely remove the last line from dotenv (Shima Ryuhei) #55982
  • [bb57e909aa] - src: remove base64 from process.versions (Richard Lau) #53442
  • [b8c89a693e] - src: add --env-file-if-exists flag (Bosco Domingo) #53060
  • [9097de073a] - src: don't match after -- in Dotenv::GetPathFromArgs (Aviv Keller) #54237
  • [ececd225b6] - src: implement IsInsideNodeModules() in C++ (Joyee Cheung) #55286
  • [18593b7d3e] - src: refactor embedded entrypoint loading (Joyee Cheung) #53573
  • [d7aefc0524] - stream: fix typo in ReadableStreamBYOBReader.readIntoRequests (Mattias Buelens) #56560
  • [fe5f7bcd47] - stream: validate undefined sizeAlgorithm in WritableStream (Jason Zhang) #56067
  • [12744c1fd4] - test: reduce number of written chunks (Luigi Pinca) #56757
  • [e121d7d62c] - test: fix invalid common.mustSucceed() usage (Luigi Pinca) #56756
  • [11b82de7ed] - test: use strict mode in global setters test (Rich Trott) #56742
  • [f9d6e35c5e] - test: cleanup and simplify test-crypto-aes-wrap (James M Snell) #56748
  • [792ce98699] - test: do not use common.isMainThread (Luigi Pinca) #56768
  • [4f0cf475e0] - test: add test that uses multibyte for path and resolves modules (yamachu) #56696
  • [3bc8d273c2] - test: add missing test for env file (Jonas) #56642
  • [ad39367712] - test: enforce strict mode in test-zlib-const (Rich Trott) #56689
  • [ca79914137] - test: test-stream-compose.js doesn't need internals (Meghan Denny) #56619
  • [08bde67101] - test: add maxCount and gcOptions to gcUntil() (Joyee Cheung) #56522
  • [40a0f6f6e3] - test: mark test-worker-prof as flaky on smartos (Joyee Cheung) #56583
  • [d17bf2f62a] - test: update test-child-process-bad-stdio to use node:test (Colin Ihrig) #56562
  • [5660b99b43] - test: disable openssl 3.4.0 incompatible tests (Jelle van der Waa) #56160
  • [861c99f351] - test: make test-crypto-hash compatible with OpenSSL > 3.4.0 (Jelle van der Waa) #56160
  • [597a39b5f9] - test: update error code in tls-psk-circuit for for OpenSSL 3.4 (sebastianas) #56420
  • [721e9e1217] - test: add initial test426 coverage (Chengzhong Wu) #56436
  • [cfe5380c44] - test: update test-set-http-max-http-headers to use node:test (Colin Ihrig) #56439
  • [51ff71a87a] - test: update test-child-process-windows-hide to use node:test (Colin Ihrig) #56437
  • [d6aca0cd89] - test: increase spin for eventloop test on s390 (Michael Dawson) #56228
  • [82461af6ec] - test: migrate message eval tests from Python to JS (Yiyun Lei) #50482
  • [5083bbb2bb] - test: remove async-hooks/test-writewrap flaky designation (Luigi Pinca) #56048
  • [b4b26e973d] - test: deflake test-esm-loader-hooks-inspect-brk (Luigi Pinca) #56050
  • [182be26b8a] - test: update WPT for url to 67880a4eb83ca9aa732eec4b35a1971ff5bf37ff (Node.js GitHub Bot) #55999
  • [e67a84902f] - test_runner: remove unused errors (Pietro Marchini) #56607
  • [4274c6a015] - test_runner: run single test file benchmark (Pietro Marchini) #56479
  • [e57004458b] - tools: update doc to new version (Node.js GitHub Bot) #56259
  • [e039f2b571] - tools: do not throw on missing create-release-proposal.sh (Antoine du Hamel) #56704
  • [9a1e314498] - tools: fix tools-deps-update (Daniel Lemire) #56684
  • [d6469b5287] - tools: do not throw on missing create-release-proposal.sh (Antoine du Hamel) #56695
  • [e162476fdc] - tools: fix permissions in lint-release-proposal workflow (Antoine du Hamel) #56614
  • [914b4675c8] - tools: edit create-release-proposal workflow (Antoine du Hamel) #56540
  • [4ff9aa7235] - tools: validate commit list as part of lint-release-commit (Antoine du Hamel) #56291
  • [589d0ae8ea] - tools: fix loong64 build failed (Xiao-Tao) #56466
  • [bc8c39bff8] - tools: disable unneeded rule ignoring in Python linting (Rich Trott) #56429
  • [3b130002bb] - tools: add release line label when opening release proposal (Antoine du Hamel) #56317
  • [73b5c16684] - (SEMVER-MINOR) worker: add postMessageToThread (Paolo Insogna) #53682

marco-ippolito and others added 30 commits February 10, 2025 15:33
PR-URL: #54897
Backport-PR-URL: #56948
Refs: #54295
Reviewed-By: Ethan Arrowood <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
PR-URL: #54982
Backport-PR-URL: #56948
Refs: aa0308d
Refs: 9010f5f
Refs: 52a40e0
Refs: b3ef289
Refs: #2564
Refs: #25819
Refs: #27311
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
PR-URL: #54897
Refs: #54295
PR-URL: #52093
Backport-PR-URL: #56927
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Refs: #52697
Instead of using an async function wrapper, just try compiling code with
unknown module format as SourceTextModule when it cannot be compiled
as CJS and the error message indicates that it's worth a retry. If
it can be parsed as SourceTextModule then it's considered ESM.

Also, move shouldRetryAsESM() to C++ completely so that
we can reuse it in the CJS module loader for require(esm).

Drive-by: move methods that don't belong to ContextifyContext
out as static methods and move GetHostDefinedOptions to
ModuleWrap.

PR-URL: #52413
Backport-PR-URL: #56927
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Refs: #52697
Avoid repetitively calling into JS callback from C++ in
`ModuleWrap::Link`. This removes the convoluted callback style of the
internal `ModuleWrap` link step.

PR-URL: #52058
Backport-PR-URL: #56927
Reviewed-By: Joyee Cheung <[email protected]>
Refs: #52697
This patch:

1. Adds ESM syntax detection to compileFunctionForCJSLoader()
  for --experimental-detect-module and allow it to emit the
  warning for how to load ESM when it's used to parse ESM as
  CJS but detection is not enabled.
2. Moves the ESM detection of --experimental-detect-module for
  the entrypoint from executeUserEntryPoint() into
  Module.prototype._compile() and handle it directly in the
  CJS loader so that the errors thrown during compilation *and
  execution* during the loading of the entrypoint does not
  need to be bubbled all the way up. If the entrypoint doesn't
  parse as CJS, and detection is enabled, the CJS loader will
  re-load the entrypoint as ESM on the spot asynchronously using
  runEntryPointWithESMLoader() and cascadedLoader.import(). This
  is fine for the entrypoint because unlike require(ESM) we don't
  the namespace of the entrypoint synchronously, and can just
  ignore the returned value. In this case process.mainModule is
  reset to undefined as they are not available for ESM entrypoints.
3. Supports --experimental-detect-module for require(esm).

PR-URL: #52047
Backport-PR-URL: #56927
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Refs: #52697
So that if there are circular dependencies in the synchronous
module graph, they could be resolved using the cached jobs.
In case linking fails and the error gets caught, reset the
cache right after linking. If it succeeds, the caller will
cache it again. Otherwise the error bubbles up to users,
and since we unset the cache for the unlinkable module
the next attempt would still fail.

PR-URL: #52868
Backport-PR-URL: #56927
Fixes: #52864
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Vinícius Lourenço Claro Cardoso <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Refs: #52697
PR-URL: #53050
Backport-PR-URL: #56927
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Refs: #52697
Unifies the CJS and ESM source map cache map with SourceMapCacheMap
and allows the CJS cache entries to be queried more efficiently with
a source url without iteration on an IterableWeakMap.

Add a test to verify that the CJS source map cache entry can be
reclaimed.

PR-URL: #51711
Backport-PR-URL: #56927
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Refs: #52697
PR-URL: #52658
Backport-PR-URL: #56927
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Daniel Lemire <[email protected]>
Refs: #52697
This patch:

1. Refactor the routines used to compile and run an embedder
  entrypoint. In JS land special handling for SEA is done
  directly in main/embedding.js instead of clobbering the CJS
  loader. Add warnings to remind users that currently the
  require() in SEA bundled scripts only supports loading builtins.
2. Don't use the bundled SEA code cache when compiling CJS
  loaded from disk, since in that case we are certainly not
  compiling the code bundled into the SEA. Use a is_sea_main
  flag in CompileFunctionForCJSLoader() (which replaces an unused
  argument) to pass this into the C++ land - the code cache is
  still read directly from C++ to avoid the overhead of
  ArrayBuffer creation.
3. Move SEA loading code into
  MaybeLoadSingleExecutableApplication() which calls
  LoadEnvironment() with its own StartExecutionCallback().
  This avoids more hidden switches in StartExecution() and
  make them explicit. Also add some TODOs about how to support
  ESM in embedded applications.
4. Add more comments

PR-URL: #53573
Backport-PR-URL: #56927
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Refs: #52697
Tooling in the ecosystem have been using the __esModule property to
recognize transpiled ESM in consuming code. For example, a 'log'
package written in ESM:

export function log(val) { console.log(val); }

Can be transpiled as:

exports.__esModule = true;
exports.default = function log(val) { console.log(val); }

The consuming code may be written like this in ESM:

import log from 'log'

Which gets transpiled to:

const _mod = require('log');
const log = _mod.__esModule ? _mod.default : _mod;

So to allow transpiled consuming code to recognize require()'d real ESM
as ESM and pick up the default exports, we add a __esModule property by
building a source text module facade for any module that has a default
export and add .__esModule = true to the exports. We don't do this to
modules that don't have default exports to avoid the unnecessary
overhead. This maintains the enumerability of the re-exported names
and the live binding of the exports.

The source of the facade is defined as a constant per-isolate property
required_module_facade_source_string, which looks like this

export * from 'original';
export { default } from 'original';
export const __esModule = true;

And the 'original' module request is always resolved by
createRequiredModuleFacade() to wrap which is a ModuleWrap wrapping
over the original module.

PR-URL: #52166
Backport-PR-URL: #56927
Refs: #52134
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Geoffrey Booth <[email protected]>
Refs: #52697
PR-URL: #53872
Backport-PR-URL: #56927
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Refs: #52697
PR-URL: #53619
Backport-PR-URL: #56927
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Refs: #52697
It was intended that warnings should only be emitted for an
existing package.json without a type. This fixes a confusing
warning telling users to update /package.json when there are
no package.json on the lookup path at all, like this:

[MODULE_TYPELESS_PACKAGE_JSON] Warning: ... parsed as an ES module
because module syntax was detected; to avoid the performance penalty
of syntax detection, add "type": "module" to /package.json

Drive-by: update the warning message to be clear about
reparsing and make it clear what's actionable.

PR-URL: #54045
Backport-PR-URL: #56927
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Refs: #52697
PR-URL: #54868
Backport-PR-URL: #56927
Fixes: #54773
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Refs: #52697
The synchronous CJS translator can handle entrypoints now, this
can be hit when --import is used, so lift the bogus assertions and
added tests.

PR-URL: #54592
Backport-PR-URL: #56927
Fixes: #54577
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Refs: #52697
PR-URL: #54980
Backport-PR-URL: #56927
Fixes: #54931
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Refs: #52697
This patch implements a "module-sync" exports condition
for packages to supply a sycnrhonous ES module to the
Node.js module loader, no matter it's being required
or imported. This is similar to the "module" condition
that bundlers have been using to support `require(esm)`
in Node.js, and allows dual-package authors to opt into
ESM-first only newer versions of Node.js that supports
require(esm) while avoiding the dual-package hazard.

```json
{
  "type": "module",
  "exports": {
    "node": {
      // On new version of Node.js, both require() and import get
      // the ESM version
      "module-sync": "./index.js",
      // On older version of Node.js, where "module" and
      // require(esm) are not supported, use the transpiled CJS version
      // to avoid dual-package hazard. Library authors can decide
      // to drop support for older versions of Node.js when they think
      // it's time.
      "default": "./dist/index.cjs"
    },
    // On any other environment, use the ESM version.
    "default": "./index.js"
  }
}
```

We end up implementing a condition with a different name
instead of reusing "module", because existing code in the
ecosystem using the "module" condition sometimes also expect
the module resolution for these ESM files to work in CJS
style, which is supported by bundlers, but the native
Node.js loader has intentionally made ESM resolution
different from CJS resolution (e.g. forbidding `import
'./noext'` or `import './directory'`), so it would be
semver-major to implement a `"module"` condition
without implementing the forbidden ESM resolution rules.
For now, this just implments a new condition as semver-minor
so it can be backported to older LTS.

Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages
PR-URL: #54648
Backport-PR-URL: #56927
Fixes: #52173
Refs: https://github.com/joyeecheung/test-module-condition
Refs: #52697
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
This lays the foundation for supporting synchronous hooks proposed
in nodejs/loaders#198 for ESM.

- Corrects and adds several JSDoc comments for internal functions
  of the ESM loader, as well as explaining how require() for
  import CJS work in the special resolve/load paths. This doesn't
  consolidate it with import in require(esm) yet due to caching
  differences, which is left as a TODO.
- The moduleProvider passed into ModuleJob is replaced as
  moduleOrModulePromise, we call the translators directly in the
  ESM loader and verify it right after loading for clarity.
- Reuse a few refactored out helpers for require(esm) in
  getModuleJobForRequire().

PR-URL: #54769
Backport-PR-URL: #56927
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Stephen Belanger <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Refs: #52697
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: #55085
Backport-PR-URL: #56927
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
PR-URL: #54563
Backport-PR-URL: #56927
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Refs: #52697
PR-URL: #55199
Backport-PR-URL: #56927
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Refs: #52697
This is faster and more consistent with other places using the
regular expression to detect node_modules.

PR-URL: #55243
Backport-PR-URL: #56927
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Refs: #52697
For detecting whether `require(esm)` is supported without triggering
the experimental warning.

PR-URL: #55241
Backport-PR-URL: #56927
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Refs: #52697
This previously compiles a script and run it in a new context
to avoid global pollution, which is more complex than necessary
and can be too slow for it to be reused in other cases. The
new implementation just checks the frames in C++ which is safe
from global pollution, faster and simpler.

The previous implementation also had a bug when the call site
is in a ESM, because ESM have URLs as their script names,
which don't start with '/' or '\' and will be skipped. The new
implementation removes the skipping to fix it for ESM.

PR-URL: #55286
Backport-PR-URL: #56927
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Refs: #52697
Previously we assumed if `--experimental-detect-module` is true, then
`--experimental-require-module` is true, which isn't the case, as
the two can be enabled/disabled separately. This patch fixes the
checks so `--no-experimental-require-module` is still effective when
`--experimental-detect-module` is enabled.

Drive-by: make the assertion messages more informative and remove
obsolete TODO about allowing TLA in entrypoints handled by
require(esm).

PR-URL: #55250
Backport-PR-URL: #56927
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Refs: #52697
When emitting the experimental warning for `require(esm)`, include
information about the parent module and the module being require()-d
to help users locate and update them.

PR-URL: #55397
Backport-PR-URL: #56927
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Stephen Belanger <[email protected]>
Reviewed-By: Chemi Atlow <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Refs: #52697
When a ESM module cannot be loaded by require due to the presence
of TLA, its module status would be stopped at kInstantiated. In
this case, when it's imported again, we should allow it to be
evaluated asynchronously, as it's also a common pattern for users
to retry with dynamic import when require fails.

PR-URL: #55502
Backport-PR-URL: #56927
Fixes: #55500
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Chemi Atlow <[email protected]>
Trim off irrelevant internal stack frames for require(esm) warnings
so it's easier to locate where the call comes from when
--trace-warnings is used.

PR-URL: #55496
Backport-PR-URL: #56927
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
Refs: #52697
@marco-ippolito
Copy link
Member Author

@joyeecheung, @GeoffreyBooth please can you review the notable change description

@marco-ippolito marco-ippolito added needs-citgm PRs that need a CITGM CI run. v8 engine Issues and PRs related to the V8 dependency. and removed doc Issues and PRs related to the documentations. build Issues and PRs related to build files or the CI. meta Issues and PRs related to the general management of the project. tools Issues and PRs related to the tools directory. dependencies Pull requests that update a dependency file. labels Mar 6, 2025
@marco-ippolito marco-ippolito marked this pull request as ready for review March 6, 2025 12:53
@marco-ippolito marco-ippolito requested a review from a team as a code owner March 6, 2025 12:53
@joyeecheung
Copy link
Member

Updated the text about require(esm) a bit because it now does not emit warnings for calls coming from node_modules.

@anonrig
Copy link
Member

anonrig commented Mar 6, 2025

@marco-ippolito Can we add URLPattern?

Notable changes:

crypto:
  * update root certificates to NSS 3.107 (Node.js GitHub Bot) #56566
module:
  * (SEMVER-MINOR) only emit require(esm) warning under --trace-require-module (Joyee Cheung) #56194
  * (SEMVER-MINOR) unflag --experimental-require-module (Joyee Cheung) #55085
  * (SEMVER-MINOR) implement the "module-sync" exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) unflag detect-module (Geoffrey Booth) #53619
  * (SEMVER-MINOR) add __esModule to require()'d ESM (Joyee Cheung) #52166
process:
  * (SEMVER-MINOR) add process.features.require_module (Joyee Cheung) #55241
worker:
  * (SEMVER-MINOR) add postMessageToThread (Paolo Insogna) #53682

PR-URL: #57349
@marco-ippolito
Copy link
Member Author

marco-ippolito commented Mar 6, 2025

@marco-ippolito Can we add URLPattern?

I'm afraid not possible for several reason

@anonrig
Copy link
Member

anonrig commented Mar 6, 2025

@marco-ippolito Can we add URLPattern?

I'm afraid not possible for several reason, the technical one is ada is stuck at 1.1.0

URLPattern is added at Ada 3.0, on main we have 3.1

@marco-ippolito
Copy link
Member Author

marco-ippolito commented Mar 6, 2025

on v20 we are at 2.9.2, but the thing is that this semver minor release was made specifically for require esm, we dont want to add too many things, there are a lot of things to backport, but since v20 is in maintainance we dont want to overdo.

@anonrig
Copy link
Member

anonrig commented Mar 6, 2025

on v20 we are at 2.9.2, but the thing is that this semver minor release was made specifically for require esm, we dont want to add too many things, there are a lot of things to backport, but since v20 is in maintainance we dont want to overdo.

Ah we don't have C++20 on v20, that's the technical limitation.

@richardlau
Copy link
Member

I mentioned in the TSC meeting yesterday but I'm concerned that this release being a semver-minor to include require(esm) as an exceptional case (Node.js 20 is in maintenance) is leading to suggestions of inclusion of other features that would not normally be backported into a release line in maintenance.

@marco-ippolito
Copy link
Member Author

marco-ippolito commented Mar 6, 2025

Other semver minor changes are:

  • src: add --env-file-if-exists flag (idk why this was not flagged as semver minor)
  • worker: add postMessageToThread

I think they are low impact and we should probably keep them

@joyeecheung
Copy link
Member

joyeecheung commented Mar 6, 2025

By the way we still have an unreleased bug fix for require(esm) #57187 that has not yet been released in v23. Without this there is a regression/UX issue for certain frameworks (mocha and prettier, for example) that load modules with Promise.all and import() - when the user code tries to load a graph that would error (because they forget to install a dependency or made a typo for example), instead of seeing the actual linking error ERR_MODULE_NOT_FOUND, they might see a shadowing ERR_REQUIRE_CYCLE_MODULE (this is a bit racy and needs the micro task queue to be in certain shape to show up). The graph is already erroring, so this isn't technically breaking something that would've worked, just making the error message a bit confusing. If possible I think it's worth delaying the minor release a bit to include this fix after it's tested in v23 and maybe 22.

@marco-ippolito
Copy link
Member Author

By the way we still have an unreleased bug fix for require(esm) #57187 that has not yet been released in v23. Without this there is a regression/UX issue for certain frameworks (mocha and prettier, for example) that load modules with Promise.all and import() - when the user code tries to load a graph that would error (because they forget to install a dependency or made a typo for example), instead of seeing the actual linking error ERR_MODULE_NOT_FOUND, they might see a shadowing ERR_REQUIRE_CYCLE_MODULE (this is a bit racy and needs the micro task queue to be in certain shape to show up). The graph is already erroring, so this isn't technically breaking something that would've worked, just making the error message a bit confusing. If possible I think it's worth delaying the minor release a bit to include this fix after it's tested in v23 and maybe 22.

that would require at least 2/3 weeks wait. Personally I'd just ship it, in the time its ready to land we can do another release

@marco-ippolito marco-ippolito added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 6, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 6, 2025
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@GeoffreyBooth
Copy link
Member

@joyeecheung, @GeoffreyBooth please can you review the notable change description

Mine looks good, thanks!

@marco-ippolito
Copy link
Member Author

marco-ippolito commented Mar 6, 2025

Some failures with watch on macos also waiting for #57330

``` ▶ test runner watch mode ✔ should run tests repeatedly (1426.943551ms) ::debug::starting to run test runner watch mode ::debug::starting to run should run tests repeatedly ::debug::completed running should run tests repeatedly ✔ should run tests with dependency repeatedly (1432.21294ms) ::debug::starting to run should run tests with dependency repeatedly ::debug::completed running should run tests with dependency repeatedly ✔ should run tests with ESM dependency (1423.797347ms) ::debug::starting to run should run tests with ESM dependency ::debug::completed running should run tests with ESM dependency ✔ should support running tests without a file (1453.174839ms) ::debug::starting to run should support running tests without a file ::debug::completed running should support running tests without a file ✖ should support a watched test file rename (2142.971018ms) Error: ENOENT: no such file or directory, rename '/Users/runner/work/node/node/test/.tmp.2468/test.js' -> '/Users/runner/work/node/node/test/.tmp.2468/test-renamed-test.js' at renameSync (node:fs:1033:11) at Timeout._onTimeout (file:///Users/runner/work/node/node/test/parallel/test-runner-watch-mode.mjs:72:40) at listOnTimeout (node:internal/timers:581:17) at process.processTimers (node:internal/timers:519:7) { errno: -2, code: 'ENOENT', syscall: 'rename', path: '/Users/runner/work/node/node/test/.tmp.2468/test.js', dest: '/Users/runner/work/node/node/test/.tmp.2468/test-renamed-test.js' }

::debug::starting to run should support a watched test file rename
::error title=should support a watched test file rename,file=test/parallel/test-runner-watch-mode.mjs,line=72,col=40::Error [ERR_TEST_FAILURE]: ENOENT: no such file or directory, rename '/Users/runner/work/node/node/test/.tmp.2468/test.js' -> '/Users/runner/work/node/node/test/.tmp.2468/test-renamed-test.js'
at process.emit (node:events:524:28) {
code: 'ERR_TEST_FAILURE',
failureType: 'uncaughtException',
cause: Error: ENOENT: no such file or directory, rename '/Users/runner/work/node/node/test/.tmp.2468/test.js' -> '/Users/runner/work/node/node/test/.tmp.2468/test-renamed-test.js'
at renameSync (node:fs:1033:11)
at Timeout._onTimeout (file:///Users/runner/work/node/node/test/parallel/test-runner-watch-mode.mjs:72:40)
at listOnTimeout (node:internal/timers:581:17)
at process.processTimers (node:internal/timers:519:7) {
errno: -2,
code: 'ENOENT',
syscall: 'rename',
path: '/Users/runner/work/node/node/test/.tmp.2468/test.js',
dest: '/Users/runner/work/node/node/test/.tmp.2468/test-renamed-test.js'
}
}
✔ should not throw when delete a watched test file (1623.756728ms)
✖ test runner watch mode (9507.846346ms)
::debug::starting to run should not throw when delete a watched test file
::debug::completed running should not throw when delete a watched test file
ℹ tests 6
ℹ suites 1
ℹ pass 5
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 9637.027375

✖ failing tests:

test at test/parallel/test-runner-watch-mode.mjs:128:3
✖ should support a watched test file rename (2142.971018ms)
Error: ENOENT: no such file or directory, rename '/Users/runner/work/node/node/test/.tmp.2468/test.js' -> '/Users/runner/work/node/node/test/.tmp.2468/test-renamed-test.js'
at renameSync (node:fs:1033:11)
at Timeout._onTimeout (file:///Users/runner/work/node/node/test/parallel/test-runner-watch-mode.mjs:72:40)
at listOnTimeout (node:internal/timers:581:17)
at process.processTimers (node:internal/timers:519:7) {
errno: -2,
code: 'ENOENT',
syscall: 'rename',
path: '/Users/runner/work/node/node/test/.tmp.2468/test.js',
dest: '/Users/runner/work/node/node/test/.tmp.2468/test-renamed-test.js'
}
::group::Test results (5 passed, 1 failed)
::notice::Total Tests: 6
Suites 📂: 1
Passed ✅: 5
Failed ❌: 1
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 9637.027ms
::endgroup::
Command: out/Release/node --expose-internals --test-reporter=spec --test-

</details>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-ci PRs that need a full CI run. needs-citgm PRs that need a CITGM CI run. release Issues and PRs related to Node.js releases. v8 engine Issues and PRs related to the V8 dependency. v20.x v20.x Issues that can be reproduced on v20.x or PRs targeting the v20.x-staging branch.
Projects
None yet
Development

Successfully merging this pull request may close these issues.