Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: compute module.paths lazily #51611

Closed
wants to merge 1 commit into from
Closed

Conversation

kvakil
Copy link
Contributor

@kvakil kvakil commented Jan 30, 2024

Make module.paths a getter which computes the property when it's
requested. The paths property is entirely deterministically computed
from the path property, and so it's not necessary to keep the relatively
large paths array in memory.

On an internal application, this reduces the used heap by around
5.5%. Furthermore (although it was not a goal of this PR) it speeds
up the execution time of vite --version by ~2% according to
hyperfine. A similar speed improvement is observed on the internal
application.

Notably, this introduces two minor changes which may be considered
breaking. The obvious one is that paths is now a getter/setter
and not a property. The other one is more subtle: the lazy computation
means that if a user changes mod.path = X and then later invokes
mod.paths, the results will reflect the updated path as opposed
to the current behavior where it would reflect the original path.
One way to fix the latter would be to force capturing paths before
path is changed, I'm open to doing that (or some cleaner approach?)
if the reviewers deem it necessary.

Make module.paths a getter which computes the property when it's
requested. The paths property is entirely deterministically computed
from the path property, so it's not necessary to keep the relatively
large paths array in memory.

On an internal application, this reduces the used heap by around 5.5%.
Furthermore (although it was not a goal of this PR) it speeds up the
execution time of `vite --version` by ~2% according to hyperfine. A
similar speed improvement is observed on the internal application.

Notably, this introduces two minor changes which may be considered
breaking. The obvious one is that `paths` is now a getter/setter and not
a property. The other one is more subtle: the lazy computation means
that if a user changes `mod.path = X` and then later invokes
`mod.paths`, the results will reflect the updated path as opposed to the
current behavior where it would reflect the original path. One way to
fix the latter would be to force capturing `paths` before `path` is
changed, I'm open to doing that (or some cleaner approach?) if the
reviewers deem it necessary.
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added esm Issues and PRs related to the ECMAScript Modules implementation. needs-ci PRs that need a full CI run. util Issues and PRs related to the built-in util module. labels Jan 30, 2024
Copy link
Member

@joyeecheung joyeecheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-wise LGTM though we should run this through CITGM to see if we are breaking any popular packages.

const lazyPathsGetter = {
__proto__: null,
enumerable: true,
get() { return this[kPaths] ?? Module._nodeModulePaths(this.path); },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could also just replace this.paths with a value property once computed so subsequent accesses are not invoking a getter again & the property type change is less observable from the user land.

@aduh95 aduh95 added needs-citgm PRs that need a CITGM CI run. dont-land-on-v18.x PRs that should not land on the v18.x-staging branch and should not be released in v18.x. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. labels Feb 2, 2024
@aduh95
Copy link
Contributor

aduh95 commented Feb 2, 2024

Added the dont-land labels for LTS because of the potential for breakage mentioned in the OP.

@joyeecheung
Copy link
Member

I would still like to see this happening...@kvakil are you still working on this?

@aduh95
Copy link
Contributor

aduh95 commented May 11, 2024

This needs a rebase.

@aduh95 aduh95 added the stalled Issues and PRs that are stalled. label Nov 3, 2024
Copy link
Contributor

github-actions bot commented Nov 3, 2024

This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open.

Copy link
Contributor

github-actions bot commented Dec 4, 2024

Closing this because it has stalled. Feel free to reopen if this issue/PR is still relevant, or to ping the collaborator who labelled it stalled if you have any questions.

@github-actions github-actions bot closed this Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dont-land-on-v18.x PRs that should not land on the v18.x-staging branch and should not be released in v18.x. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. esm Issues and PRs related to the ECMAScript Modules implementation. needs-ci PRs that need a full CI run. needs-citgm PRs that need a CITGM CI run. stalled Issues and PRs that are stalled. util Issues and PRs related to the built-in util module.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants