pnpm for Gutenberg

[manzoorwanijk]

I've been exploring using pnpm as the package manager for the Gutenberg monorepo. The proof-of-concept PR - #74135 demonstrates a full conversion with all tests passing and everything building correctly.

Because switching package managers is a significant change that touches many files, this discussion outlines a plan to break it into smaller, reviewable chunks.

Previous Efforts

• Try pnpm #37324 by @youknowriad
• Try pnpm (take two) #38624 by @sarayourfriend

Why pnpm?

pnpm offers faster installs, efficient disk usage through content-addressable storage, and stricter dependency resolution that catches missing dependencies early. We at Jetpack often have to deal with dependency issues for @wordpress/* packages. Using pnpm will prevent many such issues.

Takeaways from the POC

• We are in a good shape to adopt pnpm as there aren't any major refactors required.
• It mostly involves fixing the peer or missing dependencies for workspaces.
• The only major change is in our build script which expects modules to be available everywhere for sass build, but pnpm doesn't make workspace packages available everywhere. So, that change is necessary
• tsconfig and jest config need some love given the strict structure of pnpm dependencies. tsconfig changes can be made directly to trunk as a part of the incremental PRs.
• /storybook should be a workspace to avoid the need for all its dependencies in the root package.json

Migration Strategy

The migration is split into smaller, reviewable chunks:

1. Incremental PRs: Fix packages with missing peer dependencies or conflicting versions in separate PRs. These improvements benefit the monorepo regardless of the final switch to pnpm. It can be done by picking the chunks from the POC PR.

2. Final Switch PR: Once all packages are ready, a final PR will:

   • Update package.json files with workspace: protocol for local dependencies
   • Migrate all the patched dependencies
   • Add pnpm-lock.yaml lockfile
   • Add pnpm-workspace.yaml configuration
   • Update CI workflow configurations to use pnpm

Guidelines

When making changes, keep these in mind:

• Pin dependencies to exact versions - Avoid caret (^) or tilde (~) ranges. This ensures a more efficient and predictable dependency tree.
• Use consistent versions across packages - Use the same version of shared external dependencies across all packages to avoid duplication.
• Ensure all workspaces have a name field - Some workspaces in /routes are missing this, which is required for pnpm to work correctly.
• Some packages may need tsconfig changes - pnpm's stricter resolution may require adjustments to TypeScript configurations.

CI Considerations

Some workflows compare against previous builds stored with npm. These will need baseline updates after the switch:

• .github/workflows/bundle-size.yml - Compares bundle sizes against trunk
• .github/workflows/performance.yml - Compares performance metrics against trunk

These workflows are expected to fail for the final PR until baselines are updated post-merge.

Help Wanted

Looking for reviewers to help with the incremental PRs.

CC: @jsnajdr @aduth @sirreal @nerrad

[SirLouen]

If its passing, why don't you move out of Draft?

[manzoorwanijk]

Because it was just a POC (exploration). I am not sure if people here would be happy with such a giant PR instead of smaller ones as proposed.

[SirLouen]

I am not sure if people here would be happy with such a giant PR instead of smaller ones as proposed.

My rule of thumb: always smaller ones are preferred by reviewers.

[youknowriad]

This needs to be coordinated with Core. Especially after my PR here WordPress/wordpress-develop#10638

Not in disagreement but this likely need to be a make/core proposal for more visibility and coordination with Core IMO.

[sirreal]

I agree with what @youknowriad said, I'd expect Core to switch to pnpm if and when Gutenberg does. There's value in the two projects being aligned. It's important to understand how that impacts Core and what changes would be necessary there. It may signal to the WordPress community a preference for pnpm. This could start as a discussion topic at a #core dev chat and ultimately a proposal on make/core is a good idea.

pnpm offers faster installs, efficient disk usage through content-addressable storage, and stricter dependency resolution that catches missing dependencies early.

I'd like to see data about this for the relevant projects. How much faster? How much smaller on disk? How many dependency issues are there and what's the impact?

I'd also like to know how much of an impact it has on CI. If CI for Gutenberg and Core can get a significant speed-up from faster and smaller installation, that's a compelling motivation to switch.

Pin dependencies to exact versions - Avoid caret (^) or tilde (~) ranges. This ensures a more efficient and predictable dependency tree.

Will you explain the reasoning behind this guideline? For a project like Gutenberg, I disagree. Gutenberg publishes libraries, and when libraries use exact versions it tends to cause package duplication and inconvenience for downstream consumers. For applications (not published as libraries), exact versions do have benefits.

[anomiex]

Will you explain the reasoning behind this guideline? For a project like Gutenberg, I disagree. Gutenberg publishes libraries, and when libraries use exact versions it tends to cause package duplication and inconvenience for downstream consumers. For applications (not published as libraries), exact versions do have benefits.

I was going to comment the same thing. 🙂

Exact-version dependencies in libraries also mean that if one of the library package's dependencies releases a security fix patch version, all your downstream users have to wait for you to release your own patch version. With caret or tilde dependencies, your downstream users just need to update their own lockfiles.

[manzoorwanijk]

Yeah, I agree with that. I was thinking more about inside the monorepo, but yes, it makes complete sense to use ^ and ~ because the packages are ultimately going to be published for public use. Thank you for flagging that.

[aduth]

Security features are another factor I'd consider in weighing pnpm. Personally I feel like I'm running a gamble every time I execute npm install whether I'm about to install malware on my computer, given the recent prevelance of attacks. pnpm has several security features which, while by no means perfect, do have some sound reasoning around mitigating potential supply chain attacks:

https://pnpm.io/supply-chain-security

[SirLouen]

Surely it must be coordinated with Core. But I only see two caveats.

1. In GB, building is the bread and butter. In Core, one rarely builds. For core is interesting but being objective, core can't less about npm or pnpm. The only part that are important are the CI/CD changes but not the tool itself. Yes, surely having unified commands is always great to simply everyone's overhead.
2. In Core there are half dozen active maintainers. In GB there are two dozen. Reviewing here can be done in a week or two. Reviewing in core is going to take a version or two.

So yes, It would be a great idea to do some coordination with Core, but not mandatory to be synchronized. If possible pnpm could be added to GB earlier, and simultaneously but, not blocking, start the process to add pnpm in Core.

[manzoorwanijk]

While it is good to coordinate this with Core, the problem we are trying to solve here is to have the published packages in a good state, without missing dependencies, etc.

IIRC, Core doesn't publish any packages to npm and may not need to handle the issues we face in Gutenberg. So, it's fine to do it in Gutenberg, and then if possible/feasible, we can use pnpm in Core as well.