build, doc: use new api doc tooling

[flakey5]

Switches over to using the new doc generation tooling. For more background on this, please see #52343

Currently a draft just to get feedback on the approach to this integration.

cc @nodejs/web-infra

[nodejs-github-bot]

Review requested:

• @nodejs/nodejs-website
• @nodejs/web-infra

[ovflowd]

@flakey5 I guess you also need to check our GitHub Action Workflows, and also other mentions of these files within the source.

Like within the Contributor Docs, there is a file that describes how the legacy API doc tooling works, and I believe there are other references also.

[flakey5]

lint-js-and-md is failing because of the linting errors since it exits with a non-zero status code if there's anything wrong with the docs. I think we should skip the REPLACEME checks for normal ci runs.

It also looks like synopsis.md fails the introduced_in check because it's not under the top-level header, should it be enforced that introduced_in goes under the top-level header or should we change it to just check that it exists somewhere in the file (like it was doing previously)?

[flakey5]

Also not sure what's going on with lint-addon-docs? cc @araujogui

[ovflowd]

lint-js-and-md is failing because of the linting errors since it exits with a non-zero status code if there's anything wrong with the docs. I think we should skip the REPLACEME checks for normal ci runs.

It also looks like synopsis.md fails the introduced_in check because it's not under the top-level header, should it be enforced that introduced_in goes under the top-level header or should we change it to just check that it exists somewhere in the file (like it was doing previously)?

REPLACEME shouldn't error, imo, just give a warning. Our linter should have warn and error levels.

And yes introduced_in must be top level!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.52%. Comparing base (cbe0233) to head (d1ba077).

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #57343      +/-   ##
==========================================
- Coverage   88.53%   88.52%   -0.01%     
==========================================
  Files         703      703              
  Lines      208413   208413              
  Branches    40191    40191              
==========================================
- Hits       184521   184505      -16     
- Misses      15902    15926      +24     
+ Partials     7990     7982       -8     

see 41 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[araujogui]

lint-js-and-md is failing because of the linting errors since it exits with a non-zero status code if there's anything wrong with the docs. I think we should skip the REPLACEME checks for normal ci runs.

It also looks like synopsis.md fails the introduced_in check because it's not under the top-level header, should it be enforced that introduced_in goes under the top-level header or should we change it to just check that it exists somewhere in the file (like it was doing previously)?

Actually, the linter only returns 1 if there's an error-level issue, and I don't think that's the case here.

[araujogui]

Also not sure what's going on with lint-addon-docs? cc @araujogui

I’m not sure either, but I’ll check it out.

[ovflowd]

@flakey5:

3:1-3:9   warning Use "the Node.js" instead of "Node.js'" prohibited-strings remark-lint
4:46-4:50 warning Use "Node.js" instead of "Node"         prohibited-strings remark-lint

On the README.md file you updated (tools/doc/README.md) after updating those can you run make format-md (?)

[ovflowd]

This is the result of many months of arduous work between many awesome folks, including @flakey5 @AugustinMauroy @araujogui @ovflowd @avivkeller and others.

I'm so proud of what we are achieving here and this is a huge step towards a modern tooling and a revamped API docs within Node.js

Approving, as I believe this is ready!

[ovflowd]

cc @nodejs/collaborators can we have another approval here? 🙏

[lpinca]

RSLGTM because it is hard to review and outside of my comfort zone.

[aymen94]

LGTM, though I’d suggest removing all empty structures and data.

[ovflowd]

LGTM, though I’d suggest adding two clarifications:

• Make explicit how text-only / non-API sections should be represented in both json and json-all.

• Add a brief note on migration: how legacy JSON relates to the new schema, the expected deprecation window

Im confused, this PR doesn't adopt new schemas or changes the JSON format.

[aymen94]

LGTM, though I’d suggest adding two clarifications:

• Make explicit how text-only / non-API sections should be represented in both json and json-all.
• Add a brief note on migration: how legacy JSON relates to the new schema, the expected deprecation window

Im confused, this PR doesn't adopt new schemas or changes the JSON format.

You're right — my bad, I mixed this up with a different thread.

[ovflowd]

@aduh95 I believe we're finally ready. The JSON output is 1:1 for what matters and the build-time is incredibly small now. (8s~)

[ovflowd]

Note that I've changed the approach to if it detects any change to simply build all files, as the current main approach as it is fast enough.