Serve rustdoc static files from /-/rustdoc.static/

[jsha]

Fixes #1869

This changes the --static-root-path we pass to rustdoc, so it generates HTML that looks for files in the new location. It also changes where the shared static files are uploaded to in storage. They are uploaded to a matching path (/-/rustdoc.static/).

Files under this path can be served from an ordinary route. After this change, the special logic in SharedResourceHandler (ignore request path, serve file from storage root) will only be needed when serving documentation built prior to this PR being deployed. As such, I've renamed SharedResourceHandler to LegacySharedResourceHandler, and named the new handler SharedResourceHandler.

Note: this does not depend on rust-lang/rust#101702.

[syphar]

thanks for pushing this!

I like renaming the old handler.

[syphar]

also, if you want to copy/paste some things: master...syphar:docs.rs:static-prefix

[jsha]

Thanks for the feedback! Pushed some work-in-progress changes. For some reason with the change to the RUSTDOC_STATIC_STORAGE_PREFIX, I'm getting 404s for the static files. Will debug later, but I figured y'all might want to take a look.

[jsha]

Figured out the problem, rebased, and pushed a new revision. I think I've addressed all feedback.

[syphar]

@jsha it looks like the build & clippy is failing, could you check?

[jsha]

Fixed!

[syphar]

We're nearly there! Thanks for sticking to this!

I'm missing tests for the new static handler, also checking the canonicalization.

Since my local build-process is still broken I would love to have another manual build& server test (@Nemo157 ?)

[jsha]

I wound up removing the code that tried to detect path traversal by canonicalizing paths. The code I copy-pasted from the docs.rs static assets path was using std::fs to canonicalize paths; that technique doesn't apply to S3.

And in fact according to this, S3 doesn't treat ../ specially anyhow, so I believe we don't need to worry about path traversal here.

[syphar]

I wound up removing the code that tried to detect path traversal by canonicalizing paths. The code I copy-pasted from the docs.rs static assets path was using std::fs to canonicalize paths; that technique doesn't apply to S3.
And in fact according to this, S3 doesn't treat ../ specially anyhow, so I believe we don't need to worry about path traversal here.

TIL, I didn't know that. Then you're right removing the piece.

[syphar]

This is fine from my side,

IMO this needs a second manual test ( build a crate, check if the docs can access their static files, probably even disabling the legacy handler so we're sure the new handler is used).

( I can't test builds right now on my machine, need to fix that at.. )

[Nemo157]

I just tested https://docs.rs.dev.nemo157.com/cbor-diag/0.1.11/cbor_diag/ locally and confirmed that main-20221019-1.66.0-nightly-4b8f43199.js which it loads isn't available in the minio root, only under docsrs/rust-docs-rs/rustdoc-static/.

[jsha]

( I can't test builds right now on my machine, need to fix that at.. )

Any chance you're running into #1881 ?

[syphar]

( I can't test builds right now on my machine, need to fix that at.. )

Any chance you're running into #1881 ?

Thanks for the hint, but that's not it. It's because my computer is an M1 Mac, and the "run the build in the web container " doesn't work any more.

[syphar]

@jsha I just deployed the change and the first build uses the new path:
https://docs.rs/cozo/0.1.3/cozo/