Skip to content

File like tutorial #638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Apr 22, 2025
Merged

File like tutorial #638

merged 16 commits into from
Apr 22, 2025

Conversation

scotts
Copy link
Contributor

@scotts scotts commented Apr 12, 2025

Tutorial for streaming data through a file-like object.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Apr 12, 2025
# `Filesystem interfaces for Python <https://filesystem-spec.readthedocs.io/en/latest/?badge=latest>`_.
# Note that using these capabilities from the fsspec` library also requires the
# `aiohttp <https://docs.aiohttp.org/en/stable/>`_ module. You can install both with
# ``pip install fsspec aiohttp``.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NicolasHug, the code snippet for pip install renders kinda ugly. Rather than being one block of inline code, it renders each word separately. Do you know a better way to do it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no, this is a pytorch-sphinx-theme issue, and it has been a problem for years :(
pytorch/pytorch_sphinx_theme#107

I hope it'll be fixed in the new theme (not sure what the status of that is).

To avoid the uglyness, maybe we can just write "pip install ..." in quotes, without using double backticks. Or use single backticks so it renders in italics.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Unfortunate. I'll play around with it a little.

@scotts scotts marked this pull request as ready for review April 22, 2025 13:53
@scotts scotts changed the title [WIP] File like tutorial File like tutorial Apr 22, 2025
Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fantastic, thank you!

The example takes a bit of time to build (~3 minutes), so it does increase the duration of our docs job. I suspect it means that the doc build job now becomes our longest CI job (as it also relies on the CUDA build, which is slow).

But that's an OK price to pay for now IMHO. If it becomes too slow, we can explore reducing average_over, or avoid benchmarking the download_before_decode() function which is the obvious offender.

@scotts
Copy link
Contributor Author

scotts commented Apr 22, 2025

Time spent is a good point. Let me reduce average_over to 10 instead of 20. That still should be enough samples to reduce variation.

@scotts scotts merged commit fda54c9 into pytorch:main Apr 22, 2025
44 of 46 checks passed
@scotts scotts deleted the file_like_tutorial branch April 22, 2025 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants