Hosting illustrations in a new Jupyter Book and using xref for the book

[JimMadge]

Related to #3843 initiative and the-turing-way/images#1

From a conversation at Book Dash, an idea for how to manage images in the future.
We could host all of the illustration (and maybe other project images) in a new Jupyter Book.
We would then be able to embed these in The Turing Way using the MyST embed directive.
Here is an example of embedding a labelled image into a figure.
The only difference is we would use a inter-MyST reference, like

<!-- In the gallery project -->
(illustration)=
![](./figures/illustration.jpg)

<!-- In the book -->
:::{figure} xref:ttw-gallery#illustration
:label: fig-illustration
Caption text
:::

This way,

• The book repo can be reduced in size significantly
• We are better set up to catalogue all illustrations with metadata
• We have a single, canonical source for all illustrations (whether or not they appear in the book)
• Using images should be easier (copy/paste the tag from the gallery, we could even automatically generate template figure directives)

Tagging some interested people,

@RichardJActon
@AlexandraAAJ
@reshamas
@Arielle-Bennett

Also probably of interest to @the-turing-way/infrastructure-working-group as this is related to our initiative #3843

[sgibson91]

Are there downsides to this approach? Seems like a good idea to me.

[JimMadge]

• Effort to set up (more an investment than a downside)
• Longer process to add a new image
  • Two repos
  • Required metadata (although, this is also a feature)
• More complex build process? I expect the full sized figures would be fetched on demand rather than copied, but I'm not certain

[sgibson91]

Do we have an alternative idea that would trump these downsides?

We would probably also need to put guardrails on the book repo itself, some CI to be like "hey, looks like you're trying to add an image" (annoying clippy!)

[da5nsy]

I would add that I think it would make it more difficult if we were to ever move away to a totally different system (quarto?) - currently I'm sure we'd need to change a lot of structure/syntax but at least all the content is self-contained. Not a blocker but just adding to the list.

[JimMadge]

I would add that I think it would make it more difficult if we were to ever move away to a totally different system (quarto?) - currently I'm sure we'd need to change a lot of structure/syntax but at least all the content is self-contained. Not a blocker but just adding to the list.

That is true, but it would already be very difficult to switch. We use a lot of MyST roles and directives.

[RichardJActon]

I would add that I think it would make it more difficult if we were to ever move away to a totally different system (quarto?)

FYI quatro it looks like quatro also has a somewhat analogous embed system https://quarto.org/docs/authoring/notebook-embed.html - which now I'm looking at both of them appear to be 'run-time' embeds not 'build-time' embeds (what quarto calls includes: https://quarto.org/docs/authoring/includes.html) which probably answers one of my questions below about what get pulled down when you do a local build with this system.

[RichardJActon]

Some questions that some to mind:

• Does this fail gracefully if the content from the other book cannot be found?
• If I'm in a low bandwidth situation what actually happens when I build the book locally, do images get pulled down at build time or are they only pulled down by the browser for pages I got look at a preview for?
• How to we manage the images in the other repo we are still putting large(ish) binary files directly in git which is not optimal as discussed in Future management of images in the turing way #3843. Perhaps something like git-lfs or git-annex (https://git-annex.branchable.com/) could be used in this new repo? (Especially if we want to encourage inclusion of editable source file for images.)

[JimMadge]

If I've understood the documentation correctly the answer to my second question is that images would not be pulled down at 'build time' but only pulled down as needed when as page is loaded in the browser.

That feels like what is happening when I have tried this. At build time there will be a check to see if your label (like ![](xref:another-project#some-figure) does exist, and some data about those nodes will be fetched from the .json page (like this one). Because I believe you are essentially copying some MyST from another project into yours.

I think that would result the image path being https://... so when you open the page in a browser it is fetching the image from that source. But I think that isn't working as intended currently.

[JimMadge]

I think we probably would want to go ahead and remove the images from the book repo's history. To help speed up cloning the repo and working on the book where internet connection is slower.

[sgibson91]

I think we probably would want to go ahead and remove the images from the book repo's history. To help speed up cloning the repo and working on the book where internet connection is slower.

This should be a different Discussion thread - but how do you propose to do that? The reason we haven't in the past is because we just haven't had the resources to hold the hand of everyone who has a fork and/or PR open that'll need updating/recreating.

[JimMadge]

Ask for forgiveness 😆. But yes, rewriting history is tricky and disruptive for branches/PRs. Maybe we can find a quiet time to have least disruption.

[sgibson91]

We'd need to close/merge as many PRs as possible too

[LizHareDogs]

Wearing my accessibility hat, I really like this idea.

• We are already successful having a screen-reader-navigable Jupyter book. We might have to work on the mechanism for displaying images but we know the main site can be navigable without a mouse.
• • Metadata like alt text is not only good for image accessibility, but might be really useful for people searching for images to illustrate topics they are presenting.
• • I would not have to ask someone to help me identify images :)
• • I like the idea that having the images in a separate place would facilitate access from poor internet connections.

[JimMadge]

Tracking a relevant bug here.

Embedded figures from the local path of another project currently don't display.