A simplified & minimal template for manuscripts

[dhimmel]

This repository is an attempt to prototype a simpler repository that contains a Manubot manuscript. Compared to rootstock, it aims to:

• remove CI setup by using GitHub actions
• remove build.sh, assuming all of the processing pipeline can be handled by the Python package
• use defaults such that most things do not need to be specified or configured
• be more flexible, including compatibility with other Pandoc-centric formats like JOSS manuscripts
• introduce config.yaml for configuring manubot

CC @agitter @cgreene @slochower @vincerubinetti. Any thoughts will be helpful. It's not a complete draft yet, and I'm not sure of the timeline. But as we have ideas, we can add them.

[agitter]

This is a great idea and would make Manubot even more accessible.

Will the template still have the assets, plugins, and themes files from the build directory, or would those be hidden as well?

I haven't looked closely at your GitHub actions experiments yet. I expect that could greatly simplify the setup by removing all the CI configuration steps. Do we envision that eventually a user could create a new manuscript entirely from a web browser?

[dhimmel]

Will the template still have the assets, plugins, and themes files from the build directory, or would those be hidden as well?

Those would be moved to the manubot python package. If not overridden in config.yaml or perhaps by a specific path, the files that ship as part of the python package would be used. So you only need to add those files to your repo if you're modifying them.

Do we envision that eventually a user could create a new manuscript entirely from a web browser?

Yes, the goal is that you'd be able to copy the template and your repository would build immediately without any additional steps required.

[agitter]

Very cool. That workflow seems ideal. I like hiding most of these assets and files until a user decides to modify them.

I expect that syncing with updates to the upstream template will be possible but challenging, or at least require more technical expertise. Even if the template-derived manuscript can't be updated, this is still a very valuable direction.

[dhimmel]

I expect that syncing with updates to the upstream template will be possible but challenging

Yes, that will be the main downside compared to rootstock.

[vincerubinetti]

For what it's worth, here was my original proposal to Daniel about this "manubot refactor" we're talking about :
https://docs.google.com/document/d/15xCikkNvVyiz3-MZnPMkUEoQK6QzrMNmD9DNCBNohiw/edit?usp=sharing

This template repo has become essentially an outline like this doc was. As you'll see there's some differences, but the main concept is the same.

[dhimmel]

Roadmap

I've been thinking about when's the best point to try to create manubot/template and switch to it from manubot/rootstock as the default way to create new manuscripts.

Here's a roadmap that I think we can do before manubot/template that will make the eventual transition easier:

• convert rootstock to default to GitHub actions for new manuscripts: GitHub Actions workflow for building and deployment rootstock#297
• increase flexibility of pandoc-manubot-cite citation syntax: Increase flexibility of citation syntax manubot#192
• consider requiring --skip-citations in manubot process (beginning to separate functionality as much as possible)
• consider whether other aspects of manubot process can be moved to filters (for example, the plugins or metadata aspects)

Unrelated: jotting down an interesting project, mdBook, with a config file that we should look at for inspiration.

[slochower]

I'm basically in the same boat as @agitter. I think increasing the accessibility is great although I haven't been closely following the updates for GitHub actions. Does this tie us even more closely to GitHub though? I've run into cases where companies aren't allowed to, say, make significant use of GitHub but do have internal git repositories (mostly via GitLab) hosted internally. Would this sort of transition make it more difficult for someone to setup Manubot CI on a different platform?

[dhimmel]

Would this sort of transition make it more difficult for someone to setup Manubot CI on a different platform?

If we do it right, perhaps it could make it easier. One could imagine having multiple template repositories like manubot/github-template, manubot/gitlab-template, manubot/local-template. The differences between the templates would be the continuous integration commands. The local template might come with just a bash script to build the manuscript.

[vincerubinetti]

Clearing out my google drive and deleting the google doc I linked above. Here it is for posterity:

---

Manubot - an ecosystem, a set of tools and a prescribed workflow and methodology

• CORE - things that are integral to the Manubot ecosystem
  • engine - a singular, executable (on any system) package that turns a rootstock-formatted directory into a compiled paper. the command is simply "manubot --option1 --option2=something", instead of being "manubot process"
    • cli options
      • --theme - specify theme. options: none, clean, serif, modern. default: clean
      • --theme-mode (maybe) - specify "mode" of theme, like compact printing, or dark mode?
      • --plugins - specify which plugins to include, starting from a blank list. if not specified, all are included by default
    • injected resources - resources that we generally don't want users touching, so they are not included in the rootstock
      • plugins - users should definitely not be touching this unless they really know what they are doing. they can create issues for feature requests or bug reports
      • themes - users could more competently edit this, but the preferred way to modify the appearance would be to override specific styles with arbitrary.css, or just turn the theme off and build their own from scratch
      • template.html - our default pandoc html template. injected only if template.html doesn't exist in manuscript /config folder, such that users can possible entirely omit or forget the /config folder and it will still work
      • citation-style.csl - our default citation style. same behavior has template.html
  • template-manuscript - a convenient starting point for people writing a manuscript. CI preconfigured
    • #MASTER
      • /content - all the "meat" of a paper
        • *.md - markdown files, concatenated in alphabetical order
        • metadata.yaml - contains paper title, authors, affiliations, etc
        • references.yaml - manual references. current citation-tags.tsv should go away
      • /config - anything needed to configure the build process. this folder should be kept minimal and even removed entirely if possible
        • CI config file - github actions
        • citation-style.csl - optional. if included, overrides our default citation style. user can copy someone else's csl file and put it here with this filename. should include a comment at the top (if possible) with a link to CSL
        • template.html - pandoc template for html. should include in an HTML comment at the top that this is using the pandoc template syntax, and link to it
        • template.docx - pandoc template for docx. all pandoc template files should be called template.extension
        • arbitrary.html/css/js - any html/css/js files (except for template.html) will be inserted at the very end of the body, after all built-in things, so users can override and take precedence over built-in things
      • /output
        • /html
          • manuscript.html
          • /images
          • /assets
            • some-theme.css
            • some-plugin.js
            • some-plugin.css
        • /pdf
          • manuscript.pdf
        • /docx
          • manuscript.docx
      • readme.md - combine readme, usage, setup, and other readmes from other folders
        • Quick Start
          • basic install and setup
          • basic file structure overview
          • basic manuscript build Manubot commands
        • Full Docs
          • Full documentation of the entire Manubot ecosystem are available here [link]
      • license.md - a suggested license for the manuscript, included in the repo
    • #GH-PAGES
      • **exact file structure as what is in the /output folder**
      • /version/
        • /[some-hash]
          • **exact file structure as what is in the /output folder**
      • versions.html
• UTILITIES - things that are broadly useful to people outside of the Manubot ecosystem, released as standalone packages
  • manubot-cite - takes a list of input reference IDs (DOI, ISBN, etc), fetches metadata from online resources automatically, and returns retrieved data in the desired format (json by default) which can then be displayed and formatted however you want (eg with CSL)
• EXTRA - things that are related to the Manubot project as a whole
  • resources - logos and brand stuff
  • manufesto - the vision of the Manubot project
  • catalog - the Manubot "journal"
  • manubot.org - the Manubot website
    • Our Github explained (this very outline of products)
  • try-manubot - still dont really like the name of this. it's kinda vague and unclear what it is, though granted it's a hard thing to state concisely. it's just that when a new user looks at try-manubot and rootstock (which i want to rename to template or template-manuscript), i dont think they'll see a clear difference just from the name. you can "try manubot" with rootstock too.
    • Sandbox to practice contributing to a collaborative Manubot manuscript

goals/axioms:

1. the absolute biggest barrier to entry and adoption is the complexity of use and learning, therefore our number one priority should be making it easy to use and understand, which involves simplifying things as much as possible (without sacrificing too much configurability)
2. new users should be able to look at the file structure and file names and instantly know how manubot works and what each file does/is for
3. users should not have to hunt very hard to find information. docs should be extremely well organized, readmes should be combined into one and defer to full docs for all non-essential info
4. users shouldn't have to learn or work with 10 different file formats, even if they are easy and/or well-known already. for anything that requires a json-like structure, we should support both yaml and json
5. it should be possible for a user to jump right in without configuring anything, and get a pre-prescribed format/layout/behavior that we've set up for them
6. technologies used should be very popular and well known, so people are more likely to be familiar with it, and can more easily google for help if something goes wrong or if they need something special
7. user should be able to compile paper locally with no /config file at all. if anything is missing, like templates, default ones should be injected by the manubot engine. the bare minimum would be a content directory with some md files and metadata.yml