Improve readability and design of auto-built documentation (equinor#241)

Author: anders-kiaer

.github/workflows/webviz-config.yml

@@ -36,6 +36,11 @@ jobs:
         with:
           python-version: ${{ matrix.python-version }}
 
+      - name: 📦 Install npm dependencies
+        run: |
+          npm ci --ignore-scripts
+          npm run postinstall
+
       - name: 📦 Install webviz-config with dependencies
         run: |
           pip install 'pandas==${{ matrix.pandas-version }}'
@@ -65,24 +70,23 @@ jobs:
           webviz certificate
           webviz preferences --theme default
           pytest ./tests --headless --forked
-          pushd ./docs
-          python build_docs.py
-          popd
+          webviz docs --portable ./docs_build --skip-open
         
       - name: 🚢 Build and deploy Python package
         if: github.event_name == 'release' && matrix.python-version == '3.6' && matrix.pandas-version == '1.*'
         env:
           TWINE_USERNAME: __token__
           TWINE_PASSWORD: ${{ secrets.pypi_webviz_token }}
         run: |
+          export SETUPTOOLS_SCM_PRETEND_VERSION=${GITHUB_REF//refs\/tags\//}
           python -m pip install --upgrade setuptools wheel twine
           python setup.py sdist bdist_wheel
           twine upload dist/*
 
       - name: 📚 Update GitHub pages
         if: github.event_name != 'schedule' && github.ref == 'refs/heads/master' && matrix.python-version == '3.6' && matrix.pandas-version == '1.*'
         run: |
-          cp -R ./docs/_build ../_build
+          cp -R ./docs_build ../docs_build
 
           git config --local user.email "webviz-github-action"
           git config --local user.name "webviz-github-action"
@@ -91,7 +95,7 @@ jobs:
           git clean -f -f -d -x
           git rm -r *
 
-          cp -R ../_build/* .
+          cp -R ../docs_build/* .
 
           git add .
 

.gitignore

@@ -11,3 +11,9 @@ venv
 .DS_Store
 dist
 build
+webviz_config/_docs/static/fonts
+webviz_config/_docs/static/INTRODUCTION.md
+webviz_config/_docs/static/*.js
+webviz_config/_docs/static/*.css
+!webviz_config/_docs/static/webviz-doc.js
+!webviz_config/_docs/static/webviz-doc.css

CONTRIBUTING.md

@@ -509,15 +509,55 @@ black --check webviz_config tests
 
 ## Build documentation
 
-End-user documentation (i.e. YAML configuration file) be created
-after installation by
+`webviz-config` can automatically build documentation for all installed plugins. E.g.
+the end user can get an overview of all installed plugins, and their arguments, by
+running
+```bash
+webviz docs
+```
+in the terminal. Behind the scenes, `webviz-config` will then create a
+[`docsify`](https://github.com/docsifyjs/docsify) setup and open it `localhost`.
 
+The setup can also be deployed to e.g. GitHub Pages directly. To store the documentation
+output run
 ```bash
-pip install .[tests]  # if not already done
-cd ./docs
-python ./build_docs.py
+webviz docs --portable ./built_docs --skip-open
+```
+The `--skip-open` argument is useful in a CI/CD setting, to prevent `webviz-config`
+from automatically trying to open the created documentation in the browser.
+
+### Improve plugin documentation
+
+Auto-built `webviz` documentation will:
+- Find all installed plugins.
+- Group them according to top package name.
+- Show a `YAML` snippet with the plugin argument options.
+  - Arguments with default values will be presented with the default value, and be marked as optional.
+  - If an argument has a type annotation, that will be included in the documentation.
+
+In addition, if the plugin class has a docstring, the content in the docstring will
+be used as a short introduction to the plugin. If the docstring has several parts,
+when split by a line containing only `---`, they will be used as follows:
+1. First part is the introduction to the plugin.
+2. Second part is a more detailed explanation of the plugin arguments.
+3. Third part is information regarding plugin data input. E.g assumptions,
+prerequisites and/or required/assumed data format.
+
+Since `docsify` is used behind the scenes, you can create information boxes, warning boxes
+and use GitHub emojis :bowtie: in the plugin docstring.
+See [`docsify` documentation](https://docsify.js.org/#/) for details.
+
+[KaTeX](https://katex.org/) is also used behind the scenes, meaning that you can add
+math (TeX syntax) to your docstrings and get it rendered in the auto-built
+documentation. Remember that `\` is an escape character in Python, i.e. either
+escape it (`\\`) or use raw strings:
+```python
+class HistoryMatch(WebvizPluginABC):
+    r"""This is a docstring with some inline math $\alpha$ and some block math:
+
+$$\alpha = \frac{\beta}{\gamma}$$
+"""
 ```
 
-Officially updated built end-user documentation (i.e. information to the
-person setting up the configuration file) is
-[hosted here on github](https://equinor.github.io/webviz-config/).
+Example of auto-built documentation for `webviz-config` can be seen
+[here on github](https://equinor.github.io/webviz-config/).

INTRODUCTION.md

@@ -0,0 +1,101 @@
+# Webviz introduction
+
+### Fundamental configuration
+
+A configuration consists of some mandatory properties, e.g. app title,
+and one or more pages. A page has a title and some content.
+Each page can contain as many plugins as you want.
+
+Plugins represent predefined content, which can take one or more arguments.
+Lists and descriptions of installed plugins can be found on the other subpages.
+
+Content which is not plugins is interpreted as text paragraphs.
+
+A simple example configuration:
+```yaml
+# This is a webviz configuration file example.
+# The configuration files use the YAML standard (https://en.wikipedia.org/wiki/YAML).
+
+title: Reek Webviz Demonstration
+
+pages:
+
+ - title: Front page
+   content:
+    - BannerImage:
+        image: ./example_banner.png
+        title: My banner image
+    - Webviz created from a configuration file.
+
+ - title: Markdown example
+   content:
+    - Markdown:
+        markdown_file: ./example-markdown.md
+```
+
+### Command line usage
+
+#### Get documentation
+
+You can always run `webviz --help` to see available command line options.
+To see command line options on a subcommand, run e.g. `webviz build --help`.
+
+:books: To open the `webviz` documentation on all installed plugins, run `webviz docs`.
+
+#### Portable vs. non-portable
+
+Assuming you have a configuration file `your_config.yml`,
+there are two main usages of `webviz`:
+
+```bash
+webviz build your_config.yml
+```
+and
+```bash
+webviz build your_config.yml --portable ./some_output_folder
+python ./some_output_folder/webviz_app.py
+```
+
+**Portable**
+
+The portable way is useful when one or more plugins included in the configuration need to do
+some time-consuming data aggregation on their own, before presenting it to the user.
+The time-consuming part will then be done in the `build` step, and you can run your
+created application as many time as you want afterwards, with as little waiting
+time as possible.
+
+The `--portable` way also has the benefit of creating a :whale: Docker setup for your
+application - ready to be deployed to e.g. a cloud provider.
+
+**Non-portable**
+
+Non-portable is the easiest way if none of the plugins
+have time-consuming data aggregration to do.
+
+A feature in Dash, used by `webviz` is [hot reload](https://community.plot.ly/t/announcing-hot-reload/14177).
+When the Dash Python code file is saved, the content seen in the web browser is
+automatically reloaded (no need for localhost server restart). This feature is passed on to
+the Webviz configuration utility, meaning that if you run
+```bash
+webviz build ./examples/basic_example.yaml
+```
+and then modify `./examples/basic_example.yaml` while the Webviz application is
+still running, a hot reload will occur.
+
+#### Localhost certificate
+
+For quick local analysis, `webviz-config` uses `https` and runs on `localhost`.
+In order to create your personal :lock: `https` certificate (only valid for `localhost`), run
+```bash
+webviz certificate --auto-install
+```
+Certificate installation guidelines will be given when running the command.
+
+#### User preferences
+
+You can set preferred :rainbow: theme and/or :earth_africa: browser, such that `webviz` remembers it for later
+runs. E.g.
+
+```bash
+webviz preferences --theme equinor --browser firefox
+```

README.md

@@ -1,5 +1,5 @@
 <p align="center">
-  <img height="150" src="https://github.com/equinor/webviz-config/raw/master/docs/assets/webviz-logo.svg?sanitize=true">
+  <img height="150" src="https://github.com/equinor/webviz-config/raw/master/webviz_config/_docs/static/webviz-logo.svg?sanitize=true">
 </p>
 
 <h2 align="center">Democratizing Python web applications</h2>
@@ -42,21 +42,21 @@ Example configuration file and information about the standard plugins can be see
 
 ### Installation
 
-The simplest way of installing `webviz-config` is to run
+The recommended and simplest way of installing `webviz-config` is to run
 ```bash
 pip install webviz-config
 ```
 
-If you want to download the latest source code and install it manually you 
+If you want to develop `webviz-config` and install the latest source code manually you
+can do something along the lines of:
 can run
 ```bash
 git clone [email protected]:equinor/webviz-config.git
 cd ./webviz-config
-pip install .
+npm ci --ignore-scripts && npm run postinstall
+pip install -e .
 ```
 
-### Usage
-
 After installation, there is a console script named `webviz` available. You can test the installation by using the provided test
 configuration file,
 ```bash
@@ -74,28 +74,11 @@ The optional arguments can be seen when running
 ```bash
 webviz --help
 ```
-For example will
-```bash
-webviz build ./examples/basic_example.yaml --portable ./my_portable_app
-```
-create a portable instance (with corresponding Dockerfile) and store it in the provided folder.
 
-A feature in Dash is [hot reload](https://community.plot.ly/t/announcing-hot-reload/14177).
-When the Dash Python code file is saved, the content seen in the web browser is
-automatically reloaded (no need for localhost server restart). This feature is passed on to
-the Webviz configuration utility, meaning that if the user runs 
-```bash
-webviz build ./examples/basic_example.yaml
-```
-and then modifies `./examples/basic_example.yaml` while the Webviz application is
-still running, a hot reload will occur.
+### Usage
 
-For quick local analysis, `webviz-config` uses `https` and runs on `localhost`.
-In order to create your personal `https` certificate (only valid for `localhost`), run
-```bash
-webviz certificate --auto-install
-```
-Certificate installation guidelines will be given when running the command.
+See [the introduction](./INTRODUCTION.md) page for information on how you
+create a `webviz` configuration file and use it.
 
 ### Creating new plugins