|
| 1 | +# Documentation Publishing Process |
| 2 | + |
| 3 | +This document explains the inner workings of how our documentation is published when the GitHub Actions workflow is triggered. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +Our documentation publishing process uses a combination of Notion, GitHub Actions, and Docusaurus to create a seamless documentation pipeline. Here's a detailed breakdown of how it works. |
| 8 | + |
| 9 | +## The Publishing Pipeline |
| 10 | + |
| 11 | +### 1. Content Creation in Notion |
| 12 | + |
| 13 | +- Content is authored and maintained in three separate Notion workspaces: |
| 14 | + - Concepts documentation (main docs) |
| 15 | + - Dev Tools documentation |
| 16 | + - Contracts documentation |
| 17 | +- Each workspace has its own root page ID stored in GitHub Secrets: |
| 18 | + - `DOCU_NOTION_CONCEPT_ROOT_PAGE` |
| 19 | + - `DOCU_NOTION_DEV_ROOT_PAGE` |
| 20 | + - `DOCU_NOTION_CONTRACTS_ROOT_PAGE` |
| 21 | + |
| 22 | +### 2. GitHub Actions Workflow Trigger |
| 23 | + |
| 24 | +The workflow (`release.yml`) is triggered by either: |
| 25 | +- A push to the `main` branch |
| 26 | +- Manual trigger through GitHub Actions UI |
| 27 | + |
| 28 | +### 3. Content Synchronization Process |
| 29 | + |
| 30 | +1. **Environment Setup** |
| 31 | + - Checkout repository |
| 32 | + - Setup Node.js |
| 33 | + - Install dependencies |
| 34 | + |
| 35 | +2. **Notion Content Pull** |
| 36 | + - Clears all previously committed doc files (can remove this step if we want to preserve previously published pages) |
| 37 | + - Creates directory structure (`docs/concepts`, `docs/dev`, `docs/contracts`) |
| 38 | + - Uses `@sillsdev/docu-notion` to pull content from Notion |
| 39 | + - Converts Notion pages to Markdown files |
| 40 | + - Downloads and processes any images |
| 41 | + |
| 42 | +4. **File Organization** |
| 43 | + - Markdown files are generated with proper frontmatter |
| 44 | + - Images are stored in `static/notion_imgs` |
| 45 | + - Directory hierarchy is preserved from respective 'Outline' pages within each Notion workspace |
| 46 | + |
| 47 | +### 4. Git Operations |
| 48 | + |
| 49 | +1. **Staging Changes** |
| 50 | + - All Markdown files are staged using `git add` |
| 51 | + - Changes are committed with message "Apply changes from docu-notion" |
| 52 | + |
| 53 | +2. **Building the Site** |
| 54 | + - Docusaurus builds the site using the new content |
| 55 | + - Output is generated in the `build` directory |
| 56 | + |
| 57 | +3. **Deployment to gh-pages** |
| 58 | + - The built site is deployed to the `gh-pages` branch |
| 59 | + - This branch contains: |
| 60 | + - The compiled static site |
| 61 | + - The raw Markdown files |
| 62 | + - All assets and images |
| 63 | + |
| 64 | +## Important Implementation Details |
| 65 | +The site uses three documentation instances by running `@sillsdev/docu-notion` on each of the Notion workspaces and directing page downloads to specific folders within the directory when the workflow is ran. |
| 66 | + |
| 67 | +### Package.json Scripts |
| 68 | + |
| 69 | +The following scripts in `package.json` handle pulling content from Notion: |
| 70 | + |
| 71 | +```javascript |
| 72 | +{ |
| 73 | + "scripts": { |
| 74 | + "clear-docs": "rimraf ./docs/", |
| 75 | + "pull:concepts": "cross-var yarn docu-notion -n $DOCU_NOTION_INTEGRATION_TOKEN -r $DOCU_NOTION_CONCEPT_ROOT_PAGE -m ./docs/concepts -i ./static/notion_imgs -p /notion_imgs/", |
| 76 | + "pull:dev": "cross-var yarn docu-notion -n $DOCU_NOTION_INTEGRATION_TOKEN -r $DOCU_NOTION_DEV_ROOT_PAGE -m ./docs/dev -i ./static/notion_imgs -p /notion_imgs/", |
| 77 | + "pull:contracts": "cross-var yarn docu-notion -n $DOCU_NOTION_INTEGRATION_TOKEN -r $DOCU_NOTION_CONTRACTS_ROOT_PAGE -m ./docs/contracts -i ./static/notion_imgs -p /notion_imgs/", |
| 78 | + "pull:all": "yarn clear-docs && yarn pull:concepts && yarn pull:dev && yarn pull:contracts" |
| 79 | + } |
| 80 | +} |
| 81 | +``` |
| 82 | + |
| 83 | +Each script: |
| 84 | +- Uses `cross-var` for cross-platform environment variable support |
| 85 | +- Specifies target directory for markdown files (`-m`) |
| 86 | +- Configures image storage location (`-i`) and public path (`-p`) |
| 87 | + |
| 88 | +### Docu-notion Configuration |
| 89 | + |
| 90 | +The `.docu-notion.json` file controls how content is pulled from Notion and organized in the documentation: |
| 91 | + |
| 92 | +```javascript |
| 93 | +{ |
| 94 | + "sections": [ |
| 95 | + { |
| 96 | + "rootPage": "DOCU_NOTION_CONCEPT_ROOT_PAGE", |
| 97 | + "targetDirectory": "docs/concepts", |
| 98 | + "title": "Concepts", |
| 99 | + "preserveHierarchy": true |
| 100 | + }, |
| 101 | + { |
| 102 | + "rootPage": "DOCU_NOTION_DEV_ROOT_PAGE", |
| 103 | + "targetDirectory": "docs/dev", |
| 104 | + "title": "Dev Tools", |
| 105 | + "preserveHierarchy": true |
| 106 | + }, |
| 107 | + { |
| 108 | + "rootPage": "DOCU_NOTION_CONTRACTS_ROOT_PAGE", |
| 109 | + "targetDirectory": "docs/contracts", |
| 110 | + "title": "Contracts", |
| 111 | + "preserveHierarchy": true |
| 112 | + } |
| 113 | + ], |
| 114 | + "cleanUrls": true, |
| 115 | + "downloadImages": true, |
| 116 | + "customTransformers": [], |
| 117 | + "debug": true |
| 118 | +} |
| 119 | +``` |
| 120 | + |
| 121 | +This configuration: |
| 122 | +- **Sections**: Defines multiple documentation sections, each with: |
| 123 | + - `rootPage`: Environment variable for the Notion root page ID |
| 124 | + - `targetDirectory`: Where markdown files will be generated |
| 125 | + - `title`: Section title used in navigation |
| 126 | + - `preserveHierarchy`: Maintains Notion's page structure in the output |
| 127 | + |
| 128 | +- **Global Settings**: |
| 129 | + - `cleanUrls`: Generates URL-friendly filenames |
| 130 | + - `downloadImages`: Automatically downloads and processes Notion images |
| 131 | + - `debug`: Enables detailed logging during content pulling |
| 132 | + - `customTransformers`: Array for custom content transformation rules |
| 133 | + |
| 134 | +### Sidebar Configuration |
| 135 | + |
| 136 | +The `sidebars.js` file is configured to handle all three documentation sections: |
| 137 | + |
| 138 | +```javascript |
| 139 | +const sidebars = { |
| 140 | + docs: [ |
| 141 | + { |
| 142 | + type: 'autogenerated', |
| 143 | + dirName: '.' |
| 144 | + } |
| 145 | + ], |
| 146 | + devSidebar: [ |
| 147 | + { |
| 148 | + type: 'autogenerated', |
| 149 | + dirName: '.' |
| 150 | + } |
| 151 | + ], |
| 152 | + contractsSidebar: [ |
| 153 | + { |
| 154 | + type: 'autogenerated', |
| 155 | + dirName: '.' |
| 156 | + } |
| 157 | + ] |
| 158 | +}; |
| 159 | + |
| 160 | +module.exports = sidebars; |
| 161 | +``` |
| 162 | + |
| 163 | +This configuration: |
| 164 | +- Uses `autogenerated` type to automatically create sidebars from directory structure |
| 165 | +- `docs` sidebar for main concepts documentation |
| 166 | +- `devSidebar` for developer tools documentation |
| 167 | +- `contractsSidebar` for contracts documentation |
| 168 | +- Each sidebar uses `dirName: '.'` to reference its root directory as configured in the Docusaurus plugins |
| 169 | + |
| 170 | + |
| 171 | +### Docusaurus Configuration |
| 172 | + |
| 173 | +1. **Main Docs (Concepts)** |
| 174 | + ```javascript |
| 175 | + docs: { |
| 176 | + path: 'docs/concepts', |
| 177 | + routeBasePath: '/', |
| 178 | + sidebarPath: require.resolve('./sidebars.js'), |
| 179 | + } |
| 180 | + ``` |
| 181 | + |
| 182 | +2. **Dev Tools** |
| 183 | + ```javascript |
| 184 | + { |
| 185 | + id: 'dev', |
| 186 | + path: 'docs/dev', |
| 187 | + routeBasePath: 'dev', |
| 188 | + } |
| 189 | + ``` |
| 190 | + |
| 191 | +3. **Contracts** |
| 192 | + ```javascript |
| 193 | + { |
| 194 | + id: 'contracts', |
| 195 | + path: 'docs/contracts', |
| 196 | + routeBasePath: 'contracts', |
| 197 | + } |
| 198 | + ``` |
| 199 | + |
| 200 | +### Branch Structure |
| 201 | + |
| 202 | +- `main` branch: Contains source code and configuration |
| 203 | +- `gh-pages` branch: Contains: |
| 204 | + - Built static site in root |
| 205 | + - Raw Markdown files in respective directories |
| 206 | + - Static assets and images |
| 207 | + |
| 208 | +## Troubleshooting Common Issues |
| 209 | + |
| 210 | +1. **Missing Content** |
| 211 | + - Check Notion page permissions |
| 212 | + - Verify environment variables are set correctly |
| 213 | + - Review GitHub Actions logs for pull errors |
| 214 | + |
| 215 | +2. **Broken Links** |
| 216 | + - Ensure Notion links are properly formatted |
| 217 | + - Check for case sensitivity in file paths |
| 218 | + - Verify route configuration in `docusaurus.config.js` |
| 219 | + |
| 220 | +3. **Sidebar Issues** |
| 221 | + - Confirm sidebar IDs match plugin IDs |
| 222 | + - Check directory structure matches configuration |
| 223 | + - Verify frontmatter in Markdown files |
| 224 | + |
| 225 | +## Best Practices |
| 226 | + |
| 227 | +1. **Content Management** |
| 228 | + - Make all content changes in Notion - future builds will overwrite any content changes made in Github |
| 229 | + - Use consistent page hierarchy |
| 230 | + - Follow Notion formatting guidelines |
| 231 | + |
| 232 | +2. **Development** |
| 233 | + - Test changes locally before pushing |
| 234 | + - Monitor GitHub Actions logs |
| 235 | + - Keep configuration files in sync |
| 236 | + |
| 237 | +3. **Deployment** |
| 238 | + - Review built content before merging to main |
| 239 | + - Monitor deployment logs |
| 240 | + - Verify changes on the live site |
0 commit comments