From 7b1c102626a5619ada023e7e8d72e684f1cc66f4 Mon Sep 17 00:00:00 2001 From: Kagamihara Nadeshiko Date: Sat, 16 Dec 2023 03:21:17 +0000 Subject: [PATCH 01/11] Docs on new website - WIP --- docs/developer/website-development.mdx | 29 ++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index be3c66a74..e952baa9e 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -3,6 +3,7 @@ title: Website Development description: Development of the Lingua Franca website. --- + ## Getting Started First, for simple changes to the website, such as fixing typos, the easiest way is to scroll to the bottom of the page, follow the link at the bottom to send a pull request, edit the resulting page, and issue a pull request. @@ -12,3 +13,31 @@ For more elaborate changes, including adding new pages, you will need to clone t ## Editing the Handbook The handbook is the most updated part of the website and it includes quite a bit of infrastructure to support writing pages that describe features in any or all of the target languages. The root of the handbook pages in the `packages/documentation` part of the repo. That directory has a useful [README file](https://github.com/lf-lang/website-lingua-franca/blob/main/packages/documentation/README.md) that describes the structure and provides instructions for inserting code examples in any target language and target-specific text within a body of target-independent text. + +## Infrastructure + +### Docusaurus + +Our new website is built on top of [Docusaurus](https://docusaurus.io/), a documentation website generator maintained by Meta. Most of the functionalities are built with Docusaurus or plugins made for it to ensure that we spent minimal effort touching actual infrastructure code. You can find documentation on their website. Most of the website customisation can be done by tweaking configurations without touching any code with docusaurus. + +Below are some components we made to accomodate our special needs. + +### Syntax highlighting + +Highlighting is handled by two separate plugins: + +For languages other than Lingua Franca, highlighting is handled by Prism, which is included in Docusaurus. You can find more details in Docusaurus documentation. + +For Lingua Franca, highlighting is handled by [Shikiji](https://github.com/antfu/shikiji). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism + +Like Prism, we are running Shikiji in browser and the code is updated in browser by React after it is highlighted. Shikiji can, but is not encouraged to, run in browsers, since it is big in size and expensive in computation power. Yet running it in browser might help SEO and makes maintenance easier. + +The component we created is `ShikijiLFHighlighter` in `src/components/ShikijiLFHighlighter/index.tsx`. As Shikiji highlights code asynchorously, the component will display the unhighlighted code first, and asynchorously update the HTML to be the one with highlighted code with React. + +Upon loading the webpage, Shikiji will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikijiLFHighlighter/shikijiloader.ts`. + +We further swizzled Docusaurus' `CodeBlock` to use `ShikijiLFHighlighter` if the code language matches a pattern that indicates the code language is Lingua Franca, so using ``` notation in Markdown with language `lf` could give us the correct highlight result. + +:::info +Unlike the custom syntax highlighter in our old website, without specifying `target C`, the highlighter will not be able to highlight target language code correctly as it lacks context - even if the language speficied is `lf-c`. This will be solved in future updates to the highlighter. +::: \ No newline at end of file From 1aac99dd798c4e7dd7c1cbf58cae58662516155f Mon Sep 17 00:00:00 2001 From: Marten Lohstroh Date: Sat, 20 Jan 2024 10:13:43 -0800 Subject: [PATCH 02/11] Update docs/developer/website-development.mdx --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index e952baa9e..e2e2fe9ee 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -30,7 +30,7 @@ For languages other than Lingua Franca, highlighting is handled by Prism, which For Lingua Franca, highlighting is handled by [Shikiji](https://github.com/antfu/shikiji). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism -Like Prism, we are running Shikiji in browser and the code is updated in browser by React after it is highlighted. Shikiji can, but is not encouraged to, run in browsers, since it is big in size and expensive in computation power. Yet running it in browser might help SEO and makes maintenance easier. +Like Prism, we are running Shikiji in browser and the code is updated in browser by React after it is highlighted. Shikiji has a relatively large footprint, so its not necessarily encourage to run in browsers. Yet running it in browser might help SEO and makes maintenance easier. The component we created is `ShikijiLFHighlighter` in `src/components/ShikijiLFHighlighter/index.tsx`. As Shikiji highlights code asynchorously, the component will display the unhighlighted code first, and asynchorously update the HTML to be the one with highlighted code with React. From de5c63e303a16c306cfbc65b5327ecae8369602c Mon Sep 17 00:00:00 2001 From: Marten Lohstroh Date: Sat, 20 Jan 2024 10:13:50 -0800 Subject: [PATCH 03/11] Update docs/developer/website-development.mdx --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index e2e2fe9ee..264c1194b 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -32,7 +32,7 @@ For Lingua Franca, highlighting is handled by [Shikiji](https://github.com/antfu Like Prism, we are running Shikiji in browser and the code is updated in browser by React after it is highlighted. Shikiji has a relatively large footprint, so its not necessarily encourage to run in browsers. Yet running it in browser might help SEO and makes maintenance easier. -The component we created is `ShikijiLFHighlighter` in `src/components/ShikijiLFHighlighter/index.tsx`. As Shikiji highlights code asynchorously, the component will display the unhighlighted code first, and asynchorously update the HTML to be the one with highlighted code with React. +The component we created is `ShikijiLFHighlighter` in `src/components/ShikijiLFHighlighter/index.tsx`. As Shikiji highlights code asynchorously, the component will display the unhighlighted code first, and asynchronously update the HTML to be the one with highlighted code with React. Upon loading the webpage, Shikiji will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikijiLFHighlighter/shikijiloader.ts`. From fe20d286be56ef36bc00c8aad5ab8d8a0f2c8d39 Mon Sep 17 00:00:00 2001 From: Kagamihara Nadeshiko Date: Tue, 14 May 2024 08:41:29 +0000 Subject: [PATCH 04/11] Added docs for LinguaFrancaMultiTargetUtils, routing, and Maintenance --- docs/developer/website-development.mdx | 310 ++++++++++++++++++++++++- 1 file changed, 303 insertions(+), 7 deletions(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index 264c1194b..cf5471b6a 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -1,8 +1,15 @@ --- title: Website Development description: Development of the Lingua Franca website. +toc_max_heading_level: 5 --- +import { + LanguageSelector, + NoSelectorTargetCodeBlock, + ShowIf, ShowIfs, ShowOnly, + DynamicMultiTargetCodeblock, +} from '@site/src/components/LinguaFrancaMultiTargetUtils'; ## Getting Started @@ -28,16 +35,305 @@ Highlighting is handled by two separate plugins: For languages other than Lingua Franca, highlighting is handled by Prism, which is included in Docusaurus. You can find more details in Docusaurus documentation. -For Lingua Franca, highlighting is handled by [Shikiji](https://github.com/antfu/shikiji). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism +For Lingua Franca, highlighting is handled by [Shiki](https://github.com/shikijs/shiki). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism. -Like Prism, we are running Shikiji in browser and the code is updated in browser by React after it is highlighted. Shikiji has a relatively large footprint, so its not necessarily encourage to run in browsers. Yet running it in browser might help SEO and makes maintenance easier. +Like Prism, we are running Shiki in browser and the code is updated in browser by React after it is highlighted. Shiki has a relatively large footprint, so its not necessarily encourage to run in browsers. Yet running it in browser might help SEO and makes maintenance easier. -The component we created is `ShikijiLFHighlighter` in `src/components/ShikijiLFHighlighter/index.tsx`. As Shikiji highlights code asynchorously, the component will display the unhighlighted code first, and asynchronously update the HTML to be the one with highlighted code with React. +The component we created is `ShikiLFHighlighter` in `src/components/ShikiLFHighlighter/index.tsx`. As Shiki highlights code asynchorously, the component will display the unhighlighted code first (for a very short period of time), and asynchronously update the HTML to be the one with highlighted code with React. This will not hurt SEO as the highlight will not be considered by search engines. -Upon loading the webpage, Shikiji will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikijiLFHighlighter/shikijiloader.ts`. +Upon loading the webpage, Shiki will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikiLFHighlighter/shikiloader.ts`. -We further swizzled Docusaurus' `CodeBlock` to use `ShikijiLFHighlighter` if the code language matches a pattern that indicates the code language is Lingua Franca, so using ``` notation in Markdown with language `lf` could give us the correct highlight result. +We further swizzled Docusaurus' `CodeBlock` to use `ShikiLFHighlighter` if the code language matches a pattern that indicates the code language is Lingua Franca, so using ``` notation in Markdown with language `lf` could give us the correct highlight result. You can check `src/theme/CodeBlock/index.js` for more details. -:::info +:::note Unlike the custom syntax highlighter in our old website, without specifying `target C`, the highlighter will not be able to highlight target language code correctly as it lacks context - even if the language speficied is `lf-c`. This will be solved in future updates to the highlighter. -::: \ No newline at end of file +::: + +#### Syntaxes for Lingua Franca Syntax Highlighting +##### In Markdown + +In Markdown, the only possible way to highlight Lingua Franca code is to use the ``` notation with `lf-*` language designation, e.g.: +`````markdown +```lf-c +target C; +main reactor Troll { + // .... +} +``` +````` + +##### In MDX + +Apart from the markdown syntax, you can also use `` component which is provided by Docusaurus. You can check [here](https://docusaurus.io/docs/markdown-features/code-blocks#usage-in-jsx) for more notermation. + +Alternatively, we have created a bunch of useful utility components to aid the target selection. + +### Multi-Target Utilities + +Our whole Multi-Target infrastructure is built on top of docusaurus' [Tabs](https://docusaurus.io/docs/markdown-features/tabs) component. + +In general, different targets are just a bunch of `Tab` components with `groupId='target-languages'`. + +#### LanguageSelector +The target selector, which appears on the top, is a `Tab` component with no content. It is defined in `src/components/LinguaFrancaMultiTargetUtils/LanguageSelector.tsx`. + +##### Usage +`` where the target languages could be a subset of all targets which we support. The ordering is guaranteed to be `C C++ Python Rust TypeScript`, regardless of the ordering supplied as argument. + +##### Examples +`` +:::note[Rendered as] + + + +::: + +`` +:::note[Rendered as] + + + +::: + +#### LangSpecific +`LangSpecific` generates a `Tab` and a bunch of `TabItem`. It is defined in `src/components/LinguaFrancaMultiTargetUtils/LangSpecific.tsx`. + +##### Usage +It's not really used as a group of `ShowIfs` + `ShowIf` would look better, or, if codes are to be displayed, you should use `NoSelectorTargetCodeBlock`. In general, it looks like `hello} cpp={
std::hello
}/>` where, for the props, the keys are target languages, and values are `ReactNode` to be displayed. + +#### NoSelectorTargetCodeBlock +`NoSelectorTargetCodeBlock` is shorthand for `LangSpecific` with `CodeBlock` or `ShikiLFHighlighter` inside of it. + +##### Usage +`NoSelectorTargetCodeBlock` takes this type of props: +``` +interface prop { + [...target]: string; + lf: boolean; +} +``` +Where the target languages as key could be a subset of all targets, and the value should be the code to be rendered. +And when `lf=true`, the codes are rendered as Lingua Franca with target language set to the designated target language. + +##### Examples +```jsx + +``` +:::note[Rendered as] + + + +::: + +```jsx + +``` +:::note[Rendered as] + + + +::: + +#### ShowIfs/ShowIf +`ShowIfs` and `ShowIf` form a `Tab` and a number of `TabItem`. They are defined in `src/components/LinguaFrancaMultiTargetUtils/ShowIf.tsx`. + +Specifically - you MUST use `ShowIf` inside of a `ShowIfs`, and a `ShowIfs` MUST contain at least one `ShowIf` as its child. + +##### Usage +```jsx + + + {/* Markdown contents */} + + + {/* Markdown contents */} + + +``` +Where the target languages could be a subset of all targets which we support but must NOT collide. +As long as you have more than one `ShowIf` inside you can have as many as you like. + +##### Example +```jsx + + + This is shown when the C target is chosen. + + + C++, Python and Typescript-related content is shown here. + + + {/* No Rust here! If you choose Rust from the language selector, you will get what you chose before you chose Rust. */} +``` + +:::note[Rendered as] + + + + This is shown when the C target is chosen. + + + C++, Python and Typescript-related content is shown here. + + + +::: + +#### ShowOnly + +:::warning +`ShowOnly` uses hacky Docusaurus-internal methods for the time being. To ensure better compatibility, you should prefer `ShowIf`. +::: + +Sometimes, you want to display a piece of information only for one language. While you can use `ShowIfs` and one `ShowIf` inside of it, the issue people typically discover is that the rendered result looks miserable, because each `Tab` will add some sort of padding above and below it. + +For this, we built `ShowOnly` which reads the selected tab from `@docusaurus/theme-common/internal/useTabs` and displays the information inside if the target selected matches, and the output is a `
` or a `` so that it looks like it's blended in. This is hacky and could break any time if docusaurus decides to change how their `Tabs` work. + +##### Usage +```jsx + + {/* Markdown contents */} + +``` +Where the target languages could be a subset of all targets which we support. +If you include `inline` it will result in a ``, if not, a `
`. + +##### Examples +```jsx +The language which you used yells: +Segfault!!!!!! +``` + +:::note[Rendered as] + +The language which you used yells: +Segfault!!!!!! + +::: + +```jsx +I am not memory safe! +``` + +:::note[Rendered as] + +I am not memory safe! + +::: + +#### ShowIfsInline + +:::warning +`ShowIfsInline` uses hacky Docusaurus-internal methods for the time being. To ensure better compatibility, you should prefer `ShowIf`. +::: + +:::danger +TBD, I didn't find any usage for now; maybe drop it? +::: + + +### Code Importation + +Unlike the previous website where a preprocessor is needed for the code to be displayed correctly, for this website we used MDX with Webpack loader. This allows us to "import" the Lingua Franca source code as a string. + +To ensure proper imporation in Webpack 5 (there are other ways, by adding magic after the filename, in Webpack 4), in `docusaurus.config.ts` we have this small plugin: + +```ts +plugins: [ +() => ({ + name: 'read-lf-source-code-files', + configureWebpack: () => ({ + module: { + rules: [ + { + // Any LF file that lies in path where some directory is named "codes" + test: /codes?\/.*\.lf$/, + type: "asset/source", + }, + ], + }, + }) +}), +], +``` + +With this, if you use the statement like `import C_Schedule from '../assets/code/c/src/Schedule.lf';` you will get a TS string which Webpack loads when building. Then, you could use [`NoSelectorTargetCodeBlock`](#noselectortargetcodeblock) to show a bunch of lf code. + +:::info +Remember to use relative import! If you don't, you will mess up the versioning. +::: + +#### Example with NoSelectorTargetCodeBlock + +```tsx +import C_HelloWorld from '../assets/code/c/src/HelloWorld.lf'; +import Cpp_HelloWorld from '../assets/code/cpp/src/HelloWorld.lf'; +import Py_HelloWorld from '../assets/code/py/src/HelloWorld.lf'; +import Rs_HelloWorld from '../assets/code/rs/src/HelloWorld.lf'; +import TS_HelloWorld from '../assets/code/ts/src/HelloWorld.lf'; + + +``` + +:::note[Rendered as] + +import C_HelloWorld from '../assets/code/c/src/HelloWorld.lf'; +import Cpp_HelloWorld from '../assets/code/cpp/src/HelloWorld.lf'; +import Py_HelloWorld from '../assets/code/py/src/HelloWorld.lf'; +import Rs_HelloWorld from '../assets/code/rs/src/HelloWorld.lf'; +import TS_HelloWorld from '../assets/code/ts/src/HelloWorld.lf'; + + + +::: + +#### DynamicMultiTargetCodeblock + +:::danger +This is not working properly with versioning and will have issues with Webpack performance! +::: + +:::danger +This is not working properly now! Need to fix the file path in `content` in `DynamicMultiTargetCodeblock.tsx`. +::: + +If you are **really** lazy, there is a utility which I wrote, `DynamicMultiTargetCodeblock`, that allows you to only supply a LF filename and all files will be dynamically imported. The issue is Webpack is not intelligent enough so if you use it, it will probably load all LF files in `assets`. Also, because relative import appears to be impossible, using it will always get you the latest code examples. See `src/components/LinguaFrancaMultiTargetUtils/DynamicMultiTargetCodeblock.tsx`. + +##### Examples + +```tsx + +``` + +To suppress warning, add + +```tsx + +``` + +#### DynamicMultiTargetCodeblock with TransformDynamicLFFileImportToStatic + +:::danger +This is an experiment and should not be used in production! +::: + +A way to make `DynamicMultiTargetCodeblock` more sane is to use remark and preprocess it into a bunch of import statements and a `NoSelectorTargetCodeBlock`. + +`src/remark/TransformDynamicLFFileImportToStatic.ts` is a remark plugin that does so. It looks like a disaster and I don't understand it anymore 5 months after writing it. + +To use it, add it to `docusaurus.config.ts` as a ReMark preprocessor. + +But don't use it. + +## Routing from the Legacy Site + +`docs/legacy_routing.ts` contains the old-to-new URL mappings which allows people to access the correct content if they used a URL from the old website. It is then imported in `docusaurus.config.ts` For more, check `@docusaurus/plugin-client-redirects`. + +## Maintenance, and point-of-contact + +For docusaurus-related question, you should redirect your question to [them](https://github.com/facebook/docusaurus). + +The LF syntax file is fetched [here](https://github.com/lf-lang/vscode-lingua-franca/blob/main/syntaxes/lflang.tmLanguage.json) and should be updated from time to time. If the syntax highlighting is not working properly, check the TextMate grammar, then check with [shiki](https://github.com/shikijs/shiki). + +For website-specific utilities, you can open an issue on GitHub or [ask @axmmisaka directly](/community). The TS code quality is not great and needs to be improved. \ No newline at end of file From 14841a2845024c76c8c8d376a6f5460b269a06c6 Mon Sep 17 00:00:00 2001 From: axmmisaka <6500159+axmmisaka@users.noreply.github.com> Date: Wed, 5 Jun 2024 20:37:59 -0700 Subject: [PATCH 05/11] Update docs/developer/website-development.mdx Co-authored-by: Marten Lohstroh --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index cf5471b6a..82c44229a 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -25,7 +25,7 @@ The handbook is the most updated part of the website and it includes quite a bit ### Docusaurus -Our new website is built on top of [Docusaurus](https://docusaurus.io/), a documentation website generator maintained by Meta. Most of the functionalities are built with Docusaurus or plugins made for it to ensure that we spent minimal effort touching actual infrastructure code. You can find documentation on their website. Most of the website customisation can be done by tweaking configurations without touching any code with docusaurus. +Our website is built on top of [Docusaurus](https://docusaurus.io/), a documentation website generator maintained by Meta. Most of the functionalities are built with Docusaurus or plugins made for it to ensure that we spent minimal effort touching actual infrastructure code. You can find documentation on their website. Most of the website customization can be done by tweaking configurations, without touching any Docusaurus code. Below are some components we made to accomodate our special needs. From 2bae4471c1625f83750182c4d94b194caea94225 Mon Sep 17 00:00:00 2001 From: axmmisaka <6500159+axmmisaka@users.noreply.github.com> Date: Wed, 5 Jun 2024 20:38:05 -0700 Subject: [PATCH 06/11] Update docs/developer/website-development.mdx Co-authored-by: Marten Lohstroh --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index 82c44229a..445fe702e 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -37,7 +37,7 @@ For languages other than Lingua Franca, highlighting is handled by Prism, which For Lingua Franca, highlighting is handled by [Shiki](https://github.com/shikijs/shiki). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism. -Like Prism, we are running Shiki in browser and the code is updated in browser by React after it is highlighted. Shiki has a relatively large footprint, so its not necessarily encourage to run in browsers. Yet running it in browser might help SEO and makes maintenance easier. +Like Prism, we are running Shiki in browser and the code is updated in browser by React after it is highlighted. Shiki has a relatively large footprint, so its not necessarily encouraged to run in browsers. Yet running it in browser might help SEO and makes maintenance easier. The component we created is `ShikiLFHighlighter` in `src/components/ShikiLFHighlighter/index.tsx`. As Shiki highlights code asynchorously, the component will display the unhighlighted code first (for a very short period of time), and asynchronously update the HTML to be the one with highlighted code with React. This will not hurt SEO as the highlight will not be considered by search engines. From 6d1d92ee6ecb05a17c4975bc54bdb252dcc41a5e Mon Sep 17 00:00:00 2001 From: axmmisaka <6500159+axmmisaka@users.noreply.github.com> Date: Wed, 5 Jun 2024 20:38:09 -0700 Subject: [PATCH 07/11] Update docs/developer/website-development.mdx Co-authored-by: Marten Lohstroh --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index 445fe702e..4483bdf88 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -64,7 +64,7 @@ main reactor Troll { ##### In MDX -Apart from the markdown syntax, you can also use `` component which is provided by Docusaurus. You can check [here](https://docusaurus.io/docs/markdown-features/code-blocks#usage-in-jsx) for more notermation. +Apart from the markdown syntax, you can also use `` component which is provided by Docusaurus. You can check [here](https://docusaurus.io/docs/markdown-features/code-blocks#usage-in-jsx) for more details. Alternatively, we have created a bunch of useful utility components to aid the target selection. From 8490685e1512b6aeff5701aeb83388a018098750 Mon Sep 17 00:00:00 2001 From: axmmisaka <6500159+axmmisaka@users.noreply.github.com> Date: Wed, 5 Jun 2024 20:38:14 -0700 Subject: [PATCH 08/11] Update docs/developer/website-development.mdx Co-authored-by: Marten Lohstroh --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index 4483bdf88..ff7822f4c 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -66,7 +66,7 @@ main reactor Troll { Apart from the markdown syntax, you can also use `` component which is provided by Docusaurus. You can check [here](https://docusaurus.io/docs/markdown-features/code-blocks#usage-in-jsx) for more details. -Alternatively, we have created a bunch of useful utility components to aid the target selection. +We have also created a bunch of useful utility components to aid the target selection. ### Multi-Target Utilities From 413b98e72ddfd37b1659c67bf7718722815cb09f Mon Sep 17 00:00:00 2001 From: axmmisaka <6500159+axmmisaka@users.noreply.github.com> Date: Wed, 5 Jun 2024 20:38:29 -0700 Subject: [PATCH 09/11] Update docs/developer/website-development.mdx Co-authored-by: Marten Lohstroh --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index ff7822f4c..782301ff3 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -235,7 +235,7 @@ TBD, I didn't find any usage for now; maybe drop it? ### Code Importation -Unlike the previous website where a preprocessor is needed for the code to be displayed correctly, for this website we used MDX with Webpack loader. This allows us to "import" the Lingua Franca source code as a string. +Unlike the previous website that used a preprocessor to correctly display the code, this website we uses MDX with Webpack loader. This allows us to "import" the Lingua Franca source code as a string. To ensure proper imporation in Webpack 5 (there are other ways, by adding magic after the filename, in Webpack 4), in `docusaurus.config.ts` we have this small plugin: From 5d50b18ea83138873f35243e519d13d9248fde7e Mon Sep 17 00:00:00 2001 From: axmmisaka <6500159+axmmisaka@users.noreply.github.com> Date: Wed, 5 Jun 2024 20:38:58 -0700 Subject: [PATCH 10/11] Update docs/developer/website-development.mdx Co-authored-by: Marten Lohstroh --- docs/developer/website-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index 782301ff3..7ec388c8a 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -336,4 +336,4 @@ For docusaurus-related question, you should redirect your question to [them](htt The LF syntax file is fetched [here](https://github.com/lf-lang/vscode-lingua-franca/blob/main/syntaxes/lflang.tmLanguage.json) and should be updated from time to time. If the syntax highlighting is not working properly, check the TextMate grammar, then check with [shiki](https://github.com/shikijs/shiki). -For website-specific utilities, you can open an issue on GitHub or [ask @axmmisaka directly](/community). The TS code quality is not great and needs to be improved. \ No newline at end of file +For website-specific utilities, you can open an issue on GitHub or [ask @axmmisaka directly](/community). \ No newline at end of file From 8511fbc02b7bfed65b295397cb7d2659b2b9b5c2 Mon Sep 17 00:00:00 2001 From: Peter Donovan Date: Tue, 2 Jul 2024 14:55:05 -0700 Subject: [PATCH 11/11] Address review comments regarding website docs --- docs/developer/website-development.mdx | 24 ++++++------- .../DynamicMultiTargetCodeblock.tsx | 34 +++++++++++++++++++ 2 files changed, 46 insertions(+), 12 deletions(-) diff --git a/docs/developer/website-development.mdx b/docs/developer/website-development.mdx index 7ec388c8a..168a5732e 100644 --- a/docs/developer/website-development.mdx +++ b/docs/developer/website-development.mdx @@ -19,7 +19,7 @@ For more elaborate changes, including adding new pages, you will need to clone t ## Editing the Handbook -The handbook is the most updated part of the website and it includes quite a bit of infrastructure to support writing pages that describe features in any or all of the target languages. The root of the handbook pages in the `packages/documentation` part of the repo. That directory has a useful [README file](https://github.com/lf-lang/website-lingua-franca/blob/main/packages/documentation/README.md) that describes the structure and provides instructions for inserting code examples in any target language and target-specific text within a body of target-independent text. +The handbook is the most updated part of the website and it includes quite a bit of infrastructure to support writing pages that describe features in any or all of the target languages. The root of the handbook pages is the `packages/documentation` part of the repo. That directory has a useful [README file](https://github.com/lf-lang/website-lingua-franca/blob/main/packages/documentation/README.md) that describes the structure and provides instructions for inserting code examples in any target language and target-specific text within a body of target-independent text. ## Infrastructure @@ -37,16 +37,16 @@ For languages other than Lingua Franca, highlighting is handled by Prism, which For Lingua Franca, highlighting is handled by [Shiki](https://github.com/shikijs/shiki). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism. -Like Prism, we are running Shiki in browser and the code is updated in browser by React after it is highlighted. Shiki has a relatively large footprint, so its not necessarily encouraged to run in browsers. Yet running it in browser might help SEO and makes maintenance easier. +Like Prism, we are running Shiki in the browser, and the code is updated in the browser by React after it is highlighted. Shiki has a relatively large footprint, so its not necessarily encouraged to run in browsers. However, running it in browser might help SEO and makes maintenance easier. -The component we created is `ShikiLFHighlighter` in `src/components/ShikiLFHighlighter/index.tsx`. As Shiki highlights code asynchorously, the component will display the unhighlighted code first (for a very short period of time), and asynchronously update the HTML to be the one with highlighted code with React. This will not hurt SEO as the highlight will not be considered by search engines. +The component we created is `ShikiLFHighlighter` in `src/components/ShikiLFHighlighter/index.tsx`. As Shiki highlights code asynchronously, the component will display the unhighlighted code first (for a very short period of time), and then asynchronously update the HTML with highlighted code using React. This will not hurt SEO because the highlight will not be considered by search engines. -Upon loading the webpage, Shiki will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikiLFHighlighter/shikiloader.ts`. +Upon loading the webpage, Shiki will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikiLFHighlighter/shikiloader.ts`. -We further swizzled Docusaurus' `CodeBlock` to use `ShikiLFHighlighter` if the code language matches a pattern that indicates the code language is Lingua Franca, so using ``` notation in Markdown with language `lf` could give us the correct highlight result. You can check `src/theme/CodeBlock/index.js` for more details. +We further changed Docusaurus' `CodeBlock` to use `ShikiLFHighlighter` if the code language matches a pattern that indicates the code language is Lingua Franca, so using ``` notation in Markdown with language `lf` could give us the correct highlight result. You can check `src/theme/CodeBlock/index.js` for more details. :::note -Unlike the custom syntax highlighter in our old website, without specifying `target C`, the highlighter will not be able to highlight target language code correctly as it lacks context - even if the language speficied is `lf-c`. This will be solved in future updates to the highlighter. +Unlike the custom syntax highlighter in our old website, without specifying `target C`, the highlighter will not be able to highlight target language code correctly as it lacks context - even if the language specified is `lf-c`. This will be solved in future updates to the highlighter. ::: #### Syntaxes for Lingua Franca Syntax Highlighting @@ -70,9 +70,9 @@ We have also created a bunch of useful utility components to aid the target sele ### Multi-Target Utilities -Our whole Multi-Target infrastructure is built on top of docusaurus' [Tabs](https://docusaurus.io/docs/markdown-features/tabs) component. +Our whole Multi-Target infrastructure is built on top of Docusaurus' [Tabs](https://docusaurus.io/docs/markdown-features/tabs) component. -In general, different targets are just a bunch of `Tab` components with `groupId='target-languages'`. +In general, different targets are just a bunch of `Tab` components with `groupId='target-languages'`. #### LanguageSelector The target selector, which appears on the top, is a `Tab` component with no content. It is defined in `src/components/LinguaFrancaMultiTargetUtils/LanguageSelector.tsx`. @@ -150,7 +150,7 @@ Specifically - you MUST use `ShowIf` inside of a `ShowIfs`, and a `ShowIfs` MUST ``` -Where the target languages could be a subset of all targets which we support but must NOT collide. +Where the target languages could be a subset of all targets which we support but must NOT collide. As long as you have more than one `ShowIf` inside you can have as many as you like. ##### Example @@ -185,7 +185,7 @@ As long as you have more than one `ShowIf` inside you can have as many as you li `ShowOnly` uses hacky Docusaurus-internal methods for the time being. To ensure better compatibility, you should prefer `ShowIf`. ::: -Sometimes, you want to display a piece of information only for one language. While you can use `ShowIfs` and one `ShowIf` inside of it, the issue people typically discover is that the rendered result looks miserable, because each `Tab` will add some sort of padding above and below it. +Sometimes, you want to display a piece of information only for one language. While you can use `ShowIfs` and one `ShowIf` inside of it, the issue people typically discover is that the rendered result looks miserable, because each `Tab` will add some sort of padding above and below it. For this, we built `ShowOnly` which reads the selected tab from `@docusaurus/theme-common/internal/useTabs` and displays the information inside if the target selected matches, and the output is a `
` or a `` so that it looks like it's blended in. This is hacky and could break any time if docusaurus decides to change how their `Tabs` work. @@ -196,7 +196,7 @@ For this, we built `ShowOnly` which reads the selected tab from `@docusaurus/the {/* Markdown contents */} ``` -Where the target languages could be a subset of all targets which we support. +Where the target languages could be a subset of all targets which we support. If you include `inline` it will result in a ``, if not, a `
`. ##### Examples @@ -336,4 +336,4 @@ For docusaurus-related question, you should redirect your question to [them](htt The LF syntax file is fetched [here](https://github.com/lf-lang/vscode-lingua-franca/blob/main/syntaxes/lflang.tmLanguage.json) and should be updated from time to time. If the syntax highlighting is not working properly, check the TextMate grammar, then check with [shiki](https://github.com/shikijs/shiki). -For website-specific utilities, you can open an issue on GitHub or [ask @axmmisaka directly](/community). \ No newline at end of file +For website-specific utilities, you can open an issue on GitHub or [ask @axmmisaka directly](/community). diff --git a/src/components/LinguaFrancaMultiTargetUtils/DynamicMultiTargetCodeblock.tsx b/src/components/LinguaFrancaMultiTargetUtils/DynamicMultiTargetCodeblock.tsx index 65d32a789..0b7fe42c3 100644 --- a/src/components/LinguaFrancaMultiTargetUtils/DynamicMultiTargetCodeblock.tsx +++ b/src/components/LinguaFrancaMultiTargetUtils/DynamicMultiTargetCodeblock.tsx @@ -1,3 +1,37 @@ +/** + * #### DynamicMultiTargetCodeblock + * + * Warnings: + * - This is not working properly with versioning and will have issues with Webpack performance! + * - This is not working properly now! Need to fix the file path in `content` in `DynamicMultiTargetCodeblock.tsx`. + * + * This utility allows you to only supply a LF filename and all files will be dynamically imported. The issue is Webpack is not intelligent enough so if you use it, it will probably load all LF files in `assets`. Also, because relative import appears to be impossible, using it will always get you the latest code examples. See `src/components/LinguaFrancaMultiTargetUtils/DynamicMultiTargetCodeblock.tsx`. + * + * ##### Examples + * + * ```tsx + * + * ``` + * + * To suppress warnings, add + * + * ```tsx + * + * ``` + * + * #### DynamicMultiTargetCodeblock with TransformDynamicLFFileImportToStatic + * + * Warning: This is an experiment and should not be used in production! + * + * A way to make `DynamicMultiTargetCodeblock` more sane is to use remark and preprocess it into a bunch of import statements and a `NoSelectorTargetCodeBlock`. + * + * `src/remark/TransformDynamicLFFileImportToStatic.ts` is a remark plugin that does so. It looks like a disaster and I don't understand it anymore 5 months after writing it. + * + * To use DynamicMultiTargetCodeBlock, add it to `docusaurus.config.ts` as a ReMark preprocessor. + * + * But don't use it.` + */ + import React, { useState, useEffect } from "react"; import { TargetsType } from "./index"; import { NoSelectorTargetCodeBlock } from "./LangSpecific";