Skip to content

Commit fe20d28

Browse files
author
Kagamihara Nadeshiko
committed
Added docs for LinguaFrancaMultiTargetUtils, routing, and Maintenance
1 parent de5c63e commit fe20d28

File tree

1 file changed

+303
-7
lines changed

1 file changed

+303
-7
lines changed

‎docs/developer/website-development.mdx

+303-7
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,15 @@
11
---
22
title: Website Development
33
description: Development of the Lingua Franca website.
4+
toc_max_heading_level: 5
45
---
56

7+
import {
8+
LanguageSelector,
9+
NoSelectorTargetCodeBlock,
10+
ShowIf, ShowIfs, ShowOnly,
11+
DynamicMultiTargetCodeblock,
12+
} from '@site/src/components/LinguaFrancaMultiTargetUtils';
613

714
## Getting Started
815

@@ -28,16 +35,305 @@ Highlighting is handled by two separate plugins:
2835

2936
For languages other than Lingua Franca, highlighting is handled by Prism, which is included in Docusaurus. You can find more details in Docusaurus documentation.
3037

31-
For Lingua Franca, highlighting is handled by [Shikiji](https://github.com/antfu/shikiji). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism
38+
For Lingua Franca, highlighting is handled by [Shiki](https://github.com/shikijs/shiki). This is mainly because we only have TextMate grammar definition and it is not easy to convert from it to Prism.
3239

33-
Like Prism, we are running Shikiji in browser and the code is updated in browser by React after it is highlighted. Shikiji has a relatively large footprint, so its not necessarily encourage to run in browsers. Yet running it in browser might help SEO and makes maintenance easier.
40+
Like Prism, we are running Shiki in browser and the code is updated in browser by React after it is highlighted. Shiki has a relatively large footprint, so its not necessarily encourage to run in browsers. Yet running it in browser might help SEO and makes maintenance easier.
3441

35-
The component we created is `ShikijiLFHighlighter` in `src/components/ShikijiLFHighlighter/index.tsx`. As Shikiji highlights code asynchorously, the component will display the unhighlighted code first, and asynchronously update the HTML to be the one with highlighted code with React.
42+
The component we created is `ShikiLFHighlighter` in `src/components/ShikiLFHighlighter/index.tsx`. As Shiki highlights code asynchorously, the component will display the unhighlighted code first (for a very short period of time), and asynchronously update the HTML to be the one with highlighted code with React. This will not hurt SEO as the highlight will not be considered by search engines.
3643

37-
Upon loading the webpage, Shikiji will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikijiLFHighlighter/shikijiloader.ts`.
44+
Upon loading the webpage, Shiki will be initialised in the browser, loading grammars of all target languages and Lingua Franca, as configured in `clientModules` section in `docusaurus.config.js` and `src/components/ShikiLFHighlighter/shikiloader.ts`.
3845

39-
We further swizzled Docusaurus' `CodeBlock` to use `ShikijiLFHighlighter` if the code language matches a pattern that indicates the code language is Lingua Franca, so using ``` notation in Markdown with language `lf` could give us the correct highlight result.
46+
We further swizzled Docusaurus' `CodeBlock` to use `ShikiLFHighlighter` if the code language matches a pattern that indicates the code language is Lingua Franca, so using ``` notation in Markdown with language `lf` could give us the correct highlight result. You can check `src/theme/CodeBlock/index.js` for more details.
4047

41-
:::info
48+
:::note
4249
Unlike the custom syntax highlighter in our old website, without specifying `target C`, the highlighter will not be able to highlight target language code correctly as it lacks context - even if the language speficied is `lf-c`. This will be solved in future updates to the highlighter.
43-
:::
50+
:::
51+
52+
#### Syntaxes for Lingua Franca Syntax Highlighting
53+
##### In Markdown
54+
55+
In Markdown, the only possible way to highlight Lingua Franca code is to use the ``` notation with `lf-*` language designation, e.g.:
56+
`````markdown
57+
```lf-c
58+
target C;
59+
main reactor Troll {
60+
// ....
61+
}
62+
```
63+
`````
64+
65+
##### In MDX
66+
67+
Apart from the markdown syntax, you can also use `<CodeBlock />` component which is provided by Docusaurus. You can check [here](https://docusaurus.io/docs/markdown-features/code-blocks#usage-in-jsx) for more notermation.
68+
69+
Alternatively, we have created a bunch of useful utility components to aid the target selection.
70+
71+
### Multi-Target Utilities
72+
73+
Our whole Multi-Target infrastructure is built on top of docusaurus' [Tabs](https://docusaurus.io/docs/markdown-features/tabs) component.
74+
75+
In general, different targets are just a bunch of `Tab` components with `groupId='target-languages'`.
76+
77+
#### LanguageSelector
78+
The target selector, which appears on the top, is a `Tab` component with no content. It is defined in `src/components/LinguaFrancaMultiTargetUtils/LanguageSelector.tsx`.
79+
80+
##### Usage
81+
`<LanguageSelector c cpp py rs ts />` where the target languages could be a subset of all targets which we support. The ordering is guaranteed to be `C C++ Python Rust TypeScript`, regardless of the ordering supplied as argument.
82+
83+
##### Examples
84+
`<LanguageSelector rs c ts />`
85+
:::note[Rendered as]
86+
87+
<LanguageSelector rs c ts />
88+
89+
:::
90+
91+
`<LanguageSelector rs c ts cpp py />`
92+
:::note[Rendered as]
93+
94+
<LanguageSelector rs c ts cpp py />
95+
96+
:::
97+
98+
#### LangSpecific
99+
`LangSpecific` generates a `Tab` and a bunch of `TabItem`. It is defined in `src/components/LinguaFrancaMultiTargetUtils/LangSpecific.tsx`.
100+
101+
##### Usage
102+
It's not really used as a group of `ShowIfs` + `ShowIf` would look better, or, if codes are to be displayed, you should use `NoSelectorTargetCodeBlock`. In general, it looks like `<LangSpecific c={<div>hello</div>} cpp={<div>std::hello</div>}/>` where, for the props, the keys are target languages, and values are `ReactNode` to be displayed.
103+
104+
#### NoSelectorTargetCodeBlock
105+
`NoSelectorTargetCodeBlock` is shorthand for `LangSpecific` with `CodeBlock` or `ShikiLFHighlighter` inside of it.
106+
107+
##### Usage
108+
`NoSelectorTargetCodeBlock` takes this type of props:
109+
```
110+
interface prop {
111+
[...target]: string;
112+
lf: boolean;
113+
}
114+
```
115+
Where the target languages as key could be a subset of all targets, and the value should be the code to be rendered.
116+
And when `lf=true`, the codes are rendered as Lingua Franca with target language set to the designated target language.
117+
118+
##### Examples
119+
```jsx
120+
<NoSelectorTargetCodeBlock c='printf("hi!")' cpp='std::cout<<"hi!"' rs='println!("Hi!")' />
121+
```
122+
:::note[Rendered as]
123+
124+
<NoSelectorTargetCodeBlock c='printf("hi!");' cpp='std::cout<<"hi!";' rs='println!("Hi!");' />
125+
126+
:::
127+
128+
```jsx
129+
<NoSelectorTargetCodeBlock lf c='target C;' cpp='target Cpp;' rs='target Rust;' ts='target TypeScript;' />
130+
```
131+
:::note[Rendered as]
132+
133+
<NoSelectorTargetCodeBlock lf c='target C;' cpp='target Cpp;' rs='target Rust;' ts='target TypeScript;' />
134+
135+
:::
136+
137+
#### ShowIfs/ShowIf
138+
`ShowIfs` and `ShowIf` form a `Tab` and a number of `TabItem`. They are defined in `src/components/LinguaFrancaMultiTargetUtils/ShowIf.tsx`.
139+
140+
Specifically - you MUST use `ShowIf` inside of a `ShowIfs`, and a `ShowIfs` MUST contain at least one `ShowIf` as its child.
141+
142+
##### Usage
143+
```jsx
144+
<ShowIfs>
145+
<ShowIf c>
146+
{/* Markdown contents */}
147+
</ShowIf>
148+
<ShowIf cpp py ts>
149+
{/* Markdown contents */}
150+
</ShowIf>
151+
</ShowIfs>
152+
```
153+
Where the target languages could be a subset of all targets which we support but must NOT collide.
154+
As long as you have more than one `ShowIf` inside you can have as many as you like.
155+
156+
##### Example
157+
```jsx
158+
<ShowIfs>
159+
<ShowIf c>
160+
This is shown when the C target is chosen.
161+
</ShowIf>
162+
<ShowIf cpp py ts>
163+
C++, Python and Typescript-related content is shown here.
164+
</ShowIf>
165+
</ShowIfs>
166+
{/* No Rust here! If you choose Rust from the language selector, you will get what you chose before you chose Rust. */}
167+
```
168+
169+
:::note[Rendered as]
170+
171+
<ShowIfs>
172+
<ShowIf c>
173+
This is shown when the C target is chosen.
174+
</ShowIf>
175+
<ShowIf cpp py ts>
176+
C++, Python and Typescript-related content is shown here.
177+
</ShowIf>
178+
</ShowIfs>
179+
180+
:::
181+
182+
#### ShowOnly
183+
184+
:::warning
185+
`ShowOnly` uses hacky Docusaurus-internal methods for the time being. To ensure better compatibility, you should prefer `ShowIf`.
186+
:::
187+
188+
Sometimes, you want to display a piece of information only for one language. While you can use `ShowIfs` and one `ShowIf` inside of it, the issue people typically discover is that the rendered result looks miserable, because each `Tab` will add some sort of padding above and below it.
189+
190+
For this, we built `ShowOnly` which reads the selected tab from `@docusaurus/theme-common/internal/useTabs` and displays the information inside if the target selected matches, and the output is a `<div>` or a `<span>` so that it looks like it's blended in. This is hacky and could break any time if docusaurus decides to change how their `Tabs` work.
191+
192+
##### Usage
193+
```jsx
194+
<ShowOnly c cpp
195+
inline>
196+
{/* Markdown contents */}
197+
</ShowOnly>
198+
```
199+
Where the target languages could be a subset of all targets which we support.
200+
If you include `inline` it will result in a `<span>`, if not, a `<div>`.
201+
202+
##### Examples
203+
```jsx
204+
The language which you used yells:
205+
<ShowOnly c cpp >Segfault!!!!!!</ShowOnly>
206+
```
207+
208+
:::note[Rendered as]
209+
210+
The language which you used yells:
211+
<ShowOnly c cpp >Segfault!!!!!!</ShowOnly>
212+
213+
:::
214+
215+
```jsx
216+
I am <ShowOnly c cpp inline>not</ShowOnly> memory safe!
217+
```
218+
219+
:::note[Rendered as]
220+
221+
I am <ShowOnly c cpp inline>not</ShowOnly> memory safe!
222+
223+
:::
224+
225+
#### ShowIfsInline
226+
227+
:::warning
228+
`ShowIfsInline` uses hacky Docusaurus-internal methods for the time being. To ensure better compatibility, you should prefer `ShowIf`.
229+
:::
230+
231+
:::danger
232+
TBD, I didn't find any usage for now; maybe drop it?
233+
:::
234+
235+
236+
### Code Importation
237+
238+
Unlike the previous website where a preprocessor is needed for the code to be displayed correctly, for this website we used MDX with Webpack loader. This allows us to "import" the Lingua Franca source code as a string.
239+
240+
To ensure proper imporation in Webpack 5 (there are other ways, by adding magic after the filename, in Webpack 4), in `docusaurus.config.ts` we have this small plugin:
241+
242+
```ts
243+
plugins: [
244+
() => ({
245+
name: 'read-lf-source-code-files',
246+
configureWebpack: () => ({
247+
module: {
248+
rules: [
249+
{
250+
// Any LF file that lies in path where some directory is named "codes"
251+
test: /codes?\/.*\.lf$/,
252+
type: "asset/source",
253+
},
254+
],
255+
},
256+
})
257+
}),
258+
],
259+
```
260+
261+
With this, if you use the statement like `import C_Schedule from '../assets/code/c/src/Schedule.lf';` you will get a TS string which Webpack loads when building. Then, you could use [`NoSelectorTargetCodeBlock`](#noselectortargetcodeblock) to show a bunch of lf code.
262+
263+
:::info
264+
Remember to use relative import! If you don't, you will mess up the versioning.
265+
:::
266+
267+
#### Example with NoSelectorTargetCodeBlock
268+
269+
```tsx
270+
import C_HelloWorld from '../assets/code/c/src/HelloWorld.lf';
271+
import Cpp_HelloWorld from '../assets/code/cpp/src/HelloWorld.lf';
272+
import Py_HelloWorld from '../assets/code/py/src/HelloWorld.lf';
273+
import Rs_HelloWorld from '../assets/code/rs/src/HelloWorld.lf';
274+
import TS_HelloWorld from '../assets/code/ts/src/HelloWorld.lf';
275+
276+
<NoSelectorTargetCodeBlock c={C_HelloWorld} cpp={Cpp_HelloWorld} py={Py_HelloWorld} rs={Rs_HelloWorld} ts={TS_HelloWorld} lf />
277+
```
278+
279+
:::note[Rendered as]
280+
281+
import C_HelloWorld from '../assets/code/c/src/HelloWorld.lf';
282+
import Cpp_HelloWorld from '../assets/code/cpp/src/HelloWorld.lf';
283+
import Py_HelloWorld from '../assets/code/py/src/HelloWorld.lf';
284+
import Rs_HelloWorld from '../assets/code/rs/src/HelloWorld.lf';
285+
import TS_HelloWorld from '../assets/code/ts/src/HelloWorld.lf';
286+
287+
<NoSelectorTargetCodeBlock c={C_HelloWorld} cpp={Cpp_HelloWorld} py={Py_HelloWorld} rs={Rs_HelloWorld} ts={TS_HelloWorld} lf />
288+
289+
:::
290+
291+
#### DynamicMultiTargetCodeblock
292+
293+
:::danger
294+
This is not working properly with versioning and will have issues with Webpack performance!
295+
:::
296+
297+
:::danger
298+
This is not working properly now! Need to fix the file path in `content` in `DynamicMultiTargetCodeblock.tsx`.
299+
:::
300+
301+
If you are **really** lazy, there is a utility which I wrote, `DynamicMultiTargetCodeblock`, that allows you to only supply a LF filename and all files will be dynamically imported. The issue is Webpack is not intelligent enough so if you use it, it will probably load all LF files in `assets`. Also, because relative import appears to be impossible, using it will always get you the latest code examples. See `src/components/LinguaFrancaMultiTargetUtils/DynamicMultiTargetCodeblock.tsx`.
302+
303+
##### Examples
304+
305+
```tsx
306+
<DynamicMultiTargetCodeblock c cpp ts rs py file="HelloWorld" />
307+
```
308+
309+
To suppress warning, add
310+
311+
```tsx
312+
<DynamicMultiTargetCodeblock doNotTransformInMDX c cpp ts rs py file="HelloWorld" />
313+
```
314+
315+
#### DynamicMultiTargetCodeblock with TransformDynamicLFFileImportToStatic
316+
317+
:::danger
318+
This is an experiment and should not be used in production!
319+
:::
320+
321+
A way to make `DynamicMultiTargetCodeblock` more sane is to use remark and preprocess it into a bunch of import statements and a `NoSelectorTargetCodeBlock`.
322+
323+
`src/remark/TransformDynamicLFFileImportToStatic.ts` is a remark plugin that does so. It looks like a disaster and I don't understand it anymore 5 months after writing it.
324+
325+
To use it, add it to `docusaurus.config.ts` as a ReMark preprocessor.
326+
327+
But don't use it.
328+
329+
## Routing from the Legacy Site
330+
331+
`docs/legacy_routing.ts` contains the old-to-new URL mappings which allows people to access the correct content if they used a URL from the old website. It is then imported in `docusaurus.config.ts` For more, check `@docusaurus/plugin-client-redirects`.
332+
333+
## Maintenance, and point-of-contact
334+
335+
For docusaurus-related question, you should redirect your question to [them](https://github.com/facebook/docusaurus).
336+
337+
The LF syntax file is fetched [here](https://github.com/lf-lang/vscode-lingua-franca/blob/main/syntaxes/lflang.tmLanguage.json) and should be updated from time to time. If the syntax highlighting is not working properly, check the TextMate grammar, then check with [shiki](https://github.com/shikijs/shiki).
338+
339+
For website-specific utilities, you can open an issue on GitHub or [ask @axmmisaka directly](/community). The TS code quality is not great and needs to be improved.

0 commit comments

Comments
 (0)