internals doc

website/src/pages/internals.mdx

@@ -0,0 +1,245 @@
+---
+title: Recap
+layout: ../layouts/doc.astro
+---
+
+import { MiniRepl } from "../components/MiniRepl";
+import { Icon } from "../components/Icon";
+import Box from "../components/Box.astro";
+import DocLayout from "../layouts/doc.astro";
+
+# How kabelsalat works
+
+This site explains how kabelsalat works under the hood. **You don't need to know this to use it!**
+
+## Overview
+
+The understand how kabelsalat works, we need to understand how the user code
+is transformed to produce audio in the end. These are the basic steps:
+
+1. User evaluates code
+2. The code creates a `Node` representing the audio graph
+3. The `Node` is compiled to an optimized chunk audio callback code
+4. The code is sent to the audio thread
+5. The audio thread creates the required `AudioNode`'s and runs the code for each audio sample
+
+## From Code to Graph
+
+Let's look at this example:
+
+<MiniRepl
+  code={`sine(200)
+.log(n=>JSON.stringify(n,null,2))
+.out()`}
+  client:only="solid"
+/>
+
+If you run the above patch, the `log` method will output this:
+
+```json
+{
+  "type": "Sine",
+  "ins": [
+    {
+      "type": "n",
+      "value": 200,
+      "ins": []
+    }
+  ]
+}
+```
+
+Here we see the basic structure of a `Node` in kabelsalat:
+
+- type: which kind of node is it?
+- ins: which `Node`'s are connected to it?
+
+We can also see that the number passed to `sine` is converted to a `Node` of type `n`, which is the constant number node.
+It is special because it also has a `value` property.
+
+The clever thing about this data structure is that you only need to know the last `Node` (before .out) to know the whole graph!
+
+### Method Chaining vs Function Calling
+
+kabelsalat relies heavily on method chaining, which is only syntax sugar for regular function calls. These 2 variants are equivalent:
+
+```js
+mul(sine(200), 0.5);
+// is equal to
+sine(200).mul(0.5);
+```
+
+This will work for any node (except `out`, which is a special case). More generally, you can say
+
+```
+a.x(b) = x(a,b)
+```
+
+This syntax sugar reduces the level of parenthesis nesting alot and is also more natural way to express arithmetic:
+
+```js
+3 * 4; // infix notation
+// =
+n(3).mul(4); // method chaining
+// =
+mul(3, 4); // polish notation
+```
+
+So you could say the method chaining syntax acts as an infix operator replacement.
+
+## Compiling the Graph
+
+In the next stage, we pass the graph to the compiler, which turns it into a sequence of instructions that runs well on the audio thread.
+Before actual compilation there is preprocessing step:
+
+1. flatten the `Node`s
+2. apply topological sort
+
+### Flatten Node's
+
+This step takes the last `Node` and turns it into an Array of `Node`'s where, the `ins` are replaced with indices:
+
+<MiniRepl
+  code={`const graph = sine(200).mul(.5)
+console.log('flat graph', JSON.stringify(graph.flatten(),null,2))
+graph.out()`}
+  client:only="solid"
+/>
+
+This is the console output:
+
+```json
+[
+  // 0
+  {
+    "type": "mul",
+    "ins": ["1", "3"]
+  },
+  // 1
+  {
+    "type": "Sine",
+    "ins": ["2"]
+  },
+  // 2
+  {
+    "type": "n",
+    "ins": [],
+    "value": 200
+  },
+  // 3
+  {
+    "type": "n",
+    "ins": [],
+    "value": 0.5
+  }
+]
+```
+
+This representation of the graph contains the same info, but structured in a way suitable for...
+
+### Topological Sort
+
+[Topological Sorting](https://en.wikipedia.org/wiki/Topological_sorting) makes sure the `Node`s are ordered so that nodes without dependencies are first.
+
+<MiniRepl
+  code={`const graph = sine(200).mul(.5)
+const flat = graph.flatten()
+console.log('sorted', JSON.stringify(topoSort(flat),null,2))
+graph.out()`}
+  client:only="solid"
+/>
+
+This gives us:
+
+```json
+["3", "2", "1", "0"]
+```
+
+Reordering the flattened nodes in this order:
+
+```json
+[
+  // 3
+  {
+    "type": "n",
+    "ins": [],
+    "value": 0.5
+  }
+  // 2
+  {
+    "type": "n",
+    "ins": [],
+    "value": 200
+  },
+  // 1
+  {
+    "type": "Sine",
+    "ins": ["2"]
+  },
+  // 0
+  {
+    "type": "mul",
+    "ins": ["1", "3"]
+  },
+]
+```
+
+We can now observe that the `ins` of each `Node` preceed the `Node` itself, which means it's time to generate some code..
+
+### Compilation
+
+The compiler can now generate a chunk of code, where each line roughly corresponds to the value of one Node:
+
+<MiniRepl
+  code={`const graph = sine(200).mul(.5)
+console.log(compile(graph))
+graph.out()`}
+  client:only="solid"
+/>
+
+The compiler gives us this compiled `unit`:
+
+```js
+{
+    "src": "const n1 = nodes[0].update(200, 0); /* Sine */\nconst n0 = n1 * 0.5; /* Sine * n */\nreturn [(0*0.3), (0*0.3)]",
+    "audioThreadNodes": ["Sine"]
+}
+```
+
+Let's make the `src` more readable:
+
+```js
+const n1 = nodes[0].update(200, 0); /* Sine */
+const n0 = n1 * 0.5; /* Sine * n */
+return [0 * 0.3, 0 * 0.3];
+```
+
+This is now the actual code that runs on the audio thread.
+
+- The `n` variable numbers correspond to the original indices of the topologically sorted Array.
+- To shorten the output, the compiler inlines arithmetic Node's (add, mul,...) and constant numbers (n).
+- For `AudioNode`'s like `Sine`, the compiler inserts an `update` call with its inputs as arguments (they could also be variables).
+- In the end we just get 2 numbers for the left and right speaker.
+
+## Running the Compiled Code
+
+The `unit` returned by the compiler is sent to our `GraphWorklet` which hosts an instance of `AudioGraph`.
+The `AudioGraph` creates an `AudioNode` for each `type` in the `audioThreadNodes` Array.
+Each `AudioNode` has the ability to keep its own state, for example, `Sine` will keep track of its phase.
+The compiled `src` is used as the body of the `_genSample` function, which is ultimately called in `GraphWorklet.process` for each sample.
+
+## Credits
+
+This is the basic mechanism! It is heavily influenced by the amazing [noisecraft](https://noisecraft.app/) by Maxime Chevalier-Boisvert.
+The audio DSP code is mostly borrowed as is, but I've written a new compiler + implemented feedback and multichannel expansion + removed UI related features.
+You can read more about the specific changes in this [discussion](https://github.com/maximecb/noisecraft/discussions/109)
+
+## More
+
+What I haven't described so far:
+
+- how feedback is resolved
+- how multichannel expansion works
+- how the viz works
+
+I might write about it in the future