fixup docs

Author: Gankra

Diff for: README.md

@@ -56,15 +56,18 @@ By default, we test the following pairings:
 In theory other implementations aren't *too bad* to add. You just need to:
 
 * Add an implementation of abis::AbiImpl
-    * Specify the name, language, and source-file extension
-    * Specify supported calling conventions
-    * Specify how to generate a caller from a signature
-    * Specify how to generate a callee from a signature
+    * Specify the language and source-file extension
+    * Specify how to generate source for a caller from a signature
+    * Specify how to generate source for a callee from a signature
     * Specify how to compile a source file to a static lib
 * Register it in the `abi_impls` map in `fn main`
 * (Optional) Register what you want it paired with by default in `DEFAULT_TEST_PAIRS`
     * i.e. (ABI_IMPL_YOU, ABI_IMPL_CC) will have the harness test you calling into C
 
+The bulk of the work is specifying how to generate source code, which can be done
+incrementally by return UnimplementedError to indicate unsupported features. This
+is totally fine, all the backends have places where they give up!
+
 See the Test Harness section below for details on how to use it.
 
 
@@ -83,98 +86,37 @@ Windows Conventions:
 * cdecl
 * fastcall
 * stdcall
-* ~~vectorcall~~ (code is there, but disabled due to linking issues)
-
-Any test which specifies the "All" will implicitly combinatorically generate every known convention.
-"Nonsensical" situations like stdcall on linux are the responsibility of the AbiImpls to identify and disable.
+* vectorcall
 
 
 ## Types
 
-The test format support for the following types/concepts:
-
-* fixed-width integer types (uint8_t and friends)
-* float/double
-* bool
-* structs
-* c-like enums
-* c-like untagged unions
-* rust-like tagged unions
-* opaque pointers (void\*)
-* pass-by-ref (still checks the pointee's layout, and not the address)
-* arrays (including multi-dimensional arrays, although C often requires arrays to be wrapped in pass-by-ref)
+The abi-cafe typesystem is [defined by kdl-script](https://github.com/Gankra/abi-cafe/blob/main/kdl-script/README.md#types), see those docs for details, but, we basically support most of the types you could define/use in core Rust.
 
 
 # Adding Tests
 
-The default suite of tests can be found in `/include/tests/`
+Tests are specified as [kdl-script](https://github.com/Gankra/abi-cafe/blob/main/kdl-script/README.md) files, which are basically C header files, but with a custom syntax that avoids us tying our hands to any particular language/semantics. The syntax will feel fairly familiar to Rust programmers.
 
-There are two kinds of tests: `.kdl` ("normal") and `.procgen.kdl` ("procgen").
+The default suite of tests can be [found in `/include/tests/`](https://github.com/Gankra/abi-cafe/tree/main/include/tests), which is statically embedded in abi-cafe's binary. You don't need to register the test anywhere, we will just try to parse every file in the tests directory.
 
-The latter is sugar for the former, where you just define a type with the same name of the file (so `MetersU32.procgen.kdl` is expected to define a type named `MetersU32`), and we generate a battery of types/functions that stress it out.
+There are two kinds of tests: `.kdl` ("[normal](https://github.com/Gankra/abi-cafe/tree/main/include/tests/normal)") and `.procgen.kdl` ("[procgen](https://github.com/Gankra/abi-cafe/tree/main/include/tests/procgen)").
 
-Tests are specified as [kdl-script](https://github.com/ron-rs/ron) files in the test/ directory, because it's more compact than JSON, has comments, and is more reliable with large integers. Because C is in some sense the "lingua franca" of FFI that everyone has to deal with, we prefer using C types in these definitions.
-
-You don't need to register the test anywhere, we will just try to parse every file in that directory.
-
-The "default" workflow is to handwrite a ron file, and the testing framework will handle generating the actual code implementating that interface (example: structs.ron). Generated impls will be output to the generated_impls dir for debugging. Build artifacts will get dumped in target/temp/ if you want to debug those too.
-
-Example:
-
-```rust
-Test(
-    // name of this set of tests
-    name: "examples",
-    // generate tests for the following function signatures
-    funcs: [
-        (
-            // base function/subtest name (but also subtest name)
-            name: "some_prims",
-            // what calling conventions to generate this test for
-            // (All = do them all, a good default)
-            conventions: [All],
-            // args
-            inputs: [
-               Int(c_int32_t(5)),
-               Int(c_uint64_t(0x123_abc)),
-            ]
-            // return value
-            output: Some(Bool(true)),
-        ),
-        (
-            name: "some_structs",
-            conventions: [All],
-            inputs: [
-               Int(c_int32_t(5)),
-               // Struct decls are implicit in usage.
-               // All structs with the same name must match!
-               Struct("MyStruct", [
-                  Int(c_uint8_t(0xf1)),
-                  Float(c_double(1234.23)),
-               ]),
-               Struct("MyStruct", [
-                  Int(c_uint8_t(0x1)),
-                  Float(c_double(0.23)),
-               ]),
-            ],
-            // no return (void)
-            output: None,
-        ),
-    ]
-)
-```
-
-However, you have two "power user" options available:
+The latter is sugar for the former, where you just define a type with the same name of the file (so `MetersU32.procgen.kdl` is expected to define a type named `MetersU32`), and we generate a battery of types/functions that stress it out.
 
-* Generate the ron itself with generate_procedural_tests in main.rs (example: ui128.ron). This is good for bruteforcing a bunch of different combinations if you just want to make sure a type/feature generally works in many different situations.
+Suggested Examples:
 
-* Use the "Handwritten" convention and manually provide the implementations (example: opaque_example.ron). This lets you basically do *anything* without the testing framework having to understand your calling convention or type/feature. Manual impls go in handwritten_impls and use the same naming/structure as generated_impls.
+* [simple.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/normal/simple.kdl) - a little example of a "normal" test with explicitly defined functions to test
+* [SimpleStruct.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/struct/SimpleStruct.procgen.kdl) - similar to simple.kdl, but procgen
+* [MetersU32.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/pun/MetersU32.procgen.kdl) - an example of a ["pun type"](https://github.com/Gankra/abi-cafe/tree/main/kdl-script#pun-types), where different languages use different definitions
+* [IntrusiveList.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/fancy/IntrusiveList.procgen.kdl) - an example of how we can procgen tests for self-referential types and tagged unions
+* [i8.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/primitive/i8.procgen.kdl) - ok this one isn't instructive it's just funny that it can be a blank file because i8 is builtin so all the info needed is in the filename
 
 
 
 # The Test Harness
 
-Implementation details of dylib test harness are split up between main.rs and the contents of the top-level harness/ directory. The contents of harness/ include:
+Implementation details of dylib test harness are split up between [src/harness/run.rs](https://github.com/Gankra/abi-cafe/blob/main/src/harness/run.rs) and the contents of the top-level [/include/harness/](https://github.com/Gankra/abi-cafe/blob/main/include/harness/). The contents of /include/harness/ are:
 
 * "headers" for the testing framework for each language
 * harness.rs, which defines the entry-point for the test and sets up all the global callbacks/pointers. This is linked with the callee and caller to create the final dylib.
@@ -183,50 +125,50 @@ Ideally you shouldn't have to worry about *how* the callbacks work, so I'll just
 
 ```C
 // Caller Side
-uint64_t basic_val(struct MyStruct arg0, int32_t arg1);
+uint64_t basic_val(MyStruct arg0, int32_t arg1);
 
 // The test harness will invoke your test through this symbol!
 void do_test(void) {
     // Initialize and report the inputs
-    struct MyStruct arg0 = { 241, 1234.23 };
-    WRITE(CALLER_INPUTS, (char*)&arg0.field0, (uint32_t)sizeof(arg0.field0));
-    WRITE(CALLER_INPUTS, (char*)&arg0.field1, (uint32_t)sizeof(arg0.field1));
-    FINISHED_VAL(CALLER_INPUTS);
+    MyStruct arg0 = { 241, 1234.23 };
+    write_field(CALLER_INPUTS, arg0.field0);
+    write_filed(CALLER_INPUTS, arg0.field1);
+    finished_val(CALLER_INPUTS);
 
     int32_t arg1 = 5;
-    WRITE(CALLER_INPUTS, (char*)&arg1, (uint32_t)sizeof(arg1));
-    FINISHED_VAL(CALLER_INPUTS);
+    write_field(CALLER_INPUTS, arg1);
+    finished_val(CALLER_INPUTS);
 
     // Do the call
     uint64_t output = basic_val(arg0, arg1);
 
     // Report the output
-    WRITE(CALLER_OUTPUTS, (char*)&output, (uint32_t)sizeof(output));
-    FINISHED_VAL(CALLER_OUTPUTS);
+    write_field(CALLER_OUTPUTS, output);
+    finished_val(CALLER_OUTPUTS);
 
     // Declare that the test is complete on our side
-    FINISHED_FUNC(CALLER_INPUTS, CALLER_OUTPUTS);
+    finished_func(CALLER_INPUTS, CALLER_OUTPUTS);
 }
 ```
 
 ```C
 // Callee Side
-uint64_t basic_val(struct MyStruct arg0, int32_t arg1) {
+uint64_t basic_val(MyStruct arg0, int32_t arg1) {
     // Report the inputs
-    WRITE(CALLEE_INPUTS, (char*)&arg0.field0, (uint32_t)sizeof(arg0.field0));
-    WRITE(CALLEE_INPUTS, (char*)&arg0.field1, (uint32_t)sizeof(arg0.field1));
-    FINISHED_VAL(CALLEE_INPUTS);
+    write_field(CALLEE_INPUTS, arg0.field0);
+    write_field(CALLEE_INPUTS, arg0.field1);
+    finished_val(CALLEE_INPUTS);
 
-    WRITE(CALLEE_INPUTS, (char*)&arg1, (uint32_t)sizeof(arg1));
-    FINISHED_VAL(CALLEE_INPUTS);
+    write_field(CALLEE_INPUTS, arg1);
+    finished_val(CALLEE_INPUTS);
 
     // Initialize and report the output
     uint64_t output = 17;
-    WRITE(CALLEE_OUTPUTS, (char*)&output, (uint32_t)sizeof(output));
-    FINISHED_VAL(CALLEE_OUTPUTS);
+    write_field(CALLEE_OUTPUTS, output);
+    finished_val(CALLEE_OUTPUTS);
 
     // Declare that the test is complete on our side
-    FINISHED_FUNC(CALLEE_INPUTS, CALLEE_OUTPUTS);
+    finished_func(CALLEE_INPUTS, CALLEE_OUTPUTS);
 
     // Actually return
     return output;
@@ -235,13 +177,14 @@ uint64_t basic_val(struct MyStruct arg0, int32_t arg1) {
 
 The high level idea is that each side:
 
-* Uses WRITE to report individual fields of values (to avoid padding)
-* Uses FINISHED_VAL to specify that all fields for a value have been written
-* Uses FINISHED_FUNC to specify that the current function is done (the caller will usually contain many subtests, FINISHED_FUNC delimits those)
+* Uses write_field to report individual fields of values (to avoid padding)
+* Uses finished_val to specify that all fields for a value have been written
+* Uses finished_func to specify that the current function is done (the caller will usually contain many subtests, FINISHED_FUNC delimits those)
 
 There are 4 buffers: CALLER_INPUTS, CALLER_OUTPUTS, CALLEE_INPUTS, CALLEE_OUTPUTS. Each side should only use its two buffers.
 
-The signatures of the callbacks are:
+The signatures of the callbacks are as follows, but each language wraps these
+in functions/macros to keep the codegen readable:
 
 * `WRITE(Buffer buffer, char* input, uint32_t size_of_input)`
 * `FINISH_VAL(Buffer buffer)`

Diff for: include/tests/normal/simple.kdl

@@ -0,0 +1,22 @@
+// This is a simple little test case to demonstrate a basic normal
+// abi-cafe test that includes some custom types and functions
+
+struct "Point3" {
+    x "f32"
+    y "f32"
+    z "f32"
+}
+
+fn "print" {
+    inputs { _ "Point3"; }
+}
+
+fn "scale" {
+    inputs { _ "Point3"; factor "f32"; }
+    output { "Point3"; }
+}
+
+fn "add" {
+    inputs { _ "Point3"; _ "Point3"; }
+    outputs { _ "Point3"; }
+}

Diff for: include/tests/normal/types.kdl renamed to include/tests/normal/typemess.kdl

@@ -20,17 +20,17 @@ enum "ErrorCode" {
 
 tagged "OptionI32" {
     None
-    Some { _0 "i32"; }
+    Some { _ "i32"; }
 }
 
 tagged "MyResult" {
-    Ok { _0 "[u32;3]"; }
-    Err { _0 "ErrorCode"; }
+    Ok { _ "[u32;3]"; }
+    Err { _ "ErrorCode"; }
 }
 
 tagged "MyDeepResult" {
-    Ok { _0 "MyResult"; }
-    Err { _0 "OptionI32"; }
+    Ok { _ "MyResult"; }
+    Err { _ "OptionI32"; }
     FileNotFound { x "bool"; y "Simple"; }
 }