Added initial support for platform-specific code.

Author: TerensTare

CHANGELOG.md

@@ -6,6 +6,18 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 
+## [Unreleased]
+
+### Added
+
+- Experimental support for platform-dependent code. You should now define `start_platform_code` and `end_platform_code` on your generators to be run on each item inside a platform-specific code. The built-in generators already have support for these changes. Please refer to the documentation inside `visitor.py` for more.
+- README on the C++ generator discussing most common topics related to it.
+
+### Changed
+
+- Rewrote the C# generator to use functions from `utils`.
+
+
 ## 2024-06-28
 
 ### Added

README.md

@@ -19,7 +19,7 @@ Also, before you can run the script, you need to edit `PATH_BY_UNIT` in [setup.p
 
 This is a list of currently supported languages:
 
-- C++ ([example](./gen/cpp/example.cpp))
+- C++ ([example](./gen/cpp/example.cpp)) (for SDL2 use [this](./gen/cpp/example-sdl2.cpp) instead)
 - C# ([example](./gen/cs/Example.cs)) (NOTE: headers must be annotated as in this [PR](https://github.com/libsdl-org/SDL/pull/9907))
 - JSON
 

gen/cpp.py

@@ -54,7 +54,8 @@
     }}
 
 export namespace {2}
-{{"""
+{{
+"""
 
 
 def _cut_similarity(model: str, target: str) -> str:
@@ -121,6 +122,14 @@ def __del__(self) -> None:
         self._file.write("}\n")
         self._file.close()
 
+    def start_platform_code(self, platforms: list[str]):
+        self._file.write(
+            f"#if {' || '.join(map(lambda p: f'defined({p})', platforms))}\n"
+        )
+
+    def end_platform_code(self):
+        self._file.write("#endif\n\n")
+
     def visit_function(self, rules: dict[str, Node | list[Node]]):
         name = rules["function.name"]
         ret = rules["function.return"].text.decode()
@@ -205,8 +214,7 @@ def cast_if_enum(ty: str, name: str) -> str:
                 print(f"Note: Skipping {name.text.decode()} due to unnamed parameter")
                 return
 
-        self._file.write(f"""
-    {ret} {name.text[4:].decode()}(""")
+        self._file.write(f"\n    {ret} {name.text[4:].decode()}(")
 
         self._file.write(", ".join(f"{ty} {nm}" for ty, nm in zip(ps_types, ps_name)))
 

gen/cpp/README.md

@@ -0,0 +1,45 @@
+
+# Q&A
+
+Q: How do I use the C++ generator?
+A: Just like how you would use the other generators, by running: `py sdl_parser.py gen.cpp`. You can specify the following parameters:
+    `--module`: a format string for the module name. Defaults to `sdl.{ext}`.
+    `--namespace`: a format string for the namespace containing the generated code. Defaults to `sdl::{ext}`
+
+
+Q: What compiler do I need to use the code?
+A: You need a compiler that supports C++20 modules.
+
+
+Q: How do I use the generated code?
+A: You simply import the modules you want on your code and use the functions/types as you would with any C++ code.
+
+Q: How do I build the code?
+A:  You need to consult the documentation of your compiler on how to build modules, but below there are commands showing how to build the provided example (example.cpp) with the main SDL module (`SDL.g.cppm`) on the three main compilers: GCC, Clang and MSVC. Remember to use `example-sdl2.cpp` instead of `example.cpp` if you have generated SDL2 headers.
+```sh
+# For GCC, warning: untested code
+# Apparently with GCC you have to build everything at once
+g++ -std=c++20 -fmodules-ts -o example example.cpp SDL.g.cppm -I<your-include-paths> -l<sdl-lib>
+
+
+# For Clang, warning: untested code
+clang -c -std=c++20 SDL.g.cppm -Xclang -emit-module-interface -o SDL.g.pcm -I<your-include-paths>
+# No include paths are needed when building the executable
+clang --std=c++20 -fprebuilt-module-path=. -fmodule-file=sdl=SDL.g.pcm example.cpp SDL.g.cppm -o example -l<sdl-lib>
+
+# For MSVC
+cl /std:c++20 /c SDL.g.cppm /TP /interface /I <your-include-paths>
+# Notice no include paths are needed when building the executable
+cl /std:c++20 example.cpp /link SDL.g.obj <sdl-library>
+```
+
+
+
+The SDL modules need to be built only once and can be used freely after that.
+
+Q: What difference does the generated code have over the regular SDL code?
+A: Here's a list of changes that the generator applies:
+- Everything is located inside a namespace, depending on the unit they are part of.
+- The names have the prefix stripped (eg. `SDL_Init` is `sdl::Init`).
+- Macros are now `constexpr` functions, as macros and static variables cannot be exported from modules.
+- Enums and bitflags are strongly typed (as in `enum class`). Bitflags also use the alias type as underlying type.

gen/cs.py

@@ -220,6 +220,12 @@ def __del__(self) -> None:
         with open(self._out, "w") as f:
             f.write(self._data)
 
+    def start_platform_code(self, platforms: list[str]):
+        self._file.write(f"#if {' || '.join(platforms)}\n")
+
+    def end_platform_code(self):
+        self._file.write("#endif\n\n")
+
     def visit_function(self, rules):
         name = rules["function.name"].text.decode()
         docs = rules["function.docs"]

query.scm

@@ -1,7 +1,5 @@
 
 (
-    (comment) @function.docs
-    .
     (declaration
         (storage_class_specifier)
         type: (_) @function.return
@@ -13,8 +11,6 @@
 ) @function
 
 (
-    (comment) @function.docs
-    .
     (declaration
         (storage_class_specifier)
         type: (_) @function.return
@@ -112,3 +108,7 @@
 
 (preproc_def name: (_) @const.name value: (_) @const.value) @const
 
+[
+    (preproc_if condition: (_) @cond.text)
+    (preproc_ifdef name: (_) @cond.text)
+] @cond

sdl_parser.py

@@ -11,6 +11,11 @@
 from setup import PATH_BY_UNIT, SDL_ROOT
 import utils
 
+
+if not PATH_BY_UNIT:
+    print("Error: no units chosen to parse. Please edit PATH_BY_UNIT in setup.py.")
+    sys.exit(1)
+
 _FILE_STR: str = """
 from tree_sitter import Node
 from visitor import visitor
@@ -90,11 +95,14 @@ def os_defines() -> list[str]:
             return ["-D", "_AIX"]
 
         case "android":
-            return ["-D", "__ANDROID__"]
+            return ["-D", "__ANDROID__", "-D", "ANDROID"]
 
         case "cygwin":
             return ["-D", "_WIN32", "-D", "__CYGWIN__"]
 
+        case "darwin":
+            return ["-D", "__APPLE__", "-D", "__MACH__"]
+
         case "emscripten":
             return ["-D", "__EMSCRIPTEN__"]
 
@@ -104,6 +112,9 @@ def os_defines() -> list[str]:
         case "linux":
             return ["-D", "__linux__"]
 
+        case "wasi":
+            return ["-D", "__wasi__"]
+
         case "win32":
             return ["-D", "_WIN32"]
 
@@ -126,6 +137,28 @@ def parse_file(*args, input: str, output: str):
             "-D",
             "SDL_DECLSPEC=",  # save us some time and headaches
             "-D",
+            "SDL_DEPRECATED=",  # save us some time and headaches
+            "-D",
+            "SDL_UNUSED=",  # save us some time and headaches
+            "-D",
+            "SDL_ASSERT_LEVEL=1",  # save us some time and headaches
+            "-D",
+            "SDL_NODISCARD=",  # save us some time and headaches
+            "-D",
+            "SDL_NORETURN=",  # save us some time and headaches
+            "-D",
+            "SDL_ANALYZER_NORETURN=",  # save us some time and headaches
+            "-D",
+            "SDL_MALLOC=",  # save us some time and headaches
+            "-D",
+            "SDL_ALLOC_SIZE=",  # save us some time and headaches
+            "-D",
+            "SDL_ALLOC_SIZE2=",  # save us some time and headaches
+            "-D",
+            "SDL_BYTEORDER=SDL_LIL_ENDIAN",  # save us some time and headaches
+            "-D",
+            "SDL_FLOATWORDORDER=SDL_LIL_ENDIAN",  # save us some time and headaches
+            "-D",
             "SDL_SLOW_MEMCPY",  # save us some time and headaches
             "-D",
             "SDL_SLOW_MEMMOVE",  # save us some time and headaches
@@ -150,6 +183,8 @@ def parse_file(*args, input: str, output: str):
             "-D",
             "SDL_EndThreadFunction",
             "-D",
+            "SDL_platform_defines_h_",  # save us some time and headaches
+            "-D",
             "SDL_oldnames_h_",  # save us some time and headaches
             "-D",
             "SDL_stdinc_h_",  # save us some time and headaches
@@ -168,10 +203,9 @@ def parse_file(*args, input: str, output: str):
      (SDL_static_cast(Uint32, SDL_static_cast(Uint8, (D))) << 24))""",
             "-D",
             "SDL_static_cast(T, V)=((T)(V))",  # save us some time and headaches
-            "-N",
-            "WINAPI_FAMILY_WINRT",  # workaround
+            # skip this as we need them to detect platform-specific code
             "--passthru-defines",  # keep defines in output
-            # "--passthru-unknown-exprs",
+            "--passthru-unknown-exprs",  # NOTE: this keeps the ifdef/endif blocks
             "--passthru-unfound-includes",  # skip missing includes
             "--passthru-comments",  # keep comments in output
             "--output-encoding",

setup.py

@@ -1,5 +1,3 @@
-import sys
-
 # NOTE: this is relative to `sdl_parser.py` if a relative path.
 SDL_ROOT = "./include"
 
@@ -9,16 +7,11 @@
 # All these paths should be reasonable defaults.
 # Uncomment and change as needed.
 PATH_BY_UNIT = {
-    "SDL": "SDL2/SDL.h",
+    # "SDL": "SDL2/SDL.h",
     # "SDL": "SDL3/SDL.h",
     # "SDL_main": "SDL3/SDL_main.h",
     # "SDL_vulkan": "SDL3/SDL_vulkan.h",
     # "SDL_image": "SDL3_image/SDL_image.h",
     # "SDL_mixer": "SDL3_mixer/SDL_mixer.h",
     # "SDL_ttf": "SDL3_ttf/SDL_ttf.h",
 }
-
-# NOTE: do not touch :)
-if not PATH_BY_UNIT:
-    print("Error: no units chosen to parse.")
-    sys.exit(1)

visitor.py

@@ -2,6 +2,7 @@
 from getopt import getopt
 import inspect
 import sys
+import re
 
 from tree_sitter import Node
 
@@ -10,6 +11,8 @@
 
 _BITFLAG_FILTER = {"preproc_def", "preproc_function_def"}
 
+_PLATFORM_REGEX = re.compile(r"\bSDL_PLATFORM_\w+\b")
+
 
 class Visitor(metaclass=ABCMeta):
     def __init__(self, unit: str) -> None:
@@ -19,38 +22,115 @@ def __init__(self, unit: str) -> None:
         # ignore typedef/#defines that are used for bitflags
         # so we have to do it manually
         self._parsing_bitflag = False
+        self._platforms = []
+        self._platform_node_id = None
 
     def __del__(self) -> None:
         # empty for now
         pass
 
     def visit(self, rules: _Rules):
+        # TODO: check if this is the child of the `cond` node, if the node is not `None`
+        # when not the child, then the `cond` node becomes None
+
+        def _platform_setup(rule: str):
+            cursor = rules[rule]
+
+            if self._platform_node_id:
+                while (
+                    cursor.parent
+                    and cursor.parent.parent
+                    and cursor.parent.parent.type != "translation_unit"
+                ):
+                    cursor = cursor.parent
+
+                if cursor.id != self._platform_node_id:
+                    self._platform_node_id = None
+
+            if self._platform_node_id:
+                self.start_platform_code(self._platforms)
+
         if "function" in rules:
+            _platform_setup("function")
             self.visit_function(rules)
         elif "bitflag" in rules:
+            _platform_setup("bitflag")
             self.visit_bitflag(rules)
             self._parsing_bitflag = False
         elif "enum" in rules:
+            _platform_setup("enum")
             self.visit_enum(rules)
         elif "opaque" in rules:
+            _platform_setup("opaque")
             self.visit_opaque(rules)
         elif "struct" in rules:
+            _platform_setup("struct")
             self.visit_struct(rules)
         elif "union" in rules:
+            _platform_setup("union")
             self.visit_union(rules)
         elif "alias" in rules:
             if rules["alias"].next_sibling.type in _BITFLAG_FILTER:
                 self._parsing_bitflag = True
                 return
 
+            _platform_setup("alias")
             self.visit_alias(rules)
         elif "callback" in rules:
+            _platform_setup("callback")
             self.visit_callback(rules)
         elif "fn_macro" in rules:
+            _platform_setup("fn_macro")
             self.visit_fn_macro(rules)
         elif "const" in rules:
-            if not self._parsing_bitflag:
+            if not self._parsing_bitflag or self._platform_node_id:
+                # skip constants inside bitflags and platform-specific code
+                _platform_setup("const")
                 self.visit_const(rules)
+        elif "cond" in rules:
+            self._platforms = _PLATFORM_REGEX.findall(rules["cond.text"].text.decode())
+            if not self._platforms:
+                return
+
+            self._platform_node_id = rules["cond"].id
+            return
+
+        if self._platform_node_id:
+            self.end_platform_code()
+
+    @abstractmethod
+    def start_platform_code(self, platforms: list[str]):
+        """
+        Start a platform-specific code block.
+
+        This function is called once per platform-specific data instead of once per block.
+        Thus, in the following code block:
+
+        ```c
+        #ifdef SDL_PLATFORM_WINDOWS
+            extern void SDL_foo();
+
+            extern void SDL_bar();
+        #endif
+        ```
+
+        `start_platform_code` will be called once with `platforms` being `["SDL_PLATFORM_WINDOWS"]`
+        for both `SDL_foo` and `SDL_bar`.
+
+
+        `platforms` is a list of all the platforms that support the following code block.
+        """
+        raise NotImplementedError()
+
+    @abstractmethod
+    def end_platform_code(self):
+        """
+        End a platform-specific code block.
+
+        Similarly to `start_platform_code`, this function is called once per platform-specific data.
+        Refer to `start_platform_code` for more information.
+        """
+        raise NotImplementedError()
 
     @abstractmethod
     def visit_function(self, rules: _Rules):