Redesigned the writer (#36) and added YAML support based on the new writer (#23)

Author: liuzicheng1987

.github/workflows/test.yaml

@@ -18,11 +18,12 @@ jobs:
       - name: Run test
         run: |
           g++ --version
-          cmake -S . -B build -DREFLECTCPP_BUILD_TESTS=ON -DREFLECTCPP_FLEXBUFFERS=ON -DREFLECTCPP_XML=ON -DCMAKE_BUILD_TYPE=Release
+          cmake -S . -B build -DREFLECTCPP_BUILD_TESTS=ON -DREFLECTCPP_FLEXBUFFERS=ON -DREFLECTCPP_XML=ON -DREFLECTCPP_YAML=ON -DCMAKE_BUILD_TYPE=Release
           cmake --build build -j 4
           ./build/tests/flexbuffers/reflect-cpp-flexbuffers-tests
           ./build/tests/json/reflect-cpp-json-tests
           ./build/tests/xml/reflect-cpp-xml-tests
+          ./build/tests/yaml/reflect-cpp-yaml-tests
 
   linux-clang:
     runs-on: ubuntu-latest
@@ -46,11 +47,12 @@ jobs:
           CXX: clang++
         run: |
           clang --version
-          cmake -S . -B build -DREFLECTCPP_BUILD_TESTS=ON -DREFLECTCPP_FLEXBUFFERS=ON -DREFLECTCPP_XML=ON -DCMAKE_BUILD_TYPE=Release
+          cmake -S . -B build -DREFLECTCPP_BUILD_TESTS=ON -DREFLECTCPP_FLEXBUFFERS=ON -DREFLECTCPP_XML=ON -DREFLECTCPP_YAML=ON -DCMAKE_BUILD_TYPE=Release
           cmake --build build -j 4
           ./build/tests/flexbuffers/reflect-cpp-flexbuffers-tests
           ./build/tests/json/reflect-cpp-json-tests
           ./build/tests/xml/reflect-cpp-xml-tests
+          ./build/tests/yaml/reflect-cpp-yaml-tests
 
 
 #  The latest MSVC version on GitHub Actions has a bug, and it's difficult to switch to another version.

.gitignore

@@ -44,5 +44,7 @@
 *.json
 *.fb
 *.xml
+*.yml
+*.yaml
 
 !vcpkg.json

CMakeLists.txt

@@ -3,11 +3,12 @@ cmake_minimum_required(VERSION 3.15)
 option(REFLECTCPP_BUILD_SHARED "Build shared library" OFF)
 option(REFLECTCPP_FLEXBUFFERS "Enable flexbuffers support" OFF)
 option(REFLECTCPP_XML "Enable XML support" OFF)
+option(REFLECTCPP_YAML "Enable YAML support" OFF)
 
 option(REFLECTCPP_BUILD_TESTS "Build tests" OFF)
 
 # enable vcpkg if require features other than JSON
-if (REFLECTCPP_FLEXBUFFERS OR REFLECTCPP_XML)
+if (REFLECTCPP_FLEXBUFFERS OR REFLECTCPP_XML OR REFLECTCPP_YAML)
     set(CMAKE_TOOLCHAIN_FILE ${CMAKE_CURRENT_SOURCE_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake CACHE STRING "Vcpkg toolchain file")
 endif ()
 
@@ -33,6 +34,11 @@ if (REFLECTCPP_XML)
     target_link_libraries(reflectcpp INTERFACE pugixml::pugixml)
 endif ()
 
+if (REFLECTCPP_YAML)
+    find_package(yaml-cpp CONFIG REQUIRED)
+    target_link_libraries(reflectcpp INTERFACE yaml-cpp::yaml-cpp)
+endif ()
+
 target_compile_options(reflectcpp PRIVATE -Wall)
 
 if (REFLECTCPP_BUILD_TESTS)

README.md

@@ -23,20 +23,22 @@ Design principles for reflect-cpp include:
 - Simple extendability to custom classes
 - Standard C++ only, no compiler-specific macros
 
-## Why do we need this?
-
-Suppose your C++ program has complex data structures it needs to save and load. Or maybe it needs to interact with some kind of external API. If you do this the traditional way, you will have a lot of boilerplate code. This is annoying and error-prone.
+## Serialization formats
 
-reflect-cpp is not just a reflection library, it is for **serialization, deserialization and validation** through reflection.
+reflect-cpp provides a unified reflection-based interface across different serialization formats. It is deliberately designed in a very modular way, using [concepts](https://en.cppreference.com/w/cpp/language/constraints), to make it as easy as possible to interface various C or C++ libraries related to serialization. Refer to the [documentation](https://github.com/getml/reflect-cpp/tree/main/docs) for details.
 
-That means that you can encode your requirements about the input data in the type system and have them validated upfront. This is why the library also includes algebraic data types like tagged unions and numerous validation routines. Having your requirements encoded in the type system is the most reliable way of ensuring they are met. If your requirements are not met, the user of your software gets a very clear error message. Encoding your requirements in the type system also makes it a lot easier for anyone reading your code.
+The following table lists the serialization formats currently supported by reflect-cpp and the underlying libraries used:
 
-This increases user experience and developer experience, it makes your code safer (fewer bugs) and more secure (less prone to malicious attacks).
+| Format       | Library                                              | Version      | License    | Remarks                                              |
+|--------------|------------------------------------------------------|--------------|------------| -----------------------------------------------------|
+| JSON         | [YYJSON](https://github.com/ibireme/yyjson)          | >= 0.8.0     | MIT        | out-of-the-box support, included in this repository  |
+| flexbuffers  | [flatbuffers](https://github.com/google/flatbuffers) | >= 23.5.26   | Apache 2.0 |                                                      |
+| XML          | [pugixml](https://github.com/zeux/pugixml)           | >= 1.14      | MIT        |                                                      |
+| YAML         | [yaml-cpp](https://github.com/jbeder/yaml-cpp)       | >= 0.8.0     | MIT        |                                                      |
 
-For a more in-depth theoretical discussions of these topics, the following books are warmly recommended:
+Support for more serialization formats is in development. Refer to the [issues](https://github.com/getml/reflect-cpp/issues) for details.
 
-- *Category Theory for Programmers* by Bartosz Milewski (https://github.com/hmemcpy/milewski-ctfp-pdf/releases)
-- *Domain Modeling Made Functional* by Scott Wlaschin
+Please also refer to the *vcpkg.json* in this repository.
 
 ## Simple Example
 
@@ -66,6 +68,26 @@ The resulting JSON string looks like this:
 {"first_name":"Homer","last_name":"Simpson","age":45}
 ```
 
+Or you can use another format, such as YAML. This will work for just about 
+any example in the entire documentation or any supported format.
+
+```cpp
+#include <rfl/yaml.hpp>
+
+// ... (same as above)
+
+const std::string yaml_string = rfl::yaml::write(homer);
+auto homer2 = rfl::yaml::read<Person>(yaml_string).value();
+```
+
+The resulting YAML string looks like this:
+
+```yaml
+first_name: Homer
+last_name: Simpson
+age: 45
+```
+
 ## More Comprehensive Example
 
 ```cpp
@@ -361,19 +383,20 @@ In addition, it supports the following custom containers:
 
 Finally, it is very easy to extend full support to your own classes, refer to the [documentation](https://github.com/getml/reflect-cpp/tree/main/docs) for details.
 
-## Serialization formats
+## Why do we need this?
 
-reflect-cpp is deliberately designed in a very modular way, using [concepts](https://en.cppreference.com/w/cpp/language/constraints), to make it as easy as possible to interface C or C++ libraries for various serialization formats. Refer to the [documentation](https://github.com/getml/reflect-cpp/tree/main/docs) for details. PRs related to serialization formats are welcome.
+Suppose your C++ program has complex data structures it needs to save and load. Or maybe it needs to interact with some kind of external API. If you do this the traditional way, you will have a lot of boilerplate code. This is annoying and error-prone.
 
-The following table lists the serialization formats currently supported by reflect-cpp and the underlying libraries used:
+reflect-cpp is not just a reflection library, it is for **serialization, deserialization and validation** through reflection.
 
-| Format       | Library                                              | Version      | License    | Remarks                                              |
-|--------------|------------------------------------------------------|--------------|------------| -----------------------------------------------------|
-| JSON         | [YYJSON](https://github.com/ibireme/yyjson)          | >= 0.8.0     | MIT        | out-of-the-box support, included in this repository  |
-| flexbuffers  | [flatbuffers](https://github.com/google/flatbuffers) | >= 23.5.26#1 | Apache 2.0 |                                                      |
-| XML          | [pugixml](https://github.com/zeux/pugixml)           | >= 1.14      | MIT        |                                                      |
+That means that you can encode your requirements about the input data in the type system and have them validated upfront. This is why the library also includes algebraic data types like tagged unions and numerous validation routines. Having your requirements encoded in the type system is the most reliable way of ensuring they are met. If your requirements are not met, the user of your software gets a very clear error message. Encoding your requirements in the type system also makes it a lot easier for anyone reading your code.
 
-Please also refer to the *vcpkg.json* in this repository.
+This increases user experience and developer experience, it makes your code safer (fewer bugs) and more secure (less prone to malicious attacks).
+
+For a more in-depth theoretical discussions of these topics, the following books are warmly recommended:
+
+- *Category Theory for Programmers* by Bartosz Milewski (https://github.com/hmemcpy/milewski-ctfp-pdf/releases)
+- *Domain Modeling Made Functional* by Scott Wlaschin
 
 ## Documentation
 
@@ -427,6 +450,7 @@ add_subdirectory(reflect-cpp) # Add this project as a subdirectory
 
 set(REFLECTCPP_FLEXBUFFERS ON) # Optional
 set(REFLECTCPP_XML ON) # Optional
+set(REFLECTCPP_YAML ON) # Optional
 
 target_link_libraries(your_project PRIVATE reflectcpp) # Link against the library
 ```
@@ -470,7 +494,7 @@ git submodule update --init
 ./vcpkg/bootstrap-vcpkg.bat # Windows
 # You may be prompted to install additional dependencies.
 
-cmake -S . -B build -DREFLECTCPP_BUILD_TESTS=ON -DREFLECTCPP_FLEXBUFFERS=ON -DREFLECTCPP_XML=ON -DCMAKE_BUILD_TYPE=Release
+cmake -S . -B build -DREFLECTCPP_BUILD_TESTS=ON -DREFLECTCPP_FLEXBUFFERS=ON -DREFLECTCPP_XML=ON -DREFLECTCPP_YAML=ON -DCMAKE_BUILD_TYPE=Release
 cmake --build build -j 4 # gcc, clang
 cmake --build build --config Release -j 4 # MSVC
 ```
@@ -481,6 +505,7 @@ To run the tests, do the following:
 ./build/tests/flexbuffers/reflect-cpp-flexbuffers-tests
 ./build/tests/json/reflect-cpp-json-tests
 ./build/tests/xml/reflect-cpp-xml-tests
+./build/tests/xml/reflect-cpp-yaml-tests
 ```
 
 ## How to contribute

docs/README.md

@@ -60,6 +60,8 @@
 
 5.3) [XML](https://github.com/getml/reflect-cpp/blob/main/docs/xml.md)
 
+5.4) [YAML](https://github.com/getml/reflect-cpp/blob/main/docs/yaml.md)
+
 ## 6) Advanced topics
 
 6.1) [Supporting your own format](https://github.com/getml/reflect-cpp/blob/main/docs/supporting_your_own_format.md) - For supporting your own serialization and deserialization formats.

docs/supporting_your_own_format.md

@@ -73,29 +73,82 @@ struct Writer {
     using OutputObjectType = ...;
     using OutputVarType = ...;
 
-    /// Appends `_var` to the end of `_arr`, thus mutating it.
-    void add(const OutputVarType _var, OutputArrayType* _arr) const noexcept {...}
-
-    /// Returns an empty OutputVarType, the NULL type of the format.
-    OutputVarType empty_var() const noexcept {...}
-
-    /// Generates an OutputVarType from a basic type
-    /// (std::string, bool, floating point or integral).
-    template <class T>
-    OutputVarType from_basic_type(const T& _var) const noexcept {...}
-
-    /// Generates a new, empty array.
-    OutputArrayType new_array() const noexcept {...}
-
-    /// Generates a new, empty object.
-    OutputObjectType new_object() const noexcept {...}
-
-    /// Determines whether the var is empty (whether it is the NULL type).
-    bool is_empty(const OutputVarType& _var) const noexcept {...}
-
-    /// Adds a new field to obj, thus mutating it.
-    void set_field(const std::string& _name, const OutputVarType& _var,
-                   OutputObjectType* _obj) const noexcept {...}
+  /// Sets an empty array as the root element of the document.
+  /// Some serialization formats require you to pass the expected size in
+  /// advance. If you are not working with such a format, you can ignore the
+  /// parameter `_size`. Returns the new array for further modification.
+  OutputArrayType array_as_root(const size_t _size) const noexcept;
+
+  /// Sets an empty object as the root element of the document.
+  /// Some serialization formats require you to pass the expected size in
+  /// advance. If you are not working with such a format, you can ignore the
+  /// parameter `_size`.
+  /// Returns the new object for further modification.
+  OutputObjectType object_as_root(const size_t _size) const noexcept;
+
+  /// Sets a null as the root element of the document. Returns OutputVarType
+  /// containing the null value.
+  OutputVarType null_as_root() const noexcept;
+
+  /// Sets a basic value (bool, numeric, string) as the root element of the
+  /// document. Returns an OutputVarType containing the new value.
+  template <class T>
+  OutputVarType value_as_root(const T& _var) const noexcept;
+
+  /// Adds an empty array to an existing array. Returns the new
+  /// array for further modification.
+  OutputArrayType add_array_to_array(const size_t _size,
+                                     OutputArrayType* _parent) const noexcept;
+
+  /// Adds an empty array to an existing object. The key or name of the field is
+  /// signified by `_name`. Returns the new array for further modification.
+  OutputArrayType add_array_to_object(
+      const std::string& _name, const size_t _size,
+      OutputObjectType* _parent) const noexcept;
+
+  /// Adds an empty object to an existing array. Returns the new
+  /// object for further modification.
+  OutputObjectType add_object_to_array(
+      const size_t _size, OutputArrayType* _parent) const noexcept;
+
+  /// Adds an empty object to an existing object. The key or name of the field
+  /// is signified by `_name`. Returns the new object for further modification.
+  OutputObjectType add_object_to_object(
+      const std::string& _name, const size_t _size,
+      OutputObjectType* _parent) const noexcept;
+
+  /// Adds a basic value (bool, numeric, string) to an array. Returns an
+  /// OutputVarType containing the new value.
+  template <class T>
+  OutputVarType add_value_to_array(const T& _var,
+                                   OutputArrayType* _parent) const noexcept;
+
+  /// Adds a basic value (bool, numeric, string) to an existing object. The key
+  /// or name of the field is signified by `name`. Returns an
+  /// OutputVarType containing the new value.
+  template <class T>
+  OutputVarType add_value_to_object(const std::string& _name, const T& _var,
+                                    OutputObjectType* _parent) const noexcept;
+
+  /// Adds a null value to an array. Returns an
+  /// OutputVarType containing the null value.
+  OutputVarType add_null_to_array(OutputArrayType* _parent) const noexcept;
+
+  /// Adds a null value to an existing object. The key
+  /// or name of the field is signified by `name`. Returns an
+  /// OutputVarType containing the null value.
+  OutputVarType add_null_to_object(const std::string& _name,
+                                   OutputObjectType* _parent) const noexcept;
+
+  /// Signifies to the writer that we do not want to add any further elements to
+  /// this array. Some serialization formats require this. If you are working
+  /// with a serialization format that doesn't, just leave the function empty.
+  void end_array(OutputArrayType* _arr) const noexcept;
+
+  /// Signifies to the writer that we do not want to add any further elements to
+  /// this object. Some serialization formats require this. If you are working
+  /// with a serialization format that doesn't, just leave the function empty.
+  void end_object(OutputObjectType* _obj) const noexcept;
 };
 ```
 
@@ -145,12 +198,12 @@ struct Reader {
     /// Returns an rfl::Error if `_var` cannot be cast as an array.
     rfl::Result<InputArrayType> to_array(const InputVarType& _var) const noexcept {...}
 
-    /// fct is a function that turns the field name into the field index of the
+    /// _fct is a function that turns the field name into the field index of the
     /// struct. It returns -1, if the fields does not exist on the struct. This
     /// returns an std::array that can be used to build up the struct.
     /// See below for a more comprehensive explanation.
     template <size_t size, class FunctionType>
-    std::array<InputVarType, size> to_fields_array(
+    std::array<std::optional<InputVarType>, size> to_fields_array(
         const FunctionType _fct, const InputObjectType& _obj) const noexcept {...}
 
     /// Iterates through an object and writes the contained key-value pairs into
@@ -187,15 +240,15 @@ Consider the following struct:
 
 ```cpp
 struct Person {
-    rfl::Field<"firstName", std::string> first_name;
-    rfl::Field<"lastName", std::string> last_name;
-    rfl::Field<"birthday", rfl::Timestamp<"%Y-%m-%d">> birthday;
-    rfl::Field<"children", std::vector<Person>> children;
+    rfl::Rename<"firstName", std::string> first_name;
+    rfl::Rename<"lastName", std::string> last_name;
+    rfl::Timestamp<"%Y-%m-%d"> birthday;
+    std::vector<Person> children;
 };
 ```
 
 This struct contains four fields. `rfl::parsing::Parser` expects `to_fields_array`
-to return an `std::array<InputVarType, 4>` containing the field "firstName" in the
+to return an `std::array<std::optional<InputVarType>, 4>` containing the field "firstName" in the
 first position, "lastName" in the second position, "birthday" in the third position
 and "children" in the fourth position.
 
@@ -214,6 +267,4 @@ Your job is to implement the following:
 1. Iterate through `_obj`.
 2. Identify the required index of the field name using `_fct`.
 3. Set the corresponding field in `std::array` to the field value associated with the field name.
-4. Any field that could not be set in steps 1-3 must be set to the NULL value,
-   such that `Reader.is_empty(...)` would return `true`.
-
+4. Any field that could not be set in steps 1-3 must be set to `std::nullopt`.