Skip to content

Conversation

@koonpeng
Copy link
Collaborator

@koonpeng koonpeng commented Feb 27, 2025

This is on top of koonpeng/buffers but I will rebase it is merged.

Adds initial section support, currently only registered section is supported, templated sections will be done in another PR. The biggest change in this PR is the additional of derive based message registration first experimented in #47. Because sections can have any number of inputs, outputs and buffers, the current method of registering message operations do not work with sections.

So far I haven't found a better alternative:

  • A hashmap of CommonOperations doesn't work because CommonOperations has generic params.
    • It is also error prone as the field names need to be keyed by strings.
  • Generating a sister struct of CommonOperations could work, but get really ugly because a generic param is needed for each field.

An example of how sections with derive based registration looks like

struct OpaqueMessage;

#[derive(Section)]
struct TestSection {
    #[section(no_deserialize, no_serialize, no_clone)]
    foo: InputSlot<OpaqueMessage>,
    bar: Output<f64>,
    baz: Buffer<String>,
}

fn test_register_section() {
    let mut registry = DiagramElementRegistry::new();
    registry.register_section_builder(
        SectionBuilderOptions::new("test_section").with_name("TestSection"),
        |builder: &mut Builder, _config: ()| {
            let node = builder.create_map_block(|_: OpaqueMessage| 1_f64);
            let buffer = builder.create_buffer(BufferSettings::default());
            TestSection {
                foo: node.input,
                bar: node.output,
                baz: buffer,
            }
        },
    );
}

Compared to node registration, there is no option to opt out or add message operations when registering sections, but all the registration "rules" are still the same. e.g. there will be a compiler error if foo is not attributed with the correct attributes, opt in operations are also not automatically opt in, attempting to opt in an operation that cannot work for the type will result in a compile error.

mxgrey and others added 30 commits January 23, 2025 21:50
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Teo Koon Peng <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Co-authored-by: Michael X. Grey <[email protected]>
mxgrey added 19 commits March 28, 2025 19:10
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
@mxgrey mxgrey marked this pull request as ready for review April 21, 2025 16:58
mxgrey added 2 commits April 22, 2025 20:14
Signed-off-by: Michael X. Grey <[email protected]>
Signed-off-by: Michael X. Grey <[email protected]>
Copy link
Contributor

@mxgrey mxgrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature has turned out extremely well, and it came with some knock on effects that improved capabilities that already existed, as well as paving the way for other advanced capabilities in the future, like the scope operation.

I especially like the #[derive(Section)] macro with the attribute to enable traits for the message types. It's worth thinking about whether we can introduce a #[derive(Node)] that would let us do likewise for the node builder. Maybe if we require exactly one input field and exactly one output field for the node struct, and then the rest of the fields are assumed to be streams.

@mxgrey mxgrey merged commit 030510f into main Apr 22, 2025
6 checks passed
@mxgrey mxgrey deleted the koonpeng/sections branch April 22, 2025 12:45
@mxgrey mxgrey mentioned this pull request Apr 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants