Data-driven workflows #2430
Description
Currently data-driven workflows are created by leveraging JSON schemas and the Sgen tool.
- Allow swapping implementations at start-up time (data-driven)
- Define interfaces
- interlink this with schemas (sgen)
- externalize properties
- subjects (input / output)
- Inconsistency between externalizing subject names (allowed) and externalizing include workflow paths (not allowed)
- Goal is to have a way for data-driven instantiation of workflow modules with guarantees that they comply with a known interface
- Resist creating a general purpose language for accommodating all possible module combinations
Use case for experimental rig
Common modules:
- Rig
- Task
- Logging (included with all modules?)
Design approaches
- Automatically detect and determine compatibility between a task and a rig, e.g. need soundcard or specific response port. Both rig and task are independently configurable and tested for match.
- Task is a fixed workflow, and only rig components are configurable:
a. subject interfaces are fixed by the task
b. rig components are created from schema and connect to task subjects
c. in the end we may want to do small "tweaks" to the task structure, but this will be a slippery slope towards having to configure everything - Generate schema from a bonsai workflow to avoid duplicated work