Define structure and version management of Falaise resource files

[drbenmorgan]

As discovered in PR #29, the current system for declaring/defining versions of, and compatibility between, resource files, especially when connected by variants, needs some thought. The major issues highlighted by #29 are:

1. Even a small addition of a new variant (or any other config sections?) requires a complete new version of the category in which the variant sits to be created.
2. This new version requires all files in the category, whether changed or not, to be copied into the new version.

• This has high error potential, and seems to defeat purpose of version control system (i.e. we have multiple, content identical, copies of the same file)

3. This version must be propagated to upstream clients of the category

• Again a step with potential for errors, and might get cascading new versions?

I think there are two initial issues to discuss and address:

1. Document the current structure of the resource files, with a clear definition of what versions mean and how backward/forward compatibility work. To include a "maintainers guide" describing the workflow to edit and update resource files for the cases of bug fixes (e.g. typo or updated number), enhancements (e.g. new interface) and breaking changes (e.g. an existing interface is removed)
2. Identify the places where information from the resource files is propagated through the system (e.g. used in flsimulate, required by flreconstruct to make sense of the input data), and what consequences this has for backward/forward compatibility.

I think these are the minimum required outcome - they are needed anyway, and will highlight if further work is required.

This is a reasonably wide-ranging topic as it covers geometry/event generation/reconstruction, which impact analysis, so I've "assigned" quite a few people as input from all the working groups is needed!
Please add any comments, thoughts!

[drbenmorgan]

I think #29 quite clearly illustrates the issue - in this case all that was required was a non-breaking addition, yet look at the number of files that needed to be changed (even with reuse of older versions), and how that single change propagated through. There's also a fair bit of duplicate info (e.g. specifying files and config in more than one place), though this is more an API issue than the fundamental one of management.

[drbenmorgan]

#101 is another example of the current complexity of files needed just to add an additional event generator without any other changes. Probably needs review of structure/construction of hierarchies plus "inheritance" (reuse an existing setup/config) and "override" (add to/change existing config).

[drbenmorgan]

Closing as fixed by flattening of structure in Falaise 4. Further work on versioning will be done under the database work discussed in #90. GitCondDB seems like a reasonable technology here.