Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganise scripts and rules #176

Merged
merged 6 commits into from
Feb 6, 2024
Merged

Conversation

tomalrussell
Copy link
Member

  • workflow rules grouped by high-level concept and workflow stage
  • scripts sit next to the rules that use them
  • files should have been moved but are otherwise almost unchanged (apart from changes to script location references, and in one or two cases rules combined from two files into one)

Aim to improve on the sense that different sub-workflows were muddled together, as in the previous grouping by general stage (download/preprocess/etc.).

Aim for a clear set of workflow/ directories. We should be able to sketch a simple high-level dependency graph for the workflow between stages. Sets of rules within each stage are roughly linear.

For example, here's part of the high-level graph (ignoring contextual datasets):

power tropical-cyclone transport flood
  |       |               |       |
   \     /                 \     /
   power-tc           transport-flood  

- workflow rules grouped by high-level concept and workflow stage
- scripts sit next to the rules that use them

Aim to reduce cognitive overhead and sense that different sub-workflows
were muddled together in the previous grouping by general stage.

Aim for a clear set of workflow/<stage> directories, where we should
be able to draw a simple high-level DAG for the workflow between
stages, and sets of rules within each stage are roughly linear.
@tomalrussell tomalrussell merged commit a8a60b9 into main Feb 6, 2024
1 check passed
@tomalrussell tomalrussell deleted the feature/refactor-folder-structure branch February 6, 2024 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant