Skip to content
This repository was archived by the owner on Jul 1, 2024. It is now read-only.
This repository was archived by the owner on Jul 1, 2024. It is now read-only.

Table creation: Efficiently determine if files need to be updated #29

@everyday847

Description

@everyday847

In the status quo, we re-run build_full_model and clustering every time. This is wildly inefficient! Usually, not much has changed -- maybe only one swm_rebuild.out silent file has had added decoys, or maybe you've just changed some formatting that affects how you process the resulting data. But of course, you WANT to be sensitive to the possibility of changes, both in the input files and in the code.

Proposed strategy:

  1. Take the git commit hash for main and for this repo, plus some hash of the input silent file.
  2. If either git hash has changed, re-run all files.
  3. If the git hashes are unchanged, only rerun changed silent files.

Obvious pitfall: un-committed changes to code (either in main or here) will not trigger re-runs. Better solution, @rhiju / @calebgeniesse?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions