pfish-tools

Tools for working with pfish

Format Test Results

The script format_test_results.py converts the results.json output of pfish into a human-readable markdown. To use it, simply copy it into a pfish repo and run python3 format_test_results.py. The script will find all results.json files in subdirectories and write the output results.md in the same location.

Push and Test Dependencies

push-and-test is a Dockerized app that manages dependencies for protocols.

Installation

You should install and configure pfish before installing or using this app.

git clone https://github.com/dvnstrcklnd/pfish-tools.git
cd pfish-tools
docker build -t push_and_test .
install -m 0755 push-and-test-wrapper ~/bin/push-and-test

Usage

To run the app:

cd path/to/my_pfish_protocols
push-and-test -c 'Demo' -o 'Foo'

This will push the dependencies of OperationType Foo, then push and test Foo itself. It will also record the push time for each push in dependencies.json. The next time you run the script, it will only push the files that have been saved since the last push.

To push all updated without testing, use the -p flag:

push-and-test -c 'Demo' -o 'Foo' -p

To push all regardless of whether they have been updated, use the -f flag:

push-and-test -c 'Demo' -o 'Foo' [-p] -f

If you would like to only build or update the dependencies file, then run the app with the -B flag:

push-and-test -c 'Demo' -o 'Foo' -B

Note that the -p and -f flags have no effect when passed with the -B flag.

At this time you can only push based on the OperationType. Use pfish for pushing individual Libraries.

Local Data

The dependency information is stored in a JSON file like this one:

[
  {
    "category": "Demo",            # OperationType
    "name": "Foo",
    "last_push": {},
    "libraries": [
      {
        "category": "Demo Libs",   # Library 1
        "name": "FooBar",
        "last_push": {}
      },
      {
        "category": "Demo Libs",   # Library 2
        "name": "FooBaz",
        "last_push": {}
      }
    ]
  }
]

where Foo is the OperationType and FooBar and FooBaz are Libraries that Foo depends on. The app will push all the files for the OperationType and its dependencies only if they have been modified since the last push.

By default, dependencies are managed automatically in a hidden file. You might like to specify the location of the file, for example:

`-- my_pfish_protocols
    |-- dependencies.json <<<
    |-- demo-repo
    |   |-- demo
    |   `-- demo_libs
    `-- another-repo

You can run the app with a user-specified dependencies file by passing the -d option

cd path/to/my_pfish_protocols
push-and-test -d dependencies.json -c 'Demo' -o 'Foo'

TODO

1. Streamline installation of Docker image. Make an install script like the one for pfish.
2. Put Format Test Results into Docker app. This script is almost always run immediately following a test. This could be automatic. It would also be straightforward to have the script only format the results generated by that test, rather than scraping the entire tree.
3. Normalize library data. If operation types A and B both use libarary C, A and B shouldn't each store a different version of the last push time for C.
4. Fold into pfish. This would obviate 1-3, and may be a more straightforward path.