|
1 | | -# render-python |
| 1 | +# render-python-apps |
| 2 | + |
| 3 | +These are a set of python based processing modules that do a broad assortment of steps in various image processing workflows whose results are ultimately stored in render. They make extensive use of the render-python library (www.github.com/fcollman/render-python) for reading metadata from render and sending it back. |
| 4 | + |
| 5 | +# running a module |
| 6 | +Each module is designed to be run using a common pattern for defining inputs that is setup using the json_module pattern. |
| 7 | + |
| 8 | +module can be run in 3 different ways |
| 9 | + |
| 10 | +1) passing parameters via command line parameters |
| 11 | +python -m renderapps.module.example_module\ |
| 12 | + --render.host RENDERHOST_DNS_OR_IP\ |
| 13 | + --render.port RENDERPORT\ |
| 14 | + --render.owner OWNERNAME\ |
| 15 | + --render.project PROJECTNAME\ |
| 16 | + --render.client_scripts PATHTORENDERSCRIPTS \ |
| 17 | + --param1 P1\ |
| 18 | + --param2 P2 |
| 19 | + |
| 20 | +note for each module you can get help about the parameters by running |
| 21 | +python -m renderapps.module.example_module --help |
| 22 | + |
| 23 | +2) passing a json file from command line |
| 24 | +python -m renderapps.module.example_module --input_json path_to_json_file |
| 25 | + |
| 26 | +This will read the parameters from the json file passed in. Note that command line arguments can be mixed with json definitions, and command line arguments overwrite parameters specified in the json. |
| 27 | + |
| 28 | +3) calling the module from another python program |
| 29 | + |
| 30 | +from renderapps.module.example_module import ExampleModule |
| 31 | + |
| 32 | +module_parameters = { |
| 33 | + "render":{ |
| 34 | + "host":"RENDERHOST", |
| 35 | + "port":RENDERPORT, |
| 36 | + "owner":"OWNERNAME", |
| 37 | + "project":"PROJECTNAME", |
| 38 | + "client_scripts":"PATHTORENDERSCRIPTS" |
| 39 | + }, |
| 40 | + "param1":"P1", |
| 41 | + "param2":P2, |
| 42 | +} |
| 43 | +mod = ExampleModule(input_data=module_parameters, args = []) |
| 44 | +mod.run() |
| 45 | + |
| 46 | +Note that passing in args =[] will bypass the command line parsing. |
| 47 | + |
| 48 | +No matter how the module is run, all input parameters will be validated using the schema which each module has defined using the marshmallow framework. |
| 49 | + |
| 50 | +#Repo Organization |
| 51 | + |
| 52 | +The code is now organized in a sub-module structure arranged thematically by purpose. |
| 53 | +You can (or will when we are done with documentation) find more detailed descriptions of each submodule in the subfolders. |
| 54 | + |
| 55 | +Here are the thematic areas |
| 56 | + |
| 57 | +## dataimport |
| 58 | +These modules are related to getting data into render, including if you'd like, the generation of downsampled mipmaped image. |
| 59 | + |
| 60 | +## cross_modal_registration |
| 61 | +modules for setting up trakem2 projects to register LM and EM data of the same sections. |
| 62 | + |
| 63 | +## materialize |
| 64 | +modules for creating images on disk of materialized versions of transformed data |
| 65 | + |
| 66 | +## pointmatch |
| 67 | +modules for making plots of point matches or doing operations on the pointmatch database |
| 68 | + |
| 69 | +## registration |
| 70 | +modules for dealing with situations where you have stacks that are registered, |
| 71 | +meaning they all exist in the same space, and but you have a new transformed version of one of those stacks |
| 72 | +but you'd like to bring the rest of the registered stacks along into the new space. |
| 73 | + |
| 74 | +In particular this is useful for array tomography where you have many channels of sections that you must register |
| 75 | +and then you must align the data across sections. |
| 76 | + |
| 77 | +## section_polygons |
| 78 | +modules related to analyzing downsampled versions of section images of DAPI |
| 79 | +stained sections, and using fluorescent glue to automatically find the outline of the section |
| 80 | +Then subsequent modules that help you use those section polygons to filter point matches. |
| 81 | + |
| 82 | +## stack |
| 83 | +modules related to doing bulk operations on a stack. |
| 84 | + |
| 85 | +## stitching |
| 86 | +modules related to stitching array tomography sections, meaning within a section 2d tile montageing. |
| 87 | + |
| 88 | +## tile |
| 89 | +modules related to doing single tile operations. |
| 90 | + |
| 91 | +## TrakEM2 |
| 92 | +modules related to importing data from render>trakem2 and bringing it back again |
| 93 | + |
| 94 | +## transfer |
| 95 | +modules related to pushing render data between two render servers, |
| 96 | +and/or between local storage and the cloud. |
| 97 | + |
| 98 | +## wrinkle detection |
| 99 | +start of work on a module to detect wrinkles in sections |
| 100 | + |
| 101 | +# other folders |
| 102 | + |
| 103 | +## module |
| 104 | +the base module classes for writing json_module style modules |
| 105 | +includes a template module example to work from |
| 106 | + |
| 107 | +## json_module |
| 108 | +a subpackage related to providing a unified interface for setting parameters from a json file, or the command line, or via passing a dictionary to a class. this should go in a seperate repo eventually. |
| 109 | + |
| 110 | +## obsolete |
| 111 | +modules that were written with deprecated version of render python and would need to be rewritten in the new format |
| 112 | +hopefully will go away over time.. leaving useful code here for now. |
| 113 | + |
| 114 | +## refactor |
| 115 | +modules that don't use the consistent json_module pattern and should be refactored |
| 116 | +hopefully will go away as code is moved out of this.. |
| 117 | + |
| 118 | +# allen_utils |
| 119 | +This is a folder of miscelaneous shell scripts and files that are specific to running render python related things at the allen. they might be of interest but won't work in general for people outside of Synapse Biology at the Allen Institute. |
| 120 | + |
| 121 | +# example_json |
| 122 | +This is a folder of example json files that were used to run various modules, could be a useful reference. |
| 123 | + |
| 124 | +# integration_tests |
| 125 | +These should eventually contain tests for running against an integrated deployment environment, specifically configured to run inside an environment like the one specified here (https://github.com/fcollman/render-deploy/tree/test) |
| 126 | + |
| 127 | +# notebooks |
| 128 | +a set of ipython notebooks that show some more interactive use of render-python and render-python apps. Mostly specific to the allen in terms of data sources, but perhaps still useful. |
| 129 | + |
0 commit comments