Skip to content

Commit 253d4e3

Browse files
committed
Made small changes to repo README and sr-eval/README, added proper preds directory.
1 parent c1b0dea commit 253d4e3

File tree

3 files changed

+62831
-2
lines changed

3 files changed

+62831
-2
lines changed

README.md

+1
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Each subdirectory of the repository is an evaluation task within the project. Ea
99
* golds - The gold-truth standard, humanly-annotated files by which apps are evaluated for predictive ability.
1010
* often are `.tsv` or `.csv` or `.txt`.
1111
* preds/predictions - The app-predicted files with predicted-annotations of what phenomena are to be evaluated. (e.g. time durations for slate detection.)
12+
* each preds directory represents a batch, with naming conventions as follows:`preds@<APP_NAME><APP_VER>@<BATCH_NAME>`
1213
* are always `.mmif` files with app views.
1314
#### Outputs to Evaluations
1415
* results - This should be the result system output of the evaluation numbers from a finished evaluation.

sr-eval/README.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ To run this evaluation script, you need the following:
77

88
* Set of predictions in MMIF format (either from the preds folder in this repo
99
or generated from the [SWT app](https://github.com/clamsproject/app-swt-detection) )
10-
* Set of golds in csv format (either the golds directory in this repo or downloaded from the annotations repository
11-
using goldretriever.py, or your own set that exactly matches the format of the included golds.)
10+
* Set of golds in csv format (either downloaded from the annotations repository
11+
using goldretriever.py, or your own set that exactly matches the format present in [aapb-annotations](https://github.com/clamsproject/aapb-annotations/tree/main/scene-recognition/golds))
1212

1313
There are three arguments when running the script: `-mmif-dir`, `-gold-dir`, and `count-subtypes`.
1414
The first two are directories that contain the predictions and golds, respectively. The third is a boolean value that

0 commit comments

Comments
 (0)