Skip to content

Commit b5f4b14

Browse files
jyounekeighrim
authored andcommitted
Update [email protected]@240117-aapb-collaboration-27-d.md
changed to permalinks
1 parent 4bfcf32 commit b5f4b14

File tree

2 files changed

+8
-8
lines changed

2 files changed

+8
-8
lines changed

sr-eval/README.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ The goal is to have a simple way of comparing different results from SWT.
77
To run this evaluation script, you need the following:
88

99
* Set of predictions in MMIF format (either from the preds folder in this repo
10-
or generated from the [SWT app](https://github.com/clamsproject/app-swt-detection) )
10+
or generated from the [SWT app](https://github.com/clamsproject/app-swt-detection/tree/6b12498fc596327ec47933b7f785044da2f8cf2f)
1111
* Set of golds in csv format (either downloaded from the annotations repository
12-
using goldretriever.py, or your own set that exactly matches the format present in [aapb-annotations](https://github.com/clamsproject/aapb-annotations/tree/main/scene-recognition/golds))
12+
using goldretriever.py, or your own set that exactly matches the format present in [aapb-annotations](https://github.com/clamsproject/aapb-annotations/tree/9cbe41aa124da73a0158bfc0b4dbf8bafe6d460d/scene-recognition/golds)
1313

1414
There are three arguments when running the script: `-mmif-dir`, `-gold-dir`, and `count-subtypes`.
1515
The first two are directories that contain the predictions and golds, respectively. The third is a boolean value that
@@ -18,7 +18,7 @@ determines if the evaluation takes into account subtype labels or not.
1818
* `preds@app-swt-detection<VERSION-NUMBER>@<BATCH-NAME>`.
1919

2020
Note that only the first one is required, as `-gold-dir` defaults to the set of golds downloaded (using `goldretriever`)
21-
from the [aapb-annotations](https://github.com/clamsproject/aapb-annotations/tree/main/scene-recognition/golds) repo,
21+
from the [aapb-annotations](https://github.com/clamsproject/aapb-annotations/tree/9cbe41aa124da73a0158bfc0b4dbf8bafe6d460d/scene-recognition/golds) repo,
2222
and `count-subtypes` defaults to `False`.
2323

2424
# Usage

sr-eval/[email protected]@240117-aapb-collaboration-27-d.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,10 @@
33

44
## Report Instance of Evaluation Information
55
* 2024-07-03
6-
* [app-SWT-detection](https://github.com/clamsproject/app-swt-detection), version 5.0
7-
* [sr-eval golds](https://github.com/clamsproject/aapb-annotations/tree/main/scene-recognition/golds).
8-
* [[email protected]@240117-aapb-collaboration-27-d](https://github.com/clamsproject/aapb-evaluations/tree/sr-eval-final/sr-eval/preds%40app-swt-detection5.0%40240117-aapb-collaboration-27-d).
9-
* [sr-eval/evaluation.py](https://github.com/clamsproject/aapb-evaluations/blob/sr-eval-final/sr-eval/evaluate.py).
6+
* [app-SWT-detection](https://github.com/clamsproject/app-swt-detection/tree/6b12498fc596327ec47933b7f785044da2f8cf2f), version 5.0
7+
* [scene-recognition/golds](https://github.com/clamsproject/aapb-annotations/tree/9cbe41aa124da73a0158bfc0b4dbf8bafe6d460d/scene-recognition/golds).
8+
* [[email protected]@240117-aapb-collaboration-27-d](https://github.com/clamsproject/aapb-evaluations/tree/4bfcf3250700567aae352a710d5cdb6bc1fcdca4/sr-eval/preds%40app-swt-detection5.0%40240117-aapb-collaboration-27-d).
9+
* [sr-eval/evaluation.py](https://github.com/clamsproject/aapb-evaluations/blob/4bfcf3250700567aae352a710d5cdb6bc1fcdca4/sr-eval/evaluate.py).
1010
* `python evaluate.py -m /path/to/repo/aapb-evaluations/sr-eval/[email protected]@240117-aapb-collaboration-27-d`
1111

1212
## Metrics
@@ -50,4 +50,4 @@ overall micro-averaged set of scores (from the `all` row) is as follows:
5050
## Limitations/Issues
5151
* This script seems to be somewhat inefficient in terms of runtime. This may be due to not utilizing various evaluation
5252
libraries (like numpy), and it might be worthwhile in the future to refactor the script such that those libraries are
53-
used.
53+
used.

0 commit comments

Comments
 (0)