Skip to content

Commit 4ccc45d

Browse files
author
clams-bot
committed
adding metadata of swt-detection.v7.1
1 parent 0e0414f commit 4ccc45d

File tree

5 files changed

+383
-2
lines changed

5 files changed

+383
-2
lines changed

‎docs/_apps/swt-detection/v7.1/index.md

+180
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
---
2+
layout: posts
3+
classes: wide
4+
title: "Scenes-with-text Detection (v7.1)"
5+
date: 2024-11-25T18:27:38+00:00
6+
---
7+
## About this version
8+
9+
- Submitter: [keighrim](https://github.com/keighrim)
10+
- Submission Time: 2024-11-25T18:27:38+00:00
11+
- Prebuilt Container Image: [ghcr.io/clamsproject/app-swt-detection:v7.1](https://github.com/clamsproject/app-swt-detection/pkgs/container/app-swt-detection/v7.1)
12+
- Release Notes
13+
14+
> Release with newly trained models:
15+
> - training data is expanded with [new annotations](https://github.com/clamsproject/aapb-annotations/pull/98).
16+
> - label `U` is added, total number of "raw" labels is now 18.
17+
> - in additional to `convnext_lg` and `convnext_tiny`, `convnext_small`-based models are added. The default is now `convnext_small` model.
18+
19+
## About this app (See raw [metadata.json](metadata.json))
20+
21+
**Detects scenes with text, like slates, chyrons and credits. This app can run in three modes, depending on `useClassifier`, `useStitcher` parameters. When `useClassifier=True`, it runs in the "TimePoint mode" and generates TimePoint annotations. When `useStitcher=True`, it runs in the "TimeFrame mode" and generates TimeFrame annotations based on existing TimePoint annotations -- if no TimePoint is found, it produces an error. By default, it runs in the 'both' mode and first generates TimePoint annotations and then TimeFrame annotations on them.**
22+
23+
- App ID: [http://apps.clams.ai/swt-detection/v7.1](http://apps.clams.ai/swt-detection/v7.1)
24+
- App License: Apache 2.0
25+
- Source Repository: [https://github.com/clamsproject/app-swt-detection](https://github.com/clamsproject/app-swt-detection) ([source tree of the submitted version](https://github.com/clamsproject/app-swt-detection/tree/v7.1))
26+
27+
28+
#### Inputs
29+
(**Note**: "*" as a property value means that the property is required but can be any value.)
30+
31+
- [http://mmif.clams.ai/vocabulary/VideoDocument/v1](http://mmif.clams.ai/vocabulary/VideoDocument/v1) (required)
32+
(of any properties)
33+
34+
35+
36+
#### Configurable Parameters
37+
(**Note**: _Multivalued_ means the parameter can have one or more values.)
38+
39+
- `useClassifier`: optional, defaults to `true`
40+
41+
- Type: boolean
42+
- Multivalued: False
43+
- Choices: `false`, **_`true`_**
44+
45+
46+
> Use the image classifier model to generate TimePoint annotations.
47+
- `tpModelName`: optional, defaults to `convnext_small`
48+
49+
- Type: string
50+
- Multivalued: False
51+
- Choices: `convnext_lg`, **_`convnext_small`_**, `convnext_tiny`
52+
53+
54+
> Model name to use for classification, only applies when `useClassifier=true`.
55+
- `tpUsePosModel`: optional, defaults to `true`
56+
57+
- Type: boolean
58+
- Multivalued: False
59+
- Choices: `false`, **_`true`_**
60+
61+
62+
> Use the model trained with positional features, only applies when `useClassifier=true`.
63+
- `tpStartAt`: optional, defaults to `0`
64+
65+
- Type: integer
66+
- Multivalued: False
67+
68+
69+
> Number of milliseconds into the video to start processing, only applies when `useClassifier=true`.
70+
- `tpStopAt`: optional, defaults to `9223372036854775807`
71+
72+
- Type: integer
73+
- Multivalued: False
74+
75+
76+
> Number of milliseconds into the video to stop processing, only applies when `useClassifier=true`.
77+
- `tpSampleRate`: optional, defaults to `1000`
78+
79+
- Type: integer
80+
- Multivalued: False
81+
82+
83+
> Milliseconds between sampled frames, only applies when `useClassifier=true`.
84+
- `useStitcher`: optional, defaults to `true`
85+
86+
- Type: boolean
87+
- Multivalued: False
88+
- Choices: `false`, **_`true`_**
89+
90+
91+
> Use the stitcher after classifying the TimePoints.
92+
- `tfMinTPScore`: optional, defaults to `0.5`
93+
94+
- Type: number
95+
- Multivalued: False
96+
97+
98+
> Minimum score for a TimePoint to be included in a TimeFrame. A lower value will include more TimePoints in the TimeFrame (increasing recall in exchange for precision). Only applies when `useStitcher=true`.
99+
- `tfMinTFScore`: optional, defaults to `0.9`
100+
101+
- Type: number
102+
- Multivalued: False
103+
104+
105+
> Minimum score for a TimeFrame. A lower value will include more TimeFrames in the output (increasing recall in exchange for precision). Only applies when `useStitcher=true`
106+
- `tfMinTFDuration`: optional, defaults to `5000`
107+
108+
- Type: integer
109+
- Multivalued: False
110+
111+
112+
> Minimum duration of a TimeFrame in milliseconds, only applies when `useStitcher=true`.
113+
- `tfAllowOverlap`: optional, defaults to `false`
114+
115+
- Type: boolean
116+
- Multivalued: False
117+
- Choices: **_`false`_**, `true`
118+
119+
120+
> Allow overlapping time frames, only applies when `useStitcher=true`
121+
- `tfDynamicSceneLabels`: optional, defaults to `['credit', 'credits']`
122+
123+
- Type: string
124+
- Multivalued: True
125+
126+
127+
> Labels that are considered dynamic scenes. For dynamic scenes, TimeFrame annotations contains multiple representative points to follow any changes in the scene. Only applies when `useStitcher=true`
128+
- `tfLabelMap`: optional, defaults to `[]`
129+
130+
- Type: map
131+
- Multivalued: True
132+
133+
134+
> (See also `tfLabelMapPreset`, set `tfLabelMapPreset=nopreset` to make sure that a preset does not override `tfLabelMap` when using this) Mapping of a label in the input TimePoint annotations to a new label of the stitched TimeFrame annotations. Must be formatted as IN_LABEL:OUT_LABEL (with a colon). To pass multiple mappings, use this parameter multiple times. When two+ TP labels are mapped to a TF label, it essentially works as a "binning" operation. If no mapping is used, all the input labels are passed-through, meaning no change in both TP & TF labelsets. However, when at least one label is mapped, all the other "unset" labels are mapped to the negative label (`-`) and if `-` does not exist in the TF labelset, it is added automatically. Only applies when `useStitcher=true`.
135+
- `tfLabelMapPreset`: optional, defaults to `relaxed`
136+
137+
- Type: string
138+
- Multivalued: False
139+
- Choices: `noprebin`, `nomap`, `strict`, `simpler`, `simple`, **_`relaxed`_**, `binary-bars`, `binary-slate`, `binary-chyron-strict`, `binary-chyron-relaxed`, `binary-credits`
140+
141+
142+
> (See also `tfLabelMap`) Preset alias of a label mapping. If not `nopreset`, this parameter will override the `tfLabelMap` parameter. Available presets are:<br/>- `noprebin`: []<br/>- `nomap`: []<br/>- `strict`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`I`:`Chyron-person`', '`N`:`Chyron-person`', '`C`:`Credits`', '`R`:`Credits`', '`M`:`Main`', '`O`:`Opening`', '`W`:`Opening`', '`Y`:`Chyron-other`', '`U`:`Chyron-other`', '`K`:`Chyron-other`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`']<br/>- `simpler`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`I`:`Chyron`', '`N`:`Chyron`', '`C`:`Credits`', '`R`:`Credits`']<br/>- `simple`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`I`:`Chyron-person`', '`N`:`Chyron-person`', '`C`:`Credits`', '`R`:`Credits`', '`M`:`Other-text`', '`O`:`Other-text`', '`W`:`Other-text`', '`Y`:`Other-text`', '`U`:`Other-text`', '`K`:`Other-text`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`']<br/>- `relaxed`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`Y`:`Chyron`', '`U`:`Chyron`', '`K`:`Chyron`', '`I`:`Chyron`', '`N`:`Chyron`', '`C`:`Credits`', '`R`:`Credits`', '`M`:`Other-text`', '`O`:`Other-text`', '`W`:`Other-text`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`']<br/>- `binary-bars`: ['`B`:`Bars`']<br/>- `binary-slate`: ['`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`']<br/>- `binary-chyron-strict`: ['`I`:`Chyron-person`', '`N`:`Chyron-person`']<br/>- `binary-chyron-relaxed`: ['`Y`:`Chyron`', '`U`:`Chyron`', '`K`:`Chyron`', '`I`:`Chyron`', '`N`:`Chyron`']<br/>- `binary-credits`: ['`C`:`Credits`', '`R`:`Credits`']<br/><br/> Only applies when `useStitcher=true`.
143+
- `pretty`: optional, defaults to `false`
144+
145+
- Type: boolean
146+
- Multivalued: False
147+
- Choices: **_`false`_**, `true`
148+
149+
150+
> The JSON body of the HTTP response will be re-formatted with 2-space indentation
151+
- `runningTime`: optional, defaults to `false`
152+
153+
- Type: boolean
154+
- Multivalued: False
155+
- Choices: **_`false`_**, `true`
156+
157+
158+
> The running time of the app will be recorded in the view metadata
159+
- `hwFetch`: optional, defaults to `false`
160+
161+
- Type: boolean
162+
- Multivalued: False
163+
- Choices: **_`false`_**, `true`
164+
165+
166+
> The hardware information (architecture, GPU and vRAM) will be recorded in the view metadata
167+
168+
169+
#### Outputs
170+
(**Note**: "*" as a property value means that the property is required but can be any value.)
171+
172+
(**Note**: Not all output annotations are always generated.)
173+
174+
- [http://mmif.clams.ai/vocabulary/TimeFrame/v5](http://mmif.clams.ai/vocabulary/TimeFrame/v5)
175+
- _timeUnit_ = "milliseconds"
176+
177+
- [http://mmif.clams.ai/vocabulary/TimePoint/v4](http://mmif.clams.ai/vocabulary/TimePoint/v4)
178+
- _timeUnit_ = "milliseconds"
179+
- _labelset_ = a list of ["B", "S", "I", "C", "R", "M", "O", "W", "N", "Y", "U", "K", "L", "G", "F", "E", "T", "P"]
180+

‎docs/_apps/swt-detection/v7.1/metadata.json

+191
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
{
2+
"name": "Scenes-with-text Detection",
3+
"description": "Detects scenes with text, like slates, chyrons and credits. This app can run in three modes, depending on `useClassifier`, `useStitcher` parameters. When `useClassifier=True`, it runs in the \"TimePoint mode\" and generates TimePoint annotations. When `useStitcher=True`, it runs in the \"TimeFrame mode\" and generates TimeFrame annotations based on existing TimePoint annotations -- if no TimePoint is found, it produces an error. By default, it runs in the 'both' mode and first generates TimePoint annotations and then TimeFrame annotations on them.",
4+
"app_version": "v7.1",
5+
"mmif_version": "1.0.5",
6+
"app_license": "Apache 2.0",
7+
"identifier": "http://apps.clams.ai/swt-detection/v7.1",
8+
"url": "https://github.com/clamsproject/app-swt-detection",
9+
"input": [
10+
{
11+
"@type": "http://mmif.clams.ai/vocabulary/VideoDocument/v1",
12+
"required": true
13+
}
14+
],
15+
"output": [
16+
{
17+
"@type": "http://mmif.clams.ai/vocabulary/TimeFrame/v5",
18+
"properties": {
19+
"timeUnit": "milliseconds"
20+
}
21+
},
22+
{
23+
"@type": "http://mmif.clams.ai/vocabulary/TimePoint/v4",
24+
"properties": {
25+
"timeUnit": "milliseconds",
26+
"labelset": [
27+
"B",
28+
"S",
29+
"I",
30+
"C",
31+
"R",
32+
"M",
33+
"O",
34+
"W",
35+
"N",
36+
"Y",
37+
"U",
38+
"K",
39+
"L",
40+
"G",
41+
"F",
42+
"E",
43+
"T",
44+
"P"
45+
]
46+
}
47+
}
48+
],
49+
"parameters": [
50+
{
51+
"name": "useClassifier",
52+
"description": "Use the image classifier model to generate TimePoint annotations.",
53+
"type": "boolean",
54+
"default": true,
55+
"multivalued": false
56+
},
57+
{
58+
"name": "tpModelName",
59+
"description": "Model name to use for classification, only applies when `useClassifier=true`.",
60+
"type": "string",
61+
"choices": [
62+
"convnext_lg",
63+
"convnext_small",
64+
"convnext_tiny"
65+
],
66+
"default": "convnext_small",
67+
"multivalued": false
68+
},
69+
{
70+
"name": "tpUsePosModel",
71+
"description": "Use the model trained with positional features, only applies when `useClassifier=true`.",
72+
"type": "boolean",
73+
"default": true,
74+
"multivalued": false
75+
},
76+
{
77+
"name": "tpStartAt",
78+
"description": "Number of milliseconds into the video to start processing, only applies when `useClassifier=true`.",
79+
"type": "integer",
80+
"default": 0,
81+
"multivalued": false
82+
},
83+
{
84+
"name": "tpStopAt",
85+
"description": "Number of milliseconds into the video to stop processing, only applies when `useClassifier=true`.",
86+
"type": "integer",
87+
"default": 9223372036854775807,
88+
"multivalued": false
89+
},
90+
{
91+
"name": "tpSampleRate",
92+
"description": "Milliseconds between sampled frames, only applies when `useClassifier=true`.",
93+
"type": "integer",
94+
"default": 1000,
95+
"multivalued": false
96+
},
97+
{
98+
"name": "useStitcher",
99+
"description": "Use the stitcher after classifying the TimePoints.",
100+
"type": "boolean",
101+
"default": true,
102+
"multivalued": false
103+
},
104+
{
105+
"name": "tfMinTPScore",
106+
"description": "Minimum score for a TimePoint to be included in a TimeFrame. A lower value will include more TimePoints in the TimeFrame (increasing recall in exchange for precision). Only applies when `useStitcher=true`.",
107+
"type": "number",
108+
"default": 0.5,
109+
"multivalued": false
110+
},
111+
{
112+
"name": "tfMinTFScore",
113+
"description": "Minimum score for a TimeFrame. A lower value will include more TimeFrames in the output (increasing recall in exchange for precision). Only applies when `useStitcher=true`",
114+
"type": "number",
115+
"default": 0.9,
116+
"multivalued": false
117+
},
118+
{
119+
"name": "tfMinTFDuration",
120+
"description": "Minimum duration of a TimeFrame in milliseconds, only applies when `useStitcher=true`.",
121+
"type": "integer",
122+
"default": 5000,
123+
"multivalued": false
124+
},
125+
{
126+
"name": "tfAllowOverlap",
127+
"description": "Allow overlapping time frames, only applies when `useStitcher=true`",
128+
"type": "boolean",
129+
"default": false,
130+
"multivalued": false
131+
},
132+
{
133+
"name": "tfDynamicSceneLabels",
134+
"description": "Labels that are considered dynamic scenes. For dynamic scenes, TimeFrame annotations contains multiple representative points to follow any changes in the scene. Only applies when `useStitcher=true`",
135+
"type": "string",
136+
"default": [
137+
"credit",
138+
"credits"
139+
],
140+
"multivalued": true
141+
},
142+
{
143+
"name": "tfLabelMap",
144+
"description": "(See also `tfLabelMapPreset`, set `tfLabelMapPreset=nopreset` to make sure that a preset does not override `tfLabelMap` when using this) Mapping of a label in the input TimePoint annotations to a new label of the stitched TimeFrame annotations. Must be formatted as IN_LABEL:OUT_LABEL (with a colon). To pass multiple mappings, use this parameter multiple times. When two+ TP labels are mapped to a TF label, it essentially works as a \"binning\" operation. If no mapping is used, all the input labels are passed-through, meaning no change in both TP & TF labelsets. However, when at least one label is mapped, all the other \"unset\" labels are mapped to the negative label (`-`) and if `-` does not exist in the TF labelset, it is added automatically. Only applies when `useStitcher=true`.",
145+
"type": "map",
146+
"default": [],
147+
"multivalued": true
148+
},
149+
{
150+
"name": "tfLabelMapPreset",
151+
"description": "(See also `tfLabelMap`) Preset alias of a label mapping. If not `nopreset`, this parameter will override the `tfLabelMap` parameter. Available presets are:\n- `noprebin`: []\n- `nomap`: []\n- `strict`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`I`:`Chyron-person`', '`N`:`Chyron-person`', '`C`:`Credits`', '`R`:`Credits`', '`M`:`Main`', '`O`:`Opening`', '`W`:`Opening`', '`Y`:`Chyron-other`', '`U`:`Chyron-other`', '`K`:`Chyron-other`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`']\n- `simpler`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`I`:`Chyron`', '`N`:`Chyron`', '`C`:`Credits`', '`R`:`Credits`']\n- `simple`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`I`:`Chyron-person`', '`N`:`Chyron-person`', '`C`:`Credits`', '`R`:`Credits`', '`M`:`Other-text`', '`O`:`Other-text`', '`W`:`Other-text`', '`Y`:`Other-text`', '`U`:`Other-text`', '`K`:`Other-text`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`']\n- `relaxed`: ['`B`:`Bars`', '`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`', '`Y`:`Chyron`', '`U`:`Chyron`', '`K`:`Chyron`', '`I`:`Chyron`', '`N`:`Chyron`', '`C`:`Credits`', '`R`:`Credits`', '`M`:`Other-text`', '`O`:`Other-text`', '`W`:`Other-text`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`']\n- `binary-bars`: ['`B`:`Bars`']\n- `binary-slate`: ['`S`:`Slate`', '`S:H`:`Slate`', '`S:C`:`Slate`', '`S:D`:`Slate`', '`S:B`:`Slate`', '`S:G`:`Slate`']\n- `binary-chyron-strict`: ['`I`:`Chyron-person`', '`N`:`Chyron-person`']\n- `binary-chyron-relaxed`: ['`Y`:`Chyron`', '`U`:`Chyron`', '`K`:`Chyron`', '`I`:`Chyron`', '`N`:`Chyron`']\n- `binary-credits`: ['`C`:`Credits`', '`R`:`Credits`']\n\n Only applies when `useStitcher=true`.",
152+
"type": "string",
153+
"choices": [
154+
"noprebin",
155+
"nomap",
156+
"strict",
157+
"simpler",
158+
"simple",
159+
"relaxed",
160+
"binary-bars",
161+
"binary-slate",
162+
"binary-chyron-strict",
163+
"binary-chyron-relaxed",
164+
"binary-credits"
165+
],
166+
"default": "relaxed",
167+
"multivalued": false
168+
},
169+
{
170+
"name": "pretty",
171+
"description": "The JSON body of the HTTP response will be re-formatted with 2-space indentation",
172+
"type": "boolean",
173+
"default": false,
174+
"multivalued": false
175+
},
176+
{
177+
"name": "runningTime",
178+
"description": "The running time of the app will be recorded in the view metadata",
179+
"type": "boolean",
180+
"default": false,
181+
"multivalued": false
182+
},
183+
{
184+
"name": "hwFetch",
185+
"description": "The hardware information (architecture, GPU and vRAM) will be recorded in the view metadata",
186+
"type": "boolean",
187+
"default": false,
188+
"multivalued": false
189+
}
190+
]
191+
}

‎docs/_apps/swt-detection/v7.1/submission.json

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"time": "2024-11-25T18:27:38+00:00",
3+
"submitter": "keighrim",
4+
"image": "ghcr.io/clamsproject/app-swt-detection:v7.1",
5+
"releasenotes": "Release with newly trained models:\n\n- training data is expanded with [new annotations](https://github.com/clamsproject/aapb-annotations/pull/98).\n- label `U` is added, total number of \"raw\" labels is now 18.\n- in additional to `convnext_lg` and `convnext_tiny`, `convnext_small`-based models are added. The default is now `convnext_small` model.\n\n"
6+
}

‎docs/_data/app-index.json

+5-1
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
11
{
22
"http://apps.clams.ai/swt-detection": {
33
"description": "Detects scenes with text, like slates, chyrons and credits.",
4-
"latest_update": "2024-11-04T22:00:05+00:00",
4+
"latest_update": "2024-11-25T18:27:38+00:00",
55
"versions": [
6+
[
7+
"v7.1",
8+
"keighrim"
9+
],
610
[
711
"v7.0",
812
"keighrim"

‎docs/_data/apps.json

+1-1
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)