Skip to content

Commit 1e9b7fe

Browse files
author
clams-bot
committed
adding metadata of llava-captioner.v1.2
1 parent 14159ac commit 1e9b7fe

File tree

5 files changed

+182
-15
lines changed

5 files changed

+182
-15
lines changed

‎docs/_apps/llava-captioner/v1.2/index.md

+90
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
layout: posts
3+
classes: wide
4+
title: "LLaVA Captioner (v1.2)"
5+
date: 2025-02-05T22:52:42+00:00
6+
---
7+
## About this version
8+
9+
- Submitter: [kelleyl](https://github.com/kelleyl)
10+
- Submission Time: 2025-02-05T22:52:42+00:00
11+
- Prebuilt Container Image: [ghcr.io/clamsproject/app-llava-captioner:v1.2](https://github.com/clamsproject/app-llava-captioner/pkgs/container/app-llava-captioner/v1.2)
12+
- Release Notes
13+
14+
(no notes provided by the developer)
15+
16+
## About this app (See raw [metadata.json](metadata.json))
17+
18+
**Applies LLaVA v1.6 Mistral-7B to video frames for image captioning.**
19+
20+
- App ID: [http://apps.clams.ai/llava-captioner/v1.2](http://apps.clams.ai/llava-captioner/v1.2)
21+
- App License: Apache 2.0
22+
- Source Repository: [https://github.com/clamsproject/app-llava-captioner](https://github.com/clamsproject/app-llava-captioner) ([source tree of the submitted version](https://github.com/clamsproject/app-llava-captioner/tree/v1.2))
23+
24+
25+
#### Inputs
26+
(**Note**: "*" as a property value means that the property is required but can be any value.)
27+
28+
- [http://mmif.clams.ai/vocabulary/VideoDocument/v1](http://mmif.clams.ai/vocabulary/VideoDocument/v1) (required)
29+
(of any properties)
30+
31+
- [http://mmif.clams.ai/vocabulary/ImageDocument/v1](http://mmif.clams.ai/vocabulary/ImageDocument/v1) (required)
32+
(of any properties)
33+
34+
- [http://mmif.clams.ai/vocabulary/TimeFrame/v5](http://mmif.clams.ai/vocabulary/TimeFrame/v5) (required)
35+
(of any properties)
36+
37+
38+
39+
#### Configurable Parameters
40+
(**Note**: _Multivalued_ means the parameter can have one or more values.)
41+
42+
- `frameInterval`: optional, defaults to `30`
43+
44+
- Type: integer
45+
- Multivalued: False
46+
47+
48+
> The interval at which to extract frames from the video if there are no timeframe annotations. Default is every 30 frames.
49+
- `defaultPrompt`: optional, defaults to `Describe what is shown in this video frame. Analyze the purpose of this frame in the context of a news video. Transcribe any text present.`
50+
51+
- Type: string
52+
- Multivalued: False
53+
54+
55+
> default prompt to use for timeframes not specified in the promptMap. If set to `-`, timeframes not specified in the promptMap will be skipped.
56+
- `promptMap`: optional, defaults to `[]`
57+
58+
- Type: map
59+
- Multivalued: True
60+
61+
62+
> mapping of labels of the input timeframe annotations to new prompts. Must be formatted as "IN_LABEL:PROMPT" (with a colon). To pass multiple mappings, use this parameter multiple times. By default, any timeframe labels not mapped to a prompt will be used with the defaultprompt. In order to skip timeframes with a particular label, pass `-` as the prompt value.in order to skip all timeframes not specified in the promptMap, set the defaultPromptparameter to `-`
63+
- `config`: optional, defaults to `config/default.yaml`
64+
65+
- Type: string
66+
- Multivalued: False
67+
68+
69+
> Name of the config file to use.
70+
- `pretty`: optional, defaults to `false`
71+
72+
- Type: boolean
73+
- Multivalued: False
74+
- Choices: **_`false`_**, `true`
75+
76+
77+
> The JSON body of the HTTP response will be re-formatted with 2-space indentation
78+
79+
80+
#### Outputs
81+
(**Note**: "*" as a property value means that the property is required but can be any value.)
82+
83+
(**Note**: Not all output annotations are always generated.)
84+
85+
- [http://mmif.clams.ai/vocabulary/Alignment/v1](http://mmif.clams.ai/vocabulary/Alignment/v1)
86+
(of any properties)
87+
88+
- [http://mmif.clams.ai/vocabulary/TextDocument/v1](http://mmif.clams.ai/vocabulary/TextDocument/v1)
89+
(of any properties)
90+

‎docs/_apps/llava-captioner/v1.2/metadata.json

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
{
2+
"name": "LLaVA Captioner",
3+
"description": "Applies LLaVA v1.6 Mistral-7B to video frames for image captioning.",
4+
"app_version": "v1.2",
5+
"mmif_version": "1.0.5",
6+
"app_license": "Apache 2.0",
7+
"identifier": "http://apps.clams.ai/llava-captioner/v1.2",
8+
"url": "https://github.com/clamsproject/app-llava-captioner",
9+
"input": [
10+
{
11+
"@type": "http://mmif.clams.ai/vocabulary/VideoDocument/v1",
12+
"required": true
13+
},
14+
{
15+
"@type": "http://mmif.clams.ai/vocabulary/ImageDocument/v1",
16+
"required": true
17+
},
18+
{
19+
"@type": "http://mmif.clams.ai/vocabulary/TimeFrame/v5",
20+
"required": true
21+
}
22+
],
23+
"output": [
24+
{
25+
"@type": "http://mmif.clams.ai/vocabulary/Alignment/v1"
26+
},
27+
{
28+
"@type": "http://mmif.clams.ai/vocabulary/TextDocument/v1"
29+
}
30+
],
31+
"parameters": [
32+
{
33+
"name": "frameInterval",
34+
"description": "The interval at which to extract frames from the video if there are no timeframe annotations. Default is every 30 frames.",
35+
"type": "integer",
36+
"default": 30,
37+
"multivalued": false
38+
},
39+
{
40+
"name": "defaultPrompt",
41+
"description": "default prompt to use for timeframes not specified in the promptMap. If set to `-`, timeframes not specified in the promptMap will be skipped.",
42+
"type": "string",
43+
"default": "Describe what is shown in this video frame. Analyze the purpose of this frame in the context of a news video. Transcribe any text present.",
44+
"multivalued": false
45+
},
46+
{
47+
"name": "promptMap",
48+
"description": "mapping of labels of the input timeframe annotations to new prompts. Must be formatted as \"IN_LABEL:PROMPT\" (with a colon). To pass multiple mappings, use this parameter multiple times. By default, any timeframe labels not mapped to a prompt will be used with the defaultprompt. In order to skip timeframes with a particular label, pass `-` as the prompt value.in order to skip all timeframes not specified in the promptMap, set the defaultPromptparameter to `-`",
49+
"type": "map",
50+
"default": [],
51+
"multivalued": true
52+
},
53+
{
54+
"name": "config",
55+
"description": "Name of the config file to use.",
56+
"type": "string",
57+
"default": "config/default.yaml",
58+
"multivalued": false
59+
},
60+
{
61+
"name": "pretty",
62+
"description": "The JSON body of the HTTP response will be re-formatted with 2-space indentation",
63+
"type": "boolean",
64+
"default": false,
65+
"multivalued": false
66+
}
67+
]
68+
}

‎docs/_apps/llava-captioner/v1.2/submission.json

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"time": "2025-02-05T22:52:42+00:00",
3+
"submitter": "kelleyl",
4+
"image": "ghcr.io/clamsproject/app-llava-captioner:v1.2"
5+
}

‎docs/_data/app-index.json

+18-14
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,22 @@
11
{
2+
"http://apps.clams.ai/llava-captioner": {
3+
"description": "Applies llava to video frames.",
4+
"latest_update": "2025-02-05T22:52:42+00:00",
5+
"versions": [
6+
[
7+
"v1.2",
8+
"kelleyl"
9+
],
10+
[
11+
"v1.1",
12+
"kelleyl"
13+
],
14+
[
15+
"v1.0",
16+
"kelleyl"
17+
]
18+
]
19+
},
220
"http://apps.clams.ai/swt-detection": {
321
"description": "Detects scenes with text, like slates, chyrons and credits.",
422
"latest_update": "2025-02-03T22:07:54+00:00",
@@ -243,20 +261,6 @@
243261
]
244262
]
245263
},
246-
"http://apps.clams.ai/llava-captioner": {
247-
"description": "Applies llava to video frames.",
248-
"latest_update": "2024-05-23T19:38:32+00:00",
249-
"versions": [
250-
[
251-
"v1.1",
252-
"kelleyl"
253-
],
254-
[
255-
"v1.0",
256-
"kelleyl"
257-
]
258-
]
259-
},
260264
"http://apps.clams.ai/inaspeechsegmenter-wrapper": {
261265
"description": "inaSpeechSegmenter is a CNN-based audio segmentation toolkit. The original software can be found at https://github.com/ina-foss/inaSpeechSegmenter .",
262266
"latest_update": "2024-05-07T03:45:14+00:00",

‎docs/_data/apps.json

+1-1
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)