ValueError: fps and nframe should be set at least one valid value

[aimanu]

Hi,

My config is as follows:

{
    "model": {
        "deepseek_vl_1.3b": {
            "class": "DeepSeekVL",
            "model": "deepseek-vl-1.3b-chat"
        },
        "Qwen2-VL-7B-Instruct": {
            "class": "Qwen2VLChat",
            "model_path": "/lpai/volumes/profile-vehicle-control-vol/model/Qwen/Qwen2-VL-7B-Instruct"
        }
    },
    "data": {
        "MME-RealWorld-Lite": {
            "class": "MMERealWorld",
            "dataset": "MME-RealWorld-Lite",
            "nframe" :1,
            "fps":1
        }
    }
}

It gave the error saying "
[2025-01-27 02:05:51,584] ERROR - RUN - run.py: main - 416: Model Qwen2-VL-7B-Instruct x Dataset MME-RealWorld-Lite combination failed: fps and nframe should be set at least one valid value, skipping this combination.
Traceback (most recent call last):
File "/lpai/VLMEvalKit/run.py", line 222, in main
dataset = build_dataset_from_config(cfg['data'], dataset_name)
File "/lpai/VLMEvalKit/run.py", line 45, in build_dataset_from_config
raise ValueError('fps and nframe should be set at least one valid value')
ValueError: fps and nframe should be set at least one valid value
[2025-01-27 02:05:51] ERROR - run.py: main - 416: Model Qwen2-VL-7B-Instruct x Dataset MME-RealWorld-Lite combination failed: fps and nframe should be set at least one valid value, skipping this combination.
Traceback (most recent call last):
File "/lpai/VLMEvalKit/run.py", line 222, in main
dataset = build_dataset_from_config(cfg['data'], dataset_name)
File "/lpai/VLMEvalKit/run.py", line 45, in build_dataset_from_config
raise ValueError('fps and nframe should be set at least one valid value')
ValueError: fps and nframe should be set at least one valid value
root@lx-dev-0:/lpai#
"
I also tried the dataset config without "nframe" and "fps" parameter, the error is same.

Please help with this issue. Thanks.

[FangXinyu-0913]

Hi @aimanu, MME-RealWorld-Lite is not a video dataset, so you do not need to pass nframe or fps argument in the config.
You can take the config below to have a try:

{
    "model": {
        "deepseek_vl_1.3b": {
            "class": "DeepSeekVL",
            "model": "deepseek-vl-1.3b-chat"
        },
        "Qwen2-VL-7B-Instruct": {
            "class": "Qwen2VLChat",
            "model_path": "/lpai/volumes/profile-vehicle-control-vol/model/Qwen/Qwen2-VL-7B-Instruct"
        }
    },
    "data": {
        "MME-RealWorld-Lite": {
            "class": "MMERealWorld",
            "dataset": "MME-RealWorld-Lite"
        }
    }
}

[aimanu]

Hi @aimanu, MME-RealWorld-Lite is not a video dataset, so you do not need to pass nframe or fps argument in the config. You can take the config below to have a try:

{
    "model": {
        "deepseek_vl_1.3b": {
            "class": "DeepSeekVL",
            "model": "deepseek-vl-1.3b-chat"
        },
        "Qwen2-VL-7B-Instruct": {
            "class": "Qwen2VLChat",
            "model_path": "/lpai/volumes/profile-vehicle-control-vol/model/Qwen/Qwen2-VL-7B-Instruct"
        }
    },
    "data": {
        "MME-RealWorld-Lite": {
            "class": "MMERealWorld",
            "dataset": "MME-RealWorld-Lite"
        }
    }
}

@FangXinyu-0913 Thanks for the reply. Actually I have tried that configuration. It raised an error telling "ValueError: fps and nframe should be set at least one valid value".
The whole information is as follows: