ValueError: fps and nframe should be set at least one valid value

[aimanu]

Hi,

My config is as follows:

{
    "model": {
        "deepseek_vl_1.3b": {
            "class": "DeepSeekVL",
            "model": "deepseek-vl-1.3b-chat"
        },
        "Qwen2-VL-7B-Instruct": {
            "class": "Qwen2VLChat",
            "model_path": "/lpai/volumes/profile-vehicle-control-vol/model/Qwen/Qwen2-VL-7B-Instruct"
        }
    },
    "data": {
        "MME-RealWorld-Lite": {
            "class": "MMERealWorld",
            "dataset": "MME-RealWorld-Lite",
            "nframe" :1,
            "fps":1
        }
    }
}

It gave the error saying "
[2025-01-27 02:05:51,584] ERROR - RUN - run.py: main - 416: Model Qwen2-VL-7B-Instruct x Dataset MME-RealWorld-Lite combination failed: fps and nframe should be set at least one valid value, skipping this combination.
Traceback (most recent call last):
File "/lpai/VLMEvalKit/run.py", line 222, in main
dataset = build_dataset_from_config(cfg['data'], dataset_name)
File "/lpai/VLMEvalKit/run.py", line 45, in build_dataset_from_config
raise ValueError('fps and nframe should be set at least one valid value')
ValueError: fps and nframe should be set at least one valid value
[2025-01-27 02:05:51] ERROR - run.py: main - 416: Model Qwen2-VL-7B-Instruct x Dataset MME-RealWorld-Lite combination failed: fps and nframe should be set at least one valid value, skipping this combination.
Traceback (most recent call last):
File "/lpai/VLMEvalKit/run.py", line 222, in main
dataset = build_dataset_from_config(cfg['data'], dataset_name)
File "/lpai/VLMEvalKit/run.py", line 45, in build_dataset_from_config
raise ValueError('fps and nframe should be set at least one valid value')
ValueError: fps and nframe should be set at least one valid value
root@lx-dev-0:/lpai#
"
I also tried the dataset config without "nframe" and "fps" parameter, the error is same.

Please help with this issue. Thanks.

[FangXinyu-0913]

Hi @aimanu, MME-RealWorld-Lite is not a video dataset, so you do not need to pass nframe or fps argument in the config.
You can take the config below to have a try:

{
    "model": {
        "deepseek_vl_1.3b": {
            "class": "DeepSeekVL",
            "model": "deepseek-vl-1.3b-chat"
        },
        "Qwen2-VL-7B-Instruct": {
            "class": "Qwen2VLChat",
            "model_path": "/lpai/volumes/profile-vehicle-control-vol/model/Qwen/Qwen2-VL-7B-Instruct"
        }
    },
    "data": {
        "MME-RealWorld-Lite": {
            "class": "MMERealWorld",
            "dataset": "MME-RealWorld-Lite"
        }
    }
}

[aimanu]

Hi @aimanu, MME-RealWorld-Lite is not a video dataset, so you do not need to pass nframe or fps argument in the config. You can take the config below to have a try:

{
    "model": {
        "deepseek_vl_1.3b": {
            "class": "DeepSeekVL",
            "model": "deepseek-vl-1.3b-chat"
        },
        "Qwen2-VL-7B-Instruct": {
            "class": "Qwen2VLChat",
            "model_path": "/lpai/volumes/profile-vehicle-control-vol/model/Qwen/Qwen2-VL-7B-Instruct"
        }
    },
    "data": {
        "MME-RealWorld-Lite": {
            "class": "MMERealWorld",
            "dataset": "MME-RealWorld-Lite"
        }
    }
}

@FangXinyu-0913 Thanks for the reply. Actually I have tried that configuration. It raised an error telling "ValueError: fps and nframe should be set at least one valid value".
The whole information is as follows:

[FangXinyu-0913]

Hi @aimanu, Thank you for pointing out this issue. We have fixed it in the latest code, and you can pull the code again to try. If you have any questions, feel free to leave a message below and at me.

[aimanu]

Hi @aimanu, Thank you for pointing out this issue. We have fixed it in the latest code, and you can pull the code again to try. If you have any questions, feel free to leave a message below and at me.

Thanks a lot.