-
Notifications
You must be signed in to change notification settings - Fork 548
Description
Checklist
- I have searched for similar issues before opening this one.
- I am using the latest version of lmms-eval.
Bug Description
Describe the bug
After installing lmms-eval via pip install lmms-eval==0.6.1, running the command lmms-eval --tasks list returns an empty list for available tasks.
Root Cause Analysis
I investigated the installation directory in site-packages/lmms_eval/tasks and found that no .yaml configuration files were included in the pip package. It seems the resource files (YAML) are missing from the PyPI distribution, causing the TaskManager to find zero tasks.
Questions
Is this a known packaging issue where .yaml files are excluded from the wheel file?
Is it mandatory to install from source (pip install -e .) to get the task configurations?
If I want to keep the pip installation, is there a recommended way to sync the tasks/ directory separately?
Steps to Reproduce
1. pip install lmms-eval==0.6.1
2. lmms-eval --tasks listError Message / Traceback
Environment
- Python 3.11.14
- lmms-eval 0.6.1
Additional Context
No response