EvalHub 🚀

Discover and Review Evaluation Metrics for Your AI Models!

Description

EvalHub is a platform designed to help you discover and review evaluation metrics for your machine learning models. Explore a variety of metrics here and then head over to the lm-evaluation-harness repository by EleutherAI to evaluate your models.

Features

• Metric Discovery: Search and filter through a comprehensive list of evaluation metrics.
• Detailed Information: View detailed descriptions, GitHub links, and paper links for each metric.
• Categorization: Metrics are organized into categories for easy navigation.
• User Reviews: Coming soon - see reviews and ratings from other researchers.

Roadmap

• Review System: Implement a review system for users to rate and comment on metrics.
• Detailed Analysis: Provide in-depth analysis and comparisons of different metrics.
• Additional Features: More features to be added based on user feedback and needs.

Contributing

• Disclaimer: This platform is a work in progress. Contributions are welcome!
• Feature Requests: If you have an idea for a new feature, please open an issue to discuss it before starting development.

Acknowledgements

• Special thanks to EleutherAI for their lm-evaluation-harness repository, which inspired the creation of this platform.
• We appreciate the contributions of all open-source developers and researchers whose work is referenced and utilized within this platform.
• Thank you to our users and contributors for their valuable feedback and support in improving EvalHub.