MTEB
A Python toolkit evaluates embedding models and retrieval systems across languages and modalities, with an interactive leaderboard.
MTEB helps researchers and developers compare embedding models and retrieval systems for the tasks they need. Its benchmarks cover multiple languages and modalities.
- Task selection: Choose benchmarks, tasks and dataset splits for evaluations such as retrieval, classification, clustering and semantic similarity.
- Model comparisons: Evaluate supported models, including Sentence Transformers, or define a custom model.
- Results and leaderboard: Work with existing model results and explore benchmark performance through an interactive leaderboard.
- Custom benchmarks: Add models, datasets and benchmarks to extend the available evaluations.
MTEB is an open-source Python toolkit licensed under Apache 2.0. You can run evaluations through Python or a command-line interface.
