Top 12 Code Generation LM Evaluation Harness alternatives

Future AGI is an open-source platform designed for engineering and optimizing AI agents, integrating tools for trac…

Inspect is an open-source framework designed for evaluating large language models (LLMs), providing tools to assess…

JiWER is a Python library designed to compute Word Error Rate (WER) and other speech recognition accuracy metrics e…

Laminar is an open-source platform designed to trace, evaluate, label, and analyze large language model (LLM) data …

LangTest is an open-source toolkit designed for evaluating and monitoring the performance of natural language proce…

The Language Model Evaluation Harness is a standardized framework for evaluating the performance of generative lang…

LLMPerf is a benchmarking tool designed to measure and compare the performance of large language model (LLM) APIs, …

lmms-eval is an open-source evaluation framework designed for consistent and efficient assessment of Large Multimod…

Massive Text Embedding Benchmark (MTEB) is a standardized evaluation framework that measures the performance of tex…

Melting Pot is a suite of customizable test scenarios designed for evaluating multi-agent reinforcement learning (M…

Meta-World is an open-source benchmark environment designed for evaluating meta-reinforcement learning and multi-ta…

mir_eval is a Python library designed for evaluating Music Information Retrieval (MIR) systems by providing standar…