{"name":"Verifiers: Environments for LLM Reinforcement Learning and Evaluation","description":"Verifiers is a Python library by Prime Intellect AI for building environments to train and evaluate Large Language Models (LLMs). It enables the creation of custom environments with datasets, model harnesses, and reward functions, supporting reinforcement learning, capability evaluation, and synthetic data generation. This library is tightly integrated with the Prime Intellect ecosystem, including their Environments Hub and training framework.","github":"https://github.com/willccbb/verifiers","url":"https://osrepos.com/repo/willccbb-verifiers","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/willccbb-verifiers","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/willccbb-verifiers.md","json":"https://osrepos.com/repo/willccbb-verifiers.json","topics":["Python","Reinforcement Learning","LLM","AI","Machine Learning","Environment","Evaluation","Prime Intellect"],"keywords":["Python","Reinforcement Learning","LLM","AI","Machine Learning","Environment","Evaluation","Prime Intellect"],"stars":null,"summary":"Verifiers is a Python library by Prime Intellect AI for building environments to train and evaluate Large Language Models (LLMs). It enables the creation of custom environments with datasets, model harnesses, and reward functions, supporting reinforcement learning, capability evaluation, and synthetic data generation. This library is tightly integrated with the Prime Intellect ecosystem, including their Environments Hub and training framework.","content":"## Introduction\nVerifiers is a powerful Python library developed by Prime Intellect AI, specifically designed for creating robust environments to train and evaluate Large Language Models (LLMs). It provides a comprehensive framework where environments encapsulate everything needed to run and assess a model on a particular task. Each environment typically includes a dataset of task inputs, a harness for the model (managing tools, sandboxes, and context), and a reward function or rubric to score the model's performance. Verifiers is deeply integrated with the Prime Intellect Environments Hub, their `prime-rl` training framework, and their Hosted Training platform, offering a complete ecosystem for LLM development.\n\n## Installation\nGetting started with Verifiers is straightforward. First, ensure you have `uv` and the `prime` CLI tool installed.\n\nbash\n# install uv\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n# install the prime CLI\nuv tool install prime\n# log in to the Prime Intellect platform\nprime login\n\n\nTo set up a new workspace for environment development, use:\n\nbash\n# ~/dev/my-lab\nprime lab setup\n\n\nAlternatively, to add Verifiers to an existing project, run:\n\nbash\nuv add verifiers && prime lab setup --skip-install\n\n\n## Examples\nVerifiers allows you to easily initialize new environment templates. To create a fresh environment module, use the `prime env init` command:\n\nbash\nprime env init my-env # creates a new template in ./environments/my_env\n\n\nEnvironment modules are self-contained Python files that expose a `load_environment` function. Here's a basic example:\n\npython\n# my_env.py\nimport verifiers as vf\n\ndef load_environment(dataset_name: str = 'gsm8k') -> vf.Environment:\n    dataset = vf.load_example_dataset(dataset_name) # 'question'\n    async def correct_answer(completion, answer) -> float:\n        completion_ans = completion[-1]['content']\n        return 1.0 if completion_ans == answer else 0.0\n    rubric = vf.Rubric(funcs=[correct_answer])\n    env = vf.SingleTurnEnv(dataset=dataset, rubric=rubric)\n    return env\n\n\nYou can also install environments from the Environments Hub:\n\nbash\nprime env install primeintellect/math-python\n\n\nTo run a local evaluation with any OpenAI-compatible model:\n\nbash\nprime eval run my-env -m gpt-5-nano # run and save eval results locally\n\n\n## Why Use Verifiers\nVerifiers offers a robust solution for anyone working with LLMs, providing a structured and efficient way to:\n*   **Create Custom Environments**: Define specific tasks with tailored datasets, model interaction harnesses, and precise reward functions.\n*   **Facilitate Reinforcement Learning**: Design environments optimized for training LLMs using reinforcement learning techniques.\n*   **Evaluate LLM Capabilities**: Conduct thorough evaluations of model performance across various tasks and metrics.\n*   **Generate Synthetic Data**: Leverage environments to produce high-quality synthetic data for further model training or analysis.\n*   **Seamless Integration**: Benefit from tight integration with Prime Intellect's broader ecosystem, including their Environments Hub, `prime-rl` training framework, and Hosted Training platform.\n*   **Streamlined Workflow**: The `prime` CLI tool simplifies environment setup, installation, evaluation, and publishing.\n\n## Links\n*   [GitHub Repository](https://github.com/PrimeIntellect-ai/verifiers){:target=\"_blank\"}\n*   [Official Documentation](https://docs.primeintellect.ai/verifiers){:target=\"_blank\"}\n*   [Environments Hub](https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars){:target=\"_blank\"}\n*   [PRIME-RL Training Framework](https://github.com/PrimeIntellect-ai/prime-rl){:target=\"_blank\"}","metrics":{"detailViews":8,"githubClicks":11},"dates":{"published":null,"modified":"2026-02-14T20:01:11.000Z"}}