open-r1: Reproduce DeepSeek-R1 Training and Evaluation

Overview

Open R1 was created to make key parts of the DeepSeek-R1 pipeline reproducible: generating reasoning data, fine-tuning models, applying reinforcement learning, and evaluating results. The repository combines Python scripts, configuration recipes, and cluster job examples, while related datasets, models, and write-ups remain available through the Open R1 collections and project updates.

This is primarily a research and engineering reference for people working on language-model training, rather than a ready-to-run consumer application. The project is no longer maintained, and its README recommends TRL for current GRPO, SFT, and distillation training work.

Key Features

• Python scripts for supervised fine-tuning, GRPO training, and synthetic data generation.
• Recipes and configuration files for adapting training runs to different models and datasets.
• Example workflows for distributed training with Accelerate, DeepSpeed, and Slurm.
• Evaluation workflows using LightEval and vLLM for reasoning and code-generation benchmarks.
• Tools and examples for generating, filtering, and decontaminating datasets.
• Code-execution reward functions with sandbox-provider options for verifiable programming tasks.
• Published reasoning datasets, model artifacts, and project write-ups linked from the repository.

Use Cases

• Research teams can study an open implementation of a multi-stage reasoning-model reproduction pipeline.
• Model-training engineers can consult the recipes when adapting SFT or GRPO experiments to their own data and hardware.
• Dataset developers can explore workflows for generating synthetic reasoning traces and checking benchmark contamination.
• Evaluation practitioners can use the documented LightEval and vLLM workflows as a reference for comparing reasoning models.

Getting Started

The README documents setup with Python 3.11 and uv. A short development install is:

uv venv openr1 --python 3.11 && source openr1/bin/activate && uv pip install --upgrade pip
GIT_LFS_SKIP_SMUDGE=1 uv pip install -e ".[dev]"

See the repository README for dependency setup, training recipes, hardware-specific guidance, and evaluation commands. For active training development, see TRL.

Alternatives

• xTuring: xTuring offers high-level model customization and local or private-cloud deployment, rather than a research project focused on reproducing the DeepSeek-R1 pipeline.
• LLMBox: LLMBox provides a configurable, general-purpose training and evaluation pipeline, rather than scripts and workflows centered on reproducing DeepSeek-R1.
• torchtune: torchtune provides editable, model-specific recipes for LLM post-training, rather than a DeepSeek-R1 reproduction project with its own datasets and evaluations.

Considerations

• The project is explicitly no longer maintained, and new issues and pull requests will not be reviewed.
• The documented setup relies on GPU-oriented dependencies. The README specifies CUDA 12.4 and calls out a specific PyTorch and vLLM version pairing.
• Training examples assume substantial accelerator resources in places, including an eight-H100 configuration; other hardware may require tuning.
• Distributed training, model evaluation, and code-execution workflows require additional infrastructure and configuration.
• Treat this repository as a reproducibility reference and source of artifacts, not as the current home for training-side development.