# Open R1: An Open-Source Reproduction of DeepSeek-R1 for Advanced LLM Training

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/huggingface-open-r1
Generated for open source discovery and AI-assisted research.

Open R1 is a Hugging Face project dedicated to creating a fully open reproduction of DeepSeek-R1, a powerful reasoning language model. This initiative provides comprehensive tools and recipes for training, evaluating, and generating data for large language models. It fosters community collaboration in AI research, enabling developers to build upon and understand the complex R1 pipeline.

GitHub: https://github.com/huggingface/open-r1
OSRepos URL: https://osrepos.com/repo/huggingface-open-r1

## Summary

Open R1 is a Hugging Face project dedicated to creating a fully open reproduction of DeepSeek-R1, a powerful reasoning language model. This initiative provides comprehensive tools and recipes for training, evaluating, and generating data for large language models. It fosters community collaboration in AI research, enabling developers to build upon and understand the complex R1 pipeline.

## Topics

- Python
- LLM
- AI
- Machine Learning
- Reinforcement Learning
- Open Source
- DeepSeek-R1
- Model Training

## Repository Information

Last analyzed by OSRepos: Mon Nov 17 2025 00:01:28 GMT+0000 (Western European Standard Time)
Detail views: 10
GitHub clicks: 8

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

Open R1 is an ambitious project by Hugging Face, aiming to deliver a fully open-source reproduction of DeepSeek-R1, a state-of-the-art reasoning language model. The core goal is to build the missing pieces of the R1 pipeline, making it accessible for everyone to reproduce and innovate upon. The project is designed for simplicity, primarily consisting of `src/open_r1` for training and synthetic data generation scripts, and a `Makefile` for streamlined execution of the R1 pipeline steps.

The development follows a clear plan of attack, guided by the DeepSeek-R1 tech report. This includes replicating R1-Distill models through high-quality corpus distillation, establishing a pure Reinforcement Learning (RL) pipeline for R1-Zero, and demonstrating multi-stage training from base models to RL-tuned variants. Recent milestones include the release of the Mixture-of-Thoughts dataset, CodeForces-CoTs, and OpenR1-Math-220k, marking significant progress in replicating DeepSeek-R1's capabilities.

## Installation

To get started with Open R1, ensure you have CUDA 12.4 and Python 3.11. The project leverages `uv` for environment management and `vLLM` and `FlashAttention` for high-performance operations.

1.  **Create a virtual environment and install pip:**
    shell
uv venv openr1 --python 3.11 && source openr1/bin/activate && uv pip install --upgrade pip
    
2.  **Install vLLM and FlashAttention:**
    shell
uv pip install vllm==0.8.5.post1
uv pip install setuptools && uv pip install flash-attn --no-build-isolation
    
    *Note: PyTorch `v2.6.0` is required for vLLM binaries.* 
3.  **Install remaining dependencies (e.g., for development):**
    shell
GIT_LFS_SKIP_SMUDGE=1 uv pip install -e ".[dev]"
    
4.  **Log in to Hugging Face and Weights and Biases:**
    shell
huggingface-cli login
wandb login
    
5.  **Verify Git LFS installation:**
    shell
git-lfs --version
    
    If not installed, run: `sudo apt-get install git-lfs`.

## Examples

Open R1 provides robust tools for training, evaluating, and generating data for LLMs.

### Training Models

The project supports Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) with DDP or DeepSpeed. Training commands are configured for 8 x H100s, with flexibility for different hardware.

**SFT Example (using a YAML config):**

shell
accelerate launch --config_file recipes/accelerate_configs/zero3.yaml src/open_r1/sft.py \
    --config recipes/OpenR1-Distill-7B/sft/config_distill.yaml


**GRPO Example (single-node with vLLM colocation):**

shell
ACCELERATE_LOG_LEVEL=info \
    accelerate launch --config_file recipes/accelerate_configs/zero3.yaml \
    src/open_r1/grpo.py --config recipes/DeepSeek-R1-Distill-Qwen-1.5B/grpo/config_demo.yaml \
    --vllm_mode colocate


The project also includes a `code` reward function for training with a code interpreter, supporting E2B and Morph sandboxes, and specific reward functions for competitive programming problems like IOI and CodeForces.

### Evaluating Models

Model evaluation is performed using `lighteval` with `vLLM`, supporting both single-GPU and multi-GPU (data or tensor parallel) setups. This allows for comprehensive benchmarking against various tasks.

**Example Evaluation (AIME 2024 on a single GPU):**

shell
export VLLM_WORKER_MULTIPROC_METHOD=spawn # Required for vLLM
MODEL=deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
MODEL_ARGS="model_name=$MODEL,dtype=bfloat16,max_model_length=32768,gpu_memory_utilization=0.8,generation_parameters={max_new_tokens:32768,temperature:0.6,top_p:0.95}"
OUTPUT_DIR=data/evals/$MODEL

TASK=aime24
lighteval vllm $MODEL_ARGS "lighteval|$TASK|0|0" \
    --use-chat-template \
    --output-dir $OUTPUT_DIR


Open R1 provides detailed instructions and results for reproducing DeepSeek's reported performance on benchmarks like AIME 2024, MATH-500, GPQA Diamond, and LiveCodeBench.

### Data Generation

Synthetic data can be generated using `distilabel` with `vLLM`, enabling the creation of high-quality datasets for training. This includes generating data from smaller distilled R1 models or the larger DeepSeek-R1 model using Slurm clusters.

**Example (generating data from a smol distilled R1 model):**

python
from datasets import load_dataset
from distilabel.models import vLLM
from distilabel.pipeline import Pipeline
from distilabel.steps.tasks import TextGeneration

prompt_template = """\
You will be given a problem. Please reason step by step, and put your final answer within \boxed{}:
{{ instruction }}"""

dataset = load_dataset("AI-MO/NuminaMath-TIR", split="train").select(range(10))

model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"

with Pipeline(
    name="distill-qwen-7b-r1",
    description="A pipeline to generate data from a distilled r1 model",
) as pipeline:

    llm = vLLM(
        model=model_id,
        tokenizer=model_id,
        extra_kwargs={
            "tensor_parallel_size": 1,
            "max_model_len": 8192,
        },
        generation_kwargs={
            "temperature": 0.6,
            "max_new_tokens": 8192,
        },
    )
    prompt_column = "problem"
    text_generation = TextGeneration(
        llm=llm, 
        template=prompt_template,
        num_generations=4,
        input_mappings={"instruction": prompt_column} if prompt_column is not None else {}
    )


if __name__ == "__main__":
    distiset = pipeline.run(dataset=dataset)
    distiset.push_to_hub(repo_id="username/numina-deepseek-r1-qwen-7b")


The project also offers a script for data decontamination, using 8-grams to deduplicate and clean datasets against benchmark contamination.

## Why Use Open R1?

Open R1 offers a unique opportunity for researchers and developers to engage with the cutting-edge field of large language model development. By providing an open reproduction of DeepSeek-R1, it democratizes access to advanced reasoning capabilities. The project's comprehensive toolkit, including scalable training with GRPO, robust evaluation with `lighteval`, and flexible data generation with `distilabel`, makes it an invaluable resource. Its community-driven nature ensures continuous improvement and collaboration, pushing the boundaries of open AI research.

## Links

*   **GitHub Repository:** [huggingface/open-r1](https://github.com/huggingface/open-r1){target=_blank}
*   **DeepSeek-R1 Tech Report:** [DeepSeek-AI/DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1){target=_blank}
*   **Mixture-of-Thoughts Dataset:** [open-r1/Mixture-of-Thoughts](https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts){target=_blank}
*   **CodeForces-CoTs Dataset:** [open-r1/codeforces-cots](https://huggingface.co/datasets/open-r1/codeforces-cots){target=_blank}
*   **OpenR1-Math-220k Dataset:** [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k){target=_blank}
*   **Distilabel:** [argilla-io/distilabel](https://github.com/argilla-io/distilabel){target=_blank}
*   **E2B:** [e2b.dev](https://e2b.dev){target=_blank}
*   **Morph:** [cloud.morph.so](https://cloud.morph.so/web/){target=_blank}