{"name":"Open R1: An Open-Source Reproduction of DeepSeek-R1 for Advanced LLM Training","description":"Open R1 is a Hugging Face project dedicated to creating a fully open reproduction of DeepSeek-R1, a powerful reasoning language model. This initiative provides comprehensive tools and recipes for training, evaluating, and generating data for large language models. It fosters community collaboration in AI research, enabling developers to build upon and understand the complex R1 pipeline.","github":"https://github.com/huggingface/open-r1","url":"https://osrepos.com/repo/huggingface-open-r1","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/huggingface-open-r1","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/huggingface-open-r1.md","json":"https://osrepos.com/repo/huggingface-open-r1.json","topics":["Python","LLM","AI","Machine Learning","Reinforcement Learning","Open Source","DeepSeek-R1","Model Training"],"keywords":["Python","LLM","AI","Machine Learning","Reinforcement Learning","Open Source","DeepSeek-R1","Model Training"],"stars":null,"summary":"Open R1 is a Hugging Face project dedicated to creating a fully open reproduction of DeepSeek-R1, a powerful reasoning language model. This initiative provides comprehensive tools and recipes for training, evaluating, and generating data for large language models. It fosters community collaboration in AI research, enabling developers to build upon and understand the complex R1 pipeline.","content":"## Introduction\n\nOpen R1 is an ambitious project by Hugging Face, aiming to deliver a fully open-source reproduction of DeepSeek-R1, a state-of-the-art reasoning language model. The core goal is to build the missing pieces of the R1 pipeline, making it accessible for everyone to reproduce and innovate upon. The project is designed for simplicity, primarily consisting of `src/open_r1` for training and synthetic data generation scripts, and a `Makefile` for streamlined execution of the R1 pipeline steps.\n\nThe development follows a clear plan of attack, guided by the DeepSeek-R1 tech report. This includes replicating R1-Distill models through high-quality corpus distillation, establishing a pure Reinforcement Learning (RL) pipeline for R1-Zero, and demonstrating multi-stage training from base models to RL-tuned variants. Recent milestones include the release of the Mixture-of-Thoughts dataset, CodeForces-CoTs, and OpenR1-Math-220k, marking significant progress in replicating DeepSeek-R1's capabilities.\n\n## Installation\n\nTo get started with Open R1, ensure you have CUDA 12.4 and Python 3.11. The project leverages `uv` for environment management and `vLLM` and `FlashAttention` for high-performance operations.\n\n1.  **Create a virtual environment and install pip:**\n    shell\nuv venv openr1 --python 3.11 && source openr1/bin/activate && uv pip install --upgrade pip\n    \n2.  **Install vLLM and FlashAttention:**\n    shell\nuv pip install vllm==0.8.5.post1\nuv pip install setuptools && uv pip install flash-attn --no-build-isolation\n    \n    *Note: PyTorch `v2.6.0` is required for vLLM binaries.* \n3.  **Install remaining dependencies (e.g., for development):**\n    shell\nGIT_LFS_SKIP_SMUDGE=1 uv pip install -e \".[dev]\"\n    \n4.  **Log in to Hugging Face and Weights and Biases:**\n    shell\nhuggingface-cli login\nwandb login\n    \n5.  **Verify Git LFS installation:**\n    shell\ngit-lfs --version\n    \n    If not installed, run: `sudo apt-get install git-lfs`.\n\n## Examples\n\nOpen R1 provides robust tools for training, evaluating, and generating data for LLMs.\n\n### Training Models\n\nThe project supports Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) with DDP or DeepSpeed. Training commands are configured for 8 x H100s, with flexibility for different hardware.\n\n**SFT Example (using a YAML config):**\n\nshell\naccelerate launch --config_file recipes/accelerate_configs/zero3.yaml src/open_r1/sft.py \\\n    --config recipes/OpenR1-Distill-7B/sft/config_distill.yaml\n\n\n**GRPO Example (single-node with vLLM colocation):**\n\nshell\nACCELERATE_LOG_LEVEL=info \\\n    accelerate launch --config_file recipes/accelerate_configs/zero3.yaml \\\n    src/open_r1/grpo.py --config recipes/DeepSeek-R1-Distill-Qwen-1.5B/grpo/config_demo.yaml \\\n    --vllm_mode colocate\n\n\nThe project also includes a `code` reward function for training with a code interpreter, supporting E2B and Morph sandboxes, and specific reward functions for competitive programming problems like IOI and CodeForces.\n\n### Evaluating Models\n\nModel evaluation is performed using `lighteval` with `vLLM`, supporting both single-GPU and multi-GPU (data or tensor parallel) setups. This allows for comprehensive benchmarking against various tasks.\n\n**Example Evaluation (AIME 2024 on a single GPU):**\n\nshell\nexport VLLM_WORKER_MULTIPROC_METHOD=spawn # Required for vLLM\nMODEL=deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B\nMODEL_ARGS=\"model_name=$MODEL,dtype=bfloat16,max_model_length=32768,gpu_memory_utilization=0.8,generation_parameters={max_new_tokens:32768,temperature:0.6,top_p:0.95}\"\nOUTPUT_DIR=data/evals/$MODEL\n\nTASK=aime24\nlighteval vllm $MODEL_ARGS \"lighteval|$TASK|0|0\" \\\n    --use-chat-template \\\n    --output-dir $OUTPUT_DIR\n\n\nOpen R1 provides detailed instructions and results for reproducing DeepSeek's reported performance on benchmarks like AIME 2024, MATH-500, GPQA Diamond, and LiveCodeBench.\n\n### Data Generation\n\nSynthetic data can be generated using `distilabel` with `vLLM`, enabling the creation of high-quality datasets for training. This includes generating data from smaller distilled R1 models or the larger DeepSeek-R1 model using Slurm clusters.\n\n**Example (generating data from a smol distilled R1 model):**\n\npython\nfrom datasets import load_dataset\nfrom distilabel.models import vLLM\nfrom distilabel.pipeline import Pipeline\nfrom distilabel.steps.tasks import TextGeneration\n\nprompt_template = \"\"\"\\\nYou will be given a problem. Please reason step by step, and put your final answer within \\boxed{}:\n{{ instruction }}\"\"\"\n\ndataset = load_dataset(\"AI-MO/NuminaMath-TIR\", split=\"train\").select(range(10))\n\nmodel_id = \"deepseek-ai/DeepSeek-R1-Distill-Qwen-7B\"\n\nwith Pipeline(\n    name=\"distill-qwen-7b-r1\",\n    description=\"A pipeline to generate data from a distilled r1 model\",\n) as pipeline:\n\n    llm = vLLM(\n        model=model_id,\n        tokenizer=model_id,\n        extra_kwargs={\n            \"tensor_parallel_size\": 1,\n            \"max_model_len\": 8192,\n        },\n        generation_kwargs={\n            \"temperature\": 0.6,\n            \"max_new_tokens\": 8192,\n        },\n    )\n    prompt_column = \"problem\"\n    text_generation = TextGeneration(\n        llm=llm, \n        template=prompt_template,\n        num_generations=4,\n        input_mappings={\"instruction\": prompt_column} if prompt_column is not None else {}\n    )\n\n\nif __name__ == \"__main__\":\n    distiset = pipeline.run(dataset=dataset)\n    distiset.push_to_hub(repo_id=\"username/numina-deepseek-r1-qwen-7b\")\n\n\nThe project also offers a script for data decontamination, using 8-grams to deduplicate and clean datasets against benchmark contamination.\n\n## Why Use Open R1?\n\nOpen R1 offers a unique opportunity for researchers and developers to engage with the cutting-edge field of large language model development. By providing an open reproduction of DeepSeek-R1, it democratizes access to advanced reasoning capabilities. The project's comprehensive toolkit, including scalable training with GRPO, robust evaluation with `lighteval`, and flexible data generation with `distilabel`, makes it an invaluable resource. Its community-driven nature ensures continuous improvement and collaboration, pushing the boundaries of open AI research.\n\n## Links\n\n*   **GitHub Repository:** [huggingface/open-r1](https://github.com/huggingface/open-r1){target=_blank}\n*   **DeepSeek-R1 Tech Report:** [DeepSeek-AI/DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1){target=_blank}\n*   **Mixture-of-Thoughts Dataset:** [open-r1/Mixture-of-Thoughts](https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts){target=_blank}\n*   **CodeForces-CoTs Dataset:** [open-r1/codeforces-cots](https://huggingface.co/datasets/open-r1/codeforces-cots){target=_blank}\n*   **OpenR1-Math-220k Dataset:** [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k){target=_blank}\n*   **Distilabel:** [argilla-io/distilabel](https://github.com/argilla-io/distilabel){target=_blank}\n*   **E2B:** [e2b.dev](https://e2b.dev){target=_blank}\n*   **Morph:** [cloud.morph.so](https://cloud.morph.so/web/){target=_blank}","metrics":{"detailViews":10,"githubClicks":8},"dates":{"published":null,"modified":"2025-11-17T00:01:28.000Z"}}