{"name":"torchtune: PyTorch Native Library for LLM Post-Training and Experimentation","description":"torchtune is a PyTorch native library designed for authoring, post-training, and experimenting with Large Language Models (LLMs). It offers hackable training recipes, simple PyTorch implementations of popular LLMs, and best-in-class memory efficiency. Please note: torchtune is no longer actively maintained as of 2025.","github":"https://github.com/pytorch/torchtune","url":"https://osrepos.com/repo/pytorch-torchtune","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/pytorch-torchtune","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/pytorch-torchtune.md","json":"https://osrepos.com/repo/pytorch-torchtune.json","topics":["Python","PyTorch","LLM","Finetuning","Machine Learning","Deep Learning","AI","NLP"],"keywords":["Python","PyTorch","LLM","Finetuning","Machine Learning","Deep Learning","AI","NLP"],"stars":null,"summary":"torchtune is a PyTorch native library designed for authoring, post-training, and experimenting with Large Language Models (LLMs). It offers hackable training recipes, simple PyTorch implementations of popular LLMs, and best-in-class memory efficiency. Please note: torchtune is no longer actively maintained as of 2025.","content":"## Introduction\n\ntorchtune is a PyTorch native library designed for easily authoring, post-training, and experimenting with Large Language Models (LLMs). It provides hackable training recipes for methods like Supervised Finetuning (SFT), Knowledge Distillation (KD), Direct Preference Optimization (DPO), Proximal Policy Optimization (PPO), and Quantization-Aware Training (QAT). The library also includes simple PyTorch implementations of popular LLMs such as Llama, Gemma, Mistral, Phi, and Qwen, alongside best-in-class memory efficiency and performance improvements leveraging the latest PyTorch APIs. YAML configurations simplify the setup for training, evaluation, quantization, or inference recipes.\n\n**Important Note:** As of 2025, torchtune is no longer actively maintained. For more details, please refer to [The future of torchtune](https://github.com/meta-pytorch/torchtune/issues/2883 \"The future of torchtune issue\") on GitHub.\n\n## Installation\n\ntorchtune is tested with the latest stable PyTorch release (currently 2.6.0) as well as the preview nightly version. It leverages torchvision for finetuning multimodal LLMs and torchao for the latest in quantization techniques. You should install these dependencies as well.\n\n### Install stable release\n\nbash\n# Install stable PyTorch, torchvision, torchao stable releases\npip install torch torchvision torchao\npip install torchtune\n\n\n### Install nightly release\n\nbash\n# Install PyTorch, torchvision, torchao nightlies.\npip install --pre --upgrade torch torchvision torchao --index-url https://download.pytorch.org/whl/nightly/cu126 # full options are cpu/cu118/cu124/cu126/xpu/rocm6.2/rocm6.3/rocm6.4\npip install --pre --upgrade torchtune --extra-index-url https://download.pytorch.org/whl/nightly/cpu\n\n\nTo confirm that the package is installed correctly, you can run the following command:\n\nbash\ntune --help\n\n\nAnd should see output similar to:\n\nbash\nusage: tune [-h] {ls,cp,download,run,validate} ...\n\nWelcome to the torchtune CLI!\n\noptions:\n  -h, --help            show this help message and exit\n\n...\n\n\n## Examples\n\nTo get started with torchtune, you can follow these common steps:\n\n### Downloading a model\n\nFollow the instructions on the official `meta-llama` repository to ensure you have access to the official Llama model weights. Once access is confirmed, you can download the weights to your local machine, along with the tokenizer model and a responsible use guide. For Llama3.1, you can run:\n\nbash\ntune download meta-llama/Meta-Llama-3.1-8B-Instruct \\\n--output-dir /tmp/Meta-Llama-3.1-8B-Instruct \\\n--ignore-patterns \"original/consolidated.00.pth\" \\\n--hf-token <HF_TOKEN>\n\n\n### Running finetuning recipes\n\nYou can finetune Llama3.1 8B with LoRA on a single GPU using the following command:\n\nbash\ntune run lora_finetune_single_device --config llama3_1/8B_lora_single_device\n\n\nFor distributed training, the `tune` CLI integrates with `torchrun`. To run a full finetune of Llama3.1 8B on two GPUs:\n\nbash\ntune run --nproc_per_node 2 full_finetune_distributed --config llama3_1/8B_full\n\n\n### Modify Configs\n\nThere are two ways to modify configurations:\n\n**Config Overrides**\n\nYou can directly overwrite config fields from the command line:\n\nbash\ntune run lora_finetune_single_device \\\n--config llama2/7B_lora_single_device \\\nbatch_size=8 \\\nenable_activation_checkpointing=True \\\nmax_steps_per_epoch=128\n\n\n**Update a Local Copy**\n\nYou can also copy the config to your local directory and modify the contents directly:\n\nbash\ntune cp llama3_1/8B_full ./my_custom_config.yaml\nCopied to ./my_custom_config.yaml\n\n\nThen, run your custom recipe by directing the `tune run` command to your local files:\n\nbash\ntune run full_finetune_distributed --config ./my_custom_config.yaml\n\n\ntorchtune also supports finetuning on a variety of custom datasets, including instruct-style, chat-style, and preference datasets. It also supports various devices, including NVIDIA GPU, Intel XPU, AMD ROCm, Apple MPS, and Ascend NPU.\n\n## Why Use torchtune\n\ntorchtune offers several compelling features for LLM development:\n\n*   **Comprehensive Post-Training Capabilities**: Supports the entire post-training lifecycle with hackable recipes for Supervised Finetuning (SFT), Knowledge Distillation (KD), Reinforcement Learning from Human Feedback (RLHF) methods like DPO, PPO, GRPO, and Quantization-Aware Training (QAT).\n*   **Extensive Model Support**: Includes simple PyTorch implementations of many state-of-the-art models available on Hugging Face Hub or Kaggle Hub, such as Llama (Llama4, Llama3.3, Llama3.2-Vision, Llama3.2, Llama3.1), Mistral, Gemma2, Microsoft Phi (Phi4, Phi3), and Qwen (Qwen3, Qwen2.5, Qwen2).\n*   **Optimized Performance and Memory Efficiency**: Leverages the latest PyTorch APIs to provide best-in-class memory efficiency and performance improvements. It offers various optimization flags and techniques, such as packed datasets, `torch.compile`, chunked cross entropy, activation checkpointing, fused optimizer step into backward, activation offloading, and 8-bit AdamW, significantly reducing memory footprint and increasing tokens per second.\n*   **Robust Ecosystem Integration**: Focuses on integrating with popular tools and libraries, including Hugging Face Hub for model weights, EleutherAI's LM Eval Harness for evaluation, Hugging Face Datasets, PyTorch FSDP2 for distributed training, torchao for quantization, Weights & Biases and Comet for logging, ExecuTorch for on-device inference, bitsandbytes for low memory optimizers, and PEFT for continued finetuning.\n*   **Active Community Contributions**: Values and highlights contributions from its user community, fostering a collaborative development environment.\n\n## Links\n\n*   **GitHub Repository**: [meta-pytorch/torchtune](https://github.com/meta-pytorch/torchtune \"torchtune GitHub Repository\")\n*   **Official Documentation**: [PyTorch torchtune Documentation](https://pytorch.org/torchtune/main/index.html \"PyTorch torchtune Documentation\")\n*   **Community Discord**: [torchtune Discord Server](https://discord.gg/4Xsdn8Rr9Q \"torchtune Discord Server\")\n*   **Future of torchtune Issue**: [The future of torchtune](https://github.com/meta-pytorch/torchtune/issues/2883 \"The future of torchtune issue\")","metrics":{"detailViews":0,"githubClicks":0},"dates":{"published":null,"modified":"2026-07-05T20:58:28.000Z"}}