pytorch

torchtune: Fine-Tune and Post-Train Large Language Models

torchtune is a PyTorch library for configuring and running LLM post-training workflows, from supervised fine-tuning to preference optimization. It is aimed at developers who want editable recipes and model-specific configs, but is no longer actively maintained.

PythonBSD-3-ClausePyTorchLLMMachine Learning
View on GitHub Compare
Stars
5.8k
Forks
756
Stars, last 30 days
+1

Our take

Health score

38/100

How it is scored
Activity
19
Community
93
Issues
21
Pull requests
20

+1 stars in the last 30 days

  • More contributors than 78% of the projects we track

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

5,820

Forks

765

Change since Nov 21, 2026: +1 stars·+1 forks

Overview

torchtune is a Python library for authoring and running post-training workflows for large language models with PyTorch. It packages training recipes, model implementations, and YAML configuration files so developers can adapt fine-tuning and evaluation workflows rather than build them from scratch.

Its recipes cover supervised fine-tuning, knowledge distillation, preference optimization, reinforcement learning, and quantization-aware training. The project is best suited to practitioners comfortable with PyTorch and model-training infrastructure who need control over training code and configurations. The repository says development wound down in 2025, so it may be more appropriate for maintaining or studying existing workflows than for starting a project that depends on ongoing upstream support.

Key Features

  • Recipes for supervised fine-tuning, including full fine-tuning and LoRA/QLoRA.
  • Workflows for knowledge distillation, DPO, PPO, GRPO, and quantization-aware training.
  • PyTorch implementations and configurations for models including Llama, Gemma, Mistral, Phi, and Qwen.
  • YAML configs that can be copied, edited, or overridden from the command line.
  • Single-device, multi-device, and, for some recipes, multi-node training options.
  • Memory and performance techniques such as activation checkpointing, activation offloading, and lower-precision optimizers.
  • Integrations with Hugging Face Hub and Datasets, torchao, evaluation tools, and experiment logging services.

Use Cases

  • A model-training engineer wants to adapt a supported LLM with supervised fine-tuning and needs a configurable PyTorch recipe.
  • An ML researcher wants to experiment with methods such as DPO or knowledge distillation using recipes they can inspect and modify.
  • A team wants to fine-tune a model on constrained GPU memory and investigate options such as LoRA, QLoRA, or activation offloading.
  • A developer maintaining an existing torchtune workflow needs to reproduce training or adapt its configs and model implementations.

Getting Started

Install the package and its PyTorch dependencies:

pip install torch torchvision torchao
pip install torchtune

Then inspect the CLI with tune --help. See the repository README and documentation for setup details and recipes. The project is no longer actively maintained.

Alternatives

  • axolotl: Axolotl also configures LLM fine-tuning and post-training, with broader support for LoRA, quantization, multimodal workflows, and distributed training.
  • Ludwig: Ludwig supports declarative LLM fine-tuning alongside other ML tasks, using YAML configurations rather than torchtune’s editable PyTorch recipes.
ProjectLanguageLicenseStarsStatus
torchtunePythonBSD-3-Clause5.8kNot checked yet
axolotlPythonApache-2.012.6kActive
ludwigPythonApache-2.011.8kActive

Considerations

  • The repository states that development wound down in 2025 and is no longer actively maintained. Plan for limited or no upstream updates and support.
  • Training large models generally requires suitable accelerator hardware and model-weight access. Hardware and scaling support vary by recipe.
  • The README describes testing against specific stable and nightly PyTorch releases, so check dependency compatibility before adopting it in a new environment.
  • BSD-3-Clause covers torchtune itself; third-party model weights may have separate terms of use.

Found this useful?

Share it with someone who would like torchtune.

Comparisons