hiyouga

LlamaFactory: Fine-Tune Large Language and Vision Models

LlamaFactory provides CLI and web interfaces for fine-tuning a broad range of language and vision models. It supports parameter-efficient methods and preference training, with workflows for training, inference, and model export.

View on GitHub Compare
Stars
75.3k
Forks
9.2k
Stars, last 30 days
+533

Our take

Health score

56/100

How it is scored
Activity
54
Community
67
Issues
56
Pull requests
46

+533 stars in the last 30 days

  • More contributors than 78% of the projects we track

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

76,054

Forks

9,375

Change since Nov 15, 2026: +9 stars·+2 forks

Overview

LlamaFactory is a Python framework for adapting pretrained language and vision models. It brings multiple training methods and model families into a shared workflow, helping researchers and practitioners avoid building separate fine-tuning pipelines for each model.

It is aimed at users who want both hands-on configuration through a CLI and a graphical interface for common tasks. The project also covers inference and exporting merged model checkpoints, so it can support more of the model-adaptation workflow than training alone.

Key Features

  • Fine-tuning support for a broad selection of LLMs and multimodal models, including model families such as Llama, Qwen, Gemma, and DeepSeek.
  • Training workflows for supervised fine-tuning, pre-training, reward modeling, and preference or reinforcement-learning methods such as DPO and PPO.
  • Parameter-efficient options including LoRA and QLoRA, alongside full and freeze tuning.
  • A Gradio-based web interface, LLaMA Board, as well as CLI commands for training, chat, and export.
  • Integrations and optimizations that include vLLM and SGLang inference backends, experiment tracking, and selected training acceleration tools.
  • Support for text, image, video, and audio-related tasks, depending on the model and workflow.

Use Cases

  • Model adaptation researchers: compare training approaches or fine-tune supported base models for a task without assembling a custom pipeline from scratch.
  • Teams building domain-specific assistants: apply supervised fine-tuning or preference training to adapt a model to internal instructions and data.
  • Multimodal developers: explore fine-tuning for supported vision, video, or audio models when their task and hardware fit the available integrations.
  • Practitioners with limited GPU memory: use LoRA or quantized QLoRA workflows when full-parameter training is impractical, while checking the project’s hardware estimates and model-specific requirements.

Getting Started

Install from source and open the CLI with:

git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git
cd LlamaFactory
pip install -e .
llamafactory-cli help

For a sample training, chat, and export workflow, hardware guidance, and full installation instructions, see the README and documentation.

Alternatives

  • Axolotl: Axolotl focuses on configurable LLM post-training and fine-tuning workflows, while LlamaFactory also provides web-based training, inference, and export interfaces.
  • torchtune: torchtune uses hackable, PyTorch-native training recipes, whereas LlamaFactory offers broader model coverage and CLI and web interfaces.
  • txtinstruct: txtinstruct specializes in instruction-tuning models and datasets, while LlamaFactory supports a wider range of fine-tuning methods and workflows.
ProjectLanguageLicenseStarsStatus
LlamaFactoryPythonApache-2.075.3kNot checked yet
axolotlPythonApache-2.012.6kActive
torchtunePythonBSD-3-Clause5.8kQuiet
txtinstructPythonApache-2.0238Archived

Considerations

Fine-tuning can require substantial GPU memory, especially for full-parameter training. Actual requirements depend on model size, method, sequence length, and hardware; quantized and adapter-based methods reduce but do not eliminate those demands. The dependency stack includes PyTorch, Transformers, and related libraries, while optional backends and accelerators may need additional setup. Check model templates and installation instructions carefully, and verify that the desired model and hardware backend are supported by the version you install.

Found this useful?

Share it with someone who would like LlamaFactory.

Comparisons