Axolotl: Streamlining LLM Fine-tuning with a Powerful Open-Source Framework

Introduction

Axolotl is a powerful, open-source framework dedicated to simplifying the post-training and fine-tuning of cutting-edge large language models (LLMs). Developed by axolotl-ai-cloud, this Python-based tool provides a streamlined approach to customizing models, supporting a wide array of architectures and advanced training techniques. With over 12,000 stars on GitHub, Axolotl has become a popular choice for those looking to efficiently adapt LLMs for specific tasks and datasets.

Key features include support for numerous models from the Hugging Face Hub, multimodal training capabilities for vision-language and audio models, and a variety of training methods such as full fine-tuning, LoRA, QLoRA, and advanced preference tuning techniques like DPO and GRPO. Its emphasis on easy configuration via YAML files and extensive performance optimizations ensures efficient and scalable training across different hardware setups.

Installation

Getting started with Axolotl is straightforward. The project recommends using uv for package management and provides clear steps for setting up your environment.

Requirements:

NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or AMD GPU
Python >=3.11 (3.12 recommended)
PyTorch ?2.11.0

Steps:

Install uv (if not already installed):

curl -LsSf https://astral.sh/uv/install.sh | sh
# Restart your shell after installation

Set UV_TORCH_BACKEND (change depending on your system):
```
export UV_TORCH_BACKEND=cu130
```

Create a new virtual environment and activate it:

uv venv --python 3.12
source .venv/bin/activate

Install PyTorch and Axolotl:

uv pip install torch==2.12.0 torchvision
uv pip install --no-build-isolation axolotl[deepspeed]

Download example configurations (optional):

axolotl fetch examples
axolotl fetch deepspeed_configs # OPTIONAL

Using Docker:

For an even simpler setup, you can use the official Docker image:

docker run --gpus '"all"' --ipc=host --rm -it axolotlai/axolotl:main-latest

More detailed installation approaches are available in the official documentation.

Examples

Once Axolotl is installed, you can quickly fine-tune a model using the provided examples.

Fetch Axolotl examples:

axolotl fetch examples
# Or, specify a custom path
axolotl fetch examples --dest path/to/folder

Train a model using LoRA:

axolotl train examples/llama-3/lora-1b.yml

For a more comprehensive guide, refer to the Getting Started Guide.

Why Use Axolotl?

Axolotl stands out as a premier tool for LLM fine-tuning due to its extensive feature set and focus on performance and usability:

Broad Model Compatibility: Supports a vast range of models, including GPT-OSS, LLaMA, Mistral, Mixtral, Pythia, and many others available on the Hugging Face Hub.
Advanced Training Methods: Offers a comprehensive suite of training techniques, from full fine-tuning and LoRA/QLoRA to GPTQ, QAT, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO, GDPO), and Reward Modelling.
Multimodal Capabilities: Enables fine-tuning of vision-language models (VLMs) like LLaMA-Vision, Qwen2-VL, LLaVA, and audio models such as Voxtral, with support for image, video, and audio data.
Performance Optimizations: Integrates cutting-edge optimizations like Multipacking, Flash Attention (2/3/4), Xformers, Sequence Parallelism (SP), LoRA optimizations, and multi-GPU/multi-node training (FSDP1, FSDP2, DeepSpeed, Torchrun, Ray) to maximize training efficiency.
Simplified Workflow: Utilizes a single YAML configuration file for the entire fine-tuning pipeline, covering dataset preprocessing, training, evaluation, quantization, and inference.
Cloud Ready: Provides Docker images and PyPI packages for seamless deployment on various cloud platforms and local hardware.
AI Agent Support: Includes built-in documentation optimized for AI coding agents, making it easier to integrate with tools like Claude Code, Cursor, and Copilot.