litgpt: Train, Fine-Tune, and Serve Large Language Models

Overview

LitGPT is a Python toolkit for working with large language models across the model lifecycle, from pretraining and fine-tuning to evaluation, interactive use, and serving. It provides implementations of many model families alongside command-line workflows and YAML recipes, helping practitioners adapt models without building each training pipeline from scratch.

The project emphasizes readable implementations and configurable training, including distributed execution and parameter-efficient fine-tuning. It suits developers and researchers who want to inspect or modify model code as well as run established workflows on their own data and hardware.

Key Features

• Workflows for pretraining, continued pretraining, fine-tuning, evaluation, chat, and serving.
• Implementations and checkpoint workflows for a broad selection of LLM families.
• LoRA, QLoRA, and adapter fine-tuning options.
• Quantization and mixed-precision options to reduce memory or compute requirements.
• Multi-GPU training through fully sharded data parallelism, with optional CPU offloading and TPU support.
• YAML training recipes with command-line parameter overrides.
• Dataset support for common workflows, plus options for preparing custom datasets.
• Python API and CLI access to model loading and generation.

Use Cases

• Researchers can pretrain or continue training smaller language models on custom text and inspect or modify the implementation.
• Product teams can fine-tune a supported model on task-specific examples, then serve the resulting checkpoint through an HTTP endpoint.
• Developers can compare candidate models using the evaluation workflow before choosing one for an application.
• Teams with constrained GPU memory can explore quantization and parameter-efficient fine-tuning rather than updating all model weights.

Getting Started

Install the optional dependencies with:

pip install 'litgpt[extra]'

For model usage, workflows, and installation from source, see the README.

Alternatives

• LlamaFactory: LlamaFactory offers CLI and web-based fine-tuning workflows across many models, while LitGPT emphasizes direct control over model code and training.
• axolotl: Axolotl focuses on configurable fine-tuning and post-training, including preference optimization and reinforcement learning, rather than LitGPT's broader training toolkit.

Considerations

Training and serving LLMs can require substantial compute and storage, depending on the model and configuration. LitGPT offers quantization and memory-saving techniques, but these do not remove the need to plan for hardware and model-weight access; some model downloads require an access token. Python 3.10 or newer is required. The project is active, with recent commits and releases, and has a broad contributor base rather than a dependency on one contributor. However, the measured period recorded no issues closed, so teams should assess support expectations and review the open issue tracker. Apache-2.0 permits broad use, subject to its license terms.