xTuring: Fine-Tune and Run Personalized LLMs

Overview

xTuring provides a Python API and command-line tools for adapting open-source language models to custom data, then generating outputs or evaluating the resulting models. It brings data preparation, training, inference, and evaluation into one toolkit rather than requiring users to assemble each step themselves.

It is suited to developers and ML practitioners who want to keep model workflows in their own environment and reduce fine-tuning resource demands with options such as LoRA and low-precision model variants. The project supports models ranging from lightweight examples to much larger models, so hardware needs depend on the chosen model and method.

Key Features

• High-level APIs for datasets, model creation, fine-tuning, generation, evaluation, saving, and loading.
• Model variants for LoRA and INT8, with INT4 LoRA support through a generic model class.
• Dataset preparation and self-instruct generation utilities, including integrations with external model APIs.
• Perplexity evaluation for causal language models.
• Command-line chat and UI playground tools for interacting with models.
• Model and engine registries, with configurations organized by model.
• CPU inference support through Intel Extension for Transformers on Intel platforms.

Use Cases

• A developer can fine-tune a supported model on instruction data for an internal assistant while keeping the workflow local or in a private cloud.
• An ML practitioner can use LoRA or low-precision variants to explore customization when full-parameter fine-tuning is too resource-intensive.
• A team can evaluate causal language models on a dataset using perplexity before selecting a model for an application.
• A researcher can prototype data preparation, fine-tuning, and generation workflows through Python examples, then test results in the CLI or UI playground.

Getting Started

Install the package:

pip install xturing

For setup notes, model examples, and development instructions, see the README. Documentation is available at xturing.stochastic.ai.

Alternatives

• LLMBox: LLMBox focuses on configurable training and benchmark evaluation pipelines, while xTuring also targets data preparation and local or private-cloud model use.
• torchtune: torchtune focuses on editable PyTorch recipes for post-training, while xTuring offers a higher-level workflow spanning data preparation, fine-tuning, evaluation, and inference.
• peft: PEFT focuses on parameter-efficient adaptation methods such as LoRA, rather than xTuring’s broader workflow for preparing data, fine-tuning, evaluating, and running models.

Considerations

• Model training and inference requirements vary widely. Large models may require substantial compute and memory; the README's lightweight Qwen3 example is intended as a CPU-friendly starting point.
• The README specifies transformers>=4.36.0 and advises against upgrading to Transformers 5.x because xTuring's INT8/INT4 loading relies on arguments removed in that release.
• Qwen3-Omni support requiring Transformers 5.x is described as not yet released in the README.
• INT4 fine-tuning uses a separate generic model class rather than the standard model key variants.
• Check the supported model list and current documentation before choosing a model, precision mode, or hardware setup.