textgen: Run Local Language Models in a Desktop App

Overview

TextGen provides a desktop app and browser-based interface for running language models locally. It brings model loading, chat, generation, and API access into one tool, so users can work with local models rather than relying on hosted AI services.

It is aimed at people who want a flexible local setup, from straightforward GGUF model use to workflows involving multiple inference backends, training, or extensions. Its breadth is useful, but users should expect to choose an installation path and configure it for their hardware and models.

Key Features

• Chat and free-form text generation, with customizable prompt templates and conversation editing.
• Image and document inputs, including vision chat and attachments for text, PDF, and DOCX files.
• Support for llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM backends.
• Local API endpoints compatible with OpenAI and Anthropic formats, including tool calling.
• Custom tools, web search, page fetching, math, and support for MCP servers.
• LoRA fine-tuning and a separate image-generation interface for diffusers models.
• Extensions and interface options such as themes, code highlighting, and LaTeX rendering.

Use Cases

• Run GGUF models on a personal computer for private, offline chat and text generation.
• Provide a local model endpoint to applications that use OpenAI- or Anthropic-compatible APIs.
• Experiment with different model backends and switch models without restarting the interface.
• Build assistant workflows that use custom Python tools or MCP servers.
• Fine-tune LoRAs or generate images locally when the full installation and suitable hardware are available.

Getting Started

For the desktop experience, download a portable build from the releases page, extract it, and launch textgen. GGUF models can be placed in user_data/models for detection by the interface.

For a Python setup, the README provides installation options for portable, full, and Docker deployments. See the README for hardware-specific instructions and configuration details.

Alternatives

• jan: Jan is a desktop-first assistant with optional cloud providers, while TextGen also offers a web interface and broader model controls.

Considerations

• Performance and supported model formats depend on the selected backend, hardware, and installation type. Some backends and features require the full installation rather than portable builds.
• Full installation involves additional dependencies and downloads PyTorch. Training and image generation may require capable hardware.
• The project describes itself as offline and telemetry-free, but enabling network access, public API options, or extensions changes the deployment's exposure and behavior.
• The AGPL-3.0 license should be reviewed before incorporating or distributing modified versions.