TurboLLM: Run Local LLMs with Auto-Tuning, Any Engine, and a Polished UI

Summary

TurboLLM is a powerful, lightweight solution for running local LLM engines, offering auto-tuning for your GPU and a polished web UI. It supports any llama-server compatible binary, including community forks, and provides both OpenAI and Anthropic-compatible APIs. This offline-first tool allows users to maximize performance and flexibility with their local language models.

Use at your own risk

OSRepos shares public repositories for knowledge and discovery only. Any installation, execution, configuration, or use of code from these repositories is the user's own responsibility. Always review the repository, source code, dependencies, licenses, and security implications before running or installing anything. OSRepos is not responsible for issues, damages, or losses resulting from third-party repositories.

Introduction

TurboLLM is an innovative tool designed to revolutionize how you interact with local Large Language Models (LLMs). It stands out by allowing you to run any local LLM engine, automatically tuned to your GPU, all through a polished web user interface. With TurboLLM, you gain access to both OpenAI and Anthropic-compatible APIs, making it incredibly easy to integrate with existing tools like Claude Code on your own machine, often with just one command. Built without Electron or Python, TurboLLM is lightweight, offline-first, and engineered for maximum performance, ensuring you get the most out of your hardware without the usual complexities of manual tuning or engine limitations.

Installation

Getting started with TurboLLM is straightforward, requiring Node.js 22.13.0 or newer. You can try it out instantly or install it globally for regular use.

To run without installing (recommended for a first try):

npx turbollm

To install globally:

npm install -g turbollm
turbollm

On its first run, TurboLLM will detect your GPU, download a matching llama-server build, and guide you through a quick setup wizard to get your first model running.

Examples

TurboLLM's strength lies in its flexibility and API compatibility. You can easily launch powerful coding agents like Claude Code against your local models:

turbollm launch claude
# This command auto-loads a model if none is running, then opens Claude Code.

turbollm launch claude --model qwen3-8b
# Load a specific model first, then launch Claude Code.

TurboLLM also provides OpenAI-compatible /v1/chat/completions and Anthropic-compatible /v1/messages endpoints. This means any tool or client designed for these APIs can seamlessly interact with your local LLMs. For instance, you can query your local model using a simple curl command:

# OpenAI-compatible API example
curl http://127.0.0.1:6996/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"local","messages":[{"role":"user","content":"hello"}]}'

Why use TurboLLM

TurboLLM offers several compelling advantages over other local LLM solutions:

Unmatched Engine Flexibility: Unlike other tools that dictate the engine, TurboLLM allows you to use any llama-server-compatible binary, including cutting-edge community forks. This means you get access to the latest innovations, like low-bit KV cache or speculative decoding, on day one without manual compilation.
Performance Auto-Tuning: It automatically benchmarks and tunes dozens of launch flags to your specific GPU, providing real, measured tokens/sec. This eliminates guesswork and ensures optimal performance for every model.
Lightweight and Efficient: As a ~7 MB npm package, TurboLLM avoids the bloat of Electron or Python, offering a nimble and responsive experience. It only downloads the necessary engine components for your GPU.
Broad API Compatibility: With both OpenAI and Anthropic-compatible APIs, TurboLLM seamlessly integrates with a wide ecosystem of tools, including Claude Code, agents, and RAG pipelines.
Offline-First and Private: Designed for privacy, TurboLLM operates entirely offline, requiring no account or internet connection for core functionality. Your data, prompts, and keys never leave your machine.
Advanced Features: From a genuinely good chat UI with agentic tools and live artifacts to multi-GPU support and intelligent VRAM sharing with tools like ComfyUI, TurboLLM is packed with features for both casual users and power users.