pytorch

torchchat: Run PyTorch LLMs on Desktop, Server and Mobile

torchchat is a PyTorch-native codebase for running large language models locally with Python, C++ runners, and mobile apps. It supports desktop, server, iOS, and Android workflows, but is archived and no longer under active development.

ArchivedPythonBSD-3-ClausePyTorchLLMLocal First
View on GitHub Compare
Stars
3.6k
Forks
247
Stars, last 30 days
-4
Last commit
1.1 years ago
Contributors
62

Our take

Archivedarchived by its owners on GitHub, read-only

Health score

35/100

How it is scored
Activity
19
Community
78
Issues
21
Pull requests
20

-4 stars in the last 30 days

torchchat is a broad PyTorch example and deployment codebase, but its archived status and lack of recent maintenance make it a poor choice for teams seeking an actively supported inference stack.

Good fit if

  • You need a concrete reference for PyTorch-based local inference spanning Python, C++, and mobile execution.
  • You can maintain or adapt the code yourself, and can meet the Python 3.10 or newer requirement.
  • BSD-3-Clause licensing fits your use, with separate terms checked for third-party model weights and included components.

Look elsewhere if

  • You need ongoing upstream fixes, releases, or issue support. The repository is archived, with no commits, releases, merged pull requests, or closed issues in the recent measured period.
  • Your project depends on a maintained production interface, especially the README's explicitly work-in-progress REST server or evaluation features.
  • You need a turnkey mobile deployment without building and configuring ExecuTorch and platform-specific tooling.
All health signals
Last commit2025-09-10 (1.1 years ago)
Commits, last 90 days0
Releases, last 12 months0
Contributors62 (top contributor: 29% of commits)
Issues closed, last 90 days0
Pull requests merged, last 90 days0
Project age2 years

Checked on 2026-10-26 with the GitHub API.

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

3,609

Forks

248

Change since Nov 17, 2026: 0 stars·0 forks

Overview

torchchat is a PyTorch-native project for running large language models locally through Python, C++ applications, and mobile apps. It aims to make PyTorch LLM execution work across desktop and server environments as well as iOS and Android, rather than requiring a hosted inference service.

The repository provides workflows for chat and text generation, model export, and on-device execution. It supports PyTorch execution modes and connects with ExecuTorch for mobile deployment. The project is archived and its maintainers state that it is no longer under active development, so it is best treated as a reference or a maintained-by-you starting point rather than an actively supported dependency.

Key Features

  • Run interactive chat and prompt-based generation from Python.
  • Start a local REST server with an OpenAI-style chat completions endpoint.
  • Export models for AOT Inductor execution on desktop or server.
  • Export ExecuTorch artifacts for iOS, Android, and other supported targets.
  • Build and use C++ runners for exported model artifacts.
  • Work with a range of listed Llama, Mistral, Granite, DeepSeek, and other models, including multimodal Llama 3.2 11B.
  • Download and manage model artifacts through command-line subcommands.
  • Evaluate models through the lm-evaluation-harness integration, which the README describes as work in progress.

Use Cases

  • PyTorch developers exploring how to run LLM inference locally across Python and native execution paths.
  • Mobile developers prototyping on-device LLM experiences with ExecuTorch on iOS or Android.
  • Teams testing model export and quantization workflows for desktop or server deployment.
  • Researchers or developers comparing local model behavior using the included evaluation command, while accounting for its stated work-in-progress status.

What you need

Detected in the repository

  • Python >=3.10 (from pyproject.toml)
  • A test suite and automated checks on GitHub Actions

License in plain words

BSD-3-Clausepermissive

  • Commercial use: yes
  • Modify and redistribute: yes
  • You must keep: the copyright and license notice; the authors' names cannot endorse your product
  • Share your changes: no

A summary, not legal advice: the LICENSE file is what applies.

Getting Started

The README requires Python 3.10 or newer. Clone the repository, create a virtual environment, and install its requirements:

git clone https://github.com/pytorch/torchchat.git
cd torchchat
python3 -m venv .venv
source .venv/bin/activate
./install/install_requirements.sh

See the README for model access, download, export, and platform-specific setup details.

Alternatives

  • pocketpal-ai: PocketPal is a mobile chat app for iOS and Android, while torchchat provides runtimes and workflows across desktop, server, and mobile.
  • text-generation-inference: Text Generation Inference serves LLMs through APIs with GPU-focused features, while torchchat targets local execution across desktop, server, and mobile.
  • TensorRT-LLM: TensorRT-LLM focuses on optimized inference on NVIDIA GPUs, while torchchat supports broader local workflows, including mobile.
ProjectLanguageLicenseStarsStatus
torchchatPythonBSD-3-Clause3.6kArchived
pocketpal-aiTypeScriptMIT8.6kActive
text-generation-inferencePythonApache-2.010.9kArchived
TensorRT-LLMPython–15kActive

Considerations

  • GitHub marks the repository archived and read-only. It has had no commits or releases in over a year, and the measured recent period also shows no merged pull requests or closed issues. Although the project has 62 contributors and contributor activity is not concentrated in one person, there is no active maintenance signal.
  • Python 3.10 or newer is required. Mobile deployment also requires setting up ExecuTorch and platform tooling such as Xcode or Android Studio; the README gives additional device and memory guidance.
  • Model downloads often come from Hugging Face, may require an account and access approval, and can carry terms separate from the repository license. The repository is BSD-3-Clause, while the README notes that additional code and third-party model content may have other obligations.
  • The README identifies the REST server and evaluation workflow as incomplete or work in progress. Do not assume every API parameter or evaluation feature is implemented.

Found this useful?

Share it with someone who would like torchchat.

Comparisons