Our take
Activeregular commits and releases- Activity
- 71
- Community
- 67
- Issues
- 65
- Pull requests
- 47
+8,674 stars in the last 30 days
- Releases more often than 76% of the projects we track
Cactus is an actively maintained, broad mobile-inference stack with useful integration options, but model conversion and device-specific behavior deserve hands-on validation before adoption.
Good fit if
- You are building C++ or mobile applications that need local text, vision, or speech inference.
- You want quantization, inference, and language bindings in one project, with optional cloud handoff.
- You can validate your target models and devices; the repository has regular commits and releases, contributions from many people, and quick median issue and pull-request handling.
Look elsewhere if
- You need a clearly verified permissive license before evaluating or shipping. The repository metadata reports NOASSERTION, so confirm the applicable license terms directly.
- Your workflow depends on every Hugging Face model converting reliably. The README calls arbitrary-model conversion experimental.
- You need an established, broadly validated platform without first testing model quality and performance on your particular devices.
All health signals
| Last commit | 2026-09-26 (14 days ago) |
|---|---|
| Commits, last 90 days | 82 |
| Releases, last 12 months | 23 (latest v2.2.2, 2026-09-26) |
| Contributors | 52 (top contributor: 52% of commits) |
| Issues closed, last 90 days | 3 (typically closed in 1 day) |
| Pull requests merged, last 90 days | 12 (typically merged in 1 day) |
| Project age | 18 months |
Checked on 2026-10-11 with the GitHub API.
Project history
Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.
Stars
14,802Forks
1,041Change since Nov 17, 2026: +25 stars·+1 forks
Overview
Cactus combines model quantization, computation graphs, device-oriented kernels, and an inference engine for running AI workloads on phones, wearables, and other edge devices. Its goal is to make local inference practical across text, vision, and speech, while allowing hard queries to be routed to the cloud when configured.
It is aimed at developers integrating inference into applications, rather than only people seeking a desktop chat interface. The repository provides a command-line tool, C and C++ interfaces, and bindings for Swift, Kotlin, Flutter, React Native, Python, and Rust. Any Hugging Face model can reportedly be converted, but that path is described as experimental; several model families are more thoroughly tested.
Key Features
- Rotation-and-codebook quantization supports weight formats from 4-bit down to 1-bit, including mixed-precision options.
- A C++ computation graph provides tensor operations such as matrix multiplication and transpose.
- CPU/GPU kernels target mobile and Apple devices, including ARM NEON SIMD kernels.
- The inference engine supports chat completion, streaming, tool calling, transcription, embeddings, retrieval-augmented generation, vision, and vector indexing.
- OpenAI-compatible APIs are available, including a local HTTP server command.
- Hybrid inference can route queries to a cloud model based on local-model confidence.
- CLI commands cover running, converting, downloading, serving, benchmarking, and testing models.
- Language bindings include Swift, Kotlin, Flutter, React Native, Python, and Rust.
Use Cases
- A mobile app team can add offline or on-device chat using a quantized model and the Swift or Kotlin bindings.
- A wearable or edge-device developer can evaluate local speech transcription without sending all audio to a remote service.
- An application that handles images can use the vision inference path on supported mobile hardware.
- A team needing both local responsiveness and a fallback can use confidence-based cloud handoff for harder requests.
- Developers can expose a local model through an OpenAI-compatible server for tools and clients that already use that API style.
What you need
Detected in the repository
- C++ (main language on GitHub)
- Automated checks on GitHub Actions
Getting Started
On a Mac, install and run the CLI:
brew install cactus-compute/cactus/cactus
cactus run
For other setup options and model commands, see the repository README.
License in Plain Words
No standard license was detected. Without a license, the default copyright rules apply: you may read the code, but not reuse it. Check the repository or ask the authors.
Considerations
- The project is young, though its measured health is active: it has recent commits and releases, and issue and pull-request turnaround is short. It has a broad contributor base and is not flagged as concentrated around one contributor.
- The README describes arbitrary Hugging Face conversion as experimental. It names Liquid, Gemma, Whisper, Parakeet, and Qwen as model families that are especially tested, so compatibility should not be assumed for other models.
- Reported benchmark results cover selected models and devices. They are not a guarantee for a different model, phone, or workload; benchmark your own target hardware.
- The GitHub license field is
NOASSERTION. Review the repository's current license files and obligations before incorporating it into a product. - The supplied project facts do not identify required runtimes, and the README's Linux setup lists development packages rather than a complete deployment matrix. Verify toolchain and platform requirements for your target environment.
Found this useful?
Share it with someone who would like cactus.