cactus-compute

cactus: Run AI Models on Phones and Wearables

Cactus is a C++ inference engine for running quantized language, vision, and speech models on mobile and edge devices, with optional cloud handoff. It suits teams building on-device AI features who need native APIs and mobile bindings.

View on GitHub Compare
Stars
6.1k
Forks
515
Stars, last 30 days
+8,674
Last commit
14 days ago
Contributors
52

Our take

Activeregular commits and releases

Health score

63/100

How it is scored
Activity
71
Community
67
Issues
65
Pull requests
47

+8,674 stars in the last 30 days

  • Releases more often than 76% of the projects we track

Cactus is an actively maintained, broad mobile-inference stack with useful integration options, but model conversion and device-specific behavior deserve hands-on validation before adoption.

Good fit if

  • You are building C++ or mobile applications that need local text, vision, or speech inference.
  • You want quantization, inference, and language bindings in one project, with optional cloud handoff.
  • You can validate your target models and devices; the repository has regular commits and releases, contributions from many people, and quick median issue and pull-request handling.

Look elsewhere if

  • You need a clearly verified permissive license before evaluating or shipping. The repository metadata reports NOASSERTION, so confirm the applicable license terms directly.
  • Your workflow depends on every Hugging Face model converting reliably. The README calls arbitrary-model conversion experimental.
  • You need an established, broadly validated platform without first testing model quality and performance on your particular devices.
All health signals
Last commit2026-09-26 (14 days ago)
Commits, last 90 days82
Releases, last 12 months23 (latest v2.2.2, 2026-09-26)
Contributors52 (top contributor: 52% of commits)
Issues closed, last 90 days3 (typically closed in 1 day)
Pull requests merged, last 90 days12 (typically merged in 1 day)
Project age18 months

Checked on 2026-10-11 with the GitHub API.

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

14,802

Forks

1,041

Change since Nov 17, 2026: +25 stars·+1 forks

Overview

Cactus combines model quantization, computation graphs, device-oriented kernels, and an inference engine for running AI workloads on phones, wearables, and other edge devices. Its goal is to make local inference practical across text, vision, and speech, while allowing hard queries to be routed to the cloud when configured.

It is aimed at developers integrating inference into applications, rather than only people seeking a desktop chat interface. The repository provides a command-line tool, C and C++ interfaces, and bindings for Swift, Kotlin, Flutter, React Native, Python, and Rust. Any Hugging Face model can reportedly be converted, but that path is described as experimental; several model families are more thoroughly tested.

Key Features

  • Rotation-and-codebook quantization supports weight formats from 4-bit down to 1-bit, including mixed-precision options.
  • A C++ computation graph provides tensor operations such as matrix multiplication and transpose.
  • CPU/GPU kernels target mobile and Apple devices, including ARM NEON SIMD kernels.
  • The inference engine supports chat completion, streaming, tool calling, transcription, embeddings, retrieval-augmented generation, vision, and vector indexing.
  • OpenAI-compatible APIs are available, including a local HTTP server command.
  • Hybrid inference can route queries to a cloud model based on local-model confidence.
  • CLI commands cover running, converting, downloading, serving, benchmarking, and testing models.
  • Language bindings include Swift, Kotlin, Flutter, React Native, Python, and Rust.

Use Cases

  • A mobile app team can add offline or on-device chat using a quantized model and the Swift or Kotlin bindings.
  • A wearable or edge-device developer can evaluate local speech transcription without sending all audio to a remote service.
  • An application that handles images can use the vision inference path on supported mobile hardware.
  • A team needing both local responsiveness and a fallback can use confidence-based cloud handoff for harder requests.
  • Developers can expose a local model through an OpenAI-compatible server for tools and clients that already use that API style.

What you need

Detected in the repository

  • C++ (main language on GitHub)
  • Automated checks on GitHub Actions

Getting Started

On a Mac, install and run the CLI:

brew install cactus-compute/cactus/cactus
cactus run

For other setup options and model commands, see the repository README.

License in Plain Words

No standard license was detected. Without a license, the default copyright rules apply: you may read the code, but not reuse it. Check the repository or ask the authors.

Considerations

  • The project is young, though its measured health is active: it has recent commits and releases, and issue and pull-request turnaround is short. It has a broad contributor base and is not flagged as concentrated around one contributor.
  • The README describes arbitrary Hugging Face conversion as experimental. It names Liquid, Gemma, Whisper, Parakeet, and Qwen as model families that are especially tested, so compatibility should not be assumed for other models.
  • Reported benchmark results cover selected models and devices. They are not a guarantee for a different model, phone, or workload; benchmark your own target hardware.
  • The GitHub license field is NOASSERTION. Review the repository's current license files and obligations before incorporating it into a product.
  • The supplied project facts do not identify required runtimes, and the README's Linux setup lists development packages rather than a complete deployment matrix. Verify toolchain and platform requirements for your target environment.

Found this useful?

Share it with someone who would like cactus.

Comparisons