# SwarmLLM: Run Local AI Models and Team Up for Giant Distributed Inference

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/enapt-swarmllm
Generated for open source discovery and AI-assisted research.

SwarmLLM is a free, open-source application that allows you to run AI chat models directly on your own computer. It uniquely enables multiple computers to team up over the internet, collectively running models too large for a single machine. This platform offers an OpenAI and Anthropic-compatible API, all without requiring accounts or cryptocurrency.

GitHub: https://github.com/enapt/SwarmLLM
OSRepos URL: https://osrepos.com/repo/enapt-swarmllm

## Summary

SwarmLLM is a free, open-source application that allows you to run AI chat models directly on your own computer. It uniquely enables multiple computers to team up over the internet, collectively running models too large for a single machine. This platform offers an OpenAI and Anthropic-compatible API, all without requiring accounts or cryptocurrency.

## Topics

- ai
- llm
- decentralized
- distributed-inference
- rust
- local-llm
- openai-api
- open-source

## Repository Information

Last analyzed by OSRepos: Fri Oct 02 2026 00:17:00 GMT+0100 (Western European Summer Time)
Detail views: 2
GitHub clicks: 0

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

SwarmLLM is an innovative, open-source application designed to bring the power of AI chat models to your local machine. Developed in Rust, it provides a ChatGPT-style assistant using open AI models from various providers like Meta, Google, and Mistral. What sets SwarmLLM apart is its ability to facilitate distributed inference, allowing multiple computers to collaborate and run AI models that would otherwise be too large for any single machine. This means you can access more powerful AI, leveraging a collective network of resources.

## Why Use SwarmLLM and Key Benefits

SwarmLLM offers a compelling alternative to cloud-based AI services with several significant advantages:

*   **Local AI Chat**: Run AI models directly on your computer. Once downloaded, your conversations remain private, as there's no external company involved.
*   **Distributed Inference**: Access models too large for your machine. Computers in the swarm each hold a part of a larger model, working together to answer your questions. This is fastest when computers are geographically close.
*   **Community Contribution**: By leaving SwarmLLM running, your computer can help the swarm by sharing model parts and assisting with other users' requests, similar to seeding a torrent. In return, you gain access to the collective power of the swarm.
*   **Device Linking**: Create a private group with your own devices, such as a laptop, desktop, and home server, to utilize them as one larger machine.
*   **API Compatibility**: Integrate SwarmLLM with existing applications, coding assistants, and AI agents that are compatible with ChatGPT or Claude, using a local OpenAI- and Anthropic-compatible endpoint.
*   **No Hidden Costs**: The project is free, open-source, and operates without accounts, subscriptions, ads, or cryptocurrency. The only cost is your electricity and internet usage while contributing.
*   **Privacy Options**: While models running entirely on your machine ensure privacy, the public swarm offers partial privacy with encrypted connections. SwarmLLM also provides 'Private Mode' to keep questions within your linked devices or local network.

## Installation

Getting started with SwarmLLM is straightforward. You can find the latest releases and detailed instructions on the [SwarmLLM GitHub releases page](https://github.com/enapt/SwarmLLM/releases/latest).

**For Windows:**

1.  Download `swarmllm-windows-x86_64-gpu.zip` if you have an NVIDIA, AMD, or Intel graphics card. Otherwise, use `swarmllm-windows-x86_64-cpu.zip`.
2.  Extract the downloaded ZIP file.
3.  Double-click `swarmllm.exe`. You may need to click "More info" then "Run anyway" if Windows displays a security warning.
4.  Leave the black console window open, as it signifies SwarmLLM is running. The app will open in your web browser automatically.

**For Mac (Apple Silicon M1 or newer):**

1.  Download `swarmllm-macos-aarch64.tar.gz` and double-click to unpack.
2.  Keep the folder in `Documents` or `Downloads`.
3.  Double-click `swarmllm`. If macOS blocks the launch, go to `System Settings ? Privacy & Security`, click "Open Anyway", and double-click again.
4.  A Terminal window will open, indicating SwarmLLM is running. The app will appear in your web browser.

**For Linux:**

1.  Download `swarmllm-linux-x86_64-cuda.tar.gz` for NVIDIA RTX 30-series or newer, or `swarmllm-linux-x86_64.tar.gz` otherwise.
2.  Unpack the archive and run `./swarmllm run`.

After installation, pick a model within the app and start chatting. The first model download is a one-time process of a few gigabytes. If the browser doesn't open, navigate to `localhost:8800`.

## Examples

SwarmLLM provides an OpenAI- and Anthropic-compatible endpoint, allowing developers to integrate it into their existing tools. Here are examples for using the API:

**OpenAI-compatible API call:**

bash
export SWARMLLM_KEY=...   # from Settings ? Access Token

curl http://localhost:8800/v1/chat/completions \
  -H "Authorization: Bearer $SWARMLLM_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-3.2-3b-instruct-q4-k-m",
       "messages": [{"role": "user", "content": "Hello!"}]}'


**As a Claude Code backend:**

bash
ANTHROPIC_BASE_URL=http://localhost:8800 ANTHROPIC_AUTH_TOKEN="$SWARMLLM_KEY" \
  claude --model qwen2.5-coder-7b-instruct-q4-k-m


## Links

*   **GitHub Repository**: [https://github.com/enapt/SwarmLLM](https://github.com/enapt/SwarmLLM){:target="_blank"}
*   **Latest Releases**: [https://github.com/enapt/SwarmLLM/releases/latest](https://github.com/enapt/SwarmLLM/releases/latest){:target="_blank"}
*   **Discord Community**: [https://discord.gg/nq9be3u828](https://discord.gg/nq9be3u828){:target="_blank"}
*   **Installation Guide**: [https://enapt.github.io/SwarmLLM/getting-started/installation.html](https://enapt.github.io/SwarmLLM/getting-started/installation.html){:target="_blank"}
*   **API Reference**: [https://enapt.github.io/SwarmLLM/api/openai.html](https://enapt.github.io/SwarmLLM/api/openai.html){:target="_blank"}
*   **Issues**: [https://github.com/enapt/SwarmLLM/issues](https://github.com/enapt/SwarmLLM/issues){:target="_blank"}
*   **Contributing**: [https://github.com/enapt/SwarmLLM/blob/main/CONTRIBUTING.md](https://github.com/enapt/SwarmLLM/blob/main/CONTRIBUTING.md){:target="_blank"}