samtroberts

openhydra: Share AI Inference Peer to Peer

OpenHydra is a Rust peer-to-peer protocol for distributed AI inference, built on libp2p and designed to work with engines such as Ollama. Providers share models they already run; consumers use an OpenAI-compatible gateway without a central coordinator.

ActiveRustAILLMNetworking
View on GitHub Compare
Stars
4
Forks
0
Last commit
29 days ago
Contributors
2
Releases, 12 months
27

Our take

Activeregular commits and releases

Health score

36/100

How it is scored
Activity
81
Community
20
Issues
21
Pull requests
20
  • Releases more often than 78% of the projects we track
  • Fewer contributors than 85% of the projects we track

OpenHydra is an actively developed, unusually broad peer-to-peer inference project, but its recent activity is concentrated in one contributor and the README does not establish production reliability.

Good fit if

  • You can run a supported inference engine and want to experiment with sharing its capacity.
  • You need an OpenAI-compatible gateway and are willing to use a peer network rather than a central service.
  • You are comfortable evaluating a Rust project with recent commits and releases, while accounting for its small contributor base.

Look elsewhere if

  • You need a managed inference service, a coordinator, or an established service-level guarantee. The project describes a peer network, not those services.
  • Your deployment depends on a prebuilt Intel Mac or Linux ARM64 binary, which the README says is not yet available.
  • Your team needs evidence of broader maintenance or issue-resolution activity before adoption; measured contributor activity is concentrated in one person, with no issues closed or pull requests merged in the recent period.
All health signals
Last commit2026-10-22 (29 days ago)
Commits, last 90 days100+
Releases, last 12 months27 (latest v0.3.24, 2026-10-22)
Contributors2 (top contributor: 89% of commits)
Issues closed, last 90 days0
Pull requests merged, last 90 days0
Project age8 months

Checked on 2026-11-21 with the GitHub API.

Overview

OpenHydra is a Rust peer-to-peer protocol for distributed AI inference, built on libp2p. It sits between inference engines such as Ollama, vLLM, LM Studio, and llama.cpp and a swarm of peers: providers share models already served by their engines, while consumers route requests to a matching peer. Its goal is to make inference available through a shared network without relying on a central coordinator or subscription.

The project is aimed at people who can contribute local inference capacity as well as those who want to consume it. Its single Rust binary has separate provider and consumer roles, and the consumer gateway exposes an OpenAI-compatible API. The README describes support for macOS, Linux, and Windows binaries, with some platform and architecture gaps in prebuilt downloads.

Key Features

  • Uses libp2p discovery and transport, including Kademlia DHT, mDNS, relay support, and NAT traversal.
  • Adapts existing inference engines instead of running models itself, with named adapters for Ollama, vLLM, LM Studio, llama.cpp, Exo, OpenAI-compatible servers, and ComfyUI.
  • Provides an OpenAI-compatible consumer gateway with streaming and non-streaming chat completions.
  • Advertises provider models for discovery and routes requests to a peer offering the requested model.
  • Uses a give-to-get reciprocity model, with no tokens or subscription described in the README.
  • Co-signs inference receipts using node identities held inside the Rust daemon.
  • Offers a desktop app alongside the command-line agent.

Use Cases

  • A person already running Ollama or another supported engine can share its available models without replacing that engine.
  • A developer can point an existing OpenAI-compatible client at the gateway to try inference from peers.
  • A small group with compatible machines can share inference capacity peer to peer rather than operate a central inference coordinator.
  • A team evaluating distributed inference can inspect routing, discovery, and receipt handling in a Rust implementation.

What you need

Detected in the repository

  • Rust (from Cargo.toml)
  • Automated checks on GitHub Actions

Getting Started

Build the agent from source with a stable Rust toolchain:

git clone https://github.com/samtroberts/openhydra && cd openhydra
cargo build --release -p openhydra-agent

See the README for prebuilt downloads, engine configuration, and gateway usage.

License in Plain Words

No standard license was detected. Without a license, the default copyright rules apply: you may read the code, but not reuse it. Check the repository or ask the authors.

Alternatives

  • SwarmLLM: SwarmLLM pools computers to run larger models, while OpenHydra connects peers that share models they already run.
ProjectLanguageLicenseStarsStatus
openhydraRust–4Active
SwarmLLMRustApache-2.055Active

Considerations

  • OpenHydra is a protocol and proxy, not an inference engine. Sharing capacity requires a supported engine running locally; consuming through the gateway does not require one.
  • The project is young and maintained by a small, concentrated contributor group, even though measured activity includes regular commits and releases. Recent issue and pull-request handling data shows no closures or merges in the measured period.
  • Prebuilt binaries are not available for every platform and are not code-signed, according to the README. Building from source requires a stable Rust toolchain.
  • Token-count validation may download a model tokenizer from Hugging Face on first use. The README notes that this reveals which model's tokenizer is requested, though prompts and answers are not sent there; downloads can be disabled.
  • The repository reports no asserted license in its GitHub metadata, while the README and license badge state Apache 2.0. Confirm the repository's license file and terms before relying on that grant.

Found this useful?

Share it with someone who would like openhydra.