llm-d-router vs otari
LLM request routing and gateways compared
llm-d-router routes inference requests within Kubernetes using serving signals such as KV-cache locality, load, and priority. otari provides a self-hosted, OpenAI- and Anthropic-compatible gateway for routing requests across LLM providers, with access controls and budget tracking.
llm-d-router: Route LLM Inference with Envoy and Gateway API
llm-d-router is a Go-based router for LLM inference traffic, using an Endpoint Picker with Envoy and the Kubernetes Gateway API. It routes requests using signals such as model-server load and KV-cache locality, with options for standalone or gateway deployments.
otari: Run an OpenAI-Compatible LLM Gateway
Otari is Mozilla AI’s self-hostable LLM gateway, using any-llm to route OpenAI- and Anthropic-compatible requests to providers through one endpoint. It adds virtual keys, budgets, routing policies, and usage tracking for teams operating their own AI stack.
| llm-d-router | otari | |
|---|---|---|
| Language | Go | Python |
| License | Apache-2.0 | Apache-2.0 |
| Stars | 422 | 660 |
| Forks | 472 | 83 |
| Last analyzed | Oct 20, 2026 | Oct 19, 2026 |
Key differences
- llm-d-router focuses on inference-aware routing to serving endpoints, while otari centralizes access to multiple LLM providers through a shared API.
- llm-d-router integrates with Envoy or Gateway API infrastructure; otari runs as a self-hosted gateway with its own database or a data plane connected to otari.ai.
- llm-d-router is written in Go, while otari is written in Python.
- Both use the Apache-2.0 license; the supplied repository figures show 373 stars and 410 forks for llm-d-router, compared with 503 stars and 59 forks for otari.
- llm-d-router is aimed at Kubernetes model-serving teams; otari is aimed at teams managing provider credentials, access, and spend across applications.
Choose llm-d-router if you…
- need routing decisions based on inference-serving conditions such as cache locality, load, and request priority.
- want to connect inference backends to Envoy or Gateway API infrastructure.
- need model-name rewrites or coordination for multi-stage inference workflows.
Choose otari if you…
- want one OpenAI- and Anthropic-compatible endpoint for multiple LLM providers.
- need virtual keys, budget checks, and usage records for provider access.
- want to configure failover, weighted routing, or learned selection across providers.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.