LLM Routing Open Source Projects
Discover 3 open source LLM Routing repositories from GitHub, each with an analysis of what it does, key features, use cases and alternatives. LLM Routing projects here are most often combined with AI, LLM and LLM Gateway. Last updated October 20, 2026.
llm-d-router: Route LLM Inference with Envoy and Gateway API
llm-d-router is a Go-based router for LLM inference traffic, using an Endpoint Picker with Envoy and the Kubernetes Gateway API. It routes requests using signals such as model-server load and KV-cache locality, with options for standalone or gateway deployments.
ActiveHealth 80/100
a3m-router: Route LLM Requests Across Providers
A3M Router is a self-hosted gateway that routes OpenAI-compatible chat requests among multiple LLM providers based on request complexity and cost. It also offers provider failover, semantic caching, and parallel ensembles for teams seeking control over model choice and routing.
NewHealth 42/100
otari: Run an OpenAI-Compatible LLM Gateway
Otari is Mozilla AI’s self-hostable LLM gateway, using any-llm to route OpenAI- and Anthropic-compatible requests to providers through one endpoint. It adds virtual keys, budgets, routing policies, and usage tracking for teams operating their own AI stack.
ActiveHealth 77/100