LLM Gateway Open Source Projects
Discover 4 open source LLM Gateway repositories from GitHub, each with an analysis of what it does, key features, use cases and alternatives. LLM Gateway projects here are most often combined with LLM, AI and LLM Routing. Last updated October 20, 2026.
a3m-router: Route LLM Requests Across Providers
A3M Router is a self-hosted gateway that routes OpenAI-compatible chat requests among multiple LLM providers based on request complexity and cost. It also offers provider failover, semantic caching, and parallel ensembles for teams seeking control over model choice and routing.
NewHealth 42/100
otari: Run an OpenAI-Compatible LLM Gateway
Otari is Mozilla AI’s self-hostable LLM gateway, using any-llm to route OpenAI- and Anthropic-compatible requests to providers through one endpoint. It adds virtual keys, budgets, routing policies, and usage tracking for teams operating their own AI stack.
ActiveHealth 77/100
helicone: Monitor and Route LLM Requests
Helicone is a TypeScript platform for observing LLM requests and routing them across model providers. It helps AI teams inspect traces, track cost and latency, manage prompts, and self-host the observability stack.
ActiveHealth 56/100
kong: Route and Govern API, LLM, and MCP Traffic
Kong Gateway is a cloud-native, platform-agnostic API, LLM, and MCP gateway built in Lua. It centralizes traffic routing, security, and extensibility for teams running services on their own infrastructure or Kubernetes.
ActiveHealth 59/100