kong vs llm-d-router
API gateway and LLM inference routing compared
kong is a configurable gateway for API, AI, and MCP traffic across Kubernetes and other infrastructure. llm-d-router focuses on routing LLM inference requests in Kubernetes using serving signals such as cache locality, load, and priority.
kong: Route and Govern API, LLM, and MCP Traffic
Kong Gateway is a cloud-native, platform-agnostic API, LLM, and MCP gateway built in Lua. It centralizes traffic routing, security, and extensibility for teams running services on their own infrastructure or Kubernetes.
llm-d-router: Route LLM Inference with Envoy and Gateway API
llm-d-router is a Go-based router for LLM inference traffic, using an Endpoint Picker with Envoy and the Kubernetes Gateway API. It routes requests using signals such as model-server load and KV-cache locality, with options for standalone or gateway deployments.
| kong | llm-d-router | |
|---|---|---|
| Language | Lua | Go |
| License | Apache-2.0 | Apache-2.0 |
| Stars | 44.3k | 422 |
| Forks | 5.2k | 472 |
| Last analyzed | Oct 20, 2026 | Oct 20, 2026 |
Key differences
- kong covers general API traffic controls, including authentication, rate limiting, transformations, and observability; llm-d-router focuses on inference-aware endpoint selection and request flow control.
- kong is written in Lua and extends gateway behavior through plugins; llm-d-router is written in Go and integrates with L7 proxies such as Envoy through ext-proc.
- kong supports declarative database-less and hybrid control-plane/data-plane deployments; llm-d-router offers standalone Envoy and Gateway API modes.
- kong can route LLM traffic across providers and govern MCP traffic; llm-d-router uses InferencePool state and supports model rewrites, priorities, and performance objectives.
- Both use the Apache-2.0 license. kong reports 44.2k stars and 5.2k forks, while llm-d-router reports 373 stars and 410 forks.
Choose kong if you…
- need shared API controls such as authentication, rate limits, and request transformations across services.
- want a gateway for API, LLM, or MCP traffic across Kubernetes and other infrastructure.
- prefer extending gateway behavior with Kong or community plugins.
Choose llm-d-router if you…
- run LLM serving on Kubernetes and need routing informed by cache locality, load, or request priority.
- want to connect inference backends through Envoy or Gateway API infrastructure.
- need inference-specific priorities, performance objectives, model-name rewrites, or multi-stage workflow coordination.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.