llm-d-router: Route LLM Inference with Envoy and Gateway API
llm-d-router is a Go-based router for LLM inference traffic, using an Endpoint Picker with Envoy and the Kubernetes Gateway API. It routes requests using signals such as model-server load and KV-cache locality, with options for standalone or gateway deployments.
ActiveHealth 80/100
GoAIKubernetes
Added Sep 25, 2026 View details