helicone: Monitor and Route LLM Requests

Overview

Helicone combines LLM observability with an AI gateway. It gives teams a way to inspect requests and agent sessions, track operational metrics, and access multiple model providers through a unified API.

It is aimed at developers building LLM applications who need debugging and usage visibility, or who want provider routing and fallbacks. You can use Helicone through its hosted service or self-host its observability components.

Key Features

• Log and inspect LLM requests, traces, and sessions.
• Track request costs and latency, with quality metrics also described in the project documentation.
• Route requests to more than 100 models through a unified gateway API, with intelligent routing and automatic fallbacks.
• Test prompts and inspect sessions and traces in a playground.
• Version and deploy prompts through the gateway.
• Connect with providers and frameworks including OpenAI, Anthropic, LangChain, and the Vercel AI SDK.
• Self-host the observability platform using Docker Compose.

Use Cases

• An AI application team debugging a chatbot can inspect requests and sessions to investigate unexpected outputs.
• A team using several model providers can route requests through one API and configure fallbacks.
• Developers monitoring production usage can review cost and latency and export data to PostHog for custom dashboards.
• A team iterating on prompts can use production data to manage versions and test changes in the playground.
• Organizations that need to run observability infrastructure themselves can deploy the platform with Docker.

Getting Started

For a hosted gateway, configure the OpenAI client with Helicone's gateway URL and an API key. For self-hosting, start with the repository's Docker Compose instructions:

git clone https://github.com/Helicone/helicone.git
cd docker
cp .env.example .env
./helicone-compose.sh helicone up

See the README and documentation for setup details and integrations.

Alternatives

• opik: Opik focuses on tracing and evaluating LLM workflows and agents, while Helicone also routes requests across model providers and manages prompts.
• phoenix: Phoenix centers on self-hosted tracing and evaluation, while Helicone also provides an LLM gateway for routing requests across providers.

Considerations

The repository is active, with regular commits and contributions from many developers; contributor activity is not concentrated in one person. However, it has had no release in over a year, and issue resolution is relatively slow, so teams relying on release cadence or prompt support should account for that. Self-hosting is not a single-service deployment: the documented architecture includes a web app, logging services, Supabase, ClickHouse, and Minio. The project requires Node.js 20 or later, and Docker is available for deployment; manual deployment is explicitly not recommended. The repository is licensed under Apache-2.0.