agent-observability: Monitor AI Coding Agents Locally

Overview

agent-observability collects OpenTelemetry metrics, logs, and traces from Claude Code and OpenAI Codex, stores them in a local observability stack, and displays them in Grafana dashboards. It gives developers and teams a way to inspect agent usage and behavior without sending telemetry to a third party.

The project is aimed at people who already run these coding agents and want visibility into tokens, tool activity, sessions, and performance. It makes sense when you can operate the supporting services yourself and want dashboards ready to use rather than building telemetry queries from scratch.

Key Features

• Collects metrics, logs, and traces through an OpenTelemetry Collector.
• Provides Grafana dashboards for Claude Code and OpenAI Codex, including separate log views.
• Routes metrics to Prometheus, logs to Loki, and traces to Tempo.
• Surfaces agent-specific details such as token usage, tool activity, latency, and session events.
• Offers deployment paths for Docker Compose, Kubernetes with Kustomize, and Helm.
• Keeps telemetry and storage in the operator's environment.

Use Cases

• Individual developers can review token usage, sessions, and tool activity across their coding-agent work.
• Teams can investigate agent errors and latency using centralized dashboards and searchable logs.
• Self-hosting or privacy-conscious users can inspect agent telemetry without forwarding it to an external observability provider.
• Operators already running Kubernetes can deploy agent monitoring alongside their existing cluster services.

Getting Started

For a quick Docker Compose deployment:

git clone https://github.com/KB1SLN-Labs/agent-observability.git
cd agent-observability
docker compose up -d

Configure the agent's OpenTelemetry exporter to send data to the collector, then open Grafana at http://localhost:3000. See the README for agent configuration, Kubernetes and Helm setup, and troubleshooting.

Considerations

• You must run and maintain the collector, Prometheus, Loki, Tempo, and Grafana yourself; this is infrastructure, not a hosted service.
• Agent telemetry export must be configured and enabled. Claude Code traces require its enhanced telemetry beta flag, and Codex signal availability varies by entry point.
• Dashboard coverage depends on the data an agent emits. For example, Codex CLI does not emit the desktop-only time-to-first-token signal, and Codex telemetry does not include cost data.
• Dashboard cost figures for Claude Code are API-equivalent estimates, not subscription charges.
• The repository reports no license value in the supplied metadata, so licensing terms are not established here.