tailpipe: Query Cloud and Application Logs with SQL

Overview

Tailpipe is a Go-based command-line tool for collecting and analyzing logs locally with SQL. It uses DuckDB for analytics and Parquet for storage, aiming to make large log datasets easier to investigate without relying on a centralized SIEM.

It is suited to developers, operations teams, and security practitioners who want to query cloud, container, or application events, and optionally build detections from those queries. Plugins provide integrations for data sources, while community-maintained queries and dashboards extend the analysis workflow.

Key Features

• Query collected log data using SQL from the terminal.
• Analyze data locally using DuckDB and store it in Parquet.
• Collect logs from cloud, container, and application sources through plugins.
• Configure connections, data sources, and partitions for log collection.
• Use community-provided detections, benchmarks, and dashboards, including MITRE ATT&CK-aligned content.
• Write custom SQL queries and define detections as code.

Use Cases

• Cloud security teams can investigate activity such as AWS CloudTrail events and look for suspicious changes.
• Developers can inspect application or container logs with SQL during incident triage or debugging.
• Incident responders can collect relevant logs locally for focused analysis and forensics.
• Platform and DevOps teams can build repeatable log queries and detections around the sources they operate.

Getting Started

Install on macOS with Homebrew:

brew install turbot/tap/tailpipe

Then install a source plugin, configure credentials and a data source, and collect logs. See the repository README and documentation for setup details.

Alternatives

• nerdlog: Nerdlog queries logs across SSH-accessible hosts and visualizes results on a timeline, while Tailpipe analyzes collected events locally with DuckDB and SIEM detections.

Considerations

• Tailpipe runs locally, so users are responsible for configuring log sources, credentials, storage, and collection.
• Its workflow depends on plugins for source integrations; check the Tailpipe Hub for the sources and tables you need.
• The repository uses AGPL-3.0. The README also describes Turbot-specific commercial terms and trademark restrictions, so review the project’s licensing information before redistributing it.
• The repository is not archived. Its README describes building the core binary with make, but does not specify minimum system requirements.