Rerun: Open Source SDK for Multimodal Data Logging and Visualization

Introduction

Rerun is an open-source SDK developed by rerun-io, providing a comprehensive solution for logging, storing, querying, and visualizing multimodal and multi-rate data. It is built to help developers understand and improve complex processes involving rich data types like 2D, 3D, text, time series, and tensors. Rerun offers SDKs for C++, Python, and Rust, allowing seamless integration into various projects, with data streamed to a dedicated Rerun Viewer for live visualization or saved to file for post-mortem analysis.

This powerful tool finds its applications across diverse fields such as robotics, spatial and embodied AI, generative media, industrial processing, simulation, security, and health, making it an essential asset for anyone dealing with sensor data or evolving 2D/3D states over time.

Installation

Getting started with Rerun is straightforward. You can integrate the SDK into your projects using your preferred language:

• C++: Follow the quick start guide on the Rerun website.
• Python: Install via pip with pip install rerun-sdk or through conda. Find more details in the Python quick start guide.
• Rust: Add to your project with cargo add rerun. Refer to the Rust quick start guide for more information.

To visualize your logged data, you'll also need the rerun binary viewer. For Python users, it's bundled with pip install rerun-sdk. For C++ and Rust, you can install it separately using cargo install rerun-cli --locked --features nasm. Note that the nasm feature requires the nasm CLI to be installed for optimal video decoding performance.

Examples

Here's a quick taste of how to use the Rerun Python SDK to log 3D points:

import rerun as rr  # pip install rerun-sdk

rr.init("rerun_example_app")

rr.spawn()  # Spawn a child process with a viewer and connect
# rr.save("recording.rrd")  # Stream all logs to disk
# rr.connect_grpc()  # Connect to a remote viewer

# Associate subsequent data with 42 on the “frame” timeline
rr.set_time("frame", sequence=42)

# Log colored 3D points to the entity at `path/to/points`
rr.log("path/to/points", rr.Points3D(positions, colors=colors))
# ... more logging ...

This snippet initializes Rerun, spawns a viewer, sets a time context, and logs 3D point data, demonstrating the simplicity of integrating Rerun into your data pipelines.

Why Use Rerun

Rerun excels as a visual and temporal debugger for complex systems. Imagine a vacuum cleaning robot that keeps hitting walls. Traditional debuggers or simple text logs would fall short in explaining why. Rerun allows you to log and visualize all the robot's internal representations of the world, such as RGB camera feeds, depth images, lidar scans, segmentation images, 3D maps, detected objects, and confidence levels.

By observing how these diverse data streams evolve over time, you can pinpoint the exact moment and reason for failures. This could reveal issues like sensor glare confusing a segmentation network, a bug in lidar code, or odometry errors. Beyond debugging, Rerun helps in improving algorithms, setting up new test cases, and creating datasets for model training and evaluation through its query APIs. It transforms how you understand and iterate on systems that rely on multimodal, time-series data.

Links

Explore Rerun further with these official resources:

• High-level Documentation
• Loggable Types Reference
• Examples
• C++ API Documentation
• Python API Documentation
• Rust API Documentation
• GitHub Repository