Curie: Automated and Rigorous Scientific Experimentation with AI Agents

Introduction

Curie is the first AI-agent framework specifically engineered for automated and rigorous scientific experimentation. Its core mission is to empower scientists by accelerating research at the speed of thought. Curie helps answer complex questions through end-to-end experimentation automation, ensuring that every step, from hypothesis formulation to result interpretation, is conducted with precision, reliability, and reproducibility.

Why Use and Key Features

Curie offers a robust solution for researchers looking to enhance their experimental workflows. Its design focuses on automating tedious processes while maintaining scientific rigor.

Key Features include:

• Automated Experimentation: Manages the entire process, from hypothesis formulation and experiment implementation to execution, result analysis, and finding reflection.
• Rigor Enhancement: Incorporates built-in verification modules to enforce methodical procedures, agent reliability, and reproducibility.
• Broad Applicability: Supports various domains including ML Engineering, system analysis, and general scientific discovery.
• Use Your Starter Code: Allows integration with arbitrary user-provided starter codebases.
• Bring Your Own Dataset: Compatible with arbitrary user datasets.
• Automatic, Insightful Reporting: Generates comprehensive experiment reports automatically.

Installation

To get started with Curie, follow these installation steps. Docker is a prerequisite.

Prerequisite: Install Docker from here

sudo chmod 666 /var/run/docker.sock
docker ps  # Verify Docker installation

Option 1: Quick Install via pip

pip install curie-ai

Option 2: Manual Installation for Developers

Refer to the official documentation for detailed manual installation instructions.

Examples

Curie provides flexible ways to automate experiments, from simple question verification to complex ML strategy optimization.

It's recommended to use tmux or a similar terminal multiplexer before running Curie, as experiments can take several minutes depending on the task and budget. Do not use Jupyter Notebook.

Example 1: Verifying a Single Question

Question: How does the choice of sorting algorithm impact runtime performance across different input distributions?

import curie
# Set up your API keys, refer to curie/setup/env.sh.example
key_dict = {
    "MODEL": "claude-3-7-sonnet-20250219",
    "ANTHROPIC_API_KEY": "your-anthropic-key"
}

result = curie.experiment(api_keys=key_dict, 
                          question="How does the choice of sorting algorithm impact runtime performance across different input distributions?",
                          max_global_steps=10)

This will generate an auto-generated experiment report and an experimentation process log.

Example 2: Finding Optimal ML Strategies for Noisy Cancer Data

Question: Are ensemble methods (e.g., Random Forests, Gradient Boosting) more robust to added noise in the Breast Cancer Wisconsin dataset compared to linear models like Logistic Regression for a binary classification task?

result = curie.experiment(api_keys=key_dict, 
                          question="Are ensemble methods (e.g., Random Forests, Gradient Boosting) more robust to added noise in the Breast Cancer Wisconsin dataset compared to linear models like Logistic Regression for a binary classification task?")

Example 3: Gaining Insight from Your Dataset with Optional Starter Code

Question: How to improve my prediction accuracy on my dataset.

result = curie.experiment(
    api_keys=key_dict,
    question="E.g. How to improve my prediction accuracy on my dataset.",
    dataset_dir="/abs/path/to/your/dataset",
    codebase_dir="[Optional] /abs/path/to/your/code",
    env_requirements="[Optional] /abs/path/to/requirements.txt",
)

Curie's AutoML feature can find optimal ML solutions, as demonstrated in examples from MLE-Bench. More Machine Learning Use Cases are available here.

Links

• GitHub Repository: Just-Curieous/Curie
• arXiv Paper (Curie): Toward Rigorous and Automated Scientific Experimentation with AI Agents
• arXiv Paper (EXP-Bench): Can AI Conduct AI Research Experiments?
• PyPI: curie-ai
• Official Blog: Just-Curieous Blog
• Live Demo: Curie Demo
• Tutorial: How to use Curie
• Demo Video: Curie Overview & Demo on YouTube
• Slack Community: Join Just-Curieous Slack
• License: Apache 2.0 License