AI-Scientist-v2: Automate Scientific Experiments and Paper Drafts

Overview

AI-Scientist-v2 is an agentic research pipeline for exploring machine learning ideas without relying on human-written experiment templates. It generates hypotheses, runs and evaluates code-based experiments through a progressive tree search, then uses the results to produce a paper draft.

It is suited to researchers who want to investigate open-ended directions and can tolerate uncertain outcomes. The project describes its approach as more exploratory than its predecessor, with lower success rates than template-driven research workflows.

Key Features

• Generates and refines research ideas from a user-provided topic description.
• Can use Semantic Scholar for novelty checks and literature search.
• Explores experiment alternatives with best-first tree search and an experiment manager agent.
• Runs LLM-generated code and analyzes experiment results.
• Produces manuscript drafts and supports citation and review stages.
• Supports model providers including OpenAI, Gemini, and Claude via AWS Bedrock.
• Provides configuration for parallel exploration, search depth, debugging, and initial drafts.

Use Cases

• ML researchers can turn a broad topic into candidate hypotheses and experiments before committing to a manual study.
• Research groups can explore several experimental paths in parallel when a question does not fit a fixed template.
• Practitioners evaluating AI-generated research workflows can inspect experiment logs and tree visualizations alongside generated drafts.
• Teams preparing exploratory work can use the pipeline to create an initial manuscript draft, then review and validate its claims themselves.

Getting Started

The README documents a Linux, NVIDIA GPU, CUDA, and PyTorch setup. After installing the dependencies and configuring model API credentials, start by generating ideas:

python ai_scientist/perform_ideation_temp_free.py --workshop-file ai_scientist/ideas/my_research_topic.md --model gpt-4o-2024-05-13

See the repository README for installation, configuration, and the full experiment pipeline.

Alternatives

• autoresearch: Autoresearch iterates on a small language-model training setup, while AI-Scientist-v2 explores broader ML questions and can draft research papers.

Considerations

• The pipeline executes code written by LLMs. Run it in a controlled sandbox, such as a Docker container, and account for risks including dangerous packages, web access, and unintended processes.
• The documented setup targets Linux systems with NVIDIA GPUs, CUDA, and PyTorch. Experiments also require access to supported model APIs, and literature features may require a Semantic Scholar API key.
• Results are not guaranteed. Success depends on the model and research idea, and the project notes that v2 can be less reliable than template-based approaches.
• The README describes a project-specific AI Scientist Source Code License with a mandatory disclosure requirement for resulting manuscripts. The input metadata reports the license as NOASSERTION, so review the repository license terms before use.