{"name":"autoresearch: AI Agents for Autonomous LLM Training Research","description":"autoresearch, by Andrej Karpathy, pioneers autonomous AI research by enabling agents to experiment with LLM training on a single GPU. The system allows an AI agent to modify code, train a model for a fixed 5-minute duration, and iteratively optimize for improved performance. This innovative approach aims to automate the experimental cycle of AI research, fostering continuous discovery and optimization.","github":"https://github.com/karpathy/autoresearch","url":"https://osrepos.com/repo/karpathy-autoresearch","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/karpathy-autoresearch","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/karpathy-autoresearch.md","json":"https://osrepos.com/repo/karpathy-autoresearch.json","topics":["Python","AI","Machine Learning","LLM","Autonomous Agents","Research Automation","Deep Learning"],"keywords":["Python","AI","Machine Learning","LLM","Autonomous Agents","Research Automation","Deep Learning"],"stars":null,"summary":"autoresearch, by Andrej Karpathy, pioneers autonomous AI research by enabling agents to experiment with LLM training on a single GPU. The system allows an AI agent to modify code, train a model for a fixed 5-minute duration, and iteratively optimize for improved performance. This innovative approach aims to automate the experimental cycle of AI research, fostering continuous discovery and optimization.","content":"## Introduction\n\n**autoresearch** is a visionary project by Andrej Karpathy that introduces a paradigm shift in AI research, enabling autonomous AI agents to conduct experiments on Large Language Model (LLM) training. As envisioned in its README, dated March 2026, the era of \"meat computers\" doing research is long gone, replaced by \"autonomous swarms of AI agents.\" This repository serves as the foundational story of how this future began.\n\nThe core idea is elegantly simple: provide an AI agent with a functional LLM training setup and let it autonomously experiment overnight. The agent modifies the `train.py` file, runs a training session for a fixed 5-minute budget, evaluates the results, and decides whether to keep or discard the changes, repeating the cycle. The human researcher, in turn, programs the `program.md` Markdown file, which provides context and instructions to the AI agents, effectively setting up an \"autonomous research organization.\" The training code is a simplified, single-GPU implementation based on Karpathy's `nanochat` project.\n\n## Installation\n\nGetting started with autoresearch requires a specific environment, primarily a single NVIDIA GPU.\n\n**Requirements**\n*   A single NVIDIA GPU (tested on H100)\n*   Python 3.10+\n*   [uv](https://docs.astral.sh/uv/ \"uv project manager\" target=\"_blank\" rel=\"noopener noreferrer\") (for dependency management)\n\n**Quick Start Steps**\n\nFollow these commands to set up and run a manual training experiment:\n\nbash\n# 1. Install uv project manager (if you don't already have it)\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# 2. Install dependencies\nuv sync\n\n# 3. Download data and train tokenizer (one-time, ~2 min)\nuv run prepare.py\n\n# 4. Manually run a single training experiment (~5 min)\nuv run train.py\n\n\nIf these steps complete successfully, your setup is ready for autonomous research mode.\n\n## Examples\n\nOnce your environment is set up, you can engage an AI agent to begin autonomous experimentation. The `program.md` file acts as the baseline instructions, or \"skill,\" for your agent.\n\nTo initiate an experiment with an AI agent (e.g., Claude or Codex, ensuring all permissions are disabled for safety):\n\n\nHi have a look at program.md and let's kick off a new experiment! let's do the setup first.\n\n\nThe agent will then interpret `program.md` and begin its iterative process of modifying `train.py` and running experiments.\n\n## Why Use autoresearch?\n\nautoresearch offers a compelling vision and practical framework for the future of AI development:\n\n*   **Autonomous Experimentation:** The primary benefit is the automation of the research cycle. Agents can tirelessly explore architectural changes, hyperparameters, and optimizers, potentially discovering novel improvements faster than human-led efforts.\n*   **Fixed Time Budget:** Each training run is strictly limited to 5 minutes. This design choice ensures that experiments are directly comparable on the same compute platform, regardless of the agent's modifications (e.g., model size, batch size). It also allows for a high throughput of experiments, with approximately 100 experiments possible overnight.\n*   **Focused and Manageable Scope:** The agent interacts with a single file, `train.py`, which contains the full GPT model, optimizer (Muon + AdamW), and training loop. This focused approach keeps the scope manageable and agent-generated diffs reviewable.\n*   **Self-Contained and Simple:** The project is designed to be self-contained with minimal external dependencies beyond PyTorch. It focuses on a single GPU, one file, and one metric (`val_bpb`), reducing complexity and making it accessible for rapid iteration.\n*   **Platform Optimization:** The fixed time budget encourages the agent to find the most optimal model for *your specific platform* within that time constraint, leading to highly tailored performance.\n\n## Links\n\nExplore the autoresearch project and related resources:\n\n*   **GitHub Repository:** [karpathy/autoresearch](https://github.com/karpathy/autoresearch \"karpathy/autoresearch GitHub Repository\" target=\"_blank\" rel=\"noopener noreferrer\")\n*   **Project Context Tweet 1:** [Tweet by @karpathy](https://x.com/karpathy/status/2029701092347630069 \"Tweet by @karpathy\" target=\"_blank\" rel=\"noopener noreferrer\")\n*   **Project Context Tweet 2:** [Tweet by @karpathy](https://x.com/karpathy/status/2031135152349524125 \"Tweet by @karpathy\" target=\"_blank\" rel=\"noopener noreferrer\")\n*   **Dummy's Guide to Neural Networks:** [Tweet by @hooeem](https://x.com/hooeem/status/2030720614752039185 \"Dummy's Guide to Neural Networks\" target=\"_blank\" rel=\"noopener noreferrer\")\n\n**Notable Forks**\n\n*   **MacOS:** [miolini/autoresearch-macos](https://github.com/miolini/autoresearch-macos \"miolini/autoresearch-macos\" target=\"_blank\" rel=\"noopener noreferrer\")\n*   **MacOS (MLX):** [trevin-creator/autoresearch-mlx](https://github.com/trevin-creator/autoresearch-mlx \"trevin-creator/autoresearch-mlx\" target=\"_blank\" rel=\"noopener noreferrer\")\n*   **Windows (RTX):** [jsegov/autoresearch-win-rtx](https://github.com/jsegov/autoresearch-win-rtx \"jsegov/autoresearch-win-rtx\" target=\"_blank\" rel=\"noopener noreferrer\")\n*   **AMD:** [andyluo7/autoresearch](https://github.com/andyluo7/autoresearch \"andyluo7/autoresearch\" target=\"_blank\" rel=\"noopener noreferrer\")","metrics":{"detailViews":10,"githubClicks":15},"dates":{"published":null,"modified":"2026-05-31T08:17:02.000Z"}}