EnvHarness: Dynamically Adapting Environments for Agent Learning

Introduction

As large language models (LLMs) evolve into autonomous agents, their learning increasingly shifts from curated text to interactive environments. However, these environments are often expensive to build and remain static, failing to adapt to an agent's progress or specific weaknesses. EnvHarness addresses this challenge by applying the "agent harness" concept to environments themselves. It wraps a frozen environment with its own plug-in components, making it dynamically controllable without requiring any changes to the environment's internal code.

This dynamic layer is built from three core plug-in components:

• Setup: Reshapes the initial state of the environment.
• Rule: Modifies the interaction, including allowed actions, their effects, and agent observations.
• Link: Composes tasks from another environment into the current one.

These components operate strictly at the standard reset and step interface, ensuring that the original benchmark's trusted, human-built verifiers remain untouched. This benchmark-agnostic system works across various domains, enabling a designer agent to diagnose agent weaknesses, write components to target them, and iteratively improve the environment to teach what the agent lacks.

Why Use EnvHarness and Key Benefits

EnvHarness offers significant advantages for developing and training LLM agents:

Enhanced Agent Learning

Skills learned in EnvHarness-adapted environments consistently outperform both no-skill baselines and skills learned in original, static environments. Agents demonstrate increased effectiveness (up to +9 points on held-out tasks) and improved efficiency (~9.8% fewer interaction steps). The system also produces stronger policies under reinforcement learning, with gains compounding through repeated designer loops.

Key Features

• Frozen Environments, No Internal Edits: The benchmark's task set, dynamics, and grading remain exactly as published. Only the layer through which the agent interacts changes.
• Code as the EnvHarness: The designer agent generates real Python code, specifically a _Rules(Rules) subclass, rather than selecting from a fixed menu. This code is compiled and executed in an isolated subprocess, ensuring robustness.
• Composable by Construction: An EnvHarness itself acts as an ActionableEnv that wraps another, allowing layers to stack arbitrarily. This ensures a single, consistent interface for any benchmark.
• Benchmark-Agnostic: Integrating a new environment simply requires implementing one interface (ActionableEnv). The designer, components, learning loop, and evaluation stages require no further modifications.

Quickstart Guide

To get started with EnvHarness, you'll typically configure your LLM provider and then run one of the provided benchmarks.

LLM Configuration

EnvHarness supports various LLM providers. You'll need to set up your API keys or credentials:

• GPT (OpenAI):

  export OPENAI_API_KEY="your-openai-api-key"
  

• Claude (Vertex AI):

  gcloud auth application-default login
  export GOOGLE_CLOUD_PROJECT="your-project-id"
  pip install "google-cloud-aiplatform>=1.38"
  

• Gemini:

  export GEMINI_API_KEY="your-gemini-api-key"
  

You can specify the model in your configuration YAML or via an environment variable:

policy:
  model: openai/gpt-4.1-mini
mutator:
  type: llm
  model: vertex_ai/claude-sonnet-4-6

Or, to override for a full run:

MODEL=openai/gpt-4.1 python experiments/swebench/reproduce.py

Run a Benchmark

Each benchmark has its own environment and a dedicated driver. For detailed setup and commands, refer to the README.md file within the specific experiment folder you wish to run.

Common commands include:

• Preflight check:

  python scripts/check_env.py <benchmark>
  

• Smoke test (fewer tasks):

  bash experiments/<benchmark>/reproduce_smoke.sh
  

• Full protocol:

  python experiments/<benchmark>/reproduce.py
  

Benchmarks and Examples

EnvHarness has been tested across a range of benchmarks. You can find examples and reproduction scripts in the experiments/ directory:

• experiments/toy24
• experiments/alfworld
• experiments/swebench
• experiments/webarena
• experiments/officeqa
• experiments/spreadsheetbench

For reinforcement learning training with policies directly inside EnvHarness environments, explore the rl/ directory.

Links

• GitHub Repository: google-research/envharness
• Paper: EnvHarness: Awakening Static Worlds for Agent Learning
• Project Webpage: envharness.com