agent.cpp: Build Local AI Agents in C++

Overview

agent.cpp is a C++ library for building local agents around llama.cpp. It addresses the need for agent-building components in C++ applications without routing model inference to an external LLM API.

The library combines a model with an agent loop, instructions, tools, and lifecycle callbacks. It is aimed at C++ developers who want to run GGUF models locally and customize how agents handle context, tool execution, and stopping conditions.

Key Features

• Runs local GGUF models using llama.cpp, including configurable text generation and KV cache management.
• Provides an agent loop that alternates between model calls and tool execution.
• Defines tools with descriptions, JSON Schema parameters, and execution functions.
• Offers callbacks around agent loops, model calls, and tool execution for context changes, validation, logging, or approval flows.
• Supports GBNF grammar constraints for structured model output.
• Supports LoRA adapters, including separate adapters for models that share base weights.
• Includes examples for memory, multi-agent delegation, shell interaction, tracing, grammar, and context engineering.

Use Cases

• C++ application developers can add local model-backed tool use without relying on external LLM APIs.
• Teams can build specialized agents that share model weights while using separate LoRA adapters.
• Developers can constrain text-only agent responses to a defined format with a GBNF grammar.
• Applications needing human review or custom context handling can use callbacks to intervene in the agent lifecycle.
• Developers exploring multi-agent systems in C++ can use the delegation example as a starting point.

Project Facts

• Language: C++
• License: MIT
• Stars: 210
• Forks: 31
• Archived: No

Getting Started

Requires C++17 or higher. Add the project to a CMake build with FetchContent:

include(FetchContent)
FetchContent_Declare(
    agent-cpp
    GIT_REPOSITORY https://github.com/mozilla-ai/agent.cpp
    GIT_TAG main
)
FetchContent_MakeAvailable(agent-cpp)
target_link_libraries(my_app PRIVATE agent-cpp::agent)

See the repository README for installed-package and submodule options, model setup, examples, and build configuration.

Alternatives

• rig: Rig is a Rust library with shared integrations for multiple model providers and vector stores, rather than a C++ library centered on local llama.cpp models.
• deepagents: Deep Agents is a Python harness with planning, delegation, and file access built in, rather than a C++ library focused on a local-model agent loop.
• harness-sdk: Strands offers Python and TypeScript agent tools with model flexibility, rather than a C++ library focused on running small models through llama.cpp.

Considerations

• This is designed for local inference through llama.cpp, not for calling external LLM APIs.
• You need a GGUF model to run the examples. The README's default configuration is optimized for granite-4.0-micro; other models may need different context and sampling settings.
• llama.cpp is included as a submodule, and hardware acceleration requires the appropriate build options and supported backend.
• Grammar constraints apply to tool-call responses too. If the grammar excludes the tool-call format, the agent cannot call tools.
• Prompt cache files do not record active LoRA adapters, so separate cache files are needed for different adapter configurations.