# agent.cpp: Build Local AI Agents in C++

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/mozilla-ai-agent.cpp
Generated for open source discovery and AI-assisted research.

agent.cpp is a C++ library for building agents that run small language models locally through llama.cpp. It provides an agent loop, tools, callbacks, and support for GGUF models, grammar constraints, and LoRA adapters.

GitHub: https://github.com/mozilla-ai/agent.cpp
OSRepos URL: https://osrepos.com/repo/mozilla-ai-agent.cpp

## Summary

agent.cpp is a C++ library for building agents that run small language models locally through llama.cpp. It provides an agent loop, tools, callbacks, and support for GGUF models, grammar constraints, and LoRA adapters.

## Topics

- c++
- ai-agents
- llm
- library
- local-first
- agent-framework

## Repository Information

Last analyzed by OSRepos: Sun Oct 04 2026 00:28:15 GMT+0100 (Western European Summer Time)
Detail views: 7
GitHub clicks: 15

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Overview

agent.cpp is a C++ library for building local agents around llama.cpp. It addresses the need for agent-building components in C++ applications without routing model inference to an external LLM API.

The library combines a model with an agent loop, instructions, tools, and lifecycle callbacks. It is aimed at C++ developers who want to run GGUF models locally and customize how agents handle context, tool execution, and stopping conditions.

## Key Features

- Runs local GGUF models using llama.cpp, including configurable text generation and KV cache management.
- Provides an agent loop that alternates between model calls and tool execution.
- Defines tools with descriptions, JSON Schema parameters, and execution functions.
- Offers callbacks around agent loops, model calls, and tool execution for context changes, validation, logging, or approval flows.
- Supports GBNF grammar constraints for structured model output.
- Supports LoRA adapters, including separate adapters for models that share base weights.
- Includes examples for memory, multi-agent delegation, shell interaction, tracing, grammar, and context engineering.

## Use Cases

- C++ application developers can add local model-backed tool use without relying on external LLM APIs.
- Teams can build specialized agents that share model weights while using separate LoRA adapters.
- Developers can constrain text-only agent responses to a defined format with a GBNF grammar.
- Applications needing human review or custom context handling can use callbacks to intervene in the agent lifecycle.
- Developers exploring multi-agent systems in C++ can use the delegation example as a starting point.

## Project Facts

- Language: C++
- License: MIT
- Stars: 210
- Forks: 31
- Archived: No

## Getting Started

Requires C++17 or higher. Add the project to a CMake build with FetchContent:

```cmake
include(FetchContent)
FetchContent_Declare(
    agent-cpp
    GIT_REPOSITORY https://github.com/mozilla-ai/agent.cpp
    GIT_TAG main
)
FetchContent_MakeAvailable(agent-cpp)
target_link_libraries(my_app PRIVATE agent-cpp::agent)
```

See the [repository README](https://github.com/mozilla-ai/agent.cpp) for installed-package and submodule options, model setup, examples, and build configuration.

## Alternatives

- [rig](https://osrepos.com/repo/0xplaygrounds-rig): Rig is a Rust library with shared integrations for multiple model providers and vector stores, rather than a C++ library centered on local llama.cpp models.
- [deepagents](https://osrepos.com/repo/hwchase17-deepagents): Deep Agents is a Python harness with planning, delegation, and file access built in, rather than a C++ library focused on a local-model agent loop.
- [harness-sdk](https://osrepos.com/repo/strands-agents-sdk-python): Strands offers Python and TypeScript agent tools with model flexibility, rather than a C++ library focused on running small models through llama.cpp.

## Considerations

- This is designed for local inference through llama.cpp, not for calling external LLM APIs.
- You need a GGUF model to run the examples. The README's default configuration is optimized for granite-4.0-micro; other models may need different context and sampling settings.
- llama.cpp is included as a submodule, and hardware acceleration requires the appropriate build options and supported backend.
- Grammar constraints apply to tool-call responses too. If the grammar excludes the tool-call format, the agent cannot call tools.
- Prompt cache files do not record active LoRA adapters, so separate cache files are needed for different adapter configurations.