{"name":"agent.cpp: Build Local AI Agents in C++","description":"agent.cpp is a C++ library for building agents that run small language models locally through llama.cpp. It provides an agent loop, tools, callbacks, and support for GGUF models, grammar constraints, and LoRA adapters.","github":"https://github.com/mozilla-ai/agent.cpp","url":"https://osrepos.com/repo/mozilla-ai-agent.cpp","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/mozilla-ai-agent.cpp","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/mozilla-ai-agent.cpp.md","json":"https://osrepos.com/repo/mozilla-ai-agent.cpp.json","topics":["c++","ai-agents","llm","library","local-first","agent-framework"],"keywords":["c++","ai-agents","llm","library","local-first","agent-framework"],"stars":null,"summary":"agent.cpp is a C++ library for building agents that run small language models locally through llama.cpp. It provides an agent loop, tools, callbacks, and support for GGUF models, grammar constraints, and LoRA adapters.","content":"## Overview\n\nagent.cpp is a C++ library for building local agents around llama.cpp. It addresses the need for agent-building components in C++ applications without routing model inference to an external LLM API.\n\nThe library combines a model with an agent loop, instructions, tools, and lifecycle callbacks. It is aimed at C++ developers who want to run GGUF models locally and customize how agents handle context, tool execution, and stopping conditions.\n\n## Key Features\n\n- Runs local GGUF models using llama.cpp, including configurable text generation and KV cache management.\n- Provides an agent loop that alternates between model calls and tool execution.\n- Defines tools with descriptions, JSON Schema parameters, and execution functions.\n- Offers callbacks around agent loops, model calls, and tool execution for context changes, validation, logging, or approval flows.\n- Supports GBNF grammar constraints for structured model output.\n- Supports LoRA adapters, including separate adapters for models that share base weights.\n- Includes examples for memory, multi-agent delegation, shell interaction, tracing, grammar, and context engineering.\n\n## Use Cases\n\n- C++ application developers can add local model-backed tool use without relying on external LLM APIs.\n- Teams can build specialized agents that share model weights while using separate LoRA adapters.\n- Developers can constrain text-only agent responses to a defined format with a GBNF grammar.\n- Applications needing human review or custom context handling can use callbacks to intervene in the agent lifecycle.\n- Developers exploring multi-agent systems in C++ can use the delegation example as a starting point.\n\n## Project Facts\n\n- Language: C++\n- License: MIT\n- Stars: 210\n- Forks: 31\n- Archived: No\n\n## Getting Started\n\nRequires C++17 or higher. Add the project to a CMake build with FetchContent:\n\n```cmake\ninclude(FetchContent)\nFetchContent_Declare(\n    agent-cpp\n    GIT_REPOSITORY https://github.com/mozilla-ai/agent.cpp\n    GIT_TAG main\n)\nFetchContent_MakeAvailable(agent-cpp)\ntarget_link_libraries(my_app PRIVATE agent-cpp::agent)\n```\n\nSee the [repository README](https://github.com/mozilla-ai/agent.cpp) for installed-package and submodule options, model setup, examples, and build configuration.\n\n## Alternatives\n\n- [rig](https://osrepos.com/repo/0xplaygrounds-rig): Rig is a Rust library with shared integrations for multiple model providers and vector stores, rather than a C++ library centered on local llama.cpp models.\n- [deepagents](https://osrepos.com/repo/hwchase17-deepagents): Deep Agents is a Python harness with planning, delegation, and file access built in, rather than a C++ library focused on a local-model agent loop.\n- [harness-sdk](https://osrepos.com/repo/strands-agents-sdk-python): Strands offers Python and TypeScript agent tools with model flexibility, rather than a C++ library focused on running small models through llama.cpp.\n\n## Considerations\n\n- This is designed for local inference through llama.cpp, not for calling external LLM APIs.\n- You need a GGUF model to run the examples. The README's default configuration is optimized for granite-4.0-micro; other models may need different context and sampling settings.\n- llama.cpp is included as a submodule, and hardware acceleration requires the appropriate build options and supported backend.\n- Grammar constraints apply to tool-call responses too. If the grammar excludes the tool-call format, the agent cannot call tools.\n- Prompt cache files do not record active LoRA adapters, so separate cache files are needed for different adapter configurations.","metrics":{"detailViews":7,"githubClicks":15},"dates":{"published":null,"modified":"2026-10-03T23:28:15.000Z"}}