AgentAleph: Local-First AI Coding Agent and GGUF Model Manager

Introduction

AgentAleph is a powerful local-first AI coding agent and GGUF model manager, built with Rust and powered by llama.cpp. This desktop application offers a complete solution for developers who want to leverage AI without relying on cloud services, API keys, or external accounts. It integrates both a robust local model manager, inspired by tools like LM Studio, and a sophisticated coding agent, taking cues from Codex and Claude Code.

Why Use and Key Features

AgentAleph stands out for its commitment to privacy, local execution, and comprehensive functionality.

• Local Inference: All prompts are processed by your local model, ensuring privacy. While web searches and fetches use the network, approved shell commands can also access it, always with user control.
• An Agent, Not Just a Chat Box: Beyond simple chat, AgentAleph can read, search, edit, and create files, as well as run shell commands, all with your explicit approval.
• Simple by Design: The application features a streamlined, single-window interface for downloading, loading models, and working, eliminating the need for hosted endpoints or complex server setups.
• Safe by Default: It incorporates per-tool allow, ask, or deny permissions and a read-only plan mode that prevents project edits and shell commands, giving you full control.
• Built for Imperfect Local Models: The agent includes features like GBNF grammar-constrained tool calls, native tool calling, loop detection, and context compaction to optimize performance with various local models.
• Extensible with Skills: You can easily add task-specific local instruction packs, known as "Skills," without modifying the core code.
• Comprehensive Model Management: Browse a curated catalog of GGUF models from Hugging Face, download them with resumable transfers, and manage them locally. The app provides hardware fit badges to estimate performance on your system.
• Flexible Inference Settings: Fine-tune model behavior with configurable settings for system prompt, temperature, context size, GPU layers, and more.
• Edit Previews and Undo: Review proposed file changes before approval and undo the latest file edits, providing a safety net for your work.

Installation

AgentAleph is primarily designed for Linux x64 (Ubuntu/Debian with webkit2gtk-4.1, gtk-3). It requires Rust 1.77+ and Node 18+.

To get started, follow these steps:

git clone https://github.com/matiasA/AgentAleph.git
cd AgentAleph
npm install
./scripts/setup-llama.sh   # downloads the llama.cpp binary; it is not committed because of its size
npm run tauri dev          # development with hot reload
# or
npm run tauri build        # production build (.deb / .AppImage)

For CPU-only operation, you can specify the LLAMA_FLAVOR:

LLAMA_FLAVOR=x64 ./scripts/setup-llama.sh

The project also supports building installers via GitHub Actions. You can trigger a build manually or by pushing a version tag:

git tag v0.1.0 && git push origin v0.1.0   # triggers build + Release

Examples

Getting started with AgentAleph is straightforward:

1. Download a model from the Catalog tab, typically a Q4_K_M variant by default.
2. Load it from the Models tab and await the progress indicator.
3. Choose a mode: Select "Chat" for general conversation or "Agent" for project-specific work.
4. Work with the agent: If in Agent mode, select a project folder, describe your task, and then approve or deny any write or execute actions as prompted by the agent.

Links

• GitHub Repository: AgentAleph GitHub Repository
• License: MIT License