Repository History
3 repositories tagged with Local Inference
agent.cpp: Building Local LLM Agents with C++ and llama.cpp
agent.cpp provides essential building blocks for developing local AI agents using C++. It leverages llama.cpp to enable efficient execution of small language models directly on your hardware. This library offers a modular approach with features like agent loops, callbacks, tools, and grammar-constrained output, making it ideal for creating custom, privacy-focused agent solutions.
oobabooga/text-generation-webui: The Premier Local LLM Interface
oobabooga/text-generation-webui is a powerful and versatile web UI for running large language models (LLMs) locally. It offers a 100% offline and private environment for text generation, vision, tool-calling, and even training, all accessible through an intuitive interface and API.
llama-cpp-python: Python Bindings for llama.cpp
llama-cpp-python provides robust Python bindings for the popular llama.cpp library, enabling efficient local inference with large language models. It offers a high-level API compatible with OpenAI's API, facilitating easy integration into existing applications. The project also includes a powerful web server for local deployment and supports various hardware acceleration backends.