{"name":"ggml: A Low-Level Tensor Library for Machine Learning","description":"ggml is an innovative tensor library designed for machine learning, emphasizing low-level, cross-platform implementation. It offers features like integer quantization, automatic differentiation, and broad hardware support, all while maintaining zero third-party dependencies and efficient memory usage. This project is actively developed and forms the backbone for other popular projects like llama.cpp and whisper.cpp.","github":"https://github.com/ggml-org/ggml","url":"https://osrepos.com/repo/ggml-org-ggml","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/ggml-org-ggml","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/ggml-org-ggml.md","json":"https://osrepos.com/repo/ggml-org-ggml.json","topics":["machine-learning","tensor-library","large-language-models","automatic-differentiation","C++","AI","deep-learning","quantization"],"keywords":["machine-learning","tensor-library","large-language-models","automatic-differentiation","C++","AI","deep-learning","quantization"],"stars":null,"summary":"ggml is an innovative tensor library designed for machine learning, emphasizing low-level, cross-platform implementation. It offers features like integer quantization, automatic differentiation, and broad hardware support, all while maintaining zero third-party dependencies and efficient memory usage. This project is actively developed and forms the backbone for other popular projects like llama.cpp and whisper.cpp.","content":"## Introduction\n\nggml is a powerful tensor library specifically engineered for machine learning applications. It stands out for its low-level, cross-platform implementation, making it highly versatile and efficient. The library is under active development, with significant contributions also happening within related projects such as [llama.cpp](https://github.com/ggerganov/llama.cpp \"llama.cpp\" target=\"_blank\") and [whisper.cpp](https://github.com/ggerganov/whisper.cpp \"whisper.cpp\" target=\"_blank\").\n\nKey features of ggml include integer quantization support, broad hardware compatibility, and built-in automatic differentiation. It also incorporates ADAM and L-BFGS optimizers, all without relying on any third-party dependencies, ensuring a lean and performant codebase. A notable design principle is its commitment to zero memory allocations during runtime, which contributes to its exceptional efficiency.\n\n## Installation\n\nTo get started with ggml, clone the repository and follow the build steps:\n\nbash\ngit clone https://github.com/ggml-org/ggml\ncd ggml\n\n# install python dependencies in a virtual environment\npython3.10 -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\n\n# build the examples\nmkdir build && cd build\ncmake ..\ncmake --build . --config Release -j 8\n\n\n## Examples\n\nHere's an example of how to run GPT-2 inference using ggml:\n\nbash\n# run the GPT-2 small 117M model\n../examples/gpt-2/download-ggml-model.sh 117M\n./bin/gpt-2-backend -m models/gpt-2-117M/ggml-model.bin -p \"This is an example\"\n\n\nFor more detailed examples and usage scenarios, explore the `examples` folder within the repository.\n\n## Why Use ggml?\n\nggml offers several compelling advantages for machine learning developers:\n\n*   **Low-level and Cross-platform:** Provides fine-grained control and runs efficiently across various operating systems and architectures.\n*   **Integer Quantization Support:** Enables efficient model deployment on resource-constrained devices by reducing model size and computational requirements.\n*   **Broad Hardware Support:** Designed to work across a wide range of hardware, including specialized accelerators.\n*   **Automatic Differentiation:** Simplifies the implementation of complex machine learning models and training algorithms.\n*   **Optimizers Included:** Comes with ADAM and L-BFGS optimizers built-in, ready for use in training processes.\n*   **No Third-Party Dependencies:** Reduces complexity, potential conflicts, and improves portability.\n*   **Zero Memory Allocations During Runtime:** Ensures highly efficient memory usage, crucial for performance-critical applications.\n\n## Links\n\n*   **GitHub Repository:** [https://github.com/ggml-org/ggml](https://github.com/ggml-org/ggml \"ggml GitHub Repository\" target=\"_blank\")\n*   **Roadmap:** [https://github.com/users/ggerganov/projects/7](https://github.com/users/ggerganov/projects/7 \"ggml Roadmap\" target=\"_blank\")\n*   **Manifesto:** [https://github.com/ggerganov/llama.cpp/discussions/205](https://github.com/ggerganov/llama.cpp/discussions/205 \"ggml Manifesto\" target=\"_blank\")\n*   **Introduction to ggml (Hugging Face Blog):** [https://huggingface.co/blog/introduction-to-ggml](https://huggingface.co/blog/introduction-to-ggml \"Introduction to ggml\" target=\"_blank\")\n*   **The GGUF file format:** [https://github.com/ggerganov/ggml/blob/master/docs/gguf.md](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md \"GGUF file format documentation\" target=\"_blank\")","metrics":{"detailViews":1,"githubClicks":4},"dates":{"published":null,"modified":"2025-12-16T16:01:48.000Z"}}