ggml: Build Portable Tensor Workloads for Machine Learning

Overview

ggml provides tensor operations and optimized kernels for machine-learning software. Its goal is to make those workloads portable and efficient without requiring external dependencies or runtime memory allocations.

This is a building block for developers integrating tensor computation into their own software, rather than a ready-to-run model application. Consider it when you need a C/C++ library and want to work close to the computation and deployment details.

Key Features

• Plain C/C++ implementation with no dependencies.
• Cross-platform support for x86, ARM, RISC-V, LoongArch, PowerPC, s390x, and WebAssembly.
• SIMD-optimized kernels for x86, ARM, and RISC-V.
• Backend support for CPU, GPU, NPU, and browser environments.
• Integer quantization from 2 to 8 bits, plus MXFP4 and NVFP4 microscaling formats.
• Zero runtime memory allocations, as described by the project.

Use Cases

• C/C++ developers building machine-learning software that needs tensor operations without an external dependency.
• Teams targeting multiple processor architectures that want a portable tensor library.
• Developers implementing workloads where lower-bit quantization formats are useful.
• Engineers integrating tensor computation into CPU, GPU, NPU, or browser-based software.

Getting Started

Build from source:

git clone https://github.com/ggml-org/ggml
cd ggml
mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8

See the README and the simple matrix multiplication example for details.

Alternatives

• jax: JAX offers Python APIs for automatic differentiation and compiler-driven accelerator computing, while ggml is a low-level C/C++ tensor library focused on portable deployment.

Considerations

• ggml is a low-level library, so using it means integrating tensor operations into your own application rather than launching a complete machine-learning product.
• Backend and architecture support varies by target. Check the project documentation and code for the requirements relevant to your intended platform.
• The project asks contributors to submit changes to the core library and CMake build system through llama.cpp, rather than directly here.