faiss: Search and Cluster Dense Vectors Efficiently

Overview

Faiss is a vector search and clustering library for finding vectors closest to a query using L2 distance or dot product. It offers index types that trade search speed and quality against memory use, training time, and the cost of adding vectors, from exact search to compressed and graph-based approaches.

It is aimed at developers and researchers who need to search collections ranging from in-memory datasets to sets that may not fit in RAM. C++ is the main implementation language, with Python/NumPy wrappers and optional GPU support.

Key Features

• Supports similarity search using L2 distance and dot products, including cosine similarity on normalized vectors.
• Provides exact and approximate index types with different memory, speed, and search-quality trade-offs.
• Includes compressed representations that can reduce memory requirements without retaining original vectors.
• Supports graph-based indexing structures such as HNSW and NSG.
• Provides GPU indexes, including single- and multi-GPU use, with CPU and GPU input support.
• Includes clustering, evaluation, and parameter-tuning support.
• Offers C++ and Python/NumPy interfaces.

Use Cases

• Build nearest-neighbor retrieval for recommendation, search, or deduplication systems when records can be represented as dense vectors.
• Search very large vector collections when keeping every original vector in memory is impractical and compressed indexes are acceptable.
• Run approximate vector retrieval on GPU-equipped servers when CPU search does not meet application needs.
• Cluster vector datasets or compare index configurations as part of a machine-learning research workflow.

Project Facts

• Language: C++
• License: MIT
• Stars: 41k
• Forks: 4.5k
• Topics: none listed
• Archived: no

Getting Started

Install a precompiled package with conda install -c pytorch faiss-cpu. GPU packages and build instructions are linked from the README. See the getting-started tutorial for index and search examples.

Alternatives

• lance: Lance combines vector search with columnar storage and dataset versioning, while Faiss focuses on vector indexing and nearest-neighbor search.

Considerations

• Choosing an index requires balancing speed, accuracy, memory, and indexing costs; approximate or compressed indexes may return less precise results than exact search.
• GPU support is optional and requires an appropriate CUDA or AMD ROCm setup. The project also offers CPU-only packages.
• Building from source uses CMake and requires a BLAS implementation. The Python interface is optional.
• Faiss expects vector data and integer identifiers, so applications must handle vector creation and any associated metadata themselves.

Repository: facebookresearch/faiss · Documentation