# StringWars: Benchmarking High-Performance String Processing in Rust and Python

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/ashvardanian-stringwars
Generated for open source discovery and AI-assisted research.

StringWars is a comprehensive GitHub repository dedicated to benchmarking performance-oriented string processing libraries in Rust and Python. It meticulously compares various operations, including substring search, hashing, and edit distances, across both CPUs and GPUs. This project serves as an invaluable resource for developers seeking to identify the fastest and most efficient solutions for critical string manipulation tasks, particularly those leveraging modern SIMD instructions and GPU acceleration.

GitHub: https://github.com/ashvardanian/StringWars
OSRepos URL: https://osrepos.com/repo/ashvardanian-stringwars

## Summary

StringWars is a comprehensive GitHub repository dedicated to benchmarking performance-oriented string processing libraries in Rust and Python. It meticulously compares various operations, including substring search, hashing, and edit distances, across both CPUs and GPUs. This project serves as an invaluable resource for developers seeking to identify the fastest and most efficient solutions for critical string manipulation tasks, particularly those leveraging modern SIMD instructions and GPU acceleration.

## Topics

- benchmark
- string-processing
- Rust
- Python
- performance
- SIMD
- GPU
- bioinformatics

## Repository Information

Last analyzed by OSRepos: Tue Jul 21 2026 16:43:39 GMT+0100 (Western European Summer Time)
Detail views: 2
GitHub clicks: 1

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

StringWars is an ambitious GitHub repository that rigorously benchmarks performance-oriented string processing libraries written in Rust and Python. The project aims to provide a clear comparison of various string operations, such as substring search, multi-pattern matching, hashing, edit distances, sketching, and sorting, across different hardware architectures, including CPUs and GPUs.

Developed to accelerate the creation of the `StringZilla` C, C++, and CUDA libraries, StringWars highlights the simplicity of dependency management in Rust for benchmarking "Systems Software." It focuses on workloads where modern hardware with wide SIMD instructions (like AVX-512 on x86 or SVE/SVE2 on Arm) can offer significant performance gains, often surpassing existing library capabilities. The benchmarks include comparisons against popular libraries like `memchr`, `rapidfuzz`, `aHash`, `aho_corasick`, `polars`, and `icu`.

## Installation

To replicate the benchmark results, StringWars provides clear instructions for both Rust and Python environments.

### Replicating Results in Rust

First, clone the repository (a shallow clone is sufficient for benchmarks):

bash
git clone --depth 1 https://github.com/ashvardanian/StringWars && cd StringWars


Then, compile all dependencies with native CPU optimizations:

bash
RUSTFLAGS="-C target-cpu=native" cargo build --benches --all-features


To run a specific benchmark, for example, the hash benchmark:

bash
RUSTFLAGS="-C target-cpu=native" \
    STRINGWARS_DATASET=README.md \
    STRINGWARS_TOKENS=lines \
    cargo bench --features bench_hash --bench bench_hash --jobs $(nproc)


For GPU benchmarks, enable CUDA features explicitly:

bash
RUSTFLAGS="-C target-cpu=native" \
    STRINGWARS_DATASET=README.md \
    STRINGWARS_TOKENS=lines \
    STRINGWARS_FILTER=GPU \
    cargo bench --features "cuda bench_similarities" --bench bench_similarities --jobs 1


### Replicating Results in Python

It is recommended to use `uv` for Python dependency management. Install `uv` and then set up the environment:

bash
uv venv --python 3.12
uv pip install -r requirements.txt -r requirements-cuda.txt


To run individual Python benchmarks, for example, the hash benchmark:

bash
uv run --no-project python hash/bench.py --help


You can also run Python benchmarks directly from a URL without cloning:

bash
uv run https://raw.githubusercontent.com/ashvardanian/StringWars/main/tokenization/bench.py \
    --dataset README.md --tokens file


## Examples

StringWars provides extensive benchmarks across various string operations. Here are a few highlights demonstrating the performance differences:

### Hashing Performance (Rust, Short Words)

StringZilla consistently shows strong performance in hashing, often outperforming standard libraries.


Rust:
stringzilla::hash   ???????????????????? 1.84 GB/s
aHash::hash_one     ??????????????       1.23 GB/s
std::hash           ?????                0.43 GB/s


### Case-Insensitive UTF-8 Search (Rust, English)

For Unicode-aware, case-insensitive search, StringZilla demonstrates significant speed advantages.


Rust:
                      English
stringzilla           ???????????????????? 12.79 GB/s
icu                   ?                     0.08 GB/s


### Levenshtein Distance (Rust, 1 Core)

In computationally intensive tasks like edit distance, StringZilla, especially with GPU acceleration, achieves remarkable throughput.


Rust:
                        1 Core
rapidfuzz               ???????????????????? 14,316 MCUPS
stringzilla<384x GNR>   ???????????????????  13,084 MCUPS
bio::levenshtein        ??                      823 MCUPS


## Why Use StringWars?

*   **Performance Insight**: Gain deep insights into the performance characteristics of various string processing libraries in Rust and Python.
*   **Informed Decision-Making**: Make data-driven choices when selecting libraries for high-performance applications in areas like bioinformatics, databases, and natural language processing.
*   **Hardware Optimization**: Understand how different libraries leverage modern CPU SIMD instructions and GPU acceleration, helping you optimize for specific hardware.
*   **Contribution to StringZilla**: The project directly supports the development and optimization of the `StringZilla` library, offering a transparent comparison framework.
*   **Reproducible Benchmarks**: Follow clear instructions to reproduce benchmark results on your own hardware, ensuring reliable comparisons.

## Links

*   **GitHub Repository**: [https://github.com/ashvardanian/StringWars](https://github.com/ashvardanian/StringWars){:target="_blank"}