StringWars: Benchmarking High-Performance String Processing in Rust and Python

Introduction

StringWars is an ambitious GitHub repository that rigorously benchmarks performance-oriented string processing libraries written in Rust and Python. The project aims to provide a clear comparison of various string operations, such as substring search, multi-pattern matching, hashing, edit distances, sketching, and sorting, across different hardware architectures, including CPUs and GPUs.

Developed to accelerate the creation of the StringZilla C, C++, and CUDA libraries, StringWars highlights the simplicity of dependency management in Rust for benchmarking "Systems Software." It focuses on workloads where modern hardware with wide SIMD instructions (like AVX-512 on x86 or SVE/SVE2 on Arm) can offer significant performance gains, often surpassing existing library capabilities. The benchmarks include comparisons against popular libraries like memchr, rapidfuzz, aHash, aho_corasick, polars, and icu.

Installation

To replicate the benchmark results, StringWars provides clear instructions for both Rust and Python environments.

Replicating Results in Rust

First, clone the repository (a shallow clone is sufficient for benchmarks):

git clone --depth 1 https://github.com/ashvardanian/StringWars && cd StringWars

Then, compile all dependencies with native CPU optimizations:

RUSTFLAGS="-C target-cpu=native" cargo build --benches --all-features

To run a specific benchmark, for example, the hash benchmark:

RUSTFLAGS="-C target-cpu=native" \
    STRINGWARS_DATASET=README.md \
    STRINGWARS_TOKENS=lines \
    cargo bench --features bench_hash --bench bench_hash --jobs $(nproc)

For GPU benchmarks, enable CUDA features explicitly:

RUSTFLAGS="-C target-cpu=native" \
    STRINGWARS_DATASET=README.md \
    STRINGWARS_TOKENS=lines \
    STRINGWARS_FILTER=GPU \
    cargo bench --features "cuda bench_similarities" --bench bench_similarities --jobs 1

Replicating Results in Python

It is recommended to use uv for Python dependency management. Install uv and then set up the environment:

uv venv --python 3.12
uv pip install -r requirements.txt -r requirements-cuda.txt

To run individual Python benchmarks, for example, the hash benchmark:

uv run --no-project python hash/bench.py --help

You can also run Python benchmarks directly from a URL without cloning:

uv run https://raw.githubusercontent.com/ashvardanian/StringWars/main/tokenization/bench.py \
    --dataset README.md --tokens file

Examples

StringWars provides extensive benchmarks across various string operations. Here are a few highlights demonstrating the performance differences:

Hashing Performance (Rust, Short Words)

StringZilla consistently shows strong performance in hashing, often outperforming standard libraries.

Rust:
stringzilla::hash   ???????????????????? 1.84 GB/s
aHash::hash_one     ??????????????       1.23 GB/s
std::hash           ?????                0.43 GB/s

Case-Insensitive UTF-8 Search (Rust, English)

For Unicode-aware, case-insensitive search, StringZilla demonstrates significant speed advantages.

Rust:
                      English
stringzilla           ???????????????????? 12.79 GB/s
icu                   ?                     0.08 GB/s

Levenshtein Distance (Rust, 1 Core)

In computationally intensive tasks like edit distance, StringZilla, especially with GPU acceleration, achieves remarkable throughput.

Rust:
                        1 Core
rapidfuzz               ???????????????????? 14,316 MCUPS
stringzilla<384x GNR>   ???????????????????  13,084 MCUPS
bio::levenshtein        ??                      823 MCUPS

Why Use StringWars?

• Performance Insight: Gain deep insights into the performance characteristics of various string processing libraries in Rust and Python.
• Informed Decision-Making: Make data-driven choices when selecting libraries for high-performance applications in areas like bioinformatics, databases, and natural language processing.
• Hardware Optimization: Understand how different libraries leverage modern CPU SIMD instructions and GPU acceleration, helping you optimize for specific hardware.
• Contribution to StringZilla: The project directly supports the development and optimization of the StringZilla library, offering a transparent comparison framework.
• Reproducible Benchmarks: Follow clear instructions to reproduce benchmark results on your own hardware, ensuring reliable comparisons.

Links

• GitHub Repository: https://github.com/ashvardanian/StringWars