Repository History
3 repositories tagged with benchmark
StringWars: Benchmarking High-Performance String Processing in Rust and Python
StringWars is a comprehensive GitHub repository dedicated to benchmarking performance-oriented string processing libraries in Rust and Python. It meticulously compares various operations, including substring search, hashing, and edit distances, across both CPUs and GPUs. This project serves as an invaluable resource for developers seeking to identify the fastest and most efficient solutions for critical string manipulation tasks, particularly those leveraging modern SIMD instructions and GPU acceleration.
PromptBench: A Unified Framework for LLM Evaluation and Robustness
PromptBench is a comprehensive Python library designed for the evaluation and understanding of Large Language Models (LLMs). It provides a unified framework for assessing model performance, exploring various prompt engineering techniques, and evaluating robustness against adversarial attacks. This tool empowers researchers to conduct in-depth analyses of LLMs across diverse datasets and models.
EvalPlus: Rigorous Evaluation for LLM-Synthesized Code
EvalPlus is a robust framework designed for the rigorous evaluation of code generated by Large Language Models (LLMs). It extends standard benchmarks like HumanEval and MBPP with significantly more tests, offering precise assessment of code correctness and efficiency. This tool is crucial for developers and researchers aiming to thoroughly validate LLM-synthesized code.