LLM Reasoners: Advanced Library for Large Language Model Reasoning

Introduction

LLM Reasoners is an open-source Python library from Maitrix.org, dedicated to empowering Large Language Models (LLMs) with advanced reasoning abilities. It provides a structured framework for implementing and evaluating complex reasoning algorithms, abstracting them into key components: a world model, a search algorithm, and a reward function. This design allows users to easily experiment with and apply state-of-the-art reasoning techniques to various problems.

The library stands out for its collection of cutting-edge reasoning algorithms, including Reasoner Agent, Inference-time Scaling with PRM, Reasoning-via-Planning (MCTS), and Tree-of-Thoughts (BFS/DFS), among others. It also features an intuitive visualization tool that helps users understand the intricate reasoning processes, even for complex algorithms like Monte-Carlo Tree Search, with minimal code. Furthermore, LLM Reasoners is optimized for efficiency, integrating high-performance LLM inference frameworks like SGLang to accelerate advanced reasoning techniques. The project emphasizes rigorous implementation and reproducibility, ensuring that its algorithms are both theoretically sound and practically usable, as highlighted in its COLM2024 paper.

Installation

Ensure you are using Python 3.10 or later.

conda create -n reasoners python=3.10
conda activate reasoners

Install from pip

pip install llm-reasoners

Install from GitHub
(Recommended if you want to run the examples in the GitHub repo)

git clone https://github.com/maitrix-org/llm-reasoners --recursive
cd llm-reasoners
pip install -e .

Adding --recursive will help you clone exllama and LLM-Planning automatically. Note that some other optional modules may require additional dependencies. Please refer to the error message for details.

Examples

LLM Reasoners offers a clear and modular approach to implementing reasoning algorithms. The library abstracts an LLM reasoning algorithm into three key components: a reward function, a world model, and a search algorithm. Users define a WorldModel to handle initial states, state transitions, and terminal conditions, and a SearchConfig to specify actions and reward calculations. These are then combined with a SearchAlgorithm like MCTS to solve problems.

The repository's 'Quick Tour' provides a runnable example demonstrating how to solve BlocksWorld problems. This example illustrates how to define custom WorldModel and SearchConfig classes, and then apply a SearchAlgorithm to find optimal reasoning chains. A powerful visualization tool is also integrated, allowing users to easily diagnose and understand the reasoning process with just one line of Python code. You can explore a runnable notebook here.

Why Use LLM Reasoners?

LLM Reasoners provides several compelling reasons for its adoption in projects requiring advanced LLM reasoning:

Cutting-Edge Reasoning Algorithms: Access to a wide array of the most up-to-date search algorithms for reasoning with LLMs, including Reasoner Agent, Inference-time Scaling with PRM, Reasoning-via-Planning (MCTS), and Tree-of-Thoughts (BFS/DFS).
Intuitive Visualization and Interpretation: The library includes a visualization tool that simplifies the understanding of complex reasoning processes, enabling users to diagnose and interpret algorithm behavior effortlessly.
Efficient Reasoning with LLMs: Optimizes performance by integrating high-performance LLM inference frameworks like SGLang, offering significant speed-ups and supporting various LLM backends.
Rigorous Implementation and Reproducibility: Prioritizes precision and reliability, ensuring that implemented algorithms are faithful to their original formulations and performance, as demonstrated by successful reproductions of key research papers.
Active Development: The project is continuously evolving, with recent updates including the integration of Deepseek R1, the introduction of ReasonerAgent (a fully open-source, web-browsing research agent), and enhanced planning algorithms for web environments.