# llm-consortium: Orchestrating Multiple LLMs for Consensus and Refinement

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/irthomasthomas-llm-consortium
Generated for open source discovery and AI-assisted research.

llm-consortium is a powerful plugin for the `llm` package, designed to enhance problem-solving by orchestrating multiple large language models. It implements a parallel reasoning method that iteratively refines responses and achieves consensus through structured dialogue, evaluation, and arbitration. This system leverages the collective intelligence of diverse LLMs to tackle complex problems more effectively.

GitHub: https://github.com/irthomasthomas/llm-consortium
OSRepos URL: https://osrepos.com/repo/irthomasthomas-llm-consortium

## Summary

llm-consortium is a powerful plugin for the `llm` package, designed to enhance problem-solving by orchestrating multiple large language models. It implements a parallel reasoning method that iteratively refines responses and achieves consensus through structured dialogue, evaluation, and arbitration. This system leverages the collective intelligence of diverse LLMs to tackle complex problems more effectively.

## Topics

- ai
- llms
- Python
- artificial intelligence
- large language models
- llm orchestration
- natural language processing
- open source

## Repository Information

Last analyzed by OSRepos: Mon Dec 01 2025 00:01:26 GMT+0000 (Western European Standard Time)
Detail views: 4
GitHub clicks: 6

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

`llm-consortium` is an innovative plugin for the `llm` package that introduces a model consortium system for advanced problem-solving. Inspired by Andrej Karpathy's observation that optimal performance often comes from asking all available models and having them reach a consensus, this tool orchestrates multiple diverse language models to collaboratively solve complex problems.

The core idea revolves around parallel reasoning, where various LLMs contribute to a solution. The system then iteratively refines these responses, synthesizes them, and uses a designated arbiter model to evaluate and arbitrate until a high-confidence consensus is achieved.

## Installation

First, ensure you have the `llm` package installed. You can install it using `uv` or `pipx`:

Using `uv`:
bash
uv tool install llm


Using `pipx`:
bash
pipx install llm


Once `llm` is installed, you can install the `llm-consortium` plugin:

bash
llm install llm-consortium


## Examples

### Command Line Usage

The `consortium` command defaults to the `run` subcommand for concise usage.

**Basic Usage:**
bash
llm consortium "What are the key considerations for AGI safety?"


Or, if you have saved a consortium model (e.g., named `my-consortium`):
bash
llm -m my-consortium "What are the key considerations for AGI safety?"


This command will send your prompt to multiple models in parallel, gather responses, use an arbiter model to synthesize them, and iterate to refine the answer until a specified confidence threshold or maximum iteration count is reached.

**Conversation Continuation Usage (New in v0.3.2):**

To continue the most recent conversation:
bash
# Initial prompt
llm -m my-consortium "Tell me about the planet Mars."
# Follow-up
llm -c "How long does it take to get there?"


To continue a specific conversation using its ID:
bash
# Initial prompt (note the conversation ID, e.g., 01jscjy50ty4ycsypbq6h4ywhh)
llm -m my-consortium "Tell me about Jupiter."

# Follow-up using the ID
llm -c --cid 01jscjy50ty4ycsypbq6h4ywhh "What are its major moons?"


**Advanced Example:**
You can specify individual instance counts for models, a custom arbiter, and adjust thresholds:
bash
llm consortium "Your complex query" \
  -m o3-mini:1 \
  -m gpt-4o:2 \
  -m gemini-2:3 \
  --arbiter gemini-2 \
  --confidence-threshold 1 \
  --max-iterations 4 \
  --min-iterations 3 \
  --output results.json


### Managing Consortium Configurations

You can save a consortium configuration as a named model for reuse:
bash
llm consortium save my-consortium \
    --model claude-3-opus-20240229 \
    --model gpt-4 \
    --arbiter claude-3-opus-20240229 \
    --confidence-threshold 0.9 \
    --max-iterations 5 \
    --min-iterations 1 \
    --system "Your custom system prompt"


Once saved, invoke your custom consortium:
bash
llm -m my-consortium "What are the key considerations for AGI safety?"


### Programmatic Usage

Integrate `llm-consortium` into your Python code using the `create_consortium` helper:

python
from llm_consortium import create_consortium

orchestrator = create_consortium(
    models=["o3-mini:1", "gpt-4o:2", "gemini-2:3"],
    confidence_threshold=1,
    max_iterations=4,
    min_iterations=3,
    arbiter="gemini-2",
    raw=True
)

result = orchestrator.orchestrate("Your prompt here")
print(f"Synthesized Response: {result['synthesis']['synthesis']}")


## Why Use llm-consortium?

`llm-consortium` offers several compelling features for advanced LLM applications:

*   **Multi-Model Orchestration**: Coordinate responses from multiple models in parallel, leveraging diverse perspectives.
*   **Iterative Refinement**: Automatically refine output until a desired confidence threshold is achieved, improving answer quality.
*   **Advanced Arbitration**: Utilizes a designated arbiter model to synthesize and critically evaluate responses, ensuring robust consensus.
*   **Database Logging**: All interactions are logged to SQLite, providing a clear audit trail and debugging capabilities.
*   **Configurable Parameters**: Adjustable confidence thresholds, iteration limits, and model selection offer fine-grained control.
*   **Flexible Model Instance Counts**: Specify individual instance counts per model, allowing for optimized resource allocation.
*   **Conversation Continuation**: Seamlessly continue previous conversations, enhancing user experience for multi-turn interactions.

## Links

*   **GitHub Repository**: [https://github.com/irthomasthomas/llm-consortium](https://github.com/irthomasthomas/llm-consortium){:target="_blank"}
*   **`llm` package**: [https://github.com/simonw/llm](https://github.com/simonw/llm){:target="_blank"}