Our take
- Activity
- 37
- Community
- 12
- Issues
- 21
- Pull requests
- 20
+1 stars in the last 30 days
- Fewer contributors than 86% of the projects we track
Project history
Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.
Stars
409Forks
42Change since Nov 16, 2026: 0 stars·0 forks
Overview
llm-consortium extends the llm command-line ecosystem with a way to ask several language models the same question, compare their responses, and have an arbiter synthesize or rank them. It targets problems where comparing candidate answers may improve the result over relying on a single model.
The workflow can repeat with additional refinement rounds, stopping when its confidence threshold is met or its iteration limit is reached. It is useful when you can access multiple models and want a configurable process for combining their output, rather than a guarantee that consensus is correct.
Key Features
- Sends prompts to multiple model instances in parallel, with per-model instance counts.
- Uses a designated arbiter to synthesize responses or rank them using configurable judging methods.
- Supports iterative refinement, confidence thresholds, and iteration limits.
- Offers semantic filtering that clusters response embeddings before arbitration.
- Logs interactions and response metadata in SQLite, with optional embedding visualization.
- Saves reusable consortium configurations and supports continuing conversations through the
llmCLI. - Provides a Python
create_consortiumhelper for programmatic orchestration.
Use Cases
- Researchers can compare model-generated explanations or candidate solutions when a task benefits from multiple perspectives.
- Developers can create a reusable command-line workflow that combines models and an arbiter for recurring questions.
- Teams can experiment with ranking, synthesis, or semantic filtering strategies to examine how different arbitration approaches affect responses.
- Python developers can embed multi-model orchestration in scripts that already use the
llmecosystem.
Getting Started
Install the plugin with the llm CLI:
llm install llm-consortium
Then save a consortium configuration and invoke it as an llm model. See the README for configuration examples, optional features, and setup details.
Considerations
This is an orchestration plugin, not a language model: you need compatible model providers and access to the models you configure. Embedding-based features have optional dependencies and may require provider credentials. Agreement or geometric confidence reflects similarity between responses, not factual correctness, so outputs still need appropriate verification.
Found this useful?
Share it with someone who would like llm-consortium.