mlc-ai

xgrammar: Constrain Language Model Output to Structured Formats

XGrammar is a library for constrained decoding that helps language models produce outputs matching JSON, regular expressions, or context-free grammars. It suits teams integrating structured output into LLM inference and applications that need reliable machine-readable responses.

View on GitHub Compare
Stars
1.9k
Forks
223
Stars, last 30 days
+58

Our take

Health score

66/100

How it is scored
Activity
79
Community
75
Issues
54
Pull requests
57

+58 stars in the last 30 days

  • Merges more pull requests than 79% of the projects we track
  • Releases more often than 75% of the projects we track

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

2,024

Forks

278

Change since Nov 18, 2026: +2 stars·+2 forks

Overview

XGrammar is a structured-generation library for constraining language model decoding to a grammar. It addresses a common integration problem: generated text may be plausible but fail to follow the format an application expects. With a grammar constraint, applications can require outputs such as JSON, regular expressions, or other context-free grammar formats.

It is aimed at developers building LLM applications or inference systems that need structured responses. Consider it when output validity matters and you want a reusable decoding component rather than relying on prompts alone. The repository describes APIs for Python, C++, JavaScript, and Swift, and points to integrations with several inference engines.

Key Features

  • Constrained decoding for structurally valid output.
  • Supports JSON, regular expressions, and custom context-free grammars.
  • Provides APIs for Python, C++, JavaScript, and Swift.
  • Targets Linux, macOS, and Windows, with CPU and multiple accelerator platforms listed in the project overview.
  • Offers a Python package installable from PyPI.
  • Designed for integration with LLM inference engines, including vLLM, SGLang, TensorRT-LLM, and MLC-LLM.

Use Cases

  • LLM application developers can constrain responses to JSON for downstream parsing and tool workflows.
  • Inference engine maintainers can add grammar-based output constraints to their serving stack.
  • Teams using custom schemas or domain-specific formats can define grammars beyond standard JSON.
  • Developers building cross-platform LLM applications can evaluate a shared structured-generation library across supported APIs and platforms.

Getting Started

Install the Python package:

pip install xgrammar

See the README and documentation for setup and usage details.

Alternatives

  • jsonformer: Jsonformer constrains Hugging Face models to a supported JSON Schema subset, while XGrammar also supports regex and context-free grammar constraints.
ProjectLanguageLicenseStarsStatus
xgrammarC++Apache-2.01.9kNot checked yet
jsonformerJupyter NotebookMIT4.9kInactive

Considerations

  • The README describes support across several platforms, hardware types, and language APIs, but compatibility details may vary by setup. Check the documentation for the target environment.
  • Structured output depends on supplying an appropriate grammar. XGrammar constrains decoding; it does not replace application-level validation or define the application's schema for you.
  • The repository has 103 open issues, so review current issue reports and release notes when assessing fit.
  • The project publishes releases as needed rather than on a fixed schedule.

Found this useful?

Share it with someone who would like xgrammar.

Comparisons