langtest vs AuditNLG
Language-model testing and generated-text auditing compared
langtest generates and runs tests to assess NLP and language models across quality and safety concerns. AuditNLG focuses on auditing supplied or generated text for factualness, unsafe content, and instruction compliance, with explanations and rewrite suggestions.

langtest: Test Language Models for Safety and Quality
LangTest is a Python library for evaluating language models and NLP systems for robustness, bias, fairness, accuracy, and other quality risks. It supports NLP frameworks including Spark NLP and Hugging Face, plus several hosted LLM services.

AuditNLG: Check and Improve Trust in Generated Text
AuditNLG evaluates generated text for factuality, safety, and instruction-following, then can suggest revisions and explain flagged results. It suits teams and researchers comparing evaluation methods, but checks may depend on external APIs or models.
| langtest | AuditNLG | |
|---|---|---|
| Language | Python | Python |
| License | Apache-2.0 | BSD-3-Clause |
| Stars | 561 | 103 |
| Forks | 53 | 12 |
| Last analyzed | Oct 29, 2026 | Oct 13, 2026 |
Key differences
- langtest covers model-level tests across robustness, bias, representation, fairness, and accuracy; AuditNLG checks text for factualness, safety, and instruction compliance.
- langtest supports NLP tasks such as named entity recognition, translation, and classification, as well as LLM tasks; AuditNLG evaluates text and can use grounding knowledge for factualness checks.
- langtest can augment training data for select models; AuditNLG provides explanations and candidate rewrites and does not train or provide model weights.
- langtest integrates with Spark NLP, Hugging Face and Transformers, and supported LLM services; AuditNLG offers a Python package and a command-line workflow with JSON input, using model and service-based checks.
- langtest uses the Apache-2.0 license, while AuditNLG uses BSD-3-Clause; AuditNLG also notes that the Salesforce safety model is for non-commercial use.
- The supplied repository figures list 561 stars and 54 forks for langtest, compared with 103 stars and 12 forks for AuditNLG.
Choose langtest if you…
- need generated tests across NLP tasks and LLM concerns such as bias, robustness, and fairness.
- want to evaluate models during development and use results to guide data augmentation for supported models.
- use Spark NLP, Hugging Face and Transformers, or supported LLM services in your workflow.
Choose AuditNLG if you…
- need to assess generated text for factualness, safety, and instruction compliance.
- want explanations and candidate rewrites to investigate problematic outputs.
- need to provide reference knowledge for grounded factualness checks or run evaluations from a command line with JSON input.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.