Open Source Natural Language Processing Projects
Natural language processing (NLP) enables computers to analyze, understand, and generate human language. It addresses tasks such as classifying text, extracting information, translating, searching documents, and building conversational systems. NLP techniques range from rule-based text processing to machine learning models that learn patterns from large collections of language data. They help make unstructured text easier to organize and use, while supporting applications across languages and domains.
Open source NLP tools include libraries for tokenization and text analysis, pretrained models, data-processing utilities, evaluation frameworks, and applications for search or text generation. When choosing a tool, consider its license, documentation, maintenance, language coverage, computing requirements, and compatibility with your existing workflows. These resources can help researchers, developers, students, and organizations build language applications, study NLP methods, or adapt tools to specialized needs.
13 repositories · updated October 3, 2026
python-pinyin: A Robust Python Library for Hanzi to Pinyin Conversion
`python-pinyin` is a powerful Python library designed for converting Chinese characters (Hanzi) into Pinyin. It offers advanced features such as intelligent phrase matching, comprehensive support for polyphonic characters, and various Pinyin and Bopomofo output styles. This tool is essential for developers working on Chinese text processing tasks, including annotation, sorting, and search functionalities.
mergoo: Combine Fine-Tuned LLM Experts into Routed Models
Mergoo combines fine-tuned language models or LoRA adapters into routed expert models, then supports training the resulting model. It is aimed at teams that want one model to draw on specialized experts rather than use them separately.
ChatArena: Build Multi-Agent Language Game Environments
ChatArena is a Python framework for running language games with multiple LLM agents. It suits researchers and developers studying agent interaction, collaboration, and social behavior, but the project was deprecated in August 2025 and is no longer supported.
spacy-llm: Add LLM Tasks to spaCy NLP Pipelines
spacy-llm connects large language models to spaCy pipelines, turning model responses into structured NLP outputs without training data. It suits teams prototyping NLP tasks or combining LLM components with conventional spaCy processing.
asta-paper-finder: A Frozen-in-Time Agent for Reproducing Paper Finder Evaluations
asta-paper-finder is a standalone, "frozen-in-time" version of the AllenAI Paper Finder agent. This repository provides the code specifically for reproducing evaluation results, allowing researchers to locate sets of papers based on content and metadata criteria. It offers a stable snapshot of the agent's core paper-finding capabilities.
jieba: Segment Chinese Text in Python
jieba is a Python library for segmenting Chinese text into words, including text that does not use spaces as word boundaries. It offers configurable segmentation modes, custom dictionaries, part-of-speech tagging, and keyword extraction.
AudioSep: Separate Sounds from Natural-Language Descriptions
AudioSep is a Python foundation model for separating sounds from audio based on natural-language descriptions. It supports open-domain tasks such as isolating events, instruments, or speech, with inference, fine-tuning, and evaluation workflows.
translation-agent: Improve Translations with an LLM Reflection Workflow
translation-agent is a Python demonstration that uses an LLM to translate text, review its own draft, and revise it. It suits experimentation with style, regional language, and terminology, but is not presented as mature production software.
QueryWeaver: Transform Natural Language into SQL with Graph-Powered Text2SQL
QueryWeaver is an open-source Text2SQL tool that allows users to query databases using plain English. It leverages graph-powered schema understanding to accurately convert natural language questions into SQL. This simplifies database interaction, making data accessible without deep SQL knowledge.
EasyEdit: Edit Knowledge and Steer Large Language Models
EasyEdit is a framework for changing specific knowledge or behavior in large language models and evaluating the effects. It brings together multiple editing and inference-time steering methods for researchers and developers comparing approaches or testing targeted edits.
sumy: Summarize Text and HTML Documents
sumy is a Python library and command-line tool that extracts concise summaries from plain text and HTML. It offers several extractive summarization methods, language tokenization support, and tools for evaluating summaries.
Apple Health MCP: Query Your Apple Health Data with Natural Language and SQL
The `apple-health-mcp` project is an MCP (Model Context Protocol) server designed for querying Apple Health data. It allows users to analyze their health metrics using natural language or direct SQL queries. This server integrates with clients like Claude Desktop, providing powerful tools for health data analysis.