Open Source Text Processing Tools
Discover 25 open source Text Processing repositories from GitHub, each with an analysis of what it does, key features, use cases and alternatives. Text Processing projects here are most often combined with Python, Library and CLI. Last updated October 21, 2026.
25 repositories · updated October 21, 2026
StringWars: Compare String Processing Performance on CPUs and GPUs
StringWars benchmarks string-processing libraries across Rust and Python, from search and hashing to Unicode, sorting, and edit distance. It helps systems developers compare implementations on representative datasets and hardware before choosing or optimizing a library.
ActiveHealth 28/100
pg_textsearch: Add BM25 Ranked Search to PostgreSQL
pg_textsearch is a PostgreSQL extension that adds BM25 relevance ranking and indexed full-text search. It suits teams that want ranked text queries inside PostgreSQL, using its text search configurations and SQL workflows.
ActiveHealth 58/100
humanizer: Remove AI-Writing Tells from Text
Humanizer is an agent skill that edits AI-written text to sound more natural without changing its meaning. Built on Wikipedia’s “Signs of AI writing” guide, it works with Claude Code, Codex, and other agents that support skills.
ActiveHealth 55/100
trafilatura: Extract Text and Metadata from Web Pages
Trafilatura is a Python library and CLI for discovering web content and extracting readable text and metadata from HTML. It suits developers and researchers building crawlers, web corpora, or content-processing pipelines that need structured output.
ActiveHealth 52/100
scooter: Find and Replace Text Interactively in the Terminal
scooter is a Rust terminal app for reviewing and applying find-and-replace changes across files or piped text. It suits developers who want to select individual matches before editing, with regex, previews, and editor integration.
ActiveHealth 43/100
datatrove: Build Large-Scale Text Processing Pipelines
DataTrove is a Python library for processing, filtering, extracting, and deduplicating large text datasets. Its reusable pipeline blocks run locally or on distributed execution environments, making it useful for teams preparing data for language models.
ActiveHealth 59/100
markitdown: Convert Documents and Files to Markdown
MarkItDown converts documents and other files into Markdown for LLM and text-analysis workflows. It offers a Python library and command-line interface, with optional format-specific dependencies and plugins.
ActiveHealth 76/100
Toolkit-for-Prompt-Compression: Evaluate and Apply Prompt Compression
PCToolkit is a Python toolkit for applying and evaluating prompt-compression methods for large language models. It brings five compressors, datasets, and evaluation metrics behind modular interfaces, making it useful for comparing methods across language tasks.
InactiveHealth 20/100
espanso: Expand Text Snippets Across Your Desktop
espanso is a privacy-first text expander for Windows, macOS, and Linux. It replaces typed triggers with snippets, scripts, commands, and other content, making it useful for repetitive writing and system-wide shortcuts.
ActiveHealth 51/100
feedparser: Parse RSS, Atom, and JSON Feeds in Python
feedparser is a Python library for reading RSS, Atom, and JSON feeds. It suits applications and scripts that need to retrieve and work with feed content without implementing feed parsing themselves.
ActiveHealth 55/100
textdistance: Compare Sequences with 30+ Distance Algorithms
TextDistance is a Python library for comparing two or more sequences with a common interface to 30+ distance and similarity algorithms. It provides pure-Python implementations, with optional external libraries and NumPy for speed.
InactiveHealth 24/100
python-ftfy: Repair Mojibake and Unicode Text
ftfy is a Python library that repairs mojibake and other common Unicode glitches in text. It suits developers cleaning text from mixed or unreliable sources, with conservative fixes designed to avoid changing text that is already valid.
InactiveHealth 30/100