# Trafilatura: Advanced Web Scraping and Text Extraction in Python
This repository profile is provided by osrepos.com, an open source repository discovery platform.
Source: osrepos.com
Repository profile: https://osrepos.com/repo/adbar-trafilatura
Generated for open source discovery and AI-assisted research.
Trafilatura is a robust Python package and command-line tool designed for gathering text and metadata from the web. It simplifies web crawling, scraping, and content extraction, transforming raw HTML into structured data. Widely adopted by major companies and institutions, it offers high efficiency and accuracy for various text processing needs.
GitHub: https://github.com/adbar/trafilatura
OSRepos URL: https://osrepos.com/repo/adbar-trafilatura
## Summary
Trafilatura is a robust Python package and command-line tool designed for gathering text and metadata from the web. It simplifies web crawling, scraping, and content extraction, transforming raw HTML into structured data. Widely adopted by major companies and institutions, it offers high efficiency and accuracy for various text processing needs.
## Topics
- Python
- Web Scraping
- Text Extraction
- NLP
- Crawler
- Data Mining
- Article Extractor
- HTML to Text
## Repository Information
Last analyzed by OSRepos: Fri May 01 2026 17:33:30 GMT+0100 (Western European Summer Time)
Detail views: 1
GitHub clicks: 6
## Safety Notice
OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.
## Content
## Introduction
Trafilatura is a cutting-edge **Python package and command-line tool** designed to **gather text and metadata from the Web**. It simplifies the process of turning raw HTML into structured, meaningful data, offering essential components for **web crawling, downloads, scraping, and extraction** of main texts, metadata, and comments. Recognized for its robustness and speed, Trafilatura is widely used by companies like HuggingFace, IBM, and Microsoft Research, as well as by renowned academic institutions.
## Installation
Installing Trafilatura is straightforward and can be done using pip:
bash
pip install trafilatura
## Examples
Here's how you can use Trafilatura to extract text from a URL or an HTML string:
python
import trafilatura
from trafilatura.downloads import fetch_url
# Example 1: Extract from a URL
url = "https://adrien.barbaresi.eu/blog/trafilatura-web-scraping.html"
print(f"Extracting from: {url}")
downloaded = fetch_url(url)
if downloaded:
text = trafilatura.extract(downloaded)
print("--- Extracted Content (first 500 characters) ---")
print(text[:500])
print("----------------------------------------------------")
# Example 2: Extract from an HTML string
html_content = """
My Test Page
Main Title
This is an example paragraph with bold text.
"""
print("\nExtracting from an HTML string:")
text_from_html = trafilatura.extract(html_content)
print("--- Extracted Content ---")
print(text_from_html)
print("-------------------------")
## Why Use Trafilatura?
Trafilatura stands out for several reasons, making it an excellent choice for your web content extraction needs:
* **Efficiency and Accuracy**: It consistently outperforms other open-source libraries in text extraction benchmarks, balancing noise limitation with the inclusion of all valid parts.
* **Comprehensive Features**: It offers advanced web crawling (sitemaps and feeds support), parallel processing, and robust extraction of main text, metadata (title, author, date), and formatting (paragraphs, titles, lists).
* **Multiple Output Formats**: It supports TXT, Markdown, CSV, JSON, HTML, XML, and TEI, providing flexibility for various applications.
* **Modularity and Ease of Use**: No database is required, and it's designed to be handy and modular, facilitating integration into your projects.
* **Active Maintenance and Community Support**: It receives regular updates, feature additions, and optimizations, backed by comprehensive documentation and an active community.
* **Focus on Content Quality**: It helps focus on the actual content, avoiding noise from recurring elements like headers and footers, and making sense of data and metadata.
## Links
* **GitHub Repository**: [https://github.com/adbar/trafilatura](https://github.com/adbar/trafilatura){:target="_blank"}
* **Official Documentation**: [https://trafilatura.readthedocs.io/](https://trafilatura.readthedocs.io/){:target="_blank"}
* **PyPI Package**: [https://pypi.org/project/trafilatura/](https://pypi.org/project/trafilatura/){:target="_blank"}