unstructured: Turn Documents Into Structured Data

Overview

Unstructured provides tools for ingesting and preprocessing unstructured documents, turning files such as PDFs, HTML, Word documents, images, and emails into structured elements. Its automatic partition function detects a file type and routes it to the corresponding parser.

It is aimed at developers building document-processing and LLM data workflows. Use it when you need a library-based way to extract and organize content across different document formats; consider a managed or production platform if you need hosted workflow features beyond the open-source library.

Key Features

• Automatic file-type detection and parser selection through partition.
• Format-specific partitioning functions for documents such as PDFs and text files.
• Support for a range of document inputs, including PDFs, HTML, Word documents, images, and emails.
• Optional dependency extras let users install support for selected document types.
• Docker images and local development tooling are available.
• Connectors and integrations support broader ingestion workflows.

Use Cases

• LLM application developers can prepare documents as structured input for retrieval or other downstream processing.
• Data engineers can build ingestion pipelines that handle several document formats in one workflow.
• Teams digitizing scanned PDFs or images can use document and OCR-related processing, with required system dependencies installed.
• Developers working with office documents and email can extract content for search, analysis, or application-specific processing.

Getting Started

Install the base package with pip:

pip install unstructured

For example, parse a file with automatic type detection:

from unstructured.partition.auto import partition

elements = partition("example-docs/layout-parser-paper.pdf")

Some formats require optional package extras and system dependencies. See the README and installation documentation for details.

Alternatives

• marker: Marker focuses on converting PDFs and similar documents into Markdown, JSON, HTML, or chunks, while unstructured handles a broader range of file types.
• opendataloader-pdf: OpenDataLoader PDF specializes in PDF parsing and tagged-PDF remediation, rather than unstructured's multi-format document processing.
• text-extract-api: text-extract-api provides an API for multi-format extraction with OCR, LLM-based structuring, and PII removal, while unstructured is a Python library.
• PDF Craft: PDF Craft focuses on converting PDFs, especially scanned books, into formats such as Markdown and EPUB, while unstructured supports many document types.

Considerations

• The base install does not include every document-format dependency. Processing PDFs, images, or office files may require extras and system packages such as Tesseract, Poppler, or LibreOffice.
• The repository describes an open-source preprocessing library. Its README presents Unstructured Transform and Unstructured Pipelines separately as production-oriented offerings, so check their documentation for hosted or advanced workflow needs.
• The README says lightweight analytics are enabled by default. Set DO_NOT_TRACK or SCARF_NO_ANALYTICS to a non-empty value to disable them.