html2text: Convert HTML to Clean, Readable Markdown Text in Python

Introduction

html2text is a Python script that converts a page of HTML into clean, easy-to-read plain ASCII text. Better yet, that ASCII also happens to be valid Markdown, a popular text-to-HTML format. Originally written by Aaron Swartz, this tool provides a robust solution for transforming complex HTML structures into simple, readable Markdown.

Installation

You can easily install html2text using pip, as it is available on PyPI.

$ pip install html2text

Examples

html2text can be used both as a command-line tool and within your Python applications.

Command-line Usage

Here are some common options for command-line usage:

• --ignore-links: Don't include any formatting for links.
• --escape-all: Escape all special characters. Output is less readable, but avoids corner case formatting issues.
• --reference-links: Use reference links instead of inline links to create markdown.
• --mark-code: Mark preformatted and code blocks with [code]...[/code].

For a complete list of options, refer to the official documentation.

Python Usage

You can integrate html2text directly into your Python code:

import html2text

print(html2text.html2text("<p><strong>Zed's</strong> dead baby, <em>Zed's</em> dead.</p>"))
# Expected output: **Zed's** dead baby, _Zed's_ dead.

You can also configure its behavior with specific options:

import html2text

h = html2text.HTML2Text()
# Ignore converting links from HTML
h.ignore_links = True
print(h.handle("<p>Hello, <a href='https://www.google.com/earth/'>world</a>!</p>"))
# Expected output: Hello, world!

# Don't ignore links anymore
h.ignore_links = False
print(h.handle("<p>Hello, <a href='https://www.google.com/earth/'>world</a>!</p>"))
# Expected output: Hello, [world](https://www.google.com/earth/)!

Why Use It?

Converting HTML to Markdown offers several advantages. Markdown is lightweight, human-readable, and easy to write, making it ideal for documentation, content management, and plain text storage. html2text automates this conversion, preserving formatting like bold, italics, and links while stripping away the complexities of HTML tags, resulting in clean and portable text. This is particularly useful for data scraping, content migration, or generating reports from web pages.

Links

• GitHub Repository: https://github.com/Alir3z4/html2text
• PyPI Project: https://pypi.org/project/html2text/
• Documentation: https://github.com/Alir3z4/html2text/blob/master/docs/usage.md