MechanicalSoup

MechanicalSoup: Automate Website Interaction with Python

MechanicalSoup lets Python programs interact with websites through HTTP sessions and parsed HTML, including following links and submitting forms. It suits scripted web tasks that do not require JavaScript execution.

Stars
4.9k
Forks
401
Stars, last 17 days
+1
Last commit
5 days ago
Contributors
51

Our take

Activeregular commits

Health score

47/100

How it is scored
Activity
36
Community
75
Issues
33
Pull requests
42

+1 stars in the last 17 days

  • Slower to close issues than 98% of the projects we track

MechanicalSoup is an actively maintained, focused option for form-based web automation, though its lack of JavaScript support and slow issue closure may matter for production use.

Good fit if

  • Your task works through ordinary HTML pages, links, and forms without JavaScript.
  • You want a Python library built around HTTP sessions and HTML parsing rather than a full browser.
  • You value recent development activity: the project is active, has regular commits and merged changes, and contributions are not concentrated in one person.

Look elsewhere if

  • The target site requires JavaScript execution or browser-rendered content.
  • You need rapid issue resolution, since issues have taken a long time to close despite quick pull request merges.
  • You require frequent releases, as there has been no release in over a year.
All health signals
Last commit2026-10-06 (5 days ago)
Commits, last 90 days15
Releases, last 12 months0 (latest v1.4.0, 2025-05-30)
Contributors51 (top contributor: 34% of commits)
Issues closed, last 90 days5 (typically closed in 958 days)
Pull requests merged, last 90 days15 (typically merged in 2 days)
Project age12 years

Checked on 2026-10-12 with the GitHub API.

Overview

MechanicalSoup is a Python library for automating website interactions without a full browser. It combines HTTP session handling with HTML navigation so scripts can open pages, retain cookies, follow links, and submit forms.

It is aimed at developers who need a lightweight way to automate sites with conventional HTML forms and pages. It does not execute JavaScript, so it is not a fit for workflows that depend on client-side rendering or browser-only behavior.

Key Features

  • Maintains cookies through HTTP sessions and follows redirects.
  • Navigates HTML documents using BeautifulSoup.
  • Finds and follows links.
  • Selects forms, fills in fields, and submits them.
  • Provides a stateful browser interface for sequences of website interactions.
  • Supports Python and PyPy3, according to the project documentation.

Use Cases

  • Automate a search form and extract result links when the site serves usable HTML.
  • Submit routine forms, such as those with text fields, checkboxes, radio buttons, or textareas.
  • Write Python scripts that revisit pages while keeping session cookies.
  • Build lightweight data collection tasks for sites that do not require JavaScript to expose their content.

What you need

Detected in the repository

  • Python (from setup.py)
  • A test suite and automated checks on GitHub Actions

License in plain words

MITpermissive

  • Commercial use: yes
  • Modify and redistribute: yes
  • You must keep: the copyright and license notice
  • Share your changes: no

A summary, not legal advice: the LICENSE file is what applies.

Getting Started

Install from PyPI:

pip install MechanicalSoup

See the README and documentation for usage and examples.

Alternatives

  • splinter: Splinter automates real browsers through drivers such as Selenium, while MechanicalSoup interacts with pages using HTTP sessions and parsed HTML.
  • CloakBrowser: CloakBrowser automates a customized Chromium browser through Playwright or Puppeteer, while MechanicalSoup does not execute JavaScript.
ProjectLanguageLicenseStarsStatus
MechanicalSoupPythonMIT4.9kActive
splinterPythonBSD-3-Clause2.8kInactive
CloakBrowserPythonMIT31.9kActive

Considerations

The project explicitly does not run JavaScript, which rules it out for many modern interactive sites. Development remains active, but release cadence is slower than commit activity, and the measured issue-closure time is lengthy. The library is distributed under the MIT license; check that its terms meet your project’s requirements.

Found this useful?

Share it with someone who would like MechanicalSoup.

Comparisons