Our take
Activeregular commits and releases- Activity
- 80
- Community
- 37
- Issues
- 66
- Pull requests
- 43
+14,891 stars in the last 30 days
- Releases more often than 77% of the projects we track
- Slower to merge pull requests than 83% of the projects we track
Scrapling is a feature-rich, actively maintained Python scraping framework, but its very high contributor concentration makes the project reliant on one maintainer despite regular releases and issue handling.
Good fit if
- You need one Python library for parsing, fetching, and crawling, and can work with Python 3.10 or newer.
- Adaptive selectors, browser fetching, or resumable concurrent crawls address concrete problems in your scraping workload.
- You want MCP or RAG-oriented features alongside conventional scraping tools.
Look elsewhere if
- Your project needs a non-Python runtime.
- You need a minimal parser-only dependency but expect browser or spider functionality without installing its optional dependencies.
- You cannot accept a project whose contributor activity is concentrated heavily in one person.
All health signals
| Last commit | 2026-10-07 (today) |
|---|---|
| Commits, last 90 days | 100+ |
| Releases, last 12 months | 24 (latest v0.4.15, 2026-08-23) |
| Contributors | 31 (top contributor: 96% of commits) |
| Issues closed, last 90 days | 33+ (typically closed in 6 days) |
| Pull requests merged, last 90 days | 35 (typically merged in 3 days) |
| Project age | 24 months |
Checked on 2026-10-08 with the GitHub API.
Project history
Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.
Stars
102,537Forks
10,679Change since Nov 13, 2026: +319 stars·+40 forks
Overview
Scrapling combines HTML parsing, website fetching, and crawling in one Python framework. Its adaptive selectors can help locate elements after a site's structure changes, while its fetchers cover ordinary HTTP requests and browser-driven pages that need JavaScript or stealth features.
It is intended for developers building scrapers, from one-off extraction scripts to concurrent crawls. The project also provides a command-line interface and an MCP server for AI tools, so it can serve both Python applications and agent-based workflows.
Key Features
- Parse pages with CSS, XPath, text, and filter-based selectors, and save elements for adaptive lookup after page changes.
- Fetch pages through HTTP clients or browser-backed fetchers for dynamic content.
- Manage sessions, cookies, and rotating proxies across requests.
- Build concurrent spiders with callbacks, streaming results, and checkpoint-based pause and resume.
- Configure crawl throttling, blocked-request retries, and optional robots.txt compliance.
- Export crawl results as JSON, JSONL, CSV, or XML, or use the CLI to extract page content.
- Connect AI clients through the MCP server, or convert pages into Markdown for RAG workflows.
Use Cases
- A data engineer maintaining product or listing extraction as target pages change can use adaptive selectors to reduce selector breakage.
- A Python developer collecting data across many URLs can use spiders for concurrency, throttling, and resumable crawls rather than assembling those features separately.
- An automation team scraping JavaScript-rendered pages can use the browser fetchers when plain HTTP requests do not expose the needed content.
- An AI application developer can expose page fetching and extraction to an MCP-compatible agent, or prepare sanitized Markdown for a retrieval pipeline.
What you need
Detected in the repository
- Python >=3.10 (from pyproject.toml)
- A Dockerfile, so it can run in a container
- A test suite and automated checks on GitHub Actions
License in plain words
BSD-3-Clausepermissive
- Commercial use: yes
- Modify and redistribute: yes
- You must keep: the copyright and license notice; the authors' names cannot endorse your product
- Share your changes: no
A summary, not legal advice: the LICENSE file is what applies.
Getting Started
Install the parser package with:
pip install scrapling
For fetchers and spiders, install scrapling[fetchers] and run scrapling install to install browser dependencies. See the README and documentation for setup options and examples.
Alternatives
- grab: Grab focuses on HTTP workflows, task queues, and concurrent crawlers rather than adaptive selectors and browser-based fetching.
- requests-html: requests-html pairs Requests-style access with HTML selection and optional Chromium rendering, without Scrapling’s adaptive selectors or crawler tooling.
| Project | Language | License | Stars | Status |
|---|---|---|---|---|
| Scrapling | Python | BSD-3-Clause | 86.2k | Active |
| grab | Python | MIT | 2.5k | Inactive |
| requests-html | Python | MIT | 13.8k | Inactive |
Considerations
The project is actively maintained, with frequent commits and releases and quick reported issue and pull request handling. However, most contributor activity comes from one person, which is a continuity risk for teams planning long-term dependency on the framework.
The basic package does not include fetcher or command-line dependencies. Browser-based fetching and spider workflows require optional dependencies, and browser features also require installing browser components. Scrapling's anti-bot and stealth capabilities are technical features, not permission to disregard site terms, robots.txt, or applicable privacy and scraping laws; the project explicitly places responsibility for compliant use on the user.
Found this useful?
Share it with someone who would like Scrapling.