# Grab: A Powerful Python Web Scraping Framework

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/lorien-grab
Generated for open source discovery and AI-assisted research.

Grab is a robust Python web scraping framework designed to simplify complex data extraction tasks. It provides comprehensive tools for handling network requests, processing scraped content, and managing asynchronous operations through its powerful Spider component. Developers can leverage features like automatic cookie support, HTTP/SOCKS proxies, and XPath queries for efficient web data collection.

GitHub: https://github.com/lorien/grab
OSRepos URL: https://osrepos.com/repo/lorien-grab

## Summary

Grab is a robust Python web scraping framework designed to simplify complex data extraction tasks. It provides comprehensive tools for handling network requests, processing scraped content, and managing asynchronous operations through its powerful Spider component. Developers can leverage features like automatic cookie support, HTTP/SOCKS proxies, and XPath queries for efficient web data collection.

## Topics

- Python
- Web Scraping
- Framework
- Crawler
- Asynchronous
- HTTP Client
- Data Extraction
- Spider

## Repository Information

Last analyzed by OSRepos: Fri Jul 24 2026 17:34:26 GMT+0100 (Western European Summer Time)
Detail views: 1
GitHub clicks: 1

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction
Grab is a powerful Python web scraping framework designed to streamline the process of making network requests, scraping websites, and processing the extracted content. It offers a rich set of features including automatic cookie management, support for HTTP and SOCKS proxies, and robust tools for interacting with web forms. For advanced asynchronous scraping, Grab includes its Spider component, enabling efficient parallel processing of thousands of queries.

Recently, the Grab project underwent a significant update to ensure compatibility across a wide range of Python versions, from 2.7 up to 3.13. This update, culminating in version 1.0, focuses on stability and modern Python support, making Grab a reliable choice for current development. Please note a backward-incompatible change: `DataNotFound` and `InvalidResponseError` exceptions are now located in `grab.errors` instead of `weblib.error`.

## Installation
Installing Grab is straightforward. You can get started by running the following command:

bash
pip install -U grab


For more detailed instructions and platform-specific guidance, please refer to the official documentation: [https://grab.readthedocs.io/en/stable/usage/installation.html](https://grab.readthedocs.io/en/stable/usage/installation.html)

## Examples

### Grab Example
Here’s a basic example demonstrating how to use Grab to log into a GitHub account and list repositories:

python
import logging

from grab import Grab

logging.basicConfig(level=logging.DEBUG)

g = Grab()

g.go('https://github.com/login')
g.doc.set_input('login', '****')
g.doc.set_input('password', '****')
g.doc.submit()

g.doc.save('/tmp/x.html')

g.doc('//ul[@id="user-links"]//button[contains(@class, "signout")]').assert_exists()

home_url = g.doc('//a[contains(@class, "header-nav-link name")]/@href').text()
repo_url = home_url + '?tab=repositories'

g.go(repo_url)

for elem in g.doc.select('//h3[@class="repo-list-name"]/a'):
    print('%s: %s' % (elem.text(),
                      g.make_url_absolute(elem.attr('href'))))



### Grab::Spider Example
For asynchronous scraping, Grab's Spider component allows you to define tasks and process them in parallel:

python
import logging

from grab.spider import Spider, Task

logging.basicConfig(level=logging.DEBUG)


class ExampleSpider(Spider):
    def task_generator(self):
        for lang in 'python', 'ruby', 'perl':
            url = 'https://www.google.com/search?q=%s' % lang
            yield Task('search', url=url, lang=lang)

    def task_search(self, grab, task):
        print('%s: %s' % (task.lang,
                          grab.doc('//div[@class="s"]//cite').text()))


bot = ExampleSpider(thread_number=2)
bot.run()


## Why Use Grab?
Grab stands out as a versatile web scraping framework due to its comprehensive feature set and robust architecture. It provides:

*   **Full-featured HTTP Client**: Handles cookies, proxies (HTTP/SOCKS), Keep-Alive, and IDN support automatically.
*   **Powerful Data Extraction**: Utilizes XPath queries for precise and efficient data extraction from HTML documents.
*   **Asynchronous Capabilities**: The integrated Spider framework allows for high-performance, parallel scraping, automatically managing network errors and task queues.
*   **Web Form Interaction**: Simplifies working with web forms, including easy multipart file uploads.
*   **Active Maintenance**: Recently updated to support modern Python versions (2.7 to 3.13), ensuring ongoing compatibility and stability.
*   **Community Support**: Access to Telegram chat groups for discussions and assistance.

## Links
*   **GitHub Repository**: [https://github.com/lorien/grab](https://github.com/lorien/grab)
*   **Official Documentation**: [https://grab.readthedocs.io/en/stable/](https://grab.readthedocs.io/en/stable/)
*   **Telegram Chat (English)**: [https://t.me/grablab](https://t.me/grablab)
*   **Telegram Chat (Russian)**: [https://t.me/grablab_ru](https://t.me/grablab_ru)