{"name":"Grab: A Powerful Python Web Scraping Framework","description":"Grab is a robust Python web scraping framework designed to simplify complex data extraction tasks. It provides comprehensive tools for handling network requests, processing scraped content, and managing asynchronous operations through its powerful Spider component. Developers can leverage features like automatic cookie support, HTTP/SOCKS proxies, and XPath queries for efficient web data collection.","github":"https://github.com/lorien/grab","url":"https://osrepos.com/repo/lorien-grab","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/lorien-grab","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/lorien-grab.md","json":"https://osrepos.com/repo/lorien-grab.json","topics":["Python","Web Scraping","Framework","Crawler","Asynchronous","HTTP Client","Data Extraction","Spider"],"keywords":["Python","Web Scraping","Framework","Crawler","Asynchronous","HTTP Client","Data Extraction","Spider"],"stars":null,"summary":"Grab is a robust Python web scraping framework designed to simplify complex data extraction tasks. It provides comprehensive tools for handling network requests, processing scraped content, and managing asynchronous operations through its powerful Spider component. Developers can leverage features like automatic cookie support, HTTP/SOCKS proxies, and XPath queries for efficient web data collection.","content":"## Introduction\nGrab is a powerful Python web scraping framework designed to streamline the process of making network requests, scraping websites, and processing the extracted content. It offers a rich set of features including automatic cookie management, support for HTTP and SOCKS proxies, and robust tools for interacting with web forms. For advanced asynchronous scraping, Grab includes its Spider component, enabling efficient parallel processing of thousands of queries.\n\nRecently, the Grab project underwent a significant update to ensure compatibility across a wide range of Python versions, from 2.7 up to 3.13. This update, culminating in version 1.0, focuses on stability and modern Python support, making Grab a reliable choice for current development. Please note a backward-incompatible change: `DataNotFound` and `InvalidResponseError` exceptions are now located in `grab.errors` instead of `weblib.error`.\n\n## Installation\nInstalling Grab is straightforward. You can get started by running the following command:\n\nbash\npip install -U grab\n\n\nFor more detailed instructions and platform-specific guidance, please refer to the official documentation: [https://grab.readthedocs.io/en/stable/usage/installation.html](https://grab.readthedocs.io/en/stable/usage/installation.html)\n\n## Examples\n\n### Grab Example\nHere’s a basic example demonstrating how to use Grab to log into a GitHub account and list repositories:\n\npython\nimport logging\n\nfrom grab import Grab\n\nlogging.basicConfig(level=logging.DEBUG)\n\ng = Grab()\n\ng.go('https://github.com/login')\ng.doc.set_input('login', '****')\ng.doc.set_input('password', '****')\ng.doc.submit()\n\ng.doc.save('/tmp/x.html')\n\ng.doc('//ul[@id=\"user-links\"]//button[contains(@class, \"signout\")]').assert_exists()\n\nhome_url = g.doc('//a[contains(@class, \"header-nav-link name\")]/@href').text()\nrepo_url = home_url + '?tab=repositories'\n\ng.go(repo_url)\n\nfor elem in g.doc.select('//h3[@class=\"repo-list-name\"]/a'):\n    print('%s: %s' % (elem.text(),\n                      g.make_url_absolute(elem.attr('href'))))\n\n\n\n### Grab::Spider Example\nFor asynchronous scraping, Grab's Spider component allows you to define tasks and process them in parallel:\n\npython\nimport logging\n\nfrom grab.spider import Spider, Task\n\nlogging.basicConfig(level=logging.DEBUG)\n\n\nclass ExampleSpider(Spider):\n    def task_generator(self):\n        for lang in 'python', 'ruby', 'perl':\n            url = 'https://www.google.com/search?q=%s' % lang\n            yield Task('search', url=url, lang=lang)\n\n    def task_search(self, grab, task):\n        print('%s: %s' % (task.lang,\n                          grab.doc('//div[@class=\"s\"]//cite').text()))\n\n\nbot = ExampleSpider(thread_number=2)\nbot.run()\n\n\n## Why Use Grab?\nGrab stands out as a versatile web scraping framework due to its comprehensive feature set and robust architecture. It provides:\n\n*   **Full-featured HTTP Client**: Handles cookies, proxies (HTTP/SOCKS), Keep-Alive, and IDN support automatically.\n*   **Powerful Data Extraction**: Utilizes XPath queries for precise and efficient data extraction from HTML documents.\n*   **Asynchronous Capabilities**: The integrated Spider framework allows for high-performance, parallel scraping, automatically managing network errors and task queues.\n*   **Web Form Interaction**: Simplifies working with web forms, including easy multipart file uploads.\n*   **Active Maintenance**: Recently updated to support modern Python versions (2.7 to 3.13), ensuring ongoing compatibility and stability.\n*   **Community Support**: Access to Telegram chat groups for discussions and assistance.\n\n## Links\n*   **GitHub Repository**: [https://github.com/lorien/grab](https://github.com/lorien/grab)\n*   **Official Documentation**: [https://grab.readthedocs.io/en/stable/](https://grab.readthedocs.io/en/stable/)\n*   **Telegram Chat (English)**: [https://t.me/grablab](https://t.me/grablab)\n*   **Telegram Chat (Russian)**: [https://t.me/grablab_ru](https://t.me/grablab_ru)","metrics":{"detailViews":1,"githubClicks":1},"dates":{"published":null,"modified":"2026-07-24T16:34:26.000Z"}}