Repository History
3 repositories tagged with crawler
toapi: Declaratively Turn Any Website into a JSON API
toapi is a powerful Python library designed to transform any website into a clean JSON API declaratively. It enables users to define desired data fields using CSS selectors, fetching and parsing web pages on demand. With built-in caching and support for dynamic content, toapi simplifies web data extraction without complex crawlers or databases.
Douyin_TikTok_Download_API: High-Performance Scraper for Social Media Videos
Douyin_TikTok_Download_API is a high-performance, asynchronous tool for crawling data from Douyin, TikTok, and Bilibili. It supports API calls, online batch parsing, and no-watermark video downloads. This Python-based project offers a comprehensive solution for social media data extraction and management.
Newspaper3k: Advanced News and Article Extraction in Python
Newspaper3k is a powerful Python 3 library designed for news, full-text, and article metadata extraction. Inspired by the simplicity of 'requests' and the speed of 'lxml', it provides robust tools for scraping and curating articles from various sources. This library is ideal for developers needing to programmatically gather and process news content with advanced NLP capabilities.