Skip to content
Advertisement
Backblaze B2 Genblaze Hackathon with $10K in prizes

crawlee-python Open source

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs,

Collection & Scraping
Visit website View repo

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Open-source data tool (Apache-2.0 license, 9,348★). Source: GitHub (apify/crawlee-python).

Text

Similar tools

zdh_web

Collection & Scraping
Open source

大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批流,私域营销等模块

Multimodal
Open Source

cloudquery

Collection & Scraping
Open source

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability manageme

Multimodal
Open Source

grab

Collection & Scraping
Open source

Web Scraping Framework

Multimodal
Open Source

brightdata-mcp

Collection & Scraping
Open source

A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.

Text
Open Source

From the blog

Advertisement
Backblaze B2 Genblaze Hackathon with $10K in prizes