Scrapling

Scrapling is a Python web scraping framework for developers collecting data from individual pages or crawling whole sites. Its adaptive parser can relocate selected elements when a website's layout changes.

• Adaptive extraction: The parser tracks page changes to find elements again after an update, reducing repairs to extraction rules.
• Anti-bot handling: Fetchers include support for bypassing systems such as Cloudflare Turnstile.
• Concurrent crawling: The spider framework supports multiple sessions, automatic proxy rotation and pause/resume, with streaming results and live crawl statistics.
• Adaptive crawl speed: Crawls adjust to each website's response speed and back off when the site starts blocking requests.

Scrapling is open source under the BSD-3-Clause license. The project also includes a command-line interface and MCP support.