Search Results for "data scraper website"
Sort By:
Showing 572 open source projects for "data scraper website"
View related business solutions-
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
LLM Scraper
Extract structured data from webpages using LLM-powered scraping
LLM Scraper is a TypeScript library designed to extract structured data from webpages using large language models. Instead of relying on fragile HTML selectors or manual parsing rules, the tool interprets webpage content with language models and converts it into structured data according to a defined schema. Developers can specify the data structure using tools such as Zod or JSON Schema, enabling the model to extract relevant information directly into typed objects. ... -
2
Linkedin Scraper
A library that scrapes Linkedin for user data
Linkedin Scraper is a library that scrapes Linkedin for user data. Version 2.0.0 and before is called linkedin_user_scraper and can be installed via pip3 install --user linkedin_user_scraper. The reason is that LinkedIn has recently blocked people from viewing certain profiles without having previously signed in. So by setting scrape=False, it doesn't automatically scrape the profile, but Chrome will open the linkedin page anyways. -
3
CyberScraper 2077
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
CyberScraper 2077 is not just another web scraping tool – it's a glimpse into the future of data extraction. Born from the neon-lit streets of a cyberpunk world, this AI-powered scraper uses OpenAI, Gemini and LocalLLM Models to slice through the web's defenses, extracting the data you need with unparalleled precision and style. -
4
Colly
Elegant Scraper and Crawler Framework for Golang
Colly provides a clean interface to write any kind of crawler/scraper/spider. With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving. Clean API. Fast (>1k request/sec on a single core) Manages request delays and maximum concurrency per domain. Automatic cookie and session handling. Sync/async/parallel scraping. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
JobFunnel
Scrape job websites into a single spreadsheet with no duplicates.
...Run funnel with your settings YAML to populate your master CSV file with jobs from available providers. JobFunnel can be easily automated to run nightly with crontab. If you have a job website you'd like to write a scraper for, you are welcome to implement it, Review the Base Scraper for implementation details. JobFunnel supports scraping jobs from the same job website across locales & domains. If you are interested in adding support, you may only need to define session headers and domain strings, Review the Base Scraper for further implementation details. -
6
Spider
High-performance Rust web crawler and scraper for large-scale data
Spider is a high-performance web crawler and web scraping library written in Rust that enables developers to crawl and index websites efficiently. It focuses on speed, concurrency, and reliability by using asynchronous and multi-threaded processing to handle large volumes of web pages. It can rapidly crawl websites to collect links, retrieve page content, and extract structured information from HTML documents. Spider can operate concurrently across many pages, allowing it to gather large... -
7
ScrapeGraphAI
Python scraper based on AI
Extracting content from websites and local documents using LLM. ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). Just say which information you want to extract and the library will do it for you. -
8
Automa
A chrome extension for automating your browser by connecting blocks
Automa is a browser extension for browser automation. From auto-fill forms, doing a repetitive task, taking a screenshot, to scraping data of the website, it's up to you what you want to do with this extension. Automa has provided various kinds of blocks that will help you do automation, and all you need to do is connect them. Want your workflow to run every day or every time you visit a specific website? You can set the workflow trigger on the trigger block. Try a workflow from the marketplace. ... -
9
Crawl4AI
Open-source LLM Friendly Web Crawler & Scraper
Crawl4AI is a high-performance, AI‑ready web crawler tailored for LLM data ingestion and RAG pipelines. It supports adaptive crawling heuristics (stopping when enough info is gathered), structured markdown output, and high-speed parallel execution. Designed to operate at scale with optional Docker deployment and framework integrations. -
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
10
Matomo
Alternative to Google Analytics that gives you full control over data
Google Analytics alternative that protects your data and your customers' privacy. Take back control with Matomo – a powerful web analytics platform that gives you 100% data ownership. You could lose your customers’ trust and risk damaging your reputation if people learn their data is used for Google’s “own purposes”. By choosing the ethical alternative, Matomo, you won’t make privacy sacrifices or compromise your site. -
11
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website. ... -
12
Firecrawl
Turn entire websites into LLM-ready markdown or structured data
Crawl and convert any website into LLM-ready markdown or structured data. Built by Mendable.ai and the Firecrawl community. Includes powerful scraping, crawling, and data extraction capabilities. Firecrawl is an API service that takes a URL, crawls it, and converts it into clean markdown or structured data. We crawl all accessible subpages and give you clean data for each. -
13
Chart.js
Simple yet flexible JavaScript charting for designers & developers
Chart.js is a Javascript library that allows designers and developers to draw all kinds of charts using the HTML5 canvas element. Chart js offers a great array of simple, clean charts including animated and interactive versions. Chartjs is an easy way to include beautiful and engaging charts into your website for free. -
14
Ghostery
Ghostery Browser Extension for Firefox, Chrome, Opera and Edge
Ghostery helps you browse smarter by giving you control over ads and tracking technologies to speed up page loads, eliminate clutter, and protect your data. This is the unified code repository for the Ghostery browser extensions in Chrome, Firefox, Opera and Edge. Browse the web safer, faster & with less annoying ads. Equipped with award-winning AI anti-tracking technology to browse the websafe and quickly. Ghostery helps you stay informed about what companies are tracking you by listing the trackers on each website you visit. ... -
15
Simple.css
Simple.css is a classless CSS template to make a good website
A classless CSS framework that makes semantic HTML look good. By classless I mean that there are no CSS classes anywhere in the CSS or the HTML. So your website can look just like this using plain old vanilla HTML. When starting a new project, I wanted a CSS framework that would get me up and running quickly, and give me something I could hack on. I got sick of all these giant frameworks that include everything but the kitchen sink, 90% of which I’ll never use. For example, the minified CSS... -
16
Publii
Publii is a desktop-based CMS for Windows, Mac and Linux
Publii is a powerful blogging app perfect for anyone looking to create a privacy-focused website. Whether you're a beginner or a developer, it has all the tools you need to get started. Publii is a static site generator that makes it easy to create a personal blog, portfolio, or corporate website. With instant site switching and no databases or other credentials to remember, Publii is the perfect platform for anyone who wants a hassle-free way to build and manage an online presence. Websites... -
17
JupyterLite
Wasm powered Jupyter running in the browser
...Built using JupyterLab components and powered by WebAssembly technologies, it allows users to run Python and other language kernels directly in the browser through tools like Pyodide or Xeus. This architecture eliminates the need for installation or server infrastructure, making it highly accessible for education, demonstrations, and lightweight data science workflows. JupyterLite supports many core Jupyter features, including notebooks, code consoles, and interactive visualizations, while storing files locally using browser storage mechanisms such as IndexedDB. It is designed to be easily deployable as a static website, enabling developers to host fully functional notebook environments on platforms like GitHub Pages. -
18
LaTeX.CSS
LaTeX.css is a library that makes your website look like a LaTeX doc
This almost class-less CSS library turns your HTML document into a website that looks like a LATEX document. Write semantic HTML, and you are good to go. The source code can be found on GitHub. LaTeX.css is a minimal, almost class-less CSS library that makes any website look like a LaTeX document. Add any optional classes to elements with special styles (author subtitle, abstract, lemmas, theorems, etc.). The labels of theorems, definitions, lemmas and proofs can be changed to other... -
19
Jikan
Unofficial MyAnimeList PHP+REST API which provides functions
Jikan is a PHP API for MyAnimeList. It scrapes the website to satisfy the need for API functionality that MyAnimeList.net lacks. The raison d'être of Jikan is to assist developers easily get the data they need for their apps and projects without having to depend on the lackluster official API, unstable APIs, or sidetracking their projects to develop parsers. With nearly limitless quota on requests, you have the freedom to create without the need of writing your own parsers. ... -
20
AI-Crawler
Crawl a website starting from a URL, find relevant pages
AI Crawler is an experimental AI-powered web crawling and data extraction tool that uses natural language prompts to guide the discovery and retrieval of relevant information across websites. Unlike traditional web scrapers that rely on static selectors and manual scripting, it uses AI to dynamically identify and prioritize pages based on user intent, making it more flexible and resilient to changes in website structure. -
21
FinalRecon
All-in-one Python web reconnaissance tool for fast target analysis
FinalRecon is an all-in-one web reconnaissance tool written in Python that helps security professionals gather information about a target website quickly and efficiently. It combines multiple reconnaissance techniques into a single command-line utility so users do not need to run several separate tools to collect similar data. FinalRecon focuses on providing a fast overview of a web target while maintaining accuracy in the collected results. It includes modules for gathering server information, analyzing SSL certificates, performing WHOIS lookups, and crawling website resources. ... -
22
Dendrite
Tools to build web AI agents that can authenticate
Dendrite Python SDK is a toolkit for building web AI agents that can authenticate, interact with, and extract data from any website, facilitating web automation tasks. -
23
workflowr
Organize your project into a research website
workflowr is an R package that helps researchers organize, version, and share their data science projects in a reproducible and transparent manner. It combines R Markdown, Git, and a structured file system to create a research website that tracks analysis, results, and code changes over time. It’s ideal for academic and collaborative research workflows. -
24
SkyCrypt
A Hypixel skyblock stats website
SkyCrypt is a web-based application that allows players of Hypixel SkyBlock to view and share detailed information about their in-game profiles through a visually rich interface. It aggregates data from the Hypixel API and presents it in an organized format, including player statistics, skills, equipment, and inventory details. The project is built with a Node.js-based stack and integrates additional technologies such as MongoDB and Redis to handle data storage and caching. SkyCrypt enhances... -
25
changedetection.io
The best free open source website change detection and restock service
Loved by smart shoppers, data journalists, research engineers, data scientists, security researchers, and more. From simply monitoring website pages that have a change (such as watching prices, and restocking notifications), to deep inspection such as PDF text support, JSON and XML monitoring, and extensive text triggers. Monitor out-of-stock products and get alerts when those products are back in stock, get restock alerts via Discord, Slack, email, and many other platforms. ...
