返回排行榜

A browser testing and web crawling library for PHP and Symfony

scrapinge2e-testingwebdriverseleniumselenium-webdriversymfonyphpchromedriverhacktoberfest
Star 增长趋势
Star
3.1k
Forks
232
周增长
Issues
200
1k2k3k
2018年3月2020年12月2023年10月2026年7月
制品库Packagistcomposer require symfony/panther
README

Panther

A browser testing and web scraping library for PHP and Symfony

CI

Panther is a convenient standalone library to scrape websites and to run end-to-end tests using real browsers.

Panther is super powerful. It leverages the W3C's WebDriver protocol to drive native web browsers such as Google Chrome and Firefox.

Resources

Save the Panthers

Many of the wild cat species are highly threatened. If you like this software, help save the (real) panthers by donating to the Panthera organization.

Credits

Created by Kévin Dunglas. Sponsored by Les-Tilleuls.coop.

Panther is built on top of PHP WebDriver and several other FOSS libraries. It has been inspired by Nightwatch.js, a WebDriver-based testing tool for JavaScript.

相关仓库
firecrawl/firecrawl

The API to search, scrape, and interact with the web at scale. 🔥

TypeScriptnpmGNU Affero General Public License v3.0aicrawler
firecrawl.dev
153.6k8.8k
D4Vinci/Scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

PythonPyPIBSD 3-Clause "New" or "Revised" Licensecrawlercrawling
scrapling.readthedocs.io/en/latest/
70.4k7k
scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

PythonPyPIBSD 3-Clause "New" or "Revised" Licensepythonscraping
scrapy.org
63.3k11.8k
soxoj/maigret

🕵️‍♂️ Collect a dossier on a person by username from 3000+ sites

PythonPyPIMIT Licenseosintsocial-network
maigret.app
35.6k2.7k
feder-cr/Jobs_Applier_AI_Agent_AIHawk

AIHawk aims to easy job hunt process by automating the job application process. Utilizing artificial intelligence, it enables users to apply for multiple jobs in a tailored way.

PythonPyPIGNU Affero General Public License v3.0automationbot
30k4.6k
mendableai/firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

TypeScriptnpmGNU Affero General Public License v3.0aicrawler
firecrawl.dev
29.5k2.5k
ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

PythonPyPIMIT Licensescrapingscraping-python
scrapegraphai.com
28.5k2.8k
gocolly/colly

Elegant Scraper and Crawler Framework for Golang

GoGo ModulesApache License 2.0golangscraper
go-colly.org
25.4k1.9k
apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

TypeScriptnpmApache License 2.0web-scrapingweb-crawling
crawlee.dev
24.8k1.6k
feder-cr/Jobs_Applier_AI_Agent

Auto_Jobs_Applier_AI_Agent aims to easy job hunt process by automating the job application process. Utilizing artificial intelligence, it enables users to apply for multiple jobs in an automated and personalized way.

PythonPyPIautomationbot
24.5k3.6k
psf/requests-html

Pythonic HTML Parsing for Humans™

PythonPyPIMIT Licensehtmlscraping
html.python-requests.org
13.8k998
ultrafunkamsterdam/undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

PythonPyPIGNU General Public License v3.0chromedriverselenium
github.com/UltrafunkAmsterdam/undetected-chromedriver
12.8k1.3k