StarRankExplorer
랭킹으로 돌아가기

zorlan/skycaiji

PHPskycaiji.com

蓝天采集器是一款开源免费的爬虫系统，仅需点选编辑规则即可采集数据，可运行在本地、虚拟主机或云服务器中，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

crawlercrawlingspiderwebcrawlerphp
스타 성장
스타
2.1k
포크
606
주간 성장
—
이슈
39
1k2k
2018년 3월2020년 12월2023년 10월2026년 7월
아티팩트Packagistcomposer require zorlan/skycaiji
README

将本压缩包上传至您的服务器，如果根目录有站点建议放在子目录里，解压后打开浏览器输入您的服务器域名或ip地址（存放在子目录则加上子目录的名称）进入安装界面。

入门手册：https://www.skycaiji.com/manual

使用协议：https://www.skycaiji.com/licenses

관련 저장소
firecrawl/firecrawl

The API to search, scrape, and interact with the web at scale. 🔥

TypeScriptnpmGNU Affero General Public License v3.0aicrawler
firecrawl.dev
154.1k8.8k
D4Vinci/Scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

PythonPyPIBSD 3-Clause "New" or "Revised" Licensecrawlercrawling
scrapling.readthedocs.io/en/latest/
70.6k7k
scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

PythonPyPIBSD 3-Clause "New" or "Revised" Licensepythonscraping
scrapy.org
63.3k11.8k
NaiboWang/EasySpider

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

JavaScriptnpmGNU Affero General Public License v3.0code-freecrawler
easyspider.net
44.3k5.4k
iawia002/lux

👾 Fast and simple video download library and CLI tool written in Go

GoGo ModulesMIT Licensedownloadergo
31.6k3.3k
mendableai/firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

TypeScriptnpmGNU Affero General Public License v3.0aicrawler
firecrawl.dev
29.5k2.5k
ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

PythonPyPIMIT Licensescrapingscraping-python
scrapegraphai.com
28.5k2.8k
gocolly/colly

Elegant Scraper and Crawler Framework for Golang

GoGo ModulesApache License 2.0golangscraper
go-colly.org
25.4k1.9k
apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

TypeScriptnpmApache License 2.0web-scrapingweb-crawling
crawlee.dev
24.8k1.6k
jhao104/proxy_pool

Python ProxyPool for web spider

PythonPyPIMIT Licensecrawlerproxy
jhao104.github.io/proxy_pool/
23.5k5.4k
Evil0ctal/Douyin_TikTok_Download_API

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。

PythonPyPIApache License 2.0pythonpywebio
douyin.wtf
18.9k2.7k
projectdiscovery/katana

A next-generation crawling and spidering framework.

GoGo ModulesMIT Licensecrawlerweb-spider
17.2k1.2k