StarRank
RankingEn alzaIAStack de IATemas de IAMapa
Data Systems

Data Pipelines

978 resultados
1yt-dlp/yt-dlpA feature-rich command-line audio/video downloaderPythoncliPyPIyoutube-dlpython200.1k2ytdl-org/youtube-dlCommand-line program to download videos from YouTube.com and other video sitesPythoncliPyPIdownloadervideo141.4k3D4Vinci/Scrapling🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_devPythonlibraryPyPIcrawlercrawling89.1k4doocs/advanced-java😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲：涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识JavaMavenjavadistributed-systems79.2k5redis/redisFor developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.Cdatabasekey-value77k6NanmiCoder/MediaCrawler小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫PythonPyPIcrawlerspider67.6k7scrapy/scrapyScrapy, a fast high-level web crawling & scraping framework for Python.PythonlibraryPyPIpythonscraping64.9k8pathwaycom/pathwayPython ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.PythonPyPIbatch-processingkafka62.1k9rclone/rclone"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex FilesGocliGo Modulesgolanggo60.5k10soimort/you-get:arrow_double_down: Dumb downloader that scrapes the webPythoncliPyPIdownloadervideo56.9k11apache/airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflowsPythonPyPIairflowapache47.3k12DIYgod/RSSHub🧡 Everything is RSSibleTypeScriptnpmrsstwitter46.6k13DataTalksClub/data-engineering-zoomcampData Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼Jupyter Notebooktutorialdata-engineeringkafka46.3k14imputnet/cobaltbest way to save what you loveSvelteappdownloaderreddit45.3k15NaiboWang/EasySpiderA visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。JavaScriptappnpmcode-freecrawler44.7k16DataExpert-io/data-engineer-handbookThis is a repo with links to everything you'd ever want to learn about data engineeringJupyter Notebookawesomeapachesparkawesome44.5k17apache/sparkApache Spark - A unified analytics engine for large-scale data processingScalapythonscala44.3k18google-ai-edge/mediapipeCross-platform, customizable ML solutions for live and streaming media.C++librarymediapipec-plus-plus37.4k19restic/resticFast, secure, efficient backup programGocliGo Modulesgorestic36.7k20apache/kafkaApache Kafka - A distributed event streaming platformJavaMavenscalakafka34.1k21kestra-io/kestraEvent Driven Orchestration & Scheduling Platform for Mission Critical ApplicationsJavaMavenorchestrationdata-orchestration29.9k22alibaba/canal阿里巴巴 MySQL binlog 增量订阅&消费组件JavaMavenmysqlbinlog29.7k23celery/celeryDistributed Task Queue (development branch)PythonlibraryPyPIpythontask-manager29k24ArchiveBox/ArchiveBox🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...PythonPyPIpocketwget28.8k
Mostrando 1–24 de 978
12345