ランキングに戻る

apache/paimon

Javapaimon.apache.org

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

big-datadata-ingestionflinkpaimonreal-time-analyticssparktable-storestreaming-datalake
スター成長
スター
3.3k
フォーク
1.4k
週間成長
Issue
624
1k2k3k
2022年1月2023年7月2025年1月2026年7月
成果物Mavengit clone https://github.com/apache/paimon.git
README

Paimon

License Get on Slack

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations. Paimon innovatively combines lake format and LSM structure, bringing realtime streaming updates into the lake architecture.

Background and documentation are available at https://paimon.apache.org

Paimon's former name was Flink Table Store, developed from the Flink community. The architecture refers to some design concepts of Iceberg. Thanks to Apache Flink and Apache Iceberg.

Collaboration

Paimon tracks issues in GitHub and prefers to receive contributions as pull requests.

Mailing Lists

Name Subscribe Digest Unsubscribe Post Archive
user@paimon.apache.org
User support and questions mailing list
Subscribe Subscribe Unsubscribe Post Archives
dev@paimon.apache.org
Development related discussions
Subscribe Subscribe Unsubscribe Post Archives

Please make sure you are subscribed to the mailing list you are posting to! If you are not subscribed to the mailing list, your message will either be rejected (dev@ list) or you won't receive the response (user@ list).

Slack

You can join the Paimon community on Slack. Paimon channel is in ASF Slack workspace.

  • Anyone with an @apache.org email address can become a full member of the ASF Slack workspace. Search Paimon channel and join it.
  • If you don't have an @apache.org email address, you can email to user@paimon.apache.org to apply for an ASF Slack invitation. Then join Paimon channel.

Building

JDK 8/11 is required for building the project. Maven version >=3.6.3.

  • Run the mvn clean install -DskipTests command to build the project.
  • Run the mvn spotless:apply to format the project (both Java and Scala).
  • IDE: Mark paimon-common/target/generated-sources/antlr4 as Sources Root.

How to Contribute

Contribution Guide.

License

The code in this repository is licensed under the Apache Software License 2.

関連リポジトリ
binhnguyennus/awesome-scalability

The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

MIT Licensesystem-designbackend
72.6k7k
ClickHouse/ClickHouse

ClickHouse® is a real-time analytics database management system

C++Apache License 2.0dbmsolap
clickhouse.com
48.8k8.7k
apache/spark

Apache Spark - A unified analytics engine for large-scale data processing

ScalaApache License 2.0pythonscala
spark.apache.org
43.7k29.3k
donnemartin/data-science-ipython-notebooks

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

PythonPyPIOtherpythonmachine-learning
29.2k8k
apache/flink

Apache Flink

JavaMavenApache License 2.0scalajava
flink.apache.org
26.2k14k
thingsboard/thingsboard

Open-source IoT Platform - Device management, data collection, processing and visualization.

JavaMavenApache License 2.0visualizationiot
thingsboard.io
22.1k6.4k
amark/gun

An open source cybersecurity protocol for syncing decentralized graph data.

JavaScriptnpmOthermachine-learningartificial-intelligence
gun.eco/docs
19.1k1.2k
heibaiying/BigData-Notes

大数据入门指南 :star:

JavaMavenhadoophdfs
16.9k4.3k
prestodb/presto

The official home of the Presto distributed SQL query engine for big data

JavaMavenApache License 2.0javapresto
prestodb.io
16.7k5.5k
andkret/Cookbook

The Data Engineering Cookbook

PythonPyPIApache License 2.0data-engineerdata-engineering
learndataengineering.com
15.2k2.7k
trinodb/trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

JavaMavenApache License 2.0javapresto
trino.io
13.1k3.7k
apache/predictionio

PredictionIO, a machine learning server for developers and ML engineers.

ScalaApache License 2.0scalabig-data
predictionio.apache.org
12.5k1.9k