CNCF sandbox project, an open source distributed time-series database with high concurrency, high performance, and high scalability
go get github.com/openGemini/openGeminiREADME
English | 简体中文
Official Website | Docs | Contribution Guide | RoadMap | Performance
About openGemini
openGemini is a cloud-native distributed time series database, focusing on the storage and analysis of massive telemetry data.
If you have used openGemini, please let us know and click submit Usage Information
openGemini is an sandbox project of the Cloud Native Computing Foundation (CNCF). Please consider joining the CNCF if you are an organization that wants to take an active role in supporting the growth and evolution of the cloud native ecosystem.
Why openGemini
Compared with other time series database, openGemini has the following advantages:
High Performance
Store and query data quickly and efficiently with automatic partitioning, LSM-based storage techniques, and better data process engineering, see performance reports
High Scalability
Adopting MPP architecture, support distributed cluster deployment and can be flexibly expanded as business grows to meet higher load requirements, see Architecture
High Cardinality
A new high cardinality storage engine solves problems such as excessive index memory usage and low read and write performance, see high cardinality storage engine
Data Compression
Data is stored in column format, and different data types use dedicated data compression algorithms. The data compression ratio is as high as 15:1 or higher, see Data compression algorithm
Flexible Deployment
Supports stand-alone and cluster deployment, and supports various forms of deployment such as virtual machines, physical machines, containers, and clouds. Provides one-click container deployment tool (openGemini-operator) and one-click normal installation and deployment tool (gemix)
Ecosystem compatibility
Compatible with InfluxDB v1.x Line Protocol, Influxql and read/write APIs, also compatible with existing InfluxDB toolchain
Support for Cloud Native Scenarios
Containerized deployment, refer to openGemini-operator
openTelemetry backend storage, simplified architecture
Prometheus backend storage，support remote read/write, and Support PromQL
KubeEdge integration with openGemini, better manage edge device data
Application Scenes
Quick Start
For a more detailed introduction, please visit our official website User Guide
This section mainly contains the following:
- How to compile openGemini source code
- How to run openGemini
Compiling environment information
GO version v1.22+
Python version v3.7+
How to set GO environment variables
Open ~/.profile configuration file and add the following configurations to the end of the file:
export GOPATH=/path/to/dir
export GOBIN=$GOPATH/bin
export GO111MODULE=on
export GONOSUMDB=*
export GOSUMDB=off
Compiling
- Clone source codes from Github
> cd $GOPATH
> mkdir -p {pkg,bin,src}
> cd src
> git clone https://github.com/openGemini/openGemini.git
- Enter the home directory
> cd openGemini
- Compiling
> export CGO_LDFLAGS="-Wl,-z,now -Wl,-z,relro -Wl,-z,noexecstack -fPIE -ftrapv"
> export CGO_CFLAGS="-fstack-protector-strong -D_FORTIFY_SOURCE=2 -O2"
> python build.py
The compiled binary file is in the build directory
> ls build
ts-meta ts-monitor ts-server ts-sql ts-store
Configuration
The configuration file is in the conf directory. For more information about the configuration items, please refer to User Guide --> Reference --> Configuration Files
Run openGemini
Standalone operation
> cd openGemini
> mkdir -p /tmp/openGemini
> sh scripts/install.sh
Refer to cluster deployments in User Guide
Using openGemini
Install ts-cli
> go install github.com/openGemini/openGemini-cli/cmd/ts-cli@latest
Use the openGemini-cli to connect to openGemini
> ts-cli --host 127.0.0.1 --port 8086
After successful login, the following message will be displayed
> ts-cli --host 127.0.0.1 --port 8086
openGemini CLI 0.1.0 (rev-revision)
Please use 'quit', 'exit' or 'Ctrl-D' to exit this program
>
Create a database
> create database sensordb
> use sensordb
The openGemini supports three ways for creating a measurement.
- Implicit creation, automatic creation measurement when data is written. By default, partition is based on time.
- explicit creation, without specifying the partition key, as the same as implicit creation.
> create measurement sensor
- explicit creation, specifying partition keys during table creation, During data storage, the table will be primarily partitioned by time, and then secondarily partitioned according to the specified partition keys.
> create measurement sensor with shardkey farmID
Write data
insert sensor,farmID=f1,deviceID=d0 sensorID="s20",value=50.98
Query data
> select * from sensor
name: sensor
+---------------------+----------+--------+----------+-------+
| time | deviceID | farmID | sensorID | value |
+---------------------+----------+--------+----------+-------+
| 1657959880895515464 | d0 | f1 | s20 | 50.98 |
+---------------------+----------+--------+----------+-------+
5 columns,1 rows in set
Elapsed: 7.723332ms
Join & Contribute
Code of Conduct
openGemini follows the CNCF Code of Conduct.
Contact Us
License
openGemini is licensed under the Apache License 2.0. Refer to LICENSE for more details.
For third-party software usage notice, see Open_Source_Software_Notice