Retour au classement

sequinstream/sequin

Elixirsequinstream.com

Postgres change data capture to streams, queues, and search indexes like Kafka, SQS, Elasticsearch, HTTP endpoints, and more

cdcchange-data-captureelixirerlangkafkapostgrespostgresqlqueuestreamingsearch
Croissance des étoiles
Étoiles
2.2k
Forks
143
Croissance hebdomadaire
Issues
50
1k2k
juil. 2024mars 2025nov. 2025juil. 2026
README

Sequin

Sequin is the fastest change data capture (CDC) platform for Postgres. Sequin makes it easy to stream Postgres to streaming platforms, queues, search indexes, and more (e.g. Kafka, GCP Pub/Sub, another Postgres, etc.). You can backfill existing rows and stream new changes in real-time.

Sequin is a standalone Docker container that you can deploy next to your Postgres database.

For production deployments on AWS, see the deployment/ directory for Terraform infrastructure-as-code templates.

Sequin is open source. To help us make this project great, tell us what you're building in our Discord server or Slack community.

Getting started

See our quickstart to get up and running in your dev environment in about 5 minutes.

Performance

Sequin delivers industry-leading performance for change data capture (CDC), sustaining 50k operations per second (40 MB/s) with 55ms average latency.

Tool / Deployment Sustained throughput Avg latency 99p latency
Sequin >50 k ops / s 55 ms 253 ms
Debezium · MSK 6 k ops / s 258 ms 499 ms
Debezium · Server 23 k ops / s 210 ms 440 ms
Fivetran - 5+ minutes -
Airbyte - 1+ hours -

Read more about our benchmarking methodology and performance in our docs.

Use cases

Sequin works great for change data capture use cases like:

  • Streaming events: Stream database changes as events for other services to consume using Kafka, SQS, Pub/Sub and more.
  • Audit logging: Track, enrich, and record every change in your database for compliance or feature development.
  • Maintaining a cache: Keep application caches in sync with your database by streaming changes to Redis compliant caches.
  • Refreshing search indexes: End-to-end tooling to warm search indexes like Typesense and Elasticsearch and keep them in sync with sub-second latency.

Sinks

Icon Sink Docs Description
Azure EventHubs Azure EventHubs Reference Publish messages to Azure EventHubs
GCP Pub/Sub GCP Pub/Sub Reference Publish messages to Google Cloud Pub/Sub topics
Elasticsearch Elasticsearch Reference Index database changes with Elasticsearch
HTTP Pull Sequin Stream (HTTP GET) Reference Consume changes directly from Sequin with exactly-once processing
Kafka Kafka Reference Stream changes to Apache Kafka topics
Kinesis Kinesis Reference Send messages to Amazon Kinesis streams queues
Meilisearch Meilisearch Reference Index database changes with Meilisearch
NATS NATS Reference Stream changes to NATS subjects
RabbitMQ RabbitMQ Reference Publish messages to RabbitMQ exchanges
Redis Stream Redis Stream Reference XADD to Redis Streams
Redis String Redis String Reference SET to Redis keys
S2 S2 Reference Send messages to S2 streams
SQS SQS Reference Send messages to Amazon SQS queues
SNS SNS Reference Send messages to Amazon SNS
Typesense Typesense Reference Index database changes with Typesense
Webhook Webhooks (HTTP POST) Reference Send changes to any HTTP endpoint

Killer features

  • Bring your database: Sequin is not an extension. It works with any Postgres database version 14+.
  • Never miss a change: Sequin ensures 100% delivery of database changes to sinks with strict ordering.
  • Filters: Write custom filters to exclude or include changes to sink.
  • Transforms: Transform message payloads by writing functions in low-latency Elixir.
  • Routing: Route messages to specific topics, endpoints, or indexes.
  • Exactly-once processing: Sequin ensures exactly-once processing of database changes using idempotency keys.
  • Backfills: Backfill sinks at any point in time. Run partial backfills to only replay specific rows.
  • Infrastructure as code: Manage Sequin as code using a YAML file paired with the Sequin CLI. Or use our Management API.
  • Rich web console: Your team can configure and monitor sinks from a full feature web console.
  • Observability: Sequin ships a Prometheus metrics endpoint.

How Sequin works

Sequin connects to any Postgres database. Specify the schemas and tables you want to stream, as well as optional filters and transformations. Route changes to sinks like Kafka, Google Pub/Sub, Redis, etc.

When you setup a sink, you can opt to backfill data from the source table to the sink.

After setup, Sequin will stream new changes to the sink as they occur in real-time. If there are any issues with delivery, Sequin will automatically retry delivery with exponential backoff.

Sequin comes with a web console/UI for configuration and monitoring. You can also configure Sequin as code using YAML config files and our Management API.

Why Sequin

We all know Postgres is great for storing and querying data. But what about when you need to stream changes to other systems?

Existing tools like Debezium require significant infrastructure to operate (Kafka), endless guess-and-check configuration, and limited tooling. Other ETL tools (e.g. Fivetran) run in batches and are expensive.

This leads many teams to build brittle, homegrown solutions that are difficult to maintain and lack essential features.

Fast, simple, reliable Postgres CDC

Sequin provides fastest, simple Postgres CDC. It's tuned to Postgres to stream changes in real-time, provides strict guarantees, and doesn't require complex infrastructure to operate.

Contributing

See CONTRIBUTING.md for detailed instructions on how to contribute to Sequin.

Dépôts similaires
debezium/debezium

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

JavaMavenApache License 2.0change-data-capturekafka-connect
debezium.io
12.9k3k
apache/seatunnel

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

JavaMavenApache License 2.0data-integrationhigh-performance
seatunnel.apache.org
9.5k2.3k
supabase/realtime

Broadcast, Presence, and Postgres Changes via WebSockets

ElixirApache License 2.0elixirpostgres
supabase.com/realtime
7.6k451
apache/flink-cdc

Flink CDC is a streaming data integration tool

JavaMavenApache License 2.0change-data-capturecdc
nightlies.apache.org/flink/flink-cdc-docs-stable
6.4k2.2k
MaterializeInc/materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

Rustcrates.ioOtherrustdatabase
materialize.com
6.3k512
PeerDB-io/peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

GoGo ModulesGNU Affero General Public License v3.0etlsql
peerdb.io
3.2k198
disease-sh/API

API for Current cases and more stuff about COVID-19 and Influenza

JavaScriptnpmGNU General Public License v3.0coronacovid19
disease.sh
2.5k622
confluentinc/examples

Apache Kafka, Apache Flink and Confluent Platform examples and demos

JavaMavenApache License 2.0kafkademo
2.1k1.1k
cherry-embedded/CherryUSB

CherryUSB is a tiny and beautiful, high performance and portable USB host and device stack for embedded system with USB IP

CApache License 2.0usb-hostusb-device
cherryusb.readthedocs.io
2k428
eulerto/wal2json

JSON output plugin for changeset extraction

CBSD 3-Clause "New" or "Revised" Licensepostgresreplication
1.5k167
datazip-inc/olake

OLake - Fastest Databases, Kafka & S3 Replication to Apache Iceberg with Table optimization (Called OLake Fusion). ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supported sources : Postgres, MongoDB, MySQL, Oracle, MSSql, DB2, Kafka, S3.

GoGo ModulesApache License 2.0cdcchange-data-capture
olake.io
1.4k239
datavane/tis

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI

JavaMavenApache License 2.0javadatax
tis.pub
1.3k284