taskforcesh/bullmq

TypeScriptbullmq.io

BullMQ - Message Queue and Batch processing for NodeJS, Python, .NET, Elixir, Rust and PHP based on Redis or PostgreSQL

background-jobsnodejsqueueredistypescriptpythonelixirphprust
Star 增长趋势
Star
9.4k
Forks
682
周增长
+24
Issues
272
5k
2019年8月2021年12月2024年4月2026年9月
制品库npm
README



The fastest, most reliable, Redis-based distributed queue for Node.js, Python, Elixir, .NET, Rust, PHP, and more.
Carefully written for rock solid stability and atomicity.

Read the documentation

Follow Us for *important* Bull/BullMQ/BullMQ-Pro news and updates!

🛠 Tutorials

You can find tutorials and news in this blog: https://blog.taskforce.sh/

News 🚀

🌐 Language agnostic BullMQ

BullMQ is available natively in multiple languages:

  • Node.js / Bun — This repository (npm install bullmq)
  • Pythonpython/ directory (pip install bullmq)
  • Rustrust/ directory (cargo add bullmq-official --rename bullmq)
  • Elixirelixir/ directory ({:bullmq, "~> x.x"})
  • .Netdotnet/ directory (dotnet add package BullMQ)
  • PHPphp/ directory

For other platforms, check out the BullMQ Proxy.

Official FrontEnd

Taskforce.sh, Inc

Supercharge your queues with a professional front end:

  • Get a complete overview of all your queues.
  • Inspect jobs, search, retry, or promote delayed jobs.
  • Metrics and statistics.
  • and many more features.

Sign up at Taskforce.sh

🚀 Sponsors 🚀

Dragonfly Dragonfly is a new Redis™ drop-in replacement that is fully compatible with BullMQ and brings some important advantages over Redis™ such as massive better performance by utilizing all CPU cores available and faster and more memory efficient data structures. Read more here on how to use it with BullMQ.

Used by

Some notable organizations using BullMQ:

Microsoft Vendure Datawrapper Nest Langfuse
Curri Novu NoCodeDB Infisical

The gist

Install:

$ yarn add bullmq

If you use the node-redis adapter (createNodeRedisClient), install redis v5 or newer (redis >= 5.0.0).

If you use the Valkey Glide adapter (createValkeyGlideClient), install @valkey/valkey-glide.

Add jobs to the queue:

import { Queue } from 'bullmq';

const queue = new Queue('Paint');

queue.add('cars', { color: 'blue' });

Process the jobs in your workers:

import { Worker } from 'bullmq';

const worker = new Worker('Paint', async job => {
  if (job.name === 'cars') {
    await paintCar(job.data.color);
  }
});

Listen to jobs for completion:

import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents('Paint');

queueEvents.on('completed', ({ jobId }) => {
  console.log('done painting');
});

queueEvents.on(
  'failed',
  ({ jobId, failedReason }: { jobId: string; failedReason: string }) => {
    console.error('error painting', failedReason);
  },
);

Adds jobs with parent-child relationship:

import { FlowProducer } from 'bullmq';

const flow = new FlowProducer();

const originalTree = await flow.add({
  name: 'root-job',
  queueName: 'topQueueName',
  data: {},
  children: [
    {
      name: 'child-job',
      data: { idx: 0, foo: 'bar' },
      queueName: 'childrenQueueName',
      children: [
        {
          name: 'grandchild-job',
          data: { idx: 1, foo: 'bah' },
          queueName: 'grandChildrenQueueName',
        },
        {
          name: 'grandchild-job',
          data: { idx: 2, foo: 'baz' },
          queueName: 'grandChildrenQueueName',
        },
      ],
    },
    {
      name: 'child-job',
      data: { idx: 3, foo: 'foo' },
      queueName: 'childrenQueueName',
    },
  ],
});

This is just scratching the surface, check all the features and more in the official documentation

Feature Comparison

Since there are a few job queue solutions, here is a table comparing them:

Feature BullMQ-Pro BullMQ Bull Kue Bee Agenda
Backend redis redis redis redis redis mongo
Observables
Group Rate Limit
Group Support
Batches Support
Parent/Child Dependencies
Deduplication (Debouncing)
Deduplication (Throttling)
Priorities
Concurrency
Delayed jobs
Global events
Rate Limiter
Pause/Resume
Sandboxed worker
Repeatable jobs
Atomic ops
Persistence
UI
Optimized for Jobs / Messages Jobs / Messages Jobs / Messages Jobs Messages Jobs

Contributing

Fork the repo, make some changes, submit a pull-request! Here is the contributing doc that has more details.

Thanks

Thanks for all the contributors that made this library possible, also a special mention to Leon van Kammen that kindly donated his npm bullmq repo.

相关仓库
statelyai/xstate

State machines, statecharts, and actors for complex logic

TypeScriptnpmMIT Licensestate-machinestatechart
stately.ai/docs
30.1k1.4k
triggerdotdev/trigger.dev

Trigger.dev – build and deploy fully‑managed AI agents and workflows

TypeScriptnpmApache License 2.0automationorchestration
trigger.dev/changelog
16.2k1.4k
hibiken/asynq

Simple, reliable, and efficient distributed task queue in Go

GoGo ModuleslibraryMIT Licensetask-queuego
13.7k979
sidekiq/sidekiq

Simple, efficient background processing for Ruby

RubyRubyGemslibraryOtherrubysidekiq
sidekiq.org
13.6k2.5k
rq/rq

Simple job queues for Python

PythonPyPIlibraryOtherredispython
python-rq.org
10.7k1.5k
HangfireIO/Hangfire

An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required

C#libraryOtherdotnetdotnet-core
hangfire.io
10.1k1.8k
resque/resque

Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.

RubyRubyGemslibraryMIT Licensequeuebackground-jobs
resque.github.io
9.5k1.7k
riverqueue/river

Fast and reliable background jobs in Go

GoGo ModuleslibraryMozilla Public License 2.0background-jobsgo
riverqueue.com
5.6k178
Evernote/android-job

Android library to handle jobs in the background.

JavaMavenlibraryApache License 2.0androidandroid-job
evernote.github.io/android-job/
5.3k554
jamesmh/coravel

Near-zero config .NET library that makes advanced application features like Task Scheduling, Caching, Queuing, Event Broadcasting, and more a breeze!

C#libraryMIT Licensedotnetcorebackground-jobs
docs.coravel.net/Installation/
4.3k271
svix/svix-webhooks

The open source and enterprise-ready webhooks service 🦀

Rustcrates.iolibraryMIT Licensewebhooksrust
svix.com
3.4k279
shrinerb/shrine

File Attachment toolkit for Ruby applications

RubyRubyGemslibraryMIT Licensefile-uploadruby
shrinerb.com
3.3k277