dramatiq: Fast & Reliable Background Task Processing for Python 3

Introduction

dramatiq is a powerful and efficient Python 3 library designed for processing background tasks. It offers a fast and reliable way for applications to offload long-running or resource-intensive operations, ensuring that the main application thread remains responsive. By leveraging message brokers like RabbitMQ and Redis, dramatiq facilitates robust and scalable distributed task execution, making your applications more performant and resilient.

Installation

To get started with dramatiq, you can install it via pip. Choose the installation method based on your preferred message broker:

If you want to use it with RabbitMQ:

pip install 'dramatiq[rabbitmq, watch]'

Or if you want to use it with Redis:

pip install 'dramatiq[redis, watch]'

Examples

Here's a quickstart example demonstrating how to define and dispatch a background task using dramatiq. First, create a file named example.py:

import dramatiq
import requests
import sys


@dramatiq.actor
def count_words(url):
    response = requests.get(url)
    count = len(response.text.split(" "))
    print(f"There are {count} words at {url!r}.")


if __name__ == "__main__":
    count_words.send(sys.argv[1])

In one terminal, run your dramatiq workers:

dramatiq example

In another terminal, you can start enqueueing messages:

python example.py http://example.com
python example.py https://github.com
python example.py https://news.ycombinator.com

Why use dramatiq?

dramatiq stands out as an excellent choice for background task processing due to several key advantages:

• Speed and Reliability: Engineered for high performance and dependable task execution, even under heavy loads.
• Python 3 Focus: Built specifically for modern Python development, taking advantage of its features.
• Broker Flexibility: Seamlessly integrates with popular message brokers such as RabbitMQ and Redis, offering flexibility in your infrastructure.
• Improved Responsiveness: Offloads time-consuming tasks, preventing your main application from freezing and enhancing user experience.
• Scalability: Supports distributed task processing, allowing you to scale your background workers independently.
• Simple API: Provides a straightforward and intuitive API for defining actors and dispatching tasks.

Links

• Documentation: https://dramatiq.io
• Community: https://groups.io/g/dramatiq-users
• Changelog: https://dramatiq.io/changelog.html
• GitHub Repository: https://github.com/Bogdanp/dramatiq
• RabbitMQ: https://www.rabbitmq.com/
• Redis: https://redis.io