# Huey: A Lightweight Python Task Queue for Redis, SQLite, and More

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/coleifer-huey
Generated for open source discovery and AI-assisted research.

Huey is a lightweight and simple Python task queue designed for various storage backends like Redis, SQLite, or in-memory. It provides a clean API for scheduling tasks, retrying failures, managing recurring jobs, and executing them efficiently across multiple processes, threads, or greenlets. This library offers a robust solution for asynchronous task management in Python applications.

GitHub: https://github.com/coleifer/huey
OSRepos URL: https://osrepos.com/repo/coleifer-huey

## Summary

Huey is a lightweight and simple Python task queue designed for various storage backends like Redis, SQLite, or in-memory. It provides a clean API for scheduling tasks, retrying failures, managing recurring jobs, and executing them efficiently across multiple processes, threads, or greenlets. This library offers a robust solution for asynchronous task management in Python applications.

## Topics

- Python
- Task Queue
- Redis
- Asynchronous
- Background Jobs
- Developer Tools
- Queueing

## Repository Information

Last analyzed by OSRepos: Tue Feb 17 2026 12:01:18 GMT+0000 (Western European Standard Time)
Detail views: 5
GitHub clicks: 3

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

Huey is a lightweight and simple task queue written in Python, designed to handle asynchronous tasks efficiently. It offers a clean and straightforward API, making it easy to integrate into your Python applications. Huey supports various storage backends, including Redis, SQLite, file-system, or even in-memory, providing flexibility for different project requirements.

Key features of Huey include multi-process, multi-thread, or greenlet task execution models, allowing you to choose the best concurrency strategy for your workload. It enables scheduling tasks to execute at a given time or after a delay, and supports recurring tasks similar to a crontab. Huey also provides mechanisms for automatically retrying failed tasks, task prioritization, result storage, expiration, and locking, along with task pipelines and chains for complex workflows.

## Installation

To get started with Huey, you can install it using pip:

bash
pip install huey


Depending on your chosen storage backend, you might need to install additional dependencies, for example, `redis` for Redis storage.

## Examples

Here are some basic examples demonstrating how to define tasks, call them, and run the Huey consumer.

**Defining and Calling a Task:**

First, initialize Huey with your preferred storage backend, such as Redis:

python
from huey import RedisHuey, crontab

huey = RedisHuey('my-app', host='redis.myapp.com')

@huey.task()
def add_numbers(a, b):
    """A simple task to add two numbers."""
    return a + b

@huey.task(retries=2, retry_delay=60)
def flaky_task(url):
    """This task might fail and will be retried up to 2 times with a 60s delay."""
    # Assume this_might_fail is a function that can raise an exception
    return this_might_fail(url)

@huey.periodic_task(crontab(minute='0', hour='3'))
def nightly_backup():
    """A task that runs every night at 3 AM."""
    sync_all_data() # Assume sync_all_data performs a backup


Calling a `task`-decorated function enqueues the function call for execution by the consumer. A special result handle is returned immediately, which can be used to fetch the result once the task is finished:

python
# Assuming 'add_numbers' is defined in a module like 'demo.py'
from demo import add_numbers

res = add_numbers(1, 2)
print(f"Task enqueued, result handle: {res}")

# To get the result (this will block until the task is complete)
result = res()
print(f"Task result: {result}") # Output: Task result: 3


**Scheduling Tasks for Future Execution:**

Tasks can also be scheduled to run in the future, either after a delay or at a specific time:

python
# Schedule 'add_numbers' to run in approximately 10 seconds
res_scheduled = add_numbers.schedule((2, 3), delay=10)
print(f"Task scheduled, result handle: {res_scheduled}")

# Block until the scheduled task finishes (after ~10 seconds)
scheduled_result = res_scheduled(blocking=True)
print(f"Scheduled task result: {scheduled_result}") # Output: Scheduled task result: 5


**Running the Huey Consumer:**

The Huey consumer is responsible for picking up and executing the enqueued tasks. You can run it with different worker models:

To run the consumer with four worker processes:

bash
huey_consumer.py my_app.huey -k process -w 4


To run the consumer with a single worker thread (default):

bash
huey_consumer.py my_app.huey


For IO-bound workloads, you might prefer greenlets due to their lightweight nature:

bash
huey_consumer.py my_app.huey -k greenlet -w 32


## Why Use Huey?

Huey stands out as an excellent choice for managing asynchronous tasks in Python due to several compelling reasons:

*   **Lightweight and Simple API:** It is designed to be a lightweight alternative to more complex task queues, offering a clean and intuitive API that makes task definition and management straightforward.
*   **Flexible Storage Options:** With built-in support for Redis, SQLite, file-system, and in-memory storage, Huey provides versatility, allowing you to choose the backend that best fits your project's infrastructure and scale.
*   **Robust Feature Set:** Beyond basic task execution, Huey includes advanced features like scheduled and recurring tasks, automatic retries for transient failures, task prioritization, result storage, expiration, and locking, covering a wide range of asynchronous processing needs.
*   **Scalable Concurrency Models:** It supports various execution models, including multi-process, multi-thread, and greenlet workers, enabling you to optimize performance based on the nature of your tasks (CPU-bound vs. IO-bound).
*   **Pythonic Design:** Written entirely in Python, Huey integrates seamlessly into existing Python projects, leveraging decorators for task definition and providing a familiar development experience.

## Links

*   **Huey Documentation:** [https://huey.readthedocs.io/](https://huey.readthedocs.io/){:target="_blank"}
*   **Huey GitHub Repository:** [https://github.com/coleifer/huey](https://github.com/coleifer/huey){:target="_blank"}