rearq 0.1.1

Rewrite arq and make improvement.

Rearq

Introduction

Rearq is a distributed task queue with asyncio and redis, which rewrite from arq and make improvement.

Install

Just install from pypi:

> pip install rearq

or install latest code from github:

pip install -e git+https://github.com/long2ice/rearq.git

Quick Start

Task Definition

# main.py
rearq = ReArq()


@rearq.on_shutdown
async def on_shutdown(r):
    print("shutdown")


@rearq.on_startup
async def on_startup(r):
    print("startup")


@rearq.task(queue = "myqueue")
async def add(worker, a, b):
    return a + b

@rearq.task(cron=CronTab("*/5 * * * * * *"))
async def timer(worker):
    return "timer"

Run rearq worker

> rearq worker main:rearq -q myqueue

2020-06-04 15:37:02 - rearq.worker:92 - INFO - Start worker success with queue: myqueue
2020-06-04 15:37:02 - rearq.worker:84 - INFO - redis_version=6.0.1 mem_usage=1.47M clients_connected=25 db_keys=5

Run rearq timing worker

If you have timeing task, run another command also:

> rearq worker -t main:rearq

2020-06-04 15:37:44 - rearq.worker:346 - INFO - Start timer worker success with queue: myqueue
2020-06-04 15:37:44 - rearq.worker:84 - INFO - redis_version=6.0.1 mem_usage=1.47M clients_connected=25 db_keys=5

Why not arq

Thanks great work of arq, but that project is not so active now and lack of maintenance. On the other hand, I don't like some solution of arq and it's api, so I open this project and aims to work better.

What's the differences

Api

Rearq provide more friendly api to add register and add task, inspired by celery.

Rearq:

rearq = Rearq()

@rearq.task()
async def add(worker, a, b):
    return a + b

job = await add.delay(args=(1, 2))
print(job)

Arq:

class WorkerSettings:
    functions = [ add ]
    redis_settings = RedisSettings(**settings.ARQ)

async def add(ctx, a,b):
    return a + b

await arq.enqueue_job('add', 1, 2)

Queue implementation

Arq use redis zset to make delay queue and timing queue, and Rearq use zset and stream with ack.

ThanksTo

• arq, Fast job queuing and RPC in python with asyncio and redis.

License

This project is licensed under the MIT License.