celery-redis-prometheus 1.5.1

Exports task execution metrics in Prometheus format

Exports task execution metrics in Prometheus format: how many tasks were started and have completed successfully or with failure, and how many tasks are still in the queues (supported only for broker redis).

Inspired by https://gitlab.com/kalibrr/celery-prometheus

Usage

Start HTTP service

Start the HTTP server like this:

$ bin/celery prometheus --host=127.0.0.1 --port=9691

Configure Prometheus

scrape_configs:
  - job_name: 'celery'
    static_configs:
      - targets: ['localhost:9691']

We export the following metrics:

• celery_tasks_total{state="started|succeeded|failed|retried|retries-exceeded", queue="..."}, counter

• celery_task_queuetime_seconds{queue}, histogram (only if task_send_sent_event is enabled in Celery)

• celery_task_runtime_seconds{queue}, histogram

If you pass --queuelength-interval=x then every x seconds the queue lengths will be checked (NOTE: this only works with redis as the broker), resulting in this additional metric:

• celery_queue_length{queue="..."}, gauge

Run tests

Using tox and py.test. Maybe install tox (e.g. via pip install tox) and then simply run tox.

celery_redis_prometheus changes

1.5.1 (2023-10-27)

• Fix brown-bag release

1.5.0 (2023-10-27)

• Be explicit about unregistering default collectors

1.4.0 (2022-10-28)

• Add queue label to task_queuetime and task_runtime metrics

1.3.0 (2022-09-14)

• Add queue label to celery_tasks_total metric

• Add separate state retries-exceeded (previously used failed)

1.2.0 (2022-06-23)

• Update to celery-5.x

1.1.1 (2020-03-20)

• Fix py2 class syntax issues

1.1.0 (2020-03-20)

• Make py2 compatible.

1.0.0 (2020-03-20)

• Initial release