Skip to main content

Exports task execution metrics in Prometheus format

Project description

Exports task execution metrics in Prometheus format: how many tasks were started and have completed successfully or with failure, and how many tasks are still in the queues (supported only for broker redis).

Inspired by https://gitlab.com/kalibrr/celery-prometheus

Usage

Start HTTP service

Start the HTTP server like this:

$ bin/celery prometheus --host=127.0.0.1 --port=9691

Configure Prometheus

scrape_configs:
  - job_name: 'celery'
    static_configs:
      - targets: ['localhost:9691']

We export the following metrics:

  • celery_tasks_total{state="started|succeeded|failed|retried|retries-exceeded", queue="..."}, counter

  • celery_task_queuetime_seconds{queue}, histogram (only if task_send_sent_event is enabled in Celery)

  • celery_task_runtime_seconds{queue}, histogram

If you pass --queuelength-interval=x then every x seconds the queue lengths will be checked (NOTE: this only works with redis as the broker), resulting in this additional metric:

  • celery_queue_length{queue="..."}, gauge

Run tests

Using tox and py.test. Maybe install tox (e.g. via pip install tox) and then simply run tox.

celery_redis_prometheus changes

1.5.0 (2023-10-27)

  • Be explicit about unregistering default collectors

1.4.0 (2022-10-28)

  • Add queue label to task_queuetime and task_runtime metrics

1.3.0 (2022-09-14)

  • Add queue label to celery_tasks_total metric

  • Add separate state retries-exceeded (previously used failed)

1.2.0 (2022-06-23)

  • Update to celery-5.x

1.1.1 (2020-03-20)

  • Fix py2 class syntax issues

1.1.0 (2020-03-20)

  • Make py2 compatible.

1.0.0 (2020-03-20)

  • Initial release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

celery_redis_prometheus-1.5.0.tar.gz (6.4 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page