Easy configuration for celery app using Yaml file
Project description
Easy Configuration For Celery App Using a Yaml File
celery-yaml
is a library to inject a --yaml option to the celery worker
command in order to inject its configuration.
It also handle help to configurate this application for Pyramid application.
Usage
With Celery 4
celery worker -A my_application.module_containing_my.app --yaml development.yaml ...
With Celery 5
celery -A my_application.module_containing_my.app worker --yaml development.yaml ...
This will configure the application my_application
containing an application
app
in a submodule module_containing_my
.
The celery app must register the --yaml
using the add_yaml_option
on the
app instance, this way:
from celery import Celery
from celery_yaml import add_yaml_option
app = Celery()
add_yaml_option(app)
Yaml format
celery:
broker_url: 'amqp://guest:guest@localhost:5672//'
result_backend: 'rpc://'
imports:
- my_application.tasks
# see all settings in the celery docs:
# https://docs.celeryproject.org/en/stable/userguide/configuration.html
logging:
version: 1
# dictConfig format
# https://docs.python.org/3/library/logging.config.html#logging-config-dictschema
NOTE
The broker_url can also be override by an environment variable CELERY_BROKER_URL
to avoid password in the configuration file.
Using Celery in a Pyramid App.
The extras "pyramid" must be added to install the extras depencencies.
With poetry
[tool.poetry.dependencies]
celery-yaml = { version = "^0.1.3", extras = ["pyramid"] }
Then some entry_points have to configure, such as:
[tool.poetry.plugins."paste.app_factory"]
main = "pyramid_app.wsgi:main"
[tool.poetry.plugins."celery_yaml.app"]
main = "pyramid_app.backend:app"
[tool.poetry.plugins."plaster.loader_factory"]
"file+yaml" = "plaster_yaml:Loader"
the paste.app_factory
is used by Pyramid
itself to build the WSGI
sergivice but we add a plaster.loader_factory
to configure the usage
of a yaml file instead of an ini
file to configure it.
Then the celery_yaml.app
is used by celery-yaml
as an entrypoint to
the celery app.
Then, in the configuration file,
celery: &celery
result_backend: 'rpc://'
imports:
- pyramid_app.tasks
app:
"use": "egg:pyramid_app"
"pyramid.includes": ["celery_yaml"]
"celery":
<<: *celery
"use": "egg:pyramid_app"
See full example in the examples directory:
Pyramid with Celery 4
https://github.com/mardiros/celery-yaml/tree/master/examples/pyramid-app
Pyramid with Celery 5
https://github.com/mardiros/celery-yaml/tree/master/examples/pyramid-app5
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for celery_yaml-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 019cde36e1c6fefe692275ad6482f2e27fd01eb53e3dd16dae437416bc1f63c3 |
|
MD5 | 4d17d9b6724a1e097e343916c03f5884 |
|
BLAKE2b-256 | 3a2536fb5379047186aeae8b584f14ff2194656dd51d1188b23f8f7977659a33 |