Skip to main content

Machine Learning Orchestration

Project description

Dbnd Airflow Export plugin

The plugin

dbnd-airflow-export-plugin is a plugin for Airflow system, enables you to fetch data from Airflow database and DAG folder. This Airflow side module is one of two components allows you to observe your Airflow data with Databand system.

What does it do?

The plugin exposes a REST Api within GET /export_data which, expects since (utc). This api returns json with all the relevant information scraped from airflow system.

Configuring Airflow instance

To improve performance and reduce the runtime overhead on airflow database, we advise to add an index on end_date column to task_instance and dag_run tables i.e.

   CREATE INDEX dbnd_ti_end_date_ind ON task_instance (end_date);
   CREATE INDEX dbnd_dr_end_date_ind ON dag_run (end_date);

Installation

In order to install dbnd-airflow-export-plugin we are using Airflow plugin system.

pip install databand[airflow-export]

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbnd-airflow-export-0.40.1.tar.gz (20.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dbnd_airflow_export-0.40.1-py2.py3-none-any.whl (25.5 kB view details)

Uploaded Python 2Python 3

File details

Details for the file dbnd-airflow-export-0.40.1.tar.gz.

File metadata

  • Download URL: dbnd-airflow-export-0.40.1.tar.gz
  • Upload date:
  • Size: 20.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.6.13

File hashes

Hashes for dbnd-airflow-export-0.40.1.tar.gz
Algorithm Hash digest
SHA256 d6a81b4aa9e28727a0ee2ed835e5e924369c385956d7db9c867e45dba79a406f
MD5 79a1ff6ff5455debd24048fbac38e208
BLAKE2b-256 abb04ff0885c214f7e1b416536c0b29798af37465525650ea50114910e05bfe7

See more details on using hashes here.

File details

Details for the file dbnd_airflow_export-0.40.1-py2.py3-none-any.whl.

File metadata

  • Download URL: dbnd_airflow_export-0.40.1-py2.py3-none-any.whl
  • Upload date:
  • Size: 25.5 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.6.13

File hashes

Hashes for dbnd_airflow_export-0.40.1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 63bb12d9594846601f0e8243629880e2e31944733c99edd724f2964c7e0c0637
MD5 c67e4eebbcd923c37268cc5842f8c82d
BLAKE2b-256 cb43131a188dce284636cfb11dd5ca70f6a84b0bba7d8f68ab2015d9e7c3af27

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page