Skip to main content

Provider package apache-airflow-providers-microsoft-azure for Apache Airflow

Project description

Package apache-airflow-providers-microsoft-azure

Release: 3.2.0rc1

Microsoft Azure

Provider package

This is a provider package for microsoft.azure provider. All classes for this provider package are in airflow.providers.microsoft.azure python package.

You can find package information and changelog for the provider in the documentation.

Installation

You can install this package on top of an existing airflow 2.1+ installation via pip install apache-airflow-providers-microsoft-azure

The package supports the following python versions: 3.6,3.7,3.8,3.9

PIP requirements

PIP package

Version required

apache-airflow

>=2.1.0

azure-batch

>=8.0.0

azure-cosmos

>=3.0.1,<4

azure-datalake-store

>=0.0.45

azure-identity

>=1.3.1

azure-keyvault

>=4.1.0

azure-kusto-data

>=0.0.43,<0.1

azure-mgmt-containerinstance

>=1.5.0,<2.0

azure-mgmt-datafactory

>=1.0.0,<2.0

azure-mgmt-datalake-store

>=0.5.0

azure-mgmt-resource

>=2.2.0

azure-storage-blob

>=12.7.0,<12.9.0

azure-storage-common

>=2.1.0

azure-storage-file

>=2.1.0

Cross provider package dependencies

Those are dependencies that might be needed in order to use all the features of the package. You need to install the specified provider packages in order to use them.

You can install such cross-provider dependencies when installing from PyPI. For example:

pip install apache-airflow-providers-microsoft-azure[google]

Dependent package

Extra

apache-airflow-providers-google

google

apache-airflow-providers-oracle

oracle

Changelog

3.2.0

Features

  • Rename AzureDataLakeStorage to ADLS (#18493)

  • Creating ADF pipeline run operator, sensor + ADF custom conn fields (#17885)

  • Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator (#18168)

  • Rename FileToWasbOperator to LocalFilesystemToWasbOperator (#18109)

Bug Fixes

  • Fixed wasb hook attempting to create container when getting a blob client (#18287)

  • Removing redundant relabeling of password conn field (#18386)

  • Proper handling of Account URL custom conn field in AzureBatchHook (#18456)

Main

Changes in operators names and import paths are listed in the following table This is a backward compatible change. Deprecated operators will be removed in the next major release.

Deprecated operator name

New operator name

Deprecated path

New path

AzureDataLakeStorageListOperator

ADLSListOperator

airflow.providers.microsoft.azure.operators.adls_list

airflow.providers.microsoft.azure.operators.adls

AzureDataLakeStorageDeleteOperator

ADLSDeleteOperator

airflow.providers.microsoft.azure.operators.adls_delete

airflow.providers.microsoft.azure.operators.adls

3.1.1

Misc

  • Optimise connection importing for Airflow 2.2.0

  • Adds secrets backend/logging/auth information to provider yaml (#17625)

3.1.0

Features

  • Add support for managed identity in WASB hook (#16628)

  • Reduce log messages for happy path (#16626)

Bug Fixes

  • Fix multiple issues in Microsoft AzureContainerInstancesOperator (#15634)

3.0.0

Breaking changes

  • Auto-apply apply_default decorator (#15667)

  • Fixes AzureFileShare connection extras (#16388)

Azure Container Volume and Azure File Share have now dedicated connection types with editable UI fields. You should not use Wasb connection type any more for those connections. Names of connection ids for those hooks/operators were changed to reflect that.

Features

  • add oracle connection link (#15632)

  • Add delimiter argument to WasbHook delete_file method (#15637)

Bug Fixes

  • Fix colon spacing in ``AzureDataExplorerHook docstring (#15841)``

  • fix wasb remote logging when blob already exists (#16280)

2.0.0

Breaking changes

  • Removes unnecessary AzureContainerInstance connection type (#15514)

This change removes azure_container_instance_default connection type and replaces it with the azure_default. The problem was that AzureContainerInstance was not needed as it was exactly the same as the plain “azure” connection, however it’s presence caused duplication in the field names used in the UI editor for connections and unnecessary warnings generated. This version uses plain Azure Hook and connection also for Azure Container Instance. If you already have azure_container_instance_default connection created in your DB, it will continue to work, but the first time you edit it with the UI you will have to change it’s type to azure_default.

Features

  • Add dynamic connection fields to Azure Connection (#15159)

Bug fixes

  • Fix 'logging.exception' redundancy (#14823)

1.3.0

Features

  • A bunch of template_fields_renderers additions (#15130)

Bug fixes

  • Fix attributes for AzureDataFactory hook (#14704)

1.2.0

Features

  • Add Azure Data Factory hook (#11015)

Bug fixes

  • BugFix: Fix remote log in azure storage blob displays in one line (#14313)

  • Fix AzureDataFactoryHook failing to instantiate its connection (#14565)

1.1.0

Updated documentation and readme files.

Features

  • Upgrade azure blob to v12 (#12188)

  • Fix Azure Data Explorer Operator (#13520)

  • add AzureDatalakeStorageDeleteOperator (#13206)

1.0.0

Initial version of the provider.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distribution

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page