Back-ported apache-airflow-backport-providers-apache-spark package for Airflow 1.10.*
Project description
Package apache-airflow-backport-providers-apache-spark
Release: 2020.5.20
Backport package
This is a backport providers package for apache.spark provider. All classes for this provider package
are in airflow.providers.apache.spark python package.
Installation
You can install this package on top of an existing airflow 1.10.* installation via
pip install apache-airflow-backport-providers-apache-spark
Compatibility
For full compatibility and test status of the backport packages check Airflow Backport Package Compatibility
PIP requirements
| PIP package | Version required |
|---|---|
| pyspark |
Provider class summary
All classes in Airflow 2.0 are in airflow.providers.apache.spark package.
Operators
Moved operators
Airflow 2.0 operators: airflow.providers.apache.spark package |
Airflow 1.10.* previous location (usually airflow.contrib) |
|---|---|
| operators.spark_jdbc.SparkJDBCOperator | contrib.operators.spark_jdbc_operator.SparkJDBCOperator |
| operators.spark_sql.SparkSqlOperator | contrib.operators.spark_sql_operator.SparkSqlOperator |
| operators.spark_submit.SparkSubmitOperator | contrib.operators.spark_submit_operator.SparkSubmitOperator |
Hooks
Moved hooks
Airflow 2.0 hooks: airflow.providers.apache.spark package |
Airflow 1.10.* previous location (usually airflow.contrib) |
|---|---|
| hooks.spark_jdbc.SparkJDBCHook | contrib.hooks.spark_jdbc_hook.SparkJDBCHook |
| hooks.spark_sql.SparkSqlHook | contrib.hooks.spark_sql_hook.SparkSqlHook |
| hooks.spark_submit.SparkSubmitHook | contrib.hooks.spark_submit_hook.SparkSubmitHook |
Releases
Release 2020.5.20
| Commit | Committed | Subject |
|---|---|---|
| f3521fb0e | 2020-05-16 | Regenerate readme files for backport package release (#8886) |
| 92585ca4c | 2020-05-15 | Added automated release notes generation for backport operators (#8807) |
| 7506c73f1 | 2020-05-10 | Add default conf parameter to Spark JDBC Hook (#8787) |
| 487b5cc50 | 2020-05-06 | Add guide for Apache Spark operators (#8305) |
| 87969a350 | 2020-04-09 | [AIRFLOW-6515] Change Log Levels from Info/Warn to Error (#8170) |
| be1451b0e | 2020-04-02 | [AIRFLOW-7026] Improve SparkSqlHook's error message (#7749) |
| 4bde99f13 | 2020-03-23 | Make airflow/providers pylint compatible (#7802) |
| 7e6372a68 | 2020-03-23 | Add call to Super call in apache providers (#7820) |
| 2327aa5a2 | 2020-03-12 | [AIRFLOW-7025] Fix SparkSqlHook.run_query to handle its parameter properly (#7677) |
| 024b4bf96 | 2020-03-10 | [AIRFLOW-7024] Add the verbose parameter support to SparkSqlOperator (#7676) |
| b59042b5a | 2020-02-28 | [AIRFLOW-6949] Respect explicit spark.kubernetes.namespace conf to SparkSubmitOperator (#7575) |
| 97a429f9d | 2020-02-02 | [AIRFLOW-6714] Remove magic comments about UTF-8 (#7338) |
| 0481b9a95 | 2020-01-12 | [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package (#7142) |
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file apache-airflow-backport-providers-apache-spark-2020.5.20rc1.tar.gz.
File metadata
- Download URL: apache-airflow-backport-providers-apache-spark-2020.5.20rc1.tar.gz
- Upload date:
- Size: 119.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.6.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2692ef464522f454ca5e8676c06eda49ed1a1fec9bb0f94e3a6df4e050ec62c5
|
|
| MD5 |
1b776bf92b705b98e807c81ed74be677
|
|
| BLAKE2b-256 |
9bb130c8599dcdc91304f3aa2c852c30858942c9788e3671b0f4f34abc3ca95b
|
File details
Details for the file apache_airflow_backport_providers_apache_spark-2020.5.20rc1-py2.py3-none-any.whl.
File metadata
- Download URL: apache_airflow_backport_providers_apache_spark-2020.5.20rc1-py2.py3-none-any.whl
- Upload date:
- Size: 59.5 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.6.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3d75a2d71d02f807aa61a9a1ea6af74cebea2097f578c50539b229b9e1162828
|
|
| MD5 |
15feb5a06031fa1ed6eb4cad74fa2295
|
|
| BLAKE2b-256 |
e90e7404927b7bfe8a8e5599cb00c3fe888127564242e6f1acc335721a54a8c6
|
