jx-bigquery - JSON Expressions for BigQuery
Project description
jx-bigquery
JSON Expressions for BigQuery
Configuration
{
"table": "my_table_name",
"top_level_fields": {},
"partition": {
"field": "submit_time",
"expire": "2year"
},
"id": {
"field": "id",
"version": "last_modified"
},
"cluster": [
"id",
"last_modified"
],
"schema": {
"id": "integer",
"submit_time": "time",
"last_modified": "time"
},
"sharded": true,
"account_info": {
"private_key_id": {
"$ref": "env://BIGQUERY_PRIVATE_KEY_ID"
},
"private_key": {
"$ref": "env://BIGQUERY_PRIVATE_KEY"
},
"type": "service_account",
"project_id": "my-project-id",
"client_email": "me@my_project.iam.gserviceaccount.com",
"client_id": "12345",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test-treeherder-extract%40moz-fx-dev-ekyle-treeherder.iam.gserviceaccount.com"
}
}
table
- Any name you wish to give to this table, thetop_level_fields
- Map frompartition
- "field": "submit_time", "expire": "2year" },id
- "field": "id", "version": "last_modified" },cluster
- "id", "last_modified" ],schema
- "id": "integer", "submit_time": "time", "last_modified": "time" },sharded": true
-account_info
- The content of the application
Usage
Setup Dataset
with an applicaiotn name
destination = bigquery.Dataset(
dataset=application_name,
kwargs=settings
).get_or_create_table(settings.destination)
destination.extend(documents)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
jx-bigquery-3.34.20028.tar.gz
(27.4 kB
view hashes)