Skip to main content

Open Data Discovery Resource Name Generator

Project description

PyPI version

Open Data Discovery Resource Name Generator

Helps generate oddrn for data sources.

Requirements

  • Python >= 3.7

Installation

poetry add oddrn-generator
# or
pip install oddrn-generator

Usage and configuration

Available generators

DataSource Generator class name
cassandra CassandraGenerator
postgresql PostgresqlGenerator
mysql MysqlGenerator
glue GlueGenerator
s3 S3Generator
kafka KafkaGenerator
kafkaconnect KafkaConnectGenerator
snowflake SnowflakeGenerator
airflow AirflowGenerator
hive HiveGenerator
dynamodb DynamodbGenerator
odbc OdbcGenerator
mssql MssqlGenerator
oracle OracleGenerator
redshift RedshiftGenerator
clickhouse ClickHouseGenerator
athena AthenaGenerator
quicksight QuicksightGenerator
dbt DbtGenerator
prefect PrefectGenerator
tableau TableauGenerator
neo4j Neo4jGenerator
mongodb MongoGenerator
vertica VerticaGenerator
CubeJs CubeJsGenerator
superset SupersetGenerator
Presto PrestoGenerator
Trino TrinoGenerator

Generator properties

  • base_oddrn - Get base oddrn (without path)
  • available_paths - Get all available path of generator

Generator methods

  • get_oddrn_by_path(path_name, new_value=None) - Get oddrn string by path. You also can set value for this path using ' new_value' param
  • set_oddrn_paths(**kwargs) - Set or update values of oddrn path
  • get_data_source_oddrn() - Get data source oddrn

Generator parameters:

  • host_settings: str - optional. Hostname configuration
  • cloud_settings: dict - optional. Cloud configuration
  • **kwargs - path's name and values

Example usage

# postgresql
from oddrn_generator import PostgresqlGenerator

oddrn_gen = PostgresqlGenerator(
    host_settings='my.host.com:5432',
    schemas='schema_name', databases='database_name', tables='table_name'
)

oddrn_gen.base_oddrn
# //postgresql/host/my.host.com:5432
oddrn_gen.available_paths
# ('schemas', 'databases', 'tables', 'columns')

oddrn_gen.get_data_source_oddrn()
# //postgresql/host/my.host.com:5432/schemas/schema_name/databases/database_name

oddrn_gen.get_oddrn_by_path("schemas")
# //postgresql/host/my.host.com:5432/schemas/schema_name

oddrn_gen.get_oddrn_by_path("databases")
# //postgresql/host/my.host.com:5432/schemas/schema_name/databases/database_name

oddrn_gen.get_oddrn_by_path("tables")
# //postgresql/host/my.host.com:5432/schemas/schema_name/databases/database_name/tables/table_name

# you can set or change path:
oddrn_gen.set_oddrn_paths(tables='another_table_name', columns='new_column_name')
oddrn_gen.get_oddrn_by_path("columns")
# //postgresql/host/my.host.com:5432/schemas/schema_name/databases/database_name/tables/another_table_name/columns/new_column_name

# you can get path wih new values:
oddrn_gen.get_oddrn_by_path("columns", new_value="another_new_column_name")
# //postgresql/host/my.host.com:5432/schemas/schema_name/databases/database_name/tables/another_table_name/columns/another_new_column_name


# glue
from oddrn_generator import GlueGenerator

oddrn_gen = GlueGenerator(
    cloud_settings={'account': 'acc_id', 'region': 'reg_id'},
    databases='database_name', tables='table_name', columns='column_name',
    jobs='job_name', runs='run_name', owners='owner_name'
)

oddrn_gen.available_paths
# ('databases', 'tables', 'columns', 'owners', 'jobs', 'runs')

oddrn_gen.get_oddrn_by_path("databases")
# //glue/cloud/aws/account/acc_id/region/reg_id/databases/database_name

oddrn_gen.get_oddrn_by_path("tables")
# //glue/cloud/aws/account/acc_id/region/reg_id/databases/database_name/tables/table_name'

oddrn_gen.get_oddrn_by_path("columns")
# //glue/cloud/aws/account/acc_id/region/reg_id/databases/database_name/tables/table_name/columns/column_name

oddrn_gen.get_oddrn_by_path("jobs")
# //glue/cloud/aws/account/acc_id/region/reg_id/jobs/job_name

oddrn_gen.get_oddrn_by_path("runs")
# //glue/cloud/aws/account/acc_id/region/reg_id/jobs/job_name/runs/run_name

oddrn_gen.get_oddrn_by_path("owners")
# //glue/cloud/aws/account/acc_id/region/reg_id/owners/owner_name

Exceptions

  • WrongPathOrderException - raises when trying set path that depends on another path
from oddrn_generator import PostgresqlGenerator

oddrn_gen = PostgresqlGenerator(
    host_settings='my.host.com:5432',
    schemas='schema_name', databases='database_name',
    columns='column_without_table'
)
# WrongPathOrderException: 'columns' can not be without 'tables' attribute
  • EmptyPathValueException - raises when trying to get a path that is not set up
from oddrn_generator import PostgresqlGenerator

oddrn_gen = PostgresqlGenerator(
    host_settings='my.host.com:5432', schemas='schema_name', databases='database_name',
)
oddrn_gen.get_oddrn_by_path("tables")

# EmptyPathValueException: Path 'tables' is not set up
  • PathDoestExistException - raises when trying to get not existing oddrn path
from oddrn_generator import PostgresqlGenerator

oddrn_gen = PostgresqlGenerator(
    host_settings='my.host.com:5432', schemas='schema_name', databases='database_name',
)
oddrn_gen.get_oddrn_by_path("jobs")

# PathDoestExistException: Path 'jobs' doesn't exist in generator

Development

#Install dependencies
poetry install

#Activate shell
poetry shell

# Run tests
python run pytest

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

oddrn-generator-0.1.50.tar.gz (13.7 kB view hashes)

Uploaded Source

Built Distribution

oddrn_generator-0.1.50-py3-none-any.whl (17.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page