Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.8-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 338a50889b20ab813f782d5db3cebaa9da4b1f3b55bf542d8c5a75c87f59612c |
|
MD5 | 218d38f4055b867832ff878a2c4841ff |
|
BLAKE2b-256 | 93b4cbbb0b586da56f806e1157ff639f151328c3801560e1033ebe9dfe93e87d |
Hashes for asyncdb-2.7.8-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4befb3cf59b71ed61ceba8f21afac7050cb7c6777bc37bff2b66f0bfb04c422d |
|
MD5 | cca0b0dfb3a89dcd4a3d9580c2b12c22 |
|
BLAKE2b-256 | aa3275671e48a3643b6b36265515e10cd7981835cc8c59ed95d4eebbff3accc9 |
Hashes for asyncdb-2.7.8-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3e8f30d52742e9adc24121513005d04c61a507b4618c1c078fbc910d5df61868 |
|
MD5 | 43254bc0279e33c52ce5c89be8903de2 |
|
BLAKE2b-256 | ecc121252e90e9efad43386fa0fd8b85d8183734f3355e039740693622274ff6 |
Hashes for asyncdb-2.7.8-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 369990333641551ca8b41f38dbbaffd61f1c3b6205ebb1dbfabbffa8bc218168 |
|
MD5 | d7472cfb02ceee1966ed5c2fc3180a05 |
|
BLAKE2b-256 | 5dbb60e444364a0df0929aca3122505b74508034d3495a8ca37df932a51c92ae |
Hashes for asyncdb-2.7.8-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bea8b0f638de7056668d9c692af273a520a0368ca6f9ad4b499b7484ef8ef144 |
|
MD5 | 5b101ab41e29de996a4acdca03e0d870 |
|
BLAKE2b-256 | 7938ffa8cacc7ec04c10c2f8b7a78cef42491371fb291049138cab150f1d6b25 |
Hashes for asyncdb-2.7.8-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f72f754a3b9e29e597dbc49daf3aa1af7871db57745c526e058a36b9072e36e7 |
|
MD5 | 1c70d3596a677dd01c18555cb6f820d3 |
|
BLAKE2b-256 | 5c5da96b247afc2f3eb302ee61e852874a66808ca09cf01a48604885cb29b054 |
Hashes for asyncdb-2.7.8-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15123e924fb1c8b21a6955bfa9d3f9b1312dd14d9fe11d1815351029263801a2 |
|
MD5 | dfc538741ffcee59e12f0b8bf3905ca7 |
|
BLAKE2b-256 | bc91ade45eb9874676c7d9957ace2a60c14ce2874d42323132de18ea7fbbd1ae |
Hashes for asyncdb-2.7.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7a2bcf6e0df101454403c75cbe5e467ff5693eac4d02198c54e17e56670e7c63 |
|
MD5 | 7ec5b82364ea6ca8da40984bcca0860f |
|
BLAKE2b-256 | 2e9fc646a129b437015d3602cc2c8f803fb44106fcdc20ee681eca5ddac39c4c |
Hashes for asyncdb-2.7.8-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ab2e049cc50cd577b1eaef774503c9f6c583ac139a25aa39bbb457f728d37e2b |
|
MD5 | 944ae2c07e8256f494107fb8e974308b |
|
BLAKE2b-256 | 32f153147bd0a3ad877394f9bb655359540a8522a5f673db624f538f2428c1f2 |
Hashes for asyncdb-2.7.8-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d45ea24d29358a6991280a718aaafb1ecf9fddb605b7589fbff5202f6beae6c4 |
|
MD5 | b8309e9424c4f7e4a63b694adec6391f |
|
BLAKE2b-256 | eea01ad3f552aa23306b8b46dfd4f588ea1763188a33b12227680e17ed6e4492 |