Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.9.2-cp314-cp314-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.9.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.8 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.9.2-cp314-cp314-macosx_15_0_arm64.whl (17.5 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.9.2-cp313-cp313-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.9.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.8 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.9.2-cp313-cp313-macosx_15_0_arm64.whl (17.6 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.9.2-cp312-cp312-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.9.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.8 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.9.2-cp312-cp312-macosx_15_0_arm64.whl (17.6 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.9.2-cp311-cp311-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.9.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.8 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.9.2-cp311-cp311-macosx_15_0_arm64.whl (17.6 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.9.2-cp310-cp310-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.9.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.9.2-cp310-cp310-macosx_15_0_arm64.whl (17.6 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.9.2-cp39-cp39-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.9.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.9.2-cp39-cp39-macosx_15_0_arm64.whl (17.6 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.9.2-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.2-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.2-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 b2892b91f27d70211ad5f7f126522fd90820ae4bb85b95e38e63c09da2266669
MD5 601b1523b3ad2513b9cad2b5664d0042
BLAKE2b-256 44cf48eac3088c8f973928f1001e0555bba13ea38bd2a3bacd626a3d93ea4b93

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 b6d8c038a88142ff1941868d15238f7050ca1cb99d8bc62fe557d04c116b2b43
MD5 6858b54852f953a87da8288f895f6ace
BLAKE2b-256 32b372a3eb5045576dffc111ba6e210b955fab783772050c8adcbe340c4e9225

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 48952bd25b898d34a4d6bea44ec904ad26cc6e391064dedadf55314985d5e6ab
MD5 77cc062ce8a74155ed88171910275f77
BLAKE2b-256 69eb921dc2542644511ec8364a583775c118c866bf2eaaee4c585a5c68cc4d62

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.2-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.2-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 0b9d47ca17070d2a9a1be3128b2c81921af9e3a6236403409d92d4f4c77349e7
MD5 1302ea7cee3bb5a4937b2b02f876721b
BLAKE2b-256 9279bea112fe6fa125e85fa8e007d7018474359d202e3dfa8a65b428a297a98c

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 16189177db5e0d1c3aa99f49ebbc7bc6caddc9adb5a554d3198b884da568dc35
MD5 7b50a2d9664641da0c1de62a17da6994
BLAKE2b-256 e6b9a6b1093dc19b1f42c7c836fac2158c31008ceab13fd4f8f41589f01ec702

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 072426f912b84dc79f310eeeeda38c2f363145e3aad71b2351ecf9811afc3397
MD5 2e1cbe2c2e504cfa29fe8cdf4db03131
BLAKE2b-256 c18d45fc1289399fd57046eb374898f9745b253c236b2362e794e3010cf106c3

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.2-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.2-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 f2ef0078e13c7181ecfc385e3c45eca5deb572fee70484fbfd3e3679d74995dc
MD5 728a2e42e8bb8bb1fa2c4ce6b5734aa1
BLAKE2b-256 c8e8b3776d1e4f2575cd3c6a3a6e16cbd298a6a9f090b3ab78f5e7e440b1a4d3

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 3755d7fe03f21058d3e1af5560bdaf0a9a0f66d390d4c6045fc23c8e28ff441e
MD5 cd39ee9e3d8855ba1301af9dbfb9a906
BLAKE2b-256 4ffffd34802b6e5842be6970cd8918e4a56129c2fbf35616b877e04b16007d97

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 a7f916dd25480bdda2d1c3d1469281a97cbc411b61e36f55b1076edba7428cbe
MD5 f0ee490a9b537a8513175a5178db0446
BLAKE2b-256 b50b532ec06d551ed18623fbb540bfe14272ba7fb0664c486dc0cfa4204e4d98

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.2-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.2-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 0a3d96ad04ab5da7a0e7299a957b913f408567c57f658b5ed425a8094c874202
MD5 6c13d1067b46da2da993c5446e61338c
BLAKE2b-256 07e750743efe5199a6a43250738e75f4a33af7f08c2ad29a80bf855ce1df4cb4

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 b407605afda6646e1dc9cbd5bf9f903a52c7c07ce3ec5c219b23b19bed2a9d5d
MD5 469f6a28e01deab53f195f1d759c456f
BLAKE2b-256 2f7516bdbdcbf81c362059cfe8adce17c61a1f7afd3a2db2c8ccc23d93b9c309

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 dea647aa73bfece7419c14af33a7d9a71627b0f095b1924b3cf221a490accb9c
MD5 d10b5f931b5fe9dedd29dbc6c17651e4
BLAKE2b-256 491e742e68360f62fd33840c9bdbc4ac24d0eae076936fc5f3d2c703dd094ce0

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.2-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.2-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 fefcdb989b3d9ea57ca5cbd98c728950b00c49c84a9462075e597f3d61943e04
MD5 46df4db57a1c3fb5d02b1554f03922d3
BLAKE2b-256 7b920a784d5d26190bb805541c053dee0b15975435898b8685891c442162d18b

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 d0da9ae84d0eb69e6ea99cf2e25d1824bd1ed1445ce24505c7eb34c9aa31df99
MD5 7745b468cb1e3ed231ab38af7b4f5095
BLAKE2b-256 809abcd8ae647420b91bd9e3b059e3bfaee02782dfc677fd3477d455fed56bcc

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 d12d91ab158c291825a8b9267b7ae547fd0214e22571132d00a38988bb0b3a69
MD5 8baa73e319bebec761369e7864b11aff
BLAKE2b-256 162a5c9e90a5973ff4ad0f9a9a1a2dd3f88f7209414ef6a3081a0aefa79f529d

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.2-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.2-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 0155e45c41b9790149a2fbdca5b81e598800afaf14dbd0641e9cb0920ee4cb10
MD5 932dc4eb3a17d279e5ef1a5c18a35a64
BLAKE2b-256 71de43dc2108d79e496abbbb0f215270fbb3e3741120555bf471a260c6628946

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 8f7f6d4a47fc52657b47edb966d816bbd61e86dcd38740c7c39f5569f393eed6
MD5 fbee3ae78e513fc617578fe5c6c0ed9b
BLAKE2b-256 e5b79b7227d6e76a6c3fae606ba870a3f752ed4586edb9fb89531aa069b69051

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.2-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.2-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 7b5b4ffd9544909b8b29629bacd6109cc901c5c4bea26cb65978668f88a412f0
MD5 0851d77e2ffa96bdd65956bfc480f3bf
BLAKE2b-256 5169342de8110fd3b31584362f6741b85b1a20afdb31ecf9a7a80ba8cdced189

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page