Skip to main content

High Level Expressions for Dask

Project description

Dask Expressions

Dask DataFrames with query optimization.

This is a rewrite of Dask DataFrame that includes query optimization and generally improved organization.

More in our blog posts:

Example

import dask_expr as dx

df = dx.datasets.timeseries()
df.head()

df.groupby("name").x.mean().compute()

Query Representation

Dask-expr encodes user code in an expression tree:

>>> df.x.mean().pprint()

Mean:
  Projection: columns='x'
    Timeseries: seed=1896674884

This expression tree will be optimized and modified before execution:

>>> df.x.mean().optimize().pprint()

Div:
  Sum:
    Fused(375f9):
    | Projection: columns='x'
    |   Timeseries: dtypes={'x': <class 'float'>} seed=1896674884
  Count:
    Fused(375f9):
    | Projection: columns='x'
    |   Timeseries: dtypes={'x': <class 'float'>} seed=1896674884

Stability

This is the default backend for dask.DataFrame since version 2024.3.0.

API Coverage

Dask-Expr covers almost everything of the Dask DataFrame API. The only missing features are:

  • named GroupBy Aggregations

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dask_expr-1.1.1.tar.gz (188.5 kB view hashes)

Uploaded Source

Built Distribution

dask_expr-1.1.1-py3-none-any.whl (205.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page