Skip to main content

Macro recording and metaprogramming in Python

Project description

macro-kit

macro-kit is a package for efficient macro recording and metaprogramming in Python using abstract syntax tree (AST).

The design of AST in this package is strongly inspired by Julia metaprogramming. Similar methods are also implemented in builtin ast module but macro-kit is more focused on the macro generation and customization.

Installation

  • use pip
pip install macro-kit
  • from source
pip install git+https://github.com/hanjinliu/macro-kit

Examples

  1. Define a macro-recordable function
from macrokit import Macro, Expr, Symbol
macro = Macro()

@macro.record
def str_add(a, b):
    return str(a) + str(b)

val0 = str_add(1, 2)
val1 = str_add(val0, "xyz")
macro
[Out]
var0x24fdc2d1530 = str_add(1, 2)
var0x24fdc211df0 = str_add(var0x24fdc2d1530, 'xyz')

Use format method to rename variable names.

# substitute identifiers of variables
# var0x24fdc2d1530 -> x
macro.format([(val0, "x")]) 
[Out]
x = str_add(1, 2)
var0x24fdc211df0 = str_add(x, 'xyz')

format also support substitution with more complicated expressions.

# substitute to _dict["key"]
expr = Expr(head="getitem", args=[Symbol("_dict"), "key"])
macro.format([(val0, expr)])
[Out]
_dict['key'] = str_add(1, 2)
var0x24fdc211df0 = str_add(_dict['key'], 'xyz')
  1. Record class
macro = Macro()

@macro.record
class C:
    def __init__(self, val: int):
        self.value = val

    @property
    def value(self):
        return self._value

    @value.setter
    def value(self, new_value: int):
        if not isinstance(new_value, int):
            raise TypeError("new_value must be an integer.")
        self._value = new_value

    def show(self):
        print(self._value)

c = C(1)
c.value = 5
c.value = -10
c.show()
[Out]
-10

Note that value assignments are not recorded in duplicate.

macro.format([(c, "ins")])
[Out]
ins = C(1)
ins.value = -10     
var0x7ffed09d2cd8 = ins.show()

eval can evaluate macro.

macro.eval({"C": C})
[Out]
-10
  1. Record module
import numpy as np
macro = Macro()
np = macro.record(np) # macro-recordable numpy

arr = np.random.random(30)
mean = np.mean(arr)

macro
[Out]
var0x2a0a2864090 = numpy.random.random(30)
var0x2a0a40daef0 = numpy.mean(var0x2a0a2864090)
from dask import array as da
dask_macro = macro.format([(np, "da")])
dask_macro
[Out]
var0x2a0a2864090 = da.random.random(30)
var0x2a0a40daef0 = da.mean(var0x2a0a2864090)
output = {}
dask_macro.eval({"da": da}, output)
output
[Out]
{:da: <module 'dask.array' from 'C:\\...\\__init__.py'>,
 :var0x2a0a2864090: dask.array<random_sample, shape=(30,), dtype=float64, chunksize=(30,), chunktype=numpy.ndarray>,
 :var0x2a0a40daef0: dask.array<mean_agg-aggregate, shape=(), dtype=float64, chunksize=(), chunktype=numpy.ndarray>}
  1. String parsing

parse calls ast.parse inside so that you can safely make Expr from string.

from macrokit import parse

expr = parse("result = f(0, l[2:8])")
expr
[Out]
:(result = f(0, l[slice(2, 8, None)])
print(expr.dump())
[Out]
head: assign
args:
 0: result
 1: head: call
    args:
     0: f
     1: 0
     2: head: getitem
        args:
         0: l
         1: slice(2, 8, None)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

macro-kit-0.3.0.tar.gz (17.2 kB view hashes)

Uploaded Source

Built Distribution

macro_kit-0.3.0-py3-none-any.whl (19.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page