Parallel MPIPoolExecutor implementing the concurrent.futures interface
Project description
About
zwembad
offers an MPIPoolExecutor
class, an implementation of the
concurrent.futures.Executor
class of the standard library.
Example usage
from zwembad import MPIPoolExecutor
from mpi4py import MPI
pool = MPIPoolExecutor()
def menial_task(x):
return x ** MPI.COMM_WORLD.Get_rank()
fs = [pool.submit(menial_task, i) for i in range(100)]
print([f.result() for f in fs])
You'll see that some results will have exponentiated either by 1, 2, ..., n depending on which worker they were sent to.
Different from mpi4py
s own MPIPoolExecutor
zwembad is designed to function without
MPI.Spawn()
for cases where this approach isn't feasible, like supercomputers where
MPI.Spawn
is usually deliberatly not implemented (for example CrayMPI).
Therefor the pool can only use MPI processes that are spawned when the MPI world is
initialised and must be run from the command line using an MPI helper such as mpirun
,
mpiexec
or SLURM's srun
:
$ mpirun -n 4 python example.py
Note: Currently zwembad.MPIPoolExecutor
only implements a submit
method and lacks a
parallel map
method.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for zwembad-1.0.0b0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d9595464e7004c300ac4c471781e2bfb8f84ba7d2ef5346ddd2da40c51ecac0e |
|
MD5 | 4c3a29e11285a37921a8ea13e87b0d1e |
|
BLAKE2b-256 | b889f3840bc996b1fd6ab14fcb962ef62264ac2451ddb1626d54efb45f3c04b2 |