MPI Pool similar to futures
Project description
About
mpipool
offers a Pool
class similar multiprocessing.Pool
from the standard library.
mpipool
uses MPIPool
implementation of
schwimmbad library
and circumvents some of its limitations:
- A series of
mpipool.Pool.map
calls do not crash, - In case a worker raises an exception, the MPI tasks are shut down properly so that the full program halts and does not hang.
Example usage
from mpipool import Pool
def add(a, b):
return a + b
p = Pool()
sums = p.map(add, [(ai, bi) for ai in range(10) for bi in range(10)])
assert len(sums) == 100
assert sums[0] == 0
assert sums[-1] == 18
sums = p.map(add, [(ai, bi) for ai in range(10) for bi in range(10)])
assert len(sums) == 100
assert sums[0] == 0
assert sums[-1] == 18
The program must be run on the commandline like:
$ mpirun -n 4 python example.py
Currently mpipool.Pool
only implements a map
method.
Contrary to the MPIPool
implementation of
schwimmbad
the statements after pool = Pool()
are only executed
by the MPI process with rank 0
.
Credits
mpipool
uses of the MPIPool
implementation of
schwimmbad library.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for zwembad-0.0.1a0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce98dca3da2d9d4871023ff086c6c88832ce06c387030af5c2d8f80967fd31ef |
|
MD5 | ea8b62e9f4665c342bd8aac6823eab9d |
|
BLAKE2b-256 | d3a2cd913be6f6dccf0be0c4a7c8e89a12896799086876588a809fb12d49f70f |