Parallels is a literary powered library. It provides an async API that mimics concurrent.futures, with support for task-graph executors.
These notebooks can be viewed using nbviewer until the documentation generator is complete.
In Python there are several standard APIs for interacting with executors
. AsyncIO has the run_in_executor
API, concurrent.futures
has the Executor
API, and other
libraries like Dask and Ray have equivalent approaches. concurrent.futures
is often available within other libraries, but its reduced features-set prevents the underlying
library from implementing useful optimisations like Dask's deferred computation or task graph building.
Parallels implements a standard Executor
interface which defines a synchronous submit
method, and an asynchronous retrieve
method.
These methods operate upon value-less asyncio.Future
handles which yield True
upon task success, and raise an Exception
otherwise.
The Dask and Ray implementations accept these handles as arguments to future submit()
calls, which can be used to build task graphs
and avoid copying data to the local machine.