Python SCOOP (software)
SCOOP (Scalable Concurrent Operations in Python) is a distributed task module allowing concurrent parallel programming on various environments, from heterogeneous grids to supercomputers.
It uses ØMQ and the Greenlet package as building blocks to encapsulate and distribute tasks (named Future in its nomenclature) between processes and/or systems. Its philosophy is based on simplicity and its interface is based on the PEP-3148 which is standard from Python 3.2.
The SCOOP framework is mainly targeted for science and academic applications such as researchers using HPC resources and students learning parallelism and distributed computing.
History
SCOOP is an iterative step over the DTM module of the deap framework. While the latter is based upon MPI as its communication library, SCOOP is based upon ØMQ. Both are similar in API and aimed to be offer simple ways to distribute scientific computing.
Network topology
SCOOP uses the Broker Architecture to distribute its Futures. It is based on a central element, called the Broker, that dispatches work to its workers. The main difference between this pattern and a Master/slave topology reside in the Future origin. In the Broker architecture, the Futures emanate from a worker, which is located on the periphery of the topology, instead of the master in the Master/slave architecture. This allows higher reliability in regard to worker faults and generally better performances due to the generic function of the Broker. Since he doesn't need to serialize nor deserialize any Future to route them around the grid, his workload consists of networking or interprocess I/O and almost no CPU processing time. This lowers the bottleneck of the Broker topology.
The Broker architecture won't stress the networking fabric and elements as much as a totally distributed topology since there is only one connection required on every worker.
Performances
As of 0.5.3, the framework scales pretty well. The framework aims to support scaling up to cluster and supercomputer systems wihle keeping in mind the simpler heterogeneous networked computer systems readily available.