Interactive Parallel Computing with IPython

IPython Parallel: Interactive Parallel Computing in Python

Description

An interactive parallelization framework which is especially useful in configuring data science workload distribution. Eg. supports openMIP, MPI runs on High Performance Clusters

Code Quality Rank: L3

Programming language: Jupyter Notebook

License: GNU General Public License v3.0 or later

Tags: Science And Data Analysis     High Performance     Parallel Computing     Concurrency     Data Analysis     Jupyter     JupyterLab     3     Extensions     Prebuilt    

Latest version: v6.2.4

README

Interactive Parallel Computing with IPython

IPython Parallel (ipyparallel) is a Python package and collection of CLI scripts for controlling clusters of IPython processes, built on the Jupyter protocol.

IPython Parallel provides the following commands:

• ipcluster - start/stop/list clusters
• ipcontroller - start a controller
• ipengine - start an engine

Install

Install IPython Parallel:

pip install ipyparallel

This will install and enable the IPython Parallel extensions for Jupyter Notebook and (as of 7.0) Jupyter Lab 3.0.

Run

Start a cluster:

ipcluster start

Use it from Python:

import os
import ipyparallel as ipp

cluster = ipp.Cluster(n=4)
with cluster as rc:
    ar = rc[:].apply_async(os.getpid)
    pid_map = ar.get_dict()

See the docs for more info.