Description
An interactive parallelization framework which is especially useful in configuring data science workload distribution. Eg. supports openMIP, MPI runs on High Performance Clusters
Interactive Parallel Computing with IPython alternatives and similar packages
Based on the "Science and Data Analysis" category.
Alternatively, view Interactive Parallel Computing with IPython alternatives based on common mentions on social networks and blogs.
-
Pandas
Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
#<Sawyer::Resource:0x00007f547e829e00>
A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites. -
-
-
bcbio-nextgen
Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis -
-
-
-
-
-
-
-
PatZilla
PatZilla is a modular patent information research platform and data integration toolkit with a modern user interface and access to multiple data sources. -
-
-
-
-
ElasticBatch
Elasticsearch tool for easily collecting and batch inserting Python data and pandas DataFrames
Judoscale - Save 47% on cloud hosting with autoscaling that just works
* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.
Do you think we are missing an alternative of Interactive Parallel Computing with IPython or a related project?
README
Interactive Parallel Computing with IPython
IPython Parallel (ipyparallel) is a Python package and collection of CLI scripts for controlling clusters of IPython processes, built on the Jupyter protocol.
IPython Parallel provides the following commands:
- ipcluster - start/stop/list clusters
- ipcontroller - start a controller
- ipengine - start an engine
Install
Install IPython Parallel:
pip install ipyparallel
This will install and enable the IPython Parallel extensions for Jupyter Notebook and (as of 7.0) Jupyter Lab 3.0.
Run
Start a cluster:
ipcluster start
Use it from Python:
import os
import ipyparallel as ipp
cluster = ipp.Cluster(n=4)
with cluster as rc:
ar = rc[:].apply_async(os.getpid)
pid_map = ar.get_dict()
See the docs for more info.