If you're using Dask with tasks that use a lot of memory, RAM is your bottleneck for parallelism. That means you want to know how much memory each task uses:

1. So you can set the highest parallelism level (process or threads) for each machine, given available to RAM. 2. In order to know where to focus memory optimization efforts.

dask-memusage is an MIT-licensed statistical memory profiler for Dask's Distributed scheduler that can help you with both these problems.

dask-memusage polls your processes for memory usage and records the minimum and maximum usage in a CSV.

Programming language: Python

dask-memusage alternatives and related packages

Based on the "Science and Data Analysis" category

Do you think we are missing an alternative of dask-memusage or a related project?

Add another 'Science and Data Analysis' Package

dask-memusage Recommendations

There are no recommendations yet. Be the first to promote dask-memusage!

Have you used dask-memusage? Share your experience. Write a short recommendation and dask-memusage, you and your project will be promoted on Awesome Python.
Recommend dask-memusage

Recently added dask-memusage resources

Do you know of a usefull tutorial, book or news relevant to dask-memusage?
Be the first to add one!