All Versions
10
Latest Version
Avg Release Cycle
97 days
Latest Release
1704 days ago

Changelog History

  • v0.5.0 Changes

    July 27, 2018

    API change

    • ✂ Remove module-level api like dpark.textFile.
    • 👌 Support Streaming shuffle and Disk shuffle (Experimental, compatible).

    🛠 Fixes

    • 🐛 Bug when parsing mfs chunk info.

    👌 Improvement

    • 👍 Better broadcast impl using shared memory for tasks on the same slave to reduce memory cost.
    • 👍 Better offer-matching logic for MesosScheduler which remember bad slaves.
    • 💅 Refactor: style and layout.

    🆕 New Feature

    • Multi segment dump to save memory.
    • Gather statics for stage.
    • 👌 Support run tests/test_rdd on mesos.
    • ➕ Add colorful progress bar for dpark.
    • 👌 Support mesos role.
    • 👌 Support multi named mesos master in conf.
    • Loghub for admin.

  • v0.4.2 Changes

    March 19, 2018
    • 👌 Support Python3 & PyPy
    • 👌 Support MooseFS 3.x & refactor on file-system interface

  • v0.4.1 Changes

    March 09, 2017
    • ✨ Enhancement for the containerizer in DPark
    • 👉 Use broadcast when parallelize big dataset
    • 🛠 Fix missing line bug for bzip2 files
    • ➕ Add TopByKey in RDD
    • Other minor bugs

  • v0.4.0 Changes

    December 06, 2016
    • 🛠 Bugfix: deserialize error of old-style class.
    • 🔨 Refactor beansdb RDD
    • 💻 Web UI support for dpark
    • 👉 Use pymesos >= 0.2.0
    • Eager serialize values of ParallelCollection

  • v0.3.5

    October 27, 2016

  • v0.3.3

    September 20, 2016

  • v0.3.2

    June 07, 2016

  • v0.3.1

    May 25, 2016

  • v0.2.9

    March 15, 2016

  • v0.2.8

    March 04, 2016