« Changelog History


dpark v0.5.0 Release Notes

Release Date: 2018-07-27 // about 5 years ago

  • API change

    • ✂ Remove module-level api like dpark.textFile.
    • 👌 Support Streaming shuffle and Disk shuffle (Experimental, compatible).

    🛠 Fixes

    • 🐛 Bug when parsing mfs chunk info.

    👌 Improvement

    • 👍 Better broadcast impl using shared memory for tasks on the same slave to reduce memory cost.
    • 👍 Better offer-matching logic for MesosScheduler which remember bad slaves.
    • 💅 Refactor: style and layout.

    🆕 New Feature

    • Multi segment dump to save memory.
    • Gather statics for stage.
    • 👌 Support run tests/test_rdd on mesos.
    • ➕ Add colorful progress bar for dpark.
    • 👌 Support mesos role.
    • 👌 Support multi named mesos master in conf.
    • Loghub for admin.

Previous changes from v0.4.2

    • 👌 Support Python3 & PyPy
    • 👌 Support MooseFS 3.x & refactor on file-system interface