KQ changelog

Changelog History

• v2.0.1 Changes

  August 31, 2020

  • ➕ Added official support for Python 3.8.
  • ⬆️ Bumped up dependency versions (kafka-python>=2.0.0 and dill>=0.3.2).
  • 👷 Flush producer to ensure jobs are sent to the topic immediately.

• v2.0.0 Changes

  June 04, 2018

  • ⬇️ Dropped support for Python 2.7 and Python 3.4. This was to keep the code cleaner, reduce future development/maintenance workload for myself, and be ready for Python 2.x phaseout (albeit a little early).
  • Completely overhauled the library and improved the API. Changes are not backward compatible. Some notable changes include:
    • Ability to inject your own kafka-python producers and consumers into KQ queues and workers respectively. See here for an example.
    • Ability to specify the timeout, key and partition when enqueueing a job with new Queue.using method.
    • Ability to inject your own serializer/deserializer. See here for more details.
    • Removed the kq command line tool. Users should define an executable Python module themselves (see example here). Main reason for this was that the command line tool was not very useful: if users wanted to configure their consumers and callback functions, they had to define a configuration module and import it using the kq command line tool. This only added more room for user errors and hard-to-debug exceptions during the import, without providing much benefits over simply defining a module and calling Worker.start directly.
  • 📚 Please check out the documentation for new usage examples and API specifications.

• v1.3.2 Changes

  November 09, 2017

  • ➕ Added contributing page to the documentation.

• v1.3.1 Changes

  November 02, 2017

  • ➕ Added tests_require and license information in setup.py.

• v1.3.0 Changes

  May 10, 2017

  • ➕ Added parameter offset_policy to the Worker class to allow the user to control the Kafka consumer offset reset policy (earliest vs latest).
  • ➕ Added option offset to the kq command-line tool.
  • ➕ Added documentation on logging.

• v1.2.0 Changes

  May 09, 2017

  • ➕ Added parameter proc_ttl to the Worker class to allow re-spawning of worker process pool.
  • ➕ Added option proc-ttl to the kq command-line tool.

• v1.1.0 Changes

  December 13, 2016

  • Added Queue.enqueue_with_key(key, func, *args, **kwargs) method for adding jobs with specific Kafka message keys for sequential processing.
  • 🛠 Fixed a bug where consumer offsets are committed when jobs fail.
  • 📚 Minor documentation updates.

• v1.0.1 Changes

  December 04, 2016

  • Kafka consumer offsets are now committed each time a record is read.
  • 🛠 Fixed a couple of minors issues in the documentation and readme.