All Versions
35
Latest Version
Avg Release Cycle
70 days
Latest Release
1304 days ago

Changelog History
Page 3

  • v1.2.2 Changes

    June 21, 2016

    🛠 Bugfixes

    • ⏱ Clarify timeout unit in KafkaProducer close and flush (ms7s PR 734)
    • Avoid busy poll during metadata refresh failure with retry_backoff_ms (dpkp PR 733)
    • ⏱ Check_version should scan nodes until version found or timeout (dpkp PR 731)
    • Fix bug which could cause least_loaded_node to always return the same unavailable node (dpkp PR 730)
    • 🛠 Fix producer garbage collection with weakref in atexit handler (dpkp PR 728)
    • Close client selector to fix fd leak (msmith PR 729)
    • 👉 Tweak spelling mistake in error const (steve8918 PR 719)
    • ✅ Rearrange connection tests to separate legacy KafkaConnection

  • v1.2.1 Changes

    June 01, 2016

    🛠 Bugfixes

    • 🛠 Fix regression in MessageSet decoding wrt PartialMessages (#716)
    • 🌲 Catch response decode errors and log details (#715)
    • 🛠 Fix Legacy support url (#712 - JonasGroeger)
    • ⚡️ Update sphinx docs re 0.10 broker support

  • v1.2.0 Changes

    May 24, 2016

    🚀 This release officially adds support for Kafka 0.10

    • ➕ Add protocol support for ApiVersionRequest (dpkp PR 678)
    • KAFKA-3025: Message v1 -- add timetamp and relative offsets (dpkp PR 693)
    • 👉 Use Fetch/Produce API v2 for brokers >= 0.10 (uses message format v1) (dpkp PR 694)
    • 👉 Use standard LZ4 framing for v1 messages / kafka 0.10 (dpkp PR 695)

    Consumers

    • ⚡️ Update SimpleConsumer / legacy protocol to handle compressed messages (paulcavallaro PR 684)

    Producers

    • KAFKA-3388: Fix expiration of batches sitting in the accumulator (dpkp PR 699)
    • KAFKA-3197: when max.in.flight.request.per.connection = 1, attempt to guarantee ordering (dpkp PR 698)
    • Don't use soon-to-be-reserved keyword await as function name (FutureProduceResult) (dpkp PR 697)

    Clients

    • 🛠 Fix socket leaks in KafkaClient (dpkp PR 696)

    📚 Documentation

    Internals

    • 👌 Support SSL CRL [requires python 2.7.9+ / 3.4+] (vincentbernat PR 683)
    • 👉 Use original hostname for SSL checks (vincentbernat PR 682)
    • Always pass encoded message bytes to MessageSet.encode()
    • Raise ValueError on protocol encode/decode errors
    • 👻 Supplement socket.gaierror exception in BrokerConnection.connect() (erikbeebe PR 687)
    • BrokerConnection check_version: expect 0.9 to fail with CorrelationIdError
    • 🛠 Fix small bug in Sensor (zackdever PR 679)

  • v1.1.1 Changes

    April 26, 2016

    🛠 quick bugfixes

    • fix throttle_time_ms sensor handling (zackdever pr 667)
    • 👌 improve handling of disconnected sockets (easypost pr 666 / dpkp)
    • 📇 disable standard metadata refresh triggers during bootstrap (dpkp)
    • more predictable future callback/errback exceptions (zackdever pr 670)
    • avoid some exceptions in coordinator.del (dpkp pr 668)

  • v1.1.0 Changes

    April 25, 2016

    Consumers

    • Avoid resending FetchRequests that are pending on internal queue
    • 🌲 Log debug messages when skipping fetched messages due to offset checks
    • 👻 KAFKA-3013: Include topic-partition in exception for expired batches
    • 🌲 KAFKA-3318: clean up consumer logging and error messages
    • 👌 Improve unknown coordinator error handling
    • 👌 Improve auto-commit error handling when group_id is None
    • ➕ Add paused() API (zackdever PR 602)
    • Add default_offset_commit_callback to KafkaConsumer DEFAULT_CONFIGS

    Producers

    Clients

    • 👌 Support SSL connections
    • 👉 Use selectors module for non-blocking IO
    • 🔨 Refactor KafkaClient connection management
    • Fix AttributeError in del
    • SimpleClient: catch errors thrown by _get_leader_for_partition (zackdever PR 606)

    📚 Documentation

    • 🛠 Fix serializer/deserializer examples in README
    • ⚡️ Update max.block.ms docstring
    • ✂ Remove errant next(consumer) from consumer documentation
    • ➕ Add producer.flush() to usage docs

    Internals

    • ➕ Add initial metrics implementation (zackdever PR 637)
    • KAFKA-2136: support Fetch and Produce v1 (throttle_time_ms)
    • 👉 Use version-indexed lists for request/response protocol structs (dpkp PR 630)
    • Split kafka.common into kafka.structs and kafka.errors
    • 🖐 Handle partial socket send() (dpkp PR 611)
    • 🛠 Fix windows support (dpkp PR 603)
    • 👍 IPv6 support (TimEvens PR 615; Roguelazer PR 642)

  • v1.0.2 Changes

    March 14, 2016

    Consumers

    • 👌 Improve KafkaConsumer Heartbeat handling (dpkp PR 583)
    • 🛠 Fix KafkaConsumer.position bug (stefanth PR 578)
    • Raise TypeError when partition is not a TopicPartition (dpkp PR 587)
    • 📊 KafkaConsumer.poll should sleep to prevent tight-loops (dpkp PR 597)

    Producers

    • 🛠 Fix producer threading bug that can crash sender (dpkp PR 590)
    • 🛠 Fix bug in producer buffer pool reallocation (dpkp PR 585)
    • ✂ Remove spurious warnings when closing sync SimpleProducer (twm PR 567)
    • 🛠 Fix FutureProduceResult.await() on python2.6 (dpkp)
    • ➕ Add optional timeout parameter to KafkaProducer.flush() (dpkp)
    • KafkaProducer Optimizations (zackdever PR 598)

    Clients

    • 📇 Improve error handling in SimpleClient.load_metadata_for_topics (dpkp)
    • Improve handling of KafkaClient.least_loaded_node failure (dpkp PR 588)

    📚 Documentation

    • 🛠 Fix KafkaError import error in docs (shichao-an PR 564)
    • 🛠 Fix serializer / deserializer examples (scribu PR 573)

    Internals

    • ⚡️ Update to Kafka 0.9.0.1 for integration testing
    • 🛠 Fix ifr.future.failure in conn.py (mortenlj PR 566)
    • 👌 Improve Zookeeper / Kafka Fixture management (dpkp)

  • v1.0.1 Changes

    February 19, 2016

    Consumers

    • ➕ Add RangePartitionAssignor (and use as default); add assignor tests (dpkp PR 550)
    • ✅ Make sure all consumers are in same generation before stopping group test
    • 👌 Verify node ready before sending offset fetch request from coordinator
    • 👌 Improve warning when offset fetch request returns unknown topic / partition

    Producers

    • Warn if pending batches failed during flush
    • 🛠 Fix concurrency bug in RecordAccumulator.ready()
    • 🛠 Fix bug in SimpleBufferPool memory condition waiting / timeout
    • 👌 Support batch_size = 0 in producer buffers (dpkp PR 558)
    • Catch duplicate batch.done() calls [e.g., maybe_expire then a response errback]

    Clients

    📚 Documentation

    • 👌 Improve kafka.cluster docstrings
    • Migrate load_example.py to KafkaProducer / KafkaConsumer

    Internals

    • 🔧 Don't override system rcvbuf or sndbuf unless configured explicitly (dpkp PR 557)
    • Some attributes may not exist in del if we failed assertions
    • Break up some circular references and close client wake pipes on del (aisch PR 554)

  • v1.0.0 Changes

    February 15, 2016

    🚀 This release includes significant code changes. Users of older kafka-python 🔖 versions are encouraged to test upgrades before deploying to production as 🔧 some interfaces and configuration options have changed.

    👉 Users of SimpleConsumer / SimpleProducer / SimpleClient (formerly KafkaClient) 🚀 from prior releases should migrate to KafkaConsumer / KafkaProducer. Low-level 🚚 APIs (Simple*) are no longer being actively maintained and will be removed in a 🚀 future release.

    📚 For comprehensive API documentation, please see python help() / docstrings, 🏗 kafka-python.readthedocs.org, or run tox -e docs from source to build 📚 documentation locally.

    Consumers

    • KafkaConsumer re-written to emulate the new 0.9 kafka consumer (java client) and support coordinated consumer groups (feature requires >= 0.9.0.0 brokers)

      • Methods no longer available:
      • configure [initialize a new consumer instead]
      • set_topic_partitions [use subscribe() or assign()]
      • fetch_messages [use poll() or iterator interface]
      • get_partition_offsets
      • offsets [use committed(partition)]
      • task_done [handled internally by auto-commit; or commit offsets manually]
      • Configuration changes (consistent with updated java client):
      • lots of new configuration parameters -- see docs for details
      • auto_offset_reset: previously values were 'smallest' or 'largest', now values are 'earliest' or 'latest'
      • fetch_wait_max_ms is now fetch_max_wait_ms
      • max_partition_fetch_bytes is now max_partition_fetch_bytes
      • deserializer_class is now value_deserializer and key_deserializer
      • auto_commit_enable is now enable_auto_commit
      • auto_commit_interval_messages was removed
      • socket_timeout_ms was removed
      • refresh_leader_backoff_ms was removed

    • 🚚 SimpleConsumer and MultiProcessConsumer are now deprecated and will be removed in a future release. Users are encouraged to migrate to KafkaConsumer.

    Producers

    • 🆕 new producer class: KafkaProducer. Exposes the same interface as official java client. Async by default; returned future.get() can be called for synchronous blocking
    • 🚀 SimpleProducer is now deprecated and will be removed in a future release. Users are encouraged to migrate to KafkaProducer.

    Clients

    • 🔀 synchronous KafkaClient renamed to SimpleClient. For backwards compatibility, you will get a SimpleClient via from kafka import KafkaClient. This will change in a future release.
    • All client calls use non-blocking IO under the hood.
    • ➕ Add probe method check_version() to infer broker versions.

    📚 Documentation

    • 📚 Updated README and sphinx documentation to address new classes.
    • 📄 Docstring improvements to make python help() easier to use.

    Internals

    • 🚚 Old protocol stack is deprecated. It has been moved to kafka.protocol.legacy and may be removed in a future release.
    • Protocol layer re-written using Type classes, Schemas and Structs (modeled on the java client).
    • ➕ Add support for LZ4 compression (including broken framing header checksum).

  • v0.9.5 Changes

    December 06, 2015

    Consumers

    • 🎉 Initial support for consumer coordinator: offsets only (toddpalino PR 420)
    • 👍 Allow blocking until some messages are received in SimpleConsumer (saaros PR 457)
    • 👌 Support subclass config changes in KafkaConsumer (zackdever PR 446)
    • 👌 Support retry semantics in MultiProcessConsumer (barricadeio PR 456)
    • 👌 Support partition_info in MultiProcessConsumer (scrapinghub PR 418)
    • 👀 Enable seek() to an absolute offset in SimpleConsumer (haosdent PR 412)
    • ➕ Add KafkaConsumer.close() (ucarion PR 426)

    Producers

    • Catch client.reinit() exceptions in async producer (dpkp)
    • Producer.stop() now blocks until async thread completes (dpkp PR 485)
    • 📇 Catch errors during load_metadata_for_topics in async producer (bschopman PR 467)
    • ➕ Add compression-level support for codecs that support it (trbs PR 454)
    • 🛠 Fix translation of Java murmur2 code, fix byte encoding for Python 3 (chrischamberlin PR 439)
    • 🐳 Only call stop() on not-stopped producer objects (docker-hub PR 435)
    • 👍 Allow null payload for deletion feature (scrapinghub PR 409)

    Clients

    • 👉 Use non-blocking io for broker aware requests (ecanzonieri PR 473)
    • 📇 Use debug logging level for metadata request (ecanzonieri PR 415)
    • Catch KafkaUnavailableError in _send_broker_aware_request (mutability PR 436)
    • 🌲 Lower logging level on replica not available and commit (ecanzonieri PR 415)

    📚 Documentation

    • ⚡️ Update docs and links wrt maintainer change (mumrah -> dpkp)

    Internals

    • ➕ Add py35 to tox testing
    • ⚡️ Update travis config to use container infrastructure
    • ➕ Add 0.8.2.2 and 0.9.0.0 resources for integration tests; update default official releases
    • 🆕 new pylint disables for pylint 1.5.1 (zackdever PR 481)
    • 🛠 Fix python3 / python2 comments re queue/Queue (dpkp)
    • Add Murmur2Partitioner to kafka all imports (dpkp Issue 471)
    • Include LICENSE in PyPI sdist (koobs PR 441)

  • v0.9.4 Changes

    June 11, 2015

    Consumers

    • 🔨 Refactor SimpleConsumer internal fetch handling (dpkp PR 399)
    • Handle exceptions in SimpleConsumer commit() and reset_partition_offset() (dpkp PR 404)
    • 👌 Improve FailedPayloadsError handling in KafkaConsumer (dpkp PR 398)
    • KafkaConsumer: avoid raising KeyError in task_done (dpkp PR 389)
    • 🔧 MultiProcessConsumer -- support configured partitions list (dpkp PR 380)
    • 🛠 Fix SimpleConsumer leadership change handling (dpkp PR 393)
    • 🛠 Fix SimpleConsumer connection error handling (reAsOn2010 PR 392)
    • 👌 Improve Consumer handling of 'falsy' partition values (wting PR 342)
    • 🛠 Fix _offsets call error in KafkaConsumer (hellais PR 376)
    • 🛠 Fix str/bytes bug in KafkaConsumer (dpkp PR 365)
    • Register atexit handlers for consumer and producer thread/multiprocess cleanup (dpkp PR 360)
    • Always fetch commit offsets in base consumer unless group is None (dpkp PR 356)
    • Stop consumer threads on delete (dpkp PR 357)
    • Deprecate metadata_broker_list in favor of bootstrap_servers in KafkaConsumer (dpkp PR 340)
    • 👌 Support pass-through parameters in multiprocess consumer (scrapinghub PR 336)
    • 👀 Enable offset commit on SimpleConsumer.seek (ecanzonieri PR 350)
    • 👌 Improve multiprocess consumer partition distribution (scrapinghub PR 335)
    • Ignore messages with offset less than requested (wkiser PR 328)
    • 🖐 Handle OffsetOutOfRange in SimpleConsumer (ecanzonieri PR 296)

    Producers

    • ➕ Add Murmur2Partitioner (dpkp PR 378)
    • 🌲 Log error types in SimpleProducer and SimpleConsumer (dpkp PR 405)
    • SimpleProducer support configuration of fail_on_error (dpkp PR 396)
    • 🗄 Deprecate KeyedProducer.send() (dpkp PR 379)
    • Further improvements to async producer code (dpkp PR 388)
    • ➕ Add more configuration parameters for async producer (dpkp)
    • 🗄 Deprecate SimpleProducer batch_send=True in favor of async (dpkp)
    • 👌 Improve async producer error handling and retry logic (vshlapakov PR 331)
    • 👌 Support message keys in async producer (vshlapakov PR 329)
    • 👉 Use threading instead of multiprocessing for Async Producer (vshlapakov PR 330)
    • Stop threads on del (chmduquesne PR 324)
    • 🛠 Fix leadership failover handling in KeyedProducer (dpkp PR 314)

    KafkaClient

    • ➕ Add .topics property for list of known topics (dpkp)
    • 🛠 Fix request / response order guarantee bug in KafkaClient (dpkp PR 403)
    • Improve KafkaClient handling of connection failures in _get_conn (dpkp)
    • 📇 Client clears local metadata cache before updating from server (dpkp PR 367)
    • 👍 KafkaClient should return a response or error for each request - enable better retry handling (dpkp PR 366)
    • 👌 Improve str/bytes conversion in KafkaClient and KafkaConsumer (dpkp PR 332)
    • Always return sorted partition ids in client.get_partition_ids_for_topic() (dpkp PR 315)

    📚 Documentation

    • 📚 Cleanup Usage Documentation
    • 👌 Improve KafkaConsumer documentation (dpkp PR 341)
    • 📚 Update consumer documentation (sontek PR 317)
    • ➕ Add doc configuration for tox (sontek PR 316)
    • Switch to .rst doc format (sontek PR 321)
    • 🛠 Fixup google groups link in README (sontek PR 320)
    • 📚 Automate documentation at kafka-python.readthedocs.org

    Internals

    • ✅ Switch integration testing from 0.8.2.0 to 0.8.2.1 (dpkp PR 402)
    • 🛠 Fix most flaky tests, improve debug logging, improve fixture handling (dpkp)
    • 💅 General style cleanups (dpkp PR 394)
    • 🛰 Raise error on duplicate topic-partition payloads in protocol grouping (dpkp)
    • 👉 Use module-level loggers instead of simply 'kafka' (dpkp)
    • Remove pkg_resources check for version at runtime (dpkp PR 387)
    • 👉 Make external API consistently support python3 strings for topic (kecaps PR 361)
    • 🛠 Fix correlation id overflow (dpkp PR 355)
    • Cleanup kafka/common structs (dpkp PR 338)
    • Use context managers in gzip_encode / gzip_decode (dpkp PR 337)
    • 💾 Save failed request as FailedPayloadsError attribute (jobevers PR 302)
    • ✂ Remove unused kafka.queue (mumrah)