kombu changelog

Changelog History

Page 12

• v2.2.1 Changes

  🚀 :release-date: 2012-06-21 01:00 P.M BST 🚀 :release-by: Ask Solem

  • 0️⃣ SQS: Default visibility timeout is now 30 minutes.

    Since we have ack emulation the visibility timeout is only in effect if the consumer is abrubtly terminated.

  • retry argument to Producer.publish now works properly, when the declare argument is specified.

  • Json serializer: didn't handle buffer objects (Issue #135).

    Fix contributed by Jens Hoffrichter.

  • 👍 Virtual: Now supports passive argument to exchange_declare.

  • 0️⃣ Exchange & Queue can now be bound to connections (which will use the default channel):

    .. code-block:: pycon

    >>> exchange = Exchange('name')
    >>> bound_exchange = exchange(connection)
    >>> bound_exchange.declare()
    

  • SimpleQueue & SimpleBuffer can now be bound to connections (which will use the default channel).

  • Connection.manager.get_bindings now works for librabbitmq and pika.

  • ➕ Adds new transport info attributes:

    • Transport.driver_type

      Type of underlying driver, e.g. "amqp", "redis", "sql".

    • Transport.driver_name

      Name of library used e.g. "amqplib", "redis", "pymongo".

    • Transport.driver_version()

      Version of underlying library.

  .. _version-2.2.0:

• v2.2.0 Changes

  🚀 :release-date: 2012-06-07 03:10 P.M BST 🚀 :release-by: Ask Solem

  .. _v220-important:

  Important Notes

  • 🚚 The canonical source code repository has been moved to

    http://github.com/celery/kombu

  • Pidbox: Exchanges used by pidbox are no longer auto_delete.

    Auto delete has been described as a misfeature, and therefore we have disabled it.

    For RabbitMQ users old exchanges used by pidbox must be removed, these are named mailbox_name.pidbox, and reply.mailbox_name.pidbox.

    The following command can be used to clean up these exchanges:

    .. code-block:: text

    $ VHOST=/ URL=amqp:// python -c'import sys,kombu;[kombu.Connection(
        sys.argv[-1]).channel().exchange_delete(x)
            for x in sys.argv[1:-1]]' \
        $(sudo rabbitmqctl -q list_exchanges -p "$VHOST" \
        | grep \.pidbox | awk '{print $1}') "$URL"
    

    The :envvar:VHOST variable must be set to the target RabbitMQ virtual host, and the :envvar:URL must be the AMQP URL to the server.

  • The amqp transport alias will now use :mod:librabbitmq if installed.

    py-librabbitmq_ is a fast AMQP client for Python using the librabbitmq C library.

    It can be installed by:

    .. code-block:: console

    $ pip install librabbitmq
    

    It will not be used if the process is monkey patched by eventlet/gevent.

  .. _py-librabbitmq: https://github.com/celery/librabbitmq

  .. _v220-news:

  🆕 News

  • Redis: Ack emulation improvements.

    Reducing the possibility of data loss.

    Acks are now implemented by storing a copy of the message when the message is consumed. The copy is not removed until the consumer acknowledges or rejects it.

    This means that unacknowledged messages will be redelivered either when the connection is closed, or when the visibility timeout is exceeded.

    • Visibility timeout

      This is a timeout for acks, so that if the consumer does not ack the message within this time limit, the message is redelivered to another consumer.

      The timeout is set to one hour by default, but can be changed by configuring a transport option:

      >>> Connection('redis://', transport_options={
      ...     'visibility_timeout': 1800,  # 30 minutes
      ... })
      

    NOTE: Messages that have not been acked will be redelivered if the visibility timeout is exceeded, for Celery users this means that ETA/countdown tasks that are scheduled to execute with a time that exceeds the visibility timeout will be executed twice (or more). If you plan on using long ETA/countdowns you should tweak the visibility timeout accordingly:

    .. code-block:: python

    BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 18000}  # 5 hours
    

    Setting a long timeout means that it will take a long time for messages to be redelivered in the event of a power failure, but if so happens you could temporarily set the visibility timeout lower to flush out messages when you start up the systems again.

  • Experimental Apache ZooKeeper_ transport

    More information is in the module reference: :mod:kombu.transport.zookeeper.

    Contributed by Mahendra M.

  .. _Apache ZooKeeper: http://zookeeper.apache.org/

  • 👍 Redis: Priority support.

    The message's priority field is now respected by the Redis transport by having multiple lists for each named queue. The queues are then consumed by in order of priority.

    The priority field is a number in the range of 0 - 9, where 0 is the default and highest priority.

    The priority range is collapsed into four steps by default, since it is unlikely that nine steps will yield more benefit than using four steps. The number of steps can be configured by setting the priority_steps transport option, which must be a list of numbers in sorted order:

    .. code-block:: pycon

    >>> x = Connection('redis://', transport_options={
    ...     'priority_steps': [0, 2, 4, 6, 8, 9],
    ... })
    

    Priorities implemented in this way is not as reliable as priorities on the server side, which is why nickname the feature "quasi-priorities"; Using routing is still the suggested way of ensuring quality of service, as client implemented priorities fall short in a number of ways, e.g. if the worker is busy with long running tasks, has prefetched many messages, or the queues are congested.

    Still, it is possible that using priorities in combination with routing can be more beneficial than using routing or priorities alone. Experimentation and monitoring should be used to prove this.

    Contributed by Germán M. Bravo.

  • Redis: Now cycles queues so that consuming is fair.

    This ensures that a very busy queue won't block messages from other queues, and ensures that all queues have an equal chance of being consumed from.

    This used to be the case before, but the behavior was accidentally changed while switching to using blocking pop.

  • Redis: Auto delete queues that are bound to fanout exchanges is now deleted at channel.close.

  • 🔨 amqplib: Refactored the drain_events implementation.

  • 0️⃣ Pidbox: Now uses connection.default_channel.

  • Pickle serialization: Can now decode buffer objects.

  • Exchange/Queue declarations can now be cached even if the entity is non-durable.

    This is possible because the list of cached declarations are now kept with the connection, so that the entities will be redeclared if the connection is lost.

  • Kombu source code now only uses one-level of explicit relative imports.

  🛠 .. _v220-fixes:

  🛠 Fixes

  • eventio: Now ignores ENOENT raised by epoll.register, and EEXIST from epoll.unregister.

  • eventio: kqueue now ignores :exc:KeyError on unregister.

  • 👍 Redis: Message.reject now supports the requeue argument.

  • 🚚 Redis: Remove superfluous pipeline call.

    Fix contributed by Thomas Johansson.

  • Redis: Now sets redelivered header for redelivered messages.

  • 🚚 Now always makes sure references to :func:sys.exc_info is removed.

  • 🚚 Virtual: The compression header is now removed before restoring messages.

  • ✅ More tests for the SQLAlchemy backend.

    Contributed by Franck Cuny.

  • 📜 Url parsing did not handle MongoDB URLs properly.

    Fix contributed by Flavio Percoco Premoli.

  • 0️⃣ Beanstalk: Ignore default tube when reserving.

    Fix contributed by Zhao Xiaohong.

  👍 Nonblocking consume support

  librabbitmq, amqplib and redis transports can now be used non-blocking.

  The interface is very manual, and only consuming messages is non-blocking so far.

  The API should not be regarded as stable or final in any way. It is used by Celery which has very limited needs at this point. Hopefully we can introduce a proper callback-based API later.

  • Transport.eventmap

    Is a map of fd -> callback(fileno, event) to register in an eventloop.

  • 📊 Transport.on_poll_start()

    Is called before every call to poll. The poller must support register(fd, callback) and unregister(fd) methods.

  • 📊 Transport.on_poll_start(poller)

    Called when the hub is initialized. The poller argument must support the same interface as :class:kombu.utils.eventio.poll.

  • Connection.ensure_connection now takes a callback argument which is called for every loop while the connection is down.

  • ➕ Adds connection.drain_nowait

    This is a non-blocking alternative to drain_events, but only supported by amqplib/librabbitmq.

  • drain_events now sets connection.more_to_read if there is more data to read.

    This is to support eventloops where other things must be handled between draining events.

  .. _version-2.1.8:

• v2.1.8 Changes

  🚀 :release-date: 2012-05-06 03:06 P.M BST 🚀 :release-by: Ask Solem

  • Bound Exchange/Queue's are now pickleable.

  • Consumer/Producer can now be instantiated without a channel, and only later bound using .revive(channel).

  • ProducerPool now takes Producer argument.

  • :func:~kombu.utils.fxrange now counts forever if the stop argument is set to None. (fxrange is like xrange but for decimals).

  • 👍 Auto delete support for virtual transports were incomplete and could lead to problems so it was removed.

  • Cached declarations (:func:~kombu.common.maybe_declare) are now bound to the underlying connection, so that entities are redeclared if the connection is lost.

    This also means that previously uncacheable entities (e.g. non-durable) can now be cached.

  • compat ConsumerSet: can now specify channel.

  .. _version-2.1.7:

• v2.1.7 Changes

  🚀 :release-date: 2012-04-27 06:00 P.M BST 🚀 :release-by: Ask Solem

  • compat consumerset now accepts optional channel argument.

  .. _version-2.1.6:

• v2.1.6 Changes

  🚀 :release-date: 2012-04-23 01:30 P.M BST 🚀 :release-by: Ask Solem

  • 📜 SQLAlchemy transport was not working correctly after URL parser change.

  • maybe_declare now stores cached declarations per underlying connection instead of globally, in the rare case that data disappears from the broker after connection loss.

  • Django: Added South migrations.

    Contributed by Joseph Crosland.

  .. _version-2.1.5:

• v2.1.5 Changes

  🚀 :release-date: 2012-04-13 03:30 P.M BST 🚀 :release-by: Ask Solem

  • 🚚 The url parser removed more than the first leading slash (Issue #121).

  • SQLAlchemy: Can now specify url using + separator

    Example:

    .. code-block:: python

    Connection('sqla+mysql://localhost/db')
    

  • 👍 Better support for anonymous queues (Issue #116).

    Contributed by Michael Barrett.

  • Connection.as_uri now quotes url parts (Issue #117).

  • Beanstalk: Can now set message TTR as a message property.

    Contributed by Andrii Kostenko

  .. _version-2.1.4:

• v2.1.4 Changes

  🚀 :release-date: 2012-04-03 04:00 P.M GMT 🚀 :release-by: Ask Solem

  • 📜 MongoDB: URL parsing are now delegated to the pymongo library (Fixes Issue #103 and Issue #87).

    Fix contributed by Flavio Percoco Premoli and James Sullivan

  • SQS: A bug caused SimpleDB to be used even if sdb persistence was not enabled (Issue #108).

    Fix contributed by Anand Kumria.

  • Django: Transaction was committed in the wrong place, causing data cleanup to fail (Issue #115).

    Fix contributed by Daisuke Fujiwara.

  • 👍 MongoDB: Now supports replica set URLs.

    Contributed by Flavio Percoco Premoli.

  • Redis: Now raises a channel error if a queue key that is currently being consumed from disappears.

    Fix contributed by Stephan Jaekel.

  • All transport 'channel_errors' lists now includes kombu.exception.StdChannelError.

  • All kombu exceptions now inherit from a common :exc:~kombu.exceptions.KombuError.

  .. _version-2.1.3:

• v2.1.3 Changes

  🚀 :release-date: 2012-03-20 03:00 P.M GMT 🚀 :release-by: Ask Solem

  • 🛠 Fixes Jython compatibility issues.

  • 🛠 Fixes Python 2.5 compatibility issues.

  .. _version-2.1.2:

• v2.1.2 Changes

  🚀 :release-date: 2012-03-01 01:00 P.M GMT 🚀 :release-by: Ask Solem

  • 👍 amqplib: Last version broke SSL support.

  .. _version-2.1.1:

• v2.1.1 Changes

  🚀 :release-date: 2012-02-24 02:00 P.M GMT 🚀 :release-by: Ask Solem

  • 👍 Connection URLs now supports encoded characters.

  • 🛠 Fixed a case where connection pool could not recover from connection loss.

    Fix contributed by Florian Munz.

  • We now patch amqplib's __del__ method to skip trying to close the socket if it is not connected, as this resulted in an annoying warning.

  • 🛰 Compression can now be used with binary message payloads.

    Fix contributed by Steeve Morin.

  .. _version-2.1.0:

• « First
• ‹ Prev
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15
• 16
• Next ›
• Last »