All Versions
Latest Version
Avg Release Cycle
55 days
Latest Release
811 days ago

Changelog History
Page 1

  • v2.4.1 Changes

    November 17, 2020

    ๐Ÿ›  Fixed feed exports overwrite support

    ๐Ÿ›  Fixed the asyncio event loop handling, which could make code hang

    ๐Ÿ›  Fixed the IPv6-capable DNS resolver CachingHostnameResolver for download handlers that call reactor.resolve

    ๐Ÿ›  Fixed the output of the genspider command showing placeholders instead of the import part of the generated spider module (issue 4874)

  • v2.4.0 Changes

    October 11, 2020


    ๐Ÿ‘ Python 3.5 support has been dropped.

    ๐Ÿ“„ The file_path method of media pipelines can now access the source item.

    This allows you to set a download file path based on item data.

    The new item_export_kwargs key of the FEEDS setting allows to define keyword parameters to pass to item exporter classes.

    ๐Ÿ“„ You can now choose whether feed exports overwrite or append to the output file.

    ๐Ÿ“„ For example, when using the crawl or runspider commands, you can use the -O option instead of -o to overwrite the output file.

    ๐Ÿ‘ Zstd-compressed responses are now supported if zstandard is installed.

    In settings, where the import path of a class is required, it is now possible to pass a class object instead.

    ๐Ÿ‘€ See the full changelog

  • v2.3.0 Changes

    August 04, 2020


    ๐Ÿ“„ Feed exports now support Google Cloud Storage as a storage backend

    The new FEED_EXPORT_BATCH_ITEM_COUNT setting allows to deliver output items in batches of up to the specified number of items.

    โ†ช It also serves as a workaround for delayed file delivery, which causes Scrapy to only start item delivery after the crawl has finished when using certain storage backends (S3, FTP, and now GCS).

    ๐Ÿš€ The base implementation of item loaders has been moved into a separate library, itemloaders, allowing usage from outside Scrapy and a separate release schedule

    ๐Ÿ‘€ See the full changelog

  • v2.2.1 Changes

    July 17, 2020

    ๐Ÿ“„ The startproject command no longer makes unintended changes to the permissions of files in the destination folder, such as removing execution permissions.

  • v2.2.0 Changes

    June 24, 2020


    ๐Ÿ‘€ See the full changelog

  • v2.1.0 Changes

    April 24, 2020


    ๐Ÿ‘€ See the full changelog

  • v2.0.1 Changes

    March 18, 2020
    • ๐Ÿ‘ Response.follow_all now supports an empty URL iterable as input (#4408, #4420)
    • โœ‚ Removed top-level reactor imports to prevent errors about the wrong Twisted reactor being installed when setting a different Twisted reactor using TWISTED_REACTOR (#4401, #4406)
  • v2.0.0 Changes

    March 03, 2020


    • ๐Ÿšš Python 2 support has been removed
    • ๐Ÿ‘ Partial coroutine syntax support and experimental asyncio support
    • ๐Ÿ†• New Response.follow_all method
    • ๐Ÿ‘ FTP support for media pipelines
    • ๐Ÿ†• New Response.certificate attribute
    • ๐Ÿ‘ IPv6 support through DNS_RESOLVER

    ๐Ÿ‘€ See the full changelog

  • v1.8.0

    October 29, 2019
  • v1.7.4 Changes

    October 21, 2019

    โช Revert the fix for #3804 (#3819), which has a few undesired side effects (#3897, #3976).