All Versions
20
Latest Version
Avg Release Cycle
55 days
Latest Release
1256 days ago

Changelog History
Page 1

  • v2.4.1 Changes

    November 17, 2020

    🛠 Fixed feed exports overwrite support

    🛠 Fixed the asyncio event loop handling, which could make code hang

    🛠 Fixed the IPv6-capable DNS resolver CachingHostnameResolver for download handlers that call reactor.resolve

    🛠 Fixed the output of the genspider command showing placeholders instead of the import part of the generated spider module (issue 4874)

  • v2.4.0 Changes

    October 11, 2020

    Hihglights:

    👍 Python 3.5 support has been dropped.

    📄 The file_path method of media pipelines can now access the source item.

    This allows you to set a download file path based on item data.

    The new item_export_kwargs key of the FEEDS setting allows to define keyword parameters to pass to item exporter classes.

    📄 You can now choose whether feed exports overwrite or append to the output file.

    📄 For example, when using the crawl or runspider commands, you can use the -O option instead of -o to overwrite the output file.

    👍 Zstd-compressed responses are now supported if zstandard is installed.

    In settings, where the import path of a class is required, it is now possible to pass a class object instead.

    👀 See the full changelog

  • v2.3.0 Changes

    August 04, 2020

    Hihglights:

    📄 Feed exports now support Google Cloud Storage as a storage backend

    The new FEED_EXPORT_BATCH_ITEM_COUNT setting allows to deliver output items in batches of up to the specified number of items.

    ↪ It also serves as a workaround for delayed file delivery, which causes Scrapy to only start item delivery after the crawl has finished when using certain storage backends (S3, FTP, and now GCS).

    🚀 The base implementation of item loaders has been moved into a separate library, itemloaders, allowing usage from outside Scrapy and a separate release schedule

    👀 See the full changelog

  • v2.2.1 Changes

    July 17, 2020

    📄 The startproject command no longer makes unintended changes to the permissions of files in the destination folder, such as removing execution permissions.

  • v2.2.0 Changes

    June 24, 2020

    Highlights:

    👀 See the full changelog

  • v2.1.0 Changes

    April 24, 2020

    Highlights:

    👀 See the full changelog

  • v2.0.1 Changes

    March 18, 2020
    • 👍 Response.follow_all now supports an empty URL iterable as input (#4408, #4420)
    • ✂ Removed top-level reactor imports to prevent errors about the wrong Twisted reactor being installed when setting a different Twisted reactor using TWISTED_REACTOR (#4401, #4406)

  • v2.0.0 Changes

    March 03, 2020

    Highlights:

    • 🚚 Python 2 support has been removed
    • 👍 Partial coroutine syntax support and experimental asyncio support
    • 🆕 New Response.follow_all method
    • 👍 FTP support for media pipelines
    • 🆕 New Response.certificate attribute
    • 👍 IPv6 support through DNS_RESOLVER

    👀 See the full changelog

  • v1.8.0

    October 29, 2019

  • v1.7.4 Changes

    October 21, 2019

    ⏪ Revert the fix for #3804 (#3819), which has a few undesired side effects (#3897, #3976).