All Versions
11
Latest Version
Avg Release Cycle
63 days
Latest Release
1985 days ago
Changelog History
Page 1
Changelog History
Page 1
-
v0.6.41 Changes
June 24, 2018๐ Changed
- ๐ Restrict pycurl version to <7.43.0.1 (see #354)
-
v0.6.40 Changes
May 13, 2018 -
v0.6.39 Changes
May 09, 2018๐ Fixed
- ๐ Fix bug: task generator works incorrectly
- ๐ Fix bug: pypi package misses http api html file
- ๐ Fix bug: dictionary changed size during iteration in stat logging
- ๐ Fix bug: multiple errors in urllib3 transport and threaded network service
- ๐ Fix short names of errors in stat logging
- ๐ Improve error handling in urrllib3 transport
- ๐ Fix #299: multi-added errors
- ๐ Fix bug: pypi package misses http api html file
- ๐ Fix #285: pyquery extension parses html incorrectly
- ๐ Fix #267: normalize handling of too many redirect error
- ๐ Fix #268: fix processing of utf cookies
- ๐ Fix #241: form_fields() fails on some HTML forms
- ๐ Fix normalize_unicode issue in debug post method
- ๐ Fix #323: urllib3 transport fails with UnicodeError on some invalid URLs
- ๐ Fix #31: support for multivalue form inputs
- ๐ Fix #328, fix #67: remove hard link between document and grab
- ๐ Fix #284: option headers affects content of common_headers
- ๐ Fix #293: processing non-latin chars in Location header
- ๐ Fix #324: refactor response header processing
๐ Changed
- ๐จ Refactor Spider into set of async. services
- โ Add certifi dependency into grab[full] setup target
- ๐ Fix #315: use psycopg2-binary package for postgres cache
- Related to #206: do not use connection_reuse=False for proxy connections in spider
โ Removed
- โ Remove cache timeout option
- โ Remove structured extension
-
v0.6.38 Changes
May 17, 2017๐ Fixed
- ๐ Fix "error:None" in spider rps logging
- ๐ Fix race condition bug in task generator
โ Added
- โ Add original_exc attribute to GrabNetworkError (and subclasses) that points to original exception
๐ Changed
- โ Remove IOError from the ancestors of GrabNetworkError
- โ Add default values to --spider-transport and --grab-transport options of crawl script
-
v0.6.37 Changes
May 13, 2017โ Added
- โ Add --spider-transport and --grab-transport options to crawl script
- โ Add SOCKS5 proxy support in urllib3 transport
๐ Fixed
- ๐ Fix #237: urllib3 transport fails without pycurl installed
- ๐ Fix bug: incorrect spider request logging when cache is enabled
- ๐ Fix bug: crawl script fails while trying to process a lock key
- ๐ Fix bug: urllib3 transport fails while trying to throw GrabConnectionError exception
- ๐ Fix bug: Spider add_task method fails while trying to log invalid URL error
โ Removed
- Remove obsoleted hammer_mode and hammer_timeout config options
-
v0.6.36 Changes
February 12, 2017โ Added
- โ Add pylint to default test set
๐ Fixed
- ๐ Fix #229: using deprecated response object inside Grab
โ Removed
- โ Remove spider project template and start_project script
-
v0.6.35 Changes
February 06, 2017๐ Fixed
- ๐ Fix bug in deprecated grab.choose_form method
- โ Add default project templates files to the distribution, by @rushter
- ๐ Fix #222: debug_post option fails with big post data
- ๐ Fix #148: pycurl ignores sigint signal
-
v0.6.34 Changes
February 04, 2017โ Added
- โ Start running Grab tests in OSX environment on travis CI
๐ Changed
- ๐ Use defusedxml library to parse HTML and XML, by @kevinlondon
- Put selection, lxml and pycurl libs back to required dependencies in setup.py
- ๐ Update installation documentation
-
v0.6.33 Changes
January 28, 2017โ Added
- โ Add API documentation about few grab modules, by @rushter
- ๐ Start running Grab tests in Windows enviroment on appveyor CI
- ๐ New spider transport based on threads that allows to use Spider with any Grab network backend e.g. urllib3
- Add
remove_from_post
option to grab.doc.submit method - โ Add
random
option to grab.change_proxy method - ๐ Support for deprecated attributes Spider.items and Spider.counters
- If Spider handler raises ResponseNotValid exception, then that task goes back to task queue until task.task_try_count reaches the spider.task_try_limit
๐ Changed
- ๐จ Refactor management of internal threads, fix random test failures related to cache sub-module
- 0๏ธโฃ Disable default logging to files while running spider by
run crawl
command - Multiple improvements in urllib3 transport
- 0๏ธโฃ Set default spider network & try limits to 3 (was 10)
๐ Fixed
- Different bugs in urllib3 transport
- Different bugs
โ Removed
- Remove grab.use_next_proxy method
- โ Remove grab.dump method
- โ Remove deprecated Spider methods and attributes
-
v0.6.32 Changes
December 31, 2017๐ Fixed
- ๐ Fix setup.py