Changelog History
-
v0.12.0 Changes
August 27, 2019Main changes:
๐ Changes in official python version support: added 3.7 and dropped 3.4.
โก๏ธ Added ability to submit a form without updating
StatefulBrowser
internal state:submit_selected(..., update_state=False)
. This means you get a response from the form submission, but your browser stays on the same page. Useful for handling forms that result in a file download or open a new tab.๐ Bug fixes
๐ Improve handling of form enctype to behave like a real browser. [#242]
HTML
type
attributes are no longer required to be lowercase. [#245]Form controls with the
disabled
attribute will no longer be submitted to improve compliance with the HTML standard. If you were relying on this bug to submit disabled elements, you can still achieve this by deleting thedisabled
attribute from the element in theForm
object directly. [#248]๐ป When a form containing a file input field is submitted without choosing a file, an empty filename & content will be sent just like in a real browser. [#250]
<option>
tags without avalue
attribute will now use their text as the value. [#252]The optional
url_regex
argument tofollow_link
anddownload_link
was fixed so that it is no longer ignored. [#256]๐ Allow duplicate submit elements instead of raising a LinkNotFoundError. [#264]
๐ Our thanks to the many new contributors in this release!
-
v0.11.0 Changes
September 11, 2018๐ This release focuses on fixing bugs related to uncommon HTTP/HTML scenarios and on improving the documentation.
๐ Bug fixes
๐ Constructing a
Form
instance from abs4.element.Tag
whose tag name is notform
will now emit a warning, and may be deprecated in the future. [#228]๐ฅ Breaking Change:
LinkNotFoundError
now derives fromException
instead ofBaseException
. While this will bring the behavior in line with most people's expectations, it may affect the behavior of your code if you were heavily relying on this implementation detail in your exception handling. [#203]๐ Improve handling of
button
submit elements. Will now correctly ignore buttons of typebutton
andreset
during form submission, since they are not considered to be submit elements. [#199]๐ท Do a better job of inferring the content type of a response if the
Content-Type
header is not provided. [#195]๐ Improve consistency of query string construction between MechanicalSoup and web browsers in edge cases where form elements have duplicate name attributes. This prevents errors in valid use cases, and also makes MechanicalSoup more tolerant of invalid HTML. [#158]
-
v0.10.0 Changes
February 04, 2018Main changes:
โ Added
StatefulBrowser.refresh()
to reload the current page with the same request. [#188]๐
StatefulBrowser.follow_link
,StatefulBrowser.submit_selected()
and the newStatefulBrowser.download_link
now sets theReferer:
HTTP header to the page from which the link is followed. [#179]โ Added method
StatefulBrowser.download_link
, which will download the contents of a link to a file without changing the state of the browser. [#170]๐ป The
selector
argument ofBrowser.select_form
can now be a bs4.element.Tag in addition to a CSS selector. [#169]๐ป
Browser.submit
andStatefulBrowser.submit_selected
accept a larger number of keyword arguments. Arguments are forwarded to requests.Session.request. [#166]Internal changes:
๐ป
StatefulBrowser.choose_submit
will now ignore input elements that are missing a name-attribute instead of raising aKeyError
. [#180]๐ Private methods
Browser._build_request
andBrowser._prepare_request
have been replaced by a single methodBrowser._request
. [#166] -
v0.9.0 Changes
November 02, 2017Main changes:
0๏ธโฃ We do not rely on BeautifulSoup's default choice of HTML parser.
0๏ธโฃ Instead, we now specifylxml
as default. As a consequence, the
0๏ธโฃ default setting requireslxml
as a dependency.๐ Python 2.6 and 3.3 are no longer supported.
๐ The GitHub URL moved from
https://github.com/hickford/MechanicalSoup/ to
https://github.com/MechanicalSoup/MechanicalSoup. @moy and
@hemberger are now officially administrators of the project in
โ addition to @hickford, the original author.๐ We now have a documentation site: https://mechanicalsoup.readthedocs.io/.
The API is now fully documented, and we have included a tutorial,
๐ several more code examples, and a FAQ.๐ป
StatefulBrowser.select_form
can now be called without argument,
0๏ธโฃ and defaults to"form"
in this case. It also has a new argument,
0๏ธโฃnr
(defaults to 0), which can be used to specify the index of
the form to select if multiple forms match the selection criteria.We now use requirement files. You can install the dependencies of
MechanicalSoup with e.g.::โ pip install -r requirements.txt -r tests/requirements.txt
The
Form
class was restructured and has a new API. The behavior of
existing code is unchanged, but a new collection of methods has been
โ added for clarity and consistency with theset
method:set_input
deprecatesinput
set_textarea
deprecatestextarea
set_select
is newset_checkbox
andset_radio
together deprecatecheck
0๏ธโฃ (checkboxes are handled differently by default)
๐จ A new
Form.print_summary
method allows you to write
browser.get_current_form().print_summary()
to get a summary of the
fields you need to fill-in (and which ones are already filled-in).๐ The
Form
class now supports selecting multiple options in
a<select multiple>
element.๐ Bug fixes
๐ป Checking checkboxes with
browser["name"] = ("val1", "val2")
now
unchecks all checkbox except the ones explicitly specified.๐ป
StatefulBrowser.submit_selected
andStatefulBrowser.open
now
reset __current_page to None when the result is not an HTML page.
This fixes a bug where __current_page was still the previous page.We don't error out anymore when trying to uncheck a box which
doesn't have acheckbox
attribute.Form.new_control
now correctly overrides existing elements.Internal changes
โ The testsuite has been further improved and reached 100% coverage.
โ Tests are now run against the local version of MechanicalSoup, not
against the installed version.Browser.add_soup
will now always attach a soup-attribute.
If the response is not text/html, then soup is set to None.Form.set(force=True)
creates an<input type=text ...>
element instead of an<input type=input ...>
. -
v0.9.0-post4
November 06, 2017 -
v0.9.0-post3
November 06, 2017 -
v0.9.0-post2
November 05, 2017 -
v0.9.0-post1
November 05, 2017 -
v0.8.0 Changes
October 01, 2017Main changes:
๐ง
Browser
andStatefulBrowser
can now be configured to raise a
๐ปLinkNotFound
exception when encountering a 404 Not Found error.
This is activated by passingraise_on_404=True
to the constructor.
0๏ธโฃ It is disabled by default for backward compatibility, but is highly
recommanded.Browser
now has a__del__
method that closes the current session
when the object is deleted.A
Link
object can now be passed tofollow_link
.0๏ธโฃ The user agent can now be customized. The default includes
MechanicalSoup
and its version.๐ป There is now a direct interface to the cookiejar in
*Browser
classes ((set|get)_cookiejar
methods).๐ This is the last MechanicalSoup version supporting Python 2.6 and 3.3.
๐ Bug fixes:
We used to crash on forms without action="..." fields.
๐ The
choose_submit
method has been fixed, and thebtnName
๐ป argument ofStatefulBrowser.submit_selected
is now a shortcut for
usingchoose_submit
.Arguments to
open_relative
were not properly forwarded.Internal changes:
โ The testsuite has been greatly improved. It now uses the pytest API
โ (not only thepytest
launcher) for more concise code.โ The coverage of the testsuite is now measured with codecov.io. The
results can be viewed on:
https://codecov.io/gh/hickford/MechanicalSoupWe now have a requires.io badge to help us tracking issues with
dependencies. The report can be viewed on:
https://requires.io/github/hickford/MechanicalSoup/requirements/The version number now appears in a single place in the source code.
-
v0.7.0 Changes
May 07, 2017Summary of changes:
๐ New class
StatefulBrowser
, that keeps track of the currently visited page to make the calling code more concise.๐ป A new
launch_browser
method inBrowser
andStatefulBrowser
, that allows launching a browser on the currently visited page for easier debugging.๐ Many bug fixes.
๐ Release on Pypi: https://pypi.python.org/pypi/MechanicalSoup/0.7.0