All Versions
Latest Version
Avg Release Cycle
107 days
Latest Release
1691 days ago

Changelog History

  • v0.12.0 Changes

    August 27, 2019

    Main changes:

    ๐Ÿ”„ Changes in official python version support: added 3.7 and dropped 3.4.

    โšก๏ธ Added ability to submit a form without updating StatefulBrowser internal state: submit_selected(..., update_state=False). This means you get a response from the form submission, but your browser stays on the same page. Useful for handling forms that result in a file download or open a new tab.

    ๐Ÿ› Bug fixes

    ๐Ÿ‘Œ Improve handling of form enctype to behave like a real browser. [#242]

    HTML type attributes are no longer required to be lowercase. [#245]

    Form controls with the disabled attribute will no longer be submitted to improve compliance with the HTML standard. If you were relying on this bug to submit disabled elements, you can still achieve this by deleting the disabled attribute from the element in the Form object directly. [#248]

    ๐Ÿ’ป When a form containing a file input field is submitted without choosing a file, an empty filename & content will be sent just like in a real browser. [#250]

    <option> tags without a value attribute will now use their text as the value. [#252]

    The optional url_regex argument to follow_link and download_link was fixed so that it is no longer ignored. [#256]

    ๐Ÿ‘ Allow duplicate submit elements instead of raising a LinkNotFoundError. [#264]

    ๐Ÿš€ Our thanks to the many new contributors in this release!

  • v0.11.0 Changes

    September 11, 2018

    ๐Ÿ“š This release focuses on fixing bugs related to uncommon HTTP/HTML scenarios and on improving the documentation.

    ๐Ÿ› Bug fixes

    ๐Ÿ—„ Constructing a Form instance from a bs4.element.Tag whose tag name is not form will now emit a warning, and may be deprecated in the future. [#228]

    ๐Ÿ’ฅ Breaking Change: LinkNotFoundError now derives from Exception instead of BaseException. While this will bring the behavior in line with most people's expectations, it may affect the behavior of your code if you were heavily relying on this implementation detail in your exception handling. [#203]

    ๐Ÿ‘Œ Improve handling of button submit elements. Will now correctly ignore buttons of type button and reset during form submission, since they are not considered to be submit elements. [#199]

    ๐Ÿ‘ท Do a better job of inferring the content type of a response if the Content-Type header is not provided. [#195]

    ๐Ÿ‘Œ Improve consistency of query string construction between MechanicalSoup and web browsers in edge cases where form elements have duplicate name attributes. This prevents errors in valid use cases, and also makes MechanicalSoup more tolerant of invalid HTML. [#158]

  • v0.10.0 Changes

    February 04, 2018

    Main changes:

    โž• Added StatefulBrowser.refresh() to reload the current page with the same request. [#188]

    ๐Ÿ”— StatefulBrowser.follow_link, StatefulBrowser.submit_selected() and the new StatefulBrowser.download_link now sets the Referer: HTTP header to the page from which the link is followed. [#179]

    โž• Added method StatefulBrowser.download_link, which will download the contents of a link to a file without changing the state of the browser. [#170]

    ๐Ÿ’ป The selector argument of Browser.select_form can now be a bs4.element.Tag in addition to a CSS selector. [#169]

    ๐Ÿ’ป Browser.submit and StatefulBrowser.submit_selected accept a larger number of keyword arguments. Arguments are forwarded to requests.Session.request. [#166]

    Internal changes:

    ๐Ÿ’ป StatefulBrowser.choose_submit will now ignore input elements that are missing a name-attribute instead of raising a KeyError. [#180]

    ๐Ÿ— Private methods Browser._build_request and Browser._prepare_request have been replaced by a single method Browser._request. [#166]

  • v0.9.0 Changes

    November 02, 2017

    Main changes:

    0๏ธโƒฃ We do not rely on BeautifulSoup's default choice of HTML parser.
    0๏ธโƒฃ Instead, we now specify lxml as default. As a consequence, the
    0๏ธโƒฃ default setting requires lxml as a dependency.

    ๐Ÿ‘ Python 2.6 and 3.3 are no longer supported.

    ๐Ÿšš The GitHub URL moved from to @moy and
    @hemberger are now officially administrators of the project in
    โž• addition to @hickford, the original author.

    ๐Ÿ“š We now have a documentation site:
    The API is now fully documented, and we have included a tutorial,
    ๐Ÿ™‹ several more code examples, and a FAQ.

    ๐Ÿ’ป StatefulBrowser.select_form can now be called without argument,
    0๏ธโƒฃ and defaults to "form" in this case. It also has a new argument,
    0๏ธโƒฃ nr (defaults to 0), which can be used to specify the index of
    the form to select if multiple forms match the selection criteria.

    We now use requirement files. You can install the dependencies of
    MechanicalSoup with e.g.::

    โœ… pip install -r requirements.txt -r tests/requirements.txt

    The Form class was restructured and has a new API. The behavior of
    existing code is unchanged, but a new collection of methods has been
    โž• added for clarity and consistency with the set method:

    • set_input deprecates input
    • set_textarea deprecates textarea
    • set_select is new
    • set_checkbox and set_radio together deprecate check
      0๏ธโƒฃ (checkboxes are handled differently by default)

    ๐Ÿ–จ A new Form.print_summary method allows you to write
    browser.get_current_form().print_summary() to get a summary of the
    fields you need to fill-in (and which ones are already filled-in).

    ๐Ÿ‘ The Form class now supports selecting multiple options in
    a <select multiple> element.

    ๐Ÿ› Bug fixes

    ๐Ÿ’ป Checking checkboxes with browser["name"] = ("val1", "val2") now
    unchecks all checkbox except the ones explicitly specified.

    ๐Ÿ’ป StatefulBrowser.submit_selected and now
    reset __current_page to None when the result is not an HTML page.
    This fixes a bug where __current_page was still the previous page.

    We don't error out anymore when trying to uncheck a box which
    doesn't have a checkbox attribute.

    Form.new_control now correctly overrides existing elements.

    Internal changes

    โœ… The testsuite has been further improved and reached 100% coverage.

    โœ… Tests are now run against the local version of MechanicalSoup, not
    against the installed version.

    Browser.add_soup will now always attach a soup-attribute.
    If the response is not text/html, then soup is set to None.

    Form.set(force=True) creates an <input type=text ...>
    element instead of an <input type=input ...>.

  • v0.9.0-post4

    November 06, 2017
  • v0.9.0-post3

    November 06, 2017
  • v0.9.0-post2

    November 05, 2017
  • v0.9.0-post1

    November 05, 2017
  • v0.8.0 Changes

    October 01, 2017

    Main changes:

    ๐Ÿ”ง Browser and StatefulBrowser can now be configured to raise a
    ๐Ÿ‘ป LinkNotFound exception when encountering a 404 Not Found error.
    This is activated by passing raise_on_404=True to the constructor.
    0๏ธโƒฃ It is disabled by default for backward compatibility, but is highly

    Browser now has a __del__ method that closes the current session
    when the object is deleted.

    A Link object can now be passed to follow_link.

    0๏ธโƒฃ The user agent can now be customized. The default includes
    MechanicalSoup and its version.

    ๐Ÿ’ป There is now a direct interface to the cookiejar in *Browser
    classes ((set|get)_cookiejar methods).

    ๐Ÿ‘ This is the last MechanicalSoup version supporting Python 2.6 and 3.3.

    ๐Ÿ› Bug fixes:

    We used to crash on forms without action="..." fields.

    ๐Ÿ›  The choose_submit method has been fixed, and the btnName
    ๐Ÿ’ป argument of StatefulBrowser.submit_selected is now a shortcut for
    using choose_submit.

    Arguments to open_relative were not properly forwarded.

    Internal changes:

    โœ… The testsuite has been greatly improved. It now uses the pytest API
    โœ… (not only the pytest launcher) for more concise code.

    โœ… The coverage of the testsuite is now measured with The
    results can be viewed on:

    We now have a badge to help us tracking issues with
    dependencies. The report can be viewed on:

    The version number now appears in a single place in the source code.

  • v0.7.0 Changes

    May 07, 2017

    Summary of changes:

    ๐Ÿ†• New class StatefulBrowser, that keeps track of the currently visited page to make the calling code more concise.

    ๐Ÿ’ป A new launch_browser method in Browser and StatefulBrowser, that allows launching a browser on the currently visited page for easier debugging.

    ๐Ÿ›  Many bug fixes.

    ๐Ÿš€ Release on Pypi: