Changelog History
Page 1
-
v6.2.0.dev0
September 26, 2020 -
v6.1.2 Changes
October 28, 2020โ pytest 6.1.2 (2020-10-28)
๐ Bug Fixes
- ๐ฆ #7758: Fixed an issue where some files in packages are getting lost from
--lf
even though they contain tests that failed. Regressed in pytest 5.4.0. - โ #7911: Directories created by tmpdir are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.
๐ Improved Documentation
- โ
#7815: Improve deprecation warning message for
pytest._fillfuncargs()
.
- ๐ฆ #7758: Fixed an issue where some files in packages are getting lost from
-
v6.1.1 Changes
October 03, 2020โ pytest 6.1.1 (2020-10-03)
๐ Bug Fixes
-
v6.1.0 Changes
September 26, 2020โ pytest 6.1.0 (2020-09-26)
๐ฅ Breaking Changes
โ #5585: As per our policy, the following features which have been deprecated in the 5.X series are now
โ removed:- The
funcargnames
read-only property ofFixtureRequest
,Metafunc
, andFunction
classes. Usefixturenames
attribute. @pytest.fixture
no longer supports positional arguments, pass all arguments by keyword instead.- Direct construction of
Node
subclasses now raise an error, usefrom_parent
instead. - The default value for
junit_family
has changed toxunit2
. If you require the old format, addjunit_family=xunit1
to your configuration file. - The
TerminalReporter
no longer has awriter
attribute. Plugin authors may use the public functions of theTerminalReporter
instead of accessing theTerminalWriter
object directly. - The
--result-log
option has been removed. Users are recommended to use the pytest-reportlog plugin instead.
For more information consult
โ Deprecations and Removals in the docs.๐ Deprecations
โ #6981: The
pytest.collect
module is deprecated: all its names can be imported frompytest
directly.โ #7097: The
pytest._fillfuncargs
function is deprecated. This function was kept
๐ for backward compatibility with an older plugin.It's functionality is not meant to be used directly, but if you must replace
it, use function._request._fillfixtures() instead, though note this is not
a public API and may break in the future.โ #7210: The special
-k '-expr'
syntax to-k
is deprecated. Use-k 'not expr'
instead.๐ The special
-k 'expr:'
syntax to-k
is deprecated. Please open an issue
if you use this and want a replacement.โ #7255: The pytest_warning_captured <_pytest.hookspec.pytest_warning_captured> hook is deprecated in favor
โ of pytest_warning_recorded <_pytest.hookspec.pytest_warning_recorded>, and will be removed in a future version.๐ฆ #7648: The
gethookproxy()
andisinitpath()
methods ofFSCollector
andPackage
are deprecated;
๐ useself.session.gethookproxy()
andself.session.isinitpath()
instead.
โ This should work on all pytest versions.๐ Features
- โ
#7667: New
--durations-min
command-line flag controls the minimal duration for inclusion in the slowest list of tests shown by--durations
. Previously this was hard-coded to0.005s
.
๐ Improvements
โ #6681: Internal pytest warnings issued during the early stages of initialization are now properly handled and can filtered through filterwarnings or
--pythonwarnings/-W
.โ This also fixes a number of long standing issues: #2891, #7620, #7426.
โ #7572: When a plugin listed in
required_plugins
is missing or an unknown config key is used with--strict-config
, a simple error message is now shown instead of a stacktrace.โ #7685: Added two new attributes rootpath <_pytest.config.Config.rootpath> and inipath <_pytest.config.Config.inipath> to Config <_pytest.config.Config>.
โ These attributes are pathlib.Path versions of the existing rootdir <_pytest.config.Config.rootdir> and inifile <_pytest.config.Config.inifile> attributes,
and should be preferred over them when possible.โ #7780: Public classes which are not designed to be inherited from are now marked @final.
Code which inherits from these classes will trigger a type-checking (e.g. mypy) error, but will still work in runtime.
Currently thefinal
designation does not appear in the API Reference but hopefully will in the future.๐ Bug Fixes
โ #1953: Fixed error when overwriting a parametrized fixture, while also reusing the super fixture value.
# conftest.py import pytest @pytest.fixture(params=[1, 2]) def foo(request): return request.param # test_foo.py import pytest @pytest.fixture def foo(foo): return foo * 2
โ #4984: Fixed an internal error crash with
IndexError: list index out of range
when
collecting a module which starts with a decorated function, the decorator
raises, and assertion rewriting is enabled.๐ #7591: pylint shouldn't complain anymore about unimplemented abstract methods when inheriting from File <non-python tests>.
๐ #7628: Fixed test collection when a full path without a drive letter was passed to pytest on Windows (for example
\projects\tests\test.py
instead ofc:\projects\tests\pytest.py
).๐ #7638: Fix handling of command-line options that appear as paths but trigger an OS-level syntax error on Windows, such as the options used internally by
pytest-xdist
.โ #7742: Fixed INTERNALERROR when accessing locals / globals with faulty
exec
.๐ Improved Documentation
- ๐ #1477: Removed faq.rst and its reference in contents.rst.
Trivial/Internal Changes
- โ
#7536: The internal
junitxml
plugin has rewritten to usexml.etree.ElementTree
.
The order of attributes in XML elements might differ. Some unneeded escaping is
no longer performed. - ๐ฆ #7587: The dependency on the
more-itertools
package has been removed. - โ
#7631: The result type of capfd.readouterr() <_pytest.capture.CaptureFixture.readouterr> (and similar) is no longer a namedtuple,
but should behave like one in all respects. This was done for technical reasons. - โ
#7671: When collecting tests, pytest finds test classes and functions by examining the
attributes of python objects (modules, classes and instances). To speed up this
process, pytest now ignores builtin attributes (like__class__
,
__delattr__
and__new__
) without consulting the python_classes and
๐ง python_functions configuration options and without passing them to plugins
using the pytest_pycollect_makeitem <_pytest.hookspec.pytest_pycollect_makeitem> hook.
- The
-
v6.0.2 Changes
September 11, 2020โ pytest 6.0.2 (2020-09-04)
๐ Bug Fixes
- โ
#7148: Fixed
--log-cli
potentially causing unrelatedprint
output to be swallowed. - โช #7672: Fixed log-capturing level restored incorrectly if
caplog.set_level
is called more than once. - โ
#7686: Fixed NotSetType.token being used as the parameter ID when the parametrization list is empty.
โ Regressed in pytest 6.0.0. - ๐
#7707: Fix internal error when handling some exceptions that contain multiple lines or the style uses multiple lines (
--tb=line
for example).
- โ
#7148: Fixed
-
v6.0.1 Changes
July 30, 2020โ pytest 6.0.1 (2020-07-30)
๐ Bug Fixes
- โ
#7394: Passing an empty
help
value toParser.add_option
is now accepted instead of crashing when runningpytest --help
.
PassingNone
raises a more informativeTypeError
. - ๐ #7558: Fix pylint
not-callable
lint onpytest.mark.parametrize()
and the other builtin marks:
โskip
,skipif
,xfail
,usefixtures
,filterwarnings
. - โ
#7559: Fix regression in plugins using
TestReport.longreprtext
(such aspytest-html
) whenTestReport.longrepr
is not a string. - โ
#7569: Fix logging capture handler's level not reset on teardown after a call to
caplog.set_level()
.
- โ
#7394: Passing an empty
-
v6.0.0 Changes
July 28, 2020โ pytest 6.0.0 (2020-07-28)
๐ ( Please see the full set of changes for this release also in the 6.0.0rc1 notes below )
๐ฅ Breaking Changes
โ #5584: PytestDeprecationWarning are now errors by default.
๐ Following our plan to remove deprecated features with as little disruption as
โ possible, all warnings of typePytestDeprecationWarning
now generate errors
โ instead of warning messages.๐ The affected features will be effectively removed in pytest 6.1 , so please consult the
โ Deprecations and Removals
โก๏ธ section in the docs for directions on how to update existing code.โ In the pytest
6.0.X
series, it is possible to change the errors back into warnings as a
โ stopgap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.PytestDeprecationWarning
๐ But this will stop working when pytest
6.1
is released.If you have concerns about the removal of a specific feature, please add a
โ comment to #5584.#7472: The
exec_()
andis_true()
methods of_pytest._code.Frame
have been removed.๐ Features
- #7464: Added support for NO_COLOR and FORCE_COLOR environment variables to control colored output.
๐ Improvements
- โ
#7467:
--log-file
CLI option andlog_file
ini marker now create subdirectories if needed. - โ
#7489: The pytest.raises function has a clearer error message when
match
equals the obtained string but is not a regex match. In this case it is suggested to escape the regex.
๐ Bug Fixes
- โ
#7392: Fix the reported location of tests skipped with
@pytest.mark.skip
when--runxfail
is used. - ๐ #7491: tmpdir and tmp_path no longer raise an error if the lock to check for
stale temporary directories is not accessible. - โ
#7517: Preserve line endings when captured via
capfd
. - #7534: Restored the previous formatting of
TracebackEntry. __str__
which was changed by accident.
๐ Improved Documentation
- โ
#7422: Clarified when the
usefixtures
mark can apply fixtures to test. - โ
#7441: Add a note about
-q
option used in getting started guide.
Trivial/Internal Changes
- ๐ฆ #7389: Fixture scope
package
is no longer considered experimental.
โ pytest 6.0.0rc1 (2020-07-08)
๐ฅ Breaking Changes
โ #1316:
TestReport.longrepr
is now always an instance ofReprExceptionInfo
. Previously it was astr
when a test failed withpytest.fail(..., pytrace=False)
.โ #5965: symlinks are no longer resolved during collection and matching conftest.py files with test file paths.
๐ Resolving symlinks for the current directory and during collection was introduced as a bugfix in 3.9.0, but it actually is a new feature which had unfortunate consequences in Windows and surprising results in other platforms.
๐ The team decided to step back on resolving symlinks at all, planning to review this in the future with a more solid solution (see discussion in
โ #6523 for details).โ This might break test suites which made use of this feature; the fix is to create a symlink
โ for the entire test tree, and not only to partial files/tress as it was possible previously.โ #6505:
Testdir.run().parseoutcomes()
now always returns the parsed nouns in plural form.๐ Originally
parseoutcomes()
would always returns the nouns in plural form, but a change
โ meant to improve the terminal summary by using singular form single items (1 warning
or1 error
)
๐ caused an unintended regression by changing the keys returned byparseoutcomes()
.Now the API guarantees to always return the plural form, so calls like this:
result = testdir.runpytest() result.assert_outcomes(error=1)
Need to be changed to:
result = testdir.runpytest() result.assert_outcomes(errors=1)
โ #6903: The
os.dup()
function is now assumed to exist. We are not aware of any
๐ supported Python 3 implementations which do not provide it.โ #7040:
-k
no longer matches against the names of the directories outside the test session root.๐ฆ Also,
pytest.Package.name
is now just the name of the directory containing the package's
__init__.py
file, instead of the full path. This is consistent with how the other nodes
are named, and also one of the reasons why-k
would match against any directory containing
โ the test suite.โ #7122: Expressions given to the
-m
and-k
options are no longer evaluated using Python's eval.
๐ The format supportsor
,and
,not
, parenthesis and general identifiers to match against.
Python constants, keywords or other operators are no longer evaluated differently.โ #7135: Pytest now uses its own
TerminalWriter
class instead of using the one from thepy
library.
๐ Plugins generally access this class throughTerminalReporter.writer
,TerminalReporter.write()
โ (and similar methods), or_pytest.config.create_terminal_writer()
.The following breaking changes were made:
- Output (
write()
method and others) no longer flush implicitly; the flushing behavior
of the underlying file is respected. To flush explicitly (for example, if you
๐จ want output to be shown before an end-of-line is printed), usewrite(flush=True)
or
terminal_writer.flush()
. - Explicit Windows console support was removed, delegated to the colorama library.
- Support for writing
bytes
was removed. - The
reline
method andchars_on_current_line
property were removed. - The
stringio
andencoding
arguments was removed.
- Support for passing a callable instead of a file was removed.
๐ฒ #7224: The item.catch_log_handler and item.catch_log_handlers attributes, set by the
๐ logging plugin and never meant to be public , are no longer available.๐ The deprecated
--no-print-logs
option is removed. Use--show-capture
instead.#7226: Removed the unused
args
parameter frompytest.Function. __init__
.โ #7418: Removed the pytest_doctest_prepare_content hook specification. This hook
โ hasn't been triggered by pytest for at least 10 years.โ #7438: Some changes were made to the internal
_pytest._code.source
, listed here
๐ for the benefit of plugin authors who may be using it:- The
deindent
argument toSource()
has been removed, now it is always true. - Support for zero or multiple arguments to
Source()
has been removed. - Support for comparing
Source
with anstr
has been removed. - The methods
Source.isparseable()
andSource.putaround()
have been removed. - The method
Source.compile()
and function_pytest._code.compile()
have
๐ been removed; use plaincompile()
instead. - The function
_pytest._code.source.getsource()
has been removed; use
Source()
directly instead.
๐ Deprecations
โ #7210: The special
-k '-expr'
syntax to-k
is deprecated. Use-k 'not expr'
instead.๐ The special
-k 'expr:'
syntax to-k
is deprecated. Please open an issue
if you use this and want a replacement.โ #4049:
pytest_warning_captured
is deprecated in favor of thepytest_warning_recorded
hook.๐ Features
๐ง #1556: pytest now supports
pyproject.toml
files for configuration.๐ง The configuration options is similar to the one available in other formats, but must be defined
โ in a[tool.pytest.ini_options]
table to be picked up by pytest:# pyproject.toml [tool.pytest.ini_options] minversion = "6.0" addopts = "-ra -q" testpaths = [ "tests", "integration", ]
๐ง More information can be found in the docs.
โ #3342: pytest now includes inline type annotations and exposes them to user programs.
Most of the user-facing API is covered, as well as internal code.โ If you are running a type checker such as mypy on your tests, you may start
noticing type errors indicating incorrect usage. If you run into an error that
you believe to be incorrect, please let us know in an issue.The types were developed against mypy version 0.780. Versions before 0.750
โ are known not to work. We recommend using the latest version. Other type
checkers may work as well, but they are not officially verified to work by
โ pytest yet.โ #4049: Introduced a new hook named pytest_warning_recorded to convey information about warnings captured by the internal pytest warnings plugin.
โ This hook is meant to replace pytest_warning_captured, which is deprecated and will be removed in a future release.
โ #6471: New command-line flags:
--no-header
: disables the initial header, including platform, version, and plugins.
-
--no-summary
: disables the final test summary, including warnings.โ #6856: A warning is now shown when an unknown key is read from a config INI file.
โ The --strict-config flag has been added to treat these warnings as errors.
โ #6906: Added --code-highlight command line option to enable/disable code highlighting in terminal output.
โ #7245: New
--import-mode=importlib
option that uses importlib to import test modules.Traditionally pytest used
__import__
while changingsys.path
to import test modules (which
โ also changessys.modules
as a side-effect), which works but has a number of drawbacks, like requiring test modules
๐ฆ that don't live in packages to have unique names (as they need to reside under a unique name insys.modules
).--import-mode=importlib
uses more fine grained import mechanisms fromimportlib
which don't
โ require pytest to changesys.path
orsys.modules
at all, eliminating much of the drawbacks
of the previous mode.0๏ธโฃ We intend to make
--import-mode=importlib
the default in future versions, so users are encouraged
โ to try the new mode and provide feedback (both positive or negative) in issue #7245.๐ You can read more about this option in the documentation.
๐ง #7305: New
required_plugins
configuration option allows the user to specify a list of plugins, including version information, that are required for pytest to run. An error is raised if any required plugins are not found when running pytest.๐ Improvements
โ #4375: The
pytest
command now suppresses theBrokenPipeError
error message that
โ is printed to stderr when the output ofpytest
is piped and and the pipe is
closed by the piped-to program (common examples areless
andhead
).โ #4391: Improved precision of test durations measurement.
CallInfo
items now have a new<CallInfo>.duration
attribute, created usingtime.perf_counter()
. This attribute is used to fill the<TestReport>.duration
attribute, which is more accurate than the previous<CallInfo>.stop - <CallInfo>.start
(as these are based ontime.time()
).โ #4675: Rich comparison for dataclasses and attrs-classes is now recursive.
โ #6285: Exposed the pytest.FixtureLookupError exception which is raised by request.getfixturevalue()
(where request is a FixtureRequest fixture) when a fixture with the given name cannot be returned.โ #6433: If an error is encountered while formatting the message in a logging call, for
โ examplelogging.warning("oh no!: %s: %s", "first")
(a second argument is
โ missing), pytest now propagates the error, likely causing the test to fail.๐จ Previously, such a mistake would cause an error to be printed to stderr, which
โ is not displayed by default for passing tests. This change makes the mistake
โ visible during testing.You may supress this behavior temporarily or permanently by setting
๐logging.raiseExceptions = False
.โ #6817: Explicit new-lines in help texts of command-line options are preserved, allowing plugins better control
of the help displayed to users.โ #6940: When using the
--duration
option, the terminal message output is now more precise about the number and duration of hidden items.โ #6991: Collected files are displayed after any reports from hooks, e.g. the status from
--lf
.โ #7091: When
fd
capturing is used, through--capture=fd
or thecapfd
and
capfdbinary
fixtures, and the file descriptor (0, 1, 2) cannot be
duplicated, FD capturing is still performed. Previously, direct writes to the
file descriptors would fail or be lost in this case.โ #7119: Exit with an error if the
--basetemp
argument is empty, is the current working directory or is one of the parent directories.
This is done to protect against accidental data loss, as any directory passed to this argument is cleared.โ #7128: pytest --version now displays just the pytest version, while pytest --version --version displays more verbose information including plugins. This is more consistent with how other tools show --version.
#7133: caplog.set_level() <_pytest.logging.LogCaptureFixture.set_level> will now override any log_level set via the CLI or configuration file.
#7159: caplog.set_level() <_pytest.logging.LogCaptureFixture.set_level> and caplog.at_level() <_pytest.logging.LogCaptureFixture.at_level> no longer affect
๐ฒ the level of logs that are shown in the Captured log report report section.โ #7348: Improve recursive diff report for comparison asserts on dataclasses / attrs.
โ #7385:
--junitxml
now includes the exception cause in themessage
XML attribute for failures during setup and teardown.Previously:
<error message="test setup failure">
Now:
<error message="failed on setup with "ValueError: Some error during setup"">
๐ Bug Fixes
๐ #1120: Fix issue where directories from tmpdir are not removed properly when multiple instances of pytest are running in parallel.
โ #4583: Prevent crashing and provide a user-friendly error when a marker expression (-m) invoking of eval raises any exception.
โ #4677: The path shown in the summary report for SKIPPED tests is now always relative. Previously it was sometimes absolute.
๐ #5456: Fix a possible race condition when trying to remove lock files used to control access to folders
created by tmp_path and tmpdir.โ #6240: Fixes an issue where logging during collection step caused duplication of log
messages to stderr.โ #6428: Paths appearing in error messages are now correct in case the current working directory has
๐ changed since the start of the session.๐ #6755: Support deleting paths longer than 260 characters on windows created inside tmpdir.
โ #6871: Fix crash with captured output when using capsysbinary.
โช #6909: Revert the change introduced by #6330, which required all arguments to
@pytest.mark.parametrize
to be explicitly defined in the function signature.โช The intention of the original change was to remove what was expected to be an unintended/surprising behavior, but it turns out many people relied on it, so the restriction has been reverted.
โ #6910: Fix crash when plugins return an unknown stats while using the
--reportlog
option.โ #6924: Ensure a
unittest.IsolatedAsyncioTestCase
is actually awaited.โ #6925: Fix TerminalRepr instances to be hashable again.
โ #6947: Fix regression where functions registered with unittest.TestCase.addCleanup were not being called on test failures.
โ #6951: Allow users to still set the deprecated
TerminalReporter.writer
attribute.โ #6956: Prevent pytest from printing ConftestImportFailure traceback to stdout.
โ #6991: Fix regressions with --lf filtering too much since pytest 5.4.
โช #6992: Revert "tmpdir: clean up indirection via config for factories" #6767 as it breaks pytest-xdist.
โ #7061: When a yielding fixture fails to yield a value, report a test setup error instead of crashing.
โ #7076: The path of file skipped by
@pytest.mark.skip
in the SKIPPED report is now relative to invocation directory. Previously it was relative to root directory.โ #7110: Fixed regression:
asyncbase.TestCase
tests are executed correctly again.โ #7126:
--setup-show
now doesn't raise an error when a bytes value is used as aparametrize
parameter when Python is called with the-bb
flag.โ #7143: Fix pytest.File.from_parent so it forwards extra keyword arguments to the constructor.
#7145: Classes with broken
__getattribute__
methods are displayed correctly during failures.โ #7150: Prevent hiding the underlying exception when
ConfTestImportFailure
is raised.#7180: Fix
_is_setup_py
for files encoded differently than locale.โ #7215: Fix regression where running with
--pdb
would call unittest.TestCase.tearDown for skipped tests.โ #7253: When using
pytest.fixture
on a function directly, as inpytest.fixture(func)
,
if theautouse
orparams
arguments are also passed, the function is no longer
ignored, but is marked as a fixture.โ #7360: Fix possibly incorrect evaluation of string expressions passed to
pytest.mark.skipif
andpytest.mark.xfail
,
in rare circumstances where the exact same string is used but refers to different global values.๐ป #7383: Fixed exception causes all over the codebase, i.e. use raise new_exception from old_exception when wrapping an exception.
๐ Improved Documentation
- โ #7202: The development guide now links to the contributing section of the docs and RELEASING.rst on GitHub.
- โ
#7233: Add a note about
--strict
and--strict-markers
and the preference for the latter one. - โ #7345: Explain indirect parametrization and markers for fixtures.
Trivial/Internal Changes
- โ
#7035: The
originalname
attribute of_pytest.python.Function
now defaults toname
if not
provided explicitly, and is always set. - ๐ฆ #7264: The dependency on the
wcwidth
package has been removed. - โ
#7291: Replaced
py.iniconfig
with iniconfig. - โ
#7295:
src/_pytest/config/ __init__.py
now uses thewarnings
module to report warnings instead ofsys.stderr.write
. - #7356: Remove last internal uses of deprecated slave term from old
pytest-xdist
. - โ
#7357:
py
>=1.8.2 is now required.
-
v6.0.0.rc1
July 08, 2020 -
v5.4.3 Changes
June 02, 2020โ pytest 5.4.3 (2020-06-02)
๐ Bug Fixes
- โ
#6428: Paths appearing in error messages are now correct in case the current working directory has
๐ changed since the start of the session. - ๐ #6755: Support deleting paths longer than 260 characters on windows created inside tmpdir.
- โ #6956: Prevent pytest from printing ConftestImportFailure traceback to stdout.
- โ
#7150: Prevent hiding the underlying exception when
ConfTestImportFailure
is raised. - โ
#7215: Fix regression where running with
--pdb
would call thetearDown
methods ofunittest.TestCase
โ subclasses for skipped tests.
- โ
#6428: Paths appearing in error messages are now correct in case the current working directory has
-
v5.4.2 Changes
May 08, 2020โ pytest 5.4.2 (2020-05-08)
๐ Bug Fixes
- โ #6871: Fix crash with captured output when using the capsysbinary fixture <capsysbinary>.
- โ
#6924: Ensure a
unittest.IsolatedAsyncioTestCase
is actually awaited. - โ #6925: Fix TerminalRepr instances to be hashable again.
- โ
#6947: Fix regression where functions registered with
TestCase.addCleanup
were not being called on test failures. - โ
#6951: Allow users to still set the deprecated
TerminalReporter.writer
attribute. - โช #6992: Revert "tmpdir: clean up indirection via config for factories" #6767 as it breaks pytest-xdist.
- โ
#7110: Fixed regression:
asyncbase.TestCase
tests are executed correctly again. - โ
#7143: Fix
File.from_constructor
so it forwards extra keyword arguments to the constructor. - #7145: Classes with broken
__getattribute__
methods are displayed correctly during failures. - #7180: Fix
_is_setup_py
for files encoded differently than locale.