rename AbstractMatcher -> AbstractRaises, Matcher->RaisesExc. Add docs on RaisesGroup&RaisesExc. Add warnings to group_contains. Remove group_contains example from getting-started page

Author: jakkdl

changelog/11538.feature.rst

@@ -0,0 +1 @@
+Added :class:`pytest.RaisesGroup` (also export as ``pytest.raises_group``) and :class:`pytest.RaisesExc`, as an equivalent to :func:`pytest.raises` for expecting :exc:`ExceptionGroup`. It includes the ability to specify multiple different expected exceptions, the structure of nested exception groups, and flags for emulating :ref:`except* <except_star>`. See :ref:`assert-matching-exception-groups` and docstrings for more information.

changelog/11671.feature.rst

@@ -0,0 +1 @@
+:func:`pytest.mark.xfail` now accepts :class:`pytest.RaisesGroup` for the ``raises`` parameter when you expect an exception group. You can also pass a :class:`pytest.RaisesExc` if you e.g. want to make use of the ``check`` parameter.

changelog/12504.feature.rst

@@ -106,6 +106,8 @@
     ("py:obj", "_pytest.fixtures.FixtureValue"),
     ("py:obj", "_pytest.stash.T"),
     ("py:class", "_ScopeName"),
+    ("py:class", "BaseExcT_1"),
+    ("py:class", "ExcT_1"),
 ]
 
 add_module_names = False

doc/en/conf.py

@@ -97,30 +97,6 @@ Use the :ref:`raises <assertraises>` helper to assert that some code raises an e
         with pytest.raises(SystemExit):
             f()
 
-You can also use the context provided by :ref:`raises <assertraises>` to
-assert that an expected exception is part of a raised :class:`ExceptionGroup`:
-
-.. code-block:: python
-
-    # content of test_exceptiongroup.py
-    import pytest
-
-
-    def f():
-        raise ExceptionGroup(
-            "Group message",
-            [
-                RuntimeError(),
-            ],
-        )
-
-
-    def test_exception_in_group():
-        with pytest.raises(ExceptionGroup) as excinfo:
-            f()
-        assert excinfo.group_contains(RuntimeError)
-        assert not excinfo.group_contains(TypeError)
-
 Execute the test function with “quiet” reporting mode:
 
 .. code-block:: pytest
@@ -133,6 +109,8 @@ Execute the test function with “quiet” reporting mode:
 
     The ``-q/--quiet`` flag keeps the output brief in this and following examples.
 
+See :ref:`assertraises` for specifying more details about the expected exception.
+
 Group multiple tests in a class
 --------------------------------------------------------------
 

doc/en/getting-started.rst

@@ -145,8 +145,93 @@ Notes:
 
 .. _`assert-matching-exception-groups`:
 
-Matching exception groups
-~~~~~~~~~~~~~~~~~~~~~~~~~
+Assertions about expected exception groups
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When expecting a :exc:`BaseExceptionGroup` or :exc:`ExceptionGroup` you can use :class:`pytest.RaisesGroup`, also available as :class:`pytest.raises_group <pytest.RaisesGroup>`:
+
+.. code-block:: python
+
+    def test_exception_in_group():
+        with pytest.raises_group(ValueError):
+            raise ExceptionGroup("group msg", [ValueError("value msg")])
+        with pytest.raises_group(ValueError, TypeError):
+            raise ExceptionGroup("msg", [ValueError("foo"), TypeError("bar")])
+
+
+It accepts a ``match`` parameter, that checks against the group message, and a ``check`` parameter that takes an arbitrary callable which it passes the group to, and only succeeds if the callable returns ``True``.
+
+.. code-block:: python
+
+    def test_raisesgroup_match_and_check():
+        with pytest.raises_group(BaseException, match="my group msg"):
+            raise BaseExceptionGroup("my group msg", [KeyboardInterrupt()])
+        with pytest.raises_group(
+            Exception, check=lambda eg: isinstance(eg.__cause__, ValueError)
+        ):
+            raise ExceptionGroup("", [TypeError()]) from ValueError()
+
+It is strict about structure and unwrapped exceptions, unlike :ref:`except* <except_star>`, so you might want to set the ``flatten_subgroups`` and/or ``allow_unwrapped`` parameters.
+
+.. code-block:: python
+
+    def test_structure():
+        with pytest.raises_group(pytest.raises_group(ValueError)):
+            raise ExceptionGroup("", (ExceptionGroup("", (ValueError(),)),))
+        with pytest.raises_group(ValueError, flatten_subgroups=True):
+            raise ExceptionGroup("1st group", [ExceptionGroup("2nd group", [ValueError()])])
+        with pytest.raises_group(ValueError, allow_unwrapped=True):
+            raise ValueError
+
+To specify more details about the contained exception you can use :class:`pytest.RaisesExc`
+
+.. code-block:: python
+
+    def test_raises_exc():
+        with pytest.raises_group(pytest.RaisesExc(ValueError, match="foo")):
+            raise ExceptionGroup("", (ValueError("foo")))
+
+They both supply a method :meth:`pytest.RaisesGroup.matches` :meth:`pytest.RaisesExc.matches` if you want to do matching outside of using it as a contextmanager. This can be helpful when checking ``.__context__`` or ``.__cause__``.
+
+.. code-block:: python
+
+    def test_matches():
+        exc = ValueError()
+        exc_group = ExceptionGroup("", [exc])
+        if RaisesGroup(ValueError).matches(exc_group):
+            ...
+        # helpful error is available in `.fail_reason` if it fails to match
+        r = RaisesExc(ValueError)
+        assert r.matches(e), r.fail_reason
+
+Check the documentation on :class:`pytest.RaisesGroup` and :class:`pytest.RaisesExc` for more details and examples.
+
+``ExceptionInfo.group_contains()``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. warning::
+
+   This helper makes it easy to check for the presence of specific exceptions, but it is very bad for checking that the group does *not* contain *any other exceptions*. So this will pass:
+
+    .. code-block:: python
+
+       class EXTREMELYBADERROR(BaseException):
+           """This is a very bad error to miss"""
+
+
+       def test_for_value_error():
+           with pytest.raises(ExceptionGroup) as excinfo:
+               excs = [ValueError()]
+               if very_unlucky():
+                   excs.append(EXTREMELYBADERROR())
+               raise ExceptionGroup("", excs)
+           # this passes regardless of if there's other exceptions
+           assert excinfo.group_contains(ValueError)
+           # you can't simply list all exceptions you *don't* want to get here
+
+
+   There is no good way of using :func:`excinfo.group_contains() <pytest.ExceptionInfo.group_contains>` to ensure you're not getting *any* other exceptions than the one you expected.
+   You should instead use :class:`pytest.raises_group <pytest.RaisesGroup>`, see :ref:`assert-matching-exception-groups`.
 
 You can also use the :func:`excinfo.group_contains() <pytest.ExceptionInfo.group_contains>`
 method to test for exceptions returned as part of an :class:`ExceptionGroup`:

doc/en/how-to/assert.rst

@@ -1014,6 +1014,18 @@ PytestPluginManager
     :inherited-members:
     :show-inheritance:
 
+RaisesExc
+~~~~~~~~~
+
+.. autoclass:: pytest.RaisesExc()
+    :members:
+
+RaisesGroup
+~~~~~~~~~~~
+
+.. autoclass:: pytest.RaisesGroup()
+    :members:
+
 TerminalReporter
 ~~~~~~~~~~~~~~~~
 

doc/en/reference/reference.rst

@@ -828,6 +828,13 @@ def group_contains(
             the exceptions contained within the topmost exception group).
 
         .. versionadded:: 8.0
+
+        .. warning::
+           This helper makes it easy to check for the presence of specific exceptions,
+           but it is very bad for checking that the group does *not* contain
+           *any other exceptions*.
+           You should instead consider using :class:`pytest.raises_group <pytest.RaisesGroup>`
+
         """
         msg = "Captured exception is not an instance of `BaseExceptionGroup`"
         assert isinstance(self.value, BaseExceptionGroup), msg