"The test […] is not an async function" when asyncio marker is added via pytest_collection_modifyitems()

[mgorny]

This one I've noticed while testing IPython. The simplest reproducer is:

conftest.py:

import inspect


def pytest_collection_modifyitems(items):
    for item in items:
        if inspect.iscoroutinefunction(item.obj):
            item.add_marker("asyncio")

test_foo.py:

async def test_foo():
    pass

With pytest-asyncio 0.23.6, I'm getting:

========================================================= test session starts =========================================================
platform linux -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0
rootdir: /tmp/repro
plugins: asyncio-0.23.6
asyncio: mode=Mode.STRICT
collected 1 item                                                                                                                      

test_foo.py s                                                                                                                   [100%]

========================================================== warnings summary ===========================================================
test_foo.py::test_foo
  test_foo.py:5: PytestWarning: The test <Function test_foo> is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'.
    async def test_foo():

test_foo.py::test_foo
  /tmp/repro/.venv/lib/python3.11/site-packages/_pytest/python.py:184: PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
  You need to install a suitable plugin for your async framework, for example:
    - anyio
    - pytest-asyncio
    - pytest-tornasync
    - pytest-trio
    - pytest-twisted
    warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================================== 1 skipped, 2 warnings in 0.01s ====================================================

With pytest-asyncio-0.21.1, the test is correctly run as marked.

[mgorny]

Also affects PyZMQ.

[seifertm]

Still reproducible in pytest-asyncio v1.0.0a1.