Skip to content

Comments

Tcp socket factory 3.12#10565

Closed
TimMenninger wants to merge 2 commits intoaio-libs:3.12from
TimMenninger:tcp-socket-factory-3.12
Closed

Tcp socket factory 3.12#10565
TimMenninger wants to merge 2 commits intoaio-libs:3.12from
TimMenninger:tcp-socket-factory-3.12

Conversation

@TimMenninger
Copy link
Contributor

What do these changes do?

Replace tcp_sockopts parameter with a socket_factory parameter that
is a callback allowing the caller to own socket creation. If passed, all
sockets created by TCPConnector are expected to come from the
socket_factory callback.

Are there changes in behavior for the user?

The only users to experience a change in behavior are those who are
using the un-released tcp_sockopts argument to TCPConnector.
However, using unreleased code comes with caveat emptor, and is why I
felt entitled to remove the option entirely without warning.

Is it a substantial burden for the maintainers to support this?

The burden will be minimal and would only arise if aiohappyeyeballs
changes their interface.

Related issue number

Fixes #10520

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modification, please add yourself to
    CONTRIBUTORS.txt
    • The format is <Name> <Surname>.
    • Please keep alphabetical order, the file is sorted by names.
  • Add a new news fragment into the CHANGES/ folder
    • name it <issue_or_pr_num>.<type>.rst (e.g. 588.bugfix.rst)

    • if you don't have an issue number, change it to the pull request
      number after creating the PR

      • .bugfix: A bug fix for something the maintainers deemed an
        improper undesired behavior that got corrected to match
        pre-agreed expectations.
      • .feature: A new behavior, public APIs. That sort of stuff.
      • .deprecation: A declaration of future API removals and breaking
        changes in behavior.
      • .breaking: When something public is removed in a breaking way.
        Could be deprecated in an earlier release.
      • .doc: Notable updates to the documentation structure or build
        process.
      • .packaging: Notes for downstreams about unobvious side effects
        and tooling. Changes in the test invocation considerations and
        runtime assumptions.
      • .contrib: Stuff that affects the contributor experience. e.g.
        Running tests, building the docs, setting up the development
        environment.
      • .misc: Changes that are hard to assign to any of the above
        categories.
    • Make sure to use full sentences with correct case and punctuation,
      for example:

      Fixed issue with non-ascii contents in doctest text files
      -- by :user:`contributor-gh-handle`.

      Use the past tense or the present tense a non-imperative mood,
      referring to what's changed compared to the last released version
      of this project.


Co-authored-by: J. Nick Koston nick@koston.org

WITH CONFLICTS (resolved in subsequent commit):
        both modified:   aiohttp/__init__.py
        both modified:   docs/conf.py

Instead of TCPConnector taking a list of sockopts to be applied sockets
created, take a socket_factory callback that allows the caller to
implement socket creation entirely.

Fixes aio-libs#10520
@psf-chronographer psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Mar 16, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 16, 2025

CodSpeed Performance Report

Merging #10565 will not alter performance

Comparing TimMenninger:tcp-socket-factory-3.12 (5d55c87) with 3.12 (a3c36ad)

Summary

✅ 47 untouched benchmarks

@codecov
Copy link

codecov bot commented Mar 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.08%. Comparing base (a3c36ad) to head (5d55c87).
Report is 257 commits behind head on 3.12.

Additional details and impacted files
@@           Coverage Diff           @@
##             3.12   #10565   +/-   ##
=======================================
  Coverage   98.08%   98.08%           
=======================================
  Files         123      123           
  Lines       37450    37455    +5     
  Branches     2122     2120    -2     
=======================================
+ Hits        36732    36737    +5     
  Misses        548      548           
  Partials      170      170           
Flag Coverage Δ
CI-GHA 97.97% <100.00%> (+<0.01%) ⬆️
OS-Linux 97.67% <100.00%> (+<0.01%) ⬆️
OS-Windows 94.74% <100.00%> (+<0.01%) ⬆️
OS-macOS 96.81% <100.00%> (+<0.01%) ⬆️
Py-3.10.11 96.69% <100.00%> (+<0.01%) ⬆️
Py-3.10.16 97.26% <100.00%> (+<0.01%) ⬆️
Py-3.11.11 97.36% <100.00%> (+<0.01%) ⬆️
Py-3.11.9 96.80% <100.00%> (+<0.01%) ⬆️
Py-3.12.9 97.77% <100.00%> (+<0.01%) ⬆️
Py-3.13.2 97.75% <100.00%> (+<0.01%) ⬆️
Py-3.9.13 96.60% <100.00%> (+<0.01%) ⬆️
Py-3.9.21 97.14% <100.00%> (+<0.01%) ⬆️
Py-pypy7.3.16 90.03% <100.00%> (-3.55%) ⬇️
VM-macos 96.81% <100.00%> (+<0.01%) ⬆️
VM-ubuntu 97.67% <100.00%> (+<0.01%) ⬆️
VM-windows 94.74% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TimMenninger TimMenninger force-pushed the tcp-socket-factory-3.12 branch from 9a35bc2 to 5d55c87 Compare March 16, 2025 17:01
@TimMenninger
Copy link
Contributor Author

@bdraco @Dreamsorcerer let me know if you want me to squash the commits

@TimMenninger TimMenninger marked this pull request as ready for review March 16, 2025 17:05
Copy link
Member

@webknjaz webknjaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a few problems in the docs files.

Comment on lines +468 to +469
`socket_factory` to the :class:`~aiohttp.TCPConnector`, which implements
`SocketFactoryType`. This will be used to create all sockets for the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use double backticks in RST.

Comment on lines +1149 to +1151
Be sure to use ``aiohttp.AddrInfoType`` rather than
``aiohappyeyeballs.AddrInfoType`` to avoid import breakage, as
it is likely to be removed from ``aiohappyeyeballs`` in the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use refs here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to make these changes here, on the backport PR, or to commit some fixups to master to be backported? Or to commit them here then cherry-pick to master?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better yet, to make them in master, then add them to this PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little cleaner to do the backport with no changes, merge it, then make the changes on master and the bot should be able to backport them from master to 3.12

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry for the novice question, but how do I do that? I get

/Users/tmenninger/Downloads/aiohttp/docs/client_reference.rst:1136: WARNING: undefined label: 'aiohappyeyeballs' [ref.ref]
/Users/tmenninger/Downloads/aiohttp/docs/client_reference.rst:1150: WARNING: undefined label: 'aiohappyeyeballs' [ref.ref]

Am I supposed to define it somewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIP at #10568

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry for the novice question, but how do I do that? I get

/Users/tmenninger/Downloads/aiohttp/docs/client_reference.rst:1136: WARNING: undefined label: 'aiohappyeyeballs' [ref.ref]
/Users/tmenninger/Downloads/aiohttp/docs/client_reference.rst:1150: WARNING: undefined label: 'aiohappyeyeballs' [ref.ref]

Am I supposed to define it somewhere?

If aiohappyeyeballs' docs have these objects, it'll work. You can verify which objects are exposed/exist via python -Im sphinx.ext.intersphinx. For non-existing objects, it's probably okay to ignore for now. However, you might find that there may be objects of different roles that you can link instead. Like :doc: instead of :ref: is okay sometimes.

:param list tcp_sockopts: options applied to the socket when a connection is
created. This should be a list of 3-tuples, each a ``(level, optname, value)``.
Each tuple is deconstructed and passed verbatim to ``<socket>.setsockopt``.
:param :py:data:``SocketFactoryType`` socket_factory: This function takes an
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The correct syntax is

Suggested change
:param :py:data:``SocketFactoryType`` socket_factory: This function takes an
:param SocketFactoryType socket_factory: This function takes an

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I did this to get around

/Users/tmenninger/Downloads/aiohttp/docs/client_reference.rst:1015: WARNING: py:class reference target not found: SocketFactoryType [ref.class]

Is there a correct syntax to force it to consider SocketFactoryType a data rather than class?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. OTOH, everything in Python is an object and has a class. Param definitions expect classes / type annotations.
Also it's possible to move this into a separate :type: entry. I find that syntax cleaner.
SocketFactoryType looks like a constant and so I don't believe it to be a proper value anyway.

created. This should be a list of 3-tuples, each a ``(level, optname, value)``.
Each tuple is deconstructed and passed verbatim to ``<socket>.setsockopt``.
:param :py:data:``SocketFactoryType`` socket_factory: This function takes an
:py:data:``AddrInfoType`` and is used in lieu of ``socket.socket()`` when
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:py:data:``AddrInfoType`` and is used in lieu of ``socket.socket()`` when
:py:data:`AddrInfoType` and is used in lieu of :py:func:`socket.socket` when

want to change the conditions under which we consider a connection dead.
The following would change that to 9*7200 = 18 hours::
If the default socket is insufficient for your use case, pass an optional
`socket_factory` to the :class:`~aiohttp.TCPConnector`, which implements
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`socket_factory` to the :class:`~aiohttp.TCPConnector`, which implements
``socket_factory`` to the :class:`~aiohttp.TCPConnector`, which implements

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps symlink this change note to the new one. Both are a part of this, right?

@bdraco
Copy link
Member

bdraco commented Mar 16, 2025

I took care of the backport in #10574 so once we have the docs PR for master ready, it should cherry-pick cleanly to 3.12

@bdraco bdraco closed this in 240fb9e Mar 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants