Replace tcp_sockopts with socket_factory #10534

TimMenninger · 2025-03-10T15:45:21Z

Instead of TCPConnector taking a list of sockopts to be applied sockets created, take a socket_factory callback that allows the caller to implement socket creation entirely.

Fixes #10520

What do these changes do?

Replace tcp_sockopts parameter with a socket_factory parameter that is a callback allowing the caller to own socket creation. If passed, all sockets created by TCPConnector are expected to come from the socket_factory callback.

Are there changes in behavior for the user?

The only users to experience a change in behavior are those who are using the un-released tcp_sockopts argument to TCPConnector. However, using unreleased code comes with caveat emptor, and is why I felt entitled to remove the option entirely without warning.

Is it a substantial burden for the maintainers to support this?

The burden will be minimal and would only arise if aiohappyeyeballs changes their interface.

Related issue number

Checklist

I think the code is well written
Unit tests for the changes exist
Documentation reflects the changes
If you provide code modification, please add yourself to CONTRIBUTORS.txt
- The format is <Name> <Surname>.
- Please keep alphabetical order, the file is sorted by names.
Add a new news fragment into the CHANGES/ folder
- name it <issue_or_pr_num>.<type>.rst (e.g. 588.bugfix.rst)
- if you don't have an issue number, change it to the pull request
  number after creating the PR
  - .bugfix: A bug fix for something the maintainers deemed an
    improper undesired behavior that got corrected to match
    pre-agreed expectations.
  - .feature: A new behavior, public APIs. That sort of stuff.
  - .deprecation: A declaration of future API removals and breaking
    changes in behavior.
  - .breaking: When something public is removed in a breaking way.
    Could be deprecated in an earlier release.
  - .doc: Notable updates to the documentation structure or build
    process.
  - .packaging: Notes for downstreams about unobvious side effects
    and tooling. Changes in the test invocation considerations and
    runtime assumptions.
  - .contrib: Stuff that affects the contributor experience. e.g.
    Running tests, building the docs, setting up the development
    environment.
  - .misc: Changes that are hard to assign to any of the above
    categories.
- Make sure to use full sentences with correct case and punctuation,
  for example:
```
Fixed issue with non-ascii contents in doctest text files
-- by :user:`contributor-gh-handle`.
```
  Use the past tense or the present tense a non-imperative mood,
  referring to what's changed compared to the last released version
  of this project.

codspeed-hq · 2025-03-10T15:51:43Z

CodSpeed Performance Report

Merging #10534 will not alter performance

_{Comparing TimMenninger:tcp-socket-factory (ea62f2f) with master (e507ba3)}

Summary

✅ 46 untouched benchmarks

codecov · 2025-03-10T19:28:07Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.69%. Comparing base (45b861f) to head (ea62f2f).
Report is 6 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #10534   +/-   ##
=======================================
  Coverage   98.69%   98.69%           
=======================================
  Files         122      122           
  Lines       37230    37235    +5     
  Branches     2064     2062    -2     
=======================================
+ Hits        36745    36750    +5     
  Misses        338      338           
  Partials      147      147

Flag	Coverage Δ
CI-GHA	`98.57% <100.00%> (+<0.01%)`	⬆️
OS-Linux	`98.24% <100.00%> (+<0.01%)`	⬆️
OS-Windows	`96.18% <100.00%> (+<0.01%)`	⬆️
OS-macOS	`97.35% <100.00%> (-0.01%)`	⬇️
Py-3.10.11	`97.27% <100.00%> (+<0.01%)`	⬆️
Py-3.10.16	`97.81% <100.00%> (+<0.01%)`	⬆️
Py-3.11.11	`97.89% <100.00%> (+<0.01%)`	⬆️
Py-3.11.9	`97.35% <100.00%> (+<0.01%)`	⬆️
Py-3.12.9	`98.35% <100.00%> (+<0.01%)`	⬆️
Py-3.13.2	`98.34% <100.00%> (+<0.01%)`	⬆️
Py-3.9.13	`97.14% <100.00%> (-0.02%)`	⬇️
Py-3.9.21	`97.68% <100.00%> (+<0.01%)`	⬆️
Py-pypy7.3.16	`85.92% <100.00%> (-3.32%)`	⬇️
VM-macos	`97.35% <100.00%> (-0.01%)`	⬇️
VM-ubuntu	`98.24% <100.00%> (+<0.01%)`	⬆️
VM-windows	`96.18% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

CHANGES/10520.feature.rst

aiohttp/connector.py

kjander0 · 2025-03-11T05:09:40Z

Do you need to uprev the required version of aiohappyeyeballs in setup.cfg?

TimMenninger · 2025-03-11T14:00:20Z

Do you need to uprev the required version of aiohappyeyeballs in setup.cfg?

I think that depends on the resolution to #10534 (comment) but if only for the documentation, yeah it'd probably have to go to whatever version if/when aio-libs/aiohappyeyeballs#149 merges and released.

EDIT: Nevermind, I think as long as the latest docs are complete, which they seem to be (using https://aiohappyeyeballs--149.org.readthedocs.build/en/149/ for aiohappyeyeballs worked) the version should be fine, unless you see something I didn't.

Dreamsorcerer · 2025-03-11T14:28:37Z

I think that depends on the resolution to #10534 (comment) but if only for the documentation, yeah it'd probably have to go to whatever version if/when aio-libs/aiohappyeyeballs#149 merges and released.

We're passing a kwarg that didn't exist on older releases, won't that break on older releases of aiohappyeyeballs?

TimMenninger · 2025-03-11T15:15:15Z

We're passing a kwarg that didn't exist on older releases, won't that break on older releases of aiohappyeyeballs?

oh! good catch @kjander0 and @Dreamsorcerer, I didn't realize how new that feature is. will bump to 2.5.0 when socket_factory comes about

TimMenninger · 2025-03-11T17:07:56Z

I'm at a total loss on how to fix this

/Users/tmenninger/Downloads/aiohttp/docs/client_reference.rst:1127: WARNING: py:class reference target not found: socket.SocketKind [ref.class]
/Users/tmenninger/Downloads/aiohttp/docs/client_reference.rst:1132: WARNING: py:class reference target not found: socket.SocketKind [ref.class]

Any ideas? It's complaining about these lines...

1127:   Refer to :py:data:`aiohappyeyeballs.AddrInfoType`
...
1132:   Refer to :py:data:`aiohappyeyeballs.SocketFactoryType`

EDIT: Found a workaround, sorry I'm new to this

docs/conf.py

docs/client_reference.rst

Instead of TCPConnector taking a list of sockopts to be applied sockets created, take a socket_factory callback that allows the caller to implement socket creation entirely.

bdraco · 2025-03-12T23:07:29Z

Thanks @TimMenninger

patchback · 2025-03-15T23:58:36Z

Backport to 3.12: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 3b9bb1c on top of patchback/backports/3.12/3b9bb1cd5677a8c8443d16184ed36856ae105cd7/pr-10534

Backporting merged PR #10534 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.12/3b9bb1cd5677a8c8443d16184ed36856ae105cd7/pr-10534 upstream/3.12

Now, cherry-pick PR Replace tcp_sockopts with socket_factory #10534 contents into that branch:
```
$ git cherry-pick -x 3b9bb1cd5677a8c8443d16184ed36856ae105cd7
```
If it'll yell at you with something like fatal: Commit 3b9bb1cd5677a8c8443d16184ed36856ae105cd7 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x 3b9bb1cd5677a8c8443d16184ed36856ae105cd7
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Replace tcp_sockopts with socket_factory #10534 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.12/3b9bb1cd5677a8c8443d16184ed36856ae105cd7/pr-10534

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

bdraco · 2025-03-15T23:59:15Z

@TimMenninger Thanks for this PR. Can you take care of back-porting this PR to 3.12 using the instructions above?

TimMenninger · 2025-03-16T00:12:24Z

@TimMenninger Thanks for this PR. Can you take care of back-porting this PR to 3.12 using the instructions above?

Yep will do. It might be till tomorrow or Monday morning that I get to it.

bdraco · 2025-03-16T22:11:00Z

Since I'm planning on doing a release. I'll get a clean backport done and than we can merge your docs fixes on top.

Instead of TCPConnector taking a list of sockopts to be applied sockets created, take a socket_factory callback that allows the caller to implement socket creation entirely. Fixes #10520  Replace `tcp_sockopts` parameter with a `socket_factory` parameter that is a callback allowing the caller to own socket creation. If passed, all sockets created by `TCPConnector` are expected to come from the `socket_factory` callback.  The only users to experience a change in behavior are those who are using the un-released `tcp_sockopts` argument to `TCPConnector`. However, using unreleased code comes with caveat emptor, and is why I felt entitled to remove the option entirely without warning.  The burden will be minimal and would only arise if `aiohappyeyeballs` changes their interface.    - [x] I think the code is well written - [x] Unit tests for the changes exist - [x] Documentation reflects the changes - [x] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [x] Add a new news fragment into the `CHANGES/` folder * name it `<issue_or_pr_num>.<type>.rst` (e.g. `588.bugfix.rst`) * if you don't have an issue number, change it to the pull request number after creating the PR * `.bugfix`: A bug fix for something the maintainers deemed an improper undesired behavior that got corrected to match pre-agreed expectations. * `.feature`: A new behavior, public APIs. That sort of stuff. * `.deprecation`: A declaration of future API removals and breaking changes in behavior. * `.breaking`: When something public is removed in a breaking way. Could be deprecated in an earlier release. * `.doc`: Notable updates to the documentation structure or build process. * `.packaging`: Notes for downstreams about unobvious side effects and tooling. Changes in the test invocation considerations and runtime assumptions. * `.contrib`: Stuff that affects the contributor experience. e.g. Running tests, building the docs, setting up the development environment. * `.misc`: Changes that are hard to assign to any of the above categories. * Make sure to use full sentences with correct case and punctuation, for example: ```rst Fixed issue with non-ascii contents in doctest text files -- by :user:`contributor-gh-handle`. ``` Use the past tense or the present tense a non-imperative mood, referring to what's changed compared to the last released version of this project. --------- Co-authored-by: J. Nick Koston <nick@koston.org> (cherry picked from commit 3b9bb1c)

…factory (#10574) replaces and closes #10565 Instead of TCPConnector taking a list of sockopts to be applied sockets created, take a socket_factory callback that allows the caller to implement socket creation entirely. Fixes #10520  Replace `tcp_sockopts` parameter with a `socket_factory` parameter that is a callback allowing the caller to own socket creation. If passed, all sockets created by `TCPConnector` are expected to come from the `socket_factory` callback.  The only users to experience a change in behavior are those who are using the un-released `tcp_sockopts` argument to `TCPConnector`. However, using unreleased code comes with caveat emptor, and is why I felt entitled to remove the option entirely without warning.  The burden will be minimal and would only arise if `aiohappyeyeballs` changes their interface.    - [x] I think the code is well written - [x] Unit tests for the changes exist - [x] Documentation reflects the changes - [x] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [x] Add a new news fragment into the `CHANGES/` folder * name it `<issue_or_pr_num>.<type>.rst` (e.g. `588.bugfix.rst`) * if you don't have an issue number, change it to the pull request number after creating the PR * `.bugfix`: A bug fix for something the maintainers deemed an improper undesired behavior that got corrected to match pre-agreed expectations. * `.feature`: A new behavior, public APIs. That sort of stuff. * `.deprecation`: A declaration of future API removals and breaking changes in behavior. * `.breaking`: When something public is removed in a breaking way. Could be deprecated in an earlier release. * `.doc`: Notable updates to the documentation structure or build process. * `.packaging`: Notes for downstreams about unobvious side effects and tooling. Changes in the test invocation considerations and runtime assumptions. * `.contrib`: Stuff that affects the contributor experience. e.g. Running tests, building the docs, setting up the development environment. * `.misc`: Changes that are hard to assign to any of the above categories. * Make sure to use full sentences with correct case and punctuation, for example: ```rst Fixed issue with non-ascii contents in doctest text files -- by :user:`contributor-gh-handle`. ``` Use the past tense or the present tense a non-imperative mood, referring to what's changed compared to the last released version of this project. --------- Co-authored-by: J. Nick Koston <nick@koston.org> (cherry picked from commit 3b9bb1c)  ## What do these changes do?  ## Are there changes in behavior for the user?  ## Is it a substantial burden for the maintainers to support this?  ## Related issue number   ## Checklist - [ ] I think the code is well written - [ ] Unit tests for the changes exist - [ ] Documentation reflects the changes - [ ] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [ ] Add a new news fragment into the `CHANGES/` folder * name it `<issue_or_pr_num>.<type>.rst` (e.g. `588.bugfix.rst`) * if you don't have an issue number, change it to the pull request number after creating the PR * `.bugfix`: A bug fix for something the maintainers deemed an improper undesired behavior that got corrected to match pre-agreed expectations. * `.feature`: A new behavior, public APIs. That sort of stuff. * `.deprecation`: A declaration of future API removals and breaking changes in behavior. * `.breaking`: When something public is removed in a breaking way. Could be deprecated in an earlier release. * `.doc`: Notable updates to the documentation structure or build process. * `.packaging`: Notes for downstreams about unobvious side effects and tooling. Changes in the test invocation considerations and runtime assumptions. * `.contrib`: Stuff that affects the contributor experience. e.g. Running tests, building the docs, setting up the development environment. * `.misc`: Changes that are hard to assign to any of the above categories. * Make sure to use full sentences with correct case and punctuation, for example: ```rst Fixed issue with non-ascii contents in doctest text files -- by :user:`contributor-gh-handle`. ``` Use the past tense or the present tense a non-imperative mood, referring to what's changed compared to the last released version of this project. Co-authored-by: Tim Menninger <tmenninger22@gmail.com>

(cherry picked from commit 8ac4830)

…ocket factory (#10534) (#10637) **This is a backport of PR #10568 as merged into master (8ac4830).** Co-authored-by: Tim Menninger <tmenninger22@gmail.com>

psf-chronographer bot added the bot:chronographer:provided label Mar 10, 2025

TimMenninger force-pushed the tcp-socket-factory branch from e07d4e8 to dd9f264 Compare March 10, 2025 15:51

TimMenninger force-pushed the tcp-socket-factory branch 6 times, most recently from 212bb6f to d9a7943 Compare March 10, 2025 17:58

Dreamsorcerer reviewed Mar 11, 2025

View reviewed changes

CHANGES/10520.feature.rst Outdated Show resolved Hide resolved

Dreamsorcerer reviewed Mar 11, 2025

View reviewed changes

CHANGES/10520.feature.rst Show resolved Hide resolved

Dreamsorcerer approved these changes Mar 11, 2025

View reviewed changes

bdraco reviewed Mar 11, 2025

View reviewed changes

aiohttp/connector.py Outdated Show resolved Hide resolved

TimMenninger force-pushed the tcp-socket-factory branch from d9a7943 to 2e147ff Compare March 11, 2025 13:57

TimMenninger force-pushed the tcp-socket-factory branch from 1b467a3 to eb17499 Compare March 11, 2025 14:11

TimMenninger force-pushed the tcp-socket-factory branch 2 times, most recently from 3a87c34 to 3adf43a Compare March 11, 2025 16:46

TimMenninger commented Mar 11, 2025

View reviewed changes

docs/conf.py Outdated Show resolved Hide resolved

TimMenninger force-pushed the tcp-socket-factory branch 3 times, most recently from cdaf8ba to 09ece48 Compare March 11, 2025 22:29

TimMenninger marked this pull request as ready for review March 11, 2025 22:34

TimMenninger requested review from webknjaz and asvetlov as code owners March 11, 2025 22:34

bdraco added the backport-3.12 label Mar 12, 2025

bdraco reviewed Mar 12, 2025

View reviewed changes

docs/client_reference.rst Outdated Show resolved Hide resolved

Replace tcp_sockopts with socket_factory (aio-libs#10520)

Loading
Loading status checks…

912a3c3

Instead of TCPConnector taking a list of sockopts to be applied sockets created, take a socket_factory callback that allows the caller to implement socket creation entirely.

TimMenninger force-pushed the tcp-socket-factory branch from a5e4a81 to 912a3c3 Compare March 12, 2025 22:54

bdraco approved these changes Mar 12, 2025

View reviewed changes

bdraco changed the title ~~Replace tcp_sockopts with socket_factory (#10520)~~ Replace tcp_sockopts with socket_factory Mar 15, 2025

link to TCPConnector in the docs

Verified

This commit was signed with the committer’s verified signature.

bdraco J. Nick Koston

SSH Key Fingerprint: yg7JSgeeILNaBQZstddaVlTaDn2euAb9Ngq4jVYLiEg
Verified
Learn about vigilant mode

Loading
Loading status checks…

ea62f2f

bdraco approved these changes Mar 15, 2025

View reviewed changes

bdraco merged commit 3b9bb1c into aio-libs:master Mar 15, 2025
40 checks passed

TimMenninger added a commit to TimMenninger/aiohttp that referenced this pull request Mar 16, 2025

Docs fixups following aio-libs#10534

c41d7a4

TimMenninger added a commit to TimMenninger/aiohttp that referenced this pull request Mar 19, 2025

Docs fixups following aio-libs#10534

b2cd192

TimMenninger added a commit to TimMenninger/aiohttp that referenced this pull request Mar 19, 2025

Docs fixups following aio-libs#10534

000d120

TimMenninger added a commit to TimMenninger/aiohttp that referenced this pull request Mar 21, 2025

Docs fixups following aio-libs#10534

7ba697b

TimMenninger added a commit to TimMenninger/aiohttp that referenced this pull request Mar 22, 2025

Docs fixups following aio-libs#10534

f378fbb

patchback bot pushed a commit that referenced this pull request Mar 30, 2025

Docs fixups following implement socket factory (#10534) (#10568)

Loading
Loading status checks…

5b5fea3

(cherry picked from commit 8ac4830)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Sponsors

Replace tcp_sockopts with socket_factory #10534

Replace tcp_sockopts with socket_factory #10534

TimMenninger commented Mar 10, 2025 •

edited by bdraco

Loading

codspeed-hq bot commented Mar 10, 2025 •

edited

Loading

codecov bot commented Mar 10, 2025 •

edited

Loading

kjander0 commented Mar 11, 2025

TimMenninger commented Mar 11, 2025 •

edited

Loading

Dreamsorcerer commented Mar 11, 2025

TimMenninger commented Mar 11, 2025

TimMenninger commented Mar 11, 2025 •

edited

Loading

bdraco commented Mar 12, 2025

patchback bot commented Mar 15, 2025 •

edited

Loading

bdraco commented Mar 15, 2025

TimMenninger commented Mar 16, 2025

bdraco commented Mar 16, 2025

Replace tcp_sockopts with socket_factory #10534

Replace tcp_sockopts with socket_factory #10534

Conversation

TimMenninger commented Mar 10, 2025 • edited by bdraco Loading

What do these changes do?

Are there changes in behavior for the user?

Is it a substantial burden for the maintainers to support this?

Related issue number

Checklist

codspeed-hq bot commented Mar 10, 2025 • edited Loading

CodSpeed Performance Report

Merging #10534 will not alter performance

Summary

codecov bot commented Mar 10, 2025 • edited Loading

Codecov Report

kjander0 commented Mar 11, 2025

TimMenninger commented Mar 11, 2025 • edited Loading

Dreamsorcerer commented Mar 11, 2025

TimMenninger commented Mar 11, 2025

TimMenninger commented Mar 11, 2025 • edited Loading

bdraco commented Mar 12, 2025

patchback bot commented Mar 15, 2025 • edited Loading

Backport to 3.12: 💔 cherry-picking failed — conflicts found

Backporting merged PR #10534 into master

bdraco commented Mar 15, 2025

TimMenninger commented Mar 16, 2025

bdraco commented Mar 16, 2025

TimMenninger commented Mar 10, 2025 •

edited by bdraco

Loading

codspeed-hq bot commented Mar 10, 2025 •

edited

Loading

codecov bot commented Mar 10, 2025 •

edited

Loading

TimMenninger commented Mar 11, 2025 •

edited

Loading

TimMenninger commented Mar 11, 2025 •

edited

Loading

patchback bot commented Mar 15, 2025 •

edited

Loading