Skip to content

Fix pickling for the C extension - #838

Open
cananoo wants to merge 17 commits into
aio-libs:masterfrom
cananoo:fix/834-pickle-frozenlist
Open

Fix pickling for the C extension#838
cananoo wants to merge 17 commits into
aio-libs:masterfrom
cananoo:fix/834-pickle-frozenlist

Conversation

@cananoo

@cananoo cananoo commented Aug 27, 2026

Copy link
Copy Markdown

What do these changes do?

Add an explicit pickle reducer to the Cython implementation so the atomic
frozen state is serialized through a small reconstruction helper.

Are there changes in behavior for the user?

Yes. Pickling FrozenList instances works when the C extension is enabled,
for both frozen and unfrozen instances.

Related issue number

Fixes #834

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes (N/A; no documentation change is needed)
  • If you provide code modifications, please add yourself to CONTRIBUTORS.txt
  • Add a new news fragment into the CHANGES folder
Agent run details
  • The C-extension test suite passed: 116 tests, 100% coverage.
  • The pure-Python test suite passed: 116 tests, 100% coverage.
  • ruff check frozenlist tests passed.
  • flake8 frozenlist tests passed.
  • cython-lint --no-pycodestyle frozenlist/_frozenlist.pyx passed.
  • git diff --check passed.

Drafted with OpenAI Codex; human review is required before marking this PR ready.

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (381c91e) to head (844120c).

Additional details and impacted files
@@            Coverage Diff             @@
##            master      #838    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files            2         2            
  Lines          383       642   +259     
  Branches         9        37    +28     
==========================================
+ Hits           383       642   +259     
Flag Coverage Δ
CI-GHA 100.00% <100.00%> (ø)
OS-Linux 100.00% <100.00%> (ø)
OS-Windows 100.00% <100.00%> (ø)
OS-macOS 100.00% <100.00%> (ø)
Py-3.10.11 100.00% <100.00%> (ø)
Py-3.10.21 100.00% <100.00%> (ø)
Py-3.11.16 100.00% <100.00%> (ø)
Py-3.11.9 100.00% <100.00%> (ø)
Py-3.12.10 100.00% <100.00%> (ø)
Py-3.12.14 100.00% <100.00%> (ø)
Py-3.13.14 100.00% <100.00%> (ø)
Py-3.13.15 100.00% <100.00%> (ø)
Py-3.14.6 100.00% <100.00%> (ø)
Py-3.14.7 100.00% <100.00%> (ø)
Py-3.14.7t 100.00% <100.00%> (ø)
Py-pypy3.10.16-7.3.19 99.53% <99.23%> (-0.21%) ⬇️
VM-macos-latest 100.00% <100.00%> (ø)
VM-ubuntu-latest 100.00% <100.00%> (ø)
VM-windows-11-arm 100.00% <100.00%> (ø)
VM-windows-latest 100.00% <100.00%> (ø)
pytest 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

The functional CI matrix, Codecov, documentation, lint, and pre-commit checks are green. I also reproduced the repository's MyPy pre-commit invocation locally (mypy==1.19.0, Python 3.11 target, with tomli and types-setuptools): it reports no issues and 96.17% type-check coverage.

The remaining Coveralls statuses report 93.103% and a 0.1% decrease, which does not match the generated local Cobertura report or the 100% Codecov test coverage report. I have not added unrelated changes just to mask that external coverage discrepancy.

cananoo and others added 2 commits August 28, 2026 11:10
Used OpenAI Codex (GPT-5) to inspect the Coveralls MyPy report and make the new pickle tests type-precise. Reviewed the diff, ran MyPy with Cobertura output, and ran the pure-Python test module.
@cananoo
cananoo marked this pull request as ready for review August 28, 2026 03:18
@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

I tightened the new pickle tests so the MyPy report treats the added assertions as typed code (commit 7025fa5, with pre-commit.ci's follow-up 0a36b9). Local MyPy passes with 96.20% type-check coverage, and the pure-Python test module passes 116 tests. The existing full CI matrix and Codecov report are green; the refreshed CI run is queued now. I also marked the PR ready for review.

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Reviews (12): Last reviewed commit: "Handle mangled and mixed slot state" | Re-trigger Greptile

Comment thread frozenlist/_frozenlist.pyx Outdated
Comment thread frozenlist/_frozenlist.pyx Outdated
@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

Addressed the serialization concerns in 016b5c6. The reconstruction helper now lives in the package module so a base FrozenList pickle does not depend on the C extension module, and subclass reconstruction initializes the FrozenList base directly so subclasses with a no-argument constructor still work. Extra instance dictionary state is restored. Added regression coverage for both implementations and the reconstructor path; pure-Python tests, ruff, mypy, and Cython source validation pass locally. The C-extension matrix will provide the final wheel-level verification.

Comment thread tests/test_frozenlist.py Fixed
Comment thread tests/test_frozenlist.py Fixed
Comment thread frozenlist/_frozenlist.pyx Outdated
@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

Updated in c275abb to preserve subclass pickle state for instance dictionaries, slots, and custom getstate/setstate implementations. The pure-Python fallback unpickle path now has a regression test as well. Local full test suite passes (120 tests), plus Ruff, Mypy, and Cython validation. Please take another look when convenient.

Comment thread frozenlist/__init__.py
Comment thread frozenlist/__init__.py
@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

The latest commit 4621ce3 also has a fully green CI run: all 51 required checks passed, including Codecov, Coveralls, the full test matrix, all tested-architecture wheels, lint, and Greptile. The remaining step is maintainer review.

Comment thread frozenlist/__init__.py Outdated
@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown

Want your agent to iterate on Greptile's feedback? Try greploops.

Comment thread frozenlist/__init__.py Outdated
Comment thread frozenlist/__init__.py Outdated
Comment thread frozenlist/__init__.py Outdated
@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

Latest commit 844120c addresses the remaining slot-state feedback, including mangled private slots and mixed dict/slot restoration. All review threads are resolved, and the required CI, CodeQL, coverage, docs, lint, and pre-commit checks are green. The PR is ready for maintainer review whenever convenient. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

C extension: pickling any FrozenList raises TypeError (atomic[bint] not convertible)

2 participants