Skip to content

Declare support for Python3.12#1897

Merged
cosmicBboy merged 10 commits intounionai-oss:mainfrom
g-as:feature/python3.12
Feb 18, 2025
Merged

Declare support for Python3.12#1897
cosmicBboy merged 10 commits intounionai-oss:mainfrom
g-as:feature/python3.12

Conversation

@g-as
Copy link
Contributor

@g-as g-as commented Jan 21, 2025

  • pylint doesn't work because v2.17.3 won't install on 3.12
  • updated pylint to 3.3, I ignored some warnings, and left some others that need proper addressing
  • updated pylint to ~3.2 (latest to be run with 3.8), I ignored some warnings in the conf file, and ignored the rest (possibly-used-before-assignment) in situ.
  • pyspark doesn't support 3.12 as of 3.5.4, will do in 4.0. It crashes on import when it tries to import distutils. But besides that, it works. As suggested, I excluded 3.12 x pyspark from the matrix.

@codecov
Copy link

codecov bot commented Jan 21, 2025

Codecov Report

Attention: Patch coverage is 83.33333% with 3 lines in your changes missing coverage. Please review.

Project coverage is 93.37%. Comparing base (812b2a8) to head (82c81de).
Report is 237 commits behind head on main.

Files with missing lines Patch % Lines
pandera/api/pyspark/container.py 87.50% 1 Missing ⚠️
pandera/typing/geopandas.py 0.00% 1 Missing ⚠️
pandera/typing/pandas.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1897      +/-   ##
==========================================
- Coverage   94.28%   93.37%   -0.91%     
==========================================
  Files          91      121      +30     
  Lines        7013     9365    +2352     
==========================================
+ Hits         6612     8745    +2133     
- Misses        401      620     +219     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cosmicBboy
Copy link
Collaborator

thanks @g-as ! been wanting to do this for a while, thanks for taking the initiative on this.

pyspark doesn't support 3.12 as of 3.5.4, will do in 4.0

We can add exceptions in the CI workflow to only run pyspark on 3.12 for pyspark 4.0

@g-as
Copy link
Contributor Author

g-as commented Jan 24, 2025

Hi @cosmicBboy!

Excluded pyspark x 3.12 from the matrix as you suggested.

I ignored the rest of the pylint warnings so linters should pass (and thus tests should run). I'll let you decide on how you want to address those.

@cosmicBboy cosmicBboy marked this pull request as ready for review February 10, 2025 02:02
@g-as
Copy link
Contributor Author

g-as commented Feb 12, 2025

Hi @cosmicBboy,

as you've seen, the linting workflow fails for 3.12 because setuptools is missing from the environment.

I think it is related to this.

I thus added setuptools in the installed deps in the lint workflow.

@g-as
Copy link
Contributor Author

g-as commented Feb 17, 2025

Hi @cosmicBboy

I also added setuptools to the list of deps, since it is needed to run the tests

Capture d’écran 2025-02-17 à 15 00 22

Hopefully everything will run now.

g-as added 10 commits February 17, 2025 22:38
Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
…d addressing

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
…ce it's no longer installed by default with python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
it is required in noxfile and not automatically provided by python >= 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
typing_extensions is currently required by all python versions

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
@g-as
Copy link
Contributor Author

g-as commented Feb 18, 2025

Yay \o/

What do you want to do about the coverage?

@cosmicBboy
Copy link
Collaborator

The coverage on this project has consistently been finicky, will address that in a seperate PR. In the mean time let's go ahead and merge this.

@cosmicBboy
Copy link
Collaborator

Thanks for the contribution @g-as ! and congrats on your first PR to pandera 🚀

@cosmicBboy cosmicBboy merged commit ede8a43 into unionai-oss:main Feb 18, 2025
182 of 184 checks passed
@g-as g-as deleted the feature/python3.12 branch February 18, 2025 14:01
@g-as
Copy link
Contributor Author

g-as commented Feb 18, 2025

Thanks!

Are you interested in me adding 3.13 support? Removing 3.8?

@cosmicBboy
Copy link
Collaborator

@g-as thanks for the effort!

Re: 3.8, let's go ahead and sunset 3.8 (since it's eol'd for a few months now)

Re: 3.13+ support, 3.13 and 3.14 has introduced breaking changes to runtime type checkers. For context, see:

Besides being an entertaining read, it does provide a path forward for what library maintainers will have to do to address the breaking changes. Would you mind opening tickets for supporting 3.13 and 3.14?

m-richards pushed a commit to m-richards/pandera that referenced this pull request Feb 23, 2025
* declared support for python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* set python3.8 as min python in setup's 'python_requires'

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* bumped pylint to <3.3 (3.2.x latest to be run with 3.8)

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added pyupgrade and applied modifications

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github/tests: skipped pyspark tests on 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* TEMP: disabled pylint warnings (possibly-used-before-assignment). need addressing

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github: added setuptools to test run deps (required by noxfile), since it's no longer installed by default with python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added setuptools as an explicit test dependency

it is required in noxfile and not automatically provided by python >= 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated dependency spec of typing_extensions

typing_extensions is currently required by all python versions

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated autogen requirements files

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

---------

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
m-richards pushed a commit to m-richards/pandera that referenced this pull request Feb 23, 2025
* declared support for python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* set python3.8 as min python in setup's 'python_requires'

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* bumped pylint to <3.3 (3.2.x latest to be run with 3.8)

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added pyupgrade and applied modifications

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github/tests: skipped pyspark tests on 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* TEMP: disabled pylint warnings (possibly-used-before-assignment). need addressing

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github: added setuptools to test run deps (required by noxfile), since it's no longer installed by default with python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added setuptools as an explicit test dependency

it is required in noxfile and not automatically provided by python >= 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated dependency spec of typing_extensions

typing_extensions is currently required by all python versions

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated autogen requirements files

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

---------

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Matt Richards <mrichards7@outlook.com.au>
m-richards pushed a commit to m-richards/pandera that referenced this pull request Feb 23, 2025
* declared support for python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* set python3.8 as min python in setup's 'python_requires'

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* bumped pylint to <3.3 (3.2.x latest to be run with 3.8)

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added pyupgrade and applied modifications

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github/tests: skipped pyspark tests on 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* TEMP: disabled pylint warnings (possibly-used-before-assignment). need addressing

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github: added setuptools to test run deps (required by noxfile), since it's no longer installed by default with python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added setuptools as an explicit test dependency

it is required in noxfile and not automatically provided by python >= 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated dependency spec of typing_extensions

typing_extensions is currently required by all python versions

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated autogen requirements files

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

---------

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Matt Richards <mrichards7@outlook.com.au>
m-richards pushed a commit to m-richards/pandera that referenced this pull request Mar 4, 2025
* declared support for python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* set python3.8 as min python in setup's 'python_requires'

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* bumped pylint to <3.3 (3.2.x latest to be run with 3.8)

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added pyupgrade and applied modifications

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github/tests: skipped pyspark tests on 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* TEMP: disabled pylint warnings (possibly-used-before-assignment). need addressing

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github: added setuptools to test run deps (required by noxfile), since it's no longer installed by default with python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added setuptools as an explicit test dependency

it is required in noxfile and not automatically provided by python >= 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated dependency spec of typing_extensions

typing_extensions is currently required by all python versions

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated autogen requirements files

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

---------

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Signed-off-by: Matt Richards <mrichards7@outlook.com.au>
m-richards pushed a commit to m-richards/pandera that referenced this pull request Mar 4, 2025
* declared support for python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* set python3.8 as min python in setup's 'python_requires'

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* bumped pylint to <3.3 (3.2.x latest to be run with 3.8)

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added pyupgrade and applied modifications

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github/tests: skipped pyspark tests on 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* TEMP: disabled pylint warnings (possibly-used-before-assignment). need addressing

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* .github: added setuptools to test run deps (required by noxfile), since it's no longer installed by default with python 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* added setuptools as an explicit test dependency

it is required in noxfile and not automatically provided by python >= 3.12

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated dependency spec of typing_extensions

typing_extensions is currently required by all python versions

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

* updated autogen requirements files

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>

---------

Signed-off-by: Guillaume Andreu Sabater <guillaume.andreu.sabater@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants