deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) #337

chalmerlowe · 2025-04-17T19:49:39Z

This branch is generated automagically via AI as an experiment.
Requires a comprehensive human review and comment.
Do not merge until approved by an authorized reviewer.

Removes support for Python 3.7 and 3.8, establishing Python 3.9 as the new minimum supported version.

This change involves:

Updating python_requires and classifiers in setup.py.
Modifying Python versions in noxfile.py (default, unit tests, system tests) and ensuring constraint file logic remains correct.
Updating the GitHub Actions workflow (unittest.yml) matrix, runner, and coverage job version.
Deleting constraint files for Python 3.7 and 3.8 (testing/constraints-3.7.txt, testing/constraints-3.8.txt).
Removing Kokoro sample configuration directories (.kokoro/samples/python3.7/, .kokoro/samples/python3.8/).
Updating supported version mentions in README.rst.
Removing 3.7 and 3.8 from the ALL_VERSIONS list in samples/snippets/noxfile.py.

Removes support for Python 3.7 and 3.8, establishing Python 3.9 as the new minimum supported version. This change involves: - Updating `python_requires` and classifiers in `setup.py`. - Modifying Python versions in `noxfile.py` (default, unit tests, system tests) and ensuring constraint file logic remains correct. - Updating the GitHub Actions workflow (`unittest.yml`) matrix, runner, and coverage job version. - Deleting constraint files for Python 3.7 and 3.8 (`testing/constraints-3.7.txt`, `testing/constraints-3.8.txt`). - Removing Kokoro sample configuration directories (`.kokoro/samples/python3.7/`, `.kokoro/samples/python3.8/`). - Updating supported version mentions in `README.rst`. - Removing 3.7 and 3.8 from the `ALL_VERSIONS` list in `samples/snippets/noxfile.py`.

Linchin · 2025-04-23T17:06:42Z

Just a suggestion, maybe use deps!: so this commit shows up in release notes? (I'm not 100% sure if a breaking change "chore" will appear in the release, though)

chalmerlowe · 2025-04-23T21:13:14Z

Just a suggestion, maybe use deps!: so this commit shows up in release notes? (I'm not 100% sure if a breaking change "chore" will appear in the release, though)

Conventional Commit points at this page as reference for prefixes:

'build',
'chore',
'ci',
'docs',
'feat',
'fix',
'perf',
'refactor',
'revert',
'style',
'test'

If Google also uses deps, then sure.

Linchin · 2025-04-23T21:48:19Z

Wow, I suggested it because we used it in the commit for python-bigquery. Apparently it's not part of convetional commits, but it worked.

Edit:
It seems conventional commits doesn't enforce a whitelist of commit types

chalmerlowe · 2025-04-23T22:18:32Z

Done.

tswast · 2025-04-24T14:19:06Z

db_dtypes/__init__.py

+# Inside db_dtypes/__init__.py (TEMPORARY DEBUG)
+print(f"DEBUG: Inside __init__, JSONArray type: {type(JSONArray)}, value: {repr(JSONArray)}")
+print(f"DEBUG: Inside __init__, JSONDtype type: {type(JSONDtype)}, value: {repr(JSONDtype)}")


We'll want to remove these before merging.

Suggested change

# Inside db_dtypes/__init__.py (TEMPORARY DEBUG)

print(f"DEBUG: Inside __init__, JSONArray type: {type(JSONArray)}, value: {repr(JSONArray)}")

print(f"DEBUG: Inside __init__, JSONDtype type: {type(JSONDtype)}, value: {repr(JSONDtype)}")

tswast · 2025-04-24T14:20:30Z

db_dtypes/__init__.py

+    if sys_major == 3 and sys_minor in (7, 8):
+        warnings.warn(
+            "The python-bigquery library as well as the python-db-dtypes-pandas library no "
+            "longer supports Python 3.7 and Python 3.8. "
+            f"Your Python version is {sys_major}.{sys_minor}.{sys_micro}. We "
+            "recommend that you update soon to ensure ongoing support. For "
+            "more details, see: [Google Cloud Client Libraries Supported Python Versions policy](https://cloud.google.com/python/docs/supported-python-versions)",
+            FutureWarning,
+            stacklevel=2, # Point warning to the caller of __init__
+        )


If you use python_requires in setup.py, I'm having a hard time seeing how someone could end up in this state.

Do we want to issue this in a separate PR and release so folks see it before fully dropping?

I can do that.

tswast · 2025-04-24T14:21:01Z

db_dtypes/__init__.py

+# sys_major, sys_minor, sys_micro = _versions_helpers.extract_runtime_version()
+# if sys_major == 3 and sys_minor in (7, 8):
+#     warnings.warn(
+#         "The python-bigquery library as well as the python-db-dtypes-pandas library no "
+#         "longer supports Python 3.7 and Python 3.8. "
+#         f"Your Python version is {sys_major}.{sys_minor}.{sys_micro}. We "
+#         "recommend that you update soon to ensure ongoing support. For "
+#         "more details, see: [Google Cloud Client Libraries Supported Python Versions policy](https://cloud.google.com/python/docs/supported-python-versions)",
+#         FutureWarning,
+#     )
+


Let's remove this commented out code.

Suggested change

# sys_major, sys_minor, sys_micro = _versions_helpers.extract_runtime_version()

# if sys_major == 3 and sys_minor in (7, 8):

# warnings.warn(

# "The python-bigquery library as well as the python-db-dtypes-pandas library no "

# "longer supports Python 3.7 and Python 3.8. "

# f"Your Python version is {sys_major}.{sys_minor}.{sys_micro}. We "

# "recommend that you update soon to ensure ongoing support. For "

# "more details, see: [Google Cloud Client Libraries Supported Python Versions policy](https://cloud.google.com/python/docs/supported-python-versions)",

# FutureWarning,

# )

tswast · 2025-04-24T14:22:04Z

db_dtypes/__init__.py

+_check_python_version()
+
+# Assign __all__ by calling the function
+__all__ = _determine_all(JSONArray, JSONDtype)


This seems a bit too "magic" to me, but I suppose it could make testing easier?

This code was originally just in the file with no function def. We added _check_python_version...

To test against supported and unsupported versions we attempted to mock the version but timing became an issue, partly because dependencies also rely on version checks so our mock ended up preventing error-free installs of some dependencies.

We tried half a dozen ways to alter when the mock happened but none hit that sweet spot of reliably allowing the dependencies to do their thing and slipping in to allow our test to run.

Similarly, we ran into a mess of problems with import timing and mocking whether JSONArray JSONDtype did or did not import as expected and so ...

putting them in functions made things much easier in the testing arena.

Any hard feelings if we resolve this conversation?

db_dtypes/json.py

tswast · 2025-04-24T14:23:56Z

testing/constraints-3.7.txt

Make sure to update constraints-3.9.txt with our minimum supported package versions from setup.py.

tswast · 2025-04-24T14:24:41Z

tests/compliance/date/test_date_compliance_1_5.py

@@ -24,7 +24,7 @@
 import pytest

 # NDArrayBacked2DTests suite added in https://github.com/pandas-dev/pandas/pull/44974
-pytest.importorskip("pandas", minversion="1.5.0dev")
+pytest.importorskip("pandas")


pandas is a strict dependency. I don't think we need this line. We were just using it for the min version before.

Suggested change

pytest.importorskip("pandas")

Since we rely on pandas version >=1.5.0 this whole file feels unnecessary.

I am gonna move this test to the generic date_compliance file

I will delete this 1_5 date_compliance-specific file

I am gonna move a similar test from test_time_compliance_1_5.py to the generic time_compliance file

I will delete this 1_5 time_compliance-specific file

tswast · 2025-04-24T14:24:52Z

tests/compliance/time/test_time_compliance_1_5.py

@@ -24,7 +24,7 @@
 import pytest

 # NDArrayBacked2DTests suite added in https://github.com/pandas-dev/pandas/pull/44974
-pytest.importorskip("pandas", minversion="1.5.0dev")
+pytest.importorskip("pandas")


Same here

Suggested change

pytest.importorskip("pandas")

See comment above about date_compliance.

Done.

tswast · 2025-04-24T14:25:05Z

tests/unit/test__init__.py

@@ -0,0 +1,137 @@
+import sys


Missing license header.

tswast · 2025-04-24T14:27:11Z

tests/unit/test__init__.py

+import pytest
+import types
+import warnings
+from unittest import mock
+import pyarrow as pa


Imports should be grouped in the following order:

Standard library imports.

Related third party imports.

Local application/library specific imports.

You should put a blank line between each group of imports.

https://peps.python.org/pep-0008/#imports

Suggested change

import pytest

import types

import warnings

from unittest import mock

import pyarrow as pa

import types

import warnings

from unittest import mock

import pyarrow as pa

import pytest

chalmerlowe · 2025-05-02T13:12:45Z

@tswast or @Linchin

Can you confirm comment resolution? And if you feel comfortable, approve?

I am unable to get Owlbot to run without failing. Dunno why. Wonder if it is because the PR was generated by Jules. (But I can generally get Owlbot to run/pass, so... weird).

There is a class in pandas_backports.py that did not have tests (and which fact coverage ignored apparently). For the purposes of this PR I consider adding the tests to be out of scope. So I flagged the class as # pragma: NO COVER

There are a pair of code branches that for the life of me, no matter what I tried, I could not get coverage to acknowledge that the code was covered despite confirming that those lines of code did execute. I flagged both of those with # pragma: NO COVER

chalmerlowe added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Apr 17, 2025

chalmerlowe self-assigned this Apr 17, 2025

chalmerlowe requested review from a team as code owners April 17, 2025 19:49

chalmerlowe requested review from gericdong and Linchin April 17, 2025 19:49

product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-db-dtypes-pandas API. samples Issues that are directly related to samples. labels Apr 17, 2025

chalmerlowe added 6 commits April 18, 2025 09:46

Updates python version in lint.yml

dccef48

Updates owlbot, removing reference to 3.8

f81c738

Updates CONTRIBUTING.rst

5cb30ca

updates pytest warnings

7f44424

Removes test_samples-impl ref to older virtualenv package

3e4e6f6

Removes references to pandas older than 1.5.0

7beea0d

product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Apr 18, 2025

chalmerlowe added 6 commits April 18, 2025 12:42

Removes pandas older than 1.5 and misc changes

32d7b8f

updates pandas in setup.py

e871d73

more updates related to pandas

2bdf3ef

still broken

d902e31

Updates FutureWarning tests to account for unittest coverage

a0543b9

Updates json array type tests to account for unittest coverage

6a5d943

updates python version checks to ensure coverage

1bbd5f7

chalmerlowe changed the title ~~chore!: Drop support for Python 3.7 and 3.8 (AI Experiment)~~ deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) Apr 23, 2025

chalmerlowe added 2 commits April 23, 2025 22:19

update json test for unittest coverage

96a5e97

Update pandas_backports unittests to ensure coverage

8af9bee

tswast reviewed Apr 24, 2025

View reviewed changes

Updates per review comments

78574c0

chalmerlowe mentioned this pull request Apr 24, 2025

deps: Updates deprecation warning to FutureWarning re: 3.7 and 3.8 #338

Merged

Merge branch 'main' into remove-python-37-38

9ca34b8

chalmerlowe mentioned this pull request Apr 29, 2025

fix: updates json array registration #339

Closed

chalmerlowe added the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 30, 2025

moves class from version specific compliance file to generic file

de202f4

yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 30, 2025

chalmerlowe added 13 commits April 30, 2025 16:44

Removes weird cut and paste error

fd9dd27

fix linting errors

bfe9115

updates import statement to ensure import of JSONArrowType

91e437e

Revise required github status checks

eca1ab7

update linting

06b4e88

temporarily marking a class as no cover

d481be4

more updates

f4a9fc6

marked several snippets as pragma no cover

7585a86

updates linting

c777141

Updates constraints and setup.py

7d745b0

migrates class from one time compliance file to another

9511e7a

updating pyarrow version

9638e43

Updates linting

d433fae

chalmerlowe added the owlbot:run Add this label to trigger the Owlbot post processor. label May 1, 2025

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 1, 2025

chalmerlowe added owlbot:run Add this label to trigger the Owlbot post processor. and removed do not merge Indicates a pull request not ready for merge, due to either quality or timing. labels May 2, 2025

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) #337

deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) #337

chalmerlowe commented Apr 17, 2025

Linchin commented Apr 23, 2025

chalmerlowe commented Apr 23, 2025

Linchin commented Apr 23, 2025 •

edited

Loading

chalmerlowe commented Apr 23, 2025

tswast Apr 24, 2025

chalmerlowe Apr 29, 2025

tswast Apr 24, 2025

chalmerlowe Apr 24, 2025

chalmerlowe Apr 29, 2025

tswast Apr 24, 2025

chalmerlowe Apr 29, 2025

tswast Apr 24, 2025

chalmerlowe Apr 24, 2025

chalmerlowe Apr 30, 2025

tswast Apr 24, 2025

chalmerlowe May 2, 2025

tswast Apr 24, 2025

chalmerlowe Apr 30, 2025 •

edited

Loading

chalmerlowe May 2, 2025

tswast Apr 24, 2025

chalmerlowe May 2, 2025

tswast Apr 24, 2025

chalmerlowe May 1, 2025

tswast Apr 24, 2025

chalmerlowe May 2, 2025

chalmerlowe commented May 2, 2025 •

edited

Loading

	# Inside db_dtypes/__init__.py (TEMPORARY DEBUG)
	print(f"DEBUG: Inside __init__, JSONArray type: {type(JSONArray)}, value: {repr(JSONArray)}")
	print(f"DEBUG: Inside __init__, JSONDtype type: {type(JSONDtype)}, value: {repr(JSONDtype)}")

deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) #337

Are you sure you want to change the base?

deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) #337

Conversation

chalmerlowe commented Apr 17, 2025

Linchin commented Apr 23, 2025

chalmerlowe commented Apr 23, 2025

Linchin commented Apr 23, 2025 • edited Loading

chalmerlowe commented Apr 23, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chalmerlowe Apr 30, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chalmerlowe commented May 2, 2025 • edited Loading

Linchin commented Apr 23, 2025 •

edited

Loading

chalmerlowe Apr 30, 2025 •

edited

Loading

chalmerlowe commented May 2, 2025 •

edited

Loading