Add python->SQL handling for pyarrow strings, test python 3.9 in upstream import testing (#1206)

* Resolve upstream Dask failures

* Fix typo

* Override pyarrow string default in sql.yaml

* Move pyarrow string override to conftest.py

---------

Co-authored-by: Ayush Dattagupta <[email protected]>

Author: charlesbluca

.github/workflows/test-upstream.yml

@@ -110,7 +110,9 @@ jobs:
         with:
           miniforge-variant: Mambaforge
           use-mamba: true
-          python-version: "3.8"
+          # TODO: drop support for python 3.8, add support for python 3.11
+          # https://github.com/dask-contrib/dask-sql/pull/1143
+          python-version: "3.9"
           channel-priority: strict
       - name: Install Protoc
         uses: arduino/setup-protoc@v1

.github/workflows/test.yml

@@ -107,7 +107,9 @@ jobs:
         with:
           miniforge-variant: Mambaforge
           use-mamba: true
-          python-version: "3.8"
+          # TODO: drop support for python 3.8, add support for python 3.11
+          # https://github.com/dask-contrib/dask-sql/pull/1143
+          python-version: ${{ needs.detect-ci-trigger.outputs.triggered == 'true' && '3.9' || '3.8' }}
           channel-priority: strict
       - name: Install Protoc
         uses: arduino/setup-protoc@v1

conftest.py

@@ -12,6 +12,8 @@ def pytest_addoption(parser):
 
 
 def pytest_runtest_setup(item):
+    # TODO: explore adding support for pyarrow string columns
+    dask.config.set({"dataframe.convert-string": False})
     if "gpu" in item.keywords:
         if not item.config.getoption("--rungpu"):
             pytest.skip("need --rungpu option to run")