Skip to content

Commit 4545d7c

Browse files
lpoulainjohn-bodley
authored andcommitted
Optimize experimental_python_types and add type-mapping tests
Instead of checking the type for each row, check the type once for each fetch() call and compute a list of lambdas which are to be applied to the values from each row. A new RowMapperFactory class is created to wrap this behavior. The experimental_python_types flag is now processed in the TrinoQuery class instead of the TrinoResult class. Type mapping tests for each lambda which maps rows to Python types is added.
1 parent f4487f5 commit 4545d7c

File tree

9 files changed

+1611
-211
lines changed

9 files changed

+1611
-211
lines changed

.github/workflows/ci.yml

+5-18
Original file line numberDiff line numberDiff line change
@@ -14,21 +14,6 @@ concurrency:
1414
cancel-in-progress: true
1515

1616
jobs:
17-
checks:
18-
runs-on: ubuntu-latest
19-
steps:
20-
- name: "Checkout the source code"
21-
uses: actions/checkout@v2
22-
23-
- name: "Install Python"
24-
uses: actions/setup-python@v2
25-
26-
- name: "Install pre-commit"
27-
run: pip install pre-commit
28-
29-
- name: "Run pre-commit checks"
30-
run: pre-commit run --all-files
31-
3217
build:
3318
runs-on: ubuntu-latest
3419
strategy:
@@ -49,6 +34,7 @@ jobs:
4934
# Test with older Trino versions for backward compatibility
5035
- { python: "3.10", trino: "351" } # first Trino version
5136
env:
37+
TOX_PARALLEL_NO_SPINNER: 1
5238
TRINO_VERSION: "${{ matrix.trino }}"
5339
steps:
5440
- uses: actions/checkout@v2
@@ -59,7 +45,8 @@ jobs:
5945
run: |
6046
sudo apt-get update
6147
sudo apt-get install libkrb5-dev
62-
pip install .[tests]
63-
- name: Run tests
48+
sudo curl -sSL https://install.python-poetry.org | python3
49+
poetry install
50+
- name: Run tox
6451
run: |
65-
pytest -s tests/
52+
poetry run tox --parallel

README.md

+52-7
Original file line numberDiff line numberDiff line change
@@ -383,12 +383,12 @@ exits the *with* context and the queries succeed, otherwise
383383

384384
## Improved Python types
385385

386-
If you enable the flag `experimental_python_types`, the client will convert the results of the query to the
386+
If you enable the flag `experimental_python_types`, the client will convert the results of the query to the
387387
corresponding Python types. For example, if the query returns a `DECIMAL` column, the result will be a `Decimal` object.
388388

389-
Limitations of the Python types are described in the
390-
[Python types documentation](https://docs.python.org/3/library/datatypes.html). These limitations will generate an
391-
exception `trino.exceptions.DataError` if the query returns a value that cannot be converted to the corresponding Python
389+
Limitations of the Python types are described in the
390+
[Python types documentation](https://docs.python.org/3/library/datatypes.html). These limitations will generate an
391+
exception `trino.exceptions.DataError` if the query returns a value that cannot be converted to the corresponding Python
392392
type.
393393

394394
```python
@@ -418,6 +418,7 @@ assert cur.description[0][1] == "timestamp with time zone"
418418

419419
Start by forking the repository and then modify the code in your fork.
420420

421+
<<<<<<< HEAD
421422
We recommend that you use Python3's `venv` for development:
422423

423424
```
@@ -430,6 +431,16 @@ With `-e` passed to `pip install` above pip can reference the code you are
430431
modifying in the *virtual env*. That way, you do not need to run `pip install`
431432
again to make your changes applied to the *virtual env*.
432433

434+
=======
435+
Clone the repository and go inside the code directory.
436+
437+
Python dependencies are managed using [Poetry](https://python-poetry.org/) which helps to ensure the project is managed in a deterministic way. Poetry [creates a virtual environment](https://python-poetry.org/docs/managing-environments/) to aid with the process. Poetry should be installed via:
438+
439+
```
440+
$ curl -sSL https://install.python-poetry.org | python3
441+
```
442+
443+
>>>>>>> 1226d39 ([CI] Migrate to pyproject.toml and poetry for deterministic builds)
433444
When the code is ready, submit a Pull Request.
434445

435446
### Code style
@@ -443,25 +454,37 @@ Most of them also apply to code in trino-python-client.
443454

444455
### Running tests
445456

446-
`trino-python-client` uses [pytest](https://pytest.org/) for its tests. To run
457+
`trino-python-client` uses [tox](https://tox.wiki/en/latest/)—a tool for standardizing testing in Python—which leverages the [pytest](https://pytest.org/) testing framework. To run
447458
only unit tests, type:
448459

449460
```
450-
$ pytest tests/unit
461+
$ poetry run tox -e <environment> -- tests/unit
451462
```
452463

464+
Similarly to run only integration tests, type:
465+
466+
```
467+
$ poetry run tox -e <environment> -- tests/integration
468+
```
469+
470+
where `<environment>` denotes the Python environment (see the configuration in `tox.ini`).
471+
453472
Then you can pass options like `--pdb` or anything supported by `pytest --help`.
454473

474+
<<<<<<< HEAD
455475
To run integration tests:
456476

457477
```
458478
$ pytest tests/integration
459479
```
460480

481+
=======
482+
>>>>>>> 1226d39 ([CI] Migrate to pyproject.toml and poetry for deterministic builds)
461483
They pull a Docker image and then run a container with a Trino server:
462484
- the image is named `trinodb/trino:${TRINO_VERSION}`
463485
- the container is named `trino-python-client-tests-{uuid4()[:7]}`
464486

487+
<<<<<<< HEAD
465488
To run the tests with different versions of Python in managed *virtual envs*,
466489
use `tox` (see the configuration in `tox.ini`):
467490

@@ -470,13 +493,30 @@ $ tox
470493
```
471494

472495
## Releasing
496+
=======
497+
### pre-commit
498+
499+
`trino-python-client` leverages [pre-commit](https://pre-commit.com/) to help identify simple issues before submission to code review. Checks include the validity of the `pyproject.toml` file, type checks via [Mypy](https://github.com/python/mypy), etc. To enable `pre-commit` run:
500+
501+
```
502+
poetry run pre-commit install
503+
```
504+
505+
which will run on every commit. You can also run it anytime using:
506+
507+
```
508+
poetry run tox -e pre-commit
509+
```
510+
511+
### Releasing
512+
>>>>>>> 1226d39 ([CI] Migrate to pyproject.toml and poetry for deterministic builds)
473513

474514
- [Set up your development environment](#Getting-Started-With-Development).
475515
- Check the local workspace is up to date and has no uncommitted changes
476516
```bash
477517
git fetch -a && git status
478518
```
479-
- Change version in `trino/__init__.py` to a new version, e.g. `0.123.0`.
519+
- Change version in `trino/pyproject.toml` to a new version, e.g. `0.123.0`.
480520
- Commit
481521
```bash
482522
git commit -a -m "Bump version to 0.123.0"
@@ -487,12 +527,17 @@ $ tox
487527
```
488528
- Create release package and upload it to PyPI
489529
```bash
530+
<<<<<<< HEAD
490531
. .venv/bin/activate && \
491532
pip install twine && \
492533
rm -rf dist/ && \
493534
./setup.py sdist bdist_wheel && \
494535
twine upload dist/* && \
495536
open https://pypi.org/project/trino/ && \
537+
=======
538+
poetry publish --build &&
539+
open https://pypi.org/project/trino/ &&
540+
>>>>>>> 1226d39 ([CI] Migrate to pyproject.toml and poetry for deterministic builds)
496541
echo "Released!"
497542
```
498543
- Push the branch and the tag

0 commit comments

Comments
 (0)