Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feb 2025 Hyper API Release #149

Merged
merged 8 commits into from
Feb 13, 2025
Merged

Feb 2025 Hyper API Release #149

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0710ce0
Add placeholder for upcoming release
marcelkost Jan 17, 2025
aff9a67
Update ARRAY literal syntax (#144) (#147)
caetanosauer Jan 29, 2025
80e96d2
Update releases.md (#145)
raul-persa Jan 29, 2025
8ce9f11
Update GitHub actions
wolfroediger Feb 5, 2025
3c382b2
Update geography documentation
wolfroediger Jan 30, 2025
46f90b8
Also update deploy pages action
wolfroediger Feb 7, 2025
32be890
Update releases.md
StefanSteiner Feb 11, 2025
a06ea2d
Update config.ts
StefanSteiner Feb 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,19 @@ jobs:
id: setup-pages

- name: Checkout main
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
ref: main
path: main

- name: Checkout upcoming
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
ref: upcoming
path: upcoming

- name: Setup node
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: '20.8'
cache: yarn
Expand Down Expand Up @@ -88,10 +88,10 @@ jobs:
mv combined/lang_docs/tableauhyperapi-java-docs-* combined/lang_docs/java

- name: Upload webpage artifact
uses: actions/upload-pages-artifact@v1
uses: actions/upload-pages-artifact@v3
with:
path: 'combined'

- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
uses: actions/deploy-pages@v4
6 changes: 3 additions & 3 deletions .github/workflows/proof.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Setup node
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: '20.8'
cache: yarn
Expand All @@ -43,6 +43,6 @@ jobs:
GITHUB_BASE_PATH: ${{ steps.setup-pages.outputs.base_path }}

- name: Upload webpage artifact
uses: actions/upload-pages-artifact@v1
uses: actions/upload-pages-artifact@v3
with:
path: 'website/build'
70 changes: 34 additions & 36 deletions website/docs/guides/hyper_file/geodata.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
# Add Spatial Data to a Hyper File
# Add Geospatial Data to a Hyper File

Tableau supports spatial data (geography) in `.hyper` files.
This guide describes how you can use the Hyper API to add geography type data to the Hyper file.
Tableau supports geospatial data (`tableau.tabgeography`) in `.hyper` files.
This guide describes how you can use the Hyper API to add geospatial data to Hyper files.

The Hyper API does not directly accept Well-Known-Text for spatial data.
Instead you need to use a `CAST("column_as_text" AS GEOGRAPHY)` expression in the inserter to provide the spatial data as `text` strings.
Hyper API's inserter pushes the `CAST("column_as_text" AS GEOGRAPHY)` expression down to Hyper where the `text` strings are converted to spatial data.
The Hyper API does not directly accept Well-known text (WKT) for geospatial data.
Instead you need to use a `CAST("column_as_text" AS TABLEAU.TABGEOGRAPHY)` expression in the inserter to provide the geospatial data as `text` strings.
Hyper API's inserter pushes the `CAST("column_as_text" AS TABLEAU.TABGEOGRAPHY)` expression down to Hyper where the `text` strings are converted to geospatial data.

## Overview of inserting spatial data to a hyper file
## Overview of inserting geospatial data into a hyper file

The basic process for adding spatial data involves defining your inputs to Hyper APIs inserter and specifying how to convert the text strings to geography types using Hyper SQL expressions. Hyper APIs inserter pushes the expression down to Hyper to convert text string to spatial data on the fly during insertion
The basic process for adding geospatial data involves defining your inputs to Hyper APIs inserter and specifying how to convert the text strings to the `tableau.tabgeography` type using Hyper SQL expressions. Hyper APIs inserter pushes the expression down to Hyper to convert a text string to geospatial data on the fly during insertion.

When you add the text strings into the Hyper file, the text must be in the Well-Known-Text (WKT) format for geography data. The WKT is defined by the Open GIS Consortium, Inc, in the [*OpenGIS Simple Features Specification For SQL*](https://www.opengeospatial.org/standards/sfa). The types include **Point**, **MultiPoint**, **LineString**, **MultiLineString**, **Polygon**, and **MultiPolygon**.
When you add the text strings into the Hyper file, the text must be in the Well-known text (WKT) format for geospatial data. The WKT format is defined by the Open GIS Consortium, Inc, in the [*OpenGIS Simple Features Specification For SQL*](https://www.opengeospatial.org/standards/sfa). The types include **Point**, **MultiPoint**, **LineString**, **MultiLineString**, **Polygon**, and **MultiPolygon**.

## Create tables for text and spatial data
## Create tables for text and geospatial data

1. Define and create a table in the `.hyper` file to contain the `SqlType.geography()` data. This is the table that you will use in Tableau. For example, the following Python code snippet creates a table to hold location data. The table is called `Extract` and is in the `Extract` namespace (or schema) similar to Hyper files created with Tableau.
1. Define and create a table in the `.hyper` file with a `SqlType.tabgeography()` column. This is the table that you will use in Tableau. For example, the following Python code snippet creates a table to hold location data. The table is called `Extract` and is in the `Extract` namespace (or schema) similar to Hyper files created by Tableau.

```python
connection.catalog.create_schema('Extract')
geo_table = TableDefinition(TableName('Extract','Extract'), [
TableDefinition.Column('Name', SqlType.text(), nullability=NOT_NULLABLE),
TableDefinition.Column('Location', SqlType.geography(), nullability=NOT_NULLABLE),
TableDefinition.Column('Location', SqlType.tabgeography(), nullability=NOT_NULLABLE),
])
connection.catalog.create_table(geo_table)
```

2. Define your inputs to the inserter as a List of `TableDefinition.Column` . This definition will be similar to the TableDefinition of `Extract` table created before except that the columns with `SqlType.geography()` will be specified as `SqlType.text()` type
2. Define your inputs to the inserter as a List of `TableDefinition.Column`. This definition will be similar to the TableDefinition of the `Extract` table created before except that the columns with `SqlType.tabgeography()` will be specified as `SqlType.text()` type

```python
# Inserter definition contains the column definition for the values that are inserted
Expand All @@ -36,21 +36,20 @@ When you add the text strings into the Hyper file, the text must be in the Well-
TableDefinition.Column(name='Location_as_text', type=SqlType.text(), nullability=NOT_NULLABLE)]
```

3. Specify the conversion of `SqlType.text()` to `SqlType.geography()` using `CAST` expression in `Inserter.ColumnMapping`. Specify all columns into which data is inserter in `Inserter.ColumnMapping` list. For columns that do not require any transformations provide only the names
3. Specify the conversion of `SqlType.text()` to `SqlType.tabgeography()` using `CAST` expression in `Inserter.ColumnMapping`. Specify all columns into which data is inserter in `Inserter.ColumnMapping` list. For columns that do not require any transformations provide only the names

```python
column_mappings = [
'Name',
Inserter.ColumnMapping('Location', f'CAST({escape_name("Location_as_text")} AS GEOGRAPHY)')
Inserter.ColumnMapping('Location', f'CAST({escape_name("Location_as_text")} AS TABLEAU.TABGEOGRAPHY)')
]
```

## Insert the geospatial data as text (WKT) into the text table

## Insert the spatial data as text (WKT) into the text table
When you add the text data to the Hyper file, the text must be in the Well-known text (WKT) format for geospatial data, such as Point, Polygon, etc. For example, to specify location data, you would use `point(Longitude Latitude)`.

When you add the text data to the Hyper file, the text must be in the Well Known Text Format (WKT) format for geography data, such as Point, Polygon, etc. For example, to specify location data, you would use `point(Longitude Latitude)`.

The following Python code example inserts two rows of data with location information into a table that is defined to hold geography data.
The following Python code example inserts two rows of data with location information into a table that is defined to hold geospatial data.

```python
data_to_insert = [
Expand All @@ -63,25 +62,24 @@ with Inserter(connection, geo_table, column_mappings, inserter_definition = inse
inserter.execute()
```

Note if you have WKT data in a comma-separated value (CSV) file, you can use the [COPY](/docs/sql/command/copy_from) command to insert the data from a CSV file. The command automatically converts the WKT strings to the geography data type. For more information, see the [Example code using copy from CSV](#example-code-using-copy-from-csv) and the Help topic [Insert Data Directly from CSV Files](./insert_csv) and the CSV sample on GitHub, [hyper-api-samples](https://github.com/tableau/hyper-api-samples).
Note if you have WKT data in a comma-separated value (CSV) file, you can use the [COPY](/docs/sql/command/copy_from) command to insert the data from a CSV file. The command automatically converts the WKT strings to the `tableau.tabgeography` data type. For more information, see the [Example code using copy from CSV](#example-code-using-copy-from-csv) and the Help topic [Insert Data Directly from CSV Files](./insert_csv) and the CSV sample on GitHub, [hyper-api-samples](https://github.com/tableau/hyper-api-samples).

## Example code using the Inserter

The following example Python code illustrates how you can create a `.hyper` file that contains location (`geography`) information by using expressions in the Inserter.

The following example Python code illustrates how you can create a `.hyper` file that contains location (`tableau.tabgeography`) information by using expressions in the Inserter.

```python
from tableauhyperapi import Connection, HyperProcess, SqlType, TableDefinition, \
escape_string_literal, escape_name, NOT_NULLABLE, Telemetry, Inserter, CreateMode, TableName


with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 'myapp' ) as hyper:
with Connection(hyper.endpoint, 'TrivialExample_geo.hyper', CreateMode.CREATE_AND_REPLACE) as connection:
# Create geography table
with Connection(hyper.endpoint, 'GeospatialExample.hyper', CreateMode.CREATE_AND_REPLACE) as connection:
# Create a table with a `tableau.tabgeography` column
connection.catalog.create_schema('Extract')
geo_table = TableDefinition(TableName('Extract','Extract'), [
TableDefinition.Column('Name', SqlType.text(), nullability=NOT_NULLABLE),
TableDefinition.Column('Location', SqlType.geography(), nullability=NOT_NULLABLE),
TableDefinition.Column('Location', SqlType.tabgeography(), nullability=NOT_NULLABLE),
])
print("The geo_table is defined.")
connection.catalog.create_table(geo_table)
Expand All @@ -93,15 +91,15 @@ with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 'myapp' ) as hyper:
TableDefinition.Column(name='Location_as_text', type=SqlType.text(), nullability=NOT_NULLABLE)]

# Column 'Name' is inserted into "Extract"."Extract" as-is.
# Column 'Location' in "Extract"."Extract" of geography type is computed from Column 'Location_as_text' of text type
# using the expression 'CAST("Location_as_text") AS GEOGRAPHY'.
# Column 'Location' in "Extract"."Extract" of `tableau.tabgeography` type is computed from Column 'Location_as_text' of `text` type
# using the expression 'CAST("Location_as_text") AS TABLEAU.TABGEOGRAPHY'.
# Inserter.ColumnMapping is used for mapping the CAST expression to Column 'Location'.
column_mappings = [
'Name',
Inserter.ColumnMapping('Location', f'CAST({escape_name("Location_as_text")} AS GEOGRAPHY)')
Inserter.ColumnMapping('Location', f'CAST({escape_name("Location_as_text")} AS TABLEAU.TABGEOGRAPHY)')
]

# Format the data as well-known text (WKT)
# Format the data as Well-known text (WKT)
data_to_insert = [
[ 'Seattle', "point(-122.338083 47.647528)" ],
[ 'Munich' , "point(11.584329 48.139257)" ]
Expand All @@ -116,9 +114,9 @@ with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 'myapp' ) as hyper:

## Example code using copy from CSV

When you copy the text data from a CSV file to the Hyper file, the text data is converted to geography data. Just as with the Inserter, the data must be in the Well Known Text Format (WKT) format for geography data, such as Point, Polygon, etc. For example, to specify location data, you would use `point(Longitude Latitude)`.
When you copy the text data from a CSV file to the Hyper file, the text data is converted to geospatial data. Just as with the Inserter, the data must be in the Well-known text (WKT) format for geospatial data, such as Point, Polygon, etc. For example, to specify location data, you would use `point(Longitude Latitude)`.

The following Python code example copies two rows of data from a CSV file into a table that is defined to hold geography data. The location data is in a CSV file (`locations.csv`) that looks like the following:
The following Python code example copies two rows of data from a CSV file into a table that is defined to hold geospatial data. The location data is in a CSV file (`locations.csv`) that looks like the following:

```csv title=locations.csv
Name, Location
Expand All @@ -130,16 +128,16 @@ Munich , point(11.584329 48.139257)
from tableauhyperapi import Connection, HyperProcess, SqlType, TableDefinition, \
escape_string_literal, escape_name, NOT_NULLABLE, Telemetry, Inserter, CreateMode, TableName

# CSV file that contains,
# CSV file that contains location data in Well-known text (WKT) format
path_to_csv = "locations.csv"

with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 'myapp' ) as hyper:
with Connection(hyper.endpoint, 'TrivialExample_geo_csv.hyper', CreateMode.CREATE_AND_REPLACE) as connection:
# Create geography table
with Connection(hyper.endpoint, 'GeospatialFromCSVExample.hyper', CreateMode.CREATE_AND_REPLACE) as connection:
# Create a table with a `tableau.tabgeography` column
connection.catalog.create_schema('Extract')
geo_table = TableDefinition(TableName('Extract','Extract'), [
geo_table = TableDefinition(TableName('Extract','Extract'), [
TableDefinition.Column('Name', SqlType.text(), nullability=NOT_NULLABLE),
TableDefinition.Column('Location', SqlType.geography(), nullability=NOT_NULLABLE)])
TableDefinition.Column('Location', SqlType.tabgeography(), nullability=NOT_NULLABLE)])
connection.catalog.create_table(geo_table)

# Load all rows into the geo_table from the CSV file.
Expand Down
23 changes: 18 additions & 5 deletions website/docs/releases.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,19 @@ In case you are wondering why all our releases start with `0.0`, read [this FAQ

:::

### 0.0.21408 [Feb 13 2025]

* The `geography` type has been renamed to `tableau.tabgeography` and the geospatial functions have been moved to the `tableau` namespace.
* Existing Hyper files will continue to work; however, SQL queries and HAPI programs will need to be adjusted.
* For example, use `tableau.geo_make_point` in SQL queries instead of just `geo_make_point`.
* Use `SqlType.tabgeography()` in Python and Java, and `SqlType::tabgeography()` in C++.
* The plain `geography` type and all geospatial functions outside the `tableau` namespace are deprecated and will be removed in the near future.
* See [Geographic Functions](/docs/sql/scalar_func/geography) and [Add Geospatial Data to a Hyper File](/docs/guides/hyper_file/geodata) for more information.
* IANA released version 2024a of the Time Zone Database. Hyper’s time zone information is updated accordingly. Noteworthy changes:
* Paraguay adopts permanent -03 starting spring 2024.
* Improve historical data for Mexico, Mongolia, Philippines, and Portugal.
* Update syntax for [`ARRAY` literals](./sql/datatype/array.md) and fixed bugs with quoting and escaping of text arrays.

### 0.0.21200 [Jan 17 2025]

* Support for Microsoft Azure Blob Storage using [`azure_location`](./sql/external/location.md#microsoft-azure-blob-storage) was added.
Expand Down Expand Up @@ -359,8 +372,8 @@ Noteworthy changes in the Time Zone Database:
* Hyper now adjusts the resulting interval from a timestamp subtraction so that 24-hour time periods are represented as days.
* Hyper now supports +/-13 and +/-14 as timezone offsets.
* Python: The most commonly used Hyper API types now have `__repr__()` methods and will return a string representation of the object when printed, making interactive exploring of the Hyper API more fun.
* Improved handling of spatial types:
* Parsing GEOGRAPHY values from well-known text (WKT) format automatically adjusts the order of vertices in polygons.
* Improved handling of geospatial types:
* Parsing GEOGRAPHY values from Well-known text (WKT) format automatically adjusts the order of vertices in polygons.
* During WKT parsing, additional vertices may be added to more closely resemble the original shape specified in the WKT.

### 0.0.12514 [April 7, 2021]
Expand Down Expand Up @@ -425,10 +438,10 @@ Noteworthy changes in the Time Zone Database:

### 0.0.11074 [June 24, 2020]

* Adds several SQL functions for managing spatial data:
* Adds several SQL functions for managing geospatial data:
* For creating geography objects (`geo_make_point` and `geo_make_line`).
* For performing calculations on geography objects (`geo_distance` and `geo_buffer`).
* For manipulating the vertex order of polygons in geography objects (`geo_auto_vertex_order` and `geo_invert_vertex_order`). These functions can be used to address problems (for example, with spatial joins or to automatically zoom) where data comes from a source that uses a different winding order for polygons than the one used by Tableau. In Tableau, the interior of the polygon is considered to be on the left of the path drawn by points of the polygon ring.
* For manipulating the vertex order of polygons in geography objects (`geo_auto_vertex_order` and `geo_invert_vertex_order`). These functions can be used to address problems (for example, with geospatial joins or to automatically zoom) where data comes from a source that uses a different winding order for polygons than the one used by Tableau. In Tableau, the interior of the polygon is considered to be on the left of the path drawn by points of the polygon ring.
* See [Geographic Functions](/docs/sql/scalar_func/geography) for more information.
* Prepared queries gained support for parallelized execution. See [PREPARE](/docs/sql/command/prepare) and [EXECUTE](/docs/sql/command/execute) for more information on prepared queries in Hyper.
* Java: Fixed crashes that could occur when inserting more than 16 MB of data into a table.
Expand Down Expand Up @@ -460,7 +473,7 @@ Noteworthy changes in the Time Zone Database:

* The Hyper API `Inserter` class now allows SQL expressions to compute or transform data on the fly during insertion.

* The Hyper API `Inserter` class now allows inserting Well-Known-Text (WKT) into `Geography` columns. You can use the `CAST` expression to transform WKT data to the `Geography` type and provide WKT data as a string to the `Inserter` class. For more information, see [Add Spatial Data to a Hyper File](/docs/guides/hyper_file/geodata).
* The Hyper API `Inserter` class now allows inserting Well-known text (WKT) into `Geography` columns. You can use the `CAST` expression to transform WKT data to the `Geography` type and provide WKT data as a string to the `Inserter` class. For more information, see [Add Geospatial Data to a Hyper File](/docs/guides/hyper_file/geodata).

* Documented the available settings that can be passed to the `HyperProcess` and `Connection` constructors. See [Settings](/docs/hyper-api/hyper_process#passingprocesssettings).

Expand Down
Loading