Skip to content

Commit 9903b89

Browse files
Brooke-whiteBrooke-white
committed
docs(tutorials): add redshift datatype examples
1 parent 1a3d820 commit 9903b89

File tree

3 files changed

+238
-41
lines changed

3 files changed

+238
-41
lines changed

‎README.rst

+1
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ Tutorials
5656
- `001 - Connecting to Amazon Redshift <https://github.com/aws/amazon-redshift-python-driver/blob/master/tutorials/001%20-%20Connecting%20to%20Amazon%20Redshift.ipynb>`_
5757
- `002 - Data Science Library Integrations <https://github.com/aws/amazon-redshift-python-driver/blob/master/tutorials/002%20-%20Data%20Science%20Library%20Integrations.ipynb>`_
5858
- `003 - Amazon Redshift Feature Support <https://github.com/aws/amazon-redshift-python-driver/blob/master/tutorials/003%20-%20Amazon%20Redshift%20Feature%20Support.ipynb>`_
59+
- `004 - Amazon Redshift Datatypes <https://github.com/aws/amazon-redshift-python-driver/blob/master/tutorials/004%20-%20Amazon%20Redshift%20Datatypes.ipynb>`_
5960

6061
We are working to add more documentation and would love your feedback. Please reach out to the team by `opening an issue <https://github.com/aws/amazon-redshift-python-driver/issues/new/choose>`__ or `starting a discussion <https://github.com/aws/amazon-redshift-python-driver/discussions/new>`_ to help us fill in the gaps in our documentation.
6162

‎tutorials/003 - Amazon Redshift Feature Support.ipynb

+23-41
Original file line numberDiff line numberDiff line change
@@ -13,26 +13,27 @@
1313
},
1414
{
1515
"cell_type": "markdown",
16+
"metadata": {},
1617
"source": [
1718
"# Overview\n",
1819
"`redshift_connector` aims to support the latest and greatest features provided by Amazon Redshift so you can get the most out of your data."
19-
],
20-
"metadata": {
21-
"collapsed": false
22-
}
20+
]
2321
},
2422
{
2523
"cell_type": "markdown",
24+
"metadata": {},
2625
"source": [
2726
"## COPY and UNLOAD Support - Amazon S3\n",
2827
"`redshift_connector` provides the ability to `COPY` and `UNLOAD` data from an Amazon S3 bucket. Shown below is a sample workflow which copies and unloads data from an Amazon S3 bucket"
29-
],
30-
"metadata": {
31-
"collapsed": false
32-
}
28+
]
3329
},
3430
{
3531
"cell_type": "markdown",
32+
"metadata": {
33+
"pycharm": {
34+
"name": "#%% md\n"
35+
}
36+
},
3637
"source": [
3738
"1. Upload the following text file to an Amazon S3 bucket and name it `category_csv.txt`\n",
3839
"\n",
@@ -42,17 +43,16 @@
4243
" 14,Shows,Opera,\"All opera, light, and \"\"rock\"\" opera\"\n",
4344
" 15,Concerts,Classical,\"All symphony, concerto, and choir concerts\"\n",
4445
"```"
45-
],
46-
"metadata": {
47-
"collapsed": false,
48-
"pycharm": {
49-
"name": "#%% md\n"
50-
}
51-
}
46+
]
5247
},
5348
{
5449
"cell_type": "code",
5550
"execution_count": null,
51+
"metadata": {
52+
"pycharm": {
53+
"name": "#%%\n"
54+
}
55+
},
5656
"outputs": [],
5757
"source": [
5858
"import redshift_connector\n",
@@ -70,53 +70,35 @@
7070
" print(cursor.fetchall())\n",
7171
" cursor.execute(\"unload ('select * from category') to 's3://testing/unloaded_category_csv.txt' iam_role 'arn:aws:iam::123:role/RedshiftCopyUnload' csv;\")\n",
7272
" print('done')\n"
73-
],
74-
"metadata": {
75-
"collapsed": false,
76-
"pycharm": {
77-
"name": "#%%\n"
78-
}
79-
}
73+
]
8074
},
8175
{
8276
"cell_type": "markdown",
77+
"metadata": {},
8378
"source": [
8479
"After executing the above code block, we can see the requested data was unloaded into the following file, `unloaded_category_csv.text0000_part00`, in the specified Amazon s3 bucket\n"
85-
],
86-
"metadata": {
87-
"collapsed": false
88-
}
89-
},
90-
{
91-
"cell_type": "markdown",
92-
"source": [
93-
"## Datatype Support\n",
94-
"`redshift_connector` supports Amazon Redshift specific datatypes in order to provide users integration of their data into Python projects. Please see the projects [README](https://github.com/aws/amazon-redshift-python-driver/blob/master/README.rst) for a list of supported datatypes."
95-
],
96-
"metadata": {
97-
"collapsed": false
98-
}
80+
]
9981
}
10082
],
10183
"metadata": {
10284
"kernelspec": {
103-
"display_name": "Python 3",
85+
"display_name": "Python 3 (ipykernel)",
10486
"language": "python",
10587
"name": "python3"
10688
},
10789
"language_info": {
10890
"codemirror_mode": {
10991
"name": "ipython",
110-
"version": 2
92+
"version": 3
11193
},
11294
"file_extension": ".py",
11395
"mimetype": "text/x-python",
11496
"name": "python",
11597
"nbconvert_exporter": "python",
116-
"pygments_lexer": "ipython2",
117-
"version": "2.7.6"
98+
"pygments_lexer": "ipython3",
99+
"version": "3.9.7"
118100
}
119101
},
120102
"nbformat": 4,
121-
"nbformat_minor": 0
103+
"nbformat_minor": 1
122104
}

‎tutorials/004 - Amazon Redshift Datatypes.ipynb

+214
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {
6+
"collapsed": true
7+
},
8+
"source": [
9+
"## Datatype Support\n",
10+
"`redshift_connector` supports Amazon Redshift specific datatypes in order to provide users integration of their data into Python projects. Please see the projects [README](https://github.com/aws/amazon-redshift-python-driver/blob/master/README.rst) for a list of supported datatypes."
11+
]
12+
},
13+
{
14+
"cell_type": "markdown",
15+
"metadata": {},
16+
"source": [
17+
"### Examples\n",
18+
"The following sections provide basic examples showing how to work with Amazon Redshift datatypes.\n",
19+
"\n",
20+
"#### Geometry\n",
21+
"- **Send**: A string holding geometry data in WKB (well known binary) format.\n",
22+
"- **Receive**: A string holding geometry data in WKB format.\n",
23+
"\n",
24+
"**Note**: Geometry data can be sent and receive in formats other than WKB if Amazon Redshift spatial functions are applied. Please see the [Amazon Redshift documentation for a list of spacial functions](https://docs.aws.amazon.com/redshift/latest/dg/geospatial-functions.html).\n",
25+
"\n",
26+
"[Geometry](https://docs.aws.amazon.com/redshift/latest/dg/GeometryType-function.html)\n",
27+
"\n",
28+
"Sending data in WKB format:"
29+
]
30+
},
31+
{
32+
"cell_type": "code",
33+
"execution_count": null,
34+
"metadata": {},
35+
"outputs": [],
36+
"source": [
37+
"import redshift_connector\n",
38+
"\n",
39+
"with redshift_connector.connect(...) as conn:\n",
40+
" with conn.cursor() as cursor:\n",
41+
" cursor.execute(\"create table datatype_test (c1 geometry);\")\n",
42+
" cursor.execute(\n",
43+
" \"insert into datatype_test (c1) values (%s);\",\n",
44+
" (\n",
45+
" '0103000020E61000000100000005000000000000000000000000000000000000000000000000000000000000000000F03F000000000000F03F000000000000F03F000000000000F03F000000000000000000000000000000000000000000000000',\n",
46+
" # using WKB format\n",
47+
" )\n",
48+
" )\n",
49+
" cursor.execute(\"select c1 from datatype_test;\")\n",
50+
" result = cursor.fetchone()\n",
51+
" print(\"c1={}\\n\".format(result[0],))"
52+
]
53+
},
54+
{
55+
"cell_type": "markdown",
56+
"metadata": {},
57+
"source": [
58+
"Sending data in WKT (well known text) format:"
59+
]
60+
},
61+
{
62+
"cell_type": "code",
63+
"execution_count": null,
64+
"metadata": {},
65+
"outputs": [],
66+
"source": [
67+
"import redshift_connector\n",
68+
"\n",
69+
"with redshift_connector.connect(...) as conn:\n",
70+
" with conn.cursor() as cursor:\n",
71+
" cursor.execute(\"create table datatype_test (c1 geometry);\")\n",
72+
" cursor.execute(\n",
73+
" \"insert into datatype_test (c1) values (ST_GeomFromText(%s));\",\n",
74+
" (\n",
75+
" 'LINESTRING(1 2,3 4,5 6,7 8,9 10,11 12,13 14,15 16,17 18,19 20)', # using WKT format\n",
76+
" )\n",
77+
" )\n",
78+
" cursor.execute(\"select c1, c2 from datatype_test;\")\n",
79+
" result = cursor.fetchone()\n",
80+
" print(\"c1={}\\nc2={}\".format(result[0], result[1]))"
81+
]
82+
},
83+
{
84+
"cell_type": "markdown",
85+
"metadata": {},
86+
"source": [
87+
"#### Super\n",
88+
"- **Send**: A string containing JSON data.\n",
89+
"- **Receive**: A string containing JSON data\n",
90+
"\n",
91+
"[Super](https://docs.aws.amazon.com/redshift/latest/dg/r_SUPER_type.html)\n",
92+
"[json_parse](https://docs.aws.amazon.com/redshift/latest/dg/JSON_PARSE.html)\n",
93+
"[Unnesting SUPER arrays](https://docs.aws.amazon.com/redshift/latest/dg/query-super.html#unnest)\n",
94+
"[Querying semistructured data](https://docs.aws.amazon.com/redshift/latest/dg/query-super.html)"
95+
]
96+
},
97+
{
98+
"cell_type": "code",
99+
"execution_count": null,
100+
"metadata": {},
101+
"outputs": [],
102+
"source": [
103+
"import redshift_connector\n",
104+
"\n",
105+
"with redshift_connector.connect(...) as conn:\n",
106+
" with conn.cursor() as cursor:\n",
107+
" cursor.execute(\n",
108+
" \"CREATE TABLE foo AS SELECT json_parse(%s) AS multi_level_array;\",\n",
109+
" ('[[1.1, 1.2], [2.1, 2.2], [3.1, 3.2]]',)\n",
110+
" )\n",
111+
" cursor.execute(\"SELECT array, element FROM foo AS f, f.multi_level_array AS array, array AS element;\")\n",
112+
" result = cursor.fetchall()\n",
113+
" print(result)"
114+
]
115+
},
116+
{
117+
"cell_type": "markdown",
118+
"metadata": {},
119+
"source": [
120+
"Retrieving array elements from json array stored in super datatype"
121+
]
122+
},
123+
{
124+
"cell_type": "code",
125+
"execution_count": null,
126+
"metadata": {},
127+
"outputs": [],
128+
"source": [
129+
"import redshift_connector \n",
130+
"import json\n",
131+
"\n",
132+
"with redshift_connector.connect(...) as conn:\n",
133+
" with conn.cursor() as cursor:\n",
134+
" cursor.execute(\n",
135+
" \"CREATE TABLE foo AS SELECT json_parse(%s) AS vals;\",\n",
136+
" (json.dumps({\"x\": [1,2,3,4], \"y\": [5,6,7,8], \"z\": [9,10,11,12]}),)\n",
137+
" )\n",
138+
" cursor.execute(\"SELECT vals.x FROM foo;\")\n",
139+
" result = cursor.fetchall()\n",
140+
" print(result)"
141+
]
142+
},
143+
{
144+
"cell_type": "code",
145+
"execution_count": null,
146+
"metadata": {},
147+
"outputs": [],
148+
"source": [
149+
"import redshift_connector \n",
150+
"import json\n",
151+
"\n",
152+
"with redshift_connector.connect(...) as conn:\n",
153+
" with conn.cursor() as cursor:\n",
154+
" cursor.execute(\"create table t (s super);\")\n",
155+
" cursor.execute(\"insert into t values (json_parse(%s));\", ('[10001,10002,\"abc\"]',))\n",
156+
" cursor.execute(\"insert into t values (json_parse(%s));\", (json.dumps({\"x\": [1,2,3,4]}),))\n",
157+
" cursor.execute(\"select s from t;\")\n",
158+
" result = cursor.fetchall()\n",
159+
" print(result)"
160+
]
161+
},
162+
{
163+
"cell_type": "markdown",
164+
"metadata": {},
165+
"source": [
166+
"#### Varbyte\n",
167+
"- **Send**: A string or bytes\n",
168+
"- **Receive**: A string containing data in hexidecimal format\n",
169+
"\n",
170+
"[Varbyte](https://docs.aws.amazon.com/redshift/latest/dg/r_VARBYTE_type.html)"
171+
]
172+
},
173+
{
174+
"cell_type": "code",
175+
"execution_count": null,
176+
"metadata": {},
177+
"outputs": [],
178+
"source": [
179+
"import redshift_connector\n",
180+
"\n",
181+
"with redshift_connector.connect(...) as conn:\n",
182+
" with conn.cursor() as cursor:\n",
183+
" cursor.execute(\"create table t (v varbyte);\")\n",
184+
" cursor.execute(\"insert into t values (%s), (%s);\", ('aa', 'abc', ))\n",
185+
" cursor.execute(\"insert into t values (%s), (%s);\", (b'aa', b'abc',))\n",
186+
" cursor.execute(\"insert into t values (%s), (%s);\", (b'\\x00\\x01\\x02\\x03',b'\\x00\\x0a\\x0b\\x0c'))\n",
187+
" cursor.execute(\"select v from t;\")\n",
188+
" result = cursor.fetchall()\n",
189+
" print(result)"
190+
]
191+
}
192+
],
193+
"metadata": {
194+
"kernelspec": {
195+
"display_name": "Python 3 (ipykernel)",
196+
"language": "python",
197+
"name": "python3"
198+
},
199+
"language_info": {
200+
"codemirror_mode": {
201+
"name": "ipython",
202+
"version": 3
203+
},
204+
"file_extension": ".py",
205+
"mimetype": "text/x-python",
206+
"name": "python",
207+
"nbconvert_exporter": "python",
208+
"pygments_lexer": "ipython3",
209+
"version": "3.9.7"
210+
}
211+
},
212+
"nbformat": 4,
213+
"nbformat_minor": 1
214+
}

0 commit comments

Comments
 (0)