Skip to content

Commit 6cdca1d

Browse files
MargrietGroenendijkSarantopoulosKon
MargrietGroenendijk
and
SarantopoulosKon
authored
synch with TileDB-Viz (#50)
* streaming added * rename variables * updated yarn.lock * Update node version to v14 for azure-pipelines * tiledb-env added * Update viz-core to latest * Update widget.ts * Update widget.ts * Update viz-core * Update viz-core to latest Co-authored-by: Sarantopoulos Konstantinos <[email protected]>
1 parent 9cfe23e commit 6cdca1d

19 files changed

+468
-1156
lines changed

‎README.md

Lines changed: 42 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,12 @@
55
[![Build Status](https://dev.azure.com/TileDB-Inc/CI/_apis/build/status/TileDB-Inc.TileDB-PyBabylonJS?branchName=main)](https://dev.azure.com/TileDB-Inc/CI/_build/latest?definitionId=37&branchName=main)
66

77

8-
The TileDB-PyBabylonJS library is a geospatial data visualization Python library that interactively visualizes TileDB arrays with [Babylon.js](https://www.babylonjs.com) in a Jupyter notebook widget.
8+
The TileDB-PyBabylonJS library is a geospatial data visualization Python library that interactively visualizes TileDB arrays with [Babylon.js](https://www.babylonjs.com) in a Jupyter notebook widget.
9+
10+
The package is under development and currently contains:
11+
12+
* point cloud visualizations with the option to stream all data from a TileDB array or define a bounding box to load a slice of the array
13+
* MBRS visualization showing the minimum bounding rectangles of the [fragments](https://docs.tiledb.com/main/background/key-concepts-and-data-format#fragments) in the sparse array containing point cloud data
914

1015
## Installation
1116

@@ -15,15 +20,14 @@ This project is available from [PyPI](https://pypi.org/project/pybabylonjs/) and
1520
pip install pybabylonjs
1621
```
1722

18-
If you are using Jupyter Notebook 5.2 or earlier, you may also need to enable
19-
the nbextension:
23+
If you are using Jupyter Notebook 5.2 or earlier, you may also need to enable the nbextension:
2024
```bash
2125
jupyter nbextension enable --py [--sys-prefix|--user|--system] pybabylonjs
2226
```
2327

2428
## Development Installation
2529

26-
Create and activate a dev environment:
30+
Create and activate a development environment:
2731

2832
```bash
2933
conda install -c conda-forge mamba
@@ -35,21 +39,21 @@ conda activate pybabylonjs-dev
3539
pip install opencv-python
3640
```
3741

38-
Fork or clone the repo and go the main directory. Install the TileDB-PyBabylonJS Python package that will also build the TypeScript package:
42+
Fork or clone the repo and go to the main directory. Install the TileDB-PyBabylonJS Python package that will also build the TypeScript package:
3943

4044
```bash
4145
pip install -e ".[test, examples]"
4246
```
4347

44-
When developing your extensions, you need to manually enable your extensions with the notebook / lab frontend. For jupyter lab, this is done by the command:
48+
When developing your extensions you need to manually enable your extensions with the notebook / lab frontend. For jupyter lab this is done by the command:
4549

4650
```bash
4751
jupyter labextension install @jupyter-widgets/jupyterlab-manager
4852
yarn run build
4953
jupyter labextension install .
5054
```
5155

52-
For a classic notebook, you need to run:
56+
For a classic notebook you need to run:
5357

5458
```bash
5559
jupyter nbextension install --sys-prefix --symlink --overwrite --py pybabylonjs
@@ -58,14 +62,13 @@ jupyter nbextension enable --sys-prefix --py pybabylonjs
5862

5963
Note that the `--symlink` flag doesn't work on Windows, so you will here have to run
6064
the `install` command every time that you rebuild your extension. For certain installations
61-
you might also need another flag instead of `--sys-prefix`, but we won't cover the meaning
62-
of those flags here.
65+
you might also need another flag instead of `--sys-prefix`.
6366

6467
### How to see your changes
6568

6669
#### TypeScript
6770

68-
The TypeScript code for the visualizations can be found in the [TileDB-Viz](https://github.com/TileDB-Inc/TileDB-Viz) package. After making changes in TileDB-Viz build the package with:
71+
The TypeScript code for the visualizations can be found in the [TileDB-Viz](https://github.com/TileDB-Inc/TileDB-Viz) repository. After making changes in TileDB-Viz build the package with:
6972

7073
`yarn build`
7174

@@ -85,41 +88,49 @@ When you make a change to the Python code rebuild the package and restart the no
8588

8689
### Point clouds
8790

88-
Jupyter notebooks are provided in the [Examples folder](https://github.com/TileDB-Inc/TileDB-PyBabylonJS/tree/main/examples) for the following visualizations:
91+
Jupyter notebooks are provided in the [examples folder](https://github.com/TileDB-Inc/TileDB-PyBabylonJS/tree/main/examples) for the following visualizations:
8992

90-
* [Point cloud](/examples/point_cloud.ipynb)
91-
* [Point cloud with a time slider](/examples/point-cloud-time.ipynb)
92-
* [Point cloud with a classes slider](/examples/point-cloud-classes.ipynb)
93-
* [Point cloud with a Mapbox base map](/examples/point-cloud-topo.ipynb)
94-
* [Point cloud with gltf models](/examples/point-cloud-gltf.ipynb)
95-
* [MBRS of a point cloud](/examples/mbrs.ipynb)
93+
* [Slice of the Autzen point cloud](/examples/autzen_slice.ipynb)
94+
* [Streaming the Autzen point cloud](/examples/autzen-streaming.ipynb)
95+
* [Slice of the Boulder point cloud](/examples/point-cloud-boulder.ipynb)
9696

97-
Display a point cloud visualization from a local sparse array by specifying the bounding box of a slice of the data:
97+
Display a point cloud visualization from a TileDB cloud sparse array by specifying the bounding box of a slice of the data:
9898

9999
```python
100100
from pybabylonjs import Show as show
101101

102102
bbox = {
103-
'X': [636800, 637800],
104-
'Y': [851000, 853000],
103+
'X': [636800, 637200],
104+
'Y': [852800, 853100],
105105
'Z': [406.14, 615.26]
106106
}
107107

108108
lidar_array = "autzen-classified"
109109

110-
show.point_cloud(source="local",
111-
uri=lidar_array,
112-
bbox = bbox)
110+
show.point_cloud(source="cloud",
111+
uri = "tiledb://TileDB-Inc/autzen_classified_tiledb",
112+
token = "***",
113+
bbox = bbox,
114+
particle_size = 2.5,
115+
width = 1000,
116+
height = 900,
117+
rgb_max = 65535,
118+
camera_radius = 700)
113119
```
114120

115-
<img src="examples/pointcloud.png" width="400" height="300" />
116-
117-
To add a slider over `GpsTime` change the `mode` to `time`:
121+
Or stream all data from a group of arrays:
118122

119123
```python
120-
show.point_cloud(source="local",
121-
mode="time",
122-
uri=uri,
123-
bbox=bbox)
124-
```
124+
show.point_cloud(streaming=True,
125+
uri="***",
126+
token="***",
127+
max_levels=6,
128+
particle_size = 3,
129+
color_scheme = 'light',
130+
width = '1200px',
131+
height = '800px',
132+
rgb_max = 255,
133+
camera_radius = 800,
134+
particle_budget = 8000000)
135+
```
125136

‎azure-pipelines.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ stages:
6767

6868
- task: NodeTool@0
6969
inputs:
70-
versionSpec: '12.x'
70+
versionSpec: '14.x'
7171
- script: yarn
7272
displayName: 'Install labextension dependencies'
7373
env:
@@ -121,7 +121,7 @@ stages:
121121

122122
- task: NodeTool@0
123123
inputs:
124-
versionSpec: '12.x'
124+
versionSpec: '14.x'
125125
- script: |
126126
echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc
127127
yarn

‎examples/autzen-streaming.ipynb

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "ea5e39c9-72ce-4032-8645-67445ab34074",
6+
"metadata": {},
7+
"source": [
8+
"# Stream and visualize the Autzen point cloud"
9+
]
10+
},
11+
{
12+
"cell_type": "code",
13+
"execution_count": null,
14+
"id": "4cbfca2f-4088-4605-843f-6de49ed74b3b",
15+
"metadata": {},
16+
"outputs": [],
17+
"source": [
18+
"import requests\n",
19+
"import json\n",
20+
"\n",
21+
"import pandas as pd\n",
22+
"import pdal\n",
23+
"from pybabylonjs import Show as show\n",
24+
"import tiledb"
25+
]
26+
},
27+
{
28+
"cell_type": "markdown",
29+
"id": "e9df8cc1-8f12-46f0-86bf-5af11b76959c",
30+
"metadata": {},
31+
"source": [
32+
"### Cloud array\n",
33+
"Load and display point cloud data from cloud arrays with different levels of detail (LOD)."
34+
]
35+
},
36+
{
37+
"cell_type": "code",
38+
"execution_count": null,
39+
"id": "f68dfda1-7582-45f7-bc1f-f993448a9a1d",
40+
"metadata": {},
41+
"outputs": [],
42+
"source": [
43+
"token = \"<token>\""
44+
]
45+
},
46+
{
47+
"cell_type": "code",
48+
"execution_count": null,
49+
"id": "12fbeff2-73a2-4ff0-bc8f-bb69870948af",
50+
"metadata": {},
51+
"outputs": [],
52+
"source": [
53+
"uri = \"tiledb://norman/autzen\""
54+
]
55+
},
56+
{
57+
"cell_type": "code",
58+
"execution_count": null,
59+
"id": "c927a1a2-0371-4af3-bf96-1ab8ed94e337",
60+
"metadata": {},
61+
"outputs": [],
62+
"source": [
63+
"show.point_cloud(streaming=True,\n",
64+
" uri=uri,\n",
65+
" token=token,\n",
66+
" max_levels=6,\n",
67+
" point_size = 3,\n",
68+
" color_scheme = 'light',\n",
69+
" width = '1200px',\n",
70+
" height = '800px',\n",
71+
" rgb_max = 255,\n",
72+
" camera_radius = 800,\n",
73+
" point_budget = 8000000)"
74+
]
75+
},
76+
{
77+
"cell_type": "code",
78+
"execution_count": null,
79+
"id": "565b0786-c713-4d01-9bdf-b334db4705e3",
80+
"metadata": {},
81+
"outputs": [],
82+
"source": [
83+
"show.point_cloud(streaming=True,\n",
84+
" uri=uri,\n",
85+
" token=token,\n",
86+
" max_levels=6,\n",
87+
" point_size = 3,\n",
88+
" color_scheme = 'light',\n",
89+
" width = '1200px',\n",
90+
" height = '800px',\n",
91+
" rgb_max = 255,\n",
92+
" camera_radius = 800,\n",
93+
" point_budget = 8000000,\n",
94+
" use_shader = True)"
95+
]
96+
}
97+
],
98+
"metadata": {
99+
"kernelspec": {
100+
"display_name": "Python 3 (ipykernel)",
101+
"language": "python",
102+
"name": "python3"
103+
},
104+
"language_info": {
105+
"codemirror_mode": {
106+
"name": "ipython",
107+
"version": 3
108+
},
109+
"file_extension": ".py",
110+
"mimetype": "text/x-python",
111+
"name": "python",
112+
"nbconvert_exporter": "python",
113+
"pygments_lexer": "ipython3",
114+
"version": "3.7.10"
115+
}
116+
},
117+
"nbformat": 4,
118+
"nbformat_minor": 5
119+
}

0 commit comments

Comments
 (0)