-
Notifications
You must be signed in to change notification settings - Fork 5
/
Copy pathindex.html
251 lines (239 loc) · 10.1 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta
name="description"
content="Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs"
/>
<meta
name="keywords"
content="LP2, trajectory prediction, scene graphs, human motion, prediction"
/>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>
Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs
</title>
<link
href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet"
/>
<link rel="stylesheet" href="./assets/css/bulma.min.css" />
<link rel="stylesheet" href="./assets/css/bulma-carousel.min.css" />
<link rel="stylesheet" href="./assets/css/bulma-slider.min.css" />
<link rel="stylesheet" href="./assets/css/fontawesome.all.min.css" />
<link
rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"
/>
<link rel="stylesheet" href="./assets/css/index.css" />
<link rel="icon" href="./assets/images/favicon.svg" />
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script defer src="./assets/js/fontawesome.all.min.js"></script>
<script src="./assets/js/bulma-carousel.min.js"></script>
<script src="./assets/js/bulma-slider.min.js"></script>
<script src="./assets/js/index.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-1 publication-title">
Long-Term Human Trajectory Prediction using 3D Dynamic Scene
Graphs
</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">
<a href="https://nicolasgorlo.com">Nicolas Gorlo</a
><sup>1</sup>,</span
>
<span class="author-block">
<a href="https://schmluk.github.io/">Lukas Schmid</a
><sup>1</sup>,</span
>
<span class="author-block">
<a href="https://lucacarlone.mit.edu/">Luca Carlone</a
><sup>1</sup>
</span>
</div>
<div class="is-size-5 publication-authors">
<span class="author-block"
><sup>1</sup
><a href="https://mit.edu/sparklab/">MIT SPARK Lab</a></span
>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<span class="link-block">
<a
href="https://ieeexplore.ieee.org/document/10720207"
class="external-link button is-normal is-rounded is-dark"
>
<span class="icon">
<i class="fas fa-file-pdf"></i>
</span>
<span>Paper</span>
</a>
</span>
<span class="link-block">
<a
href="https://arxiv.org/abs/2405.00552"
class="external-link button is-normal is-rounded is-dark"
>
<span class="icon">
<i class="ai ai-arxiv"></i>
</span>
<span>arXiv</span>
</a>
</span>
<span class="link-block">
<a
href="https://www.youtube.com/watch?v=mzumT3T0dYw"
class="external-link button is-normal is-rounded is-dark"
>
<span class="icon">
<i class="fab fa-youtube"></i>
</span>
<span>Video</span>
</a>
</span>
<span class="link-block">
<a
href="https://github.com/MIT-SPARK/LP2"
class="external-link button is-normal is-rounded is-dark"
>
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Code</span>
</a>
</span>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<section class="hero teaser">
<div class="container is-max-desktop">
<div class="hero-body">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<img
src="https://github.com/MIT-SPARK/LP2/blob/main/assets/project_page_title.gif?raw=true"
alt="LP2 Demo"
/>
<h2 class="subtitle has-text-centered">
Our method, LP2, Language-based Probabilistic Long-term Prediction,
predicts a spatio-temporal distribution over long-term human
trajectories in complex environments by reasoning about the humans
interactions with the scene, represented as a 3D Dynamic Scene
Graph.
</h2>
</div>
</div>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Abstract</h2>
<div class="content has-text-justified">
<p>
We present a novel approach for long-term human trajectory
prediction in indoor human-centric environments, which is
essential for long-horizon robot planning in these environments.
State-of-the-art human trajectory prediction methods are limited
by their focus on collision avoidance and short-term planning,
and their inability to model complex interactions of humans with
the environment. In contrast, our approach overcomes these
limitations by predicting sequences of human interactions with
the environment and using this information to guide trajectory
predictions over a horizon of up to 60s. We leverage Large
Language Models (LLMs) to predict interactions with the
environment by conditioning the LLM prediction on rich
contextual information about the scene. This information is
given as a 3D Dynamic Scene Graph that encodes the geometry,
semantics, and traversability of the environment into a
hierarchical representation. We then ground these interaction
sequences into multi-modal spatio-temporal distributions over
human positions using a probabilistic approach based on
continuous-time Markov Chains. To evaluate our approach, we
introduce a new semi-synthetic dataset of long-term human
trajectories in complex indoor environments, which also includes
annotations of human-object interactions. We show in thorough
experimental evaluations that our approach achieves a 54% lower
average negative log-likelihood and a 26.5% lower Best-of-20
displacement error compared to the best non-privileged (i.e.,
evaluated in a zero-shot fashion on the dataset) baselines for a
time horizon of 60s.
</p>
</div>
</div>
</div>
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Video</h2>
<div class="publication-video">
<iframe
width="560"
height="315"
src="https://www.youtube.com/embed/mzumT3T0dYw"
title="YouTube video player"
frameborder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen
></iframe>
</div>
</div>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Dataset</h2>
<div class="content is-centered has-text-centered">
Our dataset is available for download on
<a
href="https://drive.google.com/drive/folders/1-ThhNFVzWQvtsCLv1fo0R7Oo8fTX5uMT?usp=sharing"
>Google Drive</a
>.
</div>
</div>
</div>
</div>
</section>
<footer class="footer">
<div class="container">
<div class="columns is-centered">
<div class="column is-8">
<div class="content">
<p>
This website is licensed under a
<a
rel="license"
href="http://creativecommons.org/licenses/by-sa/4.0/"
>Creative Commons Attribution-ShareAlike 4.0 International
License</a
>.
</p>
<p>
This webpage template is borrowed from
<a href="https://github.com/nerfies/nerfies.github.io"
>nerfies</a
>. We highly appreciate them open-sourcing their template.
</p>
</div>
</div>
</div>
</div>
</footer>
</body>
</html>