FreeAD is an end-to-end (E2E) robot navigation system designed to operate in unstructured environments such as auxiliary roads, campus paths, and indoor settings. Unlike traditional E2E autonomous driving models that focus on structured roads, FreeAD aims to improve navigation capabilities in these challenging scenarios.
This project introduces the FreeWorld Dataset, a comprehensive dataset combining real-world robot data and synthetic data generated using the Isaac Sim simulator. This dataset is tailored for training and evaluating E2E autonomous driving models in unstructured environments.
To validate the dataset's effectiveness, we fine-tuned the efficient E2E driving model VAD using FreeWorld. Our results demonstrate that this fine-tuning significantly enhances navigation performance in unstructured environments.
This repository provides the first dataset specifically for E2E robot navigation in unstructured scenarios and offers a benchmark for vision-based E2E navigation technology, supporting the development of logistics and service robots.
We modified some APIs from the nuScenes dataset to enhance flexibility and support a wider variety of data and map scenarios. The modified code has been localized and named FreeWorld. The FreeWorld Dataset is available for access.
The FT-VAD model (FT_VAD_s1e6_s2e3.pth) was trained for 6 epochs in stage 1 and 3 epochs in stage 2, based on the pre-trained VAD-Base model.
The FT-VAD model is available on Hugging Face.
- Open-loop planning results on FreeWorld.
| Method | L2 (m) 1s | L2 (m) 2s | L2 (m) 3s | Avg. | AP divider | FPS | Collision (Avg. %) |
|---|---|---|---|---|---|---|---|
| VAD-Tiny | 0.891 | 1.600 | 2.449 | 1.647 | 0.000 | 8.7 | 0.00 |
| VAD-Base | 0.499 | 0.759 | 1.040 | 0.766 | 0.001 | 5.0 | 0.00 |
| FT-VAD | 0.421 | 0.595 | 0.753 | 0.589 | 0.480 | 5.0 | 0.00 |
- Open-loop planning results on nuScenes.
| Method | L2 (m) 1s | L2 (m) 2s | L2 (m) 3s | Col. (%) 1s | Col. (%) 2s | Col. (%) 3s | FPS |
|---|---|---|---|---|---|---|---|
| VAD-Tiny | 0.46 | 0.76 | 1.12 | 0.21 | 0.35 | 0.58 | 16.8 |
| VAD-Base | 0.41 | 0.70 | 1.05 | 0.07 | 0.17 | 0.41 | 4.5 |
| FT-VAD | 3.93 | 6.54 | 9.16 | 0.004 | 0.017 | 0.031 | 5.0 |
If you have any questions or suggestions about this repo, please feel free to contact us ([email protected]).
All code in this repository is under the Apache License 2.0.