AI2Thor Keyboard Player

AI2Thor Data Collection Tool Via Keyboard Interaction

✅ Both Navigation and Interaction

✅ First View / Third View Camera

✅ Rotate Scene

✅ Generate GIF or MP4

✅ ProcTHOR🏘️ Supported

Quick Start

Installation 💻

pip install ai2thor

Follwing the instuctions from AI2Thor to install https://ai2thor.allenai.org/ithor/documentation

Start 👋

python keyboard_player.py

Enter "P" to end the game.

And an interaction video and GIF will be generated automatically.

Navigation 🏃

Press on the keyboard with the mouse on the screen.

Walk	LookAround	Pitching

Interaction 🦾

Press on the keyboard with the mouse on the screen.

And then choose the expected interactive object index from the object lists on the console.

Enter "P" to interupt interaction.

Pick&Put	Open&Close	Toggle&Slice

Demo 📹

View	Demo
First View	first_view_FloorPlan19_physics_3x.mp4
Third View w/o Rotate	third_view_FloorPlan19_physics_3x.mp4
Bird's Eye View	third_view_FloorPlan19_physics_BEV_3x.mp4
Third View w/ Rotate	third_view_FloorPlan19_physics_rotate_3x.mp4

Ackonwledgements

We are grateful to AI2-Thor and ProcTHOR which provide us the useful embodied agent playground.

AI2-THOR

@article{ai2thor,
  author={Eric Kolve and Roozbeh Mottaghi and Winson Han and
          Eli VanderBilt and Luca Weihs and Alvaro Herrasti and
          Daniel Gordon and Yuke Zhu and Abhinav Gupta and
          Ali Farhadi},
  title={{AI2-THOR: An Interactive 3D Environment for Visual AI}},
  journal={arXiv},
  year={2017}
}

ProcTHOR

@inproceedings{procthor,
  author={Matt Deitke and Eli VanderBilt and Alvaro Herrasti and
          Luca Weihs and Jordi Salvador and Kiana Ehsani and
          Winson Han and Eric Kolve and Ali Farhadi and
          Aniruddha Kembhavi and Roozbeh Mottaghi},
  title={{ProcTHOR: Large-Scale Embodied AI Using Procedural Generation}},
  booktitle={NeurIPS},
  year={2022},
  note={Outstanding Paper Award}
}

Contact Us

If you have any questions, please feel free to contact us

We are from Visual Information Processing and Learning (VIPL) research group, which attached to the Institute of Computing Technology of the Chinese Academy of Sciences (ICT, CAS) and the key Intelligent Information Processing Laboratory of the Chinese Academy of Sciences.