[Question] How to run Hierarchical RL (HIRO) and Isaac Sim Lifecycle Control with IsaacLab RL class like RLGames

[huangjiancong1]

Question

Hierarchical RL (HIRO-style)
How can we implement a HIRO-like hierarchical reinforcement learning setup in IsaacLab (high-level policy for subgoals, low-level policy for control)?
Does IsaacLab or RL-Games support multi-policy training natively, or is a custom environment/training loop required? Any recommended structure or examples would be appreciated.

Build Info

Describe the versions that you are currently using:

• Isaac Lab Version: 2.3.1
• Isaac Sim Version: 5.1.0

[RandomOakForest]

Thank you for posting this. IsaacLab and RL-Games do not currently provide a plug‑and‑play HIRO-style hierarchical RL runner; you will need a custom multi‑policy setup that uses the IsaacLab RL wrappers but orchestrates two policies yourself. I will move this post to our Discussions section for follow up. @Toni-SM for vis.

[huangjiancong1]

Thank you for the reply. Yes, a personal wrapper with isaaclab_rl will be a good choice. It is fine to use an abstract env for a high level agent and only have the low level agent interact with an isaacsim env. A personal wrapper should be a good choice.