Issue about fine-tuning with hpt-xlarge results in unnatural prediction transitions and fallbacks

[ZixuanLi2004]

Hello,
First of all, I would like to express my sincere gratitude for your contributions. Your work has been incredibly helpful.
I am currently fine-tuning the hpt-xlarge model on my own dataset. However, I have encountered an issue where the predictions are not transitioning smoothly and there are fallbacks. My configuration is as follows: observation_horizon: 5, action_horizon: 8.
I would greatly appreciate any suggestions or advice you could offer to improve the prediction transitions and reduce fallbacks.
Thank you very much for your time and assistance.

a test video: https://github.com/user-attachments/assets/6e875576-5bb8-420d-9094-596853bab598)

[AliBehdar]

Hi
Was this problem before fine tuning the model? I can see in the simulation there a bit of fallbacks and what is your FPS?
I think your problem can have 3 reason first one is GPU speed 2. bad fine tuning and 3 is conde implementation of the code spatially in setting up the model