We need to implement "soft canceling" of fine-tuning jobs, so that the hardware returns in the correct state after canceling.

It could be implemented with a flag which should be repeatedly checked at the end of each step during fine-tuning. In case of canceling, the flag is first turned to False, and the fine-tuning job is canceled once it ends the step.