Cancelled and failed e2e jobs don't clean up resources

**What steps did you take and what happened:**
1. Start an e2e job
2. Cancel it (or let it fail)
3. Notice the created resources linger.

**What did you expect to happen:**
Resources get cleaned up no matter the result.

**Anything you would like to add:**
This makes e2e unreliable as the runner runs out of resources.

**Environment:**
Our internal e2e runners.