Release concurrency locks when supervisor fails the jobs handled by workers #547
+39
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When workers 'die' because of various reasons, the supervisor unlocks the jobs that were being handled by those workers by deleting the
claimed_executions
for the job and creatingfailed_executions
. However, it does not release any concurrency locks taken by the jobs. This can be an issue if the concurrency limit was reached for the job because no further jobs can run until the concurrency duration is reached.In this PR, I call
job.unblock_next_blocked_job
when the supervisor releases all claimed executions of the worker.Fixes #546