[persist] Allow disabling the sink progress collection optimization #31464
+14
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Discussed live: easy and possibly useful.
Tips for reviewer
Note that this doesn't allow disabling the creation etc. of the new shard or the writes in the Kafka sink itself, since that involves wider changes to the controller that are hard to separate out. Still, this may be useful in the case where that stuff is fine but the actual optimization itself is unsafe...
I still have a couple TODOs I'd like to get to ~soonish, but I'll split those out to a separate PR to increase the odds this one merges by the release cut.
Checklist
$T ⇔ Proto$T
mapping (possibly in a backwards-incompatible way), then it is tagged with aT-proto
label.