🐛 Use SSA instead of Update for finalizer operations #2328
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
The ClusterExtension and ClusterCatalog finalizer code used Update, the ClusterExtensionRevision used Apply, so the CE and CE finalizer code was rewritten to be like CER. There is common finalizer code between them now. |
|
Ping @joelanford |
|
catalogd pod is panicing... it only fails due to the summary generated, it's not "noticeable" during a regular run! |
If we want to remove conflicts, we should use patch, but adding/removing finalizer we do not touch removed fields (including the annotation). Controller-runtime cache is write-through cache, so an update or patch operation does not perform any direct modification on the cache - all changes are made by the informer. In what situations did we see conflicts, given that only single controller is responsible for a resource type? |
This is not a conflict issue. The problem was that we are using Update(), which uses the cached version of the resource to make the changes. The cached version no longer includes the This causes a problem with e.g. kubectl applying a ClusterExtension a second time. If you apply a CE twice without this code (i.e. current The problem is that the |
tmshort
force-pushed
the
apply-finalizers
branch
from
21e90d9 to
0608a72
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2328 +/- ##
==========================================
- Coverage 74.30% 74.18% -0.13%
==========================================
Files 91 91
Lines 7083 7108 +25
==========================================
+ Hits 5263 5273 +10
- Misses 1405 1412 +7
- Partials 415 423 +8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| // If the finalizer already exists, this is a no-op and returns (false, nil). | ||
| // Returns (true, nil) if the finalizer was added. | ||
| // Note: This function will update the passed object with the server response. | ||
| func EnsureFinalizer(ctx context.Context, c client.Client, obj client.Object, finalizer string) (bool, error) { |
There was a problem hiding this comment.
If we want to add multiple finalizers, it will mean multiple Patch calls to the apiserver as this is written. Perhaps we should have this function accept multiple finalizers so that we can limit the number of patch calls (and watch events produced/received by apiserver/informers)?
| currentFinalizers := obj.GetFinalizers() | ||
| newFinalizers := make([]string, len(currentFinalizers), len(currentFinalizers)+1) | ||
| copy(newFinalizers, currentFinalizers) | ||
| newFinalizers = append(newFinalizers, finalizer) |
There was a problem hiding this comment.
Sort the new finalizers to ensure deterministic ordering?
| return false, fmt.Errorf("marshalling patch to add finalizer: %w", err) | ||
| } | ||
| // Note: Patch will update obj with the server response, including ResourceVersion | ||
| if err := c.Patch(ctx, obj, client.RawPatch(types.MergePatchType, patchJSON)); err != nil { |
There was a problem hiding this comment.
that could be a good improvement, that would simplify the reconcile logic a lot, i.e. we do not need to care about appending/sorting finalizers at all.
|
|
||
| // RemoveFinalizer removes a finalizer from the object using Patch. | ||
| // If the finalizer doesn't exist, this is a no-op. | ||
| func RemoveFinalizer(ctx context.Context, c client.Client, obj client.Object, finalizer string) error { |
There was a problem hiding this comment.
Same comment here: re accept multiple finalizers to avoid multiple patch calls?
| newFinalizers := make([]string, 0, len(currentFinalizers)) | ||
| for _, f := range currentFinalizers { | ||
| if f != finalizer { | ||
| newFinalizers = append(newFinalizers, f) | ||
| } | ||
| } |
There was a problem hiding this comment.
newFinalizers := slices.Delete(slices.Clone(currentFinalizers), finalizer)?
There was a problem hiding this comment.
Or if we reorganize to accept multiple finalizers, then slices.DeleteFunc could be more appropriate (e.g. by building a set of finalizers to remove and then checking set presence in the predicate function).
Thanks for explaining it, make sense. Given that I think we should update PR description to reflect that, currently we have:
Given the explanation above, we are not reducing conflict or improving performances - we are ensuring correctness, i.e. not removing the annotation. |
tmshort
force-pushed
the
apply-finalizers
branch
from
0608a72 to
cf27d6e
Compare
Refactor all controllers to use server-side-apply instead of Update() when adding or removing finalizers to improve performance, and to avoid removing non-cached fields erroneously. Create shared finalizer utilities to eliminate code duplication across controllers. This is necesary because we no longer cache the`last-applied-configuration` annotation, so when we add/remove the finalizers, we are removing that field from the metadata. This causes issues with clients when they don't see that annotation (e.g. apply the same ClusterExtension twice). - Add shared finalizer.EnsureFinalizer() and RemoveFinalizer() utilities - Update ClusterCatalog, ClusterExtension, and ClusterExtensionRevision controllers to use Patch-based finalizer management - Maintain early return behavior after adding finalizers on create - Remove unused internal/operator-controller/finalizers package - Update all unit tests to match new behavior 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> Signed-off-by: Todd Short <[email protected]>
tmshort
force-pushed
the
apply-finalizers
branch
from
cf27d6e to
bfb7a16
Compare
tmshort
changed the title
3 participants
Refactor all controllers to use server-side-apply instead of Update() when adding or removing finalizers to improve performance, and to avoid removing non-cached fields erroneously. Create shared finalizer utilities to eliminate code duplication across controllers.
This is necesary because we no longer cache the
last-applied-configurationannotation, so when we add/remove the finalizers, we are removing that field from the metadata. This causes issues with clients when they don't see that annotation (e.g. apply the same ClusterExtension twice).controllers to use Patch-based finalizer management
🤖 Generated with Claude Code
Co-Authored-By: Claude [email protected]
Description
Reviewer Checklist