OCPBUGS-33894: Avoid dumping Go struct in potentially user-facing reason annotation

[djoshy]

- What I did

• Use nodeName instead of node in error message for setting node annotations. On a degraded node, such an error would result in the entire node object to be dumped in the node's annotation which was not user friendly.
• I also changed retry mechanism to use a backoff of retry.DefaultRetry, instead of retry.DefaultBackoff which is recommended for conflicts.

- How to verify it

• Reading through the bug, this seems to surface during upgrades/installs when the APIServer goes unavailable/nodes have high activity. It may be useful to simulate such conditions and see if the errors resurface.
• I suppose one could also explore ways to make the API update calls from the MCO to fail to simulate a retry error scenario.

[openshift-ci-robot]

@djoshy: This pull request references Jira Issue OCPBUGS-33894, which is invalid:

• expected the bug to target the "4.19.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

- What I did

• Use nodeName instead of node in error message for setting node annotations. On a degraded node, such an error would result in be stored in the node's annotation which was not user friendly.
• I also changed retry mechanism to use a backoff of retry.DefaultRetry, instead of retry.DefaultBackoff which is recommended for conflicts.

- How to verify it

• Reading through the bug, this seems to surface during upgrades/installs when the APIServer goes unavailable/nodes have high activity. It may be useful to simulate such conditions and see if the errors resurface.
• I suppose one could also explore ways to make the API update calls from the MCO to fail to simulate a retry error scenario.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[djoshy]

/jira refresh

This may potentially alleviate https://issues.redhat.com/browse/OCPBUGS-33891, but I am unsure at this time

[openshift-ci-robot]

@djoshy: This pull request references Jira Issue OCPBUGS-33894, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.19.0) matches configured target version for branch (4.19.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

In response to this:

/jira refresh

This may potentially alleviate https://issues.redhat.com/browse/OCPBUGS-33891, but I am unsure at this time

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

@djoshy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-azure-ovn-upgrade-out-of-change	fb86e86	link	false	/test e2e-azure-ovn-upgrade-out-of-change
ci/prow/okd-scos-e2e-aws-ovn	fb86e86	link	false	/test okd-scos-e2e-aws-ovn
ci/prow/bootstrap-unit	fb86e86	link	false	/test bootstrap-unit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-ci-robot]

@djoshy: This pull request references Jira Issue OCPBUGS-33894, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.19.0) matches configured target version for branch (4.19.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

In response to this:

- What I did

• Use nodeName instead of node in error message for setting node annotations. On a degraded node, such an error would result in the entire node object to be dumped in the node's annotation which was not user friendly.
• I also changed retry mechanism to use a backoff of retry.DefaultRetry, instead of retry.DefaultBackoff which is recommended for conflicts.

- How to verify it

• Reading through the bug, this seems to surface during upgrades/installs when the APIServer goes unavailable/nodes have high activity. It may be useful to simulate such conditions and see if the errors resurface.
• I suppose one could also explore ways to make the API update calls from the MCO to fail to simulate a retry error scenario.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[isabella-janssen]

/hold
Holding for qe if it is needed @djoshy

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djoshy, isabella-janssen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [djoshy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[djoshy]

Chatted with @sergiordlr and we think this won't need pre merge QE testing

/unhold

/label acknowledge-critical-fixes-only

This is a minor change to error text and a retry mechanism; shouldn't affect payloads.

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 0afa42b and 2 for PR HEAD fb86e86 in total

[openshift-ci-robot]

@djoshy: Jira Issue OCPBUGS-33894: All pull requests linked via external trackers have merged:

• openshift/machine-config-operator#4814

Jira Issue OCPBUGS-33894 has been moved to the MODIFIED state.

In response to this:

- What I did

• Use nodeName instead of node in error message for setting node annotations. On a degraded node, such an error would result in the entire node object to be dumped in the node's annotation which was not user friendly.
• I also changed retry mechanism to use a backoff of retry.DefaultRetry, instead of retry.DefaultBackoff which is recommended for conflicts.

- How to verify it

• Reading through the bug, this seems to surface during upgrades/installs when the APIServer goes unavailable/nodes have high activity. It may be useful to simulate such conditions and see if the errors resurface.
• I suppose one could also explore ways to make the API update calls from the MCO to fail to simulate a retry error scenario.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: ose-machine-config-operator
This PR has been included in build ose-machine-config-operator-container-v4.19.0-202502150007.p0.gc26bdb0.assembly.stream.el9.
All builds following this will include this PR.