🌱 Proposed way to unskip some tests #874

bentito · 2024-05-20T20:09:00Z

Description

TestClusterExtensionChannelVersionExists mostly restored here.

This is a proposed way to change the other skipped tests as well.

Items skipped by the last conditional are:

Bundle Deployment related (can likely just be removed);
Invariant or other conditions on code paths we're avoiding due to a desire to lessen the mocking.

Reviewer Checklist

API Go Documentation
Tests: Unit Tests (and E2E Tests, if appropriate)
Comprehensive Commit Messages
Links to related GitHub Issue(s)

internal/controllers/clusterextension_controller_test.go

internal/controllers/suite_test.go

internal/controllers/clusterextension_controller_test.go

internal/controllers/clusterextension_registryv1_validation_test.go

internal/controllers/suite_test.go

varshaprasad96 · 2024-05-21T18:44:36Z

@bentito: d649164 - I tried making changes to your branch, but GH doesn't let me create a PR against your repo 😅 With this change, we don't need to mock the loader itself. We use the mock unpacker to return UnpackPending and test relevant conditions which is equivalent on what the test did previously.

internal/controllers/clusterextension_controller_test.go

joelanford · 2024-05-21T19:45:06Z

internal/controllers/clusterextension_controller_test.go


-	verifyInvariants(ctx, t, reconciler.Client, clusterExtension)


Curious why we're losing this verifyInvariants piece of the test?

Looks like verifyInvariants was removed from a few of the tests. I wasn't aware of what that function does so I went exploring. After learning what it's used for I agree that it should remain because IIUC we should be setting all of our status conditions on every reconcile.

After taking a look at the rest of the PR though, adding verifyInvariants back would be my only requested change (with the assumption that we currently expect some tests to be failing and will address them in another way).

We have way more condition types and reasons now - so setting all of the conditions whenever the reconciler exits is an issue by itself. We are working on fixing this and brining verifyInvariants back for now. But we would strongly urge for us to evaluate the whole design of what conditions we want. #842 deals with some of it, but have also created an issue to track it.

verifyInvarients is restored here and everywhere now.

joelanford · 2024-05-21T19:50:29Z

internal/controllers/suite_test.go


 	"github.com/operator-framework/operator-controller/internal/controllers"
 	"github.com/operator-framework/operator-controller/pkg/scheme"
 	testutil "github.com/operator-framework/operator-controller/test/util"
 )

+// MockUnpacker is a mock of Unpacker interface


Nit: move the mocks to a separate package and/or file to make them easier to find?

Added as #895.

TestClusterExtensionChannelVersionExists mostly restored here Signed-off-by: Brett Tofel <[email protected]>

Signed-off-by: Brett Tofel <[email protected]>

In just one test, for now. Signed-off-by: Brett Tofel <[email protected]>

Unskips all in clusterextension_controller_test.go Signed-off-by: Brett Tofel <[email protected]>

Still failing tests (up|down)grade should err b/c we don't have an installedBundle to check against Signed-off-by: Brett Tofel <[email protected]>

Signed-off-by: Varsha Prasad Narsing <[email protected]>

Signed-off-by: Brett Tofel <[email protected]>

Signed-off-by: dtfranz <[email protected]>

dtfranz · 2024-05-28T22:30:51Z

test/e2e/cluster_extension_install_test.go

@@ -207,6 +207,7 @@ func TestClusterExtensionBlockInstallNonSuccessorVersion(t *testing.T) {
 		assert.Equal(ct, ocv1alpha1.ReasonSuccess, cond.Reason)
 		assert.Contains(ct, cond.Message, "resolved to")
 		assert.Equal(ct, &ocv1alpha1.BundleMetadata{Name: "prometheus-operator.1.0.0", Version: "1.0.0"}, clusterExtension.Status.ResolvedBundle)
+		assert.Equal(ct, &ocv1alpha1.BundleMetadata{Name: "prometheus-operator.1.0.0", Version: "1.0.0"}, clusterExtension.Status.InstalledBundle)


If we don't actually check that this version has been installed, we may run into a scenario where we've resolved to 1.0.0 but haven't installed it yet. Then, if the user immediately changes the ClusterExtension to 1.2.0, we're essentially installing 1.2.0 from scratch, and the non-successor limitation doesn't apply in that case. I don't personally think this is a bug.

This check is not necessary on main because when we finish resolve the BundleDeployment is immediately created with the installed version flags present, regardless of whether or not the 1.0.0 operator has finished installing yet. I think this is actually less desirable compared to the above scenario; for instance if the user quickly changed their mind about which version they wanted they wouldn't be able to install 1.2.0 even though 1.0.0 hasn't even installed yet.

varshaprasad96

/lgtm

varshaprasad96 · 2024-05-29T00:03:30Z

internal/controllers/common_controller.go

+func updateStatusUnpackPending(status *ocv1alpha1.ClusterExtensionStatus, result *source.Result, generation int64) {
+	status.InstalledBundle = nil
+	apimeta.SetStatusCondition(&status.Conditions, metav1.Condition{
+		Type:               rukpakv1alpha2.TypeUnpacked,


Same for the other two unpack status conditions below.

Suggested change

Type: rukpakv1alpha2.TypeUnpacked,

Type: ocv1alpha1.TypeUnpacked,

This is a status condition, didn't we say that another PR will deal with that stuff? (Although the above suggestion is correct.)

…s into the cluster (#846) * Helm POC rebase-a-thon * Unpack Image (#811) Builds and loads the unpacker binary into the operator-controller image so we don't need to use the rukpak image. Signed-off-by: dtfranz <[email protected]> * ✨ OPRUN-3293 Fixes several unit tests - not cache ones (#831) * Fix, rm or skip BundleDeployment related tests Signed-off-by: Brett Tofel <[email protected]> * Address lint errors Signed-off-by: Brett Tofel <[email protected]> --------- Signed-off-by: Brett Tofel <[email protected]> * Fix e2e (#843) Signed-off-by: Varsha Prasad Narsing <[email protected]> * Make goreleaser build unpack Signed-off-by: Brett Tofel <[email protected]> * Rm commented code Signed-off-by: Brett Tofel <[email protected]> * Rebase required changes Signed-off-by: Brett Tofel <[email protected]> * Cleanups (#854) Signed-off-by: Varsha Prasad Narsing <[email protected]> * Expand comments in Dockerfile Signed-off-by: Brett Tofel <[email protected]> * Address Reviews Signed-off-by: Varsha Prasad Narsing <[email protected]> * Back out packagerrors It's agreed that this makes debugging and reasoning about log messages and runtime errors much harder. Signed-off-by: Brett Tofel <[email protected]> * Helm POC e2e fixes (#870) Fixes the e2e tests in helm-poc branch and restores feature-gate switching. Signed-off-by: dtfranz <[email protected]> * Fix defer location (#883) Fixes the defer location to make sure that all resources are cleaned up properly. Signed-off-by: dtfranz <[email protected]> * Remove BundleDeployments gathering from e2e's gatherArtifacts (#886) When tests fail, gatherArtifacts is run to collect cluster state. This PR removes gathering of BundleDeployments since we don't use them anymore and it's generating additional misleading error messages. Signed-off-by: dtfranz <[email protected]> * Address reviews (#887) Signed-off-by: Varsha Prasad Narsing <[email protected]> * 🌱 Proposed way to unskip some tests (#874) * Proposed way to unskip some tests TestClusterExtensionChannelVersionExists mostly restored here Signed-off-by: Brett Tofel <[email protected]> * Refactors global variable names Signed-off-by: Brett Tofel <[email protected]> * Removes BundleDeployment related checking In just one test, for now. Signed-off-by: Brett Tofel <[email protected]> * fix unit test - TestClusterExtensionChannelVersionExists * Unskip all - failing tests (up|down)grade should err Unskips all in clusterextension_controller_test.go Signed-off-by: Brett Tofel <[email protected]> * Unskip all clusterextension_registryv1_validation_test.go Still failing tests (up|down)grade should err b/c we don't have an installedBundle to check against Signed-off-by: Brett Tofel <[email protected]> * debugging Signed-off-by: Varsha Prasad Narsing <[email protected]> * Fixes and additions pending & unpack path Signed-off-by: Brett Tofel <[email protected]> * Fix linter, remove debug logs and extraneous file Signed-off-by: dtfranz <[email protected]> * Fix e2e failure due to race condition Signed-off-by: dtfranz <[email protected]> --------- Signed-off-by: Brett Tofel <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: dtfranz <[email protected]> Co-authored-by: Varsha Prasad Narsing <[email protected]> Co-authored-by: dtfranz <[email protected]> * Revert goreleaser to use buildx Signed-off-by: Brett Tofel <[email protected]> * Revert all of goreleaser to use buildx Signed-off-by: Brett Tofel <[email protected]> * Revert errorf->printf & rm some logging in tests Signed-off-by: Brett Tofel <[email protected]> --------- Signed-off-by: dtfranz <[email protected]> Signed-off-by: Brett Tofel <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> Co-authored-by: Todd Short <[email protected]> Co-authored-by: Daniel Franz <[email protected]> Co-authored-by: Varsha <[email protected]>

bentito requested a review from a team as a code owner May 20, 2024 20:09

bentito changed the title ~~Proposed way to unskip some tests~~ 🌱 Proposed way to unskip some tests May 20, 2024

bentito mentioned this pull request May 20, 2024

✨ Modify ClusterExtension to use Helm under the hood to apply contents into the cluster #846

Merged

4 tasks

bentito commented May 21, 2024

View reviewed changes