✨ Modify ClusterExtension to use Helm under the hood to apply contents into the cluster (#846)

* Helm POC rebase-a-thon

* Unpack Image (#811)

Builds and loads the unpacker binary into the operator-controller image so we don't need to use the rukpak image.

Signed-off-by: dtfranz <[email protected]>

* ✨ OPRUN-3293 Fixes several unit tests - not cache ones  (#831)

* Fix, rm or skip BundleDeployment related tests

Signed-off-by: Brett Tofel <[email protected]>

* Address lint errors

Signed-off-by: Brett Tofel <[email protected]>

---------

Signed-off-by: Brett Tofel <[email protected]>

* Fix e2e (#843)

Signed-off-by: Varsha Prasad Narsing <[email protected]>

* Make goreleaser build unpack

Signed-off-by: Brett Tofel <[email protected]>

* Rm commented code

Signed-off-by: Brett Tofel <[email protected]>

* Rebase required changes

Signed-off-by: Brett Tofel <[email protected]>

* Cleanups (#854)

Signed-off-by: Varsha Prasad Narsing <[email protected]>

* Expand comments in Dockerfile

Signed-off-by: Brett Tofel <[email protected]>

* Address Reviews

Signed-off-by: Varsha Prasad Narsing <[email protected]>

* Back out packagerrors

It's agreed that this makes debugging and reasoning about log messages and runtime errors much harder.

Signed-off-by: Brett Tofel <[email protected]>

* Helm POC e2e fixes (#870)

Fixes the e2e tests in helm-poc branch and restores feature-gate switching.

Signed-off-by: dtfranz <[email protected]>

* Fix defer location (#883)

Fixes the defer location to make sure that all resources are cleaned up properly.

Signed-off-by: dtfranz <[email protected]>

* Remove BundleDeployments gathering from e2e's gatherArtifacts (#886)

When tests fail, gatherArtifacts is run to collect cluster state. This PR removes gathering of BundleDeployments since we don't use them anymore and it's generating additional misleading error messages.

Signed-off-by: dtfranz <[email protected]>

* Address reviews (#887)

Signed-off-by: Varsha Prasad Narsing <[email protected]>

* 🌱 Proposed way to unskip some tests (#874)

* Proposed way to unskip some tests

TestClusterExtensionChannelVersionExists mostly restored here

Signed-off-by: Brett Tofel <[email protected]>

* Refactors global variable names

Signed-off-by: Brett Tofel <[email protected]>

* Removes BundleDeployment related checking

In just one test, for now.

Signed-off-by: Brett Tofel <[email protected]>

* fix unit test - TestClusterExtensionChannelVersionExists

* Unskip all - failing tests (up|down)grade should err

Unskips all in clusterextension_controller_test.go

Signed-off-by: Brett Tofel <[email protected]>

* Unskip all clusterextension_registryv1_validation_test.go

Still failing tests (up|down)grade should err b/c we don't have an installedBundle to check against

Signed-off-by: Brett Tofel <[email protected]>

* debugging

Signed-off-by: Varsha Prasad Narsing <[email protected]>

* Fixes and additions pending & unpack path

Signed-off-by: Brett Tofel <[email protected]>

* Fix linter, remove debug logs and extraneous file

Signed-off-by: dtfranz <[email protected]>

* Fix e2e failure due to race condition

Signed-off-by: dtfranz <[email protected]>

---------

Signed-off-by: Brett Tofel <[email protected]>
Signed-off-by: Varsha Prasad Narsing <[email protected]>
Signed-off-by: dtfranz <[email protected]>
Co-authored-by: Varsha Prasad Narsing <[email protected]>
Co-authored-by: dtfranz <[email protected]>

* Revert goreleaser to use buildx

Signed-off-by: Brett Tofel <[email protected]>

* Revert all of goreleaser to use buildx

Signed-off-by: Brett Tofel <[email protected]>

* Revert errorf->printf & rm some logging in tests

Signed-off-by: Brett Tofel <[email protected]>

---------

Signed-off-by: dtfranz <[email protected]>
Signed-off-by: Brett Tofel <[email protected]>
Signed-off-by: Varsha Prasad Narsing <[email protected]>
Co-authored-by: Todd Short <[email protected]>
Co-authored-by: Daniel Franz <[email protected]>
Co-authored-by: Varsha <[email protected]>

Author: 4 people

.github/workflows/tilt.yaml

@@ -22,10 +22,6 @@ jobs:
     - uses: actions/checkout@v4
       with:
         path: operator-controller
-    - uses: actions/checkout@v4
-      with:
-        repository: operator-framework/rukpak
-        path: rukpak
     - uses: actions/checkout@v4
       with:
         repository: operator-framework/catalogd

.goreleaser.yml

@@ -16,6 +16,19 @@ builds:
     - arm64
     - ppc64le
     - s390x
+  - id: unpack
+    main: ./cmd/unpack/main.go  # Update this path to the actual location of your unpack source
+    binary: bin/unpack
+    asmflags: "{{ .Env.GO_BUILD_ASMFLAGS }}"
+    gcflags: "{{ .Env.GO_BUILD_GCFLAGS }}"
+    ldflags: "{{ .Env.GO_BUILD_LDFLAGS }}"
+    goos:
+      - linux
+    goarch:
+      - amd64
+      - arm64
+      - ppc64le
+      - s390x
 dockers:
 - image_templates:
   - "{{ .Env.IMAGE_REPO }}:{{ .Env.IMAGE_TAG }}-amd64"

CONTRIBUTING.md

@@ -19,9 +19,8 @@ The user experience captured in the OLM V1 PRD introduces many requirements that
 - [The Operator-Controller project](https://github.com/operator-framework/operator-controller/), which is the top level component allowing users to specify operators they'd like to install.
 - [The Catalogd project](https://github.com/operator-framework/catalogd/), which hosts operator content and helps users discover installable content.
 - [The Deppy project](https://github.com/operator-framework/deppy/), which enables the operator-controller to identify valid installs and upgrades from the list of installable content provided via the Catalogd project.
-- [The RukPak project](https://github.com/operator-framework/rukpak/), which facilitates the installation of operators.
 
-Each of the projects listed above have their own governance, release milestones, and release cadence. However, from a technical perspective, the "OLM V1 experience" matches the experienced offered by the operator-controller project, the top level component which introduces dependencies on RukPak, Deppy, and the Catalogd projects.
+Each of the projects listed above have their own governance, release milestones, and release cadence. However, from a technical perspective, the "OLM V1 experience" matches the experienced offered by the operator-controller project, the top level component which introduces dependencies on Deppy and the Catalogd projects.
 
 ## How do we collaborate
 
@@ -62,7 +61,7 @@ Ongoing or previous Operator-Controller milestones can always be found in the [m
 
 ### How are Subproject Issues Tracked?
 
-As discussed earlier, the operator-controller adheres to a microservice architecture, where multiple projects contribute to the overall experience. As such, when designing an operator-controller milestone, the community may need to file an issue against RukPak, Deppy, or Catalogd. Unfortunately, the operator-controller milestone cannot contain issues from one of its subprojects. As such, we've introduced the concept of a "Dependency Issue", described below:
+As discussed earlier, the operator-controller adheres to a microservice architecture, where multiple projects contribute to the overall experience. As such, when designing an operator-controller milestone, the community may need to file an issue against Deppy or Catalogd. Unfortunately, the operator-controller milestone cannot contain issues from one of its subprojects. As such, we've introduced the concept of a "Dependency Issue", described below:
 
 > Dependency Issues: An issue tracked in a milestone that "points" to an issue in another project with a URL.
 
@@ -72,7 +71,6 @@ Unsure where to submit an issue?
 - [The Operator-Controller project](https://github.com/operator-framework/operator-controller/), which is the top level component allowing users to specify operators they'd like to install.
 - [The Catalogd project](https://github.com/operator-framework/catalogd/), which hosts operator content and helps users discover installable content.
 - [The Deppy project](https://github.com/operator-framework/deppy/), which enables the operator-controller to identify valid installs and upgrades from the list of installable content provided via the Catalogd project.
-- [The RukPak project](https://github.com/operator-framework/rukpak/), which facilitates the installation of operators.
 
 Don't worry if you accidentally submit an issue against the wrong project, if we notice that an issue would fit better with a separate project we'll move it to the correct repository and mention it in the #olm-dev slack channel.
 

Dockerfile

@@ -1,14 +1,21 @@
 # Note: This dockerfile does not build the binaries 
 # required and is intended to be built only with the
 # 'make build' or 'make release' targets.
+# Stage 1:
+FROM gcr.io/distroless/static:debug-nonroot AS builder
+
+# Stage 2: 
 FROM gcr.io/distroless/static:nonroot
 
+# Grab the cp binary so we can cp the unpack
+# binary to a shared volume in the bundle image (rukpak library needs it)
+COPY --from=builder /busybox/cp /cp
+
 WORKDIR /
 
 COPY manager manager
+COPY unpack unpack
 
 EXPOSE 8080
 
 USER 65532:65532
-
-ENTRYPOINT ["/manager"]

Makefile

@@ -22,7 +22,6 @@ IMG := $(IMAGE_REPO):$(IMAGE_TAG)
 # Define dependency versions (use go.mod if we also use Go code from dependency)
 export CERT_MGR_VERSION := v1.9.0
 export CATALOGD_VERSION := $(shell go list -mod=mod -m -f "{{.Version}}" github.com/operator-framework/catalogd)
-export RUKPAK_VERSION := $(shell go list -mod=mod -m -f "{{.Version}}" github.com/operator-framework/rukpak)
 export WAIT_TIMEOUT := 60s
 
 # By default setup-envtest will write to $XDG_DATA_HOME, or $HOME/.local/share if that is not defined.
@@ -165,10 +164,16 @@ e2e-coverage:
 kind-load: $(KIND) #EXHELP Loads the currently constructed image onto the cluster.
 	$(CONTAINER_RUNTIME) save $(IMG) | $(KIND) load image-archive /dev/stdin --name $(KIND_CLUSTER_NAME)
 
-kind-deploy: export MANIFEST := ./operator-controller.yaml
+.PHONY: kind-deploy
+kind-deploy: export MANIFEST="./operator-controller.yaml"
 kind-deploy: manifests $(KUSTOMIZE) #EXHELP Install controller and dependencies onto the kind cluster.
 	$(KUSTOMIZE) build $(KUSTOMIZE_BUILD_DIR) > operator-controller.yaml
-	envsubst '$$CATALOGD_VERSION,$$CERT_MGR_VERSION,$$RUKPAK_VERSION,$$MANIFEST' < scripts/install.tpl.sh | bash -s
+	envsubst '$$CATALOGD_VERSION,$$CERT_MGR_VERSION,$$MANIFEST' < scripts/install.tpl.sh | bash -s
+
+.PHONY: kind-redeploy
+kind-redeploy: generate docker-build kind-load kind-deploy #EXHELP Redeploy newly built executables
+	kubectl delete pod -l control-plane=controller-manager -n $(OPERATOR_CONTROLLER_NAMESPACE)
+	envsubst '$$CATALOGD_VERSION,$$CERT_MGR_VERSION,$$MANIFEST' < scripts/install.tpl.sh | bash -s
 
 .PHONY: kind-cluster
 kind-cluster: $(KIND) #EXHELP Standup a kind cluster.
@@ -215,22 +220,25 @@ export GO_BUILD_FLAGS :=
 export GO_BUILD_LDFLAGS := -s -w \
     -X '$(VERSION_PATH).version=$(VERSION)' \
 
-BUILDCMD = go build $(GO_BUILD_FLAGS) -ldflags '$(GO_BUILD_LDFLAGS)' -gcflags '$(GO_BUILD_GCFLAGS)' -asmflags '$(GO_BUILD_ASMFLAGS)' -o $(BUILDBIN)/manager ./cmd/manager
+BINARIES=manager unpack
+
+$(BINARIES):
+	go build $(GO_BUILD_FLAGS) -tags '$(GO_BUILD_TAGS)' -ldflags '$(GO_BUILD_LDFLAGS)' -gcflags '$(GO_BUILD_GCFLAGS)' -asmflags '$(GO_BUILD_ASMFLAGS)' -o $(BUILDBIN)/$@ ./cmd/$@
 
 .PHONY: build-deps
 build-deps: manifests generate fmt vet
 
 .PHONY: build go-build-local
 build: build-deps go-build-local #HELP Build manager binary for current GOOS and GOARCH. Default target.
 go-build-local: BUILDBIN := bin
-go-build-local:
-	$(BUILDCMD)
+go-build-local: $(BINARIES)
 
 .PHONY: build-linux go-build-linux
 build-linux: build-deps go-build-linux #EXHELP Build manager binary for GOOS=linux and local GOARCH.
 go-build-linux: BUILDBIN := bin/linux
-go-build-linux:
-	GOOS=linux $(BUILDCMD)
+go-build-linux: export GOOS=linux
+go-build-linux: export GOARCH=amd64
+go-build-linux: $(BINARIES)
 
 .PHONY: run
 run: docker-build kind-cluster kind-load kind-deploy #HELP Build the operator-controller then deploy it into a new kind cluster.
@@ -258,7 +266,7 @@ release: $(GORELEASER) #EXHELP Runs goreleaser for the operator-controller. By d
 quickstart: export MANIFEST := https://github.com/operator-framework/operator-controller/releases/download/$(VERSION)/operator-controller.yaml
 quickstart: $(KUSTOMIZE) manifests #EXHELP Generate the installation release manifests and scripts.
 	$(KUSTOMIZE) build $(KUSTOMIZE_BUILD_DIR) | sed "s/:devel/:$(VERSION)/g" > operator-controller.yaml
-	envsubst '$$CATALOGD_VERSION,$$CERT_MGR_VERSION,$$RUKPAK_VERSION,$$MANIFEST' < scripts/install.tpl.sh > install.sh
+	envsubst '$$CATALOGD_VERSION,$$CERT_MGR_VERSION,$$MANIFEST' < scripts/install.tpl.sh > install.sh
 
 ##@ Docs
 

README.md

@@ -17,7 +17,6 @@ OLM v1 is the follow-up to OLM v0, located [here](https://github.com/operator-fr
 OLM v1 consists of four different components:
 * operator-controller (this repository)
 * [deppy](https://github.com/operator-framework/deppy)
-* [rukpak](https://github.com/operator-framework/rukpak)
 * [catalogd](https://github.com/operator-framework/catalogd)
 
 For a more complete overview of OLM v1 and how it differs from OLM v0, see our [overview](./docs/olmv1_overview.md).
@@ -76,7 +75,7 @@ Install the CRDs and the operator-controller into a new [KIND cluster](https://k
 make run
 ```
 This will build a local container image of the operator-controller, create a new KIND cluster and then deploy onto that cluster.
-This will also deploy the catalogd, rukpak and cert-manager dependencies.
+This will also deploy the catalogd and cert-manager dependencies.
 
 ### Modifying the API definitions
 If you are editing the API definitions, generate the manifests such as CRs or CRDs using:

Tiltfile

@@ -5,7 +5,7 @@ load('../tilt-support/Tiltfile', 'deploy_repo')
 
 config.define_string_list('repos', args=True)
 cfg = config.parse()
-repos = cfg.get('repos', ['operator-controller', 'rukpak', 'catalogd'])
+repos = cfg.get('repos', ['operator-controller', 'catalogd'])
 
 repo = {
     'image': 'quay.io/operator-framework/operator-controller',

api/v1alpha1/clusterextension_types.go

@@ -22,6 +22,11 @@ import (
 	"github.com/operator-framework/operator-controller/internal/conditionsets"
 )
 
+var (
+	ClusterExtensionGVK  = SchemeBuilder.GroupVersion.WithKind("ClusterExtension")
+	ClusterExtensionKind = ClusterExtensionGVK.Kind
+)
+
 type UpgradeConstraintPolicy string
 
 const (
@@ -77,47 +82,58 @@ type ClusterExtensionSpec struct {
 
 const (
 	// TODO(user): add more Types, here and into init()
-	TypeInstalled = "Installed"
-	TypeResolved  = "Resolved"
+	TypeInstalled      = "Installed"
+	TypeResolved       = "Resolved"
+	TypeHasValidBundle = "HasValidBundle"
+
 	// TypeDeprecated is a rollup condition that is present when
 	// any of the deprecated conditions are present.
 	TypeDeprecated        = "Deprecated"
 	TypePackageDeprecated = "PackageDeprecated"
 	TypeChannelDeprecated = "ChannelDeprecated"
 	TypeBundleDeprecated  = "BundleDeprecated"
+	TypeUnpacked          = "Unpacked"
+
+	ReasonErrorGettingClient = "ErrorGettingClient"
+	ReasonBundleLoadFailed   = "BundleLoadFailed"
 
-	ReasonBundleLookupFailed        = "BundleLookupFailed"
 	ReasonInstallationFailed        = "InstallationFailed"
 	ReasonInstallationStatusUnknown = "InstallationStatusUnknown"
 	ReasonInstallationSucceeded     = "InstallationSucceeded"
-	ReasonInvalidSpec               = "InvalidSpec"
 	ReasonResolutionFailed          = "ResolutionFailed"
-	ReasonResolutionUnknown         = "ResolutionUnknown"
-	ReasonSuccess                   = "Success"
-	ReasonDeprecated                = "Deprecated"
+
+	ReasonSuccess               = "Success"
+	ReasonDeprecated            = "Deprecated"
+	ReasonUpgradeFailed         = "UpgradeFailed"
+	ReasonHasValidBundleUnknown = "HasValidBundleUnknown"
+	ReasonUnpackPending         = "UnpackPending"
 )
 
 func init() {
 	// TODO(user): add Types from above
 	conditionsets.ConditionTypes = append(conditionsets.ConditionTypes,
 		TypeInstalled,
 		TypeResolved,
+		TypeHasValidBundle,
 		TypeDeprecated,
 		TypePackageDeprecated,
 		TypeChannelDeprecated,
 		TypeBundleDeprecated,
+		TypeUnpacked,
 	)
 	// TODO(user): add Reasons from above
 	conditionsets.ConditionReasons = append(conditionsets.ConditionReasons,
 		ReasonInstallationSucceeded,
 		ReasonResolutionFailed,
-		ReasonResolutionUnknown,
-		ReasonBundleLookupFailed,
 		ReasonInstallationFailed,
-		ReasonInstallationStatusUnknown,
-		ReasonInvalidSpec,
 		ReasonSuccess,
 		ReasonDeprecated,
+		ReasonUpgradeFailed,
+		ReasonBundleLoadFailed,
+		ReasonErrorGettingClient,
+		ReasonInstallationStatusUnknown,
+		ReasonHasValidBundleUnknown,
+		ReasonUnpackPending,
 	)
 }
 
@@ -132,7 +148,6 @@ type ClusterExtensionStatus struct {
 	InstalledBundle *BundleMetadata `json:"installedBundle,omitempty"`
 	// +optional
 	ResolvedBundle *BundleMetadata `json:"resolvedBundle,omitempty"`
-
 	// +patchMergeKey=type
 	// +patchStrategy=merge
 	// +listType=map