Mle-28303 Dynamic Host Scaling Feature Implementation

[pengzhouml]

This pull request introduces significant improvements to the E2E testing pipeline and Makefile, with a focus on supporting new E2E test scopes, better handling of local images for Minikube, and stricter validation for dynamic group configurations. It also updates the default version to 1.3.0 and enhances developer experience with more flexible and robust Makefile targets.

E2E Pipeline and Testing Enhancements:

• Added support for selecting E2E test scopes (cluster, dynamic-host, volume-resize) via the E2E_SCOPE parameter in the Jenkins pipeline, including validation and error handling for unsupported combinations (e.g., restricting non-cluster scopes when running on EKS). Istio and Helm namespace-scoped tests are now only run for the cluster scope. (Jenkinsfile) [1] [2] [3] [4] [5] [6] [7]

• Introduced new Makefile targets for focused E2E tests:

  • e2e-test-dynamic-host and e2e-test-dynamic-host-local for dynamic-host lifecycle tests, with logic to build/load local images for Minikube contexts.
  • e2e-test-volume-resize-local for volume-resize tests with local image and Minikube storage class setup.
  • Improved all E2E targets to load images into Minikube when appropriate, reducing remote pull failures and making local development easier. (Makefile)

• Increased E2E test timeouts to 60 minutes for most targets and made timeouts configurable via E2E_TEST_TIMEOUT. (Makefile) [1] [2] [3] [4] [5]

Image and Build Improvements:

• Updated default operator image version to 1.3.0 and added a LOCAL_E2E_IMG variable for local E2E testing. (Makefile) [1] [2]
• Changed Docker build command to use --load for compatibility with local image loading. (Makefile)

Kustomize and Tooling:

• Enhanced the kustomize Makefile target to respect externally set paths and provide better error messaging if the binary is not executable. (Makefile)

API Validation and CRD Changes:

• Added a DynamicGroupConfig struct with strict ISO-8601 duration validation for the tokenDuration field. (api/v1/common_types.go)
• Added CRD-level validation to ensure dynamic hosts require image tag latest or MarkLogic major version 12+. Also, added a max length validation for the image field. (api/v1/marklogiccluster_types.go) [1] [2]

Developer Experience:

• Added a kill-envtest Makefile target to clean up stale test processes. (Makefile)

These changes collectively improve the reliability, flexibility, and developer usability of the E2E testing and build workflow, while enforcing important validation rules for dynamic group configurations and image selection.

[Copilot]

Pull request overview

Adds dynamic MarkLogic host pool support to the operator, including API fields, CRD/status schema, controller reconciliation, management API client logic, dynamic pod startup behavior, and a functional spec.

Changes:

• Adds dynamic group API fields/status and generated CRD/deepcopy updates.
• Adds dynamic host lifecycle reconciliation for group configuration, token join, scale-down cleanup, restart recovery, and finalizers.
• Updates StatefulSet/Service generation, startup scripts, controller watches, and tests/spec docs for dynamic groups.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
pkg/mlmanage/client.go	Adds MarkLogic Management API client for dynamic host operations.
pkg/k8sutil/statefulset.go	Adds dynamic labels, env var, readiness probe, and scale-down delay behavior.
pkg/k8sutil/service.go	Selects dynamic pods via component labels.
pkg/k8sutil/secret.go	Creates shared manage-admin credentials for dynamic groups.
pkg/k8sutil/scripts/cluster-init-wrapper.sh	Skips static init/join in dynamic mode.
pkg/k8sutil/scripts/cluster-config.sh	Adds dynamic-mode guard.
pkg/k8sutil/marklogicServer.go	Propagates dynamic config/defaults to child MarklogicGroup resources.
pkg/k8sutil/handler.go	Invokes dynamic reconciliation for dynamic groups.
pkg/k8sutil/dynamic_reconcile.go	Implements dynamic host lifecycle reconciliation.
pkg/k8sutil/common.go	Adds dynamic/static component label helpers.
internal/controller/marklogicgroup_controller.go	Adds pod update handling and pod ownership watch.
internal/controller/marklogiccluster_controller_test.go	Adds dynamic group propagation tests.
docs/spec/Dynamic Host.md	Adds functional specification for dynamic host support.
config/crd/bases/marklogic.progress.com_marklogicgroups.yaml	Adds dynamic spec/status schema for MarklogicGroup.
config/crd/bases/marklogic.progress.com_marklogicclusters.yaml	Adds dynamic fields/validation for MarklogicCluster groups.
api/v1/zz_generated.deepcopy.go	Adds deepcopy support for dynamic structs/fields.
api/v1/marklogicgroup_types.go	Adds dynamic fields and status structs to MarklogicGroup API.
api/v1/marklogiccluster_types.go	Adds dynamic fields to cluster group entries.
api/v1/common_types.go	Adds dynamic group configuration type.

Files not reviewed (1)

• api/v1/zz_generated.deepcopy.go: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[vitalykorolev]

@pengzhouml please respond to Copilot's comments and mark conversations as Resolved.

[Copilot]

Pull request overview

Copilot reviewed 20 out of 22 changed files in this pull request and generated 5 comments.

Files not reviewed (1)

• api/v1/zz_generated.deepcopy.go: Language not supported

Comments suppressed due to low confidence (1)

pkg/k8sutil/statefulset.go:115

• patchDiff.String() is logged before checking err from patch.DefaultPatchMaker.Calculate. If Calculate fails, patchDiff may be unusable and calling methods on it can panic or log misleading output. Please check/return on err before using patchDiff (including after the second Calculate call in the dynamic scale-down delay branch).

	patchDiff, err := patch.DefaultPatchMaker.Calculate(currentSts, statefulSetDef,
		patch.IgnoreStatusFields(),
		patch.IgnoreVolumeClaimTemplateTypeMetaAndStatus(),
		patch.IgnoreField("kind"))
	if shouldDelayDynamicEmptyDirScaleDown(cr, currentSts) {
		statefulSetDef.Spec.Replicas = currentSts.Spec.Replicas
		patchDiff, err = patch.DefaultPatchMaker.Calculate(currentSts, statefulSetDef,
			patch.IgnoreStatusFields(),
			patch.IgnoreVolumeClaimTemplateTypeMetaAndStatus(),
			patch.IgnoreField("kind"))
	}
	logger.Info("Patch Diff:", "Diff", patchDiff.String())
	logger.Info("statefulSetDef Spec:", "Spec", statefulSetDef.Spec.Replicas)
	if err != nil {
		logger.Error(err, "Error calculating patch")
		return result.Error(err).Output()

[rwinieski]

Pending Jenkins to be green.

Otherwise the code looks good to me.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.