Skip to content

Commit 0249ba1

Browse files
stuggistuggi
committed
Allow customize http vhost config using HttpdCustomization.CustomConfigSecret
This change allows to customize the httpd vhost config using this parameter to specify a secret that contains service config data. The content of each provided snippet gets rendered as a go template and placed into /etc/httpd/conf/httpd_custom_<endpoint>_<key> . At the end of the vhost config in the default httpd template these custom configs get included using `Include conf/httpd_custom_<endpoint>_*`. For information on how sections in httpd configuration get merged, check section "How the sections are merged" in https://httpd.apache.org/docs/current/sections.html#merging All possible parameters which can be use in a template can be looked up in the <service>-config-data secret of the service like: $ oc get secret -n openstack glance-config-data -o json | jq -r .data.TemplateParameters | base64 -d or in the running pod of the service in the file: $ cat /var/lib/config-data/default/TemplateParameters The content is a versioned dump of the parameters of the service operator, like: ~~~ DatabaseConnection: mysql+pymysql://user:[email protected]/keystone?read_default_file=/etc/my.cnf KeystoneEndpointInternal: https://keystone-internal.openstack.svc:5000 KeystoneEndpointPublic: https://keystone-public-openstack.apps-crc.testing TransportURL: rabbit://user:[email protected]:5671/?ssl=1 VHosts: internal: Override: false SSLCertificateFile: /etc/pki/tls/certs/internal.crt SSLCertificateKeyFile: /etc/pki/tls/private/internal.key ServerName: glance-internal.openstack.svc TLS: true public: Override: false SSLCertificateFile: /etc/pki/tls/certs/public.crt SSLCertificateKeyFile: /etc/pki/tls/private/public.key ServerName: glance-public.openstack.svc TLS: true ... ~~~ Depends-On: openstack-k8s-operators/lib-common#591 Depends-On: openstack-k8s-operators/lib-common#593 Jira: https://issues.redhat.com/browse/OSPRH-13100 Signed-off-by: Martin Schuppert <[email protected]>
1 parent 72c189b commit 0249ba1

19 files changed

+354
-21
lines changed

‎api/bases/glance.openstack.org_glanceapis.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -546,6 +546,11 @@ spec:
546546
- extraVol
547547
type: object
548548
type: array
549+
httpdCustomization:
550+
properties:
551+
customConfigSecret:
552+
type: string
553+
type: object
549554
imageCache:
550555
properties:
551556
cleanerScheduler:

‎api/bases/glance.openstack.org_glances.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -559,6 +559,11 @@ spec:
559559
items:
560560
type: string
561561
type: array
562+
httpdCustomization:
563+
properties:
564+
customConfigSecret:
565+
type: string
566+
type: object
562567
imageCache:
563568
properties:
564569
cleanerScheduler:
@@ -708,6 +713,11 @@ spec:
708713
type: object
709714
default: {}
710715
type: object
716+
httpdCustomization:
717+
properties:
718+
customConfigSecret:
719+
type: string
720+
type: object
711721
imageCache:
712722
properties:
713723
cleanerScheduler:

‎api/go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ go 1.21
44

55
require (
66
github.com/google/go-cmp v0.6.0
7-
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241216113837-d172b3ac0f4e
7+
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20250116145727-01a8948d5dd7
88
github.com/openstack-k8s-operators/lib-common/modules/storage v0.5.1-0.20241216113837-d172b3ac0f4e
99
k8s.io/api v0.29.12
1010
k8s.io/apimachinery v0.29.12

‎api/go.sum

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,8 @@ github.com/onsi/gomega v1.34.1 h1:EUMJIKUjM8sKjYbtxQI9A4z2o+rruxnzNvpknOXie6k=
7575
github.com/onsi/gomega v1.34.1/go.mod h1:kU1QgUvBDLXBJq618Xvm2LUX6rSAfRaFRTcdOeDLwwY=
7676
github.com/openshift/api v0.0.0-20240830023148-b7d0481c9094 h1:J1wuGhVxpsHykZBa6Beb1gQ96Ptej9AE/BvwCBiRj1E=
7777
github.com/openshift/api v0.0.0-20240830023148-b7d0481c9094/go.mod h1:CxgbWAlvu2iQB0UmKTtRu1YfepRg1/vJ64n2DlIEVz4=
78-
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241216113837-d172b3ac0f4e h1:hf4kVQBkyG79WcHBxdQ25QrDBbGFdarebS1Tc0Xclq4=
79-
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241216113837-d172b3ac0f4e/go.mod h1:YpNTuJhDWhbXM50O3qBkhO7M+OOyRmWkNVmJ4y3cyFs=
78+
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20250116145727-01a8948d5dd7 h1:vXHpH93PjbAgg5ZN6n5WmxkybVQOs0nhXvVw62o7aZs=
79+
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20250116145727-01a8948d5dd7/go.mod h1:YpNTuJhDWhbXM50O3qBkhO7M+OOyRmWkNVmJ4y3cyFs=
8080
github.com/openstack-k8s-operators/lib-common/modules/storage v0.5.1-0.20241216113837-d172b3ac0f4e h1:Qz0JFEoRDUyjEWorNY3LggwxTsmpMtQkcpmZDQulGHQ=
8181
github.com/openstack-k8s-operators/lib-common/modules/storage v0.5.1-0.20241216113837-d172b3ac0f4e/go.mod h1:tfgBeLRqmlH/NQkLPe7396rj+t0whv2wPuMb8Ttvh8w=
8282
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=

‎api/v1beta1/common_types.go

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,10 @@ type GlanceAPITemplate struct {
106106
// +kubebuilder:validation:Minimum=1
107107
// APITimeout for HAProxy and Apache defaults to GlanceSpecCore APITimeout
108108
APITimeout int `json:"apiTimeout,omitempty"`
109+
110+
// +kubebuilder:validation:Optional
111+
// HttpdCustomization - customize the httpd service
112+
HttpdCustomization HttpdCustomization `json:"httpdCustomization,omitempty"`
109113
}
110114

111115
// Storage -
@@ -144,6 +148,19 @@ type APIOverrideSpec struct {
144148
Service map[service.Endpoint]service.RoutedOverrideSpec `json:"service,omitempty"`
145149
}
146150

151+
// HttpdCustomization - customize the httpd service
152+
type HttpdCustomization struct {
153+
// +kubebuilder:validation:Optional
154+
// CustomConfigSecret - customize the httpd vhost config using this parameter to specify
155+
// a secret that contains service config data. The content of each provided snippet gets
156+
// rendered as a go template and placed into /etc/httpd/conf/httpd_custom_<key> .
157+
// In the default httpd template at the end of the vhost those custom configs get
158+
// included using `Include conf/httpd_custom_<endpoint>_*`.
159+
// For information on how sections in httpd configuration get merged, check section
160+
// "How the sections are merged" in https://httpd.apache.org/docs/current/sections.html#merging
161+
CustomConfigSecret *string `json:"customConfigSecret,omitempty"`
162+
}
163+
147164
// SetupDefaults - initializes any CRD field defaults based on environment variables (the defaulting mechanism itself is implemented via webhooks)
148165
func SetupDefaults() {
149166
// Acquire environmental defaults and initialize Glance defaults with them

‎api/v1beta1/glance_types.go

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,10 @@ type GlanceSpecCore struct {
125125
// +kubebuilder:validation:Minimum=1
126126
// Default APITimeout for HAProxy and Apache, defaults to 60 seconds
127127
APITimeout int `json:"apiTimeout"`
128+
129+
// +kubebuilder:validation:Optional
130+
// HttpdCustomization - customize the httpd service of all GlanceAPIs
131+
HttpdCustomization HttpdCustomization `json:"httpdCustomization,omitempty"`
128132
}
129133

130134
// GlanceSpec defines the desired state of Glance

‎api/v1beta1/zz_generated.deepcopy.go

Lines changed: 22 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎config/crd/bases/glance.openstack.org_glanceapis.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -546,6 +546,11 @@ spec:
546546
- extraVol
547547
type: object
548548
type: array
549+
httpdCustomization:
550+
properties:
551+
customConfigSecret:
552+
type: string
553+
type: object
549554
imageCache:
550555
properties:
551556
cleanerScheduler:

‎config/crd/bases/glance.openstack.org_glances.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -559,6 +559,11 @@ spec:
559559
items:
560560
type: string
561561
type: array
562+
httpdCustomization:
563+
properties:
564+
customConfigSecret:
565+
type: string
566+
type: object
562567
imageCache:
563568
properties:
564569
cleanerScheduler:
@@ -708,6 +713,11 @@ spec:
708713
type: object
709714
default: {}
710715
type: object
716+
httpdCustomization:
717+
properties:
718+
customConfigSecret:
719+
type: string
720+
type: object
711721
imageCache:
712722
properties:
713723
cleanerScheduler:

‎controllers/glance_common.go

Lines changed: 23 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ import (
2323

2424
networkv1 "github.com/k8snetworkplumbingwg/network-attachment-definition-client/pkg/apis/k8s.cni.cncf.io/v1"
2525
"github.com/openstack-k8s-operators/lib-common/modules/common/condition"
26+
"gopkg.in/yaml.v2"
2627
"k8s.io/apimachinery/pkg/types"
2728

2829
glancev1 "github.com/openstack-k8s-operators/glance-operator/api/v1beta1"
@@ -45,10 +46,11 @@ import (
4546

4647
// fields to index to reconcile when change
4748
const (
48-
passwordSecretField = ".spec.secret"
49-
caBundleSecretNameField = ".spec.tls.caBundleSecretName"
50-
tlsAPIInternalField = ".spec.tls.api.internal.secretName"
51-
tlsAPIPublicField = ".spec.tls.api.public.secretName"
49+
passwordSecretField = ".spec.secret"
50+
caBundleSecretNameField = ".spec.tls.caBundleSecretName"
51+
tlsAPIInternalField = ".spec.tls.api.internal.secretName"
52+
tlsAPIPublicField = ".spec.tls.api.public.secretName"
53+
httpdCustomServiceConfigSecretField = ".spec.httpdCustomization.customServiceConfigSecret"
5254
)
5355

5456
var (
@@ -60,6 +62,7 @@ var (
6062
caBundleSecretNameField,
6163
tlsAPIInternalField,
6264
tlsAPIPublicField,
65+
httpdCustomServiceConfigSecretField,
6366
}
6467
)
6568

@@ -162,18 +165,27 @@ func GenerateConfigsGeneric(
162165
customData map[string]string,
163166
cmLabels map[string]string,
164167
scripts bool,
168+
customTemplates map[string]string,
165169
) error {
166170

171+
// Marshal the templateParameters map to YAML
172+
yamlData, err := yaml.Marshal(templateParameters)
173+
if err != nil {
174+
return fmt.Errorf("Error marshalling to YAML: %w", err)
175+
}
176+
customData[common.TemplateParameters] = string(yamlData)
177+
167178
cms := []util.Template{
168179
// Templates where the GlanceAPI config is stored
169180
{
170-
Name: fmt.Sprintf("%s-config-data", instance.GetName()),
171-
Namespace: instance.GetNamespace(),
172-
Type: util.TemplateTypeConfig,
173-
InstanceType: instance.GetObjectKind().GroupVersionKind().Kind,
174-
ConfigOptions: templateParameters,
175-
CustomData: customData,
176-
Labels: cmLabels,
181+
Name: fmt.Sprintf("%s-config-data", instance.GetName()),
182+
Namespace: instance.GetNamespace(),
183+
Type: util.TemplateTypeConfig,
184+
InstanceType: instance.GetObjectKind().GroupVersionKind().Kind,
185+
ConfigOptions: templateParameters,
186+
CustomData: customData,
187+
StringTemplate: customTemplates,
188+
Labels: cmLabels,
177189
},
178190
}
179191
// TODO: Scripts have no reason to be secrets, should move to configmap

‎controllers/glance_controller.go

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -838,6 +838,10 @@ func (r *GlanceReconciler) apiDeploymentCreateOrUpdate(
838838
apiSpec.GlanceAPITemplate.NodeSelector = instance.Spec.NodeSelector
839839
}
840840

841+
if apiSpec.GlanceAPITemplate.HttpdCustomization.CustomConfigSecret == nil {
842+
apiSpec.GlanceAPITemplate.HttpdCustomization.CustomConfigSecret = instance.Spec.HttpdCustomization.CustomConfigSecret
843+
}
844+
841845
// Inherit the ImageCacheSize from the top level if not specified
842846
if apiSpec.ImageCache.Size == "" {
843847
apiSpec.ImageCache.Size = instance.Spec.ImageCache.Size
@@ -967,7 +971,7 @@ func (r *GlanceReconciler) generateServiceConfig(
967971
}
968972

969973
// Generate both default 00-config.conf and -scripts
970-
return GenerateConfigsGeneric(ctx, h, instance, envVars, templateParameters, customData, labels, true)
974+
return GenerateConfigsGeneric(ctx, h, instance, envVars, templateParameters, customData, labels, true, map[string]string{})
971975
}
972976

973977
// ensureRegisteredLimits - create registered limits in keystone that will be

‎controllers/glanceapi_controller.go

Lines changed: 32 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,18 @@ func (r *GlanceAPIReconciler) SetupWithManager(mgr ctrl.Manager) error {
241241
return err
242242
}
243243

244+
// index httpdOverrideSecretField
245+
if err := mgr.GetFieldIndexer().IndexField(context.Background(), &glancev1.GlanceAPI{}, httpdCustomServiceConfigSecretField, func(rawObj client.Object) []string {
246+
// Extract the secret name from the spec, if one is provided
247+
cr := rawObj.(*glancev1.GlanceAPI)
248+
if cr.Spec.HttpdCustomization.CustomConfigSecret == nil {
249+
return nil
250+
}
251+
return []string{*cr.Spec.HttpdCustomization.CustomConfigSecret}
252+
}); err != nil {
253+
return err
254+
}
255+
244256
// Watch for changes to any CustomServiceConfigSecrets. Global secrets
245257
svcSecretFn := func(_ context.Context, o client.Object) []reconcile.Request {
246258
var namespace string = o.GetNamespace()
@@ -1011,6 +1023,14 @@ func (r *GlanceAPIReconciler) generateServiceConfig(
10111023
}
10121024
customData[glance.CustomServiceConfigSecretsFileName] = customSecrets
10131025

1026+
httpdOverrideSecret := &corev1.Secret{}
1027+
if instance.Spec.HttpdCustomization.CustomConfigSecret != nil && *instance.Spec.HttpdCustomization.CustomConfigSecret != "" {
1028+
httpdOverrideSecret, _, err = secret.GetSecret(ctx, h, *instance.Spec.HttpdCustomization.CustomConfigSecret, instance.Namespace)
1029+
if err != nil {
1030+
return err
1031+
}
1032+
}
1033+
10141034
keystoneAPI, err := keystonev1.GetKeystoneAPI(ctx, h, instance.Namespace, map[string]string{})
10151035
// KeystoneAPI not available we should not aggregate the error and continue
10161036
if err != nil {
@@ -1043,6 +1063,7 @@ func (r *GlanceAPIReconciler) generateServiceConfig(
10431063
if instance.Spec.APIType != glancev1.APISingle {
10441064
endptName = fmt.Sprintf("%s-api", instance.Name)
10451065
}
1066+
customTemplates := map[string]string{}
10461067
httpdVhostConfig := map[string]interface{}{}
10471068
for endpt := range glanceEndpoints {
10481069
endptConfig := map[string]interface{}{}
@@ -1055,6 +1076,16 @@ func (r *GlanceAPIReconciler) generateServiceConfig(
10551076
endptConfig["SSLCertificateFile"] = fmt.Sprintf("/etc/pki/tls/certs/%s.crt", endpt.String())
10561077
endptConfig["SSLCertificateKeyFile"] = fmt.Sprintf("/etc/pki/tls/private/%s.key", endpt.String())
10571078
}
1079+
1080+
endptConfig["Override"] = false
1081+
if len(httpdOverrideSecret.Data) > 0 {
1082+
endptConfig["Override"] = true
1083+
for key, data := range httpdOverrideSecret.Data {
1084+
if len(data) > 0 {
1085+
customTemplates["httpd_custom_"+endpt.String()+"_"+key] = string(data)
1086+
}
1087+
}
1088+
}
10581089
httpdVhostConfig[endpt.String()] = endptConfig
10591090
}
10601091

@@ -1113,7 +1144,7 @@ func (r *GlanceAPIReconciler) generateServiceConfig(
11131144
// 00-default.conf will be regenerated as we have a ln -s of the
11141145
// templates/glance/config directory
11151146
// Do not generate -scripts as they are inherited from the top-level CR
1116-
return GenerateConfigsGeneric(ctx, h, instance, envVars, templateParameters, customData, labels, false)
1147+
return GenerateConfigsGeneric(ctx, h, instance, envVars, templateParameters, customData, labels, false, customTemplates)
11171148
}
11181149

11191150
// createHashOfInputHashes - creates a hash of hashes which gets added to the resources which requires a restart

‎go.mod

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,13 @@ require (
1313
github.com/openstack-k8s-operators/glance-operator/api v0.0.0-00010101000000-000000000000
1414
github.com/openstack-k8s-operators/infra-operator/apis v0.5.1-0.20250108092548-58707fa645ce
1515
github.com/openstack-k8s-operators/keystone-operator/api v0.5.1-0.20250107165241-16c3ed8e549f
16-
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241216113837-d172b3ac0f4e
16+
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20250116145727-01a8948d5dd7
1717
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.5.1-0.20241216113837-d172b3ac0f4e
1818
github.com/openstack-k8s-operators/lib-common/modules/storage v0.5.1-0.20241216113837-d172b3ac0f4e
1919
github.com/openstack-k8s-operators/lib-common/modules/test v0.5.1-0.20241216113837-d172b3ac0f4e
2020
github.com/openstack-k8s-operators/mariadb-operator/api v0.5.1-0.20250108071621-aa59f25c5b1a
2121
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56
22+
gopkg.in/yaml.v2 v2.4.0
2223
gopkg.in/yaml.v3 v3.0.1
2324
k8s.io/api v0.29.12
2425
k8s.io/apimachinery v0.29.12
@@ -74,7 +75,6 @@ require (
7475
google.golang.org/appengine v1.6.8 // indirect
7576
google.golang.org/protobuf v1.34.1 // indirect
7677
gopkg.in/inf.v0 v0.9.1 // indirect
77-
gopkg.in/yaml.v2 v2.4.0 // indirect
7878
k8s.io/apiextensions-apiserver v0.29.12 // indirect
7979
k8s.io/component-base v0.29.12 // indirect
8080
k8s.io/klog/v2 v2.120.1 // indirect

‎go.sum

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,8 +84,8 @@ github.com/openstack-k8s-operators/infra-operator/apis v0.5.1-0.20250108092548-5
8484
github.com/openstack-k8s-operators/infra-operator/apis v0.5.1-0.20250108092548-58707fa645ce/go.mod h1:TDaE7BVQvJwJGFm33R6xcPTeF8LKAnMh+a1ho+YqJHs=
8585
github.com/openstack-k8s-operators/keystone-operator/api v0.5.1-0.20250107165241-16c3ed8e549f h1:jlUo93FAwlDll1bJRxJO5B1Vi3t3wCoHQuy5HEO96ME=
8686
github.com/openstack-k8s-operators/keystone-operator/api v0.5.1-0.20250107165241-16c3ed8e549f/go.mod h1:CyuEOM1TpXKNUR1n8cudNtRzTEwkzv90JFkpDPPId8E=
87-
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241216113837-d172b3ac0f4e h1:hf4kVQBkyG79WcHBxdQ25QrDBbGFdarebS1Tc0Xclq4=
88-
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20241216113837-d172b3ac0f4e/go.mod h1:YpNTuJhDWhbXM50O3qBkhO7M+OOyRmWkNVmJ4y3cyFs=
87+
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20250116145727-01a8948d5dd7 h1:vXHpH93PjbAgg5ZN6n5WmxkybVQOs0nhXvVw62o7aZs=
88+
github.com/openstack-k8s-operators/lib-common/modules/common v0.5.1-0.20250116145727-01a8948d5dd7/go.mod h1:YpNTuJhDWhbXM50O3qBkhO7M+OOyRmWkNVmJ4y3cyFs=
8989
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.5.1-0.20241216113837-d172b3ac0f4e h1:HFo4OqPY0x4ZQeaWI2YGonTXAGTQFt+rOEJlfZVhS7s=
9090
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.5.1-0.20241216113837-d172b3ac0f4e/go.mod h1:IASoGvp5QM/tBJUd/8i8uIjj4DBnI+64Ydh4r7pmnvA=
9191
github.com/openstack-k8s-operators/lib-common/modules/storage v0.5.1-0.20241216113837-d172b3ac0f4e h1:Qz0JFEoRDUyjEWorNY3LggwxTsmpMtQkcpmZDQulGHQ=

‎templates/common/config/10-glance-httpd.conf

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,5 +33,10 @@
3333
SSLCertificateFile "{{ $vhost.SSLCertificateFile }}"
3434
SSLCertificateKeyFile "{{ $vhost.SSLCertificateKeyFile }}"
3535
{{- end }}
36+
37+
{{- if $vhost.Override }}
38+
Include conf/httpd_custom_{{ $endpt }}_*
39+
{{- end }}
40+
3641
</VirtualHost>
3742
{{ end }}

‎templates/common/config/glance-api-config.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,13 @@
6161
"dest": "/usr/local/bin/kolla_extend_start",
6262
"owner": "root:root",
6363
"perm": "0755"
64+
},
65+
{
66+
"source": "/var/lib/config-data/default/httpd_custom_*",
67+
"dest": "/etc/httpd/conf/",
68+
"owner": "apache",
69+
"perm": "0444",
70+
"optional": true
6471
}
6572
],
6673
"permissions": [

0 commit comments

Comments
 (0)