Skip to content

Commit a71d121

Browse files
gkechtjmoore4cbandydsessler7benjaminjb
authored
K8SPG 741 Merge upstream changes for 2.7.0 (#1094)
* Remove Postgres 10 from APIs and Tests Issue: PGO-614 * Remove major-upgrade-missing-image KUTTL test from e2e-other This test duplicates what's in the main e2e folder. * Remove json:omitempty from required fields This version of controller-gen ignores "validation:Required" markers when the struct tag has "json:omitempty". Issue: PGO-1748 * October updates to workflows and README Issue: PGO-738 Issue: PGO-1829 * run make generate * Update field manager for deployment id / configmap (#4020) For some reason, this was originally created without PGO listed as the manager for the configmap used by upgrade check. * Check that snapshot.Status is not nil when checking Status properties. * Move SQL escape functions to the Postgres package These are specific to Postgres. Add tests and remove unused functions. * Enable the "errchkjson" linter This is part of the "bugs" preset of golangci-lint. The "check-error-free-encoding" setting is recommended upstream. * Quiet a number of "gosec" warnings in tests These are ignored by golangci-lint by default. * Create warning event when postgres version is EOL. * Consistently use either pointer or value receivers Subtle bugs can arise when mixing the two. A type is more coherent when it behaves as only one of (1) a value or (2) a reference. A new linter identifies this situation but does not yet account for unmarshal methods on value types. See: https://go.dev/wiki/CodeReviewComments#receiver-type See: https://go.dev/wiki/MethodSets * fix errors and tests * Add functions to safely convert unstructured types The default unstructured converter does not complain if you try to convert a list to an object or vice versa. It also expects to be called with an empty target object. * fix library * Run upgrade tests in isolated namespaces * fix test * Remove github.com/pkg/errors from tests Tests already report the line on which an assert fails, and the standard "errors" packages unwraps errors. * fix imports * Load VolumeSnapshot CRDs from the client Go module during tests We only use these CRD files in Go tests, and Go has already downloaded them as part of the module we import for serializing API objects. * fix issues * Add default container annotation (#4031) * Add default container annotation Since K8s 1.21, the kubectl.kubernetes.io/default-container annotation can be added to set a default container, removing the need for a -c flag or eliminating the message about defaulting Note: the `-c` flag can still be used to refer to a different, non-default container. Issues: [PGO-1941] * bring back GOBIN='/hack/tools', run make clean-deprecated * fix one more test * commit generated * revert change on aws sdk go * Add an interface for querying available Kubernetes APIs * conflicts fixing * Explicitly enable "hot_standby" during restore This parameter has been enabled by default since Postgres 10 and is unlikely to change, but we want the behavior, so we should set it. * Consider pg_ctl successful when progress is made There is a race when using pg_ctl start --wait: - pg_ctl starts Postgres - Postgres begins recovery, detects a parameter requires restart, and exits - pg_ctl reports that Postgres did not start Now we look at the LSN reported by pg_controldata to determine if Postgres made any progress during a "failed" start. Issue: PGO-1945 * Tidy the pgBackRest restore command using slices Having these lines broken into string slices allows for Go comments that explain them without presenting those comments in YAML at runtime. This also: - Uses the postgres.ParameterSet type to accumulate Postgres settings. A new String method renders those values safely for use in postgresql.conf. - Disables localization using LC_ALL=C in calls to pg_controldata before we parse its output. - Removes commands to change permissions on tablespace directories; pgBackRest handles this for us now. - Passes command line parameters to Postgres using "-c" rather than "--" long flags. Both work on Linux, but the former works on all systems. - Explains why we need a large timeout for "pg_ctl --wait" and configures it once using the PGCTLTIMEOUT environment variable. * run make generate for main * features and conflicts and main * Stop turning runners context.Canceled error into nil. * Add a Comparison for YAML that contains a string * remove k8s helper functions that are not used * Use multiple processors when PGUpgrade is given CPU resources The --jobs flag allows for some aspects of pg_upgrade to operate in parallel. The documentation says: This option can dramatically reduce the time to upgrade a multi-database server running on a multiprocessor machine. Issue: PGO-1958 See: https://www.postgresql.org/docs/current/pgupgrade.html * bring back the error on marshaling * fix tests * run go fmt * larget timeout on assert 01 * add sts default annotation if version over 2.7 * fix annotations for pgbouncer and pgbackrest * fix unit test internal/controller/postgrescluster/pgbouncer_test.go * run go fmt --------- Co-authored-by: TJ Moore <[email protected]> Co-authored-by: Chris Bandy <[email protected]> Co-authored-by: Drew Sessler <[email protected]> Co-authored-by: Benjamin Blattberg <[email protected]>
1 parent a2dbe30 commit a71d121

File tree

102 files changed

+1144
-3787
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

102 files changed

+1144
-3787
lines changed

‎.golangci.next.yaml

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ linters:
1010
enable:
1111
- contextcheck
1212
- err113
13-
- errchkjson
1413
- gocritic
1514
- godot
1615
- godox
@@ -28,13 +27,16 @@ linters:
2827
- wastedassign
2928

3029
issues:
30+
exclude-rules:
31+
# We call external linters when they are installed: Flake8, ShellCheck, etc.
32+
- linters: [gosec]
33+
path: '_test[.]go$'
34+
text: 'G204: Subprocess launched with variable'
35+
3136
# https://github.com/golangci/golangci-lint/issues/2239
3237
exclude-use-default: false
3338

3439
linters-settings:
35-
errchkjson:
36-
check-error-free-encoding: true
37-
3840
thelper:
3941
# https://github.com/kulti/thelper/issues/27
4042
tb: { begin: true, first: true }

‎.golangci.yaml

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
linters:
44
disable:
55
- contextcheck
6-
- errchkjson
76
- gci
87
- gofumpt
98
- goimports
@@ -47,6 +46,15 @@ linters-settings:
4746
- pkg: github.com/percona/percona-postgresql-operator/internal/testing/*
4847
desc: The "internal/testing" packages should be used only in tests.
4948

49+
tests:
50+
files: ['$test']
51+
deny:
52+
- pkg: github.com/pkg/errors
53+
desc: Use the "errors" package unless you are interacting with stack traces.
54+
55+
errchkjson:
56+
check-error-free-encoding: true
57+
5058
exhaustive:
5159
default-signifies-exhaustive: true
5260

@@ -73,6 +81,10 @@ run:
7381
build-tags:
7482
- envtest
7583
issues:
76-
exclude-dirs:
77-
- pkg/generated
78-
- hack
84+
exclude-generated: strict
85+
exclude-rules:
86+
# These value types have unmarshal methods.
87+
# https://github.com/raeperd/recvcheck/issues/7
88+
- linters: [recvcheck]
89+
path: internal/pki/pki.go
90+
text: 'methods of "(Certificate|PrivateKey)"'

‎Makefile

Lines changed: 2 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,6 @@ PGMONITOR_DIR ?= hack/tools/pgmonitor
99
PGMONITOR_VERSION ?= v4.11.0
1010
QUERIES_CONFIG_DIR ?= hack/tools/queries
1111

12-
EXTERNAL_SNAPSHOTTER_DIR ?= hack/tools/external-snapshotter
13-
EXTERNAL_SNAPSHOTTER_VERSION ?= v8.0.1
14-
1512
# Buildah's "build" used to be "bud". Use the alias to be compatible for a while.
1613
BUILDAH_BUILD ?= buildah bud
1714

@@ -56,12 +53,6 @@ get-pgmonitor:
5653
cp -r '$(PGMONITOR_DIR)/postgres_exporter/common/.' '${QUERIES_CONFIG_DIR}'
5754
cp '$(PGMONITOR_DIR)/postgres_exporter/linux/queries_backrest.yml' '${QUERIES_CONFIG_DIR}'
5855

59-
.PHONY: get-external-snapshotter
60-
get-external-snapshotter:
61-
git -C '$(dir $(EXTERNAL_SNAPSHOTTER_DIR))' clone https://github.com/kubernetes-csi/external-snapshotter.git || git -C '$(EXTERNAL_SNAPSHOTTER_DIR)' fetch origin
62-
@git -C '$(EXTERNAL_SNAPSHOTTER_DIR)' checkout '$(EXTERNAL_SNAPSHOTTER_VERSION)'
63-
@git -C '$(EXTERNAL_SNAPSHOTTER_DIR)' config pull.ff only
64-
6556
.PHONY: clean
6657
clean: ## Clean resources
6758
clean: clean-deprecated
@@ -209,7 +200,7 @@ check: get-pgmonitor
209200
check-envtest: ## Run check using envtest and a mock kube api
210201
check-envtest: ENVTEST_USE = hack/tools/setup-envtest --bin-dir=$(CURDIR)/hack/tools/envtest use $(ENVTEST_K8S_VERSION)
211202
check-envtest: SHELL = bash
212-
check-envtest: get-pgmonitor get-external-snapshotter
203+
check-envtest: get-pgmonitor tools/setup-envtest
213204
GOBIN='$(CURDIR)/hack/tools' $(GO) install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
214205
@$(ENVTEST_USE) --print=overview && echo
215206
source <($(ENVTEST_USE) --print=env) && PGO_NAMESPACE="postgres-operator" QUERIES_CONFIG_DIR="$(CURDIR)/${QUERIES_CONFIG_DIR}" \
@@ -221,7 +212,7 @@ check-envtest: get-pgmonitor get-external-snapshotter
221212
# make check-envtest-existing PGO_TEST_TIMEOUT_SCALE=1.2
222213
.PHONY: check-envtest-existing
223214
check-envtest-existing: ## Run check using envtest and an existing kube api
224-
check-envtest-existing: get-pgmonitor get-external-snapshotter
215+
check-envtest-existing: get-pgmonitor
225216
check-envtest-existing: createnamespaces
226217
kubectl apply --server-side -k ./config/dev
227218
USE_EXISTING_CLUSTER=true PGO_NAMESPACE="postgres-operator" QUERIES_CONFIG_DIR="$(CURDIR)/${QUERIES_CONFIG_DIR}" \

‎build/crd/crunchy/generated/postgres-operator.crunchydata.com_pgupgrades.yaml

Lines changed: 0 additions & 2703 deletions
This file was deleted.

‎build/crd/crunchy/generated/postgres-operator.crunchydata.com_postgresclusters.yaml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20829,10 +20829,6 @@ spec:
2082920829
type:
2083020830
description: The pgBackRest backup type for this Job
2083120831
type: string
20832-
required:
20833-
- cronJobName
20834-
- repo
20835-
- type
2083620832
type: object
2083720833
type: array
2083820834
type: object

‎cmd/postgres-operator/main.go

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,12 @@ func main() {
9292
assertNoError(err)
9393

9494
assertNoError(features.Set(os.Getenv("PGO_FEATURE_GATES")))
95-
log.Info("feature gates enabled", "PGO_FEATURE_GATES", features.String())
95+
ctx = feature.NewContext(ctx, features)
96+
log.Info("feature gates",
97+
// These are set by the user
98+
"PGO_FEATURE_GATES", feature.ShowAssigned(ctx),
99+
// These are enabled, including features that are on by default
100+
"enabled", feature.ShowEnabled(ctx))
96101

97102
cruntime.SetLogger(log)
98103

‎config/crd/bases/postgres-operator.crunchydata.com_postgresclusters.yaml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20727,10 +20727,6 @@ spec:
2072720727
type:
2072820728
description: The pgBackRest backup type for this Job
2072920729
type: string
20730-
required:
20731-
- cronJobName
20732-
- repo
20733-
- type
2073420730
type: object
2073520731
type: array
2073620732
type: object

‎config/manager/manager.yaml

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -18,28 +18,28 @@ spec:
1818
fieldPath: metadata.namespace
1919
- name: CRUNCHY_DEBUG
2020
value: "true"
21-
- name: RELATED_IMAGE_POSTGRES_15
22-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres:ubi8-15.7-1"
23-
- name: RELATED_IMAGE_POSTGRES_15_GIS_3.3
24-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres-gis:ubi8-15.7-3.3-1"
2521
- name: RELATED_IMAGE_POSTGRES_16
26-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres:ubi8-16.3-1"
22+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres:ubi8-16.4-2"
2723
- name: RELATED_IMAGE_POSTGRES_16_GIS_3.3
28-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres-gis:ubi8-16.3-3.3-1"
24+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres-gis:ubi8-16.4-3.3-2"
2925
- name: RELATED_IMAGE_POSTGRES_16_GIS_3.4
30-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres-gis:ubi8-16.3-3.4-1"
26+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres-gis:ubi8-16.4-3.4-2"
27+
- name: RELATED_IMAGE_POSTGRES_17
28+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres:ubi8-17.0-0"
29+
- name: RELATED_IMAGE_POSTGRES_17_GIS_3.4
30+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres-gis:ubi8-17.0-3.4-0"
3131
- name: RELATED_IMAGE_PGADMIN
32-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgadmin4:ubi8-4.30-26"
32+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgadmin4:ubi8-4.30-31"
3333
- name: RELATED_IMAGE_PGBACKREST
34-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgbackrest:ubi8-2.51-1"
34+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgbackrest:ubi8-2.53.1-0"
3535
- name: RELATED_IMAGE_PGBOUNCER
36-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgbouncer:ubi8-1.22-1"
36+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgbouncer:ubi8-1.23-0"
3737
- name: RELATED_IMAGE_PGEXPORTER
3838
value: "registry.developers.crunchydata.com/crunchydata/crunchy-postgres-exporter:latest"
3939
- name: RELATED_IMAGE_PGUPGRADE
4040
value: "registry.developers.crunchydata.com/crunchydata/crunchy-upgrade:latest"
4141
- name: RELATED_IMAGE_STANDALONE_PGADMIN
42-
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgadmin4:ubi8-8.6-1"
42+
value: "registry.developers.crunchydata.com/crunchydata/crunchy-pgadmin4:ubi8-8.12-0"
4343
securityContext:
4444
allowPrivilegeEscalation: false
4545
capabilities: { drop: [ALL] }

‎deploy/bundle.yaml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47215,10 +47215,6 @@ spec:
4721547215
type:
4721647216
description: The pgBackRest backup type for this Job
4721747217
type: string
47218-
required:
47219-
- cronJobName
47220-
- repo
47221-
- type
4722247218
type: object
4722347219
type: array
4722447220
type: object

‎deploy/crd.yaml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47215,10 +47215,6 @@ spec:
4721547215
type:
4721647216
description: The pgBackRest backup type for this Job
4721747217
type: string
47218-
required:
47219-
- cronJobName
47220-
- repo
47221-
- type
4722247218
type: object
4722347219
type: array
4722447220
type: object

‎deploy/cw-bundle.yaml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47215,10 +47215,6 @@ spec:
4721547215
type:
4721647216
description: The pgBackRest backup type for this Job
4721747217
type: string
47218-
required:
47219-
- cronJobName
47220-
- repo
47221-
- type
4722247218
type: object
4722347219
type: array
4722447220
type: object

‎e2e-tests/tests/upgrade-consistency/01-assert.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
apiVersion: kuttl.dev/v1beta1
22
kind: TestAssert
3-
timeout: 180
3+
timeout: 300
44
---
55
kind: StatefulSet
66
apiVersion: apps/v1

‎examples/postgrescluster/postgrescluster.yaml

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ kind: PostgresCluster
33
metadata:
44
name: hippo
55
spec:
6-
image: registry.developers.crunchydata.com/crunchydata/crunchy-postgres:ubi8-16.3-1
76
postgresVersion: 16
87
instances:
98
- name: instance1
@@ -15,7 +14,6 @@ spec:
1514
storage: 1Gi
1615
backups:
1716
pgbackrest:
18-
image: registry.developers.crunchydata.com/crunchydata/crunchy-pgbackrest:ubi8-2.51-1
1917
repos:
2018
- name: repo1
2119
volume:
@@ -34,5 +32,4 @@ spec:
3432
requests:
3533
storage: 1Gi
3634
proxy:
37-
pgBouncer:
38-
image: registry.developers.crunchydata.com/crunchydata/crunchy-pgbouncer:ubi8-1.22-1
35+
pgBouncer: {}

‎go.mod

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ require (
5555
github.com/x448/float16 v0.8.4 // indirect
5656
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
5757
go.uber.org/automaxprocs v1.6.0 // indirect
58+
golang.org/x/mod v0.24.0 // indirect
5859
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
5960
)
6061

@@ -122,7 +123,7 @@ require (
122123
golang.org/x/term v0.31.0 // indirect
123124
golang.org/x/text v0.24.0 // indirect
124125
golang.org/x/time v0.8.0 // indirect
125-
golang.org/x/tools v0.31.0 // indirect
126+
golang.org/x/tools v0.31.0
126127
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
127128
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb // indirect
128129
google.golang.org/genproto/googleapis/rpc v0.0.0-20250311190419-81fb87f6b8bf // indirect

‎go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -287,6 +287,8 @@ golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHl
287287
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
288288
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
289289
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
290+
golang.org/x/mod v0.24.0 h1:ZfthKaKaT4NrhGVZHO1/WDTwGES4De8KtWO0SIbNJMU=
291+
golang.org/x/mod v0.24.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
290292
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
291293
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
292294
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=

0 commit comments

Comments
 (0)