Skip to content

Commit fd6a245

Browse files
VineethReddy02sdudoladov
VineethReddy02
authored and
sdudoladov
committed
[GSoC 2019] kubectl plugin for the Postgres operator (#579)
* Add a prototype of a kubectl plugin for the Postgres operator Work done by Vineeth Pothulapati <[email protected]> during Google Summer of Code 2019
1 parent 056b222 commit fd6a245

19 files changed

+2191
-0
lines changed

‎kubectl-pg/README.md

+125
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Kubectl Plugin for Zalando's Postgres Operator
2+
3+
## Google Summer of Code 2019
4+
5+
This plugin is a prototype developed as a part of GSoC 2019 under the organisation
6+
**The Postgres Operator**
7+
8+
### GSoC Proposal
9+
10+
[kubectl pg proposal](https://docs.google.com/document/d/1-WMy9HkfZ1XnnMbzplMe9rCzKrRMGaMz4owLVXXPb7w/edit)
11+
12+
### Weekly Reports
13+
14+
https://github.com/VineethReddy02/GSoC-Kubectl-Plugin-for-Postgres-Operator-tracker
15+
16+
### Final Project Report
17+
18+
https://gist.github.com/VineethReddy02/159283bd368a710379eaf0f6bd60a40a
19+
20+
21+
### Installtion of kubectl pg plugin
22+
23+
This project uses Go Modules for dependency management to build locally
24+
Install go and enable go modules ```export GO111MODULE=on```
25+
From Go >=1.13 Go modules will be enabled by default
26+
```
27+
# Assumes you have a working KUBECONFIG
28+
$ GO111MODULE="on"
29+
# As of now go by default doesn't support Go mods. So explicit enabling is required.
30+
$ GOPATH/src/github.com/zalando/postgres-operator/kubectl-pg go mod vendor
31+
# This generate a vendor directory with all dependencies needed by the plugin.
32+
$ $GOPATH/src/github.com/zalando/postgres-operator/kubectl-pg go install
33+
# This will place the kubectl-pg binary in your $GOPATH/bin
34+
```
35+
36+
### Before using the kubectl pg plugin make sure to set KUBECONFIG env varibale
37+
38+
Ideally KUBECONFIG is found in $HOME/.kube/config else specify the KUBECONFIG path here.
39+
40+
```export KUBECONFIG=$HOME/.kube/config```
41+
42+
### To list all commands available in kubectl pg
43+
44+
```kubectl pg --help``` (or) ```kubectl pg```
45+
46+
### This basically means the operator pod managed to start, so our operator is installed.
47+
48+
```kubectl pg check```
49+
50+
### To create postgresql cluster using manifest file
51+
52+
```kubectl pg create -f acid-minimal-cluster.yaml```
53+
54+
### To update existing cluster using manifest file
55+
56+
```kubectl pg update -f acid-minimal-cluster.yaml```
57+
58+
### To delete existing cluster using manifest file
59+
60+
```kubectl pg delete -f acid-minimal-cluster.yaml```
61+
62+
### To delete existing cluster using cluster name
63+
64+
```kubectl pg delete acid-minimal-cluster```
65+
66+
--namespace or -n flag to specify namespace if cluster is in other namespace.
67+
68+
```kubectl pg delete acid-minimal-cluster -n namespace01```
69+
70+
### To list postgres resources for the current namespace
71+
72+
```kubectl pg list```
73+
74+
### To list postgres resources across namespaces
75+
76+
```kubectl pg list all```
77+
78+
### To add-user and it's roles to an existing pg cluster
79+
80+
```kubectl pg add-user USER01 -p CREATEDB,LOGIN -c acid-minimal-cluster```
81+
82+
Privileges can only be [SUPERUSER, REPLICATION, INHERIT, LOGIN, NOLOGIN, CREATEROLE, CREATEDB, BYPASSURL]
83+
84+
Note: A login user is created by default unless NOLOGIN is specified, in which case the operator creates a role.
85+
86+
### To add-db and it's owner to an existing pg cluster
87+
88+
```kubectl pg add-db DB01 -o OWNER01 -c acid-minimal-cluster```
89+
90+
### To extend volume for an existing pg cluster
91+
92+
```kubectl pg ext-volume 2Gi -c acid-minimal-cluster```
93+
94+
### To find the version of postgres operator and kubectl plugin
95+
96+
```kubectl pg version (optional -n NAMESPACE allows to know specific to a namespace)```
97+
98+
### To connect to the shell of a postgres pod
99+
100+
```kubectl pg connect -c CLUSTER``` #This connects to a random pod
101+
```kubectl pg connect -c CLUSTER -m``` #This connects the master
102+
```kubectl pg connect -c CLUSTER -r 0``` #This connects to the desired replica
103+
104+
### To connect to the psql prompt
105+
106+
```kubectl pg connect -c CLUSTER -p -u username``` #This connects to a random pod. Not master
107+
```kubectl pg connect -c CLUSTER -m -p -u username``` #This connects the master
108+
```kubectl pg connect -c CLUSTER -r 0 -p -u username``` #This connects to the desired replica
109+
110+
Note: -p represents psql prompt
111+
112+
### To get the logs of postgres operator
113+
114+
```kubectl pg logs -o```
115+
116+
### To get the logs of the postgres cluster
117+
118+
```kubectl pg logs -c CLUSTER``` #Fetches the logs of a random pod. Not master
119+
```kubectl pg logs -c CLUSTER -m``` #Fetches the logs of master
120+
```kubectl pg logs -c CLUSTER -r 2``` #Fecthes the logs of specified replica
121+
122+
## Development
123+
124+
- When making changes to plugin make sure to change the major or patch version
125+
of plugin in ```build.sh``` and run ```./build.sh```

‎kubectl-pg/build.sh

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
2+
VERSION=1.0
3+
sed -i "s/KubectlPgVersion string = \"[^\"]*\"/KubectlPgVersion string = \"${VERSION}\"/" cmd/version.go
4+
go install

‎kubectl-pg/cmd/addDb.go

+112
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
/*
2+
Copyright © 2019 Vineeth Pothulapati <[email protected]>
3+
4+
Permission is hereby granted, free of charge, to any person obtaining a copy
5+
of this software and associated documentation files (the "Software"), to deal
6+
in the Software without restriction, including without limitation the rights
7+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
8+
copies of the Software, and to permit persons to whom the Software is
9+
furnished to do so, subject to the following conditions:
10+
11+
The above copyright notice and this permission notice shall be included in
12+
all copies or substantial portions of the Software.
13+
14+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
17+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
18+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
19+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
20+
THE SOFTWARE.
21+
*/
22+
23+
package cmd
24+
25+
import (
26+
"encoding/json"
27+
"fmt"
28+
"github.com/spf13/cobra"
29+
PostgresqlLister "github.com/zalando/postgres-operator/pkg/generated/clientset/versioned/typed/acid.zalan.do/v1"
30+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
31+
"k8s.io/apimachinery/pkg/types"
32+
"log"
33+
)
34+
35+
// addDbCmd represents the addDb command
36+
var addDbCmd = &cobra.Command{
37+
Use: "add-db",
38+
Short: "Adds a DB and its owner to a Postgres cluster. The owner role is created if it does not exist",
39+
Long: `Adds a new DB to the Postgres cluster. Owner needs to be specified by the -o flag, cluster with -c flag.`,
40+
Run: func(cmd *cobra.Command, args []string) {
41+
if len(args) > 0 {
42+
dbName := args[0]
43+
dbOwner, _ := cmd.Flags().GetString("owner")
44+
clusterName, _ := cmd.Flags().GetString("cluster")
45+
addDb(dbName, dbOwner, clusterName)
46+
} else {
47+
fmt.Println("database name can't be empty. Use kubectl pg add-db [-h | --help] for more info")
48+
}
49+
50+
},
51+
Example: `
52+
kubectl pg add-db db01 -o owner01 -c cluster01
53+
`,
54+
}
55+
56+
// add db and it's owner to the cluster
57+
func addDb(dbName string, dbOwner string, clusterName string) {
58+
config := getConfig()
59+
postgresConfig, err := PostgresqlLister.NewForConfig(config)
60+
if err != nil {
61+
log.Fatal(err)
62+
}
63+
64+
namespace := getCurrentNamespace()
65+
postgresql, err := postgresConfig.Postgresqls(namespace).Get(clusterName, metav1.GetOptions{})
66+
if err != nil {
67+
log.Fatal(err)
68+
}
69+
70+
var dbOwnerExists bool
71+
dbUsers := postgresql.Spec.Users
72+
for key, _ := range dbUsers {
73+
if key == dbOwner {
74+
dbOwnerExists = true
75+
}
76+
}
77+
var patch []byte
78+
// validating reserved DB names
79+
if dbOwnerExists && dbName != "postgres" && dbName != "template0" && dbName != "template1" {
80+
patch = dbPatch(dbName, dbOwner)
81+
} else if !dbOwnerExists {
82+
log.Fatal("The provided db-owner doesn't exist")
83+
} else {
84+
log.Fatal("The provided db-name is reserved by postgres")
85+
}
86+
87+
updatedPostgres, err := postgresConfig.Postgresqls(namespace).Patch(postgresql.Name, types.MergePatchType, patch, "")
88+
if err != nil {
89+
log.Fatal(err)
90+
}
91+
92+
if updatedPostgres.ResourceVersion != postgresql.ResourceVersion {
93+
fmt.Printf("Created new database %s with owner %s in PostgreSQL cluster %s.\n", dbName, dbOwner, updatedPostgres.Name)
94+
} else {
95+
fmt.Printf("postgresql %s is unchanged.\n", updatedPostgres.Name)
96+
}
97+
}
98+
99+
func dbPatch(dbname string, owner string) []byte {
100+
ins := map[string]map[string]map[string]string{"spec": {"databases": {dbname: owner}}}
101+
patchInstances, err := json.Marshal(ins)
102+
if err != nil {
103+
log.Fatal(err, "unable to parse patch for add-db")
104+
}
105+
return patchInstances
106+
}
107+
108+
func init() {
109+
addDbCmd.Flags().StringP("owner", "o", "", "provide owner of the database.")
110+
addDbCmd.Flags().StringP("cluster", "c", "", "provide a postgres cluster name.")
111+
rootCmd.AddCommand(addDbCmd)
112+
}

‎kubectl-pg/cmd/addUser.go

+142
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
/*
2+
Copyright © 2019 Vineeth Pothulapati <[email protected]>
3+
4+
Permission is hereby granted, free of charge, to any person obtaining a copy
5+
of this software and associated documentation files (the "Software"), to deal
6+
in the Software without restriction, including without limitation the rights
7+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
8+
copies of the Software, and to permit persons to whom the Software is
9+
furnished to do so, subject to the following conditions:
10+
11+
The above copyright notice and this permission notice shall be included in
12+
all copies or substantial portions of the Software.
13+
14+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
17+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
18+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
19+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
20+
THE SOFTWARE.
21+
*/
22+
23+
package cmd
24+
25+
import (
26+
"encoding/json"
27+
"fmt"
28+
"github.com/spf13/cobra"
29+
PostgresqlLister "github.com/zalando/postgres-operator/pkg/generated/clientset/versioned/typed/acid.zalan.do/v1"
30+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
31+
"k8s.io/apimachinery/pkg/types"
32+
"log"
33+
"strings"
34+
)
35+
36+
var allowedPrivileges = []string{"SUPERUSER", "REPLICATION", "INHERIT", "LOGIN", "NOLOGIN", "CREATEROLE", "CREATEDB", "BYPASSURL"}
37+
38+
// addUserCmd represents the addUser command
39+
var addUserCmd = &cobra.Command{
40+
Use: "add-user",
41+
Short: "Adds a user to the postgres cluster with given privileges",
42+
Long: `Adds a user to the postgres cluster. You can add privileges as well with -p flag.`,
43+
Run: func(cmd *cobra.Command, args []string) {
44+
clusterName, _ := cmd.Flags().GetString("cluster")
45+
privileges, _ := cmd.Flags().GetString("privileges")
46+
47+
if len(args) > 0 {
48+
user := args[0]
49+
var permissions []string
50+
var perms []string
51+
52+
if privileges != "" {
53+
parsedRoles := strings.Replace(privileges, ",", " ", -1)
54+
parsedRoles = strings.ToUpper(parsedRoles)
55+
permissions = strings.Fields(parsedRoles)
56+
var invalidPerms []string
57+
58+
for _, userPrivilege := range permissions {
59+
validPerm := false
60+
for _, privilege := range allowedPrivileges {
61+
if privilege == userPrivilege {
62+
perms = append(perms, userPrivilege)
63+
validPerm = true
64+
}
65+
}
66+
if !validPerm {
67+
invalidPerms = append(invalidPerms, userPrivilege)
68+
}
69+
}
70+
71+
if len(invalidPerms) > 0 {
72+
fmt.Printf("Invalid privileges %s\n", invalidPerms)
73+
return
74+
}
75+
}
76+
addUser(user, clusterName, perms)
77+
}
78+
},
79+
Example: `
80+
kubectl pg add-user user01 -p login,createdb -c cluster01
81+
`,
82+
}
83+
84+
// add user to the cluster with provided permissions
85+
func addUser(user string, clusterName string, permissions []string) {
86+
config := getConfig()
87+
postgresConfig, err := PostgresqlLister.NewForConfig(config)
88+
if err != nil {
89+
log.Fatal(err)
90+
}
91+
92+
namespace := getCurrentNamespace()
93+
postgresql, err := postgresConfig.Postgresqls(namespace).Get(clusterName, metav1.GetOptions{})
94+
if err != nil {
95+
log.Fatal(err)
96+
}
97+
98+
setUsers := make(map[string]bool)
99+
for _, k := range permissions {
100+
setUsers[k] = true
101+
}
102+
103+
if existingRoles, key := postgresql.Spec.Users[user]; key {
104+
for _, k := range existingRoles {
105+
setUsers[k] = true
106+
}
107+
}
108+
109+
Privileges := []string{}
110+
for keys, values := range setUsers {
111+
if values {
112+
Privileges = append(Privileges, keys)
113+
}
114+
}
115+
116+
patch := applyUserPatch(user, Privileges)
117+
updatedPostgresql, err := postgresConfig.Postgresqls(namespace).Patch(postgresql.Name, types.MergePatchType, patch, "")
118+
if err != nil {
119+
log.Fatal(err)
120+
}
121+
122+
if updatedPostgresql.ResourceVersion != postgresql.ResourceVersion {
123+
fmt.Printf("postgresql %s is updated with new user %s and with privileges %s.\n", updatedPostgresql.Name, user, permissions)
124+
} else {
125+
fmt.Printf("postgresql %s is unchanged.\n", updatedPostgresql.Name)
126+
}
127+
}
128+
129+
func applyUserPatch(user string, value []string) []byte {
130+
ins := map[string]map[string]map[string][]string{"spec": {"users": {user: value}}}
131+
patchInstances, err := json.Marshal(ins)
132+
if err != nil {
133+
log.Fatal(err, "unable to parse number of instances json")
134+
}
135+
return patchInstances
136+
}
137+
138+
func init() {
139+
addUserCmd.Flags().StringP("cluster", "c", "", "add user to the provided cluster.")
140+
addUserCmd.Flags().StringP("privileges", "p", "", "add privileges separated by commas without spaces")
141+
rootCmd.AddCommand(addUserCmd)
142+
}

0 commit comments

Comments
 (0)