Unable to load configuration: failed to create store (externaldb)

[andrewklau]

Summary

Unable to store external postgres configuration on version 6 and above with read only error

Using this config:

  database:
    external:
      secret: mattermost

kubectl logs -f -n mattermost pod/mattermost-7cc6dbcdfb-9bmd2
Error: failed to load configuration: failed to create store: unable to load on store creation: failed to persist: failed to write file: open /mattermost/config/postgresql:/user:password@mypostgresdb:25060/defaultdb?sslmode=require: no such file or directory
Usage:
  mattermost [flags]
  mattermost [command]

Available Commands:
  db          Commands related to the database
  export      Export data from Mattermost
  help        Help about any command
  import      Import data.
  jobserver   Start the Mattermost job server
  server      Run the Mattermost server
  version     Display version information

Flags:
  -c, --config string   Configuration file to use.
  -h, --help            help for mattermost

Use "mattermost [command] --help" for more information about a command.

Steps to reproduce

• Latest mattermost operator
• Deploy mattermost using helm operator and external DB secret

Expected behavior

mattermost starts

[Szymongib]

Hi @andrewklau,

Could you share your Mattermost CR manifest? I have a hard time understanding what is the problem exactly. Are you just trying to run Mattermost with the external database, or is there something specific to your setup?

It looks like Mattermost is trying to interpret the connection string as a file patch which seems a little weird.

I would appreciate some details so that we can hopefully figure it out.

[andrewklau]

Hi @Szymongib

I'm trying to run mattermost with an external database. Here's the full yaml file:

apiVersion: installation.mattermost.com/v1beta1
kind: Mattermost
metadata:
  name: mattermost
  namespace: mattermost
spec:
  size: 100users
  ingress:
    enabled: true
    host: chat.example.com
    annotations:
      kubernetes.io/ingress.class: nginx
  version: 6.0.1
  database:
    external:
      secret: mattermost
  fileStore:
    external:
      url: sgp1.digitaloceanspaces.com
      bucket: mymattermost
      secret: mattermost
  mattermostEnv:
  - name: MM_FILESETTINGS_AMAZONS3SSE
    value: "true"
  - name: MM_FILESETTINGS_AMAZONS3SSL
    value: "true"

The weird thing is yes it comes up as a file name. When I run the command kubectl logs -f -n mattermost pod/mattermost-7cc6dbcdfb-9bmd2 this comes up in the logs:

Error: failed to load configuration: failed to create store: unable to load on store creation: failed to persist: failed to write file: open /mattermost/config/postgresql:/user:password@mypostgresdb:25060/defaultdb?sslmode=require: no such file or directory
Usage:
  mattermost [flags]
  mattermost [command]

Available Commands:
  db          Commands related to the database
  export      Export data from Mattermost
  help        Help about any command
  import      Import data.
  jobserver   Start the Mattermost job server
  server      Run the Mattermost server
  version     Display version information

Flags:
  -c, --config string   Configuration file to use.
  -h, --help            help for mattermost

Use "mattermost [command] --help" for more information about a command.

The secret file:

apiVersion: v1
kind: Secret
metadata:
  name: mattermost
  namespace: mattermost
type: Opaque
stringData:
  DB_CONNECTION_STRING: 'postgresql://admin:[email protected]:25060/defaultdb?sslmode=require'

[Szymongib]

Thanks a lot for details @andrewklau.

It looks like the issue is on the Mattermost Server side, apparently, the server only accepts connection schema postgres and treats others as a file.

Try changing your connection string to:

postgres://admin:[email protected]:25060/defaultdb?sslmode=require

Hopefully, that should solve the issue.

[andrewklau]

Thanks it works now!

[jpnorenam]

In Kubernetes v1.26, the secret data can't be created as plain text.

Error from server (BadRequest): error when creating "STDIN": Secret in version "v1" cannot be handled as a Secret: illegal base64 data at input byte <n>

And in base64, as the documentation suggests:

apiVersion: v1
data:
  DB_CONNECTION_CHECK_URL: cG9zdGdyZXM6Ly91c2VyOnN1cGVyX3NlY3JldF9wYXNzd29yZEBteS1kYXRhYmFzZS5jbHVzdGVyLWFiY2QudXMtZWFzdC0xLnJkcy5hbWF6b25hd3MuY29tOjU0MzIvbWF0dGVybW9zdD9jb25uZWN0X3RpbWVvdXQ9MTAK
  DB_CONNECTION_STRING: cG9zdGdyZXM6Ly91c2VyOnN1cGVyX3NlY3JldF9wYXNzd29yZEBteS1kYXRhYmFzZS5jbHVzdGVyLWFiY2QudXMtZWFzdC0xLnJkcy5hbWF6b25hd3MuY29tOjU0MzIvbWF0dGVybW9zdD9jb25uZWN0X3RpbWVvdXQ9MTAK
  MM_SQLSETTINGS_DATASOURCEREPLICAS: cG9zdGdyZXM6Ly91c2VyOnN1cGVyX3NlY3JldF9wYXNzd29yZEBteS1kYXRhYmFzZS5jbHVzdGVyLXJvLWFiY2QudXMtZWFzdC0xLnJkcy5hbWF6b25hd3MuY29tOjU0MzIvbWF0dGVybW9zdD9jb25uZWN0X3RpbWVvdXQ9MTAK
kind: Secret
metadata:
  name: my-postgres-connection
type: Opaque

the operator seems to add a '/n' at the end of the connection string, resulting in the following error:

Defaulted container "mattermost" out of: mattermost, init-check-database (init)
Error: failed to load configuration: failed to initialize: failed to create Configurations table: parse "postgres://user:super_secret_password@my-database.cluster-abcd.us-east-1.rds.amazonaws.com:5432/mattermost?connect_timeout=10\n": net/url: invalid control character in URL

[jpnorenam]

Opened a new issue: #342

[Szymongib]

@jpnorenam It is likely not the Operator that adds a new line but the way you are encoding the string.

Try this if you are on Linux:

echo "postgres://user:super_secret_password@my-database.cluster-abcd.us-east-1.rds.amazonaws.com:5432/mattermost?connect_timeout=10" | base64 -w 0

Or on MacOS:

echo "postgres://user:super_secret_password@my-database.cluster-abcd.us-east-1.rds.amazonaws.com:5432/mattermost?connect_timeout=10" | tr -d \\n | base64