Update Helm README (#27)

Co-authored-by: Martin Kysel <[email protected]>
xmtp · Feb 7, 2025 · dc3c9ec · dc3c9ec
1 parent dd0744b
commit dc3c9ec
Showing 1 changed file with 171 additions and 43 deletions.
diff --git a/helm/README.md b/helm/README.md
@@ -1,87 +1,211 @@
-# XMTPD Helm Charts
+# Install xmtpd on Kubernetes using Helm charts
 
-**⚠️ Experimental:** This software is in early development. Expect frequent changes and unresolved issues.
+**xmtpd (XMTP daemon)** is software that operators run to provide nodes in the decentralized XMTP network. The responsibilities of an xmtpd node include message storage and replication, censorship resistance, network resilience and recovery, and much more.
 
-`xmtpd` (XMTP daemon) is an experimental version of XMTP node software. It is **not** the node software that currently forms the XMTP network.
+This repository includes Helm charts that you can use to deploy xmtpd on Kubernetes. Charts are curated application definitions for Helm. 
 
-This repository includes Helm Charts that can be used to deploy XMTPD in Kubernetes. You may also contribute by submitting enhancement requests if you like. Charts are curated application definitions for Helm. For more information about installing and using Helm, see its
-[README.md](https://github.com/helm/helm/tree/master/README.md). To get a quick introduction to Helm Charts see this [chart document](https://github.com/helm/helm/blob/master/docs/charts.md). For more information on using Helm, refer to the [Helm's documentation](https://github.com/kubernetes/helm#docs).
+- To learn how to install Helm, see the Helm [README](https://github.com/helm/helm/tree/master/README.md). 
 
-## XMTPD Helm Chart Installation
+- To learn how to use Helm, see the Helm [Quickstart Guide](https://helm.sh/docs/intro/quickstart/).
 
-To add the XMTPD charts to your local helm repository, clone this repository locally.
+- To learn more about Helm charts, see [Charts](https://helm.sh/docs/topics/charts/). 
 
-Eventually XMTP will provide a public Helm Charts release.
+## What does the xmtpd Helm chart contain?
 
-## 0) Alchemy Account
+xmtpd is composed of two key deployments: 
 
-To run a node, you will need to have an [Alchemy Account](https://www.alchemy.com/).
-You can follow their docs on the [API Key Creation](https://docs.alchemy.com/docs/alchemy-quickstart-guide#1key-create-an-alchemy-api-key).
+1. **Synchronization service**
+
+    Handles data replication from other nodes to the local node, ensuring consistency across the network. 
+
+2. **API service**
+
+    Provides a client-facing gRPC endpoint for both nodes and client applications (like Converse) to interact with the system.
+
+Both services support multiple replicas for high availability. However, adding replicas doesn't enhance the overall throughput of the service. As such, we recommend running exactly two replicas of each.
 
-Once logged in, you can find and enable the [XMTP Chain](https://dashboard.alchemy.com/chains/xmtp).
-Find and copy the XMTP app HTTP endpoint along with its API key. It should be in the format `https://xmtp-testnet.g.alchemy.com/v2/<apikey>`
+## Prerequisites
 
-## 1) Dependencies
+Before diving into the installation process, ensure you have the following:
 
-XMTPD needs a Postgres database to be running accessible from the cluster.
-For example, you can use [Bitnami PG Helm Charts](https://bitnami.com/stack/postgresql/helm), or any other tooling to provision a Postgres database.
+1. **Kubernetes cluster**: A running Kubernetes cluster with sufficient resources. To learn more, see [Kubernetes documentation](https://kubernetes.io/docs/home/). If you want to test the charts locally, you can use [Docker Desktop](https://www.docker.com/get-started/)
+2. **Helm**: A package manager for Kubernetes. To learn more, see [Helm documentation](https://helm.sh/docs/).
+   ```bash
+   brew install helm
+   ```
+3. **PostgreSQL database**: xmtpd relies on a PostgreSQL database. Tools like Bitnami’s [PostgreSQL Helm Charts](https://github.com/bitnami/charts/tree/main/bitnami/postgresql) can simplify database provisioning. xmtpd has been tested with Postgres versions 13 and newer. To learn more, see [PostgreSQL documentation](https://www.postgresql.org/docs/).
+   ```bash
+    helm repo add bitnami https://charts.bitnami.com/bitnami
+    helm repo update
+    ```
+4. Clone the [xmtpd-infrastructure](https://github.com/xmtp/xmtpd-infrastructure) repo.
 
-## 2) Installing MLS Validation Service
+## Step 1. Get an Alchemy account
 
-The XMTP MLS Validation Service does not have any external dependencies.
-It is a stateless horizontally-scalable service.
+To run xmtpd, you need an Alchemy account.
+
+1. See [Create an Alchemy API Key](https://docs.alchemy.com/docs/alchemy-quickstart-guide#1key-create-an-alchemy-api-key) and log in to Alchemy.
+
+2. Go to the [XMTP Chain page](https://dashboard.alchemy.com/chains/xmtp) and set the **Network Status** to ***Enabled***.
+
+3. In the **API URL** column, click **Copy** to copy the XMTP app HTTPs endpoint along with its API key. It should use the format `https://xmtp-testnet.g.alchemy.com/v2/<apikey>`.
+
+4. You will use this endpoint for the `XMTPD_CONTRACTS_RPC_URL` config option
+
+## Step 2: Register your node
+
+The xmtpd node software is open source, allowing anyone to run a node. However, only registered nodes can join the XMTP testnet.
+
+To enable your node to join the XMTP testnet, register its public key and DNS address on the blockchain.
+
+The node registration process is currently managed by [Ephemera](https://ephemerahq.com/), stewarding the development and adoption of XMTP. To learn more about the XMTP network node operator qualification criteria and selection process, see [XIP-54](https://community.xmtp.org/t/xip-54-xmtp-network-node-operator-qualification-criteria/868).
+
+### **Step 2.1: Get all keys**
+
+If you need a new private key, you can generate one using the xmtpd CLI Docker image. 
+
+```bash
+docker run ghcr.io/xmtp/xmtpd-cli:latest generate-key | jq
+```
+
+Example response:
+
+```json
+{
+  "level": "INFO",
+  "time": "2024-10-15T13:21:14.036-0400",
+  "message": "generated private key",
+  "private-key": "0x7d3cd4989b92c593db9a4b3ac1c2a5d542efad058b2a83e26c3467392b29c6f9",
+  "public-key": "0x03da53968d81f4eb3c9dd8b96617575767ec0cccbd28103b2cfd7f1511bb282d30",
+  "address": "0x9419db765e6b469edc028ffa72ba2944f2bad169"
+}
+```
+
+If you already have a private key, you can extract relevant public details using the xmtpd CLI Docker image.
+
+```bash
+docker run ghcr.io/xmtp/xmtpd-cli:latest get-pub-key --private-key 0xa9b48d687f450ea99a5faaae1be096ddb49487cb28393d3906d7359ede6ea460 | jq
+```
+
+ Example response:
+
+```json
+{
+  "level": "INFO",
+  "time": "2024-10-15T13:21:51.276-0400",
+  "message": "parsed private key",
+  "pub-key": "0x027a64295b98e48682cb77be1b990d4ecf8f1a86badf051df0af123e6fe3790e3f",
+  "address": "0x9419db765e6b469edc028ffa72ba2944f2bad169"
+}
+```
+
+### **Step 2.2: Provide your node public key and address**
+
+Send your node public key and address to `securitycouncil.xmtp.eth`.
+
+Ensure that public key and address values are correct because once registered, they are immutable and cannot be changed.
+
+## Step 3: Set up dependencies
+
+### Check out the repository
+
+```bash
+git clone [email protected]:xmtp/xmtpd-infrastructure.git
+cd xmtpd-infrastructure/helm
+```
+
+### Install the PostgreSQL database
+
+xmtpd requires a PostgreSQL database that's accessible from the Kubernetes cluster. Use a Helm chart or your preferred tool to set up the database.
+
+For example, run:
+
+```bash
+helm install postgres bitnami/postgresql --set auth.postgresPassword=postgres
+```
+
+> [!INFO]
+> For your convenience while testing this flow, this command sets the PostgresSQL database password to `postgres`. Be sure to set a secure password before going live on the XMTP testnet.
+
+You’ll use values in the response to update the `XMTPD_DB_WRITER_CONNECTION_STRING` value in your `xmtpd.yaml` configuration.
+
+### Install the MLS validation service
+
+The MLS validation service is a stateless, horizontally scalable service that xmtpd depends on.
+
+Install it using the Helm chart. To do this, run:
 
-To install, run:
 ```bash
 helm install mls-validation-service mls-validation-service/
 ```
 
-## 3) Installing the XMTPD Node
+## Step 4: Install the xmtpd node
 
-The XMPT daemon depends on the following:
-- a PG database running locally
-- the MLS validation service running locally. XMTP Expects the format `http|https://<DNS|IP>[:port]`
-- a blockchain with the [Nodes contract](https://github.com/xmtp/xmtpd)
-- a private key
-- the private key needs to be registered with the blockchain smart contract and a NodeID has to be issued. For more info see [Onboarding](https://github.com/xmtp/xmtpd/blob/main/doc/onboarding.md)
+### Configure the xmtpd node
+
+Before deploying xmtpd, create a configuration file (`xmtpd.yaml`) to specify the required environment variables. 
+
+Here is a sample configuration:
 
-Create a `xmtpd.yaml` file and fill out all required variables:
 ```yaml
 env:
   secret:
-    XMTPD_DB_WRITER_CONNECTION_STRING: "<postgres://postgres:[email protected]:5432/postgres?sslmode=disable>"
+    XMTPD_DB_WRITER_CONNECTION_STRING: "postgres://<username>:<password>@<host-service>:<port>/<database>?sslmode=disable"
     XMTPD_SIGNER_PRIVATE_KEY: "<private-key>"
     XMTPD_PAYER_PRIVATE_KEY: "<private-key>"
     XMTPD_CONTRACTS_RPC_URL: "https://xmtp-testnet.g.alchemy.com/v2/<apikey>"
-    XMTPD_MLS_VALIDATION_GRPC_ADDRESS: "<http://mls-validation-service.default.svc.cluster.local:50051>"
+    XMTPD_MLS_VALIDATION_GRPC_ADDRESS: "http://mls-validation-service.default.svc.cluster.local:50051"
     XMTPD_CONTRACTS_CHAIN_ID: "241320161"
-    XMTPD_CONTRACTS_NODES_ADDRESS: "<0x390D339A6C0Aa432876B5C898b16287Cacde2A0A>"
-    XMTPD_CONTRACTS_MESSAGES_ADDRESS: "<0x162f2d4d96565437F47bfB7a0BF8AC4FF481Bbf6>"
-    XMTPD_CONTRACTS_IDENTITY_UPDATES_ADDRESS: "<0x00e92e15AB0D7d3aA5c76bceCcE675DcAf311189>"
+    XMTPD_CONTRACTS_NODES_ADDRESS: "0x390D339A6C0Aa432876B5C898b16287Cacde2A0A"
+    XMTPD_CONTRACTS_MESSAGES_ADDRESS: "<0x162f2d4d96565437F47bfB7a0BF8AC4FF481Bbf6"
+    XMTPD_CONTRACTS_IDENTITY_UPDATES_ADDRESS: "0x00e92e15AB0D7d3aA5c76bceCcE675DcAf311189"
     XMTPD_METRICS_ENABLE: "true"
     XMTPD_REFLECTION_ENABLE: "true"
     XMTPD_LOG_LEVEL: "debug"
 ```
 
-Install the helm chart
+Replace placeholder values with actual credentials and configurations:
+
+- The `XMTPD_DB_WRITER_CONNECTION_STRING` is constructed as `postgres://<username>:<password>@<host-service>:<port>/<database>?sslmode=disable`
+    - Replace `<username>` with the username for pg-postgresql. The default value is usually `postgres`.
+    - Replace `<password>` with the password for pg-postgresql. If the password includes a special character, be sure to URL encode it.
+    - Replace `<host-service>` with `<helm-chart-name>-postgresql.<namespace>.svc.cluster.local`. The default value is usually `postgres-postgresql.default.svc.cluster.local`.
+    - Replace `<port>` with the port for pg-postgresql. The default value is usually `5432`.
+    
+    If you are following the setup steps in this document, the full connection string will be: `postgres://postgres:[email protected]:5432/postgres?sslmode=disable`
+- Replace `<apikey>` with your full Alchemy URL    
+- Replace `<private-key>` with the private key for your registered node.
+- Ask the XMTP team to confirm the following address values:
+    - Nodes address
+    - Messages address
+    - Identity updates address
+
+### Install xmtpd
+
+Use Helm to deploy the xmtpd node. In the directory where you created your `xmtpd.yaml` configuration file, run:
+
 ```bash
 helm install xmtpd xmtpd/ -f xmtpd.yaml
 ```
 
-### Validating the installation
+## Step 5: Validate the installation
+
+After installation, verify that all the required pods are running:
+
+```bash
+kubectl get pods
+```
+
+Expected output:
 
-Once you have successfully installed all charts, including a DB, you should see 4 pods running.
-You can confirm via:
 ```bash
-$ kubectl get pods
 NAME                                      READY   STATUS    RESTARTS   AGE
 mls-validation-service-75b6b96f79-kp4jq   1/1     Running   0          6h30m
 pg-postgresql-0                           1/1     Running   0          6h26m
 xmtpd-api-7dd49f6b88-mfwls                1/1     Running   0          4h56m
-xmtpd-sync-7dd49f6b88-dn28d               1/1     Running   0          4h56m
+xmtpd-sync-7dd49f6b88-mfwls               1/1     Running   0          4h56m
 ```
 
-
 ## XMTP Payer Helm Chart Installation
 
 If you are a XMTPD Node operator, you will not need to run the Payer service.
@@ -92,19 +216,18 @@ It does require access to the public internet to read state from the blockchain.
 It also requires write access to all known XMTP nodes in the cluster.
 
 As of XMTPD docker image v0.1.1, all services require the same configuration options even though some might be unused.
-As such, you can reuse the same config that we described in [Step 3](#3-Installing-the-XMTPD-Node).
+As such, you can reuse the same config that we described in [Step 4](#Step-4-Install-the-xmtpd-node).
 
 Install the helm chart
 ```bash
 helm install xmtp-payer xmtp-payer/ -f xmtpd.yaml
 ```
-Once you have successfully installed the chart, you should see 2 pods running.
+Once you have successfully installed the chart, you should see 1 pod running.
 You can confirm via:
 ```bash
 $ kubectl get pods
 NAME                                      READY   STATUS    RESTARTS   AGE
 xmtp-payer-7978dbcb8-mnvxx                1/1     Running   0          9m48s
-xmtp-payer-7978dbcb8-vkwsv                1/1     Running   0          9m48s
 ```
 
 ## Kubernetes Ingress
@@ -148,3 +271,8 @@ ingress:
 ```
 
 For a comprehensive guide on how to terminate TLS in GKE, you can read our [GKE+Let'Encrypt+NGINX Howto](../doc/nginx-cert-gke.md)
+
+## What’s next
+
+> [!TIP]
+> Have feedback or ideas for improving xmtpd? [Open an issue](https://github.com/xmtp/xmtpd/issues) in the xmtpd GitHub repo.