Skip to content

Commit dc3c9ec

Browse files
jhaaaamkysel
andauthored
Update Helm README (#27)
Co-authored-by: Martin Kysel <[email protected]>
1 parent dd0744b commit dc3c9ec

File tree

1 file changed

+171
-43
lines changed

1 file changed

+171
-43
lines changed

helm/README.md

Lines changed: 171 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -1,87 +1,211 @@
1-
# XMTPD Helm Charts
1+
# Install xmtpd on Kubernetes using Helm charts
22

3-
**⚠️ Experimental:** This software is in early development. Expect frequent changes and unresolved issues.
3+
**xmtpd (XMTP daemon)** is software that operators run to provide nodes in the decentralized XMTP network. The responsibilities of an xmtpd node include message storage and replication, censorship resistance, network resilience and recovery, and much more.
44

5-
`xmtpd` (XMTP daemon) is an experimental version of XMTP node software. It is **not** the node software that currently forms the XMTP network.
5+
This repository includes Helm charts that you can use to deploy xmtpd on Kubernetes. Charts are curated application definitions for Helm.
66

7-
This repository includes Helm Charts that can be used to deploy XMTPD in Kubernetes. You may also contribute by submitting enhancement requests if you like. Charts are curated application definitions for Helm. For more information about installing and using Helm, see its
8-
[README.md](https://github.com/helm/helm/tree/master/README.md). To get a quick introduction to Helm Charts see this [chart document](https://github.com/helm/helm/blob/master/docs/charts.md). For more information on using Helm, refer to the [Helm's documentation](https://github.com/kubernetes/helm#docs).
7+
- To learn how to install Helm, see the Helm [README](https://github.com/helm/helm/tree/master/README.md).
98

10-
## XMTPD Helm Chart Installation
9+
- To learn how to use Helm, see the Helm [Quickstart Guide](https://helm.sh/docs/intro/quickstart/).
1110

12-
To add the XMTPD charts to your local helm repository, clone this repository locally.
11+
- To learn more about Helm charts, see [Charts](https://helm.sh/docs/topics/charts/).
1312

14-
Eventually XMTP will provide a public Helm Charts release.
13+
## What does the xmtpd Helm chart contain?
1514

16-
## 0) Alchemy Account
15+
xmtpd is composed of two key deployments:
1716

18-
To run a node, you will need to have an [Alchemy Account](https://www.alchemy.com/).
19-
You can follow their docs on the [API Key Creation](https://docs.alchemy.com/docs/alchemy-quickstart-guide#1key-create-an-alchemy-api-key).
17+
1. **Synchronization service**
18+
19+
Handles data replication from other nodes to the local node, ensuring consistency across the network.
20+
21+
2. **API service**
22+
23+
Provides a client-facing gRPC endpoint for both nodes and client applications (like Converse) to interact with the system.
24+
25+
Both services support multiple replicas for high availability. However, adding replicas doesn't enhance the overall throughput of the service. As such, we recommend running exactly two replicas of each.
2026

21-
Once logged in, you can find and enable the [XMTP Chain](https://dashboard.alchemy.com/chains/xmtp).
22-
Find and copy the XMTP app HTTP endpoint along with its API key. It should be in the format `https://xmtp-testnet.g.alchemy.com/v2/<apikey>`
27+
## Prerequisites
2328

24-
## 1) Dependencies
29+
Before diving into the installation process, ensure you have the following:
2530

26-
XMTPD needs a Postgres database to be running accessible from the cluster.
27-
For example, you can use [Bitnami PG Helm Charts](https://bitnami.com/stack/postgresql/helm), or any other tooling to provision a Postgres database.
31+
1. **Kubernetes cluster**: A running Kubernetes cluster with sufficient resources. To learn more, see [Kubernetes documentation](https://kubernetes.io/docs/home/). If you want to test the charts locally, you can use [Docker Desktop](https://www.docker.com/get-started/)
32+
2. **Helm**: A package manager for Kubernetes. To learn more, see [Helm documentation](https://helm.sh/docs/).
33+
```bash
34+
brew install helm
35+
```
36+
3. **PostgreSQL database**: xmtpd relies on a PostgreSQL database. Tools like Bitnami’s [PostgreSQL Helm Charts](https://github.com/bitnami/charts/tree/main/bitnami/postgresql) can simplify database provisioning. xmtpd has been tested with Postgres versions 13 and newer. To learn more, see [PostgreSQL documentation](https://www.postgresql.org/docs/).
37+
```bash
38+
helm repo add bitnami https://charts.bitnami.com/bitnami
39+
helm repo update
40+
```
41+
4. Clone the [xmtpd-infrastructure](https://github.com/xmtp/xmtpd-infrastructure) repo.
2842

29-
## 2) Installing MLS Validation Service
43+
## Step 1. Get an Alchemy account
3044

31-
The XMTP MLS Validation Service does not have any external dependencies.
32-
It is a stateless horizontally-scalable service.
45+
To run xmtpd, you need an Alchemy account.
46+
47+
1. See [Create an Alchemy API Key](https://docs.alchemy.com/docs/alchemy-quickstart-guide#1key-create-an-alchemy-api-key) and log in to Alchemy.
48+
49+
2. Go to the [XMTP Chain page](https://dashboard.alchemy.com/chains/xmtp) and set the **Network Status** to ***Enabled***.
50+
51+
3. In the **API URL** column, click **Copy** to copy the XMTP app HTTPs endpoint along with its API key. It should use the format `https://xmtp-testnet.g.alchemy.com/v2/<apikey>`.
52+
53+
4. You will use this endpoint for the `XMTPD_CONTRACTS_RPC_URL` config option
54+
55+
## Step 2: Register your node
56+
57+
The xmtpd node software is open source, allowing anyone to run a node. However, only registered nodes can join the XMTP testnet.
58+
59+
To enable your node to join the XMTP testnet, register its public key and DNS address on the blockchain.
60+
61+
The node registration process is currently managed by [Ephemera](https://ephemerahq.com/), stewarding the development and adoption of XMTP. To learn more about the XMTP network node operator qualification criteria and selection process, see [XIP-54](https://community.xmtp.org/t/xip-54-xmtp-network-node-operator-qualification-criteria/868).
62+
63+
### **Step 2.1: Get all keys**
64+
65+
If you need a new private key, you can generate one using the xmtpd CLI Docker image.
66+
67+
```bash
68+
docker run ghcr.io/xmtp/xmtpd-cli:latest generate-key | jq
69+
```
70+
71+
Example response:
72+
73+
```json
74+
{
75+
"level": "INFO",
76+
"time": "2024-10-15T13:21:14.036-0400",
77+
"message": "generated private key",
78+
"private-key": "0x7d3cd4989b92c593db9a4b3ac1c2a5d542efad058b2a83e26c3467392b29c6f9",
79+
"public-key": "0x03da53968d81f4eb3c9dd8b96617575767ec0cccbd28103b2cfd7f1511bb282d30",
80+
"address": "0x9419db765e6b469edc028ffa72ba2944f2bad169"
81+
}
82+
```
83+
84+
If you already have a private key, you can extract relevant public details using the xmtpd CLI Docker image.
85+
86+
```bash
87+
docker run ghcr.io/xmtp/xmtpd-cli:latest get-pub-key --private-key 0xa9b48d687f450ea99a5faaae1be096ddb49487cb28393d3906d7359ede6ea460 | jq
88+
```
89+
90+
Example response:
91+
92+
```json
93+
{
94+
"level": "INFO",
95+
"time": "2024-10-15T13:21:51.276-0400",
96+
"message": "parsed private key",
97+
"pub-key": "0x027a64295b98e48682cb77be1b990d4ecf8f1a86badf051df0af123e6fe3790e3f",
98+
"address": "0x9419db765e6b469edc028ffa72ba2944f2bad169"
99+
}
100+
```
101+
102+
### **Step 2.2: Provide your node public key and address**
103+
104+
Send your node public key and address to `securitycouncil.xmtp.eth`.
105+
106+
Ensure that public key and address values are correct because once registered, they are immutable and cannot be changed.
107+
108+
## Step 3: Set up dependencies
109+
110+
### Check out the repository
111+
112+
```bash
113+
git clone [email protected]:xmtp/xmtpd-infrastructure.git
114+
cd xmtpd-infrastructure/helm
115+
```
116+
117+
### Install the PostgreSQL database
118+
119+
xmtpd requires a PostgreSQL database that's accessible from the Kubernetes cluster. Use a Helm chart or your preferred tool to set up the database.
120+
121+
For example, run:
122+
123+
```bash
124+
helm install postgres bitnami/postgresql --set auth.postgresPassword=postgres
125+
```
126+
127+
> [!INFO]
128+
> For your convenience while testing this flow, this command sets the PostgresSQL database password to `postgres`. Be sure to set a secure password before going live on the XMTP testnet.
129+
130+
You’ll use values in the response to update the `XMTPD_DB_WRITER_CONNECTION_STRING` value in your `xmtpd.yaml` configuration.
131+
132+
### Install the MLS validation service
133+
134+
The MLS validation service is a stateless, horizontally scalable service that xmtpd depends on.
135+
136+
Install it using the Helm chart. To do this, run:
33137
34-
To install, run:
35138
```bash
36139
helm install mls-validation-service mls-validation-service/
37140
```
38141
39-
## 3) Installing the XMTPD Node
142+
## Step 4: Install the xmtpd node
40143
41-
The XMPT daemon depends on the following:
42-
- a PG database running locally
43-
- the MLS validation service running locally. XMTP Expects the format `http|https://<DNS|IP>[:port]`
44-
- a blockchain with the [Nodes contract](https://github.com/xmtp/xmtpd)
45-
- a private key
46-
- the private key needs to be registered with the blockchain smart contract and a NodeID has to be issued. For more info see [Onboarding](https://github.com/xmtp/xmtpd/blob/main/doc/onboarding.md)
144+
### Configure the xmtpd node
145+
146+
Before deploying xmtpd, create a configuration file (`xmtpd.yaml`) to specify the required environment variables.
147+
148+
Here is a sample configuration:
47149
48-
Create a `xmtpd.yaml` file and fill out all required variables:
49150
```yaml
50151
env:
51152
secret:
52-
XMTPD_DB_WRITER_CONNECTION_STRING: "<postgres://postgres:[email protected]:5432/postgres?sslmode=disable>"
153+
XMTPD_DB_WRITER_CONNECTION_STRING: "postgres://<username>:<password>@<host-service>:<port>/<database>?sslmode=disable"
53154
XMTPD_SIGNER_PRIVATE_KEY: "<private-key>"
54155
XMTPD_PAYER_PRIVATE_KEY: "<private-key>"
55156
XMTPD_CONTRACTS_RPC_URL: "https://xmtp-testnet.g.alchemy.com/v2/<apikey>"
56-
XMTPD_MLS_VALIDATION_GRPC_ADDRESS: "<http://mls-validation-service.default.svc.cluster.local:50051>"
157+
XMTPD_MLS_VALIDATION_GRPC_ADDRESS: "http://mls-validation-service.default.svc.cluster.local:50051"
57158
XMTPD_CONTRACTS_CHAIN_ID: "241320161"
58-
XMTPD_CONTRACTS_NODES_ADDRESS: "<0x390D339A6C0Aa432876B5C898b16287Cacde2A0A>"
59-
XMTPD_CONTRACTS_MESSAGES_ADDRESS: "<0x162f2d4d96565437F47bfB7a0BF8AC4FF481Bbf6>"
60-
XMTPD_CONTRACTS_IDENTITY_UPDATES_ADDRESS: "<0x00e92e15AB0D7d3aA5c76bceCcE675DcAf311189>"
159+
XMTPD_CONTRACTS_NODES_ADDRESS: "0x390D339A6C0Aa432876B5C898b16287Cacde2A0A"
160+
XMTPD_CONTRACTS_MESSAGES_ADDRESS: "<0x162f2d4d96565437F47bfB7a0BF8AC4FF481Bbf6"
161+
XMTPD_CONTRACTS_IDENTITY_UPDATES_ADDRESS: "0x00e92e15AB0D7d3aA5c76bceCcE675DcAf311189"
61162
XMTPD_METRICS_ENABLE: "true"
62163
XMTPD_REFLECTION_ENABLE: "true"
63164
XMTPD_LOG_LEVEL: "debug"
64165
```
65166
66-
Install the helm chart
167+
Replace placeholder values with actual credentials and configurations:
168+
169+
- The `XMTPD_DB_WRITER_CONNECTION_STRING` is constructed as `postgres://<username>:<password>@<host-service>:<port>/<database>?sslmode=disable`
170+
- Replace `<username>` with the username for pg-postgresql. The default value is usually `postgres`.
171+
- Replace `<password>` with the password for pg-postgresql. If the password includes a special character, be sure to URL encode it.
172+
- Replace `<host-service>` with `<helm-chart-name>-postgresql.<namespace>.svc.cluster.local`. The default value is usually `postgres-postgresql.default.svc.cluster.local`.
173+
- Replace `<port>` with the port for pg-postgresql. The default value is usually `5432`.
174+
175+
If you are following the setup steps in this document, the full connection string will be: `postgres://postgres:[email protected]:5432/postgres?sslmode=disable`
176+
- Replace `<apikey>` with your full Alchemy URL
177+
- Replace `<private-key>` with the private key for your registered node.
178+
- Ask the XMTP team to confirm the following address values:
179+
- Nodes address
180+
- Messages address
181+
- Identity updates address
182+
183+
### Install xmtpd
184+
185+
Use Helm to deploy the xmtpd node. In the directory where you created your `xmtpd.yaml` configuration file, run:
186+
67187
```bash
68188
helm install xmtpd xmtpd/ -f xmtpd.yaml
69189
```
70190
71-
### Validating the installation
191+
## Step 5: Validate the installation
192+
193+
After installation, verify that all the required pods are running:
194+
195+
```bash
196+
kubectl get pods
197+
```
198+
199+
Expected output:
72200
73-
Once you have successfully installed all charts, including a DB, you should see 4 pods running.
74-
You can confirm via:
75201
```bash
76-
$ kubectl get pods
77202
NAME READY STATUS RESTARTS AGE
78203
mls-validation-service-75b6b96f79-kp4jq 1/1 Running 0 6h30m
79204
pg-postgresql-0 1/1 Running 0 6h26m
80205
xmtpd-api-7dd49f6b88-mfwls 1/1 Running 0 4h56m
81-
xmtpd-sync-7dd49f6b88-dn28d 1/1 Running 0 4h56m
206+
xmtpd-sync-7dd49f6b88-mfwls 1/1 Running 0 4h56m
82207
```
83208
84-
85209
## XMTP Payer Helm Chart Installation
86210
87211
If you are a XMTPD Node operator, you will not need to run the Payer service.
@@ -92,19 +216,18 @@ It does require access to the public internet to read state from the blockchain.
92216
It also requires write access to all known XMTP nodes in the cluster.
93217
94218
As of XMTPD docker image v0.1.1, all services require the same configuration options even though some might be unused.
95-
As such, you can reuse the same config that we described in [Step 3](#3-Installing-the-XMTPD-Node).
219+
As such, you can reuse the same config that we described in [Step 4](#Step-4-Install-the-xmtpd-node).
96220
97221
Install the helm chart
98222
```bash
99223
helm install xmtp-payer xmtp-payer/ -f xmtpd.yaml
100224
```
101-
Once you have successfully installed the chart, you should see 2 pods running.
225+
Once you have successfully installed the chart, you should see 1 pod running.
102226
You can confirm via:
103227
```bash
104228
$ kubectl get pods
105229
NAME READY STATUS RESTARTS AGE
106230
xmtp-payer-7978dbcb8-mnvxx 1/1 Running 0 9m48s
107-
xmtp-payer-7978dbcb8-vkwsv 1/1 Running 0 9m48s
108231
```
109232
110233
## Kubernetes Ingress
@@ -148,3 +271,8 @@ ingress:
148271
```
149272
150273
For a comprehensive guide on how to terminate TLS in GKE, you can read our [GKE+Let'Encrypt+NGINX Howto](../doc/nginx-cert-gke.md)
274+
275+
## What’s next
276+
277+
> [!TIP]
278+
> Have feedback or ideas for improving xmtpd? [Open an issue](https://github.com/xmtp/xmtpd/issues) in the xmtpd GitHub repo.

0 commit comments

Comments
 (0)