Skip to content

Commit c1259a8

Browse files
authored
feat: add guide for manually deployment (#2439)
1 parent 9b8a16a commit c1259a8

File tree

13 files changed

+910
-4
lines changed

13 files changed

+910
-4
lines changed

docs/en/guides/10-deploy/01-deploy/02-production/01-preparing-storage.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Go to the AWS Management Console, select the S3 service, enter the bucket name,
3333

3434
### Bucket Lifecycle Policies
3535

36-
Lifecyle rule is needed when Bucket Versioning is enabled. You can configure lifecycle policies to automatically delete old versions of objects or transition objects to different storage classes.
36+
Lifecycle rule is needed when Bucket Versioning is enabled. You can configure lifecycle policies to automatically delete old versions of objects or transition objects to different storage classes.
3737

3838
- Configure lifecycle rule to delete old versions of objects.
3939

docs/en/guides/10-deploy/01-deploy/02-production/10-metasrv-deploy.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Before you start, make sure you have completed the following preparations:
1717

1818
- Plan your deployment. This topic is based on the following cluster deployment plan, which involves setting up a meta cluster comprising three meta nodes and a query cluster consisting of two query nodes:
1919

20-
| Node # | IP Address | Leader Meta Node? | Tenant ID | Cluster ID |
20+
| Node # | IP Address | First Meta Node? | Tenant ID | Cluster ID |
2121
| ------- | ----------------- | ----------------- | --------- | ---------- |
2222
| Meta-1 | 172.16.125.128/24 | Yes | - | - |
2323
| Meta-2 | 172.16.125.129/24 | No | - | - |
Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
---
2+
title: Prepare Package Environment
3+
---
4+
5+
## Prerequisites
6+
7+
- A Linux-based operating system
8+
- `wget` or `curl` for downloading files
9+
- `tar` for extracting the package
10+
- `sudo` privileges for system-wide installation
11+
12+
## Check System Architecture
13+
14+
1. Check your system architecture:
15+
```bash
16+
uname -m
17+
```
18+
19+
The output will help you determine which package to download:
20+
- If the output is `x86_64`, download the x86_64 package
21+
- If the output is `aarch64`, download the aarch64 package
22+
23+
## Download the Package
24+
25+
1. Visit the [Databend GitHub Releases](https://github.com/datafuselabs/databend/releases) page.
26+
27+
2. Choose the latest stable release version. For example, if you want to install version v1.2.755-nightly, you'll need to download:
28+
- `databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz` for x86_64 Linux systems
29+
- `databend-v1.2.755-nightly-aarch64-unknown-linux-gnu.tar.gz` for aarch64 Linux systems
30+
31+
3. Download the package using wget (replace `v1.2.755-nightly` with your desired version):
32+
```bash
33+
wget https://github.com/datafuselabs/databend/releases/download/v1.2.755-nightly/databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz
34+
```
35+
36+
Or using curl (replace `v1.2.755-nightly` with your desired version):
37+
```bash
38+
curl -L -O https://github.com/datafuselabs/databend/releases/download/v1.2.755-nightly/databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz
39+
```
40+
41+
Note: Make sure to replace `v1.2.755-nightly` in both the URL and filename with your desired version number.
42+
43+
## Extract the Package
44+
45+
1. Extract the package in the current directory:
46+
```bash
47+
tar xzf databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz
48+
```
49+
50+
## Verify the Installation
51+
52+
1. Check the extracted files:
53+
```bash
54+
ls --tree
55+
```
56+
57+
You should see the following directory structure:
58+
```
59+
.
60+
├── bin
61+
│ ├── bendsql
62+
│ ├── databend-bendsave
63+
│ ├── databend-meta
64+
│ ├── databend-metactl
65+
│ └── databend-query
66+
├── configs
67+
│ ├── databend-meta.toml
68+
│ └── databend-query.toml
69+
├── readme.txt
70+
├── scripts
71+
│ ├── postinstall.sh
72+
│ └── preinstall.sh
73+
└── systemd
74+
├── databend-meta.default
75+
├── databend-meta.service
76+
├── databend-query.default
77+
└── databend-query.service
78+
```
79+
80+
2. Verify the binaries are executable:
81+
```bash
82+
./bin/databend-meta --version
83+
./bin/databend-metactl --version
84+
./bin/databend-query --version
85+
./bin/bendsql --version
86+
```
87+
88+
## Create Databend User
89+
90+
1. Run the preinstall script to create the databend user and group:
91+
```bash
92+
sudo ./scripts/preinstall.sh
93+
```
94+
95+
This script will:
96+
- Create a `databend` user and group if they don't exist
97+
- Set up necessary system configurations
98+
- Create required directories with proper permissions
99+
100+
## Next Steps
101+
102+
Now that you have prepared the environment, you can proceed to:
103+
- [Deploy Meta Service](02-deploy-metasrv.md)
104+
- [Deploy Query Service](03-deploy-query.md)
Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
---
2+
title: Deploy Meta Service
3+
---
4+
5+
## Overview
6+
7+
The Meta Service is a critical component of Databend that manages metadata and cluster coordination. This guide will walk you through the process of deploying a Meta Service node.
8+
9+
## Prerequisites
10+
11+
- Completed the [Prepare Package Environment](01-prepare.md) steps
12+
- Have the extracted Databend package ready
13+
- Have sudo privileges
14+
15+
## Install Binary Files
16+
17+
1. Copy the Meta Service binary to the system binary directory:
18+
```bash
19+
sudo cp bin/databend-meta /usr/bin/
20+
sudo chmod +x /usr/bin/databend-meta
21+
```
22+
23+
2. Copy the Meta Service control tool binary:
24+
```bash
25+
sudo cp bin/databend-metactl /usr/bin/
26+
sudo chmod +x /usr/bin/databend-metactl
27+
```
28+
29+
## Configure Meta Service
30+
31+
1. Navigate to the extracted package directory and copy the default configuration:
32+
```bash
33+
sudo mkdir -p /etc/databend
34+
sudo cp configs/databend-meta.toml /etc/databend/databend-meta.toml
35+
```
36+
37+
2. Edit the configuration file:
38+
```bash
39+
sudo vim /etc/databend/databend-meta.toml
40+
```
41+
42+
The default configuration looks like this:
43+
```toml
44+
admin_api_address = "0.0.0.0:28002"
45+
grpc_api_address = "0.0.0.0:9191"
46+
grpc_api_advertise_host = "localhost" # change this
47+
48+
[log]
49+
[log.file]
50+
level = "WARN"
51+
format = "text"
52+
dir = "/var/log/databend"
53+
54+
[raft_config]
55+
id = 0 # keep this as 0 for single-node deployment or first node in cluster
56+
raft_dir = "/var/lib/databend/raft"
57+
raft_api_port = 28004
58+
raft_listen_host = "0.0.0.0"
59+
raft_advertise_host = "localhost" # change this
60+
single = true # keep this as true for single-node deployment or first node in cluster
61+
```
62+
63+
Modify the following settings based on your environment:
64+
- `grpc_api_advertise_host`: The hostname or IP address for gRPC communication, should be the same as the host name or IP address of the machine.
65+
- `raft_advertise_host`: The hostname or IP address that other nodes will use to connect, should be the same as the host name or IP address of the machine.
66+
67+
## Set Up Systemd Service
68+
69+
1. Copy the systemd service file:
70+
```bash
71+
sudo cp systemd/databend-meta.service /etc/systemd/system/
72+
```
73+
74+
2. Copy the default environment file:
75+
```bash
76+
sudo cp systemd/databend-meta.default /etc/default/databend-meta
77+
```
78+
79+
3. Edit the environment file (Optional):
80+
```bash
81+
sudo vim /etc/default/databend-meta
82+
```
83+
84+
Set the following variables when needed (Optional):
85+
```bash
86+
RUST_BACKTRACE=1 # enable backtrace for debugging
87+
SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt # set the path to the CA certificate file if you are using custom CA certificate
88+
```
89+
90+
4. Reload systemd to recognize the new service:
91+
```bash
92+
sudo systemctl daemon-reload
93+
```
94+
95+
5. Enable the service to start on boot:
96+
```bash
97+
sudo systemctl enable databend-meta
98+
```
99+
100+
## Start Meta Service
101+
102+
1. Start the Meta Service:
103+
```bash
104+
sudo systemctl start databend-meta
105+
```
106+
107+
2. Check the service status:
108+
```bash
109+
sudo systemctl status databend-meta
110+
```
111+
112+
3. View the logs:
113+
```bash
114+
sudo journalctl -u databend-meta -f
115+
```
116+
117+
## Verify Meta Service
118+
119+
1. Check if the Meta Service is listening on the configured ports:
120+
```bash
121+
sudo netstat -tulpn | grep databend-meta
122+
```
123+
124+
2. Test the admin API endpoint:
125+
```bash
126+
curl http://127.0.0.1:28002/v1/health
127+
```
128+
129+
You should receive a response indicating the service is healthy.
130+
131+
3. Check the Meta Service status using metactl:
132+
```bash
133+
databend-metactl status
134+
```
135+
136+
You should see the current status of the Meta Service, including:
137+
- Node ID
138+
- Raft status
139+
- Leader information
140+
- Cluster configuration
141+
142+
## Troubleshooting
143+
144+
If you encounter issues:
145+
146+
1. Check the service status:
147+
```bash
148+
sudo systemctl status databend-meta
149+
```
150+
151+
2. View the logs for detailed error messages:
152+
```bash
153+
# View systemd logs
154+
sudo journalctl -u databend-meta -f
155+
156+
# View log files in /var/log/databend
157+
sudo tail -f /var/log/databend/databend-meta-*.log
158+
```
159+
160+
3. Common issues and solutions:
161+
- Permission denied: Ensure the databend user has proper permissions in previous steps
162+
- Port already in use: Check if another service is using the configured ports
163+
- Configuration errors: Verify the configuration file syntax and paths
164+
165+
## Next Steps
166+
167+
Now that you have deployed the Meta Service, you can proceed to:
168+
- [Deploy Query Service](03-deploy-query.md)
169+
- [Scale Meta Service Nodes](04-scale-metasrv.md) (for multi-node deployment)

0 commit comments

Comments
 (0)