Skip to content

Commit 38d8d28

Browse files
authored
Merge pull request #7543 from tangledbytes/utkarsh/improv/archive-docs
Update Spectrum Archive deployment guide
2 parents f24daa5 + a6e34e7 commit 38d8d28

File tree

1 file changed

+54
-5
lines changed

1 file changed

+54
-5
lines changed

src/deploy/spectrum_archive/deployment_guide.md

Lines changed: 54 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ This deployment guide goes over the steps that needs to be taken to get NooBaa r
55
NooBaa has the following prerequisites and expects them to be in place before proceeding with its installation and deployment.
66
1. Host is a Red Hat OS like Centos/RHEL.
77
2. Spectrum Scale along with Spectrum Archive must be in installed on the host machine.
8-
3. libboost RPM packages must be already installed. In particular, `boost-system` and `boost-thread` packages are required. Without them in place, NooBaa installation will fail.
8+
3. libboost RPM packages must be already installed. In particular, `boost-system` and `boost-thread` packages are required. Without them in place, NooBaa installation will fail. Can be checked by running `rpm -q boost-system boost-thread`.
99

1010
## NooBaa Installation
1111
NooBaa is packaged as a RPM which needs to be installed in order to be able to use NooBaa.
@@ -20,6 +20,8 @@ NooBaa is packaged as a RPM which needs to be installed in order to be able to u
2020
NooBaa needs some configurations to be in place before we start up the NooBaa process and it is important to ensure that this is done before starting up the service.
2121

2222
### Configure NooBaa User
23+
Before proceeding, please ensure there are no stale items in `/etc/noobaa.conf.d` from a previous (not in use) NooBaa setup. If there is this directory with stale data then remove it by runnign `rm -r /etc/noobaa.conf.d`.
24+
2325
In order to be able to access NooBaa, the user should create a account. This can be done in the following way.
2426
```console
2527
$ cd /usr/local/noobaa-core
@@ -41,7 +43,7 @@ $ bin/node src/cmd/manage_nsfs.js account add --email [email protected] --access_
4143

4244
### Configure NooBaa
4345
```console
44-
$ cat >/usr/local/noobaa-core/.env <<EOF
46+
$ cat >/etc/noobaa.conf.d/.env <<EOF
4547
ENDPOINT_PORT=80
4648
ENDPOINT_SSL_PORT=443
4749
ENDPOINT_FORKS=8
@@ -93,13 +95,58 @@ $ systemctl status noobaa_nsfs # You should see status "Active" in green color
9395
Now that NooBaa has been installed and is active, we can test out the deployment.
9496
These AWS commands will read `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` from the environment, ensure that these are available in the environment and should be the same that we used in [configure NooBaa user](#configure-noobaa-user).
9597

98+
### Basic Testing
9699
```console
97100
$ aws s3 --endpoint https://localhost:443 --no-verify-ssl mb s3://first.bucket # Create a bucket named first.bucket
98101
make_bucket: first.bucket
99102
$ aws s3 --endpoint https://localhost:443 --no-verify-ssl ls # List all of the buckets
100103
2023-10-05 21:18:45 first.bucket
101104
```
102105

106+
### Test `GLACIER` Storage Class
107+
```console
108+
$ # Upload "somefile" to "first.bucket" bucket that we created in the previous step. Here "--storage-class GLACIER"
109+
$ # tells the S3 server to store the file onto tape instead of disk.
110+
$ aws s3 --endpoint https://localhost:443 --no-verify-ssl cp somefile s3://first.bucket --storage-class GLACIER
111+
upload: ./somefile to s3://first.bucket/somefile
112+
$ # Trying to copy file back without restoring will fail because the file is in GLACIER storage class
113+
$ aws s3 cp --endpoint https://localhost:443 --no-verify-ssl s3://first.bucket/somefile somefile.cp
114+
warning: Skipping file s3://first.bucket/somefile. Object is of storage class GLACIER. Unable to perform download operations on GLACIER objects. You must restore the object to be able to perform the operation. See aws s3 download help for additional parameter options to ignore or force these transfers.
115+
$ # Let's issue a restore request. This will return immediately but will take about 15 mins for the restore to finish.
116+
$ aws s3api --endpoint https://localhost:443 --no-verify-ssl restore-object --bucket first.bucket --key somefile --restore-request '{"Days": 1}'
117+
$ # We can check the restore request status
118+
$ aws s3api head-object --endpoint https://localhost:443 --no-verify-ssl --bucket first.bucket --key somefile
119+
{
120+
"AcceptRanges": "bytes",
121+
"Restore": "ongoing-request=\"true\"",
122+
"LastModified": "2023-10-12T11:01:06+00:00",
123+
"ContentLength": 8883,
124+
"ETag": "\"mtime-cw6eqqo588w0-ino-5n5k\"",
125+
"ContentType": "application/octet-stream",
126+
"Metadata": {
127+
"storage_class": "GLACIER"
128+
},
129+
"StorageClass": "GLACIER"
130+
}
131+
$ # Checking again after 15 minutes - The object has been restored
132+
$ aws s3api head-object --endpoint https://localhost:443 --no-verify-ssl --bucket first.bucket --key somefile
133+
{
134+
"AcceptRanges": "bytes",
135+
"Restore": "ongoing-request=\"false\", expiry-date=\"Thu, 26 Oct 2023 00:00:00 GMT\"",
136+
"LastModified": "2023-10-12T11:01:06+00:00",
137+
"ContentLength": 8883,
138+
"ETag": "\"mtime-cw6eqqo588w0-ino-5n5k\"",
139+
"ContentType": "application/octet-stream",
140+
"Metadata": {
141+
"storage_class": "GLACIER"
142+
},
143+
"StorageClass": "GLACIER"
144+
}
145+
$ # Once the restore succeeds we can download the file
146+
$ aws s3 cp --endpoint https://localhost:443 --no-verify-ssl s3://first.bucket/somefile somefile.cp
147+
download: s3://first.bucket/somefile to ./somefile.cp
148+
```
149+
103150
## Log and Logrotate
104151
Noobaa logs are configured using rsyslog and logrotate. RPM will configure rsyslog and logrotate if both are already running.
105152

@@ -108,9 +155,9 @@ Rsyslog status check
108155
systemctl status rsyslog
109156
```
110157

111-
Noobaa logs are pushed to `var/log/noobaa.log` and the log is rotated and compressed daily.
158+
Noobaa logs are pushed to `/var/log/noobaa.log` and the log is rotated and compressed daily.
112159

113-
Verify the rsyslog and logrotate rpm configuration is complete by checking the files `etc/rsyslog.d/noobaa_syslog.conf` and `etc/rsyslog.d/noobaa_rsyslog.conf` for rsyslog and `etc/logrotate.d/noobaa/logrotate_noobaa.conf` for logrotate.These files contain the noobaa specific configuration for rsyslog and logrotate.
160+
Verify the rsyslog and logrotate rpm configuration is complete by checking the files `/etc/rsyslog.d/noobaa_syslog.conf` and `/etc/rsyslog.d/noobaa_rsyslog.conf` for rsyslog and `/etc/logrotate.d/noobaa/logrotate_noobaa.conf` for logrotate.These files contain the noobaa specific configuration for rsyslog and logrotate.
114161

115162
Rotate the logs manually.
116163

@@ -122,4 +169,6 @@ logrotate /etc/logrotate.d/noobaa/logrotate_noobaa.conf
122169
- What happens if I forget the credentials used to generate NooBaa User?
123170
- You can find all of the NooBaa accounts details here: `/etc/noobaa.conf.d/accounts`.
124171
- How do I add new users?
125-
- You can repeat the command that we ran in the section [configure NooBaa User](#configure-noobaa-user). You need to make sure that the access key **must not be reused**.
172+
- You can repeat the command that we ran in the section [configure NooBaa User](#configure-noobaa-user). You need to make sure that the access key **must not be reused**.
173+
- My migrations/restores aren't working!
174+
- You can find the migration/restore related logs in your crontab logs.

0 commit comments

Comments
 (0)