Dtspo 23334 documentation (#386)

* Added documenation for Atlassian

* image

* added some changes

* added some spell checks

* further changes

* added known words

Author: cpareek

.github/actions/spelling/expect.txt

@@ -1,33 +1,49 @@
+ACNF
+ACRD
+AJRA
+atl
+atlassian
+atlassiannonprod
+atlasssian
 Bance
 cdef
 Chirag
 CVP
+Datlassian
 deevanapalli
 devrot
+dpdb
 dsl
 EDB
 Enda
 geolocationfile
 GEOLOCATIONUPDATES
+gluster
 IMZ
 Jacomb
 Jenkinsfiles
 jfmd
 jfrou
 jfrt
+jira
 journalctl
 kalyan
 mdv
 MDVADMVPNHA
 MDVDMZJUMPL
 NONLINE
+NSGs
 paramatised
 pmd
 prd
+PRDATL
 prepatch
+privatekeyfilename
 PRX
 pvc
 Qube
+recoverypoint
+Resouce
 Rns
 ruxit
 SBZ
@@ -39,3 +55,4 @@ TTLs
 utilisation
 Virtualbox
 virtualwan
+VNETs

source/Services/atlassian.html.md.erb

@@ -0,0 +1,198 @@
+---
+title: Atlassian Apps - Jira, Confluence, Crowd and GlusterFS
+last_reviewed_on: 2025-01-17
+review_in: 12 months
+weight: 159
+---
+
+# <%= current_page.data.title %>
+
+We have deployed Atlassian apps Jira, Confluence, Crowd on the staging environment. The [repository](https://github.com/hmcts/atlassian-infrastructure) has all the IaC as well as automation scripts.
+
+Our goal was to create replica of the production atlassian environment on the staging environment.
+
+This documentation provides a detailed outline of the procedure, offering guidance for anyone tasked with rolling out atlassian apps from scratch.
+
+## Prerequisites
+
+The atlassian production environment is deployed on the [MOJ DCD Atlassian LVE](https://portal.azure.com/#@HMCTS.NET/resource/subscriptions/79898897-729c-41a0-a5ca-53c764839d95/overview) subscription and the staging environment is deployed on the [MOJ DCD Atlassian NLE](https://portal.azure.com/#@HMCTS.NET/resource/subscriptions/b7d2bd5f-b744-4acc-9c73-e068cec2e8d8/overview) subscription. so make sure you have access to these subscriptions as contributor.
+
+
+## Existing Production Setup
+
+Please take a look at this [documentation](https://tools.hmcts.net/confluence/pages/viewpage.action?spaceKey=DTSPO&title=ODP+0002+-+Atlassian+Interim+Hosting+Deployment+Approach) on confluence which has detailed list of VMs and existing setup and the diagram.
+
+There are total 9 VMs in the production environment, 3 VMs for Jira, 2 VMs for Confluence, 1 VM for Crowd and 3 VMs for GlusterFS.
+
+All the VMs are on this resource group [RG-PRD-ATL-INT-01](https://portal.azure.com/#@HMCTS.NET/resource/subscriptions/79898897-729c-41a0-a5ca-53c764839d95/resourceGroups/RG-PRD-ATL-INT-01/overview).
+
+#### Jira VMs
+* PRDATL01AJRA01.cp.cjs.hmcts.net
+* PRDATL01AJRA02.cp.cjs.hmcts.net
+* PRDATL01AJRA03.cp.cjs.hmcts.net
+
+#### Confluence VMs
+* PRDATL01ACNF02.cp.cjs.hmcts.net
+* PRDATL01ACNF04.cp.cjs.hmcts.net
+
+#### Crowd VM
+* PRDATL01ACRD01.cp.cjs.hmcts.net
+
+#### GlusterFS VMs
+* PRDATL01DGST01.cp.cjs.hmcts.net
+* PRDATL01DGST02.cp.cjs.hmcts.net
+* PRDATL01DGST03.cp.cjs.hmcts.net
+
+The database is available on the [RG-PRD-ATL-01](https://portal.azure.com/#@HMCTS.NET/resource/subscriptions/79898897-729c-41a0-a5ca-53c764839d95/resourceGroups/RG-PRD-ATL-01) resource group.
+
+#### Postgres Single Server
+ps-prd-atl-dpdb
+
+Please note that there are other important resources like Application Gateways, Load Balancer, NSGs, VNETs, etc. which are not mentioned here.
+
+## Approach for Staging deployment
+
+We decided to use the same setup as production for the staging environment. We have first of all used backup copies of the above VMs to deploy the VMs on staging. We restored a backup of the production database to a new staging single server.
+
+In order to restore VMs from the Backup and give them new names, we have used the following script:
+
+```bash
+az account set --subscription 79898897-729c-41a0-a5ca-53c764839d95
+
+TargetSubnetName=<TargetSubnetName> # eg "atlassian-int-subnet-dat"
+TargetVmName=<TargetVmName> # eg "atlassian-nonprod-gluster-03"
+ItemName=<ItemName> # eg"PRDATL01DGST03.cp.cjs.hmcts.net"
+
+VaultName="BK-PRD-ATL-INT-01"
+SourceResourceGroup="RG-PRD-ATL-INT-01"
+SourceSubscription="79898897-729c-41a0-a5ca-53c764839d95"
+TargetResouceGroup="atlassian-nonprod-rg"
+TargetSubscription="b7d2bd5f-b744-4acc-9c73-e068cec2e8d8"
+StorageAccountName="atlassiannonprod"
+TargetVNetName="atlassian-int-nonprod-vnet"
+
+# az backup container list --resource-group $SourceResourceGroup --vault-name $VaultName --backup-management-type AzureIaasVM --query '[].{Name:name, ItemName:properties.friendlyName}' -o table
+
+ContainerName=$(az backup container list --resource-group $SourceResourceGroup --vault-name $VaultName --backup-management-type AzureIaasVM --query "[?properties.friendlyName=='$ItemName'].{Name:name}" -o tsv)
+
+# az backup recoverypoint list --vault-name $VaultName --resource-group $SourceResourceGroup --container-name $ContainerName --item-name $ItemName --query '[].{Name:properties.recoveryPointTime, ID:name}' -o table
+
+RecoverypointName=$(az backup recoverypoint list --vault-name $VaultName --resource-group $SourceResourceGroup --container-name $ContainerName --item-name $ItemName --query '[0].name' -o tsv)
+
+echo $RecoverypointName
+
+az backup restore restore-disks \
+    --resource-group $SourceResourceGroup \
+    --vault-name $VaultName \
+    --item-name $ItemName \
+    --rp-name $RecoverypointName \
+    --storage-account $StorageAccountName \
+    --restore-to-staging-storage-account true \
+    --target-resource-group $TargetResouceGroup \
+    --target-subscription-id $TargetSubscription \
+    --target-vm-name $TargetVmName \
+    --target-vnet-name $TargetVNetName \
+    --target-subnet-name $TargetSubnetName \
+    --target-vnet-resource-group $TargetResouceGroup \
+    --container-name $ContainerName \
+    --subscription $SourceSubscription \
+    --storage-account-resource-group $TargetResouceGroup
+```
+
+Above script will trigger the restore of the VM from the backup and will create a new VM in the staging environment.  You can able to see progress of the restore in the [backup vault](https://portal.azure.com/#view/Microsoft_Azure_DataProtection/V1JobsListBlade/vaultId/%2Fsubscriptions%2F79898897-729c-41a0-a5ca-53c764839d95%2FresourceGroups%2FRG-PRD-ATL-INT-01%2Fproviders%2FMicrosoft.RecoveryServices%2Fvaults%2FBK-PRD-ATL-INT-01/status/InProgress).
+
+Please note that each backup is separate job so its fine to run restore for multiple VMs at the same time.
+
+### Set new public key for access
+
+Once the VMs are restored, you need to set the public key for the access. You can use the following script to set the public key for the VMs:
+
+```bash
+TargetVmName=<TargetVmName> # eg "atlassian-nonprod-gluster-03"
+
+az account set --subscription b7d2bd5f-b744-4acc-9c73-e068cec2e8d8
+TargetSubscription="b7d2bd5f-b744-4acc-9c73-e068cec2e8d8"
+KeyvaultName="atlasssian-nonprod-kv"
+SecretName="public-key"
+TargetResouceGroup="atlassian-nonprod-rg"
+
+PublicKey=$(az keyvault secret show \
+  --vault-name $KeyvaultName \
+  --name $SecretName \
+  --subscription $TargetSubscription \
+  --query value -o tsv)
+
+username="atlassian-admin"
+
+az vm user update \
+  --resource-group $TargetResouceGroup \
+  --name $TargetVmName \
+  --username $username \
+  --ssh-key-value $PublicKey \
+  --subscription $TargetSubscription
+```
+
+This is the time it has taken to restore the VMs last time when we did it:
+
+<img src=images/AtlassianVMRestore.png  width="600">
+
+### Accessing the VMs
+
+Download the private key from the key vault, and save it to a file.  There are two private keys on this keyvault secret atlasssian-nonprod-kv  but they are both same, private-key and test-private-key.  There was some formatting issue with the private-key so we have created a new secret test-private-key.
+
+```bash
+# run chmod 600 <privatekeyfilename>
+
+ssh -i <privatekeyfilename> atlassian-admin@<VM-IP>
+```
+
+### Restore the databases
+
+We then have to restore the databases from the production to the staging environment.  We have used the backup and restore from vault feature within Azure
+
+Backup Vault containing [backups from prod](https://portal.azure.com/#@HMCTS.NET/resource/subscriptions/79898897-729c-41a0-a5ca-53c764839d95/resourceGroups/RG-PRD-ATL-01/providers/Microsoft.DataProtection/BackupVaults/ATL-Backup-Vault/overview)
+
+Restored the latest backup via the portal to target server atlassian-nonprod-server 
+
+Jobs can be [viewed here](https://portal.azure.com/#@HMCTS.NET/resource/subscriptions/79898897-729c-41a0-a5ca-53c764839d95/resourceGroups/RG-PRD-ATL-01/providers/Microsoft.DataProtection/BackupVaults/ATL-Backup-Vault/backupJobs)
+
+Crowd took 2min
+Confluence took 1hr30min
+Jira took 2hr5min
+
+### Import VM to the terraform
+
+Once the VMs are restored and you have access to the VMs, you can import the VMs to the terraform.  You can use the following PR to see what changes you may have to do to import the VMs to the terraform.
+
+[Pull Request](https://github.com/hmcts/atlassian-infrastructure/pull/48)
+
+Terraform already has the automation script which would make config changes on the VMs.  Please note that the script is only triggered when there is any change on the script so please make sure to make any change on the script to trigger the script.
+
+
+### Post Deployment Steps
+
+**1.** Post merging PR above and having the automation script run, add the recovery admin password on crowd and make your account Admin account
+
+**2.** Added password here  - vi /opt/crowd/apache-tomcat/bin/setenv.sh  --> append  ``` -Datlassian.recovery.password=<any password> ```
+
+**3.** Login to crowd using recovery_admin username and above password and add yourself Administrator groups.
+
+Your account then should sync with Jira and Confluence and you can then able to make any Administrative changes.
+
+**4.** Change the Base URLs on all the apps to staging.tools.hmcts.net
+
+**5.** Change colours of the environment to differentiate between Live and Staging
+
+
+## Troubleshooting
+
+**1.** If you are unable to access the VMs, please make sure you are connected to F5 VPN and using the correct private key and the private key is in the correct format.
+
+**2.** For some reason, if you see errors on the application, please make sure GlusterFS shares are mounted correctly on the VMs.
+e.g  `jira_shared` should be mounted here `/var/atlassian/application-data/jira/shared`
+
+Please use `mount -a` command to mount them correctly.
+
+There was problem where the share was not mounted correctly after auto shutdown, we have got cronjob on the staging VMs to run the mounting script everyday at 8am.
+
+

source/Services/images/AtlassianVMRestore.png

@@ -93,7 +93,7 @@ If you are happy with the range you are getting, run the reservation command to
   ```json
   {
       "cidr": "10.0.100.0/24",
-      "desc" "New CIDR"
+      "desc": "New CIDR"
   }
   ```