Additional documentation

Author: DenisBiondic

Build-Containers.ps1

@@ -23,12 +23,26 @@ Function Build-Container([string]$ContainerTag, [string]$BuildContextFolderName,
     $dockerfile = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($PSScriptRoot, "$BuildContextFolderName/Dockerfile"))
 
     docker build -t $ContainerTag -f $dockerfile $buildContext
+
+    if ($LastExitCode -ne 0) {
+        throw "Could not build the image $ContainerTag from file: $dockerfile for context: $buildContext"
+    }
+
     docker tag $ContainerTag $registryUrl/$ContainerTag
 
+    if ($LastExitCode -ne 0) {
+        throw "Could not tag the image $ContainerTag for $registryUrl/$ContainerTag"
+    }
+
     Write-Host "[PUSH TO REPOSITORY] Pushing $ContainerTag container to $RegistryUrl..." -foreground "green"
 
     docker push $RegistryUrl/$ContainerTag
 
+    if ($LastExitCode -ne 0) {
+        throw "Could not push the image $RegistryUrl/$ContainerTag"
+    }
+
+
     Write-Host "[FINISHED] Finished building $ContainerTag container" -foreground "green"
 }
 
@@ -47,6 +61,10 @@ $registryUrl = ("cadevcache" + $EnvironmentTag + "registry.azurecr.io")
 
 docker login $registryUrl -u $registryUsername -p $registryPassword
 
+if ($LastExitCode -ne 0) {
+    throw "Could not login to the docker registry"
+}
+
 Build-Container -ContainerTag "devicecache-frontend:$VersionTag" -BuildContextFolderName "DeviceCache.Frontend" -RegistryUrl $registryUrl
 Build-Container -ContainerTag "devicecache-processor:$VersionTag" -BuildContextFolderName "DeviceCache.Processor" -RegistryUrl $registryUrl
 Build-Container -ContainerTag "devicecache-simulator:$VersionTag" -BuildContextFolderName "DeviceCache.Simulator" -RegistryUrl $registryUrl

Create-Prerequisites.ps1 Create-CloudClusterPrerequisites.ps1

@@ -33,13 +33,6 @@ $eventHubTemplateFile = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine(
 $registryTemplateFile = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($PSScriptRoot, "DeviceCache.Infrastructure/Registry.json"))
 $clusterTemplateFile = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($PSScriptRoot, "DeviceCache.Infrastructure/Cluster.json"))
 
-$automationKeyVaultName = "ca-automation-$EnvironmentTag"
-$automationKeyVault = Get-AzureRmKeyVault -VaultName $automationKeyVaultName -ErrorAction SilentlyContinue
-
-if (-not $automationKeyVault) {
-    throw "Automation key vault not found. Make sure you run the Create-Prerequisites.ps1 script first."
-}
-
 $keyVaultName = "ca-devcache-$EnvironmentTag"
 Create-KeyVault -KeyVaultName $keyVaultName -ResourceGroupName $resourceGroupName -ResourceGroupLocation $ResourceGroupLocation
 
@@ -54,6 +47,13 @@ $registryTemplateParameters["EnvironmentTag"] = $EnvironmentTag
 DeployTemplate -ResourceGroupName $resourceGroupName -TemplateFileFullPath $registryTemplateFile -TemplateParameters $registryTemplateParameters
 
 if (-not $SkipCluster) {
+    $automationKeyVaultName = "ca-automation-$EnvironmentTag"
+    $automationKeyVault = Get-AzureRmKeyVault -VaultName $automationKeyVaultName -ErrorAction SilentlyContinue
+
+    if (-not $automationKeyVault) {
+        throw "Automation key vault required for the cluster not found. Make sure you run the Create-CloudClusterPrerequisites script first."
+    }
+
     $clusterManagerId = (Get-AzureKeyVaultSecret -VaultName $automationKeyVaultName -SecretName servicePrincipalId).SecretValueText
     $clusterManagerKey = (Get-AzureKeyVaultSecret -VaultName $automationKeyVaultName -SecretName servicePrincipalPassword).SecretValue
     $sshPublicKey = (Get-AzureKeyVaultSecret -VaultName $automationKeyVaultName -SecretName machineSshPublicKey).SecretValueText

Create-Infrastructure.ps1

@@ -0,0 +1,40 @@
+# Prerequisites for cloud cluster deployment
+
+In case you will be also deploying a remote cluster with ACS (Create-Infrastructure.ps1 script without the -SkipCluster flag), you will need to setup some prerequisites for such deployment:
+
+- Subscription admin service principal credentials (**you need to provide this**)
+- Public SSH key (**you need to provide this**)
+
+All the prerequistes will be automatically saved in a key vault which can be accessed later in other scripts.
+
+## Getting a service principal credentials
+
+To create and get the service principal credentials, simply follow these [instructions](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal)(scroll down to "Create an Azure Active Directory application" section). You will need the application ID and the key from your principal.
+
+## Getting a public SSH key
+
+For SSH access, you will need a private / public key pair. For setting these up under windows, you can find [some instructions here](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/ssh-from-windows).
+
+You could also use some of the tools like **Bitvise** which are capable of creating private / public keys themselves. 
+
+**Tip**: if you used Azure CLI to provision ACS clusters already, you can simply reuse the public key found in `C:\Users\<<your user>>\.ssh`
+
+## Running the Create-CloudClusterPrerequisites.ps1 script
+
+First, execute this line to securely enter your credentials for the service principal:
+
+```powershell
+$servicePrincipalCredentials = Get-Credential
+```
+
+Afterwards, you can execute the Create-Prerequisites.ps1 script itself: 
+
+``` powershell
+.\Create-CloudClusterPrerequisites.ps1 -EnvironmentTag <<your_env_tag>> -MachineSshPublicKey "ssh-rsa AAAA...6SkIQ0opBt" -ServicePrincipalCredentials $servicePrincipalCredentials
+```
+
+If everything was setup correctly, you can verify the contents of your key vault in the cloud:
+
+![Secrets inside key vault](keyvault_with_secrets.png)
+
+Key vault name will be `ca-automation-<<your_env_tag>>`

docs/cloud-prerequisites.md

@@ -1,3 +1,5 @@
+# Roadmap
+
 ~~Infrastructure incl. scripting~~
 
 ~~Packaging / parametrization with Helm~~
@@ -14,6 +16,6 @@
 
 Start to finish documentation with commands (in progress)
 
-Prometheus / Grafana instance
+Prometheus / Grafana instance (in progress)
 
 Exposing load metrics from services

docs/keyvault_with_secrets.png

@@ -1,3 +1,5 @@
+# Setup
+
 ## Required Software:
 
 **Editor**: Visual Studio 2017 / Visual Studio Code
@@ -12,23 +14,119 @@
 
 **Minikube** (for local development): https://github.com/kubernetes/minikube
 
-**Azure CLI 2.0**
+**Azure CLI 2.0** (can be run as [docker image](https://hub.docker.com/r/microsoft/azure-cli/))
+
+## Concept of environments
+
+All the PS scripts have the mandatory `-EnvironmentTag` flag which needs to be set. This flag is one of the best practices when writing Infrastructure-as-Code scripts to be able to reuse same scripts between multiple environments. It also helps when multiple people are using the same scripts for their "local" environments so that they get no conflicts in the cloud (each has its own environment).
+
+Important here to know is:
+- choose a 2-4 letter [a-Z] tag which you want
+- provide the same tag to all the scripts you will be executing
+
+## Prerequisites
+
+This project can be run in two ways:
+- with a local cluster (e.g. minikube) while the PaaS components are in cloud (e.g. Event Hub)
+- completely in cloud (cluster with Azure Container Services)
+
+Depending whether you want local or cloud deployment, there are some small differences in setup.
+
+If you want to deploy the sample to Azure Container Services (cloud), you need to run the `Create-CloudClusterPrerequisites.ps1` script which should setup a KeyVault with deployment-time secrets (service principal, SSH Key, passwords and such) neccessary for Azure Container Services cluster to created.
+
+More info here: [Runnning Create-CloudClusterPrerequisites script](cloud-prerequisites.md)
 
 ## Infrastructure Setup
 
-For setting up the neccessary Azure Infrastructure (Infrastructure-as-Code) for the code to run, you can use the Create-Infrastructure.ps1 script. However, this script has a dependency on a Key Vault which should contain deployment-time secrets (service principal, SSH Key, passwords and such).
+For setting up the neccessary Azure Infrastructure (Infrastructure-as-Code) for the code to run, you can use the `Create-Infrastructure.ps1` script. 
+
+First, make sure you log in to your Azure Subscription with
+
+```powershell
+Login-AzureRmAccount
+```
+
+and, if necessary, switch to the correct subscription using
+
+```powershell
+# to find out the subscirption id, run Get-AzureRmSubscription
+Select-AzureRmSubscription
+```
+
+You can execute the `Create-Infrastructure.ps1` script now. In case if you will be working with a local cluster (e.g. minikube), run the following:
+
+```powershell
+.\Create-Infrastructure.ps1 -EnvironmentTag <<set_tag_here>> -SkipCluster
+```
+
+If you are going to be using a cloud cluster (ACS), omit the -SkipCluster flag.
+
+```powershell
+# make sure you execute the Create-CloudClusterPrerequisites.ps1 script first!
+# more info above in this document, or read cloud-prerequisites.md
+.\Create-Infrastructure.ps1 -EnvironmentTag <<your_tag_here>>
+```
+
+Script should finish without any errors.
 
-To create such Key Vault with all required secrets, run the Create-Prerequisites.ps1 script.
-First, execute this like to securely enter your credentials for the service principal:
+## Initialize the cluster
+
+Before you start deploying the microservices, your cluster needs to be "initialized" first. What this actually means is that we need to write in a secret for private docker registry connections and that we need to initialize [Helm](https://helm.sh/) for doing the actual deployments.
+
+#### Local cluster
+
+If you already installed minikube and are planning to deploy locally, you can go ahead and run the script.
+
+```powershell
+.\Initialize-Cluster.ps1 -EnvironmentTag <<your_tag_here>>
+```
+
+#### Cloud cluster
+
+If you are planning to use the cloud cluster (ACS), you need to configure your kubectl tool first. Easiest way to do this is through Azure CLI:
+
+```bash
+az acs kubernetes get-credentials --resource-group=... --name=...
+```
+
+Afterwards, simple run the script:
+
+```powershell
+.\Initialize-Cluster.ps1 -EnvironmentTag <<your_tag_here>>
+```
+
+#### Kubectl & contexts
+
+Important thing to realise here is that many of the tools like Helm and scripts you will be using in this project are supporting [Kubectl contexts](https://kubernetes.io/docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/) directly. One such context for your local cluster is setup when starting minikube, and the `az acs kubernetes get-credentials ... ` command above also sets up a context for your remote ACS cluster.
+
+## Building the containers
+
+**Short answer:**
 
 ```powershell
-$servicePrincipalCredentials = Get-Credential
+# builds all the containers and pushes them to a remove environment specific docker registry (ACR)
+.\Build-Containers.ps1 -EnvironmentTag <<your_tag_here>>
 ```
 
-Afterwards, you can execute the Create-Prerequisites.ps1 script itself: 
+**Long answer:**
+
+One of the things that was setup in previous step, was a private Docker Registry in form of Azure Container Registry. For both scenarios, local and cloud, we will be using this registry to roll out the containers onto the Kubernetes cluster. 
+
+Each of the microservices in this project has a Dockerfile which can be used out of the box and could do the classic docker build / docker tag / docker login / docker push if you wanted (or you can use something like [Draft](https://github.com/Azure/draft))
 
-``` powershell
-.\Create-Prerequisites.ps1 -EnvironmentTag "white" -MachineSshPublicKey "ssh-rsa AAAA...6SkIQ0opBt" -ServicePrincipalCredentials $servicePrincipalCredentials
+## Deploying the application
+
+Simply use the Deploy-Application.ps1 script:
+
+```powershell
+# will use your current kubectl context for deployment target
+.\Deploy-Application.ps1 -EnvironmentTag <<your_tag_here>>
 ```
 
-Warning: make sure to delete the resources since they incure Azure costs! :)
+Once installed, you will be able to see the pods with `kubectl get pods`. 
+
+`helm list` will also show you your new release.
+
+## Deleting the application
+
+Simply run `helm delete <<release_name>>` with the release name you got from `helm list`.