Merge branch 'master' into release-1.0

Author: dvavili

README.md

@@ -2,7 +2,8 @@
 Install Contiv on your Docker Swarm or Kubernetes cluster.
 ## Docker Swarm Installation
 ### Prerequisites
-* CentOS 7.x operating system
+* CentOS 7.x operating system.
+* Python installed on the master and worker nodes.
 * Docker installed on the host where you are running the installer.
 * Install a Docker Swarm cluster or use the Contiv installer to install the scheduler stack after installing Docker on a node external to the cluster where the scheduler stack is being installed.
 
@@ -16,34 +17,38 @@ If your access to the Internet is limited or slow and you want to download the f
 Note: The full image contains only Contiv components. Installing Docker Swarm will need Internet connectivity.
 * Extract the install bundle <br>`tar oxf contiv-$VERSION.tgz`.
 * Change directories to the extracted folder <br>`cd contiv-$VERSION`
-* To install Contiv with Docker Swarm:<br> `./install/ansible/install_swarm.sh -f cfg.yml -e <ssh key> -u <username> -i`
-* To install Contiv with Docker Swarm and ACI:<br> `./install/ansible/install_swarm.sh -f aci_cfg.yml -e <ssh key> -u <username> -i -m aci`
+* To install Contiv with Docker Legacy Swarm:<br> `./install/ansible/install_swarm.sh -f cfg.yml -e <ssh key> -u <username> -i`
+* To install Contiv with Docker Legacy Swarm and ACI:<br> `./install/ansible/install_swarm.sh -f aci_cfg.yml -e <ssh key> -u <username> -i -m aci`
+* To install Contiv v2plugin:<br> `./install/ansible/install_swarm.sh -f cfg.yml -e <ssh key> -u <username> -p`
 * Example host config files are available at install/ansible/cfg.yml and install/ansible/aci_cfg.yml
 * To see additional install options and examples, run <br>`./install/ansible/install_swarm.sh -h`.
 
 ### Removing Contiv
 
 If you need to remove Contiv from Docker Swarm and return to your original state, you can uninstall Contiv with the following commands:
-* To uninstall Contiv and Docker Swarm:<br>
+* To uninstall Contiv and Docker Legacy Swarm:<br>
 `./install/ansible/uninstall_swarm.sh -f cfg.yml -e <ssh key> -u <username> -i`
-* To uninstall Contiv and Docker Swarm with ACI support:<br>
+* To uninstall Contiv and Docker Legacy Swarm with ACI support:<br>
 `./install/ansible/uninstall_swarm.sh -f aci_cfg.yml -e <ssh key> -u <username> -i -m aci`
-* To uninstall Contiv and not Docker Swarm:<br>
+* To uninstall Contiv and not Docker Legacy Swarm:<br>
 `./install/ansible/uninstall_swarm.sh -f cfg.yml -e <ssh key> -u <username>`
+* To uninstall Contiv v2plugin:<br>
+`./install/ansible/uninstall_swarm.sh -f cfg.yml -e <ssh key> -u <username> -p`
 * Note: Adding the `-r` flag, will cleanup any Contiv state.
 
-## Kubernetes 1.4 Installation
+## Kubernetes Installation
 
 ### Prerequisites
 
+* Kubernetes 1.4.x, 1.5.x and 1.6.2+ are supported with the following instructions.
 * CentOS 7.x operating system
-* Install Kubernetes 1.4:
-  1. Contiv service-cidr is currently 10.254.0.0/16. `kubeadm init` step needs to be called with the `--service-cidr 10.254.0.0/16` parameter.
-  2. kubeadm installs the latest Kubernetes version. To install the latest supported version use the `--use-kubernetes-version v1.4.7` parameter.<br>
-     See an example script [here.](https://github.com/contiv/install/blob/master/cluster/k8s1.4/k8smaster.sh)
-  3. Replace step (3/4) in the kubeadm install guide with the following Contiv Installation Instructions. Contiv installation can be done after completing step (4/4).
-  4. Instructions to install Kubernetes 1.4 are available [here.](http://kubernetes.io/docs/getting-started-guides/kubeadm/)
-  5. **NOTE:** A recent kubeadm update has rendered the latest kubeadm unable to install Kubernetes 1.4. To workaround this issue, please use kubeadm version 1.6.0-0.alpha or earlier.
+* Install Kubernetes:
+  1. kubeadm installs the latest Kubernetes version. <br>
+     For Kubernetes 1.4, see an example script [here.](https://github.com/contiv/install/blob/master/cluster/k8s1.4/k8smaster.sh)
+     For Kubernetes 1.6, see an example script [here.](https://github.com/contiv/install/blob/master/cluster/k8s1.6/k8smaster.sh)
+  1. Replace step (3/4) in the kubeadm install guide with the following Contiv Installation Instructions. Contiv installation can be done after completing step (4/4).
+  1. Instructions to install Kubernetes are available [here.](http://kubernetes.io/docs/getting-started-guides/kubeadm/)
+  1. **Kubernetes 1.4 NOTE:** A recent kubeadm update has rendered the latest kubeadm unable to install Kubernetes 1.4. To workaround this issue, please use kubeadm version 1.6.0-0.alpha or earlier.
      Example pre-built RPMs are available as shown [here.](https://github.com/contiv/install/blob/master/cluster/k8s1.4/bootstrap_centos.sh#L22-L26)
 
 ### Contiv Installation

install/ansible/Dockerfile

@@ -1,5 +1,7 @@
-FROM alpine
-
-RUN apk add --no-cache python openssl libffi \
-    py-pip ansible nmap-ncat 
+FROM alpine:3.6
 
+RUN DEV_PACKAGES="python-dev gcc musl-dev openssl-dev libffi-dev" \
+ && apk add --no-cache python openssl libffi nmap-ncat py-pip $DEV_PACKAGES \
+ && pip install --upgrade pip \
+ && pip install ansible==2.3.1.0 \
+ && apk del $DEV_PACKAGES

install/ansible/env.json

@@ -13,6 +13,7 @@
   "etcd_cleanup_state": "False",
   "auth_proxy_local_install": "False",
   "contiv_network_local_install": "False",
+  "netctl_url": "http://netmaster:9999",
   "contiv_v2plugin_install": "False",
   "contiv_v2plugin_image": "contiv/v2plugin:__CONTIV_V2PLUGIN_VERSION__"
 }

install/ansible/install.sh

@@ -1,5 +1,7 @@
 #!/bin/sh
 
+set -xeuo pipefail
+
 # This scripts runs in a container with ansible installed.
 . ./install/ansible/install_defaults.sh
 
@@ -19,6 +21,7 @@ install_scheduler=false
 
 # This is the netmaster IP that needs to be provided for the installation to proceed
 netmaster=""
+contiv_v2plugin_install=""
 
 usage() {
 	echo "Usage:"
@@ -90,14 +93,18 @@ if [ "$netmaster" = "" ]; then
 	usage
 fi
 
+if [ "$install_scheduler" = "true" ] && [ "$contiv_v2plugin_install" = "true" ]; then
+	echo "ERROR: -p and -i are mutually exclusive"
+	usage
+fi
+
 ansible_path=./ansible
 env_file=install/ansible/env.json
 # Verify ansible can reach all hosts
 
 echo "Verifying ansible reachability"
-ansible all $ans_opts -i $host_inventory -m setup -a 'filter=ansible_distribution*' >&$inventory_log
-egrep 'FAIL|UNREACHABLE' $inventory_log >&/dev/null
-if [ $? -eq 0 ]; then
+ansible all -vvv $ans_opts -i $host_inventory -m setup -a 'filter=ansible_distribution*' | tee $inventory_log
+if [ egrep 'FAIL|UNREACHABLE' $inventory_log > /dev/null ]; then
 	echo "WARNING Some of the hosts are not accessible via passwordless SSH"
 	echo " "
 	echo "This means either the host is unreachable or passwordless SSH is not"
@@ -131,22 +138,30 @@ cp /var/contiv/key.pem /ansible/roles/auth_proxy/files/
 if [ "$aci_image" != "" ]; then
 	sed -i.bak "s#.*aci_gw_image.*#\"aci_gw_image\":\"$aci_image\",#g" "$env_file"
 fi
+if [ "$contiv_v2plugin_install" == "true" ]; then
+	sed -i.bak "s#.*contiv_v2plugin_install.*#\"contiv_v2plugin_install\":\"True\",#g" "$env_file"
+else
+	sed -i.bak "s#.*contiv_v2plugin_install.*#\"contiv_v2plugin_install\":\"False\",#g" "$env_file"
+fi
 
 echo "Installing Contiv"
 # Always install the base, install the scheduler stack/etcd if required
-echo '- include: install_base.yml' >$ansible_path/install_plays.yml
 
-if [ "$install_scheduler" == "true" ] ; then
+rm -f $ansible_path/install_plays.yml
+touch $ansible_path/install_plays.yml
+
+if [ "$install_scheduler" == "true" ]; then
+	echo '- include: install_base.yml' >$ansible_path/install_plays.yml
 	echo '- include: install_docker.yml' >>$ansible_path/install_plays.yml
 	echo '- include: install_etcd.yml' >>$ansible_path/install_plays.yml
 	echo '- include: install_scheduler.yml' >>$ansible_path/install_plays.yml
 else
 	if [ "$install_etcd" == "true" ]; then
-		echo '- include: install_etcd.yml' >>$ansible_path/install_plays.yml
+		echo '- include: install_etcd.yml' >$ansible_path/install_plays.yml
 	fi
 fi
 # Install contiv & API Proxy
-if [ "$contiv_v2plugin_install" == "true" ] ; then
+if [ "$contiv_v2plugin_install" == "true" ]; then
 	echo '- include: install_v2plugin.yml' >>$ansible_path/install_plays.yml
 	echo '- include: install_auth_proxy.yml' >>$ansible_path/install_plays.yml
 else
@@ -172,6 +187,8 @@ chmod 666 $inventory_log
 chmod 666 $env_file
 chmod 666 $log_file
 
+set +x
+
 if [ "$unreachable" = "" ] && [ "$failed" = "" ]; then
 	echo "Installation is complete"
 	echo "========================================================="
@@ -188,6 +205,7 @@ if [ "$unreachable" = "" ] && [ "$failed" = "" ]; then
 	echo " For example, netctl net create -t default --subnet=20.1.1.0/24 default-net"
 	echo " "
 	echo "========================================================="
+	exit 0
 else
 	echo "Installation failed"
 	echo "========================================================="

install/ansible/install_swarm.sh

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+set -euo pipefail
+
 # This is the installation script for Contiv.
 
 . ./install/ansible/install_defaults.sh
@@ -9,6 +11,9 @@ ans_opts=""
 ans_user="root"
 ans_key=$src_conf_path/insecure_private_key
 install_scheduler=""
+netmaster=""
+v2plugin_param=""
+contiv_v2plugin_install=""
 
 # Check for docker
 

install/ansible/uninstall.sh

@@ -1,5 +1,6 @@
 #!/bin/sh
 
+set -xeuo pipefail
 # This scripts runs in a container with ansible installed.
 . ./install/ansible/install_defaults.sh
 
@@ -103,9 +104,8 @@ env_file=install/ansible/env.json
 # Verify ansible can reach all hosts
 
 echo "Verifying ansible reachability"
-ansible all $ans_opts -i $host_inventory -m setup -a 'filter=ansible_distribution*' >&$inventory_log
-egrep 'FAIL|UNREACHABLE' $inventory_log >&/dev/null
-if [ $? -eq 0 ]; then
+ansible all -vvv $ans_opts -i $host_inventory -m setup -a 'filter=ansible_distribution*' | tee $inventory_log
+if [ egrep 'FAIL|UNREACHABLE' $inventory_log > /dev/null ]; then
 	echo "WARNING Some of the hosts are not accessible via passwordless SSH"
 	echo " "
 	echo "This means either the host is unreachable or passwordless SSH is not"
@@ -135,13 +135,17 @@ sed -i.bak "s/.*docker_reset_container_state.*/\"docker_reset_container_state\":
 sed -i.bak "s/.*docker_reset_image_state.*/\"docker_reset_image_state\":$reset_images,/g" $env_file
 sed -i.bak "s/.*etcd_cleanup_state.*/\"etcd_cleanup_state\":$reset,/g" $env_file
 
-sed -i.bak "s#__CLUSTER_STORE__#$cluster#g" $env_file
-
 if [ "$aci_image" != "" ]; then
 	sed -i.bak "s#.*aci_gw_image.*#\"aci_gw_image\":\"$aci_image\",#g" "$env_file"
 fi
+if [ "$uninstall_v2plugin" == "true" ]; then
+	sed -i.bak "s#.*contiv_v2plugin_install.*#\"contiv_v2plugin_install\":\"True\",#g" "$env_file"
+else
+	sed -i.bak "s#.*contiv_v2plugin_install.*#\"contiv_v2plugin_install\":\"False\",#g" "$env_file"
+fi
 
 echo "Uninstalling Contiv"
+rm -f $ansible_path/uninstall_plays.yml
 
 # Uninstall contiv & API Proxy
 if [ $uninstall_v2plugin == true ]; then
@@ -179,8 +183,11 @@ chmod 666 $inventory_log
 chmod 666 $env_file
 chmod 666 $log_file
 
+set +x
+
 if [ "$unreachable" = "" ] && [ "$failed" = "" ]; then
 	echo "Uninstallation is complete"
+	exit 0
 else
 	echo "Uninstallation failed"
 	echo "========================================================="

install/ansible/uninstall_swarm.sh

@@ -1,4 +1,7 @@
 #!/bin/bash
+
+set -euo pipefail
+
 # This is the uninstall script for Contiv.
 
 . ./install/ansible/install_defaults.sh
@@ -8,6 +11,7 @@ ans_opts=""
 ans_user="root"
 ans_key=$src_conf_path/insecure_private_key
 uninstall_scheduler=""
+netmaster=""
 uninstall_v2plugin=""
 reset_params=""
 

install/k8s/install.sh

@@ -207,6 +207,8 @@ fi
 
 if [ "$apic_url" != "" ]; then
 	cat $contiv_aci_gw_template >>$contiv_yaml
+	# We do not support routing in ACI mode
+	fwd_mode="bridge"
 fi
 
 # We will store the ACI key in a k8s secret.
@@ -221,9 +223,10 @@ fi
 
 $kubectl create secret generic aci.key --from-file=$aci_key -n kube-system
 
+mkdir -p /var/contiv
+
 if [ "$tls_cert" = "" ]; then
 	echo "Generating local certs for Contiv Proxy"
-	mkdir -p /var/contiv
 	mkdir -p ./local_certs
 
 	chmod +x ./install/generate-certificate.sh
@@ -274,8 +277,10 @@ for i in {0..30}; do
 done
 set -e
 
-netctl global set --fwd-mode $fwd_mode
-netctl net create -n infra -s $infra_subnet -g $infra_gateway contivh1
+if [ "$fwd_mode" == "routing" ]; then
+	netctl global set --fwd-mode $fwd_mode
+	netctl net create -n infra -s $infra_subnet -g $infra_gateway contivh1
+fi
 
 echo "Installation is complete"
 echo "========================================================="

install/k8s/k8s1.6/contiv.yaml

@@ -137,8 +137,6 @@ spec:
       tolerations:
       - key: node-role.kubernetes.io/master
         effect: NoSchedule
-      nodeSelector:
-        node-role.kubernetes.io/master: ""
       serviceAccountName: contiv-netplugin
       containers:
         # Runs netplugin container on each Kubernetes node.  This

scripts/build.sh

@@ -13,15 +13,21 @@ fi
 DEV_IMAGE_NAME="devbuild"
 VERSION=${BUILD_VERSION-$DEV_IMAGE_NAME}
 
-contiv_version=${CONTIV_VERSION:-"1.0.2"}
-
+contiv_version=${CONTIV_VERSION:-"1.0.3"}
+pull_images=${CONTIV_CI_HOST:-"false"}
 aci_gw_version=${CONTIV_ACI_GW_VERSION:-"latest"}
 ansible_image_version=${CONTIV_ANSIBLE_IMAGE_VERSION:-$contiv_version}
 auth_proxy_version=${CONTIV_API_PROXY_VERSION:-$contiv_version}
-v2plugin_version=${CONTIV_V2PLUGIN_VERSION:-"1.0.2"}
 docker_version=${CONTIV_DOCKER_VERSION:-1.12.6}
 etcd_version=${CONTIV_ETCD_VERSION:-v2.3.8}
 
+# the installer currently pulls the v2plugin image directly from Docker Hub, but
+# this will change to being downloaded from the Docker Store in the future.
+# because of this, the default value for this variable will become the latest
+# version that is available in the Docker Store and should be considered
+# independent of $contiv_version above.
+v2plugin_version=${CONTIV_V2PLUGIN_VERSION:-"1.0.3"}
+
 function usage() {
 	echo "Usage:"
 	echo "./release.sh -a <ACI gateway image> -c <contiv version> -e <etcd version> -p <API proxy image version> "
@@ -121,13 +127,13 @@ mkdir -p $binary_cache
 tar czf $tmp_output_file -C $release_dir contiv-$VERSION
 
 # Save the auth proxy & aci-gw images for packaging the full docker images with contiv install binaries
-if [ "$(docker images -q contiv/auth_proxy:$auth_proxy_version 2>/dev/null)" == "" ]; then
+if [[ "$(docker images -q contiv/auth_proxy:$auth_proxy_version 2>/dev/null)" == "" || "$pull_images" == "true" ]]; then
 	docker pull contiv/auth_proxy:$auth_proxy_version
 fi
 proxy_image=$(docker images -q contiv/auth_proxy:$auth_proxy_version)
 docker save $proxy_image -o $binary_cache/auth-proxy-image.tar
 
-if [ "$(docker images -q contiv/aci-gw:$aci_gw_version 2>/dev/null)" == "" ]; then
+if [[ "$(docker images -q contiv/aci-gw:$aci_gw_version 2>/dev/null)" == "" || "$pull_images" == "true" ]]; then
 	docker pull contiv/aci-gw:$aci_gw_version
 fi
 aci_image=$(docker images -q contiv/aci-gw:$aci_gw_version)
@@ -139,7 +145,7 @@ curl -sL -o $binary_cache/netplugin-$contiv_version.tar.bz2 https://github.com/c
 
 env_file=$output_dir/install/ansible/env.json
 sed -i.bak "s#.*auth_proxy_local_install.*#  \"auth_proxy_local_install\": True,#g" $env_file
-sed -i.bak "s#.*contiv_network_local_install.*#  \"contiv_network_local_install\": True#g" $env_file
+sed -i.bak "s#.*contiv_network_local_install.*#  \"contiv_network_local_install\": True,#g" $env_file
 
 # Create the full tar bundle
 tar czf $tmp_full_output_file -C $release_dir contiv-$VERSION