Merge branch 'master' of https://github.com/OpenVisualCloud/CDN-Transcode-Sample

Author: Pengfei Qu

README.md

@@ -5,24 +5,6 @@
 [![License](https://img.shields.io/badge/license-BSD_3_Clause-green.svg)](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/blob/master/LICENSE)
 [![Contributions](https://img.shields.io/badge/contributions-welcome-blue.svg)](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/wiki)
 
-Table of Contents
-=================
- * [Open Visual Cloud CDN Transcode Sample](#open-visual-cloud-cdn-transcode-sample)
-   * [Architecture](#architecture)
-   * [What's in this project](#whats-in-this-project)
-   * [System requirements](#system-requirements)
-      * [Operating system](#operating-system)
-   * [How to setup The CDN Transcode Sample](#how-to-setup-the-cdn-transcode-sample)
-      * [Setup the CDN Transcode Sample OS environment(Both of master and slave nodes)](#setup-the-cdn-transcode-sample-os-environmentboth-of-master-and-slave-nodes)
-         * [Install ubuntu18.04.2/CentOS 7.6](#install-ubuntu18042centos-76)
-      * [Setup CDN environment(Both of master and slave nodes)](#setup-cdn-environmentboth-of-master-and-slave-nodes)
-         * [Install the third-party dependency Libraries and tools](#install-the-third-party-dependency-libraries-and-tools)
-         * [Setup docker proxy as follows if you are behind a firewall](#setup-docker-proxy-as-follows-if-you-are-behind-a-firewall)
-      * [Build(Both of master and slave nodes)](#buildboth-of-master-and-slave-nodes)
-      * [Deploy](#deploy)
-         * [Auto deployment using Kubernetes](#auto-deployment-using-kubernetes)
-         * [See Also](#see-also)
-
 The CDN Transcode Sample is an Open Visual Cloud software stack with all required open source ingredients well integrated to provide out-of-box CDN media transcode service, including live streaming and video on demand. It also provides Docker-based media delivery software development environment upon which developer can easily build their specific applications.
 
 # Architecture
@@ -31,30 +13,15 @@ The sample implements a reference server-side transcode system over CDN infrastr
 
 <IMG src="doc/CDN-Transcode-Sample-Arch.png" height="450">
 
-# What's in this project
-The CDN Transcode Sample contains below components:
--  Dockerfiles
--  Python web services source code
--  BASH glue scripts
--  HTML web pages
--  CMakefiles
--  Configuration files
--  Documents
-
 # System requirements
 ## Operating system
 The CDN Transcode Sample may run on Linux* 64 bit operating systems. The list below represents the operating systems that the transcode application and library were tested and validated on:
 - Ubuntu* 18.04.2 Server LTS
 - CentOS* 7.6
 
 # How to setup The CDN Transcode Sample
-## Setup the CDN Transcode Sample OS environment(Both of master and slave nodes)
-Install Ubuntu 18.04.2/CentOS 7.6 on CDN-Transcode Server, and configure the IP address & proxy properly.
-### Install ubuntu18.04.2/CentOS 7.6
--  [Download Ubuntu and Install](https://ubuntu.com/download)
--  [Download CentOS and install](https://www.centos.org/download/)
 
-## Setup CDN environment(Both of master and slave nodes)
+## Setup CDN environment  
 ### Install the third-party dependency Libraries and tools
 ```
 sudo -E ./script/install_dependency.sh
@@ -66,7 +33,7 @@ printf "[Service]\nEnvironment=\"HTTPS_PROXY=$https_proxy\" \"NO_PROXY=$no_proxy
 sudo systemctl daemon-reload
 sudo systemctl restart docker
 ```
-## Build(Both of master and slave nodes)
+## Build   
 
 Run below commands to build docker images
 ```
@@ -88,10 +55,9 @@ To deploy without a private registry, run `make update` after each build to push
 ## Deploy
 ### Auto deployment using Kubernetes
 
-**Tips:** It divides into two parts: master or slave ones
-- [Setup Kubernetes master environment for CentOS](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/wiki/Setup-Kubernetes-master-environment-for-CentOS)
-- [Setup Kubernetes master environment for Ubuntu](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/wiki/Setup-Kubernetes-master-environment-for-Ubuntu)
-- [Setup Kubernetes slave environment](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/wiki/Setup-Kubernetes-slave-environment)
+**Tips:** It divides into two parts:   
+- [Setup Kubernetes for CentOS](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/wiki/Setup-Kubernetes-for-CentOS)
+- [Setup Kubernetes for Ubuntu](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/wiki/Setup-Kubernetes-for-Ubuntu)
 - [Setup NFS environment](https://github.com/OpenVisualCloud/CDN-Transcode-Sample/wiki/Setup-NFS-environment)
 
 Start CDN transcode service

cdn-server/Dockerfile

@@ -1,5 +1,5 @@
 
-FROM openvisualcloud/xeon-ubuntu1804-media-nginx:20.3.1
+FROM openvisualcloud/xeon-ubuntu1804-media-nginx:20.7
 
 Run DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y -q --no-install-recommends python3-setuptools python3-redis python-celery-common python3-tornado python3-kafka python3-kazoo openssh-server && rm -rf /var/lib/apt/lists/*
 

deployment/docker-swarm/docker-compose.yml.m4

@@ -7,6 +7,7 @@ services:
         restart: always
         deploy:
             replicas: 1
+        user: redis
         command:
             redis-server
 
@@ -20,12 +21,13 @@ services:
             ZOOKEEPER_MAX_CLIENT_CNXNS: '20000'
             ZOOKEEPER_LOG4J_LOGGERS: 'zookeepr=ERROR'
             ZOOKEEPER_LOG4J_ROOT_LOGLEVEL: 'ERROR'
+        user: zookeeper
         restart: always
         deploy:
             replicas: 1
 
     kafka-service:
-        image: wurstmeister/kafka:latest
+        image: defn(`REGISTRY_PREFIX')ovc_kafka_service:latest
         depends_on:
             - zookeeper-service
         environment:
@@ -45,6 +47,7 @@ services:
             KAFKA_HEAP_OPTS: '-Xmx1024m -Xms1024m'
             KAFKA_LOG4J_LOGGERS: 'kafka=ERROR,kafka.controller=ERROR,state.change.logger=ERROR,org.apache.kafka=ERROR'
             KAFKA_LOG4J_ROOT_LOGLEVEL: 'ERROR'
+        user: kafka
         restart: always
         deploy:
             replicas: 1

deployment/kubernetes/helm/cdn-transcode/templates/kafka-service-deployment.yaml

@@ -48,8 +48,12 @@ spec:
           value: "1"
         - name: KAFKA_ZOOKEEPER_CONNECT
           value: zookeeper-service:2181
-        image: wurstmeister/kafka:latest
+        image: {{ $.Values.registryPrefix }}ovc_kafka_service:latest
+        imagePullPolicy: IfNotPresent
         name: kafka-service
         ports:
         - containerPort: 9092
+        securityContext:
+          runAsUser: 1000
       restartPolicy: Always
+

deployment/kubernetes/helm/cdn-transcode/templates/redis-service-deployment.yaml

@@ -19,7 +19,10 @@ spec:
       - args:
         - redis-server
         image: redis:latest
+        imagePullPolicy: IfNotPresent
         name: redis-service
         ports:
         - containerPort: 6379
+        securityContext:
+          runAsUser: 999
       restartPolicy: Always

deployment/kubernetes/helm/cdn-transcode/templates/zookeeper-service-deployment.yaml

@@ -32,7 +32,10 @@ spec:
         - name: ZOOKEEPER_TICK_TIME
           value: "2000"
         image: zookeeper:latest
+        imagePullPolicy: IfNotPresent
         name: zookeeper-service
         ports:
         - containerPort: 2181
+        securityContext:
+          runAsUser: 1000
       restartPolicy: Always

deployment/kubernetes/yaml/kafka-deploy.yaml.m4

@@ -20,7 +20,7 @@ spec:
       enableServiceLinks: false
       containers:
         - name: kafka
-          image: wurstmeister/kafka:2.12-2.4.0
+          image: defn(`REGISTRY_PREFIX')ovc_kafka_service:latest
           imagePullPolicy: IfNotPresent
           ports:
             - containerPort: 9092
@@ -53,6 +53,8 @@ spec:
               value: "`-Xmx'defn(`KAFKA_MEMORY')m -`Xms'defn(`KAFKA_MEMORY')m"
             - name: "KAFKA_LOG4J_ROOT_LOGLEVEL"
               value: "ERROR"
+          securityContext:
+            runAsUser: 1000
           resources:
               requests:
                   cpu: defn(`KAFKA_CPU')

deployment/kubernetes/yaml/redis-deploy.yaml.m4

@@ -25,6 +25,8 @@ spec:
             - redis-server 
           ports:
             - containerPort: 6379
+          securityContext:
+            runAsUser: 999
           resources:
               requests:
                   cpu: defn(`REDIS_CPU')

deployment/kubernetes/yaml/zookeeper-deploy.yaml.m4

@@ -33,6 +33,8 @@ spec:
               value: "1"
             - name: "ZOO_LOG4J_PROP"
               value: "ERROR"
+          securityContext:
+            runAsUser: 1000
           resources:
               requests:
                   cpu: defn(`ZOOKEEPER_CPU')

kafka/CMakeLists.txt

@@ -0,0 +1,2 @@
+set(service "ovc_kafka_service")
+include("${CMAKE_SOURCE_DIR}/script/service.cmake")