Merge pull request #8 from intel-iot-devkit/upload_svet_2020.2.0

upload code for svet linux 2020.2.0 with following features
1) Support using VPP instead of SFC for scaling and color format conversion in decoding session.
2) Support H265 RTSP stream as input.
3) Support new inference options “-infer::interval” and “-infer::max_detect”.
4) Support disabling composition by adding new sink session type “-fake_sink”.
5) Upgrade the MediaSDK to version 2020.1.1 and OpenVINO to version 2020.3.
6) Support pure multiple decoding performance testing.

Author: maozhong1

README.md

@@ -14,7 +14,7 @@ Sample par files can be found in par_files directory. Verfied on i7-8559U. Perfo
 The sample application depends on [Intel® Media SDK](https://github.com/Intel-Media-SDK/), [Intel® OpenVINO™](https://software.intel.com/en-us/openvino-toolkit) and [FFmpeg](https://www.ffmpeg.org/)
 
 # FAQ
-See doc/FAQ.md
+See [FAQ](./doc/FAQ.md)
 
 # Table of contents
 
@@ -33,27 +33,27 @@ The sample application is licensed under MIT license. See [LICENSE](./LICENSE) f
 See [CONTRIBUTING](./doc/CONTRIBUTING.md) for details. Thank you!
 
 # Documentation
-See [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.1.0.pdf)
+See [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.2.0.pdf)
 
 # System requirements
 
 **Operating System:**
 * Ubuntu 18.04.02
 
 **Software:**
-* [MediaSDK 19.4.0](https://github.com/Intel-Media-SDK/MediaSDK/releases/tag/intel-mediasdk-19.4.0)
-* [OpenVINO™ 2019 R3](https://software.intel.com/en-us/openvino-toolkit)
+* [MediaSDK 20.1.1](https://github.com/Intel-Media-SDK/MediaSDK/releases/tag/intel-mediasdk-20.1.1)
+* [OpenVINO™ 2020.3](https://software.intel.com/en-us/openvino-toolkit)
 
 **Hardware:** 
-* Intel® platforms supported by the MediaSDK 19.4.0 and OpenVINO 2019 R3. 
+* Intel® platforms supported by the MediaSDK 20.1.1 and OpenVINO 2020.3. 
 * For Media SDK, the major platform dependency comes from the back-end media driver. https://github.com/intel/media-driver
 * For OpenVINO™, see details from here: https://software.intel.com/en-us/openvino-toolkit/documentation/system-requirements
 
 # How to build
 
 Run build_and_install.sh to install dependent software packages and build sample application video_e2e_sample. 
 
-Please refer to ”Installation Guide“ in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.1.0.pdf) for details.
+Please refer to ”Installation Guide“ in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.2.0.pdf) for details.
 
 ## Build steps
 
@@ -68,7 +68,7 @@ cd cva_sample
 ```
 This script will install the dependent software packages by running command "apt install". So it will ask for sudo password. Then it will download libva, libva-util, media-driver and MediaSDK source code and install these libraries. It might take 10 to 20 minutes depending on the network bandwidth.
 
-After the script finishing, the sample application video_e2e_sample can be found under ./bin. Please refer to "Run sample application" in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.1.0.pdf) for details.
+After the script finishing, the sample application video_e2e_sample can be found under ./bin. Please refer to "Run sample application" in [user guide](./doc/concurrent_video_analytic_sample_application_user_guide_2020.2.0.pdf) for details.
 
 # Known limitations
 

build_and_install.sh

@@ -8,7 +8,7 @@ then echo "Please make sure openvino has been installed and enviroment variables
     exit -1;
 fi
 
-./msdk_pre_install.py
+./msdk_pre_install_internal.py
 
 git_projects="MediaSDK media-driver libva"
 for i in $git_projects;
@@ -22,9 +22,8 @@ do
 done
 
 cd "MediaSDK"
-git reset --hard
-patch -p1 < ../patches/0001-MSDK-Enable-AVC-decode-SFC-RGB4.patch
-patch -p1 < ../patches/0002-sample_common-Add-support-to-multiple-displays.patch
+patch -N --no-backup-if-mismatch -p1 < ../patches/0001-MSDK-Enable-AVC-decode-SFC-RGB4.patch
+patch -N --no-backup-if-mismatch -p1 < ../patches/0002-sample_common-Add-support-to-multiple-displays.patch
 
 if [ $? != 0 ]; then
     echo "Apply patch failed!"
@@ -33,7 +32,7 @@ fi
 cd ../
 cp -rf video_e2e_sample  MediaSDK/samples/
 
-./msdk_pre_install.py -b cfl
+./msdk_pre_install_internal.py -b cfl
 
 if [[ -f MediaSDK/build/__bin/release/video_e2e_sample ]];
 then
@@ -63,6 +62,17 @@ then
         echo 'export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/intel/mediasdk/lib"' >> ~/.bashrc
     fi
 
+    if [[ -d $cl_cache_dir ]];
+    then
+        echo "cl_cache is enabled. \$cl_cache_dir : $cl_cache_dir"
+    else
+        echo "Add enabling cl_cache commands to ~\.bashrc"
+        set -x
+        mkdir -p ~/cl_cache
+        export cl_cache_dir=~/cl_cache
+        echo "mkdir -p ~/cl_cache" >> ~/.bashrc
+        echo "export cl_cache_dir=~/cl_cache" >> ~/.bashrc
+    fi
     echo "Sample application building has completed!"
     echo "Please use ./bin/video_e2e_sample for testing"
 fi

doc/CONTRIBUTING.md

@@ -1,8 +1,8 @@
-We welcome community contributions to SVET sample application. Thank you for your time!
+We welcome community contributions to the concurrent video analytic sample application. Thank you for your time!
 
 Please note that review and merge might take some time at this point.
 
-SVET sample application is licensed under MIT license. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
+The sample application is licensed under MIT license. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
 
 Steps:
  - In the commit message, explain what bug is fixed or what new feature is added in details.
@@ -19,7 +19,7 @@ Software Requirements: Ubuntu 18.04, MediaSDK 19.4.0 and OpenVINO 2019 R3
 
 ## How to test your changes
 
-### 1. Build the SVET sample applicaton
+### 1. Build the sample applicaton
 
 ```sh
 ./build_and_install.sh
@@ -51,7 +51,7 @@ n16_1080p_rtsp_simu.pa
 n4_1080p_rtsp_simu_dp.par
 n16_1080p_rtsp_simu_dump.par
 
-Multiple display tests:
+Multiple display test:
 ```sh
 ./bin/video_e2e_sample -par par_file/n16_1080p_1080p_dp_noinfer.par -par par_file/n16_1080p_1080p_dp.par
 ```

doc/FAQ.md

@@ -4,15 +4,21 @@
 See chapter 2.4 in doc/svet_sample_application_user_guide_2020.1.0.pdf
 Running the SVET sample applicton with option "-?" can show the usage of options.
 
-## Why does the system need to be switched to text mode before running the sample application
-The sample application uses libDRM to render the video directly to display, so it needs to act as master of the DRM display, which isn't allowed when X server is running. 
+## Why does the system need to be switched to text console mode before running the sample application
+The sample application uses libDRM to render the video directly to display, so it needs to act as master of the DRM display, which isn't allowed when X client is running. If there is any VNC session, please also close it. Because VNC session also starts X client. 
 If the par file doesn't include display session, there is no need to switch to text mode.
 
 ## Why it needs "su -p" to switch to root user before running the sample application
 To become DRM master, it needs root privileges. With option "-p", it will preserve environment variables, like LIBVA_DRIVERS_PATH, LIBVA_DRIVER_NAME and LD_LIBRARY_PATH. If without "-p", these environment variables will be reset and the sample application will run into problems.
 
+## Is it possible to use X11 instead of DRM display?
+If user doesn't want to switch to text console mode or switch to root for using DRM display, user can replace "-rdrm-DisplayPort" with "-rx11" in the par file. However, the X11 rendering isn't as effcient as DRM rendering. According to our 16-channel face detection 1080p test on CFL , the time cost of each frame increased by around 6ms. Example [par file](./par_file/inference/n16_face_detection_1080p_x11.par) using X11 as rendering method. 
+
+## Is there any limitation of the order of decoding, encoding and dislay sessions in par file
+Yes. The decoding dessions must be descripted firstly. If there is display dession, it must be the last line in par file.
+
 ## The loading time of 16-channel face detection demo is too long
-Please enable cl_cache by running command "export cl_cache_dir=/tmp/cl_cache" and "mkdir -p /tmp/cl_cache". Then after the first running of 16-channel face detection demo, the compiled OpenCL kernles are cached and the model loading time of next runnings of 16-channel face detection demo will only take about 10 seconds.
+Please make sure cl_cache is enabled by command "echo $cl_cache_dir". If this environment isn't set, please enable cl_cache by running command "export cl_cache_dir=/tmp/cl_cache" and "mkdir -p /tmp/cl_cache". Then after the first running of 16-channel face detection demo, the compiled OpenCL kernles are cached and the model loading time of next runnings of 16-channel face detection demo will only take about 10 seconds.
 More details about cl_cache can be found at https://github.com/intel/compute-runtime/blob/master/opencl/doc/FAQ.md
 
 ## Can sources number for "-vpp_comp_only" or "-vpp_comp" be different from number of decoding sessions?  
@@ -21,15 +27,22 @@ No. The sources number for "-vpp_comp_only" or "-vpp_comp" must be equal to the
 ## How to limit the fps of whole pipeline to 30?
 Add "-fps 30" to every decoding session.
 
+## "-fps 30" doesn't work with "-fake_sink"
+Fake sink session doesn't support "-fps 30". Please add "-fps 30" to every decoding session instead.
+
+Add "-fps 30" to every decoding session.
 ## How to limit the frame number of input to 1000?
-Add "-n 1000" to every decoding dessions. However this option won't work if both "-vpp_comp_only" and "-vpp_comp" are set.
+Add "-n 1000" to every decoding dessions. But please not add "-n" to encode, display and fake sink session. These sink sessions will automatically stop when the source session stops. Note, this option won't work if both "-vpp_comp_only" and "-vpp_comp" are set. 
 
 ## Where can I find tutorials for inference engine?
 Please refer to https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html
 
+## Why HDDL card usage ratio is low for face detection inference?
+It can be caused by the decoded frames aren't fed to inference engine efficiently. The default inference interval of face detection is 6. You try to set the inference interval to a lower valuer when using HDDL as inference target device. For example, with 3 HDDL L2 card, adding "-infer::inverval 1" to 16-channel face detection par file can increase the HDDL usage ratio to 100%.  
+
 ## Where can I find information for the models?
 Please refer to https://github.com/opencv/open_model_zoo/tree/master/models/intel. The names of models used in sample application are
 face-detection-retail-0004, human-pose-estimation-0001, vehicle-attributes-recognition-barrier-0039, vehicle-license-plate-detection-barrier-0106.
 
-## Can I use other OpenVINO version rather than 2019 R3?
+## Can I use other OpenVINO version rather than 2020.3 ?
 Yes, but you have to modify some code due to interfaces changing. And also you need to download the IR files and copy them to ./model manually. Please refer to script/download_and_copy_models.sh for how to download the IR files.

doc/concurrent_video_analytic_sample_application_release_notes_2020.2.0.pdf

@@ -127,7 +127,7 @@ def fnParseCommandline():
 
         print "Please input the sudo password to proceed\n"
         cmd ="sudo apt-get -y install git libssl-dev dh-autoreconf cmake libgl1-mesa-dev libpciaccess-dev build-essential curl imagemagick unzip yasm libjpeg-dev libavcodec-dev libavutil-dev libavformat-dev;"
-        cmd+="sudo apt-get -y install libopencv-dev checkinstall pkg-config libgflags-dev"
+        cmd+="sudo apt-get -y install checkinstall pkg-config"
         os.system(cmd)
 
         print ""
@@ -138,41 +138,38 @@ def fnParseCommandline():
         # Pull all the source code
         print "libva"
         if not os.path.exists("%s/libva"%(WORKING_DIR)):
-            cmd = "cd %s; git clone https://github.com/intel/libva.git;"%(WORKING_DIR) 
-            cmd+= "cd libva;"
-            cmd+="git checkout 5f693d9e603e0e83928cec67c30b6ac902d7aa85;" # 04/12/18
+            cmd = "cd %s; rm -f libva.tar.gz; wget https://github.com/intel/libva/archive/2.7.1.tar.gz -O libva.tar.gz;"%(WORKING_DIR) 
+            cmd+= "tar zxf libva.tar.gz; mv libva-2.7.1 libva"
             print cmd
             os.system(cmd);
 
         print "libva-utils"
         if not os.path.exists("%s/libva-utils"%(WORKING_DIR)):
-            cmd = "cd %s; git clone https://github.com/intel/libva-utils.git;"%(WORKING_DIR)
-            cmd += "cd libva-utils;"
-            cmd+="git checkout 8ea1eba433dcbceb0e5dcb54b8e3f984987f7a17; " # 03/20/18
+            cmd = "cd %s;rm -f libva-utils.tar.gz;"%(WORKING_DIR)
+            cmd += "wget https://github.com/intel/libva-utils/archive/2.7.1.tar.gz -O libva-utils.tar.gz;"
+            cmd += "tar zxf libva-utils.tar.gz; mv libva-utils-2.7.1 libva-utils;"
             print cmd
             os.system(cmd);
 
         print "media-driver"
         if not os.path.exists("%s/media-driver"%(WORKING_DIR)): 
-            cmd = "cd %s; git clone https://github.com/intel/media-driver.git; "%(WORKING_DIR)
-            cmd += "cd media-driver;"
-            cmd+= "git checkout 12b7fcded6c74377ecf57eb8258f5e3d55ca722e" # 04/16/18
+            cmd = "cd %s; rm -f intel-media-20.1.1.tar.gz;"%(WORKING_DIR)
+            cmd += "wget https://github.com/intel/media-driver/archive/intel-media-20.1.1.tar.gz; "
+            cmd += "tar zxf intel-media-20.1.1.tar.gz; mv media-driver-intel-media-20.1.1 media-driver"
             print cmd
             os.system(cmd);
 
         print "gmmlib"
         if not os.path.exists("%s/gmmlib"%(WORKING_DIR)): 
-            cmd = "cd %s; git clone https://github.com/intel/gmmlib.git; "%(WORKING_DIR)
-            cmd += "cd gmmlib;"
-            cmd+= "git checkout ebfcfd565031dbd7b45089d9054cd44a501f14a9" # 04/05/18
+            cmd = "cd %s;rm -f intel-gmmlib-20.1.1.tar.gz; wget https://github.com/intel/gmmlib/archive/intel-gmmlib-20.1.1.tar.gz; "%(WORKING_DIR)
+            cmd += "tar zxf intel-gmmlib-20.1.1.tar.gz; mv gmmlib-intel-gmmlib-20.1.1 gmmlib"
             print cmd
             os.system(cmd);
 
         print "MediaSDK"
         if not os.path.exists("%s/MediaSDK"%(WORKING_DIR)): 
-            cmd = "cd %s; git clone https://github.com/Intel-Media-SDK/MediaSDK.git; "%(WORKING_DIR)
-            cmd+= "cd MediaSDK;"
-            cmd+= "git checkout f8f6646afcaec410ca037f0176814dd9d7ca4900" # 04/16/18
+            cmd = "cd %s; rm -f intel-mediasdk-20.1.1.tar.gz;  wget https://github.com/Intel-Media-SDK/MediaSDK/archive/intel-mediasdk-20.1.1.tar.gz; "%(WORKING_DIR)
+            cmd+= "tar zxf intel-mediasdk-20.1.1.tar.gz; mv MediaSDK-intel-mediasdk-20.1.1 MediaSDK"
             print cmd
             os.system(cmd);
 
@@ -209,9 +206,9 @@ def fnParseCommandline():
         print cmd
         os.system(cmd)
 
-        cmd = "mkdir -p %s/media-driver/build; "%(WORKING_DIR)
-        cmd+= "cd %s/media-driver/build; "%(WORKING_DIR)
-        cmd+= "cmake ../; "
+        cmd = "mkdir -p %s/media_build; "%(WORKING_DIR)
+        cmd+= "cd %s/media_build; "%(WORKING_DIR)
+        cmd+= "cmake ../media-driver; "
         cmd+= "make -j4; sudo make install"
         print cmd
         os.system(cmd)
@@ -227,7 +224,6 @@ def fnParseCommandline():
             sys.exit() 
 
         # Build and install Media SDK library and samples
-        # MediaSDK origin: 22dae397ae17d29b4734adb0e5a998f32a9c25b2 (022218)
         cmd ="cd %s/MediaSDK; "%(WORKING_DIR)
         cmd+="mkdir -p build; "
         cmd+="cd build; "
@@ -243,22 +239,6 @@ def fnParseCommandline():
             print "MediaSDK source code doen't exist!"
             sys.exit()
 
-        r=subprocess.check_output("grep -c \"export LIBVA_DRIVER_NAME=iHD\" ~/.bashrc", shell=True)
-
-        if r >= 1:
-            sys.exit()
-
-        print ""
-        print "************************************************************************"
-        print_info("Set environment variables. ", loglevelcode.INFO)
-        print "************************************************************************"
-        cmd = "echo 'export MFX_HOME=/opt/intel/mediasdk' >> ~/.bashrc; "
-        cmd+= "echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/usr/lib' >> ~/.bashrc; "
-        cmd+= "echo 'export LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri' >> ~/.bashrc; "
-        cmd+= "echo 'export LIBVA_DRIVER_NAME=iHD' >> ~/.bashrc"
-        print cmd
-        os.system(cmd)
-
         cmd = "sudo echo '/usr/lib/x86_64-linux-gnu' > /etc/ld.so.conf.d/libdrm-intel.conf; "
         cmd+= "sudo echo '/usr/lib' >> /etc/ld.so.conf.d/libdrm-intel.conf; "
         cmd+= "sudo ldconfig"

doc/concurrent_video_analytic_sample_application_user_guide_2020.2.0.pdf

@@ -0,0 +1 @@
+-i::h264 /home/intel/Downloads/classroom.h264 -join -hw -async 4 -o::jpeg ./test_#.jpeg -w 1920 -h 1080 -ext_allocator -fps 30 -q 100 -frameskip 9

msdk_pre_install.py

@@ -0,0 +1,4 @@
+-i::h264 /home/intel/Downloads/classroom.h264 -join -hw -async 4 -o::jpeg ./test_1_#.jpeg -w 1920 -h 1080 -ext_allocator -fps 30 -q 100 -frameskip 9
+-i::h264 /home/intel/Downloads/classroom.h264 -join -hw -async 4 -o::jpeg ./test_2_#.jpeg -w 720 -h 480 -ext_allocator -fps 30 -q 100 -frameskip 19
+-i::h264 /home/intel/Downloads/classroom.h264 -join -hw -async 4 -o::jpeg ./test_3_#.jpeg -w 960 -h 540 -ext_allocator -fps 30 -q 100 -frameskip 99
+-i::h264 /home/intel/Downloads/classroom.h264 -join -hw -async 4 -o::jpeg ./test_4_#.jpeg -w 480 -h 256 -ext_allocator -fps 30 -q 100 -frameskip 999