Skip to content

Latest commit

 

History

History
116 lines (97 loc) · 4.51 KB

README.md

File metadata and controls

116 lines (97 loc) · 4.51 KB

SSD-MobileNet BFloat16 inference

This document has instructions for running SSD-MobileNet BFloat16 inference using Intel-optimized TensorFlow.

Datasets

The COCO validation dataset is used in these SSD-Mobilenet quickstart scripts. The accuracy quickstart script require the dataset to be converted into the TF records format. See the COCO dataset for instructions on downloading and preprocessing the COCO validation dataset.

Set the DATASET_DIR to point to the dataset directory that contains the TF records file coco_val.record when running accuracy script.

Quick Start Scripts

Script name Description
inference_realtime.sh Runs multi instance realtime inference using 4 cores per instance for the specified precision (fp32 or bfloat16). Waits for all instances to complete, then prints a summarized throughput value.
inference_throughput.sh Runs multi instance batch inference using 1 instance per socket for the specified precision (fp32 or bfloat16). Waits for all instances to complete, then prints a summarized throughput value.
accuracy.sh Measures the inference accuracy (providing a DATASET_DIR environment variable is required) for the specified precision (fp32 or bfloat16).

Run the model

Setup your environment using the instructions below, depending on if you are using AI Kit:

Setup using AI Kit on Linux Setup without AI Kit on Linux

To run using AI Kit on Linux you will need:

  • numactl
  • wget
  • build-essential
  • Cython
  • contextlib2
  • jupyter
  • lxml
  • matplotlib
  • pillow>=8.1.2
  • pycocotools
  • Activate the `tensorflow` conda environment 
    conda activate tensorflow

To run without AI Kit on Linux you will need:

  • Python 3
  • git
  • numactl
  • wget
  • intel-tensorflow>=2.5.0
  • build-essential
  • Cython
  • contextlib2
  • jupyter
  • lxml
  • matplotlib
  • pillow>=8.1.2
  • pycocotools
  • A clone of the Model Zoo repo
    git clone https://github.com/IntelAI/models.git

For more information on the dependencies, see the documentation on prerequisites in the TensorFlow models repo.

Download the pretrained model and set the PRETRAINED_MODEL environment variable to the path of the frozen graph. For Linux, run:

wget https://storage.googleapis.com/intel-optimized-tensorflow/models/v1_8/ssdmobilenet_fp32_pretrained_model_combinedNMS.pb
export PRETRAINED_MODEL=$(pwd)/ssdmobilenet_fp32_pretrained_model_combinedNMS.pb

After installing the prerequisites and downloading the pretrained model, set the environment variables for the paths to your PRETRAINED_MODEL, an OUTPUT_DIR where log files will be written, DATASET_DIR for COCO raw dataset directory or tf_records file based on whether you run inference or accuracy scripts and PRECISION to bfloat16. Navigate to your model zoo directory and then run a quickstart script on Linux.

Run on Linux

# cd to your model zoo directory
cd models

# Set environment variables
export PRETRAINED_MODEL=<path to the downloaded frozen graph>
export DATASET_DIR=<path to the directory where coco tf record file is stored>
export OUTPUT_DIR=<path to the directory where log files will be written>
export PRECISION=bfloat16
# For a custom batch size, set env var `BATCH_SIZE` or it will run with a default value.
export BATCH_SIZE=<customized batch size value>

# Run a quickstart script (for example, inference realtime.sh)
./quickstart/object_detection/tensorflow/ssd-mobilenet/inference/cpu/<script name>.sh

Additional Resources

  • To run more advanced use cases, see the instructions here for calling the launch_benchmark.py script directly.