[torch deploy] Update deploy.rst with working simple example (pytorch#76538)

Summary:
Pull Request resolved: pytorch#76538

when running the example from the docs, I found that these steps were not working.

These are the updates necessary to get the example working.

Test Plan: n/a

Reviewed By: PaliC

Differential Revision: D35998155

fbshipit-source-id: d78bb2886f94889abae5a3af5239fcd306cd5e09
(cherry picked from commit 6893812)

Author: s4ayub

docs/source/deploy.rst

@@ -29,8 +29,7 @@ When running ``setup.py``, you will need to specify ``USE_DEPLOY=1``, like:
 
     export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
     export USE_DEPLOY=1
-    python setup.py bdist_wheel
-    python -mpip install dist/*.whl
+    python setup.py develop
 
 
 Creating a model package in Python
@@ -53,28 +52,39 @@ For now, let's create a simple model that we can load and run in ``torch::deploy
     # Package and export it.
     with PackageExporter("my_package.pt") as e:
         e.intern("torchvision.**")
+        e.extern("numpy.**")
         e.extern("sys")
+        e.extern("PIL.*")
         e.save_pickle("model", "model.pkl", model)
 
+Note that since "numpy", "sys" and "PIL" were marked as "extern", `torch.package` will
+look for these dependencies on the system that loads this package. They will not be packaged
+with the model.
+
 Now, there should be a file named ``my_package.pt`` in your working directory.
 
-.. note::
 
-    Currently, ``torch::deploy`` supports only the Python standard library and
-    ``torch`` as ``extern`` modules in ``torch.package``. In the future we plan
-    to transparently support any Conda environment you point us to.
+Loading and running the model in C++
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Set an environment variable (e.g. $PATH_TO_EXTERN_PYTHON_PACKAGES) to indicate to the interpreters
+where the external Python dependencies can be found. In the example below, the path to the
+site-packages of a conda environment is provided.
 
+.. code-block:: bash
 
+    export PATH_TO_EXTERN_PYTHON_PACKAGES= \
+        "~/anaconda/envs/deploy-example-env/lib/python3.8/site-packages"
 
-Loading and running the model in C++
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Let's create a minimal C++ program to that loads the model.
 
 .. code-block:: cpp
 
-    #include <torch/deploy.h>
+    #include <torch/csrc/deploy/deploy.h>
+    #include <torch/csrc/deploy/path_environment.h>
     #include <torch/script.h>
+    #include <torch/torch.h>
 
     #include <iostream>
     #include <memory>
@@ -86,14 +96,19 @@ Let's create a minimal C++ program to that loads the model.
         }
 
         // Start an interpreter manager governing 4 embedded interpreters.
-        torch::deploy::InterpreterManager manager(4);
+        std::shared_ptr<torch::deploy::Environment> env =
+            std::make_shared<torch::deploy::PathEnvironment>(
+                std::getenv("PATH_TO_EXTERN_PYTHON_PACKAGES")
+            );
+        torch::deploy::InterpreterManager manager(4, env);
 
         try {
             // Load the model from the torch.package.
             torch::deploy::Package package = manager.loadPackage(argv[1]);
             torch::deploy::ReplicatedObj model = package.loadPickle("model", "model.pkl");
         } catch (const c10::Error& e) {
             std::cerr << "error loading the model\n";
+            std::cerr << e.msg();
             return -1;
         }
 
@@ -105,6 +120,9 @@ This small program introduces many of the core concepts of ``torch::deploy``.
 An ``InterpreterManager`` abstracts over a collection of independent Python
 interpreters, allowing you to load balance across them when running your code.
 
+``PathEnvironment`` enables you to specify the location of Python
+packages on your system which are external, but necessary, for your model.
+
 Using the ``InterpreterManager::loadPackage`` method, you can load a
 ``torch.package`` from disk and make it available to all interpreters.
 
@@ -120,20 +138,55 @@ an free interpreter to execute that interaction.
 Building and running the application
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+Locate `libtorch_deployinterpreter.o` on your system. This should have been
+built when PyTorch was built from source. In the same PyTorch directory, locate
+the deploy source files. Set these locations to an environment variable for the build.
+An example of where these can be found on a system is shown below.
+
+.. code-block:: bash
+
+    export DEPLOY_INTERPRETER_PATH="/pytorch/build/torch/csrc/deploy/"
+    export DEPLOY_SRC_PATH="/pytorch/torch/csrc/deploy/"
+
+As ``torch::deploy`` is in active development, these manual steps will be removed
+soon.
+
 Assuming the above C++ program was stored in a file called, `example-app.cpp`, a
 minimal CMakeLists.txt file would look like:
 
 .. code-block:: cmake
 
-    cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
+    cmake_minimum_required(VERSION 3.19 FATAL_ERROR)
     project(deploy_tutorial)
 
+    find_package(fmt REQUIRED)
     find_package(Torch REQUIRED)
 
-    add_executable(example-app example-app.cpp)
-    target_link_libraries(example-app "${TORCH_LIBRARIES}")
-    set_property(TARGET example-app PROPERTY CXX_STANDARD 14)
+    add_library(torch_deploy_internal STATIC
+        ${DEPLOY_INTERPRETER_PATH}/libtorch_deployinterpreter.o
+        ${DEPLOY_DIR}/deploy.cpp
+        ${DEPLOY_DIR}/loader.cpp
+        ${DEPLOY_DIR}/path_environment.cpp
+        ${DEPLOY_DIR}/elf_file.cpp)
+
+    # for python builtins
+    target_link_libraries(torch_deploy_internal PRIVATE
+        crypt pthread dl util m z ffi lzma readline nsl ncursesw panelw)
+    target_link_libraries(torch_deploy_internal PUBLIC
+        shm torch fmt::fmt-header-only)
+    caffe2_interface_library(torch_deploy_internal torch_deploy)
+
+    add_executable(example-app example.cpp)
+    target_link_libraries(example-app PUBLIC
+        "-Wl,--no-as-needed -rdynamic" dl torch_deploy "${TORCH_LIBRARIES}")
+
+Currently, it is necessary to build ``torch::deploy`` as a static library.
+In order to correctly link to a static library, the utility ``caffe2_interface_library``
+is used to appropriately set and unset ``--whole-archive`` flag.
 
+Furthermore, the ``-rdynamic`` flag is needed when linking to the executable
+to ensure that symbols are exported to the dynamic table, making them accessible
+to the deploy interpreters (which are dynamically loaded).
 
 The last step is configuring and building the project. Assuming that our code
 directory is laid out like this:
@@ -152,8 +205,9 @@ We can now run the following commands to build the application from within the
     mkdir build
     cd build
     # Point CMake at the built version of PyTorch we just installed.
-    SITE_PACKAGES="$(python -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())')"
-    cmake -DCMAKE_PREFIX_PATH="$SITE_PACKAGES/torch" ..
+    cmake -DCMAKE_PREFIX_PATH="$(python -c 'import torch.utils; print(torch.utils.cmake_prefix_path)')" .. \
+        -DDEPLOY_INTERPRETER_PATH="$DEPLOY_INTERPRETER_PATH" \
+        -DDEPLOY_DIR="$DEPLOY_DIR"
     cmake --build . --config Release
 
 Now we can run our app:

torch/csrc/deploy/README.md

@@ -20,3 +20,8 @@ Because CPython builds successfully when optional dependencies are missing, the
 To be safe, install the [complete list of dependencies for CPython](https://devguide.python.org/setup/#install-dependencies) for your platform, before trying to build torch with USE_DEPLOY=1.
 
 If you already built CPython without all the dependencies and want to fix it, just blow away the CPython folder under torch/csrc/deploy/third_party, install the missing system dependencies, and re-attempt the pytorch build command.
+
+# Example
+
+Read the [getting started guide](https://github.com/pytorch/pytorch/blob/master/docs/source/deploy.rst) for an
+example on how to use `torch::deploy`.