Connecting debugger and TensorFlow

[technologiclee]

I have followed the instructions and changed the example on two computers:

sys.path.append('/home/evjang/thesis/tensor_debugger')

sys.path.append('/home/lee/softwareinstalled/anaconda3-5/tdb_ext')

sys.path.append('/home/mylao/tdb')

Is this the correct location?
I get this message:
“Waiting for TDB to connect...”

The MNIST exapmle was not in the tdb_ext download so I cloned TDB from Git also.
https://github.com/ericjang/tdb/releases/download/tdb_ext_v0.1/tdb_ext.zip
https://github.com/ericjang/tdb.git

---

import notebook.nbextensions
import urllib
import zipfile
SOURCE_URL = 'https://github.com/ericjang/tdb/releases/download/tdb_ext_v0.1/tdb_ext.zip'
urllib.urlretrieve(SOURCE_URL, 'tdb_ext.zip')
with zipfile.ZipFile('tdb_ext.zip', "r") as z:
z.extractall("")
notebook.nbextensions.install_nbextension('tdb_ext',user=True)

There has been some change, I think it is supposed to be like this now:
http://stackoverflow.com/questions/17960942/attributeerror-module-object-has-no-attribute-urlretrieve

import urllib.request
data = urllib.request.urlretrieve("http://...")

---

I foolishly thought this comment was changing the location was was trying to modify it there!

sys.path.append('/home/

Now I think it means to change /home/.bashrc
Here is a helpful note for noobs like me:

add this line to the bottom of /home/.bashrc

export PATH="/home/lee/softwareInstalled/anaconda3-5/tdb_ext:$PATH"

refresh .bashrc with . ~/.bashrc or logout and logback in

It seems to load tensorflow and urllib but not the other imports.

This is the bottom of /home/.bashrc

added by Anaconda3 2.4.1 installer

export PATH="/home/lee/anaconda3/bin:$PATH"
export PATH="/home/lee/softwareInstalled/anaconda3-5/tdb_ext:$PATH"
export PATH="/home/lee/softwareInstalled/anaconda3-5/tdb_ext/tdb:$PATH"
export PATH="/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/examples:$PATH"

lee@lee-VGN-NR21E-S:~$ echo $PATH
/home/lee/softwareInstalled/anaconda3-5/tdb_ext/tdb:/home/lee/softwareInstalled/anaconda3-5/tdb_ext:/home/lee/anaconda3/bin:/home/lee/softwareInstalled/anaconda3-5/tdb_ext/tdb:/home/lee/anaconda3/bin:/home/lee/softwareInstalled/anaconda3-5/tdb_ext:/home/lee/anaconda3/bin:/home/lee/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

import tdb
from tdb.examples import mnist, viz
import matplotlib.pyplot as plt
import tensorflow as tf

import urllib

ImportError Traceback (most recent call last)
in ()
5 #refresh .bashrc with . ~/.bashrc or logout and logback in
6
----> 7 import tdb
8 from tdb.examples import mnist, viz
9 import matplotlib.pyplot as plt

/home/lee/anaconda3/lib/python3.5/site-packages/tdb/init.py in ()
6 """
7
----> 8 from interface import debug, c, s, get_exe_queue, get_value
9 import op_store
10 from plot_op import plot_op

ImportError: No module named 'interface'

---

I uncommented this line, now I get:
sys.path.append('/home/lee/softwareinstalled/anaconda3-5/tdb_ext')

----> 2 sys.path.append('/home/lee/softwareinstalled/anaconda3-5/tdb_ext')

NameError: name 'sys' is not defined

---

I uncommented this line so the 'sys' error goes away.
import sys

Now I am back to this error:

7 import tdb
8 from tdb.examples import mnist, viz
9 import matplotlib.pyplot as plt

/home/lee/anaconda3/lib/python3.5/site-packages/tdb/init.py in ()
6 """
7
----> 8 from interface import debug, c, s, get_exe_queue, get_value
9 import op_store
10 from plot_op import plot_op

ImportError: No module named 'interface'

---

Now I have this:
import sys
sys.path.append('/home/lee/softwareinstalled/anaconda3-5/tdb_ext')
sys.path.append('/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/')
sys.path.append('/usr/local/lib/python2.7/dist-packages/tensorflow')

/home/lee/anaconda3/lib/python3.5/site-packages/tdb/init.py in ()
6 """
7
----> 8 from interface import debug, c, s, get_exe_queue, get_value
9 import op_store
10 from plot_op import plot_op

/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/interface.py in ()
4 """
5
----> 6 import debug_session
7
8 # default session

/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/debug_session.py in ()
1
2 from ht_op import HTOp
----> 3 import op_store
4 import tensorflow as tf
5

/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/op_store.py in ()
1 from toposort import toposort, toposort_flatten
2 from transitive_closure import transitive_closure
----> 3 import tensorflow as tf
4
5 _ops={} # Map<string,tdb.PythonOp>

ImportError: No module named 'tensorflow'

---

lee@lee-VGN-NR21E-S:~$ echo $PATH
/usr/local/lib/python2.7/dist-packages/tensorflow:
/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/examples:
/home/lee/softwareInstalled/anaconda3-5/tdb_ext/tdb:
/home/lee/softwareInstalled/anaconda3-5/tdb_ext:
/home/lee/anaconda3/bin:/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/examples:
/home/lee/softwareInstalled/anaconda3-5/tdb_ext/tdb:/home/lee/softwareInstalled/anaconda3-5/tdb_ext:
/home/lee/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:
/usr/bin:/sbin:/bin:/usr/games:
/usr/local/games

lee@lee-VGN-NR21E-S:~$ echo $PYTHONPATH

---

Now it looks like this, still not finding TensorFlow:

import sys
sys.path.append('/home/lee/softwareinstalled/anaconda3-5/tdb_ext')
sys.path.append('/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/')
sys.path.append('/usr/local/lib/python2.7/dist-packages/tensorflow')
print (sys.path)

import tdb
from tdb.examples import mnist, viz
import matplotlib.pyplot as plt
import tensorflow as tf
import urllib

['', '/home/lee/anaconda3/lib/python35.zip', '/home/lee/anaconda3/lib/python3.5', '/home/lee/anaconda3/lib/python3.5/plat-linux', '/home/lee/anaconda3/lib/python3.5/lib-dynload', '/home/lee/anaconda3/lib/python3.5/site-packages/Sphinx-1.3.1-py3.5.egg', '/home/lee/anaconda3/lib/python3.5/site-packages/setuptools-19.4-py3.5.egg', '/home/lee/anaconda3/lib/python3.5/site-packages', '/home/lee/anaconda3/lib/python3.5/site-packages/cryptography-1.0.2-py3.5-linux-x86_64.egg', '/home/lee/anaconda3/lib/python3.5/site-packages/IPython/extensions', '/home/lee/.ipython', '/home/lee/softwareinstalled/anaconda3-5/tdb_ext', '/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/', '/usr/local/lib/python2.7/dist-packages/tensorflow']

ImportError Traceback (most recent call last)
in ()
8 #refresh .bashrc with . ~/.bashrc or logout and logback in
9
---> 10 import tdb
11 from tdb.examples import mnist, viz
12 import matplotlib.pyplot as plt

/home/lee/anaconda3/lib/python3.5/site-packages/tdb/init.py in ()
6 """
7
----> 8 from interface import debug, c, s, get_exe_queue, get_value
9 import op_store
10 from plot_op import plot_op

/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/interface.py in ()
4 """
5
----> 6 import debug_session
7
8 # default session

/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/debug_session.py in ()
1
2 from ht_op import HTOp
----> 3 import op_store
4 import tensorflow as tf
5

/home/lee/softwareinstalled/anaconda3-5/tdb_ext/tdb/tdb/op_store.py in ()
1 from toposort import toposort, toposort_flatten
2 from transitive_closure import transitive_closure
----> 3 import tensorflow as tf
4
5 _ops={} # Map<string,tdb.PythonOp>

ImportError: No module named 'tensorflow'

Any advice would be appreciated
Thanks,
Lee

[jesuisnicolasdavid]

There is an issue with python3, try all of this in python2 (works for me)

[technologiclee]

@jesuisnicolasdavid Thanks. I tried it again with Anaconda2. It still does not import tdb or tensorflow.

Jupyter was already installed but I started Python in the anaconda2 directory and ran this part again. It seemed to install.

import notebook.nbextensions
import urllib
import zipfile
SOURCE_URL = 'https://github.com/ericjang/tdb/releases/download/tdb_ext_v0.1/tdb_ext.zip'
urllib.urlretrieve(SOURCE_URL, 'tdb_ext.zip')
with zipfile.ZipFile('tdb_ext.zip', "r") as z:
    z.extractall("")
notebook.nbextensions.install_nbextension('tdb_ext',user=True)

Note, for anaconda2 I did not need to change:

import urllib.request
data = urllib.request.urlretrieve("http://...")

I also git cloned TDB into the anaconda2 directory to get the MNIST notebook.

import sys
sys.path.append('/home/lee/anaconda2')
sys.path.append('/home/lee/anaconda2/tdb')
sys.path.append('/home/lee/anaconda2/tdb/tdb')
print (sys.path)
import tdb
from tdb.examples import mnist, viz
import matplotlib.pyplot as plt
import tensorflow as tf
import urll

['', '/home/lee/anaconda2/lib/python27.zip', '/home/lee/anaconda2/lib/python2.7', '/home/lee/anaconda2/lib/python2.7/plat-linux2', '/home/lee/anaconda2/lib/python2.7/lib-tk', '/home/lee/anaconda2/lib/python2.7/lib-old', '/home/lee/anaconda2/lib/python2.7/lib-dynload', '/home/lee/.local/lib/python2.7/site-packages', '/home/lee/anaconda2/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg', '/home/lee/anaconda2/lib/python2.7/site-packages/setuptools-18.5-py2.7.egg', '/home/lee/anaconda2/lib/python2.7/site-packages', '/home/lee/anaconda2/lib/python2.7/site-packages/cryptography-1.0.2-py2.7-linux-x86_64.egg', '/home/lee/anaconda2/lib/python2.7/site-packages/IPython/extensions', '/home/lee/.ipython', '/home/lee/anaconda2', '/home/lee/anaconda2/tdb', '/home/lee/anaconda2/tdb/tdb']

ImportError                               Traceback (most recent call last)
<ipython-input-2-e802af79025a> in <module>()
      4 sys.path.append('/home/lee/anaconda2/tdb/tdb')
      5 print (sys.path)
----> 6 import tdb
      7 from tdb.examples import mnist, viz
      8 import matplotlib.pyplot as plt

/home/lee/.local/lib/python2.7/site-packages/tdb/__init__.py in <module>()
      6 """
      7 
----> 8 from interface import debug, c, s, get_exe_queue, get_value
      9 import op_store
     10 from plot_op import plot_op

/home/lee/.local/lib/python2.7/site-packages/tdb/interface.py in <module>()
      4 """
      5 
----> 6 import debug_session
      7 
      8 # default session

/home/lee/.local/lib/python2.7/site-packages/tdb/debug_session.py in <module>()
      1 
      2 from ht_op import HTOp
----> 3 import op_store
      4 import tensorflow as tf
      5 

/home/lee/.local/lib/python2.7/site-packages/tdb/op_store.py in <module>()
      1 from toposort import toposort, toposort_flatten
      2 from transitive_closure import transitive_closure
----> 3 import tensorflow as tf
      4 
      5 _ops={} # Map<string,tdb.PythonOp>

ImportError: No module named tensorflow

[jesuisnicolasdavid]

Maybe you can try to redo everything in python 2 but within a clean install aka a virtual environnement.

[technologiclee]

@jesuisnicolasdavid Thanks. I tried a virtualenv and ended with 'Too many levels of symbolic links'
So I deleted that environment and tried again with the same results. I think the problem is that it is that it is using the Anaconda2 Python even though I use the command: /usr/bin/python2.7

mkdir vtflow
cd vtflow/
virtualenv vtflow

New python executable in /home/lee/vtflow/vtflow/bin/python
Installing setuptools, pip, wheel...done.

which python
/home/lee/anaconda2/bin/python
virtualenv -p /usr/bin/python2.7 vtflow

Running virtualenv with interpreter /usr/bin/python2.7
Overwriting /home/lee/vtflow/vtflow/lib/python2.7/orig-prefix.txt with new content
New python executable in /home/lee/vtflow/vtflow/bin/python2.7
Not overwriting existing python script /home/lee/vtflow/vtflow/bin/python (you must use /home/lee/vtflow/vtflow/bin/python2.7)
Traceback (most recent call last):
  File "/home/lee/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 2279, in <module>
    main()
  File "/home/lee/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 703, in main
    symlink=options.symlink)
  File "/home/lee/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 884, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/home/lee/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 1311, in install_python
    os.symlink(py_executable_base, full_pth)
OSError: [Errno 17] File exists

source vtflow/bin/activate
pip freeze > requirements.txt

bash: /home/lee/vtflow/vtflow/bin/pip: /home/lee/vtflow/vtflow/bin/python: bad interpreter: Too many levels of symbolic links`

I looked in vtflow/bin
there is a link to python2.7
I will try to find out how to see where this link is to and how to change it.

lrwxrwxrwx 1 lee lee    9  1月 30 23:12 python -> python2.7
lrwxrwxrwx 1 lee lee    6  1月 30 23:11 python2 -> python
lrwxrwxrwx 1 lee lee    6  1月 30 23:11 python2.7 -> python

[ericjang]

@technologiclee did you install Tensorflow using https://www.tensorflow.org/versions/0.6.0/get_started/os_setup.html#virtualenv_install?

It's likely an issue with TensorFlow installation rather than TDB since the latter is pretty simple when it comes to dependencies.

[technologiclee]

@ericjang @jesuisnicolasdavid Thanks!
TDB connected: success !
In the next exciting issue: Where are the images?

I have been using the master branch page not the 0.6.0 version, the pip install link is the same as I used before. This time I used the --system-site-packages option.
virtualenv --system-site-packages ~/tensorflow

Here are my install notes:

sudo apt-get install python-pip python-dev python-virtualenv
virtualenv --system-site-packages ~/tensorflow
source ~/tensorflow/bin/activate

pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl

Collecting tensorflow==0.5.0 from https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
  Downloading https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl (10.9MB)
    100% |████████████████████████████████| 10.9MB 13kB/s 
Requirement already up-to-date: six>=1.10.0 in ./anaconda2/lib/python2.7/site-packages (from tensorflow==0.5.0)
Collecting numpy>=1.9.2 (from tensorflow==0.5.0)
  Downloading numpy-1.10.4.tar.gz (4.1MB)
    100% |████████████████████████████████| 4.1MB 28kB/s 
Building wheels for collected packages: numpy
  Running setup.py bdist_wheel for numpy ... done
  Stored in directory: /home/lee/.cache/pip/wheels/66/f5/d7/f6ddd78b61037fcb51a3e32c9cd276e292343cdd62d5384efd
Successfully built numpy
Installing collected packages: numpy, tensorflow
  Found existing installation: numpy 1.10.1
    Not uninstalling numpy at /home/lee/anaconda2/lib/python2.7/site-packages, outside environment /home/lee/tensorflow

which python
/home/lee/tensorflow/bin/python

`pip install tfdebugger
Requirement already satisfied (use --upgrade to upgrade): tfdebugger in ./.local/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): toposort>=1.4 in ./.local/lib/python2.7/site-packages (from tfdebugger)

Prerequisite: Jupyter installation requires Python 3.3 or greater, or Python 2.7. Older releases of IPython are available here.

Using pip
If you already have Python installed and are not using conda or Anaconda, you may install the Jupyter Notebook using Python’s package manager, pip:
(Use pip instead of pip3 for legacy Python 2.)

pip install jupyter
Requirement already satisfied (use --upgrade to upgrade): jupyter in ./anaconda2/lib/python2.7/site-packages

python
import notebook.nbextensions
import urllib
import zipfile
SOURCE_URL = 'https://github.com/ericjang/tdb/releases/download/tdb_ext_v0.1/tdb_ext.zip'
urllib.urlretrieve(SOURCE_URL, 'tdb_ext.zip')

('tdb_ext.zip', <httplib.HTTPMessage instance at 0x7fe43d215e60>)

with zipfile.ZipFile('tdb_ext.zip', "r") as z:
    z.extractall("")
notebook.nbextensions.install_nbextension('tdb_ext',user=True)

Now exit from the Python interpretor with:
exit()

cd tensorflow/
git clone https://github.com/ericjang/tdb.git
jupyter notebook

This will start the Juypter server in your browser. You can open the floder tbd by clicking on it.
The MNIST example is in the folder Notebooks
To exit the juypter notebook server press
Ctrl+C Crtrl+C
To close the virtual environment type
`deactivat

[technologiclee]

This is fully functional now with virtual environment and Python2. Thanks everyone!

I was wrong about the images. I did not wait long enough for the processing to finish.

[kirangavini]

Hi i am using import tdb using jupyter in windows. I changed path to my system path, i am unable to import tdb, i am thinking tdb as a folderand mnist, viz are the sub files in the folder. Can you help me get pass through this?

[AndyDeNike]

@technologiclee so I finally got TBD Connected: success after executing Run All from the Cell menu, but unfortunately nothing has loaded beyond that.

I have noticed some errors (AttributeError: 'module' object has no attribute 'histogram_summary') appeared after running all as follows:

MNIST Visualization Example

Real-time visualization of MNIST training on a CNN, using TensorFlow and TensorDebugger

The visualizations in this notebook won't show up on http://nbviewer.ipython.org. To view the widgets and interact with them, you will need to download this notebook and run it with a Jupyter Notebook server.
Step 1: Load TDB Notebook Extension

%%javascript

Jupyter.utils.load_extensions('tdb_ext/main')

#import sys

#sys.path.append('/home/evjang/thesis/tensor_debugger')

import tdb

from tdb.examples import mnist, viz

import matplotlib.pyplot as plt

import tensorflow as tf

import urllib

/anaconda2/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters

Step 2: Build TensorFlow Model

(train_data_node,

    train_labels_node,

    validation_data_node,

    test_data_node,

    # predictions

    train_prediction,

    validation_prediction,

    test_prediction,

    # weights

    conv1_weights,

    conv2_weights,

    fc1_weights,

    fc2_weights,

    # training

    optimizer,

    loss,

    learning_rate,

    summaries) = mnist.build_model()

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-f9617cffa412> in <module>()
     16     loss,
     17     learning_rate,
---> 18     summaries) = mnist.build_model()

/anaconda2/lib/python2.7/site-packages/tdb/examples/mnist.pyc in build_model()
    146 
    147   # TRAINING LOSS / REGULARIZATION NODES
--> 148   logits = build_lenet(train_data_node, True)
    149   loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits, train_labels_node))
    150 

/anaconda2/lib/python2.7/site-packages/tdb/examples/mnist.pyc in build_lenet(data, train)
    139       fc1 = tf.nn.dropout(fc1, 0.5, seed=SEED)
    140       # append summary ops to train
--> 141       _activation_summary(conv1)
    142       _activation_summary(fc1)
    143 

/anaconda2/lib/python2.7/site-packages/tdb/examples/mnist.pyc in _activation_summary(x)
     88   # Remove 'tower_[0-9]/' from the name in case this is a multi-GPU training
     89   # session. This helps the clarity of presentation on tensorboard.
---> 90   tf.histogram_summary(x.name + '/activations', x)
     91   tf.scalar_summary(x.name + '/sparsity', tf.nn.zero_fraction(x))
     92 

AttributeError: 'module' object has no attribute 'histogram_summary'

Any idea how I can troubleshoot this?