Monday 23 December 2019

How to install TensorRT Python package on NVIDIA Jetson Nano

Source code of the following Python script contains:

import tensorrt as trt

...and its execution fails:


(tensorflow-demo) nvidia@nvidia-nano:~/dev/nvidia/uff_ssd$ python detect_objects.py images/image1.jpg
Traceback (most recent call last):
  File "detect_objects.py", line 58, in <module>
    import tensorrt as trt
ModuleNotFoundError: No module named 'tensorrt'

TensorRT Pyton module was not installed. I want to share here my experience with the process of setting up TensorRT on Jetson Nano as described here:
A Guide to using TensorRT on the Nvidia Jetson Nano - Donkey Car

$ sudo find / -name nvcc
[sudo] password for nvidia:
find: ‘/run/user/1000/gvfs’: Permission denied
find: ‘/run/user/120/gvfs’: Permission denied
/usr/local/cuda-10.0/bin/nvcc
/home/nvidia/.cache/bazel/_bazel_nvidia/0c75cc683915a1db7f4f8a4da90fb148/execroot/org_tensorflow/bazel-out/host/bin/external/local_config_cuda/cuda/cuda/bin/nvcc
/home/nvidia/.cache/bazel/_bazel_nvidia/0c75cc683915a1db7f4f8a4da90fb148/execroot/org_tensorflow/bazel-out/aarch64-py2-opt/bin/external/local_config_cuda/cuda/cuda/bin/nvcc
/home/nvidia/.cache/bazel/_bazel_nvidia/0c75cc683915a1db7f4f8a4da90fb148/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/external/local_config_cuda/cuda/cuda/bin/nvcc


nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ /usr/local/cuda-10.0/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Mon_Mar_11_22:13:24_CDT_2019
Cuda compilation tools, release 10.0, V10.0.326


nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ cd /usr/local/cuda

nvidia@nvidia-nano:/usr/local/cuda$ ls
bin  extras   lib64    NsightCompute-1.0  nvvm    samples  targets  version.txt
doc  include  LICENSE  nvml               README  share    tools

nvidia@nvidia-nano:/usr/local/cuda$ cd bin

nvidia@nvidia-nano:/usr/local/cuda/bin$ ls
bin2c     cuda-gdbserver                fatbinary     nvlink
crt       cuda-install-samples-10.0.sh  nvcc          nvprof
cudafe++  cuda-memcheck                 nvcc.profile  nvprune
cuda-gdb  cuobjdump                     nvdisasm      ptxas

nvidia@nvidia-nano:/usr/local/cuda/bin$ ls -la
total 57272
drwxr-xr-x  3 root root     4096 Aug 16 19:42 .
drwxr-xr-x 12 root root     4096 Aug 16 19:43 ..
-rwxr-xr-x  1 root root    60600 Mar 12  2019 bin2c
drwxr-xr-x  2 root root     4096 Aug 16 19:36 crt
-rwxr-xr-x  1 root root  3960000 Mar 12  2019 cudafe++
-rwxr-xr-x  1 root root  6143896 Mar 12  2019 cuda-gdb
-rwxr-xr-x  1 root root   472429 Mar 12  2019 cuda-gdbserver
-rwxr-xr-x  1 root root      784 Mar 12  2019 cuda-install-samples-10.0.sh
-rwxr-xr-x  1 root root   259368 Mar 12  2019 cuda-memcheck
-rwxr-xr-x  1 root root   270128 Mar 12  2019 cuobjdump
-rwxr-xr-x  1 root root   119280 Mar 12  2019 fatbinary
-rwxr-xr-x  1 root root   180816 Mar 12  2019 nvcc
-rw-r--r--  1 root root      393 Mar 12  2019 nvcc.profile
-rwxr-xr-x  1 root root 22607128 Mar 12  2019 nvdisasm
-rwxr-xr-x  1 root root  8979096 Mar 12  2019 nvlink
-rwxr-xr-x  1 root root  6597104 Mar 12  2019 nvprof
-rwxr-xr-x  1 root root    77280 Mar 12  2019 nvprune
-rwxr-xr-x  1 root root  8874104 Mar 12  2019 ptxas

nvidia@nvidia-nano:/usr/local/cuda/bin$ ./nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Mon_Mar_11_22:13:24_CDT_2019
Cuda compilation tools, release 10.0, V10.0.326

nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ sudo gedit ~/.bashrc

# Add this to your .bashrc file
export CUDA_HOME=/usr/local/cuda
# Adds the CUDA compiler to the PATH
export PATH=$CUDA_HOME/bin:$PATH
# Adds the libraries
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ source ~/.bashrc

nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Mon_Mar_11_22:13:24_CDT_2019
Cuda compilation tools, release 10.0, V10.0.326

Just after I issued pip install pycuda:

nvidia@nvidia-nano:~$ sudo find / -name *pycuda*
[sudo] password for nvidia: 
find: ‘/run/user/1000/gvfs’: Permission denied
find: ‘/run/user/120/gvfs’: Permission denied
/tmp/pip-install-sf96wq0d/pycuda
/tmp/pip-install-sf96wq0d/pycuda/pycuda.egg-info
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda/cuda/pycuda-helpers.hpp
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda/cuda/pycuda-complex.hpp
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda/cuda/pycuda-complex-impl.hpp
/tmp/pip-install-sf96wq0d/pycuda/pycuda
/tmp/pip-install-sf96wq0d/pycuda/pycuda/cuda/pycuda-helpers.hpp
/tmp/pip-install-sf96wq0d/pycuda/pycuda/cuda/pycuda-complex.hpp
/tmp/pip-install-sf96wq0d/pycuda/pycuda/cuda/pycuda-complex-impl.hpp
/tmp/pip-install-sf96wq0d/pycuda/pip-egg-info/pycuda.egg-info
/tmp/pip-install-sf96wq0d/pycuda/bpl-subset/bpl_subset/pycudaboost

Installing pycuda:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ pip install pycuda
Collecting pycuda
  Using cached https://files.pythonhosted.org/packages/5e/3f/5658c38579b41866ba21ee1b5020b8225cec86fe717e4b1c5c972de0a33c/pycuda-2019.1.2.tar.gz
Requirement already satisfied: pytools>=2011.2 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (2019.1.1)
Requirement already satisfied: decorator>=3.2.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (4.4.1)
Requirement already satisfied: appdirs>=1.4.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (1.4.3)
Requirement already satisfied: mako in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (1.1.0)
Requirement already satisfied: six>=1.8.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pytools>=2011.2->pycuda) (1.13.0)
Requirement already satisfied: numpy>=1.6.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pytools>=2011.2->pycuda) (1.17.4)
Requirement already satisfied: MarkupSafe>=0.9.2 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from mako->pycuda) (1.1.1)
Building wheels for collected packages: pycuda
  Building wheel for pycuda (setup.py) ... done
  Created wheel for pycuda: filename=pycuda-2019.1.2-cp36-cp36m-linux_aarch64.whl size=4497309 sha256=e7bbecd3bfa86a84f2ed505966a51123d97073cb4fd981ed434959f20a7507f3
  Stored in directory: /home/nvidia/.cache/pip/wheels/a6/60/f0/b1c430c73d281ac3e46070480db50f7907364eb6f6d3188396
Successfully built pycuda
Installing collected packages: pycuda
Successfully installed pycuda-2019.1.2

Let's see where is now pycuda:

nvidia@nvidia-nano:~$ sudo find / -name *pycuda*
find: ‘/run/user/1000/gvfs’: Permission denied
find: ‘/run/user/120/gvfs’: Permission denied
/home/nvidia/.cache/pip/wheels/a6/60/f0/b1c430c73d281ac3e46070480db50f7907364eb6f6d3188396/pycuda-2019.1.2-cp36-cp36m-linux_aarch64.whl
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda-2019.1.2.dist-info
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda/cuda/pycuda-helpers.hpp
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda/cuda/pycuda-complex.hpp
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda/cuda/pycuda-complex-impl.hpp

Let's find TensorRT:

nvidia@nvidia-nano:~$ ls -la /usr/lib/python3.6/dist-packages
total 4272
drwxrwxr-x 10 root root    4096 Nov 28 22:43 .
drwxr-xr-x 33 root root   20480 Dec  7 23:46 ..
-rw-r--r--  1 root root 2831864 Feb  6  2019 cv2.cpython-36m-aarch64-linux-gnu.so
drwxr-xr-x  2 root root    4096 Aug 16 19:52 graphsurgeon
drwxr-xr-x  2 root root    4096 Aug 16 19:52 graphsurgeon-0.4.1.dist-info
drwxr-xr-x  4 root root    4096 Nov 28 22:43 jetson
drwxr-xr-x  4 root root    4096 Nov 28 22:43 Jetson
-rw-r--r--  1 root root  746496 Nov 28 22:43 jetson_inference_python.so
-rw-r--r--  1 root root  725512 Nov 28 22:43 jetson_utils_python.so
drwxr-xr-x  3 root root    4096 Aug 16 19:52 tensorrt
drwxr-xr-x  2 root root    4096 Aug 16 19:52 tensorrt-5.1.6.1.dist-info
drwxr-xr-x  5 root root    4096 Aug 16 19:52 uff
drwxr-xr-x  2 root root    4096 Aug 16 19:52 uff-0.6.3.dist-info

nvidia@nvidia-nano:~$ ls -la /usr/lib/python3.6/dist-packages/tensorrt
total 2156
drwxr-xr-x  3 root root    4096 Aug 16 19:52 .
drwxrwxr-x 10 root root    4096 Nov 28 22:43 ..
-rw-r--r--  1 root root    2914 Jun  4  2019 __init__.py
drwxr-xr-x  6 root root    4096 Aug 16 19:52 legacy
-rw-r--r--  1 root root 2190376 Jun  4  2019 tensorrt.so


TensorRT is still not visible to Python in virtual environment though:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ python -c "import tensorrt as trt"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'tensorrt'

We need to add the path to TensorRT package to PYTHONPATH in virtual environment:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ export PYTHONPATH=/usr/lib/python3.6/dist-packages:$PYTHONPATH

TensorRT is now visible to Python:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ python -c "import tensorrt as trt"

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ python
Python 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorrt as trt
>>> trt.__version__
'5.1.6.1'
>>> 


1 comment:

Sachin said...

Thanks! This was so helpful!