Monday, 23 December 2019

How to install TensorRT Python package on NVIDIA Jetson Nano

Source code of the following Python script contains:

import tensorrt as trt

...and its execution fails:


(tensorflow-demo) nvidia@nvidia-nano:~/dev/nvidia/uff_ssd$ python detect_objects.py images/image1.jpg
Traceback (most recent call last):
  File "detect_objects.py", line 58, in <module>
    import tensorrt as trt
ModuleNotFoundError: No module named 'tensorrt'

TensorRT Pyton module was not installed. I want to share here my experience with the process of setting up TensorRT on Jetson Nano as described here:
A Guide to using TensorRT on the Nvidia Jetson Nano - Donkey Car

$ sudo find / -name nvcc
[sudo] password for nvidia:
find: ‘/run/user/1000/gvfs’: Permission denied
find: ‘/run/user/120/gvfs’: Permission denied
/usr/local/cuda-10.0/bin/nvcc
/home/nvidia/.cache/bazel/_bazel_nvidia/0c75cc683915a1db7f4f8a4da90fb148/execroot/org_tensorflow/bazel-out/host/bin/external/local_config_cuda/cuda/cuda/bin/nvcc
/home/nvidia/.cache/bazel/_bazel_nvidia/0c75cc683915a1db7f4f8a4da90fb148/execroot/org_tensorflow/bazel-out/aarch64-py2-opt/bin/external/local_config_cuda/cuda/cuda/bin/nvcc
/home/nvidia/.cache/bazel/_bazel_nvidia/0c75cc683915a1db7f4f8a4da90fb148/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/external/local_config_cuda/cuda/cuda/bin/nvcc


nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ /usr/local/cuda-10.0/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Mon_Mar_11_22:13:24_CDT_2019
Cuda compilation tools, release 10.0, V10.0.326


nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ cd /usr/local/cuda

 nvidia@nvidia-nano:/usr/local/cuda$ ls
bin  extras   lib64    NsightCompute-1.0  nvvm    samples  targets  version.txt
doc  include  LICENSE  nvml               README  share    tools

 nvidia@nvidia-nano:/usr/local/cuda$ cd bin

 nvidia@nvidia-nano:/usr/local/cuda/bin$ ls
bin2c     cuda-gdbserver                fatbinary     nvlink
crt       cuda-install-samples-10.0.sh  nvcc          nvprof
cudafe++  cuda-memcheck                 nvcc.profile  nvprune
cuda-gdb  cuobjdump                     nvdisasm      ptxas

 nvidia@nvidia-nano:/usr/local/cuda/bin$ ls -la
total 57272
drwxr-xr-x  3 root root     4096 Aug 16 19:42 .
drwxr-xr-x 12 root root     4096 Aug 16 19:43 ..
-rwxr-xr-x  1 root root    60600 Mar 12  2019 bin2c
drwxr-xr-x  2 root root     4096 Aug 16 19:36 crt
-rwxr-xr-x  1 root root  3960000 Mar 12  2019 cudafe++
-rwxr-xr-x  1 root root  6143896 Mar 12  2019 cuda-gdb
-rwxr-xr-x  1 root root   472429 Mar 12  2019 cuda-gdbserver
-rwxr-xr-x  1 root root      784 Mar 12  2019 cuda-install-samples-10.0.sh
-rwxr-xr-x  1 root root   259368 Mar 12  2019 cuda-memcheck
-rwxr-xr-x  1 root root   270128 Mar 12  2019 cuobjdump
-rwxr-xr-x  1 root root   119280 Mar 12  2019 fatbinary
-rwxr-xr-x  1 root root   180816 Mar 12  2019 nvcc
-rw-r--r--  1 root root      393 Mar 12  2019 nvcc.profile
-rwxr-xr-x  1 root root 22607128 Mar 12  2019 nvdisasm
-rwxr-xr-x  1 root root  8979096 Mar 12  2019 nvlink
-rwxr-xr-x  1 root root  6597104 Mar 12  2019 nvprof
-rwxr-xr-x  1 root root    77280 Mar 12  2019 nvprune
-rwxr-xr-x  1 root root  8874104 Mar 12  2019 ptxas

 nvidia@nvidia-nano:/usr/local/cuda/bin$ ./nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Mon_Mar_11_22:13:24_CDT_2019
Cuda compilation tools, release 10.0, V10.0.326

nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ sudo gedit ~/.bashrc

# Add this to your .bashrc file
export CUDA_HOME=/usr/local/cuda
# Adds the CUDA compiler to the PATH
export PATH=$CUDA_HOME/bin:$PATH
# Adds the libraries
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ source ~/.bashrc

 nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Mon_Mar_11_22:13:24_CDT_2019
Cuda compilation tools, release 10.0, V10.0.326

Just after I issued pip install pycuda:

nvidia@nvidia-nano:~$ sudo find / -name *pycuda*
[sudo] password for nvidia: 
find: ‘/run/user/1000/gvfs’: Permission denied
find: ‘/run/user/120/gvfs’: Permission denied
/tmp/pip-install-sf96wq0d/pycuda
/tmp/pip-install-sf96wq0d/pycuda/pycuda.egg-info
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda/cuda/pycuda-helpers.hpp
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda/cuda/pycuda-complex.hpp
/tmp/pip-install-sf96wq0d/pycuda/build/lib.linux-aarch64-3.6/pycuda/cuda/pycuda-complex-impl.hpp
/tmp/pip-install-sf96wq0d/pycuda/pycuda
/tmp/pip-install-sf96wq0d/pycuda/pycuda/cuda/pycuda-helpers.hpp
/tmp/pip-install-sf96wq0d/pycuda/pycuda/cuda/pycuda-complex.hpp
/tmp/pip-install-sf96wq0d/pycuda/pycuda/cuda/pycuda-complex-impl.hpp
/tmp/pip-install-sf96wq0d/pycuda/pip-egg-info/pycuda.egg-info
/tmp/pip-install-sf96wq0d/pycuda/bpl-subset/bpl_subset/pycudaboost

Installing pycuda:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ pip install pycuda
Collecting pycuda
  Using cached https://files.pythonhosted.org/packages/5e/3f/5658c38579b41866ba21ee1b5020b8225cec86fe717e4b1c5c972de0a33c/pycuda-2019.1.2.tar.gz
Requirement already satisfied: pytools>=2011.2 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (2019.1.1)
Requirement already satisfied: decorator>=3.2.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (4.4.1)
Requirement already satisfied: appdirs>=1.4.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (1.4.3)
Requirement already satisfied: mako in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pycuda) (1.1.0)
Requirement already satisfied: six>=1.8.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pytools>=2011.2->pycuda) (1.13.0)
Requirement already satisfied: numpy>=1.6.0 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from pytools>=2011.2->pycuda) (1.17.4)
Requirement already satisfied: MarkupSafe>=0.9.2 in /home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages (from mako->pycuda) (1.1.1)
Building wheels for collected packages: pycuda
  Building wheel for pycuda (setup.py) ... done
  Created wheel for pycuda: filename=pycuda-2019.1.2-cp36-cp36m-linux_aarch64.whl size=4497309 sha256=e7bbecd3bfa86a84f2ed505966a51123d97073cb4fd981ed434959f20a7507f3
  Stored in directory: /home/nvidia/.cache/pip/wheels/a6/60/f0/b1c430c73d281ac3e46070480db50f7907364eb6f6d3188396
Successfully built pycuda
Installing collected packages: pycuda
Successfully installed pycuda-2019.1.2

Let's see where is now pycuda:

nvidia@nvidia-nano:~$ sudo find / -name *pycuda*
find: ‘/run/user/1000/gvfs’: Permission denied
find: ‘/run/user/120/gvfs’: Permission denied
/home/nvidia/.cache/pip/wheels/a6/60/f0/b1c430c73d281ac3e46070480db50f7907364eb6f6d3188396/pycuda-2019.1.2-cp36-cp36m-linux_aarch64.whl
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda-2019.1.2.dist-info
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda/cuda/pycuda-helpers.hpp
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda/cuda/pycuda-complex.hpp
/home/nvidia/python-envs/tensorflow-demo/lib/python3.6/site-packages/pycuda/cuda/pycuda-complex-impl.hpp

Let's find TensorRT:

nvidia@nvidia-nano:~$ ls -la /usr/lib/python3.6/dist-packages
total 4272
drwxrwxr-x 10 root root    4096 Nov 28 22:43 .
drwxr-xr-x 33 root root   20480 Dec  7 23:46 ..
-rw-r--r--  1 root root 2831864 Feb  6  2019 cv2.cpython-36m-aarch64-linux-gnu.so
drwxr-xr-x  2 root root    4096 Aug 16 19:52 graphsurgeon
drwxr-xr-x  2 root root    4096 Aug 16 19:52 graphsurgeon-0.4.1.dist-info
drwxr-xr-x  4 root root    4096 Nov 28 22:43 jetson
drwxr-xr-x  4 root root    4096 Nov 28 22:43 Jetson
-rw-r--r--  1 root root  746496 Nov 28 22:43 jetson_inference_python.so
-rw-r--r--  1 root root  725512 Nov 28 22:43 jetson_utils_python.so
drwxr-xr-x  3 root root    4096 Aug 16 19:52 tensorrt
drwxr-xr-x  2 root root    4096 Aug 16 19:52 tensorrt-5.1.6.1.dist-info
drwxr-xr-x  5 root root    4096 Aug 16 19:52 uff
drwxr-xr-x  2 root root    4096 Aug 16 19:52 uff-0.6.3.dist-info

 nvidia@nvidia-nano:~$ ls -la /usr/lib/python3.6/dist-packages/tensorrt
total 2156
drwxr-xr-x  3 root root    4096 Aug 16 19:52 .
drwxrwxr-x 10 root root    4096 Nov 28 22:43 ..
-rw-r--r--  1 root root    2914 Jun  4  2019 __init__.py
drwxr-xr-x  6 root root    4096 Aug 16 19:52 legacy
-rw-r--r--  1 root root 2190376 Jun  4  2019 tensorrt.so


TensorRT is still not visible to Python in virtual environment though:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ python -c "import tensorrt as trt"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'tensorrt'

We need to add the path to TensorRT package to PYTHONPATH in virtual environment:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ export PYTHONPATH=/usr/lib/python3.6/dist-packages:$PYTHONPATH

TensorRT is now visible to Python:

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ python -c "import tensorrt as trt"

(tensorflow-demo) nvidia@nvidia-nano:/usr/src/tensorrt/samples/python$ python
Python 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorrt as trt
>>> trt.__version__
'5.1.6.1'
>>> 


Posted by at
Labels: , , , ,

1 comment:

micheal pan said...

BE SMART AND BECOME RICH IN LESS THAN 3DAYS....It all depends on how fast 
you can be to get the new PROGRAMMED blank ATM card that is capable of
hacking into any ATM machine,anywhere in the world. I got to know about 
this BLANK ATM CARD when I was searching for job online about a month 
ago..It has really changed my life for good and now I can say I'm rich and 
I can never be poor again. The least money I get in a day with it is about 
$50,000.(fifty thousand USD) Every now and then I keeping pumping money 
into my account. Though is illegal,there is no risk of being caught 
,because it has been programmed in such a way that it is not traceable,it 
also has a technique that makes it impossible for the CCTVs to detect 
you..For details on how to get yours today, email the hackers on : (
atmmachinehackers1@gmail.com ). Tell your 
loved once too, and start to live large. That's the simple testimony of how 
my life changed for good...Love you all ...the email address again is ;
atmmachinehackers1@gmail.com

15 June 2021 at 15:17

Post a Comment

Subscribe to: Post Comments (Atom)