Friday, 10 May 2019

Introduction to Python

Python versions

To find out which version the executable python refers to, execute this:

$ python --version
Python 2.7.15rc1

Installing packages

$ pip install jsonschema2db

pip - How do I install a Python package with a .whl file? - Stack Overflow

$ pip install --user  /path/to/package.whl

Importing packages

import datetime
from jsonschema2db import JSONSchemaToPostgres

Getting Package Info

$ python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import JSONSchema2DB
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'JSONSchema2DB'
>>> import jsonschema2db
>>> help(jsonschema2db)
>>> exit()

help(package_name) opens a documentation mode which lists all classes, functions etc.:

Help on module jsonschema2db:


    class JSONSchemaToDatabase(builtins.object)
     |  JSONSchemaToDatabase is the mother class for everything
     |  :param schema: The JSON schema, as a native Python dict
     |  :param database_flavor: Either "postgres" or "redshift"


How to find package version programmatically?

Almost every normal package in python assigns the variable .__version__ or VERSION to the current version. So if you want to find the version of some package you can do the following

import a
a.__version__ # or a.VERSION




def add(a, b):
    return a+b


class MyClass:

In Python 3 each class implicitly inherit from object.


class Derived(Base):


Data Types


Python List


Python Dictionary Methods
python dictionaries and JSON (crash course)


Dependencies & Package Management

pip -  Package Manager

$ pip

  pip <command> [options]

  install                     Install packages.
  download                    Download packages.
  uninstall                   Uninstall packages.
  freeze                      Output installed packages in requirements format.
  list                        List installed packages.
  show                        Show information about installed packages.
  check                       Verify installed packages have compatible dependencies.
  search                      Search PyPI for packages.
  wheel                       Build wheels from your requirements.
  hash                        Compute hashes of package archives.
  completion                  A helper command used for command completion.
  help                        Show help for commands.

General Options:
  -h, --help                  Show help.
  --isolated                  Run pip in an isolated mode, ignoring environment variables and user configuration.
  -v, --verbose               Give more output. Option is additive, and can be used up to 3 times.
  -V, --version               Show version and exit.
  -q, --quiet                 Give less output. Option is additive, and can be used up to 3 times (corresponding to WARNING, ERROR, and CRITICAL logging levels).
  --log <path>                Path to a verbose appending log.
  --proxy <proxy>             Specify a proxy in the form [user:passwd@]proxy.server:port.
  --retries <retries>         Maximum number of retries each connection should attempt (default 5 times).
  --timeout <sec>             Set the socket timeout (default 15 seconds).
  --exists-action <action>    Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.
  --trusted-host <hostname>   Mark this host as trusted, even though it does not have valid or any HTTPS.
  --cert <path>               Path to alternate CA bundle.
  --client-cert <path>        Path to SSL client certificate, a single file containing the private key and the certificate in PEM format.
  --cache-dir <dir>           Store the cache data in <dir>.
  --no-cache-dir              Disable the cache.
                              Don't periodically check PyPI to determine whether a new version of pip is available for download. Implied with --no-index.

To check its version and for which Python version it has been installed:

$ pip -V
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

To list packages installed via pip:

$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
asn1crypto (0.24.0)
change-case (0.5.2)
cryptography (2.1.4)
dnspython (1.15.0)
enum34 (1.1.6)
gyp (0.1)
idna (2.6)
ipaddress (1.0.17)
iso8601 (0.1.12)
JSONSchema2DB (1.0.1)

To install some package via pip:

$ pip install jsonschema2db

To list properties of the installed package (including its location on disk!) use pip show:

$ pip show jsonschema2db
Name: JSONSchema2DB
Version: 1.0.1
Summary: Generate database tables from JSON schema
Author: Erik Bernhardsson
License: MIT
Location: /home/bojan/.local/lib/python2.7/site-packages
Requires: iso8601, change-case, psycopg2

To uninstall a package, use:

$ pip uninstall JSONSchema2DB
Uninstalling JSONSchema2DB-1.0.1:
Proceed (y/n)? y
  Successfully uninstalled JSONSchema2DB-1.0.1


$ pip freeze

Dockerfile before requirements.txt:

RUN python3 -m pip install psycopg2
RUN python3 -m pip install jsonschema2db



Dockerfile after requirements.txt:

RUN python3 -m pip install -r requirements.txt

We can only list names of packages in this file but sometimes it is necessary to specify their versions as otherwise we might get pip error like this:

ERROR: jsonschema2db 1.0.1 has requirement psycopg2==2.7.2, but you'll have psycopg2 2.8.2 which is incompatible.


Pip3 is the Python3 version of pip. If you just use pip, then only the python2.7 version will be installed. You have to use pip3 for it to be installed on Python3. [source]

If pip3 is not installed:

$ python3 -m pip install JSONSchema2DB
/usr/bin/python3: No module named pip

$ pip3
Command 'pip3' not found, but can be installed with:
sudo apt install python3-pip

Let's install it:

$ sudo apt install python3-pip
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up python3.6-dev (3.6.7-1~18.04) ...
Setting up python3-lib2to3 (3.6.7-1~18.04) ...
Setting up python3-distutils (3.6.7-1~18.04) ...
Setting up libpython3-dev:amd64 (3.6.7-1~18.04) ...
Setting up python3-pip (9.0.1-2.3~ubuntu1) ...
Setting up python3-setuptools (39.0.1-2) ...
Setting up dh-python (3.20180325ubuntu2) ...
Setting up python3-dev (3.6.7-1~18.04) ...

Now we have:

$ pip -V
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
$ pip3 -V
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

We can now use it to install some package for python3.

$ python3 -m pip install JSONSchema2DB
Collecting JSONSchema2DB
  Using cached 
Installing collected packages: change-case, iso8601, psycopg2, JSONSchema2DB
Successfully installed JSONSchema2DB-1.0.1 change-case-0.5.2 iso8601-0.1.12 psycopg2-2.7.2

Let's verify it is installed for Python3:

$ pip3 show JSONSchema2DB
Name: JSONSchema2DB
Version: 1.0.1
Summary: Generate database tables from JSON schema
Author: Erik Bernhardsson
License: MIT
Location: /home/bojan/.local/lib/python3.6/site-packages
Requires: change-case, iso8601, psycopg2

An Introduction To Venv

$ source ./venv/bin/activate

How to rename virtual environment?

directory - How to rename a virtualenv in Python? - Stack Overflow

From inside active virtual environment:

$ pip freeze > requirements.txt
$ deactivate

Then delete old environment directory. E.g.

$ rm -r ~/python-envs/tensorrt-

...and create a new one with correct name:

$ python3 -m virtualenv -p python3 ~/python-envs/tensorrt-
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/nvidia/python-envs/tensorrt-
Also creating executable in /home/nvidia/python-envs/tensorrt-
Installing setuptools, pkg_resources, pip, wheel...done.

Activate it and install all packages from the saved list:

$ source ~/python-envs/tensorrt-
$ pip install -r requirements.txt
$ pip3 install --extra-index-url tensorflow-gpu==1.15.0+nv19.12

Installing Python in Docker

How to compare version strings?

How do I compare version numbers in Python?

from packaging import version
>>> version.parse("2.3.1") < version.parse("10.1.2")

If module packaging is not installed, you might get the following error:

No module named 'packaging'

Solution is to install it via pip (after optionally updating pip itself):

$ pip3 install --upgrade pip
$ pip3 install packaging

References and Further Reading

Learn Python Programming - The Definitive Guide

How to install multiple python packages at once using pip

No comments: