Friday 8 March 2019

Running PostgreSQL in Docker inside VirtualBox Ubuntu VM

The last time I used PostgreSQL (and databases in general) was in my Brand Communications days, around 2010-2011. I was then designing and implementing DB support for customer care events coming from VPN clients connected to Apollo server. Today, 9 years later I am coming back to data bases and am witnessing how one software product and overall environment has drastically changed.

Back in 2012, pgAdmin used to be a standalone native desktop application (written in C++ and wxWidgets) and PostgreSQL DB would be running directly on top of the OS. Today, pgAdmin runs in a browser as a web application (written in Python and JavaScript) and connects to a DB instance running more often inside a Docker container in a cloud then on some in-house server...What a nice example of technology evolution! :)

I wanted to catch up with pgAdmin evolution and found the article written by its lead developer where he said:

Aside from it being extremely hard to find C++ developers these days (...) the world is shifting to a web based model these days. Cloud deployments are becoming more widely used, as well as simple instances supplied through hosting providers. Users are getting used to being able to pickup their laptop or a tablet and do whatever they need to do without having to install software – open a browser, read and write email, build a spreadsheet or create a database.

Let's then dive into new paradigms of using PostgreSQL:
  • PostgreSQL image is available in Docker Hub
  • pgAdmin 4 Windows installer can be downloaded from postgresql.org but host has to have some browser installed

Running PostgreSQL in Docker inside VirtualBox Ubuntu image


To pull the official PostgreSQL image from the DockerHub (if not already downloaded) and launch it on your local system, execute the following command:

$ docker run -d -p 5432:5432 --name my-postgres -e POSTGRES_PASSWORD=mysecretpassword postgres

Let's dissect it:

docker run
-d  // runs container in detached mode
-p 5432:5432 // opens port 5432 for incoming connections from external applications
--name my-postgres // sets container name "my-postgres"
-e POSTGRES_PASSWORD=mysecretpassword // sets environment variable
postgres // image name in Docker Hub

To start a terminal in the container use docker exec. From now on, the prompt will show the current user in the container and container ID:

$ sudo docker exec -it my-postgres bash
root@a4a7485fea59:/# ls
bin  boot  dev docker-entrypoint-initdb.d  docker-entrypoint.sh  etc  home  lib  lib64  media mnt  opt  proc root  run  sbin  srv  sys  tmp usr  var
root@a4a7485fea59:/# pwd
/

To launch PostgreSQL interactive terminal use psql.  
To connect to the database as some specific user, use -U <USERNAME>.

root@a4a7485fea59:/# psql -U postgres
psql (11.2 (Debian 11.2-1.pgdg90+1))
Type "help" for help.

To see the help menu, type help:

postgres=# help
You are using psql, the command-line interface to PostgreSQL.
Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help with psql commands
       \g or terminate with semicolon to execute query
       \q to quit


To see the list of SQL commands for which help is provided, use \h:

postgres=# \h
Available help:
  ABORT                            ALTER TEXT SEARCH TEMPLATE       CREATE PUBLICATION               DROP FUNCTION                    IMPORT FOREIGN SCHEMA
  ALTER AGGREGATE                  ALTER TRIGGER                    CREATE ROLE                      ...

To see help for some particular SQL command type help <COMMAND>:

postgres-# \help abort
Command:     ABORT
Description: abort the current transaction
Syntax:
ABORT [ WORK | TRANSACTION ]

To list all psql commands use \?:

postgres-# \?
General
  \copyright             show PostgreSQL usage and distribution terms
  \crosstabview [COLUMNS] execute query and display results in crosstab
  \errverbose            show most recent error message at maximum verbosity
  \g [FILE] or ;         execute query (and send results to file or |pipe)
  \gdesc                 describe result of query, without executing it
  \gexec                 execute query, then execute each value in its result
  \gset [PREFIX]         execute query and store results in psql variables
  \gx [FILE]             as \g, but forces expanded output mode
  \q                     quit psql
  \watch [SEC]           execute query every SEC seconds

Help
  \? [commands]          show help on backslash commands
  \? options             show help on psql command-line options
  \? variables           show help on special variables
  \h [NAME]              help on syntax of SQL commands, * for all commands

Query Buffer
  \e [FILE] [LINE]       edit the query buffer (or file) with external editor
  \ef [FUNCNAME [LINE]]  edit function definition with external editor
  \ev [VIEWNAME [LINE]]  edit view definition with external editor
  \p                     show the contents of the query buffer
  \r                     reset (clear) the query buffer
  \s [FILE]              display history or save it to file
  \w FILE                write query buffer to file

Input/Output
  \copy ...              perform SQL COPY with data stream to the client host
  \echo [STRING]         write string to standard output
  \i FILE                execute commands from file
  \ir FILE               as \i, but relative to location of current script
  \o [FILE]              send all query results to file or |pipe
  \qecho [STRING]        write string to query output stream (see \o)

Conditional
  \if EXPR               begin conditional block
  \elif EXPR             alternative within current conditional block
  \else                  final alternative within current conditional block
  \endif                 end conditional block

Informational
  (options: S = show system objects, + = additional detail)
  \d[S+]                 list tables, views, and sequences
  \d[S+]  NAME           describe table, view, sequence, or index
  \da[S]  [PATTERN]      list aggregates
  \dA[+]  [PATTERN]      list access methods
  \db[+]  [PATTERN]      list tablespaces
  \dc[S+] [PATTERN]      list conversions
  \dC[+]  [PATTERN]      list casts
  \dd[S]  [PATTERN]      show object descriptions not displayed elsewhere
  \dD[S+] [PATTERN]      list domains
  \ddp    [PATTERN]      list default privileges
  \dE[S+] [PATTERN]      list foreign tables
  \det[+] [PATTERN]      list foreign tables
  \des[+] [PATTERN]      list foreign servers
  \deu[+] [PATTERN]      list user mappings
  \dew[+] [PATTERN]      list foreign-data wrappers
  \df[anptw][S+] [PATRN] list [only agg/normal/procedures/trigger/window] functions
  \dF[+]  [PATTERN]      list text search configurations
  \dFd[+] [PATTERN]      list text search dictionaries
  \dFp[+] [PATTERN]      list text search parsers
  \dFt[+] [PATTERN]      list text search templates
  \dg[S+] [PATTERN]      list roles
  \di[S+] [PATTERN]      list indexes
  \dl                    list large objects, same as \lo_list
  \dL[S+] [PATTERN]      list procedural languages
  \dm[S+] [PATTERN]      list materialized views
  \dn[S+] [PATTERN]      list schemas
  \do[S]  [PATTERN]      list operators
  \dO[S+] [PATTERN]      list collations
  \dp     [PATTERN]      list table, view, and sequence access privileges
  \drds [PATRN1 [PATRN2]] list per-database role settings
  \dRp[+] [PATTERN]      list replication publications
  \dRs[+] [PATTERN]      list replication subscriptions
  \ds[S+] [PATTERN]      list sequences
  \dt[S+] [PATTERN]      list tables
  \dT[S+] [PATTERN]      list data types
  \du[S+] [PATTERN]      list roles
  \dv[S+] [PATTERN]      list views
  \dx[+]  [PATTERN]      list extensions
  \dy     [PATTERN]      list event triggers
  \l[+]   [PATTERN]      list databases
  \sf[+]  FUNCNAME       show a function's definition
  \sv[+]  VIEWNAME       show a view's definition
  \z      [PATTERN]      same as \dp

Formatting
  \a                     toggle between unaligned and aligned output mode
  \C [STRING]            set table title, or unset if none
  \f [STRING]            show or set field separator for unaligned query output
  \H                     toggle HTML output mode (currently off)
  \pset [NAME [VALUE]]   set table output option
                         (NAME := {border|columns|expanded|fieldsep|fieldsep_zero|
                         footer|format|linestyle|null|numericlocale|pager|
                         pager_min_lines|recordsep|recordsep_zero|tableattr|title|
                         tuples_only|unicode_border_linestyle|
                         unicode_column_linestyle|unicode_header_linestyle})
  \t [on|off]            show only rows (currently off)
  \T [STRING]            set HTML <table> tag attributes, or unset if none
  \x [on|off|auto]       toggle expanded output (currently off)

Connection
  \c[onnect] {[DBNAME|- USER|- HOST|- PORT|-] | conninfo}
                         connect to new database (currently "postgres")
  \conninfo              display information about current connection
  \encoding [ENCODING]   show or set client encoding
  \password [USERNAME]   securely change the password for a user

Operating System
  \cd [DIR]              change the current working directory
  \setenv NAME [VALUE]   set or unset environment variable
  \timing [on|off]       toggle timing of commands (currently off)
  \! [COMMAND]           execute command in shell or start interactive shell

Variables
  \prompt [TEXT] NAME    prompt user to set internal variable
  \set [NAME [VALUE]]    set internal variable, or list all if no parameters
  \unset NAME            unset (delete) internal variable

Large Objects
  \lo_export LOBOID FILE
  \lo_import FILE [COMMENT]
  \lo_list
  \lo_unlink LOBOID      large object operations


To check the PostgreSQL version use:

postgres=# select version();
                                                             version                                                              
----------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 11.2 (Debian 11.2-1.pgdg90+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit
(1 row)


To list all databases, use \l:

postgres-# \l
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges   
-----------+----------+----------+------------+------------+-----------------------
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
(3 rows)

To create a new database, execute this SQL command:

postgres=# CREATE DATABASE mytestdb;
CREATE DATABASE

This new database appears in the list of databases:

postgres=# \l
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges   
-----------+----------+----------+------------+------------+-----------------------
 mytestdb  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
(4 rows)


To connect to a database, use \c:

postgres-# \c postgres
You are now connected to database "postgres" as user "postgres".

To list all relations, use \dS:

postgres-# \dS
                        List of relations
   Schema   |              Name               | Type  |  Owner   
------------+---------------------------------+-------+----------
 pg_catalog | pg_aggregate                    | table | postgres
 pg_catalog | pg_am                           | table | postgres
 pg_catalog | pg_amop                         | table | postgres
 pg_catalog | pg_amproc                       | table | postgres
 pg_catalog | pg_attrdef                      | table | postgres
 pg_catalog | pg_attribute                    | table | postgres
 pg_catalog | pg_auth_members                 | table | postgres
 pg_catalog | pg_authid                       | table | postgres
 pg_catalog | pg_available_extension_versions | view  | postgres
 pg_catalog | pg_available_extensions         | view  | postgres
 pg_catalog | pg_cast                         | table | postgres
 pg_catalog | pg_class                        | table | postgres
 pg_catalog | pg_collation                    | table | postgres
 pg_catalog | pg_config                       | view  | postgres
 pg_catalog | pg_constraint                   | table | postgres
 pg_catalog | pg_conversion                   | table | postgres
 pg_catalog | pg_cursors                      | view  | postgres
 pg_catalog | pg_database                     | table | postgres
 pg_catalog | pg_db_role_setting              | table | postgres
 pg_catalog | pg_default_acl                  | table | postgres
 pg_catalog | pg_depend                       | table | postgres
 pg_catalog | pg_description                  | table | postgres
 pg_catalog | pg_enum                         | table | postgres
 pg_catalog | pg_event_trigger                | table | postgres
 pg_catalog | pg_extension                    | table | postgres
 pg_catalog | pg_file_settings                | view  | postgres
 pg_catalog | pg_foreign_data_wrapper         | table | postgres
 pg_catalog | pg_foreign_server               | table | postgres
 pg_catalog | pg_foreign_table                | table | postgres
 pg_catalog | pg_group                        | view  | postgres
 pg_catalog | pg_hba_file_rules               | view  | postgres
 pg_catalog | pg_index                        | table | postgres
 pg_catalog | pg_indexes                      | view  | postgres
 pg_catalog | pg_inherits                     | table | postgres
 pg_catalog | pg_init_privs                   | table | postgres
 pg_catalog | pg_language                     | table | postgres
 pg_catalog | pg_largeobject                  | table | postgres
 pg_catalog | pg_largeobject_metadata         | table | postgres
 pg_catalog | pg_locks                        | view  | postgres
 pg_catalog | pg_matviews                     | view  | postgres
 pg_catalog | pg_namespace                    | table | postgres
 pg_catalog | pg_opclass                      | table | postgres
 pg_catalog | pg_operator                     | table | postgres
 pg_catalog | pg_opfamily                     | table | postgres
 pg_catalog | pg_partitioned_table            | table | postgres
 pg_catalog | pg_pltemplate                   | table | postgres
 pg_catalog | pg_policies                     | view  | postgres
 pg_catalog | pg_policy                       | table | postgres
 pg_catalog | pg_prepared_statements          | view  | postgres
 pg_catalog | pg_prepared_xacts               | view  | postgres
 pg_catalog | pg_proc                         | table | postgres
 pg_catalog | pg_publication                  | table | postgres
 pg_catalog | pg_publication_rel              | table | postgres
 pg_catalog | pg_publication_tables           | view  | postgres
 pg_catalog | pg_range                        | table | postgres
 pg_catalog | pg_replication_origin           | table | postgres
 pg_catalog | pg_replication_origin_status    | view  | postgres
 pg_catalog | pg_replication_slots            | view  | postgres
 pg_catalog | pg_rewrite                      | table | postgres
 pg_catalog | pg_roles                        | view  | postgres
 pg_catalog | pg_rules                        | view  | postgres
 pg_catalog | pg_seclabel                     | table | postgres
 pg_catalog | pg_seclabels                    | view  | postgres
 pg_catalog | pg_sequence                     | table | postgres
 pg_catalog | pg_sequences                    | view  | postgres
 pg_catalog | pg_settings                     | view  | postgres
 pg_catalog | pg_shadow                       | view  | postgres
 pg_catalog | pg_shdepend                     | table | postgres
 pg_catalog | pg_shdescription                | table | postgres
 pg_catalog | pg_shseclabel                   | table | postgres
 pg_catalog | pg_stat_activity                | view  | postgres
 pg_catalog | pg_stat_all_indexes             | view  | postgres
 pg_catalog | pg_stat_all_tables              | view  | postgres
 pg_catalog | pg_stat_archiver                | view  | postgres
 pg_catalog | pg_stat_bgwriter                | view  | postgres
 pg_catalog | pg_stat_database                | view  | postgres
 pg_catalog | pg_stat_database_conflicts      | view  | postgres
 pg_catalog | pg_stat_progress_vacuum         | view  | postgres
 pg_catalog | pg_stat_replication             | view  | postgres
 pg_catalog | pg_stat_ssl                     | view  | postgres
 pg_catalog | pg_stat_subscription            | view  | postgres
 pg_catalog | pg_stat_sys_indexes             | view  | postgres
 pg_catalog | pg_stat_sys_tables              | view  | postgres
 pg_catalog | pg_stat_user_functions          | view  | postgres
 pg_catalog | pg_stat_user_indexes            | view  | postgres
 pg_catalog | pg_stat_user_tables             | view  | postgres
 pg_catalog | pg_stat_wal_receiver            | view  | postgres
 pg_catalog | pg_stat_xact_all_tables         | view  | postgres
 pg_catalog | pg_stat_xact_sys_tables         | view  | postgres
 pg_catalog | pg_stat_xact_user_functions     | view  | postgres
 pg_catalog | pg_stat_xact_user_tables        | view  | postgres
 pg_catalog | pg_statio_all_indexes           | view  | postgres
 pg_catalog | pg_statio_all_sequences         | view  | postgres
 pg_catalog | pg_statio_all_tables            | view  | postgres
 pg_catalog | pg_statio_sys_indexes           | view  | postgres
 pg_catalog | pg_statio_sys_sequences         | view  | postgres
 pg_catalog | pg_statio_sys_tables            | view  | postgres
 pg_catalog | pg_statio_user_indexes          | view  | postgres
 pg_catalog | pg_statio_user_sequences        | view  | postgres
 pg_catalog | pg_statio_user_tables           | view  | postgres
 pg_catalog | pg_statistic                    | table | postgres
 pg_catalog | pg_statistic_ext                | table | postgres
 pg_catalog | pg_stats                        | view  | postgres
 pg_catalog | pg_subscription                 | table | postgres
 pg_catalog | pg_subscription_rel             | table | postgres
 pg_catalog | pg_tables                       | view  | postgres
 pg_catalog | pg_tablespace                   | table | postgres
 pg_catalog | pg_timezone_abbrevs             | view  | postgres
 pg_catalog | pg_timezone_names               | view  | postgres
 pg_catalog | pg_transform                    | table | postgres
 pg_catalog | pg_trigger                      | table | postgres
 pg_catalog | pg_ts_config                    | table | postgres
 pg_catalog | pg_ts_config_map                | table | postgres
 pg_catalog | pg_ts_dict                      | table | postgres
 pg_catalog | pg_ts_parser                    | table | postgres
 pg_catalog | pg_ts_template                  | table | postgres
 pg_catalog | pg_type                         | table | postgres
 pg_catalog | pg_user                         | view  | postgres
 pg_catalog | pg_user_mapping                 | table | postgres
 pg_catalog | pg_user_mappings                | view  | postgres
 pg_catalog | pg_views                        | view  | postgres
(121 rows)


To list tables only use \d:

my_database=# \d
                     List of relations
 Schema    |              Name          | Type  | Owner 
---------- +----------------------------+-------+-------
 public    | customers                  | table | bojan
 public    | audience                   | table | bojan
 public    | campaign                   | table | bojan


To leave psql interactive terminal use \q:

postgres=# \q
root@a4a7485fea59:/# 


If docker container is stopped (by executing docker stop my-postgres,  in some other terminal) prompt of the Ubuntu will appear:

root@a4a7485fea59:/# bojan@bojan-VirtualBox:~$ 

Using pgAdmin on Windows to query PostgreSQL running in Docker on Ubuntu VirtualBox VM


PostgreSQL by default listens on port 5432 and that's why we opened a port on Docker and mapped 5432 onto it. We chose the same port number to be opened on Docker. Now, we need to open that port on our Ubuntu Virtual Machine:


Now we can go back to Windows and after pgAdmin is installed we can add this DB server:


We can name this server as we wish, this name is arbitrary:


We'll use localhost, port 5432, and username and password as we used when we were launching the PostgreSQL:


As soon as we hit "Save" pgAdmin will try to connect to the server and will load all databases in it. We can see our databases here:




TBC...

References:

Connect from local machine to PostgreSQL database in Docker
https://wiki.postgresql.org/wiki/Apt

No comments: