Wednesday, 2 November 2022

Exploring Docker commands: docker run

 

docker run


docker run = docker create + docker start

container is an instance of an image and docker pull only downloads the image, it doesn't create a container.

docker run creates a container from the image you choose.

If you run

$ docker run myImage

...N times, docker ps -a will show N containers based on that image.

Why “Docker run” creates new container every time?

Containers do not modify images. Any changes made to a container you've started via docker run won't affect another container run from the same image

Container process that runs is isolated in that it has its own:
  • file system
  • networking
  • isolated process tree separate from the host

$ docker run --help

Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]


Run a command in a new container


Options:

      --add-host list                  Add a custom host-to-IP mapping (host:ip)
  -a, --attach list                    Attach to STDIN, STDOUT or STDERR
      --blkio-weight uint16            Block IO (relative weight), between 10 and 1000, or 0 to disable (default 0)
      --blkio-weight-device list       Block IO weight (relative device weight) (default [])
      --cap-add list                   Add Linux capabilities
      --cap-drop list                  Drop Linux capabilities
      --cgroup-parent string           Optional parent cgroup for the container
      --cidfile string                 Write the container ID to the file
      --cpu-count int                  CPU count (Windows only)
      --cpu-percent int                CPU percent (Windows only)
      --cpu-period int                 Limit CPU CFS (Completely Fair Scheduler) period
      --cpu-quota int                  Limit CPU CFS (Completely Fair Scheduler) quota
      --cpu-rt-period int              Limit CPU real-time period in microseconds
      --cpu-rt-runtime int             Limit CPU real-time runtime in microseconds
  -c, --cpu-shares int                 CPU shares (relative weight)
      --cpus decimal                   Number of CPUs
      --cpuset-cpus string             CPUs in which to allow execution (0-3, 0,1)
      --cpuset-mems string             MEMs in which to allow execution (0-3, 0,1)
  
      -d, --detach                     Run container in background and print container ID

      --detach-keys string             Override the key sequence for detaching a container

      --device list                    Add a host device to the container
      --device-cgroup-rule list        Add a rule to the cgroup allowed devices list
      --device-read-bps list           Limit read rate (bytes per second) from a device (default [])
      --device-read-iops list          Limit read rate (IO per second) from a device (default [])
      --device-write-bps list          Limit write rate (bytes per second) to a device (default [])
      --device-write-iops list         Limit write rate (IO per second) to a device (default [])
      --disable-content-trust          Skip image verification (default true)
      --dns list                       Set custom DNS servers
      --dns-option list                Set DNS options
      --dns-search list                Set custom DNS search domains
      --entrypoint string              Overwrite the default ENTRYPOINT of the image

      -e, --env list                   Set environment variables


      --env-file list                  Read in a file of environment variables

      --expose list                    Expose a port or a range of ports
      --group-add list                 Add additional groups to join
      --health-cmd string              Command to run to check health
      --health-interval duration       Time between running the check (ms|s|m|h) (default 0s)
      --health-retries int             Consecutive failures needed to report unhealthy
      --health-start-period duration   Start period for the container to initialize before starting health-retries countdown (ms|s|m|h) (default 0s)
      --health-timeout duration        Maximum time to allow one check to run (ms|s|m|h) (default 0s)
      --help                           Print usage
      -h, --hostname string            Container host name
      --init                           Run an init inside the container that forwards signals and reaps processes

      -i, --interactive                
Keep STDIN open even if not attached

      --io-maxbandwidth bytes          Maximum IO bandwidth limit for the system drive (Windows only)

      --io-maxiops uint                Maximum IOps limit for the system drive (Windows only)
      --ip string                      IPv4 address (e.g., 172.30.100.104)
      --ip6 string                     IPv6 address (e.g., 2001:db8::33)
      --ipc string                     IPC mode to use
      --isolation string               Container isolation technology
      --kernel-memory bytes            Kernel memory limit
  -l, --label list                     Set meta data on a container
      --label-file list                Read in a line delimited file of labels
      --link list                      Add link to another container
      --link-local-ip list             Container IPv4/IPv6 link-local addresses
      --log-driver string              Logging driver for the container
      --log-opt list                   Log driver options
      --mac-address string             Container MAC address (e.g., 92:d0:c6:0a:29:33)
  -m, --memory bytes                   Memory limit
      --memory-reservation bytes       Memory soft limit
      --memory-swap bytes              Swap limit equal to memory plus swap: '-1' to enable unlimited swap
      --memory-swappiness int          Tune container memory swappiness (0 to 100) (default -1)
      
      --mount mount                    Attach a filesystem mount to the container

      --name string                    Assign a name to the container

      
      --network string                 Connect a container to a network (default "default")

      --network-alias list             Add network-scoped alias for the container

      --no-healthcheck                 Disable any container-specified HEALTHCHECK
      --oom-kill-disable               Disable OOM Killer
      --oom-score-adj int              Tune host's OOM preferences (-1000 to 1000)
      --pid string                     PID namespace to use
      --pids-limit int                 Tune container pids limit (set -1 for unlimited)
      --platform string                Set platform if server is multi-platform capable
      --privileged                     Give extended privileges to this container

      -p, --publish list               Publish a container's port(s) to the host


  -P, --publish-all                    Publish all exposed ports to random ports

      --read-only                      Mount the container's root filesystem as read only
      --restart string                 Restart policy to apply when a container exits (default "no")

      --rm                             
Automatically remove the container when it exits; this means it will not be listed in "docker ps -a" output after it terminates.

      --runtime string                 Runtime to use for this container

      --security-opt list              Security Options
      --shm-size bytes                 Size of /dev/shm
      --sig-proxy                      Proxy received signals to the process (default true)
      --stop-signal string             Signal to stop a container (default "SIGTERM")
      --stop-timeout int               Timeout (in seconds) to stop a container
      --storage-opt list               Storage driver options for the container
      --sysctl map                     Sysctl options (default map[])
      --tmpfs list                     Mount a tmpfs directory

      -t, --tty                        
Allocate a pseudo-TTY
(If you wander WTF is TTY read this)

      --ulimit ulimit                  Ulimit options (default [])


      -u, --user string                
Username or UID (format: <name|uid>[:<group|gid>])

      --userns string                  User namespace to use

      --uts string                     UTS namespace to use

      -v, --volume list                
Bind mount a volume

      --volume-driver string           Optional volume driver for the container

      --volumes-from list              Mount volumes from the specified container(s)
  -w, --workdir string                 Working directory inside the container


For interactive processes (like a shell), you must use -i -t together in order to allocate a tty for the container process. -i -t is often written -it. [source]

Confused about Docker -t option to Allocate a pseudo-TTY

It is possible to pipe (continuously) data from some command running in host's terminal into an app running in Docker container:

e.g.

$ ping 8.8.8.8 | docker run -i --rm -p 8000:8000 --name http_server http_server


Example: 

This example shows how it's easy to fetch, spin and use a specific version of Golang (v1.11) with Docker:

docker run --rm -it golang:1.11
Unable to find image 'golang:1.11' locally
1.11: Pulling from library/golang
e79bb959ec00: Already exists 
... 
963c818ebafc: Already exists 
66831bab8cde: Pull complete 
0ac2e04178ce: Pull complete 
Digest: sha256:c096aaab963e25f78d226e841ad258ab3cba8c6a4c345c24166a5755686734f1
Status: Downloaded newer image for golang:1.11
root@bee14b1ffe61:/go# go version
go version go1.11.6 linux/amd64
root@bee14b1ffe61:/go# exit
exit

Example:

docker run -p 443:443 \
        -v "/private/var/lib/pgadmin:/var/lib/pgadmin" \
        -v "/path/to/certificate.cert:/certs/server.cert" \
        -v "/path/to/certificate.key:/certs/server.key" \
        -v "/tmp/servers.json:/servers.json" \
        -e "PGADMIN_DEFAULT_EMAIL=user@domain.com" \
        -e "PGADMIN_DEFAULT_PASSWORD=SuperSecret" \
        -e "PGADMIN_ENABLE_TLS=True" \
        -d dpage/pgadmin4


Pulling the image from Docker Hub


docker run checks if image exists locally and if not it performs docker pull and downloads it from Docker Hub. [1] Once the image is present docker run would not attempt to download its newer version next time [docker run doesn't pull down latest image if the image exists locally · Issue #13331 · moby/moby]. We need to pull it manually:

$ docker pull my_image && docker run my_image

Container identification


Container can be identified by:

  • UUID
    • assigned by the daemon
    • can be:
      • long
      • short
  • name
    • set by the developer via --name argument OR
    • if --name is not used, Docker assigns automatically some arbitrary but human readable name

Container Modes (-d) 


Docker can run processes (containers) in one of these two modes:
  • foreground
    • default mode
    • console/terminal is attached to process' stdin, stdout, stderr; we can see all the terminal output, log messages etc... created by the applications running inside the container
    • only one container can run in the current terminal 
  • background (detached)
    • -d has to be passed to docker run
    • in a single terminal (tab) we can run as many container as we wish
    • container exits when 
      • the root process used to run container exits OR
      • when daemon exits (if this happens first)


Exposing Ports


By default all Docker container run in a default bridge network. We can't access the container unless its port(s) are exposed outside. 

If some process in the container is listening on some port and we want to allow connections from outside the container, we can open (expose) these incoming ports via -p option which publishes single port or range of ports to the host. Port number that service listens inside the container does not need to be the same as the port exposed to the host (where external clients connect). These mappings are specified like in these examples:

-p <host_port(s)>:<container_port(s)>/protocol

-p 1234:1234 // single port, all protocols
-p 1234:1234/tcp // allow only TCP protocol
-p 1234-1240:1234-1240 // range
-p 1234-1240:1234-1240/tcp // range, TCP protocol only
-p 127.0.0.1:80:8080/tcp // tcp port 8080 is mapped onto host's port 80

Host port is bound onto local host address 0.0.0.0.

Networking

Use case:
  • DB is running in one container and DB client is running in another container
  • DB client is using host name (not IP address) of the first container
Setup:

  • Both containers have to be on the same network. This can be achieved in two ways:
    • DB container does not specify any network but DB client container is using --network=container:<DB_container_name>
    • a custom network is created and both containers are using --network=<network_name>
  • DB has to have set network alias so DB client can use it as DB's host name
Creating custom network:

$ docker network create \
--subnet=172.18.0.0/24 \
--gateway 172.18.0.1 \
custom_network

Starting DB container:

$ docker run \
--name postgres \
--network custom_network \
--publish 5432:5432 \
--hostname db \ <-- this is irrelevant for DNS resolution
--network-alias db \ <-- this is relevant for DNS resolution
--ip 172.18.0.2 \
...
postgres

Starting DB client container:

$ docker run \
-e PGHOST=db \
--name db_client \
--network custom_network \
...
dockerhub.com/mydbclient

From the docs: --hostname option specifies the hostname a container uses for itself. Defaults to the container’s ID if not specified.

To check container's network do:

$ docker inspect <container_name_or_id>

or

$ docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $INSTANCE_ID

Container's IPv4 address has to be found in NetworkSettings.Networks.<network_name>.IPAddress. For backward compatibility and only for the default bridge network, the IPv4 address will also be listed in NetworkSettings.IPAddress.


Can't resolve set hostname from another docker container in same network

If (Node.js) app for some reason can't resolve hostname, its networking stack might error with:
Error: getaddrinfo EAI_AGAIN

Error: getaddrinfo EAI_AGAIN
My Docker container does not have IP address. Why?
docker inspect <container-id> returns blank IP address
Troubleshooting Container Networking


Useful for testing: blocking particular domain for Docker container

Use --add-host. Example:

$ docker run -e APP_ENV=dev -e DB_HOST=1.2.3.4 -e DB_PORT=5432 -e DB_NAME=my_db_dev -e DB_USER=postgres -e DB_PASSWORD=postgres -e OUTPUT_DIR=./data-vol --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol --network=db_default --add-host dev.example.com:0.0.0.0 --name my-app-container my-app-image --param0=arg0

Adding something like

RUN echo "0.0.0.0 dev.example.com" >> /etc/hosts && cat /etc/hosts

to Dockerfile won't work.


How to make Docker using Proxy server?

One of the options is to use environment variables:

docker run \
...
-e HTTP_PROXY=10.21.32.40:8080 \
-e HTTPS_PROXY=10.21.32.40:8080 \
...

Configure Docker to use a proxy server | Docker Documentation

Running Docker Containers as Specified User


Processes In Containers Should Not Run As Root
Running a Docker container as a non-root user
How to use Docker without sudo on Ubuntu

Add current user to docker group (which members can talk to docker service which is running as root):

$ sudo gpasswd -a $USER docker
# or sudo usermod -aG docker $USER
$ newgrp docker

Running Docker Containers as Current Host User
What's the default user for docker exec?
Permission denied on accessing host directory in docker



Environment variables


Use -e option followed by NAME=VALUE like in this example:

-e POSTGRES_PASSWORD=mysecretpassword


Using volumes

Use volumes

From Volume mapping making directories out of files:
When using the -v <host-path>:<container-path> option is used for a bind-mount, docker will look for <host-path> on the host where the daemon runs; if that path does not exist, it will create a directory on that location (as it doesn't know if you wanted to mount a file or a directory; a directory is the default). (Windows-specific: If you don't have a shared drive set-up, docker won't be able to access that path from your host, thus won't find the path inside the VM (thus creates a directory)).
To prevent docker from creating the path if it cannot find it (and produce an error instead), use the --mount option instead of -v. The --mount flag uses a different syntax, which can be found on this page; https://docs.docker.com/engine/admin/volumes/bind-mounts/#start-a-container-with-a-bind-mount


Volumes are the preferred mechanism for persisting data generated by and used by Docker containers.

New users should try --mount syntax which is simpler than --volume syntax.

--mount 'type=volume,src=<VOLUME-NAME>,dst=<CONTAINER-PATH>,volume-driver=local,volume-opt=type=nfs...'


type - type of the mount, which can be bind, volume, or tmpfs
src - For named volumes, this is the name of the volume. May be specified as source or src. It can be either absolute or relative path.
dst - destination takes as its value the path where the file or directory is mounted in the container; May be specified as destination, dst, or target. It has to be absolute path otherwise an error similar to this is issued:

$ docker run --rm -it --mount type=volume,src=./data-vol-src,target=./data-vol-target --name go-demo go-demo
docker: Error response from daemon: invalid mount config for type "volume": invalid mount path: './data-vol-target' mount path must be absolute.
See 'docker run --help'.


Example:

$ docker run  --mount type=volume,source=myvol,target=/app ...


Docker Tip #33: Should You Use the Volume or Mount Flag?

BK: I could not make docker run --mount working when having both type=volume and source=`pwd`/myvol (or $(pwd)/myvol, ${pwd}/myvol, "$(pwd)/myvol", "$(pwd)"/myvol...etc...).

Example:

$ docker run --rm -it --mount type=volume,src="$(pwd)/data-vol",target=/go/src/github.com/BojanKomazec/go-demo/data-vol --name go-demo go-demo
docker: Error response from daemon: create /home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/data-vol: "/home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/data-vol" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.

This is very likely a consequence of Docker's way of handling volumes: When -v (or -- mount type=volume) is specified Docker will create a new volume in the docker host's /var/lib/docker/volumes/new_volume_name/_data and mount to the container. We can't use an arbitrary absolute path for the volume (the one that includes $(pwd)) if volume has to be in a directory predefined by Docker.

Nevertheless, if we switch to binding local host's arbitrary directory to some container's one then $(pwd) works well.

Example:

$ docker run --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/go/src/github.com/BojanKomazec/go-demo/data-vol --name go-demo go-demo

Our application can pick up target directory via environment variable: we need to add e.g. OUTPUT_DIR=./data-vol to .env file or to docker run -e argument.

If some file is created and saved to disk by the application which runs within Docker container, that file will actually be saved on the host's disk, in the directory in container's volume. E.g.

/var/lib/docker/volumes/f84dd5236429fc84c31104335394bd08f5292316c93fb04552e68349877ba0ca/_data/storage/test_example.com/demo_table_from_db.csv

To access (e.g. view) this file, attach to container's terminal as:

$ docker exec -it container_name /bin/bash

or

$ docker exec -it container_name /bin/sh

...if bash is not available in the image.

Example:

$ docker exec -it pgadmin /bin/bash
OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown
$ docker exec -it pgadmin /bin/sh
/pgadmin4 # ls -la
...

Troubleshooting #1


If you use docker run and then see that container is not running (e.g. the output of docker container ls shows its status as Exited(1)) check the log of the container:

$ docker inspect <container_name>

In the output json look for LogPath attribute:

"LogPath":"/var/lib/docker/containers/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6-json.log"

Then see the content of the log file:

$ sudo cat /var/lib/docker/containers/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6-json.log

{"log":"You need to specify PGADMIN_DEFAULT_EMAIL and PGADMIN_DEFAULT_PASSWORD environment variables\n","stream":"stdout","time":"2019-06-05T10:14:22.078604896Z"}

Troubleshooting #2


If Dockerfile specifies an executable to be run via CMD but some of the dependencies of that executable are missing (e.g. base image is alpine so doesn't have some libs installed...) running this container might report an error:

$ docker run  --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol  --name myapp docker.example.com/myapps/myapp:build-28
standard_init_linux.go:211: exec user process caused "no such file or directory"

To verify that it is indeed binary stated in CMD that is making problems, we can override CMD by specifying /bin/sh as the executable so we can browse the file system of the container:

$ docker run  --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol  --name myapp docker.example.com/myapps/myapp:build-28 /bin/sh
/ # ls -la
total 8404
drwxr-xr-x    1 root     root          4096 Jul 25 13:27 .
drwxr-xr-x    1 root     root          4096 Jul 25 13:27 ..
-rwxr-xr-x    1 root     root             0 Jul 25 13:27 .dockerenv
drwxr-xr-x    2 root     root          4096 Jul 11 17:29 bin

-rwxr-xr-x    1 root     root       8532786 Jul 24 17:22 myapp
/ # ./myapp 
/bin/sh: ./myapp: not found
/ # myapp

/bin/sh: myapp: not found
...

Troubleshooting #3

If docker run shows that app started with CMD doesn't have access privileges, it will output an error similar to this:

error TS5033: Could not write file '/home/node/myapp/build/index.js': EACCES: permission denied, open '/home/node/myapp/build/index.js'.

To find out the permissions on the /home/node/myapp/ you can temporarily change CMD app in Dockerfile:

CMD [ "ls", "-la", "/home/node/myapp" ]

Now rebuild the image and run the container - the output will show owners of that directory and files in it:

$ docker build -t bkomazec/myapp .
$ docker run --user node --rm -it --name running-myapp bkomazec/myapp

No comments: