Docker is a platform which allows some application and all its dependencies to be packed in a container, shipped within this monolithic bundle and then deployed on any other machine with no fear that some dependency would be missing or not compatible with the new host.
Here are some terms:
host
container
Docker runs processes in isolated containers.
Lifecycle of Docker Container
Here are some terms:
host
- may be local or remote
container
- process which runs on a host
Docker runs processes in isolated containers.
Lifecycle of Docker Container
Image Source and Credits: http://docker-saigon.github.io/post/Docker-Internals/
image
Docker images can consist of multiple layers. Layers can be reused by images. If a layer is downloaded as part of imageA and it is also part of imageB then when we pull/download imageB, this layer will not be downloaded again but will be reused.
- image ID is a SHA256 digest covering the image’s configuration and layers.
Dockerfile
Multi-stage build
This is one of the use cases which shows benefits of multi-staged build:
Docker container running golang http.Client getting error `certificate signed by unknown authority`
---
If we already have app binary built and can simply copy it onto an image but still have to add certificates, Dockerfile can look like:
FROM golang:alpine as build
RUN apk --no-cache add ca-certificates
FROM scratch
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY ./bin/my_app /my_app
CMD ["/my_app"]
---
If we set working directory in the image created at first stage, it will remain set to the same path in the following images:
FROM python:3-alpine as base
# Required for installing psycopg2 (see https://github.com/psycopg/psycopg2/issues/684)
RUN apk update && apk add --no-cache postgresql-dev python3-dev gcc musl-dev
ENV appDir /usr/local/src/python-demo
WORKDIR ${appDir}
COPY ./requirements.txt ./
RUN pip install -r requirements.txt
FROM base
COPY . .
COPY src/ ./src
RUN pwd // /usr/local/src/python-demo
CMD [ "python", "./python_demo.py" ]
---
If we set working directory in the image created at first stage, it will remain set to the same path in the following images:
FROM python:3-alpine as base
# Required for installing psycopg2 (see https://github.com/psycopg/psycopg2/issues/684)
RUN apk update && apk add --no-cache postgresql-dev python3-dev gcc musl-dev
ENV appDir /usr/local/src/python-demo
WORKDIR ${appDir}
COPY ./requirements.txt ./
RUN pip install -r requirements.txt
FROM base
COPY . .
COPY src/ ./src
RUN pwd // /usr/local/src/python-demo
CMD [ "python", "./python_demo.py" ]
---
FROM scratch
If we use FROM scratch then we can't use/execute bash/sh terminal if we attach to the container from the outside. The lightest image which provides bash terminal is alpine:
FROM alpine
alpine contains /bin/sh so we can use it via
$ docker exec -it <container_name> sh
Attaching to a docker container with base image scratch?
Why we needed to run apk add ca-certificates?
Install Certificates in Alpine Image to establish Secured Communication (SSL/TLS)
---
.dockerignore
- added manually in the same directory where Dockerfile resides
- a newline-separated list of patterns that define a set of files and/or directories that should be ignored by ADD and COPY commands in Dockerfile
- should contain any directory/file present in the same directory as .dockerfile but which should not be in the newly built image e.g. application's output directory (present on the dev machine and populated during testing) which can be large in size. This can cause sending unnecessary files to Docker daemon and long times for building an image:
$ docker build -t my_image .
...
Sending build context to Docker daemon 20.66GB
.dockerignore Example:
# comment
.git
.gitignore
*/temp*
*/*/temp*
temp
*.md
!README.md
Can dockerfile be put in .dockerignore?
Docker Tips : about the build context
Do not ignore .dockerignore (it's expensive and potentially dangerous) - Codefresh
Do not ignore .dockerignore (it's expensive and potentially dangerous) - Codefresh
Linter
Example:
$ docker run --rm -i hadolint/hadolint < Dockerfile
Unable to find image 'hadolint/hadolint:latest' locally
latest: Pulling from hadolint/hadolint
447f1c00f9d8: Pull complete
Digest: sha256:6e67b08b2f9b3cf57616cfc92adb8864b74b02f079d70eabc944b29f05aff5f9
Status: Downloaded newer image for hadolint/hadolint:latest
/dev/stdin:27 DL3025 Use arguments JSON notation for CMD and ENTRYPOINT arguments
Docker CLI Commands
---
$ docker --helpUsage: docker [OPTIONS] COMMAND
A self-sufficient runtime for containers
Options:
--config string Location of client config files (default "/home/bojan/.docker")
-D, --debug Enable debug mode
-H, --host list Daemon socket(s) to connect to
-l, --log-level string Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info")
--tls Use TLS; implied by --tlsverify
--tlscacert string Trust certs signed only by this CA (default "/home/bojan/.docker/ca.pem")
--tlscert string Path to TLS certificate file (default "/home/bojan/.docker/cert.pem")
--tlskey string Path to TLS key file (default "/home/bojan/.docker/key.pem")
--tlsverify Use TLS and verify the remote
-v, --version Print version information and quit
Management Commands:
builder Manage builds
config Manage Docker configs
container Manage containers
engine Manage the docker engine
image Manage images
network Manage networks
node Manage Swarm nodes
plugin Manage plugins
secret Manage Docker secrets
service Manage services
stack Manage Docker stacks
swarm Manage Swarm
system Manage Docker
trust Manage trust on Docker images
volume Manage volumes
Commands:
attach Attach local standard input, output, and error streams to a running container
build Build an image from a Dockerfile
commit Create a new image from a container's changes
cp Copy files/folders between a container and the local filesystem
create Create a new container
diff Inspect changes to files or directories on a container's filesystem
events Get real time events from the server
exec Run a command in a running container
export Export a container's filesystem as a tar archive
history Show the history of an image
images List images
import Import the contents from a tarball to create a filesystem image
info Display system-wide information
inspect Return low-level information on Docker objects
kill Kill one or more running containers
load Load an image from a tar archive or STDIN
login Log in to a Docker registry
logout Log out from a Docker registry
logs Fetch the logs of a container
pause Pause all processes within one or more containers
port List port mappings or a specific mapping for the container
ps List containers
pull Pull an image or a repository from a registry
push Push an image or a repository to a registry
rename Rename a container
restart Restart one or more containers
rm Remove one or more containers
rmi Remove one or more images
run Run a command in a new container
save Save one or more images to a tar archive (streamed to STDOUT by default)
search Search the Docker Hub for images
start Start one or more stopped containers
stats Display a live stream of container(s) resource usage statistics
stop Stop one or more running containers
tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
top Display the running processes of a container
unpause Unpause all processes within one or more containers
update Update configuration of one or more containers
version Show the Docker version information
wait Block until one or more containers stop, then print their exit codes
Run 'docker COMMAND --help' for more information on a command.
---
$ docker build --help
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
Options:
--add-host list Add a custom host-to-IP mapping (host:ip)
--build-arg list Set build-time variables
--cache-from strings Images to consider as cache sources
--cgroup-parent string Optional parent cgroup for the container
--compress Compress the build context using gzip
--cpu-period int Limit the CPU CFS (Completely Fair Scheduler) period
--cpu-quota int Limit the CPU CFS (Completely Fair Scheduler) quota
-c, --cpu-shares int CPU shares (relative weight)
--cpuset-cpus string CPUs in which to allow execution (0-3, 0,1)
--cpuset-mems string MEMs in which to allow execution (0-3, 0,1)
--disable-content-trust Skip image verification (default true)
-f, --file string Name of the Dockerfile (Default is 'PATH/Dockerfile')
--force-rm Always remove intermediate containers
--iidfile string Write the image ID to the file
--isolation string Container isolation technology
--label list Set metadata for an image
-m, --memory bytes Memory limit
--memory-swap bytes Swap limit equal to memory plus swap: '-1' to enable unlimited swap
--network string Set the networking mode for the RUN instructions during build (default "default")
--no-cache Do not use cache when building the image
--pull Always attempt to pull a newer version of the image
-q, --quiet Suppress the build output and print image ID on success
--rm Remove intermediate containers after a successful build (default true)
--security-opt strings Security options
--shm-size bytes Size of /dev/shm
-t, --tag list Name and optionally a tag in the 'name:tag' format
--target string Set the target build stage to build.
--ulimit ulimit Ulimit options (default [])
The PATH specifies where to find the files for the “context” of the build on the Docker daemon.
Example (has to be run in a directory which contains Dockerfile):
$ docker build -t <my username>/<my-app-name> .
Example of how to build and run a Docker image:
$ docker build -t jsonschema2db . && docker run jsonschema2db
If Docker has to fetch some resources from remote host whose domain name can't be resolved from within Docker native network but can from your local/dev host network use --network host:
$ docker build --network host -t my-image .
$ docker build -t jsonschema2db . && docker run jsonschema2db
If Docker has to fetch some resources from remote host whose domain name can't be resolved from within Docker native network but can from your local/dev host network use --network host:
$ docker build --network host -t my-image .
--pull instructs docker to pull the latest version of any base image(s) instead of reusing whatever you already have tagged locally. From https://stackoverflow.com/questions/58488535/whats-the-purpose-of-docker-build-pull:
Take for instance an image based on a moving tag (such as ubuntu:bionic). Upstream makes changes and rebuilds this periodically but you might have a months old image locally. Docker will happily build against the old base. --pull will pull as a side effect so you build against the latest base image.It's usually a best practice to use it to get upstream security fixes as soon as possible (instead of using stale, potentially vulnerable images). Though you have to trade off breaking changes (and if you use immutable tags then it doesn't make a difference).
--no-cache forces rebuilding of layers already available.
In normal use you shouldn’t need --no-cache. If the base image is updated (and --pull gets a new version) that automatically invalidates the cache; similarly if you COPY different code that will invalidate the cache. The only thing this will usually affect is if you’re doing something like apt-get install of a network-hosted package, in which case --no-cache will cause it to check for a newer version even if the base image hasn’t updated.
--build-arg arg=value - used to pass to Dockerfile an argument from command line. arg needs to be defined in Dockerfile via ARG command:
ARG arg[=default_value]
If arg is http_proxy or https_proxy there is no need to define these arguments as they are defined by default (predefined args).
---
$ docker container
Usage: docker container COMMAND
Manage containers
Commands:
attach Attach local standard input, output, and error streams to a running container
commit Create a new image from a container's changes
cp Copy files/folders between a container and the local filesystem
create Create a new container
diff Inspect changes to files or directories on a container's filesystem
exec Run a command in a running container
export Export a container's filesystem as a tar archive
inspect Display detailed information on one or more containers
kill Kill one or more running containers
logs Fetch the logs of a container
ls List containers
pause Pause all processes within one or more containers
port List port mappings or a specific mapping for the container
prune Remove all stopped containers
rename Rename a container
restart Restart one or more containers
rm Remove one or more containers
run Run a command in a new container
start Start one or more stopped containers
stats Display a live stream of container(s) resource usage statistics
stop Stop one or more running containers
top Display the running processes of a container
unpause Unpause all processes within one or more containers
update Update configuration of one or more containers
wait Block until one or more containers stop, then print their exit codes
Run 'docker container COMMAND --help' for more information on a command.
---
If we try to start some container which requires opening some port on the localhost while another running container has already opened it, we might get the following error:
$ docker-compose up --build
Starting my_app_1 ...
ERROR: for my-app_db_1 Cannot start service db: driver failed programming external connectivity on endpoint my-app_db_1 (a456b10867b734493c831aa99f227147110f61233652c4984415c12ecdf9a9b3): Bind for 0.0.0.0:5432 failed: port is Starting my-app_my-service_1 ... done
ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint my-app_db_1 (a456b10867b734493c831aa99f227147110f61233652c4984415c12ecdf9a9b3): Bind for 0.0.0.0:5432 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
To resolve this, we first need to check which containers are running and then to stop them:
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
499c43a9c28b postgres:latest "docker-entrypoint.s…" 3 days ago Up 3 days 0.0.0.0:5432->5432/tcp my-app-2_db_1
$ docker container stop my-app-2_db_1
my-app-2_db_1
Let's verify that no containers are running:
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
---
$ docker container ls --help
Usage: docker container ls [OPTIONS]
List containers
Aliases:
ls, ps, list
Options:
-a, --all Show all containers (default shows just running)
-f, --filter filter Filter output based on conditions provided
--format string Pretty-print containers using a Go template
-n, --last int Show n last created containers (includes all states) (default -1)
-l, --latest Show the latest created container (includes all states)
--no-trunc Don't truncate output
-q, --quiet Only display numeric IDs
-s, --size Display total file sizes
To list all containers:
$ docker container ls -a
To list only numeric IDs of all containers:
$ docker container ls -a -q
or
$ docker container ls -aq
---
$ docker container rm --help
Usage: docker container rm [OPTIONS] CONTAINER [CONTAINER...]
Remove one or more containers
Options:
-f, --force Force the removal of a running container (uses SIGKILL)
-l, --link Remove the specified link
-v, --volumes Remove the volumes associated with the container
To remove all containers:
$ docker container rm $(docker container ls -aq)
---
$ docker create --help
Usage: docker create [OPTIONS] IMAGE [COMMAND] [ARG...]
Create a new container
Options:
--add-host list Add a custom host-to-IP mapping (host:ip)
-a, --attach list Attach to STDIN, STDOUT or STDERR
--blkio-weight uint16 Block IO (relative weight), between 10 and 1000, or 0 to disable (default 0)
--blkio-weight-device list Block IO weight (relative device weight) (default [])
--cap-add list Add Linux capabilities
--cap-drop list Drop Linux capabilities
--cgroup-parent string Optional parent cgroup for the container
--cidfile string Write the container ID to the file
--cpu-period int Limit CPU CFS (Completely Fair Scheduler) period
--cpu-quota int Limit CPU CFS (Completely Fair Scheduler) quota
--cpu-rt-period int Limit CPU real-time period in microseconds
--cpu-rt-runtime int Limit CPU real-time runtime in microseconds
-c, --cpu-shares int CPU shares (relative weight)
--cpus decimal Number of CPUs
--cpuset-cpus string CPUs in which to allow execution (0-3, 0,1)
--cpuset-mems string MEMs in which to allow execution (0-3, 0,1)
--device list Add a host device to the container
--device-cgroup-rule list Add a rule to the cgroup allowed devices list
--device-read-bps list Limit read rate (bytes per second) from a device (default [])
--device-read-iops list Limit read rate (IO per second) from a device (default [])
--device-write-bps list Limit write rate (bytes per second) to a device (default [])
--device-write-iops list Limit write rate (IO per second) to a device (default [])
--disable-content-trust Skip image verification (default true)
--dns list Set custom DNS servers
--dns-option list Set DNS options
--dns-search list Set custom DNS search domains
--entrypoint string Overwrite the default ENTRYPOINT of the image
-e, --env list Set environment variables
--env-file list Read in a file of environment variables
--expose list Expose a port or a range of ports
--group-add list Add additional groups to join
--health-cmd string Command to run to check health
--health-interval duration Time between running the check (ms|s|m|h) (default 0s)
--health-retries int Consecutive failures needed to report unhealthy
--health-start-period duration Start period for the container to initialize before starting health-retries countdown (ms|s|m|h) (default 0s)
--health-timeout duration Maximum time to allow one check to run (ms|s|m|h) (default 0s)
--help Print usage
-h, --hostname string
Container host name
--init Run an init inside the container that forwards signals and reaps processes
-i, --interactive Keep STDIN open even if not attached
--ip string IPv4 address (e.g., 172.30.100.104)
--ip6 string IPv6 address (e.g., 2001:db8::33)
--ipc string IPC mode to use
--isolation string Container isolation technology
--kernel-memory bytes Kernel memory limit
-l, --label list Set meta data on a container
--label-file list Read in a line delimited file of labels
--link list Add link to another container
--link-local-ip list Container IPv4/IPv6 link-local addresses
--log-driver string Logging driver for the container
--log-opt list Log driver options
--mac-address string Container MAC address (e.g., 92:d0:c6:0a:29:33)
-m, --memory bytes Memory limit
--memory-reservation bytes Memory soft limit
--memory-swap bytes Swap limit equal to memory plus swap: '-1' to enable unlimited swap
--memory-swappiness int Tune container memory swappiness (0 to 100) (default -1)
--mount mount Attach a filesystem mount to the container
--name string Assign a name to the container
--network string Connect a container to a network (default "default")
--network-alias list Add network-scoped alias for the container
--no-healthcheck Disable any container-specified HEALTHCHECK
--oom-kill-disable Disable OOM Killer
--oom-score-adj int Tune host's OOM preferences (-1000 to 1000)
--pid string PID namespace to use
--pids-limit int Tune container pids limit (set -1 for unlimited)
--privileged Give extended privileges to this container
-p, --publish list Publish a container's port(s) to the host
-P, --publish-all Publish all exposed ports to random ports
--read-only Mount the container's root filesystem as read only
--restart string Restart policy to apply when a container exits (default "no")
--rm Automatically remove the container when it exits
--runtime string Runtime to use for this container
--security-opt list Security Options
--shm-size bytes Size of /dev/shm
--stop-signal string Signal to stop a container (default "SIGTERM")
--stop-timeout int Timeout (in seconds) to stop a container
--storage-opt list Storage driver options for the container
--sysctl map Sysctl options (default map[])
--tmpfs list Mount a tmpfs directory
-t, --tty Allocate a pseudo-TTY
--ulimit ulimit Ulimit options (default [])
-u, --user string Username or UID (format: <name|uid>[:<group|gid>])
--userns string User namespace to use
--uts string UTS namespace to use
-v, --volume list Bind mount a volume
--volume-driver string Optional volume driver for the container
--volumes-from list Mount volumes from the specified container(s)
-w, --workdir string Working directory inside the container
To execute a command inside a running container, use docker exec. For example, we can run a bash terminal as:
$ docker exec -it my-postgres bash
root@a4a7486fea59:/#
-i // keeps stdin opened
-t // allocates pseudo TTY
In this terminal we can run usual Linux commands as we were inside the container:
root@a4a7486fea59:/# ls
bin boot dev docker-entrypoint-initdb.d docker-entrypoint.sh etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root@a4a7486fea59:/# pwd
/
not able to use docker exec shell
Try the alpine tag of image:
$ docker run --name gorush -d appleboy/gorush:alpine
$ docker exec -ti gorush /bin/sh
To list all images:
$ docker image ls
To remove some image:
$ docker image rm image_id
$ docker inspect --help
Usage: docker inspect [OPTIONS] NAME|ID [NAME|ID...]
Return low-level information on Docker objects
Options:
-f, --format string Format the output using the given Go template
-s, --size Display total file sizes if the type is container
--type string Return JSON for specified type
What to Inspect When You're Inspecting
Usage: docker logs [OPTIONS] CONTAINER
Fetch the logs of a container
Options:
--details Show extra details provided to logs
-f, --follow Follow log output
--since string Show logs since timestamp (e.g. 2013-01-02T13:23:37) or relative (e.g. 42m for 42 minutes)
--tail string Number of lines to show from the end of the logs (default "all")
-t, --timestamps Show timestamps
--until string Show logs before a timestamp (e.g. 2013-01-02T13:23:37) or relative (e.g. 42m for 42 minutes)
$ docker logs my_app >& my_app.log
---
$ docker network --help
Usage: docker network COMMAND
Manage networks
Commands:
connect Connect a container to a network
create Create a network
disconnect Disconnect a container from a network
inspect Display detailed information on one or more networks
ls List networks
prune Remove all unused networks
rm Remove one or more networks
Run 'docker network COMMAND --help' for more information on a command.
List all networks:
Destroying a network
$ docker network rm mynet123
---
To list all running containers use docker ps:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b9f263f2b517 postgres "some.command…" 9 hours ago Up 5 hours 0.0.0.0:5432->5432/tcp my_project_postgres_1
To list all containers, use docker ps -a.
docker pull pulls an image from the repository.
Use:
$ docker pull <image_name>:<tag_name>
If no tag is provided, Docker Engine uses the :latest tag as a default.
Apart from pulling some image for the first time, this command is used to pull a new version of the existing image. [Update a docker container to the latest version]
In my case, I have been running pgAdmin4 from within a container but after some time I started getting a popup on pgAdmin4 main web page that I am using the old version of it. My course of actions was:
$ docker images | grep pgadmin
dpage/pgadmin4 latest c32b95c07a00 5 weeks ago 237MB
$ docker pull dpage/pgadmin4
Using default tag: latest
latest: Pulling from dpage/pgadmin4
e7c96db7181b: Already exists
...
44bc3cf004be: Already exists
aa774cf9fd8b: Pull complete
...
660e231737f7: Pull complete
Digest: sha256:2c46b3e631f33434246a2a51134e3a43951b00c5639f7fc158b4b973879f9ea3
Status: Downloaded newer image for dpage/pgadmin4:latest
---
How to upgrade docker container after its image changed
---
docker run = docker create + docker start
A container is an instance of an image and pull only downloads the image, it doesn't create a container.
docker run creates a container from the image you choose.
If you run
$ docker run myImage
...N times, docker ps -a will show N containers based on that image.
Why “Docker run” creates new container every time?
Containers do not modify images. Any changes made to a container you've started via docker run won't affect another container run from the same image
Container process that runs is isolated in that it has its own:
$ docker run --help
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Options:
--add-host list Add a custom host-to-IP mapping (host:ip)
-a, --attach list Attach to STDIN, STDOUT or STDERR
--blkio-weight uint16 Block IO (relative weight), between 10 and 1000, or 0 to disable (default 0)
--blkio-weight-device list Block IO weight (relative device weight) (default [])
--cap-add list Add Linux capabilities
--cap-drop list Drop Linux capabilities
--cgroup-parent string Optional parent cgroup for the container
--cidfile string Write the container ID to the file
--cpu-count int CPU count (Windows only)
--cpu-percent int CPU percent (Windows only)
--cpu-period int Limit CPU CFS (Completely Fair Scheduler) period
--cpu-quota int Limit CPU CFS (Completely Fair Scheduler) quota
--cpu-rt-period int Limit CPU real-time period in microseconds
--cpu-rt-runtime int Limit CPU real-time runtime in microseconds
-c, --cpu-shares int CPU shares (relative weight)
--cpus decimal Number of CPUs
--cpuset-cpus string CPUs in which to allow execution (0-3, 0,1)
--cpuset-mems string MEMs in which to allow execution (0-3, 0,1)
-d, --detach Run container in background and print container ID
--detach-keys string Override the key sequence for detaching a container
--device list Add a host device to the container
--device-cgroup-rule list Add a rule to the cgroup allowed devices list
--device-read-bps list Limit read rate (bytes per second) from a device (default [])
--device-read-iops list Limit read rate (IO per second) from a device (default [])
--device-write-bps list Limit write rate (bytes per second) to a device (default [])
--device-write-iops list Limit write rate (IO per second) to a device (default [])
--disable-content-trust Skip image verification (default true)
--dns list Set custom DNS servers
--dns-option list Set DNS options
--dns-search list Set custom DNS search domains
--entrypoint string Overwrite the default ENTRYPOINT of the image
-e, --env list Set environment variables
--env-file list Read in a file of environment variables
--expose list Expose a port or a range of ports
--group-add list Add additional groups to join
--health-cmd string Command to run to check health
--health-interval duration Time between running the check (ms|s|m|h) (default 0s)
--health-retries int Consecutive failures needed to report unhealthy
--health-start-period duration Start period for the container to initialize before starting health-retries countdown (ms|s|m|h) (default 0s)
--health-timeout duration Maximum time to allow one check to run (ms|s|m|h) (default 0s)
--help Print usage
-h, --hostname string Container host name
--init Run an init inside the container that forwards signals and reaps processes
-i, --interactive Keep STDIN open even if not attached
--io-maxbandwidth bytes Maximum IO bandwidth limit for the system drive (Windows only)
--io-maxiops uint Maximum IOps limit for the system drive (Windows only)
--ip string IPv4 address (e.g., 172.30.100.104)
--ip6 string IPv6 address (e.g., 2001:db8::33)
--ipc string IPC mode to use
--isolation string Container isolation technology
--kernel-memory bytes Kernel memory limit
-l, --label list Set meta data on a container
--label-file list Read in a line delimited file of labels
--link list Add link to another container
--link-local-ip list Container IPv4/IPv6 link-local addresses
--log-driver string Logging driver for the container
--log-opt list Log driver options
--mac-address string Container MAC address (e.g., 92:d0:c6:0a:29:33)
-m, --memory bytes Memory limit
--memory-reservation bytes Memory soft limit
--memory-swap bytes Swap limit equal to memory plus swap: '-1' to enable unlimited swap
--memory-swappiness int Tune container memory swappiness (0 to 100) (default -1)
--mount mount Attach a filesystem mount to the container
--name string Assign a name to the container
--network string Connect a container to a network (default "default")
--network-alias list Add network-scoped alias for the container
--no-healthcheck Disable any container-specified HEALTHCHECK
--oom-kill-disable Disable OOM Killer
--oom-score-adj int Tune host's OOM preferences (-1000 to 1000)
--pid string PID namespace to use
--pids-limit int Tune container pids limit (set -1 for unlimited)
--platform string Set platform if server is multi-platform capable
--privileged Give extended privileges to this container
-p, --publish list Publish a container's port(s) to the host
-P, --publish-all Publish all exposed ports to random ports
--read-only Mount the container's root filesystem as read only
--restart string Restart policy to apply when a container exits (default "no")
--rm Automatically remove the container when it exits; this means it will not be listed in "docker ps -a" output after it terminates.
--runtime string Runtime to use for this container
--security-opt list Security Options
--shm-size bytes Size of /dev/shm
--sig-proxy Proxy received signals to the process (default true)
--stop-signal string Signal to stop a container (default "SIGTERM")
--stop-timeout int Timeout (in seconds) to stop a container
--storage-opt list Storage driver options for the container
--sysctl map Sysctl options (default map[])
--tmpfs list Mount a tmpfs directory
-t, --tty Allocate a pseudo-TTY
(If you wander WTF is TTY read this)
--ulimit ulimit Ulimit options (default [])
-u, --user string Username or UID (format: <name|uid>[:<group|gid>])
--userns string User namespace to use
--uts string UTS namespace to use
-v, --volume list Bind mount a volume
--volume-driver string Optional volume driver for the container
--volumes-from list Mount volumes from the specified container(s)
-w, --workdir string Working directory inside the container
For interactive processes (like a shell), you must use -i -t together in order to allocate a tty for the container process. -i -t is often written -it. [source]
Confused about Docker -t option to Allocate a pseudo-TTY
It is possible to pipe (continuously) data from some command running in host's terminal into an app running in Docker container:
e.g.
$ ping 8.8.8.8 | docker run -i --rm -p 8000:8000 --name http_server http_server
Example:
This example shows how it's easy to fetch, spin and use a specific version of Golang (v1.11) with Docker:
$ docker run --rm -it golang:1.11
Unable to find image 'golang:1.11' locally
1.11: Pulling from library/golang
e79bb959ec00: Already exists
d4b7902036fe: Already exists
1b2a72d4e030: Already exists
d54db43011fd: Already exists
963c818ebafc: Already exists
66831bab8cde: Pull complete
0ac2e04178ce: Pull complete
Digest: sha256:c096aaab963e25f78d226e841ad258ab3cba8c6a4c345c24166a5755686734f1
Status: Downloaded newer image for golang:1.11
root@bee14b1ffe61:/go# go version
go version go1.11.6 linux/amd64
root@bee14b1ffe61:/go# exit
exit
Example:
docker run -p 443:443 \
-v "/private/var/lib/pgadmin:/var/lib/pgadmin" \
-v "/path/to/certificate.cert:/certs/server.cert" \
-v "/path/to/certificate.key:/certs/server.key" \
-v "/tmp/servers.json:/servers.json" \
-e "PGADMIN_DEFAULT_EMAIL=user@domain.com" \
-e "PGADMIN_DEFAULT_PASSWORD=SuperSecret" \
-e "PGADMIN_ENABLE_TLS=True" \
-d dpage/pgadmin4
docker run checks if image exists locally and if not it performs docker pull and downloads it from Docker Hub. [1]
Container can be identified by:
Docker can run processes (containers) in one of these two modes:
From the docs: --hostname option specifies the hostname a container uses for itself. Defaults to the container’s ID if not specified.
To check container's network do:
$ docker inspect <container_name_or_id>
or
$ docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $INSTANCE_ID
Container's IPv4 address has to be found in NetworkSettings.Networks.<network_name>.IPAddress. For backward compatibility and only for the default bridge network, the IPv4 address will also be listed in NetworkSettings.IPAddress.
Can't resolve set hostname from another docker container in same network
If (Node.js) app for some reason can't resolve hostname, its networking stack might error with:
Error: getaddrinfo EAI_AGAIN
Error: getaddrinfo EAI_AGAIN
My Docker container does not have IP address. Why?
docker inspect <container-id> returns blank IP address
How to make Docker using Proxy server?
One of the options is to use environment variables:
docker run \
...
-e HTTP_PROXY=10.21.32.40:8080 \
-e HTTPS_PROXY=10.21.32.40:8080 \
...
Configure Docker to use a proxy server | Docker Documentation
Processes In Containers Should Not Run As Root
Running a Docker container as a non-root user
How to use Docker without sudo on Ubuntu
Add current user to docker group (which members can talk to docker service which is running as root):
$ sudo gpasswd -a $USER docker
# or sudo usermod -aG docker $USER
$ newgrp docker
Running Docker Containers as Current Host User
What's the default user for docker exec?
Permission denied on accessing host directory in docker
Use -e option followed by NAME=VALUE like in this example:
-e POSTGRES_PASSWORD=mysecretpassword
From Volume mapping making directories out of files:
When using the -v <host-path>:<container-path> option is used for a bind-mount, docker will look for <host-path> on the host where the daemon runs; if that path does not exist, it will create a directory on that location (as it doesn't know if you wanted to mount a file or a directory; a directory is the default). (Windows-specific: If you don't have a shared drive set-up, docker won't be able to access that path from your host, thus won't find the path inside the VM (thus creates a directory)).
To prevent docker from creating the path if it cannot find it (and produce an error instead), use the --mount option instead of -v. The --mount flag uses a different syntax, which can be found on this page; https://docs.docker.com/engine/admin/volumes/bind-mounts/#start-a-container-with-a-bind-mount
Volumes are the preferred mechanism for persisting data generated by and used by Docker containers.
New users should try --mount syntax which is simpler than --volume syntax.
--mount 'type=volume,src=<VOLUME-NAME>,dst=<CONTAINER-PATH>,volume-driver=local,volume-opt=type=nfs...'
Example:
$ docker run --mount type=volume,source=myvol,target=/app ...
Docker Tip #33: Should You Use the Volume or Mount Flag?
BK: I could not make docker run --mount working when having both type=volume and source=`pwd`/myvol (or $(pwd)/myvol, ${pwd}/myvol, "$(pwd)/myvol", "$(pwd)"/myvol...etc...).
Example:
$ docker run --rm -it --mount type=volume,src="$(pwd)/data-vol",target=/go/src/github.com/BojanKomazec/go-demo/data-vol --name go-demo go-demo
docker: Error response from daemon: create /home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/data-vol: "/home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/data-vol" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.
This is very likely a consequence of Docker's way of handling volumes: When -v (or -- mount type=volume) is specified Docker will create a new volume in the docker host's /var/lib/docker/volumes/new_volume_name/_data and mount to the container. We can't use an arbitrary absolute path for the volume (the one that includes $(pwd)) if volume has to be in a directory predefined by Docker.
Nevertheless, if we switch to binding local host's arbitrary directory to some container's one then $(pwd) works well.
Example:
$ docker run --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/go/src/github.com/BojanKomazec/go-demo/data-vol --name go-demo go-demo
Our application can pick up target directory via environment variable: we need to add e.g. OUTPUT_DIR=./data-vol to .env file or to docker run -e argument.
If some file is created and saved to disk by the application which runs within Docker container, that file will actually be saved on the host's disk, in the directory in container's volume. E.g.
/var/lib/docker/volumes/f84dd5236429fc84c31104335394bd08f5292316c93fb04552e68349877ba0ca/_data/storage/test_example.com/demo_table_from_db.csv
To access (e.g. view) this file, attach to container's terminal as:
$ docker exec -it container_name /bin/bash
or
$ docker exec -it container_name /bin/sh
...if bash is not available in the image.
Example:
$ docker exec -it pgadmin /bin/bash
OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown
$ docker exec -it pgadmin /bin/sh
/pgadmin4 # ls -la
...
If you use docker run and then see that container is not running (e.g. the output of docker container ls shows its status as Exited(1)) check the log of the container:
$ docker inspect <container_name>
In the output json look for LogPath attribute:
"LogPath":"/var/lib/docker/containers/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6-json.log"
Then see the content of the log file:
$ sudo cat /var/lib/docker/containers/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6-json.log
{"log":"You need to specify PGADMIN_DEFAULT_EMAIL and PGADMIN_DEFAULT_PASSWORD environment variables\n","stream":"stdout","time":"2019-06-05T10:14:22.078604896Z"}
If Dockerfile specifies an executable to be run via CMD but some of the dependencies of that executable are missing (e.g. base image is alpine so doesn't have some libs installed...) running this container might report an error:
$ docker run --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol --name myapp docker.example.com/myapps/myapp:build-28
standard_init_linux.go:211: exec user process caused "no such file or directory"
To verify that it is indeed binary stated in CMD that is making problems, we can override CMD by specifying /bin/sh as the executable so we can browse the file system of the container:
$ docker run --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol --name myapp docker.example.com/myapps/myapp:build-28 /bin/sh
/ # ls -la
total 8404
drwxr-xr-x 1 root root 4096 Jul 25 13:27 .
drwxr-xr-x 1 root root 4096 Jul 25 13:27 ..
-rwxr-xr-x 1 root root 0 Jul 25 13:27 .dockerenv
drwxr-xr-x 2 root root 4096 Jul 11 17:29 bin
-rwxr-xr-x 1 root root 8532786 Jul 24 17:22 myapp
/ # ./myapp
/bin/sh: ./myapp: not found
/ # myapp
/bin/sh: myapp: not found
...
To start a container use docker start. In the following example container name is my-postgres:
$ docker start my-postgres
my-postgres
To stop running one or more containers use docker stop:
$ docker stop my_service_postgres_1
my_service_postgres_1
Usage: docker image COMMAND
Manage images
Commands:
build Build an image from a Dockerfile
history Show the history of an image
import Import the contents from a tarball to create a filesystem image
inspect Display detailed information on one or more images
load Load an image from a tar archive or STDIN
ls List images
prune Remove unused images
pull Pull an image or a repository from a registry
push Push an image or a repository to a registry
rm Remove one or more images
save Save one or more images to a tar archive (streamed to STDOUT by default)
tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
Run 'docker image COMMAND --help' for more information on a command.
---
$ docker image ls --help
Usage: docker image ls [OPTIONS] [REPOSITORY[:TAG]]
List images
Aliases:
ls, images, list
Options:
-a, --all Show all images (default hides intermediate images)
--digests Show digests
-f, --filter filter Filter output based on conditions provided
--format string Pretty-print images using a Go template
--no-trunc Don't truncate output
-q, --quiet Only show numeric IDs
To list IDs of all images:
$ docker image ls -aq
$ docker image rm --help
Usage: docker image rm [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Aliases:
rm, rmi, remove
Options:
-f, --force Force removal of the image
--no-prune Do not delete untagged parents
To delete all images:
$ docker image rm $(docker image ls -aq)
---
To see which images are present locally, use docker images:
$ docker images --help
Usage: docker images [OPTIONS] [REPOSITORY[:TAG]]
List images
Options:
-a, --all Show all images (default hides intermediate images)
--digests Show digests
-f, --filter filter Filter output based on conditions provided
--format string Pretty-print images using a Go template
--no-trunc Don't truncate output
-q, --quiet Only show numeric IDs
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
my_app latest 477db2641216 12 minutes ago 193MB
<none> <none> 7f82c84cc05a 12 minutes ago 193MB
$ docker volume
Usage: docker volume COMMAND
Manage volumes
Commands:
create Create a volume
inspect Display detailed information on one or more volumes
ls List volumes
prune Remove all unused local volumes
rm Remove one or more volumes
Run 'docker volume COMMAND --help' for more information on a command.
---
$ docker volume ls --help
Usage: docker volume ls [OPTIONS]
List volumes
Aliases:
ls, list
Options:
-f, --filter filter Provide filter values (e.g. 'dangling=true')
--format string Pretty-print volumes using a Go template
-q, --quiet Only display volume names
---
$ docker volume ls
DRIVER VOLUME NAME
local 1a2a9d0f43ce94ad1e176f9bb318f46829fcd34354d4d648d70434efa0f43f60
local 1a66b905bcd4be936b8df26cacf018d7f863e758773a11be5faf9d9949311eb0
local 1c6af16b8d8b1041ad3a46e3910bde9121b64cfae65f8299cedd81793d8fce37
...
$ docker volume ls -q
1a2a9d0f43ce94ad1e176f9bb318f46829fcd34354d4d648d70434efa0f43f60
1a66b905bcd4be936b8df26cacf018d7f863e758773a11be5faf9d9949311eb0
1c6af16b8d8b1041ad3a46e3910bde9121b64cfae65f8299cedd81793d8fce37
---
$ docker volume rm --help
Usage: docker volume rm [OPTIONS] VOLUME [VOLUME...]
Remove one or more volumes
Aliases:
rm, remove
Examples:
$ docker volume rm hello
hello
Options:
-f, --force Force the removal of one or more volumes
---
To remover all volumes:
$ docker volume rm $(docker volume ls -q)
TBC...
Lifecycle of Docker Container
Hey Docker! Why do you hate Windows so much?
Exploring Docker container's file system
docker container
$ docker container Usage: docker container COMMAND
Manage containers
Commands:
attach Attach local standard input, output, and error streams to a running container
commit Create a new image from a container's changes
cp Copy files/folders between a container and the local filesystem
create Create a new container
diff Inspect changes to files or directories on a container's filesystem
exec Run a command in a running container
export Export a container's filesystem as a tar archive
inspect Display detailed information on one or more containers
kill Kill one or more running containers
logs Fetch the logs of a container
ls List containers
pause Pause all processes within one or more containers
port List port mappings or a specific mapping for the container
prune Remove all stopped containers
rename Rename a container
restart Restart one or more containers
rm Remove one or more containers
run Run a command in a new container
start Start one or more stopped containers
stats Display a live stream of container(s) resource usage statistics
stop Stop one or more running containers
top Display the running processes of a container
unpause Unpause all processes within one or more containers
update Update configuration of one or more containers
wait Block until one or more containers stop, then print their exit codes
Run 'docker container COMMAND --help' for more information on a command.
---
If we try to start some container which requires opening some port on the localhost while another running container has already opened it, we might get the following error:
$ docker-compose up --build
Starting my_app_1 ...
ERROR: for my-app_db_1 Cannot start service db: driver failed programming external connectivity on endpoint my-app_db_1 (a456b10867b734493c831aa99f227147110f61233652c4984415c12ecdf9a9b3): Bind for 0.0.0.0:5432 failed: port is Starting my-app_my-service_1 ... done
ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint my-app_db_1 (a456b10867b734493c831aa99f227147110f61233652c4984415c12ecdf9a9b3): Bind for 0.0.0.0:5432 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
To resolve this, we first need to check which containers are running and then to stop them:
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
499c43a9c28b postgres:latest "docker-entrypoint.s…" 3 days ago Up 3 days 0.0.0.0:5432->5432/tcp my-app-2_db_1
$ docker container stop my-app-2_db_1
my-app-2_db_1
Let's verify that no containers are running:
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
---
docker container ls
$ docker container ls --help
Usage: docker container ls [OPTIONS]
List containers
Aliases:
ls, ps, list
Options:
-a, --all Show all containers (default shows just running)
-f, --filter filter Filter output based on conditions provided
--format string Pretty-print containers using a Go template
-n, --last int Show n last created containers (includes all states) (default -1)
-l, --latest Show the latest created container (includes all states)
--no-trunc Don't truncate output
-q, --quiet Only display numeric IDs
-s, --size Display total file sizes
To list all containers:
$ docker container ls -a
To list only numeric IDs of all containers:
$ docker container ls -a -q
or
$ docker container ls -aq
---
docker container rm
$ docker container rm --help
Usage: docker container rm [OPTIONS] CONTAINER [CONTAINER...]
Remove one or more containers
Options:
-f, --force Force the removal of a running container (uses SIGKILL)
-l, --link Remove the specified link
-v, --volumes Remove the volumes associated with the container
To remove all containers:
$ docker container rm $(docker container ls -aq)
To display detailed information on one or more containers:
$ docker container inspect <container id or name>
---
docker create
---
$ docker create --help
Usage: docker create [OPTIONS] IMAGE [COMMAND] [ARG...]
Create a new container
Options:
--add-host list Add a custom host-to-IP mapping (host:ip)
-a, --attach list Attach to STDIN, STDOUT or STDERR
--blkio-weight uint16 Block IO (relative weight), between 10 and 1000, or 0 to disable (default 0)
--blkio-weight-device list Block IO weight (relative device weight) (default [])
--cap-add list Add Linux capabilities
--cap-drop list Drop Linux capabilities
--cgroup-parent string Optional parent cgroup for the container
--cidfile string Write the container ID to the file
--cpu-period int Limit CPU CFS (Completely Fair Scheduler) period
--cpu-quota int Limit CPU CFS (Completely Fair Scheduler) quota
--cpu-rt-period int Limit CPU real-time period in microseconds
--cpu-rt-runtime int Limit CPU real-time runtime in microseconds
-c, --cpu-shares int CPU shares (relative weight)
--cpus decimal Number of CPUs
--cpuset-cpus string CPUs in which to allow execution (0-3, 0,1)
--cpuset-mems string MEMs in which to allow execution (0-3, 0,1)
--device list Add a host device to the container
--device-cgroup-rule list Add a rule to the cgroup allowed devices list
--device-read-bps list Limit read rate (bytes per second) from a device (default [])
--device-read-iops list Limit read rate (IO per second) from a device (default [])
--device-write-bps list Limit write rate (bytes per second) to a device (default [])
--device-write-iops list Limit write rate (IO per second) to a device (default [])
--disable-content-trust Skip image verification (default true)
--dns list Set custom DNS servers
--dns-option list Set DNS options
--dns-search list Set custom DNS search domains
--entrypoint string Overwrite the default ENTRYPOINT of the image
-e, --env list Set environment variables
--env-file list Read in a file of environment variables
--expose list Expose a port or a range of ports
--group-add list Add additional groups to join
--health-cmd string Command to run to check health
--health-interval duration Time between running the check (ms|s|m|h) (default 0s)
--health-retries int Consecutive failures needed to report unhealthy
--health-start-period duration Start period for the container to initialize before starting health-retries countdown (ms|s|m|h) (default 0s)
--health-timeout duration Maximum time to allow one check to run (ms|s|m|h) (default 0s)
--help Print usage
-h, --hostname string
Container host name
--init Run an init inside the container that forwards signals and reaps processes
-i, --interactive Keep STDIN open even if not attached
--ip string IPv4 address (e.g., 172.30.100.104)
--ip6 string IPv6 address (e.g., 2001:db8::33)
--ipc string IPC mode to use
--isolation string Container isolation technology
--kernel-memory bytes Kernel memory limit
-l, --label list Set meta data on a container
--label-file list Read in a line delimited file of labels
--link list Add link to another container
--link-local-ip list Container IPv4/IPv6 link-local addresses
--log-driver string Logging driver for the container
--log-opt list Log driver options
--mac-address string Container MAC address (e.g., 92:d0:c6:0a:29:33)
-m, --memory bytes Memory limit
--memory-reservation bytes Memory soft limit
--memory-swap bytes Swap limit equal to memory plus swap: '-1' to enable unlimited swap
--memory-swappiness int Tune container memory swappiness (0 to 100) (default -1)
--mount mount Attach a filesystem mount to the container
--name string Assign a name to the container
--network string Connect a container to a network (default "default")
--network-alias list Add network-scoped alias for the container
--no-healthcheck Disable any container-specified HEALTHCHECK
--oom-kill-disable Disable OOM Killer
--oom-score-adj int Tune host's OOM preferences (-1000 to 1000)
--pid string PID namespace to use
--pids-limit int Tune container pids limit (set -1 for unlimited)
--privileged Give extended privileges to this container
-p, --publish list Publish a container's port(s) to the host
-P, --publish-all Publish all exposed ports to random ports
--read-only Mount the container's root filesystem as read only
--restart string Restart policy to apply when a container exits (default "no")
--rm Automatically remove the container when it exits
--runtime string Runtime to use for this container
--security-opt list Security Options
--shm-size bytes Size of /dev/shm
--stop-signal string Signal to stop a container (default "SIGTERM")
--stop-timeout int Timeout (in seconds) to stop a container
--storage-opt list Storage driver options for the container
--sysctl map Sysctl options (default map[])
--tmpfs list Mount a tmpfs directory
-t, --tty Allocate a pseudo-TTY
--ulimit ulimit Ulimit options (default [])
-u, --user string Username or UID (format: <name|uid>[:<group|gid>])
--userns string User namespace to use
--uts string UTS namespace to use
-v, --volume list Bind mount a volume
--volume-driver string Optional volume driver for the container
--volumes-from list Mount volumes from the specified container(s)
-w, --workdir string Working directory inside the container
---
Examples:
$ docker create \
-e DB_HOST=121.1.0.1 \
-e DB_PORT=5432 \
-e DB_NAME=test_db \
-e DB_USER=postgres \
-e DB_PASSWORD=postgres \
-e OUTPUT_DIR=./data-vol \
--rm \
-it \
--mount type=bind,src="$(pwd)/data-vol",target=/data-vol \
--network=bridge \
--name my_app_test_container \
docker.example.com/bojan/myapp:build-28
If environment variable contains a question mark character which we want to preserve, this string has to be enclosed in single quotation mark.
Example: If we have:
docker run \
--rm \
-e NODE_ENV=dev \
-e API_BASE_URL=https://dev.example.com \
-e API_QUERY_TEMPLATE=?post_type={post_type}&json=1&count=1000 \
-e SCHEMAS_DIR=${SCHEMAS_DIR} \
-e PGHOST=postgres \
-e PGDATABASE=test_dev \
-e PGUSER=postgres \
-e PGPASSWORD=postgres \
-v "$(pwd)/${SCHEMAS_DIR}":/usr/local/my-app/${SCHEMAS_DIR} \
--name my-app \
--network myappnet \
--user $(id -u):$(id -g) \
example.com/docker/myapp:build-%docker-image.build.wpsync%
it would fail with error:
-e: not found
Because docker run would interpret/expand ? as wildcard (?). Fix is to enclose env var value in single quotes:
docker run \
...
-e API_QUERY_TEMPLATE='?post_type={post_type}&json=1&count=1000' \
...
shell - How to escape "!" and "&" in docker's environment varibles - Super User
Example: If we have:
docker run \
--rm \
-e NODE_ENV=dev \
-e API_BASE_URL=https://dev.example.com \
-e API_QUERY_TEMPLATE=?post_type={post_type}&json=1&count=1000 \
-e SCHEMAS_DIR=${SCHEMAS_DIR} \
-e PGHOST=postgres \
-e PGDATABASE=test_dev \
-e PGUSER=postgres \
-e PGPASSWORD=postgres \
-v "$(pwd)/${SCHEMAS_DIR}":/usr/local/my-app/${SCHEMAS_DIR} \
--name my-app \
--network myappnet \
--user $(id -u):$(id -g) \
example.com/docker/myapp:build-%docker-image.build.wpsync%
it would fail with error:
-e: not found
Because docker run would interpret/expand ? as wildcard (?). Fix is to enclose env var value in single quotes:
docker run \
...
-e API_QUERY_TEMPLATE='?post_type={post_type}&json=1&count=1000' \
...
shell - How to escape "!" and "&" in docker's environment varibles - Super User
docker exec
To execute a command inside a running container, use docker exec. For example, we can run a bash terminal as:
$ docker exec -it my-postgres bash
root@a4a7486fea59:/#
-i // keeps stdin opened
-t // allocates pseudo TTY
In this terminal we can run usual Linux commands as we were inside the container:
root@a4a7486fea59:/# ls
bin boot dev docker-entrypoint-initdb.d docker-entrypoint.sh etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root@a4a7486fea59:/# pwd
/
not able to use docker exec shell
Try the alpine tag of image:
$ docker run --name gorush -d appleboy/gorush:alpine
$ docker exec -ti gorush /bin/sh
docker image
To list all images:
$ docker image ls
To remove some image:
$ docker image rm image_id
docker inspect
$ docker inspect --help
Usage: docker inspect [OPTIONS] NAME|ID [NAME|ID...]
Return low-level information on Docker objects
Options:
-f, --format string Format the output using the given Go template
-s, --size Display total file sizes if the type is container
--type string Return JSON for specified type
Examples:
How to find mounting points of a (stopped) container? E.g. find local directory that is mounted as a volume on some container.
$ docker inspect -f {{.Mounts}} my_container
[{bind /home/bojan/tmp/data-vol /data-vol true rprivate}]
docker logs
$ docker logs --helpUsage: docker logs [OPTIONS] CONTAINER
Fetch the logs of a container
Options:
--details Show extra details provided to logs
-f, --follow Follow log output
--since string Show logs since timestamp (e.g. 2013-01-02T13:23:37) or relative (e.g. 42m for 42 minutes)
--tail string Number of lines to show from the end of the logs (default "all")
-t, --timestamps Show timestamps
--until string Show logs before a timestamp (e.g. 2013-01-02T13:23:37) or relative (e.g. 42m for 42 minutes)
Example:
$ docker logs my_app >& my_app.log
---
docker network
$ docker network --help
Usage: docker network COMMAND
Manage networks
Commands:
connect Connect a container to a network
create Create a network
disconnect Disconnect a container from a network
inspect Display detailed information on one or more networks
ls List networks
prune Remove all unused networks
rm Remove one or more networks
Run 'docker network COMMAND --help' for more information on a command.
List all networks:
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
110d23a9b23f bridge bridge local
b32aba0ecc85 my_app1 bridge local
5a91d6b95b25 my_app2 bridge local
7422e6165f7a my_app3 bridge local
b2977102daa4 my_app4_test-network bridge local
e829cad5a344 host host local
6b0d605ed4a5 none null local
Connect a container to the network (of some other container):
$ docker network connect --help
Usage: docker network connect [OPTIONS] NETWORK CONTAINER
Connect a container to a network
Options:
--alias strings Add network-scoped alias for the container
--ip string IPv4 address (e.g., 172.30.100.104)
--ip6 string IPv6 address (e.g., 2001:db8::33)
--link list Add link to another container
--link-local-ip strings Add a link-local address for the container
Use case example:
Running Postgres DB in one container (based on postgres image; container runs in its own network) and connecting to it via pgAdmin which runs in another container (based on dpage/pgadmin4 image).
docker container ls shows that my_app_db_1 is up and running with port mapping as 0.0.0.0:5432->5432/tcp. This container runs Postgres DB.
To find the name of the network this container is connected to use docker inspect <container_name> and in the output json look for "NetworkSettings" > "Networks". There should be a network name like "my_app_default".
Let's inspect that network:
$ docker network inspect my_app_default
[
{
"Name": "my_app_default",
"Id": "5a91d6b95b25fdd6d24e921e1ccdb69f24ae4b1c509188ccfc087acd330375ca",
"Created": "2019-06-05T10:19:11.576495786+01:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.19.0.0/16",
"Gateway": "172.19.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"9ae5dfe822348906d2bdf2b55314a323f7ba03360eb2edf8e5d7d42afca40e34": {
"Name": "my_app_service",
"EndpointID": "a5230696caabc7d230df8f589066b8028c3b03493a2f05ed0dc5e224944952fc",
"MacAddress": "02:42:ac:13:00:03",
"IPv4Address": "172.19.0.3/16",
"IPv6Address": ""
},
"e52e5d6de6b6a062d2cbfaf921f3365f20db0208511a906cab6143528a14ee6d": {
"Name": "my_app_db_1",
"EndpointID": "bbe87692635abe7cd45a3980d0982ca1dbc6c9a5718a267935659cfb036b2da1",
"MacAddress": "02:42:ac:13:00:02",
"IPv4Address": "172.19.0.2/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {
"com.docker.compose.network": "default",
"com.docker.compose.project": "my_app",
"com.docker.compose.version": "1.23.2"
}
}
]
my_app_db_1 is the name of the container which is running db service (Postgres) as I specified in docker-compose.yml.
Now, I want to run pgAdmin DB client (which is also a web server...) from its own container, based on dpage/pgadmin4 image:
$ docker run -p 5051:5051 -d -e "PGADMIN_DEFAULT_EMAIL=bojan.komazec@example.com" -e "PGADMIN_DEFAULT_PASSWORD=postgres" -e "PGADMIN_LISTEN_PORT=5051" --name pgadmin dpage/pgadmin4
baf9343368f9cb20a0dd9ab1a4763eaf08df03f2ee667f979b4e3466bc20d7b1
We can use docker container ls to verify that this container is now up and running, with port mappings as 80/tcp, 443/tcp, 0.0.0.0:5051->5051/tcp. We can now open in browser on the local host the address http://localhost:5051/ which should open pgAdmin login page. If we try to add DB server with IP address 172.19.0.2 (as listed in the postgres container network spec above), connecting to it will fail as pgAdmin runs in its own network. We need to connect pgadmin container to my_app_default network:
$ docker network connect my_app_default pgadmin
We can now use 172.19.0.2 address to connect to DB from pgAdmin web app.
If we again inspect my_app_default network we can verify that there is one more container attached to it: pgadmin.
To avoid issuing two commands, we can specify network container shall connect to in docker run command:
$ docker run -p 5051:5051 -d -e "PGADMIN_DEFAULT_EMAIL=bojan.komazec@example.com" -e "PGADMIN_DEFAULT_PASSWORD=postgres" -e "PGADMIN_LISTEN_PORT=5051" --network=my_app_default --name pgadmin dpage/pgadmin4
$ docker network create --help
Usage: docker network create [OPTIONS] NETWORK
Create a network
Options:
--attachable Enable manual container attachment
--aux-address map Auxiliary IPv4 or IPv6 addresses used by Network driver (default map[])
--config-from string The network from which copying the configuration
--config-only Create a configuration only network
-d, --driver string Driver to manage the Network (default "bridge")
--gateway strings IPv4 or IPv6 Gateway for the master subnet
--ingress Create swarm routing-mesh network
--internal Restrict external access to the network
--ip-range strings Allocate container ip from a sub-range
--ipam-driver string IP Address Management Driver (default "default")
--ipam-opt map Set IPAM driver specific options (default map[])
--ipv6 Enable IPv6 networking
--label list Set metadata on a network
-o, --opt map Set driver specific options (default map[])
--scope string Control the network's scope
--subnet strings Subnet in CIDR format that represents a network segment
$ docker run -p 5051:5051 -d -e "PGADMIN_DEFAULT_EMAIL=bojan.komazec@example.com" -e "PGADMIN_DEFAULT_PASSWORD=postgres" -e "PGADMIN_LISTEN_PORT=5051" --network=my_app_default --name pgadmin dpage/pgadmin4
Creating a network:
$ docker network create --help
Usage: docker network create [OPTIONS] NETWORK
Create a network
Options:
--attachable Enable manual container attachment
--aux-address map Auxiliary IPv4 or IPv6 addresses used by Network driver (default map[])
--config-from string The network from which copying the configuration
--config-only Create a configuration only network
-d, --driver string Driver to manage the Network (default "bridge")
--gateway strings IPv4 or IPv6 Gateway for the master subnet
--ingress Create swarm routing-mesh network
--internal Restrict external access to the network
--ip-range strings Allocate container ip from a sub-range
--ipam-driver string IP Address Management Driver (default "default")
--ipam-opt map Set IPAM driver specific options (default map[])
--ipv6 Enable IPv6 networking
--label list Set metadata on a network
-o, --opt map Set driver specific options (default map[])
--scope string Control the network's scope
--subnet strings Subnet in CIDR format that represents a network segment
Example:
Create a network:
$ docker network create --subnet=172.18.0.0/24 --gateway=172.18.0.1 mynet123
Attach container to that network and assign a static IP address to it:
$ docker run --net mynet123 --ip 172.18.0.22 -it ubuntu bash
Destroying a network
$ docker network rm mynet123
---
docker ps
To list all running containers use docker ps:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b9f263f2b517 postgres "some.command…" 9 hours ago Up 5 hours 0.0.0.0:5432->5432/tcp my_project_postgres_1
If no containers are running:
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
To list all containers, use docker ps -a.
docker pull
docker pull pulls an image from the repository.
Use:
$ docker pull <image_name>:<tag_name>
If no tag is provided, Docker Engine uses the :latest tag as a default.
Apart from pulling some image for the first time, this command is used to pull a new version of the existing image. [Update a docker container to the latest version]
In my case, I have been running pgAdmin4 from within a container but after some time I started getting a popup on pgAdmin4 main web page that I am using the old version of it. My course of actions was:
$ docker images | grep pgadmin
dpage/pgadmin4 latest c32b95c07a00 5 weeks ago 237MB
$ docker pull dpage/pgadmin4
Using default tag: latest
latest: Pulling from dpage/pgadmin4
e7c96db7181b: Already exists
...
44bc3cf004be: Already exists
aa774cf9fd8b: Pull complete
...
660e231737f7: Pull complete
Digest: sha256:2c46b3e631f33434246a2a51134e3a43951b00c5639f7fc158b4b973879f9ea3
Status: Downloaded newer image for dpage/pgadmin4:latest
---
How to upgrade docker container after its image changed
---
docker run
A container is an instance of an image and pull only downloads the image, it doesn't create a container.
docker run creates a container from the image you choose.
If you run
$ docker run myImage
...N times, docker ps -a will show N containers based on that image.
Why “Docker run” creates new container every time?
Containers do not modify images. Any changes made to a container you've started via docker run won't affect another container run from the same image
- file system
- networking
- isolated process tree separate from the host
$ docker run --help
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Options:
--add-host list Add a custom host-to-IP mapping (host:ip)
-a, --attach list Attach to STDIN, STDOUT or STDERR
--blkio-weight uint16 Block IO (relative weight), between 10 and 1000, or 0 to disable (default 0)
--blkio-weight-device list Block IO weight (relative device weight) (default [])
--cap-add list Add Linux capabilities
--cap-drop list Drop Linux capabilities
--cgroup-parent string Optional parent cgroup for the container
--cidfile string Write the container ID to the file
--cpu-count int CPU count (Windows only)
--cpu-percent int CPU percent (Windows only)
--cpu-period int Limit CPU CFS (Completely Fair Scheduler) period
--cpu-quota int Limit CPU CFS (Completely Fair Scheduler) quota
--cpu-rt-period int Limit CPU real-time period in microseconds
--cpu-rt-runtime int Limit CPU real-time runtime in microseconds
-c, --cpu-shares int CPU shares (relative weight)
--cpus decimal Number of CPUs
--cpuset-cpus string CPUs in which to allow execution (0-3, 0,1)
--cpuset-mems string MEMs in which to allow execution (0-3, 0,1)
-d, --detach Run container in background and print container ID
--detach-keys string Override the key sequence for detaching a container
--device list Add a host device to the container
--device-cgroup-rule list Add a rule to the cgroup allowed devices list
--device-read-bps list Limit read rate (bytes per second) from a device (default [])
--device-read-iops list Limit read rate (IO per second) from a device (default [])
--device-write-bps list Limit write rate (bytes per second) to a device (default [])
--device-write-iops list Limit write rate (IO per second) to a device (default [])
--disable-content-trust Skip image verification (default true)
--dns list Set custom DNS servers
--dns-option list Set DNS options
--dns-search list Set custom DNS search domains
--entrypoint string Overwrite the default ENTRYPOINT of the image
-e, --env list Set environment variables
--env-file list Read in a file of environment variables
--expose list Expose a port or a range of ports
--group-add list Add additional groups to join
--health-cmd string Command to run to check health
--health-interval duration Time between running the check (ms|s|m|h) (default 0s)
--health-retries int Consecutive failures needed to report unhealthy
--health-start-period duration Start period for the container to initialize before starting health-retries countdown (ms|s|m|h) (default 0s)
--health-timeout duration Maximum time to allow one check to run (ms|s|m|h) (default 0s)
--help Print usage
-h, --hostname string Container host name
--init Run an init inside the container that forwards signals and reaps processes
-i, --interactive Keep STDIN open even if not attached
--io-maxbandwidth bytes Maximum IO bandwidth limit for the system drive (Windows only)
--io-maxiops uint Maximum IOps limit for the system drive (Windows only)
--ip string IPv4 address (e.g., 172.30.100.104)
--ip6 string IPv6 address (e.g., 2001:db8::33)
--ipc string IPC mode to use
--isolation string Container isolation technology
--kernel-memory bytes Kernel memory limit
-l, --label list Set meta data on a container
--label-file list Read in a line delimited file of labels
--link list Add link to another container
--link-local-ip list Container IPv4/IPv6 link-local addresses
--log-driver string Logging driver for the container
--log-opt list Log driver options
--mac-address string Container MAC address (e.g., 92:d0:c6:0a:29:33)
-m, --memory bytes Memory limit
--memory-reservation bytes Memory soft limit
--memory-swap bytes Swap limit equal to memory plus swap: '-1' to enable unlimited swap
--memory-swappiness int Tune container memory swappiness (0 to 100) (default -1)
--mount mount Attach a filesystem mount to the container
--name string Assign a name to the container
--network string Connect a container to a network (default "default")
--network-alias list Add network-scoped alias for the container
--no-healthcheck Disable any container-specified HEALTHCHECK
--oom-kill-disable Disable OOM Killer
--oom-score-adj int Tune host's OOM preferences (-1000 to 1000)
--pid string PID namespace to use
--pids-limit int Tune container pids limit (set -1 for unlimited)
--platform string Set platform if server is multi-platform capable
--privileged Give extended privileges to this container
-p, --publish list Publish a container's port(s) to the host
-P, --publish-all Publish all exposed ports to random ports
--read-only Mount the container's root filesystem as read only
--restart string Restart policy to apply when a container exits (default "no")
--rm Automatically remove the container when it exits; this means it will not be listed in "docker ps -a" output after it terminates.
--runtime string Runtime to use for this container
--security-opt list Security Options
--shm-size bytes Size of /dev/shm
--sig-proxy Proxy received signals to the process (default true)
--stop-signal string Signal to stop a container (default "SIGTERM")
--stop-timeout int Timeout (in seconds) to stop a container
--storage-opt list Storage driver options for the container
--sysctl map Sysctl options (default map[])
--tmpfs list Mount a tmpfs directory
-t, --tty Allocate a pseudo-TTY
(If you wander WTF is TTY read this)
--ulimit ulimit Ulimit options (default [])
-u, --user string Username or UID (format: <name|uid>[:<group|gid>])
--userns string User namespace to use
--uts string UTS namespace to use
-v, --volume list Bind mount a volume
--volume-driver string Optional volume driver for the container
--volumes-from list Mount volumes from the specified container(s)
-w, --workdir string Working directory inside the container
For interactive processes (like a shell), you must use -i -t together in order to allocate a tty for the container process. -i -t is often written -it. [source]
Confused about Docker -t option to Allocate a pseudo-TTY
It is possible to pipe (continuously) data from some command running in host's terminal into an app running in Docker container:
e.g.
$ ping 8.8.8.8 | docker run -i --rm -p 8000:8000 --name http_server http_server
Example:
This example shows how it's easy to fetch, spin and use a specific version of Golang (v1.11) with Docker:
$ docker run --rm -it golang:1.11
Unable to find image 'golang:1.11' locally
1.11: Pulling from library/golang
e79bb959ec00: Already exists
d4b7902036fe: Already exists
1b2a72d4e030: Already exists
d54db43011fd: Already exists
963c818ebafc: Already exists
66831bab8cde: Pull complete
0ac2e04178ce: Pull complete
Digest: sha256:c096aaab963e25f78d226e841ad258ab3cba8c6a4c345c24166a5755686734f1
Status: Downloaded newer image for golang:1.11
root@bee14b1ffe61:/go# go version
go version go1.11.6 linux/amd64
root@bee14b1ffe61:/go# exit
exit
Example:
docker run -p 443:443 \
-v "/private/var/lib/pgadmin:/var/lib/pgadmin" \
-v "/path/to/certificate.cert:/certs/server.cert" \
-v "/path/to/certificate.key:/certs/server.key" \
-v "/tmp/servers.json:/servers.json" \
-e "PGADMIN_DEFAULT_EMAIL=user@domain.com" \
-e "PGADMIN_DEFAULT_PASSWORD=SuperSecret" \
-e "PGADMIN_ENABLE_TLS=True" \
-d dpage/pgadmin4
Pulling the image from Docker Hub
docker run checks if image exists locally and if not it performs docker pull and downloads it from Docker Hub. [1]
Container identification
Container can be identified by:
- UUID
- assigned by the daemon
- can be:
- long
- short
- name
- set by the developer via --name argument
Container Modes (-d)
Docker can run processes (containers) in one of these two modes:
- foreground
- default mode
- console is attached to process' stdin, stdout, stderr
- background (detached)
- -d has to be passed to docker run
- container exits when
- the root process used to run container exits OR
- when daemon exits (if this happens first)
Exposing Ports
If some process in the container is listening on some port and we want to allow connections from outside the container, we can open (expose) these incoming ports via -p option which publishes single port or range of ports to the host. Port number that service listens inside the container does not need to be the same as the port exposed to the host (where external clients connect). These mappings are specified like in these examples:
-p <host_port(s)>:<container_port(s)>/protocol
-p <host_port(s)>:<container_port(s)>/protocol
-p 1234:1234 // single port, all protocols
-p 1234:1234/tcp // allow only TCP protocol
-p 1234-1240:1234-1240 // range
-p 1234-1240:1234-1240/tcp // range, TCP protocol only
-p 127.0.0.1:80:8080/tcp // tcp port 8080 is mapped onto host's port 80
-p 127.0.0.1:80:8080/tcp // tcp port 8080 is mapped onto host's port 80
Networking
Use case:- DB is running in one container and DB client is running in another container
- DB client is using host name (not IP address) of the first container
- Both containers have to be on the same network. This can be achieved in two ways:
- DB container does not specify any network but DB client container is using --network=container:<DB_container_name>
- a custom network is created and both containers are using --network=<network_name>
- DB has to have set network alias so DB client can use it as DB's host name
Creating custom network:
$ docker network create \
--subnet=172.18.0.0/24 \
--gateway 172.18.0.1 \
custom_network
Starting DB container:
$ docker network create \
--subnet=172.18.0.0/24 \
--gateway 172.18.0.1 \
custom_network
Starting DB container:
$ docker run \
--name postgres \
--network custom_network \
--publish 5432:5432 \
--hostname db \ <-- this is irrelevant for DNS resolution
--network-alias db \ <-- this is relevant for DNS resolution
--ip 172.18.0.2 \
...
postgres
Starting DB client container:
$ docker run \
-e PGHOST=db \
--name db_client \
--network custom_network \
...
dockerhub.com/mydbclient
From the docs: --hostname option specifies the hostname a container uses for itself. Defaults to the container’s ID if not specified.
To check container's network do:
$ docker inspect <container_name_or_id>
or
$ docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $INSTANCE_ID
Container's IPv4 address has to be found in NetworkSettings.Networks.<network_name>.IPAddress. For backward compatibility and only for the default bridge network, the IPv4 address will also be listed in NetworkSettings.IPAddress.
Can't resolve set hostname from another docker container in same network
If (Node.js) app for some reason can't resolve hostname, its networking stack might error with:
Error: getaddrinfo EAI_AGAIN
Error: getaddrinfo EAI_AGAIN
My Docker container does not have IP address. Why?
docker inspect <container-id> returns blank IP address
Troubleshooting Container Networking
Useful for testing: blocking particular domain for Docker container
Use --add-host. Example:
$ docker run -e APP_ENV=dev -e DB_HOST=1.2.3.4 -e DB_PORT=5432 -e DB_NAME=my_db_dev -e DB_USER=postgres -e DB_PASSWORD=postgres -e OUTPUT_DIR=./data-vol --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol --network=db_default --add-host dev.example.com:0.0.0.0 --name my-app-container my-app-image --param0=arg0
Adding something like
RUN echo "0.0.0.0 dev.example.com" >> /etc/hosts && cat /etc/hosts
to Dockerfile won't work.
Useful for testing: blocking particular domain for Docker container
Use --add-host. Example:
$ docker run -e APP_ENV=dev -e DB_HOST=1.2.3.4 -e DB_PORT=5432 -e DB_NAME=my_db_dev -e DB_USER=postgres -e DB_PASSWORD=postgres -e OUTPUT_DIR=./data-vol --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol --network=db_default --add-host dev.example.com:0.0.0.0 --name my-app-container my-app-image --param0=arg0
Adding something like
RUN echo "0.0.0.0 dev.example.com" >> /etc/hosts && cat /etc/hosts
to Dockerfile won't work.
How to make Docker using Proxy server?
One of the options is to use environment variables:
docker run \
...
-e HTTP_PROXY=10.21.32.40:8080 \
-e HTTPS_PROXY=10.21.32.40:8080 \
...
Configure Docker to use a proxy server | Docker Documentation
Running Docker Containers as Specified User
Processes In Containers Should Not Run As Root
Running a Docker container as a non-root user
How to use Docker without sudo on Ubuntu
Add current user to docker group (which members can talk to docker service which is running as root):
$ sudo gpasswd -a $USER docker
# or sudo usermod -aG docker $USER
$ newgrp docker
Running Docker Containers as Current Host User
What's the default user for docker exec?
Permission denied on accessing host directory in docker
Environment variables
Use -e option followed by NAME=VALUE like in this example:
-e POSTGRES_PASSWORD=mysecretpassword
Using volumes
Use volumesFrom Volume mapping making directories out of files:
When using the -v <host-path>:<container-path> option is used for a bind-mount, docker will look for <host-path> on the host where the daemon runs; if that path does not exist, it will create a directory on that location (as it doesn't know if you wanted to mount a file or a directory; a directory is the default). (Windows-specific: If you don't have a shared drive set-up, docker won't be able to access that path from your host, thus won't find the path inside the VM (thus creates a directory)).
To prevent docker from creating the path if it cannot find it (and produce an error instead), use the --mount option instead of -v. The --mount flag uses a different syntax, which can be found on this page; https://docs.docker.com/engine/admin/volumes/bind-mounts/#start-a-container-with-a-bind-mount
Volumes are the preferred mechanism for persisting data generated by and used by Docker containers.
New users should try --mount syntax which is simpler than --volume syntax.
--mount 'type=volume,src=<VOLUME-NAME>,dst=<CONTAINER-PATH>,volume-driver=local,volume-opt=type=nfs...'
type - type of the mount, which can be bind, volume, or tmpfs
src - For named volumes, this is the name of the volume. May be specified as source or src. It can be either absolute or relative path.
dst - destination takes as its value the path where the file or directory is mounted in the container; May be specified as destination, dst, or target. It has to be absolute path otherwise an error similar to this is issued:
$ docker run --rm -it --mount type=volume,src=./data-vol-src,target=./data-vol-target --name go-demo go-demo
docker: Error response from daemon: invalid mount config for type "volume": invalid mount path: './data-vol-target' mount path must be absolute.
See 'docker run --help'.
Example:
$ docker run --mount type=volume,source=myvol,target=/app ...
Docker Tip #33: Should You Use the Volume or Mount Flag?
BK: I could not make docker run --mount working when having both type=volume and source=`pwd`/myvol (or $(pwd)/myvol, ${pwd}/myvol, "$(pwd)/myvol", "$(pwd)"/myvol...etc...).
Example:
$ docker run --rm -it --mount type=volume,src="$(pwd)/data-vol",target=/go/src/github.com/BojanKomazec/go-demo/data-vol --name go-demo go-demo
docker: Error response from daemon: create /home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/data-vol: "/home/bojan/dev/go/src/github.com/BojanKomazec/go-demo/data-vol" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.
This is very likely a consequence of Docker's way of handling volumes: When -v (or -- mount type=volume) is specified Docker will create a new volume in the docker host's /var/lib/docker/volumes/new_volume_name/_data and mount to the container. We can't use an arbitrary absolute path for the volume (the one that includes $(pwd)) if volume has to be in a directory predefined by Docker.
Nevertheless, if we switch to binding local host's arbitrary directory to some container's one then $(pwd) works well.
Example:
$ docker run --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/go/src/github.com/BojanKomazec/go-demo/data-vol --name go-demo go-demo
Our application can pick up target directory via environment variable: we need to add e.g. OUTPUT_DIR=./data-vol to .env file or to docker run -e argument.
If some file is created and saved to disk by the application which runs within Docker container, that file will actually be saved on the host's disk, in the directory in container's volume. E.g.
/var/lib/docker/volumes/f84dd5236429fc84c31104335394bd08f5292316c93fb04552e68349877ba0ca/_data/storage/test_example.com/demo_table_from_db.csv
To access (e.g. view) this file, attach to container's terminal as:
$ docker exec -it container_name /bin/bash
or
$ docker exec -it container_name /bin/sh
...if bash is not available in the image.
Example:
$ docker exec -it pgadmin /bin/bash
OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown
$ docker exec -it pgadmin /bin/sh
/pgadmin4 # ls -la
...
Troubleshooting #1
If you use docker run and then see that container is not running (e.g. the output of docker container ls shows its status as Exited(1)) check the log of the container:
$ docker inspect <container_name>
In the output json look for LogPath attribute:
"LogPath":"/var/lib/docker/containers/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6-json.log"
Then see the content of the log file:
$ sudo cat /var/lib/docker/containers/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6/62fdb19e88ded813969915891f6a2dec6a83cb3a28c1054fbdd1e001f033ccf6-json.log
{"log":"You need to specify PGADMIN_DEFAULT_EMAIL and PGADMIN_DEFAULT_PASSWORD environment variables\n","stream":"stdout","time":"2019-06-05T10:14:22.078604896Z"}
Troubleshooting #2
If Dockerfile specifies an executable to be run via CMD but some of the dependencies of that executable are missing (e.g. base image is alpine so doesn't have some libs installed...) running this container might report an error:
$ docker run --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol --name myapp docker.example.com/myapps/myapp:build-28
standard_init_linux.go:211: exec user process caused "no such file or directory"
To verify that it is indeed binary stated in CMD that is making problems, we can override CMD by specifying /bin/sh as the executable so we can browse the file system of the container:
$ docker run --rm -it --mount type=bind,src="$(pwd)/data-vol",target=/data-vol --name myapp docker.example.com/myapps/myapp:build-28 /bin/sh
/ # ls -la
total 8404
drwxr-xr-x 1 root root 4096 Jul 25 13:27 .
drwxr-xr-x 1 root root 4096 Jul 25 13:27 ..
-rwxr-xr-x 1 root root 0 Jul 25 13:27 .dockerenv
drwxr-xr-x 2 root root 4096 Jul 11 17:29 bin
-rwxr-xr-x 1 root root 8532786 Jul 24 17:22 myapp
/ # ./myapp
/bin/sh: ./myapp: not found
/ # myapp
/bin/sh: myapp: not found
...
Troubleshooting #3
If docker run shows that app started with CMD doesn't have access privileges, it will output an error similar to this:
error TS5033: Could not write file '/home/node/myapp/build/index.js': EACCES: permission denied, open '/home/node/myapp/build/index.js'.
To find out the permissions on the /home/node/myapp/ you can temporarily change CMD app in Dockerfile:
CMD [ "ls", "-la", "/home/node/myapp" ]
Now rebuild the image and run the container - the output will show owners of that directory and files in it:
$ docker build -t bkomazec/myapp .
$ docker run --user node --rm -it --name running-myapp bkomazec/myapp
If docker run shows that app started with CMD doesn't have access privileges, it will output an error similar to this:
error TS5033: Could not write file '/home/node/myapp/build/index.js': EACCES: permission denied, open '/home/node/myapp/build/index.js'.
To find out the permissions on the /home/node/myapp/ you can temporarily change CMD app in Dockerfile:
CMD [ "ls", "-la", "/home/node/myapp" ]
Now rebuild the image and run the container - the output will show owners of that directory and files in it:
$ docker build -t bkomazec/myapp .
$ docker run --user node --rm -it --name running-myapp bkomazec/myapp
docker start
To start a container use docker start. In the following example container name is my-postgres:
$ docker start my-postgres
my-postgres
docker stop
To stop running one or more containers use docker stop:
$ docker stop my_service_postgres_1
my_service_postgres_1
docker image
$ docker image --helpUsage: docker image COMMAND
Manage images
Commands:
build Build an image from a Dockerfile
history Show the history of an image
import Import the contents from a tarball to create a filesystem image
inspect Display detailed information on one or more images
load Load an image from a tar archive or STDIN
ls List images
prune Remove unused images
pull Pull an image or a repository from a registry
push Push an image or a repository to a registry
rm Remove one or more images
save Save one or more images to a tar archive (streamed to STDOUT by default)
tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
Run 'docker image COMMAND --help' for more information on a command.
---
docker image ls
$ docker image ls --help
Usage: docker image ls [OPTIONS] [REPOSITORY[:TAG]]
List images
Aliases:
ls, images, list
Options:
-a, --all Show all images (default hides intermediate images)
--digests Show digests
-f, --filter filter Filter output based on conditions provided
--format string Pretty-print images using a Go template
--no-trunc Don't truncate output
-q, --quiet Only show numeric IDs
To list IDs of all images:
$ docker image ls -aq
docker image rm
$ docker image rm --help
Usage: docker image rm [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Aliases:
rm, rmi, remove
Options:
-f, --force Force removal of the image
--no-prune Do not delete untagged parents
To delete all images:
$ docker image rm $(docker image ls -aq)
---
docker images
To see which images are present locally, use docker images:
$ docker images --help
Usage: docker images [OPTIONS] [REPOSITORY[:TAG]]
List images
Options:
-a, --all Show all images (default hides intermediate images)
--digests Show digests
-f, --filter filter Filter output based on conditions provided
--format string Pretty-print images using a Go template
--no-trunc Don't truncate output
-q, --quiet Only show numeric IDs
Example:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
my_app latest 477db2641216 12 minutes ago 193MB
<none> <none> 7f82c84cc05a 12 minutes ago 193MB
docker volume
$ docker volume
Usage: docker volume COMMAND
Manage volumes
Commands:
create Create a volume
inspect Display detailed information on one or more volumes
ls List volumes
prune Remove all unused local volumes
rm Remove one or more volumes
Run 'docker volume COMMAND --help' for more information on a command.
---
docker volume ls
$ docker volume ls --help
Usage: docker volume ls [OPTIONS]
List volumes
Aliases:
ls, list
Options:
-f, --filter filter Provide filter values (e.g. 'dangling=true')
--format string Pretty-print volumes using a Go template
-q, --quiet Only display volume names
---
$ docker volume ls
DRIVER VOLUME NAME
local 1a2a9d0f43ce94ad1e176f9bb318f46829fcd34354d4d648d70434efa0f43f60
local 1a66b905bcd4be936b8df26cacf018d7f863e758773a11be5faf9d9949311eb0
local 1c6af16b8d8b1041ad3a46e3910bde9121b64cfae65f8299cedd81793d8fce37
...
$ docker volume ls -q
1a2a9d0f43ce94ad1e176f9bb318f46829fcd34354d4d648d70434efa0f43f60
1a66b905bcd4be936b8df26cacf018d7f863e758773a11be5faf9d9949311eb0
1c6af16b8d8b1041ad3a46e3910bde9121b64cfae65f8299cedd81793d8fce37
---
docker volume rm
$ docker volume rm --help
Usage: docker volume rm [OPTIONS] VOLUME [VOLUME...]
Remove one or more volumes
Aliases:
rm, remove
Examples:
$ docker volume rm hello
hello
Options:
-f, --force Force the removal of one or more volumes
---
To remover all volumes:
$ docker volume rm $(docker volume ls -q)
TBC...
References
Lifecycle of Docker Container
Hey Docker! Why do you hate Windows so much?
Exploring Docker container's file system
1 comment:
Thanks for sharing the valuable information. Keep on sharing.
Docker and Kubernetes Training
Docker and Kubernetes Online Training
Kubernetes Online Training
Docker Online Training
Post a comment