Wednesday 6 March 2019

Introduction to docker-compose


Helps managing Docker containers and their dependencies. Its configuration is in docker-compose.yml file

It can also be used in a single project with Dockerfile as instead of manually running docker build <args> and then docker run <args> we can put all these args in docker-compose.yml and then call docker-compose up.


To upgrade it, first uninstall and then install it again. I described the whole process in this article: How to upgrade docker-compose on Linux | My Public Notepad.

Configuration file

docker-compose reads its configuration from a file called docker-compose.yml.

Directories specified in volumes section in yml file don't have to be created manually - docker-compose up will create them.


$ docker-compose
Define and run multi-container applications with Docker.


  docker-compose [-f <arg>...] [options] [COMMAND] [ARGS...]
  docker-compose -h|--help


  -f, --file FILE             Specify an alternate compose file
                              (default: docker-compose.yml)
  -p, --project-name NAME     Specify an alternate project name
                              (default: directory name)
  --verbose                   Show more output
  --log-level LEVEL           Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
  --no-ansi                   Do not print ANSI control characters
  -v, --version               Print version and exit
  -H, --host HOST             Daemon socket to connect to

  --tls                       Use TLS; implied by --tlsverify

  --tlscacert CA_PATH         Trust certs signed only by this CA
  --tlscert CLIENT_CERT_PATH  Path to TLS certificate file
  --tlskey TLS_KEY_PATH       Path to TLS key file
  --tlsverify                 Use TLS and verify the remote
  --skip-hostname-check       Don't check the daemon's hostname against the
                              name specified in the client certificate
  --project-directory PATH    Specify an alternate working directory
                              (default: the path of the Compose file)
  --compatibility             If set, Compose will attempt to convert deploy
                              keys in v3 files to their non-Swarm equivalent


  build              Build or rebuild services
  bundle             Generate a Docker bundle from the Compose file
  config             Validate and view the Compose file
  create             Create services
  down               Stop and remove containers, networks, images, and volumes
  events             Receive real time events from containers
  exec               Execute a command in a running container
  help               Get help on a command
  images             List images
  kill               Kill containers
  logs               View output from containers
  pause              Pause services
  port               Print the public port for a port binding
  ps                 List containers
  pull               Pull service images
  push               Push service images
  restart            Restart services
  rm                 Remove stopped containers
  run                Run a one-off command
  scale              Set number of containers for a service
  start              Start services
  stop               Stop services
  top                Display the running processes
  unpause            Unpause services
  up                 Create and start containers
  version            Show the Docker-Compose version information

Its configuration file is docker-compose.yaml (or .yml). This file is also known as Compose file. It contains:
  • services - list of service names; Each service can have:
    • build
    • command
    • environment
    • image
    • ports
    • restart
    • volumes
docker-compose up will read Compose filecreate and start containers (services) start listed there.

To check the state of all services, run docker-compose ps from the working directory (which is by default the one that contains Compose file):

$ docker-compose ps

         Name                   Command           State           Ports         
my-project_service1_1    /bin/sh -c yarn          Up>3001/tcp
                         install && ...                                         
my-project__postgres_1              Up>5432/tcp                                             
my-project_service3_1    /usr/local/bin/example   Up>4567/tcp                              

To list the IDs of all running containers:

$ docker-compose ps -q

To list all images:

$ docker-compose images
       Container       Repository   Tag       Image Id      Size 
my-project_service1_1   service1   latest   58f28f305f80   875 MB
my-project__postgres_1  postgres   latest   f97c959a7d9c   298 MB
my-project_service3_1   service3   latest   4e02459e1040   159 MB

Another way to find the association between running containers and their IDs:

$ docker ps
CONTAINER ID    IMAGE   COMMAND                     CREATED       STATUS     PORTS                    NAMES
a1403d3f917c  postgres  "postgres.s…"               29 hours ago  Up 3 hours>5432/tcp   postgres
e6aecf2e79b1  service3  "/usr/local/bin/example…"   29 hours ago  Up 3 hours>4567/tcp   browser-engagement_s3_1
1e8a12167a4a  service1  "/bin/sh -c 'yarn in…"      29 hours ago  Up 3 hours>3001/tcp   my-project_service1_1

Stops containers and removes containers, networks, volumes, and images
created by `up`.

docker-compose down

It completely destroys containers (but not images).

$ docker-compose down --help
Stops containers and removes containers, networks, volumes, and images
created by `up`.

By default, the only things removed are:

- Containers for services defined in the Compose file

- Networks defined in the `networks` section of the Compose file
- The default network, if one is used

Networks and volumes defined as `external` are never removed.

Usage: down [options]


    --rmi type              Remove images. Type must be one of:
                              'all': Remove all images used by any service.
                              'local': Remove only images that don't have a
                              custom tag set by the `image` field.
    -v, --volumes           Remove named volumes declared in the `volumes`
                            section of the Compose file and anonymous volumes
                            attached to containers.
    --remove-orphans        Remove containers for services not defined in the
                            Compose file
    -t, --timeout TIMEOUT   Specify a shutdown timeout in seconds.
                            (default: 10)


$ docker-compose down --rmi all --volumes

docker-compose: option to automaticaly remove container after run in docker-compose.yml

docker-compose pull

Let's assume our docker-compose.yml contains two services, db and db_sync and each of them has some Dockerfile which defines base image for their containers. When we do docker-compose up first time, Docker will pull the latest version of those images but each subsequent docker-compose up will not be checking if base images have newer version in the remote image repository - we need to update these images manually. We can look Dockerfile and update each base image manually, like:

$ docker pull node:alpine

We might make mistake and omit some images. Better way is to call docker-compose pull (from a directory which contains yml file) - it will iterate through each service listed in yml and update its base image(s). Example:

$ docker-compose pull
Pulling db     ... done
Pulling db_sync ... done

docker-compose pull
docker-compose up doesn't pull down latest image if the image exists locally #3574

docker-compose restart

It is possible to restart a single container (service). Eg. if we have an app container and db container and we want to restart only app container we can do:

$ docker-compose restart my-app

Caveat: this does not pick up any changes in yml config file or image itself. To rebuild the image and restart the container:

$ docker-compose build my-app && docker-compose restart my-app

docker-compose stop

It only hibernate containers.

docker-compose up

$ docker-compose up --help
Builds, (re)creates, starts, and attaches to containers for a service.

Unless they are already running, this command also starts any linked services.

The `docker-compose up` command aggregates the output of each container. When
the command exits, all containers are stopped. Running `docker-compose up -d`
starts the containers in the background and leaves them running.

If there are existing containers for a service, and the service's configuration
or image was changed after the container's creation, `docker-compose up` picks
up the changes by stopping and recreating the containers (preserving mounted
volumes). To prevent Compose from picking up changes, use the `--no-recreate`

If you want to force Compose to stop and recreate all containers, use the
`--force-recreate` flag.

Usage: up [options] [--scale SERVICE=NUM...] [SERVICE...]

    -d, --detach               Detached mode: Run containers in the background,
                               print new container names. Incompatible with
    --no-color                 Produce monochrome output.
    --quiet-pull               Pull without printing progress information
    --no-deps                  Don't start linked services.
    --force-recreate           Recreate containers even if their configuration
                               and image haven't changed.
    --always-recreate-deps     Recreate dependent containers.
                               Incompatible with --no-recreate.
    --no-recreate              If containers already exist, don't recreate
                               them. Incompatible with --force-recreate and -V.
    --no-build                 Don't build an image, even if it's missing.
    --no-start                 Don't start the services after creating them.
    --build                    Build images before starting containers.
    --abort-on-container-exit  Stops all containers if any container was
                               stopped. Incompatible with -d.
    -t, --timeout TIMEOUT      Use this timeout in seconds for container
                               shutdown when attached or when containers are
                               already running. (default: 10)
    -V, --renew-anon-volumes   Recreate anonymous volumes instead of retrieving
                               data from the previous containers.
    --remove-orphans           Remove containers for services not defined
                               in the Compose file.
    --exit-code-from SERVICE   Return the exit code of the selected service
                               container. Implies --abort-on-container-exit.
    --scale SERVICE=NUM        Scale SERVICE to NUM instances. Overrides the
                               `scale` setting in the Compose file if present.


$ docker-compose up --build

How to make docker-compose read .env file?

How to use environment variables in docker compose




    MY_KEY: ${MY_KEY}

Both $VARIABLE and ${VARIABLE} syntax are supported. [source]

How to get docker run -it ... via docker-compose?

Interactive shell using Docker Compose

Include the following lines in your docker-compose.yml:

stdin_open: true
tty: true

How to make docker-compose remove the container after running?

Use docker-compose down

Auto remove container with docker-compose.yml

How do I mount a host directory as a volume in docker compose

How to set network name, IP range and static IP addresses for each service in yml file?

docker-compose.yml example:

version: '3.7'
    name: postgres-demo-net
      driver: default
        - subnet:

By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.

Your app’s network is given a name based on the “project name”, which is based on the name of the directory it lives in.

Instead of just using the default app network, you can specify your own networks with the top-level networks key.

Each service can specify what networks to connect to with the service-level networks key, which is a list of names referencing entries under the top-level networks key.

source: Networking in Compose

Containers running on the same network can use each other's service names as specified in docker-compose.yml as hostnames (for IP connections). This is handy as we don't need to inspect the network in order to determine their IP addresses. Example: running pgAdmin on the same network as Postgres. When adding a DB server we can use DB service name from yml as DB server's hostname.

Set default network name for compose


How to copy file from one container into another?

$ COPY --from=<service_name> service1/path/to/executable service2/path/to/executable


References and Further Reading Links

How to Use Docker Compose

No comments: