Docker Compose

An open-source platform called Docker makes designing, shipping, and deploying applications simple. It runs an application in an isolated environment by compiling its dependencies into a so-called container. for additional information on Docker. In a normal case, several services, such as a database and load balancing, are required to support an application.

We'll look at Docker Compose's assistance with setting up many services in this article. Also, we will see a demonstration of installing and utilizing Docker Compose. Let's try to understand docker-compose simply.

Docker Compose will execute a YAML-based multi-container application. The YAML file consists of all configurations needed to deploy containers Docker Compose, which is integrated with Docker Swarm, and provides directions for building and deploying containers. With Docker Compose, each container is constructed to run on a single host.

Key Concepts in Docker Compose

Docker Compose is a powerful tool for managing multi-container applications, and mastering its key components—like services, networks, volumes, and environment variables—can greatly enhance its usage. Let’s break down these concepts and how they work within a Docker Compose file.

Docker Compose File (YAML Format)

Docker Compose configurations are mainly stored in a file named docker-compose.yml, which uses YAML format to define an application’s environment. This file includes all the necessary details to set up and run your application, such as services, networks, and volumes. To use Docker Compose in an effective way you have to know the structure of this file.

Key Elements of the YAML Configuration

• Version: It defines the format of the Compose file, by ensuring compatibility with specific Docker Compose features and syntax.
• Services: The services section lists each containerized service required for the application. Each service can have its configuration options, such as which image to use, environment variables, and resource limits.
• Networks: In network section, you can define custom networks that enable communication between containers. Additionally, it allows you to specify network drivers and custom settings for organizing container interactions.
• Volumes: Volumes allow for data persistence across container restarts and can be shared between containers if needed. They enable you to store data outside the container's lifecycle, making it useful for shared storage or preserving application state.

Example docker-compose.yml

Here’s a sample Compose file that defines two services, a shared network, and a volume:

version: '3.8'

services: 
  web:    
    image: nginx:latest   
    ports:     
      - "80:80"   
    networks:      
      - frontend    
    volumes:      
      - shared-volume:/usr/share/nginx/html
    depends_on:
      - app

  app:   
    image: node:14   
    working_dir: /app  
    command: node server.js   # Specify a command
    networks:     
      - frontend    
    volumes:      
      - shared-volume:/app/data

networks:  
  frontend:   
    driver: bridge

volumes:  
  shared-volume:  # Remove incorrect syntax

Explanation:

• The web service runs an Nginx container, and app runs a Node.js container.
• Both services connect through the frontend network, allowing them to communicate.
• The shared-volume volume is mounted in both containers, providing shared storage for files.

Docker Compose Services

In Docker Compose, every component of your application operates as a separate service, with each service running a single container tailored to a specific role—such as a database, web server, or cache. These services are defined within the `services` section of the `docker-compose.yml` file. This section lets you configure each service individually, specifying details like the Docker image to pull, environment variables, network connections, and storage options. Through this setup, you can control how each part of your application interacts, ensuring smooth communication and resource management across the services.

Key Service Configuration Options

• Image: Image option defines which Docker image we are going to use by the service from the Docker Hub or any other registry.
• Build: Instead of pulling an image, you can build one locally by specifying a directory containing a Dockerfile. The build is ideal for including custom code in your application.
• Ports: This setting maps a container's internal ports to those on the host machine, enabling access to services from outside the container.
• Volumes: Volumes attach persistent storage to a service ensuring that the data remains accessible even when a container restart.
• Environment: Environment variables allow you to pass configurations or sensitive information, like database credentials or API keys, to the service.
• Depends_on: Depends_on controls the startup order of services, ensuring that certain containers are running before others begin.

Example of docker-compose.yml Configuration

Here’s a sample configuration that demonstrates how these options are used:

version: '3.8'
services:    
    db:         
         image: postgres:13        
         environment:             
             - POSTGRES_USER=user            
             - POSTGRES_PASSWORD=pass        
    volumes:           
        - db_data:/var/lib/postgresql/data   

    web:         
         build: ./web        
         ports:            
             - "5000:5000"        
   volumes:             
       - web_data:/usr/src/app         
   environment:             
       - DATABASE_URL=postgres://user:pass@db:5432/mydb        
   depends_on:             
        - db

volumes:  
    db_data:  
    web_data:

Explanation:

• The db service runs a PostgreSQL container. It uses environment variables to set up a database username and password, and stores data on the db_data volume to ensure it’s retained.
• The web service is built from a Dockerfile in the ./web directory and exposes port 5000. The web_data volume is mounted to store application files persistently. It depends on the db service, ensuring the database is available when the web service starts.

Docker Compose Networks

Docker Compose deployments use networks to allow secure communications between the services. Services defined in a docker-compose.yml file are by default placed on one network and are able to connect to each other without any additional setup. For more strict control, you can create additional networks and assign services to them in order to control the way they communicate or to separate some groups of services as the need arises.

Key Network Configuration Options

• Driver: Driver are used in the network driver type, such as bridge (the default for local networks) or overlay (for multi-host networks in Docker Swarm), which determines how services connect to each other.
• Driver Options (driver_opts): Driver Options(driver_opts) allows for additional settings on the network driver, useful for fine-tuning network behavior to meet specific needs.
• IP Address Management (ipam):IP address management configures network-level IP settings, like subnets and IP ranges, to give you greater control over the IP address space assigned to your services.

Example docker-compose.yml with Custom Networks

Below is an example Compose file that sets up two networks, one for database communication and another for web access.

version: '3.8'

services:  
   db:    
        image: postgres:13    
        networks:      
            - backend  

   web:    
         image: nginx:latest   
         networks:     
             - frontend      
             - backend    
         ports:      
             - "80:80"

networks: 
    frontend:    
       driver: bridge 

backend:    
     driver: bridge   
     ipam:      
         config:        
             - subnet: 172.16.238.0/24

Explanation:

• The db service uses thebackend network, isolating it from the frontend network to limit access.
• The web service is connected to both frontend and backend networks, allowing it to communicate with the db service while remaining accessible via the frontend network.
• The backend network includes IPAM settings with a specific subnet range, ensuring custom IP address management.

Docker Compose Volumes

Volumesin docker compose are used to persist data created or used by the docker containers. By doing so they enable the data to persist even if containers are stopped or removed in your docker-compose. Within a docker-compose. yml file, the volumes section describes all the volumes that are attached to the services allowing you to manage data that exists independently of the container lifecycle.

Key Volume Configuration Options

• External: Set true to signify that the volume was created externally, outside of Docker Compose (such as via docker volume create) and simply referenced in the configuration.
• Driver: This indicates which volume driver the volume should use, and it controls how are these volumes being handled. By default, the driver is local, but other options are also available.
• Driver Options (driver_opts): Additional options to customize the volume driver like filesystem type or different storage parameters.

Example docker-compose.yml with Volumes

Here’s a practical example showing how to configure a volume for a Pos-tgreSQL database, ensuring that its data is stored persistently.

version: '3.8'

services:    
    db:         
         image: postgres:13        
         environment:              
              - POSTGRES_USER=user              
              - POSTGRES_PASSWORD=pass   
         volumes:      
             - db_data:/var/lib/postgresql/data

volumes:  
    db_data:    
         driver: local    
         driver_opts:      
             type: none      
             o: bind      
             device: /path/to/local/db_data

Explanation

• The db service runs a PostgreSQL container, with its data stored in the db_data volume. This setup ensures that the database information remains intact across restarts or removals of the container.
• The db_data volume is configured to use the localdriver, and it has driver options set to create a bind mount pointing to a specific path on the host system (/path/to/local/db_data). This means that all database files are saved in that designated directory on the host.
• By using volumes in this way, you can keep essential data safe and easily accessible, separate from the container itself.

Docker Compose Environment Variables

Environment variables are a simple and effective way to pass configuration settings from your host operating system through Docker Compose in order to get to your services. You can set these variables directly on the service definition by using the environment section or load them from an external file.

How to Set Environment Variables in Docker Compose?

• Inline: you may declare env vars directly in the service definition.This approach is simple and gives everything in one place.
• env_file: This option allows you to load environment variables from an external file, making it easier to manage configuration, especially when dealing with many variables.

Example docker-compose.yml Using Environment Variables

Here’s an example that demonstrates both methods of setting environment variables for a web application and a database service.

version: '3.8'

services:  
    db:    
         image: postgres:13    
         environment:      
             - POSTGRES_USER=user      
             - POSTGRES_PASSWORD=pass    
         volumes:      - db_data:/var/lib/postgresql/data  


    web:    image: my-web-app:latest    
                 build: ./web    
                 environment:      
                     - DATABASE_URL=postgres://user:pass@db:5432/mydb    
                 env_file:      
                      - .env

volumes: 
    db_data:

Explanation

• In the db service, the POSTGRES_USER and POSTGRES_PASSWORD environment variables are defined inline, specifying the database credentials directly.
• The web service uses an inline variable for DATABASE_URL, which connects to the PostgreSQL database. Additionally, it loads environment variables from an external file named .env. This file can contain various settings, such as API keys, application configurations, and other sensitive information.

With a good understanding of these basic principles, developers are ready to use Docker Compose to manage and orchestrate applications that can be quite complex and involve many Docker containers.

Install Docker Compose

We can run Docker Compose on macOs, Widows, and 64-bit Linux. 

• For any significant activity, Docker Compose depends on Docker Engine. Depending on your arrangement, we must ensure that Docker Engine is installed either locally or remotely.
• A desktop system such as Docker for Mac and Windows comes with Docker Compose preinstalled. 
• Install Docker first as instructed in Docker installation on the Linux system before beginning the installation of Docker Compose. 

Install Docker Compose on Ubuntu - A Step-By-Step Guide

Step 1: Update the package Manager

• The following scripts will install the most recent version of Docker Compose and update the package management.

sudo apt-get update

Step 2: Download the Software

• Here we are using the Ubuntu flavor in the Linux Operating system. So the package manager is "apt-get" If you want to install it in Redhat Linux then the package manager will be "yum".

Step 3: Apply Permissions

• Apply the Executable permissions to the software with the following commands:

sudo chmod +x /usr/local/bin/docker-compose

Step 4: Verify the Download Software

• Verify the whether the docker compose is successfully installed or not with the following command:

docker-compose --version

Docker Container

A docker container is a lightweight Linux-based system that packages all the libraries and dependencies of an application, prebuilt and ready to be executed. It is an isolated running image that makes the application feel like the whole system is dedicated to it. Many large organizations are moving towards containers from VMs as they are light and simple to use and maintain. But when it comes to using containers for real-world applications, usually one container is not sufficient. For example, Let's assume Netflix uses a microservices architecture. Then it needs services for authentication, Login, Database, Payment, etc, and for each of these services, we want to run a separate container. It is preferred for a container to have only a single purpose.

Now, imagine writing separate docker files, and managing configuration and networks for each container. This is where Docker Compose comes into the picture and makes our lives easy.

Why Docker Compose?

As discussed earlier,  a real-world application has a separate container for each of its services. And we know that each container needs to have a Dockerfile. It means we will have to write maybe hundreds of docker files and then manage everything about the containers individually, That's cumbersome. 

Hence we use docker-compose, which is a tool that helps in the definition and running of multi-container applications. With the help of Docker Compose you can start and stop the services by using its YAML file. Docker-compose allows us to start and stop all of the services with just a few simple commands and a single YAML file for each configuration.

In contrast to utilizing a prebuilt image from Docker Hub, which you may configure with the docker-compose.yaml file, if you are using a custom image, you will need to declare its configurations in a separate Dockerfile. These are the features that docker-compose support:

• All the services are isolated running on a single host.
• Containers are recreated only when there is some change.
• The volume data is not reset when creating new containers, volumes are preserved.
• Movement of variables and composition within environments.
• It creates a virtual network for easy interaction within the environments.

Now, let's see how we can use docker-compose, using a simple project.

How to Use Docker Compose?

In this project, we will create a straightforward Restfull API that will return a list of fruits. We will use a flask for this purpose. And a PHP application will request this service and show it in the browser. Both services will run in their own containers.

Step 1: Create Project Directory

• First, Create a separate directory for our complete project. Use the following command.

mkdir dockerComposeProject

• Move inside the directory.

cd dockerComposeProject

Step 2: Create API

we will create a custom image that will use Python to serve our Restful API defined below. Then the service will be further configured using aDockerfile.

• Then create a subdirectory for the service we will name it product. and move into the same.

mkdir product
cd product

• Create requirements.txt

Inside the product folder, create a file named requirements.txt and add the following dependencies:

flask
flask-restful

Step 3: Build Python  api.py

• The following is the python file that helps in making an API call:

• Create a Dockerfile to define the container in which the above API will run.

from flask import Flask
from flask_restful import Resource, Api

# create a flask object
app = Flask(__name__)
api = Api(app)

# creating a class for Fruits that will hold
# the accessors
class Fruits(Resource):
    def get(self):
        # returns a dictionary with fruits
        return {
            'fruits': ['Mango', 'Pomegranate', 'Orange', 'Litchi']
        }

# adds the resources at the root route
api.add_resource(Fruits, '/')

# if this file is being executed then run the service
if __name__ == '__main__':
    # run the service
    app.run(host='0.0.0.0', port=80, debug=True)

Step 4: Create Dockerfile For Python API 

FROM python:3
WORKDIR /usr/src/app
COPY requirements.txt . 
RUN pip install --no-cache-dir -r requirements.txt
COPY . . 
CMD ["python", "api.py"]

FROM accepts an image name and a version that the docker will download from the docker hub. The current working directory's contents can be copied to the location where the server expects the code to be by using the copy command. Moreover, the CMD command takes a list of commands to start the service once the container has been started.

Step 5: Create PHP HTML Website 

Let's create a simple website using PHP that will use our API.

• Move to the parent directory and create another subdirectory for the website.

cd ..
mkdir website
cd website

index.php

<!DOCTYPE html>
<html lang="en">
<head>
   <title>Fruit Service</title>
</head>
<body>
   <h1>Welcome to India's Fruit Shop</h1>
   <ul>
       <?php
           $json = file_get_contents('http://fruit-service');
           $obj = json_decode($json);
           $fruits = $obj->fruits;
           foreach ($fruits as $fruit){
               echo "<li>$fruit</li>";
           }
       ?>
   </ul>
</body>
</html>

• Now create a compose file where we will define and configure the two services, API and the website.
• Move out of the website subdirectory using the following code.

cd ..

• And then create the file name as . docker-compose.yaml 

Step 6: Create Docker-compose.yaml file 

• The following is the sample docker compose file code:

version: "3"

services:
  fruit-service:
    build: ./product
    volumes:
      - ./product:/usr/src/app
    ports:
      - 5001:80

  website:
    image: php:apache
    volumes:
      - ./website:/var/www/html
    ports:
      - 5000:80
    depends_on:
      - fruit-service

Docker-compose.yaml File

The first line is optional where we specify the version of the docker-compose tool. Next services define a list of services that our application is going to use. The first service is fruit service which is our API and the second one is our website. The fruit service has a property build that contains the dockerfile that is to be built and created as an image. Volumes define storage mapping between the host and the container so that we can make live changes. Finally, port property exposes the containers port 80 through the host's 5001.

The website service does not use a custom image but we download the PHP image from the Docker hub and then map the websites folder that contains our index.php to /var/www/html (PHP expects the code to be at this location). Ports expose the container port. Finally, the depends_on specifies all the services on which the current service depends.

• The folder structure after creating all the required files and directory will be as follows:

Run the application stack with Docker Compose

• Now that we have our docker-compose.yml file, we can run it.
• To start the application, enter the following command.

docker-compose up -d 

Now all the services will start and our website will be ready to be used at localhost:5000.

• Open your browser and enter localhost:5000.

Output

• To stop the application, either press CTRL + C or

docker-compose stop

Advantages of Docker Compose

The following are the advantages of Docker Compose:

• Simplifies Multi-Container Management: Docker Compose facilitates with features such as define, configure, and run multiple containers with a single YAML file, streamlining the management of complex applications.
• Facilitates Environment Consistency: It facilitates with the development, testing, and production environments that are consistent with reducing the risk of environment-related issues.
• Automates Multi-Container Workflows: With Docker Compose, you can easily automate the setup and teardown of multi-container environments, making it ideal for CI/CD pipelines and development workflows.
• Efficient Resource Management: It enables efficient allocation and management of resources across multiple containers, improving application performance and scalability.

Disadvantages of Docker Compose

The following are the disadvantages of Docker Compose:

• Limited Scalability: Docker Compose is not developed for large scaling mechanism which can limit its effectiveness for managing complex deployments.
• Single Host Limitation: Docker Compose will operate on a single host, making it unsuitable for distributed applications with requiring multi-host orchestration.
• Basic Load Balancing: It lacks with advanced load balancing and auto-scaling features found in more robust orchestration tools like Kubernetes.
• Less Robust Monitoring: Docker Compose provides minimal built-in monitoring and logging capabilities compared to more comprehensive solutions.

Important Docker Compose Commands

Best Practices of Docker Compose

The following are the some of the best practices of Docker Compose:

• Use Environment Variables: It is suggestable to store configuration values and secrets in environment variables to keep your docker-compose.yml clean and secure.
• Keep Services Lightweight: It is preferred to design each service to handle a single responsibility to ensure modularity and ease of maintenance.
• Leverage Volumes: Usage of volumes with enhancing in maintaining the persistent data storage, allowing data to persist across container restarts and updates.
• Version Control Your Compose Files: It is preferred to maintain your docker-compose.yml file in version control (e.g., Git) to track changes and collaborate with your team effectively.

Features of Docker Compose

The following are the features of Docker Compose:

• Multi-Container Deployment: it facilitates with easily define and run applications with multiple containers using a single YAML file.
• Service Isolation: Each service runs in its own container, with ensuring the isolation and reducing conflicts between services.
• Simplified Configuration: It helps in centralizing all the configurations, including networking, volumes, and dependencies, in the docker-compose.yml file.
• Scalability: It provides the effortlessly scaling of services up or down with a single command, allowing for flexible and dynamic resource management.

Conclusion

In this article, we learned about Docker Compose, and why and when to use it. And demonstrated its usuage through a simple project. It helps in automating the creating process of containers through services, networks and volumes with through respective keywords availability. Through using docker compose management and automation of containers and its volumes, networks will be easier.

ganesh227

Follow

Improve

Article Tags :

• Docker
• DevOps
• docker