What Is Docker Volume?

Docker Volumes are persistent storage mechanisms managed by Docker that allow containers to retain data beyond their lifecycle. They provide a reliable and efficient way to preserve data generated by containers by mounting a dedicated filesystem into the container. This ensures data remains intact even if the container is stopped, removed, or recreated.

Persistent Storage with Docker Volumes

Docker containers are ephemeral—once a container is removed, any data stored within its internal filesystem is lost. This is ideal for stateless apps but problematic for databases or services that require data persistence.

Docker Volumes solve this by decoupling the data from the container's lifecycle. By storing data in a dedicated location on the host machine managed by Docker, you ensure:

• Data Durability: Information persists even if the container is stopped, deleted, or updated.
• Decoupling: Application logic stays in the container while persistent data lives independently.
• Seamless Updates: You can swap containers or upgrade images without losing your database or user uploads.

Independent Lifecycle of Docker Volumes

Unlike a container, which has a specific lifecycle that ends when it’s stopped or removed, a volume's lifecycle is independent of any particular container. This means that the data stored in a volume persists beyond the container that uses it, making it ideal for retaining important information like databases, configuration files, or logs.

• Persistence Beyond Containers: When a container is destroyed, its writable layer is also deleted, but any data stored in a volume remains intact. This ensures that your data is not lost even if the container is removed.
• Shared Access: A volume can be mounted into multiple containers at the same time, enabling easy data sharing across different applications or services.
• Volume Cleanup: Even if no container is currently using a volume, it remains available to Docker and won’t be automatically removed. If you need to free up space, you can remove unused volumes using the docker volume prune command.

This behavior allows you to manage and protect your data independently from the container lifecycle, making volumes an essential tool for ensuring data persistence and flexibility.

Bind Mounts Vs. Docker Volumes

Types Of Mounts in Docker:

The data appears the same from within the container in all mount modes. In the filesystem of the container, it is shown as a directory or a single file.

• Volumes: Docker manages volumes kept in a section of the host filesystem (/var/lib/docker/volumes on Linux). This portion of the filesystem shouldn't be altered by non-Docker processes. In Docker, volumes are the most effective way to store data. Using the docker volume create command, we may directly create a volume, or Docker can do it for us when it creates a container or service.
• Named Pipes: To facilitate communication between a container and the Docker host, a named pipe mount can be employed. Using a named pipe to connect to the Docker Engine API while running a third-party program inside a container is the typical use case.
• Bind Mounts: On the host system, bind mounts can be kept anywhere. These might be crucial system folders or files. They are always modifiable by non-Docker processes running on a Docker host or in a Docker container. Comparatively speaking, bind mounts are less useful than volumes.
• Tmpfs Mounts: These mounts are stored in the host system's RAM only. They are never written to the computer's hard drive (disk). While they appear as a normal folder inside the container, the data is volatile it is lost instantly if the container stops. This makes them perfect for storing sensitive data (like security tokens) or high-speed temporary files that you don't want persisting on the disk.

Named Vs. Anonymous Volumes

Docker Volume Plugins

Docker Engine volume plugins link Engine installations with external storage systems such as Amazon EBS, allowing data volumes to survive beyond the lifespan of a single Docker host. For further details, please refer to the plugin documentation.

Command-Line Changes

Use the --volume and --volume-driver options on the docker container run command to grant a container access to a volume. The host's volume name and path are accepted by the --volume (or -v) flag, whereas the driver type is accepted by the --volume-driver flag.

 $ docker volume create --driver=flocker volumename

 $ docker container run -it --volume volumename:/data busybox sh

Volume Plugin Protocol

If a plugin registers itself as a VolumeDriver when activated, it must provide the Docker Daemon with writeable paths on the host filesystem. The Docker daemon provides these paths to containers to consume. The Docker daemon makes the volumes available by bind-mounting the provided paths into the containers.

{
    "Name": "volume_name",
    "Opts": {}
}

Using Docker Volumes

Manually Creating and Linking Volumes with Proper Naming And Labeling Conventions

• Make sure you use appropriate name and labelling practices when establishing Docker volumes.

docker volume create \
--label description="my_vol" \
--label version="1.0.1" \
my_vol

Using Volumes in Dockerfiles with Controlling Permissions For Volumes

• In order to preserve data security and integrity, make sure the appropriate permissions are specified for Docker volumes.

FROM baseimage
RUN mkdir /app/data
RUN chown -R 1000:1000 /app/data
RUN chmod 755 /app/data
VOLUME /app/data

Mounting Volumes as Read-Only

Mounting volumes as read-only in Docker allows for the protection of sensitive or critical data from unintended modifications. By setting the volume option to read-only, you ensure that any changes made within the container are not persisted to the underlying volume, preserving data integrity and security.

docker run -d \
  -v /path/on/host:/path/in/container:ro \
  --name my_container \
  my_image

• -v /path/on/host:/path/in/container:ro mounts the directory /path/on/host on the host machine to /path/in/container in the container as read-only (ro).
• --name my_container assigns the name my_container to the Docker container.
• my_image is the name of the Docker image used to create the container.

Tracking And Controlling Volume Consumption

• To maximise resource consumption, track and adjust Docker volume usage on a regular basis.

$ docker system df -v

Populating Volume Content

When mounting volumes to container paths with existing data, Docker ensures data integrity by copying the existing container data into the new volume. Consequently, neighboring mount points and other containers using the volume will also access the populated content, preventing inadvertent data loss.

Reusing Volumes When Containers Start

Instead of manually specifying each volume with the -v flag, you can use --volumes-from to inherit volumes from an existing container when starting a new container:

# Create the first container
$ docker run -d --name test -v my_vol:/data image:latest

# Create the second container
$ docker run -d --name backup --volumes-from test image:latest

This command automatically mounts all volumes from the "test" container into the "backup" container, simplifying the setup process. It's handy for tasks like backing up data from one container to another.

Interacting With Docker Volumes

Each volume's name and the storage driver that supports it will be shown. Use docker volume inspect to obtain more in-depth details about a particular volume instead:

Inspecting Volumes

To inspect volumes in Docker, you can use the docker volume inspect command followed by the name or ID of the volume you want to inspect. For example:

docker volume inspect my_vol

Removing Volumes

To remove volumes in Docker, you can use the docker volume rm command followed by the name or ID of the volume you want to remove. For example:

docker volume rm my_vol

Pruning Volumes

To prune volumes in Docker, you can use the docker volume prune command. This command removes all volumes not used by at least one container. Here's how you can use it:

docker volume prune

Starting a Container with a Volume

On Using -v Option

• We may start a container with a bind mount using the -v option:

$ docker run -v $(pwd):/var/opt/project bash:latest \
  bash -c "ls /var/opt/project"

• This shows nothing from the mount position. However, if we write to the volume within a single execution of the container:

$ docker run -v data-volume:/var/opt/project bash:latest \
  bash -c "echo 'Hello World' > /var/opt/project/test.txttmpfs"

Using The –mount Option

• To indicate the volume we want to mount, we might find it easier to use the more obvious –mount option:

$ docker run --mount \
  'type=volume,src=data-volume,\
  dst=/var/opt/project,volume-driver=local,\
  readonly' \ 
  bash -c "ls /var/opt/project"

On Using Shared Volumes

Assume that we used the data-volume mount in a container to run our echo script. Afterwards, we could make a list of every container we've used:

$ docker ps -a
CONTAINER ID   IMAGE             COMMAND                  CREATED              STATUS                          PORTS      NAMES
5774502f857   bash              "docker-entrypoint.s…"   8 minutes ago        Exited (0) 8 minutes ago                   exciting_payne

How to use Docker Volumes?

The following command launches a fresh Ubuntu 22.04 container and connects your terminal to it (-it), enabling you to execute example commands in the ensuing stages. Within the container, a volume named demo_volume is mounted to /data. Use the following command right now:

$ docker run -it -v demo_volume:/data ubuntu:22.06

• Give a list of everything in the /data directory of your container:

$ ls /data

• Include a test file with any random content:

$ echo "foobar" > /data/foo
$ cat /data/foo
foobar

• Launch a fresh container with the same volume attached now:

$ docker run -it -v demo_volume:/app alpine:latest

• Add the --mount option to the docker run command in order to mount a data volume to a container.
• It stores the data created inside the virtual environment by adding volume to the designated container.
• To launch a container and mount a data drive to it, use the following syntax:

$ docker run --mount source=[volume_name],destination=[path_in_container] [docker_image]  

Troubleshooting Common Docker Volume Issues

Permission Denied When Mounting Volumes

• If you are mounting a local disc or host directory with the -v option while operating a Docker container, as follows:

docker run -it --rm \
    -p 8888:8888 \
    -v <my-vol>:<container-dir> \
    quay.io/jupyter/minimal-notebook:latest

Incompatible CPU detected

A processor (CPU) that supports virtualization—more especially, the Apple Hypervisor framework—is necessary for Docker Desktop to function. Only Mac computers with CPUs that support the Hypervisor framework may use Docker Desktop.

$ sysctl kern.hv_support

Path Conversion On Windows

When using Linux, mounting a route to another path is handled by the system. For instance, when executing the subsequent command on a Linux system:

 $ docker run --rm -ti -v /home/user/work:/work alpine

Permissions Errors On Data Directories For Shared Volumes

Docker Desktop defaults the read, write, and execute permissions for both users and groups on shared volumes to 0777 when sharing files from Windows.On shared discs, the default permissions are not customisable. You must either utilise non-host-mounted volumes or figure out a means to get the programmes to operate with the default file permissions if you are working with applications that need permissions different from the shared volume defaults during container runtime.