Demystifying Docker & Kubernetes

Networking
Who am I?

• Senior Devops Engineer at Onmobile Global

• Part of Bangalore Docker Community
• One of the Contributors in Docker Labs
• Have travelled extensively from work.
• A Linux, Docker Enthusiast, Avid Runner and Cyclist

LinkedIn profile https://in.linkedin.com/in/balasundaram-natarajan-43471115

Agenda

•Overview of Container Networking Standards

•Docker CNM-Container Networking Model Dive
•Kubernetes CNI- Container Networking Interface Dive
Container Standards
Container Standards

Many different standards

Putting all Container Standards together

Allows users to build container The container engine is OCI compliant runtimes can OCI compliant runtimes can
images with any tool they choose. responsible for creating the consume the config.json and consume the config.json and
Different tools are good for config.json file and unpacking root filesystem, and tell the root filesystem, and tell the
different use cases. images into a root file system. kernel to create a container. kernel to create a container.
Container in comparison with OSI
Container Building Blocks
Namespace
• Linux provides seven different namespaces
(Cgroup, IPC, Network, Mount, PID, User and UTS).
• Network namespaces (CLONE_NEWNET) determine the network resources that are
available to a process,
• Each network namespace has its own network devices, IP addresses, IP routing
tables, /proc/net directory, port numbers, and so on.

cgroups:
• blkio, cpu, cpuacct, cpuset, devices, hugetlb, memory,
• net_cls,net_prio, pids, Freezer,Perf_events,ns
• xt_cgroup(cgroupv2)
Container Building Blocks

In cgroups v1, you could assign threads of the same process to different cgroups.But in Cgroup v2, this is not
possible. Rhel8 by default comes up with cgroupv2.
Note: Kernel version has to be 4.5 and above
Container Networking
High Level Abstractions

CTR 1 CTR2
Container Network Model
CNM VS CNI

Note: https://kubernetes.io/blog/2016/01/why-kubernetes-doesnt-use-libnetwork/
Containers and the CNM
Endpoint Sandbox Network Container

Container C1 Container C2 Container C3

Network A Network B
CNM Driver Interfaces
Docker Default Network Drivers
Null/None Network
Default Bridge Network(docker0)

Docker host Docker host

Cntnr 1 Cntnr 2 Cntnr 3 Cntnr 4 Cntnr 5 Cntnr 6 Cntnr 7

bridgenet1 bridgenet2 bridgenet3

docker network create -d bridge --name bridgenet1

Docker Bridge Networking and Port Mapping

Docker host 1
Host port Container port
Cntnr1

10.0.0.8 :80
$ docker container run -p 8080:80 ...

Bridge

172.14.3.55 :8080

L2/L3 physical network

Custom Bridge Network
Host Network
 DEMO
https://labs.play-with-docker.com
Typical On-Premise Deployment
Macvlan Network
Ipvlan Mode L2
Ipvlan Mode L3
Overlay Mode
The overlay driver enables simple and secure multi-host networking
Overlay Mode
What is Service Discovery
The ability to discover services within a Swarm

• Every service registers its name with the Swarm

• Every task registers its name with the Swarm
• Clients can lookup service names
• Service discovery uses the DNS resolver embedded inside each
container and the DNS server inside of each Docker Engine
Service Discovery Big Picture
Docker host 1 Docker host 2
task1.myservice task2.myservice task3.myservice

“mynet” network (overlay)

task1.myservice 10.0.1.19
task2.myservice 10.0.1.20
task3.myservice 10.0.1.21
myservice 10.0.1.18
Swarm DNS (service discovery)
 Service Virtual IP (VIP) Load Balancing
• Every service gets a VIP when it’s created
• This stays with the service for its entire life
• Lookups against the VIP get load-balanced across all healthy
tasks in the service
• Behind the scenes it uses Linux kernel IPVS to perform transport
layer load balancing
• docker service inspect <service> (shows the service VIP)
NAME HEALTHY IP
Service myservice 10.0.1.18
VIP task1.myservice Y 10.0.1.19
task2.myservice Y 10.0.1.20
Load balance task3.myservice Y 10.0.1.21
group task4.myservice Y 10.0.1.22
task5.myservice Y 10.0.1.23
32
What is the Routing Mesh
Native load balancing of requests coming from an external source

• Services get published on a single port across the entire Swarm

• Incoming traffic to the published port can be handled by all Swarm
nodes
• A special overlay network called “Ingress” is used to forward the
requests to a task in the service
• Traffic is internally load balanced as per normal service VIP load
balancing

33
Routing Mesh Example

Docker host 1 Docker host 2 Docker host 3

1. Three Docker hosts
task1.myservice task2.myservice
2. New service with 2 tasks
3. Connected to the mynet overlay
network
IPVS IPVS IPVS
4. Service published on port 8080
swarm-wide
5. External LB sends request to Docker “mynet” overlay network
host 3 on port 8080
6. Routing mesh forwards the request to Ingress network
a healthy task using the ingress network
8080 8080 8080

LB
Swarm Topology
Node

● Each Node has a role

● Roles are dynamic Node
Node
● Programmable Topology
Node
Node

Node Node

Node

Node
Node Node
Node
Manager
Worker
Swarm Topology: High Availability
Swarm Topology: High Availability

Follower Leader Follower

Swarm Swarm Swarm
Manager Manager Manager

Swarm Swarm Swarm Swarm Swarm Swarm

Worker Worker Worker Worker Worker Worker
Swarm Topology: High Availability

Follower Leader Follower

Swarm Swarm Swarm
Manager Manager Manager

Swarm Swarm Swarm Swarm Swarm Swarm

Worker Worker Worker Worker Worker Worker
Swarm Topology: High Availability

Leader Follower Follower

Swarm Swarm Swarm
Manager Manager Manager

Swarm Swarm Swarm Swarm Swarm Swarm

Worker Worker Worker Worker Worker Worker
Swarm Topology: High Availability

Leader Follower Follower

Swarm Swarm Swarm
Manager Manager Manager

Swarm Swarm Swarm Swarm Swarm Swarm

Worker Worker Worker Worker Worker Worker
Services \ Tasks
• Services provide a piece of functionality
• Based on a Docker image

• Replicated Services and Global Services

• Tasks are the containers that actually do the work

• A service has 1-n tasks
How service deployment works

$ docker service create declares

Declare the service name, network, image:tag
and scale

Reconcile Schedule

Engines check to see what is running Managers break down service into
and compared to what was declared tasks, schedules them and
to “true up” the environment workers execute tasks
Services

Engine
Engine

Engine
mynet

Engine
Engine Engine

$ docker service create --replicas 3 --name frontend --network

mynet
--publish 80:80/tcp frontend_image:latest
Services

Engine
Engine

Engine
mynet

Engine
Engine Engine

$ docker service create --replicas 3 --name frontend --network

mynet --publish 80:80/tcp frontend_image:latest
$ docker service create --name redis --network mynet redis:latest
Node Failure

Engine
Engine

Engine
mynet

Engine
Engine Engine

$ docker service create --replicas 3 --name frontend --network

mynet --publish 80:80/tcp frontend_image:latest
$ docker service create --name redis --network mynet redis:latest
Desired State ≠ Actual State

Engine
Engine

Engine
mynet

Engine
Engine

$ docker service create --replicas 3 --name frontend --network

mynet --publish 80:80/tcp frontend_image:latest
$ docker service create --name redis --network mynet redis:latest
Converge Back to Desired State
mynet

Engine
Engine

Engine

Engine
Engine

$ docker service create --replicas 3 --name frontend --network

mynet --publish 80:80/tcp frontend_image:latest
$ docker service create --name redis --network mynet redis:latest
Container Network Interface
Kubernetes At a High Level
Kubernetes Fundamentals
Kubernetes Fundamentals
Kubernetes Networking Fundamentals
Kubernetes Networking Fundamentals
Network Landscape in Kubernetes
CNI
Kube-Proxy

Alternatives to Kube-proxy
Kubernetes Networking Model

Given the above constraints , below problems to be solved in Kubernetes Networking

Container to container networking
Pod Networking
Pod to Pod Networking
Pod to Pod Networking same node
Pod to Pod Networking different node
Overlay approach
Service
Kubernetes service concept
Pod to Service Networking
Service to Pod Networking
Service Networking Options
Nodeport
Load Balancer
Ingress Layer7 Load balancing
DENY all traffic to an application

LIMIT traffic to an application

DENY all non-whitelisted traffic in a namespace

DENY all traffic from other namespaces

ALLOW traffic from other namespaces

ALLOW traffic from external clients

Multi networking pods
Reference

• https://github.com/collabnix/dockerlabs
• https://docs.docker.com
• http://www.collabnix.com
• https://kubernetes.io/docs/concepts/cluster-administration/networking/
• https://sookocheff.com/post/kubernetes/understanding-kubernetes-networking-model/
• https://www.digitalocean.com/community/tutorials/how-to-inspect-kubernetes-networking
• https://success.docker.com/article/docker-ee-best-practices#astandarddeploymentarchitecture
• https://success.docker.com/article/networking
• https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/
• https://medium.com/@reuvenharrison/an-introduction-to-kubernetes-network-policies-for-
security-people-ba92dd4c809d
• https://sreeninet.wordpress.com/2016/05/29/docker-macvlan-and-ipvlan-network-plugins/
Thank You