Beruflich Dokumente
Kultur Dokumente
Networking
Hyunsoo Kim(hykim@redhat.com)
Senior Solution Architect
05.Mar.2017
1
Docker Network
Container Networking on a Plain Docker Host
Docker Host
vethXXX
Plain ContainerA
ethØ
IP Stack dockerØ
IP Tables vethYYY
Plain ContainerB
ethØ
eth2 eth1 ethØ
Default LAN
Admin LAN
Storage LAN
3
Access to NAS/SAN storage from a container
Docker Host
vethXXX
Plain ContainerA
ethØ
IP Stack dockerØ
IP Tables vethYYY
Plain ContainerB
ethØ
eth2 eth1 ethØ
Linux Kernel
nfs & network attached
block storage
Default LAN
Admin LAN
Storage LAN
IP Stack dockerØ
IP Tables vethYYY
Plain ContainerB
ethØ
eth2 eth1 ethØ
Default LAN
Admin LAN
Storage LAN
IP Stack dockerØ
IP Tables vethYYY
Plain ContainerB
ethØ
eth2 eth1 ethØ
Default LAN
Admin LAN
Storage LAN
Outbound : Container B eth0 vethYYY docker0 bridge IPTables NAT host network 6
Inbound : host network IPTables DNAT docker0 bridge vethYYY Container B eth0
Host network
7
Container network
8
Host network after a container running
9
Host bridge
10
OpenShift Network
Pod/Container Networking on a OpenShift Node
https://access.redhat.com/documentation/en-us/openshift_enterprise/3.1/html/cluster_administration/admin-guide-sdn-troubleshooting
12
Pod/Container Networking on a OpenShift Node
OpenShift Node
vethXXX
lbrØ Plain ContainerA
ethØ
vlinuxbr
IP Stack vovsbr
IP Tables tunØ
port3 vethYYY
Pod B
port2 brØ port4 ethØ
eth2 eth1 ethØ (OVS) vethZZZ
port1 Pod C
ethØ
vxlanØ
Default LAN
Admin LAN
Storage LAN
13
Access between containers on same Pod
OpenShift Node
vethXXX
lbrØ Plain ContainerA
ethØ
vlinuxbr
IP Stack Pod B
vovsbr lo
IP Tables tunØ Container1
vethYYY
brØ ethØ Container2
eth2 eth1 ethØ (OVS) vethZZZ
Pod C
ethØ
vxlanØ
Default LAN
Admin LAN
Storage LAN
Default LAN
Admin LAN
Storage LAN
Default LAN
Admin LAN
Storage LAN
Pod to Container : Pod B eth0 vethYYY (OVS) br0 vovsbr vlinuxbr lbr0 vethXXX Container A eth0 16
Container to Pod : Container A eth0 vethXXX lbr0 vlinuxbr vovsbr (OVS) br0 vethYYY Pod B eth0
Access Pod from external
OpenShift Node
vethXXX
lbrØ Plain ContainerA
ethØ
vlinuxbr
IP Stack vovsbr
IP Tables tunØ vethYYY
brØ Pod B
ethØ
eth2 eth1 ethØ (OVS) vethZZZ
Pod C
ethØ
vxlanØ
Default LAN
Admin LAN
Storage LAN
Outbound : Pod B eth0 vethYYY (OVS) br0 tun0 (IPTables NAT) host network 17
Inbound : host network IPTables DNAT tun0 (OVS) br0 vethYYY Pod B eth0
Host network
18
Host network(Cont’d)
19
lbr0
Even though a number of pods (resulting in about 20 containers) are running on the node,
no vethXXXX device is connected to the lbr0
20
ovs-vsctl
21
Pod/Container Networking across OpenShift Nodes
OpenShift Node 1 OpenShift Node 2
vethXXX Plain vethXXX Plain
lbrØ lbrØ
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
IP Stack vovsbr IP Stack vovsbr
IP Tables tunØ vethYYY IP Tables tunØ vethYYY
Pod B Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Pod Z
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
22
Pod Networking across OpenShift Nodes
OpenShift Node 1 OpenShift Node 2
vethXXX Plain vethXXX Plain
lbrØ lbrØ
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
IP Stack vovsbr IP Stack vovsbr
IP Tables tunØ vethYYY IP Tables tunØ vethYYY
Pod B Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Pod Z
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
Pod B eth0 vethYYY (OVS) br0 vxlan0 (L3 encapsulation) (tunnel via host network) 23
vxlan0 (L3 deencapsulation) (OVS) br0 vethZZZ Pod Z eth0
Pod and Container Networking across OpenShift Nodes
This capability does not extend to plain docker containers, i.e. they cannot communicate
with either pods or other plain docker containers on another node.
This means plain docker containers are limited to communicate with other containers
and pods running on the same node as well as the any host connected to the physical
network(s).
24
Pod Networking by Service
OpenShift Node 1 OpenShift Node 2
kube- vethXXX kube- vethXXX
lbrØ Plain lbrØ Plain
proxy proxy
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
vovsbr vovsbr
IP Stack tunØ vethYYY IP Stack tunØ vethYYY
IP Tables Pod B IP Tables Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Pod Z
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
25
Pod Networking on same OpenShift Node by Service
OpenShift Node 1 OpenShift Node 2
kube- vethXXX kube- vethXXX
lbrØ Plain lbrØ Plain
proxy proxy
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
vovsbr vovsbr
IP Stack tunØ vethYYY IP Stack tunØ vethYYY
IP Tables Pod B IP Tables Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Pod Z
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
Pod B eth0 vethYYY (OVS) br0 tun0 IPTables NAT kube-proxy tun0 (OVS) br0 26
vethZZZ Pod C eth0
Pod Networking across OpenShift Nodes by Service
OpenShift Node 1 OpenShift Node 2
kube- vethXXX kube- vethXXX
lbrØ Plain lbrØ Plain
proxy proxy
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
vovsbr vovsbr
IP Stack tunØ vethYYY IP Stack tunØ vethYYY
IP Tables Pod B IP Tables Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Pod Z
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
Pod B eth0 vethYYY (OVS) br0 tun0 IPTables NAT kube-proxy tun0 (OVS) br0
vxlan0 (L3 encapsulation) (tunnel via host network) vxlan0 (L3 deencapsulation) (OVS) br0 27
vethZZZ Pod Z eth0
OpenShift Router Networking
OpenShift Node 1 OpenShift Node 2
kube- vethXXX kube- vethXXX
lbrØ Plain lbrØ Plain
proxy proxy
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
vovsbr vovsbr
IP Stack tunØ vethYYY IP Stack tunØ vethYYY
IP Tables Pod B IP Tables Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Router Pod
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
28
Router and Pod networking on same node
OpenShift Node 1 OpenShift Node 2
kube- vethXXX kube- vethXXX
lbrØ Plain lbrØ Plain
proxy proxy
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
vovsbr vovsbr
IP Stack tunØ vethYYY IP Stack tunØ vethYYY
IP Tables Pod B IP Tables Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Router Pod
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
External client network Node2 eth0 (IPTables DNAT) tun 0 (OVS) br0 vethZZZ
Router Pod eth0 (userspace router) Router Pod eth0 vethZZZ (OVS) br0 vethYYY 29
Pod Y eth0
Router and Pod networking across nodes
OpenShift Node 1 OpenShift Node 2
kube- vethXXX kube- vethXXX
lbrØ Plain lbrØ Plain
proxy proxy
ethØ ContainerA ethØ ContainerX
vlinuxbr vlinuxbr
vovsbr vovsbr
IP Stack tunØ vethYYY IP Stack tunØ vethYYY
IP Tables Pod B IP Tables Pod Y
brØ ethØ brØ ethØ
(OVS) (OVS)
eth2 eth1 ethØ vethZZZ eth2 eth1 ethØ vethZZZ
Pod C Router Pod
ethØ ethØ
vxlanØ vxlanØ
Default LAN
Admin LAN
Storage LAN
External client network Node2 eth0 (IPTables DNAT) tun 0 (OVS) br0 vethZZZ Router
Pod eth0 (userspace router) Router Pod eth0 vethZZZ (OVS) br0 vxlan0 (L3 encapsulation) 30
(tunnel via host network) vxlan0 (L3 deencapsulation) (OVS) br0 vethYYY Pod B eth0
EGRESS Pod
OpenShift Node
vethXXX
lbrØ Plain ContainerA
ethØ
vlinuxbr
IP Stack vovsbr
IP Tables tunØ vethYYY
brØ Pod B
ethØ
eth2 eth1 ethØ (OVS) vethZZZ
Egress Pod
ethØ
vxlanØ
macvlanØ
Default LAN
Admin LAN
Storage LAN
31
EGRESS Traffic
OpenShift Node
vethXXX
lbrØ Plain ContainerA
ethØ
vlinuxbr
IP Stack vovsbr
IP Tables tunØ vethYYY
brØ Pod B
ethØ
eth2 eth1 ethØ (OVS) vethZZZ
Egress Pod
ethØ
vxlanØ
macvlanØ
Default LAN
Admin LAN
Storage LAN
32
+OpenShift facebook.com/openshift
rhopenshift @OpenShift
33