Validating overlay network when docker swarm running on Centos VMs on vSphere

I got a chance to revisit my docker swarm deployment this week after a bit of a break. I was a little curious about my setup because when I spoke to some of our ‘Project Hatchway‘ engineers, I was told that I should be able to launch a single instance of Nginx in Docker Swarm (“docker service create –replicas 1 -p 8080:80 –name web nginx”) and I should be able to access the web service using the following command from any swarm node – “curl 127.0.0.1:8080”. This was not what I was seeing. When I launched the Nginx service, the curl command was successful on the container host where the service was running, but on every other host/node in the swarm cluster, I got a “Failed connect/connection refused”. So why wasn’t it working?

Eventually I traced it to yet another firewall issue on the container hosts/swarm nodes (using Centos 7). It seems that the overlay network needed some ports opened to work as well. These are the ports that I figured out needed to be opened on the firewall of my swarm nodes:

  • 7946/tcp – port for “control plane” discovery communication
  • 7946/udp – port for “control plane” discovery communication
  • 4789/udp – port for “data plane” overlay network traffic

I used the following command on Centos 7 to modify the firewall:

[root@centos-swarm-master ~]# firewall-cmd --zone=public --add-port=7946/tcp --permanent
[root@centos-swarm-master ~]# firewall-cmd --zone=public --add-port=7946/udp --permanent
[root@centos-swarm-master ~]# firewall-cmd --zone=public --add-port=4789/udp --permanent
[root@centos-swarm-master ~]# firewall-cmd --reload

To verify that the changes took place, I used the following command:

[root@centos-swarm-master ~]# firewall-cmd --list-all
public (active)
 target: default
 icmp-block-inversion: no
 interfaces: ens192
 sources:
 services: dhcpv6-client ssh
 ports: 2379/tcp 4789/udp 2377/tcp 7946/udp 7946/tcp 2380/tcp
 protocols:
 masquerade: no
 forward-ports:
 sourceports:
 icmp-blocks:
 rich rules:

The other ports related to Swarm, which is discussed here , and ETCD, which is for vFile (which I haven’t yet blogged about – watch this space). With these ports opened, we have allowed our docker overlay network to communicate between Swarm nodes. Now if I launch a single replica for the Nginx web service and retry the curl test on different nodes, lets see what happens:

[root@centos-swarm-master ~]# docker service ls
 ID           NAME                 MODE       REPLICAS IMAGE                PORTS
 rxspku5i98cc vFileServerSharedVol replicated 1/1      luomiao/samba-debian *:30000->445/tcp

[root@centos-swarm-master ~]# docker service create --replicas 1 -p 8080:80 --name web nginx
 xvtzr79sb0fdut85yssxd7z1n
 overall progress: 1 out of 1 tasks
 1/1: running [==================================================>]
 verify: Service converged
 
[root@centos-swarm-master ~]# docker service ls
 ID           NAME                 MODE       REPLICAS IMAGE                PORTS
 rxspku5i98cc vFileServerSharedVol replicated 1/1      luomiao/samba-debian *:30000->445/tcp
 xvtzr79sb0fd web                  replicated 1/1      nginx:latest         *:8080->80/tcp

[root@centos-swarm-master ~]# curl 127.0.0.1:8080
 
 
 
 Welcome to nginx!
 
  body {
  width: 35em;
  margin: 0 auto;
  font-family: Tahoma, Verdana, Arial, sans-serif;
  }
 
 
 
 

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

[root@centos-swarm-master ~]#

Let’s switch to a worker node, and retry the same test.

[root@centos-swarm-w1 ~]# curl 127.0.0.1:8080



Welcome to nginx!

 body {
 width: 35em;
 margin: 0 auto;
 font-family: Tahoma, Verdana, Arial, sans-serif;
 }



Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

[root@centos-swarm-w1 ~]#

Success! Now that my overlay network is working successfully, I can reach a single instance of a service working on docker swarm from any of the nodes in the cluster.

责编内容来自:cormachogan (源链) | 更多关于

阅读提示:酷辣虫无法对本内容的真实性提供任何保证,请自行验证并承担相关的风险与后果!
本站遵循[CC BY-NC-SA 4.0]。如您有版权、意见投诉等问题,请通过eMail联系我们处理。
酷辣虫 » 后端存储 » Validating overlay network when docker swarm running on Centos VMs on vSphere

喜欢 (0)or分享给?

专业 x 专注 x 聚合 x 分享 CC BY-NC-SA 4.0

使用声明 | 英豪名录