This is a creation in
Article, where the information may have evolved or changed.
"Editor's note" Flannel is an overlay network (Overlay) tool designed by the CoreOS team for Kubernetes to help each kuberentes host with CoreOS have a complete subnet. This sharing will be introduced from the introduction of flannel, working principle and installation and configuration of three aspects to introduce the use of this tool.
The first part: Flannel introduction
Flannel is a network planning service designed by the CoreOS team for kubernetes, which simply means that the Docker container created by the different node hosts in the cluster has a unique virtual IP address for the complete cluster.
In the Kubernetes network model, it is assumed that each physical node should have a "dedicated subnet IP" belonging to the same intranet IP segment. For example:
Node a:10.0.1.0/24
Node b:10.0.2.0/24
Node c:10.0.3.0/24
In the default Docker configuration, however, the Docker service on each node is responsible for the IP assignment of the node container. One problem with this is that containers on different nodes may get the same internal and external IP addresses. and enables these containers to be able to find each other through the IP address, that is, ping each other.
Flannel is designed to re-plan the use of IP addresses for all nodes in the cluster, allowing containers on different nodes to have "one intranet" and "non-duplicated" IP addresses, and allow containers belonging to different nodes to communicate directly through the intranet IP.
Part II: How the Flannel Works
Flannel is essentially an "overlay network", which means that TCP data is packaged in another network packet for routing and forwarding and communication, and data forwarding such as UDP, VxLAN, AWS VPC, and GCE routing is now supported.
The default inter-node data communication mode is UDP forwarding, in the Flannel GitHub page as a schematic diagram:
This picture is full of information, the following simple interpretation.
After the data is emitted from the source container, it is forwarded to the FLANNEL0 virtual network card via the DOCKER0 virtual network card of the host, which is a peer virtual network card, and the Flanneld service listens on the other end of the network card.
Flannel maintains a routing table between nodes through the ETCD service, which we'll cover in a later configuration section.
The Flanneld service of the source host will have the original data content UDP encapsulated in accordance with its own routing table to the destination node's Flanneld service, the data arrives after being unpacked, and then directly into the destination node of the FLANNEL0 virtual network card, It is then forwarded to the destination host's Docker0 virtual network card, and finally, as the native container communicates, there is a DOCKER0 route to reach the target container.
This completes the delivery of the entire packet, and there are three questions to explain.
The first question, what is UDP encapsulation?
Let's take a look at the following figure, which is the ping command traffic packet that is fetched on one of the communication nodes. You can see that the data content portion of UDP is actually a packet of another ICMP (that is, the ping command).
The raw data is UDP encapsulated on the flannel service of the starting node, and the flannel service at the other end is restored to the original packet after it is posted to the destination node, and the Docker service on both sides does not feel the existence of the process.
second question, why does Docker on each node use a different IP address segment?
This thing looks strange, but the truth is very simple. In fact, simply because flannel through the ETCD assigned to each node available IP address segment, secretly modified the Docker startup parameters, see.
This is the Docker service process running parameter that is seen on the node running the Flannel service.
Note the "--bip=172.17.18.1/24" parameter, which restricts the IP range obtained by the node container in which it is located.
This IP range is automatically assigned by flannel and is ensured by flannel by the records stored in the ETCD service to ensure that they are not duplicated.
The third question is why the data on the sending node is routed from Docker0 to flannel0 virtual network card, and the destination node is routed from Flannel0 to DOCKER0 virtual network card?
Let's take a look at the routing table on the node where flannel is installed. The following is the routing table for the data sending node:
This is the routing table for the data receiving node:
For example, there is now a packet to be sent from a container with IP 172.17.18.2 to a container with IP 172.17.46.2. Based on the routing table of the data sending node, it matches the record only with 172.17.0.0/16, so the data is posted to Flannel0 after it is docker0 out. Similarly in the target node, because the address of the delivery is a container, so the destination address must fall in DOCKER0 for the 172.17.46.0/24 this record, the natural is delivered to the DOCKER0 network card.
Part III: Installation and configuration of the flannel
Flannel is a program written by Golang, so the installation is simple.
Download the latest version of the binary package for flannel and ETCD, respectively, from Https://github.com/coreos/flannel/releases and https://github.com/coreos/etcd/releases.
After decompression, the flannel binaries "Flanneld" and the script file "mk-docker-opts.sh", as well as the ETCD binaries "Etcd" and "Etcdctl" are placed under the system's path directory, even if the installation is complete.
The configuration section is a bit more complicated.
Start Etcd First, refer to Https://github.com/coreos/etcd ... overy.
Visit this address: https://discovery.etcd.io/new?size=3 get a "discovery address"
Run the following startup commands on each node:
Etcd-initial-advertise-peer-urls http://< Current node Ip>:2380-listen-peer-urls http://< current node ip>:2380- Listen-client-urls http://< Current node ip>:2379,http://< current node Ip>:2379-advertise-client-urls http://< current node IP >:2379-discovery < just got discovery address > &
After you start ETCD, you can configure the flannel.
Flannel configuration information is all recorded in the ETCD, to ETCD write the following the simplest configuration, only specify the flannel can be used to assign to each Docker node of the proposed IP address segment:
Etcdctl set/coreos.com/network/config ' {"Network": "172.17.0.0/16"} '
Then start flannel on each node separately:
Flanneld &
Finally, you need to give Docker a little bit, modify its startup parameters and Docker0 address.
Execute on each node:
sudo mk-docker-opts.sh-i
Source/run/flannel/subnet.env
sudo rm/var/run/docker.pid
sudo ifconfig Docker0 ${flannel_subnet}
Reboot the Docker once so the configuration is complete.
Now a Docker container is started on two nodes, each of which has been directly pinging each other through the IP address.
This is where the entire flannel cluster is functioning.
Finally, it is mentioned repeatedly that flannel has a routing table stored in ETCD that can be found in ETCD data, such as.
Q&a
Q: After the data from the source container issued, through the host's Docker0 virtual network card forwarding to the FLANNEL0 virtual network card, this peer-to-real production in the presence of packet loss, or the mechanism is highly available to protect it?
A: Just the peer network card, not through external networks, it should be relatively stable. But I don't have any specific data here.
Q: UDP data encapsulation, the form of forwarding is also UDP? We generally know that UDP send data is stateless, reliable?
A: Forwarding is UDP, high concurrency data flow may be a problem, I also have no data here.
Q: In fact, Kubernates is to dilute the container IP, the peripheral users only need to focus on the services invoked, do not care about the specific IP, here fannel IP separate and unique, what is the benefit of doing this? Is there a business scenario that is actually applied?
A: IP is the only kubernetes can be set up one of the conditions, do not put the network pull-through behind the things are not good whole.
Q: Flannel has secretly modified the Docker startup parameters after assigning the IP address segments available to each node through ETCD: So if you add nodes, or delete nodes, will these address segments (ETCD) change dynamically? If it is not dynamic change, will cause the waste of IP address?
The answer will cause some waste, generally using 10.x.x.x IP segment.
Q: What exactly does sudo mk-docker-opts.sh-i do with this command? What is the difference between using flannel on non-CoreOS?
A: A docker-initiated environment variable file was generated, which added the boot parameters to Docker.
Nothing different, just coreos integrated flannel, start flannel on CoreOS is just one line of command: Systemctl start Flanneld.
Q: Are the container IPs fixed? Can the external network and the physical host ping through and ping all the container IPs of the Docker cluster?
A: Not fixed, IP assignment or docker doing, flannel just assigned subnets.
Q: Can I implement a VPN for flannel? Have you ever studied?
A: It should not, it requires that these containers are already in an intranet.
Q: Who developed the FLANNL? Is it all about the development of k8s two times?
A: CoreOS company, not k8s two times development, independent open source project, to k8s provide the basic network environment.
Q: Does the flannel support pure forwarding for non-packets? So there's no loss of performance?
A: How does the non-encapsulation route? The TCP packet emitted does not have the information that is routed between the networks, and remember that two flannel are not directly connected and are separated by a common local area network.
Q: Which version is Flanel now, and what is the focus of the subsequent version? Performance optimizations, or feature extensions?
A: It's not yet 1.0, and on GitHub there's a big part of their development plan.
Q: Is the customer still required to install flannel in CoreOS?
A: Do not need, in the boot cloudinit configuration to Etcd write flannel configuration, and then add Flanneld.service command:start on it, start directly available, document Connection I did not find, there is this configuration, ready-made.
Q: Can i specify the IP range of each host directly with a command, and then do the GRE tunnel to communicate between nodes? This can also be implemented on different hosts of the container IP is different and can communicate with each other?
A: It is not supported to specify which node to use the IP, but seemingly can be changed in the Etcd hand.
Q: Flannel is only responsible for the communication service, is that also to install k8s?
Answer: Yes, k8s is separate.
Q: What else can I choose or recommend for the network components of Docker?
Answer: Overlay network is commonly used is flannel and weave, other OvS and other said.
===========================
The above content is organized according to the August 25, 2015 Night Group sharing content. Share people
Linfan, ThoughtWorks Chengdu Cloud&devops Consultant, the main research content is the application of containerized and CoreOS system related fields. The author of the CoreOS Practice Guide and the CoreOS articles series. The CoreOS topic will be shared at the container technology conference on August 28. Dockone Weekly will organize the technology to share, welcome interested students add: LIYINGJIESX, into group participation, you want to listen to the topic can give us a message.
