Kuberntes Service Quality assurance (QoS)

"Editor's words" kubernetes as the current mainstream container cluster management platform, the need for overall co-ordination of platform resource use, fair and reasonable allocation of resources to the relevant pod containers to use, and to ensure that the container life cycle has sufficient resources to ensure its operation. At the same time, due to the exclusive nature of resources, that is, resources have been allocated to a container, the same resources are not allocated to other containers, for a relatively low resource utilization of containers, resources are not actually used (such as CPU, memory) caused a serious waste of resources, Kubernetes need to improve resource utilization from the perspective of priority and fairness. In order to realize the resource being efficiently dispatched and allocated while increasing the resource utilization, kubernetes the service quality of pod through QoS to the expectation of different service qualities, and provided a requests and limits two kinds of resource allocation and use restrictions. For a pod, the quality of service is embodied in two 2 specific metrics: CPU and memory. In practice, when memory resources are tight on node nodes, Kubernetes is processed according to the different QoS classes that are set up beforehand. Set the reason for resource restrictions

If you do not have a node nodeselector, affinity (node affinity) or pod affinity, anti affinity (pod affinity/anti-affinity) and other pod advanced scheduling policy settings, we have no way to specify the service deployment to the specified machine, This can result in resource competition due to the simultaneous allocation of the CPU or memory-intensive pod to the same node. On the other hand, if resources are not restricted, some key services may be killed due to resource competition for Oom, or limited CPU usage. Resource requirements (Requests) and limitations (Limits)

For each resource, container can specify the specific resource requirements (requests) and restrictions (limits), requests the maximum configuration of 0 to node nodes, and the limits application range is requests to infinity, or 0 <= Requests <=node allocatable <= limits <=, requests.
For CPUs, if the service in the pod uses the CPU over the setting, the limits,pod is not killed but is limited. If Limits,pod is not set, all idle CPU resources can be used.
For memory, a process in which a pod uses more than the set of memory in a limits,pod container is kernel killed by Oom. When container because Oom is killed, the system tends to reboot the container or native or another pod on its original machine. QoS classification

Kubelet provides QoS service quality management and supports system-level Oom control. In Kubernetes, the QoS level of the pod: guaranteed, burstable, and Best-effort. The following is a description of each level:
All containers in guaranteed:pod must be set limits uniformly and set with the same parameters, and if there is a container to set requests, all containers are set and the parameters are set to match the limits. Then the QoS of this pod is the guaranteed level.
Note: If a container only indicates limit and the request is not set, the value of the request is equal to the limit value.
Guaranteed Example 1: The container only indicates the limits and does not indicate the requests).

Containers:
name:foo Resources
:
limits:
cpu:10m
memory:1gi
name:bar resources:
   limits:
cpu:100m
memory:100mi

Guaranteed examples 2:requests and limit are both specified and have equal values.

Containers:
name:foo Resources
:
limits:
cpu:10m
memory:1gi
requests:
cpu:10m
memory:1gi

name:bar Resources
:
limits:
cpu:100m
memory:100mi
   requests:
cpu:100m
memory:100mi

The QoS of the pod is burstable as long as the requests and limits settings of a container are not the same in burstable:pod. Examples are as follows:
Container Bar does not specify resources

Containers:
name:foo Resources
:
limits:
cpu:10m memory:1gi requests
:
cpu:1 0m
Memory:1gi

Name:bar

burstable Example 2: For container foo and bar different resources (foo is memory, and bar is CPU) set limits.

Containers:
name:foo:
limits:
memory:1gi name:bar Resources
:
limit S:
cpu:100m

burstable Example 3:container foo has no limits set, and bar requests and limits are not set.

Containers:
name:foo Resources
:
requests:
cpu:10m
memory:1gi

Name:bar

Best-effort: If requests and limits are not set for all resources, the pod's QoS is best-effort. Examples are as follows:

Containers:
name:foo resources:
Name:bar
:

Compressible Resources and Incompressible resources

Kubernetes according to the scale of resources can be classified into compressible resources and can not be compressed resources 2 kinds. CPU resources are currently supported as a compressible resource, while memory resources and disk resources are currently supported uncompressed resources. QoS Priority

3 QoS priorities from low to High (left-to-right):

Best-effort pods-> burstable pods-> guaranteed pods

Static pod

There is a daemonset type pod in the Kubernetes, a pod that can run on a node for a long time, managed directly by the Kubelet service on the node, without the need for API server intervention, and static pod does not need to correlate any RC, is completely monitored by the Kubelet service, and Kubelet will restart static pod when Kubelet finds static pod stops. Resource Recovery policy

When the available resources on a node in the Kubernetes cluster are less than an hour, Kubernetes provides a resource recovery policy to ensure that nodes operate properly in this pod. When memory or CPU resources are exhausted on a node, the pod service that is scheduled to run on the node may be unstable. Kubernetes through the Kubelet to control the recovery strategy to ensure that the node pod in the node resources can be stable operation.

Compressible Resource CPU: In the Compressed Resources section, the CPU is referred to as a compressible resource, and when the pod uses the limits value of the set, the process using the CPU in the pod will be limited, but not kill.

Uncompressed resource memory: When node memory resources are low, there will be processes because Oom will be killed by kernel, 3 kinds of QoS pods are killed in the order and the scene is as follows: Best-effort type of pods: When the system ran out of all memory, The type pods will be killed first. Burstable type pods: When the system has run out of all memory and there is no best-effort container can be killed, the type pods will be killed. Guaranteed pods: The system ran out of all memory and no burstable and Best-effort container could be killed, and pods of that type would be killed.

Note: If the pod process is using more than pre-set limites rather than node resource constraints, the system tends to reboot the container or native or other to recreate a pod on its original machine. Use recommendations if you have sufficient resources, you can set the QoS pods type to guaranteed. Use computational resources to change business performance and stability, reducing the time and cost of troubleshooting. If you want to improve resource utilization better, business services can be set to guaranteed, while other services can be set to burstable or Best-effort, for example, Filebeat, depending on the degree of importance. Known Issues

Swap reference is not supported

Resource Quality of Service in kubernetes:https://github.com/kubernetes/... os.md
Introduction to Quality of service and service level Agreement:http://www.loriotpro.com/produ ... N.htm
Kubelet pod level resource management:https://github.com/kubernetes/... nt.md

Kubelet–eviction policy:https://github.com/kubernetes/... on.md

Author: freewheel Lead Engineer Zhangxia

Kuberntes Service Quality assurance (QoS)