Kubernetes Cluster management platform based on Python+django

the original is from the "Listen to Cloud Technology blog": http://blog.tingyun.com/web/article/detail/1345

Today, contact Kubernetes also has a period of time, and most of our business has been stably run in different sizes of kubernetes clusters, we have to say, whether from the application deployment, iteration, or resource scheduling management and other aspects have their unspeakable advantages, But as business continues to grow and services diversify, the volume and management of containers increases.

Some pain points in daily management and maintenance of kubernetes cluster:

1. Relatively large size of the cluster and the number of container maintenance management.

The business scenario of our company is typical of multi-line parallel operation. At the same time, in order to facilitate classification management, avoid port conflicts and rational use of resources. We have also adopted a number of strategies, such as:

Label Labels: by label, on the one hand can identify which product line which application is located on which node, perhaps someone would like to why to do so, suppose you have a data on the disk application and the application always change with the start to change is not fun. On the one hand through the label can be balanced equipment load, such as the CPU and comparative consumption of memory together, not only the full utilization of resources and also effectively prevent the same type (such as high consumption of CPU) accidentally ran a node on the cause of resource scramble and port conflict.

So the question is, how to get an OPS person to face the vast number of tags and maintain management (Kubectl get node–show-labels? , and how to let an operation and maintenance personnel, failure occurs, the face of the vast number of nodes/pods, immediately and quickly locate the correspondence between the two, thus solving the problem.

2. Test environment Maintenance management issues.

General application deployment and on-line process more cumbersome

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133553776.jpg&filename=1.jpg "title=" Listen to Cloud Blog reference image "alt=" 1.jpg "style=" border:0px; Vertical-align:middle; "/>

This mode allows each developer to debug the beta environment, whether it is changing the configuration or the code update will need to communicate operations personnel to operate, so that each operator to use more energy to maintain a set of additional systems and even more environment, daily walk in beta, online between. It's a bit of a headache.

Would prefer to have such a pattern

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133630650.jpg&filename=2.jpg "title=" Listen to Cloud Blog reference image "alt=" 2.jpg "style=" border:0px; Vertical-align:middle; "/>

This greatly reduces the communication costs between departments. But the question is, how can a researcher be able to independently develop and maintain its own beta environment, without needing to worry too much about something other than code debugging? (like how to write a YAML or JSON based on the Kubernetes service)

Take this, then sprouted out an attempt to write a management service idea, the purpose is to let operators more convenient management of their own kubernetes line cluster, so that developers can also independently write and maintain their own test environment applications, the initial stage, for reference only, if there are shortcomings, We welcome your valuable comments at any time.

Python Admin (Beta) is an operations management system based on the Python+django and Kubernetes APIs. The front end uses open source SB (start bootstrap) Admin-2 template (fresh, simple).

1. Version information:

python2.7.5+django1.8.13+kubernetes1.2.4+docker1.10.3

2.Kubernetes API Related:

Create and update a label

Curl-x patch-i-H "Content-type:application/merge-patch+json" http://k8smaster:8080/api/v1/nodes/{nodename}-d ' {"Me Tadata ": {" labels ": {" tag ":" Application "}} '

Create Configmap

Curl-x post-i-H "Content-type:application/json" http://k8smaster:8080/api/v1/namespaces/default/configmaps/-D "$ ( Cat Configmaptest.json) "

Update Configmap

Curl-x patch-i-H "Content-type:application/merge-patch+json" http://k8smaster:8080/api/v1/namespaces/default/ configmaps/{Configmapname}-D "$ (cat Configmapupdate.json)"

Delete Configmap

Curl-x DELETE http://k8smaster:8080/api/v1/namespaces/default/configmaps/{Configmapname}

Basic JSON template for Configmap

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133710403.jpg&filename=3.jpg "title=" Listen to Cloud Blog reference image "alt=" 3.jpg "style=" border:0px; Vertical-align:middle; "/>

Create Daemonset

Curl-x post-i–h "Content-type:application/json" http://k8smaster:8080/apis/extensions/v1beta1/namespaces/default/ Daemonsets-d "$ (cat Daemonset.json)"

Update Daemonset

Curl-x patch-i-H "Content-type:application/merge-patch+json" http://k8smaster:8080/apis/extensions/v1beta1/ Namespaces/default/daemonsets/{daemonsetname}-D "$ (cat Daemonsetupdate.json)"

Delete Daemonset

Curl-x DELETE Http://k8smaster:8080/apis/extensions/v1beta1/namespaces/default/daemonsets/{daemonsetname}

Daemonset Basic JSON template

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133737173.jpg&filename=4.jpg "title=" Listen to Cloud Blog reference image "alt=" 4.jpg "style=" border:0px; Vertical-align:middle; "/>

The above is listed as part of the API operation, other related operations please refer to Kubernetes official documentation

http://kubernetes.io/docs/api-reference/v1/operations/

3. Platform Operating Interface Overview

1..Kubernets Cluster Resource management interface (clear display of cluster resource information and affiliated project group for easy Classification Management)

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133800050.jpg&filename=5.jpg "title=" Listen to Cloud Blog reference image "alt=" 5.jpg "style=" border:0px; Vertical-align:middle; "/>

2. Project Application Configuration Management interface (configuration files are managed separately, using database to store configuration file contents. Re-reload when creating and updating Configmap, and synchronizing profile usage status in real time. ）

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133826377.jpg&filename=6.jpg "title=" Listen to Cloud Blog reference image "alt=" 6.jpg "style=" border:0px; Vertical-align:middle; "/>

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133843546.jpg&filename=7.jpg "title=" Listen to Cloud Blog reference image "alt=" 7.jpg "style=" border:0px; Vertical-align:middle; "/>

3. Service Deployment and Management interface (application template creation, add System log function, record the execution of each stage after service start, convenient error tracking, have certain operation audit function)

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133901713.jpg&filename=8.jpg "title=" Listen to Cloud Blog reference image "alt=" 8.jpg "style=" border:0px; Vertical-align:middle; "/>

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133913237.jpg&filename=9.jpg "title=" Listen to Cloud Blog reference image "alt=" 9.jpg "style=" border:0px; Vertical-align:middle; "/>

4.Kubernetes Container Resource Management interface (all node for each cluster, and all pods information for each node, and WebSocket exec to get inside the container to avoid improper permission control issues)

650) this.width=650; "Src=" http://blog.tingyun.com/dynamic/transitionResourcePath?key=image/ Forumimage20161129133924383.jpg&filename=10.jpg "title=" Listen to Cloud Blog reference image "alt=" 10.jpg "class=" Last "style=" border:0 Px;vertical-align:middle; "/>

If you do not confirm that the service will start properly, you can go inside the container by using debug mode (command: ["Sleep", "long enough"]) after the container is established./run.sh Adjustment service, after the problem, then the normal mode starts.

Some small ideas for future optimizations:

1.kubernets Cluster One-click Deployment, node resources immediately join.

2. Monitoring, on the basis of system level monitoring, increase the container service level monitoring and corresponding alarm strategy.

3. Integration into the Jenkins interface, let the service deployment and update, more simple and transparent.

This article is from the "Listen to Cloud Technology blog" blog, make sure to keep this source http://tingyunblog.blog.51cto.com/12385276/1881250

Kubernetes Cluster management platform based on Python+django