Kubernetes Cluster Log Management

Kubernetes has developed a Elasticsearch add-on to enable log management of the cluster. This is a combination of Elasticsearch, FLUENTD and Kibana. Elasticsearch is a search engine that is responsible for storing logs and providing query interfaces; Fluentd is responsible for collecting logs from Kubernetes and sending Elasticsearch;kibana a Web GUI that users can browse and search for stored in Logs in the Elasticsearch.

Deployment

The Elasticsearch add-on itself is run as a Kubernetes application in the cluster, and its YAML configuration file is available from the https://github.com/kubernetes/kubernetes/tree/master/ Cluster/addons/fluentd-elasticsearch get.

These YAML files can be downloaded to a local directory, for example addons , by kubectl apply -f addons/ deployment.

One thing to note here is that we will be accessing Kibana through Nodeport, and we need to comment out kibana-deployment.yaml the environment variables SERVER_BASEPATH , otherwise we won't be able to access them.

All the resources are deployed in kube-system Namespace.

Daemonset fluentd-es collects logs from each node and sends them to Elasticsearch.

Elasticsearch runs with Statefulset Resources and elasticsearch-logging provides interfaces externally through the Service. The type of Service has been kubectl edit modified to Nodeport.

You can http://192.168.56.106:32607/ verify that the Elasticsearch is working correctly.

Kibana runs as a Deployment resource, and users can kibana-logging access their Web GUI through a Service. The type of Service has been modified to nodeport.

by http://192.168.56.106:30319/ accessing Kibana.

Kibana will display the Index Pattern creation page. Direct Click Create , Kibana will automatically complete the subsequent configuration.

At this point, click on the top left corner Discover to view and retrieve the Kubernetes log.

Kubernetes Log Management system is ready, users can create their own Dashboard as needed, specific methods can refer to Kibana official documentation.

Summary

The Elasticsearch add-on itself is run as a Kubernetes application in the cluster to enable log management of the cluster. It is a combination of Elasticsearch, FLUENTD and Kibana.
Elasticsearch is a search engine that is responsible for storing logs and providing query interfaces.
FLUENTD is responsible for collecting logs from Kubernetes and sending them to Elasticsearch.
Kibana provides a Web GUI that allows users to browse and search for logs stored in Elasticsearch.

Written in the last

As a practical tutorial for Kubernetes, we have come to the end of the place.

This tutorial covers the most important technologies of Kubernetes: Cluster architecture, containerized application deployment, scale up/down, rolling update, monitoring and inspection, cluster networking, data management, monitoring, and log management, and the operation mechanism of Kubernetes is explored through a large number of experiments.

The goal of this tutorial is to enable readers to master the skills necessary to implement and manage Kubernetes, and to really use Kubernetes.

In order to achieve this goal, each chapter has designed a large number of practical operations, through and logs to help readers understand the various technical points, at the same time for the reader to practice Kubernetes to provide detailed reference.

This tutorial should have two roles for readers:

1. Beginners can systematically learn Kubernetes in the order of chapters, and Master Kubernetes's theoretical knowledge and practical skills through the experiments in the tutorials.

2. Experienced OPS personnel can use this tutorial as a reference material to view relevant knowledge points in the actual work.

It is hoped that readers will be able to lay a solid foundation through this tutorial, to be able to operate the Kubernetes calmly, and to build a practical container management platform in combination with the actual needs of the company and organization.

Finally wish everyone to use Kubernetes happy!

Books:

1. "5 minutes a day kubernetes"
Https://item.jd.com/26225745440.html

2.5 minutes of Docker container technology per day
Https://item.jd.com/16936307278.html

3.5 minutes of play with OpenStack per day
Https://item.jd.com/12086376.html

Kubernetes Cluster Log Management