How do I use the kubernetes API in the go language?

Alena Prokharchyk, Chief software engineer at Rancher Labs, was invited to give a presentation at the Kubernetes + Kubecon 2017 North American Summit, hosted by CNCF in December 2017 6-8th. This article is organized by the content of the lecture.

As Kubernetes is becoming more and more popular, the number of integration and monitoring services around it is growing. The key component of all such services written by Golang is kubernetes/client-go--a package for communicating with the Kubernetes Cluster API. In this article, we'll discuss the basics of using client-go, and how to save developers the time it takes to write actual application logic. We'll also show you the best practices for using the package and share our experience from the perspective of a developer who integrates with kubernetes every day. The content will include:

client Authentication vs. out-of-cluster client authentication in a cluster

basic list, using Client-go to create and delete Kubernetes object operations

How to use Listwatch and informers to monitor and react to k8s events

How to manage software package dependencies

Kubernetes is a platform

Kubernetes has a lot of popular places. Users like its rich functionality, stability, and performance. For contributors, the Kubernetes open source community is not only large, but also easy to get started and quick to respond to. What really makes kubernetes attractive to third-party developers is its extensibility. The project provides a number of ways to add new features, extend existing functionality, and not disrupt the main code base. It is these that make kubernetes become a platform.

Here are some ways to extend kubernetes:

, you can see that each kubernetes cluster component, whether it's a kubelet or an API server, can be scaled in some way. Today we will focus on a "custom controller" approach, from now on I call it a Kubernetes controller (Kubernetes), or simply called a controller.

Kubernetes What exactly is a controller? ?

The most common definition of a controller is the code that causes the current state of the system to reach the desired state. But what exactly does that mean? We take the ingress controller as an example. Ingress is a kubernetes resource that is able to define external access to services in the cluster. HTTP is usually used and load balancing is supported. However, there is no ingress implementation in Kubernetes's core code. The implementation of the third-party controller will contain:

1. Monitoring events for Ingress/services/endpoint resources (create, update, delete)

2. load balancer inside or outside the program

3. Update ingress with the address of the load balancer

The "desired state" here in ingress refers to the IP address pointing to the running load balancer, which is implemented by the user according to the rules defined by the Ingress specification. and the external Ingress Controller is responsible for transferring the ingress resources to this state.

The implementation of the Controller and the manner in which they are deployed may vary for the same resources. You can select the Nginx controller and deploy it to each node in the cluster as a daemon set (Daemon set), or you can choose to run the Ingress controller outside the Kubernetes cluster and F5 programming as a load balancer. There are no strict rules here, and kubernetes is so flexible.

Client-go

Here are a few ways to get information about kubernetes clusters and their resources, and you can use Dashboard, kubectl, or programmatic access to the Kubernetes API. Client-go all the most widely used libraries in the go language, as well as many other language versions (Java, Python, etc.). If you haven't written your own controller yet, I recommend you try Go/client-go first. Kubernetes was written with go, and I found it more convenient to develop plugins in the same language as the main project.

Let's Build it ...

To be familiar with the relevant platforms and tools, the best way is to practice, to achieve something. We start with a simple, first implementation of the following controller:

1. monitoring Kubernetes nodes

2. alerts when a mirror on a node occupies storage space, and can change

This part of the implementation, source code can be found here : https://github.com/alena1108/kubecon2017

Basic process

Configuration Items

As a developer, my colleagues and rancher labs are more willing to use lightweight and easy tools, where I will share 3 of my favorite tools that will help us complete our first project.

1. Go-skel–go Language micro-service skeleton, just execute run./skel.sh test123, it will create test123 for the new Go Project skeleton.

2. Vendor management tools for Trash–go languages. There are actually a lot of dependency management tools, but in terms of temporary dependency management, trash is very good and simple to use.

3. dapper– A tool for encapsulating any existing build tools in a consistent environment

Add Client-go as a dependency

To make it easier to use Client-go's code, we have to set it as a dependency on the project. Add it to the vendor.conf file:

Then run the trash. It pulls all dependencies defined in vendor.conf to the vendor folder of the project. Here you need to make sure that the Client-go is compatible with the Kubernetes version of your cluster.

Create a client

Before creating a client that communicates with the Kubernetes API, we must first decide how to run our tool: inside or outside the Kubetnetes cluster. When the application is running inside the cluster, it is containerized and deployed as a kubernetes pod. It also provides some additional functionality: You can choose how to deploy it (Deamon set runs on each node, or as a deployment of n replicas), configure health checks for it, and so on. When an application runs outside the cluster, it needs to manage it itself. The following configuration can make our tools more flexible and support two ways of defining clients based on Config flag:

We'll use the way the cluster runs outside when we debug the application, so you don't have to build the mirror every time and redeploy it to the Kubernetes pod. After testing the application, we can build the image and deploy it to the cluster.

As you can see in, the configuration is being built and passed to kubernetes. Newforconfig to generate the client.

Use the basic Cruds

Our tools need to monitor the nodes. Before implementing the logic process, let's familiarize ourselves with using client-go to perform CRUD operations:

The above shows:

1. list node Minikube, is implemented by Fieldselector filter

2. update nodes with new annotations

3. Delete the node using the graceperios=10 sec instruction-meaning that the delete operation will not be performed for 10 seconds after the command is executed

All of the above steps are done using the user set (Clientset) that we created earlier.

We also need information about the image of the node, which can be retrieved by accessing the appropriate fields:

Use Informer for monitoring/notification

Now we know how to get the nodes from the Kubernetes APIs and get the image information from them. So how do we monitor the change in the size of the mirror? The simplest approach is to periodically poll the nodes, calculate the current mirrored storage capacity, and compare them to the results of the previous poll. The disadvantage here is that, regardless of whether the node has changed, the list calls we perform will fetch all the nodes, which can be very resource-especially when the polling interval is short. What we really want to achieve is to be notified when a node changes, and only then do our logic flow. These are the Client-go informer to do.

In this example, we pass the watchlist instruction to create informer for the node object to monitor the node and set the object type to API. node and the 30-second synchronization cycle periodically poll the nodes, regardless of whether the node has changed-this is a good way to recall that the update event terminated for some reason. In the last argument, we passed 2 callback functions--handlenodeadd and Handlenodeupdate. These callback functions have the actual logic and are triggered when the mirroring on the node changes when the storage has changed. Newinformer returns 2 objects--controller and store. Once the controller is started, the monitoring of Node.update and Node.add is initiated and the callback function is called. This part of the code is stored in the memory cache, updated by informer, and you can get the node object in the cache instead of calling the Kubernetes APIs directly:

There is only one controller in our project and it is sufficient to use regular informer. However, if the future of your project eventually has multiple controllers for the same object, I recommend that you use Sharedinformer. So you don't have to informer each controller with one more, just register a shared informer and have each controller register its own set of callback functions to return the shared cache, which can reduce memory consumption:

Deployment time

It's time to deploy and test the code! For the first run, we just need to create a go binary and run it in out-of-cluster mode:

To change the message output, use a mirror to deploy a pod, which is a mirror that is not displayed on the current node.

After the basic functionality has passed the test, the next step is to try to run it in cluster mode. To do this we must first create the image and define its dockerfile:

and create a mirror using Docker build, which generates a mirror that can be used to deploy pods in kubernetes. Now your application can run as a pod on the kubernetes cluster. here is an example of a deployment definition, in which I used the example to deploy our application:

In this article we have done the following:

1. Create a Go Project

2. Add a Client-go package dependency to the project

3. Create a client for communication with the Kubernetes API

4. Define a informer that is used to monitor the change of the node object and execute the callback function once it occurs

5. Implement an actual logic in the callback function

6. Run the binaries outside the cluster to test the code and deploy it to the cluster

How do I use the kubernetes API in the go language?