Analysis of rolling update mechanism of kubernetes

Commit:d577db99873cbf04b8e17b78f17ec8f3a27eca30
Date:fri APR 10 23:45:36 2015-0700

0. Basic knowledge of command line and dependenciesSynopsis

Perform a rolling update of the given ReplicationController.Replaces the specified controller with new controller, updating one pod at a time to use thenew PodTemplate. The new-controller.json must specify the same namespace as theexisting controller and overwrite at least one (common) label in its replicaSelector.kubectl rolling-update OLD_CONTROLLER_NAME -f NEW_CONTROLLER_SPEC

Examples

// Update pods of frontend-v1 using new controller data in frontend-v2.json.$ kubectl rolling-update frontend-v1 -f frontend-v2.json// Update pods of frontend-v1 using JSON data passed into stdin.$ cat frontend-v2.json | kubectl rolling-update frontend-v1 -f -

Replicationcontroller, referred to as RC, is the kubernet system of a certain type of pod, RC has a key parameter called replicas, is also the number of pods.

So what's the use of RC? This is to solve a bunch of pods in the cluster if they hang up, then start the container on the other host and let the traffic into the pod that is properly started. That is, RC guarantees the availability of the Cluster service, and when you have many services that start up in a cluster, you need to use the program to monitor the health of these services and to dynamically ensure that the service is available.

What is the corresponding relationship between RC and pod? RC uses the selector to select some pods as his control range. As long as the label of the pod conforms to Seletor, it belongs to this RC, and the following is an example of pod and RC.

Xx-controller.json

   "spec":{      "replicas":1,      "selector":{         "name":"redis",         "role":"master"      },

Xx-pod.json

  "labels": {    "name": "redis"  },

Kubernetes is referred to as k8s, if you are interested in the basic concept of which can be seen in this article

1.kubctl Entrance

/cmd/kubectl/kubctl.go

func main() {    runtime.GOMAXPROCS(runtime.NumCPU())    cmd := cmd.NewKubectlCommand(cmdutil.NewFactory(nil), os.Stdin, os.Stdout, os.Stderr)    if err := cmd.Execute(); err != nil {        os.Exit(1)    }}

2. Actual invocation

The source code in the pkg package,/pkg/kubectl/cmd/cmd.go, each sub-command to achieve a unified interface, rollingupdate this line is:

    cmds.AddCommand(NewCmdRollingUpdate(f, out))

This function is implemented in:/pkg/kubectl/cmd/rollingupdate.go

func NewCmdRollingUpdate(f *cmdutil.Factory, out io.Writer) *cobra.Command {    cmd := &cobra.Command{        Use: "rolling-update OLD_CONTROLLER_NAME -f NEW_CONTROLLER_SPEC",        // rollingupdate is deprecated.        Aliases: []string{"rollingupdate"},        Short:   "Perform a rolling update of the given ReplicationController.",        Long:    rollingUpdate_long,        Example: rollingUpdate_example,        Run: func(cmd *cobra.Command, args []string) {            err := RunRollingUpdate(f, out, cmd, args)            cmdutil.CheckErr(err)        },    }}

You can see the execution function at the actual call. RunRollingUpdate

func RunRollingUpdate(f *cmdutil.Factory, out io.Writer, cmd *cobra.Command, args []string) error {...    mapper, typer := f.Object()    // TODO: use resource.Builder instead    obj, err := resource.NewBuilder(mapper, typer, f.ClientMapperForCommand()).        NamespaceParam(cmdNamespace).RequireNamespace().        FilenameParam(filename).        Do().        Object()    if err != nil {        return err    }    newRc, ok := obj.(*api.ReplicationController)    if !ok {        return cmdutil.UsageError(cmd, "%s does not specify a valid ReplicationController", filename)    }

This is the code that builds a new RC, where resource is the base class for all resources (POD,SERVICE,RC) of Kubneter. You can see that the new RC gets all the information from the JSON parameter file and then escapes the class as Replicationcontroller.

    if oldName == newName {        return cmdutil.UsageError(cmd, "%s cannot have the same name as the existing ReplicationController %s",            filename, oldName)    }    var hasLabel bool    for key, oldValue := range oldRc.Spec.Selector {        if newValue, ok := newRc.Spec.Selector[key]; ok && newValue != oldValue {            hasLabel = true            break        }    }    if !hasLabel {        return cmdutil.UsageError(cmd, "%s must specify a matching key with non-equal value in Selector for %s",            filename, oldName)    }

As can be seen here, for the new RC and the old RC, there are 2 restrictions, one is the new and old names need to be different, the other is the RC selector need at least one of the values are not the same.

    updater := kubectl.NewRollingUpdater(newRc.Namespace, client)    // fetch rc    oldRc, err := client.ReplicationControllers(newRc.Namespace).Get(oldName)    if err != nil {        return err    }...    err = updater.Update(out, oldRc, newRc, period, interval, timeout)    if err != nil {        return err    }

When doing rolling update, there are two conditions, one is the new RC name needs to be different from the old, the second is at least one tag value is not the same. Where namespace is k8s used to do long tenant resource isolation, can be ignored first.

3. Data structure and implementation

This code appears NewRollingUpdater , in the previous layer of the/pkg/kubectl/rollingupdate.go this file, more close to the main

// RollingUpdater provides methods for updating replicated pods in a predictable,// fault-tolerant way.type RollingUpdater struct {    // Client interface for creating and updating controllers    c client.Interface    // Namespace for resources    ns string}

The

can see that the rollingupdater inside is a K8S client structure to send commands to the API server

Func (R *rollingupdater) Update (out IO. Writer, OLDRC, NEWRC *api. Replicationcontroller, UpdatePeriod, interval, timeout time. Duration) Error {oldname: = oldRc.ObjectMeta.Name newName: = NewRc.ObjectMeta.Name Retry: = &retryparams{int Erval, timeout} waitforreplicas: = &retryparams{interval, timeout} if NewRc.Spec.Replicas <= 0 {retur N FMT. Errorf ("Invalid controller spec for%s; Required: > 0 Replicas, Actual:%s\n ", NewName, Newrc.spec)} desired: = NewRc.Spec.Replicas SourceID: = Fmt.s printf ("%s:%s", Oldname, OldRc.ObjectMeta.UID)//Look for existing NEWRC, incase this update was previously started BU T interrupted RC, existing, err: = R.GETEXISTINGNEWRC (SourceID, newName) if existing {fmt.        fprintf (out, "Continuing update with existing controller%s.\n", newName) if err! = Nil {return err } Replicas: = RC. Objectmeta.annotations[desiredreplicasannotation] desired, err = StrConv. Atoi (rePLICAS) If err! = Nil {return FMT.        Errorf ("Unable to parse annotation for%s:%s=%s", NewName, Desiredreplicasannotation, Replicas)} NEWRC = rc} else {FMT. fprintf (out, "Creating%s\n", newName) if newRc.ObjectMeta.Annotations = nil {NewRc.ObjectMeta.Annotat ions = map[string]string{}} newrc.objectmeta.annotations[desiredreplicasannotation] = Fmt.        Sprintf ("%d", desired) newrc.objectmeta.annotations[sourceidannotation] = SourceID NewRc.Spec.Replicas = 0 NEWRC, err = R.c.replicationcontrollers (R.NS). Create (NEWRC) if err! = Nil {return err}}//+1,-1 on OLDRC, NEWRC until NEWRC have desi  Red number of replicas or OLDRC have 0 replicas for NewRc.Spec.Replicas < desired && OldRc.Spec.Replicas! = 0 {NewRc.Spec.Replicas + = 1 OldRc.Spec.Replicas-= 1 fmt. Printf ("At Beginning of loop:%s Replicas:%d,%s replicas:%d\ n ", Oldname, OldRc.Spec.Replicas, NewName, NewRc.Spec.Replicas) fmt. fprintf (out, "Updating%s replicas:%d,%s replicas:%d\n", Oldname, OldRc.Spec.Replicas, NewName, n EwRc.Spec.Replicas) NEWRC, err = r.resizeandwait (NEWRC, Retry, Waitforreplicas) if err! = Nil {R Eturn Err} time. Sleep (UpdatePeriod) OLDRC, err = r.resizeandwait (OLDRC, Retry, Waitforreplicas) if err! = Nil {R Eturn Err} fmt. Printf ("At End of Loop:%s Replicas:%d,%s replicas:%d\n", Oldname, OldRc.Spec.Replicas, NewName, NEWRC.SPEC.REPLICAS)}//delete remaining replicas on OLDRC if OldRc.Spec.Replicas! = 0 {fmt.  fprintf (out, "stopping%s replicas:%d-%d\n", Oldname, OldRc.Spec.Replicas, 0) OldRc.Spec.Replicas = 0 OLDRC, err = r.resizeandwait (OLDRC, Retry, Waitforreplicas)//OLDRC, err = r.resizeandwait (OLDRC, inTerval, timeout) if err! = Nil {return err}}//Add remaining replicas on NEWRC if NE WRc.Spec.Replicas! = Desired {FMT. fprintf (out, "resizing%s replicas:%d-%d\n", NewName, NewRc.Spec.Replicas, desired) newRc.Spec.Re Plicas = desired NEWRC, err = r.resizeandwait (NEWRC, Retry, Waitforreplicas) if err! = Nil {Retu RN Err}}//clean up annotations if NEWRC, err = R.c.replicationcontrollers (R.NS). Get (NewName); Err! = Nil {return err} delete (newRc.ObjectMeta.Annotations, sourceidannotation) Delete (Newrc.objectmeta .        Annotations, Desiredreplicasannotation) NEWRC, err = r.updateandwait (NEWRC, interval, timeout) if err! = Nil { return err}//delete old RC Fmt. fprintf (out, "Update succeeded. Deleting%s\n ", oldname) return R.c.replicationcontrollers (R.NS). Delete (Oldname)}

This code is very long, but the thing to do is simple:

1. If the new RC is not created, create it first if it has been created (created in the last rolling_update but timed out)
2. With a few loops, add the new RC replicas up, the old RC replicas down, the main function is resizeandwait and updateandwait

4. Underlying call

Next to the section of the resizeandwait, code in/pkg/kubectl/resize.go, where the specific code is not posted
All the rest of the calls occur/pkg/client this directory, this is a Http/json client, the main function is to send a request to Api-server
Overall, the wait on the above is the implementation of the soil, is to send an update request in the past, the subsequent polling call get to detect whether the state is in line with the final need.

5. Summary

Let's talk about the effects of these three time parameters:

Update-period: After adding a pod to the new RC, wait for this period and then reduce the pod from the old RC
Poll-interval: The function name is derived from the poll call on Linux, which is to initiate a request to the server for each poll-interval, until the request succeeds or the report fails.
Timeout: Time-out for total operations

Rolling update is mainly implemented by the client side, the analysis is done, but there are still some unknown problems, such as:

1. Api-server, Cadvisor, Kubelet, proxy, etcd How are these server-side components interacting? How do I make sure to increase or decrease pods when the service is always available?
2. Is it possible to insert some of your own code or process when the pod increases or decreases? Because we do not use the K8s proxy in our current architecture, we need to call the load-balanced system to give these pod diversion
3. For the specific pod, how do we do the internal program health check? Send a message to the K8S system when the business is unavailable, kill the pod, and create a new one on another machine instead.

Analysis of rolling update mechanism of kubernetes