Kubernetes 1.8

The Cluster Lifecycle layer adds self-hosted functionality to Kubeadm, which means that Kubernetes can be run on Kubernetes, which is bootstrap. Bootstrap is considered to be a system of "elegance" of a embodiment, in fact, Kubernetes in the early attempt to bootstrap, now Kubeadm directly add this feature, is undoubtedly a step closer. In addition, the KUBEADM community has been trying to deploy a highly available (HA) Kubernetes cluster directly, although it has not been released externally. It can be foreseen that the KUBEADM will further solve the Kubernetes most criticized deployment problem, kubernetes  volume reduce the use threshold.

The update of the storage function is a relatively large number of modules, is also the cloud technology attaches great importance to and continue to invest in the group. First of all, Kubernetes's storage model is stable, starting with 1.8 of releases to add more extensibility and additional features, such as support for Mount selection, support for Storageclass parameterization, etc. Snapshots, expansions, and local storage are undoubtedly the highlights of this release of the enclosure, although it is now alpha, even pre-alpha. Because the file system expansion program is not unified, kubernetes persistent volume the expansion in this release only support GlusterFS, but cloud technology has been on the ceph, cinder on the prototype, I believe in 1.9 can be launched in the community. Local temporary storage has been in a stable alpha version after 1.7 and 1.8 two iterations, and subsequent scenarios will no longer change and we will continue to enhance their stability. Local persistent storage progress is slow due to the need to discuss the design with the scheduler. The snapshot is now in the prototype phase,kubernetes volume mounts  and Kubernetes is inconsistent with the Volume snapshot in use.

Another notable case is the convergence of the SIG. Currently, Kubernetes Node, Storage, scheduling and other groups are collaborating to form the Resource group, which is designed to enable Kubernetes to support more types of applications. Auth, Node and other groups to establish Container Identity Group, to ensure that the container in the external communication of the security and reliability. In collaboration with various teams, kubernetes volume mount Kubernetes now proposes Device Plugin, CPU Manager, Hugepage, Resource Claas to support a variety of hardware. Kubernetes 1.8 is the first large-scale feature release after the founding of such a group and looks forward to more progress.

Let's take a look at what's published in Kubernetes 1.8.kubernetes vs docker

Publish a topic

persistent volume kubernetes

Kubernetes through the Interest Group (SIG) to manage the community and development, kubernetes ingress the following according to the interest group to interpret Kubernetes 1.8 release content.

SIG Apps

The work of SIG apps is focused on the Kubernetes API, what is kubernetes providing a variety of basic tools for managing different types of applications.

In the release version of 1.8, SIG apps migrated some of the APIs associated with the application to APPS/V1BETA2, including Daemonset,deployment,replicaset and Statefulset. In Apps/v1beta2, some of the content has been deprecated or changed, and this is done to provide developers with a stable and consistent set of APIs that allow developers to build applications based on Kubernetes. In subsequent releases, the SIG Apps will gradually push that version towards a stable version.

SIG Auth

SIG Auth is responsible for Kubernetes certification,kubernetes tutorial  authorization and cluster security policy related work.

In Kubernetes 1.8, the SIG Auth focuses primarily on the features introduced in previous releases of stabilization. The RBAC (role-based access control) feature has been promoted to the v1,advanced Auditing (advanced auditing) feature has been released in the beta version. The encryption of resources at rest (static resource encryption) feature is still alpha version and begins to attempt to integrate with the external key management system.

SIG Cluster Lifecycle

The SIG Cluster Lifecycle is responsible for deploying the user experience of upgrading and deleting clusters.

In the 1.8 release, SIG Cluster Lifecycle continues to focus on the functionality of the extended Kubeadm, which is both a user-oriented cluster management tool as well as a build unit for high-level systems. Starting with version 1.8, KUBEADM supports a new upgrade command and provides alpha support for the self hosting of the cluster control component.

SIG Node

SIG node is responsible for the components of resource interaction between Pod and host kubernetes certification host and the lifecycle of pods on the management node

For the 1.8 release, SIG Node continues to focus on supporting a wider range of workload types, kubernetes persistent volume claim including support for hardware and performance-sensitive workloads such as data analysis and deep learning, while continuously enhancing Node reliability. The SIG network is responsible for the networking components, APIs, and plugins in Kubernetes.

SIG Network

The SIG network is responsible for the networking components, APIs, and plugins in Kubernetes.

For version 1.8, the SIG Network enhanced the Networkpolicy API to support Pod egress traffic policies, as well as the matching criteria that allow policy rules to match source or target CIDR. Both of these enhancements are designed to be beta versions. The SIG Network is also focused on improving kube-proxy, in addition to the current iptables and userspace modes, Kube-proxy also introduces an alpha version of IPVS mode.

SIG Storage

Storage interest groups primarily contain storage and various storage volume plugins.

1.8, the storage interest group extends the kubernetes storage API, no longer simply provides the available volumes, but also adds volume expansion and snapshot capabilities. In addition to these alpha/prototype features, the storage interest group focuses on giving users better control over their storage, providing the ability to set temporary storage requests & limits, to specify mount options, to expose more storage metrics, kubernetes persistent volumes example and to improve Flex Driv ER deployment.

SIG Scheduling

The SIG scheduling is primarily responsible for common scheduler and scheduling related components.

In release 1.8, SIG scheduling extends the concept of shared clusters by introducing Pod priority and preemption features. These features allow mixing of different types of applications and tasks in a single cluster, increasing the utilization and availability of the cluster. These features are currently alpha versions. SIG scheduling will also progressively optimize scheduling-related internal APIs, making it easy for other components and external schedulers to use these APIs.

SIG autoscaling

The SIG autoscaling is primarily responsible for elastic scaling related components such as horizontal Pod Autoscaler and Cluster Autoscaler.

In the release version of 1.8, the SIG autoscaling is primarily about improving the stability and functionality of existing components. For example, volume mount kubernetes the new horizontal Pod Autoscaler will support custom metrics, Cluster Autoscaler improves performance and error reporting capabilities.

SIG instrumentation

The SIG instrumentation is responsible for the output and collection of indicators.

In version 1.8, the focus of SIG instrumentation is to support the new version of the HPA API and to upgrade the dependent APIs and components to a stable version, including: Resource metrics Api,custom Metrics API and Metric S-server (Metrics-server will be in the alternative heapster in the default monitoring pipeline role).

SIG Scalability

SIG Scalability is responsible for scalability testing, measuring and improving system performance, and answering questions about scalability.

For the 1.8 release, SIG Scalability focuses on automating large cluster scalability testing in a continuous integration (CI) environment. In addition to defining the specific process for scalability testing, the SIG Scalability creates documentation for the current scalability thresholds and defines a new set of service level metrics (SLI) and service level objectives (SLO) across the system.

Scalability Validation report for this release:

Https://github.com/kubernetes/features/blob/master/release-1.8/scalability_validation_report.md

Main content

Workload API (APPS/V1BETA2)

Kubernetes 1.8 adds Apps/v1beta2, which includes Daemonset,deployment,replicaset and Statefulset. These APIs will gradually stabilize in future releases.

API Additions and Migrations

• The current version of Daemonset,deployment,replicaset and Statefulset is apps/v1beta2.

• In Apps/v1beta2, Statefulset increases the scale sub-resource.

• All types in apps/v1beta2 have the appropriate Condition type added.

Behavioral changes

• For all types in Apps/v1beta2, Spec.selector is disabled by default because it is incompatible with KUBECTL apply and strategic merge patches. The user must explicitly set the Spec.selector, and if Spec.selector does not match the labels in spec.template, then this object is not valid.

• Because these types of controllers are not handled differently for selector, these types of selector cannot be modified in APPS/V1BETA2. This restriction may be removed in the future, but it may also be retained to a stable version. If the user has some code that relies on mutable selector, then the code can continue to use the APPS/V1BETA1 version, but it should start to modify the code and no longer depend on the modifiable selector.

• Extended Resource is a legal domain name other than Kubernetes.io. The value of Extended Resource must be an integer. The user can use any legitimate resource name, such as [AAA.] MY-DOMAIN.BBB/CCC, instead of continuing to use Opaque Integer Resource. Extended Resource is not dynamic, so the same Extended Resource values must be the same in request and limit.

• The default Bootstrap Token created by the KUBEADM init v1.8 version is deleted by default after 24 hours, preventing the cluster from leaking important information. Users can create a new Bootstrap token by Kubeadm token create or by setting –token-ttl 0 to Kubeadm init so Bootstrap token does not expire. The default token can be deleted by Kubeadm token Delete.

• Instead of implementing the process yourself, the KUBEADM join now gives the TLS boot to Kubelet to complete. The KUBEADM join writes the startup Kubeconfig file to/etc/kubernetes/bootstrap-kubelet.conf.

Default value

• The Spec.updatestrategy default values for Statefulset and Daemonset are rollingupdate in Apps/v1beta2. If necessary, the user can be manually set to OnDelete.

• Selector is disabled by default.

• The default value for the related type of spec.revisionhistorylimit in Apps/v1beta2 is 10.

• The default value for the Cronjob spec.successfuljobshistorylimit defaults to 3,spec.failedjobshistorylimit is 1.

Workload API (Batch)

• Cronjob has migrated to Batch/v1beta1.

• Batch/v2alpha. The cronjob has been deprecated and removed in a future release.

• The Job can now set the failure policy through Spec.backofflimit. The default value for this field is 6.

• Batch/v2alpha1. Scheduledjobs has been removed.

• The JOB controller now creates pods in batches instead of the previous one-time creation.

• The Job can now set a shorter spec. Activedeadlineseconds.

Scheduling

• [Alpha] supports Pod priority and PriorityClass

• [Alpha] supports POD priority-based preemption

• [Alpha] hit node taint by condition

Storage

• [Stable] Mount options

  • Elevate the specified mount option capability from beta to stable version

  • Add a new variable mountoptions to the Persistentvolume spec to specify mount options in place of the original alias setting

  • The mountoptions is also added to the Storageclass spec, allowing the Mount option to be configured for dynamically supplied volumes. Allows K8S administrators to control the mount options used within their clusters

• [Stable] provides attach/detach for RWO volumes such as ISCSI and FC

• [Stable] exposing storage usage information

  • Kubernetes metric API exposes how much storage space is available for PV

• [Stable] Volume plug-in information

  • The Kubernetes metric API exposes successful or deferred information for certain operations, including the following: Mount/unmount/attach/detach/provision/delete

• [Stable] Modify the corresponding PV spec for Azure File, CEPHFS, ISCSI, Glusterfs so that they can reference namespace resources

• [Stable] supports iSCSI volume plug-in customization for each volume iSCSI initiator name

• [Stable] supports WWID for FC volume identifiers

• [Beta] Storageclass Recovery Strategy

  • Run the recycle policy in configuration storageclass, unlike before, for dynamically supplied volumes, the default is only delete

• [Alpha] Volume expansion

  • Run the volume with the Kubernetes API

  • The alpha version only expands on specific volume, but does not do file system expansion

  • In alpha version, only the expansion of the Gluster is implemented.

• [Alpha] provides isolation and management capabilities for local temporary storage

  • For new resource Ephemeral-storage, run the set Requests/limits and node reservation for the container

  • Ephemeral-stroage contains all the disk space that the container may use

• [Alpha] Mount space propagation

  • In the pod declaration, add a new entry for the container Volumemount volumemount.propagation

  • This new item can be set to bidirectional, allowing one of the container's mounts to propagate to the host or other container

• [Alpha] Improved Flexvolume deployment

  • Simplifies deployment of Flex volume driver

    • Automatically discovers and initializes new driver files, instead of having to restart Kubelet and Controller-manager as before

    • Provides a daemonset sample that can be used to deploy Flexvolume drivers

• [Prototype] Volume snapshot

  • Allow creation of volume snapshots via kubernetes API Trigger

  • Because the service is stopped before the snapshot is not supported, it is possible that the snapshot data is inconsistent

  • This project is not in the core kubernetes repo inside, in https://github.com/kubernetes-incubator/external-storage/tree/master/snapshot here

Node Component

Kubelet

• [Alpha] Kubelet now supports container-level CPU affinity policies that are replaced with the new CPU manager.

• The [Alpha] application can now request pre-allocated hugepages by using the new hugepages resource in the container resource request.

• [Alpha] Increased support for Kubelet dynamic configuration

• [Alpha] Increase CRI checksum test set and CRI CLI

• [Alpha] Add API for hardware device plug-ins

• [Stable] Support Cri-o, has passed all the e2es

Automatic Scaling and metrics

• Horizontal Pod Autoscaler Upgrades the custom metrics to the beta version. The APIs for the associated metrics are upgraded to the V1BETA1 version. Review the required actions before upgrading.

• It is recommended to use Metrics-server as the component that provides the resource metrics API. It can be deployed as a plug-in, similar to how heapster is deployed.

Cluster Auto Extender

• Cluster Auto Extender upgraded to GA

• Extended Cluster supports 1000 nodes

• Pod Graceful Stop time is 10 minutes

• Handling area inventory and failures

• Improved monitoring and Error reporting

Auth

• [GA] The RBAC API Group has been upgraded from V1beta1 to V1. No API-related modifications were introduced.

• [Beta] Advanced auditing has been from alpha to beta. The sections related to Webhook and the logging policy format are changed in comparison to Alpha and may need to be modified.

• [Beta] Kubelet certificate rotation through the Certificates API (to rotate certificates through the certificate API) has been changed from alpha to Beta. RBAC's cluster role (cluster roles) configured for certificates controllers (certificate controller) has been created to access the generic certificate APIs such as Kubelet.

• [Beta] Selfsujectrulesreview is an API that lets a user know what permissions he has under a namespace and has been added to the Authorization.k8s.io API group. This batch query is designed for UIs to show and hide some functions according to the user. And this API allows users to quickly understand their own permissions.

• [Alpha] Based on the 1.7 version of the work allows to encrypt resources such as secrets, the resource encryption with the key stored in the external KMS system. The implementation of this mechanism allows integration with various KMS systems in addition to supporting the initial file-based storage. The Google Cloud KMS plugin has been added and can be used once the Google-side integration is complete.

• Websocket requests can now be authenticated through API server by setting bearer tokens in the Websocket subprotocol Base64url.bearer.authorization.k8s.io.

• Advanced Audit is now able to correctly report impersonated user (impersonated users) information.

• The advanced audit policy now supports matching sub-resources and resource names, but the top-level resources no longer match child resources. For example, "pods" can no longer match a request for a logs sub-resource. To use "pods/logs" to match child resources.

• A previously deleted service account or bootstrapping token secret will be considered valid until they are really recycled. Now when Deletiontimestamp is set they will fail.

• The –insecure-allow-any-token parameter has been removed from API server. Users using this parameter should use the impersonation header instead to debug it.

• The Noderestriction admission plugin now allows a node to expel pods bound to itself.

• Ownerreferencespermissionenforcement Admission Plug-in in order to set up blockownerdeletion on an owner reference, it is now necessary to referenced The Finalizers child resource for owner (the referenced owner) has the update permission.

• The Subjectaccessreview API under the Authorization.k8s.io API group now allows the user's UID to be provided.

• After Kubelet rotates its client certificate, it closes the link to the API server to force the new certificate handshake. The previous kubelet will keep the existing connection always on, even if the certificate used by the connection has expired and is rejected by the API server.

• Podsecuritypolicies can now specify a whitelist that records the path that is allowed as a host data volume.

• API Server Authentication now caches the successful certified bearer token for a few seconds.

• The OpenID Connect authentication plug-in now has the ability to add custom prefixes before username and groups two claim, or use the default prefix. Two parameter settings via –oidc-username-prefix and –oidc-groups-prefix. For example, the authentication plugin can map a user named "Jane" to "Google:jane" by setting the "Google:" username prefix.

• The bootstrap token authentication plugin is now able to configure groups in addition to system:bootstrappers in tokens.

• Advanced audit allows logging of failed login requests.

• The Kubectl auth RECONCILE subcommand has been added to apply the RBAC resource. When passing in a file including RBAC Roles,rolebindings,clusterroles, or clusterrolebindings, the command can calculate the override permissions and add the missing rule.

Cluster Lifecycle

Kubeadm

• [Beta] A new upgrade subcommand allows you to automatically upgrade a self-hosted cluster using KUBEADM.

• [Alpha] with KUBEADM init, you can easily create an experimental self-hosted cluster. This feature is enabled by configuring the Selfhosting feature for True:–feature-gates=selfhosting=true.

  • Note: In the next release version, version 1.9, self-hosting will be the default way to deploy control components.

• [Alpha] A new phase subcommand supports subtasks that run the KUBEADM init process. Combined with more accessible configurations, Kubeadm is now easy to integrate into advanced deployment platforms such as Kops and Gke.

  • Note: This command is currently placed in the KUBEADM alpha Phase subcommand and will be placed in the top-level command in future versions.

Kops

• [Alpha] supports bare metal (non-cloud provider) environments.

• [Alpha] Kops now supports running as a service.

• [Beta] GCE supports upgrading from alpha to beta.

Cluster Discovery/bootstrap

• [Beta] Bootstrap Tokens This authentication and identification method is further optimized. Use Bootstrap Tokens to add a lot of new nodes to the container.

Multi-Platform

• [Alpha] A consistent e2e test suite began to support arm, arm64, and Ppc64le architectures.

Cloud Providers

• [Alpha] supports pluggable, Out-of-tree and Out-of-core cloud provider features have been significantly improved.

Network

Network-policy

• [Beta] CIDR-based Networkplicy policy support.

• [Beta] Egressrules is supported in Networkpolicy.

Kube-proxy Ipvs Mode

• [Alpha] Kube-proxy supports Ipvs mode.

API Machinery

Kube-apiserver

• Fixed an issue with Apiservice autoenrollment. This issue has affected the rolling restart of the HA API to add or remove API groups.

• [Alpha] The Kubernetes API now supports list queries that specify criteria. The client can specify the number of results returned, and if more results are present, a token token is returned for repeated calls until all results are retrieved. Due to the functionality provided by ETCD3, the result list is the same as a list call that does not perform a chunked. This allows the server to respond to very large lists with less memory and CPU. The entry for this feature is apilistchunking and is not turned on by default. Version 1.9, all informer will be used by default.

• Ignores pods that are marked for deletion when the grace period is exceeded in Resourcequota.

Dynamic Admission Control

• Pod spec is changed when the pod is not initialized. The API server requires pod spec to be valid even if the pod is not initialized. Updating the status of an uninitialized pod is not valid.

• The initialization function using Alpha now requires the initializers function entry to be turned on. This feature is enabled automatically if the Initializersadmission plugin is enabled.

• The validation rules for the [Action required] metadata.initializers.pending[x].name are tightened. The name of the initializer needs to include at least three segments separated by a point. Depending on the build rule, you can create objects with pending initializers and do not rely on API server to add pending initializers. If you do, update the existing objects and initializer name in the configuration file to conform to the new validation rules.

• Even if the API servers and nodes are in two separate networks, the Webhook admission plug-in will work as expected. For example, in Gke. Webhook author can use DNS names such as services to generate Wehhook server certificates as a common name.

• Action Required:

• Previously, the value of CN could be ignored for admission webhook to regenerate the server certificate. Now it must be set to the DNS name of the Webhook service: <service. Name>.<service. Namespace>.svc.

Custom Resource Definitions (CRDs)

• [Alpha] The Customresourcedefinition API now has the option to validate the JSON-schema-based custom object provided by CRD spec

• You can enable this feature through the Customresourcevalidation feature entry in Kube-apiserver.

Garbage Collector

• Garbage collector now supports servers custom APIs via customresourcedefinition or aggregated API. Garbage collector controller periodically refreshes, so expect a 30-second interval between adding an API and garbage collector start managing it.

Monitoring/prometheus

• [Action required] in Prometheus's Apiserver_request_* series, watchlist as a verb WATCH. Depending on the level of the query, a new "scope" tag will be added to all apiserver_request_*, which can be ' cluster ', ' resource ', or ' namespace '.

Go Client

• Add support for client junk mail filtering

Reprinted: 78250552

Kubernetes 1.8