Custom Storage plug-in-flexvolume vs Csi_docker

Starting with version 1.8, Kubernetes Storage SIG stopped accepting tree volume Plug-ins and recommended that all storage providers implement the tree plug-in. There are currently two recommended implementations: the container Storage Interface (CSI) and the Flexvolume. Flexvolume Introduction

Lexvolume enables users to write their own drivers and add support for volumes in kubernetes. If –enable-controller-attach-detach enables the Kubelet option, the vendor driver should be installed in the volume plug-in path on each kubelet node and master node.

Flexvolume is the GA feature after the Kubernetes 1.8 version. Prerequisite

Install the vendor driver on all nodes in the plug-in path, –enable-controller-attach-detach set to True,
Path to install plugin: \ Dynamic Plugin Discovery

Flexvolume the ability to support dynamic detection of drivers from v1.8.
You do not need to have a driver in the system initialization, or you need to restart the Kubelet or controller manager.
You can install, upgrade/demote, and uninstall drivers while the system is running. For more information, see Designing a Document Automatic plug-in installation/upgrade

One possible way to install and upgrade the Flexvolume driver is to use Daemonset. See the recommended driver deployment method for more information. Plug-in details

The plug-in wants to implement the following call for the back-end driver. Some annotations are optional.
Calls are called from the Kubelet and controller manager nodes.
Only when the "–enable-controller-attach-detach" kubelet option is enabled,
is invoked from the Controller-manager call. Driver Call Model Init

Initialize the driver. Called during the initialization of Kubelet&controller manager.
When successful, the function returns a function map that shows whether the driver supports each Flexvolume feature. Current function: Attach-Boolean field indicating whether the drive requires additional and detached operations.
This field is required, but for backward compatibility, the default value is set to true, which requires additional and detached.
For a feature graph format, see driver output.

<driver executable> Init

Attach

Attaches the volume specified by the given specification on a given host. When successful, returns the device path that the device connects to the node.
If the "–enable-controller-attach-detach" kubelet option is enabled,
Then the nodename parameter is valid/relevant. From Kubelet&controllermanager.

This call does not pass the "secrets" specified in the Flexvolume specification. If your driver needs secrets,
Instead of performing this call, use the mount to pull up and execute attach and calls in the pull.

<driver executable> Attach <json options> <node

Detach

Detach the volume from the Kubelet node. Only when the "–enable-controller-attach-detach" kubelet option is enabled
The nodename parameter is valid/relevant. Called by Kubelet & Controller Manager

<driver executable> Detach <mount device> <node

Wait for attach

Wait for the volume to connect to the remote node. When successful, returns the path to the device. Called from Kubelet & Controller Manager with a timeout of 10 milliseconds (code),

<driver executable> Waitforattach <mount device> <json

Volume is attached

Check that the volume is connected on the node. Called from the Kubelet & Controller Manager.

<driver executable> isattached <json options> <node

Mount Device

The Mount device mounts the device to the global path, and then the individual containers can be dynamically bound and can only be invoked from Kubelet.

This call does not pass the "secrets" specified in the Flexvolume specification. If your driver needs secrets,
Instead of performing this call, use the mount to pull up and execute attach and calls in the pull.

<driver executable> mountdevice <mount dir> <mount device> <json options>

Unmount device

Cancels all mounts, which are invoked once all the binding mounts have been uninstalled. Can only be invoked from Kubelet.

<driver executable> unmountdevice <mount device>

Mount

Mount the volume to the Mount directory. This is the default for binding mounted drivers that implement attach and Mount-device. Can only be invoked from Kubelet.

<driver executable> Mount <mount dir> <json options>

Unmount

Unmount the volume by default, which is the driver for attaching and mounting devices that the binding mount implements. Can only be invoked from Kubelet.

<driver executable> unmount <mount dir>

For a simple example of how to write a simple flexvolume driver, see Lvm&nfs. Drive output

Flexvolume want the driver to return the operation status in the following format.

{
    ' status ': ' <success/failure/not supported> ', ' Message
    ': ' <reason for success/failure> ',
    ' Device ":" <path to the device attached. This is valid a for attach & Waitforattach call-outs> "
    volumename": "<cluster wide unique name of The volume. Valid only to Getvolumename call-out> "
    attached": <true/false (return True if volume was attached on the node. Valid only to isattached call-out) >
    "Capabilities": <only included as part of the Init response>
    {
        " Attach ": <true/false (return True If the driver implements attach and Detach) >
    }
}

Default JSON option

In addition to the flags specified by the user in the Flexvolumesource options field, the following flags are passed to the executable file. Note: The secret is only passed to "Mount/umount" for an example of Flexvolume

For a quick example of how to use Flexvolume in pod, see Nginx.yaml&nginx-nfs.yaml.

Https://github.com/sigma/cifs_k8s_plugin
Https://github.com/kubernetes/kubernetes/blob/master/examples/volumes/flexvolume/lvm
Introduction to Https://github.com/kubernetes/kubernetes/blob/master/examples/volumes/flexvolume/nfs CSI

CSI provides a single interface where storage vendors can implement their storage solutions to work across multiple different container layouts, and the volume plug-in is designed to be out-of-tree. This is a huge effort, and CSI's full implementation takes several quarters and requires immediate solutions for storage vendors to continue adding volume plug-ins.
It enables many different types of storage systems to:
-Create the storage automatically when needed.
-Make storage available wherever you plan.
-delete the store automatically when it is no longer needed. Reasons to create a CSI

Kubernetes volume plug-ins are currently "in-tree", meaning they are linked, compiled, built, and published with Core Kubernetes binaries.
Adding support for the kubernetes (volume plug-in) to the new storage system requires that code be checked into the core kubernetes repository.
But the Kubernetes release process is painful for many plug-in developers.

The existing Flex volume plug-in attempts to address this problem by exposing the exec-based APIs of the external volume plug-in.
Although it enables Third-party storage vendors to write drivers outside the tree, in order to deploy Third-party driver files,
It requires access to the root file system of the node and the host.

In addition to being difficult to deploy, Flex does not address the pain of plug-in dependencies: Plug-ins often have many external requirements (for example, on mounts and file system tools).
It is assumed that these dependencies are available on the underlying host operating system, which is often not the case (and that they require access to the root file system of the node machine).

CSI solves all of these problems, enabling storage plug-ins to be kubernetes, containerized, deployed,
and is used by the Kubernetes storage Primitives (persistentvolumeclaims,persistentvolumes,storageclasses), which are familiar and favorite to users. CSI Drive

Https://kubernetes-csi.github.io/docs/Drivers.html usage Enable CSI

CSI is alpha in 1.9, and to use it, set the following parameters:

API Server Binary:–feature-gates=csipersistentvolume=true–runtime-config=storage.k8s.io/v1alpha1=true

API server binary and Kubelet binaries:–feature-gates=mountpropagation=true–allow-privileged=true preconfigured volume

Preconfigured drivers work the same way as before, the administrator creates a Persistentvolume specification that describes the volume to use. The Persistentvolume specification needs to be set according to your driver, but the difference is that there is a new part called CSI that needs to be set accordingly. Please refer to the Kubernetes documentation on CSI volumes (LINK TBD).

The following is an example of a persistentvolume specification for preconfigured volumes managed by the CSI driver:

APIVERSION:V1
kind:persistentvolume
metadata:
  NAME:MANUALLY-CREATED-PV
Spec:
  capacity:
    storage:5gi
  accessmodes:
    -readwriteonce
  persistentvolumereclaimpolicy:retain
  CSI:
    Driver:com.example.team/csi-driver
    volumehandle:existingvolumename
    readonly:false

Dynamic Pre-matching

To set up the system for dynamic configuration, the administrator needs to set the external configurator that Storageclass points to the CSI driver and specify any parameters required by the driver. This is an example of a storageclass:

Kind:storageclass
apiversion:storage.k8s.io/v1
metadata:
  name:fast-storage
Provisioner: Com.example.team/csi-driver
parameters:
  TYPE:PD-SSD

Provider: The name of the CSI driver must be set
Parameter: must contain any parameters specific to the CSI driver.
The user can then use this storageclass to create a persistentvolumeclaim, as follows:

APIVERSION:V1
kind:persistentvolumeclaim
metadata:
  name:request-for-storage
Spec:
  Accessmodes:
  -readwriteonce Resources
  :
    requests:
      storage:5gi
  storageclassname: Fast-storage

Using in k8s

This section describes how to deploy the CSI driver to the k8s 1.9 cluster

In Kubernetes 1.9, three new components plus kubelet enable CSI drivers to provide storage for kubernetes.
The new component is a side-car container that is responsible for communicating with the kubernetes and CSI drivers, monitoring events so that the CSI interface is invoked in due course. External-attacher

External-attacher is a side vehicle container that monitors kubernetes volumeattachment objects and triggers the CSI for driver endpoints Controllerpublish and Controllerunpublish operations. When writing this article, the external assistant does not support leader elections, so each CSI driver can only run once. For more information, please read attach and detach.

Note that even if this is called an external attachment, its function is to invoke the CSI API to invoke Controllerpublish and Controllerunpublish. These calls are most likely to occur in a node that is not the volume that will be installed. As a result, many CSI drivers do not support these calls, but instead perform Attach/detach and Mount/unmount in the CSI Nodepublish and Nodeunpublish calls completed by the Kubelet on the node to be installed. External-provisioner

External-provisioner is a sidecar container that monitors kubernetes Persistentvolumeclaim objects and triggers the CSI for driver endpoints Createvolume and Deletevolume operations. For more information, please read the supply and delete. Driver-registrar

Driver-registrar is a side vehicle container that registers the CSI driver with Kubelet and adds driver customization Nodeid to the label on the Kubernetes Node API object. This is done by communicating with the identity service on the CSI driver and invoking the CSI Getnodeid operation. The driver registrar must have the Kubernetes name of the node set by the environment variable, kube_node_name as follows:

        -Name:csi-driver-registrar
          imagepullpolicy:always
          image:docker.io/k8scsi/driver-registrar
          args:
            -"--v=5"
            -"--csi-address=$"
          env:
            -name:address
              value:/csi/csi.sock
            -Name: Kube_node_name
              valuefrom:
                fieldref:
                  fieldPath:spec.nodeName
          volumemounts:
            -NAME: Socket-dir
              Mountpath:/CSI

Pod Configuration

          Volumemounts:
            -name:socket-dir
              mountpath:/csi
            -Name:mountpoint-dir mountpath
              :/var/lib/kubelet/ Pods
              mountpropagation: "Bidirectional"
      volumes:
        -name:socket-dir
          hostpath:
            path:/var/lib/ Kubelet/plugins/csi-hostpath
            type:directoryorcreate
        -name:mountpoint-dir
          hostpath:
            path:/ Var/lib/kubelet/pods
            Type:directory

RBAC Configuration

Kind:clusterrole
apiversion:rbac.authorization.k8s.io/v1
metadata:
  name:csi-hostpath-role
Rules:
  -apigroups: ["] Resources
    : [" persistentvolumes "]
    verbs: [" create "," delete "," Get "," list "," Watch "," Update "]
  -apigroups: [" "] Resources
    : [" persistentvolumeclaims "]
    verbs: [" get "," list "," Watch ", "Update"
  -apigroups: [""] Resources
    : ["nodes"]
    verbs: ["get", "list", "Watch", "Update"]
  - Apigroups: ["Storage.k8s.io"] resources
    : ["storageclasses"]
    verbs: ["get", "list", "Watch"]
  - Apigroups: ["Storage.k8s.io"] resources
    : ["volumeattachments"]
    verbs: ["get", "list", "Watch", "Update"]

Reference

Https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md
Https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/flexvolume-deployment.md
Https://github.com/container-storage-interface/spec/blob/master/spec.md
Https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md
Http://blog.kubernetes.io/2018/01/introducing-container-storage-interface.html
Https://github.com/container-storage-interface
https://kubernetes-csi.github.io/docs/

Welcome to join QQ Group: k8s Development and Practice (482956822) Exchange k8s Technology