How to build Kubernetes Platform (build Kubernetes Platform Scenario Reference)

• Architecture

• Architecture Diagram

• Non-prod Environment
• Prod Environment

• Cluster Networking
• Container Repository Setup

• Minikube for Dev ENV
• Kubeadm for Non-prod ENV
• Other IaaC for Prod ENV

• Devops

• Ci/cd
• Source Code Management
• PipeLine
• Monitoring

• Dashboard
• Metrics

• Security

• Pod Service Account
• Pod Security Policy

• Auth

• Authentication
• Authenrization
• Multi-tenancy

• Application Architecture
• Microservice
• Servcie Mesh

• Training

• Knowledge

• Kubernets
• ETCD
• Containter
• Networking

• Orgnization & People

ArchitectureArchitecture DiagramNon-prod Environment

Istio architecture

Prod Environment

Cluster Networking

Kubernetes supports for Third-party netwroking the cluster via the MLM plugin. Istio architecture diagram.For more infromation, please see Cluster Networking.

Istio service mesh architecture

According to this Chinese blog, Principles and Solutions of Kubernetes Networking from Yourongyun the three top of the MLM Pro Viders for kubernetes cluster based on VMs is Project Calico, flannel, Weave Net.

And also, here is comparison of variable networking solutions, Https://github.com/xelatex/homepage/blob/master/source/_ Posts/battlefield-calico-flannel-weave-and-docker-overlay-network.md

If just only considering performance, Project Calico should is perferred.

Container Repository

The options as follows,

Option 1, Private Repository, such as Harbor.

Option 2, repositories from Cloud Providers, such as AWS ECR.

Setup

We can refer to picking the right solution in Kubernetes offical document to select a setup solution. Considering our goal of building a kuberletes platform based on VMs in our on-premises data center.

minikube for Dev Env

Minikube can engaed to create a kubernets development in a local local single.

For more information about Minikube, please read Running Kubernetes locally via Minikube.

kubeadm for non-prod Env

Using Kubeadm, we can build a kubernetes cluster for Non-prod environment, which runs master key components as containers.

For more kubeadm information, please see the Using kubeadm to Create a Cluster.

Other IaaC for Prod Env

For creating a kubernetes cluster on VMs for Prod ENV, We need to a infrastructure automation tool, such as Ansible, to ha ve this is done.

Here is a reference of Creating kubernets Cluster via Ansible

DevopsCI/CDSource Code Management

For small and agile web projects, such SaaS applications, considering adopting GitHub Flow. For more information, please refer to GitHub Flow.

For desk or client applications, such PC desk application, iso/android apps, or being different time windows for delivery a ND release of applications, considering GitLab Flow. For more information-refer to Gitlab Flow.

PipeLine

MonitoringDashboard

Using Kubernetes Dashboard as web-based UI for kubernets clusters to manage the cluster itself along with its attendant re Sources.

For more information, please see Web-ui (Dashboard).

Metrics

Here is official suggested solution as follows. For more informaton, please see Tools for monitoring Compute, Storage, and Network Resources

Grafana + heapster/prometheus + cadvisor + InfluxDB

Heapster As a Metircs aggregator and processor

InfluxDB Time Series database for storage

Grafana As a dashboarding and alerting solution

Cadvisor Have been built in Kubelet, which collects host metrics like CPUs, disk space, and memory utilization, in addition to Container metrics.

And also, here's a practical example, how to utilize the ' Heapster + InfluxDB + grafana ' Stack in Kubernetes for Monitori Ng Pods.

Logging

ELK

APM

Zipkin

Pinpoint

SecurityPod Service Account

For more information, please see the User Guide to Service Accounts.

Pod Security Policy

For more information, please see Pod Security policies

AuthAuthentication

For more information, please see authenticating

Support SSO integration, such as Saml,ad OpenID, Auth2?

authenrization

For more information, please see https://kubernetes.io/docs/admin/authorization/

Abac/rbac

multi-tenancy

Hypernetes

For more information, please see hypernetes:bringing Security and Multi-tenancy to Kubernetes

Application ArchitectureMicroservice

Here's a Chinese blog about what to select Open Source tools for building a micorservice.

Https://mp.weixin.qq.com/s/bsuveX-E6E2fKZ24mj03nQ

Servcie Mesh

Linkerd

Envoy

Istio

TrainingKnowledgekubernets

Document

Kubernetes Handbook (Chinese Version)

ETCD

For more information, please see ETCD Document.

ContainterDocker

For Moe information, please see https://docs.docker.com/.

Cri-o

For more information, please see http://cri-o.io/.

OCI

OCI is a container specification named Open container Initiative, consisting of OCI Runtime specification and OCI Image Fo Rmat

NetworkingIp/route/iptables/ipvs etcContainer netwrokingOpenvswitchCni-calico/flannelCNM bridge/host/none/(overlay) PluginOrgnization & People

How to build Kubernetes Platform (build Kubernetes Platform scenario Reference)