What are the underlying issues that the container private cloud and continuous publishing have to solve? Second episode

Zheng was founded in 2015/10/30 last updated on 2015/11/20 keywords: Docker, container, continuous integration, continuous release, private cloud, Jenkins,mesos,marathon This document applies to people: the broad sense of the technical staff outline:

1. Container or volume mount?
2. Host Networking or Bridge Networking?
3. Does the container want to secure the IP?
4. How do I get the host's IP inside the container?
5. How are container logs collected?
6. Apache Mesos or Google k8s?
7. How to ensure Registry mirror Pull/push security?
8. How do I secure the Marathon API and Docker API calls?

These underlying issues also need to be addressed when building a container-based private cloud and the corresponding continuous release. Continuation of the previous article.How is the 0x04 container log collected? We are still using our usual ELK scheme. That

1. with the developer's agreement on the path specification for the log file, the log falls on the local disk of the Mesos Slave host where the container resides, and the
• log location is unified:/data/application/logs
• Log Text Unified Name: log type-project name-environment-container tag-hostname , so Logstash You can extract important information from the file name:
• Application log location:/data/application/logs/log Type _${appname}_${appenv}_${appimgtag}_${hostname }_%d{yyyymmdd}.log,  such as Aether's log location is:/data/appliaction/logs/weberror_aether_nor_8_778283_20150819.log 
•  trace (Eagle Eye) log location:/data/application/logs/tracing/tracing_${appname}_${appenv}_${appimgtag}_${ Hostname}_%d{yyyymmddhh}.log, such as the trace log location for aether:/data/appliaction/logs/tracing/tracing_aether_nor_8_778283_ 2015081912.log 
• for ENV in shorthand form, take the first 3 characters of each environment to refer to the environment:
• develop the Union (DEV)
• General (NOR)
• Emergency (EME)
• Special (SPE)
• Mirroring (MIR)
• production (pro)
2. start Log Stash Agent to collect logs, to Logstash,
3. We are based on Kibana query and analysis log, especially our Exception Log analysis and summary.

0x05 Apache mesos or Google k8s? VS K8S is the acronym for Kubernetes. In November 2014, k8s also needed some open source components to complete the network configuration, so we chose the more mature and easy-to-use Mesos+marathon at that time.   Since k8s as a container orchestration tool can be erected on the Mesos, k8s is more and more mature, so the follow-up does not exclude the selection mesos+k8s. Next, call security. How does 0x06 guarantee registry mirroring Pull/push security? The storage and management of Docker images, which correspond to Docker-registry, are developed in the Python language. It consists of three components:

1. Docker Index
• Web UI
• Meta-data Metadata Storage (notes, stars, public library listings)
• Access authentication
• Token management
2. Docker Registry
• Storage mirroring, and the family pedigree of the mirroring layer
• No user account data
• Do not know the user's account and security
• Entrust security and certification to Docker-hub, with token to ensure safe delivery
• No need to reinvent the wheel to support multiple storage back-end
• No local database
3. Back-End Storage
• Because mirroring is ultimately stored statically on the server in a tar.gz way
• For object storage instead of block storage
• Registry Storage Driver
• Official supported drivers are files, Amazon AWS S3, CEPH-S3, Google GCs, OpenStack swift,glance

can see that docker-registry Default no security permission settings, anyone can pull and push, so security and authentication is handled by Docker Index (ie docker-hub). But at the moment we haven't introduced the Docker Index role, just a layer of Nginx in front of the Docker Registry. Responsible for Basic-auth authentication and SSL encryption transmission, this is because Docker 1.3, released in October 2014, started to force Basic authentication and must use HTTPS to access Registry, as shown in: That is, Jenkins needs to hold a for basic Auth. User name password and SSL certificate. The following is a reference to Larrycaiyu's mapping to illustrate how Nginx+registry's services are composed: About SSL self-signed certificates (serf-signed certification), which you can refer to when building docker-registry with these two articles Self-signed SSL certificate authentication issues and Building private Docker registry with basic authentication.0x07 How to ensure the security of Marathon API and Docker API calls? Our Continuous Integration management platform (Codename:touchstone) calls the Marathon REST API for container deployment work, as shown in. The Marathon REST API supports basic Auth and SSL, as shown in the official document SSL and Basic Access authentication, so Touchstone needs to hold the user name for basic Auth  Password. In April 2014, Docker 0.10 introduced the TLS auth with the built-in TLS/SSL certificate security so that the Docker Remote API no longer ran naked. Currently, we have--tlsverify enabled in the Docker Daemon configuration file/etc/sysconfig/docker of the Mesos Slave host:

options= '--tlsverify --tlscacert=/root/.docker/ca.pem--tlscert=/root/.docker/cert.pem--tlskey= /root/.docker/key.pem

.......

thusJava-developed touchstone is configured through the interface and stores the following information in its database:

• User name password and SSL certificate required to access the remote Marathon REST API;
• The SSL certificate required to access the remote Docker remote API.

---Welcome to subscribe to my subscription number "veteran notes", please scan the QR code attention: Please specify "reprint from onlookers-blog Park" or give the original link of this article. Appendix A: Reference Resources 1,2015, the main features and experience sharing of chat Kubernetes, Http://www.dockone.io/article/578;2,2015,Swarm, Fleet, Kubernetes, Mesos- Comparative analysis of Orchestration tools, customization and performance analysis of Http://dockone.io/article/823;3,2015,Docker registry, Http://dockone.io/article/375;4, 2014, use Nginx to do the private Docker registry security control, HTTP://WWW.LARRYCAIYU.COM/2014/12/01/ private-docker-registry-with-nginx.html;5,2014, use self-signed SSL certificate authentication problem when building Docker-registry, https://www.webmaster.me/ Server/docker-registry-with-self-signed-ssl-certificate.html;6,2015, from the Docker hub and Docker-registry to see Excellent backend service design implementations, Http://dockone.io/article/142;7, Ma Quan, Https://github.com/containerops/wharf;8,Marathon API doc,https:// Mesosphere.github.io/marathon/docs/ssl-basic-access-authentication.html;9,2014,docker Getting Started tutorial of Docker Remote api;10, 2015, Zheng, the container private cloud and the continuous release of the basic problems to solve what the first episode;-eof-

What are the underlying issues that the container private cloud and continuous publishing have to solve? Second episode