This is a creation in Article, where the information may have evolved or changed.
"Editor's words" This time the main share of Dockercon on the information, share all the content can be divided into three parts: 1. Docker 1.7.0 depth resolution; 2. Dockercon hackathon, 3. Dockercon Festival.
First step: Docker 1.7.0 Depth Analysis
From the Docker version change log, Docker 1.7.0 will have more or less changes in four areas: Docker runtime, Docker code changes, Docker Builder module, and Docker bug fixes.
This share mainly involves the runtime of the Docker 1.7.0.
1.1. Added a feature that is still in the experimental phase: Data Volume plug-in supporting out of process
What is the experimental nature, in other words, this part of the Docker feature is not yet supported in a production environment, and these features are more desirable for users to use only in test environments and sandbox environments. The experimental features are completely a highlight of the Docker 1.7.0.
On the basis of understanding out-of-process, it is much easier, plug-in itself and Docker daemon no coupling, Plug and Play, in the Docker daemon outside the scope of play.
At present, the experimental characteristics of Docker can be described from two aspects, first Docker has now supported the user to customize the use of third-party plug-ins, and on this basis, Docker itself supports the container data volume volume plug-in. In addition, Docker defines a complete set of plug-in-related APIs for user-friendly use. Of course, the follow-up in this area, whether it is the official Docker, or the entire community, will continue to have new plug-ins are born. It is worth mentioning, in the data volume volume plug-in aspect, appeared the Flocker figure, this also means the container data storage question, really is put on the table, and has the corresponding reasonable solution.
1.2. From the Docker daemon angle, add the userland-proxy start-stop switch
First of all, the role of Userland-proxy has always been introduced. As we all know, in the bridge-bridged network mode of Docker, the Docker container establishes the communication with the world outside the host by the NAT mode on the host. On the host, however, in general, the process can access the container in three ways, namely: <eth0ip>:
When Userland-proxy was introduced, it was probably because the designer realized that 0.0.0.0 address and localhost have a functional defect on the container access. However, it is quite a long time since Docker-proxy joined Docker. Docker enthusiasts generally feel that in many scenarios, docker-proxy is not necessary, and even brings some other drawbacks.
There are two main kinds of scenes with large impact.
First, a single container needs to have multiple ports mapped to the host. In this scenario, if a container needs to map 1000 ports or more, 1000 or more docker-proxy processes are created on the host. According to the incomplete test, each docker-proxy occupies the memory is 4-10MB unequal. As a result, it can be a huge burden to consume at least 4-10GB memory directly, and at least 1000 processes, whether from system memory or from system CPU resources.
Second, many containers exist at the same time in the host, a single container mapping port very few. In this scenario, the consumption of host resources is not as violent as in the first scenario, and is a more chronic way of invading resources.
Today, the Docker Daemon introduces--userland-proxy this flag, giving the control of the above scenes to the user completely, the user decides whether to open, and also provides the flexibility for the proxy agent of the user's scene.
1.3. Docker EXEC command Add--user parameter, user controls the user in which Docker exec executes commands in the container
Since Docker 1.3.0 introduced Docker exec, the user's ability to manipulate the container has been greatly freed, and the container is no longer a running black box for the user. However, while Docker exec brings great benefits, we can see some of these flaws, and of course the Docker community is constantly perfecting Docker exec.
First, the process that Docker exec runs in the container runs with root privileges, and the security of the container is likely to get out of control while the permissions are inflexible. Parameter--user just compensates for this shortcoming. Secondly, the presence of Docker exec breaks the status of tree-like relationships in the container, and many of the concepts of Docker containers in the early stages of the design are based on the idea of the tree from the init process, so the current Docker The exec process is not managed by Docker Daemon exactly as the original ecosystem container process.
1.4. Enhance the configuration breadth of the Docker container gateway address
Before the Docker 1.7.0 was released, in bridge bridging mode, the gateway address of the Docker container was generated by default, typically the Docker0 bridge address in the Docker environment. From the point of view of container communication, the default mode is already sufficient. However, we can still see that this model has some drawbacks, such as the flexibility of network configuration and network security.
The network of Docker containers, which has been widely watched and lacks configurable features, is almost always closed in today's software development. –default-gateway and –default-gateway-v6 These two parameters, to a large extent, improve the flexibility of user-defined container network, the user more scene coverage, seems to be from the development of Docker if the shadow. Combined with recent releases, enhancements and richness of functionality, it is not difficult to guess, Docker's enterprise and production, has been a new level.
The default gateway setting, why is it related to the network security of the container? Over a long period of time, Docker0 as the gateway address of the container, which embodies the relationship between the container and the host in a very thorough way. The DOCKER0 acts as a network interface on the host, acting as a bridge between the container and the host. However, it is also the existence of a bridge that makes it easy for the container's internal processes to traverse the gateway to reach the host, which is not transparent to the user.
1.5. Container CFS quota Support
Perfect Docker support for kernel cgoups, refers to the time limit for a group of process groups to be dispatched by the kernel CFS scheduling algorithm within a single cycle, in microseconds. The corresponding file for this configuration item in Cgroups is/sys/fs/cgroup/cpu/cpu.cfs_quota_us.
1.6. Support for container disk IO limits
As we all know, the container will provide the user with an isolated running environment, the container internal process or process group use resources will be limited, such resources, including: memory resources (physical memory and swap), CPU resources (CPU time slices and CPU cores, etc.), disk space resources, etc. This part of the content is more or less, the new version of Docker can be implemented more or less, but the isolation dimension is still not perfect, this time Docker added the-blkio-weight parameter, to implement the container disk IO limit support. Isolation is more complete, and users no longer need to worry about the competition between containers for disk IO resources.
1.7. ZFS Support
Docker 1.7.0 officially announces support for ZFS file systems. This also means that the support for the Docker container file system has increased from 5 to 6 types. Previously, Docker supported AUFS,DEVMAPPER,BTRFS,OVELAYFS,VFS (for support of volume) and now adds support for ZFS. ZFS support makes it reminiscent of the flocker of volume plug-ins with the data volume of Docker. Wrong in the wrong, it seems that the relationship is more subtle.
It is worth mentioning that in addition to supporting ZFS, I found in the graph module responsible for the container file system, the addition of Driver_windows.go, although the content is extremely simple, not fully realize the overall support of Windows, But at least let everyone see the pace of Docker's support for Windows moving forward.
1.8. Feature extensions for Docker logs
Looking at the container log, I believe many Docker enthusiasts have experienced it, which is also an important basis for users to view the state of the container operation.
It's easy to understand how Docker container logs work: For each Docker container you create, Docker daemon creates a goroutine internally to listen to the standard output stdout of the container's internal processes and the standard error stderr, and deliver the content to the log file. Docker Daemon passes the contents of the log file to the Docker client display whenever the user initiates a request to view the container log through Docker client, Docker logs.
The development of Docker logs can be divided into 4 stages: Original eco-Log printing at the beginning of Docker, allowing the user to follow the log of the container, turning on the tail function of the container log, and the since function of the container log, printing the container log from the beginning of a timestamp.
Although the functionality of the container log is growing, it is undeniable that the container log is one of the largest modules of the container itself and the Docker daemon, and this involves the planning of the first Docker design, which is not perfect, but it is the most easy-to-use solution in a short time.
1.9. Container and host share the UTS namespace support
In different scenarios, the container can be completely isolated from the host, and the container may share information with the host, and the host mode of the Docker network is a good example of a container sharing a host's network namespace.
The support for sharing the UTS namespace means that the relationship between the container and the host is becoming more subtle. There may be many Docker enthusiasts who are accustomed to the complete isolation of containers from the host, and of course there are users who have complained that a completely isolated environment does not smoothly container legacy business. So, at present, Docker in the case of both, more to meet the needs of the latter, in the near future, the application of Docker container will be more abundant, which is the way Docker into the enterprise and production must be a journey.
Overall, the Docker 1.7.0 to the author's feelings are: functional gradually to the needs of enterprises, in the Production-ready on the road to optimize, in addition to security in the core is not involved in the basis of continuous improvement.
Just a few days after the release of the 1.7.0 Dockercon, Docker announced Production-ready and take a serious view of security, in fact, from the 1.7.0 changes in view, you can feel this.
In addition, it is necessary to mention that: the above 3, 5, 63 is a strong impetus from the domestic company Huawei to complete, as a docker developer, heartfelt thanks to Huawei and the numerous Docker committer contributions.
The second link: Dockercon Hackathon
June 7 8th, we Chinese developers in Beijing held a golang&docker hackathon,dockercon Eve, San Francisco also held a Docker Hackathon.
This time the hackathon was born with a lot of new ideas, and the Geek spirit of whale powder was vividly played. Because the content is too rich, I'll just introduce two projects that I'm very impressed with.
2.1. Entry: Swarm-sec
This is a swarm cluster security assessment tool
Scan the entire cluster with swarm, pre-defined security specifications. The security dimension of the evaluation is mainly three points: the security configuration of Docker daemon in Swarm, swarm the specific security policy usage of Docker node in the cluster, swarm the runtime security of the cluster.
This is how the Swarm-sec starts:
Docker run-it--net host--pid host--cap-add Audit_control \
-v/var/lib:/var/lib \
-v/var/run/docker.sock:/var/run/docker.sock \
-V/USR/LIB/SYSTEMD:/USR/LIB/SYSTEMD \
-v/etc:/etc--label swarm-sec \
Swarm-sec <token-id>
The network namespace, the PID namespace, almost all docker information is mounted inside the container, and a capability capability Audit_control is added to obtain information about the audit daemon.
My main introduction is SWARM-SEC's focus on swarm daemon and what security issues Docker daemon on Docker node.
Swarm Daemon
1. See if the swarm is running in a container,
2. See if Swarm is the latest stable version
3. Check the log level of swarm and do not recommend the use of debug mode
4. Verify that Docker's default port is occupied
5. Verify that the TLS secure transport protocol is enabled for swarm daemon
6. Verify that the container dispatch driver is Mesos and the Mesos drive is still in the pilot phase
7. See if SELinux and AppArmor are enabled
......
Docker Daemon
Verify that the label settings for the Docker node are standardized, and verify that the Docker daemon configuration hand is secure, such as not using AUFS.
Project Source: Https://github.com/snrism/swarm-sec
2.2. Entries: Sherdock Rancheros's works
The main features are:
Based on regular expressions, the Docker image is automatically recycled.
The reason why the GC is mirrored is because the build fails to produce a series of none mirrors, which may not be useful. You can now clean up such a mirror based on a given regular expression.
Do the orphan data volume cleanup based on Docker 1.7.0. (Orphan data Volume: A container with data volume does not specify the-v parameter when it is deleted)
Provides a UI interface.
Hackathon's reports share too much space, if you are interested, you can read: Dockercon's hacker Marathon News
The third Link: Dockercon festival
Sharing Dockercon, Dockercon the content of two days is the highlight. This part of the share, I mainly divided into two parts: first, keynotes content, and through a few key words to introduce; second, introduce the topic content of several sub-venues I attended.
3.1 keynotes Keywords
3.1.1 OCP
Docker, a large number of it giants, is building a completely open container standard called OCP, which means "open container project", and Huawei, China's company, is also in the list. The OCP is the world's first open and unified standard for the container technology community.
The birth of the OCP means that the dispute between Docker and CoreOS of the standard of container has been suspended. The openness of the OCP makes the standard of the container better.
The OCP enables the development of the container to be widely recognized and supported, the individual or team to extend the sharing of the container technology can be more thought, can be more practice, its development on the impact of the container is absolutely not a small estimate.
3.1.2 New Project
Experimental binary
Docker's experimental binary is a functional component that is very powerful but unlikely to be added to the official Docker version. This experimental construction allows users to self-reflect the new functionality of Docker and provide feedback to Docker maintainers. In this way, Docker officials want to pre-expose some of the new features to developers around the world, allowing them to reinvent these important features in practice and feedback.
At present, the network plug-in of Docker and the data volume volume plugin are the first experimental function plug-ins that meet with everyone.
Docker Plugins
The scalability of Docker was once again put on the agenda and practiced by Solomom and his team. At present, Docker's network module, database volume module, dispatch module in Swarm and Service discovery module have been gradually plug-in, which embodies the expansibility ability of the software in Docker ecosystem, and embodies the openness of Docker ecology. Of course, in addition to these already supported plug-ins, the future of plug-ins will certainly be more.
In addition, the use of plug-ins, there is no other patch introduction for Docker daemon and other software modules, however, there is still a need to restart Docker daemon or other software. In addition, plug-in support is completely transparent to Docker applications, and the introduction of plugins does not change the way the original Docker application is used.
Docker Notary for Security
The notary consists of the server side and client side, and the server side primarily runs and maintains a container's trusted collection, which the client side uses to interact with the server to eventually distribute the security tasks.
Overall, notary wants to provide a simple way for users to publish and validate network data more securely. In principle, we can guarantee the security of both sides of the communication through the TLS secure Transport layer protocol, however, this is inherently flawed, the reason is simple, any negligence on the server side will cause malicious content to replace the legitimate content, the former endangers network communication.
With the help of notary, publishers of network content can sign and encrypt user content with strong security keys. Once completed and ready to be published, the content is first pushed to the notary trusted collection. Only when a visitor obtains the publisher's public key through a secure tunnel can it communicate with any server in notary and access the appropriate and accurate resources.
Runc.io
Runc is a common runtime environment at the OS level and is only operating at the OS level. Project address: [Https://runc.io]
Docker officially claims Runc as well as through comprehensive testing, and production-ready.
In terms of functionality, RUNC supports all Linux security features, such as Selinux,apparmor,cgroups,seccomp.
- Support for user namespace currently not supported by Docker
- Supports hot migration of containers.
- Microsoft is actively supporting Windows for Runc.
- The support of the arm system is also being actively progressing.
- Intel is also contributing to DPDK
- A standardized, portable, and operational format is defined
- Compatible with command line forms and programmatically
production of 3.1.3
Enhancements and improvements to Docker hub Online Services
Upgrade to V2 Open source registry
Optimized backend for faster download times, fewer customer service/server-side interactions
Improve stability and improve overall user experience
The new UI interface
Docker has made a lot of efforts to improve security, including
Authentication Micro Services
Content-addressable Images
One time use build hosts
On-ging scanning and audits and so on.
Ocra
Project Orca's vision is to provide a top-down consolidation stack that captures all tools and plumbing (Docker Engine, Docker Swarm, Networking, GUI, Docker Compose, security, installation tools, deployment, configuration, etc.). Project Orca is ideal for the "run" part of "Build-ship-run".
Represents Docker's real foray into the enterprise market, the privatization of container management solutions Docker launches solution.
3.2 minutes to share the venue
3.2.1Docker of Security
The security coverage provided by Docker will cover the following 7 points:
- Isolation of namespace protection system resources
- Cgroup to complete the Process group resource limit
- Linux Security module provides Mac (AppArmor, SELinux)
- Capabilities to subdivide root permissions into multiple types
- Ulimit features more restricted resources for dimensions (after Docker 1.6 support)
- User namespace protection inside the container root is not root on host (expected Docker 1.8 support)
- Seccomp allows the process within the container to be controlled by system invoke permissions
Docker also does the following in terms of security:
Reduces the security direction of the mirror
To set up a more streamlined Linux distribution:
Removing unwanted package, user, and binary tool files
Proposed Tailored Profiles
Containers can create Least-privildge files
Files have the ability to pass with containers
All permissions are indicated by a separate file configuration
has been planned into Runc
Describe all the isolation characteristics
It is worth mentioning: Dockercon The security topic of the audience, thanks to the Huawei company from China.
3.2.2 Netflix's Docker enterprise-class practices
Why Choose Docker?
1. There is a high demand for internal process isolation within Netflix
2. Docker has the ability to combine many features into a single Docker binary that users can do without needing to be concerned with the dependencies between the Linux kernel and other features, simply by manipulating Docker's simple API
3. The development of the Docker's peripheral toolchain and the scalability of application releases between clusters have greatly attracted Netflix
Q&a
1. Previous Docker-proxy This process seems to be just for port mapping, and whether each container occupies a process, is it possible for daemon to have only one process, responsible for maintaining all port mappings on this daemon?
A: After shutting down the userland-proxy, there is no effect on the communication between the containers and the container's access to the outside world. But the influence always exists, and the existence of docker-proxy is always valuable. The effect is that the port of the container cannot be accessed through 0.0.0.0 and localhost on a single host, but it can be communicated through the host IP. A single process to maintain, the load will be very large, and so that the process and all containers have a large coupling, there are some drawbacks.
2. "added a feature that is still in the experimental phase: the data volume plug-in that supports out of process" what does this specifically provide?
A: Simply put, this plugin has the ability to manage the data volume of the container, including the functions of storage backup, recovery, etc.
3. Will the final presentation of experimental binary be docker plugin?
A: My understanding is "not exactly", experimental binary can also be a function of the interior of Docker Daemon, and then announce that it is experimental binary, let all the Docker users to experience, to test, To help improve Docker, plug-ins are also a form.
4. Can you describe the progress of the container support situation on the Windows platform and look at the display of Microsoft?
A: Microsoft has its own container solution, not Docker, and now it's time for Microsoft to start slowly supporting Docker, a program that is fully visible at the start of the Windows Support Docker API, The actual implementation of Docker has also appeared in Windows, but to a full realization, it may take a long time.
5. Will the next default gateway be the new feature of 1.7.0, can you give me a little more detailed introduction?
A: The original Docker container gateway address is generally fixed, are Docker0 bridge address, Docker0 is located in the host, so that the bridge between the container and host Docker0 is easy to use, through the container to the host's access. Set Default-gateway for the container, you can bypass the DOCKER0, although the physical traffic is still going through DOCKER0, at least the user is not visible.
Learn more about Dockercon Live content
1.Docker 1.7.0 Depth Analysis
2. Dockercon's hacking marathon
3. What else did you say besides Ocp,dockercon's keynote speech?
4. There is a picture of the truth: Dockercon Netflix's Docker enterprise-class practice
5. Docker practice in the US financial industry
6. Dockercon Day 2: Enter the production environment!
7. Dockercon Day 2: Security first
8. Dockercon Day 2: City dwellers play mirror
9. Dockercon Day2: Closing + eggs
=====================================================
The above content is organized according to the July 2, 2015 night Group sharing content. Share people
Liang, Daocloud software engineer, InfoQ "Docker source analysis" columnist. Contact Docker for nearly two years, love to delve into the principles of system implementation, understand a little Linux operating system. Dockone Weekly will organize a targeted technology sharing, welcome interested students Please add: liyingjiesx participate in sharing and learning.
