Build a kafka cluster environment in a docker container
Kafka cluster management and status saving are implemented through zookeeper. Therefore, you must first set up a zookeeper cluster.
Zookeeper cluster Construction
I. software environment:
The zookeeper cluster requires more than half of the nodes to survive for
Kafka Cluster management, state saving is realized through zookeeper, so we should build zookeeper cluster first
Zookeeper Cluster setup
First, the SOFTWARE environment:
The zookeeper cluster requires more than half of the node to survive to be externally serviced, so the number of servers should be 2*n+1, where 3 node is used to build the zookeeper cluster.
1.3 Linux servers are created using the Docker
Implementation Architecture
A scenario implementation architecture is shown in the following illustration:
Analysis of 3.1 producer layer
Service assumptions within the PAAs platform are deployed within the Docker container, so to meet non-functional requirements, another process is responsible for collecting logs, thus not intruding into service frameworks and processes. Using flume ng for log collection,
not start the server;5. Kafka_server,message_producer,message_consumer are connected to the Zookeeper container via the link parameter, and Message_producer is also connected to Kafka server, Because the IP address of Kafka server is used when sending messages;
Now open the terminal, in the DOCKER-COMPOSE.YML director
1, from the container inside copy files to host.
Answer: Execute the following command in the host
Docker CP container name: The path of the file to be copied inside the container to be copied to the appropriate path of the host
Example: Assuming that the container is name
After the front upload files to the server, the server returned to the foreground of the file list in Chinese garbled, all the Chinese file name into? , the English file name is displayed normally.
The problem is positioned to find that the Multipartfile class of the background code has this exception when executing the Transterto method, but Multipartresolver and encodingfilter in the configuration file set are already UTF-8. Excludes code exceptions. KUBECTL exec enters into the
Docker container data management and operation tutorial, docker container Data Management
Docker data volumes
What is a data volume?
Features of data volumes
Relationship between data volumes and containers
Add volumes for containers
Add permissions for data vo
I just spent 3 hours last night reading the Journal: a unified concept of real-time data that every software engineer should know about.Today, Kafka is running in a Docker container, several on GitHub, but it's all too complicated.I'll write the simplest Python demo experience for you: Https://github.com/xuqinghan/docker
Install vim In the docker container, and install vim In the docker container
When using docker containers, sometimes vim is not installed. When you press the vim command, the system prompts vim: command not found. At this time, you need to install vim, however, when you pre
(1) Pull a new CentOS image for Docker pulls centos:7.2
(2) Running a container, ready to install the necessary environment
Docker run--privileged--dns 8.8.8.8--dns 8.8.4.4-h controler--name slurm_control-i-t-v/container_data/:/data C Entos:centos7/bin/bash
(3) Install the SSH environment inside the container
Yum Insta
port 80 on the respective container.
User namespaces
Starting with Linux 2.6.23 completed on Linux 3.8)
User and group ID spaces
The user and group IDs of the processes in user namespace can be different from the host, and each container can have a different user and group ID, and a non-privileged user on a host can be a privileged user in the user namespace ;
The conc
nvidia-dockeris a can be GPU used docker , nvidia-docker is docker done in a layer of encapsulation, through nvidia-docker-plugin , and then call to docker on, its final implementation or on docker the start command to carry some
Install, run, and remove docker applications to run and save docker containers
1. Run and save the nginx Server Based on the Ubuntu docker container. Install the nginx daemon to the Ubuntu startup container:
# Docker run Ubuntu ba
Three major docker components: container and docker1. containers are another core concept of Docker.
Simply put, containers are one or more applications that run independently and Their runtime environments. The virtual machine can be understood as a simulation.A complete set of operating systems (including the running environment and other system environments) a
start container docker run-ti Ubuntu/bin/bash
Docker run = First Docker Create and then Docker start
-T means to have Docker assign a pseudo terminal and bind to the container's standard input
When the-T is used, the command
-round.
3 Implementing the Architecture
A schema implementation architecture is shown in the following figure:
Analysis of 3.1 producer layer
The service assumptions within the PAAs platform are deployed within the Docker container, so in order to meet the non-functional requirements, another process is responsible for collecting logs and therefore does not invade the service framework and processes. U
Docker Quick Start series (2): container concepts and related operations, docker Quick StartWhat is container
To put it simply, an image requires a carrier, which is a container. As mentioned in the previous article, an additional writable file layer will be started after th
.
Network namespaces
Started with Linux 2.6.24 completed on Linux 2.6.29
Network-related system resources
Each container has its own network device, IP address, IP routing table,/proc/net directory, port number, and so on. This also allows the same application in multiple containers on a host to be bound to port 80 on the respective container.
User namespaces
one directly in:
Docker images view downloaded Docker mirrors:
Docker Run-tid--name ubuntu-p 23:22 ubuntu: Name the new Docker as Ubuntu and map to Port 23
NETSTAT-APNL | grep 23 See if Port 23 is open
You can see that port 23 is open.
Docker Exec-ti Ubuntu/bin/bash
This article focuses on the Docker image, container, warehouse basic concepts of knowledge.
Docker concept
Docker is an open source application container engine that allows developers to package their apps and dependencies into a portable
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.