Mesos cluster installation Deployment planning, preparation (1)

A: Introduction

Mesos, a research project that was born in UC Berkeley, has now become a project in Apache incubator. Mesos COMPUTE Framework A cluster manager that provides efficient, resource isolation and sharing across distributed applications or frameworks that can run Hadoop, MPI, hypertable, and Spark. Use zookeeper for fault-tolerant replication, use Linux containers to isolate tasks, and support multiple resource planning allocations.

1: Overall architecture

Apache Mesos consists of four components, namely mesos-master,mesos-slave,framework and executor.

Explanation of basic terms in Mesos

(1) Mesos-master: Is the core of the entire system, responsible for the management of access to the Mesos of the various framework (by Frameworks_manager Management) and slave (managed by Slaves_manager), The resources on the slave are assigned to the framework (managed by the independent plug-in module allocator) according to a policy.

(2) Mesos-slave: Responsible for receiving and executing commands from Mesos-master, managing mesos-task on nodes, and assigning resources to individual tasks. Mesos-slave sends its own amount of resources to mesos-master, which is determined by the allocator module in Mesos-master to which framework the resource is allocated, and the resources currently under consideration are both CPU and memory, that is, Mesos-slave will send the number of CPUs and the amount of memory to Mesos-master, and when the user submits the job, you need to specify the number of CPUs and the amount of memory required for each task, so that when the task runs, Mesos-slave will put the task into Linux with fixed resources Run in container to achieve the effect of resource isolation.

(3) Framework: refers to the external computing framework, such as Hadoop,mesos, these computing frameworks can be registered to access the Mesos, in order to Mesos unified management and resource allocation. Mesos requires that the accessible framework must have a scheduler module that is responsible for task scheduling within the framework. When a framework wants to access mesos, it needs to modify its scheduler to register with Mesos and get Mesos assigned to its own resources so that its own scheduler assigns those resources to tasks in the framework, which means The entire Mesos system uses a two-tier scheduling framework:

(4) Executor: Actuator, mounted on mesos-slave, to start a task in the compute frame.

The composition of the Mesos frame is as follows:

II: Installation Deployment

1: Introduction to the deployment environment:

Kernel version:

[Email protected] ~]# uname-a

Linux ctn-7-11.ptmind.com 3.10.0-327.4.4.el7.x86_64 #1 SMP Tue Jan 5 16:07:00 UTC x86_64 x86_64 x86_64 gnu/linux

System version:

[Email protected] ~]# more/etc/redhat-release

CentOS Linux release 7.2.1511 (Core)

Docker version:

[email protected] ~]# Docker--version

Docker version 1.10.3, build 20F81DD

Cluster all hosts add host name

[Email protected] ~]# more/etc/hosts

---slightly---

172.16.7.12 ctn-7-12.ptmind.com ctn-7-12

172.16.7.13 ctn-7-13.ptmind.com ctn-7-13

172.16.7.14 ctn-7-14.ptmind.com ctn-7-14

For configuration easy to speak server ssh password-free login

[[email protected] workspace]# for IP in $ (cat/workspace/mesoslist);d o echo------$ip-----; ssh [email protected] $ip HOSTN Ame;done

------172.16.7.11-----

Ctn-7-11.ptmind.com

------172.16.7.12-----

Ctn-7-12.ptmind.com

------172.16.7.13-----

Ctn-7-13.ptmind.com

------172.16.7.14-----

Ctn-7-14.ptmind.com

2: Prepare before installation:

Shutting down the firewall

sudo systemctl stop Firewalld

sudo systemctl disable FIREWALLD

Installing Docker

sudo yum update

Curl-ssl https://get.docker.com/| Sh

Install common software packages

Yum install git lrzsz wget elinks nc-y

Add Yum source (all hosts, install one of them)

RPM-UVH http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm

RPM-UVH http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-2.noarch.rpm

RPM-UVH http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-3.noarch.rpm

3: Cluster service deployment plan:

Server IP address host name installation service

172.16.7.12ctn-7-12.ptmind.com mesos-master, zookeeper, Mesos-slave, Chronos

172.16.7.13ctn-7-13.ptmind.com mesos-master, zookeeper, Mesos-slave,

172.16.7.14ctn-7-14.ptmind.com mesos-master, zookeeper, Mesos-slave, marathon

Installation Software Release Description:

Install common package versions

marathon 0.15.3-1.0.463.el7

Mesos-maser/mesos-slave 0.28.0-2.0.16.centos701406

Mesosphere-zookeeper 3.4.6-0.1.20141204175332.

Install Mesos and related software (all hosts)

Yum-y Install Mesos Mesosphere-zookeeper

Mesos cluster installation Deployment planning, preparation (1)

Mesos cluster Installation Deployment Zookeeper (2)

Mesos cluster Installation Deployment Mesos-master (3)

Mesos cluster Installation Deployment Marathon (4)

Mesos cluster Installation Deployment Chronos (5)

Mesos cluster Installation Deployment Mesos-slave (6)

Mesos cluster Installation Deployment Mesos-dns (7)

Mesos cluster basic function test (8)

Mesos cluster Cleanup (9)

This article from "Kang Jianhua" blog, reproduced please contact the author!

Mesos cluster installation Deployment planning, preparation (1)