Overview and architecture of puppet

1. Overview

Puppet is an open source software automation configuration and Deployment tool, which is easy to use and powerful, and is getting more and more attention, and now many large IT companies are using puppet to manage and deploy the software in the cluster. If Google uses puppet to manage more than 6000 platform Mac desktops (2007 data).

This paper mainly introduces puppet installation method, design structure and usage method.

2. Design Architecture

Puppet is based on the C/S architecture. The server side holds all the configuration code for the client server, which is called manifest in puppet. After the client downloads manifest, the server can be configured according to manifest, such as package management, user management, file management, and so on.

650) this.width=650; "class=" AlignCenter size-full wp-image-632 "title=" Infrastructure "src=" http://dongxicheng.org/ Wp-content/uploads/2011/05/infrastructure.jpg "height=" width= "408"/>

As shown, the puppet workflow is as follows: (1) The client puppetd calls Facter,facter to detect some variables of the host, such as hostname, memory size, IP address, etc. Pupppetd this information through the SSL connection to the server side, (2) the server side of the puppetmaster detect the host name of the client, and then find the corresponding node in the manifest configuration, and this part of the content to parse, Facter sent to the information can be used as variable processing, node involved in the code to parse, and other non-involved code does not parse. Parsing is divided into several stages, grammar checking, and error if grammatical errors. If the syntax is correct, continue to parse, the result of the resolution generates an intermediate "pseudo-code", and then send the pseudo-code to the client, (3) The client receives "pseudo-code", and executes, the client sends execution results to the server, and (4) the server side writes the execution result of the client to the log.

Puppet two points worth noting in the process of work, first, in order to ensure security, between the client and master is based on SSL and certificate, only the Master Certificate authentication client can communicate with master; Puppet will keep the system in a state that you expect to persist, such as detecting a file and ensuring it persists, ensuring that the SSH service is always on, if the file is deleted, or if the SSH service is turned off, puppet next execution (default 30 minutes), The file is recreated or the SSH service is started.

3. Software Installation

Installing with the Apt-get command is not recommended because there is a bug in the puppet downloaded by the command. Can be installed directly from the source code, need to install the software has Ruby,facter and puppet.

3.1 Installation Steps

Edit/etc/host to modify the hostname, because puppet is certificate-based and the certificate contains the host name;

Install Ruby, Facter, and Puppet on Master and slave, and use Ruby Puppet when installing Facter and install.rb.

3.2 Post-installation directory structure

(1) Installation directory

The installation directory is saved as/etc/puppet by default, and manifests in this directory holds manifest files.

Other executables under/user/sbin, mainly include:

Puppet: Used to perform independent mainfests files written by the user, such as:

Puppet-l/tmp/manifest.log manifest.pp

PUPPETD: A client program running on a managed host, such as:

Puppet–server Servername–waitforcert 60

Puppetmasterd: A server program running on a management machine, such as:

Puppetmasterd–debug

PUPPETCA Puppet certification program, mainly used for certification of slave certificates, such as:

View Slave:puppetca–list to be certified

To certify these slave: puppetca-s–a

Puppetrun is used to connect the client, forcing the local configuration file to run, such as:

Puppetrun-p 10–host host1–host host2-t remotefile-t webserver

(2) configuration file

Puppet.conf

Puppet master configuration file, if it is root user, profile is/etc/puppet/puppet.conf, normal user, profile is: ~user/.puppet/puppet.conf

For specific configuration parameters, see:

Http://docs.puppetlabs.com/references/stable/configuration.html#configuration-files

Fileserver.conf

Puppet the file server configuration file. Using path to configure the file path, Allow/deny Configure access rights, see: Http://docs.puppetlabs.com/guides/file_serving.html

3.3 Verifying that the installation is successful

Select a slave to validate with master, assuming that the host for the Slave is Masterhost, and the Slave00,master is entered on the slave00:

Puppetd–test–server servername

Then look at the certified slave on the Masterhost:

Puppetca–list

If this is not a problem, you can now see slave00, signing the slave certificate:

Puppetca-s-A

This allows the slave00 to pass the certificate validation and can further interact with master.

Write the site.pp file under Masterhost's/etc/puppet/manifests directory, as follows:

Node Default {

File {

"/tmp/test":

Content=> "hello\n",

mode = 0644;

}

}

At the same time on the slave00 input: puppetd–test–server servername, view slave00 's/tmp folder, generated a new file test, the content is hello, the file's permissions are-rw-r-r-. This proves that the puppet installation was successful and if an error occurs, see section sixth.

4. Configuration scripting

This section describes the configuration scripting method for Puppet, mainly refers to the puppet manifest writing method. Puppet abstracts the content that needs to be managed into a resource, each resource has different attributes, so puppet language is the language that describes the properties of these resources and the relationships between the resources.

For ease of management, puppet the resource modularity, i.e. the manifest of each function module is placed in a single directory. Each module contains a main manifest file (INIT.PP, which is the module's entry, similar to the C language of the main function), which contains several classes to encapsulate the module's resources, common resources such as file,package,service, each resource by its own attributes, such as Fil E has attribute name,owner,mode and so on.

This section mainly introduces the writing methods of manifest in puppet, and then introduces the methods of writing resource attributes, resources, node management, functions and modules in turn.

4.1 Resource Properties

There are two kinds of resource attributes, one is resource-exclusive attribute, the other is the common resource attribute, and the resource-exclusive attribute is described in the next section, and the resource common attribute is a property shared by all resources, mainly:

Before

Used to control the execution order relationship of different objects (resources), which means that an object (resource) Occurs after another object (require, in contrast, represents a previous occurrence). Such as:

file {"/var/nagios/configuration":

Source = "...",

Recurse = True,

before = exec["Nagios-rebuid"]

}

exec {"Nagios-rebuild":

Command = "/usr/bin/make",

CWD = "/var/nagios/configuration"

}

This code guarantees that all code is up-to-date before it is compiled with make. You can also before multiple resources, such as:

before = [file["/usr/local"], file["/usr/local/scripts"]

Subscribe

Detects a resource that, when it changes, reloads the resource, such as:

Class Nagios {

file {"/etc/nagios/nagios.conf":

Source = "Puppet://server/module/nagios.conf",

alias = nagconf # just to make things easier for me

}

Service {nagios:

ensure = running,

Subscribe = file[nagconf]

}

}

When a file nagconf is detected, the service Nagios is updated accordingly. It is important to note that the resources currently supporting subscribe are only Exec,service and mount.

For more information, see: http://docs.puppetlabs.com/references/latest/metaparameter.html

4.2 Resources

The main resources are as follows:

File: Document management

Package: Packages Management

Service: System service Management

Cron: Configuring Recurring tasks

EXEC: Run shell command

(1) file resource

Further details can be found at: http://puppet.wikidot.com/file

(2) Package Resources

Further details can be found at: http://puppet.wikidot.com/package

(3) Service resources

Further details can be found at: http://puppet.wikidot.com/srv

(4) Exec resources

Further details can be found at: http://puppet.wikidot.com/exec

(5) Cron Resources

Further details can be found at: Http://puppet.wikidot.com/cron

4.3 Node management

Puppet how to differentiate between different clients and assign manifest to different service sides? Puppet uses node resources to do this, and node is followed by the host name of the client, for example:

Node ' slave00 ' {

Include SSH

}

Node ' Slave11 ' {

$networktype = "Tele"

$nagioscheckport = "80,22,3306″

Include Apache, MySQL, PHP

}

Variables can be used in resource node, and other manifest can be included directly through the include.

Further details can be found at: http://docs.puppetlabs.com/references/latest/type.html

4.4 Classes and functions

A class can have multiple related resources defined together to form a class. Classes can inherit, see also: http://docs.puppetlabs.com/guides/language_guide.html#resource-collections

Functions (called "Defination" in puppet) can wrap multiple resources into a single resource, or package a resource into a model for ease of use. For example, it is very easy to manage an Apache virtual machine in Debian, put a virtual host's configuration file inside/etc/sites-available/, and then make a symbolic link to the/etc/sites-enabled directory. You can copy the same configuration code for each of your virtual hosts, but it's better and easier if you use the following code:

Define Virtual_host ($docroot, $ip, er = $, $ensure = "Enabled") {

$file = "/etc/sites-available/$name. conf"

# The template fills in the Docroot, IP, and name.

File {$file:

Content = Template ("Virtual_host.erb"),

notify = Service[apache]

}

file {"/etc/sites-enabled/er-$name. conf":

ensure = $ensure? {

Enabled = $file,

Disabled = Absent

}

}

}

You can then use this definition to manage an Apache virtual host, as shown in the following code:

virtual_host {"reductivelabs.com":

Order = 100,

ip = "192.168.0.100″,

Docroot = "/var/www/reductivelabs.com/htdocs"

}

4.5 Modules

A module is a directory under the/etc/puppet/modules directory and its subdirectories, in the puppet main file site.pp with import modulename can be inserted into the module. The new version of Puppet can be automatically inserted into the module under the/etc/puppet/modules directory. The introduction of modules, can be structured code, easy to share and manage. For example, all configurations of Apache are written under the Apache module. A module directory typically consists of three directories: Files,manifests,templates. Manifests inside must include a init.pp file, which is the initial (entry) file of the module, when importing a module, will be executed from INIT.PP. All the code can be written into the init.pp, can also be divided into several pp files, init to include other files. The files directory is the file publishing directory for this module, and Puppet provides a file distribution mechanism similar to the Rsync module. The templates directory contains the ERB model file, which is related to the template property of the file resource.

Puppet after installation, the modules directory is not, you build a line, and then in the inside can add new modules.

5. Programming examples

5.1 Hello World

This section describes a very simple programming example: a slave obtains its manifest from master, and the maniftest requires slave to do the following: Install GCC, create a folder/home/dxc/test, Download file hello.c program, compile hello.c.

(1) Code structure Organization

The directory structure of the code on Master is as follows:

|–auth.conf

|–fileserver.conf #puppet文件服务器配置文件

|–manifests #puppet主文件所在目录

| |–MODULES.PP #puppet各个模块汇总

| |–nodes #各个slave要处理的模块

|   | ' –exechello.pp #hello模块对应由那些slave处理

| ' –site.pp #puppet主文件 (entry file)

|–modules #puppet的各个模块所在文件

| ' –hello #hello模块

| |–files #该模块对应的文件资源, possibly a configuration file to send to slave, etc.

|   | ' –HELLO.C

| ' –manifests #模块的manifest文件

| ' –init.pp #模块入口文件

' –ssl #puppet的证书文件目录

(2) Program execution flow

The sequence of code calls is:

Slave Initiating connection request ÀSITE.PPÀNODESÀMODULESÀINIT.PP

First, the slave initiates a master connection request and authenticates the certificate;

Then, after the certificate is validated, Master will directly find the site.pp file in the manifests directory of the portal file, which may contain some global variables, the default values of the parameters (when the modules do not have these parameters set, their default values), and other PP file calls (in this case, the modules.pp and nodes are called for each pp file);

The master then navigates through the various PP files under the nodes to the module to be executed by the slave (INIT.PP is the entrance to each module), and summarizes the module code back to slave;

Finally, slave the configuration information according to the manifest that the master sent over.

(3) Code interpretation

Download the code directly here .

5.2 A more complex instance

This section describes a more complex company that is working with instances where the puppet code layout is consistent with the previous instance, except that the instance involves more modules and more complex dependency management. The specifics of the code are not explained in this section, see Code .

6. Issues that may be encountered

Q:puppet's certificate mechanism

A:puppet certificate problem is the most difficult problem for beginners, here is how to deal with. The puppet server will automatically produce a root certificate and a server certificate when it is installed or on its first boot, and the certificate and hostname are related, so if you change the hostname after the certificate is generated, there will be a problem. The puppet client will automatically generate the certificate at the first boot, but the certificate needs to be signed by the puppet server, so the puppet client will send a certificate request when it first connects to the server, and the server side needs to sign the certificate. The puppet client will download the signed certificate the next time it connects to the server.

Q:ubuntu The following certificate error, how to resolve?

A: This method is provided to the novice test environment, the generation environment is not recommended to do so. Delete the/var/lib/puppet/ssl directory first on the puppetmaster (server side), then start Puppetmasterd, and then delete the/var/lib/puppet/ssl directory on the client side. Write the hostname of the puppetmaster machine and the corresponding IP address to the/etc/hosts of the client machine.

Then execute: Puppetd–test–server server.example.com. Replace the server.example.com

Replace it with your own server host name. Execute this command, there will be a message, do not bother.

Then log on to the puppetmaster server machine, execute the puppetca–list command to see if there is a client's certificate request, and if not, check that the previous steps are correct and that the network connection is healthy. If Puppetca–list can see the request, execute the puppetca-s-a command and sign all certificate requests. Finally go back to the puppet client machine and execute

Puppetd–test–server server.example.com.

You'll be able to establish a connection if your site.pp is written. You can test the puppet.

Add: If the time between client and server is inconsistent, it will cause certificate authentication to fail, so it is necessary to check whether the time of the two machines is consistent when the certificate problem occurs, if inconsistent with the date command or the ntpdate command to keep the time of the two machines consistent.

Q: Error [Puppet Users] err:could not retrieve catalog; Skipping run

A: This may be due to the fact that two versions of Ruby or Facter have been installed, as shown in the solution:

https://projects.puppetlabs.com/issues/5279

7. Summary

As server clusters grow larger, automating the configuration and deployment of these servers can make management very easy and significantly reduce the cost of managing deployment, so it is highly valued by IT companies.

This document describes puppet, a new type of software automation configuration and Deployment tool. The main content of this paper relates to the architecture, installation and usage of puppet, and gives two usages.

In a large-scale build environment, if only one puppetmaster will not be busy, because Puppet is written in Ruby, Ruby is an analytic language, each client to access, all to parse once, when the client is too busy to come, so need to expand into a server group. Puppetmaster can be seen as a Web server, which is actually done by a Web server module provided by Ruby. Therefore, the Web proxy software can be used to do cluster setup with puppetmaster, see: Http://puppet.wikidot.com/puppetnginx.

8. References

Puppet official website: http://www.puppetlabs.com/

Puppet Chinese wiki:http://puppet.chinaec2.com/

Puppet Chinese Blog: http://www.comeonsa.com

9. Code download

(1) 5.1 section example code download

(2) 5.2 Section example code download

Overview and architecture of puppet