Puppet uses Nginx multiple ports for load balancing

As the company's application demand increases, the number of servers is also increasing. as the number of servers increases, we will find that a puppetmaster is under great pressure and resolution is slow, in addition, there may be errors such as "timeout" from time to time. Is there any optimization method? We found a solution on the official Puppet website. we found that puppetmaster can configure multiple ports and combine them with WEB proxy (Nginx is recommended). In this way, puppetmaster's affordability can be improved by at least several times, as the company's application demand increases, the number of servers also increases. as the number of servers increases, we will find that a puppetmaster is under high pressure and resolution is slow, in addition, there may be errors such as "time out" from time to time. Is there any optimization solution? We found a solution on the official Puppet website. we found that puppetmaster can configure multiple ports and combine them with WEB proxy (Nginx is recommended). In this way, puppetmaster's affordability can be improved by at least several times, this is equivalent to greatly optimizing puppet's processing capabilities. 1. following the preceding environment settings, the server environment and software version here are: Server System: CentOS5.8 x86_64 Ruby Version: ruby-1.8.5 Puppet version: puppet-2.7.9 Nginx version: nginx-0.8.462.Mongrel installation to use puppet multi-port configuration, you need to specify the mongrel type, the default is not installed, you need to install: yum install-y rubygem-mongrel3. configure puppetmaster to add the following two lines at the end of the/etc/sysconfig/puppetmaster file, representing the multi-port and mongrel types, as shown in the following figure: PUPPETMASTER_PORTS = (8141 8142 8143 8144 8145)
PUPPETMASTER_EXTRA_OPTS = "-- servertype = mongrel -- ssl_client_header = HTTP_X_SSL_SUBJECT" 4. before installing the Nginx service, make sure that the pcre-devel regular library has been installed in the system, and then compile and install Nginx. you need to add the SSL module parameter support. The Nginx installation process is as follows: yum-y install pcre-devel
Cd/usr/local/src
Wget http://nginx.org/download/nginx-0.8.46.tar.gz
Tar zxvf nginx-0.8.46.tar.gz
Cd nginx-0.8.46
./Configure -- prefix =/usr/local/nginx -- with-http_ssl_module
Make & make install & cd ../add the www user group and user. the command is as follows: groupadd www
Useradd-g www www5. we modify the configuration file nginx. conf based on puppet requirements. the content is as follows: user www;
Worker_processes 8;
Events {
Worker_connections 65535;
}

Http {
Include mime. types;
Default_type application/octet-stream;

Sendfile on;
Tcp_nopush on;

Keepalive_timeout 65;
# Define puppet client access puppet-server log format
Log_format main '$ remote_addr-$ remote_user [$ time_local] "$ request" $ request_length $ request_time $ time_local'
'$ Status $ body_bytes_sent $ bytes_sent $ connection $ msec "$ http_referer "'
'"$ Http_user_agent" $ http_x_forwarded_for $ upstream_response_time $ upstream_addr $ upstream_status ';
Access_log/usr/local/nginx/logs/access. log main;

Upstream puppetmaster {
Server 127.0.0.1: 8141;
Server 127.0.0.1: 8142;
Server 127.0.0.1: 8143;
Server 127.0.0.1: 8144;
Server 127.0.0.1: 8145;
}

Server {
Listen 8140;
Root/etc/puppet;
Ssl on;
Ssl_session_timeout 5 m;
# The following is the puppetmaster server certificate address.
Ssl_certificate/var/lib/puppet/ssl/certs/server.cn7788.com. pem;
Ssl_certificate_key/var/lib/puppet/ssl/private_keys/server.cn7788.com. pem;
Ssl_client_certificate/var/lib/puppet/ssl/ca/ca_crt.pem;
Ssl_crl/var/lib/puppet/ssl/ca/ca_crl.pem;
Ssl_verify_client optional;
# File sections
Location/production/file_content/files /{
Types {}
Default_type application/x-raw;
# Define the puppet push path alias
Alias/etc/puppet/files /;
}
# Modules files sections
Location ~ /Production/file_content/modules/. + /{
Root/etc/puppet/modules;
Types {}
Default_type application/x-raw;
Rewrite ^/production/file_content/modules/(. +)/(. +) $/$1/files/$2 break;
}
Location /{
# Redirect to puppetmaster server load balancer
Proxy_pass http: // puppetmaster;
Proxy_redirect off;
Proxy_set_header Host $ host;
Proxy_set_header X-Real-IP $ remote_addr;
Proxy_set_header X-Forwarded-For $ proxy_add_x_forwarded_for;
Proxy_set_header X-Client-Verify $ ssl_client_verify;
Proxy_set_header X-SSL-Subject $ ssl_client_s_dn;
Proxy_set_header X-SSL-Issuer $ ssl_client_ I _dn;
Proxy_buffer_size 10 m;
Proxy_buffers 1024 10 m;
Proxy_busy_buffers_size 10 m;
Proxy_temp_file_write_size 10 m;
Proxy_read_timeout 120;
}
}
} 6. after modifying the nginx. conf file, how should we start nginx and puppet-server? 1. shut down the puppetmaster process and start nginx first. otherwise, nginx will fail to be started. the command is as follows: after/usr/local/nginx/sbin/nginxnginx occupies the default port 8140 of puppetmaster, run the following command to check whether Port 8140 is taken over by nginx: lsof-I: 8140 the COMMAND shows that 8140 is taken over by the nginx process, as shown below: command pid user fd type device size/OFF NODE NAME
Nginx 4121 root 6u IPv4 20668 0t0 TCP *: 8140 (LISTEN)
Nginx 4122 www 6u IPv4 20668 0t0 TCP *: 8140 (LISTEN) Let's start puppetmaster again. The command is as follows: service puppetmaster start if ruby version is 1.8.5, the following warning will be reported when puppetmaster is run, starting puppetmaster:
Port: 8141 ** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
[OK]
Port: 8142 ** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
[OK]
Port: 8143 ** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
[OK]
Port: 8144 ** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
[OK]
Port: 8145 ** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
[OK] this warning value indicates: It's just a warning. mongrel wants a Ruby version of at least 1.8.6.But it still runs just fine with previous versions. just ignore the warning. mongrel requires ruby at least 1.8.6 and later versions, but it is still running in the current version. Please ignore the current warning. to ensure the stability of the entire puppet runtime environment, I will continue to use ruby 1.8.5 here.