Keepalived + nginx high-availability environment, keepalivednginx

Keepalived + nginx high-availability environment, keepalivednginx

There are actually many high-availability solutions, and the cost is very low and simple. For example, if you have a website, the simplest high availability is nds + nginx + two machines.

Because we are a customer's project, dns application is very troublesome, so we need to consider using keepalived + nginx for high availability solutions.

Common HA combinations are sorted out from the Internet.
Heartbeat v2 + crm
Heartbeat v3 + pacemaker
Corosync + pacemaker
Cman + rgmanager
Keepalived + lvs.

Here we don't use lvs, but directly use nginx. We feel that there are only two machines, so there is no need to make it too complicated. It can also meet the requirements.

I think it is important to plan before installation. We didn't plan well before, but now it feels messy and the customer upgrading is also troublesome. A bunch of processes.

Our system is developed in java, so tomcat is used for background parsing.

The first is directory planning, such as where tomcat is put and where static files are put. Generally, static files after nginx are parsed by nginx, which can relieve a lot of tomcat pressure.

Preparation before nginx installation, because nginx installation is still relatively simple, but the problem is that nginx has many plug-ins. Whether these plug-ins need to be installed needs to be studied. Nginx is not very good at session processing, so we need to add a session persistence plug-in here. We recommend that you do not install too many other plug-ins. After all, stability and efficiency are the first, unofficial plug-ins are afraid of problems and cannot be found at that time.

Dependent on yum-y install gcc pcre-devel kernel-devel

If yum install openssl-*-y is left, check the prompt. If not, install it.

The following is the installation:

Tar-zxvf nginx-1.7.9.tar.gz
./Configure -- prefix =/usr/local/nginx -- with-http_stub_status_module -- with-http_ssl_module -- with-http_realip_module -- add-module = ../nginx-stick
Make
Make install

Here, a stick session persistence plug-in is used. For example, this plug-in is sometimes not easy to find. It is basically on google servers and there is no way to download it.

Https://github.com/ezbake/nginx-sticky-module

Install keepalived:

Tar-zxvf keepalived-1.1.20.tar.gz
Ln-s/usr/src/kernels/2.6.9-78. EL-i686/usr/src // linux
./Configure make install
Cp/usr/local/etc/rc. d/init. d/keepalived/etc/rc. d/init. d/
Cp/usr/local/etc/sysconfig/keepalived/etc/sysconfig/
Mkdir/etc/keepalived
Cp/usr/local/etc/keepalived. conf/etc/keepalived/
Cp/usr/local/sbin/keepalived/usr/sbin/
Set to service and start upon startup
Vi/etc/rc. local
/Etc/init. d/keepalived start

Start and Stop the two software.

Service keepalived start/stop

./Nginx-s stop
./Nginx-c/usr/local/nginx/conf/nginx. conf

Keepalived Configuration:

Global_defs {
Router_id LVS_DEVEL // ID of the Server Load balancer, which can be the same within the same network segment
}
Vrrp_instance VI_1 {// define a vrrp instance
State MASTER // master lvs is the MASTER, from the BACKUP
Interface eth0 // Network interface monitored by LVS
Virtual_router_id 51 // virtual_router_id must be the same for the same instance
Priority 100 // defines the priority. A larger number indicates a higher priority.
Advert_int 5 // interval between the MASTER and BACKUP Server Load balancer for synchronization check, in seconds
Authentication {// authentication type and password
Auth_type PASS
Auth_pass 1111
}
Virtual_ipaddress {// virtual IP Address
192.168.1.8
}
}

I think other things are mainly related to Server Load balancer lvs, because nginx has replaced lvs, so it has been removed.

The main configuration of nginx is as follows:

# User nobody;
Worker_processes 1;
Events {
Worker_connections 1024;
}
Http {
Include mime. types;
Default_type application/octet-stream;
Log_format secisland '$ remote_addr-$ remote_user [$ time_local] "$ request "'
'$ Status $ body_bytes_sent "$ http_referer "'
'"$ Http_user_agent" "$ http_x_forwarded_for "';
Sendfile on;
Keepalive_timeout 65;
Gzip on;
Gzip_min_length 1000;
Gzip_types text/plain text/css application/x-javascript;
Upstream tomcat {
Sticky;
Server 127.0.0.1: 8081;
Server 127.0.0.1: 8082;
}
Server {
Listen 80;
Server_name localhost;
Access_log logs/secisland. log secisland;
Location/nginx_status {
Stub_status on;
Access_log off;
}
Location ~ \. (Htm | html | gif | jpg | jpeg | png | ico | rar | css | js | zip | txt | flv | swf | doc | ppt | xls | pdf) $ {
Root/opt/cmdat1/webapps/ROOT;
Access_log off;
Expires 24 h;
}
Location /{
Proxy_pass http: // tomcat;
# Root/html;
Index index.html index.htm index. jsp;
}

Basically, load configuration of keepalived + nginx + tomcat is completed. ngix can be configured on the first server to forward data to the second tomcat server, the nginx capability is used for load balancing. When keepalived is mounted on the first machine, the dynamic IP address is switched to the second machine, enabling high availability of the two machines.

The problem here is that when nginx fails, keepalived does not know that the service is unavailable, so it will continue to provide services, but it does not work at this time.

Therefore, you need to add a daemon. When nginx fails, keepalived is also killed, And keepalived is also started when the process starts.

You can set a 30-second check in the crontab, that is, when nginx fails, the maximum time is 30 seconds.

*: 30 */opt/nginx_pid.sh

#! /Bin/bash
# Varsion 1.0
A = 'ps-C nginx -- no-header | wc-l'
B = 'ps-C keepalived -- no-header | wc-l'
If [$ A-eq 0]; then
#/Usr/local/nginx/sbin/nginx
Sleep 1
If ['ps-C nginx -- no-header | wc-l'-eq 0]; then
Killall keepalived
Fi
Elif [$ B-eq 0]; then
/Etc/rc. d/init. d/keepalived start
Fi

Finally, do not forget to add executable permissions to the script. Otherwise, the script will not be executed.