Remember database connection issues once

Recently when the project found a problem, when the site opened, a stop in the turn, about 5 minutes later returned 504 Gateway Timeout

At that time the first reaction is my network architecture problem, so from the front-end LVS has been looking down, my architecture is lvs+nginx+tomcat, until the Nginx this piece is no problem, from the Nginx error log can not see anything, the same newspaper 504 net timeout, At that time was thinking 502 Bad Gateway is that nginx to throw the request to Tomcat when the port can not find Tomcat, 504 will not be the same?

[Email protected] ~]# Netstat-lntup|grep 800
TCP 0 0::: 8000:::* LISTEN 4208/java
TCP 0 0::: 8002:::* LISTEN 4208/java
Don't see any problems, and then compare with the online server to find the problem

[Email protected] ~]# Netstat-lntup|grep 800
TCP 0 0::: 8000:::* LISTEN 4208/java
TCP 0 0:: ffff:127.0.0.1:8001:::* LISTEN 4208/java
TCP 0 0::: 8002:::* LISTEN 4208/java
Discover that Tomcat is missing a port, why didn't it cause the problem?

View the tomcat logs and find that Tomcat is not up, stuck in the connection connection database that block

So I looked at the MySQL port on the web side and found that I could

[[Email protected] server]# telnet 10.10.0.126 3306
Trying 10.10.0.126 ...
Connected to 10.10.0.126.
Escape character is ' ^] '.
^]
Telnet> \q
Connection closed.

Come here to find their ideas broken!!

Twists and turns, inadvertently on the DB2 on the MySQL port, found and the results on the web is not the same

[[Email protected] root]# telnet 10.10.0.126 3306
Trying 10.10.0.126 ...
Connected to 10.10.0.126.
Escape character is ' ^] '.
N
5.5.32-log0pkhjw=db!>migixaww,s "Mysql_native_password
What is this for?

So a little bit of memory, thinking about the difference between the web and DB, finally found the problem

10.10.0.126 and 10.10.0.127 is my data read VIP and write VIP, only these two machines have 10.10.0.0 network segment IP

There is only one extranet IP on the web, so route-n a look at the Web and DB gateways

[[email protected] server]# route-n
Kernel IP routing table
destination      gateway         genmask          flags Metric ref    use Iface
202.192.168.0   0.0.0.0          255.255.255.0   u     0       0        0 eth1
169.254.0.0     0.0.0.0          255.255.0.0     u     1002    0        0 eth1
0.0.0.0          202.192.168.1   0.0.0.0         UG     0      0        0 eth1

[email protected] ~]# route-n
Kernel IP Routing Table
Destination Gateway genmask Flags Metric Ref use Iface
202.192.168.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.10.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
0.0.0.0 202.192.168.1 0.0.0.0 UG 0 0 0 eth0

Obviously, a 10.10.0.0 gateway is missing on the web

A gateway was added to the web side
Route add-net 10.10.0.0 netmask 255.255.255.0 Dev eth1
Look again.

[email protected] server]# route-n
Kernel IP Routing Table
Destination Gateway genmask Flags Metric Ref use Iface
202.192.168.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.10.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth1
0.0.0.0 202.192.168.1 0.0.0.0 UG 0 0 0 eth1
It's the same as the DB end.

Check out the MySQL port

[[Email protected] server]# telnet 10.10.0.126 3306
Trying 10.10.0.126 ...
Connected to 10.10.0.126.
Escape character is ' ^] '.
N
5.5.32-log±vj3[skes! Q{9h=9e0]fjamysql_native_password
SUCCESS,!!!!!.

Reboot Tomcat to view Tomcat ports

[email protected] server]# Netstat-lntup|grep
TCP 0 0::: 8000:::* LISTEN 4208/java
TCP 0 0:: ffff:127.0.0.1:8001:::* LISTEN 4208/java
TCP 0 0::: 8002:::* LISTEN 4208/java
Port up!

Page access is also normal!!!

This article is from the "Wanliang" blog, make sure to keep this source http://wlmint.blog.51cto.com/10931984/1826740

Remember database connection issues once