F5 Network Load Balancing Service access fault resolution (CISCO OTV+LISP-MTU problem)

First, the problem phenomenon

Recently in a customer because of the holiday core Cisco 6509 hardware failure problem, and then found that F5 released 3 application access problems, some people access to the application is not available problems, when bad, the intranet use F5 GTM+LTM domain name double live, The internal co-city dual live DC is built using Cisco's sophomore layer technology Otv+lisp technology through three-tier routing;

F5 above check the application whether the VS or the pool member are normal, the health check or monitor algorithm uses TCP, and cross-DC member offline,gslb through the member of the LTM dual-machine on the face-to-side DC business Disable resolution only diversion to the main DC, when business access is normal, form a single live to troubleshoot

The problem is that the business accesses the exception after the cross-DC access, but the magic is that only some of the VLANs are problematic, and most of the VLANs across the DC are not problematic!

Through the initial troubleshooting, the application personnel said that the application is no problem, network personnel said the network is no problem (can ping from the main center application IP, can cross the Dctelnet business application port, and other VLANs do not have problems), F5 personnel also said F5 log all aspects of normal, no abnormal log!

Second, the cause of the problem

F5 personnel recommends direct access (without F5 load) to the 443 ports accessed across the DC to test and grab packets, check packet traffic

By grasping the packet, found that the TCP three times the handshake is normal, but the SSL protocol handshake is abnormal, the client sends the customer hello, the server side loopback a 1050byte SSL data (non-server Hello) package and prompts the leading segment is missing! Then the client fin dropped the connection!

Then through the normal application access to this DC capture packet, clear SSL negotiation is normal, SSL handshake packet up to hundreds of byte, so this is an application-level anomaly, not a simple network-level problem

But whether it is an application problem, let the application staff replace a VLAN, access to normal! Proving not an application-level configuration exception problem! It's probably a problem with the network impact app!

In view of the hardware failure when the machine, path changes, the application of SSL protocol interaction packet size anomalies, and prompted Previos fragment lead segment loss and other network problems, F5 personnel recommended to check the MTU settings, and then the customer management and network personnel to say before the MTU problem, let Cisco The TAC makes a check through several hours of checks, and finally confirms that the MTU is not changed to a 9216-byte MTU due to the Cisco 6509 when the machine causes a partial VLAN OTV path Transformation!

Business access is OK after the change!

Third, the solution

Replace the path in the OTV MTU after resolution, F5 related configuration restore, the application test is normal!

F5 Network Load Balancing Service access fault resolution (CISCO OTV+LISP-MTU problem)