Problem Description: Data calculation generation is not started until the daily necessary data is imported from the CRM database. The start of the calculation this morning was 2 hours later than normal. Maintenance personnel seizure is the process of data import that was completed in about 300 seconds and now takes more than 1 hours. The general troubleshooting process is as follows: (1) using the OSW tool to record the Linux system history data, excluding the corresponding time period CPU, IO activity anomalies. (2) to export the corresponding time period AWR report and the past normal situation of the corresponding time period AWR report to do comparative analysis. Normally the corresponding time period of the AWR report:
The corresponding time period in the exception of the AWR report
You can see that the status of the database itself is fairly idle. The only anomaly is that sql*net more data from client wait time 11.1K is normally 2470.2 times more than 5 times.
(3) to capture network-related information at the system level
3 Ten
Gigabit Network card traffic stability in 10mb/s, far from reaching the peak of the network card.
It came to mind that it was supposed to be a full-time weekend, due to the unusual interruption of backup software, there was a re-launch last night. Verify that the full backup is still not complete at the time of troubleshooting .
and network management personnel to confirm that the server connected to the hundred gigabit Switch, the problem is basically located. Accidental full backups occupy the Business time window causing network congestion, the appearance of the performance of the database is abnormal!
Troubleshooting of abnormal performance of database due to network congestion