Server access failure records and Experience Summary In February 13, 2015, and experience summary in 2015
Zookeeper
I plan to write an article every day in 2015.
Step 2
Http://weibo.com/p/1001603810113986105909
If you do not want to see the entire process, read the experience summary and relay the record! The IP address settings are manually modified for major accidents. Therefore, an error occurs when you use the graphic interface. The consequence is that the route table is incorrect.
- Pay attention to the allowable maintenance time of the data center, especially during public holidays.
- We need to prepare detailed transportation routes. For example, taxi drivers in this IDC room may be more familiar with the original name: Panasonic TV factory.
- You need to pay attention to the access control management of the data center. The ID card must be included. In the previous data center, you need to make a self-made work card in advance.
- You need to pay attention to the internal access control management of the data center. If you need to make a call to make a partition, You need to test the cell phone signal power in the partition.
- The management method of the IDC is different. The IDC cannot be labeled with IP addresses on the server. Therefore, you need to add tags or other obvious labels on your own.
- The keyboard and display in the IDC are normal, and the mouse is broken. If you need to enter the Linux graphic interface or Windows, you can use your mouse.
- It is recommended that you download the relevant information to your laptop or tablet computer in advance to facilitate on-site query. On-site internet access is not convenient.
During the detailed process, the server was recently migrated to the data center, and the access was unstable. After the network of the data center was ruled out, it was determined that the server was faulty. Therefore, it was necessary to go to the data center for on-site maintenance. I am a Full Stack engineer. My skills include server O & M, backend software development, front-end software development, and product design. I have recently written many articles on product design. Here I would like to explain it so that many friends may think that I have changed my mind. The cause of the accident is: ping failure, ssh failure, and website access failure. Because similar situations have been encountered before, it is basically determined that the route table has an error. When I started designing products, many product designers talked about the experiences of product design. At this time, I could only say, you have never seen how miserable O & M work is! Product design is no longer before it is launched, and O & M incidents are after it is launched, all people will stop working and wait for you to solve the problem, therefore, every empirical judgment is almost like this. Next, let's talk about the data center. The former data center is located in Hebei and provides KVM support. KVM is short for the Keyboard Video Mouse, that is, you can use a virtual network to connect the Keyboard, Mouse, and display. The current data center is located in Beijing and the Fifth Ring Road in Northeast China. Without such support, the transportation is convenient. If servers are deployed outside China, you must check whether it is convenient to provide KVM. If some data centers do not provide KVM, the traffic cost for one maintenance operation will exceed the difference between the two data centers, therefore, do not store them in other places to save hundreds of dollars, especially the main production servers. Once an accident occurs, it takes half a day to run. The loss is more than a year's cost, not worth the candle, and the transportation in other places is inconvenient, you may need to stay there that day, and there are no convenient living facilities nearby some data centers. In my previous work, the nearest one was from the office building in xierqi to the data center in xierqi. However, the larger the data center, the fewer and fewer data centers in office buildings or office buildings. Therefore, distance is one of the main factors for Data Center selection. Before going to the data center, you need to know the time for maintenance. For example, during the Spring Festival holiday, on-site maintenance is prohibited from today (February 14), so yesterday afternoon is the last maintenance period before the festival, otherwise, you need to wait until the spring festival. Even in Beijing, you also need to pay attention to traffic. Although manual maintenance does not require equipment, the location of the IDC is generally not very convenient. For example, the IDC is near Jiuxianqiao, I took a taxi according to the address. As a result, the driver did not hear about the park. Later, I learned that this place was the former Panasonic TV factory. The old driver only knew the place name. Of course, the new driver may not know the new name or the original Panasonic TV factory. The IDC is located in a large campus and needs to be driven in. Access control management for each data center is different. You should say hello in advance. This data center only checks the registered ID card. In the past, the data center must check both the ID card and the work card, for temporary maintenance, you need to prepare the self-made work card in advance. The internal management of each data center is different. Staff are required to lead the partition door in and out of the data center. To maintain the data center for a long time, we need to send out the contact information and test the mobile phone signal, otherwise, it would be really troublesome to shut it down in the IDC. In the past, there was also a data center that only needed to be led in the door. You can leave it on your own. After partitioning, finding the server also depends on the management measures of the data center. The previous management is to add an IP tag to the server. This IDC cannot be pasted. Therefore, the server should be clearly identified, especially when it is maintained by different personnel. Then ask the Data Center Administrator for peripherals. Because it is a linux Command Line, you only need a keyboard display. This is the case for most maintenance, so the two devices should be good. If you need to enter the linux graphical interface or windows, you may encounter a Bad Mouse condition. In this case, you need to ask for a mouse from the data center. Of course, you 'd better bring your own mouse. Then you can log on smoothly, which gives you peace of mind. You can check the information and save it offline, although the data center has convenient network access, it may not be able to access the data center easily because at least an idle iP address is required, so do not get used to accessing the Internet. If you need to access the Internet, you may need a mobile phone. The mobile phone signal is not necessarily good, and when the mobile phone is used as a hotspot, the computer may run as wifi traffic. Similar to the previous situation, an error occurred when locating the route table directly. After querying the ip configuration, you can find that an error occurs when you use the graphical configuration interface again, after modification, restart the network service and restart the computer. Finally, call the IDC administrator to take the lead.
I plan to write an article every day in 2015.
Step 2
Http://weibo.com/p/1001603810113986105909
