Notes on Linux physical server migration

Notes on Linux physical server migration

Two days ago, we performed online migration on the two servers in the production system. Therefore, we would like to summarize some of the operations and lessons learned before and after the migration. In a professional style, I have read some official migration guides from Microsoft and RedHat before and after that, but I found that there are not many practical sections, therefore, after referring to the Migration Guide of Microsoft and RedHat, I decided to provide my own server Migration Guide for reference only.

Migration background:

Because the two servers need to be used for other purposes (it can be understood as server hardware upgrades, the new server hardware performs better than the old server hardware, such as stronger CPU, memory, and storage ), therefore, we need to migrate the original two servers to the new two servers. Due to the large number of active users during the day, the concurrency of each server is high, so migration should be performed at night (we will discuss how to avoid this situation later)

Some migration concepts:

System migration refers to moving the operating system and applications on the source host to the target host and running properly on the target host. In the era when there is no virtual machine, the migration between physical machines relies on system backup and recovery technologies. Back up the operating system and application status on the source host in real time, connect the storage media to the target host, and restore the system on the target host. With the development of virtual machine technology, the system migration is more flexible and diversified. Therefore, if it is a virtualized environment, it is much simpler, because virtual machines can be considered as stateless computing.

Some migration principles:

1. Do not interrupt your business as much as possible, and do not affect your use. If this affects your use, inform the user of your acceptance and preparation.
2. unified server hardware resources, such as hardware architecture, chipset, and processor
3. Keep the operating environment of system software as consistent as possible
4. Use the most appropriate person to do this, and communicate in a timely manner if any problem occurs.

Some migration indicators:

1. Overall migration time: the time from the source host to the migration end
2. downtime: the time when the source host and target host are unavailable during migration
3. Impact on application performance: the impact of migration on the performance of services running on the migrated host.

An excellent migration tool aims to minimize the overall migration time and downtime and minimize the impact of migration on the performance of services running on the migrated host. Of course, these factors affect each other. The implementers need to measure the application requirements for migration and select appropriate tools and software. For Linux system migration, because it is an open-source system, you can use some open-source tools and scripts to manually migrate the system. This method requires the administrator responsible for migration to have relevant experience, and familiar with Linux systems running on physical machines.
Overall migration steps and procedures:

1. Prepare for Migration

1) collect the hardware information and software environment of the source host, and prepare relevant software and hardware documents of the source host.
2) configure the target host to the same software environment as the source host based on the above information and documents (under the condition that the hardware resources of the server are unified)
3). Test the above configuration results (test method reference verification migration)
4). The above work should be done well before migration

2. implement migration

1) inform relevant personnel in advance to prepare for Migration
2). switch network parameters, such as IP addresses.
3) Disable business applications on the source host
4) start business applications on the target host
5) restart or smoothly restart the associated services for the relevant host.
6) The migration is completed on the current day.

3. Verify migration

1). Self-Test migration results
2) ask other personnel to test the migration results, such as the R & D team or test team members.
3) Migration completed on the same day

4. Post-migration tasks

1) report to relevant persons in charge
2) report to relevant users
3). Update relevant documents and records

5. Complete migration (same as "post-migration task ")
6. rollback

1). switch network parameters, such as IP addresses.
2) close business applications on the target host
3) start business applications on the source host
4). Update the configuration files of associated services running on the relevant host, and restart the associated services running on the relevant host.
5) fix the problem and prepare for the next migration

Note:

1. some applications need to be tested before implementation, but may conflict with some servers in the production environment after startup, for example, a message processing application may compete for messages in a message queue with other message processing programs. Therefore, you must eliminate such conflicts during testing, this also requires applications to be loosely coupled and modularized as much as possible to facilitate changes.
2. During the migration process, to start business applications on the target host, you must disable business applications on the source host, and vice versa.
3. No matter which type of switch, try to avoid omissions and make everything very fine.

The following lists precautions and considerations for Linux system migration:

1. Sort out the migration list.

1 ). including the source host operating system name, version, host name, environment variables, drivers, processes, installed software, installed services, firewalls, kernel parameters, security settings, user configuration, certificate keys, file permissions, and other basic system environments
2). Based on the process name, port number, firewall configuration, or system maintenance management log, find the services currently being provided by the server, the installation manual of the service, and the software environment on which the service depends.
3 ). according to the document or the current network connection status of the source host, this mainly refers to the hosts that use the Intranet address of the source host. For example, find the relevant hosts of the proxy source server and pay attention to the configuration on these proxy hosts, for example, you need to change the Intranet IP address and domain name information together. A typical example is nginx and HAProxy reverse proxy.
4). Some special file directories, software compilation directories,/etc/profile ,~ /. Bash_profile,/etc/rc. local ,~ /. Profile ,~ /. Ssh,/etc/fstab,/etc/hosts,/etc/sysconfig, etc.
5) if there are too many migration projects or steps, you can create a list to facilitate reference and prevent omissions. For more information, see the appendix.

2. Prepare the target host.

1) manually (one operation can be performed one by one or a script can be written). Configure the target host to be tested in the same software environment as the source host.

3. switch network parameters. If you only switch the public IP address and do not switch the Intranet IP address, because the Intranet IP address may be coupled within the network, you need to pay attention to other related hosts.

Migration summary:

1. The implementation of docized standardization is very important. Although it may be very hard and tiring, we should try our best to do it.
2. If any change has occurred, You need to modify the relevant documents and notify the relevant persons in a timely manner.
3. Standardized deployment, such as fixed compilation and installation directories, user-friendly soft links, and strict reference documents
4. make proper use of tools and skills to reduce operation time
5. High Cohesion and low coupling as much as possible during the software design process, supporting horizontal expansion as much as possible, achieving stateless software design is a very idealized design goal
6. Some hardware manufacturers provide some migration tools or information collection tools for reference, such as Dell's SystemManager, dset, Cisco's UCS Manager and other related tools.

How to avoid overnight migration, upgrade, or business changes:

1. To avoid upgrading in the middle of the night, you must adopt the HA System Architecture (no spof), that is, when one or more servers go down, there is no impact on users.
2. Software loose coupling as much as possible and support horizontal expansion as much as possible
3. Make Rational Use of automation, scripts, and scheduled tasks
4. perform multiple write tests and perform more drills, such as whether the rollback can be performed normally.
5. communicate with each other in advance and discuss matters well ,;)

Appendix:

For example, "migration data collection worksheet" and "xxx change table", the migration data collection worksheet can be referred to as follows:

#	Source server settings	Set ID	Result of setting status of the target server	Remarks
1	Key1	Value1	Success
2	Key2	Value1	Failed	Causes, solutions, etc.

Other references:

1. Red Hat documents: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/

Microsoft TechNet: https://technet.microsoft.com/en-us/library/dd365353 (v = ws.10). aspx

3. Migration theme in the Microsoft Forum: https://social.technet.microsoft.com/Forums/windowsserver/en-US/home? Forum = winserverMigration

4. Virtual Machine Migration Technology:

-- End --

This article permanently updates the link address: