methods, steps and experiences on data migration

methods, steps and experiences on data migration

In the project will often encounter the system after the complete replacement of the historical data migration issues, to show respect for the customer's historical work, not to mention a lot of data are still necessary to retain.
How do you do the historical data migration?

System Analysis:
1, analysis of the original business system
Accurate to the approximate system function module, the approximate processing flow can
2, analysis of existing business systems
Accurate to the approximate system function module, the approximate processing flow can
3, analysis of their own differences and differences
A rough analysis of the differences between two business systems helps to determine workload and work progress
4, analysis of the user needs of the old data
Analysis of the need for old data to be not blind to the whole of the migration
5, analysis of the user's old data processing rules

The processing rules of old data are generally divided into the following categories:
1, the basic data, usually this kind of easy to migrate, the data format is simple, but will affect all the relevant business data, focus on the data as the primary key and unique key way.
2, Pure historical data import, only for reference, this kind of data import easy
2.1 Pure Historical data
This kind of data processing will be relatively easy, one-time import can be followed by incremental data import.
2.2 Flow of data
This kind of data only after the record closes completely, needs to carry on the increment import and the data update, but also must carry on the related inquiry interface development, guarantees the old data to be able to inquire in the new system.
3, the old and new system table structure changes large historical data
The workload of this kind of data is heaviest, need to study the data structure of new and old business system carefully.
1, as far as possible through the unit to collect a complete set of relevant original system design documents, this is helpful to data analysis, through the feeling of people and the observation of the data is not very reliable after all.
2, on the original system to observe the relevant data to understand the changes in data and data table data relationship (for more difficult to understand related fields are very helpful)
3, compare the new and old system data differences, if it is very unreliable, it is recommended to press 2.2 to deal with.

System Design:
1, after the system analysis, the relevant data classification, basic data, pure historical data, change the larger historical data
2, first from the simple start, give yourself some confidence
3, in the Excel table in the relevant table data dictionary control, sketch out the corresponding fields, conversion logic, dependencies, when necessary in the new system table to do the corresponding redundancy, and so on after the data migration and then clear.
Key points:
Matching problems with the field types of different databases, such as SQL Server text, should correspond to CLOB in Oracle, but would rather be converted to several varchar2, which is relatively easy to implement.
About primary key issues, consistent data types as far as possible to maintain the existing state, inconsistent as far as possible with the Oracle sequence or SQL Server identity int, but after the migration, you should pay attention to update the sequence values

System implementation:
1, Pl/sql or T-sql,dts, SSIS, or PowerBuilder pipelines as a means of data migration
2, related business logic migration, data integrity and business-related stored procedures, triggers, function rewriting, and even the rewriting of SQL
3, the preparation of Business Inquiry page

Something:
1. No special or advanced methods for data migration
2, the Business data analysis is the most critical, need to constantly try on the new system constantly testing
3, data migration technology, mainly through the SQL, stored procedures, and even cursors to achieve, priority also above

There is also a data migration is only a database of the flat or heterogeneous database migration
Database leveling, that is, for performance expansion needs to migrate from one server to another server, with the export of the database import or Backup recovery tool processing can, of course, also consider the migration of some sequence field initial values.
Heterogeneous data migration, that is, from one database platform to another database platform, with ETL tools or SQL can be implemented, but pay attention to the business logic migration, that is, stored procedures, functions, triggers and so on