DTS development record (6) -- Data ing

In DTS development record (4)- ing column, we have discussed in detail the source and target columns involved in ing. On the basis of the basic concept of ing columns, we can discuss data ing DataMapping.
Data ing consists of a set of mappingitems, which contain detailed and complete ing relationships between the source data outline and the target data outline.
Note that data ing is bound to the data outline, but not to any specific database table. In this way, DataMapping can be applied to different database tables, as long as the corresponding database table structure is compatible with the data outline required by DataMapping.

1. MappingItem ing
A ing item corresponds to a target column. If a target column participates in the ing, it can carry only one MappingItem. MappingItem contains how one or more source columns are mapped to the target column. If MappingItem contains only one source column, it is called "simple ing". If MappingItem contains multiple source columns, it is called "merge ing ".
The following is a class diagram of the MappingItem interface:

The Source attribute of IMappingItem is a set of isourcemappingcolumns. When the number of elements in the set is greater than 1, the logs are merged and mapped. When the number of elements in the set is equal to 1, the logs are simple ing.
The Source of IMappingItem can contain both common Source columns and Source subcolumns, which we have already emphasized in the article "data transmission development record (4) -- ing column.
We noticed that IMappingItem also has a Verify method, which is used to Verify whether the ing rules in this ing item are valid. For example, the Source of IMappingItem contains multiple Source (child) columns, but the merge expression carried by the corresponding idemo-appingcolumn is empty. Verify will Verify that the ing item is invalid. Invalid ing items cannot participate in data transmission tasks.

2. DataMapping
In addition to a set of mappingitems, data ING also includes some important elements, such as the source data outline and target data outline and primary key information. Is the DataMapping class diagram:

DataMapping carries the source data outline and target data outline. When started, the data engine uses these two outlines to verify whether the data source and data target are compatible with the outline. If not, data cannot be transmitted.
We can see that DataMapping also has a Verify method. This method will first call the Verify method of each MappingItem in sequence. If it passes, it will continue according to the export form (all export and incremental export) perform other verification, such as primary key verification and primary key ing verification.
It is very important that DataMapping has a SourceMappingColumnManager attribute. SourceMappingColumnManager is used to manage all source columns and source subcolumns involved in the ing. Not all source columns are involved in the ing, and the source columns involved in the ing may be very complex, such as source column splitting, deformation, and child column involved in deformation ing. In this way, when we reload DataMapping for editing, we can obtain detailed information about splitting, deformation, and ing of each source column from SourceMappingColumnManager.
DataMapping is serializable because we need to save it as a file so that it can be edited and used later. Since DataMapping is serializable, there is no doubt that SourceMappingColumnManager also needs to be serializable. Therefore, the serialized DataMapping stores all the details about a complete ing, so that the deserialization can restore DataMapping to the original state to edit or control the data transfer process.

The Data Engine is based on the connection pair (you can create the corresponding import data pipeline and export data pipeline Based on the connection pair) and Data ing. So far, we have finished introducing the connection pair and data ing, next, you can enter the data engine, the core part of the data transmission system.
Before transferring data, we may need a checking tool to check and analyze the data in the data source to see which data does not meet the conversion conditions or meet the requirements. You may need to make important decisions related to data transmission based on the analysis results of the checking tool.
Therefore, the content of the next section is "Data check", which is a very "trivial" section, because you need to consider the possibility of any subtle conversion failure, it's nothing!

Go to: DTS development records (sequence)