Basic concepts of data mapping (mapping)

Data Mapping (Mapping) : Given two data models, the corresponding relationship between the data elements is established between the models, which is called data mapping. Data mapping is the first step in many data integration tasks, such as data migration (migration), data cleansing (database cleaning), data integration, Semantic Web architecture, Peer-to-peer information System.

Data maps can be done in two ways: manual encoding (hand-coded) and visualization (graphical manual). Manual coding is the direct definition of data correspondence in a programming language such as xslt,java,c++. Visualization typically enables users to draw a line between data items to define a corresponding relationship between data items. Some tools that support visualization can automatically establish this correspondence. This automatically established correspondence typically requires the data item to have the same name. Regardless of whether the manual operation or the automatic establishment of the relationship, the tool will eventually need to automatically transform the graphical representation of the corresponding relationship to xslt,java,c++ such an executable program.

Currently, there are two frontier research directions in the field of data mapping:
Data-driven mapping: using statistical methods to analyze the actual data of source and target database, and to excavate the data correspondence relationship. This method can be found in the data between the "substring", "concatenations", "Arithmetic", "case statements" and other conversion logic. This method can also be used to detect anomalies, that is, data that does not conform to the defined transformation logic.

So what is the relationship between data mapping and data migration?

Typically, data migrations include three phases: Data extraction (extract), Data conversion (transform), data loading (load), commonly known as ETL. But how to extract, how to transform, and where to load these issues require a clear rule of instruction. So this master needs a data map to define these rules. It's kind of like design and development in the software development process. Data mapping is equivalent to software design, ETL execution code implementation process is equivalent to software development.