ETL is responsible for the scattered, heterogeneous data sources such as relational data, flat data files, such as the extraction of the temporary middle layer after the cleaning, transformation, integration, and finally loaded into the data warehouse or data mart, as the basis for online analysis processing, data mining. The term ETL often appears in the Data warehouse, but its object is not confined to the data warehouse. ETL is a very important link in data Warehouse. It is a necessary step in the past. Compared with the relational database, the Data Warehouse technology has no strict mathematical theory Foundation, it is more oriented to practical engineering application. Therefore, from the point of view of engineering application, loading data according to the requirements of physical data model and processing the data in some series, the process is directly related to the experience, and this part directly relates to the quality of data in the Data Warehouse, which affects the quality of the results of on-line Analytical processing and data mining. The Data Warehouse is a separate data environment, it needs to import data from the online transaction processing environment, external data source and offline data storage media into the data warehouse through the extraction process, and in the technology, the ETL mainly involves the correlation, the transformation, the increment, the dispatch and the monitoring and so on several aspects , data in the Data Warehouse system does not require real-time synchronization with the data in the online transaction processing system, so ETL can be timed. However, the operation time, sequence and success of multiple ETL are critical to the validity of information in Data Warehouse.
ETL data extraction, transformation, and loading
