Learning plan: SSIS

I have participated in a system, collecting and summarizing data, extracting common data from databases with different structures, and performing data validation and filtering, there is also data type conversion (in fact, all are converted to string processing), using graphical configuration, reflecting database tables and fields, at that time, the most creative thing was to drag a field from a table structure to another table in many databases and draw a line between them, similar to building a primary foreign key, we constructed a field ing relationship to extract data (it took a long time for GDI + to achieve this ). Finally, we need to monitor changes in the database in real time. Due to business requirements, we finally adopt the round-robin read and compare method.

Two or three years later, new technologies have emerged. I have heard that Oracle has acquired a set of middleware for data extraction and transfer. I don't know much about oracle, recently, I heard from my colleagues that SQL Server's SSIS is very useful. It also extracts, processes, and transfers data from the source data to the target data. It can also develop intermediate programs, and debugging interruption is also very convenient. As A. Net programmer, I naturally need to pay attention to it. Besides, there are many similarities with the systems I once implemented, that is, I don't know if there is any real-time update detection? In any case, Microsoft's things are naturally far more powerful than our Xiaomi and rifle, so we should take a good look.

The following are extracted learning resources for preliminary understanding:

From: http://www.cnblogs.com/shanyou/archive/2009/12/29/1634974.html
SSIS data stream
Data flow is a new concept introduced in SQL Server 2005. Data flow is a workflow dedicated to data operations. Data streams are also called pipelines. The data flow can be considered as an assembly line, which contains multiple operations in sequence. Each node in the data stream is called a conversion. Data Flow usually starts with source conversion and ends with target conversion. Between the two transformations, the predefined data stream transformations are applied to the data in sequence. Some conversions are synchronous, such as search, conditional splitting, and data conversion. These synchronous conversions can be executed in parallel.

Once the conversion has been applied to a data row, the next conversion can start to process the data row without waiting for the upper-level conversion to process the complete data set. Some conversions are asynchronous, such as aggregation and sorting. These conversions must obtain all rows from the previous output to process and generate output for subsequent conversions.

SSIS Learning (2): Data Flow task (I)

Integration Services Learning (3): Data Flow task (2)

SSIS engineers reveal data streams to you

Writing a custom data flow component (dataflow component) for SSIS: Custom Editor