Transform and integrate data using WebSphere DataStage XML and Web Services packages

This article supporting source code

Ibm®websphere®datastage™xml and Web Services packages are components in DataStage that deliver fast data integration solutions when XML and Web services are involved. Study the main functions and operations of DataStage Web Services and XML packages. Learn how to use these two packages to transform, integrate data, and get Google search Web services. This article includes 4 examples, each with a simple overview, detailed steps, instructions, and pictures.

Brief introduction

DataStage XML and Web Services Packages overview

Ascential DataStage was acquired by IBM in 2005 and is now known as IBM WebSphere DataStage. It is a convenient, GUI-based extraction, transformation, and loading (ETL) tool that can be used to integrate data in different structures and formats and effectively support customer relationship management (CRM) analysis, Decision Support Systems (DSS), and e-business.

DataStage is based on a client-server infrastructure, including designers, managers, pilot controls (director), and administrator components, while providing robust functionality. DataStage can greatly simplify the task from development to maintenance.

DataStage's XML Package and Web Services package are the focus of this article. With these two packages, it is easy to integrate XML and WEB services into the DataStage solution. The advantages of these two packages using DataStage are:

Shorten development time

Speed up user response

User friendly development, management and maintenance of GUI environment

DataStage XML (DataStage XML package) consists of XML Digest, XML Publishing, and transformation capabilities. The XML section of this article describes one of the main transformations provided by XML packages: the conversion between XML documents and table data.

WEB Services packages enable DataStage to invoke remote services and treat those remote services as simple data sources, data destinations, or interactive data interfaces. The Web Services Section of this article describes the Web service Transformer that provides an interactive data interface.

This article includes 4 about using DataStage (Version. 7.5.1A, with the XML and Web Services package installed, an example of developing a data solution is analyzed and annotated for each example. These 4 examples are based on the following typical scenarios:

Publish an XML document based on table data

Parsing an XML document into table data

Accessing Web services with input and output data

Prerequisite

The WebSphere DataStage v7.5.1a is installed with db2® packages, XML packages, and WEB services packages.

There are ibm®db2 for linux®, UNIX®, and windows®v8.2 (or later) installed.

The reader has some practical experience in the fields of XML, Web service and DB2, and has the basic knowledge of DataStage.

1th part. Publish an XML document based on table data

Publishing an XML document based on existing table data is a common scenario. Sometimes, you need to convert relational tables or sequential files into XML hierarchies, such as XML documents or XML blocks. In this example, you can use the XML output stage to generate the XML output. It uses an XPath expression to map an input table field to some location in the output document.

Example 1. Generating XML files from two tables using XML output stage

Figure 1. Task Map for XML Publishing

Example 1 overview

In Example 1, customer data and contact data were extracted from two corresponding DB2 tables, as shown in Figure 1. Next, the converter replaces the complex SQL, integrates the data, and provides the connected data to the XML output stage through DSLINK6. The XML output stage then generates the XML results and saves the results to the file system. Figure 1 provides a brief description of the entire application demo.