Add BPEL to the Enterprise Integrated Hybrid Environment

Copyright: This article from Oracle technology network original address: http://www.oracle.com/technology/global/cn/pub/articles/bpel_cookbook/chandran.html

 

Add BPEL to the Enterprise Integrated Hybrid Environment
Author: Praveen CHANDRAN and Arun poduval

Use the orchestration function of Oracle BPEL Process Manager to implement standard-based business process integration that complements the traditional EAI middleware.

 

Most enterprises have a very different application infrastructure, including a variety of applications provided by multiple vendors, running on different platforms, and using completely different technologies. To solve these integration problems, companies such as tibco, webmethods, Vitria, and seebeyonda have successively launched Traditional Enterprise Application Integration (EAI) products over the past decade. Over the past few years, many companies have invested heavily in these EAI solutions. Therefore, business integration in the EAI field is usually locked to a single vendor, and the Integration components are tightly coupled.

The maintenance cost of these dedicated integration links brings heavy burden to enterprises. Specialized skills are essential to increase cost and stability. In addition, it is not a desirable method for companies that protect the large investments in EAI to destroy and replace existing EAI solutions.

BPEL provides a standard-based, platform-independent solution to solve all these problems. The loosely coupled BPEL process removes vendor lock, reduces integration costs, and provides interoperability. In addition, it also adds a comprehensive security, exception management, and logging layer. Most importantly, companies can leverage their existing infrastructure to make full use of them and orchestrate them using BPEL.

In this section of the "BPEL Guide", we will introduce an architecture blueprint that describes how to use the Oracle BPEL Process Manager to develop new integrated solutions and connect to existing integrated solutions. This includes a case study. In this case study, you must integrate the compiled web service with existing heterogeneous EAI solutions based on tibco businessworks and webmethods.

In addition, because any complete business process implementation is inseparable from the well-designed error management, security, and record framework, therefore, I will introduce how to enhance the robustness and fault tolerance of the process by the BPEL scope and compensation handler, and how to protect the process and related services.

Background knowledge of case studies

EAI is a powerful driving force to make full use of existing applications. It can publish existing middleware processes as Web services and then orchestrate the web services through BPEL.

Demonstrate a common method for orchestrating existing EAI interfaces and integrating new applications using the Oracle BPEL Process Manager. This method assumes that the middleware can publish business processes as Web Services and that the application server itself has web service interfaces.

 

Figure 1Oracle BPEL Process Manager in EAI (complete)

 

The analysis of specific case studies involving two traditional EAI middleware Products demonstrates how BPEL plays an important role in integrating the two products.

In many enterprises, changes made in a customer's "record system" are not filled into other systems that maintain customer data. Let's look at this situation: for various business reasons, an enterprise eventually uses two middleware Products from different vendors, for example, Siebel CRM integrated with tibco businessworks and sap r/3 integrated with webmethods (see figure 2 ).

 

Figure 2Customer Details Management Module and middleware interface

 

In this model, inconsistent customer data between SAP and Siebel Systems may have a negative impact on the customer service level and reduce the revenue of departments and enterprises. The consistency can be maintained through the general Customer Details Management Module (this module establishes multiple interface points between tibco and webmethods integration. For example, when Siebel receives customer data, it checks whether the customer is a new customer or an existing customer, and then adds the new data to sap or updates existing customer data in two applications.

You can use existing middleware tools (tibco and webmethods) to implement this integration, but this will only increase the size of the dedicated integration and cause vendor lock. This situation provides an opportunity for applications to take full advantage of the potential to implement a standard-based, vendor-independent solution.

To implement standard-based interfaces in EAI, first, publish the process as a web service. Most middleware platforms can communicate with each other through web services, but when a group of interface services must be bonded with the corresponding business logic, the situation will become complicated.

You can use other middleware processes or even complex Java code to orchestrate Web Services. However, this process must provide the following functions:

• Concurrent web service calls
• Asynchronous web service calls
• Loose coupling and portability between services
• Use standard-based interfaces to publish the entire orchestration
• Orchestration monitoring

As shown in 3, standard-based premade orchestration solutions (based on BPEL) can solve these problems.

 

Figure 3Customer Details Management Module with Web service interfaces

 

Introducing BPEL to this scenario has the following potential benefits:

• BPEL supports loosely coupled Web Service orchestration.
• You can use simple XML tags to represent business logic (or even parallel streams !).
• Easy to useAssign(CopyRules) andInvokeThe statement easily transfers data between services.
• You can call the customer's detailed information management module in the form of independent web service components from other orchestration, middleware tools, or web applications.
• You can use a simple gui (such as the GUI provided by the Oracle BPEL Process Manager) to manage processes.

Most middleware tools can expose their business processes as web services, making it easier to connect existing integration with the BPEL orchestration. In fact, you can use a common message format for all the related middleware service interfaces.

Now let's take a look at how to use the Oracle BPEL Process Manager to synchronize customer data between SAP and Siebel.

Implement the Customer Details Management Module

BPEL is helpful in the automation of customer data synchronization processes between SAP and Siebel. The process involves the following steps:

1. Publish the tibco and webmethods processes as Web Services.
2. Orchestrate web services by using the BPEL process.
3. Add the exception management function to the BPEL process.
4. Protects communication between Oracle BPEL Process Manager, application adapter, and EAI tools.
5. Centralized recording and notification processes.

Step 2: publish the tibco and webmethods processes as Web Services.Customer information is presented in a standard format and contains the fields required by SAP and Siebel. If this general format is passed between tibco and webmethods, these platforms convert the standard format to Siebel and SAP customer records accordingly.

The following steps are taken to publish the businessworks process as a Web Service:

1. Analyzes the functions provided by the businessworks process to determine whether it can be used as an independent component in an integration scenario.
2. Determine the process input and output.
3. IfInputAndOutputComplex. Use the W3C XML Schema (XSD) to define them. You can also use XSD to customize the failure mode.
4. Use the WSDL template to create and define the WSDLInputAndOutputMessage format (if needed, associate them with the predefined XSD ). You can also import the existing WSDL.
5. ConfigurationHTTP ConnectionResources.
6. Add soapEvent SourceUsed as the first activity, followed by the business logic, and then used soapSend replyPublish the process as a service.
7. SetHTTP ConnectionResources andEvent SourceAssociation.
8. Convert the WSDL andSend replyAndEvent SourceAssociation.
9. Handle possible exceptions and use soapSend faultSend exceptions to the service client.
10. If the computer name is mymachine, port 8000 is used for HTTP connection resources; if the process name isSoapserviceYou can use URL http: // mymachine: 8000/soapservice to access the service.
11. SlaveEvent SourceObtain the final WSDL of the service from the activity's WSDL tab.

The following describes how to use webmethods to perform the same operation:

1. Determine whether the webmethods stream service can be an independent component in the integration scenario.
2. If you want to call this web service outside the webmethods environment without any authentication, specify "execute ACL to anonymous" on the permissions tab ".
3. In webmethods developer, select flow service and clickGenerate WSDL.
4. Specify the Protocol (SOAP-RPC/SOAP-MSG/HTTP-GET/HTTP-POST) and transport mechanism (HTTP/https) when generating the WSDL document ).
5. Define the target namespace of the WSDL document.
6. InHostEnter the IP address or name of the computer hosting the webmethods Integration Server in the domain.
7. InPortEnter the port number to connect to the current integrated server.
8. Save the WSDL document to the local file system. You can find the service endpoint in the generated WSDL document.

Note: The webmethods integration server sends predefined soap Faults Based on specific error conditions. If a custom soap fault must be sent, a custom soap processor should be used. The packaging program service or custom soap processor should also be used to publish the service as a document/text web service.

Now, assume that you have the following three tibco Web Services (implemented using the tibco businessworks process and tibco adapter for Siebel ).

• Siebel add
• Siebel update
• Siebel Query

Similarly, you have deployed the following webmethods web services using the webmethods Integration Server and webmethods sap r/3 adapter.

• SAP add
• SAP update

The architecture of this solution can be summarized as follows:

 

Figure 4Solution architecture

 

As shown above, the BPEL process is converted between the frontend call center and the backend sap and Siebel CRM applications through the EAI tool.

The following are some best practices for exposing middleware processes to Web Services:

• Comply with WS-I standards as much as possible.
• Try to publish the service in the Document Style with text encoding. If this method is not available, use a text-encoded "RPC" style. Although both are recommended methods for WS-I, you will find that document styles are simpler (at least when you areAssignActivity CreationCopyThis is the case when the rules are met)-with the RPC style, all the mode elements appear as separate Message parts, and with the document style, the entire message is transmitted in one part. You can use a singleCopyThe rule copies the entire pattern, reducing the development workload and verifying the style and encoding in the final WSDL document.
• Avoid using soap encoding.ActionAttribute is an empty string.
• Make sure that the middleware uses WSDL 1.1 to describe the Web service interface.
• Use http binding of soap.
• Make sure that all XSD used to describe this schema complies with the XML Schema Specification recommended by W3C. For example, the Global element declaration should not use references to other global elements, that is, use the type attribute instead of the ref attribute.

Step 2: orchestrate Web Services.After the middleware process is published as a web service, you can use the Oracle BPEL Process Manager (which has a powerful Gui-based BPEL Creation Interface) to orchestrate the middleware process.

As we mentioned earlier, an important aspect of the Customer Details Management Module is that it synchronizes customer data between SAP and Siebel. This process is created visually in the BPEL designer, as shown below.

 

Figure 5Orchestrate the customer details module using the Oracle BPEL Process Manager

 

The following is a summary of the process:

1. The receive activity accepts the customer details (Enterprise mode ).
2. PassAssignAndInvoke(Siebel QueryService) activities send detailed information to Siebel.
3. PassPickActivity, useSiebel QueryResult: Determine whether the customer is an existing customer or a new customer.
4. If the customer is a new customer, it calls parallel execution and passesFlowThe activity adds the customer to Siebel and SAP; otherwise, the other parallel stream updates the customer details in the two applications.
5. If SAP does not contain customer detailsSiebel QueryThese fields will be obtained from Siebel. This sap field and other fields to be updated may use a groupAssign copyThe rule is passedSAP update.
6. By using explicitReplyThe activity returns the final state of customer update/addition.

As you can see, both sides of the business process are Web services used to add and update Siebel and SAP data. These web services designed in step 1 call the EAI process internally.

This BPEL process meets the business needs managed by the customer, but it still cannot handle exceptions. For example, if a customer is successfully added to Siebel but fails to be added to sap, what will happen? To solve this problem, you must enable Exception management in the business process.

Step 2: add the exception management function.Through Exception management, the BPEL process can handle error messages or other exceptions returned by external web services, and generate error messages to respond to service or runtime faults.

The following table lists the exceptions that need to be handled to make the customer manage the BPEL process more secure.

No.	Situation	Solution
1	Siebel QueryFailed	Terminate the process and try again
2	Siebel addFailed;SAP addSuccessful	Compensate to delete sap records and try again
3	Siebel addSucceeded;SAP addFailed	Normal stream; retry
4	Siebel addFailed;SAP addFailed	Normal stream; retry
5	Siebel updateSucceeded;SAP updateFailed	Normal stream; retry
6	Siebel updateFailed;SAP updateSuccessful	Perform compensation to roll back sap records, and then try again
7	Siebel updateFailed;SAP updateFailed	Normal stream; retry

Except for situations 1, 2, and 6 (as shown below), Data Consistency can be maintained without explicit processing of other situations.

You must track the status of the Web service to capture exceptions and take appropriate actions. Before introducing how to handle situations 1, 2, and 6, let's take a look at how to track the status of a specific web service.

The whole BPEL process has the following reply mode attributes:

• Siebel_add_status
• Siebel_update_status
• Sap_add_status
• Sap_update_status

All these attributes can contain values.Failed,SuccessOrNaThe value can be set in each location in the BPEL process. To setFailedStatus, the process can capture the soap Fault Caused by the target Web Service (for eachInvokeActivity uses a capture handler ). In the event of any fault, the client that calls the Customer Details Management module can resend the details.

The following describes how to manage exceptions:

Scenario 1
If the Siebel query fails, terminate the process and call the client to retry.

Scenario 2
When the customer's detailed information fails to be inserted in Siebel but is successfully inserted in SAP (they occur at the same time), data will not be consistent. In addition, the following problems may occur during retry:

• If you try to call the BPEL process to insert customer details into Siebel, the customer details may be repeated in SAP.
• If you try to call the BPEL process to update the customer details, the update in Siebel fails because there is no corresponding record in Siebel.

To deal with the above situations, you must use another webmethods web service with a BPEL compensation handler and scope to delete sap customer records before trying.

Scope and compensation activities are important tools for developing BPEL.ScopeIs the container and context of other activities;Scope ActivitySimilar{} Block. The scope simplifies the BPEL stream by grouping the functional structure, and provides fault, event, and compensation handling programs as well as data variables and associations.

The Oracle BPEL Process Manager provides two compensation processing structures:

• Compensation Handler-This handler is the business logic for rollback. You can define a handler for the process and scope.
• Compensation activity-this activity calls the compensation handler for a successfully completed internal scope activity and can only call it from the internal of the Fault Handler or another compensation handler.

Exceptions are captured at the scope level by the capture handler. In turn, the capture handler calls the compensation handler in the internal scope using the compensation activity. The compensation handler will perform the desired rollback.

Return to the example in this article. Assume that the BPEL process has two scopes: one internal scope and one external scope.SAP addAndSiebel addThe service calling activity comes from the external scope, but onlySAP addSource: internal scope. The compensation handler can be associated with an internal scope and called by the Compensation HandlerSAP DeleteService activities.

The capture block must be associated with an external scope to captureSiebeladdfault. Every occurrenceSiebeladdfaultThe compensation activity will compensate the internal scope and delete sap customer records. Note that compensation is successful only when all activities in the internal scope are successful.

Figure 6 shows the modified BPEL process with a scope and a compensation handler.

 

Figure 6Compensation logic used to delete sap customer records

 

Case 6
If the Siebel update fails and the SAP update succeeds, the transaction also fails. This will result in data inconsistency; therefore, you must use the compensation logic to roll back transactions in SAP. Compensation handling procedure andSAP updateService Association, and callSAP rollbackService. Modifications to the BPEL process follow the guidelines given above.

The explicit call of the compensation activity function is the basis of the error handling framework of BPEL. Unlike the traditional EAI compensation mechanism, BPEL provides a standardized rollback processing method.

After creating a BPEL process to orchestrate tibco and webmethods web services, let's take a look at how to make the communication between the BPEL, adapter and EAI tools safer.

Step 2: secure business communication.Security can be achieved at two levels: Outbound (calling secure tibco and webmethods services) and inbound (protecting the BPEL process ).

Outbound Security
The tibco and webmethods services need to be protected to prevent unauthorized access. Oracle BPEL Process Manager supports HTTP Basic Authentication and WS-Security Authentication for calling external services. This example demonstrates how to use HTTP authentication to protect tibco and webmethods services and mechanisms to call them from the BPEL process.

Web services deployed on the tibco businessworks and webmethods integration servers support basic HTTP authentication. When designing tibco Web Services, selectUse Basic AuthenticationCheck box. When deploying web services using tibco administrator, you can set service access levels for different users/roles. When using webmethods developer to design web services, you can set the ACL for different operations (Access Control List ).

If basic authentication is used when tibco and webmethods services are deployed, the Service performs this authentication each time it is called and the certificate should be transferred by the BPEL process. You can complete this task by setting the link property of two partners:HttpusernameAndHttppassword. You can set these two attributes as follows.

 

Figure 7Set httpusername and httppassword

 

To dynamically pass the certificate, useCopyRules.

  <copy><from variable="varUsername"<to partnerLink="p1" bpelx:property="httpUsername"</copy>

In addition, you can use WS-Security to protect tibco and webmethods services. The BPEL process can pass the WS-Security authentication header to the Web Service protected by WS-Security. You need to define the WS-Security headers supported by the Service in the Service's WSDL document. These header fields can then be processed as variables in the BPEL process (just like the message body data element. You canElastic shopflowFor more information about WS-Security Authentication, see the example (which can be downloaded from OTN.

Inbound Security
You can use HTTP authentication to prevent unauthorized users from calling the BPEL process. You can also set different certificates for different BPEL processes.

To achieve this security, you must enable HTTP Basic Authentication at the application server level. Then, the certificate can be extracted from the BPEL process and passed to the tibco and webmethods web services in the form of partner Link Attributes.

For more details about the security of BPEL, please refer to the "Protect BPEL process and service" network seminar on OTN.

Step 2: Record and handle errors in a centralized manner.Although business processes are secure and robust, building records and notification functions are equally important. Building a centralized record and error handling framework will make applications more robust, enhance reusability, and reduce development costs. You can use web services to build such a framework from the BPEL process and middleware. With the Oracle BPEL Process Manager, you can use the File adapter to add records to the service and require an email notification when an error occurs.

The following sample mode can be used in this framework:

 

Mode Element	Description
Role	Error, debug, warning, info, and other roles
Code	Error Code
Description	Error description
Source	Error Source
Email	ID of the email to which the notification must be sent

This process can be exposed as an asynchronous one-way Web Service, which will allow the service client to continue running without any delay. The following shows the centralized record and notification process.Logpolicy.

 

Figure 8Use the record and error handling framework of Oracle BPEL Process Manager

 

8,LogpolicyPerform the following operations:

1. Receive Information from external processes to be recorded
2. CallLog ServiceTo write the data to the log file. To this end,Log ServiceThe file adapter will be used.
3. If the record is successful, the process will be completed. In addition, ifRoleField inclusionError, The service will notify the relevant users by email. Retrieve email information from the received original message (see sample mode ).
4. If the notification fails, the process ends after the notification error is recorded in the file.

Each call activity in the main BPEL process can have a separateTry-catchBlock. The soap fault sent by the middleware process (or even contains exceptions caused by the final application)CatchBlock, and send it to the common record and error handling framework.

Figure 9 shows howSiebel addCall upon failureLogpolicyProcess.

 

Figure 9Call record and error handling framework from the Customer Details Management Module

 

Conclusion

The integration market is filled with powerful EAI products, and many of them are integrated through these products. BPEL is the only choice to make full use of these existing EAI solutions. By exposing existing middleware processes to Web Services and orchestrating them using the Oracle BPEL Process Manager, enterprises can begin to make SOA decisions.