"Data real-time access" solution for large systems

Recently, as one of the many external vendors, the company needs to rely on a large platform system (hereinafter referred to as BIG-S) to provide some services to specific users.

As a WEB application developed by an external manufacturer (hereinafter referred to as SMALL-S), it is necessary to extract the basic data from the Big-s, including the user, organization structure, code table ... Part of the field into the Local data table.

Fusing Small-s own features as a prerequisite for building small-s WEB projects.

Small-s needs to be consistent in real time with Big-s's key underlying data, focusing on the following features in Big-s data interactions.

1. BIG-S provides external manufacturers with roughly three forms of interaction (EJB,WEBSERVICE,JMS), with data tables close to 4000 +.

2. Big-s is provided to the external vendor Oracle database Db_select, of course Db_select only contains Select permissions.

As for which databases db_select real-time synchronization big-s, how to load deployment ..... These are not the things to be concerned about.

A few rounds of the internal PK, the final determination of a better solution, this article will be a step forward to describe several proposed methods, to other counterparts to do reference, record summary supplemented.

Perhaps some people will be puzzled, directly to the basic data needed to extract it all at once?

Big-s in real-time in the operation of these data, extracted data will certainly and big-s error, has such data as the basis of the Web project, waiting to be led please go to tea.

1. Use of external services provided by Big-s

Using the external services provided by big-s, the process of obtaining data is processed in real time, and the processing speed is quite good.

When getting data, you need some preconditions to organize your request to get exactly the data you need, such as you need a user information, a pre-condition may be a user ID, an organization ID ...

• The first thing to consider is that Small-s needs to expand its business to find out if Big-s has a similar service for you to use, if any.
• What if not? Tell Big-s to give you a custom development?
• Second, when Small-s needs to do similar query functions, it is assumed that when querying all user information, you do not have preconditions.
• Finally, while it's good to get data response events, SMALL-S needs to organize requests, remove irrelevant, and convert to the expected SMALL-S data structure objects, which are time consuming.
• Ejb/webservice service calls, interaction rules are not the focus of this article to be elaborated, interested can see for themselves.

2. Dual Data Source Configuration

Use the services provided by Big-s to obtain data in real time, after being broken up by the crowd.

Then someone put their eyes on the big-s to provide the database db_select above.

The specific idea is to link both small-s and big-s Db_select, the same database resource pool as the common one.

<BeanID= "Small-s"class= "Com.mchange.v2.c3p0.ComboPooledDataSource"Destroy-method= "Close">        < Propertyname= "Driverclass"value= "${smalls.driverclassname}" />        < Propertyname= "Jdbcurl"value= "${smalls.url}" />        < Propertyname= "User"value= "${smalls.username}" />        < Propertyname= "Password"value= "${smalls.password}" />        ........    </Bean>    <BeanID= "Big-s"class= "Com.mchange.v2.c3p0.ComboPooledDataSource"Destroy-method= "Close">        < Propertyname= "Driverclass"value= "${bigs.driverclassname}" />        < Propertyname= "Jdbcurl"value= "${bigs.url}" />        < Propertyname= "User"value= "${bigs.username}" />        < Propertyname= "Password"value= "${bigs.password}" />......</Bean>

• With Spring, you can easily do dual database configuration, resource pools can use common c3p0, DPCP ....
• Development can be in the specific Dao layer by the programmer to determine the specific needs of the data source injection, or do point design to inject two data sources into the parent DAO, the base class DAO inherit.
• At this time, as long as you know the specific business, the specific table name, you can get easy access to the data in the Big-s, this is still quite satisfactory.
• But the part of the underlying data that changes in real time, reads big-s data into small-s?
• Do timed tasks? How much is the time interval set to fit? Interval big-s user data changes, small-s users in business operations what should be done?
• This solution provides some ideas, and when the end of the death in the synchronization, time consuming power.

3. Oracle Db_link + synonyms

This idea is a seasoned DBA of the database, and on Small-s it is recommended that the database be linked to Big-s, and then a synonym for the table should be established on the small-s.

Like views, synonyms do not occupy the actual storage space, only the definition of the synonym is saved in the data dictionary, and the query big-s table is real-time data.

To establish a database link and synonym process:

    --CREATE DATABASE link    Create  Public DatabaseLink Big-S_link Connect to 'User'Identified by 'Password'using'( Description = (address_list (address = (protocol = TCP) (host ='Ip') (port ='Port')) ) (Connect_data = (SID ='Instance Name') (server = dedicated )))'; --CREATE database synonym   Create or ReplaceSynonym S_a forB_a@big-S_link;

• In the actual analysis, big-s common 4000+ data table, small-s at most will use only 200 +.
• With the obligation extension, you can choose the data table in Big-s, which is very convenient to expand.
• In this way, you do not need to configure two data sources, only the Small-s database can be linked.
• The use of synonyms guarantees the real-time consistency of the data, and does not result in interest disputes due to user information errors.

So attached big-s data real-time access to the solution should have a relatively good landing, as for the use of what the situation, need to run the project online for a period of time to know.

Big-s after all is a big company, specifically I do not mention, and is a multi-vendor synergy, remember each other, there are problems, in their own side of the problem after the location to ask the other party.

Pay attention to the minimum respect for people, speak a little, good relations, so that work will be more efficient.

"Data real-time access" solution for large systems