Share runtime data through enterprise distributed cache

Many enterprises use Microsoft. NET Framework and Java applications in combination.ProgramEspecially for large and medium-sized enterprises that cannot rely only on a single technology for various reasons. Generally, enterprises use web applications, Service-Oriented Architecture (SOA) Web Services, and other server applications to process a large number of transactions.

Many applications need to share data with each other during running. Generally, these applications operate on common business data stored in the database. They generally face continuous data streams (such as financial transaction applications) and need to process data multiple times at runtime and share the results with other applications.

Although the database is the primary storage area for permanent data storage, it is not suitable for data sharing during running hours. One of the reasons is that the high performance cannot always be guaranteed when reading data from the database. Furthermore, database scalability is not good in transaction processing, so it is likely to become a bottleneck soon and reduce the speed of all applications dependent on it.

In addition, data cannot be shared in real time and effectively. Real-time data sharing requires that once an application updates data, all other applications interested in the data should receive a notification immediately. Similarly, some applications may be waiting for some data types to be created and available. Once these operations occur, these applications should be notified immediately.

Whether the applications that need to share data are all based on. NET Framework, or some are Based on. NET while others are based on Java, This is a common problem. In fact, if the application uses both. NET and Java, the problem is more serious, because for such applications, there is no automatic way to share data locally at the application level.

Solution: enterprise distributed cache

Fortunately, enterprise distributed cache can solve these problems. In-memory storage can span multiple servers, and the server memory can be centralized in one unit, so the memory storage capacity is scalable. The transaction capacity is also scalable. The more servers you add, the larger the transaction load that can be processed.

Enterprise distributed cache also provides an event notification mechanism for applications to notify each other after updating data. Therefore, you can have an asynchronous event notification mechanism, where one application generates data and other applications can use this data to create a producer/user model or a publishing/subscription model. Multiple applications can subscribe to certain data types. when the data is published, these applications will receive notifications.

There is also a read/write mechanism, that is, the enterprise distributed cache itself can read a large amount of data from the data source and application. Whether the application is based on Java or. NETCodeCan be easier, because you can read data from the enterprise distributed cache without embedding the database access code.Figure 1Is a simple example of A. NET Framework application that uses enterprise distributed cache.

Figure 1. NET applications that use enterprise distributed cache

  
  
 
   
 
   
Using system;
  
   
 
   
...
  
   
 
   
Using alachisoft. ncache. Web. caching;
  
   
 
   
 
  
   
 
   
Namespace Client
  
   
 
   
{
  
   
 
   
Class Program
  
   
 
   
{
  
   
 
   
Static string _ scachename = "myappcache ";
  
   

   
Static cache _ scache = ncache. initializecache (_ scachename );
  
   
 
   
 
  
   
 
   
Static void main (string [] ARGs)
  
   
 
   
{
  
   
 
   
String employeeid = "1000 ";
  
   
 
   
String key = "Maid: employeeid:" + employeeid;
  
   
 
   
  
   
 
   
// First check the cache for this employee
  
   
 
   
Employee EMP = _ scache. Get (key );
  
   
 
   
 
  
   
 
   
// If cache doesn' t have it then make database call
  
   
 
   
If (EMP = NULL)
  
   
 
   
{
  
   

   
EMP = loademployeefromdb (employeeid );
  
   
 
   
 
  
   
 
   
// Now add it to the cache for next time
  
   
 
   
_ Scache. insert (Key, EMP );
  
   
 
   
}
  
   
 
   
}
  
   
 
   
}
  
   
 
   
}
  
   
 
   
  
  

In addition, enterprise distributed cache can synchronize any data changes made to the database based on other third-party applications. It is connected to the database, so as long as a data type in the database changes, it will receive a notification.Figure 2The illustration shows how. NET and Java applications share data with each other through the enterprise distributed cache at runtime.

Figure 2. NET and Java applications share data through distributed cache

. NET and Java applications share data

With enterprise distributed cache, multiple applications (whether Based on. NET or Java) can access the same cache and share data through the cache. If only. Net Applications (or Java applications) share data through the distributed cache, the application can store the object as a local binary format and serialize/deserialize it. However, if two types of applications share data with each other, the data needs to be stored in the distributed cache in a portable data format.

This is because when the. NET application stores an object in the distributed cache, it will actually convert the object to an XML document and store the XML. On the other hand, when a Java application reads the data from the distributed cache, the XML is converted to a Java object. In fact, XML is used as a portable data storage mechanism because. Net objects are converted to XML and then from XML to Java, and vice versa.

There are many openSource codeLibrary can help you convert. Net or Java objects to XML, and then convert back to the object format. Of course, you can also develop it on your own, but I suggest you select the open source code library. I personally prefer web objects in XML (wox, woxserializer.sourceforge.net) developed by Carlos jaimez and Simon Lucas ). In this article, examples of conversion from Java to. Net from their website will be used (with their consent ).Figure 3The student and course classes defined in Java and C # are displayed.

Figure 3Student and course classes written in Java and C #

  
  
 
   
 
    
 // Java classes 
 
    
 public class student 
 
    
{
 
    
 private string name; 
 
    
 private int registrationnumber; 
 
    
 private course [] courses; 
 
    
}
 
    
 public class course 
 
    
{
 
    
 private int code; 
 
    
 private string name; 
 
    
 private int term; 
 
    
}
 
    
 
 
    
 //*********************************** **************** 
 
    
 //. Net classes in C # 
 
    
 public class student 
 
    
{
 
    
 private string name; 
 
    
 private int32 registrationnumber; 
 
    
 private course [] courses; 
 
    
}
 
    
 public class course 
 
    
{
 
    
 private int32 code; 
 
    
 private string name; 
 
    
 private int32 term; 
 
    
}
 
    
 
 
   
 
  

If we use the. NET and Java applications to store the above student and course objects in the enterprise distributed cache, then we can use the wox library to convert these objects to XML. Then, if the application wants to read these objects from the enterprise distributed cache, it will read the wox library again and convert the XML to Java or. NET object format.Figure 4Displays student and course classes converted to XML format.

Figure 4Java and. Net classes converted to XML

  
  
 
   
 
   
<Object type = "student" id = "0">
  
   
 
   
<Field name = "name" type = "string" value = "Carlos jaimez"/>
  
   
 
   
<Field name = "registrationnumber" type = "int" value = "76453"/>
  
   
 
   
<Field name = "courses">
  
   
 
   
<Object type = "array" elementtype = "Course" length = "3" id = "1">
  
   
 
   
<Object type = "Course" id = "2">
  
   
 
   
<Field name = "code" type = "int" value = "6756"/>
  
   

   
<Field name = "name" type = "string"
  
   
 
   
Value = "XML and Related Technologies"/>
  
   
 
   
<Field name = "term" type = "int" value = "2"/>
  
   
 
   
</Object>
  
   
 
   
<Object type = "Course" id = "3">
  
   
 
   
<Field name = "code" type = "int" value = "9865"/>
  
   
 
   
<Field name = "name" type = "string"
  
   
 
   
Value = "Object Oriented Programming"/>
  
   
 
   
<Field name = "term" type = "int" value = "2"/>
  
   

   
</Object>
  
   
 
   
<Object type = "Course" id = "4">
  
   
 
   
<Field name = "code" type = "int" value = "1134"/>
  
   
 
   
<Field name = "name" type = "string" value = "E-Commerce programming"/>
  
   
 
   
<Field name = "term" type = "int" value = "3"/>
  
   
 
   
</Object>
  
   
 
   
</Object>
  
   
 
   
</Field>
  
   
 
   
</Object>
  
   
 
   
  
  

In your application, wox should be called from the cache layer or data access layer.

Item-based Event Notification

The event notification mechanism is powerful. Multiple applications (. NET and Java) can use this mechanism to coordinate asynchronous data sharing. With the help of this mechanism, applications can avoid costly database polling. This mechanism can be shared between. NET and Java applications, so they can seamlessly notify each other.

A common type of event notification is item-based notification. In this type, the application registers various cache item keys of interest (may or may not exist in the cache ), the application will receive a notification once this item is added, updated, or deleted in the distributed cache for any reason. For example, if an item is deleted due to expiration or eviction, a notification of the deletion event is triggered.

Both. NET and Java applications can register interest in the same cache item and receive notifications about the item. Notifications usually include affected cache items, as described in the preceding section, which are converted to. Net or Java format based on the application type.

Custom Event Notifications generated by the application

For. NET and Java applications, enterprise distributed cache is also a powerful event propagation platform. Any application that is connected to the enterprise distributed cache can trigger a custom event in the cache. Then, no matter where the application is located, as long as you register an interest in these custom events, the cache notification will be received. This provides a powerful event propagation mechanism independent of languages and platforms in enterprise distributed cache.

Applications can use this function to coordinate asynchronous data sharing. For example, if an application puts data in the distributed cache and then triggers a Custom Event, other applications that plan to use or process the data will receive a notification immediately.

Event Notification based on continuous Query

Although item-based event notification is powerful, the application is required to know the key of the cache item. If you use item-based event notifications in conjunction with other group functions (such as tags, groups, and sub-groups) that are often provided in the enterprise distributed cache, then, the application can be notified of almost all the situations based on the occurrence of each cache item.

However, there are two restrictions on item-based events. First, as mentioned above, the application must be aware of the keys of all cache items to be notified. Second, the application will receive notifications regardless of the changes to these items. Applications cannot set more detailed standards so that they receive notifications only when specific data changes occur.

To cope with such situations, enterprise distributed cache provides continuous queries, which are similar to SQL queries and can capture business rules related to data of interest to applications. Continuous query is not a search query, but a "standard" maintained by the enterprise distributed cache ". As long as the content is added or updated in the distributed cache, the operation is compared with the continuous query standard. If the criteria match, an event is triggered and a standard application for continuous query is released.

Through continuous queries, applications can wait for more complex changes and receive notifications only when these changes occur.

Read-And write-through handlers

In many cases, the data that the application tries to read is not in the enterprise distributed cache and must be read from the database. In this case, the application can directly access the database and read the data, but this means that all applications must copy the same data access code (especially in. NET and Java ). Alternatively, you can require the enterprise distributed cache to read the data from the database when you need the data.

With the read/write function, enterprise distributed cache can directly read data from data sources. Applications can simplify their code without accessing the database. They only need to require the enterprise distributed cache to provide data for it. If the data is not in the cache, access the data source and read the data.Figure 5Shows how enterprise distributed cache uses the Read and Write Functions.

Figure 5Usage principle of read/write

Note one thing. Although it is of great benefit to allow the distributed cache to read data from the database, there are still many data types that are better to be read directly from the database by applications. If you want to read a dataset that contains a complex join, you 'd better read it by the application and put it in the distributed cache.

Database Synchronization

Because a large amount of data is stored in the enterprise distributed cache, it is necessary to ensure that the data is synchronized with the primary data source (usually relational database. Enterprise distributed cache provides this function.

Through the Database Synchronization function, the application can specify the relationship (dependency) between the cache entry and the row in the database table ). As long as the data in the database changes, the database server will trigger a. Net event (even the SQL Server 2005/2008 database) and notify the enterprise of this change in distributed cache. Not supported. for other databases of the net event, enterprise distributed cache also provides configurable Round Robin so that the distributed cache can poll the database (for example, once every 15 seconds) and synchronize the data when the data is changed.

The distributed cache then deletes the data from the cache or reads a new copy of the data (if the read-through function is configured ).Figure 6Shows how enterprise distributed cache is synchronized with SQL Server.

Figure 6Database Synchronization in distributed cache

High Availability: self-repairing dynamic clusters

Enterprise distributed cache can be used as a runtime database sharing platform between multiple applications (. NET to. net,. Net to Java, and Java to Java ). In many cases, these applications are critical to your company.

Because many key task applications rely on enterprise distributed cache, distributed cache must have high availability. Enterprise distributed cache cannot be paralyzed or stopped, and it should be completely out of service for maintenance or other normal operations.

Enterprise distributed cache achieves high availability through a self-healing dynamic cache server cluster. Self-repair indicates that the cluster knows all its members and dynamically adjusts when a member leaves or joins the cluster. In addition, data can be duplicated to ensure reliability. If a cluster member leaves, the backup data can be automatically used by the application. All these functions must be executed quickly without any interruption to applications that use enterprise distributed cache.

Scalability: cache partitioning and Replication

Many applications that use enterprise distributed cache are highly transactional applications. Therefore, the load on the cache cluster increases rapidly. However, if the response time of enterprise distributed cache is prolonged, its value will be greatly reduced. In fact, enterprise distributed cache is better than relational databases within a certain range. Because it can add more servers in the dynamic group, it processes more transactions per second than the database. However, the scalability cannot be achieved unless the data in the distributed cache is stored intelligently. This can be achieved through data partitions. Each partition is replicated to ensure reliability.

Thanks to the enterprise distributed cache, you can use the partition topology for expansion.Figure 7Displays the partition replication topology.

Figure 7Partition replication topology for reliable Scaling

Enterprise distributed cache automatically partitions all data stored in the cache. Each partition is stored on a different server, and the backup of the partition is created and stored on another server. This ensures that data will not be lost even if any server is shut down.

In short, you can use the Partitioning technology to add more cache servers to a dynamic group to expand the storage capacity. As the number of servers increases, the amount of transactions processed per second will also increase. In addition, replication ensures data reliability because server downtime does not cause data loss.

Functions and collaboration

All in all, enterprise distributed cache is an ideal way for high transaction. NET and Java applications to share data with each other. Its powerful event propagation mechanism, including item-based Event Notifications, Custom Event Notifications generated by applications, and Event Notifications based on continuous queries, ensures real-time data sharing.

In terms of design, enterprise distributed cache is not only fast but also scalable. High-speed source operations in the memory. Scalability comes from the ability to add multiple servers. It partitions the actual storage, stores each partition on different servers, and stores the backup of the partition on another server (such as a raid disk.

Today's applications have higher functional requirements than in the past. They need to share data and interact with each other in a more collaborative manner. They not only require high speed, but also meet extremely high load requirements to avoid compromising performance and scalability. Moreover, they must perform operations across multiple platforms so that. NET applications can work with Java applications transparently and effectively. Enterprise distributed cache can help us achieve all of the above goals.

Iqbal Khan As the President and technology promoter of alachisoft (alachisoft.com), The ncache (. NET distributed cache) provided by alachisoft can improve the performance and scalability of enterprise applications. Khan received a master's degree in computer science from the University of danale in 1990. You can contact him by email: iqbal@alachisoft.com.

Link: http://msdn.microsoft.com/zh-cn/magazine/gg232763.aspx