Anatomy PetShop Series 3: Message Processing at the data access layer of PetShop)

3. Message Processing at the PetShop data access layer

In system design, in addition to paying enough attention to security and transaction issues, performance is also an inevitable problem, especially a B/S structure software system, traffic, data traffic, and server load must be fully considered. To solve the performance bottleneck, in addition to upgrading the hardware system, the rationality of the software design is particularly important.
As I mentioned earlier, hierarchical structure design may affect the performance of data access to a certain extent. However, it is almost negligible compared with the benefits it brings to designers. To provide the performance of the entire system, you can also start with database optimization, such as using the connection pool, creating indexes, and optimizing query policies. For example, the database Cache is used in PetShop, for Order data with a large amount of data, the Order and Inventory databases are created separately by means of database sharding. In terms of software design, the most useful method is multithreading and asynchronous processing.
In PetShop4.0, Microsoft Messaging Queue (MSMQ) technology is used to complete asynchronous processing and the message Queue is used to temporarily store the data to be inserted, data Access provides access performance because it does not need to access the database. As for the data in the queue, the system waits for free before processing, and inserts the data into the database.
Message Processing in PetShop4.0 is divided into the following parts: message interface IMessaging, message factory MessagingFactory, MSMQ implementation MSMQMessaging, and OrderProcessor.
In terms of modularity, PetShop implements the "interface-oriented design" principle from the beginning to the end, separates Message Processing interfaces from implementations, and encapsulates messages in the factory mode to create objects, to achieve the goal of loose coupling.
As PetShop only uses an asynchronous Processing Method for order processing, only one IOrder interface is defined in the message interface IMessaging. The class diagram is as follows:
 
In the implementation of the message interface, considering that other data objects will use MSMQ in the future expansion, a basic Queue class is defined to implement the basic operations of message Receive and Send:
Public virtual object Receive ()
{
Try
{
Using (Message message = queue. Receive (timeout, transactionType ))
Return message;
}
Catch (MessageQueueException mqex)
{
If (mqex. MessageQueueErrorCode = MessageQueueErrorCode. IOTimeout)
Throw new TimeoutException ();
Throw;
}
}
Public virtual void Send (object msg)
{
Queue. Send (msg, transactionType );
}
The queue object is of the System. Messaging. MessageQueue type and serves as the queue for storing data. The MSMQ queue is a persistent queue. Therefore, you do not have to worry about the loss of order data when placing orders continuously. When the timeout value is set in PetShopQueue, OrderProcessor periodically scans the order data in the queue based on the timeout value.
In the MSMQMessaging module, the Order object implements the IOrder interface defined in the IMessaging module. It also inherits the basic class PetShopQueue, which is defined as follows:
Public class Order: PetShopQueue, PetShop. IMessaging. IOrder
The implementation code of the method is as follows:
Public new OrderInfo Receive ()
{
// This method involves in distributed transaction and need Automatic Transaction type
Base. transactionType = MessageQueueTransactionType. Automatic;
Return (OrderInfo) (Message) base. Receive (). Body;
}

Public OrderInfo Receive (int timeout)
{
Base. timeout = TimeSpan. FromSeconds (Convert. ToDouble (timeout ));
Return Receive ();
}

Public void Send (OrderInfo orderMessage)
{
// This method does not involve in distributed transaction and optimizes performance using Single type
Base. transactionType = MessageQueueTransactionType. Single;
Base. Send (orderMessage );
}
Therefore, the class diagram should be as follows:
 
Note that in the Receive () method of the Order class, use the new Keyword instead of the override keyword to override the Receive () virtual method of the parent class PetShopQueue. Therefore, if the following object is instantiated, The Receive () method of PetShopQueue will be called, instead of the Receive () method of the subclass Order:
PetShopQueue queue = new Order ();
Queue. Receive ();
From the design point of view, because PetShop adopts the "interface-oriented design" principle, if we want to create an Order object, we should adopt the following method:
IOrder order = new Order ();
Order. Receive ();
Considering possible changes to the implementation of IOrder, PetShop still uses the factory mode to encapsulate the creation of IOrder objects with specialized factory modules:
 
In the QueueAccess class, use the CreateOrder () method to use reflection technology to create the correct IOrder type object:
Public static PetShop. IMessaging. IOrder CreateOrder ()
{
String className = path + ". Order ";
Return PetShop. IMessaging. IOrder) Assembly. Load (path). CreateInstance (className );
}
The path value is obtained through the configuration file:
Private static readonly string path = ConfigurationManager. deleettings ["OrderMessaging"];
In the configuration file, the value of OrderMessaging is set as follows:
<Add key = "OrderMessaging" value = "PetShop. MSMQMessaging"/>
The factory mode is used to create objects, which is convenient for calling at the business layer. For example, the OrderAsynchronous class in The BLL module:
Public class OrderAsynchronous: IOrderStrategy
{
Private static readonly PetShop. IMessaging. IOrder asynchOrder = PetShop. MessagingFactory. QueueAccess. CreateOrder ();
Public void Insert (PetShop. Model. OrderInfo order)
{
AsynchOrder. Send (order );
}
}
Once the implementation of the IOrder interface changes, this implementation method allows the customer to modify the configuration file without modifying the Code, so that the re-compilation and deployment of the Assembly can be avoided, this allows the system to flexibly respond to changes in requirements. For example, to define a SpecialOrder that implements the IOrder interface, you can add a module, such as PetShop. specialMSMQMessaging, while the class name is still Order, then we only need to modify the value of OrderMessaging in the configuration file:
<Add key = "OrderMessaging" value = "PetShop. SpecialMSMQMessaging"/>
OrderProcessor is a console application, but it can be designed as a Windows Service as needed. It is designed to receive the Order data in the message queue and insert it into the Order and Inventory databases. It uses multithreading technology to improve system performance.
In the OrderProcessor application, the Main function is used to control the thread, and the core execution task is implemented by the ProcessOrders () method:
Private static void ProcessOrders ()
{
// The transaction timeout shocould be long enough to handle all of orders in the batch
TimeSpan tsTimeout = TimeSpan. FromSeconds (Convert. ToDouble (transactionTimeout * batchSize ));

Order order = new Order ();
While (true)
{
// Queue timeout variables
TimeSpan datetimeStarting = new TimeSpan (DateTime. Now. Ticks );
Double elapsedTime = 0;

Int processedItems = 0;

ArrayList queueOrders = new ArrayList ();

Using (TransactionScope ts = new TransactionScope (transactionscospontion. Required, tsTimeout ))
{
// Receive the orders from the queue
For (int j = 0; j <batchSize; j ++)
{
Try
{
// Only receive more queued orders if there is enough time
If (elapsedTime + queueTimeout + transactionTimeout) <tsTimeout. TotalSeconds)
{
QueueOrders. Add (order. ReceiveFromQueue (queueTimeout ));
}
Else
{
J = batchSize; // exit loop
}

// Update elapsed time
ElapsedTime = new TimeSpan (DateTime. Now. Ticks). TotalSeconds-datetimeStarting. TotalSeconds;
}
Catch (TimeoutException)
{
// Exit loop because no more messages are waiting
J = batchSize;
}
}
// Process the queued orders
For (int k = 0; k <queueOrders. Count; k ++)
{
Order. Insert (OrderInfo) queueOrders [k]);
ProcessedItems ++;
TotalOrdersProcessed ++;
}

// Batch complete or MSMQ receive timed out
Ts. Complete ();
}

Console. writeLine ("(Thread Id" + Thread. currentThread. managedThreadId + ") batch finished," + processedItems + "items, in" + elapsedTime. toString () + "seconds. ");
}
}
First, it uses PetShop. BLL. the public method ReceiveFromQueue () of the Order class is used to obtain the Order data in the message queue and put it into an ArrayList object. However, PetShop is called. BLL. the Insert method of the Order class inserts it into the Order and Inventory databases.
In the PetShop. BLL. Order class, the Insert () method of the IOrderStrategy interface is called instead of directly executing the Insert Order operation:
Public void Insert (OrderInfo order)
{
// Call credit card procesor
ProcessCreditCard (order );

// Insert the order (a) synchrounously based on configuration
OrderInsertStrategy. Insert (order );
}
Here, a policy mode is used. The class diagram is as follows:
 
In the PetShop. BLL. Order class, the configuration file is still used to dynamically create the IOrderStategy object:
Private static readonly PetShop. IBLLStrategy. IOrderStrategy orderInsertStrategy = LoadInsertStrategy ();
Private static PetShop. IBLLStrategy. IOrderStrategy LoadInsertStrategy ()
{
// Look up which strategy to use from config file
String path = ConfigurationManager. receivettings ["OrderStrategyAssembly"];
String className = ConfigurationManager. etettings ["OrderStrategyClass"];

// Using the evispongiven in the config file load the appropriate assembly and class
Return (PetShop. IBLLStrategy. IOrderStrategy) Assembly. Load (path). CreateInstance (className );
}
As OrderProcessor is a separate application, the configuration file used by OrderProcessor is different from that used by PetShop. It is stored in the App of the application. in the config file, the configuration of IOrderStategy is as follows:
<Add key = "OrderStrategyAssembly" value = "PetShop. BLL"/>
<Add key = "OrderStrategyClass" value = "PetShop. BLL. OrderSynchronous"/>
Therefore, the process of Asynchronously inserting orders is shown in:
 
In addition to asynchronous processing, Microsoft Messaging Queue (MSMQ) is mainly a distributed processing technology. In distributed processing, an important technical element is message processing. the Message class has been provided in the Messaging namespace and can be used to transmit messages. on the premise that the sender and receiver of a Message should have a unified interface specification in terms of data definition.
The Application of MSMQ in distributed processing has been implemented in projects I participated in. Dealer, a distributor, is used to develop a large system for an automobile manufacturer.. Net client, which needs to pass data to the management center, and the data will be used by Oracle's EBS (E-Business System. The distributor management system (DMS) adopts the C/S structure. The database is SQL Server, and the EBS database of the Management Center of the automobile manufacturer is Oracle. Data transmission between two systems is involved.
The implementation architecture is as follows:

First, Dealer data is transmitted to MSMQ Server through MSMQ. In this case, you can insert the data into the SQL Server database and transmit the data to a dedicated File Server using FTP. Then, use IBM's EAI technology (Enterprise Application Integration, Enterprise Application Itegration) to regularly write files on the file server to the EAI database server using interface specifications, and finally wrote in the Oracle database of EBS.
The above architecture is a typical distributed processing structure, and the core of technology implementation is MSMQ and EAI. Since we have defined a unified interface specification, after a file is formed through the message queue, the data has nothing to do with the platform. the distributor management system under the. Net platform can be integrated with Oracle's EBS to complete data processing.