MSMQ proxy service conception for load and Fault Handling

In the process of processing highly concurrent data applications, applications such as message queues are distributed to different servers for data processing, the purpose is to split the process data into N links and distribute them to different servers to improve the overall data processing capability.

Some time ago I was doing MSMQ load and intentional processing. During the test, I found that the problem that caused me a lot of trouble was passed. when the other party blocks the remote MQ operation. net will not report errors, but will write to the local outgoing queue, so it is difficult to transfer messages to other MQ servers in real time and effectively. on the other hand. NET remote MQ local machine must also be installed with the MQ service, another is. NET remote MQ operation is also very expensive.

To solve the above problem, we plan to implement an MSMQ Proxy component. The components are divided into two parts: server and client. The function of the server is to enable the listener to perform local MQ operations on requests.The client is relatively complex. It provides functions such as load migration and fault migration.

Msmqnode Structure

Both the application service and Message Processing Service write and read messages through msmqnode.

Load Processing

In order to adapt to future concurrency needs, do not adjustCodeYou can easily configure the load processing capability.

In addition to the multi-host load processing capability, the msmqnode client performs a simple round-robin operation on different services.

Troubleshooting

Multi-node connection pool

Msmqnode is designed to implement load and migration through software. Therefore, you do not need to set up load services such as LVS in the middle of deployment. A multi-node connection pool is designed to implement this function component.

 

When a connection exception occurs to a node, the connection pool can exclude the currently damaged node from the pool. In addition to the exclusion, the connection pool also regularly creates TCP connections for the damaged node, once the connection is recovered, the node will return to the connection pool again.