[Short MMS] C # development and design of the short MMS module (1) -- Architecture

We will discuss the implementation of the Short MMS module in the following aspects:

[Short MMS] C # Short MMS module development and design (1) -- Architecture (http://www.cnblogs.com/CopyPaster/archive/2012/12/07/2806776.html)
[Short MMS] C # Short MMS module development and design (2) -- configuration (http://www.cnblogs.com/CopyPaster/archive/2012/12/10/2811626.html)
[Short MMS] C # Short MMS module development and design (3) -- protocol (http://www.cnblogs.com/CopyPaster/archive/2012/12/12/2814918.html)
[Short MMS] C # Short MMS module development and design (4) -- Other (http://www.cnblogs.com/CopyPaster/archive/2012/12/17/2821715.html)

Before I wrote a MMS Message blog (http://www.cnblogs.com/CopyPaster/archive/2012/04/28/2475240.html), in fact, the main meaning is to paste the MMS Message for everyone to see. As I knew at the time, mobile provided Java APIs, but there was no.. net, so if you need to use. to develop the MMS gateway Adapter Service, you must construct your own MMS messages. Therefore, paste a simple MMS Message for your reference. However, when I read the comments, more people are concerned about the sending efficiency. In fact, from the process of developing short MMS, the short MMS protocol itself is only part of the research and learning process, in fact, we are more focused on how to interact with and connect systems with a large amount of data faster, better, and more secure. Here we will talk about some of my ideas about the short MMS module, I plan to talk about it from a few points. I will introduce it one by one in the future. The first article will talk about the overall architecture.In fact, the overall structure and some processing methods described in this article are not only applicable to short MMS applications, but also suitable for interaction and processing of various modules in large data volumes.

Let's take a look at the following figure:

Note:

Upper-layer applications

Upper-layer applications refer to all subsystems that need to send downstream messages. Of course, for short messages, it is generally necessary to check whether the business requirements need to process upstream messages. (For MMS, the protocol supports upstream messages, however, it is generally difficult to come across such practical applications? I think it may only work on TV stations and may encounter MMS uplink. For example, if a user uploads an image MMS, the host will display the image to the audience );

In the preceding figure, all the modules that need to send short MMS messages can send one message to the AMQ message queue, however, the actual application in one of my projects is: there is a layer of service (short MMS platform) in the middle. for upper-level applications, if you want to send short MMS messages, you only need to send a message to the short MMS platform, short MMS sends messages to the corresponding AMQ message queue. As for this short MMS platform, I think some practices and ideas are helpful for processing and interacting with large amounts of data. Here I will give you some tips for your reference. 1) The upper-layer applications and short MMS platform adopt the RPC Method for message communication; provide the DLL interface to the upper-layer applications for direct use (in reality, due to the large amount of downstream data, the short MMS platform is a group of services, and there is a mechanism to add a new platform (I call this processing method a sub-pool, the new platform is actually a plus pool; about the sub-pool, in short, each pool is a set of applications and databases. A global pool on top of the pool contains global services and data (such as pool partitioning rules )), each platform only processes the downlink of a specific tail number. to isolate this change, it provides a DLL Interface for upper-layer applications to use directly, upper-layer applications do not need to pay attention to the situation where a downstream message will be processed on the short MMS platform); 2) in addition to sending messages to AMQ message queues, the short MMS platform is also a group of clusters, therefore, some other issues need to be considered when sending/receiving messages. In this case, the upper-layer applications are isolated from this change.) There are also some other logics (blacklist filtering; de-duplication verification (de-duplication consumes resources, and each platform maintains a B + tree processing );

 

AMQ Message Queue Cluster

AMQ refers to activemq. Why do we use activemq? Because 1) free; 2) it is inconvenient for MSMQ to start when handling cross-host messages; I have posted a blog on activemq configuration and performance tuning, if you do not understand activemq can go to the reference (http://www.cnblogs.com/CopyPaster/archive/2012/04/27/2473179.html); about the AMQ queue cluster message sending and receiving principles need to explain:

1) long connections are used for message sending and receiving. the sender and receiving sides must maintain a connection pool. If the connection fails, the connection status is changed to unavailable; the self-maintained connection pool needs to start another thread to regularly check whether there are available connections and try to reconnect. If the connection is successful, change the connection status to available. (Note: Use AMQ message queue, in fact, there is a key point to note: fast message consumption to avoid a large backlog of messages. Of course, if there are many Queue Server Clusters and their memory is large, the threshold value of this backlog can be gradually increased. For AMQ queues, AMQ has no efficiency in short connections, therefore, persistent connections are generally used for processing ;)

2) The message sender randomly selects an available connection for sending each message;

3) The Message Receiver listens to all connections (if there is no message sleep for a short time; If the connection is broken, sleep for a long time );

According to the above rules, the load effect can be achieved, and the hot master/backup effect can also be achieved;

 

Short MMS Service Cluster

Key points:

1) Each short MMS Service must work with a set of MSMQ message queues on the local machine. After receiving a message from an AMQ queue, the short MMS service first caches the message to the MSMQ on the local machine, in this way, the rapid consumption of AMQ is further achieved;

2) For the received receipt/status report/upstream information, the service first caches the message to the memory queue. (generally, if the downstream traffic is large in another application scenario, moreover, a certain transmission speed is required. The short MMS service is generally connected by province. In this way, a short MMS service is connected to multiple gateways to receive such information; of course, sometimes the data volume is not large, and the method of connecting to a gateway only );

3) For short MMS services, we generally need to consider flow control and sliding windows (sliding windows are not described in the MMS protocol. To improve the transmission speed of MMS, we can use this method for reference, in addition, the MMS is http post, which is generally asynchronous, otherwise it is difficult to control the flow control), connections (the short message is based on the socket method, and the protocol uses cmpp_active_test for Link detection; the MMS is not connected to httppost, you do not need to consider this)

4) for the SMS service, if it is accessed separately, and not every service can handle all the gateways, you need to consider the message jump problem (in the case of provincial access, on the one hand, gateways in different provinces have different capabilities. On the one hand, the actual business volume varies in different provinces. Or, due to insufficient connections and other factors, some "small provinces" do not need to run all the text message services, this problem may occur );

 

Upstream/downstream history/receipt/status report import Service

Generally, if the downstream traffic is large, the amount of receipt/status reports is also large. The import Service is responsible for importing the messages in the message queue to the database for a long time. Generally, when the data volume is large, you can import data in batches (for example, you can write data to the database only after receiving 10 thousand messages using batchinsert. In practice, you can import data in batches, it takes less time to write data to the database than to receive the batch of messages );