Some summarization and thoughts on the design of Push system (I.)

Discussion on push, MQTT and Mosquitto can join group: 221779856 message and notification

The message in this article refers to the string to be sent to the push system, the notification string that is sent to the client via a long connection service, which is used only within the push system and is not perceived by the upper application using the push system; security

In a push system, security is most concerned with the safety control of long connections and the security of data transmitted over long connected channels. Give an example of security controls for long connections: Some people know that your long connection service IP address and the port after the forgery of a large number of TCP connections, will be long connected to the server side of the serious security and performance problems, such as the use of common protocol long Connection service is particularly serious, For example, many companies now use the MQTT protocol as long connection services, if you do not check the connection of long connections, you can get the IP and port of long connection service by grasping the packet on the client, and can analyze the type of protocol, then can forge a large number of connections to mount to the long connection server, send and receive data, This creates a security risk and creates incalculable pressure on the server's performance. For the data transmission security on long connection, it refers to the security problems such as whether or not it can be stolen and forged by others in the long connection. 1.1 Safety and efficiency

long connection uses TCP or TLS.
Personally think that this is more based on the push of the application scenario, the development of business follow-up to the comprehensive judgment; If the client has a few long connections for the current and foreseeable time in the scenario, but the security requirements for the data transfer are high, TLS can be used; if the current and foreseeable extent, The number of clients cannot be judged, and the growth of long connections is unpredictable, for example, to tens of millions of or even billions of online connections, but the security requirements for data are relatively low, and TCP connections can be used.
Using TLS in a long connection service provides a secure transport channel, but TLS can severely affect the performance of the server, and based on previous testing, using TLS to reduce the performance of long connection services in the context of mobile Internet 50%~80%, in previous experience, The effect of this is not the addition and decryption of data but the frequent disconnection and establishment of the connection, because in the mobile internet situation, the client network exception is a normal, when the long connection service supports 100,000 connections, the number of disconnection per second will likely reach 1000 times, if the use of TLS may also be higher, Of course these values are for reference only and may vary depending on the application scenario.
can there be a way to balance security and service-side efficiencies.
Personal advice is: Long connection services using TCP connections, data encryption by the long connection on the service to ensure that the long connection to the service is encrypted after the data, so as to ensure the security of data, but not too much loss of long connection service itself performance. Here to note that the data encryption and decryption of the symmetric key transmission, can be applied to the client through HTTPS long connection service address by the server generated and delivered to the user, the symmetric key should be each client in each landing long connection service is not the same time. 1.2 access Control

A personal view: Many long connection services provide a username and password to control access to the connection, in the TCP transmission, this measure can hardly play any role, these user name or password data (whether or not encrypted) will be easily intercepted by people to obtain, Someone else can take your encrypted username and password. The secret connection to the long connection server.
then, how can the long connection service with TCP protocol control the client access effectively?
Personal advice is: Do not use the user + password method, but the client id+ dynamic password (or session) way to access control, that is, every time the client login long connection service, by the server for this ID client dynamically generate a password or session, When the client logs in to take this session, the service side officers transferred Guevara the ID of the client and the validity of the session and whether the two match, in this way, each ID in each landing push will dynamically assign a different password or session.
In a long connection service, each connection corresponds to a unique client ID, and if a two-ID connection comes in, the incoming connection kicks the previous one, in short, the Long Connection service maintains only one connection for a client ID. If someone grabbed the analysis to get a connection ID and password, then its forged connection can only affect that one connection, then the normal client and forged client to play each other phenomenon, in the client in this way a little control can be resolved, For example, after the client has to go back to the server to request a new long connection service, the server will assign the long connection ID new long connection service and the corresponding connection password.
The use of client-side id+ dynamic password (or session) way to access control can make illegal connection to the existing push to reduce the scope of the impact to a connection, combined with the previous section of the application layer encryption measures to allow illegal access to the connection even if the incoming message can not be correctly resolved, and The mutual kick mechanism of long connection can also let the normal connection quickly detect another client to impersonate itself to connect the long connection service.
how to maintain the uniqueness of a connection in a long connected cluster.
In the previous description, a single long connection service would not allow two connections with the same ID to work at the same time. However, if in the long connected cluster environment, the same ID connection to the different long connection services, the single long connection service itself can not do different "long connection service" between the same ID mutually exclusive, At this point, the cluster management level is required for processing, for example, when a client obtains a long connection service address, check to see if a long connection service has been assigned to the ID, and if there is an assigned record, and the long connection is still online, then the assigned long connection will return directly to the last assigned service. At the same time, the call client is told in the return value that the ID is already online, and the client layer is then processed to handle this mutual kick scene to avoid the need for a long connection service to generate an inter kick while the upper application is not perceived.
processing of pseudo connection
This is the security of the Protocol, if the custom protocol must pay attention to this issue, if the adoption of sophisticated protocols such as MQTT, XMPP and other protocols will not occur when the basic problems. In a long connection service based on TCP, Application layer protocol generally requires TCP three times after the handshake, the application layer protocol initialization operations, such as establishing a connection, etc. in the case of the MQTT protocol, the first message that the server receives from the connection must be a MQTT connection message after the TCP connection is established. If someone forges a connection to TCP and does nothing, then the long connection service ensures that the connection is not involved in any business and that the connection is cleaned up in due time.Second, the rate of message delivery

There are many factors that affect the delivery rate of push system, and there are any problems between server, client and network, which will affect the timeliness of push message and even cause the message to be lost. The following describes several measures that can increase the message delivery rate. 2.1 Push Compatibility

At present, many mobile phone manufacturers have customized their own ROM. These diverse custom ROM for the purpose of power saving, provincial traffic, etc., will apply to the application in the state of the screen special processing, the development of their own long connection service will often be customized ROM to hang up or be directly killed. Therefore, no matter how perfect the push system design, once the long connection of the client by ROM such processing, can not work properly, resulting in the message can not be pushed down in time.
However, many mobile phone manufacturers have developed their own push system, for example, millet, their ROM will be their push clients to add to the whitelist, so in these custom ROM and have their own push, the delivery rate of their own push system is generally higher than other third-party push message delivery rate.
In this case, if the self-learning push system can be compatible with these handset manufacturer ROM, will be able to effectively improve the entire push system message delivery rate. So how to combine. What technologies or technologies may be involved in the process of integration.
It is recommended to use a third-party push as a long connection service, to be compatible with a third party push within a push system, and to block these internal details for use with push-top applications. When implemented, the client detects and decides which third party push to use, and will determine the result in the login push system passed past, push the system inside to record the client uses which Third-party push system, it in the third party push system identification is what information; When the message is pushed to the client, The push system will also first query which push the client uses, and then send the push message to the push client SDK, push the client's SDK to receive the message, and then deliver the message to the client's application, so as a whole, Any business that uses push does not perceive the processing logic within the push system, including the push backend and push client SDK.
problems and countermeasures in the process of pushing with the third party
(1) Do not believe that the third party push, many companies themselves to develop the push system is because do not believe that the third-party push security, unwilling to send their own data to others to push. In addition, the method of application layer encryption proposed in this paper third-party push status is like the use of TCP connection long connection services, in the entire system design is assumed that they are unsafe, in the message to the third party is already encrypted when pushed, so you can guarantee the security of data.
(2) Custom ROM, self-propelled mobile phone manufacturers too many, in the push system compatible with the Third-party push. Compatible with how many third party push is appropriate. You can choose the domestic sales TOP2~TOP3 two or three handset manufacturers, and then integrate their push into their own push system, the development of their own push SDK to integrate the handset manufacturer's SDK. The integration of third party push is not easy, otherwise the client SDK will be pushed very large.2.2 Service delivery rate for extended connection Services

In the push system, long connection service is mainly responsible for sending notifications to online users, due to mobile internet environment, network anomaly is the normal, frequent disconnection of clients-reconnection is a common situation, therefore, the promotion of long connection services in these scenes of the notification service rate will effectively ensure the timeliness of message delivery. Then, how to improve the long connection service delivery rate of notification.
In the long connection service that you develop, it is recommended that you defer the method of releasing the Connection object. Each client has a unique ID in the long Connection service, and when the long Connection service is developed, each client is assigned an object that holds the ID of the client, the notification to be sent to it, the socket that the server uses to communicate with the client, and so on. When the socket connection is disconnected, the connection object should not be released immediately, but rather be deferred for a period of time, and if the object has a new connection coming in during the deferred release, direct the object to use the connection directly, and the message sent to the client during the break is first given to the object for saving, so Messages during these break breaks can be sent to the client using a new socket connection, thereby raising the notification delivery rate for the long connection service at the code implementation level.