I. Preamble
In recent months has been engaged in a distributed asynchronous communication system, today to organize and blog a bit.
This is a nationwide communications platform, the performance, massive data, fault tolerance and scalability have very high requirements, so in the system architecture can not be simple to adopt centralized. A simple summary is:
1. Data Distributed Storage
2. Request distributed scheduling
3. Multi-node distributed deployment
4. Double backup, eager swap
The core of the system is nothing more than network architecture, distributed operators and communication, the requirements are as follows:
Distributed operators:
1. For any input, the output is evenly distributed
2. The number of output results can be controlled
Communication:
1. High concurrency
2. Multithreading
Distributed operators we choose the hash function of sun company, the communication is Cindy socket communication. The network architecture and the specific description will be given in the following blog.
Second, the network architecture
The architecture of the entire system consists of four layers, each of which can be diverted by several nodes to data and requests:
1. Interface server (Interface server):
1). Provide access to the external interface and accept the request, in view of the extensive HTTP, generally built an HTTP server process
2). Monitor the working status of each dispatcher server
3). Forward the request to one of the best dispatcher, where the optimal judgment is based on the working state of each dispatcher server, of course, the specific content of the request on this layer can simply adopt the polling or random algorithm.
2. Message distributor (Dispatcher server):
1). Accept requests from the interface server
2). Parse the request, extract the feature parameters (typically something like a user account, the data under an account is distributed to the same node), execute a hash function on the parameter, calculate the app server where the target data resides, and then forward the request to the app server.
3). In fact, the processing in the actual project is more complicated than the above introduction, but the scalability is greatly enhanced.
3. Application servers (APP server):
1). The execution of business logic, equivalent to the application server in a centralized system, has no distributed features. The data being processed is the data in your own database, regardless of other nodes on the network.
2). is divided into multiple logical groups (group), Server load Balancing in the same group
3). The Multi-database single-read multi-write strategy is used in the case of dual backup, hot swap and load balancing. For reading, monitoring the working state of each database, selecting an optimal database to provide the data, and writing all the databases at the same time, it is necessary to ensure the transactional operation.
4. Database server (DB server):
1). Provide data access, nothing to say, for non-transactional databases need to provide support measures at the app server level;
5. Communication between nodes
1). Data (Request, response, exception) asynchronous concurrent transmission in network format
Third, distributed algorithm
The interface server (Interface server) and the message distributor (Dispatcher server) differ in the policy of distributing the request.
Auxiliary functions and variables:
Public string[] Gettargetserverips ()//IP of the target server, such as 193.243.15.45:8080
Public int[] Gettargetserverids ()//target server ID, corresponding to the above server IP one by one, can be freely configured
public boolean isserverworking (index);//Determine the status of the target server
int currenttargetserverindex=0;//Index of the current target server in Targetserverids
The interface server (Interface server) employs a polling algorithm:
Public String Gettargetserverip () {//Gets the target server to be distributed for the request
String[] Targetserverips=gettargetserverips ();
Int[] Targetserverids=gettargetserverids ();
int index=currenttargetserverindex;
Boolean isworking=false;
while (!isworking) {
Index=targetserverids.length ()% (currenttargetserverindex+1);
Isworking=isserverworking (index);
if (!isworking&&index==currenttargetserverindex) {//no target server available
return "0:0";
}
}
Currenttargetserverindex=index;
return Targetserverips[index];
}
hash algorithm used by message distributor (Dispatcher server) Distribution request
Hash algrithm from JDK ' s String, hash algorithm from JDK
public int hash (byte[] bs) {
int hash = 0;
for (int i = 0; i < bs.length; i++) {
hash = * hash + bs[i];
}
return hash;
}
public int Gettargetservergroupindexbyhash (String hashparam) throws Btirexception {//Returns a group of target servers based on hash
Byte[] Hashinfo=hashparam.getbytes ("Utf-8");
int framecount=2://The number of fragments by the latter two bits of the hash value, i.e. the number of hash results, the number of target server groups
int step = 100/framecount;
int hash = Math.Abs (hash (hashparam)% 100);
for (int i=0, beg=0, end=step; i<framecount; i++) {
if (Beg <= hash && hash < end)
return 2*i;
Beg = end;
End + = step;
}
return (FRAMECOUNT-1); If you set it up well, you're not going to get here.
}
Public String gettargetserveripingriuo (int groupindex) {//Based on the polling algorithm, compute the optimal server in the server group
String[] Targetserverips=gettargetserverips ();//The polling algorithm within the group, the code is slightly
Int[] Targetserverids=gettargetserverids ();
int Index=gettargetserverindexingroup (GROUPINDEX);//polling algorithm code slightly
return Targetserverips[index];
}
Four, communication node design model
Communication is the way the request responds, which is always the case for the interface server, the message Distributor, and the application server, so the three can be described by a consistent model.
Includes two parts: client and server. Here is a description of the structure and network communication.
The client constructs and sends the request, and in the asynchronous system it is possible to construct and send the uncoupling,
The requestbuilder generates a request to put the request into the request queue (Requestqueue) in a separate thread Requestscanner scans the request queue and calls Requestsender to send requests. different queues and different sender can be constructed for different types of requests, and the priority policy of request in the queue can be customized as needed. The server receives the request and processes it,
The server receives the request and processes it,
Requestaccepter receives the request and puts it into the requests queue independent thread scans the queue and requesthandler the call to processing RequestHandler returns between Response.client and server after processing is complete Communication: There are many communication protocols and technologies, such as Web SERVICE,EJB,JMS, where single Java NIO-based sockets are used, because of their asynchrony and high concurrency. The two basic criteria for using sockets are: 1. The number of threads on the server can be controlled, and the number of requests should not be linearly increased by 2. Separating processing requests from receiving requests, otherwise reducing throughput and volume
Java Distributed Communication System (EE distributed server architecture)--(reprint)