Java builds distributed Network Service __java

Original address: http://blog.csdn.net/jesseshen/article/details/6659543

This is a national communication platform, the performance, massive data, fault tolerance and scalability have very high requirements, so the system architecture can not be simple to use centralized. A simple summary is:

1. Distributed data storage

2. Request distributed scheduling

3. Multi-node distributed deployment

4. Double backup, eager to change

The core of the system is nothing but network architecture, distributed operators and communication requirements as follows:

Distributed operators:

1. For arbitrary input, the output evenly distributed

2. The output result number is controllable

Communication:

1. High concurrent Volume

2. Multithreading

Distributed operators we choose Sun's hash function, which is Cindy socket communication.

The architecture of the entire system, as shown in the figure, consists of four layers, each of which can be diverted from data and requests by a number of nodes:

One. Interface server (Interface server):

1. Provide access to the external interface and accept the request, taking into account the breadth of HTTP, generally built-in an HTTP server process

2. Monitor the working status of each dispatcher server

3. Forwarding requests to one of the optimal dispatcher, where the optimality judgment is based on the working status of each dispatcher server, of course, the specific content of the request in this layer can be simply using polling or random algorithms.

two. Message distributor (Dispatcher server):

1. Accept requests from the interface server

2. Parsing requests, extracting feature parameters (typically similar to the user account, where the data is distributed to the same node), then executing the hash function on the parameter, calculating the app server where the target data resides, and forwarding the request to the app server.

3. In fact, the processing in the actual project is more complex than the above introduction, but the scalability is greatly enhanced.

three. Application servers (APP server):

1. The execution of business logic, equivalent to the application server in a centralized system, is no longer distributed. The data being processed is the data in your own database, regardless of the other nodes on the network.

2. Divided into multiple logical groups (group), Server load Balancing in the same group

3. Taking into account the dual backup of the database, eager to change and load balance, the use of multiple database single read multiple write strategy. For reading, monitoring the working status of each database, select an optimal database to provide data, for write, write all the database, so must guarantee the operation of the transaction.

four. Database servers (DB server):

1. Provide data access, there is nothing to say, for non-transactional databases need to provide ancillary measures in the app server layer;

Five. Communication between nodes

1. Data (Request, response, exception) in network format asynchronous concurrent transmission. *********************************************************************************************************** **********

Distributed Algorithm

The interface server (Interface server) and the message distributor (Dispatcher server) differ in the policy for distributing requests.

Auxiliary functions and variables:

Public string[] Gettargetserverips ()//IP of the target server, such as 193.243.15.45:8080

Public int[] Gettargetserverids ()//target server ID, corresponding to the above server IP one by one, can be configured freely

public boolean isserverworking (index);//Determine the status of the target server

int currenttargetserverindex=0;//Index of the current target server in Targetserverids

The interface server (Interface server) employs a polling algorithm:

Public String Gettargetserverip () {//Get the target server to be distributed for this request

String[] Targetserverips=gettargetserverips ();

Int[] Targetserverids=gettargetserverids ();

int index=currenttargetserverindex;

Boolean isworking=false;

while (!isworking) {

Index=targetserverids.length ()% (currenttargetserverindex+1);

Isworking=isserverworking (index);

if (!isworking&&index==currenttargetserverindex) {//no target server available

return "0:0";
}

}

Currenttargetserverindex=index;

return Targetserverips[index];
}

hash algorithm used by the message distributor (Dispatcher server) Distribution request

Hash algrithm from JDK ' s String, hash algorithm from JDK
public int hash (byte[] bs) {
int hash = 0;
for (int i = 0; i < bs.length; i++) {
hash = * hash + bs[i];
}
return hash;
}

public int Gettargetservergroupindexbyhash (String hashparam) throws Btirexception {//Return target server group computed from hash

Byte[] Hashinfo=hashparam.getbytes ("Utf-8");

int framecount=2://The number of bytes that are segmented by the second digit of the hash value, that is, the number of hash results and the number of target server groups

int step = 100/framecount;

int hash = Math.Abs (hash (hashparam)% 100);
for (int i=0, beg=0, end=step; i<framecount; i++) {
if (Beg <= hash && hash < end)
return 2*i;
Beg = end;
End + = step;
}

Return 2* (frameCount-1); If set well, it should not come here
}

Public String gettargetserveripingriuo (int groupindex) {//Based on polling algorithm, compute the optimal server in a server group

String[] Targetserverips=gettargetserverips ()//group polling algorithm, code slightly

Int[] Targetserverids=gettargetserverids ();

int Index=gettargetserverindexingroup (GROUPINDEX);//polling algorithm code slightly

return Targetserverips[index];

} ********************************************************************************************************** ***********

Communication node Design model

Communication is the way to request a response, which is constant for interface servers, messaging servers, and application servers, so the three can be described by a consistent model.

Includes two parts: client and server. Here is a description of both the structure and network communication. The client constructs and sends the request, in the asynchronous system can construct and send the uncoupling, as shown

Requestbuilder Generate request

Put the request into the request queue (Requestqueue)

The standalone thread Requestscanner scans the request queue and calls Requestsender to send the request.

For different types of requests, different queues and different sender can be constructed, and the priority policy for the request in the queue can be tailored to suit the needs.

The server receives the request and processes it, as shown in

Requestaccepter receive requests and put them in the request queue

A standalone thread scans the queue and processes the call RequestHandler

RequestHandler after processing, return to response.

Communication between client and server:

There are many communication protocols and technologies, such as Web SERVICE,EJB,JMS, where Java NIO based sockets are used alone because of their asynchrony and high concurrency.

The two basic standards for using sockets are:

1. The number of threads on the server can be controlled, avoid and request the number of linear growth

       2. Separate processing and receiving requests, otherwise the throughput and the amount of the method will be reduced