Alibaba has several distributed frameworks, including: remote invocation (similar to this remote invocation of RMI) (Dubbo, HSF), JMS messaging Service (Napoli, notify), KV database (Tair), and so on. this framework/tool/product, when implemented, takes into account the disaster tolerance, scaling, and load balancing, so there is a configuration center (configserver) to solve these problems. Fundamentals in our system, there are often some cross-system calls, such as in a system to invoke a service B system, we may use RMI directly, B system issued an RMI interface service, and then a system to call this interface through RMI, in order to solve the disaster tolerance, expansion, load balancing problems , we may think a lot of ways, Alibaba this method feels good. This article only says Dubbo, the principle is as follows:
Configuration Center,and a real-time heartbeat test between each server/client (because they're allsocket Long connection ), such as a few seconds of detection. collect information about the services provided by each server, each client's information, and sort out a list of services such as:
| ServiceName |
Serveraddresslist |
Clientaddresslist |
| UserService |
192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4 |
172.16.0.1,172.16.0.2 |
| Productservice |
192.168.0.3,192.168.0.4,192.168.0.5,192.168.0.6 |
172.16.0.2,172.16.0.3 |
| OrderService |
192.168.0.10,192.168.0.12,192.168.0.5,192.168.0.6 |
172.16.0.3,172.16.0.4 |
When aServer is not available, thenUpdate the affected service corresponding to the
serveraddresslist, which is to take this server from
serveraddresslistKicked out of the(Removed from address list), while thePush
serveraddresslist to all the clients within the clientaddresslist of these affected services . such as:192.168.0.3 Hung, then UserService and Productservice
serveraddresslist to remove the 192.168.0.3, At the same time to the new list to tell the corresponding client 172.16.0.1,172.16.0.2,172.16.0.3; when a client is hung up, the clientaddresslist that corresponds to the affected service
is updated Configserver based on the list of services, you can provide a Web management interface to view the providers and users of managed services. when a new server is added, because it will actively contact Configserver, and Configserver will send this information to the client actively, soWhen you add a new server, you only need to start the server, and in a few seconds, the client will use the services it provides
the machine that invokes the service, each client starts, initiates the socket long connection with configserver, and sends the corresponding information such as its own IP to Configserver. when the client uses the service, it obtains the service provider information according to the service name Configserver (so that Configserver knows that a service iscurrently which client is in use), the client to get these service provider information, and they are connected, can directly invoke the service, when there are multiple service providers, the client according to certain rules to load balance, such as polling, random, by weight, etc. Once the client uses a service that has a change in its corresponding service provider (the service provider has a new, deleted case), Configserver will push the latest list of service providers to the client, The client will reestablish the connection based on the latest service provider list, the new provider establishes the connection, and the deleted provider discards the connection
A machine that really provides services, each server starts , the initiative to establish a scoket long connection with Configserver, and send their own IP, service name, port and other information directly to Configserver,configserver will collect information about the services provided by each server. Advantages:1, as long as the client and server start, Configserver is good, the service can be called, if the followingConfigserver hung up, that only affect the Configserver after the service provider has changed, and the client is not aware of the change. 2,client Each invocation of the service is not configserver, the client simply contacts it and gets a list of service providers from it .3, call service- load balancer:When a client invokes a service, it can call services in turn by rules across multiple service providers. 4, service provider- disaster recovery: A server hangs, the client is still able to invoke the service correctly, when the premise is that the service has at least 2 service providers, the client can quickly perceive the service provider changes, and respond accordingly. 5, service provider-extension: It is easy to add a service provider, and the client will quickly perceive its presence and use it.
By the way, the central node in Hadoop is similar to the configserver here, while maintaining the node list, however, its calculations need to be made through the center to assign tasks.
Because the Dubbo bottom uses the socket to communicate, their own theory of communication is not very clear, so by the way the knowledge of communication also learn.
N Communication Theory
The exchange of information between computers and the outside world is called communication. The basic method of communication has two kinds: parallel communication and serial communication.
1. A group of information (usually bytes) of data that is transmitted simultaneously is called parallel communication. Parallel communication relies on the parallel I/O interface implementation. Parallel communication is fast, but the number of transmission lines is more than the root, only for close-up (several meters apart) of communication.
2. The data of a group of information is transmitted in a bitwise sequential manner called serial communication. Serial communication can be achieved through a serial interface. Serial communication speed is slow, but the transmission line is small, suitable for long-distance communication.
The serial communication is divided into the following 3 kinds of information transmission direction:
1) simplex
Data can only be transferred in one Direction
2) Half Duplex
Information can be transmitted in two directions, but not in both directions
3) Full Duplex
Bidirectional transmission and can be transmitted in both directions simultaneously
N Socket
Socket is an application interface, TCP/IP is a network transport protocol, although the interface is the same, but different protocols will have different service nature. When you create a socket connection, you can specify the transport layer protocol used, which can support different transport layer protocols (TCP or UDP), which is a TCP connection when a connection is made using the TCP protocol. Soket and TCP/IP are not necessarily linked. The socket programming interface is designed to adapt to other network protocols as well. Therefore, the socket is only more convenient to use the TCP/IP protocol stack.
Citation: http://hi.baidu.com/lewutian/blog/item/b28e27fd446d641d09244d08.html
The last communication theory is actually to say socket (TCP) communication is the way of full duplex
N Dubbo Analysis of the principle of remote synchronous invocation
Learn from the Dubbo Open source document that a calling procedure such as
Http://code.alibabatech.com/wiki/display/dubbo/User+Guide#UserGuide-APIReference
Also documented in the documentation: theDubbo default protocol uses a single long connection and NIO asynchronous communication , suitable for small data volume concurrent service invocation, and the number of service consumer machines is much larger than the number of service provider machines.
Dubbo the default protocol, using mina1.1.7+hessian3.2.1-based tbremoting interactions.
- Number of connections: Single connection
- Connection mode: Long connection
- Transport protocol: TCP
- Transmission mode: NIO asynchronous transmission
- Serialization: Hessian binary serialization
- Scope of application: Incoming outgoing parameter packet is small (recommended less than 100K), consumers more than the number of providers, a single consumer can not be full of providers, try not to transfer large files or super-large strings with the Dubbo protocol .
- Applicable scenario: General remote Service method invocation
Typically, a typical synchronous remote invocation should look like this:
1, the client thread calls the remote interface, sends the request to the server, and the current thread should be in a "paused" state, that is, the thread cannot be executed backwards, and must take the server to its own results before it can be executed backwards.2, the service ends up with a client request, processes the request, and gives the result to the client3, the client receives the result, and the current thread continues to execute Dubbo uses the socket (using the Apache Mina framework to make the underlying call) to establish long connections, send and receive data, and send messages using the iosession of the Apache Mina framework. View Dubbo documentation and source code it is known that the Dubbo uses the socket to send a message in the form of data transfer, in combination with the Mina framework, using the Iosession.write () method, which is called after the entire remote call (from the request to the reception of the results) is an asynchronous, that is, for the current thread, the request is sent out, the thread can be executed back, as for the server end, the service end processing is completed, and then sent to the client in the form of a message. So there are 2 questions:
- How does the current thread let it "pause" and wait until the result comes back and then executes backwards?
- As mentioned earlier, the socket communication is a full-duplex way, if there are multiple threads making remote method calls at the same time, there will be a lot of messages sent on the socket connection between client server, and the order may be messy, After the server finishes processing the result, sends the result message to Client,client to receive a lot of messages, how to know which message result is which thread was originally called?
Analyze the source code, the basic principle is as follows:
- The client one thread calls the remote interface, generates a unique ID (such as a random string, uuid, etc.), and Dubbo is a cumulative number using Atomiclong starting from 0.
- The packaged method invocation information (such as the calling interface name, method name, parameter value list, and so on), and the callback object that handles the result callback, are all encapsulated together to form an object
- Put (ID, object) into the global concurrenthashmap that specifically holds the call information
- Encapsulates ID and packaged method invocation information into an object connrequest, sent asynchronously using Iosession.write (connrequest)
- The current thread then uses the Get () method of callback to try to get the result of the remote return, inside get (), using synchronized to get the lock of the callback object callback, and then first to detect if the result has been obtained, if not, Then call callback's Wait () method to release the lock on the callback and leave the current thread in a wait state.
- After the server receives the request and processes, sends the result (the result contains the preceding ID, that is, the callback) to the client, the client socket connection on the special listener message received the message, parse the result, take the ID, and then from the previous concurrenthashmap inside get (ID), The callback is then found, and the result of the method invocation is set to the callback object.
- The listener thread then uses synchronized to get the lock of the callback object callback (because the previous call to wait (), that thread has freed the callback lock), and then Notifyall (), Wake up the thread that is waiting before continuing (the callback get () method continues execution to get the result of the call), at this point, the entire process is finished.
You
need to draw a big picture to describe it, and then fill it up. It is important to note that the callback object here is each call to produce a new one that cannot be shared, otherwise there will be a problem, and the ID must at least be guaranteed to be unique within a socket connection. now, the first two questions already have an answer,
- How does the current thread let it "pause" and wait until the result comes back and then executes backwards?
a
: sir, as an object, obj, put (id,obj) in a global map, get the obj lock with synchronized, and then call Obj.wait () to let the current thread wait. Then another message listener thread waits until the server end results come up, then Map.get (ID) finds obj, then uses synchronized to obtain the obj lock, and then calls Obj.notifyall () to wake up the thread in front of the waiting state.
- As mentioned earlier, the socket communication is a full-duplex way, if there are multiple threads making remote method calls at the same time, there will be a lot of messages sent on the socket connection between client server, and the order may be messy, After the server finishes processing the result, sends the result message to Client,client to receive a lot of messages, how to know which message result is which thread was originally called?
a
: use an ID to make it unique, then pass it to the server, and then return it back to the server, so that the result is the original thread.
This is not the first time to see, 10 in the previous company, but also the remote interface call, but the message of the middleware RABBITMQ, synchronous call principle similar to this, see:RABBITMQ Learning-9- Rpcclient send message and synchronous receive message principle
Alibaba Remote Call Frame Dubbo principle
