Architecture Design: Inter-system Communication (43)-Self-designed ESB (4)

==============================
"Architecture Design: Inter-system Communication (42)--DIY ESB (3)"

5. Borker Server Selection

In the previous three articles, we introduced the design of the top-level design of ESB middleware, introduced the main control service how to log collection and monitoring of multiple esb-brokers dynamic nodes, and also introduced the Esb-broker node how to carry on the dynamic route definition load management. In this article we mainly discuss some key designs for esb-client.

In our self-designed ESB middleware to ensure that ESB services do not become a bottleneck for the entire software system, we designed the ESB service in a way that multiple Esb-broker server nodes work together. multiple Esb-client requests will be assigned to these Esb-broker nodes in accordance with certain rules, And these esb-client nodes will dynamically switch the corresponding Esb-broker nodes based on the immediate state of the Esb-brokers cluster they serve . So the question we're going to talk about is: Why do esb-client need these basic features and how to implement them?

Let us first discuss the most probable form of esb-client. While we know that multiple esb-broker exist in an ESB service to allow multiple esb-client requests to be evenly distributed to these esb-broker as much as possible, does one esb-client represent a business system that participates in ESB integration? The answer is: Certainly not, each business system in order to ensure the processing performance of its construction structure is certainly a number of processing nodes . This means that a business system may contain multiple esb-client nodes. As shown in the following:

As shown in business system A and business system B, respectively, there are three running nodes, as to whether these three running nodes still have a synergistic action to complete some special business request processing, we do not discuss here, because this is the business system internal things do not affect the design of the ESB middleware. The nodes of these business systems start with the state of the Esb-broker nodes that are already in the ESB service running module and select a Esb-broker node to provide service response services for themselves. This is a common health condition on our formal system, which also explains why in the top-level design of a business system, ESB Middleware services are easily a performance bottleneck- not every business system has only one esb-client node request to Esb-brokers to run the cluster .

At the peak of the business, the OPS team observed that the module with three esb-brokers running nodes had reached peak performance through the main control module and then started a new Esb-broker node, "Broker Server4". If the ESB middleware does not have any methods to reallocate the service relationships of existing esb-client nodes and Esb-broker nodes, it is not possible to reduce the operating pressure of the ESB service even if many new Esb-broker nodes are started .

And in order to be able to quickly alleviate the performance pressure of the Esb-brokers module, we designed the allocation scheme should be full dynamic. Fully dynamic means that as long as the Esb-brokers module has changed, all esb-client will immediately be notified and immediately re-select a new Esb-broker server node for themselves. The "Changes" described here include not only the addition of a new Esb-broker server node to the Esb-brokers module, but also the removal of an existing Esb-broker server node, including a Esb-broker Although the server node has not been removed from the Esb-brokers module, it has itself failed to provide services temporarily.

5-1, the solution process

Judging from the hard design requirements described above, using zookeeper to coordinate the state of each esb-client node and Esb-broker node is a feasible scheme. To do this, we need to make adjustments to the zookeeper-end data storage structure already mentioned in the previous article. Please see below:

The path node named "Active_system" is the original data structure used to describe the business system information. The red Circle range is a new data structure that describes the currently active Esb-broker node. Note that these nodes are temporary nodes, meaning that the path node information about the Esb-broker node on zookeeper is automatically deleted when the Esb-broker node is completely stopped. And the original has been recorded in the Zookeeper service end of the business system information, is all moved to the "Active_system" under the path node, the reason is to avoid esb-broker will "Active_brokers" Mistakenly identified as a business system information.

After we have finished adjusting the data structure on the zookeeper, we can refine the coordination and invocation relationship of Esb-client, Esb-broker and zookeeper:

is architecture design: inter-system Communication (40)-Self-designed ESB (1) This side of the article "ESB Top-level design" describes the process refinement of the section. The coordinated process of esb-client, Esb-broker and zookeeper are described in detail:

2. When a Esb-broker server node finishes the boot process (that is, the process described in the "4-3" section above), a temporary node that represents itself and works properly is created under "Active_brokers" Path node on the zookeeper service side. Note that it must be a temporary node (ephemeral type), so that this temporary node automatically falls off when the current Esb-broker server node is fully operational, and notifies the event to all Esb-client (7th or 9th steps shown in). This processing step also represents the Esb-broker server node boot process that has been described earlier that needs to be adjusted accordingly.

4. When a esb-client node is started, the zookeeper client is connected to the ZK server to query all Esb-broker server node information that already exists under the current "Active_brokers" Path node. It is important to note that these Esb-broker server node information is not always available , because when the Esb-broker server node terminates abnormally, the corresponding temporary node on the Zookeeper service may not be immediately deleted. Another possibility is that although some Esb-broker server nodes work properly, the ports provided to esb-client nodes on the Esb-broker server node are blocked by firewalls.

6. The [Data] section of these Esb-broker server nodes will record the working state of this node. However, in this step, the zookeeper server will return all the currently existing Esb-broker node information to esb-client. The latter will select a node to serve itself based on the current Esb-broker server node information (this article is referred to as the "main selection process") in terms of "redundancy". This process is important because the selection process may be selected to an "unavailable" Esb-broker server node. This "main selection process" will be explained in detail below.

8. Next, Esb-client will be able to access the "available" Esb-broker server nodes selected in the previous steps as normal. And at the last step of the Esb-client node boot process, the Esb-client node also establishes a listener for the "Active_brokers" Path node to receive the change events for the immediate child nodes under the node.

10. When the Esb-client node determines the Esb-broker server node that can be accessed, and the former does not receive any event notifications about changes to the Esb-broker cluster, the esb-client node and Esb-broker The correspondence between server nodes will not change. However, when access to the Esb-broker server node is blocked by the Esb-client node, the esb-client node initiates the retry process.

12. If the retry process fails all, the Esb-client node will temporarily block all of its own requests for the ESB service (until the Esb-broker server information corresponding to the Esb-client node is updated). The Esb-client node then proactively modifies the Esb-broker server node's status to "unavailable" to the zookeeper service, so that all esb-client nodes receive this event.

14. As mentioned in the previous step, when the availability of a Esb-broker server node changes (the change may be switched to the "available" state, or it may be switched to the "unavailable" state), All esb-client nodes will receive this event (including the Esb-client node that triggered the event). They will then re-esb-broker the server node selection.

16. Steps 5 through 7 above are the process of proactively triggering a "re-esb-broker server node selection" After a failed Esb-broker server node is discovered by the Esb-client node. When the operational personnel of the formal system actively terminate the operation of a Esb-broker server node, the temporary node corresponding to the Esb-broker server node on the zookeeper service is immediately deleted and "re-esb-broker" is triggered. The server node selection process.

18. The Esb-broker server node change event triggered by the above conditions can be reused at the esb-client end of the process.

5-2. Modify the boot process of Esb-broker server

For the first procedure mentioned in the previous section, we need to modify the Bootstartup class on the Esb-broker server that was previously given. Primarily after it has completed its normal boot process, it will go into the "Camelcontextoperatequeue.take ()" Queue before it waits, through the curator component on Esb-broker Server on the zookeeper service side "Active_ Brokers "Path node creates a temporary nodes that represent this esb-broker. The code snippet looks like this, and note that because the main code snippet in the Bootstartup class has been given before, this gives only the part of the code that is added to the code:

/** * Our own design of Esb-broker server application main Launcher <br> * This initiator will coordinate the Zookeeper Operation module and Camelcontext context during the boot process. * @author Yinwenjie * * Public  class BootStartupV2 {....../** * Main start-up process here */    @SuppressWarnings("Unchecked")protected void Start()throwsException {/* * Boot order: * * * 4.1, under the Active_brokers node, increase the temporary node representing this broker server node (New Step) * * ... * */......// ========================        //The code here adds a temporary node representing the broker server on the zookeeper service        //Note Check the existence of "active_brokers" Path node        // ========================        //If the conditions are true, it means that the active_brokers must be created permanently        if(Zkclient.checkexists (). Forpath ("/active_brokers") ==NULL) {zkclient.create (). Forpath ("/active_brokers",NULL); }//Get a Esb-broker valid IP here to provide esb-client use        //Ipchoiceutils is a tool class that is used to select an available IP register to ZK. String sourceIp = Ipchoiceutils.getsourceip ();//If the condition is true, the description does not take any IP. Stop running!         if(SourceIp = =NULL) {System.exit (-1); }//Establish temporary nodeZkclient.create (). Withmode (createmode.ephemeral). Forpath ("/active_brokers/broker_"+ SourceIp, string.valueof (true). GetBytes ());    ......    } ......}

What is not clear in the code snippet above is "How to choose an IP to provide to Esb-client". In general, a physical server will have multiple IPs, such as a loopback IP value of 127.0.0.1, and an intranet IP value of 192.168.1.100. So which IP is given to esb-client so that the latter can make service requests? Here are two of the most straightforward ways to handle this: the first is that Esb-broker server provides a configuration file, and in the configuration file, a developer/OPS person assigns an IP. This way of implementation is relatively simple, only need to read the configuration file in the code, but the shortcomings of this approach is more obvious: the developer/OPS personnel need to maintain a configuration file for each esb-broker node, adding a certain amount of maintenance and management difficulties.

Another way to handle this is by Esb-broker server to automatically select an IP at startup. The advantage of this approach is that in general, developers/OPS are not required to specifically maintain a configuration file that calls IP, reducing the workload of staff deployment and maintenance. However, because the physical server may exist in a relatively complex LAN, this approach does not necessarily guarantee that the IP selected by the program is esb-client the best call IP, or even the wrong choice. Imagine a situation where a physical server has two network cards, each specifying two intranet IP addresses. But one of the intranet IPs took a three-tier switch and the other intranet IP only went through a layer of switches. Therefore, the best way in the formal system is to synthesize the above two ways: automatic recognition of the program as the Main, with manual configuration designated as a supplement . Here is the core code for the "Ipchoiceutils" tool class:

....../** * IP Select tool for selecting an IP * @author Yinwenjie * * * for esb-client access in multiple IPs on the machine * / Public  class ipchoiceutils {    /** * Log * *    Private Static FinalLog LOGGER = Logfactory.getlog (Interfacelister.class);/** * Intranet IP range * /    Private Static FinalString localip_ranges[] =NewString[] {"10.0.0.0-10.255.255.255","172.16.0.0-172.131.255.255","192.168.0.0-192.168.255.255"};/** * A, B, Class C IP range, not including broadcast IP, but includes intranet IP range */    Private Static FinalString remoteip_ranges[] =NewString[] {"1.0.0.1-126.255.255.254","128.0.0.1-191.255.255.254","192.0.0.1-223.255.255.254"};/** * This method is used to select a IP that provides access to the esb-client in the native multiple IPs * @return  */     Public StaticStringGetsourceip() {enumeration<networkinterface> interfaces =NULL;Try{interfaces = Networkinterface.getnetworkinterfaces (); }Catch(SocketException e) {Logger.error (E.getmessage (), E);return NULL; }//First get the IPV4 address available on the machine. List<string> allips =NewArraylist<string> (); while(Interfaces.hasmoreelements ()) {NetworkInterface networkinterface = interfaces.nextelement ();//Note that a network device may be bound to multiple IPs or it may not have an IP            //There may be a IPV6 format            //is a IPV4 format, or it may be a loop-type IPenumeration<inetaddress> inetaddresss = networkinterface.getinetaddresses (); while(Inetaddresss.hasmoreelements ()) {InetAddress inetaddress = inetaddresss.nextelement ();if(! (inetaddressinstanceofinet4address) | | Inetaddress.isloopbackaddress () | | Inetaddress.ismulticastaddress ()) {Continue;                } String IP = inetaddress.gethostaddress ();            Allips.add (IP); }        }//Start selecting an IP from the available IP first        //Selection rule: First select a low-level intranet IP. If the intranet IP is not valid, then first select Class A, Class B, Class C IP        //Developers can make changes to the selection rules, depending on their needs         for(String sourceip:allips) { for(String localip_range:localip_ranges) {if(Ipexistsinrange (SourceIP, Localip_range)) {Logger.info ("====== Select to IP:"+ SourceIP);returnSourceIP; }            }        } for(String sourceip:allips) { for(String remoteip_range:remoteip_ranges) {if(Ipexistsinrange (SourceIP, Remoteip_range)) {Logger.info ("====== Select to IP:"+ SourceIP);returnSourceIP; }            }        }return NULL; }/** * Determine if a given IP is within a range * @param IP * @param ipsection * @return If it is in a range, Returns true if other conditions return false * /    Private Static Boolean Ipexistsinrange(String ip,string ipsection) {string[] ipsections = Ipsection.split ("\\-"); String Beginip = ipsections[0]; String EndIP = ipsections[1];returnGetip2long (Beginip) <= getip2long (IP) && getip2long (IP) <= getip2long (ENDIP); }/** * This private method converts the IP in the form of a string to a corresponding long integer. Why See basic knowledge of IP infrastructure * For comparison * @param IP * @return  * *    Private Static Long Getip2long(String IP) {string[] ips = Ip.split ("\\.");LongIp2long =0L for(inti =0; I <4; ++i) {Ip2long = Ip2long <<8|        Integer.parseint (Ips[i]); }returnIp2long; }}

5-3, esb-client the start-up process

After discussing the changes in Esb-broker server, let's discuss the main work esb-client needs to do to select and maintain a stable service request for itself and Esb-broker nodes. Since Esb-client represents a business system that typically needs to be completed at startup with relatively tight business data initialization, various esb-client start-up processes are different. For example, the logistics system may need to complete the loading of the province-city-county area information when it starts up; the CRM system needs to load the basic information of the administrator into memory at startup, and the financial system must first establish a long connection with the payment system. So what we're going to discuss here is the Esb-client startup process, which refers to the start-up steps of the ESB middleware related to the business system that esb-client represents after completing the business-related startup process .

Esb-client mainly through the passive monitoring zookeeper service under the registered Esb-broker temporary sub-node change way, to know the Esb-brokers Cluster service status changes. These changes include the addition of Esb-broker nodes, reduced esb-broker nodes, and a change in the state of a Esb-broker node.

When Esb-client completes the startup process for the business section, it begins to select the available Esb-broker node for itself. By proactively querying the Zookeeper service for all current esb-broker node information, Esb-client will be able to generate a list locally. Then the selection is made based on this list (it can also be a weighted selection algorithm). However, the Esb-broker node that has been selected for the first time is not necessarily available, and the data area of the node may have been identified by another esb-client as "false" (that is, state invalidation). If this is the case, Esb-client will recheck the availability of the node through a test address that is provided by default for all Esb-broker. If the test succeeds, esb-client changes the availability of the Esb-broker node to "true", and if the test fails, the esb-client will take the base +1 of the remainder and re-esb-broker the selection.

===============================
(next)

Architecture Design: Inter-system Communication (43)-Self-designed ESB (4)