(vii): C + + Distributed real-time application Framework 2.0

C + + Distributed real-time application Framework 2.0

Technical cooperation QQ Group: 436466587 Welcome to discuss Exchange

Previous: (vi): Large-scale project container transformation

Copyright NOTICE: This article copyright and the technology used belong to Smartguys team all, for plagiarism, not by the consent to reprint and other acts to retain the right to legal investigation!

　　After the release of the C + + distributed real-time application framework (CDRAF) 1.0 release, we made a number of improvements to the framework. Support micro-service architecture, microservices orchestration at the architectural level. The coupling of CDRAF and business code is further clarified, and all distributed functions no longer need to be cared for by the business side, and the unification is realized by Cdrfa. The ultimate simplification of the business side of the configuration file, perhaps the next step is to do a code generation function, according to the configuration to generate the corresponding business code framework. Re-implementation such as node time-delay statistics (CDRFA automatic implementation of each packet delay statistics, business need not care), single-number log tracking (changed to message packet dyeing scheme, CDRFA encountered the dyeing packet automatic printing log), Application command function ( Through the System Management module's restful interface to the application within the node to release instructions) and so on. Provides an interface to fully display the cluster node real-time topology diagram, and support the cluster node micro-service real-time orchestration. Added new logging framework capabilities to provide high performance, multi-scene log output capabilities. The corresponding unit tests should be added to these improvements synchronously. The C + + distributed real-time application Framework 2.0 version also came into being!

first, the cluster real-time topology map

The real-time topology diagram shows each node of the cluster (container instance), the line represents the direction of communication, and the isolated nodes represent the nodes that are not connected. Interface real-time display TPS (traffic), RTT (delay) Two data, point open node can enter the container node management interface, do more management of container nodes.

　　

　　

Second, micro-service orchestration

In the fifth chapter, "Evolution of the MicroServices architecture", we have added a graphical interface to support the orchestration of the MicroServices architecture. Each large circle represents a type of node, and a small circle represents a microservices port. Orchestration from one node to another node, and select two nodes to connect to the micro-service, you can complete the orchestration. Can be organized in real-time during the operation of the cluster, the completion of the orchestration, the cluster status will be displayed in real-time in the topology diagram above. Of course, depending on your business needs, you can add new types of nodes or add new microservices ports to your nodes. You can see that each node is independent and non-coupled, and that the interaction between nodes is done entirely through the microservices ports, and that these ports are in effect or not controlled through microservices orchestration.

Three, time-delay statistics function

The time-delay statistic function is one of the core data of distributed framework, and it is used to detect the performance of nodes and adopt corresponding strategies. Originally this function is implemented on the business side, in CDRAF2.0, we refer to this function in the framework, can calculate the delay data of all nodes, and the business does not need to be concerned about (the business does not need to do anything to get this function).

In CDRAF2.0, we unify the communication model of the node, and all the nodes are composed of dis (data distribution) and business processing process. After a business package arrives at DIS, the framework will hit the packet header at the time of the moment, after the business process has finished processing the message back to dis, the dis calculates two time difference to get the delay data. The data of each package is averaged, escalated to the status center, and can be displayed in a real-time topology diagram. All this work is done by the framework.

　　

Four, single-number log tracking (packet dyeing scheme)

　　The original log tracking scheme after receiving the number tracking command, all the nodes will be notified all the process to pay attention to this number, encountered this number of packets after the log trace opened. There are several major drawbacks to this scheme:

A) All business processes need to implement a number detection, log tracking function, the code will be extremely redundant.

b) When this feature is turned on, all nodes in the cluster, all processes in the node, will be processed to monitor if there is a status of this number. The entire cluster is in a very unstable state.

c) It may not be possible to track multiple numbers simultaneously on the business.

For this we adjusted the single-number log tracking scheme, using packet dyeing scheme. The location of the message at the entrance of the detection number, once the conditions will be the message packet Baotou dyeing, after the processing link framework received packet will precede the business detection header, if found Baotou is dyed, will log tracking open, the package is processed and then closed. As each link goes down, the framework automatically dyes the next packet again to ensure that every step of the process receives a dyed package. The advantages of the new scheme are as follows:

A) It is no longer necessary for the business to implement the corresponding code for this function, as long as the number is monitored at the entry location, and if the call frame is dyed in conformance, all subsequent operations are done by the framework.

b) When this feature is turned on, the cluster no longer handles the unstable state, and the business level does not even feel the existence of the feature, and the framework handles all the problems.

c) When new business nodes are added, this function is automatically acquired without any modification.

D) In addition to the single-number log tracking feature, this solution can also be applied to other business scenarios.

　　

V. General Instruction Delivery Scheme

　　In Cdraf, if the outside world is going to give an instruction to a process of a node in the cluster, it passes through the restful interface of the System Management module, and then through the State center, the communication platform is eventually passed to the corresponding process. But in our previous scenario, each additional command needed to be supported by adding code to each module (System Management, status Center, communication platform).

In the new scenario, we designed a generic message path for passing instructions.

The communication framework automatically adds monitormsginterface endpoints to the smartagent process without configuration, and the communication framework automatically adds monitormsginterface endpoints to the smartmonitor process, without configuration; smartagent The business control message is emitted from its own monitormsginterface endpoint and Smartmonitor received and processed from its own monitormsginterface endpoint;

Unified command Format

message monitor_msg{    string1;     2 ;}

　　Where the dest is to control the destination that the message wants to send, fill in the process name of the specific destination process, such as "Olcproxy", "Ocpro", "Ocdis", etc., if you need to send the command to all processes, then dest fill in the string all. The MSG field is the specific control command text.

　　The framework itself provides a number of common control commands, enumerated as follows:
Adjust log levels
Format: Log Logging Level
Where the log level contains off, critical, err, warn, info, debug, trace
Example: Log Debug
Stop thread
Format: Stop
Overloaded Communication link Information
Format: Reload
In addition to the public control commands provided by the above framework, Smartmonitor can receive any message and pass it to the specified process, which does not care about the content of the message, and the content of the message is handled entirely by the business process. In principle smartagent should be read directly from ZK when filling in the value of the field, and the text of the control command should be available on ZK.

as you can see from the above adjustments, CDRAF2.0 is committed to decoupling distributed-related functions and services completely. In our design and implementation, there is a clear dividing line between the business and the framework. All functions can be done on the frame side, the business side line of code is not written, it can be automatically obtained. I think this is a big step for CDRAF2.0 to the general-purpose framework!

(vii): C + + Distributed real-time application Framework 2.0