Simple communication process

Simple communication process https://www.cnblogs.com/SteveLee/p/9860507.html

In the previous article, we have fully introduced the grpc-based service discovery and service governance solution. let's first review the RPC call process (I will repeatedly emphasize this process call solution in several articles in this section ).

Based on the figure above, the service principle can be divided into three steps:
The server starts and sends service information to the Registration Center. After receiving the service information, the Registration Center regularly monitors the service status (Common heartbeat detection );
When the client needs to start calling the service, first go to the Registration Center to obtain service information;
The client creates a remote call connection. After the connection, the server returns the processing information;

Steps 1 can be further subdivided. The following describes the principle of Remote Process calling:
Objective: How can a client call a public method on a remote machine?
Service Discovery: obtain services from the Registration Center (many services are required here: load balancing is required when multiple services are obtained, same data center filtering, version filtering, service route filtering, and unified gateway );
The client initiates a call and assembles the services, methods, and parameters to be called;
Serialized and encoded messages. JSON, XML, protobuf, and Hessian can be used here, the serialization speed and the size of bytes occupied after serialization in several schemes are all important indicators for selection. I suggest using efficient protobuf for internal serialization based on TCP/IP binary, which is small in size and fast.
Transmission protocol, which can use traditional Io to block transmission, or use efficient NiO transmission (netty );
After receiving the message, the server deserializes it and then processes it accordingly;
The server serializes the response information and returns it;
The client receives the response information and deserializes it;

As mentioned in Article 4th in step 3 above, when class C is called to class S, RPC or restful can be selected for internal communication, I strongly recommend that you use RPC to call all services in the S class. The comparison between RPC and restful is as follows:
Advantages:
Binary messages are used for serialization, with good performance and high efficiency (Good space and time efficiency );
Serialization and deserialization directly correspond to the data classes in the program and do not need to be parsed before ing (XML and JSON are both in this way );
Compared with the HTTP protocol, there is no useless header, which simplifies data transmission and is based on the TCP layer for faster transmission and smaller capacity;
Netty and other frameworks (the focus is also the main framework introduced in this article );
Disadvantages:
Complicated use, high maintenance cost and learning cost, and difficult debugging;
Based on http2, most HTTP servers and nginx are not yet supported. That is, nginx cannot use grpc requests as HTTP requests for load balancing, but as common TCP requests. (Nginx1.9 is supported );
Poor binary readability or almost no direct readability, special tools are required for deserialization;
Dynamic features are not available by default (you can dynamically define the generation of message types or support for dynamic compilation. The dynamic compilation feature using rosyln will be introduced later );

Introduction to the communication and transmission tool netty (net is dotnetty)
(Blame Microsoft first) We are doing net development and envy the excellent frameworks such as netty, spring, struts, and Dubbo on Java, while net is just dumpfounded, the ecosystem was not ready before the rogue. However, due to the recent release of net core, it has gradually pulled back a small part of the world that belongs to Microsoft.
Dotnetty is modeled by the azure Team (almost as you can say) Java's netty (a part of netty has been implemented currently). Currently, star on GitHub has 1.8 K +. Address: Ghost. Although it is much later than netty, we should be glad that our net programmers can finally use a powerful communication framework like netty on their own platforms.
Traditional communication problems:
We use common applications or class libraries for mutual communication. For example, we often use an HTTP client library to obtain information from the web server, or use the Web service to execute a remote call.
However, sometimes a General Protocol or its implementation does not meet the requirements well. For example, we cannot use a common HTTP server to handle large files, emails, and near real-time messages, such as financial information and multiplayer game data. We need a highly optimized protocol to deal with some special scenarios. For example, you may want to implement an optimized Ajax chat application, media stream transmission, or a large file transmitter, you can even design and implement a new protocol to meet your needs accurately.
Another unavoidable situation is when you have to deal with legacy proprietary protocols to ensure interoperability with old systems. In this case, the important thing is how we can quickly implement the Protocol without sacrificing the stability and performance of the application.
Solution:
Netty is a network application framework that provides asynchronous event-driven (asynchronous event-driven). It is a server and client for fast development of high-performance and scalable protocols.
In other words, netty is a NiO Client Server framework that can be used to quickly and easily develop network applications, such as server and client protocols. Netty greatly simplifies the development process of network programs, such as the development of TCP and UDP socket services.
"Fast and simple" does not mean that applications are difficult to maintain and have low performance. netty is a well-designed framework, it draws a lot of experience from the implementation of many protocols, such as FTP, SMTP, HTTP, many binary and text-based traditional protocols. therefore, netty has successfully found a way to achieve simplicity, high performance, and stability of development without losing flexibility.
Some users may have discovered that some other network frameworks also claim that they have the same advantages, so you may ask what is the difference between netty and them. The answer is netty's philosophical design philosophy. From the very beginning, netty provides users with the best API and implementation design for user experience. Thanks to netty's philosophy, you can easily read this guide and use netty.

(I am not sure about the framework and implementation of dotnetty, But I can refer to the official documentation of netty to learn and use the dotnetty-related API interfaces)

Several important Libraries (assembly) in dotnetty ):
Dotnetty. buffers: encapsulation of memory buffer management.
Dotnetty. codecs: encapsulation of codec, including the implementation of some basic base classes. The protocols we customize in the project must inherit the specific base classes and implementations of the project.
Dotnetty. codecs. mqtt: mqtt (Message Queue telemetry transmission) codec is encapsulated, including the implementation of some basic classes.
Dotnetty. codecs. protobuf: protobuf codec is encapsulation, including the implementation of some basic classes.
Dotnetty. codecs. protocolbuffers: protocolbuffers codec is encapsulation, including the implementation of some basic base classes.
Dotnetty. codecs. redis: The redis protocol codec is encapsulated, including the implementation of some basic classes.
Dotnetty. Common: encapsulation of common class library projects, packaging thread pools, parallel tasks, and common help classes.
Dotnetty. handlers: encapsulates common MPs queues, such as TLS codec, timeout, heartbeat check, and logs.
Dotnetty. Transport: Core Implementation of dotnetty, basic socket framework, communication mode: asynchronous non-blocking.
Dotnetty. Transport. libuv: dotnetty implements the core implementation based on libuv (high-performance, event-driven I/O library.
Common libraries include codecs, common, handlers, buffers, and Transport. Currently, the azure team is implementing APIs in other netty libraries (including non-public netty APIs). Let's wait and see.

Direct point-to-point communication
In the dotnetty example folder, there are many official instances, such as discard, ECHO, and telnet instances. To achieve direct point-to-point communication, the author uses the echo demo, and subsequent RPC calls will also be implemented based on Echo. Note the details and directly upload the code of the receiving end (server:
View code

Ieventloopgroup is a multithreaded event loop device used to process I/O operations. dotnetty provides many different eventloopgroup implementations to process different transmissions. In this example, we implement a server application, so two ieventloopgroups will be used. The first connection is often called 'bos', which is used to receive incoming connections. The second is often called 'worker', which is used to process received connections. Once 'boss' receives the connection, it registers the connection information to 'worker.
Serverbootstrap is an auxiliary startup class for starting the transport service. You can directly use the channel in this service, but this is a complicated processing process. In many cases, you do not need to do this.
Here we use the tcpserverchannel class to illustrate how a new channel receives incoming connections.
Channelinitializer is a special processing class designed to help users configure a new channel. When your program becomes more complex, you may add more processing classes to pipline and extract these anonymous classes to the top-level classes.
You can set the configuration parameters for the channel implementation specified here. We are writing a TCP/IP server, so we are allowed to set socket Parameter options such as tcpnodelay and keepalive.
Bind the port and start the service. Here we bind the configured port on the machine Nic. Of course, you can call the BIND () method multiple times (based on different binding addresses ).
After use, release the specified working group thread elegantly. Of course, you can choose to close the program, but this is not recommended.

Server-side event processing code:
Implementation of adding a rough area in the previous Code

View code
Discardserverhandler inherits from channelinboundhandleradapter. This class implements the ichannelhandler interface. ichannelhandler provides many interface methods for event processing, and then you can override these methods. Now you only need to inherit the channelinboundhandleradapter class instead of implementing the interface method yourself.
Here we overwrite the chanelread () event processing method. Every time a new data is received from the client, this method is called when a message is received. In this example, the type of message received is bytebuf.
To respond to or display the information sent from the client, we will print the data sent from the client in the console.
Then, we write the message sent from the client back to the client through context. writeasync.
Of course, step 4 only caches the stream to the context and does not perform the real write operation. It writes the stream data to the pipeline by executing flush and sends it back to the client through context.

Client code:
Focus on the comments. There is no difference between other comments and the server.

View code
Initialize the size of a buffer.
The data type accepted by the default buffer is bytes []. Of course, this makes it easier to serialize the data into a stream.
Write the stream of the buffer directly into the channel pipeline. This pipeline is generally the other end of the Link communication (client C ).
An endless loop is created to test the data that must be input from the client each time. The next input operation is performed after the server loop.

Client-side event processing code:
View code
It is very simple to display data streams to the console.

Implementation result
So far, we have built a simple response server using the dotnetty framework, which is very simple and has the following implementation results:
After the C end actively sends data to the S end, the S end receives the data, prints the data on the console, and returns the data to the C end. Of course, the S end can do a lot of things.

Dotnetty internal debugging Record Analysis
Although dotnetty does not provide any technical documentation, it provides detailed debugging records. In many cases, our learners can analyze the implementation process of a function through debugging records. We can print the internal input and output records of dotnetty to the console.

Internalloggerfactory. defaultfactory. addprovider (New consoleloggerprovider (S, level) => true, false ));
We can see that there are a lot of printing records on the server, most of which are the printing records during dotnetty internal debugging. We only focus on the following parts.

Copy code
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1] handler_added
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1] registered (1)
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1] BIND: 0.0.0.0: 8007 (2)
Wait the client input
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1, 0.0.0.0: 8007] active (3)
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1, 0.0.0.0: 8007] Read (4)
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1, 0.0.0.0: 8007] received: [ID: 0x7bac2775, 127.0.0.1: 64073:> 127.0.0.1: 8007] (5)
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1, 0.0.0.0: 8007] received_complete (6)
Dbug: SRV-LSTN [0]
[ID: 0x3e8afca1, 0.0.0.0: 8007] Read (7)
Dbug: SRV-CONN [0]
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] handler_added (8)
Dbug: SRV-CONN [0]
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] registered (9)
Dbug: SRV-CONN [0]
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] active (10)
Dbug: SRV-CONN [0]
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] Read (11)
Dbug: dotnetty. buffers. abstractbytebuffer [0] (12)
-Dio. netty. Buffer. bytebuf. checkaccessible: True
Dbug: SRV-CONN [0]
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] received: 14B (13)
+ ------------------------------------------------- +
| 0 1 2 3 4 5 6 7 8 9 a B c d e f |
+ -------- + --------------------------------------------------- + ---------------- +
| 100000000 | 00 0C 68 65 6C 6C 6f 20 77 6f 72 6C 64 21 |... Hello world! |
+ -------- + --------------------------------------------------- + ---------------- +
Received From Client: Hello world!
Dbug: SRV-CONN [0] (14)
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] Write: 2b
+ ------------------------------------------------- +
| 0 1 2 3 4 5 6 7 8 9 a B c d e f |
+ -------- + --------------------------------------------------- + ---------------- +
| 100000000 | 00 0C |... |
+ -------- + --------------------------------------------------- + ---------------- +
Dbug: SRV-CONN [0] (15)
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] Write: 12B
+ ------------------------------------------------- +
| 0 1 2 3 4 5 6 7 8 9 a B c d e f |
+ -------- + --------------------------------------------------- + ---------------- +
| 100000000 | 68 65 6C 6C 6f 20 77 6f 72 6C 64 21 | Hello world! |
+ -------- + --------------------------------------------------- + ---------------- +
Dbug: SRV-CONN [0] (16)
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] received_complete
Dbug: SRV-CONN [0] (17)
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] flush
Dbug: SRV-CONN [0] (18)
[ID: 0x7bac2775, 127.0.0.1: 64073 => 127.0.0.1: 8007] Read
Copy code
At first glance, there are 18 operations, which seem to be a bit too many. Otherwise, there are still a lot of internal debugging details that are not printed on the console.

By manually establishing a working thread group and registering this group of threads into the pipeline, this pipeline can be based on socker, and can be based on ichannel (1 );
Bind the custom IP address and port number to the custom MPs Queue (2 );
Activate custom MPs Queue (3 );
Start reading (actually listening) (4 );
After receiving a client connection request whose ID is 0x7bac2775, establish the connection and start listening (5) (6) (7 );
Starting from step 1, the log has changed to a record with the ID 0x7bac2775. Of course, the log contains the same operations as the S-end, such as registering the MPs queue, activating the MPs queue, and starting the listener (8) (9) (10) (11)
When I enter a "Hello world! "After the data, dotnetty. buffers. abstractbytebuffer checks the data type to confirm that the data can be placed in the pipeline. (12)
Send data to the S end. The data size is 14B. There are two vertices before Hello world. This indicates that this is the data header, and then two vertices are sent, but no data exists, indicating that the data has ended. Dotnetty presents the hexadecimal storage space of data in a user-friendly manner. (13) (14)
No processing or processing is performed on the received data on the send. The data is immediately uploaded back to the C end. (15) (16)
Finally, after this process is completed, the data in the cache area needs to be forcibly written to the pipeline, so a flush operation is executed, and the entire transmission is completed. Next, you can change your status to read on both the C and S ends to listen to various situations in the pipeline, such as connection status and data transmission (17 ).

Summary
This is a nightmare for friends who are new to socket programming, because the complexity of socket programming is not easier or even more complex than multithreading. Protocol, compression, transmission, multithreading, listening, stream control, and so on. Therefore, netty is an excellent open-source framework, but netty is a semi-finished product, because you need to implement the desired protocol, transmission, and other custom operations based on it, and you do not need to care about the underlying content. Unlike some frameworks, such as functional frameworks such as newtonsoft. JSON, you can simply use them without configuration or customization.
Although dotnetty has helped us implement a large number of operations at the underlying layer, if we are not familiar with or do not know network communication at all, we are also confused about the above Code. Why? The market needs. Our programmers are catching up with business every day. How can we learn and learn more details... by printing out the debugging records and analyzing the comparison code one by one, we will start to understand the simplest communication process.
This article only implements the simplest communication process based on dotnetty. It just makes a loop of data and does not implement any RPC-related calls. The next article will introduce this example in depth, this section describes dotnetty-based RPC calls.

Simple communication process