How does python implement rpc through protobuf?

This article describes in detail how to implement rpc through protobuf in python. If you are interested, refer to this article because the rpc currently used by the project team is implemented based on the google protobuf rpc protocol, so it took some time to understand protobuf rpc. Rpc is certainly no stranger to people who are working on distributed systems. If you are not familiar with rpc, You can google your own shoes. Here is a brief introduction. The main function of rpc is to make the implementation of Distributed Systems simpler. It provides powerful remote calls without compromising the simplicity of local call semantics. To achieve this goal, the rpc framework needs to provide a transparent call mechanism so that users do not need to distinguish between local calls and remote calls. The components involved in the rpc architecture are as follows:

The client calls the remote interface method just like calling a local method. The RPC framework provides the proxy Implementation of the interface, and the actual call will be delegated to the proxy RpcProxy. The proxy encapsulates the call information and transfers the call to RpcInvoker for actual execution. The client RpcInvoker maintains the RpcChannel with the server through the connector RpcConnector, uses RpcProtocol to execute protocol encoding (encode), and sends the encoded request message to the service provider through the channel. The RPC server receiver RpcAcceptor receives client call requests and uses RpcProtocol to perform protocol decoding (decode ). The decoded call information is passed to RpcProcessor to control the process of processing the call, and then the call is called to RpcInvoker for actual execution and return the call result.

Protobuf rpc mainly plays the role of RpcProtocol in the above components, which saves the protocol design and the protobuf protocol is very efficient in coding and space efficiency, this is why many companies adopt protobuf as data serialization and communication protocols. Protobuf rpc also defines an abstract rpc framework, as shown in:

The RpcServiceStub and RpcService classes are the classes generated by the protobuf compiler according to the proto definition. RpcService defines the function interfaces exposed to the client by the server. You must inherit the class to implement the specific implementation. RpcServiceStub defines the description of the function exposed on the server, and converts the client's call to the function call in RpcServiceStub to call the CallMethod method in RpcChannel, callMethod uses the function Descriptor and function parameter passed by RpcServiceStub to encode this rpc call and sends it to the service provider through RpcConnecor. The opposite process of the client finally calls the function defined in RpcSerivice. In fact, the protobuf rpc framework only defines an empty CallMethod in RpcChannel. Therefore, you must implement the encode and call RpcConnector by yourself. RpcConnector is not defined in protobuf, so this is implemented by the user. Its function is to send and receive rpc message packets. On the server side, RpcChannel calls the CallMethod in RpcService to call the function exposed to the client in RpcService.

I have introduced so much about how to use protobuf rpc to implement an rpc, but it is still a fog. The following describes how to use protobuf rpc to implement a simple python version of rpc demo.

The demo describes the PRC proto file. For the preparation rules of the proto file, refer to the protobuf official website.

Common. proto file:

package game;message RequestMessage{  required string message = 1;}message ResponseMessage{  required string message = 1;}

Game_service.proto file:

package game;import "common.proto";option py_generic_services = true;service GameService{  rpc connect_server(RequestMessage) returns(RequestMessage);}

The common. proto file describes the messages sent and received in RPC. game_service.proto describes the connect_server function exported by the server. This function accepts the RequestMessage object as a parameter and returns the RequestMessage object. When using the PRC protocol, you must add option py_generic_services = true; optional; otherwise, the compiler will not generate the GameService description that contains the connect_server function.

Use the compiler protoc to compile the proto file. The specific command is:
Protoc.exe -- python_out =. game_service.proto
The compiled file is game_service_pb2.py, which implements the GameService and GameService_Stub classes. The GameService_Stub class is used by client callers to call the GameService service.
As mentioned above, on the client side, RpcChannel only implements an empty CallMethod. Therefore, you need to inherit RpcChannel and re-use this function to encode messages and send messages. In the server RpcChannel, CallMethod must be called to call functions in the Service. The specific implementation is as follows:

Class MyRpcChannel (service. rpcChannel): def _ init _ (self, rpc_service, conn): super (MyRpcChannel, self ). _ init _ () self. logger = LogManager. get_logger ("MyRpcChannel") def CallMethod (self, method_descriptor, rpc_controller, request, response_class, done): "functions required by protol buffer rpc, used to send an rpc call "self.logger.info ('callmethod') pai_index = method_descriptor.index assert (pai_index <65535) data = request. serializeToString () total_len = len (data) + 2 self. conn. send_data (''. join ([pack ('
 
  

  
The last step is to inherit GameService and implement the connect_server function.
  

  
class GameService(game_service_pb2.GameService):  def __init__(self):    self.logger = LogManager.get_logger("GameService")  def connect_server(self, rpc_controller, request, callback):    self.logger.info('%s', request.message)
  

  
For the RpcConnector used for sending and receiving messages on the network, you can use the asyncore library of python. The specific implementation will not be discussed here.
  
From the above implementation, the implementation of protobuf rpc mainly includes compiling the proto file and compiling and generating the corresponding service_k2file, inheriting the RpcChannel and implementing the CallMethod and calling the Service CallMethod, inherits services to implement functions exposed to the client.
  
The above is all the content of this article, hoping to help you learn.