[Turn] Boot Camp series--Platform RPC Framework Introduction

Original: http://weibo.com/p/1001643875439147097368

Course Outline

1. Introduction to RPC

1.1 What is RPC

1.2 RPC compared to other remote invocation methods

2. Motan RPC Framework

2.1 RPC Service Framework

2.2 Roles in the Motan RPC framework

2.3 Motan RPC Call flow

3, Motan RPC in the role of the introduction

3.1 Registration Center

3.2 RPC Service

3.3 RPC Client

4, Motan actual combat

How to use 4.1 motan

4.2 Using Motan to provide RPC services

4.3 Calling the RPC service using Motan

5. Summary

6. After-school thinking

I. Introduction of RPC

RPC protocol is language-independent, communication protocol independent in large and medium-sized distributed systems, remote service invocation is a key link. RPC is widely used in remote service invocation because of its language-independent, simple and efficient features. This article will briefly introduce the basic concepts of RPC and the rationale and usage of the RPC service framework for the microblogging platform.

1. What is RPC?

RPC (remote Procedure Call) is a protocol that invokes a remote procedure (or method) over a network. RPC is based on the Client/server mode, the client side carries the necessary parameters to call the server-side method, and gets the results of the method execution returned by the server side.

, high efficiency and reliability, it is well suited as a basic protocol for remote interaction such as distributed computing and distributed services. RPC protocol generally includes protocol message processing, network transmission two parts, a simple RPC call process such as:

When the client calls the remote method, the method name, parameters, and other information to be invoked are converted to RPC messages by the RPC protocol, and then the RPC message is transferred to the server side through some transport protocol (TCP, HTTP, etc.). After the server has received the request, the RPC message is converted to the calling request by the Protocol, and the execution method is returned to the client by a similar procedure, which completes the RPC call.

2, RPC and HTTP, RMI and other remote communication mode comparison

1), RMI and RPC:

RMI (remote method invocation) refers to the method invocation in the Java language, in which each method in the RMI has an approach signature, and the RMI client and server side make remote method calls through the method signature. RMI can be used only in the Java language to treat RMI as an object-oriented Java RPC.

2), HTTP and RPC

HTTP (hypertext Transfer Protocol) is an application-layer communication protocol that uses standard semantics to access a specified resource (picture, interface, and so on), and the brokered server in the network can identify the protocol content. The HTTP protocol is a resource access protocol that allows you to complete remote requests and return request results through the HTTP protocol. The advantages of HTTP are simple, easy to use, understandable, and language-independent, the disadvantage of HTTP protocol is that the protocol header is heavier, the general request to the specific server link is longer (there may be DNS resolution, Nginx proxy, etc.).

As opposed to HTTP, the communication layer is only part of the RPC protocol, and RPC can use TCP or HTTP as a communication protocol. In addition to the communication protocol, RPC also includes the request serialization protocol, the general RPC framework will also include some scheduling management capabilities. The advantage of RPC over HTTP is the ability to provide scheduling, management capabilities, the use of RPC services is relatively high degree of automation, performance is excellent. The disadvantage is relatively complex, the learning cost is slightly higher.

The following is a platform RPC framework Motan as an example, the basic implementation of the RPC service and how to use.

Second, Motan RPC service Framework

1. RPC Service Framework

The General RPC protocol contains only the point-to-point call between the client and server, in the actual use, also need to consider the service discovery and service logoff, the provision of services of multiple servers load balancing, high availability of services and a series of problems.

At present, the industry more mature RPC Framework has Ali Open source Dubbo, Facebook Open source Thrift, as well as Avro, Hetty, Grpc and so on, thrift, Avro, Hetty, Grpc and so on are excellent performance RPC framework, It focuses on cross-language serialization transmission, in the serialization mode, protocol extension, synchronous asynchronous invocation and other aspects have different advantages, but in the RPC service management features less, in the Distributed service application also need to do some auxiliary development. Dubbo is a comprehensive service governance framework that provides high-performance, transparent RPC calls and powerful service management capabilities. Dubbo is powerful, but the whole frame is a bit overweight and slightly more complex to apply.

Considering the requirements of micro-service, dynamic service governance, and the high concurrency of its own business, service performance and stability requirements, the microblog platform has developed its own Motan RPC framework. Motan emphasis on simple and practical service governance functions and excellent RPC protocol expansion capabilities, can provide efficient RPC remote call, but also provide service discovery, service high availability (Hi Available), load Balancing, service monitoring, management and other service governance functions. The SPI mechanism provides powerful scalability to support different RPC protocols and transport protocols. Motan can seamlessly support spring configuration using RPC services, providing or using RPC services with a simple, flexible configuration. By using the Motan framework, service splitting and distributed service deployment can be very convenient.

2. Three characters in the MOTANRPC framework

In the Motan framework, RPC service calls are performed by the interaction of three roles. First, you need a service provider (also called provider), and the service provides a variety of methods that can be remotely called (method), and another role called client (also called referer or consumer). The client needs to use the remote service provided by the service. If you have only these two roles, you cannot implement an automated RPC service, and you need a third role-the registry registry,registry is used to provide service discovery functionality.

When the service is ready to provide an RPC service externally, it first needs to register with registry. When the client needs to use the RPC service, it needs to subscribe to the RPC service with registry, Registry will give the corresponding service information to the client, and notify the client when the service information changes. The client requests a specific server that can provide the specified RPC service, based on the service information provided by registry, to complete the remote call. Service, Client, registry The interaction of three roles such as:

For example, Server1 this server and registry registered "s" This RPC service, registry saved a "Server1 provide S service" such a message. Server2 can also provide "s" services, and register the "s" Service with register. The information in register becomes "Server1 and Server2 provide s service". When Client1 needed to use S service, it subscribed to register S service, Register told Client1: "Server1 and Server2 provide s service", Client1 has chosen to invoke Server1 's S service to complete the RPC call based on a policy. Later, Server1 did not provide the S service, and register told the change to Client1,client1 to no longer use Server1 's s service, but instead used the Server2 s service to complete the RPC call.

When using Motan, the Java language uses the Java interface class for the client and service to make RPC calls. For example, the service side implements the Com.weibo.demo.TestService interface, and the client can use the method declared in this interface class through Motan.

3. Motan RPC Call Flow

The modules included in the registry, Service, and client three roles in the Motan framework and the interaction flow of each module are as follows:

The register module is used to interact with the registry, including registration service, subscription service, service change notification, service heartbeat sending and other functions; the transport module is used for remote communication; The Serialize module serializes and deserializes objects such as parameters, results, and so on in RPC requests , which is to convert the object to the byte stream; The Protocol module is used to describe the RPC service and the configuration management of the RPC service; The cluster module contains a set of specific servers that can provide RPC services. The actual request selects one of the available originating remote calls in a set of servers based on different ha and load balancing policies. The details of each module are described in the next section.

Three, the role of Motan in the introduction

1, Registry

Motan can support a variety of registry modules, such as zookeeper, Consul, etc. can be used as a registry, the default registry module using platform-developed vintage configuration services. Vintage is a Redis-based lightweight kv storage system capable of providing namespace services, service registration, service subscriptions, and more. The services in vintage are stored in groups (group), typically a grouping is named by the room + line of business, such as YF-USER-RPC. A grouping contains several service, and a service represents an interface class in Java that can provide multiple methods.

There is a set of servers under each service that can provide the corresponding services, and each server, after registering with registry, sends a heartbeat periodically and reports health status to registry. The server can control whether the RPC service is available externally by opening and closing the heartbeat switch.

After the client subscribes to the service, a set of server,client that provide the corresponding service is obtained from the registry to treat this set of servers as a cluster to serve. When the server in the cluster changes, the client's register module notifies the client to update.

2. Motan Service

The main steps of the service side when starting the RPC service are as follows:

1), first through the configuration file to generate the corresponding protocol information, including the external service port, service name, number of worker threads, etc.

2), register the RPC service to vintage (i.e., registry), including the protocol information that provides the service, the IP of the server, the port that provides the service, and so on.

3), open and listen for the declared RPC service port. At this point the service is in a state that has been registered but not formally serviced.

4), after the service verification, open the heartbeat switch, the server will regularly report to the Vintage Registry health status, at this time can formally provide RPC services.

After serving the service side, the transport module is responsible for listening to the RPC service's port and receiving the RPC request sent by the client side. A complete service-side processing RPC request process is as follows:

1), listen to RPC service port, receive RPC Request,transport module by Transport module by default, use Netty NiO to communicate with client via TCP long link. The transport module receives the request and then transfers the request to the Serialize module for parsing.

2), the Serialize module uses Hession to decode RPC requests by default, transferring requests from bytes to Java objects.

3), the Fliter module will be based on the parameters of the request to do a filter policy, such as call statistics, logs, request restrictions and so on.

4), protocol is responsible for parsing through the filter of the request object, according to the protocol analysis of the request to invoke the method and parameters, and then find the specific object to provide the corresponding service.

5), the specific service object through the reflection mechanism to complete the call of the method in the service and return processing results.

6), the return result through the filter, the result object is converted to a byte stream through the Serialize module, sent by the transport module to the client side, complete the processing of the RPC request.

3. Motan Client

The initialization process for the Motan client side is as follows:

1), as with the service side, first generate the corresponding protocol information through the configuration file.

2), the RPC service required to subscribe to the registry, get all the servers that provide the corresponding service, this set of server and HA policy module, load balancer module together constitute a cluster.

3), establish an initial link for each server.

After the client has completed the initialization process, when invoking the RPC service, there is a process for selecting the server, which is similar to server-side processing when the server is selected. The process of client calling RPC is as follows:

1), when making RPC requests, first select a highly available policy through the HA module, such as fast failure, failure to select other server retry policies, the parameters of various policies can be set as required.

2), then select an available server from a set of servers that provide services through the Load Balancer module. The policies for load balancing include consistent hashing, polling, concurrency, weights, and so on.

3), select the server, the RPC request for this server, the figure Netty client is used for point-to-point RPC request, including the Protocol, serialize, transport module, The processing process of the request is basically consistent with the process of the service side, and is also the transport module to communicate through the Serialize module to convert the Request object.

Four, Motan actual combat

1. Motan RPC Basic usage mode

Motan RPC is an interface class that provides and invokes services by interface, in the case of Java services, typically the service provider declares the service, and provides a default RPC client invocation configuration, and then encapsulates the interface class with the default client configuration XML file as a jar package. The service provider and the service consumer introduce this jar package, the service provider implements the interface class and provides RPC service externally; The service consumer loads the default client-side configuration file and uses the RPC service directly.

In addition to the way XML is configured, Motan also supports the API approach because the XML configuration is flexible and fully compatible with the spring configuration load, and it is generally recommended to configure the Motan service in XML mode. The following is an XML configuration that describes the specific steps for using Motan to provide and invoke RPC services.

2. Using Motan to provide RPC services

Using Motan to provide RPC services takes roughly the following steps:

1), declares the RPC interface class, and implements the interface class.

2), introduce Motan related jar package.

3), configure the server-side XML configuration.

4), deploy, start the service, verify the service.

5), open the heartbeat switch, formally provide RPC service

A basic service-side XML configuration is as follows:

Motan uses a custom XML schema, which needs to be introduced in XML. The XML configuration of Motan includes registry, Protocol, service, Referer parts, where registry and protocol are used by both the service side and client side.

Registry is configured with registry-related properties, including registry IP, prot, timeouts, and so on, protocol configures RPC service-related properties, such as number of worker threads, request timeouts, message lengths, and so on, Basicservice configured public properties for different services. When a project provides more than one RPC service, the public properties of these services can be configured in Basicservice, and the service is configured with specific properties, including ports for external services, specific service implementation class beans for external services, etc.

In addition to these underlying properties, a large number of advanced properties are supported under each Configuration tab, such as access control, filter policy, and so on, which is not explained in one by one.

3. Call RPC Service using Motan

Using the Motan RPC service is relatively straightforward, and the General Service provider has given the default client configuration by simply loading the XML configuration into the project and using the RPC service directly as if it were a local spring bean. The steps are as follows:

1), introduce Motan related jar package and RPC Service Interface jar package.

2), load the default XML configuration in the RPC interface jar package

3), directly using the refer Bean declared in XML.

The client-side basic configuration is as follows:

Where the registry, protocol parts are consistent with the configuration of the service side. The Basicrefer configuration is a common configuration for multiple RPC services, such as request time-out control, whether the server exception is thrown, and so on, Referer represents a remote reference to the service implementation class, and properties such as the Service interface class used are configured.

If you want to use a custom client-side configuration, you can also configure the XML separately by the attributes in the schema, which can take effect after loading. In addition, in the client side can also do some more detailed configuration, including the different methods in an interface class configuration, can be configured according to the specific business scenario.

V. Summary

RPC service is more and more widely used in distributed system, this paper briefly introduces the basic concept of RPC, the basic structure of platform RPC framework, the Motan of three roles in Motan, and the basic usage mode. The RPC service can be used with a simple configuration. Of course, if you need finer control over the RPC service, you can configure it with a specific attribute in the schema or use a different extension module.

Six, after-school thinking

1, the different call mode of the remote service, the difference between RPC and HTTP, and the applicable scenario.

2. Motan the three roles in the RPC framework and the interaction process between the roles.

3. Call procedure for RPC request.

------------------Boot Camp Introduction------------------

Micro-blogging Platform Boot camp activities are organized within the microblog platform for new recruits to the team into the training course, the goal is the integration of the team, including the integration of people, atmosphere, technology integration. At present, there are 4 activities, many students quickly grow into the backbone of the platform technology.

Weibo platform is very focused on team members into the growth of the team, where someone to help you into, someone with you to grow up, but also welcome the small partners to join the microblogging platform, welcome private messages consultation.

------------------Instructor Profile------------------

Zhang Lei, @rayzhang0603 Weibo platform and Big Data Department-platform Research and development senior system development engineer, July 2012 graduated from Taiyuan University of Science and Technology, 13, joined the microblogging platform, has been responsible for open platform, powder clothing developers, MOTANRPC and other backend service architecture work, Has a wealth of system architecture design and optimization experience. The second phase of the boot camp.

[Turn] Boot Camp series--Platform RPC Framework Introduction