Analysis of HSF Source code

Preface

HSF is a distributed remote service invocation framework, in fact, I prefer to remove the distributed word, because HSF itself is not a separate service (refers to a process), he is a component attached to your application, an RPC component (Remote procedure call--remote Procedure Call, a protocol that requests services from a remote computer program over a network without needing to know the underlying network technology. In the OSI network communication model, RPC spans the transport and application tiers. RPC makes it easier to develop distributed applications, and of course HSF's full content is certainly more than that.

What is the full name of the HSF in that long? High-speed Service Framework

RPC

Let's take a look at a picture:

 Many students read this picture may think this and the process of HTTP what is the difference?

There is such a scene (would like to give a specific business example, think or have the technology to achieve relative good), monitoring platform : Monitoring the status of all hosts, at this time each host has an agent, every few seconds to the monitoring platform to upload data (host memory utilization, hard disk status, CPU, load, process information, and so on).

Perhaps the simplest way to develop is that the monitoring platform has an HTTP interface, the agent requests every few seconds, to meet the demand, but if the host number of rapid growth, monitoring more and more, the request body is getting larger, you will find that the efficiency of the HTTP transmission decreased The time spent on each call increases.

At this point we will study the HTTP protocol, to optimize the process, the discovery of the HTTP process is: To establish a connection, send request information, send response information, close the connection, see this process first of all want to optimize is not every time to establish a connection to close the connection, because the data escalation is a continuous process , and then to study the HTTP header, found that many protocols do not use, miscellaneous, in vain to increase the message body; Later, I think the HTTP protocol parsing and restore process is very complex, you can develop a performance to improve ...

RPC came, he can meet these needs, but the premise is the need for development, need upfront costs, so want to measure the project design, but nothing, we have HSF ah.

We'll change it a little bit:

 Now you can see that there is a long connection between the client and the server, and we have our own protocol body: Rpcrequest and Rpcresponse.

RPC is here, after all, the focus is HSF, want to know more about RPC, can be on the wiki or online query.

HSF Architecture

In fact, in our application, your application is not only the client, but also the server, because you need not only to call the services provided by other applications, but also to provide services to other applications, so that the entire HSF service call link will be very complex.

From the two images above, we have obviously found a problem, that is, how the service provider informs the client of the services he provides , so there is a need for a service to register and discover where it is provided in the HSF architecture configserver, such as:

 It can be seen from the server side of the boot when the configserver to register their own services, the client will be to Configserver Subscribe to the services required by Configserver to push the address of the relevant service provider and other critical information to the clientvia subscription information.

Basic capabilities have been implemented above, but how to dynamically configure the load (thread pool size), default configuration (Configserver address, etc.), and some feature features (such as routing rules) require a persistent configuration center, such as:

 when the client and server start, they will go to the diamond to obtain the required configuration information, such as the most critical service registry type and address, in addition to the type and address of service governance, etc.

Let's focus on routing rules , for example: Routing rules are configured to call only the server in the same room when the service is called, which is certainly less time consuming than spanning the room. In addition to the HSF also wrote a separate Unitservice service unit released to distinguish the center release, these things have time to write a foreign article, here is not too much elaboration, after all, these somewhat biased scene of the content of the business may be changed to another way.

 I believe that we have used the HSF service governance site, through this site can see what services, the address of the service provider, how many providers, the specific consumers who, HSF through Configserver, Redis, diamond storage information to obtain this information.

Redis Features: HSF uses Redis storage metadata, each HSF consumer/provider in the start, every once in a while to the Redis report metadata, these metadata data collected and provided to Hsfops do service governance, Includes the application name and service mapping, metadata for the service, and so on.

registration and publication of servicesNext we'll unpack the server and see what's inside.

<bean id= "Hsftestservice"        class= "Com.test.service.impl.HsfTestServiceImpl"/> <bean    class= " Com.taobao.hsf.app.spring.util.HSFSpringProviderBean "        init-method=" init ">        <property name=" ServiceName "value=" Hsftestservice "/>        <property name=" target "ref=" Hsftestservice "/>        < Property name= "Serviceinterface" >            <value>com.test.service.hsftestservice            </value>        </property>        <property name= "serviceversion" >            <value>${hsf.common.provider.version} </value>        </property></bean>

I believe that the students are familiar with the above configuration code, then the service is how to register it, why the configuration of the service here can be called?

From the configuration file to see a key Bean--hsfspringproviderbean, there is a key initialization method Init, in fact, the process of Init is the service release process, we look at the Hsfspringproviderbean part of the code:

     public void Init () throws Exception {        //avoid being initialized multiple times        if (!providerbean.getinited (). Compareandset (False, True)) { C3/>return;        }        Loggerinit.inithsflog ();        Spasinit.initspas ();        Providerbean.checkconfig ();        Publishifnotinspringcontainer ();    }    private void Publishifnotinspringcontainer () {        if (!isinspringcontainer) {            Logger.warn ("[ Springproviderbean] is not created in the spring container, it is not recommended ");            Providerbean.publish ();        }    }

　　

It is obvious from the code to see the Service Release Providerbean.publish (), first look at the approximate class diagram, some of the class diagram is not very critical first omitted:

Basically explain the class diagram, which is also a process for service publishing:

2. Service initialization requires a service implementation class (Spring Bean) and interfaces that provide services first;
4. Initialize the Hsfspringproviderbean, get the service name, interface, implementation class, version, etc. from the configuration file;
6. Providerbean is a variable hsfapiproviderbean in Hsfspringproviderbean, Hsfspringproviderbean will get the service name, interface, implementation class from the configuration file, version, etc. assigned to Providerbean;
8. There is a service entity class in Providerbean that Servicemetadata,providerbean all the information that the service publishes, such as interfaces, implementation classes, versions, and so on, throughout the publishing process, Servicemetadata is the transfer object between all objects;
10. Here first to explain why there are Hsfspringproviderbean and Hsfapiproviderbean, in fact, two can be merged into one, but why separate it? My understanding is that different implementations of different environments, such as the spring environment now, require a spring adaptation class Hsfspringproviderbean to get configuration information, and if it is another environment then there will be another adaptation class. Finally, the unification of information to Hsfapiproviderbean,hsfapiproviderbean is to realize the concrete operation;
12. When executing providerbean.publish (), the Publish method of Processservice is called, and the implementation class is processcomponent;
14. The specific process for publishing is processcomponent:
    • The first step is to call Rpcprotocolservice to register the RPC service, which is to publish a thread pool locally on the server, and each service will request a thread pool to fetch executor from the thread pool to execute and return when requested.
    • The second step, check the cellular release, the Unitservice in the pre-release check is a central release or unit release, the Servicemetadata set different release routes;
    • The third step is to publish servicemetadata to configserver through MetadataService;
    • The fourth step is to save Servicemetadata to Redis for service governance or other purposes through metadatainfostoreservice.

A service registration release is roughly the process.

the client of HSF

Now look at how the client is going to invoke the service.

<bean id= "Hsftestservice" class= "Com.taobao.hsf.app.spring.util.HSFSpringConsumerBean" init-method= "init" >    <property name= "InterfaceName" value= "Com.test.service.hsfTestService"/> <property    name= " Version "Value=" 1.0.0.daily "/></bean>

The above configuration file is sure to be very common in the project, so how does he work? When spring is injected, there is no specific implementation class Ah, there is only one interface? How do I implement a call?

In fact, this is a curiosity of my place, I would like to see exactly how the HSF is the way to achieve.

Let's start by thinking about the question, that is, how does HSF implement registering services in spring without a specific implementation class? The answer is the dynamic agent, similar to the MyBatis way, mybatis writing the DAO layer just write an interface, and not specifically implemented, HSF is similar to this way.

The client is divided into two parts: the subscription of the service and the call to be pushed, and the service.

subscriptions and tweets for services

First look at the class diagram:

 

As we look at the subscription and receive process for a service through class diagrams:

2. Service initialization, the first need to introduce the service interface related to the POM, and then write the configuration file;

3. Registers the service that needs to be called as a spring bean, which is the contents of the above configuration file.

   • This uses the dynamic agent, through the class diagram we can see Hsfspringconsumerbean realizes the Factorybean;

   • Factorybean: Is a Java bean, but it is a factory bean that can produce objects, returning the specific bean through the GetObject method, in the process of instantiating the bean by the spring bean, it will judge whether it is Factorybean, If it is not to return the bean, otherwise return factorybean production of beans, specific students can go to see abstractbeanfactory Dogetbean method, inside will call Getobjectforbeaninstance method, There are concrete implementations in this method;

   • hsfspringconsumerbean implemented Factorybean, what exactly did the GetObject method return? How to return it?

      @Overridepublic Object GetObject () throws Exception {return Consumerbean.getobject ();} 

     From the code to see that Consumerbean (Hsfapiconsumerbean) called the GetObject method returned, then we look at the GetObject method:

     public Object GetObject () throws Exception {return metadata.gettarget ();} 

     This method returns the target of metadata (servicemetadata), so how does target get it? The following emphasis is shown;

   • The Init method of the

   • Hsfspringconsumerbean calls the Init method of Consumerbean (Hsfapiconsumerbean), and we look at a piece of code for the Init method in Consumerbean:

     processservice processservice = hsfservicecontainer.getinstance (    Processservice.class); try {metadata.settarget (Processservice.consume (metadata)); Logger.warn ("successfully generated to the interface for [" + metadata.getinterfacename () + "] version of the HSF service call for [" + metadata.getversion () + "]! ");}            catch (Exception e) {logger.error ("", "generated to interface for [" + metadata.getinterfacename () + "] version for [" + Metadata.getversion ()    + "] The proxy for the HSF service invocation failed", e); Since 2007, throws a throw e once the exception is initialized;} int waitTime = metadata.getmaxwaittimeforcsaddress (); if (WaitTime > 0) {try {Metadata.getcsaddresscountdown    Latch (). Await (WaitTime, timeunit.milliseconds); } catch (Interruptedexception e) {//Ignore}} 

     This piece of code contains the specific build of the dynamic proxy object and the service subscription and Service information received;

   • First of all, the code logic, the service subscription and the service information received (pushed) in the Processservice, the dynamic proxy object generated in the Processservice, the following wait I guess is used to wait for target service information push (when the target of receiving the subscription of the specific service implementation , the next call process can walk through);

   • It seems that Processservice is a very important component, this way through processservice.consume (metadata) Such a method call to achieve so many steps, the target is also generated in this area, Let's talk about the logic inside this method:

     • First go to the cache to find whether the previous target has been generated, there is a return;

     • No object is generated from Java proxy;

     • Subscription service information (return callable address);

     • Save the client metadata to Redis and return to target.

So far, the build of the service proxy object, the subscription to the service is complete, and then the call to the service.

invocation of the service

In fact, through the above two parts of the entire framework has been set, the service information has been registered to publish, the client also gets the call address of the service, then the call is called on the line, the call is the real RPC request, HSF RPC is implemented through Netty.

Directly above the class diagram:

Before the dynamic agent is said, then when the method executes, the line enters the proxy class execution, executes the Hsfserviceproxy invoke method, the Invoke method calls the Trueinvoke method:

• Call Rpcprotocoltemplateservice in Trueinvoke to encapsulate hsfrequest here, execute the specific Invoke method;

• The specific invoke method call Rpcprotocolservice, here is mainly based on Invoketype to determine the specific Invokeservice implementation, the most basic we know that the HSF service has synchronous calls and asynchronous calls, the concrete implementation is here;

• Finally, in the specific implementation of the class to get nettyclient, and the server to communicate, return to Hsfresponse.

Simply say the service-side process:

• The server will start the nettyserver, specifically nettyserverhandler to handle all RPC requests;

• Nettyserverhandler will find specific handler according to Hsfrequest, this side is rpcserverhandler, besides there is heartbeat ah and so on handler;

• Get the specific execution of executor through handler (this is said on the previous service registration side, each service will apply for a thread pool, threadpoolexecutor);

• New A handlerrunnable is put into executor execution Executor.execute (new handlerrunnable);

• Finally, calling Providerprocessor,providerprocessor in handler will find the specific service implementation class and execute, encapsulate the execution result into Hsfresponse, and return Hsfresponse to the client.

 written in the last

I speak more here is the main link, there are a lot of specific details such as routing, Eagle Eye tracking, logs, load, etc. did not unfold, in fact, each point can be written an article, may be for the development of the HSF students, every point will have a very fun story, then about the HSF on the first to talk about here.

Analysis of HSF Source code