[Reproduced] Boot Camp series courses-platform services deployment and web framework

Original: http://weibo.com/p/1001643875679132642345

Outline

Weibo platform is mainly responsible for the basic functions of Weibo. Next you will be introduced

• The role of the platform, as well as the form of service delivery

• Deployment of Platform Web services

• Introduction to the Platform Web framework

Background

The overall structure is broadly divided into three tiers

• Presentation layer: Mobile, master and third-party applications that bear the front-end of related business

• Adaptation layer: Responsible for interface adaptation of the service end and multiple display terminals

• Service layer: Provide basic function services, including feed service, user relationship, open Platform and message box, etc.

The platform serves as the basic functional service layer of the entire microblog architecture, and provides services in the form of HTTP interfaces. The interface complies with the RESTful specification. The interface examples are as follows:

About restful is more like an architectural design style than a specification. It is mainly to provide a set of design principles and constraints, widely used in the C/s or/b architecture. To understand what is restful, you can start with its full name--representational state Transfer, translated into Chinese is the representation of the status transformation. This obscure text omits the subject, "The expression layer" actually refers to "the resources" the "Expression layer". Core concepts include

• A resource is specified by a URI.

• Operations on resources include acquiring, creating, modifying, and deleting resources that correspond exactly to the get, POST, put, and Delete methods provided by the HTTP protocol.

• Manipulate resources by manipulating their representations.

To sum up, the platform provides services externally through the HTTP interface to access the underlying resources.

Platform Services Deployment

After we have a better understanding of the platform's positioning and service form, we look at the Web service deployment structure of the platform.

The services of the platform are deployed in multiple engine rooms. Take Beijing For example, there are AX, BX and CX three room. The self-built DNS service switches the user's traffic to a different machine room depending on the operator.

After a user request arrives at the server, it first passes through the reverse proxy. The reverse proxy method refers to the proxy server to accept the connection request on the public network, and then forwards the request to the server on the internal networks, and returns the results from the server to the Reverse client, at which point the proxy server appears as a reverse proxy server. The reverse proxy currently used by the platform has LVS and Nginx.

• Lvs:linux Virtual Server, IP-based load balancing and reverse proxy technology

• Nginx: HTTP-based load balancing and reverse proxy server

About Nginx, in addition to the above mentioned, also responsible for cluster health check. This is achieved mainly through the Nginx's own health check module. Nginx Server polls The index.jsp page of the backend cluster, and if return 200 considers the server to be normal, the request is forwarded to the server normally, and 503 is removed, and the request will no longer reach the server.

After the reverse proxy is forwarded, the request arrives at the application server where the Web app is deployed. The platform currently uses Tomcat primarily as an application container. The request is then parsed and processed by the Unified web framework. The contents of the web framework are described in more detail later.

For different requests upstream and downstream, the link to request processing is also different.

Take the microblog core business feed stream as an example. When the application server receives a downstream request (such as querying the content of a microblog), it accesses the cache resource directly, and if hit returns the result directly to the client, it continues to query the DB and returns the result to the client.

When an upstream request (such as a tweet) is received, the application writes the upstream request to a message queue. A separate processing application reads the message queue, performing resource operations on upstream requests, such as write cache, update db, and so on.

The uplink request mode of this queue processor is widely used by the platform, and the main advantages are as follows:

• Decoupling front-end applications from back-end resources

• Peak load shifting: The queue can be used as a buffer when the volume of requests is large, easing the pressure on the backend resources

As the request is assigned to a different room, the data between the multi-machine rooms also needs to be synchronized. At present, we use the virtual message pipeline WMB to synchronize the data between the room: all the upstream request after reaching a room will be replayed through WMB to other rooms, so as to ensure that the room back-end resources consistent. In addition, in order to disaster tolerance, backend resources such as cache, DB of the master-slave cluster will be distributed in different computer room. Synchronize master-slave data between each other by applying itself (Redis, MYSQL) or the client (Memcached).

Platform Web Framework

Here is a brief introduction to the Web framework we use. As we mentioned earlier, after the request arrives at the application container, it is first processed by the Unified web framework. The entire processing link for the user request in the application container is as follows.

The process of web framework is primarily to convert requests in HTTP form to requests understood by the application runtime environment (such as the JVM), including interface routing, parameter processing, parameter validation, and so on. The platform currently uses Credus as a unified web framework, which is a self-research framework based on Jersey transformation.

Jersey is a jax-rs (JSR311) Open source reference implementation for building RESTful WEB service. Features are rich, including

• Interface Routing

• Feature-rich filter

• HTTP parameter Check

• Document Generation

In addition, Jersey provides some additional APIs and extension mechanisms, so developers can scale jersey as they want.

On the extension mechanism provided by Jersey, we have developed the Credus, the main functions are

• Encapsulated jersey Frame

• customizing content

• Wiki templates

On the filter mechanism provided by the jersey, the Credus framework customizes a series of common interface policies and functions. Includes user authentication, interface frequency limit, interface information statistics and return interface JSONP encapsulation. In addition, the original parameter check of Jersey is further expanded, and more parameter checking methods are added. The process of Web requests in the Credus framework is as follows

Summarize

This time we shared the knowledge of the microblogging platform, including platform features, platform service deployment, and Platform Web framework introduction. Hope that through this sharing, we can understand the micro-Bo platform, at the same time on the service structure has a holistic understanding of the future work has helped.

Due to space and theme limitations, there is a lot of content is not specifically launched, interested to pay attention to the @ Platform technology Salon.

------------------Boot Camp Introduction------------------

Micro-blogging Platform Boot camp activities are organized within the microblog platform for new recruits to the team into the training course, the goal is the integration of the team, including the integration of people, atmosphere, technology integration. At present, there are 4 activities, many students quickly grow into the backbone of the platform technology.

Weibo platform is very focused on team members into the growth of the team, where someone to help you into, someone with you to grow up, but also welcome the small partners to join the microblogging platform, welcome private messages consultation.

------------------Instructor Profile------------------

Su Shingyu, @zhuidawugui, Weibo platform and big Data department-communications Systems Research and Development engineer, January 2014 graduated from China University of Science and Technology, school recruit into the micro Bo. Successively responsible for powder clothing platform, group chat, micro-BO camera message box and other business back-end design and development work. Focus on distributed system design and service assurance, storm streaming. Recruit Training Camp Phase III trainees

[Reproduced] Boot Camp series courses-platform services deployment and web framework