from 0 to 1, a shop. Recommended system for general recommendation platform

Author: Chen Yu, No. 1th store recommended team Architecture leader, 2010 Master's degree, has been working in Alibaba, No. 1th store. In the past 6 years of work has been engaged in system architecture development work, the current focus on micro-services, Lucene index, hbase and other technologies.
Zebian: Chang (qianshg@csdn.net)
This article for the "programmer" original article, not allowed to reprint, more wonderful articles please subscribe to the 2016 "programmer"

A store of precision recommendation department through continuous exploration, and gradually build a real-time, highly available, recommended process traceability of the general recommendation Platform, currently the platform is being used more and more people in the company. This paper starts from the background of the general recommendation platform of the first shop, discusses the overall architecture design of the platform, the design of the visualization system of the recommended process, the design of the visualization system of the recommendation results, and concludes the conclusion. Objective

Personalized recommendation system is currently the most popular technical field, it is the use of E-commerce site to provide customers with merchandise information and advice to help users decide what products should be purchased, simulation sales staff to help customers complete the purchase process. A good personalized recommendation system not only can provide users with personalized service, but also can respond to the user's personalized needs in real time, and can provide the recommended commodity predictability, interpretative. The historical background of the general recommendation platform

In the traditional Electronic Business website, to the user recommended products of colorful fields. For example, the first page of the PC in a shop will recommend the products that users may be interested in based on the user's purchase record, which we call guessing you like the field, as shown in Figure 1.

Figure 11th Shop PC Side Home User guess you like the field

At the PC end of the one shop, the product Details page will recommend products similar to the product that the user is currently browsing, which we call the purchase and the field, as shown in Figure 2.

Figure 21st Shop PC side bought and bought a field

With the rapid expansion of the electric business, the more and more recommended fields, the business logic becomes more and more complex, the urgent need for logical change is more and more, in this scenario, developers will be tired of coping with the complexity of the recommended requirements and maintenance of complex business logic, code redundancy is very large. I use Figure 3 to express this situation.

Fig. 3 The problem of recommending field before the general recommendation platform is built

In response to this background, a store precision department developed an online, visual configuration field and modify the field logic of the recommended visualization platform for real-time responses to the recommended foreground field requirements.

And in order for the recommendation to be explanatory, we will recommend that the field of logical calculation log through the self-developed data backflow framework, in the form of HDFs precipitation, and through the data cleaning steps to write to the HBase, through the Web interface to the site operators to explain how the line of the field is to recommend the product, This platform is referred to as the recommended traceability platform.

The recommended process visualization system provides the function of what kind of logic to recommend the product to the user, the recommended traceability system can reproduce the product recommendation process, these two platforms work together, it is recommended that the process visualization system provides the user with what logic to recommend the product function, The recommended traceability system can reproduce the recommended process for the product, and these two platforms work together to solve the recommended field problem shown in Figure 3. We combined these two systems together as a common recommendation platform. General recommendation platform Overall design

The overall architectural design of the general recommendation platform is shown in Figure 4.

Fig. 4 Overall architecture design of general recommendation platform

As can be seen from Figure 4, the common recommendation platform consists of two systems, the recommended process visualization system, and the recommended results traceability system. In the recommended process visualization system, site operation in advance to recommend the field configuration, and then the user access to the completed configuration of the field, the request process visualization system in the unified recommendation interface, in the interface logic process, will recommend steps through the data reflux system into the hbase, The recommended results can be traced back to the system; In the recommended results traceability system, through the Analog online field request, according to the request ID and the product ID to backtrack the recommendation process to determine whether the recommendation process is justified. Detailed design of visualization system for recommended process

The detailed design of the recommended process visualization system is shown in Figure 5.

Fig. 5 Detailed design diagram of the recommended process visualization system

From Figure 5 You can see that the recommended process visualization system is divided into the front and back two parts, the front section provides online, real-time recommended process configuration functions, the background part of the external exposure of the unified recommendation interface, the interface will be accessed according to the recommended field ID, access to the field of recommended process configuration, Then according to the process configuration to calculate the corresponding recommended products.

In the calculation process, the independent research and development of the data reflux system will recommend steps through Kafka, Camus sent to the hive, and then through the steps of data cleaning into the hbase. The flow chart of the data reflux system is shown in Figure 6.

Fig. 6 Flow chart of data reflow frame

Figure 6 describes the recommended step message content through the centralized compression push, Kafka transmission, precipitation in the hive, data cleaning and other processes, and eventually formed a trail like the recommended steps of the flow log. Recommended results Traceability system detailed design

The recommended process can be traced back to the detailed design of the system as shown in Figure 7.

Fig. 7 The recommended results can be traced to the detailed design diagram of the system

From Figure 7, you can see that the site operators through the Simulation online field request, thus calling the recommended process visualization system of the unified recommendation interface to get the real recommended products. At the same time, according to the recommended process visualization system in the precipitation of recommended steps in the flow log, with the commodity ID and request ID to do combinatorial key query, so as to reproduce the recommended steps. This makes our personalized recommendation system of the recommendation process transparent, the recommended results are explanatory, but also to facilitate the first time to solve the user complaints of the recommendations of inaccurate results. Summary of General recommendation platform

After the general recommendation platform is completed, the precision Recommendation Department of a shop solves the common problems of the recommendation system (see Figure 3), such as the repetitive development workload, which greatly improves the development efficiency and operational efficiency of the recommended team. The positive impact of this platform on the recommended team, as shown in Figure 8.

Fig. 8 The status quo of recommendation field after the general recommendation platform is built

As you can see from Figure 8, the recommended process visualization system for the generic recommendation platform is visualized through a field configuration, the development of students to avoid duplication of development work, and can be real-time adjustment of the field configuration, to respond to the field logic adjustment needs; recommended results of the universal Precision recommendation platform traceability system by reducing the referral process, It provides the guarantee of the correctness of the recommended results.

Finally, a set of data operations on the general recommendation platform: a shop PC end + Mobile End has a total of more than 50 recommended fields, nearly 80% of the recommended field use the platform; the average development time of the recommended requirement is shortened from the previous week to the current 10 minutes, and the recommended results have increased the user satisfaction rate by nearly 50%.

Subscribe to the 2016 programmer (including iOS, Android, and print edition) please visit http://dingyue.programmer.com.cn

Subscribe to the Advisory:

• Online Consulting (QQ): 2251809102
• Telephone Consultation: 010-64351436
• More news, Welcome to "Programmer's editorial department"