Recommendation engine Note 1

Recommendation Engine

The receng uses special information filtering technologies to recommend different items or content to users who may be interested in them.

Figure 1. recommendation engine working principle

Figure 1 shows the working principle of the recommendation engine. Here, we first regard the recommendation engine as a black box. The input it receives is the recommendation data source. Generally, the data sources required by the recommendation engine include:

• Metadata of the item or content to be recommended, such as keywords and genetic descriptions;
• Basic information of system users, such as gender and age
• Users' preferences on items or information may include users' ratings on items, users' viewing records of items, and users' purchase records based on different applications. In fact, these user preferences can be divided into two types:

• Explicit user feedback: This type of feedback is explicitly provided, such as the user's score on the item or comment on the item, outside of the user's natural browsing or use of the website.
•   Implicit user feedback: This type of data is generated by users who use the website. It implicitly reflects users' preferences for items. For example, if a user buys an item, you can view the information of an item.

In-depth recommendation Mechanism

This chapter details the working principles, advantages and disadvantages, and application scenarios of each recommendation mechanism.

Demographic-based recommendation

Demographic-based recommendation is the most easy-to-Implement recommendation method. It simply discovers user Relevance Based on the basic information of system users, then, we recommend other items that are favored by similar users to the current user. Figure 2 shows how the recommendation works.

Figure 2. Working Principle of demographic-based recommendation Mechanism

We can clearly see that, first, the system creates a profile for each user, including the user's basic information, such as the user's age and gender. Then, the system calculates the user similarity based on the user profile. We can see that the profile of user a is the same as that of user C. Then, the system considers that user a and user C are similar users. In the recommendation engine, they can be called "neighbors". Finally, some items are recommended to the current user based on the preferences of the "neighbors" user group. In the figure, item A liked by user a is recommended to user C.

The benefits of this demographic-based recommendation mechanism are:

1. Because the current user's preferences for items are not used, there is no "Cold Start" problem for new users.
2. This method does not depend on the item data, so it can be used in different item fields. It is a domain-independent ).

So what are the shortcomings and problems of this method? This method of classifying users based on the basic information of users is too rough, especially for fields with high taste requirements, such as books, movies, and music, which cannot obtain good recommendation results. This method may provide some simple recommendations on some e-commerce websites. Another limitation is that this method may involve sensitive information that is irrelevant to the information discovery problem, such as the user's age. The user information is not well obtained.

Content-based recommendation

Content-based recommendation is the most widely used recommendation mechanism at the beginning of its appearance. Its core idea is to discover the relevance of items or content based on the metadata of recommended items or content, then, similar items are recommended to users based on users' preferences. Figure 3 shows the basic principles of content-based recommendation.

Figure 3. Basic principles of content-based recommendation

Figure 3 shows a typical example of content-based recommendation. For a movie recommendation system, we need to model the metadata of the movie. Here we only briefly describe the type of the movie; then, the similarity between movies is discovered through the metadata of the movie, because the types are "Love, romantic" movies A and C are considered to be similar movies (of course, only the type is not enough, for better recommendations, we can also consider movie directors, actors, and so on.) Finally, we recommend that user a like movie, then the system can recommend a similar movie C to him.

The benefit of this content-based recommendation mechanism is that it can well model user tastes and provide more accurate recommendations. However, it also has the following problems:

1. The product needs to be analyzed and modeled. The recommendation quality depends on the completeness and comprehensiveness of the product model. In our current application, we can see that keywords and tags are considered a simple and effective method to describe item metadata.
2. The Analysis of item similarity only depends on the characteristics of the item. The attitude of the person to the item is not considered here.
3. Because we need to make recommendations based on the user's past preferences, there is a "Cold Start" problem for new users.

Although this method has many shortcomings and problems, it is still successfully applied to some social websites of movies, music, and books. Some websites also require professional personnel to perform genetic code on the items, for example, Pandora said in a report that in Pandora's recommendation engine, each song has more than 100 metadata features, including the style, year, and singer of the song.

 

Collaborative Filtering-based recommendation

With the development of Web, Web sites advocate user participation and user contribution. Therefore, the collaborative filtering-based recommendation mechanism is born. The principle is very simple, that is, discovering the relevance of the item or content, or discovering the relevance of the user based on the user's preference for the item or information, and then making recommendations based on the relevance. Collaborative Filtering-based recommendation can be divided into three sub-categories: user-based recommendation and project-based recommendation) and model-based recommendation ). Next we will introduce three collaborative filtering recommendation mechanisms in detail.

User-based collaborative filtering and recommendation

The basic principle of user-based collaborative filtering recommendation is to find the "Neighbor" user group similar to the current user's taste and preferences based on the preferences of all users on items or information, in general, the algorithm for calculating "K-Neighbor" is used. Then, based on the historical preference information of the K-neighbor, the algorithm is recommended for the current user. 4. The schematic diagram is provided.

Figure 4. Basic principles of user-based collaborative filtering and recommendation Mechanism

The following figure shows the basic principles of the user-based collaborative filtering and recommendation mechanism. Assume that user a prefers item A, item C, and user B, and user C prefers item, item C and item D. from the user's historical preferences, we can find that user a and user C have similar tastes and preferences, and user C also like item D, so we can infer that user a may also like item d, so we can recommend item d to user.

User-based collaborative filtering recommendation mechanism and demographic-based recommendation mechanism both calculate user similarity and calculate recommendations based on the "Neighbor" user group, however, what they differ from is how to calculate user similarity. The demographic-based mechanism only takes into account user characteristics, however, the user-based collaborative filtering mechanism computes user similarity on the user's historical preference data. Its basic assumption is, users who like similar items may have the same or similar tastes and preferences.

Project-based collaborative filtering and recommendation

The basic principle of project-based collaborative filtering and recommendation is similar. It only means that it uses all users' preferences on items or information to discover similarity between items, then, similar items are recommended to the user based on the user's historical preferences. Figure 5 shows the basic principles of the item.

Assume that user a prefers item A and item C, user B prefers item A, item B, and item C, and user C prefers item, from the historical preferences of these users, we can analyze the similarities between item A and item C. People who like item A like item C, based on this data, we can infer that user C may also like item C, so the system will recommend item C to user C.

Similar to the above, project-based collaborative filtering recommendation and content-based recommendation are both based on item similarity prediction recommendation, but the similarity calculation method is different, the former is based on the preference of the user's history, and the latter is based on the attribute feature information of the item itself.

Figure 5. Basic principles of project-based collaborative filtering and recommendation

At the same time, how should we choose collaborative filtering based on user and project? In fact, the project-based collaborative filtering and recommendation mechanism is an improved method of Amazon's user-based mechanism, because in most web sites, the number of items is far smaller than the number of users, and the number and similarity of items are relatively stable. At the same time, the project-based mechanism is better than the user-based Real-time performance. However, this is not the case in all scenarios. In some news recommendation systems, the number of items, that is, news, may be larger than the number of users, in addition, news are updated quickly, so its shape is still unstable. Therefore, we can see that the selection of Recommendation policies is closely related to specific application scenarios.

 

Model-based collaborative filtering recommendation

Model-based collaborative filtering recommendation is to train a recommendation model based on sample user preferences, and then predict and calculate recommendations based on real-time user preferences.

The recommendation mechanism based on collaborative filtering is the most widely used recommendation mechanism today. It has the following significant advantages:

1. It does not need to strictly model items or users, and does not require that item descriptions be understandable by machines. Therefore, this method is also irrelevant to the field.
2. The recommendation calculated in this way is open and can share the experience of others, so that users can find potential interests and preferences.

It also has the following problems:

1. The core of the method is based on historical data, so there is a "Cold Start" problem for new items and new users.
2. The recommendation results depend on the quantity and accuracy of user historical preference data.
3. In most implementations, the user's historical preferences are stored using a sparse matrix, while the computing on the sparse matrix has some obvious problems, including the possibility that a small number of people may have a great impact on the accuracy of recommendations.
4. Users with special tastes cannot give good recommendations.
5. Based on historical data, it is difficult to modify and model users' preferences after capturing and modeling the user's preferences or according to the user's usage evolution, resulting in the inflexible method.

Hybrid recommendation Mechanism

The recommendation on the current web site often does not simply adopt a recommendation mechanism and strategy. They often combine multiple methods, to achieve better recommendation results. Here are several popular combination methods for how to combine various recommendation mechanisms.

1. Weighted hybridization: uses a linear formula (linear formula) to combine several different recommendations based on a certain weight. The specific weight value needs to be tested repeatedly on the test dataset, to achieve the best recommendation results.
2. Switching Hybridization ), recommendation policies may vary greatly, so the Mixed Mode of switching is to allow the most suitable recommendation mechanism to calculate recommendations under different circumstances.
3. Mixed hybridization: uses multiple recommendation mechanisms and displays different recommendation results in different regions. In fact, Amazon, Dangdang, and many other e-commerce websites use this method. Users can get comprehensive recommendations and find what they want.
4. Meta-level hybridization: uses multiple recommendation mechanisms and uses the results of one recommendation mechanism as another input to integrate the advantages and disadvantages of each recommendation mechanism, get more accurate recommendations.
   Source: http://www.ibm.com/developerworks/cn/web/1103_zhaoct_recommstudy1/index.html>