Recommendation System (i)--Good recommendation system

ML direction of the preliminary plan to learn the recommendation system, thanks to the spirit of Go recommended Book-"Recommendation System Practice", book a hand, can't wait to read the first chapter, it is really interesting, it is worth investing in learning. The book is not intended to be written in the handbook-style masterpiece or the basic textbook, the content is relatively trivial and loose, of course, this should be due to the limitations of the domain characteristics, therefore, I intend to learn, take time to learn the contents of the summary, made to read notes, but also convenient for their later access to the supplement.
--Preface

First, the recommendation system introduction 1. Why use a referral system

The core is the problem of information overload, the solution from the Classification directory (Yahoo) to the search engine (Google), due to the following two advantages, but also to the recommendation system, of course, the recommendation system does not replace the search engine, the two can be complementary to use.
(1) The referral system can help users find new content that they are interested in when they don't have a definite purpose.
(2) Recommendation system to better explore the long tail of items

2. What is the referral system?

The basic task of the Recommender system is to contact the user and the item (collectively, the things that can be consumed by the user) to solve the problem of information overload. The essence of the recommendation algorithm is to connect the user to the item in a certain way, with roughly three different ways:
(1) Social referrals
(2) Content-based recommendations
(3) Recommendation based on collaborative filtering

Ii. Fields of Application

Almost all of the recommended system applications are made up of the following three parts:
-Display page of the front desk
-Log system in the background
-Recommended Algorithm System
Widely used in the field of personalized recommender system
(1) e-commerce--amazon
Personalized recommendation list, related recommendation list (packaged sales)
(2) Film and video website--netflix
(3) Personalized music network Radio--pandora watercress
Note that music recommendations have many special points
(4) Social network--facebook
Facebook has a referral API
(5) Personalized reading--googlereader Zite
(6) Location-based services
Location is a very important contextual information
(7) Personalized mail--tapestry
(8) Personalized advertising--facebook
Ad targeting is the core technology of many internet companies

Third, the evaluation Method 1. Recommended system Participants

(1) User
(2) Items
(3) Websites that provide referral systems

2. Recommended system test Method--evaluation recommendation effect

In general, a new recommendation algorithm is finally on-line, and the following tests need to be completed sequentially:
(1) The offline experiment proves that it is superior to the existing algorithm in many off-line indexes.
(2) User survey to determine their user satisfaction is not less than the existing algorithm
(3) The online AB test determines its superiority over the existing algorithm on the indicators we care about

2.1 Offline experiments

(1) Features: Through the log system to obtain user behavior data set can be
(2) Advantages: Do not need real user participation, as long as there is a data set extracted from the actual system log can be calculated quickly
(3) Shortcomings: Suitable for evaluation of "predictive accuracy", many commercial concerns of the indicators "user Satisfaction" "CTR" "conversion rate" and so on can not be obtained

2.2 User Survey

(1) Features: In the user survey, on the one hand to control costs, on the other hand, but also to ensure the statistical significance of the results
(2) Advantages: Can obtain a lot of user subjective feelings of the indicators, relatively low risk of online experiments, errors easy to compensate
(3) Shortcomings: The organization of large-scale testing is difficult, resulting in inadequate statistical significance of test results; Design double blind experiment difficult, test environment and real environment user behavior difference, resulting in test environment collected test indicators may be difficult to reproduce in the real environment

2.3 Online experiments

(1) Features: AB testing is a very common test method for on-line evaluation algorithms. Flow segmentation in AB testing is critical, and the teams that control these layers need to get the flow of their AB tests from a single place, and the flow between the different tiers should be orthogonal.
(2) Advantages: It is fair to obtain the performance index of different algorithms in actual online.
(3) Disadvantage: The period is long, must carry on the long-term experiment to obtain the reliable result

3. Recommended System Evaluation Indicator 3.1 user satisfaction

(1) Most important indicators
(2) Only user survey (primary) or online experiments (through user behavior statistics, such as CTR, user dwell time and conversion rate, etc.) to obtain

3.2 Forecast Accuracy

(1) The most important offline evaluation indicators
(2) Off-line test and questionnaire survey can be
(3) Rating evaluation system--RMSE or MAE


The RMSE standard is more stringent. If the scoring system is based on integers (that is, the user-given scores are integers), then rounding the predictions results in a decrease in the error of Mae.
(4) TOPN recommendation system--accuracy or recall rate


(5) Choose the rating evaluation or TOPN mainly look at the content of the recommended site.

3.3 Coverage

(1) Coverage describes a referral system for the long tail of the item excavation ability. Available in three different ways. It is the content provider's concern that the indicator can be used to describe the distribution of the number of items in the recommendation list. The ability of the recommendation system to explore the long tail
(2) The distribution of the number of items in the recommended list can be used to describe the ability of the recommendation system to explore the long tail, which can be measured by the information entropy and the Gini coefficient two indicators.


where P (i) is the prevalence of item I's prevalence ratio of all items (the calculation formula for popularity has not yet appeared); I_j is the article J in the Logistics table sorted by item prevalence p () from small to large
(3) Now the mainstream recommendation algorithm has Matthew Effect, the original intention of the recommendation system is to eliminate Matthew effect.
The simple way to evaluate whether the recommendation system has Matthew effect is to use the Gini coefficient, if G1 is the Gini coefficient of the item popularity calculated from the initial user behavior, G2 is the Gini coefficient of the item popularity calculated from the recommendation list, then when G2>G1, it is suggested that the proposed algorithm has Matthew effect.

3.4 Multi-sample sex

(1) Diversity describes the similarity between item 22 in the recommendation list, which is mainly obtained by questionnaires.
(2) The overall diversity of the recommendation system can be defined as the average of the diversity of all user recommendations, i.e.

, where the diversity of user U under the recommended list R (U) is

where S (i,j) defines the similarity between the item I and J.

3.5 Novelty

(1) Novel recommendation refers to the user to recommend those they have not heard of the items, mainly by the survey to obtain
(2) The simplest way to do this is to filter out the list of items that the user has previously acted on in the site.
(3) If the average popularity of items in the recommended results is lower, the recommended results can be of high novelty.

3.6 Surprising sex

The distinction between attention and novelty, there is no recognized definition of indicators, can only be obtained through a questionnaire survey

3.7 Trust

(1) can only be obtained through a questionnaire survey
(2) To improve the trust of the recommendation system mainly lies in the need to increase the transparency of the recommendation system, that is, to provide interpretation of the recommendation system, so that users understand its operating mechanism
Consider the user's social network information, use friend information to recommend, use friends to explain, etc.

3.8 Real-time sex

(1) The recommendation system needs to update the recommendation list in real-time to meet the user's new behavioral changes-by evaluating the rate of change in the recommended list
(2) The recommendation system needs to be able to recommend the new system to the user-the use of the user referral List of the proportion of items are newly added to the same day to evaluate

3.9 Robustness

(1) Robustness measures the ability of a referral system to combat cheating (man-made attacks)
(2) evaluation mainly using simulated attack
(3) You can improve the robustness of the system by:
Use high-cost user behavior when designing recommender systems, such as changing browsing behavior to purchase behavior
Perform attack testing before data is used to clean up the data

3.10 Business Goals

Related to the profit model of the website

3.11 Summary

(1) for off-line optimization, the accuracy of prediction is generally optimized under the constraints of given coverage, diversity and novelty.
(2) In order to fully understand the performance of an algorithm, it should be evaluated in several dimensions:
User dimension: Mainly includes the user's demographic information, activity and is not new users, etc.
Item Dimensions: Includes attribute information, popularity, average points, and new items added from time to time.
Time dimension: Including season, weekday or weekend, day or night

Recommendation System (i)--Good recommendation system