Recommended system Diversity

From: Joseph A. Konstan & John Riedl

The recommendation system also has two other features that have a significant impact on the recommendations you end up seeing: first, before figuring out how similar you are to other shoppers, the recommendation system must first understand what you really like; second, the referral system runs in accordance with a set of business rules to ensure that the recommendations are not only useful, Also make businesses profitable.

How does the recommendation algorithm win your trust and allow the merchant to earn money?

First, collect your data on the Internet

For example, look at Amazon's art shop, where last time we went to see 900多万册 prints and posters were on sale. Amazon's Art shop has several ways to evaluate your preferences. It will give you a rating of 1 to 5 stars on a certain piece of art, it will also record which of your pictures you click to enlarge the view, which paintings you have seen repeatedly, which you put into the wish list, as well as you end up actually buy what. It also tracks which paintings are displayed on every page you visit. Online retailers will use the path you have on their website (the page you've visited and the link to the product) to refer you to the associated product. In addition, it combines your purchase history and scoring information to create a profile of your long-term buying preferences.

Companies like Amazon collect a lot of such data about customers. During your login, almost every action you make on its website will be recorded and reserved for future use. Thanks to browser cookies, even the online record merchant of an anonymous shopper can be maintained, and eventually the data will be linked to the customer's profile when an anonymous shopper creates an account or logs in. This explosive data collection is not unique to online merchants, and Wal-Mart is known for its deep mining of cash receipts. But online stores are in a more advantageous position to view and record, not just what consumers buy, but also what you have considered, browsed, and decided not to buy. In most parts of the world, all such activities are monitored and recorded by any person, and only in Europe does the Data Privacy Act limit this to some extent.

Of course, no matter how the law, customers find their data is abused, will have a strong resistance. As early as September 2000, Amazon had a rough time: some of the customers found that they received a higher price because the site identified them as old customers, rather than anonymously entering or transferring customers from a parity site. Amazon claims it is just a random price test, and the link between the results and the identity of the old customer is purely coincidental. That said, it stopped the operation.

II. operating under commercial rules

The various business rules added to the recommendation algorithm are designed to prevent the algorithm from giving foolish recommendations and to help online retailers maximize turnover without losing your trust. At the very least, the recommendation system should avoid what people say about the supermarket paradox (supermarket Paradox). For example, almost everyone who goes to the supermarket likes bananas and often buys some. So shouldn't the recommendation system recommend bananas to every customer? The answer is no-it doesn't help the customer, nor does it boost the sales of bananas. Therefore, the smart supermarket recommendation system will always include a rule that explicitly excludes bananas from the recommended results.

This example may sound like nothing, but in one of our early projects, our referral system recommended the Beatles ' white Album ' to almost everyone who visited our website. Statistically speaking, this is a great recommendation: Customers have not previously bought the album from this e-commerce site, and most customers have high ratings for the White Album. Nonetheless, the recommendation is still ineffective--anyone interested in the White Album already has one.

Of course, most of the recommendation rules are more subtle. For example, when John was in the Netflix action film in September, there was no "Avengers" in the results, because the blockbuster didn't have a rental version at the time, and the recommendation would not make Netflix Avengers. As a result, John was directed to Iron Man 2 (Iron Man. 2), which is already available in streaming media.

Other rules include the prohibition of recommending goods that are sold at a loss to attract customers (loss leader), which in turn encourages the recommendation of unsalable goods. During the run of net perceptions, we worked with a client who used the referral system to identify potential customers for the inventory backlog and achieved considerable success.

Third, to win your trust

However, such things will soon become tricky. A recommended algorithm that only sells high-margin products will not win the trust of customers. It's like going to a restaurant where the waiter is trying to recommend a fish to you. Is this fish really what he thinks is the most delicious? Or is the chef urging the people below to sell it before the fish goes bad?

In order to build trust, the more complex recommendation algorithm will try to maintain a certain degree of transparency, so that customers have a general idea of why the system would recommend this product to themselves, and can change their profile if they do not like the recommended results they receive. For example, you can delete the shopping records you bought on Amazon, after all, those things don't reflect your personal preferences. You can also know why the system is recommending certain products to you. When Amazon chose Jonathan Franzen's novel "Freedom" for John, John clicked the link on the label "Why did you recommend it to me?" ”。 A brief description of the book that he placed on the wishlist triggered the recommendation. " However, since he had not read the books on his wish list, John did not take the recommendation of freedom. Explanations like this will let the user know if the recommended results are useful.

However, the improvement of personal data and interpretation of recommended results are often insufficient to ensure that the system is not wrong. Recently, Amazon bombed Joe with promotional emails from high-definition big-screen TVs (HDTV)-3 a week, one months in a row. In addition to sending too many emails to Joe, the retailer was unaware that Joe had bought a TV set with his wife's account. In addition, these emails did not provide a very obvious way for Joe to say "Thank you, but I am not interested". In the end, Joe canceled some of his mail subscriptions at Amazon; he didn't care about receiving all sorts of information, and he had more time to really watch his TV.

Iv. How big is the recommended algorithm?

Well, how does the recommendation algorithm really work? They have, of course, been increasing online sales; Jack Aaronson, an analyst at the Aaronson Group, estimates that investment in the recommended algorithm will yield 10% to 30% of revenue as the recommended algorithm drives sales growth. And they're just getting started. Now, for those of us who study recommender systems, the biggest challenge is figuring out how to judge new methods and algorithms best. This is not as simple as benchmarking a microprocessor, because different recommender systems have very different goals.

The simplest way to evaluate an algorithm is to see how much difference there is between its predictions and the user's actual evaluation. For example, if John gave a star to the romantic novel "Twilight" (Twilight), Amazon might notice that the algorithm, based on other similar users, had predicted that John would give two stars, and that there was a star bias. However, sellers are more concerned about the error of the algorithm in the user-rated product, because the more popular items are more likely to be purchased by the customer; John will not buy twilight anyway. So it's not helpful to think about how much it does to understand the recommendation algorithm.

Another common method is to look at the recommended results given by the algorithm and what the customer actually buys, and how high the matching degree is. However, this method can also be misleading, because such analysis will be the user's own efforts to find the product mistakenly counted on the head of the recommendation algorithm, and the user find something is the most should not be recommended! Given the shortcomings of these methods, researchers have been looking at new indicators, not just precision, but also other attributes like discovering unexpected surprises and diversity.

Unexpected surprises (serendipity) are weighted with unusual recommendations, especially those that are valuable to a particular user, but are not useful for other users of the same category. The algorithm that adjusts to surprise surprises will notice that the White Album appears to be a good recommendation for almost everyone, and will instead look for a less common choice-perhaps Joan Armatrading's love and emotion. This less-popular recommendation is unlikely to hit the target, but once it encounters, it will give the user a much bigger surprise.

The diversity of recommended results can also be a good indication of the problem. For example, a user who loves to watch Dick Francis's mystery fiction can still be disappointed when he sees all of Dick Francis's work in the referral form. A truly diverse recommendation form will include different authors and different types of books, as well as movies, games and other products.

The recommendation system research needs to break through a variety of obstacles, much more than the existing system to fine-tune. What researchers are considering right now is how the recommendation algorithm should help users discover parts of a site's content collection that they never knew about. For example, send the person who buys the book to the Amazon clothing department, rather than give some safe, customer more likely to accept the recommendation result. Outside the retail world, the recommended algorithms can help people reach new ideas, and even if we disagree with some of them, the overall effect will probably be positive, as it will help reduce the Balkanization of the society (i.e., fragmentation). It is still to be seen whether the recommended algorithm can do this, or if it is boring or distrustful.

But one thing is clear: the referral system will only get better, collect more and more data about you, and show it in other, unexpected places. If you like this article, Amazon will be happy to recommend all the other books you might like about the referral system.

Recommended system Diversity