Unveil the mystery of the Recommendation System

Opening

I recommend several articles on recommendation first. I personally think it is of practical significance for getting started. It was written by IBM engineers as follows:

Explore the secrets inside the receng, Part 1: Initial Exploration of receng

Explore the secrets inside the receng, Part 1: go deep into receng-Collaborative Filtering

Explore the secrets inside receng, Part 1: go deep into receng-related algorithms-Clustering

What is a recommendation system?

Recommendation is to push the products you may like to you. Building a recommendation system is the process of building how to push commodities to you.

I often hear people say that recommendation is an algorithm. Before getting in touch with the recommendation system, I thought that recommendation is an algorithm. When it comes to algorithms, I may think that they are very advanced and very considerate, A sense of worship immediately becomes mysterious.

When I was fully connected to the Recommendation System and reached an entry-level, I could independently build a recommendation system for a PV e-commerce website, and my previous cognition also changed, my current point is:

Recommendation is an overall computing process. In encoding, the workload of algorithms may be less than 1%;

It is relatively easy to build a recommendation system with a PV level of tens of millions. The daily log is only several hundred MB, and the data in the computing process can be stored in the memory of a single machine. When the PV Reaches hundreds of millions of billions, it requires a slightly more complex distributed computing;

There are many recommended calculation methods. It is hard to predict the effects of selection. It makes sense to do more results analysis either horizontally or vertically.

Recommended computing process

Computing Data Source

Web access logs, purchase, and favorites are actually user behavior data;

This is the basic data for analysis;

Commodities, which are the basic data for analysis;

Plan log storage format

How to mark the same Unlogged user; how to identify the Unlogged user and the login user is a person.

This is very important. This is the basis for log analysis and computing in the future.

Computing process step 1

Analyzes the relationship between users and products based on user behavior data. Users <--> browsing, users <--> purchasing, users <--> adding to favorites, and so on.

Step 2 of the computing process

Analyze the commonly used recommendation results based on the data calculated in the first step. For example, you can calculate the "shopping and viewing" based on the browsing data and calculate the "shopping and buying" based on the purchase data.

Algorithm (or rule) of the computing process)

Algorithms are generalized and mathematical formulas. Rules are niche. business rules defined by the company and complex scenarios are used to calculate the final recommendation results in the second step of the computing process, most of them use custom business rules.

For example, how can we recommend other products based on one product:

According to the basic meaning of this recommendation type, a product ---> many people who have read this product and read it ---> many products. This is the recommendation result, but there are many results for this recommendation. How can we recommend it?

You can recommend the final number of purchases. The latest ones are recommended. The view groups of the two products are the most similar ......

The recommendation result interface provides

This is nothing, and it is common.

Core of the Recommendation System

Business-based, recommendation effectiveness evaluation system;

Technology-based Distributed Computing for large data volumes

Code Description

Pre-Project:There are a lot of related projects, such as websites, products, and orders.

Latest source code:Git clone [email protected]: pumadong/cl-recommend.git.

Unveil the mystery of the Recommendation System