Notes for the Baidu technology salon

This technology salon was presented by Baidu's senior architect Chen tianjian and Douban's chief scientist Wang shouyu. The main topic of this salon is the recommendation system.

Chen tianjian's main topic is the stream computing architecture in the Baidu recommendation engine computing platform architecture. Some text messages are missing in the middle. When you wait for the video to come out and listen again, this note is basically not sorted out. It is mainly a backup file. If you are interested, you can directly go to infoq to watch the video. Copy the notes below:

NLP-current analysis hotspot;

The traditional architecture focuses on hadoop and streaming computing accelerates data processing;

Queueworker;

Stream computing system, topology S4, dag;

Diverse Indexes

Timely computing to multiply user access.

Engine-This part needs to be replayed

Many things of the recommendation system need to be verified and improved.

Baidu's recommendation engine computing platform may publish services

The following is part of Wang Shouyi, Chief Scientist of Douban. He mainly focuses on the choice of algorithms.

Algorithm complexity Selection

Incremental update

Algorithms change based on user groups, products, and computing frameworks

The early user group is different from the public user group

Douban's recommendation has an item saturation period-this is just what I thought of at the scene, not the content of the speech.

Missing Value data also plays a role

Matrix decomposition and generation model

Text Analysis: generate model, hidden horse model, Gaussian mixture model, Bayesian model, Lda, RBM.

Entry growth tends to be stable

Improving long-term indicators depends on people

From traditional media information economy to modern app Experience Economy

Information is gradually private and closed, either a platform or a part of the platform.