Sky-cup Big Data algorithm application contest sentiment

The competition used to be a long time ago, wanted to write feelings and experiences, but has been dragged not written. Today finally has the time, writes.

The title of the contest is recommended by the video site, which predicts the user's access to 10 video sites for the eighth week based on the number of user visits to 10 video sites per day for the first seven weeks and other online behaviors.

I see this is a time series prediction problem, yes, give the first 49 days user access data, let you predict the next seven days user access to data. This should really be a time series prediction problem.

But I use the time Prediction algorithm model Holt-winter (3 times exponential smoothing), the result of Arima run is not very good.

I later analyzed the data and counted the number of users who clicked on 10 sites in seven days of each week for seven weeks. I was shocked to find that it was evenly distributed.

Originally I thought that for some websites, the number of weekend visits will be higher, or 7 weeks 49 days, the number of visits will have a certain regularity. But there was no discovery.

The only discovery is that there will be a mutation in the fifth week, and the 567-week number of visits is drastically halved, as is true for all sites.

And for individuals, the site's clicks are more akin to random distributions, and we sampled 50 users.

There is no regularity in the week and there is no regularity between weeks. And although 7 weeks looks very long, for one day, for example, Monday, there are only 7 valid data, so there is less effective data.

Therefore, it is not suitable for the time series prediction algorithm to grasp the periodic characteristics.

But the recommendation, there is always a way. I think of collaborative filtering. I divided the problem into two steps, the first step, to predict the daily user access or not, this is a 01 classification problem. The second step is to predict the number of user visits, which can be solved by collaborative filtering.

For the first-step classification problem, selecting features is the most important. I selected: seven weeks prior to each day of visit or not, whether the last two weeks of continuous access, whether the number of visits to the site exceeded the threshold, the number of days per week and whether or not to exceed a threshold value as a feature

The previous six weeks as a data set, the seventh week as a test set, and then run the results with a neural network as a preliminary result.

Collaborative filtering we use a user-specific collaborative filtering algorithm.

Cond

Sky-cup Big Data algorithm application contest sentiment