The application of deep learning in short text similarity (sentence2vector)--qjzcy Blog _ Deep Learning

We often encounter how to find the similarity of two sentences in our work, such as how to judge the similarity between search query and ad query, search for similarities of query and app, and so on. There is no good way, here is a personal summary of it.

Directory:
First, put the results
Two, short text similar commonly used method
Iii. application of the theme model
Four, the depth study model constructs

(a) The old appearance of the first paste results bar, the sample is processed after the search query and ads Click Query, accurate rate of about 95%

The format of Figure 2 is labeled (similar to 1, not similar to 0), a prediction label, a value (greater than 0 is the forecast label 1), search query @ AD Query

(b) Our common approach: Semantic related app search (ii) on the similar--qjzcy blog
1. Session Related method
2, sentence vector method: A vector space model for a sentence (such as using TFIDF as the weight), distance formula (such as cosine) to find the distance
3, the multi-level jump method
4, algorithm model method: such as the theme model and the Word2vector model of comparative fire

(iii) application of the theme model
To the theme model here, we actually have a problem with the topic model how to use. Take the Word2vector model, it provides a vector of each word, using this vector can be very good to calculate the similarity between two words, but there are several words in a sentence how, how to calculate it.
Several methods:
1, the vector of each word superimposed, this method is a bit rough, but simple.
2, the first method is feasible, it is easy to think if we can find the key words in the sentence, to give it a weighted effect is not better, but the key word how to come, but also a problem (TFIDF is obviously not optimal, Amway one I do the key word method: http://blog.csdn.net/ qjzcy/article/details/51737059).
3, the vector directly strung up into a long vector, this method did not experiment, but think about a lot of problems, or lazy to say such a simple method has not heard that people use the affirmation effect is not ^_^.
4, if the sentence itself can form a 1-dimensional space vector, the subject of each word has a vector of dimensions. It's natural to think that we can use convolution to solve this problem.
(iv) The model of deep learning to build
Question: Since we want to use a deep learning model, then how do we let the model identify our initial data.
We can do this:
1, each sentence is convolution into a vector, using this vector to find the distance
Like the Microsoft model.

2, the sentence also as a feature added to the phrase to train together
For example, this paper follows the Word2vector idea:
Distributed representations of sentences and Documents

3, I do: put two sentences into a sentence, separated by a logo, so as to form a 2D structure of data as input, with CNN Training
I prefer this approach because the data structure is simpler and allows us to focus our attention on other structural aspects of the model.
The problem is solved, we put it in the model run, Duang. Almost did not do what model optimization can have 93% or so accurate rate, sure enough to deep study or have some superstition.

Another kind of lstm attempt:
Using LSTM model to realize the blog of similar--qjzcy in this topic