Li Yu 17th "word vector-glove and fasttext"

Glove uses the co-occurrence (co-occurrence) information between the words and the words, assuming that the element Xij is the number of times the word J appears in the Environment (context) of the word I. There are many possible definitions of the "environment" here. For example, in a text sequence, if the word J appears on the left side of the word I or on the right side of the distance of no more than 10 words, we think that the word J appears in the environment of the word I once. The probability that the word J appears in the environment of the word I is also called the co-occurrence probability of the word I and the word j, as shown below:

The following figure is calculated as follows:




Note that the last glove of the word vector is generated using the word vector of the center word and the background word as the word vector of the word, which is different from the Word2vec just using the center word as the word vector. At the same time follow-up to see the paper glove

Fasttext

Reference Address: http://zh.gluon.ai/chapter_natural-language-processing/glove-fasttext.html