Word co-occurrence implementation of Hadoop

Word Co-occurrence has not known how to translate correctly, word similarity or symbiotic word or co-occurrence matrix of words?

This is a commonly used text processing algorithm in statistics, which is used to measure all the phrases with the closest frequency of occurrence in a set of documents.He is actually a context phrase, not a word.It can be a more commonly used algorithm that can derive other statistics Algorithm can be used as a recommendation because it can provide the result that "people look at this and look at that." For example, do some collaborative filtering other than shopping recommendations, credit risk analysis, or calculate everyone What do you like?

For example, I love you, appear "I love" at the same time often accompanied by "love you" appeared, but the Chinese deal with English is not the same, you need to use the word segmentation library to do pretreatment.

Split the code as Mapper, Reducer and Driver do

Mapper program:

Reducer program:

Driver program does not explain, the world Driver are the same:

In fact, the core of the algorithm is the word and the last word out at the same time as a key plus a word value word count, the frequency of symbiosis of statistical words to the text clustering.View k-means a lot of online, in fact, many of the time algorithm Depending on the needs of the k-means or fuzzy k-value is not necessarily tall, wordcount is not necessarily poor.