Golang Realization Hamming Distance Demo

This is a creation in Article, where the information may have evolved or changed.

Simhash's algorithm is simply, from the massive text fast search and known Simhash difference is less than K bit Simhash collection, here each text can be represented by a Simhash value, a simhash has 64bit, similar text, 64bit is similar, The empirical value of K in this paper is 3. The disadvantage of the method is as obvious as the advantages, there are two points, for short text, K value is very sensitive; the other is because the algorithm is space-changing time, the system memory is too much.

Demo

Package Mainimport ("FMT" "Math" "StrConv" "strings") type Simhash struct {intsimhash Int64 hashbitsint}func Main () {str:= "is the brightest star in the night sky to remember the loneliness and sigh of the one who looks up in the sky the brightest star in the night is to remember the loneliness and sigh in the sky the brightest star in the night is to remember the loneliness and sigh of the one who is looking at the brightest star in the night. The loneliness and sigh in the sky the brightest star in the night is to remember the loneliness and sigh of the one who looks up in the sky the brightest star in the night is to remember the loneliness and sigh in the heart of those who look up.str2:= "is the brightest star in the night sky to remember the loneliness and sigh of the one who looks up in the sky the brightest star in the night is to remember the loneliness and sigh in the sky the brightest star in the night is to remember the loneliness and sigh of the one who is looking at the brightest star in the night. The loneliness and sigh in the sky the brightest star in the night is to remember the loneliness and sigh of the one who looks up in the sky the brightest star in the night is to remember the loneliness and sigh of the one who looks up to the heart "//str3:=" Nai Nai ge Xiong Cao "S:=params ()//STR hash value hash:=S.simhash (str) FMT. Println (hash)//str2 distance HASH2:=S.simhash (str2) fmt. Println (HASH2)//To calculate the similarity of SM:=s.similarity (hash, HASH2) fmt. PRINTLN (SM)//Distance TS:=s.hammingdistance (hash, HASH2) fmt. PRINTLN (TS)}/**Distance compensation*/Func (S*simhash) Hammingdistance (hash, other int64)int{x:= (hash ^ other) & ((1 << uint64 (s.hashbits))-1) Tot:= 0For x! = 0{tot+ = 1x&= x-1} return tot}/**Similarity degree*/Func (S*Simhash) Similarity (hash, other Int64) float64 {a:=float64 (hash) B:=float64 (Other)ifA >B {return B/A} return a/B}/**Hamming distance Hash*/Func (S*Simhash) Simhash (str string) Int64 {m:= Strings. Split (str, "") Token_int:= Make ([]int, s.hashbits) for I:= 0; I < Len (m); i++{temp:=M[i] t:=S.hash (temp) for J:= 0; J < S.hashbits; J + +{bitmask:= Int64 (1 <<UINT (j))ifT&bitmask! = 0{Token_int[j]+ = 1} else {Token_int[j]-= 1}}} var fingerprint int64= 0For I:= 0; i < s.hashbits; i++ {        ifToken_int[i] >= 0{fingerprint+ = 1 <<UInt64 (i)}} Return fingerprint}/**Initialize*/func params (s)*Simhash) {s= &simhash{} s.hashbits= 64return S}/**Hash Value*/Func (S*Simhash) Hash (token string) Int64 {iftoken = = ""{return0} else {x:= Int64 (int(Token[0]) << 7) m:= Int64 (1000003) Mask:= Math. Pow (2, float64 (s.hashbits-1)) S:= StrConv. Formatfloat (Mask, ' f ',-1, 64) Tsk, _:= StrConv. parseint (S, 10, 64) for I:= 0; I < Len (token); i++{tokens:= Int64 (int(token[0])) x= ((x * m) ^ tokens) &tsk} x^=Int64 (len (token))ifx = =-1{x=-2} return Int64 (x)}}