Problem Description:
Need to tune out a third-party interface, the interface returned by the encoding format is GBK, but Golang built only recognize UTF8, so you need to convert the GBK string to UTF8, in order to perform subsequent data processing.
Problem handling:
Introduction to open source package go get Github.com/axgle/mahonia
The open source protocol for this package is BSD, so you can safely reference and use
Mahonia Source Interpretation:
Convertstring converts a string from d ' s encoding to UTF-8.
Func (d Decoder) convertstring (s string) string {
bytes: = []byte (s)
runes: = Make ([]rune, Len (s))
Destpos: = 0
for Len (bytes) > 0 {
c, size, Status: = d (bytes)
if status = = state_only {
bytes = bytes[size:]
C Ontinue
}
If status = = No_room {
c = 0xfffd
size = Len (bytes)
status = Invalid_char
}
bytes = bytes[size:]
Runes[destpos] = c
destpos++
}
return string (runes[:d Estpos])
}
thought to call the above function to complete the conversion, and found that the Chinese part of the returned string is still garbled. After checking the data online, it is found that a translate function needs to be called.
Translate enables a Decoder to implement Go-charset ' s Translator interface.
Func (d Decoder) Translate (data []byte, EOF bool) (n int, CDATA []byte, err error) {CDATA = make ([]byte, Len (data) +1) Destpos: = 0 for n < len (data) {rune, size, Status: = D (data[n:]) switch Status {C ASE State_only:n + = size Continue case no_room:if!eof {RE
Turn N, cdata[:d Estpos], nil} rune = 0xfffd n = len (data) Default: n + = size} If Rune <, {if Destpos >= len (CDATA) {CDATA = Doubl Elength (CDATA)} Cdata[destpos] = Byte (rune) destpos++} else {if Destpos+utf8. Runelen (rune) > len (CDATA) {CDATA = Doublelength (CDATA)} Destpos + = UTF8. Encoderune (Cdata[destpos:], Rune)}} reTurn N, cdata[:d Estpos], nil}
The internal encoding conversion code is not analyzed, looked at the next not how to understand ... As soon as a take doctrine, look at the function of the parameter and return value, directly called. found that the return value has three parameters, N should be the capacity or length of the string, err I find the return nil anyway, so these two parameters for me I do not care, only need to get the second return parameter []byte on the line.
As a result, the conversion function is encapsulated as follows
SRC is the string to be converted, Srccode is the encoding format to be converted, Targetcode is the encoding format to convert
func converttobyte (src string, srccode string, Targetcode string) []byte {
Srccoder: = Mahonia. Newdecoder (Srccode)
Srcresult: = srccoder.convertstring (src)
tagcoder: = Mahonia. Newdecoder (Targetcode)
_, CDATA, _: = Tagcoder.translate ([]byte (Srcresult), true)
return CDATA
}
Use
Response = util. Converttobyte (String (response), "GBK", "UTF8")
err = json. Unmarshal (response, &result)
if err! = Nil {
logger. Error ("Error:", err)
return result, err
}
return Result,nil
accomplished