Linux bulk GDK encoded articles into UTF-8 encoded articles and "ICONV: illegal input sequence unknown XXX" error handling

Batch conversion GBK file to UTF-8 encoding method under Linux

View File Encoding

File-i filename

1. Recursive conversions (including subfolders)

Find Default-type d-exec mkdir-p utf/{} \;

Find Default-type f-exec iconv-f gbk-t UTF-8 {}-o utf/{} \;

These two lines of command convert the file under the default directory from GBK encoding to UTF-8 encoding, the directory structure is unchanged, and the transcoded file is saved in the Utf/default directory.

2. Conversion

The command format is as follows

$enca-L Current language-x target encoded file name

For example, to turn all files in the current directory into Utf-8

$enca-L zh_cn-x utf-8 *

ICONV: illegal input sequence at unknown XXX

Under the Linux command line bash, there was a problem with iconv conversion UTF-8 encoding to gb2312: iconv: Illegal input sequence at unknown XX

Adding the-C option to iconv, ignoring invalid characters, succeeded.

/*--------------------------------I am the split line--------------------------------*/

Above reference http://hi.baidu.com/maleung/blog/item/a75403ed6ea19fd8b21cb1db.html

-----------------------------------------------------------------

You can also add//ignor to the parameters as in PHP, as follows

Iconv-f gbk//ignore-t Utf8//ignore hufu.csv> hufu-utf8.csv

can also be performed normally