Encoding for Japanese
Windows: shift-jis
Linux: 2.4 kernel using EUC encoding, 2.6 kernel using UTF8 encoding
Check file encoding nkf-g filename
The Iconv command is usually used to handle character encodings, but the iconv command can only be used to process filenames, but the encoding of text content cannot be processed.
In order to convert the character encoding of the text content, we need to use the NKF.
-j : Convert to JIS Code (ISO-2022-JP), Default-E : Convert to EUC encoding-s : Convert to Shift-jis encode-W : Convert to UTF-8 encoding (no BOM)-lu : Convert to UN IX newline Format (LF)-LW : Convert to Windows Wrap Format (CRLF)-LM : Convert to Macintosh newline format (CR)-G (--guess) : Automatically determine the encoding and display the--version : Show version--help : Show Help
convert Linux to window:nkf-sxlw NKF-SWLW
Window converted to Linux:nkf-wxlu
Conversion of Japanese encoding using the NKF command in Linux
For the processing of Japanese characters encoding, Windows uses different character encodings for Linux, and the Japanese characters used in Windows are encoded as Shift-jis. The 2.4 kernel in Linux uses EUC encoding, UTF8 encoding is used in the 2.6 kernel, and in order to solve this problem, the NKF command can be used. Usually processing character encoding use ICONV this command, but the ICONV command can only be used to handle the file name, but the text content of the encoding can not be processed, to the text content of the character encoding to convert, it is necessary to use the NKF.
First download the installation, I use Ubuntu8.10, with APT can be very convenient to install. If you use the source code for installation, you can find the relevant information yourself.
The installation is ready to start using. Here is a brief introduction to the use of NKF. When the NKF command is used, the specified text content is printed and output to the screen in the encoding format set by the parameter, and the original text is not changed, to generate new encoded text, you can use the pipeline command to redirect the results to another file. First introduce a few basic parameters: J,s,e,w, these parameters are JIS,SHIFT-JIS,EUC,UTF encoding, if you want to use UTF8 W8 parameters, to use UTF16 with W16 parameters, note that these letters are lowercase, there are four uppercase parameters, J,s,e , W corresponds to the same character encoding format as before, the difference between uppercase and lowercase is that the lowercase letter denotes the encoding of the output, the uppercase letter indicates the character encoding that is assumed to be entered, the word "assumed" is used here, which is what the man document says, and I guess that means, If a shift-jis encoded file needs to be converted to UTF8 format, if the specified parameter is e, then the character encoding of the file is treated as a EUC. The help documentation says you can specify the input file and output file, but according to my own experiment it seems that I cannot make the result into an output file and can only print to the screen, so you can use only pipeline commands to redirect to other files. For example, I'm now going to convert a shift-jis encoded file into a UTF8 format, so I can use the following command, which is something we often use.
$NKF-W myfile.txt >> test
The meaning of this command is to convert the contents of the file MyFile.txt file text into a UTF8 encoding and then output it to the file test, and this time we will look at the contents of the file test file should be able to display the normal.
NKF also has a more interesting parameter, which is the ability to convert between hiragana and katakana. That is the-h parameter, note that the parameters of the Help file command for many commands are also-H,NKF's help command for the--HELP,-H1 parameter for the conversion between katakana to hiragana,-h2 for the conversion between hiragana to katakana,-h3 for bidirectional conversion, the first two parameters okay to understand, for a third bidirectional conversion, I have not done the actual verification, do not do other ideas.
Here is a simple way to use the NKF command, specific usage please refer to the man document or help command, remember, NKF's help command is--help, not-H.
From for notes (Wiz)
Linux NKF Japanese encoding conversion command [reprint]
