Simple method of processing BOM head and ^m under Linux

Linux has a large share of Web servers and embedded devices, and Microsoft windows has a large share of desktop operating systems, so a lot of people like to use Windows to control the operation of Linux.

Since the use of Windows to control Linux, it is inevitable that the resulting files on the Windows system in some way to the Linux system, resulting in display problems or garbled situation.

For example, a file saved by default with a Windows-brought "notepad" (Notepad) program will have a ^m tag at the end of each line.

PS: Some common error Examples: Some people may have doubts, why I use Notepad to save the file as UTF-8 is also not good. There are people in the implementation of "Sed-i '/^$/d ' filename" when it was found that there is no space but not deleted.

Therefore, in the daily use process, regardless of what tool to edit files uploaded to the Linux server, you need to pay attention to the line break problem.

Phenomenon:

1.cat program shows a problem

Show "? t" at the beginning of the file

2.bash, Python and other file execution times wrong

such as-bash:./someshname.sh:/bin/bash^m:bad interpreter:no such file or directory

However, if you do not want to convert a newline character, you can execute the file directly with the appropriate interpreter, as

/bin/bash./someshname.sh

3. Other applications such as PHP, Java and other run times wrong

Explain:

NewLine characters (newline, lines ending, end of Line (EOL), or line break) are a control character used to differentiate between the end of each row. NewLine characters are usually represented by one of the line feeds (LF) and carriage return (CR) or their combination in a computer system, and there are three common newline characters:

1. LF (UNIX and OS X \ n)

2. CR (Classic Mac \ R)

3. CRLF (Windows \ r \ n)

LF is the line feed abbreviation, CR is carriage return abbreviation, their control character (\r,\n or \ r \ n) is represented by the corresponding anscii.

The reason why "^m" occurs because ^m in anscii means that carriage return is \ r So if ^m appears in a printout of Linux, the newline character is in Windows format.

The BOM is the abbreviation for byte order mark, which is interpreted as a "byte sequence mark" to make it clear that this file is Unicode encoded, and that other functions can be referred to in the Wikipedia English page (obviously the Chinese page is not fully explained).

Workaround:

This problem is simply not necessary to use the length of an article to express, Jiang said a word: You can use the Dos2unix tool, the Windows format of the text file into the format available under Linux. but in order to facilitate those who need detailed understanding of the people, specifically write a few more words as follows.

# Remove BOM and ^m (BOM and ^m can come from Windows Notepad program and Save as ' ANSI ' or ' UTF-8 ')
# such as "example^m$", ANSI, ASCII text, with CRLF line terminators
# such as "m-om-; M-?example^m$ ", UTF-8, UTF-8 Unicode (with BOM) text, with CRLF line terminators
# Linux Right format is "example$", ASCII text
# Other method is using Vim [Noeol][dos]: Set Ff=unix

# Determine file type

File Testfilename

# Display $ at end of each line, display TAB characters as ^i, use ^ and m-notation, except for LFD and TABo

Cat-a Testfilename

#Text file format converters. Convert text files with DOS or Mac line endings to Unix line endings and vice versa.
# Debian & ubuntu:apt-get Install Dos2unix
# RHEL & Centos:yum Install Dos2unix

# Dos/mac to Unix and vice versa text file format converter

Dos2unix Testfilename

About display garbled problems

1. It may be related to the terminal (SSH connection tool), try to adjust the character encoding to Utf-8

2. May be a system cause, such as lack of Chinese support

Some of the available references:

Byte Order Mark Https://en.wikipedia.org/wiki/Byte_order_mark

Newline Https://en.wikipedia.org/wiki/Newline

Anscii https://en.wikipedia.org/wiki/ASCII#ASCII_printable_characters

Tag:bad interpreter, remove Bom,dos2unix command, Crlf,linux line break

--end--

This article is from "Communication, My Favorites" blog, please make sure to keep this source http://dgd2010.blog.51cto.com/1539422/1762675

Simple method of processing BOM head and ^m under Linux