Partition log large file under Linux

There are two common commands for partitioning logs under Linux

1. Dd

2. Split ( recommended )

1.DD:

The function is to copy a file with a block of the specified size and make the specified conversion at the same time as the copy.

Parameter comment:

If=filename: The file name entered

Of=finename: The file name of the output

Bs=bytes: Number of bytes read and write, default is 512bytes

Skip=blocks: Pre-blocks block of input file skipped before copy, size of block has BS decision

Count=blocks: Copy only the first blocks block of the input file

Like what

There is now a file, with a size of 116616 bytes:

[root]# du-b file 116616 file

Divide it into two files file1 and file2, then we set each block to 1024 bytes, the first 60 blocks of file into File1, the rest into file2:

[root]# DD if=file bs=1024 count=60 skip=0 of=file1 [root]# dd if=file bs=1024 count=60 skip=60 of=file2

This enables large file segmentation.

2.split

This is more commonly used as a log split

Syntax: Split [--help][--version][-< rows >][-b < bytes >][-c < bytes >][-l < rows >][files to be cut [output file name]

Parameters

-< lines > or-l< > Specify how many lines to cut into a small file.

-b< byte > Specify how many words you want to cut into a small file. Support Unit: M,k

The-c< byte > is similar to the-B parameter, but maintains the integrity of each line as much as possible when cutting.

--help display Help.

--version Displays version information.

[Output file name] Sets the file's predecessor file name after cutting, and split automatically adds a number to the pre-file name.

　　

Examples of Use:

[[email protected] split]# split-b 100m 1111.log (//separated by byte, one file per 100mb) [[email protected] split]# split-l 1000000 1 111.log (//Divide by line, split every 100w line)

Another example:

Mister into 100kb Size file

[[email protected] split]# dd If=/dev/zero bs=100k count=1 of=date.file 1+0 Records in 1+0 Records out 102400 bytes (102 k B) copied, 0.00043 seconds, 238 MB/s

Use the split command to split the Date.file file created above into small files of size 10KB:

[Email protected] split]# Split-b 10k Date.file

The resulting effect is as follows

[Email protected] split]# lsdate.file xaa xab xac xad xae xaf xag xah Xai Xaj

The file is split into multiple suffix files with letters, and if you want to use the-D parameter with a numeric suffix, you can specify the length of the suffix by using-a length. You can also specify a filename prefix for the split file:

[[email protected] split]# Split-b 10k date.file-d A 3 split_file[[email protected] split]# lsdate.file split_file000 split_file001 split_file002 split_file003split_file004 split_file005 split_file006 split_file007 split_file008 SPL it_file009

This article is from the "Yue Dance God" blog, please be sure to keep this source http://hui1943.blog.51cto.com/4195144/1967221

Partition log large file under Linux