8th: Shell script Archiving and compression

Tag: One put BSP compresses rect permissions. GZ mnt gzip

8th: Shell script Archiving and compression

Regular backups are not trivial, and we can automate backup through shell scripting. Where data backup is generally used to archive and compression, archiving and compression for the system administrator or ordinary users are more commonly used tools, there are many different compression format, to combine different use methods to obtain the best compression effect. File encryption is also a common method of protecting data, which is usually archived and compressed before it is encrypted.

8.1 , Tar Archive

The tar command is primarily used to archive files, and you can use tar to save multiple files and folders as a single file, while preserving the properties, permissions, and so on for all files.

Example 1 : Create tar Archive

[Email protected] home]# tar--help | grep ' \-f\, '

-F,--file=archive using an archive file or ARCHIVE device

[Email protected] home]# tar--help | grep "\-c\," #查看系统tar帮助 description of the –C option

-C,--create create a new archive

[[email protected] home]# for ((i=1;i<4;i++));d o ifconfig > Font$i.txt;done #使用for循环创建3个测试文本

[[email protected] home]# ls font*

Font1.txt Font2.txt Font3.txt

[Email protected] home]# tar-cf font.tar font1.txt font2.txt font3.txt

Example 2: View archive and append archive files

[Email protected] home]# tar--help | grep ' \-v\, '

-V,--verbose lists the processed files in detail

[Email protected] home]# tar--help | grep ' \-r\, '

-R,--append append file to end of archive

[Email protected] home]# TAR-TF Font.tar #查看归档文件

Font1.txt

Font2.txt

Font3.txt

[Email protected] home]# TAR-RVF Font.tar font4.txt

Font4.txt

[Email protected] home]# TAR-TF Font.tar

Font1.txt

Font2.txt

Font3.txt

Font4.txt

[Email protected] home]# TAR-TVF Font.tar #查看更加详细的归档文件

-rw-r--r--root/root 10240 2017-04-29 18:05 font1.txt

-rw-r--r--root/root 884 2017-04-29 18:03 font2.txt

-rw-r--r--root/root 884 2017-04-29 18:03 font3.txt

-rw-r--r--root/root 884 2017-04-29 18:14 font4.txt

Example 3: extract text or folders from an archive file

[Email protected] home]# tar--help | grep ' \-x\, '

-X,--extract,--get to extract files from the archive

[Email protected] home]# tar--help | grep ' \-c\, '

-C,--directory=dir change to directory dir

[Email protected] home]# TAR-XF Font.tar

[[email protected] home]# ls

Font1.txt font2.txt font3.txt font4.txt font.tar lost+found test.sh

[Email protected] home]# TAR-XF font.tar-c/mnt

[Email protected] home]# ls/mnt/

Font1.txt font2.txt font3.txt Font4.txt

[Email protected] home]# TAR-TVF Font.tar

-rw-r--r--root/root 884 2017-04-29 18:27 font1.txt

-rw-r--r--root/root 884 2017-04-29 18:27 font2.txt

-rw-r--r--root/root 884 2017-04-29 18:27 font3.txt

-rw-r--r--root/root 884 2017-04-29 18:27 font4.txt

[Email protected] home]# TAR-XVF Font.tar font2.txt

Font2.txt

Example 4 : Two tar Archive files for merging

[Email protected] home]# tar--help | grep "\-a\,"

-A,--catenate,--concatenate append tar file to archive

[Email protected] home]# TAR-CF Test01.tar font1.txt font2.txt

[Email protected] home]# TAR-CF Test02.tar font3.txt font4.txt

[Email protected] home]# tar-af test01.tar test02.tar #把test02. tar files merged into Test01.tar

Example 5: Delete a file from an archive file

[Email protected] home]# tar--help | grep "\-delete"

--delete from archive (non-tape!) ) in the Delete

[Email protected] home]# TAR-TF Test01.tar

Font1.txt

Font2.txt

Font3.txt

Font4.txt

[Email protected] home]# tar--delete--file test01.tar font2.txt

[Email protected] home]# TAR-TF Test01.tar

Font1.txt

Font3.txt

Font4.txt

Example 6: The tar command archives and compresses files,-Z to. GZ compression format,-j to bz2 compression format

[Email protected] home]# TAR-ZCVF file.tar.gz font1.txt font3.txt font4.txt

Font1.txt

Font3.txt

Font4.txt

[Email protected] home]# Ls-l | Head-n 3

-rw-r--r--. 1 root root 589 April 22:59 file.tar.gz

-rw-r--r--. 1 root root 45 April 22:56 Font1.txt

Example 7: tar command to extract archive files

[Email protected] home]# TAR-ZXVF file.tar.gz

Font1.txt

Font3.txt

Font4.txt

8.2 , gzip Compression and Gunzip Unzip

Gzip only compresses a single file and cannot archive directories and multiple files, because it requires tar to be archived and then compressed using gzip, generating a compressed format suffix of. gz

Example 1: gzip Deletes the original file by default to generate a compressed file test01.tar.gz

[Email protected] home]# gzip font1.txt

[Email protected] home]# ls-l

-rw-r--r--. 1 root root 401 April 18:27 font1.txt.gz

-rw-r--r--. 1 root root 884 April 18:27 font3.txt

Example 2: Gunzip Decompression will delete the original compressed files extracted from the file

[Email protected] home]# gunzip font1.txt.gz

[Email protected] home]# ls-l font1.txt

-rw-r--r--. 1 root root 884 April 18:27 font1.txt

Example 3: gzip-l option to view compressed file property information

[Email protected] home]# gzip-l font1.txt.gz

Compressed uncompressed ratio uncompressed_name

401 884 58.7% Font1.txt

Example 4: font4.txt content input, gzip packaged output rewrite to test.gz

[email protected] home]# Cat Font4.txt | Gzip-c > test.gz

Example 5: archive with Tar first and then gzip compress

[Email protected] home]# TAR-CVF File.tar font3.txt font4.txt

Font3.txt

Font4.txt

[Email protected] home]# gzip File.tar

[Email protected] home]# ls-l file.tar.gz

-rw-r--r--. 1 root root 515 April 23:35 file.tar.gz

Example 6: first, gzip unzip. GZ then unzip the tar with tar

[Email protected] home]# ls-l file.tar.gz

-rw-r--r--. 1 root root 515 April 23:35 file.tar.gz

[Email protected] home]# gunzip file.tar.gz

[Email protected] home]# ls-l File.tar

-rw-r--r--. 1 root root 10240 April 23:35 File.tar

[Email protected] home]# TAR-XVF File.tar

Font3.txt

Font4.txt

8.3 , bzip2 Compression and bunzip2 Unzip

BUNZIP2 is another compression technique similar to bZIP, which compresses the format suffix. bz2

Example 1: bzip2 Default deletes the original file to generate a compressed file font3.txt.bz2

[Email protected] home]# bzip2 font3.txt

[Email protected] home]# ls-l font3.txt.bz2

-rw-r--r--. 1 root root 462 April 18:27 font3.txt.bz2

Example 2: BUNZIP2 Decompression will delete the original compressed files extracted from the file

[Email protected] home]# BUNZIP2 font3.txt.bz2

[Email protected] home]# ls-l font3.txt

-rw-r--r--. 1 root root 884 April 18:27 font3.txt

Example 3: font3.txt content input, bzip2 packaging output rewrite to test.bz2

[email protected] home]# Cat Font3.txt | Bzip2-c > test.bz2

Example 4: first use tar for archiving and bzip2 compression

[Email protected] home]# TAR-CVF File.tar font3.txt

[Email protected] home]# ls-l File.tar

-rw-r--r--. 1 root root 10240 April 00:16 File.tar

[Email protected] home]# bzip2 File.tar

[Email protected] home]# ls-l file.tar.bz2

-rw-r--r--. 1 root root 545 April 00:16 file.tar.bz2

Example 5: first bzip2 to decompress. bz2 then unzip the tar with tar

[Email protected] home]# ls-l file.tar.bz2

-rw-r--r--. 1 root root 545 April 00:16 file.tar.bz2

[Email protected] home]# BUNZIP2 file.tar.bz2

[Email protected] home]# ls-l File.tar

-rw-r--r--. 1 root root 10240 April 00:16 File.tar

[Email protected] home]# TAR-XVF File.tar

Font3.txt

8.4 , Zip Archiving and compression

Zip, as a popular compression format, is less widely used in gzip or bzip2, but files on the Internet are usually used in this format.

Example 1: zip compression does not delete the original file, and the-r option recursively compresses directories and files.

[Email protected] home]# zip file.zip font3.txt

Adding:font3.txt (deflated 58%)

[Email protected] home]# zip-r file.zip file/

adding:file/(stored 0%)

Adding:file/file.zip (stored 0%)

adding:file/abc/(stored 0%)

Adding:file/abc/file.zip (stored 0%)

[Email protected] home]# ls-l file.zip

-rw-r--r--. 1 root root 1704 April 00:46 file.zip

Example 2: unzip extracts content from a ZIP file and does not remove zip compression after completion

[[email protected] home] #unzip File.zip

Archive:file.zip

creating:file/

Extracting:file/file.zip

creating:file/abc/

Extracting:file/abc/file.zip

Example 3: - u option to update compressed archive contents,-D to delete deleted contents from compressed archive file

[Email protected] home]# zip file.zip-u font4.txt #更新

Adding:font4.txt (deflated 58%)

[Email protected] mnt]# zip-d file.zip/file/abc/file.zip #删除指定压缩归档文件目录下的文件

Deleting:file/abc/file.zip

8.5 , cryptographic tools and hashing

Example 1: GPG encrypts and decrypts text, which uses key signature technology to protect the contents of a file.

[Email protected] home]# gpg-c font4.txt #提示输入加密密码

[Email protected] home]# ls-l FONT4.TXT.GPG

-rw-r--r--. 1 root root 429 April 01:36 font4.txt.gpg

[Email protected] home]# GPG font4.txt.gpg #输入解密密码进行解密

Gpg:3des Encrypted data

Can ' t connect to '/root/.gnupg/s.gpg-agent ': no file or directory

AGPG: Encrypted with 1 passwords

GPG: Warning: Messages are not fully protected

Example 2: Base64 is a set of similar coding schemes that describe binary data in ASCII strings by converting ASCII characters to a 64 base. Base64 can be used to encode and decode base64 strings.

[email protected] home]# cat file.txt

abc123

abc123

abc123

[[email protected] home]# base64 file.txt > Outputfile.txt #加密生成outputfile. txt Encrypted file

[email protected] home]# cat outputfile.txt

Ywjjmtizcmfiyzeymwphymmxmjmk

[[email protected] home]# base64-d outputfile.txt > Jiemifile.txt #解密outputfile. txt generate jimifile.txt Decrypt file

[email protected] home]# cat Jiemifile.txt

abc123

abc123

abc123

8.6 , Raync Backup System Snapshot

Rsync can back up files and directories located in different locations, and it can minimize the amount of data transferred using differential computing and compression techniques. Rsync also supports network data transfer, and rsync also compares the source and destination files and replicates only when the files are updated.

Example 1: Copy source directory to destination,-a option to archive,-V to show details or progress

Callout: in the path format, if you use/,rsync at the end of/bin, all files in the/bin/directory will be copied to the destination. If/bin is not used/, rsync will copy the/bin directory itself to the destination.

[[email protected] home]# rsync-av/bin//home/file #将/bin/directory in/home/file directory

[[email protected] home]# rsync-av/bin/home/#将/bin directory copied to/home/directory

[[email protected] home]# rsync–av/bin/[email protected]:/home/file #将/bin/directory copied to remote host/home/file directory

Example 2: --exclude exclude portions of files that are not in the archive scope

[Email protected] home]# rsync-av/bin/home/--exclude "*.txt"

8.7 , DD Command Clone Disk

DD command when dealing with a hard disk and partition, we may need to create a copy or backup of all partitions, not just the contents (not just the partitions of each hard disk, but also information such as the record of the log, partition table, etc.). At this point, we can use the DD command, which can be used to clone any type of disk if hard disk, Flash, CD, DVD, etc.

The DD format is as follows:

DD If=source of=target bs=block_size count=count

which

If represents input file or input device path

of represents the destination file or target device path

BS represents the block size (usually given in the form of a power of 2, if 512, 1024, 2048, etc.). Count is the number of blocks that need to be copied.

Total number of bytes to copy = block Size * Count

By specifying count, we can limit the number of bytes copied from the input file to the destination. If you do not specify the COUNT,DD command, the input file will be copied until you meet the end of file (EOF) sign.

Example 1: back up the SDA1 partition and use the backup to restore the partition.

[Email protected] home]# df-h

File system capacity has been used with available percent mount points

/dev/sda2 97G 4.5G 87G 5%/

Tmpfs 996M 112K 996M 1%/DEV/SHM

/DEV/SDA1 485M 30M 430M 7%/boot

/dev/sda3 58G 188M 55G 1%/Home

/dev/sr0 4.0G 4.0G 0 100%/media/centos_6.0_final

[email protected] home]# DD if=/dev/sda1 of=/home/sda1_boot.img #备份分区

Recorded 1024000+0 read-in

Recorded the writing of 1024000+0.

524288000 bytes (524 MB) Replicated, 35.5123 sec, 14.8 mb/sec

[Email protected] home]# ls-l sda1_boot.img

-rw-r--r--. 1 root root 524288000 May 3 17:31 sda1_boot.img

[email protected] home]# DD if=/home/sda1_boot.img of=/dev/sda1 #恢复分区

Recorded 1024000+0 read-in

Recorded the writing of 1024000+0.

524288000 bytes (524 MB) Replicated, 9.14704 sec, 57.3 Mb/sec

Example 2: permanently deletes all data from a partition, writes 0 embodied to the partition using DD,/dev/zero is a character device, it always returns the character ' \ s '

[email protected] home]# DD If=/dev/zero of=/dev/sda1

Example 3: in the same capacity of the hard disk cloning,/DEV/SDA is the original data end,/DEV/SDB destination data.

[email protected] home]# DD IF=/DEV/SDA of=/dev/sdb

[[email protected] home]# dd if=/dev/cdrom of=cdrom.iso #制作CD ROM image (ISO file)

8th: Shell script Archiving and compression