Use of lftp in open source software

Lftp is a very famous character interface file transfer tool. A powerful download tool that supports ftp, ftps, http, https, hftp, and fish. (ftps and https must include the openssl library during compilation ). The llftp interface has a shell: It has command completion, history records, and allows multiple background tasks to be executed. It is very convenient to use. It also provides functions such as bookmarks, queues, images, resumable data transfer, and multi-process download.

LFTP is sophisticated ftp/http client, file transfer program supporting a number of network protocols. like BASH, it has job control and uses readline library for input. it has bookmarks, built-in mirror, can transfer several files in parallel. it was designed with reliability in mind. LFTP is free software.

The Code on different servers needs to be synchronized yesterday, but the default ftp tool cannot be used to recursively download folders. I found this software on the Internet, so I wrote my own operation steps, hope to help you.

1. Download The lftp installation package
The version I downloaded is: lftp-4.0.1.tar.bz2,: http://download.chinaunix.net/down.php? Id = 26738 & ResourceID = 5628 & site = 1

The installation package with the extension tar.bz2 is decompressed, and "bzip2-d lftp-4.0.1.tar.bz2”is used to generate a “lftp-4.0.1.tar" tar package, and then release the files. The procedure is as follows:

[/Opt/opthb/work/ymm/OpenSource] $ bzip2-d lftp-4.0.1.tar.bz2 [/opt/opthb/work/ymm/OpenSource] $ tar-xzvf lftp-4.0.1.tar [/opt/opthb/ work/ymm/OpenSource] $ lslftp-4.0.1/Of course, the-j parameter of tar can also be extracted. The difference with bzip2 is that the source file no longer exists when "bzip2" is used, but "tar-xjvf" is used ", the source file will be retained as follows:

[/opt/opthb/work/ymm/OpenSource]$lslftp-4.0.1.tar.bz2

The following is an explanation of parameters in tar in the man manual:

-J, -- bzip2 filter the archive through bzip2. We can see that using the "-j" parameter, we still use "bzip2" for decompression. 2. Install lftp to modify the configure file execution permission, generate the makefile file, run the "configure" command to check and configure the system environment required for installation, and generate the installation configuration file. You can use the -- prefix path to execute the path of the final generated database. # Modify the path of the generated library to "/opt/opthb/work/ymm/usr/local"

[/opt/opthb/work/ymm/OpenSource/lftp-4.0.1]$ ./configure --prefix=“/opt/opthb/work/ymm/usr/local”

After the makefile of the environment is generated, the source file is compiled and the executable file is generated.
The command is as follows:

[/Opt/opthb/work/ymm/OpenSource/lftp-4.0.1] $ make use of make install to install compiled files

[/opt/opthb/work/ymm/OpenSource/lftp-4.0.1]$ make install

After the installation is successful, the required executable file is generated under the specified directory "/opt/opthb/work/ymm/usr/local", as follows:

[/opt/opthb/work/ymm/OpenSource/lftp-4.0.1]$ cd /opt/opthb/work/ymm/usr/local[/opt/opthb/work/ymm/usr/local]$lsbin  etc  lib  share[/opt/opthb/work/ymm/usr/local]$ls binhelp  lftp  lftpget

Through the above steps, we can use lftp.
If we want to use lftp in other paths, we only need to add the bin PATH to the environment variable PATH, as shown below:

[/opt/opthb/work/ymm/usr/local]$lftp-bash: lftp: command not found[/opt/opthb/work/ymm/usr/local]$export PATH=/opt/opthb/work/ymm/usr/local/bin:${PATH}                [/opt/opthb/work/ymm/usr/local]$lftplftp :~> 

3. log on to the remote server
Log on to the remote server in the same way as normal ftp logon, as shown below:

[/opt/opthb/work/ymm/usr/local]$lftplftp :~> open 172.21.0.21lftp 172.21.0.21:~> user billing_dxPassword: lftp billing_dx@172.21.0.21:~>

4. Use an image for file transmission
You can use help to view how to use all commands supported by lftp.

Lftp billing_dx@172.21.0.21 :~ /Work/ymm> help mirrorUsage: mirror [OPTS] [remote [local] Mirror specified remote directory to local directory-c, -- continue a mirror job if possible-e, -- delete files not present at remote site -- delete-first delete old files before transferring new ones-s, -- allow-suid set suid/sgid bits according to remote site -- allow-chown try to set owner and group on files -- ignore-time ignore time when deciding whether to download-n, -- only-newer download only newer files (-c won't work)-r, -- no-recursion don't go to subdirectories-p, -- no-perms don't set file permissions -- no-umask don't apply umask to file modes-R, -- reverse mirror (put files)-L, -- dereference download symbolic links as files-N, -- newer-than = SPEC download only files newer than specified time-P, -- parallel [= N] download N files in parallel-I RX, -- include RX include matching files-x RX, -- exclude RX exclude matching files RX is extended regular expression-v, -- verbose [= N] verbose operation -- log = FILE write lftp commands being executed to FILE -- script = FILE write lftp commands to FILE, but don't execute them -- just-print, -- dry-run same as -- script =-When using-R, the first directory is local and the second is remote. if the second directory is omitted, basename of first directory is used. if both directories are omitted, current local and remote directories are used. you can use the mirror command to download folders. With the LCD command (local path), you can download files to the specified directory:

Lftp billing_dx@172.21.0.21: ~> LCD/opt/opthb/work/ymm/test/LCD OK, local cwd =/opt/opthb/work/ymm/testmirror folder, you can recursively download this folder to the directory specified by the LCD, as shown below:

lftp billing_dx@172.21.0.21:~/work/ymm> mirror  DataGen/Total: 3 directories, 15 files, 0 symlinks                                 Modified: 1 file, 0 symlinks195 bytes transferred

Mirror-R upload folders (separate files are not allowed)

Lftp billing_dx@172.21.0.21:/tmp/temp> mirror-R ymm.txt # File Upload error mirror: Access failed:/opt/opthb/work/ymm/test/ymm.txt: not a directory1 error detectedlftp billing_dx@172.21.0.21:/tmp/temp> mirror-R test # upload directory normal Total: 38 directories, 289 files, 0 symlinks New: 289 files, 0 symlinks3047762 bytes transferred in 2 seconds (1.89 M/s) To be removed: 0 directories, 1 file, 0 symlinks

Resumable data transfer and Multithreading
Mirror-c -- parallel = number of threads remote path local path

Lftp billing_dx@172.21.0.21:/tmp/temp> mirror-c -- parallel = 2 sub sub2Total: 1 directory, 4 files, 0 symlinksNew: 4 files, 0 symlinks if the local path does not exist, the specified directory is automatically created. Using the above method, the migration of user directories is very effective when the number of files is large. In addition, I found that the executable permissions of files downloaded through ftp will be removed, while the executable files downloaded through lftp will be retained. Http://download.chinaunix.net/download/0006000/5628.shtml lftp introduction and software download http://blog.csdn.net/xiaotuni/article/details/2099609 tar.bz2 files unzip http://bbs.chinaunix.net/thread-1325666-1-1.html lftp download entire folder http://blog.csdn.net/helloworldtoo/article/details/7022291 lftp command Introduction