Introduction to the use of multi-threaded download tool mwget and Axel under Linux

Linux Operations in the operation of the Linux process, the most used Linux download tool must be wget, do not see which server has not been installed wget, perhaps someone use ftp download, also someone use multi-threaded axel and Prozilla, There is no doubt that these two tools are very good, but for the habit of using wget, it seems not suitable, so the foreigner developed a multi-threaded version of the wget tool mwget,m means multi multi-threading meaning.

Mwget
Official address: HTTP://SOURCEFORGE.NET/PROJECTS/KMPHPFM/?SOURCE=DLP

# cd/usr/local/src/
# wget HTTP://JAIST.DL.SOURCEFORGE.NET/PROJECT/KMPHPFM/MWGET/0.1/MWGET_0.1.0.ORIG.TAR.BZ2
# TAR-XJVF MWGET_0.1.0.ORIG.TAR.BZ2
# CD Mwget_0.1.0.orig
#./configure
# make
# make Install

If the following error occurs during configure, install a version of Intltool that is greater than or equal to 0.35

Checking for Intltool >= 0.35.0 ... ./configure:line 6824:intltool-update:command Not found
Found
Configure:error:Your Intltool is too old. You need Intltool 0.35.0 or later.
# yum Install Intltool

wget vs Mwget Speed test
The following uses the download 2.4.5 version of MongoDB to sample.
Mwget Speed Test

# Mwget Http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.4.5.tgz
Begin to Download:mongodb-linux-x86_64-2.4.5.tgz
filesize:90m
36% [====> =====> ===> ====>] [33M] [2.0m/s] [eta:00:28]

The Mwget speed is as follows:

wget Speed Test

# wget Http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.4.5.tgz
--2013-08-16 10:55:23--http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.4.5.tgz
Resolving fastdl.mongodb.org ... 54.230.114.206, 54.230.113.36, 54.230.114.144, ...
Connecting to fastdl.mongodb.org|54.230.114.206|:80 ... Connected.
HTTP request sent, awaiting response ... OK
length:94827295 (90M) [Application/x-tar]
Saving to: "Mongodb-linux-x86_64-2.4.5.tgz.1"

6% [==>] 6,470,547 210k/s ETA 6m 26s

Wget speed is as follows

Mwget usage:

# mwget-h
GNU Mwget 0.1.0, a non-interactive and multiline network retriever of all Postix Systems.
Usage:mwget [Options] ... [URL] ...
Options:
-B,--debug Show the debug message
-C,--count=num Set the retry count to [num], no limit when ' 0 ', the default is ' 99 '
-D,--directory=dir Set the local direcotry to [dir], and the default is "."
-F,--file=file Rename the file to [file]
-H,--help A Brief summary of all the options
-I,--interval=num Set the FTP retry interval to [num] seconds, the default is "5"
-N,--number=num use [num] connections instead of the default (4)
-R,--referer=url Include ' referer: [URL] ' header in HTTP request.
-T,--timeout=num Set the connection timeout to [num] seconds, the default is "30"
-V,--version Show The version of the Mwget and exit
-X,--proxy=url Set the proxy [URL]

Mail bug reports and suggestions to <[email Protected]><[email protected]>
On website Bugs reports and suggestions to We Welcome your BUG report!

Axel
Operation and maintenance work, often need to download files, because the website download speed limit or network and other reasons cause the download unbearable, the previous introduction multithreading wget tool Mwget, speed up a lot, and the use is very simple. Axel use more people, not only the installation method is simple, the command is more simple. Look down.
Installing Axel
RPM Installation Axel (method I)
: http://pkgs.repoforge.org/axel/
, choose the RPM that's right for your server

[Email protected] # wget http://pkgs.repoforge.org/axel/axel-2.4-1.el6.rf.x86_64.rpm
[Email protected] # RPM-IVH axel-2.4-1.el6.rf.x86_64.rpm
Preparing ... ########################################### [100%]
1:axel ########################################### [100%]

Yum installation Axel (method two)

[Email protected] # Yum install Axel

Apt-get installation Axel
If you are an Ubuntu system

[Email protected] # axel-h
Usage:axel [Options] url1 [url2] [url ...]

--max-speed=x-s x Max Speed (Bytes/sec)
--num-connections=x-n x Maximum number of connections
--output=f-o F Specify file name
--SEARCH[=X]-s [x] search for mirrors and download from x servers
--header=x-h x Add header
--user-agent=x-u x Setting up the user agent
--no-proxy-n do not use any proxy
--quiet-q Leave stdout alone
--verbose-v Show more status information
--alternate-a Show Simple progress bar
--help-h Help
--version-v Version Information
Axel Use instances
Download CentOS

[Email protected] # Axel Http://mirrors.btte.net/centos/6.5/isos/x86_64/CentOS-6.5-x86_64-bin-DVD1.iso
[1%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [4654.5KB/S]
[1%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [4657.1KB/S]
[1%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [4656.5KB/S]
[1%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [4659.3KB/S]

A picture of the life time of the carrying dimension

[Email protected] # Axel Http://www.ttlsa.com/wp-content/uploads/2014/04/<a href= "http://www.ttlsa.com/monitor/ zabbix/"title=" Zabbix "target=" _blank ">zabbix</a>-adduser-01.jpg
Initializing download:http://www.ttlsa.com/wp-content/uploads/2014/04/zabbix-adduser-01.jpg
File size:73009 bytes
Opening Output File Zabbix-adduser-01.jpg
Starting download</p> <p>[email protected] # Axel HTTP://WWW.TTLSA.COM/WP-CONTENT/UPLOADS/2014/04/ Zabbix-adduser-01.jpg
Initializing download:http://www.ttlsa.com/wp-content/uploads/2014/04/zabbix-adduser-01.jpg
File size:73009 bytes
Opening Output File Zabbix-adduser-01.jpg
Starting download

[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [598.0KB/S]
[70%] ........ .......
Connection 0 Finished

Downloaded 71.3 kilobytes in 0 seconds. (710.02 kb/s)

Simple progress bar
By default, the entire screen is a progress bar, plus the-a parameter, and the progress bar is similar to wget.

[Email protected] # axel-a Http://mirrors.btte.net/centos/6.5/isos/x86_64/CentOS-6.5-x86_64-bin-DVD1.iso
Initializing Download:http://mirrors.btte.net/centos/6.5/isos/x86_64/centos-6.5-x86_64-bin-dvd1.iso
File size:4467982336 bytes
Opening Output File Centos-6.5-x86_64-bin-dvd1.iso
Starting download</p> <p>[1%] [0 1 2 3] [3.2mb/s] [21:47]

Download speed limit
Ttlsa website Bandwidth is limited, I want to ensure that the download without affecting the user's access, such as limited to 1mb/s download speed, the following can be seen below the basic speed guaranteed within 1MB.

[Email protected] # axel-s 1024000 http://mirrors.btte.net/centos/6.5/isos/x86_64/CentOS-6.5-x86_64-bin-DVD1.iso
Initializing Download:http://mirrors.btte.net/centos/6.5/isos/x86_64/centos-6.5-x86_64-bin-dvd1.iso
File size:4467982336 bytes
Opening Output File Centos-6.5-x86_64-bin-dvd1.iso
State file found:30522952 bytes downloaded, 4437459384 to go.
Starting download

,,,,,,,... .......... .......... .......... .......... [394.8KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [689.2KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [879.5KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [974.5KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [1086.4KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [967.5KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [943.8KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [1021.0KB/S]
[0%] ... .... ..... ..... ..... ..... ..... ..... ..... ....... ........ [1066.1KB/S]

At last
Complete, the method is very simple, the function is very simple. Operational survival time of the server and installed Axel, you installed it?

Introduction to the use of multi-threaded download tool mwget and Axel under Linux