Today I tried a tool (like azcopy) to import data from a Linux cluster into a hdinsight cluster in Azure (Microsoft Cloud)
about the Linux version of similar to Azcopy Upload Tool, or you can change the parameters to support the Chinese area Azure transfer of the file.
so this approach should be feasible. You can download this tool from the following GitHub .
Https://github.com/matthchr/azure-batch-samples/tree/master/Python/Storage
support for the Chinese region Azure the parameters are as follows :
--BLOBEP BLOBEP default is Global BLOB Storage endpoint [core.windows.net]
Forchina, use:core.chinacloudapi.cn
First, install the Blobxfer
Https://github.com/matthchr/azure-batch-samples/tree/master/Python/Storage
PIP command System is not, because the previously installed python2.7 does not have this command, you need to install PIP first
Second, PIP installation, reference
https://www.ttlsa.com/python/how-to-install-and-use-pip-ttlsa/
The steps are as follows:
wget "HTTPS://PYPI.PYTHON.ORG/PACKAGES/SOURCE/P/PIP/PIP-1.5.4.TAR.GZ#MD5=834B2904F92D46AAA333267FB1C922BB"-- No-check-certificate
TAR-ZXVF pip-1.5.4.tar.gz
CD pip-1.5.4
chmod +x setup.py
# python setup.py Install
The error is as follows:
Traceback (most recent):
File "setup.py", line 6, <module>
From Setuptools import Setup, find_packages
Importerror:no module named Setuptools
Tip There is no setuptools module, you need to install the Setuptools module first,
The final display is shown below to calculate the installation success:
Installed/usr/local/python2.7/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg
Processing dependencies for pip==1.5.4
Finished processing dependencies for pip==1.5.4
Third, found no setuptols
Refer to the following document: Http://www.letuknowit.com/post/147.html
wget Https://pypi.python.org/packages/source/s/setuptools/setuptools-5.7.zip--no-check-certificate
wget http://peak.telecommunity.com/dist/ez_setup.py
wget Http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
Unzip Setuptools-5.7.zip
CD setuptools-5.7
Need to add X Execute permission
chmod +x setup.py
# python ez_setup.py or python2.7 ez_setup.py install
The final display follows the installation success:
Installed/usr/local/python2.7/lib/python2.7/site-packages/setuptools-5.7-py2.7.egg
Processing dependencies for setuptools==5.7
Finished processing dependencies for setuptools==5.7
Need to add execute permission, otherwise the error is as follows:
Traceback (most recent):
File "ez_setup.py", line 278, in <module>
Main (sys.argv[1:])
File "ez_setup.py", line 212, in main
From Setuptools.command.easy_install Import Main
Zipimport. Zipimporterror:can ' t decompress data; Zlib Not available
Or the error is as follows:
# python ez_setup.py
Downloading Http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
Traceback (most recent):
File "ez_setup.py", line 278, in <module>
Main (sys.argv[1:])
File "ez_setup.py", line +, in main
Egg = Download_setuptools (version, delay=0)
File "ez_setup.py", line 158, in Download_setuptools
src = urllib2.urlopen (URL)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 154, in Urlopen
Return Opener.open (URL, data, timeout)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 437, in open
Response = Meth (req, response)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 550, in Http_response
' HTTP ', request, response, code, MSG, HDRs)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 469, in error
result = Self._call_chain (*args)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = Func (*args)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 656, in http_error_302
Return Self.parent.open (new, Timeout=req.timeout)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 431, in open
Response = Self._open (req, data)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 454, in _open
' Unknown_open ', req)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = Func (*args)
File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 1265, in Unknown_open
Raise Urlerror (' unknown URL type:%s '% type)
Urllib2. Urlerror: <urlopen error Unknown URL type:https>
Note: Sometimes you will be prompted no moudule zlib
CD python-2.7.11/
chmod +x update.sh
CD modules/
Vim Setup.dist will # Delete, then recompile install Python, note add update.sh (Install script) Execute permissions
zlib zlibmodule.c-i$ (prefix)/include-l$ (exec_prefix)/lib-lz
[Email protected] ~]# blobxfer azureaccount azurecontiner nginx+php1.9.tar.gz
Traceback (most recent):
File "/usr/local/python2.7/bin/blobxfer", line 7, <module>
From Blobxfer Import Main
File "/usr/local/python2.7/lib/python2.7/site-packages/blobxfer.py", line at <module>
Import Azure.common
Importerror:no module named Azure.common
[[Email protected] ~]# pip install Azure-storage
Requirement already satisfied (use--upgrade to upgrade): Azure-storage in/usr/local/python2.7/lib/python2.7/ Site-packages
Cleaning up ...
[[Email protected] ~]# pip install Azure-common
Requirement already satisfied (use--upgrade to upgrade): Azure-common in/usr/local/python2.7/lib/python2.7/ Site-packages
Cleaning up ...
[[Email protected] ~]# PIP install requests
Downloading/unpacking Requests
Downloading REQUESTS-2.9.1-PY2.PY3-NONE-ANY.WHL (501kB): 501kB downloaded
Installing collected Packages:requests
Successfully installed requests
This article comes from "Plum blossom fragrance from bitter cold!" "Blog, be sure to keep this provenance http://daixuan.blog.51cto.com/5426657/1767325
Python pip, setuptools installation