How to use the seven kn Python SDK to write a synchronization script and use tutorials

The Python language version of the seven Qiniu storage SDK (hereinafter referred to as PYTHON-SDK) is a layer of encapsulation of the seven Cow cloud Storage API protocol to provide a set of easy-to-use development tools for Python developers. Python developers do not need to understand the details of the seven cloud storage API protocol when docking python-sdk, and in principle do not require a very deep understanding of the HTTP protocol and principles, but if you have the basic HTTP knowledge, the processing of the error scenario can be more efficient.

Recently, I've just set up a markdown static blog to put pictures in cloud storage.

After a survey that seven of cows can meet my personal needs, I chose it.

To refer to a picture, upload the image to the cloud first.

Although the seven cattle website backstage can upload files, but each upload need to log in first, then select the picture, set the connection address, to upload.

This process is a bit cumbersome, so I would like to use the SDK provided by the seven cow Cloud to write a synchronization tool to facilitate the incremental synchronization of files.

With this idea, act immediately. It took me about a morning to write this tool and put it on Gitosc and GitHub.

#!/usr/bin/env python#-*-coding:utf-8-*-# # AUTHOR = "Heqingpan" # author_email = "heqingpan@126.com" # URL = "Http://git. Oschina.net/hqp/qiniu_sync "Import qiniufrom qiniu import authfrom qiniu import bucketmanagerimport osimport Reaccess_ Key = ' Secret_key = ' Bucket_name = ' Bucket_domain = ' q = Auth (Access_key, secret_key) bucket = Bucketmanager (q) basedir= Os.path.realpath (Os.path.dirname (__file__)) Filename=__file__ignore_paths=[filename, "{0}c". Format (filename)] Ignore_names=[". Ds_store ",". Git ",". Gitignore "]charset=" UTF8 "diff_time=2*60def list_all (Bucket_name, Bucket=none, prefix=" ", limit=  : rlist=[] If bucket is none:bucket = Bucketmanager (q) marker = None EOF = False when EOF is False:ret, EOF, info = Bucket.list (Bucket_name, Prefix=prefix, Marker=marker, limit=limit) marker = ret.get (' marker ', None) for item in ret[ ' Items ': Rlist.append (item["key"]) if EOF is not True: # error handling #print "error" Pass return rlistdef Get_files (basedir= "", fix= "", rlist=none,ignore_paths=[], ignore_names=[]): If Rlist is none:rlist=[] for subfile in Os.listdir (basedir): Temp_path=os.path.join (basedir,subfile ) Tp=os.path.join (fix,subfile) if TP in Ignore_names:continue if TP in Ignore_paths:continue if Os.path.isfile (t Emp_path): Rlist.append (TP) elif Os.path.isdir (Temp_path): Get_files (Temp_path,tp,rlist,ignore_paths,ignore_names) Return rlistdef Get_valid_key_files (subdir= "): Basedir=subdir or basedir files = get_files (basedir=basedir,ignore_ Paths=ignore_paths,ignore_names=ignore_names) return map (lambda F: (f.replace ("\ \", "/"), F), files) def sync (): Qn_keys =list_all (Bucket_name,bucket) qn_set=set (Qn_keys) l_key_files=get_valid_key_files (basedir) k2f={} update_keys=[] U_ count=500 u_index=0 for k,f in l_key_files:k2f[k]=f str_k=k if Isinstance (k,str): K=k.decode (CharSet) if K in Qn_se     T:update_keys.append (str_k) u_index+=1 if U_index > U_count:u_index-=u_count update_file (K2f,update_keys) Update_keys=[] Else: # upload Upload_fIle (K,os.path.join (basedir,f)) if Update_keys:update_file (k2f,update_keys) print "Sync End" Def update_file (k2f,ulist  ): Ops=qiniu.build_batch_stat (bucket_name,ulist) Rets,infos = Bucket.batch (OPS) for I in Xrange (Len (ulist)): K=ulist[i] F=k2f.get (k) ret=rets[i]["data"] size=ret.get ("Fsize", None) put_time = Int (Ret.get ("Puttime")/10000000) local_size=o S.path.getsize (f) local_time=int (Os.path.getatime (f)) if Local_size==size:continue if Put_time >= local_time-di Ff_time: # is new continue # Update Upload_file (K,os.path.join (basedir,f)) def upload_file (key,localfile): print "UPL Oad_file: "Print Key token = Q.upload_token (Bucket_name, key) Mime_type = Get_mime_type (localfile) params = {' x:a ': ' A '} p Rogress_handler = lambda Progress, total:progress ret, info = qiniu.put_file (token, key, LocalFile, params, Mime_type, PR Ogress_handler=progress_handler) def get_mime_type (path): Mime_type = "Text/plain" return Mime_typedef Main (): Sync () if __name__== "__main__": Main ()

This synchronization script supports batch comparison of files, differential incremental updates, and batch updates.

How to use

Installing the seven KN python SDK

Pip Install Qiniu

Fill in the configuration information for the script file (qiniusync.py)

Access_key = ' Secret_key = ' Bucket_name = '

You can get the corresponding information after registering.

Copy the script file (qiniusync.py) to the root you want to synchronize

Run the script

Python qiniusync.py

Postscript

After writing the submission only found that seven cows have provided the appropriate tools, I am a repeat of the wheel it.

Since it has been written, it is issued, as familiar with the seven-ox SDK is also good, perhaps in the future can be used.

Seven Qiniu storage Python SDK Usage tutorials

This tutorial is intended to show you how to use the seven kn Python sdk to quickly file upload, download, process, manage, and more.

Installation

First, the SDK to use Python must be installed first. The seven-Ox Python SDK is open source, hosted on GitHub and has a project address of HTTPS://GITHUB.COM/QINIU/PYTHON-SDK.

The installation can be done using PIP install Qiniu, as stated in the project description. Of course, you can directly clone a copy of the source code down the direct use. I generally like the direct clone source code, so if you want to make some changes to the SDK is very easy.

The latest version of the Python SDK relies on the requests library, so it needs to be installed in advance. The installation method can also use PIP install requests.

Development environment

Python's development environment has a lot of choices, if you like the way text, such as vim,emacs,sublime text is a good choice, if you like the IDE, then the most popular is pycharm. The latest version of Pycharm to download here.

Access Key and secret key

We know the seven Qiniu. The permissions check mechanism is based on a pair of keys, called access key and Secret key, respectively. Where Access key is the public key, Secret key is the private key. This pair of keys can be obtained from the background of seven cows.

Small trial Sledgehammer

Well, doing the above preparatory work, we will go to upload a simple file, practice practiced hand.

python#coding=utf-8__author__ = ' Jemy '

This example demonstrates a simple file upload.

In this example, the SDK chooses whether to upload a form or a shard based on the size of the file.

"Import Qiniuaccesskey ="
 
  
   
  "Secretkey ="
  
   
    
   "#解析结果def parseret (Retdata, Respinfo): If Retdata ! = None:print ("Upload file success!") print ("hash:" + retdata["hash"]) print ("Key:" + retdata["key"]) #检查扩展参数 for K, V  In Retdata.items ():  if k[:2] = = "x:":  print (k + ":" + V) #检查其他参数 for K, V in Retdata.items ():  if k[:2] = = "x:" or k = = "Hash" or k = = "key":  continue  Else:  print (k + ":" + str (v)) else:print ("Upload file failed!") Print ("Error:" + respinfo.text_body) #无key上传, the key parameter is not specified in the HTTP request def upload_without_key (bucket, FilePath): #生成上传凭证 auth = Qiniu. Auth (AccessKey, secretkey) Uptoken = Auth.upload_token (bucket, key=none) #上传文件 retdata, respinfo = Qiniu.put_file ( Uptoken, None, FilePath) #解析结果 Parseret (Retdata, Respinfo) def main (): bucket = "IF-PBL" FilePath = "/users/jemy/documents/ Jemy.png "Upload_without_key (bucket, filePath) if __name__ = =" __main__ ": Main (
  )
   
 
  

The result of the operation is:

Upload file success!
Hash:fp0xr6tm4yzmeikxw7ezzmeyysq8
Key:fp0xr6tm4yzmeikxw7ezzmeyysq8

As we can see from the above, the most basic steps to uploading a file using the seven KN Python SDK are:

1. Generate Upload Voucher

2. Uploading Files

3. Parse the reply result

Summary

To sum up, in fact, the use of seven Ox SDK to upload files is very simple, the next tutorial, we will be on the basis of this example to learn more about file upload knowledge.

