1. Preparations
After installing pymongo through pip or easy_install, you can use Python to call mongodb.
Then install a flask to serve as a web server.
Of course, mongo has to be installed. For Ubuntu users, especially those who use Server 12.04, it takes a little time to install the latest version.
Sudo apt-key adv -- keyserver hkp: // keyserver.ubuntu.com: 80 -- recv 7F0CEB10
Echo 'destroy http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen '| sudo tee/etc/apt/sources. list. d/mongodb. list
Sudo apt-get update
Sudo apt-get install mongodb-10gen
If you think that, just like me, you can tell users by uploading the file name suffix that the uploaded file is completely holding yam and deceiving yourself as a cucumber, you 'd better prepare a Pillow library.
Pip install Pillow
Or (more suitable for Windows users)
Easy_install Pillow
2. Opening scene
2.1 Flask file Upload
The example on the Flask official website has been split into two parts, which makes it impossible to speak out. Here we will first get the simplest one, regardless of what files are first made up.
Import flask
App = flask. Flask (_ name __)
App. debug = True
@ App. route ('/upload', methods = ['post'])
Def upload ():
F = flask. request. files ['uploaded _ file']
Print f. read ()
Return flask. redirect ('/')
@ App. route ('/')
Def index ():
Return '''
<! Doctype html>
<Html>
<Body>
<Form action = '/upload' method = 'post' enctype = 'multipart/form-data'>
<Input type = 'file' name = 'uploaded _ file'>
<Input type = 'submit 'value = 'upload'>
</Form>
'''
If _ name _ = '_ main __':
App. run (port = 7777)
Note: in the upload function, use flask. request. files [KEY] to obtain the object to be uploaded. The KEY is the name value of the input in the form.
Because the content is output in the background, it is best to test it with a plain text file.
2.2 save to mongodb
If you are not so specific, you only need
Import pymongo
Import bson. binary
From cStringIO import StringIO
App = flask. Flask (_ name __)
App. debug = True
Db = pymongo. MongoClient ('localhost', 27017). test
Def save_file (f ):
Content = StringIO (f. read ())
Db. files. save (dict (
Content = bson. binary. Binary (content. getvalue ()),
))
@ App. route ('/upload', methods = ['post'])
Def upload ():
F = flask. request. files ['uploaded _ file']
Save_file (f)
Return flask. redirect ('/')
Feed the content into a bson. binary. Binary object and then throw it into mongodb.
Now try to upload another file. In mongo shell, you can see it through db. files. find.
However, the content field is almost invisible to the naked eye. Even in plain text files, mongo will display Base64 encoding.
2.3 provide file access
Given the ID (as part of the URI) of the file stored in the database, the content of the file is returned to the browser, as shown below:
Def save_file (f ):
Content = StringIO (f. read ())
C = dict (content = bson. binary. Binary (content. getvalue ()))
Db. files. save (c)
Return c ['_ id']
@ App. route ('/f/<fid> ')
Def serve_file (fid ):
F = db. files. find_one (bson. objectid. ObjectId (fid ))
Return f ['content']
@ App. route ('/upload', methods = ['post'])
Def upload ():
F = flask. request. files ['uploaded _ file']
Fid = save_file (f)
Return flask. redirect ('/f/' + str (fid ))
After the file is uploaded, the upload function will jump to the corresponding file browsing page. In this way, the content of the text file can be properly previewed, if not so picky line breaks and consecutive spaces are eaten by the browser.
2.4 when the file cannot be found
In either case, the database ID format is incorrect. In this case, pymongo throws an exception bson. errors. InvalidId. Second, the object (!) cannot be found (!), In this case, pymongo will return None.
This is the case for simplicity.
@ App. route ('/f/<fid> ')
Def serve_file (fid ):
Import bson. errors
Try:
F = db. files. find_one (bson. objectid. ObjectId (fid ))
If f is None:
Raise bson. errors. InvalidId ()
Return f ['content']
Failed T bson. errors. InvalidId:
Flask. abort (404)
2.5 correct MIME
From now on, you have to strictly control the uploaded files. Text files, dogs and scissors cannot be uploaded.
Before determining the image file, let's say that we use Pillow for real scenarios.
From PIL import Image
Allow_formats = set (['jpeg ', 'PNG', 'GIF'])
Def save_file (f ):
Content = StringIO (f. read ())
Try:
Mime = Image. open (content). format. lower ()
If mime not in allow_formats:
Raise IOError ()
Handle T IOError:
Flask. abort (400)
C = dict (content = bson. binary. Binary (content. getvalue ()))
Db. files. save (c)
Return c ['_ id']
Then, try uploading a text file. The image file can be uploaded normally. no, it is not normal, because after the jump is completed, the server does not provide the correct mimetype, so we still preview a piece of binary garbled text.
To solve this problem, you must store the MIME in the database together. In addition, the mimetype must be correctly transmitted when the file is given.
Def save_file (f ):
Content = StringIO (f. read ())
Try:
Mime = Image. open (content). format. lower ()
If mime not in allow_formats:
Raise IOError ()
Handle T IOError:
Flask. abort (400)
C = dict (content = bson. binary. Binary (content. getvalue (), mime = mime)
Db. files. save (c)
Return c ['_ id']
@ App. route ('/f/<fid> ')
Def serve_file (fid ):
Try:
F = db. files. find_one (bson. objectid. ObjectId (fid ))
If f is None:
Raise bson. errors. InvalidId ()
Return flask. Response (f ['content'], mimetype = 'image/'+ f ['Mime'])
Failed T bson. errors. InvalidId:
Flask. abort (404)
Of course, the original object does not have the mime attribute, so it is best to clear the original data with db. files. drop () in mongo shell first.
2.6 not modified based on the upload time
HTTP 304 not modified can be used to squeeze the browser cache and save bandwidth as much as possible. Three operations are required.
Record the last File upload time
When the browser requests this file, a timestamp string is inserted into the request header.
When the browser requests a file, it tries to get the timestamp from the request header. If it is consistent with the file timestamp, it will directly 304
The code is
Import datetime
Def save_file (f ):
Content = StringIO (f. read ())
Try:
Mime = Image. open (content). format. lower ()
If mime not in allow_formats:
Raise IOError ()
Handle T IOError:
Flask. abort (400)
C = dict (
Content = bson. binary. Binary (content. getvalue ()),
Mime = mime,
Time = datetime. datetime. utcnow (),
)
Db. files. save (c)
Return c ['_ id']
@ App. route ('/f/<fid> ')
Def serve_file (fid ):
Try:
F = db. files. find_one (bson. objectid. ObjectId (fid ))
If f is None:
Raise bson. errors. InvalidId ()
If flask. request. headers. get ('if-Modified-Since ') = f ['Time']. ctime ():
Return flask. Response (status = 304)
Resp = flask. Response (f ['content'], mimetype = 'image/'+ f ['Mime'])
Resp. headers ['last-modified'] = f ['Time']. ctime ()
Return resp
Failed T bson. errors. InvalidId:
Flask. abort (404)
Then, you have to get a script to add a timestamp to the existing image in the database.
In addition, NoSQL DB does not have any advantages in this environment, and it is almost the same as RDB.
2.7 use SHA-1 to remove duplicates
Unlike Cola in the refrigerator, in most cases, you certainly do not want a big wave of identical images in the database. images, along with their EXIFF and other data information, should be unique in the database. In this case, it is better to use a slightly stronger hash technology to detect images.
To achieve this goal, the simplest thing is to create a unique SHA-1 index, so that the database will prevent the same thing from being put in.
Create a unique index in the MongoDB table and execute it (Mongo console)
Db. files. ensureIndex ({sha1: 1}, {unique: true })
If your database contains multiple records, MongoDB reports an error. this seemingly harmonious and harmless index operation is told that there are repeated null values in the database (in fact, the existing entries in the database do not have this attribute at all ). different from general RDB, MongoDB requires null or non-existent attribute values, so these ghost attributes will cause the unique index to fail to be created.
There are three solutions:
Delete all the data now (it must be an irresponsible way to test the database !)
Create a sparse index. This index does not require the ghost attribute to be unique. However, when multiple null values exist, it will still be determined to be repeated (this can be done regardless of the existing data)
Write a script to run the database once, translate all the stored data, re-calculate the SHA-1, and then store it in.
The specific method is random. Assuming that the problem has been fixed and the index has been completed, the rest is the Python code.
Import hashlib
Def save_file (f ):
Content = StringIO (f. read ())
Try:
Mime = Image. open (content). format. lower ()
If mime not in allow_formats:
Raise IOError ()
Handle T IOError:
Flask. abort (400)
Sha1 = hashlib. sha1 (content. getvalue (). hexdigest ()
C = dict (
Content = bson. binary. Binary (content. getvalue ()),
Mime = mime,
Time = datetime. datetime. utcnow (),
Sha1 = sha1,
)
Try:
Db. files. save (c)
Failed T pymongo. errors. DuplicateKeyError:
Pass
Return c ['_ id']
It's okay to upload files. however, according to the above logic, if an existing file is uploaded, the returned c ['_ id'] will be a non-existent data id. to solve this problem, it is best to return sha1. In addition, when accessing a file, modify it to access the file SHA-1 instead of the ID.
The final modification result and complete source code of this article are as follows:
Import hashlib
Import datetime
Import flask
Import pymongo
Import bson. binary
Import bson. objectid
Import bson. errors
From cStringIO import StringIO
From PIL import Image
App = flask. Flask (_ name __)
App. debug = True
Db = pymongo. MongoClient ('localhost', 27017). test
Allow_formats = set (['jpeg ', 'PNG', 'GIF'])
Def save_file (f ):
Content = StringIO (f. read ())
Try:
Mime = Image. open (content). format. lower ()
If mime not in allow_formats:
Raise IOError ()
Handle T IOError:
Flask. abort (400)
Sha1 = hashlib. sha1 (content. getvalue (). hexdigest ()
C = dict (
Content = bson. binary. Binary (content. getvalue ()),
Mime = mime,
Time = datetime. datetime. utcnow (),
Sha1 = sha1,
)
Try:
Db. files. save (c)
Failed T pymongo. errors. DuplicateKeyError:
Pass
Return sha1
@ App. route ('/f/<sha1> ')
Def serve_file (sha1 ):
Try:
F = db. files. find_one ({'sha1': sha1 })
If f is None:
Raise bson. errors. InvalidId ()
If flask. request. headers. get ('if-Modified-Since ') = f ['Time']. ctime ():
Return flask. Response (status = 304)
Resp = flask. Response (f ['content'], mimetype = 'image/'+ f ['Mime'])
Resp. headers ['last-modified'] = f ['Time']. ctime ()
Return resp
Failed T bson. errors. InvalidId:
Flask. abort (404)
@ App. route ('/upload', methods = ['post'])
Def upload ():
F = flask. request. files ['uploaded _ file']
Sha1 = save_file (f)
Return flask. redirect ('/f/' + str (sha1 ))
@ App. route ('/')
Def index ():
Return '''
<! Doctype html>
<Html>
<Body>
<Form action = '/upload' method = 'post' enctype = 'multipart/form-data'>
<Input type = 'file' name = 'uploaded _ file'>
<Input type = 'submit 'value = 'upload'>
</Form>
'''
If _ name _ = '_ main __':
App. run (port = 7777)
3. REF
Developing RESTful Web APIs with Python, Flask and MongoDB
Http://www.slideshare.net/nicolaiarocci/developing-restful-web-apis-with-python-flask-and-mongodb
Https://github.com/nicolaiarocci/eve
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
