Shelve, shelves

Shelve, shelves

These days, I have come into contact with the shelve module in Python, which is easier to use than pickle. It is also a simple tool for persistence of Python objects. If you don't want to use a relational database to store data when writing a program, try shelve. Shelf is also accessed with a key, which is similar to a dictionary. Shelve uses anydbm to create databases and manage persistent objects.

 

Create a new shelf

You can directly use shelve. open () to create

1 import shelve2 3 s = shelve.open('test_shelf.db')4 try:5     s['key1'] = { 'int': 10, 'float':9.5, 'string':'Sample data' }6 finally:7     s.close()

 

If you want to access this shelf again, you only need to re-access the shelve. open (). Then we can use this shelf like using a dictionary.

1 import shelve2 3 s = shelve.open('test_shelf.db')4 try:5     existing = s['key1']6 finally:7     s.close()8 9 print existing

 

When we run the above two py files, we will get the following output:

$ python shelve_create.py$ python shelve_existing.py{'int': 10, 'float': 9.5, 'string': 'Sample data'}

 

The dbm module has a limitation. It does not support multiple applications to write data to the same database at the same time. So when we know that our application only performs read operations, we can let shelve open the database in read-only mode:

1 import shelve2 3 s = shelve.open('test_shelf.db', flag='r')4 try:5     existing = s['key1']6 finally:7     s.close()8 9 print existing

When our program tries to modify a database opened in read-only mode, an access error will be thrown. The specific type of the exception depends on the DB used by the anydbm module when creating the database.

 

Write-back)

Because shelve does not record any modifications to the objects to be persisted by default, we need to modify the default parameters during shelve. open (). Otherwise, the changes to the objects will not be saved.

 1 import shelve 2  3 s = shelve.open('test_shelf.db') 4 try: 5     print s['key1'] 6     s['key1']['new_value'] = 'this was not here before' 7 finally: 8     s.close() 9 10 s = shelve.open('test_shelf.db', writeback=True)11 try:12     print s['key1']13 finally:14     s.close()

In the above example, because the default parameter shelve. open () is used at the beginning, the modified value of row 6th will not be saved even if s. close () is used.

The execution result is as follows:

$ python shelve_create.py$ python shelve_withoutwriteback.py{'int': 10, 'float': 9.5, 'string': 'Sample data'}{'int': 10, 'float': 9.5, 'string': 'Sample data'}

 

So when we try to allow shelve to automatically capture object changes, we should set writeback to True when we open shelf. When we set the flag of writeback to True, shelf will store all the objects read from the DB to a memory cache. When we close () to open the shelf, all objects in the cache will be re-written to the DB.

 1 import shelve 2  3 s = shelve.open('test_shelf.db', writeback=True) 4 try: 5     print s['key1'] 6     s['key1']['new_value'] = 'this was not here before' 7     print s['key1'] 8 finally: 9     s.close()10 11 s = shelve.open('test_shelf.db', writeback=True)12 try:13     print s['key1']14 finally:15     s.close()

The writeback method has advantages and disadvantages. The advantage is that it reduces the probability of errors and makes object persistence more transparent to users. However, this method is not required in all cases. First, after using writeback, shelf will increase memory consumption during open (), and when DB closes (), it will write every object in the cache to the DB, this will also lead to additional waiting time. Because shelve has no way to know which objects are modified in the cache and which objects are not modified, all objects will be written.

1 $ python shelve_create.py2 $ python shelve_writeback.py3 4 {'int': 10, 'float': 9.5, 'string': 'Sample data'}5 {'int': 10, 'new_value': 'this was not here before', 'float': 9.5, 'string': 'Sample data'}6 {'int': 10, 'new_value': 'this was not here before', 'float': 9.5, 'string': 'Sample data'}

 

The following is a complex example:

 1 #!/bin/env python 2  3 import time 4 import datetime 5 import md5 6 import shelve 7  8 LOGIN_TIME_OUT = 60 9 db = shelve.open('user_shelve.db', writeback=True)10 11 def newuser():12     global db13     prompt = "login desired: "14     while True:15         name = raw_input(prompt)16         if name in db:17             prompt = "name taken, try another: "18             continue19         elif len(name) == 0:20             prompt = "name should not be empty, try another: "21             continue22         else:23             break24     pwd = raw_input("password: ")25     db[name] = {"password": md5_digest(pwd), "last_login_time": time.time()}26     #print '-->', db27 28 def olduser():29     global db30     name = raw_input("login: ")31     pwd = raw_input("password: ")32     try:33         password = db.get(name).get('password')34     except AttributeError, e:35         print "\033[1;31;40mUsername '%s' doesn't existed\033[0m" % name36         return37     if md5_digest(pwd) == password:38         login_time = time.time()39         last_login_time = db.get(name).get('last_login_time')40         if login_time - last_login_time < LOGIN_TIME_OUT:41             print "\033[1;31;40mYou already logged in at: <%s>\033[0m" % datetime.datetime.fromtimestamp(last_login_time).isoformat()42 43         db[name]['last_login_time'] = login_time44         print "\033[1;32;40mwelcome back\033[0m", name45     else:46         print "\033[1;31;40mlogin incorrect\033[0m"47 48 def md5_digest(plain_pass):49    return md5.new(plain_pass).hexdigest()50 51 def showmenu():52     #print '>>>', db53     global db54     prompt = """55 (N)ew User Login56 (E)xisting User Login57 (Q)uit58 Enter choice: """59     done = False60     while not done:61         chosen = False62         while not chosen:63             try:64                 choice = raw_input(prompt).strip()[0].lower()65             except (EOFError, KeyboardInterrupt):66                 choice = "q"67             print "\nYou picked: [%s]" % choice68             if choice not in "neq":69                 print "invalid option, try again"70             else:71                 chosen = True72 73         if choice == "q": done = True74         if choice == "n": newuser()75         if choice == "e": olduser()76     db.close()77 78 if __name__ == "__main__":79     showmenu()