Python Memory Database Pydblite

Pure-python engine

Recently, due to the discovery of the Python informixdb module in the project development of multi-threaded support is very bad, when the two threads connected to the Informix database at the same time, the database will error, showing that SQL process is in progress, according to Python Multi-threading mechanism we suspect that when the database is connected, Informix identifies two threads of the cursor as the same, so the database error. All operations on the database are changed to multiple processes through Python in the Multiprocess module.

But this leads to another problem, in the case of previous multithreading, a queue is maintained in the project, which stores a number of instance information that has been created for the Informix tenant pool, for customers to quickly get the available database resources. But with a multi-process presence, each time a client takes a instance, the main process needs to take one out of the queue, and the take () action triggers the creation of a tenant instance, Contains a process that stores instance information in the queue again. This will involve the issue of inter-process communication between queue. You need to change the queue to Multiprocess.queue to avoid data loss. Here I want to try the steps of using a memory database to try to simplify the process communication.

Import class fromBasemodule Pydblite: from pydblite import Base

Create a database instance, passing it a path in the file system: db = Base (' dummy ')

For a new database, define the field names: db.create (' name ', ' age ', ' size ')

You don't have to define the field types. Pydblite would accept any value that can is serialized bycPicklethe module:strings, Unicode strings, integers, floats, da TES and DateTimes (instances ofdatedatetimedatetimethe and classes in the module), user-defined classes, etc

If the base exists, open it: db.open ()

you can pass a parameter "mode" to the Create () method, to specify what's want to do if the B ASE already exists in the file system

mode = "Open":  db.create (' name ', ' age ', ' size ', Mode= "Open")  opens the database and ignores the field definition

mode = "override":  db.create (' name ', ' age ', ' size ', mode= "override")  erases the existing base and creates a new one with the F Ield definition

If mode is not specified and the base already exists, an IOError  is raised

Insert a new record

By keywords: db.insert (name= ' Homer ', age=23,size=1.84)

If Some fields is missing, they is initialized with the valueNone

By positional arguments: db.insert (' Homer ', 23,1.84)

The arguments must is provided in the same order as in thecreate()method

Save the changes on disk: Db.commit ()

If you don ' t commit the changes, the insertion, deletion and update operations'll is not being saved on disk. To return to the previous version, just open () It again (this was equivalent to rollback in transactional databases)

Besidescreate()the passed to the method, an internal field called is__id__added. It is a integer which are guaranteed to being unique and unchanged for each record in the base so that it can used as the Record identifier

Another internal field called is__version__also managed by the database engine. It is a integer which are set to 0 when the record is created, then incremented by 1 each time the record is updated. This is used to detect concurrency control, for instance in a Web application where 2 users select the same record and WAN T to update it at the same time

The selection of records uses Python list comprehension syntax:

RECs = [R for r in db if > r[' age '] >= and r[' size '] < 2]

Returns the records in the base where the an is between, and a size is below 2 meters. The record is a dictionary, where the key is the field name and value are the field value

Python Generator expression syntax can also be used:

For R in (R for R in db if r[' name "] in (' Homer ', ' Marge ')):

Do_something_with (R)

Iterates on the records where the name is one of ' Homer ' or ' Marge '

To iterate on all the records:

for R in DB:

Do_something_with (R)

A record can be accessed by it identifier: record = db[rec_id] Returns the record such that record[' __id__ '] = = Rec_ ID

Finally, a shortcut can be used for simple selections: db (Key1=val1,key2=val2) returns the list of records where the Keys take the given value. It is equivalent to [R for R in db if r[' Key1 "]==val1 and r[" Key2 "]==val2], but much more concise

To speed up selections, an index can is created on a field: db.create_index (' age ')

When a index is created, the database instance have an attribute (here_age: note the heading underscore, to avoid name conflicts with internal names). This attribute is a Dictionary-like object, where keys was the values taken by the field, and values were the the records whose Field values is egal to the key:

Records = db._age[23] Returns the list of records with age = = 23

If no record has the this value, the lookup by this value returns an empty list

The index supports iteration on the field values, and thekeys()method returns all existing values for the field

Number of records in the base: len (db)

To delete a record: Db.delete (record) or, if know the record identifier: del db[rec_id]

To delete a list of records: Db.delete (List_of_records)

list_of_recordsCan is any iterable (list, tuple, set, etc) yielding records

To update a record: db.update (record,age=24)

To add a new field to an existing base and specify a default value: Db.add_field (' New_field ' [, Default=v]). If No default is provided, the field valueNoneis

To drop an existing field: Db.drop_field (' name ')

To get the list of fields: db.fields

import pydblite
# Use in-memory database
pydb = pydblite.Base ("address")
# Create three fields a, b, and c
pydb.create (‘a’, ‘b’, ‘c’)
# Create indexes for fields a and b
pydb.create_index (‘a’, ‘b’)
# Insert a piece of data

pydb.insert (a = 0, b = 0, c = 1)
pydb.insert (a = 1, b = 0, c = 1)
pydb.insert (a = 1, b = 0, c = 1)
pydb.insert (a = 1, b = 0, c = 1)
pydb.update (records = pydb [1], a = 2, c = "li")
pydb.delete (pydb [0])
# Query data that meets specific requirements
results = pydb (a = 2)
for i in results:
     print results [i] 

Python Memory Database Pydblite