A tutorial for transactional transaction management using the Python Django framework

If you spend a lot of time working on Django database transactions, you'll see that it's confusing.

In the past, just providing a simple base document, to know exactly how it was used, must also be done by creating and executing Django transactions.

There are a number of Django transaction nouns, such as commit_on_success, commit_manually, Commit_unless_maneged,rollback_unless_managed,enter_ Transaction_management,leace_transaction_management, these are just a few of them.

The most fortunate thing is that after the Django 1.6 release. Now you can use a few functions to implement the transaction, before we enter a few seconds to learn these functions. First, we want to make clear the following questions:

• What is transaction processing?
• What's new in Django 1.6 that deals with priorities?

Before entering the "How to make the correct transaction in Django 1.6 version", please look at the following detailed case:

Ribbon case

Transaction

• Recommended Way
• Using separators
• Every HTTP request transaction

Save Point

Nested transactions

What is a transaction?

According to SQL-92, "a SQL transaction (sometimes referred to as a transaction) is a sequence of SQL statement execution that can be atomically recovered." In other words, all SQL statements are executed and committed together. Similarly, when rolling back, all statements are rolled back.

For example:

# startnote = Note (title= "My first Note", text= "yay!") Note = Note (title= "My second note", text= "whee!") Address1.save () Address2.save () # COMMIT

So a transaction is a single unit of work in the database. It is started by a start transaction and a commit or explicit rollback ends.

What's wrong with the transaction management before Django 1.6?

To answer this question completely, we have to explain how transactions are handled in the database, client, and Django.

Database

Each statement in the database runs in a single transaction, which can even contain only one statement.

Almost all databases have a autocommit setting, which is usually set to True by default. Autocommit wraps all the statements into a single transaction, and as long as the statement executes successfully, the transaction is immediately committed. Of course you can also call start_transaction manually, which will temporarily suspend autocommit until you call commit_transaction or rollback.

This, in turn, will invalidate the implicit submission of the AUTOCOMMIT setting after each statement.

However, there are Python client libraries such as Sqlite3 and mysqldb that allow Python programs to connect to the database itself. These libraries follow a set of criteria for how to access and query a database. The DB API 2.0 standard, described in Pep 249. Although it may make people read a little bit more dry. One important thing to take away is that in THE PEP 249 State, the default database should turn off the autocommit feature.

This clearly conflicts with what's happening in the database:

• Database statements are always run in one transaction. This database will typically turn on auto-submit for you.
• However, according to Pep 249, this should not happen.
• What happens to the client library that must be reflected in the database? However, because they do not allow the auto-submit feature to be turned on by default. They simply wrap the SQL statements in a single transaction. Just like a database.

All right, stay with me for a while.

Django

Entering the Django,django also has to say about transaction processing. In Django 1.5 and earlier versions. When you write data to a database, Django basically runs an open transaction and automatically submits the transaction functionality. So every time you name something like Model.save () or Model.update (), Django generates the appropriate SQL statement and commits the transaction.

There are also in Django 1.5 and earlier versions, it is recommended that you use Transactionmiddleware to bind HTTP request transactions. Each request provides a transaction. If the response returned is not an exception, Django commits the transaction. But if your view function throws an error, the rollback will be called. This actually means that it turns off the auto-commit feature. If you want to standardize, the database level automatically submits the style of transaction management, you have to manage your own transactions-usually by using transactions to decorate your view features, such as @transaction.commit_manually, or @transaction.commit_on_success.

Breathe a breath, or two mouth.

What does that mean?

Yes, there are a lot of things to do, and it turns out that most developers are in need of this standard database-level autocommit feature-meaningful transactions are often left behind. Do your own thing until you need to manually adjust them.

What is correct about transaction management in the Django 1.6 release?

Now, welcome to Django 1.6. Try to forget everything, we just talk about it, just remember in Django 1.6, you can use the database, and you can manually commit and manage transactions automatically when needed. In essence, we have a simpler model, basically to put the design of what kind of database in the first place.

All right! Let's write the code, okay?

Stripe case

Below, we use an example of handling a user registration, calling stripe to handle the credit card process.

def register (Request): User = None if Request.method = = ' POST ': Form = UserForm (Request.       POST) if Form.is_valid (): Customer = customer.create ("subscription", email = form.cleaned_data[' email '),       Description = form.cleaned_data[' name '], card = form.cleaned_data[' Stripe_token '], plan= "Gold",) cd = Form.cleaned_data Try:user = user.create (cd[' name '), cd[' email ', cd[' password '], cd[' Last_4_digits ']) if customer:user.stripe_id = Customer.id user.save () ELSE:UNP       Aidusers (email=cd[' email '). Save () except IntegrityError:form.addError (cd[' email ' + ' is already a member ')   else:request.session[' user '] = user.pk return httpresponseredirect ('/') Else:form = UserForm () Return Render_to_response (' register.html ', {' form ': Form, ' months ': Range (1, +), ' publishable ': Sett Ings. Stripe_publishable, ' soon ': Soon (), ' useR ': User, ' Years ': Range (2036),}, Context_instance=requestcontext (Request)) 

The example calls Customer.create first, actually calls Stripe to handle the credit card process, and then we create a new user. If we get a response from stripe, we'll update the newly created user with stripe_id. If we don't get a response (stripe is off), we'll add a new entry to the Unpaidusers table with the newly created user's email, so we can have them retry their credit card information later.

The idea is this: if Stripe doesn't respond, the user can still sign up and start using our website. We will let the user provide the credit card information at a later time.

"I understand that this is a special example, and this is not the way I want to do the function, but it is intended to show the trade"

Consider trading, keeping in mind that the "autocommit" functionality of the database is provided in Django1.6. Next look at the database-related code:

cd = form.cleaned_datatry:  user = user.create (cd[' name '], cd[' email '), cd[' password '], cd[' last_4_digits ')   If customer:    user.stripe_id = customer.id    user.save ()  Else:    unpaidusers (email=cd[' email '). Save () Except Integrityerror:

Can you see the problem? What happens if "unpaidusers (email=cd[' email"). Save () "fails to run?

There was a user who registered the system, and then the system thought that the credit card had been checked. But in fact, the system has not been checked.

We just want one of the results:

1. Create a user in the database, and have the stripe_id

2. The user was created in the database, but there is no stripe_id. At the same time, the same e-mail address exists in the relevant "unpaidusers" line.

This means that what we want is a separate database statement header to complete the task or rollback. This example is a good illustration of the deal.

First, let's write some test cases to verify that things work the way we think:

@mock. Patch (' payments.models.UnpaidUsers.save ', Side_effect = integrityerror) def test_registering_user_when_strip_ Is_down_all_or_nothing (self, save_mock): #create the request used to test the view self.request.session = {} self.requ Est.method= ' POST ' Self.request.POST = {' email ': ' python@rocks.com ', ' name ': ' Pyrock ', ' stripe_t  Oken ': ' ... ', ' last_4_digits ': ' 4242 ', ' Password ': ' Bad_password ', ' Ver_password ': ' Bad_password ',} #mock out stripe and ask it for throw a connection error with Mock.patch (' stripe. Customer.create ', Side_effect = Socket.error ("can ' t connect to Stripe")) as Stripe_mock: #run the test res    p = Register (self.request) #assert There is no record in the database without stripe ID. Users = User.objects.filter (email= "python@rocks.com") self.assertequals (Len (users), 0) #check The associated table Also didn ' t get updated unpaid = UnpaidUsers.objects.filter (Email= "python@rocks.com") self.assertequals (len (unpaid), 0) 

When we try to save "unpaidusers", the interpreter above the test will run out of the exception ' Integrityerror '.

The next step is to explain the answer to this question, "when" unpaidusers (email=cd[' email '). Save () "What happened at the time of the run?" "The following section of code creates a conversation, and we need to give some appropriate information in the registration function." Then "with Mock.patch" forces the system to think that stripe is not responding and eventually jumps to our test case.

RESP = Register (self.request)

The above remark simply calls our registered view to pass the request. Then we just need to check to see if there are any updates:

#assert there is no record in the database without stripe_id.users = User.objects.filter (email= "python@rocks.com") self.as Sertequals (users), 0) #check The associated table also didn ' t get Updatedunpaid = UnpaidUsers.objects.filter (email= "p Ython@rocks.com ") self.assertequals (len (unpaid), 0)

So if we run the test case, it will fail to run:

======================================================================fail:test_registering_user_when_strip_is _down_all_or_nothing (tests.payments.testViews.RegisterPageTests)---------------------------------------------- ------------------------Traceback (most recent): File "/users/j1z0/.virtualenvs/django_1.6/lib/python2.7/ site-packages/mock.py ", line 1201, in patched  return func (*args, **keywargs) File"/USERS/J1Z0/CODE/REALPYTHON/MVP _for_adv_python_web_book/tests/payments/testviews.py ", line 266, in Test_registering_user_when_strip_is_down_all_ Or_nothing  self.assertequals (len (users), 0) Assertionerror:1! = 0--------------------------------------------- -------------------------

Praise. That's what we want to end up with.

Remember: We have practiced the "test-driven development" capability here. The error message prompts US: The user information has been saved to the database, but this is not what we want, because we do not pay!

Transactional transactions are used to save such problems ...

Transaction

For Django1.6, there are many ways to create transactions.

Here is a brief introduction to several.
Recommended method

According to Django1.6 's documentation, "Django provides a simple API to control transactional transactions of the database ... Atomic operations are used to define the properties of a database transaction. Atomic operations allow us to create a bunch of code with the assurance of a database. If the code is executed successfully, the corresponding changes are committed to the database. If an exception occurs, the operation is rolled back. ”

Atomic operations can be used to interpret operations or content management. So if we use it as content management, the code for our registration function will be as follows:

From django.db Import transaction try:  with Transaction.atomic ():    user = user.create (cd[' name '], cd[' email '), cd[' password '], cd[' last_4_digits '))     if customer:      user.stripe_id = customer.id      user.save ()    Else:      unpaidusers (email=cd[' email '). Save () except Integrityerror:  form.adderror (cd[' email ') + ' is already a Member ')

Notice the line in "with Transaction.atomic ()". This code will be executed inside the transaction. So if we rerun our tests, they will all pass.

Remember: A transaction is a unit of work, so when the "unpaidusers" call fails, all content management operations are rolled back together.

Using adorners

In addition to the above approach, we can use the Python adorner feature to use the transaction.

@transaction. Atomic ():d EF Register (Request):  snip ....   Try:    user = user.create (cd[' name '], cd[' email '), cd[' password '], cd[' last_4_digits '])     if customer:      user.stripe_id = customer.id      user.save ()    Else:        unpaidusers (email=cd[' email '). Save ()   except Integrityerror:    form.adderror (cd[' email ') + ' is already a member ')

If we run a test again, it will still be the same failure.

Why is it? Why does the transaction not roll back correctly? The reason is that with transaction.atomic will try to catch some kind of exception, and our code in human capture (such as try-except code block Integrityerror exception), so transaction.atomic never see this exception, So the standard autocommit process is void.

However, deleting the Try-catch statement causes the exception to not be caught, and then the code flow is lost in all likelihood. So, that means you can't get rid of Try-catch.

So, the trick is to put the atomic context manager into our Try-catch code snippet in the first solution.

Take a look at the correct code:

From django.db Import transaction try:  with Transaction.atomic ():    user = user.create (cd[' name '], cd[' email '), cd[' password '], cd[' last_4_digits '))     if customer:      user.stripe_id = customer.id      user.save ()    Else:      unpaidusers (email=cd[' email '). Save () except Integrityerror:  form.adderror (cd[' email ') + ' is already a Member ')

When unpaidusers triggers Integrityerror, the context Manager transaction.atomic () captures it and performs a rollback operation. At this point our code executes in exception handling (theform.adderrorline), the rollback operation is completed, and the database call can be made safely if necessary. Also note that any database call before or after the context Manager thetransaction.atomic () will not be affected by the result of its execution.

Transactional transactions for each HTTP request

Both Django1.5 and version 1.6 allow the user to manipulate the request transaction mode. In this mode, Django will automatically process your view functions in the transaction. If the view function throws an exception, Django automatically rolls back the transaction, otherwise Django commits the transaction.

In order to do this, you need to set "Atomic_request" to true in the configuration of the database you want to have this capability. So in our "settings.py" need to have the following settings:

DATABASES = {'  default ': {    ' ENGINE ': ' Django.db.backends.sqlite3 ',    ' NAME ': Os.path.join (Site_root, ' Test.db '),    ' atomic_request ': True,  }}

If we put the interpreter in the view function, the above setting will take effect. So that doesn't fit our idea.

However, it is still worth noting that the interpreter: "Atomic_requests" and "@transaction. ATOMIC" are still likely to handle these errors when there are exceptions thrown. To capture these errors, you need to complete some regular middleware, or you need to overwrite "urls.hadler500" or create a new "500.html" template.

Save Point

Although the transaction is atomic, it can be broken up into multiple "savepoint"-you can be seen as a "partial transaction."

For example, you have a transaction that contains 4 SQL statements, and you can create a savepoint after the second SQL. Once the savepoint is created successfully, even if the third or fourth SQL execution fails, you can still do a partial rollback, ignoring the next two SQL, leaving only the previous two.

Basically, it's like providing a cutting capability: a trivial transaction can be cut into more "lightweight" transactions and then partially rolled back or partially committed.

Be careful, however, that the entire transaction is rolled back for no reason (for example, by throwing a integrityerror exception but not catching it, all the savepoint will be rolled back)

Take a look at the sample code to learn how to play the save point.

@transaction. Atomic () def save_points (self,save=true):   user = User.create (' JJ ', ' Inception ', ' JJ ', ' 1234 ')  SP1 = Transaction.savepoint ()   user.name = ' Zheli hui Guadiao, t.t '  user.stripe_id = 4  user.save ()   if Save:    transaction.savepoint_commit (SP1)  else:    transaction.savepoint_rollback (SP1)

In the example, the entire function belongs to a transaction. After the new user object is created, we create and get a savepoint. Then the next 3 lines of statements--

  User.Name = ' Zheli hui Guadiao, t.t '  user.stripe_id = 4  user.save ()

--not belonging to the save point just now, so they may be part of the following savepoint_rollback or Savepoint_commit. Assuming it is savepoint_rollback, the code line User = User.create (' JJ ', ' Inception ', ' JJ ', ' 1234 ') will still be successfully committed to the database, and the following three lines will not succeed.

Using another approach, the following two test cases describe how the SavePoint works:

def test_savepoint_rollbacks (self):   self.save_points (False)   #verify This everything was stored  users = User.objects.filter (email= "Inception")  Self.assertequals (len (Users), 1) #note The values here is from the   Original Create call  self.assertequals (users[0].stripe_id, ')  self.assertequals (Users[0].name, ' JJ ')  def test_savepoint_commit (self):  self.save_points (True)   #verify This everything was stored  users = User.objects.filter (email= "Inception")  Self.assertequals (len (Users), 1) #note The values here is from the   Update calls  self.assertequals (users[0].stripe_id, ' 4 ')  self.assertequals (Users[0].name, ' starting down The Rabbit Hole ')

Again, after we commit or roll back the savepoint, we can still continue working in the same transaction. At the same time, the result of this operation is not affected by the output of the previous save point.

For example, if we update the "save_points" function according to the following example,

@transaction. Atomic () def save_points (self,save=true):   user = User.create (' JJ ', ' Inception ', ' JJ ', ' 1234 ')  SP1 = Transaction.savepoint ()   user.name = ' Starting down the rabbit Hole '  user.save ()   user.stripe_id = 4  User.save ()   if Save:    transaction.savepoint_commit (SP1)  else:    Transaction.savepoint_ Rollback (SP1)   user.create (' Limbo ', ' illbehere@forever ', ' mind blown ',      ' 1111 ')

Even if "savepoint_commit" or "Savepoint_rollback" is called by the "limbo" user, the transaction will still be created successfully. If the creation is not successful, the entire transaction will be rolled back.

Nested transactions

Using "SavePoint ()", "Savepoint_commit" and "Savepoint_rollback" to manually specify the savepoint, a nested transaction is automatically created, and the nested transaction automatically creates a savepoint for us. And, if we encounter an error, the transaction will be rolled back.

Here is an example of an extension that illustrates:

@transaction. Atomic () def save_points (self,save=true):   user = User.create (' JJ ', ' Inception ', ' JJ ', ' 1234 ')  SP1 = Transaction.savepoint ()   user.name = ' Starting down the rabbit Hole '  user.save ()   user.stripe_id = 4  User.save ()   if Save:    transaction.savepoint_commit (SP1)  else:    Transaction.savepoint_ Rollback (SP1)   try:    with Transaction.atomic ():      user.create (' Limbo ', ' illbehere@forever ', ' mind Blown ',          ' 1111 ')      if not save:raise databaseerror  except Databaseerror:    Pass

Here we can see: after we have processed the savepoint, we use the context management area of "thetransaction.atomic" to wipe out the "limbo" user we created. When context management is called, it creates a savepoint (because we are already in the transaction) and the savepoint is executed or rolled back according to the existing context manager.

So the following two test cases describe the wording:

def test_savepoint_rollbacks (self): self.save_points (False) #verify This everything was stored users = User.ob Jects.filter (email= "Inception") Self.assertequals (Len (Users), 1) #savepoint was rolled back so we should had Origi  NAL values Self.assertequals (users[0].stripe_id, ') self.assertequals (Users[0].name, ' JJ ') #this save point was Rolled back because of databaseerror limbo = User.objects.filter (email= "Illbehere@forever") Self.assertequals (Len (l Imbo), 0) def test_savepoint_commit (self): self.save_points (True) #verify This everything was stored users = Us Er.objects.filter (email= "Inception") Self.assertequals (Len (Users), 1) #savepoint was committed self.assertequals (users[0].stripe_id, ' 4 ') self.assertequals (Users[0].name, ' starting down the Rabbit hole ') #save point was Committ Ed by exiting the Context_manager without an exception limbo = User.objects.filter (email= "Illbehere@forever") self.a Ssertequals (Len (Limbo), 1) 

So, in reality, you can use atoms or create a savepoint within a transaction to save points. With atoms, you don't have to be very careful about committing and rolling, and when this happens, you have complete control over the savepoint.
Conclusion

If you have any previous experience using Django earlier transaction processing, you can see many more simple transactional models. As below, by default, there is also autocommit, which is a good example of the "sensible" default values that both Django and Python are proud to provide. For so many systems, you will not need to deal directly with the transaction. Just let the auto-submit feature do its job, but if you do, I would like this post to provide you with the transactions you need to manage in Django like an expert.

