Tutorial on implementing transaction management using the Python Django framework

This article describes how to use the Python Django framework to implement transaction management, and perform a series of operations on the transaction behavior of the database, for more information, see. if you spend a lot of time processing Django database transactions, you will understand that this is confusing.

In the past, only a simple basic document was provided. to understand how it is used, you must create and execute Django transactions.

There are many terms for Django transaction processing, such as commit_on_success, commit_manually, commit_unless_maneged, rollback_unless_managed, enter_transaction_management, and leace_transaction_management.

Fortunately, after the release of Django 1.6. Now you can use several functions to implement transaction processing, several seconds before learning these functions. First, we need to clarify the following issues:

• What is transaction processing?
• What new transaction processing priorities appear in Django 1.6?

Before you go to "how is the correct transaction processing in Django 1.6", please first learn the following detailed cases:

Case study

Transactions

• Recommended methods
• Use separators
• Each HTTP request transaction

Save Point

Nested transactions

What is a transaction?

According to the SQL-92, "an SQL transaction (sometimes referred to as a transaction) is a sequence of SQL statement execution that can be atomically restored ". In other words, all SQL statements are executed and submitted together. Similarly, when rollback is performed, all statements are rolled back.

For example:

# STARTnote = Note(title="my first note", text="Yay!")note = Note(title="my second note", text="Whee!")address1.save()address2.save()# COMMIT

Therefore, a transaction is a single unit of work in the database. It starts with a start transaction and ends with a commit or an explicit rollback.

What is the problem with transaction management before Django 1.6?

To answer this question completely, we must explain how transactions are processed in databases, clients, and Django.

Database

Each statement in the database runs in one transaction. this transaction can even contain only one statement.

Almost all databases have an AUTOCOMMIT setting, which is usually set to True by default. AUTOCOMMIT encapsulates all statements in a transaction. as long as the statement is successfully executed, the transaction is committed immediately. Of course, you can also manually call START_TRANSACTION, which will temporarily suspend AUTOCOMMIT until you call COMMIT_TRANSACTION or ROLLBACK.

Then, this method will invalidate the implicit commit after the AUTOCOMMIT settings apply to each statement.

However, there are python client libraries such as sqlite3 and mysqldb that allow python programs to be connected to the database itself. These databases follow a set of standards for database access and query. This db api 2.0 standard is described in PEP 249. Although it may be easier to read. One important thing to take away is that in the PEP 249 status, the automatic submission function should be disabled by default in the database.

This is clearly in conflict with the database:

• Database statements always run in a transaction. This database generally enables the automatic submission function for you.
• However, according to PEP 249, this should not happen.
• What happened in the database must be reflected in the client database? However, they are not allowed to enable the automatic submission function by default. They simply wrap SQL statements in a transaction. It is like a database.

Well, stay with me for a long time.

Django

When you enter Django, Django also has something to say about transaction processing. In Django 1.5 and earlier versions. When you write data to the database, Django basically runs an open transaction and automatically commits this transaction function. So every time you name something like model. save () or model. update (), Django generates the corresponding SQL statement and submits the transaction.

There are also versions in Django 1.5 and earlier. it is recommended that you use TransactionMiddleware to bind http request transactions. Each request provides a transaction. If no exception is returned, Django submits this transaction. However, if your view function throws an error, rollback will be called. This actually indicates that the automatic submission function is disabled. If you want to standardize, database-level automatic commit-style transaction management, you must manage your own transactions-typically by decorating your view functions using transactions, such as @ transaction. commit_manually, or @ transaction. commit_on_success.

Take one breath or two.

What does this mean?

Yes, there are a lot of things to do there, and it turns out that most developers are needing this standard database-level automatic commit function-meaningful transactions are often processed behind the scenes. Do your own thing until you need to manually adjust them.

In Django 1.6, what is correct about transaction management?

Now, welcome to Django 1. 6. try to forget everything. let's just talk about it. just remember that in Django 1.6, you can use the database and manually submit and manage transactions as needed. In essence, we have a simpler model, basically putting the database we designed first.

Okay! Let's write the code?

Stripe case

In the following example, Stripe is called to process the credit card process.

def register(request):  user = None  if request.method == 'POST':    form = UserForm(request.POST)    if form.is_valid():       customer = Customer.create("subscription",       email = form.cleaned_data['email'],       description = form.cleaned_data['name'],       card = form.cleaned_data['stripe_token'],       plan="gold",      )       cd = form.cleaned_data            try:        user = User.create(cd['name'], cd['email'], cd['password'],          cd['last_4_digits'])         if customer:          user.stripe_id = customer.id          user.save()        else:          UnpaidUsers(email=cd['email']).save()       except IntegrityError:        form.addError(cd['email'] + ' is already a member')      else:        request.session['user'] = user.pk        return HttpResponseRedirect('/')   else:   form = UserForm()   return render_to_response(    'register.html',    {     'form': form,     'months': range(1, 12),     'publishable': settings.STRIPE_PUBLISHABLE,     'soon': soon(),     'user': user,     'years': range(2011, 2036),    },    context_instance=RequestContext(request)  )

In this example, Customer. create is called to process the credit card process, and then a new user is created. If we get a response from Stripe, we will use stripe_id to update the newly created user. If we don't get a response (Stripe is closed), we will add a new entry to the UnpaidUsers table using the email of the newly created user so that they can retry their credit card information later.

The idea is as follows: if Stripe does not respond, users can still register and start using our website. We will ask the user to provide credit card information later.

"I understand this is a special example, and this is not the way I want to complete the function, but it aims to show transactions"

Consider transactions. remember that the database AUTOCOMMIT feature is provided in Django1.6. Next, let's take a look at the database-related code:

cd = form.cleaned_datatry:  user = User.create(cd['name'], cd['email'], cd['password'], cd['last_4_digits'])   if customer:    user.stripe_id = customer.id    user.save()  else:    UnpaidUsers(email=cd['email']).save() except IntegrityError:

Can you find the problem? What happens if "UnpaidUsers (email = cd ['email ']). save ()" fails to run?

One user registered the system and thought that the credit card had been verified. But in fact, the system has not been checked.

We just want to get one of the results:

1. create a user in the database with the stripe_id

2. create a user in the database, but there is no stripe_id. At the same time, the relevant "UnpaidUsers" row contains the same email address

This means that we want a separate database statement header to complete the task or roll back. This example shows the transaction.

First, we will write some test cases to verify whether things run as we imagined:

@mock.patch('payments.models.UnpaidUsers.save', side_effect = IntegrityError)def test_registering_user_when_strip_is_down_all_or_nothing(self, save_mock):   #create the request used to test the view  self.request.session = {}  self.request.method='POST'  self.request.POST = {'email' : 'python@rocks.com',             'name' : 'pyRock',             'stripe_token' : '...',             'last_4_digits' : '4242',             'password' : 'bad_password',             'ver_password' : 'bad_password',            }       #mock out stripe and ask it to throw a connection error  with mock.patch('stripe.Customer.create', side_effect =          socket.error("can't connect to stripe")) as stripe_mock:     #run the test    resp = register(self.request)     #assert there is no record in the database without stripe id.    users = User.objects.filter(email="python@rocks.com")    self.assertEquals(len(users), 0)     #check the associated table also didn't get updated    unpaid = UnpaidUsers.objects.filter(email="python@rocks.com")    self.assertEquals(len(unpaid), 0)

When we try to save "UnpaidUsers", the interpreter above the test will run out of an exception 'integrityerror '.

Next, I will explain the answer to this question. "What happened when" UnpaidUsers (email = cd ['email ']). save () "was running ?" The following code creates a dialog. we need to provide some suitable information in the registration function. Then "with mock. patch" will force the system to think that Stripe has no response, and finally jump to our test case.

resp = register(self.request)

The above section only calls our registration view to pass the request. Then we only need to check whether the table has been updated:

#assert there is no record in the database without stripe_id.users = User.objects.filter(email="python@rocks.com")self.assertEquals(len(users), 0) #check the associated table also didn't get updatedunpaid = UnpaidUsers.objects.filter(email="python@rocks.com")self.assertEquals(len(unpaid), 0)

So if we run the test case, it should fail:

======================================================================FAIL: test_registering_user_when_strip_is_down_all_or_nothing (tests.payments.testViews.RegisterPageTests)----------------------------------------------------------------------Traceback (most recent call last): File "/Users/j1z0/.virtualenvs/django_1.6/lib/python2.7/site-packages/mock.py", line 1201, in patched  return func(*args, **keywargs) File "/Users/j1z0/Code/RealPython/mvp_for_Adv_Python_Web_Book/tests/payments/testViews.py", line 266, in test_registering_user_when_strip_is_down_all_or_nothing  self.assertEquals(len(users), 0)AssertionError: 1 != 0 ----------------------------------------------------------------------

Like. This is the final result we want.

Remember: Here we have practiced "test-driven development. The error message prompts us that the user information has been saved to the database, but this is not what we want, because we have not paid!

Transaction is used to save such problems...

Transactions

For Django1.6, there are many ways to create transactions.

Here is a brief introduction to several types.
Recommended methods

According to Django1.6, "Django provides a simple API to control database transaction... atomic operations are used to define the attributes of database transactions. Atomic operations allow us to create a bunch of code while the database guarantees. If the code is successfully executed, the corresponding changes will be submitted to the database. If an exception occurs, the operation will be rolled back ."

Atomic operations can be used for interpreting operations or content management. So if we use it as content management, the code for registering a function will be as follows:

from django.db import transaction try:  with transaction.atomic():    user = User.create(cd['name'], cd['email'], cd['password'], cd['last_4_digits'])     if customer:      user.stripe_id = customer.id      user.save()    else:      UnpaidUsers(email=cd['email']).save() except IntegrityError:  form.addError(cd['email'] + ' is already a member')

Note that in the line "with transaction. atomic. This code will be executed within the transaction. So if we re-run our test, all of them will pass.

Remember: a transaction is a unit of work, so when the "UnpaidUsers" call fails, all operations in content management will be rolled back together.

Use decorator

In addition to the above practice, we can use the Python modifier feature to use transactions.

@transaction.atomic():def register(request):  ...snip....   try:    user = User.create(cd['name'], cd['email'], cd['password'], cd['last_4_digits'])     if customer:      user.stripe_id = customer.id      user.save()    else:        UnpaidUsers(email=cd['email']).save()   except IntegrityError:    form.addError(cd['email'] + ' is already a member')

If we run the test again, it will still fail.

Why? Why is the transaction not correctly rolled back? The reason lies in the relationship with transaction. atomic will try to catch some exceptions, while the people in our code will catch them (for example, the IntegrityError exception in the try-try T code block), so transaction. atomic will never see this exception, so the standard AUTOCOMMIT process will not work.

However, deleting the try-catch statement will cause exceptions not to be captured, and the code flow will be out of order. Therefore, try-catch cannot be removed.

Therefore, the trick is to put the atomic context manager into the try-catch code segment in our first solution.

Let's look at the correct code:

from django.db import transaction try:  with transaction.atomic():    user = User.create(cd['name'], cd['email'], cd['password'], cd['last_4_digits'])     if customer:      user.stripe_id = customer.id      user.save()    else:      UnpaidUsers(email=cd['email']).save() except IntegrityError:  form.addError(cd['email'] + ' is already a member')

When UnpaidUsers triggers the IntegrityError, the context manager transaction. atomic () will capture it and perform the rollback operation. In this case, if our code is executed in the exception handling process (theform. addErrorline), the rollback operation will be completed and, if necessary, the database can be safely called. Note: Any database call before and after the context manager thetransaction. atomic () will not be affected by its execution results.

Transaction for each HTTP request

Both Django1.5 and 1.6 allow users to operate the request transaction mode. In this mode, Django automatically processes your view functions in the transaction. If a View function throws an exception, Django automatically rolls back the transaction; otherwise, Django commits the transaction.

To implement this function, you need to set "ATOMIC_REQUEST" to true in the configuration of the database you want to have this function. Therefore, the following settings are required in our "settings. py:

DATABASES = {  'default': {    'ENGINE': 'django.db.backends.sqlite3',    'NAME': os.path.join(SITE_ROOT, 'test.db'),    'ATOMIC_REQUEST': True,  }}

If we put the interpreter into The View function, the above settings will take effect. So this is not in line with our ideas.

However, it is worth noting that the interpreters "ATOMIC_REQUESTS" and "@ transaction. atomic" may still handle these errors when exceptions are thrown. Templates.

Save Point

Although transactions are atomic, they can still be divided into multiple "storage points"-you can think of them as "some transactions ".

For example, if a transaction contains four SQL statements, you can create a save point after the second SQL statement. Once the storage point is created successfully, even if the execution of the third or fourth SQL statement fails, you can still perform a partial rollback, ignoring the last two SQL statements and retaining only the first two.

Basically, this is like providing a cutting capability: a common transaction can be divided into multiple more "lightweight" transactions, and then be able to perform partial rollback or partial commit.

However, be careful when the entire transaction is rolled back for no reason (for example, because an IntegrityError exception is thrown but not captured, all the storage points will be rolled back)

Let's take a look at the sample code to learn how to play with the save point.

@transaction.atomic()def save_points(self,save=True):   user = User.create('jj','inception','jj','1234')  sp1 = transaction.savepoint()   user.name = 'zheli hui guadiao, T.T'  user.stripe_id = 4  user.save()   if save:    transaction.savepoint_commit(sp1)  else:    transaction.savepoint_rollback(sp1)

In this example, the entire function belongs to a transaction. After the new User object is created, a save point is created and obtained. Then the following three lines of statements --

  user.name = 'zheli hui guadiao, T.T'  user.stripe_id = 4  user.save()

-- Does not belong to the save point. Therefore, they may belong to the following savepoint_rollback or savepoint_commit part. Assume it is savepoint_rollback, and the code line is user = User. create ('JJ ', 'exception', 'JJ', '000000') will still be successfully submitted to the database, and the following three rows will not be successful.

Using another method, the following two test cases describe how the save point works:

def test_savepoint_rollbacks(self):   self.save_points(False)   #verify that everything was stored  users = User.objects.filter(email="inception")  self.assertEquals(len(users), 1)   #note the values here are from the original create call  self.assertEquals(users[0].stripe_id, '')  self.assertEquals(users[0].name, 'jj')  def test_savepoint_commit(self):  self.save_points(True)   #verify that everything was stored  users = User.objects.filter(email="inception")  self.assertEquals(len(users), 1)   #note the values here are from the update calls  self.assertEquals(users[0].stripe_id, '4')  self.assertEquals(users[0].name, 'starting down the rabbit hole')

Similarly, after we commit or roll back the storage point, we can continue to work in the same transaction. At the same time, the running result is not affected by the output result of the previously saved vertex.

For example, if we update the "save_points" function as follows,

@transaction.atomic()def save_points(self,save=True):   user = User.create('jj','inception','jj','1234')  sp1 = transaction.savepoint()   user.name = 'starting down the rabbit hole'  user.save()   user.stripe_id = 4  user.save()   if save:    transaction.savepoint_commit(sp1)  else:    transaction.savepoint_rollback(sp1)   user.create('limbo','illbehere@forever','mind blown',      '1111')

Even if "savepoint_commit" or "savepoint_rollback" is called by the "limbo" user, the transaction will still be created successfully. If it is not successfully created, the entire transaction will be rolled back.

Nested transactions

When "savepoint ()", "savepoint_commit", and "savepoint_rollback" are used to manually specify the storage point, a nested transaction is automatically generated, and the nested transaction automatically creates a storage point for us. In addition, if we encounter an error, this transaction will be rolled back.

The following is an example of an extension:

@transaction.atomic()def save_points(self,save=True):   user = User.create('jj','inception','jj','1234')  sp1 = transaction.savepoint()   user.name = 'starting down the rabbit hole'  user.save()   user.stripe_id = 4  user.save()   if save:    transaction.savepoint_commit(sp1)  else:    transaction.savepoint_rollback(sp1)   try:    with transaction.atomic():      user.create('limbo','illbehere@forever','mind blown',          '1111')      if not save: raise DatabaseError  except DatabaseError:    pass

Here we can see that after processing the save point, we use the context management area of "thetransaction. atomic" to erase the user we created "limbo. When context management is called, it creates a save point (because we are already in the transaction ), at the same time, this save point will be executed or rolled back based on the existing context manager.

The following two test cases describe this article:

def test_savepoint_rollbacks(self):      self.save_points(False)     #verify that everything was stored    users = User.objects.filter(email="inception")    self.assertEquals(len(users), 1)     #savepoint was rolled back so we should have original values    self.assertEquals(users[0].stripe_id, '')    self.assertEquals(users[0].name, 'jj')     #this save point was rolled back because of DatabaseError    limbo = User.objects.filter(email="illbehere@forever")    self.assertEquals(len(limbo),0)   def test_savepoint_commit(self):    self.save_points(True)     #verify that everything was stored    users = User.objects.filter(email="inception")    self.assertEquals(len(users), 1)     #savepoint was committed    self.assertEquals(users[0].stripe_id, '4')    self.assertEquals(users[0].name, 'starting down the rabbit hole')     #save point was committed by exiting the context_manager without an exception    limbo = User.objects.filter(email="illbehere@forever")    self.assertEquals(len(limbo),1)

Therefore, in reality, you can use an atom or create a storage point in a transaction. When using atoms, you do not have to worry carefully about submission and rollback. when this happens, you can completely control the storage points.
Conclusion

If you have any previous experience in using Django's earlier version of transaction processing, you can see a lot of simpler transaction processing models. As follows, by default, the automatic submission function is also available. this is a good example. both Django and python are proud to provide the "rational" default value. For so many systems, you do not need to process transactions directly. I only want the "auto-submit function" to complete the job, but if you do, I hope this post will provide the transaction processing you need to manage in Django like an expert.