Python--struct, Cpickle, PYAMF performance comparison

In large network applications, serialization is a necessary means, so be sure to understand the serialization of performance consumption, in order to write high-performance programs. Commonly used serialization is the use of pickle, unpopular is the use of PYAMF. The AMF data can be used as a part of the custom protocol in the network communication, and it can compress the binary data well. Let's test the cost of these two serialization.

Serialization: Pickle

Packing: struct

first of all, we need to refer to the reference data first. Empty run while loop:

Import time

start_time = Time.time ()
print start_time

j = 1 while
True:
    j = 1
    end_time = Time.ti Me ()
    if End_time-start_time >= 1:
        break
print J
Print End_time

On the Apple machine, the I5 processor, the frequency 2.3GHz, ran out of the data results as follows:

>>>
1343818757.17
2174001
1343818758.17
>>>
That is, a rough calculation can run 2.1 million times per second of +1 operations, and time function calls . (GIL, Test here is run only on one core)
Here is the code to test struct packaging performance: The amount of data packaged is 1116 bytes (8+1108)

#! /usr/bin/env python #coding =utf-8 import time import struct Test_data = {' Baihe ': {' name ': Unicode (' Lily ', 
        ' Utf-8 '), ' say ': Unicode (' Fresh, elegant, floral ', ' utf-8 '), ' grow_time ': 0.5, ' fruit_time ': 0.5, ' Super_time ': 0.5, ' total_time ': 1, ' buy ': {' Gold ': 2,}, ' Harvest_fruit ': 1, ' 
        Harvest_super ': 1, ' Sale ': 1, ' level_need ': 0, ' experience ': 2, ' exp_fruit ': 1,
            ' Exp_super ': 1, ' used ': True,}, ' 1 ': {' interval ': 0.3, ' probability ': { ' n ': {' Chips ': (5,),}, ' 2 ': {' gem ': (1,1),},},}, ' 2 ': {' unlock ': {' ch
            IPs ': 1000, ' FC ': Ten,}, ' interval ': A, ' probability ': {': {' chips ': (120, 250),}, ': {' gem ': (1,1),}, ' ten ': {' gem ': (2,2),},},}, ' one ': {' 10,5 ': {' id ': ' M01 ', ' Y ': 1, ' msg ': U' A silver coin was found in the jar. ',}, ' 3,7 ': {' id ': ' m02 ', ' Y ': ' msg ': U ' found 10 pieces of silver. A great deal of money. 
        ',}, ' 15,5 ': {' id ': ' m03 ', ' Y ': 2, ' msg ': U ' a mouse ran past ',}, ' 7,4 ': {' id ': ' m04 ', ' Y ': 4, ' msg ': U ' found four rusty silver coins ... ',}, ' 2,12 ': {' id ': ' m05 ', ' Y ': 6, ' msg ': U ' six shiny silver coins.


',},},} str_data = str (test_data) str_data_length = Len (str_data) # 1108 print '-----', str_data_length Start_time = Time.time () print "Start_time:", start_time j = 1 while true:j + = 1 Xulie_data = Struct.pack (">i i1108s ", 888, 333, str_data) End_time = Time.time () if End_time-start_time >= print" l  Oop_num: ", J print" End_time: ", end_time print len (xulie_data) ii_tuple = Struct.unpack (" >ii1108s ", Xulie_data) print

 Ii_tuple

cpickle Test:

Test the original data size, about 1k.

Test results:

start_time:1343873648.89
loop_num:34514
end_time:1343873649.89
868

A rough calculation, can run 35,000 times per second pickle.dumps () operation, and time function call .

The code refers to the latter blog post.

PYAMF Test:

Test the original data size, about 1k.

Test results:

start_time:1343897700.92
loop_num:6805
end_time:1343897701.92
656

Rough calculation, only 6,000 times per second can run AMF3 encoding operations, and time function calls .

The code refers to the latter blog post.

struct test:

Test Struct.pack (">ih", 55, 66)

Test results:

start_time:1343964526.48
loop_num:730989
end_time:1343964527.48

Rough calculation, can run 70多万次 simple pack operation per second, and time function call .

Summarize:

It can be seen that PYAMF coding efficiency is not high, purely from the point of view of serialization, Cpickle dumps () at Protocol 1 o'clock, efficiency is more than 5 times times the PYAMF, but PYAMF compression ratio than pickle.dumps () higher than 20%.

Besides packing struct, simple test, itself is very efficient, the performance bottleneck is not big. And, when unpack, the efficiency is more than packing time, high out of 15-20%.