Comparison of dataset compression methods for large data volumes (16 MB)

Source: Internet
Author: User
It is the best way to compress dataset when a large amount of data must be transmitted on WebService (which cannot be avoided, this reduces the amount of time used during network transmission.
Next, let's take a test on the dataset compression method that I have mastered. I hope that the experts will have an axe and a reference for those who need it.
Note: The WebService that has been compressed into dataset lacks universality. Please weigh the use.
 
The test source code (only compressed part) and result are as follows:
Condition: the size of the dataset before compression is 16891323 bytes (about 16.12 MB ).
Environment:. Net formatwork 1.1
System: Windows XP SP1
Hardware: P4 (2.4 GB) + 256ddr
 
Solution 1: datasetsurrogate + binaryformatter
Public Function datasettobyte (byval dataset as dataset) as byte ()
Dim DSS as new datasetsurrogate (Dataset)
Dim MS as new IO. memorystream
Dim BF as new binaryformatter
BF. serialize (MS, DSS)
Dim reval () as byte = Ms. toarray ()
Ms. Close ()
DSS = nothing
MS = nothing
BF = nothing
Return reval
End Function
 
Solution 2: datasetsurrogate + binaryformatter + sharpziplib
Public Function datasettobyte (byval dataset as dataset) as byte ()
Dim MS as new IO. memorystream
Dim ZOS as new zipoutputstream (MS)
ZOS. putnextentry (New zipentry (Dataset. datasetname ))
Dim BF as new binaryformatter
Dim DSS as datasetsurrogate = new datasetsurrogate (Dataset)
BF. serialize (ZOS, DSS)
ZOS. closeentry ()
ZOS. Close ()
Dim reval as byte () = Ms. toarray
Ms. Close ()
MS = nothing
ZOS = nothing
BF = nothing
DSS = nothing
Return reval
End Function
 
Solution 3: zlib
Public Function datasettobyte (byval dataset as dataset) as byte ()
Dim MS as new system. Io. memorystream
Dataset. writexml (MS, xmlwritemode. writeschema)
Dim reval () as byte = Ms. toarray
Ms. Close ()
MS = nothing
Dim zlib as new vbzlib. Compress
Zlib. compressbyte (reval)
Zlib = nothing
Return reval
End Function
 
Result (after compression ):
Solution 1: 4420881 bytes (compressed by about 73.83%), time consumed: 23200 Ms
Solution 2: 696881 bytes (compressed by about 95.87%), time consumed: 26621 Ms
Solution 3: 422990 bytes (compressed by about 97.50%), time consumed: 680 Ms
 
It seems that no matter the compression rate or time consumption, solution 3 is the best, but a third-party component is used. The CPU usage of the first two solutions is unacceptable.
Author: zhongnian_wei & 163.com)

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.