A small test of multi-thread image file writing

In processing remote sensing images, it is often found that time-consuming data is more time-consuming in I/O, especially in O (write. Gdal supports multi-threaded image writing. The following is a simple test with examples to see if the actual effect will be improved.

Here I use the thread library of the boost library to create multiple threads. The following describes how to use pi to calculate the boost thread. The timing here uses the boost progress_timer. The following is a small piece of code that uses multi-threaded PI computing. The 10 parts of multi-threaded computing are not added up.

# Include <stdio. h> # include <boost/progress. HPP> // boost timing function # include <boost/thread. HPP> // boost multithreading # include <boost/bind. HPP> // boost bind library using namespace boost; // calculate the number of PI int isize = 1000000000; int isize1 = 100000000; // calculate double calcpi_s () in normal mode () {double DPI = 0.0; int iflag = 1; for (int K = 0; k <= isize; k ++) {DPI = DPI + iflag/(2 * k + 1.0); iflag =-iflag;} return DPI * 4;} // void threadpi (INT istart, int iend) {double DPI = 0.0; int iflag = 1; for (int K = istart; k <= iend; k ++) {DPI = DPI + iflag/(2 * k + 1.0); iflag =-iflag;} printf ("% 18.16lf \ n", DPI * 4 );} // void calcpi_m () {for (INT I = 0; I <10; I ++) {boost: thread thrd (boost: BIND (& threadpi, I * isize1, (I + 1) * isize1); thrd. join () ;}} int main () {// do not use multithreading to process progress_timer * ptime = new progress_timer (); // start timing printf ("one-thread pi \ n"); double dspi = calcpi_s (); printf ("computing end, time consumed: % F Pi = % 18.16lf \ n ", ptime-> elapsed (), dspi); // use multiple threads to process ptime-> restart (); // start timing printf ("multi-thread pi \ n"); calcpi_m (); printf ("computing end, time consumed: % F \ n ", ptime-> elapsed (); Delete ptime; System ("pause"); Return 0 ;}

The above code is tested and the result of release compilation is as follows:

The computation of PI in a single thread ends. Time consumed: 9.643000 Pi = 3.14159265458805069.64 s multi-threaded computation ends. Time consumed: 8.4980008.50 S. Press any key to continue...

Second:

The computation of PI in a single thread ends. Time consumed: 12.039000 Pi = 3.1415926545880506 multithread computation ends. Time consumed: 8.5520.8.55 S. Press any key to continue...

Third time:

The computation of PI in a single thread ends. Time consumed: 14.473000 Pi = 3.1415926545880506 multithread computation ends. Time consumed: 8.5000008.50 S. Press any key to continue...

Fourth:

The computation of PI in a single thread ends. Time consumed: 10.898000 Pi = 3.1415926545880506 multithread computation ends. Time consumed: 8.5100008.51 S. Press any key to continue...

Through four tests, I found that multithreading can improve the speed slightly, but I don't know why, during Single-thread computing, the time jump is large, the ups and downs are large, and I don't know why, if you have any children's shoes, please let us know.

The following is a 10000x10000 single-band image, in the IMG format of ERDAS. The image content is the result of the remainder of the 255 Image Based on the row number, the result is a black and white ripple. Multithreading uses 10 threads to write different parts of the image. The Code is as follows:

# Include <stdio. h> # include "gdal_priv.h" # include <boost/progress. HPP> // boost timing function # include <boost/thread. HPP> # include <boost/bind. HPP> using namespace boost; # pragma comment (Lib, "gdal_ I .lib") typedef unsigned char dt_8u;/*** @ brief create output image */bool createimage (const char * pszfile) {gdalallregister (); gdaldriverh hdriver = gdalgetdriverbyname ("HFA"); If (hdriver = NULL) return false; gdaldataseth hdstds = gdalcreate (hdriver, pszfile, 10000,100 00, 1, gdt_byte, null); // create the output file if (hdstds = NULL) return false; gdalclose (hdstds); Return true;} bool singleprocess (const char * pszfile) {gdalallregister (); gdaldataset * posrcds = (gdaldataset *) gdalopen (pszfile, ga_update); If (posrcds = NULL) return false; int iwidth = posrcds-> getrasterxsize (); int iheight = posrcds-> getrasterysize (); gdalrasterband * pband = posrcds-> getrasterband (1); dt_8u * pbuf = new dt_8u [iwidth]; memset (pbuf, 0, sizeof (dt_8u) * iwidth); // For (INT I = 0; I <iheight; I ++) {int ivalue = I % 255; memset (pbuf, ivalue, sizeof (dt_8u) * iwidth); // outside AOI pband-> rasterio (gf_write, 0, I, iwidth, 1, pbuf, iwidth, 1, gdt_byte, 0, 0);} gdalclose (gdaldataseth) posrcds); Return true;} void threadfun (const char * pszfile, int istart, int iend/*, int Index */) {gdalallregister (); gdaldataset * posrcds = (gdaldataset *) gdalopen (pszfile, ga_update); If (posrcds = NULL) return; int iwidth = posrcds-> getrasterxsize (); int iheight = posrcds-> getrasterysize (); gdalrasterband * pband = posrcds-> getrasterband (1); dt_8u * pbuf = new dt_8u [iwidth]; memset (pbuf, 0, sizeof (dt_8u) * iwidth); // For (INT I = istart; I <iend; I ++) {int ivalue = I % 255; memset (pbuf, ivalue, sizeof (dt_8u) * iwidth); // outside AOI pband-> rasterio (gf_write, 0, I, iwidth, 1, pbuf, iwidth, 1, gdt_byte, 0, 0);} gdalclose (gdaldataseth) posrcds); // printf ("thread % N ended \ n", index);} bool Multiprocess (const char * pszfile) {for (INT I = 0; I <10; I ++) {boost: thread thrd (boost: BIND (& threadfun, pszfile, I * 1000, (I + 1) * 1000/*, I */); thrd. join ();} return true;} int main () {// do not use multithreading to process progress_timer * ptime = new progress_timer (); // start timing const char * pszfilesingle = "F: \ data \ test \ single. IMG "; printf (" image creation start \ n "); createimage (pszfilesingle); printf (" image creation end, time consumed: % F \ n ", ptime-> elapsed (); ptime-> restart (); // start timing singleprocess (pszfilesingle); printf ("processing image by single thread ends, time consumed: % F \ n ", ptime-> elapsed (); // use multithreading to process ptime-> restart (); // start timing const char * pszfilemulti =" F: \ data \ test \ multi. IMG "; printf (" image creation start \ n "); createimage (pszfilemulti); printf (" image creation end, time consumed: % F \ n ", ptime-> elapsed (); ptime-> restart (); // start time Multiprocess (pszfilemulti); printf ("multi-thread processing image ends, time consumed: % F \ n ", ptime-> elapsed (); Delete ptime; System ("pause"); Return 0 ;}

The result is still compiled using release. The running result is as follows:

Image Creation starts and image creation ends. Time consumed: 2.852000 image processing ends in a single thread; time consumed: 20.579000 image creation begins and image creation ends; time consumed: 3.261000 image processing ends in multiple threads; time consumed: 29.34300029.34 S. Press any key to continue...

Second:

Image Creation starts and image creation ends. Time consumed: 2.034000 image processing ends in a single thread; time consumed: 22.311000 image creation begins and image creation ends; time consumed: 2.217000 image processing ends in multiple threads; time consumed: 30.572.1630.57 S. Press any key to continue...

Third time:

Image Creation starts and image creation ends. Time consumed: 2.007000 image processing ends in a single thread; time consumed: 20.285000 image creation begins and image creation ends; time consumed: 2.267000 image processing ends in multiple threads; time consumed: 28.72300028.73 s press any key to continue...

Paste it three times. I tested it for less than ten times on my computer and found that it was slow to write multiple threads, but the result image was correct. This is also unexpected. It is reasonable to say that multithreading should be faster, but the opposite is true here, I don't know whether the multi-thread problem of the boost library or the problem of multi-thread image writing. No matter what the situation is, this path may be more difficult for people who want to use multi-thread image creation.

If you have a better way, please let us know. Thank you.