Through the simple modification Libjpeg source code, realizes the inside memory bitmap the compression and the decompression-[linux]__linux

Tag:linux Tips JPEG

Copyright notice: When reprinted, please indicate the original source and author information of the article by hyperlink form and this statement
Http://rtfsc.blogbus.com/logs/23148378.html
Believe that used friends should like to Libjpeg, it is easy to use, compression quality can be arbitrary control, and stability is good, but, the official website to provide Libjpeg library,
Whether it is compression or decompression, you need to use file, so that we want to directly compress or decompress the image in memory, we have to implement the corresponding structure,
In short, more trouble, especially for beginners, but also do not know where to start, fortunately, libjpeg to provide us with the source code, today I will introduce to you, how to modify the source code,
Enables Libjpeg to handle images in memory very easily, without the use of file manipulation.

First, the establishment of their own Libjpeg project
In order to modify the convenience of compiling, but also in the VC environment in order to easily use the Libjpeg library, we follow the following steps to convert Libjpeg to VC environment under the project.
1, in the VC environment to re-establish an empty static library project, the project named Libjpeg, here Note that the new project does not include MFC, do not precompile the header file;
2, then the libjpeg under the jcapimin.c jcapistd.c jccoefct.c jccolor.c jcdctmgr.c JCHUFF.C
jcinit.c jcmainct.c jcmarker.c jcmaster.c jcomapi.c jcparam.c
JCPHUFF.C jcprepct.c jcsample.c jctrans.c jdapimin.c jdapistd.c
jdatadst.c jdatasrc.c jdcoefct.c jdcolor.c jddctmgr.c JDHUFF.C
jdinput.c jdmainct.c jdmarker.c jdmaster.c jdmerge.c JDPHUFF.C
jdpostct.c jdsample.c jdtrans.c jerror.c jfdctflt.c jfdctfst.c
jfdctint.c jidctflt.c jidctfst.c jidctint.c jidctred.c jquant1.c
jquant2.c JUTILS.C jmemmgr.c
Jchuff.h jconfig.h jdhuff.h jdct.h jerror.h jinclude.h jmemsys.h jmorecfg.h
Jpegint.h jpeglib.h jversion.h files are copied to the new project folder, and the. c file is renamed. cpp;
3, all the source files and header files added to the new project;
4, compile the new project, at this time can generate Libjpeg.lib.
Second, analyze and modify the source code
We know that Libjpeg is using file to access image data, next, we will analyze how libjpeg is using file to access image data,
Then we replace all the file operations (I/O) with the memory copy, and realize the goal of image compression and decompression in memory.
Below, the first analysis of compressed image libjpeg is how to use file to store data. Let's first look at the functions we call when we do image compression, which are related to the file:
Jpeg_stdio_dest (j_compres_ptr cinfo, FILE *outfile);
We found the source code for this function (line 130th of the Jdatadst.cpp file):
1 GLOBAL (void)
2 Jpeg_stdio_dest (j_compress_ptr cinfo, FILE * outfile)
3 {
4 My_dest_ptr dest;
5/* The destination object is made permanent I multiple JPEG images
6 * Can be written to the same file without re-executing jpeg_stdio_dest.
7 * This makes it dangerous to use this manager and a different destination
8 * Manager serially with the same JPEG object, because their private object
9 * sizes may be different. Caveat programmer.
10 */
One if (cinfo->dest = = NULL) {/*/* A For this JPEG object? *
Cinfo->dest = (struct jpeg_destination_mgr *)
(*cinfo->mem->alloc_small) (j_common_ptr) Cinfo, Jpool_permanent,
SIZEOF (my_destination_mgr));
15}
Dest = (my_dest_ptr) cinfo->dest;
Dest->pub.init_destination = init_destination;
Dest->pub.empty_output_buffer = Empty_output_buffer;
Dest->pub.term_destination = term_destination;
Dest->outfile = outfile;
21}

Let's look at line 20th, the function assigns a pointer to a file type to Dest->outfile, and it is clear that later operations on the files turn to the dest->outfile operation,
We just find all the reference outfile function, we can know how libjpeg compressed the image to the file, so we continue to search outfile, the results are as follows:

Find All "outfile", subfolders, find Results 1, "Entire Solution"
E:/vs2005/libjpeg/libjpeg/jpeglib.h (910): EXTERN (void) jpeg_stdio_dest JPP (j_compress_ptr cinfo, FILE * outfile));
E:/vs2005/libjpeg/libjpeg/jdatadst.cpp (a): FILE * outfile; /* Target Stream * *
E:/vs2005/libjpeg/libjpeg/jdatadst.cpp (Jfwrite): if (Dest->outfile, Dest->buffer, OUTPUT_BUF_SIZE)!=
E:/vs2005/libjpeg/libjpeg/jdatadst.cpp (113): if (Jfwrite (Dest->outfile, Dest->buffer, Datacount)!= Datacount)
E:/vs2005/libjpeg/libjpeg/jdatadst.cpp (116): Fflush (Dest->outfile);
E:/vs2005/libjpeg/libjpeg/jdatadst.cpp (118): if (ferror (dest->outfile))
E:/vs2005/libjpeg/libjpeg/jdatadst.cpp (130): Jpeg_stdio_dest (j_compress_ptr cinfo, FILE * outfile)
E:/vs2005/libjpeg/libjpeg/jdatadst.cpp (): Dest->outfile = outfile;
Matching lines:8 Matching files:2 Total Files searched:57

As you can see, there are 8 references to the outfile variable, the first is the function declaration, the second is the variable declaration, the 第三、四、五、六 is the file operation, the seventh and eighth place we
Already met, we only need to change these eight places to achieve our goal. As follows:

EXTERN (void) jpeg_stdio_dest JPP ((j_compress_ptr cinfo, char* outdata)); by extern (void) jpeg_stdio_dest JPP (j_compress_ptr cinfo, FILE * outfile);
char * OUTDATA;  /* Target stream///by FILE * outfile; /* Target stream */overwrite

Jdatadst.cpp file line 87th Empty_output_buffer (j_compress_ptr cinfo) function
memcpy (dest->outdata,dest->buffer,output_buf_size);//by Jfwrite (Dest->outfile, Dest->buffer, OUTPUT_ buf_size) rewrite

Jdatadst.cpp file line 114th Term_destination (j_compress_ptr cinfo)
memcpy (Dest->outdata,dest->buffer,datacount); Rewritten by Jfwrite (Dest->outfile, Dest->buffer, Datacount)

Delete Fflush (dest->outfile); and if (Ferror (Dest->outfile)) and other related statements.
Peg_stdio_dest (j_compress_ptr cinfo, char* outdata)//rewritten by peg_stdio_dest (j_compress_ptr cinfo, FILE * outfile)
Dest->outdata = Outdata; by dest->outfile = outfile; rewrite

We change here, we can compile, there should be no mistakes, but, you will not feel a problem. Yes, we found that we did not provide an offset for the memory area (the offset points to the current position each time the image data is appended).
In addition, because only after the compression to know the image compressed data volume size, we also need a to indicate the size of the image data variables.

We add these two variables to the outdata, like Outdata, as Dest member variables, as follows:
typedef struct {
struct Jpeg_destination_mgr pub; /* Public Fields * *

char * OUTDATA; /* Target Stream * *
int *psize; New variable, which is supplied by the caller, and returns the image size after compression
int noutoffset; New Add Variable
Joctet * buffer; /* Start of buffer * *
} my_destination_mgr;

We will provide the psize pointer through the Jpeg_stdio_dest function and initialize the newly added variable in the Jpeg_stdio_dest implementation function as follows:
GLOBAL (void)
Jpeg_stdio_dest (j_compress_ptr cinfo, char * outdata, int *psize)
{
My_dest_ptr dest;

 /* The destination object is made permanent and multiple JPEG images
   * Can be written to the Same file without re-executing jpeg_stdio_dest.
   * This makes it dangerous to use this manager and a different destination
   * Manager seria Lly with the same JPEG object, because their private object
   * Sizes May is different.  caveat program Mer.
   */
  if (cinfo->dest = = NULL) { /*-A for this JPEG object? */
 &NBSP ;  cinfo->dest = (struct jpeg_destination_mgr *)
      (*cinfo->mem->alloc _small) ((j_common_ptr) Cinfo, jpool_permanent,
      SIZEOF (my_destination_mgr));
 }

Dest = (my_dest_ptr) cinfo->dest;
Dest->pub.init_destination = init_destination;
Dest->pub.empty_output_buffer = Empty_output_buffer;
Dest->pub.term_destination = term_destination;
/* Modified Code * *
Dest->outdata = Outdata;
Dest->noutoffset = 0;
Dest->psize = psize;
* (dest->psize) = 0;
}

Overwrite declaration function
EXTERN (void) jpeg_stdio_dest JPP (j_compress_ptr cinfo, char* outdata, int *psize));

Jdatadst.cpp file line 87th Empty_output_buffer (j_compress_ptr cinfo) function
memcpy (dest->outdata+dest->noutoffset,dest->buffer,output_buf_size);//by Jfwrite (Dest->outfile, dest- >buffer, output_buf_size) rewrite
dest->noutoffset+=output_buf_size;
* (dest->psize) =dest->noutoffset;

Jdatadst.cpp file line 114th Term_destination (j_compress_ptr cinfo)
memcpy (Dest->outdata+dest->noutoffset,dest->buffer,datacount); Rewritten by Jfwrite (Dest->outfile, Dest->buffer, Datacount)
dest->noutoffset+=datacount;
* (dest->psize) =dest->noutoffset;

Recompile the project so that we realize a compressed BMP bitmap into memory, of course, before we call jpeg_stdio_dest, we need to allocate enough memory and pass the memory pointer to the Jpeg_stdio_dest function.
Well, let's analyze how Libjpeg reads the image data from a JPG file when extracting the JPG image.

Let's first look at the function of the file operation that we called when extracting the image, as follows:
Jpeg_stdio_src (j_decompress_ptr cinfo, File * infile).
The MY_SRC_PTR structure is found in the implementation code of the function, and we find that the member variable of the struct related to the file operation is infile, referring to the above, we search for infile, and the search results are as follows:
Find All "infile", Subfolders, find Results 1, "Entire Solution"
  e:/vs2005/libjpeg/libjpeg/jpeglib.h (911): EXTERN (void) jpeg_ Stdio_src JPP (j_decompress_ptr cinfo, FILE * infile));
  E:/vs2005/libjpeg/libjpeg/jdatasrc.cpp (d):  FILE * infile;  /* Source Stream */
  E :/vs2005/libjpeg/libjpeg/jdatasrc.cpp ():  nbytes = Jfread (Src->infile, Src->buffer, INPUT_BUF_SIZE);
  E:/vs2005/libjpeg/libjpeg/jdatasrc.cpp (): Jpeg_stdio_src (j_decompress_ptr cinfo, FILE * infile)
  E:/vs2005/libjpeg/libjpeg/jdatasrc.cpp (209):  src->infile = infile;
  Matching lines:5    Matching files:2    Total Files searched:57

Based on the experience above, we consider adding two variables, image size variables and image offsets in addition to changing the file * type variable to a char * type variable, which is the same as image compression, except that
Image compression, image size is returned by the Libjpeg library, so the call is provided to the Libjpeg library is a pointer, while uncompressed, image data size is by the caller through the variable (not the pointer) to the LIBJPEG library.
Since I explained in detail the work we did in image compression, I think it's easy for readers to understand the changes that were made when they were uncompressed, so I'll just list the code we've rewritten and no longer explain it in detail.

Jpeglib.h Line No. 911
EXTERN (void) jpeg_stdio_src JPP (j_decompress_ptr cinfo, char * indata,int nsize));

Jdatasrc.cpp Line 33rd
/* Expanded Data source object for stdio input */

typedef struct {
struct Jpeg_source_mgr pub; /* Public Fields * *

char * INDATA; /* Source Stream * *
int ninoffset;
int nsize;
Joctet * buffer; /* Start of buffer * *
Boolean start_of_file; /* Have we gotten any data yet? */
} my_source_mgr;

Jdatasrc.cpp Line 183th
GLOBAL (void)
JPEG_STDIO_SRC (j_decompress_ptr cinfo, char * indata, int nsize)
{
My_src_ptr src;

/* The source object and input buffer are made permanent so that a series
* of JPEG images can be read to same file by calling Jpeg_stdio_src
* Only before the one. (If we discarded the buffer at the end of
* One image, we ' d likely lose the start of the next one.
* This makes it unsafe to use this manager and a different source
* Manager serially with the same JPEG object. Caveat programmer.
*/
if (cinfo->src = = NULL) {/* * * * the "*" A For this JPEG object? * *
CINFO->SRC = (struct jpeg_source_mgr *)
(*cinfo->mem->alloc_small) ((j_common_ptr) Cinfo, Jpool_permanent,
SIZEOF (my_source_mgr));
src = (my_src_ptr) cinfo->src;
Src->buffer = (Joctet *)
(*cinfo->mem->alloc_small) ((j_common_ptr) Cinfo, Jpool_permanent,
Input_buf_size * SIZEOF (joctet));
}

src = (my_src_ptr) cinfo->src;
Src->pub.init_source = Init_source;
Src->pub.fill_input_buffer = Fill_input_buffer;
Src->pub.skip_input_data = Skip_input_data;
Src->pub.resync_to_restart = Jpeg_resync_to_restart; /* Use default method * *
Src->pub.term_source = Term_source;
Src->indata = Indata; New Add row
Src->nsize = nsize; New additions
Src->ninoffset = 0; New additions
Src->pub.bytes_in_buffer = 0; /* Forces fill_input_buffer on A/
Src->pub.next_input_byte = NULL; /* Until buffer loaded * *
}

Jdatasrc.cpp Line 91st
METHODDEF (Boolean)
Fill_input_buffer (J_decompress_ptr cinfo)
{
my_src_ptr src = (my_src_ptr) cinfo->src;
size_t nbytes;

Nbytes = Jfread (Src->infile, Src->buffer, input_buf_size);
Nbytes = src->nsize-src->ninoffset;
if (nbytes>input_buf_size) nbytes = input_buf_size;

if (nbytes <= 0) {
if (src->start_of_file)/* Treat empty input file as fatal error * *
Errexit (Cinfo, jerr_input_empty);
Warnms (Cinfo, jwrn_jpeg_eof);
/* Insert a fake EOI marker * *
Src->buffer[0] = (joctet) 0xFF;
SRC->BUFFER[1] = (joctet) jpeg_eoi;
Nbytes = 2;
}

memcpy (src->buffer,src->indata+src->ninoffset,nbytes);
src->ninoffset+=nbytes;

Src->pub.next_input_byte = src->buffer;
Src->pub.bytes_in_buffer = nbytes;
Src->start_of_file = FALSE;

return TRUE;
}

At this point, the source code of the Libjpeg library to change everything we have done, the rest of the thing is that we write a test program test.

Third, write test code

For the detailed call steps for the Libjpeg library, refer to my article, "Using Jpeglib compressed images as JPG format," which details the use of the Libjpeg library
To do the image compression and decompression steps, in the writing of this example of the test code, we last provided the test code based on the following improvements, as follows:

Whether compression or decompression, and the original Libjpeg library call is only one place, is the jpeg_stdio_dest and jpeg_stdio_src these two functions of the call,
When you call the original Libjpeg library, you need to provide the open JPG file handle for both functions, and for the new Libjpeg library, you do not need to open the JPG file, when you compress it,
We need to provide a large enough memory area to the Libjpeg library, decompression, only to be stored with JPEG image of the memory area provided to the Libjpeg library on the line, the following details
For the rewritten jpeg_stdio_dest and JPEG_STDIO_SRC, the invocation method of these two functions.

1, Jpeg_stdio_dest
The prototype of the function is: void Jpeg_stdio_dest (J_compress_ptr cinfo, char * outdata, int *psize);
Here, Outdata points to the memory area we provide to the LIBJPEG library for storing compressed image data, which should be applied before we call the function, as you can see,
We do not have access to the memory area in the Libjpeg library across the border to check and be large enough, otherwise there will be the risk of memory cross-border access, when the entire image compression work is completed, psize
Returns the size of the JPG image data. The test code is as follows:
Char outdata[1000000]; For caching, this is set to 1000K, which can be applied dynamically when used in real time
int nsize; For storing the size of image data after compression

..........
Jpeg_stdio_dest (&JCS, outdata,&nsize);
..........

The prototype of the

    2, jpeg_stdio_src
     function is: void Jpeg_stdio_src (j_decompress_ptr Cinfo, char * indata,int nsize);
     here, Indata points to the JPG data that we are going to decompress, which we can read directly from the JPG file, or through the Libjpeg library, which is directly compressed in memory
to generate the data. Nsize Of course is the size of this JPG data. The test code is as follows:
  .....
        Char indata[1000000];//for storing image data before decompression, which reads directly from a JPG file
         FILE *f = fopen (Strsourcefilename, "RB");
 if (f==null)
 {
  printf ("Open file error!/n");
  return;
 }
 int nsize = fread (outdata,1,1000000,f);//read JPG image data, nsize for actual read image data size
 fclose (f);
 //The following code is used to decompress the
 jpeg_stdio_src (&cinfo, outdata,nsize) from the beginning of the bank;
  ....      all of our work has been completed, the complete test program please download from my resources, in order to increase compatibility, this test program deliberately added byte adjustment function to solve some of the readers
Test image when the image of the tilt problem, OK, compile and run the test program to see the effect of it.