ImageIO and image decoding in iOS

ImageIO to open up the object has Cgimagesourceref, Cgimagedestinationref, not open to the object has cgimagemetadataref. There are cgimageref and Cgdataprovider objects in coregraphics that often deal with ImageIO, and then look at the roles that these five objects assume in creating a uiimage.

Using Timeprofiler Step-by-step to create a function called internally during the UIImage process can help us solve the problem, because the Timeprofiler statistical function stack is counted once for a period of time, resulting in no calls to all functions, and each function stack may be inconsistent, So it's impossible to tell exactly how a function stack is called, but it's possible to guess what each step does.

The code from CFDATAREF to UIImage is as follows

NSString *resource = [[nsbundle mainbundle] pathforresource:@ "the_red _batman " ofType:@" PNG "];

NSData *data = [nsdata datawithcontentsoffile: Resource options:0 Error : nil];

Cfdataref Dataref = (__bridge cfdataref)data;

Cgimagesourceref Source = cgimagesourcecreatewithdata(dataref, Nil );

Cgimageref cgimage = cgimagesourcecreateimageatindex(source, 0, Nil);

UIImage *image = [UIImage imagewithcgimage: Cgimage];

Cgimagsourcecreatewithdata

The intrinsic function _cgimagereadcreate is called, which means that cgimagesourceref is related to reading the image data.

Cgimagesourcecreateimageatindex

_cg_png_read_info and cgimagemetadatacreatemutable are called, and when the Cgimageref is built, the underlying data and metadata of the image are read, and the underlying data includes the header chunk of the image, such as PNG's IHDR. Meta data is abstracted by Cgimagemetadataref. And did not read the image of other data, but also do not do the decoding action.

Interestingly, if you call Cgimagesourcecopypropertiesatindex

Cgimagesourcecopypropertiesatindex's intrinsic function calls the cgimagemetadataref, and if you add a comment to the imageio/cgimagemetadata.h file

@description cgimagemetadata APIs allow clients to view and modify metadata

For popular image formats. ImageIO supports the EXIF, IPTC, and XMP

Metadata specifications. Refer to CGImageSource.h for functions to

Read metadata from a cgimagesource, and CGImageDestination.h for functions to

Write metadata to a cgimagedestination. Cgimagedestinationcopyimagesource can

be used to modify metadata without recompressing the image.

The Description Cgimagemetadataref abstracts the metadata of the Exif, IPTC, and XMP formats in the image, and if you want to get cgimagemetadataref you have to go through cgimagesourceref.

Similarly, look at the internal function call for Cgdataproviderref, the code is as follows

Change Cgimagesourceref from Cgdataproviderref to create

Cfdataref Dataref = (__bridge cfdataref)data;

Cgdataproviderref Dataprovider = cgdataprovidercreatewithcfdata(dataref);

Cgimagesourceref Source = cgimagesourcecreatewithdataprovider(dataprovider, nil);

The test is obtained by Cgimageref Cgdataproviderref and created by Cgdataproviderref Cgimageref

Cgdataproviderref Newdataprovider = cgimagegetdataprovider(cgimage);

size_t Width = cgimagegetwidth(cgimage);

size_t Height = cgimagegetheight(cgimage);

Cgcolorspaceref Space = cgimagegetcolorspace(cgimage);

size_t bitspercomponent = cgimagegetbitspercomponent(cgimage);

size_t bitsperpixel = cgimagegetbitsperpixel(cgimage);

size_t Bytesperrow = cgimagegetbytesperrow(cgimage);

Cgbitmapinfo bitmapinfo = cgimagegetbitmapinfo(cgimage);

Cgdataproviderref Newdataprovider = cgimagegetdataprovider(cgimage);

Cgimageref newimage = cgimagecreate(width, height, Bitspercomponent, bitsperpixel, bytesperrow, space, Bitmapinfo, newdataprovider, NULL, false, Kcgrenderingintentdefault);

Unfortunately, there is no function call to find out about Cgdataproviderref. Could not conclude what Cgdataproviderref had done.

Look at the internal function calls for Cgimagedestinationref, and the code and intrinsic functions are called as follows

Cfmutabledataref Buffer = cfdatacreatemutable(kcfallocatordefault, 0); cgimagedestinationref Destination = cgimagedestinationcreatewithdata(buffer , kuttypepng, 1, NULL);

Cgimagedestinationaddimage(destination, cgimage, nil);

Cgimagedestinationfinalize(destination);

Cgimagedestinationref writes the image data to the destination, and is responsible for image coding or image compression.

Test conclusion

CGIMAGESOURCEREF abstract the channel to read the image data, read the image through it, it itself does not read any data of the image, when you call Cgimagesourcecopypropertiesatindex will read the image metadata.

Cgimagemetadataref abstract The image of Exif, IPTC, XMP format metadata, obtained through the cgimagesourceref.

CGIMAGEREF abstract the basic data and metadata of the image, when it is created, it will read the basic data and metadata of the image through Cgimagesourceref, but no other data of the image is read, and no image decoding action is done.

Cgdataproviderref did not come up with useful information.

Cgimagedestinationref Abstract writes the channel of the image data, writes the image to pass it, when writes the picture The time also is responsible for the picture encoding.

Image decoding

You can see from the cfdataref until you create the uiimage, have not called the image decoding function, only read some of the image base data and metadata.

When did the image decode occur? In the Imageio/cgimagesource.h file Kcgimagesourceshouldcache the above actually has a clear comment.

Specifies whether image decoding and caching should happen at image creation time.

The value of this key must is a cfbooleanref. The default value is Kcfbooleanfalse (image decoding would happen at rendering time).

If you do not set the image manually, it will only wait until the screen is rendered and then decoded. Tested, that's true. This kcgimagesourceshouldcacheimmediately might as well be named kcgimagesourceshoulddecodeimmediately.

Where exactly is image decoding done?

If you render a picture on the canvas, the picture must be decoded. The following code runs the test again

size_t Width = cgimagegetwidth(cgimage);

size_t Height = cgimagegetheight(cgimage);

Cgimagealphainfo alphainfo = cgimagegetalphainfo(cgimage) & Kcgbitmapalphainfomask;

BOOL Hasalpha = NO;

If (alphainfo = = Kcgimagealphapremultipliedlast | |

alphainfo = = Kcgimagealphapremultipliedfirst | |

alphainfo = = Kcgimagealphalast | |

alphainfo = = Kcgimagealphafirst) {

Hasalpha = YES;

}

Cgbitmapinfo bitmapinfo = kcgbitmapbyteorder32host;

Bitmapinfo |= hasalpha ? kcgimagealphapremultipliedfirst : Kcgimagealphanoneskipfirst ;

Cgcontextref Context = cgbitmapcontextcreate(NULL, width, height , 8, 0, cgcolorspacecreatedevicergb(), bitmapinfo );

Cgcontextdrawimage(context, cgrectmake(0, 0, Width, height), cgimage);

Cgimageref newimage = cgbitmapcontextcreateimage(context);

There are no obvious decompress or decode words in the function names of all calls. And the above four function calls are the highest frequency. Based on the png_compress_idat of the cgimagedestinationref call, guessing Png_read_idat_data is the function of decoding.

The above is a PNG image as a test case, below to see the JPEG image

It is obvious that the Decode method of Applejpeg is to do the decoding function. JPEG and PNG call two of the same functions, while different images tune different decoding functions. When you draw a picture on the canvas, you call Imageprovidercopyimageblocksetcallback settings callback, then call Copyimageblock, and then call the set callback, However, the decoding function is called by Copyimageblock or by callback and cannot be verified.

Does that imageprovidercopyimageblocksetcallback have anything to do with Cgdataprovidercopydata? After testing, Imageprovidercopyimageblocksetcallback and Copyimageblock are also called inside the Cgdataprovidercopydata. And the cfdataref that Cgdataprovidercopydata gets is a decoded array of pixels.

Conclusion: Image decoding takes place within the Cgdataprovidercopydata function called the callback or Copyimageblock function of the Imageprovidercopyimageblocksetcallback setting, Different methods are called depending on the picture format.

Initialization method of Image

Imagewithdata from the invocation of an intrinsic function, Cgimagesourceref accesses the image data and creates a cgimageref.

Imagewithcontentsoffile internal calls are as follows

The file creates cgimageref by Mmap to memory and then accessing the image data through Cgimagesourceref.

Imagenamed first finds the resource path from the bundle, then also mmap the file into memory, and then accesses the image data through cgimagesourceref, creating cgimageref.

the cache for image

See if different methods have cached behavior by calling different UIImage initialization methods and then creating Uiimageview to display on the screen.

Imagenamed does not call ImageIO any method when the same image is displayed for the second time.

Imagewithdata and Imagewithcontentsoffile Call the ImageIO decoding method the second time they display the same image.

The uiimage created by the Imagewithdata and Imagewithcontentsoffile initialization methods will not invoke the ImageIO decoding method as long as they are not released.

The conclusion is that there are two kinds of caches for uiimage, one is the cache of the UIImage class, which guarantees that the uiimage of imagenamed initialization will only be decoded once. The other is the cache of the UIImage object, which guarantees that the uiimage is not decoded as long as it is not released.

The Kcgimagesourceshouldcache option for the Cgimagesourcecreateimageatindex method refers to the second kind of cache, and if set to false, I test out that the image is still not decoded when it is rendered again. It's a little strange. If you have a detailed knowledge of what is going on, please enlighten me.

Finally attach ImageIO's family photo

ImageIO and image decoding in iOS