Camera ISP (Image Signal Processor)

Transferred from: http://blog.csdn.net/shenbin1430/article/details/6065444

1. Target [52rd.com]
Mobile phone camera module with ISP function module market trend and research and development direction. To be able to correctly understand the mobile phone camera module industry to provide technical and market basis. [52rd.com]
2. Application principle of ISP in module [52rd.com]
2.1 Functional Area [52rd.com]
Regardless of the digital camera, camera or camera phone, the image data from the front-end sensor must pass through the ASP (Analog Signal processing), ADC (Analog-digital Converter), After the pre-image processing (PRE-ISP) and back-end image processing (POST-ISP) Four stages, the image data can be finally presented on the terminal equipment (figure I).

However, due to the high and low pixel height of the image sensor, and other cost considerations, the ISP functional areas will be distributed according to the characteristics of the mobile phone market or integrated processing, for example, the low-end camera integrates pre-isp with the sensor, 2.0M Pixel mobile phone will all the ISP functions of a single chip, etc. [52rd.com]
2.1.1 ASP [52rd.com]
ASP (Analog Signal Processor) is mainly for the image sensor acquisition of voltage or current signal processing, the main role is signal amplification, automatic exposure adjustment, timing control, pixel sampling control. Because of its absolute correlation with the initial signal, the general image sensor manufacturer will make this function directly with the sensor. In Figure one is the blue part. After the sensing data has been processed by ASP, the output data is raw. [52rd.com]
2.1.2 Pre-isp [52rd.com]
Pre-isp (Image Signal Processor) is a front-end image processing, focusing on digital data (Raw data) after ADC conversion, for Image bad patch, white balance, gamma correction, sharpness, color interpolation, and more. In low-pixel products, such as 0.3M pixels, because the image data is small, do not need large-scale complex processing, will be pre-isp and image sensor in the same chip, but high-pixel CMOS sensor, due to the need to process more and more pixels, although the pre-isp integrated in the sensor Internal manufacturing technology is not difficult, but due to cost and imaging quality reasons, some mobile design companies in the design will be integrated in the sensor inside the PRE-ISP function to shield off, and maintain the original data of the ADC output, Processed by a separate ISP chip or an ISP integrated in the baseband. [52rd.com]
The data after processing by PRE-ISP is divided into RGB and YUV,RGB for three primary colors, the data is larger than YUV, so it can be convenient for the subsequent processing unit (POST-ISP) to make more multivariate changes. YUV is the RGB three primary color by interpolation method, the data volume is small, but it is not conducive to the subsequent Processing unit processing. [52rd.com]
For the strong mobile phone research and development companies tend to use raw data or RGB data, so that you can adapt to the needs of more perfect picture quality. Raw data is also used for the future high-end market, but the domestic 2.0M pixel module application is still in the primary stage. Therefore, the main output of YUV. [52rd.com]
2.1.3 Post-isp [52rd.com]
Post-isp, although also known as the back-end impact processing, but its imaging-related work is not much, mainly responsible for data compression and back-end interface control, as well as transmission, control and other work, including LCD image preview, lens focus control, the use of interface and so on. [52rd.com]
2.2 ISP development stage [52rd.com]
2.2.1 First stage [52rd.com]
The first stage because the pixel is not high (CIF grade), so its structure and general mobile phone is not much different, pre-isp function and image sensor integration, and post-isp is dependent on the cell phone baseband chip, the display is provided by the system, the structure is as follows [52rd.com]
[52rd.com]
Figure two mobile camera module backend chip first stage schematic [52rd.com]
2.2.2 Phase II [52rd.com]
With the increase in pixels (CIF to VGA), in order to maintain the back-end cell phone baseband chip communication function, the original responsibility of the JPEG compression, data flow coordination and other functions independently into a single chip to complete, and image processing Pre-isp to mobile phone design company to decide, Integrated into the image sensor and integrated with the Post-isp two kinds. [52rd.com]
This phase of the scheme has been phased out and the POST-ISP functionality of the VGA pixel has been baseband integrated.

Figure three mobile phone camera module back-end chip schematic diagram of the second stage [52rd.com]
2.2.3 Phase III [52rd.com]
At this point the pixel has been converted from VGA to megapixel, high-pixel pre-isp, because of the required memory and power requirements too large, in the technical conditions at that time, the sensor terminal can not be fully processed, so the Pre-isp and Post-isp integrated into a complete ISP. Because of the higher processing requirements for the display screen, this phase also integrates the Display control function to reduce the processing burden of the baseband, as shown in Figure four.

Figure four cell phone camera module backend chip phase III schematic [52rd.com]
2.2.4 Fourth stage [52rd.com]
Main Panel [52rd.com]
(240*320) [52rd.com]
[52rd.com]
With the increase of pixels to 2.0M and above, manufacturers will be more features such as MP3, 3D sound processor, audio processor and so on are also integrated into a separate ISP chip, this is called MMP (Mobile multimedia processor),

Figure five mobile phone camera module back end chip phase IV schematic [52rd.com]
Table one mobile phone camera module back end chip development trend [52rd.com]
Phase with pixel trend project [52rd.com]
The first stage CIFL structure and ordinary mobile phone difference is similar [52rd.com]
L PRE-ISP function can be integrated with image sensor [52rd.com]
L Post-isp rely on cell phone baseband chip [52rd.com]
Phase II Vgal Post-isp independent into a single chip [52rd.com]
L PRE-ISP or integrated by sensor or integrated with Post-isp [52rd.com]
Phase III Megal image sensor cannot be fully processed pre-isp, integrated with Post-isp into a separate ISP chip [52rd.com]
Phase IV 2Mega and above L separate MMP (ISP) chip integrates more features to cater for 2.5G and 3G development [52rd.com]
[52rd.com]
3. Current ISP's main application scheme [52rd.com]
Table Two current ISP's main application scheme [52rd.com]
Pixel output format ISP or mobile phone design notes [52rd.com]
vgayuv/rgb/[52rd.com]
Raw datal sensor integrated pre-isp; [52rd.com]
L Baseband integrated Post-isp. OV is the preferred sensor manufacturer of VGA, its mainstream products: ov7660/ov7663/ov7670 are integrated pre-isp, and MTK will post-isp part integrated into the baseband. [52rd.com]
L Baseband not integrated post-isp; [52rd.com]
The ISP (PRE-ISP&POST-ISP) function is integrated in MMP, and a certain proportion of baseband vendors (e.g. Infineon) do not integrate Post-isp into the baseband. Instead, it is handled in a way that uses MMP. [52rd.com]
2.0Mega and above YUV/RGBL sensor integrated pre-isp; [52rd.com]
The ISP (PRE-ISP&POST-ISP) function is integrated into the MMP. Baseband does not participate in image processing mobile phone design companies can use the sensor output YUV or RGB, or directly to the Pre-isp shield off, all the image processing work by the MMP chip to do. (in theory, the MMP processing effect should be better, in fact, sometimes because of the level of debugging engineers, can not fully play the potential of MMP.) ) [52rd.com]
Pre-isp not integrated in Raw datal sensor; [52rd.com]
The ISP (Pre-isp & POST-ISP) functionality is integrated into the MMP. Baseband does not participate in image processing for all image processing work done by the MMP chip. [52rd.com]
[52rd.com]
In the future, the baseband chip integrates all the image processing functions for cost reasons. [52rd.com]
4. Rating [52rd.com]
LIn terms of cost, the relationship between the ISP and sensor is a departure from integration. The cost of integrating the ISP on the sensor is much higher than the ISP integration on the MMP or baseband. As the resolution of the sensor increases, more and more sensors will only integrate the circuitry necessary to output the digital signal, and integrate functions such as image processing and compression into the MMP or baseband. [52rd.com]
In terms of imaging quality, the trend of development is also a departure from integration, which integrates powerful image processing capabilities in MMP, and this trend is becoming more pronounced with resolution. [52rd.com]
L The above two reviews are the development trend, but now the market situation of various application programs exist, and there is no obvious strength of the points. MTK Soc is dominated by low-end VGA products, and there are a large number of advocates, but there are still a large number of platform companies still do not take the road of baseband integration. and high-end products, from my understanding of the ISP has not been integrated into the baseband, most still use the MMP scheme. However, the data show that the manufacturing cost of integrating ISP functions into baseband is negligible, and there is still a cost space compared to the MMP scheme. Big players in baseband products and application processors are looking to add ISP functionality to their chips. [52rd.com]
In any case, whether it is the current low-end market situation, or the future of high-end products, independent ISP (MMP) chip has a larger market space. The era of true high-pixel SOC baseband solutions is still a long way off.