Camera ISP.

 

1. Target [52rd.com]

The Market Trend and R & D direction of the mobile phone camera module's ISP function module. It provides technical and market basis for a correct understanding of the mobile phone camera module industry. [52rd.com]

2. Application Principle of ISP in the module [52rd.com]
2.1 functional areas [52rd.com]
Regardless of the digital camera, camera, or mobile phone, after sensing the image data from the front end, it must go through ASP (analog signal processing), ADC (analog-digital converter) after the four phases of pre-ISP and post-ISP, the image data can be finally presented on the terminal device (figure 1 ).

However, due to the different pixel levels and other costs of image sensors, ISP functional areas are configured or integrated according to the characteristics of the mobile phone market. For example, the low-end camera integrates pre-ISP and sensor, and the M pixel mobile phone makes all ISP functions into a single chip.
[52rd.com]
2.1.1 ASP [52rd.com]

ASP (analog signal processor) is mainly used to process the voltage or current signals collected by image sensors. It is mainly used for signal amplification, Automatic Exposure adjustment, timing control, and pixel sampling control. Because of its absolute correlation with the initial signal, generally image sensor manufacturers will directly integrate this function with the sensor. In Figure 1, it is the blue part. After ASP processing, raw data is output.
[52rd.com]
2.1.2 pre-ISP [52rd.com]

Pre-ISP (image signal processor) is used for front-end image processing, mainly for raw data transmitted after ADC conversion ), repairs bad image points, white balance, Gamma Correction, sharpness, and color interpolation. In low-pixel products, for example, 0.3m pixels, pre-ISP and image sensor are deployed in the same chip because there is little image data and no complicated processing is required; however, the high-pixel CMOS sensor requires an increasing number of workers to be processed. Although it is not difficult to integrate pre-ISP into the sensor in terms of manufacturing technology, due to the cost and imaging quality, some mobile phone design companies are designed to block the pre-ISP function integrated into the sensor and maintain the sensor.
Raw data output by the ADC is processed by a separate ISP chip or an ISP Integrated In baseband. [52rd.com]

Data Processed by pre-ISP can be divided into RGB and YUV. RGB is the primary color, which is larger than YUV and can be easily changed by post-ISP; YUV is obtained by interpolation of RGB primary colors, with a small amount of data, but it is not conducive to subsequent processing units.
[52rd.com]
Powerful mobile phone R & D companies tend to use raw data or RGB data, so that they can adjust the image quality as needed. Raw data is also the future direction of use in the high-end market, but the current domestic M pixel module application is still in its infancy. Therefore, YUV is the main output.
[52rd.com]
2.1.3 post-ISP [52rd.com]

Although post-ISP is also called backend impact processing, it does not have much imaging-related work. It is mainly responsible for data compression, interface control of backend interfaces, and data transmission and control, it also includes LCD image preview, lens focus control, and user interface.
[52rd.com]
2.2 ISP development phase [52rd.com]

2.2.1 Phase 1 [52rd.com]

In the first phase, because the pixels are low (CIF level), the structure is similar to that of General mobile phones. Pre-ISP functions are integrated with image sensors, while post-ISP relies on cell phone baseband chips, the display is provided by the system. Its structure is as follows:
[52rd.com]
[52rd.com]
Figure 2 schematic diagram of the first phase of the backend chip of the mobile phone camera Module
[52rd.com]
2.2.2 Stage 2 [52rd.com]

With the increase of pixels (CIF to VGA), to maintain the communication function of the Baseband Chip of the backend mobile phone, the functions such as JPEG compression and coordinated data stream processing are independently completed into a single chip, the pre-ISP of image processing is determined by the mobile phone design company, which is integrated into the image sensor and post-ISP.
[52rd.com]
This phase of solution has been eliminated, and the post-ISP function of VGA pixels can be integrated with baseband.

Figure 3 schematic diagram of the second phase of the backend chip of the mobile phone camera module [52rd.com]

2.2.3 Stage 3 [52rd.com]
At this time, pixels have been converted from VGA to millions of pixels, and high-pixel pre-ISP. Due to the large demand for memory and power, the sensor end could not be processed in full under the current technical conditions, therefore, pre-ISP and post-ISP are integrated into a complete ISP. In addition, the display control function is integrated at this stage to reduce the processing burden of baseband. See figure 4.

Figure 4 schematic diagram of the third phase of the backend chip of the mobile phone camera Module
[52rd.com]
2.2.4 Stage 4 [52rd.com]

Main Panel [52rd.com]

(240*320) [52rd.com]

[52rd.com]
As pixels increase to m and above, manufacturers also integrate more functions such as MP3, 3D sound processor, and audio processor into separate ISP chips, mobile Multimedia processor ),

Figure 5 schematic diagram of the fourth phase of the backend chip of the mobile phone camera Module
[52rd.com]
Table 1 Development Trend of mobile phone camera module backend chips
[52rd.com]
Phase-based pixel trend project [52rd.com]

The cifl structure of the first phase is the same as that of a general mobile phone.
[52rd.com]
L pre-ISP functions can be integrated with Image Sensors
[52rd.com]
L post-ISP relies on cell phone Baseband Chip
[52rd.com]
Phase 2 vgal post-ISP independently becomes a Single Chip
[52rd.com]
L pre-ISP or integrated by sensor or with post-ISP
[52rd.com]
In stage 3, megal image sensors cannot fully process pre-ISP, and are integrated with post-ISP into a separate ISP chip.
[52rd.com]
Stage 4: 2 mega and above l separate MMP ms (ISP) chips integrate more functions to cater to the Development of 2G and 3G
[52rd.com]
[52rd.com]
3. Main ISP application solutions at present [52rd.com]

Table 2 main ISP application solutions at present [52rd.com]

Description of the pixel output format ISP or mobile phone design scheme
[52rd.com]
Vgayuv/RGB/[52rd.com]

Raw datal sensor integrates pre-ISP;
[52rd.com]
L baseband integrates post-ISP. OV is the preferred sensor manufacturer for VGA. Its mainstream products: ov7660, ov7663, and ov7670 are all integrated with pre-ISP, while MTK integrates post-ISP into baseband.
[52rd.com]
L baseband is not integrated with post-ISP;
[52rd.com]
L ISP (pre-ISP & post-ISP) functions are integrated into MMP; a certain proportion of baseband vendors (such as infineon) do not integrate post-ISP into baseband. Instead, it adopts the matrix-based processing (MMP ms) method.
[52rd.com]
Integrate pre-ISP in 2.0mega and above YUV/rgbl sensor;
[52rd.com]
L ISP (pre-ISP & post-ISP) functions are integrated into MMP. Baseband is not involved in image processing. Mobile phone design companies can use YUV or RGB output by sensor, or directly shield pre-ISP. All image processing work is done by the MMP chip. (In theory, the processing performance of MMP is better. In fact, due to the level of debugging engineers, the potential of MMP is not fully realized .)
[52rd.com]
Raw datal sensor does not integrate pre-ISP;
[52rd.com]
L ISP (pre-ISP & post-ISP) functions are integrated into MMP. Baseband is not involved in all image processing tasks.
[52rd.com]
[52rd.com]
In the future, the Baseband Chip will integrate all the image processing functions for cost reasons.
[52rd.com]
4. Comment [52rd.com]

LIn terms of cost, the relationship between ISP and sensor deviates from integration. The cost of integrating ISP on sensor is much higher than that of integrating ISP on MMP or baseband. As the resolution of sensors increases, more and more sensors will integrate only the circuits necessary to output digital signals, and integrate functions such as image processing and compression into MMP or baseband. [52rd.com]

L from the perspective of imaging quality, the development trend is also different from integration. The powerful image processing functions are integrated in MMP, and this trend will become more and more obvious as resolution increases. [52rd.com]

L the above two ratings are the development trend, but the current market conditions of various application solutions exist, and there is no obvious strength or weakness. Mtk soc is dominant in low-end VGA products and has a large number of advocates. However, a large number of platform companies are still not on the path of baseband integration. In addition, I have not integrated ISP functions into baseband from my understanding of high-end products, and most of them still adopt the MMP solution. However, it is suggested that the manufacturing cost of integrating ISP functions into baseband is negligible. This solution is still more costly than the MMP solution. Major players in the baseband product and application processor fields are looking to add ISP functions to their chips. [52rd.com]

L no matter what the current low-end market situation or future high-end products, independent ISP (MMP) chips have a large market space. There is still a long way to go in the era of real high-pixel SOC baseband solutions.