Filter development: filter, filter development: Filter

Filter development: filter, filter development: Filter
Filter development: Filter

Filter

Generally, an object made of glass, resin, polycarbonate is used to handle the impact of cameras and other sensors on the external environment, or to achieve a certain visual effect. For images in the digital age, we can use algorithms to achieve the effects of previous lenses.

Filter development on the Android platform

Processing Method

Java layer: Bitmap pixel points and matrix operations; ColorMatrix and other systems apinative layer: Use the c/c ++ local language, with higher performance, can operate the overall processing of Bitmap data imported into the java layer: that is, the entire image processing, matrix processing, ColorMatrix or system api local processing: Single or partial processing of special pixels cpu processing: The above processing is based on cpu, strong cpu computing capabilities, however, there are few cores and poor gpu processing in parallel: There are many cores and high concurrency. In Android, opengl es can be used to call gpu acceleration to process images in parallel.

For opengl, we can refer to relevant materials and do more than filter.
His core work mode is almost parallel processing of image pixels, while the cpu generally can only process pixels serially.

Gpu processing, better performance, faster. For a static image, simple filter processing can still be performed in the cpu mode. However, when the camera filter and video dynamic filter are used, the cpu may be choppy obviously. In this case, the gpu filter processing is required and competent.

For various processing methods with the cpu, we use Bitmap or system api to implement various filter algorithms, which is easier than gpu.

The implementation scheme of various special effects is usually pixel judgment, overall pixel judgment and processing, mask assisted graph, various image processing algorithms, and color space conversion.

Filters can be used to process images on mobile platforms, such as filters, face-to-face makeup cameras, dynamic pasters, and special video effects.

Processing Method of camera data

YUV is directly drawn by OpenGL without conversion, and Shader is used for image processing (due to processing algorithms and rendering of most images ).
The RGB format is used. This scheme is not considered for the time being. Only the possibility is proposed)

Use C/C ++ to implement YUV-> RGB and image processing, and synthesize Bitmap, which is drawn on the Canvas by the CPU.

YUV-> RGB and image processing are implemented through C/C ++, which are directly drawn on SurfaceView on the ndk layer.

Use C/C ++ to implement YUV-> RGB, use Shader to implement image processing, and use OpenGL to draw images (android-gpuimage)

Use Shader to implement YUV-> RGB and image processing, and use OpenGL to draw

Application

Image: For an image, we can perform various Processing Based on the desired effect and the above methods.

Video: video processing is based on the video source,
The first method is to obtain video data through the camera, usually in YUV format, and then process the data frame by frame;
The second type is filter processing through one or more clips, such as transfer, animation, and slides, and then generate a video;
Third, you can use MediaDecoder or FFmpeg to obtain the video frame for processing the source video.
Note: videos require a high frame rate and cannot be choppy.

Camera filter: YUV video frame-by-frame Processing

Beautify makeup:
For a single static image, face recognition, regional judgment, beauty, coloring, etc.
For cameras: Face Recognition and processing by the number of YUV videos frame by frame

Face dynamic Paster: Same as face Filter

........ Other applications

About face recognition Solutions

You can also implement simple face recognition on your own, but the accuracy and speed are demanding for the production environment. Generally, you can purchase a third-party service. Fortunately, these third-party services are currently available for free testing. However, if they are released to the application market or other commercial applications, You need to purchase services. These identification services are integrated into the apk package in sdk mode and are billed on time or by Time

Follow-up

I am currently engaged in image processing in this area. I hope that through this method, I will summarize and systematically learn relevant knowledge during this period of time and implement the above ideas, methods, and applications.