Introduction to the camera framework

Camera applications are typical Android multimedia systems designed using the C/S architecture. The client and server communicate with each other through the Binder Mechanism between two independent threads. The following figure shows the overall architecture of camera.

From the camera. java (packages \ app \ camera), to the camera of the Framework Java section. java (Framework \ base \ core \ Java \ Android \ hardware), go to android_hardware_camera (Framework \ base \ core) on the JNI layer, and go to camera in framework C ++. CPP and cameraservice. CPP, camera2client. CPP. The binder data communication structure of framework C ++ is as follows:

The following describes the initialization of camera and the preview process of camera.

Camera Initialization

Because camera communicates through the binder, it is bound to register its service in servicemanager. The corresponding path is a main function under frameworks \ AV \ media \ mediaserver \ main_mediaserver.cpp, where cameraservice: instantiate () is used to register the cameraservice service. Next, let's look back at this process from the initialization of cameraapp.

After clicking the camera application icon, the camera application starts a thread named camerastartupthread in the oncreate () method of the activity. In this thread, there are several steps from start to preview:

1. Open camera

2. applyxxxcallbacks

3. applyparameters

4. startpreview

The registration callback function is mainly related to the clicking of the icon in the camera preview window, And applyparameter is mainly related to the parameter settings of camera (such as whether to enable Flash and what is the picture size ). The two will be expanded later. Here we mainly analyze opencamera and startpreview.

Util. opencamera (cameraid)

--> Cameraholder. instance (). Open (cameraid)

--> Cameramanager. instance (). cameraopen (cameraid)

--> Frameworksclassfactory. opencamera (cameraid)

--> Camera. Open (cameraid)

Camera is the camera of the Framework Java section, that is, camera. Java under Framework \ base \ core \ Java \ Android \ hardware. Here, the open action creates a camera instance, and the instance initializes and calls native_setup (newweakreference <camera> (this), cameraid, packagename ). From the function name, we can see that the call is transferred to the JNI layer. Through the JNI naming rule, we find android_hardware_camera.cpp in android_hardware_camera_native_setup (). Call relationship:

Camera. Open (cameraid)

--> New camera (cameraid)

--> Native_setup (...);

--> Sp <camera> camera = camera: connect (...)

--> Camerabaset: connect (...)

--> Const sp <icameraservice> & cs = getcameraservice ();

--> (CS. Get ()-> * fnconnectservice )(...)

I noticed that camera. in CPP, the value of fnconnectservice is cameratraits <camera >:: tcamconnectservicecameratraits <camera >:: fnconnectservice = & icameraservice: connect; from the above icon, we know that icameraservice communicates with binder, the connect branch is used in the bpcameraservice (client), bncameraservice (server), bpcameraservice: connect à bncameraservice: ontransact, and switch statements. Note the following two statements:

Sp <icameraclient> cameraclient =

Interface_cast <icameraclient> (data. readstrongbinder ());

Status_tstatus = connect (cameraclient, cameraid,

Clientname, clientuid,/* out */camera );

From binder knowledge, connect to connect in cameraservice, where cameraclient is created as camera2client. When the server cameraservice is created. CPP is created when obtaining the server object from iservicemanager. constsp <icameraservice> & cs = getcameraservice (); As mentioned above, an instance is created during cameraservice initialization and called onfirstref ().Hw_get_moduleThe function is used to obtain the Hal stub of the module.Camera_hardware_module_idObtainCamera halLayer proxyStubAnd assign the value to the mmodule.MmoduleComplete pairCameraModule control. The above basically completes the initialization of camera. Next, let's take a look at the preview process of camera.

 

Camerapreview Process

The camera at the application layer generally does not set the previewcallback callback. Unless setpreviewcallback is set in your app, you can see that the data in the preview can still be called back to the upper layer, but the system does not call back by default, in addition, the buffer Preview data delivery in the data collection area and display area is completed at the Hal layer to complete the real-time display of the preview. Next we will continue to track the code process from the upper layer of the application.

 

Camera. Java (CAMERA Application)

--> Camera. Java (frameworokjava)

--> Android_hardware_camera.cpp (JNI)

--> Camera. cpp (client)

--> Cameraservice. cpp (server)

--> Cameraharwareinterface (halinterface)

The Code flow from startpreview to JNI layer on the upper layer is roughly as follows:

Startpreview ()

1) --> mcamera. getcameradevice (). startpreviewasync ()

2) --> startpreviewasync ()

3) --> mcamera. startpreview () (Framework android_hardware_camera)

4) --> android_hardware_camera_startpreview (...)

 

(2) mcamera. getcameradevice is the util returned during camera initialization described above. opencamera (), get cameramanager. for the cameraproxy instance, calling startpreviewasync () actually sends a message start_preview_async. After receiving the message, 3) Call icamera's startpreview. Here, the startpreview corresponding to android_hardware_camera in frameworkjava is forwarded to the JNI layer for processing. The general process is as follows:

Icamera: startpreview ()

--> Icamera: bpcamerastartpreview ()

--> Icamera: bncameraontransact () start_preview case Branch

In the camera2client, the preview request is processed and enters the Hal layer.

Status_t camera2client: startpreview (){...... Return startpreviewl (L. mparameters, false);} the Data Processing of camera2client is related to streamingprocessor. Status_t camera2client: startpreviewl (parameters milliseconds ms, bool restart ){...... If (! Mstreamingprocessor-> havevalidpreviewwindow ()){......} Res = mstreamingprocessor-> updatepreviewstream (Params );...... If (! Params. recordinghint ){...... Res = mstreamingprocessor-> updatepreviewrequest (Params); Res = mstreamingprocessor-> startstream (streamingprocessor: preview, outputstreams);} else {...... Res = mstreamingprocessor-> updaterecordingrequest (Params); Res = mstreamingprocessor-> startstream (streamingprocessor: Record, outputstreams );}}

If the upper-layer application wants to obtain the Preview data, it needs to call setpreviewdisplay (surfaceholder), and the data will be transmitted to the upper-layer surfaceview. In the red part of the above function, in mstreamingprocessor-> updatepreviewstream (Params), The surfaceview on the upper layer is converted to the igraphicbufferproducer type, converted to the surface on the lower layer in camera2client, and passed streamingprocessor, point to mpreviewwindow. The Preview data is also specified to be passed to the surface.

Finally, create a previewrequest in updatepreviewrequest (Params) and update stream to outputstreams.

 

Introduction to the camera framework