Android Camera System Framework Analysis

1. What work can developers do in Android?
Application Development: Use the powerful SDK provided by Android to develop a variety of novel applications.
System development: In Android, Google implements all the code unrelated to the hardware, but the hardware abstraction layer closely related to the hardware cannot be provided, the underlying hardware of different device providers for mobile devices is ever-changing. It is impossible to provide a unified hardware driver and interface implementation, and only standard interfaces can be provided. Therefore, hardware providers need to develop device drivers by themselves,
And implement the interfaces provided by the android framework.
Ii. camera system source code analysis in the android framework
Each Android phone has a camera application for photo taking. Different hardware providers may change this application to suit their UI style,
Here we only analyze the android native camera application and framework (Android 4.0)
The native camera application code is in camera. Java (android4.0 \ packages \ apps \ camera \ SRC \ com \ Android \ camera). This should be regarded as the upper-layer and application-layer Implementation of the camera system.
The following is the code of the camera class.

public class Camera extends ActivityBase implements FocusManager.Listener,        View.OnTouchListener, ShutterButton.OnShutterButtonListener,        SurfaceHolder.Callback, ModePicker.OnModeChangeListener,        FaceDetectionListener, CameraPreference.OnPreferenceChangedListener,        LocationManager.Listener, ShutterButton.OnShutterButtonLongPressListener

As shown above, camera inherits many listening interfaces to listen to various events (Focus events, user touch events, etc ). This application inherits activitybase,
You can reload the oncreate, onresume, and other interfaces to complete initialization in these interfaces, basically Initializing Various listening objects and obtaining camera parameters.
The key is in the doonresume function:

    @Override    protected void doOnResume() {        if (mOpenCameraFail || mCameraDisabled) return;        mPausing = false;        mJpegPictureCallbackTime = 0;        mZoomValue = 0;        // Start the preview if it is not started.        if (mCameraState == PREVIEW_STOPPED) {            try {                mCameraDevice = Util.openCamera(this, mCameraId);                initializeCapabilities();                resetExposureCompensation();                startPreview();                if (mFirstTimeInitialized) startFaceDetection();            } catch (CameraHardwareException e) {                Util.showErrorAndFinish(this, R.string.cannot_connect_camera);                return;            } catch (CameraDisabledException e) {                Util.showErrorAndFinish(this, R.string.camera_disabled);                return;            }        }        if (mSurfaceHolder != null) {            // If first time initialization is not finished, put it in the            // message queue.            if (!mFirstTimeInitialized) {                mHandler.sendEmptyMessage(FIRST_TIME_INIT);            } else {                initializeSecondTime();            }        }        keepScreenOnAwhile();        if (mCameraState == IDLE) {            mOnResumeTime = SystemClock.uptimeMillis();            mHandler.sendEmptyMessageDelayed(CHECK_DISPLAY_ROTATION, 100);        }    }

In this function, we can see that this function is used to obtain the underlying camera object.
Mcameradevice = util. opencamera (this, mcameraid). The util class is used here. The implementation of this class is
In util. Java (android4.0 \ packages \ apps \ camera \ SRC \ com \ Android \ camera), find the opencamera function implementation:

public static android.hardware.Camera openCamera(Activity activity, int cameraId)            throws CameraHardwareException, CameraDisabledException {        // Check if device policy has disabled the camera.        DevicePolicyManager dpm = (DevicePolicyManager) activity.getSystemService(                Context.DEVICE_POLICY_SERVICE);        if (dpm.getCameraDisabled(null)) {            throw new CameraDisabledException();        }        try {            return CameraHolder.instance().open(cameraId);        } catch (CameraHardwareException e) {            // In eng build, we throw the exception so that test tool            // can detect it and report it            if ("eng".equals(Build.TYPE)) {                throw new RuntimeException("openCamera failed", e);            } else {                throw e;            }        }    }

From this function, we can see that the lower-layer camera management in the Android system is managed through a singleton mode cameraholder,
Locate the implementation of this class cameraholder. Java (android4.0 \ packages \ apps \ camera \ SRC \ com \ Android \ camera) and call the OPEN function to obtain a camera hardware device object,
Because the camera device is an exclusive device and cannot be occupied by two processes at the same time, and the entire android system is a multi-process environment, you need to add some methods for mutual exclusion and synchronization between processes.
Find the OPEN function of this class:

    public synchronized android.hardware.Camera open(int cameraId)            throws CameraHardwareException {        Assert(mUsers == 0);        if (mCameraDevice != null && mCameraId != cameraId) {            mCameraDevice.release();            mCameraDevice = null;            mCameraId = -1;        }        if (mCameraDevice == null) {            try {                Log.v(TAG, "open camera " + cameraId);                mCameraDevice = android.hardware.Camera.open(cameraId);                mCameraId = cameraId;            } catch (RuntimeException e) {                Log.e(TAG, "fail to connect Camera", e);                throw new CameraHardwareException(e);            }            mParameters = mCameraDevice.getParameters();        } else {            try {                mCameraDevice.reconnect();            } catch (IOException e) {                Log.e(TAG, "reconnect failed.");                throw new CameraHardwareException(e);            }            mCameraDevice.setParameters(mParameters);        }        ++mUsers;        mHandler.removeMessages(RELEASE_CAMERA);        mKeepBeforeTime = 0;        return mCameraDevice;    }

Android. hardware. camera. open (cameraid) calls enter the next layer of encapsulation, namely the JNI layer, which is the lowest layer of Java code. It encapsulates the lower-layer camerac ++ code in JNI and encapsulates the implementation class in camera. java (android4.0 \ frameworks \ base \ core \ Java \ Android \ hardware) is part of the implementation of this class, which defines a number of callback functions:

public class Camera {    private static final String TAG = "Camera";    // These match the enums in frameworks/base/include/camera/Camera.h    private static final int CAMERA_MSG_ERROR            = 0x001;    private static final int CAMERA_MSG_SHUTTER          = 0x002;    private static final int CAMERA_MSG_FOCUS            = 0x004;    private static final int CAMERA_MSG_ZOOM             = 0x008;    private static final int CAMERA_MSG_PREVIEW_FRAME    = 0x010;    private static final int CAMERA_MSG_VIDEO_FRAME      = 0x020;    private static final int CAMERA_MSG_POSTVIEW_FRAME   = 0x040;    private static final int CAMERA_MSG_RAW_IMAGE        = 0x080;    private static final int CAMERA_MSG_COMPRESSED_IMAGE = 0x100;    private static final int CAMERA_MSG_RAW_IMAGE_NOTIFY = 0x200;    private static final int CAMERA_MSG_PREVIEW_METADATA = 0x400;    private static final int CAMERA_MSG_ALL_MSGS         = 0x4FF;    private int mNativeContext; // accessed by native methods    private EventHandler mEventHandler;    private ShutterCallback mShutterCallback;    private PictureCallback mRawImageCallback;    private PictureCallback mJpegCallback;    private PreviewCallback mPreviewCallback;    private PictureCallback mPostviewCallback;    private AutoFocusCallback mAutoFocusCallback;    private OnZoomChangeListener mZoomListener;    private FaceDetectionListener mFaceListener;    private ErrorCallback mErrorCallback;

Find the OPEN function:
Public Static camera open (INT cameraid ){
Return new camera (cameraid );
}
The open function is a static method that constructs a camera object:

    Camera(int cameraId) {        mShutterCallback = null;        mRawImageCallback = null;        mJpegCallback = null;        mPreviewCallback = null;        mPostviewCallback = null;        mZoomListener = null;        Looper looper;        if ((looper = Looper.myLooper()) != null) {            mEventHandler = new EventHandler(this, looper);        } else if ((looper = Looper.getMainLooper()) != null) {            mEventHandler = new EventHandler(this, looper);        } else {            mEventHandler = null;        }        native_setup(new WeakReference<Camera>(this), cameraId);    }

Call the native_setup method in the constructor. This method corresponds to the android_hardware_camera_native_setup method of the c ++ code,
The implementation is in android_hardware_camera.cpp (android4.0 \ frameworks \ base \ core \ JNI). The specific code is as follows:

static void android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz,    jobject weak_this, jint cameraId){    sp<Camera> camera = Camera::connect(cameraId);    if (camera == NULL) {        jniThrowRuntimeException(env, "Fail to connect to camera service");        return;    }    // make sure camera hardware is alive    if (camera->getStatus() != NO_ERROR) {        jniThrowRuntimeException(env, "Camera initialization failed");        return;    }    jclass clazz = env->GetObjectClass(thiz);    if (clazz == NULL) {        jniThrowRuntimeException(env, "Can't find android/hardware/Camera");        return;    }    // We use a weak reference so the Camera object can be garbage collected.    // The reference is only used as a proxy for callbacks.    sp<JNICameraContext> context = new JNICameraContext(env, weak_this, clazz, camera);    context->incStrong(thiz);    camera->setListener(context);    // save context in opaque field    env->SetIntField(thiz, fields.context, (int)context.get());}

The Connect Method of the camera object is called in the android_hardware_camera_native_setup method. The camera class declaration is in camera. H (android4.0 \ frameworks \ base \ include \ camera)
Locate the connect method:

sp<Camera> Camera::connect(int cameraId){    LOGV("connect");    sp<Camera> c = new Camera();    const sp<ICameraService>& cs = getCameraService();    if (cs != 0) {        c->mCamera = cs->connect(c, cameraId);    }    if (c->mCamera != 0) {        c->mCamera->asBinder()->linkToDeath(c);        c->mStatus = NO_ERROR;    } else {        c.clear();    }    return c;}

The following code is critical, involving the implementation mechanism of the camera framework. The camera system uses the server-client mechanism, and the service and client are in different processes, processes use the Binder Mechanism for communication,
The service actually implements camera-related operations, and the client calls the corresponding operations of the Service through the binder interface.
Continue to analyze the code. The above function calls the getcameraservice method to obtain reference from cameraservice. icameraservice has two subclasses: bncameraservice and bpcameraservice.
The ibinder interface is inherited. The two subclasses implement the two ends of the binder communication. bnxxx implements the specific functions of icameraservice. bpxxx encapsulates the icameraservice method using the communication functions of the binder, as follows:

class ICameraService : public IInterface{public:    enum {        GET_NUMBER_OF_CAMERAS = IBinder::FIRST_CALL_TRANSACTION,        GET_CAMERA_INFO,        CONNECT    };public:    DECLARE_META_INTERFACE(CameraService);    virtual int32_t         getNumberOfCameras() = 0;    virtual status_t        getCameraInfo(int cameraId,                                          struct CameraInfo* cameraInfo) = 0;    virtual sp<ICamera>     connect(const sp<ICameraClient>& cameraClient,                                    int cameraId) = 0;};// ----------------------------------------------------------------------------class BnCameraService: public BnInterface<ICameraService>{public:    virtual status_t    onTransact( uint32_t code,                                    const Parcel& data,                                    Parcel* reply,                                    uint32_t flags = 0);};}; // na

class BpCameraService: public BpInterface<ICameraService>{public:    BpCameraService(const sp<IBinder>& impl)        : BpInterface<ICameraService>(impl)    {    }    // get number of cameras available    virtual int32_t getNumberOfCameras()    {        Parcel data, reply;        data.writeInterfaceToken(ICameraService::getInterfaceDescriptor());        remote()->transact(BnCameraService::GET_NUMBER_OF_CAMERAS, data, &reply);        return reply.readInt32();    }    // get information about a camera    virtual status_t getCameraInfo(int cameraId,                                   struct CameraInfo* cameraInfo) {        Parcel data, reply;        data.writeInterfaceToken(ICameraService::getInterfaceDescriptor());        data.writeInt32(cameraId);        remote()->transact(BnCameraService::GET_CAMERA_INFO, data, &reply);        cameraInfo->facing = reply.readInt32();        cameraInfo->orientation = reply.readInt32();        return reply.readInt32();    }    // connect to camera service    virtual sp<ICamera> connect(const sp<ICameraClient>& cameraClient, int cameraId)    {        Parcel data, reply;        data.writeInterfaceToken(ICameraService::getInterfaceDescriptor());        data.writeStrongBinder(cameraClient->asBinder());        data.writeInt32(cameraId);        remote()->transact(BnCameraService::CONNECT, data, &reply);        return interface_cast<ICamera>(reply.readStrongBinder());    }};IMPLEMENT_META_INTERFACE(CameraService, "android.hardware.ICameraService");// ----------------------------------------------------------------------status_t BnCameraService::onTransact(    uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags){    switch(code) {        case GET_NUMBER_OF_CAMERAS: {            CHECK_INTERFACE(ICameraService, data, reply);            reply->writeInt32(getNumberOfCameras());            return NO_ERROR;        } break;        case GET_CAMERA_INFO: {            CHECK_INTERFACE(ICameraService, data, reply);            CameraInfo cameraInfo;            memset(&cameraInfo, 0, sizeof(cameraInfo));            status_t result = getCameraInfo(data.readInt32(), &cameraInfo);            reply->writeInt32(cameraInfo.facing);            reply->writeInt32(cameraInfo.orientation);            reply->writeInt32(result);            return NO_ERROR;        } break;        case CONNECT: {            CHECK_INTERFACE(ICameraService, data, reply);            sp<ICameraClient> cameraClient = interface_cast<ICameraClient>(data.readStrongBinder());            sp<ICamera> camera = connect(cameraClient, data.readInt32());            reply->writeStrongBinder(camera->asBinder());            return NO_ERROR;        } break;        default:            return BBinder::onTransact(code, data, reply, flags);    }}// ----------------------------------------------------------------------------}; // namespace android

Next, analyze the sp <camera> camera: connect (INT cameraid) method and locate the getcameraservice method.

const sp<ICameraService>& Camera::getCameraService(){    Mutex::Autolock _l(mLock);    if (mCameraService.get() == 0) {        sp<IServiceManager> sm = defaultServiceManager();        sp<IBinder> binder;        do {            binder = sm->getService(String16("media.camera"));            if (binder != 0)                break;            LOGW("CameraService not published, waiting...");            usleep(500000); // 0.5 s        } while(true);        if (mDeathNotifier == NULL) {            mDeathNotifier = new DeathNotifier();        }        binder->linkToDeath(mDeathNotifier);        mCameraService = interface_cast<ICameraService>(binder);    }    LOGE_IF(mCameraService==0, "no CameraService!?");    return mCameraService;}

Locate mcameraservice = interface_cast <icameraservice> (binder); mcameraservice is an icamerservice type. More specifically, it should be bpcameraservice,
Because the icameraservice method is implemented in this class.

To sum up the above Binder Mechanism, we only need to analyze the binder usage and do not go into the underlying implementation. The basic steps are as follows:
1. Define the interface for inter-process communication, for example, icameraservice;
2. implement this interface in bncameraservice and bpcamaraservice. These two interfaces also inherit from bninterface and bpinterface respectively;
3. The server registers the binder with servicemanager, and the client obtains the binder from servicemanager;
4. Then we can implement two-way inter-process communication;

After icameraservice is referenced by icameraservice through getcameraservice, call the Connect Method of icameraservice to obtain icamera reference,

c->mCamera = cs->connect(c, cameraId);
Follow up the connect method further. Here is the specific implementation of the Connect Method in the bpcameraservice class.
    virtual sp<ICamera> connect(const sp<ICameraClient>& cameraClient, int cameraId)    {        Parcel data, reply;        data.writeInterfaceToken(ICameraService::getInterfaceDescriptor());        data.writeStrongBinder(cameraClient->asBinder());        data.writeInt32(cameraId);        remote()->transact(BnCameraService::CONNECT, data, &reply);        return interface_cast<ICamera>(reply.readStrongBinder());    }
The icamera object returned here is actually a bpcamera object. Here we use an anonymous binder. We get the famous binder used by cameraservice. The famous binder needs to get the binder by using servicemanager, the anonymous binder can be obtained through the established communication channel (famous binder. The preceding section shows the implementation of the camera framework. The specific method for implementing the camera is the interface related to icamera. The following is the definition of the interface:
class ICamera: public IInterface{public:    DECLARE_META_INTERFACE(Camera);    virtual void            disconnect() = 0;    // connect new client with existing camera remote    virtual status_t        connect(const sp<ICameraClient>& client) = 0;    // prevent other processes from using this ICamera interface    virtual status_t        lock() = 0;    // allow other processes to use this ICamera interface    virtual status_t        unlock() = 0;    // pass the buffered Surface to the camera service    virtual status_t        setPreviewDisplay(const sp<Surface>& surface) = 0;    // pass the buffered ISurfaceTexture to the camera service    virtual status_t        setPreviewTexture(            const sp<ISurfaceTexture>& surfaceTexture) = 0;    // set the preview callback flag to affect how the received frames from    // preview are handled.    virtual void            setPreviewCallbackFlag(int flag) = 0;    // start preview mode, must call setPreviewDisplay first    virtual status_t        startPreview() = 0;    // stop preview mode    virtual void            stopPreview() = 0;    // get preview state    virtual bool            previewEnabled() = 0;    // start recording mode    virtual status_t        startRecording() = 0;    // stop recording mode    virtual void            stopRecording() = 0;    // get recording state    virtual bool            recordingEnabled() = 0;    // release a recording frame    virtual void            releaseRecordingFrame(const sp<IMemory>& mem) = 0;    // auto focus    virtual status_t        autoFocus() = 0;    // cancel auto focus    virtual status_t        cancelAutoFocus() = 0;    /*     * take a picture.     * @param msgType the message type an application selectively turn on/off     * on a photo-by-photo basis. The supported message types are:     * CAMERA_MSG_SHUTTER, CAMERA_MSG_RAW_IMAGE, CAMERA_MSG_COMPRESSED_IMAGE,     * and CAMERA_MSG_POSTVIEW_FRAME. Any other message types will be ignored.     */    virtual status_t        takePicture(int msgType) = 0;    // set preview/capture parameters - key/value pairs    virtual status_t        setParameters(const String8& params) = 0;    // get preview/capture parameters - key/value pairs    virtual String8         getParameters() const = 0;    // send command to camera driver    virtual status_t        sendCommand(int32_t cmd, int32_t arg1, int32_t arg2) = 0;    // tell the camera hal to store meta data or real YUV data in video buffers.    virtual status_t        storeMetaDataInBuffers(bool enabled) = 0;};
The icamera interface has two subclasses: bncamera and bpcamera, which are the two ends of the binder communication. bpcamera provides the client call interface, which encapsulates the specific implementation of bncamera, bncamera does not actually implement icamera-related interfaces, but is implemented in the bncamera subclass cameraservice: client. In the cameraservice: client class, the relevant methods in the hardware abstraction layer will be called to implement the specific camera function. Now we will take a look at how the camera classes in Android are connected .... Unfinished