Analysis of the creation process of Android4.2.2 Multimedia architecture Mediaplay (I.)

This article is the source of their own reading a bit summary, transfer Please specify the source thank you.

Welcome to communicate with you. qq:1037701636 Email:[email protected]

Android source version version:4.2.2; Hardware platform Full Chi A31

Frontier:
Looking back on the past, remember the 2012, when still young do not know, the study of Android Multimedia framework, then still 2.3 of the source code, read the source of Stagefright, remember that was a special pain. Now, once again look at the code of this multimedia module, suddenly feel enlightened, the level of the module between the clear, according to, the question encountered often can be solved. I think, perhaps this is more than 2 years of progress and experience it. Thanks for the time, let me learn to become a talent.
Dengfanping told me: no matter when to enter the Internet is not too late, indeed standing on the shoulders of giants to re-comb the old things, or can learn more deeply.

The next period of time, intends to focus on the Android multimedia framework, camera architecture, Surfaceflinger and other framework layers, HAL layer module and related Android system customization and class tablet development, The lower level is the video capture and display driver in the Linux kernel, which is the most serious part of looking for a job. Self-feeling in the mobile internet and embedded world, these aspects should still be indispensable now.

Android Multimedia frame Familiar people are very familiar with, like me and so on rookie, can only slowly chew. This is the background of the source code of 4.2.2, documenting the core modules of the multimedia framework I am familiar with for later use.

The function of multimedia frame in Android mainly reflects the playback and recording of the noisy video, the former corresponds to decoding, the latter corresponds to the encoding. In Android, a Mediaplay class is used as the basic class for audio and video playback, and a series of processing is carried out around him. To learn a new module, the simplest step is to find a typical application that analyzes the data flow and control flow of the entire module through its implementation. Studies such as Surfaceflinger can be studied with bootanmation initiation. The typical Mediaplay interface in Java includes the video playback class Videoview and the audio-specific Mediaplay class.

1.APP Flash of the Videoview class, its essence is to use the Mediaplay class to achieve, just because it is video playback, have to and Surfaceview hang enough to separate it out. Make it like the following structure:

public class Videoview extends Surfaceview implements MediaPlayerControl {    private String TAG = "Videoview";    Settable by the client    private Uri         MUri;    Private map<string, string> mheaders;

In the app, the typical simple use of Videoview is as follows:

                Video = (videoview) Findviewbyid (r.id.videoview1) Mediacontroller =new Mediacontroller (this); Video.setvideopath ( VIDEO_FD); Mediacontroller.setanchorview (video);        Control and video bindings Video.setmediacontroller (Mediacontroller);  Set the video playback controller video.start ();

Through the Setvideopath API processing, make the following calls in turn:

  public void Setvideopath (String path) {        Setvideouri (uri.parse (path));    }    public void Setvideouri (Uri uri) {        Setvideouri (URI, NULL);    }    /**     * @hide * *    /public void Setvideouri (Uri Uri, map<string, string> headers) {        MUri = URI;        Mheaders = headers;        mseekwhenprepared = 0;        Openvideo ();        Requestlayout ();        Invalidate ();    }

Openvideo processing, let the final processing power to the MediaPlayer.

    private void Openvideo () {        if (MUri = = NULL | | msurfaceholder = = NULL) {//not-ready            for playback just yet, would Try again later            return;        }        Tell the music playback service to pause        //Todo:these constants need to being published somewhere in the framework.< C7/>intent i = new Intent ("Com.android.music.musicservicecommand");        I.putextra ("command", "pause");        Mcontext.sendbroadcast (i);        We shouldn ' t clear the target State, because somebody might has        //called Start () previously        release (false);        try {            mmediaplayer = new MediaPlayer ();. */            Mmediaplayer.setdatasource (Mcontext, MUri, Mheaders) ;.........}

The above two architectures are called more tightly in the audio playback app, as follows:

MediaPlayer = new MediaPlayer (); Mediaplayer.setdatasource (MUSIC_FD);  Set the audio file to play Mediaplayer.prepare (); Mediaplayer.seekto (0);

Here the basic audio and video framework is basically done in the app call.

2. Enter the world of Mediaplay

2.1 First focus on Mediaplay object creation process, which is also an analysis of Android source code a basic requirement. Enter the process flow of the framework (libmedia.so) by Java,jni (libmedia_jni.so) in turn.

New videoview--> New Mediaplay-->native_setup: A typical establishment of an object, and traditional to JNI. Native_setup primarily for the establishment of native C + + layer objects

Into the JNI to do android_media_mediaplayer_native_setup processing, so that eventually into the C + + world.

Android_media_mediaplayer_native_setup (jnienv *env, Jobject thiz, Jobject weak_this) {    ALOGV ("Native_setup");    Sp<mediaplayer> MP = new MediaPlayer ();    if (MP = = NULL) {        jnithrowexception (env, "java/lang/runtimeexception", "Out of Memory");        return;    }    Create new listener and give it to MediaPlayer    sp<jnimediaplayerlistener> listener = new Jnimediaplayerlisten ER (env, thiz, weak_this);    Mp->setlistener (listener);    Stow our new C + + MediaPlayer in a opaque field in the Java object.    Setmediaplayer (env, thiz, MP);}

Well, so far, the real establishment of a so-called native MediaPlayer object. Of course, Java also has this object class.

2.2 Setdatasource

Mediaplay C + + code is located under/home/a31_android4.2.2/android/frameworks/av/media/libmedia to form a libmedia.so.

The following is a look at the processing of this API, and then only analysis of the FW layer of C + + processing flow, Java flow and the above analysis is similar.

status_t mediaplayer::setdatasource (int fd, int64_t offset, int64_t length) {    ALOGV ("Setdatasource (%d,%lld,%LLD)" , FD, offset, length);    status_t err = unknown_error;    Const sp<imediaplayerservice>& Service (Getmediaplayerservice ());    if (service! = 0) {        sp<imediaplayer> player (Service->create (Getpid (), this, Maudiosessionid));// Returns a Bpmediaplayer        if ((No_error! = dosetretransmitendpoint (player)) | |            (No_error! = Player->setdatasource (fd, offset, length))) {//Set video source            player.clear ();        }        Err = Attachnewplayer (player);    }    return err;}

Typical binder c/s architecture, get Mediaplayerservice (MPS) proxy, submitted to the MPs processing.

3. Mediaplayerservice's work.

MPs, like the service of entry, exists as a server, and he is a typical representation of the framework and principles of binder driving. Launch in MediaServer, and other Cameraservice and Audioflinger as multimedia services.

int main (int argc, char** argv) {    signal (sigpipe, sig_ign);    sp<processstate> proc (processstate::self ());    sp<iservicemanager> sm = Defaultservicemanager ();    Alogi ("ServiceManager:%p", Sm.get ());    Audioflinger::instantiate ()///Multimedia service start-up including audio, camera and other    mediaplayerservice::instantiate ();    Cameraservice::instantiate ();    Audiopolicyservice::instantiate ();    Processstate::self ()->startthreadpool ();    Ipcthreadstate::self ()->jointhreadpool ();}

3.1 Processing the Create request:

Sp<imediaplayer> mediaplayerservice::create (pid_t pid, const sp<imediaplayerclient>& Client,        int Audiosessionid)//Create a MediaPlayer, Fanfan a client{    alogv ("Mediaplayerservice::create");    int32_t Connid = Android_atomic_inc (&mnextconnid);    sp<client> C = new Client (this            , PID, Connid, Client, Audiosessionid,            ipcthreadstate::self () Getcallinguid ());//internal class creation, implementation bnmediaplayer ...   }

Create an internal client class for MPs (inherited from Binder native Interface class Bnmediaplay), which looks similar to Cameraservice. With this native Client class, the Mediaplay of the application side only needs to interact with the client, and this is where the anonymous binder works.

3.2 Player->setdatasource ()

The player is a bpbinder derived class that is returned after calling MPs and eventually calls the Setdatasource () of the internal class client of MPs.

status_t mediaplayerservice::client::setdatasource (int fd, int64_t offset, int64_t length) {  ...    Player_type Playertype = Mediaplayerfactory::getplayertype (this,                                                     fd,                                            offset,                                                               length,        true);// Get the type        of player you want to use based on the video source ... Sp<mediaplayerbase> p = setdatasource_pre (playertype);    Now set data source    Setdatasource_post (p, P->setdatasource (FD, offset, length));    return mstatus;}

There is a mediaplayerfactory in this, let's just understand the player manufacturer class. Through it to obtain the current incoming video source of video source suffix format, such as MP4,AVI,3GP. The final assumption is that the currently returned Playertype is Stagefright_player. Then analyze the Setdatasource_pre function:

Sp<mediaplayerbase> mediaplayerservice::client::setdatasource_pre (        player_type playerType) {    // Create the right type of player    sp<mediaplayerbase> p = createplayer (playertype);//Creates a player    if (p = = NULL) { C4/>return p;    }    if (!p->hardwareoutput ()) {        maudiooutput = new Audiooutput (Maudiosessionid);        Static_cast<mediaplayerinterface*> (P.get ())->setaudiosink (maudiooutput);//Forced conversion to Mediaplayerinterface    }    return p;}

3.3 Really create a player:createplayer that is appropriate for the current video file playback needs.

Sp<mediaplayerbase> Mediaplayerservice::client::createplayer (Player_type playerType) {    //determine if we The right player type    sp<mediaplayerbase> p = mPlayer;    if ((P! = NULL) && (P->playertype ()! = Playertype)) {        alogv ("delete player");        P.clear ();    }    if (p = = null) {        p = mediaplayerfactory::createplayer (Playertype, this, notify);//Create a new player    }    if (P! = null) {        p->setuid (muid);    }    return p;}

For the first time, the MPlayer must be empty, so you can see that the vendor player class is eventually returned, because this class maintains the type of player supported by the current platform (speaking of low is the type of decoder).

Sp<mediaplayerbase> Mediaplayerfactory::createplayer (Player_type playertype, void* cookies, not    Ify_callback_f notifyfunc) {sp<mediaplayerbase> p;    Ifactory* Factory;    status_t Init_result;    Mutex::autolock Lock_ (&slock);              if (Sfactorymap.indexofkey (Playertype) < 0) {Aloge ("Failed to create player object of type%d, no registered"        "Factory", Playertype);    return p;    } factory = Sfactorymap.valuefor (Playertype);//Gets a stagefrightplayerfactory CHECK (NULL! = Factory) based on type types; p = Factory->createplayer ();//Call to create a true palyer stagefrightplayerplay if (p = = NULL) {aloge ("Failed to create        Player object of type%d, create failed ", Playertype);    return p;    } Init_result = P->initcheck ();    if (Init_result = = No_error) {p->setnotifycallback (cookie, notifyfunc);            } else {Aloge ("Failed to create player object of type%d, Initcheck Failed"  "(res =%d)", Playertype, Init_result);    P.clear (); } return p;}

A global variable Sfactorymap variable is presented here:

The actual type of Mediaplayerfactory::tfactorymap sfactorymap;//is typedef keyedvector<player_type, ifactory*> TFactoryMap; Keyedvector is a vector class, similar to an array, indexed by an index lookup. He represents a list of currently supported players. So where is the table initialized, and we need to go back to the Mediaplyerservice constructor.

3.4 Registration of information related to the type of player supported by the system:

Mediaplayerservice::mediaplayerservice () {    ALOGV ("Mediaplayerservice created");    Mnextconnid = 1;    Mbatteryaudio.refcount = 0;    for (int i = 0; i < num_audio_devices; i++) {        mbatteryaudio.deviceon[i] = 0;        Mbatteryaudio.lasttime[i] = 0;        Mbatteryaudio.totaltime[i] = 0;    }    Speaker is on by default    mbatteryaudio.deviceon[speaker] = 1;    Mediaplayerfactory::registerbuiltinfactories ();//register to build the maker's play

void Mediaplayerfactory::registerbuiltinfactories () {    mutex::autolock lock_ (&slock);    if (sinitcomplete)        return;    registerfactory_l (New Cedarxplayerfactory (), cedarx_player);    registerfactory_l (New Cedaraplayerfactory (), cedara_player);    registerfactory_l (New Tplayerfactory (), thumbnail_player);    registerfactory_l (New Stagefrightplayerfactory (), stagefright_player);    registerfactory_l (New Nuplayerfactory (), nu_player);    registerfactory_l (New Sonivoxplayerfactory (), sonivox_player);    registerfactory_l (New Testplayerfactory (), test_player);//different player registration    Sinitcomplete = true;}

Here is an example of the stagefright_player we want to use for example to analyze:

A.new Stagefrightplayerfactory () creates a new player class that has the following structure:

Class Stagefrightplayerfactory: Public    mediaplayerfactory::ifactory {public  :    Virtual float Scorefactory (const sp<imediaplayer>& client,                               int fd,                               int64_t offset,                               int64_t length,                               Float curscore) {        char buf[20];        Lseek (FD, offset, seek_set);        Read (FD, buf, sizeof (BUF));        Lseek (FD, offset, seek_set);        Long ident = * ((long*) buf);        Ogg Vorbis?        if (ident = = 0x5367674f)//' Oggs '            return 1.0;        return 0.0;    }    Virtual sp<mediaplayerbase> Createplayer () {        ALOGV ("Create Stagefrightplayer");        return new Stagefrightplayer ();//Create a new Stagefrightplayer    }};

It is obvious that this class is characterized by inheriting and implementing the related functions of the Ifactory interface class.

B. Register the newly created player object, add the index value: type of player, and save its corresponding factory to Sfactorymap in this vector table.

status_t mediaplayerfactory::registerfactory_l (ifactory* factory,                                               player_type type) {    if (NULL = = Factory) {        Aloge ("Failed to register mediaplayerfactory of type%d, factory is"              "NULL.", type);        return bad_value;    }    if (Sfactorymap.indexofkey (type) >= 0) {        aloge ("Failed to register mediaplayerfactory of type%d, type is"              "Al Ready registered. ", type);        return already_exists;    }    if (Sfactorymap.add (type, Factory) < 0) {        aloge ("Failed to register mediaplayerfactory of type%d, Failed to add" c13/> "to map.", type);        return unknown_error;    }    return OK;}

We go back to the program Central 3.3, Through factory = Sfactorymap.valuefor (Playertype);//Gets a stagefrightplayerfactory, which is the factory object that was previously registered, based on type. The actual call to his virtual function, Create_player (), is to implement:

    Virtual sp<mediaplayerbase> Createplayer () {        ALOGV ("Create Stagefrightplayer");        return new Stagefrightplayer ();//Create a new Stagefrightplayer    }

What we're going to see next is the process that really goes into stagefright:

Stagefrightplayer::stagefrightplayer ()    : MPlayer (new Awesomeplayer) {//New Awesomeplayer class, which belongs to Stagefright    ALOGV ("Stagefrightplayer");    Mplayer->setlistener (this);//register Stagefrightplayer to Awesomeplayer class}

3.4 Awesimeplayer into Stagefright interior

Awesomeplayer::awesomeplayer () : Mqueuestarted (False), Muidvalid (false), Mtimesource (NULL), mvideorenderingstarted (false), mvideorend Ererispreview (False), Maudioplayer (NULL), Mdisplaywidth (0), mdisplayheight (0), Mvideoscalingmode (NATIV E_window_scaling_mode_scale_to_window), Mflags (0), mextractorflags (0), Mvideobuffer (NULL), Mdecrypthan Dle (NULL), Mlastvideotimeus ( -1), mtextdriver (null) {Check_eq (Mclient.connect (), (status_t) OK);//omxclient,co    Nnect after maintenance of a momx:bpomx datasource::registerdefaultsniffers ();    Mvideoevent = new Awesomeevent (this, &awesomeplayer::onvideoevent);//Register onvideoevent Event mvideoeventpending = false; Mstreamdoneevent = new Awesomeevent (this, &awesomeplayer::onstreamdone);//Register Onstreamdone event Mstreamdoneeventpend    ing = false; Mbufferingevent = new Awesomeevent (This, &awesomeplayer::onbufferingupdate);//Register Onbufferingupdate mbufferingeventpending = false;    Mvideolagevent = new Awesomeevent (this, &awesomeplayer::onvideolagupdate);    Mvideoeventpending = false;    Mcheckaudiostatusevent = new Awesomeevent (this, &awesomeplayer::oncheckaudiostatus);    Maudiostatuseventpending = false; Reset ();}

Awesomeplay constructor, the main process is to establish a number of event processing registration, the specific event processing mechanism to share in the next article