Mediaplayer architecture of Android

This article mainly introduces the architecture of mediaplayer, which is very important and most complex in Android. For a complete and relatively complex system such as Android, the implementation of a mediaplayer function is not specific to its functions, but how to adapt specific functions to Android mediaplayer is mainly implemented in the player of opencore, which is not the focus of this article. This article focuses on the mediaplayer system architecture. Other Android applications also use a similar architecture.

Overview of mediaplayer
Android mediaplayer includes the playing functions of audio and video. On the android interface, both the music and video applications call mediaplayer.
MediaplayerAt the underlying layerOpencore (packetvideo) LibraryTo build a mediaplayer program, the upper layer also contains inter-process communication and other content. The basis for inter-process communication is the Binder Mechanism in the basic Android library.
Taking open-source android as an example,Mediaplayer codeIt is mainly in the following directory:
Java program path:
Packages/apps/music/src/COM/Android/music/
Java class path:
Frameworks/base/Media/Java/Android/Media/mediaplayer. Java
Java local call (JNI ):
Frameworks/base/Media/JNI/android_media_mediaplayer.cpp
This part of content is compiledLibmedia_jni.so.
The main header file is in the following directory:
Frameworks/base/include/Media/
The multimedia underlying library is in the following directory:
Frameworks/base/Media/libmedia/
This part of content is compiled into a libraryLibmedia. So.
Multimedia Service Section:
Frameworks/base/Media/libmediaplayerservice/
The files are mediaplayerservice. h and mediaplayerservice. cpp.
This part of content is compiled into a libraryLibmediaplayerservice. So.
Opencore-based multimedia player
External/opencore/
This part of content is compiled into a libraryLibopencoreplayer. So.

From the perspective of program scale, libopencoreplayer. So is the main implementation part, while other libraries are basically encapsulated on it and the mechanism for establishing inter-process communication.

Part 2 mediaplayer interfaces and architecture
2.1 Overall framework
The structure of each mediaplayer library is complex and can be expressed

In each library, libmedia. so is at the core. It provides the mediaplayer class for interfaces on the upper layer. libmedia_jni.so provides Java interfaces by calling the mediaplayer class, and implements android. media. mediaplayer class.
Libmediaplayerservice. so is the media server, which inherits libmedia. so class implements server functions, while libmedia. in so, another part of the content is through inter-process communication and libmediaplayerservice. so for communication. Libmediaplayerservice. So is implemented by calling opencore player.
The header file of mediaplayer is in the frameworks/base/include/Media/directory, which corresponds to the directory of the libmedia. So Library source file frameworks/base/Media/libmedia. The main header files include the following:
Imediaplayerclient. h
Mediaplayer. h
Imediaplayer. h
Imediaplayerservice. h
Mediaplayerinterface. h

In these header files, mediaplayer. H provides interfaces for the upper layer, while several other header files provide some interface classes (that is, classes containing pure virtual functions ), these interface classes must be inherited by implementation classes before they can be used.
Shows the relationship between the entire mediaplayer Library and the call:

When the entire mediaplayer is running, it can be roughly divided into two parts: client and server. They run in two processes respectively, and the binder mechanism is used between them to implement IPC communication. From the framework structure, imediaplayerservice. h. imediaplayerclient. H and mediaplayer. the three Class H defines the interface and architecture of meidaplayer, mediaplayerservice. CPP and mediaplayer. the CoO files are used to implement the meidaplayer architecture. The specific features of meidaplayer are in pvplayer (library libopencoreplayer. so.

2.2 header file imediaplayerclient. h
Imediaplayerclient. h is used to describe the interface of a mediaplayer client. The description is as follows:

Class imediaplayerclient: Public iinterface <br/>{< br/> Public: <br/> declare_meta_interface (mediaplayerclient); <br/> virtual void y (int msg, int ext1, int ext2) = 0; <br/>}; </P> <p> class bnmediaplayerclient: Public bninterface <imediaplayerclient> <br/>{< br/> public: <br/> virtual status_t ontransact (uint32_t code, <br/> const parcel & Data, <br/> parcel * reply, <br/> uint32_t flags = 0 ); <br/> };In the definition, the imediaplayerclient class inherits the iinterface and defines an interface of the mediaplayer client. bnmediaplayerclient inherits the bninterface <imediaplayerclient>, this is built for the Android-based basic class Binder Mechanism to implement process communication. In fact, according to the definition of the bninterface class template, the bninterface <imediaplayerclient> class is equivalent to inheriting the bninterface and imediaplayerclient. This is a common definition method for Android.

2.3 header file mediaplayer. h
Mediaplayer. H is an external interface class. It mainly defines a mediaplayer class:
Class mediaplayer: Public bnmediaplayerclient <br/>{< br/> Public: <br/> mediaplayer (); <br/> ~ Mediaplayer (); <br/> void onfirstref (); <br/> void disconnect (); <br/> status_t setdatasource (const char * URL ); <br/> status_t setdatasource (int fd, int64_t offset, int64_t length); <br/> status_t setvideosurface (const sp <surface> & surface ); <br/> status_t setlistener (const sp <mediaplayerlistener> & listener); <br/> status_t prepare (); <br/> status_t prepareasync (); <br/> status_t start (); <br/> sta Tus_t stop (); <br/> status_t pause (); <br/> bool isplaying (); <br/> status_t getvideowidth (int * w ); <br/> status_t getvideoheight (int * H); <br/> status_t seekto (INT msec); <br/> status_t getcurrentposition (int * msec ); <br/> status_t getduration (int * msec); <br/> status_t reset (); <br/> status_t setaudiostreamtype (INT type ); <br/> status_t setlooping (INT loop); <br/> status_t setvolume (float leftvo Lume, float rightvolume); <br/> void notify (int msg, int ext1, int ext2); <br/> static sp <imemory> decode (const char * URL, optional * psamplerate, int * pnumchannels); <br/> static sp <imemory> decode (int fd, int64_t offset, int64_t length, uint32_t * psamplerate, int * pnumchannels ); <br/> //...... <Br/>}From the interface, we can see that the mediaplayer class just implements basic mediaplayer operations, such as start, stop, and pause.
The deathnotifier class is defined in the mediaplayer class and inherits the deathrecipient class in the ibinder class:
Class deathnotifier: Public ibinder: deathrecipient <br/>{< br/> Public: <br/> deathnotifier () {}< br/> virtual ~ Deathnotifier (); <br/> virtual void binderdied (const WP <ibinder> & WHO); <br/> };In fact, the mediaplayer class indirectly inherits ibinder, while the mediaplayer: deathnotifier class inherits ibinder: deathrecipient, which is built to implement inter-process communication.

2.4 header file imediaplayer. h
The main content of imediaplayer. H is an interface for implementing the mediaplayer function. Its main definition is as follows:
Class imediaplayer: Public iinterface <br/>{< br/> Public: <br/> declare_meta_interface (mediaplayer); <br/> virtual void disconnect () = 0; <br/> virtual status_t setvideosurface (const sp <isurface> & surface) = 0; <br/> virtual status_t prepareasync () = 0; <br/> virtual status_t start () = 0; <br/> virtual status_t stop () = 0; <br/> virtual status_t pause () = 0; <br/> virtual status_t isplaying (bool * State) = 0; <br/> virtual status_t getvideosize (int * w, int * H) = 0; <br/> virtual status_t seekto (INT msec) = 0; <br/> virtual status_t getcurrentposition (int * msec) = 0; <br/> virtual status_t getduration (int * msec) = 0; <br/> virtual status_t reset () = 0; <br/> virtual status_t setaudiostreamtype (INT type) = 0; <br/> virtual status_t setlooping (INT loop) = 0; <br/> virtual status_t setvolume (float leftvolume, float rightvolume) = 0; <br/>}; <br/> class bnmediaplayer: public bninterface <imediaplayer> <br/> {<br/> Public: <br/> virtual status_t ontransact (uint32_t code, <br/> const parcel & data, <br/> parcel * reply, <br/> uint32_t flags = 0); <br/> };The imediaplayer class mainly defines the functional interfaces of the mediaplayer. This class must be inherited before it can be used. It is worth noting that these interfaces are similar to those of the mediaplayer class, but they are not directly related. In fact, various implementations of the mediaplayer class are generally completed by calling the implementation class of the imediaplayer class.

2.5 header file imediaplayerservice. h
Imediaplayerservice. h is used to describe a mediaplayer service. The definition is as follows:
Class imediaplayerservice: Public iinterface <br/>{< br/> Public: <br/> declare_meta_interface (mediaplayerservice); <br/> virtual sp <imediaplayer> Create (pid_t PID, const sp <imediaplayerclient> & client, const char * URL) = 0; <br/> virtual sp <imediaplayer> Create (pid_t PID, const sp <imediaplayerclient> & client, int FD, int64_t offset, int64_t length) = 0; <br/> virtual sp <imemory> decode (const char * URL, uint32_t * psamplerate, int * pnumchannels) = 0; <br/> virtual sp <imemory> decode (int fd, int64_t offset, int64_t length, uint32_t * psamplerate, int * pnumchannels) = 0; <br/> }; <br/> class bnmediaplayerservice: Public bninterface <imediaplayerservice> <br/>{< br/> Public: <br/> virtual status_t ontransact (uint32_t code, <br/> const parcel & Data, <br/> parcel * reply, <br/> uint32_t flags = 0); <br/>}; <br/>Because of pure virtual functions, imediaplayerservice and bnmediaplayerservice must be inherited and implemented before they can be used. interfaces such as create and decode defined by imediaplayerservice must actually be implemented by the successor. Note that the return value type of create is sp <imediaplayer>. This imediaplayer provides the interface for implementing functions.

Part 3 Main Implementation Analysis of mediaplayer

3.1 Java program

The mediaplaybackservice. Java file in the packages/apps/music/src/COM/Android/music/directory contains the call to the mediaplayer.
The mediaplaybackservice. Java file contains a reference to the package:
Import Android. Media. mediaplayer;
In the mediaplaybackservice class, the multiplayer class is defined:
Private class multiplayer {<br/> private mediaplayer mmediaplayer = new mediaplayer (); <br/>}The mediaplayer class is used in the multiplayer class. Some of them call this mediaplayer. The call process is as follows:
Mmediaplayer. Reset (); <br/> mmediaplayer. setdatasource (PATH); <br/> mmediaplayer. setaudiostreamtype (audiomanager. stream_music );Interfaces such as reset, setdatasource, and setaudiostreamtype are implemented through Java local call (JNI.

3.2 Java local call of mediaplayer
The Java local call of mediaplayer is implemented in the file frameworks/base/Media/JNI/android_media_mediaplayer.cpp.
Android_media_mediaplayer.cpp defines a jninativemethod (Java local call method) type array gmethods, as shown below:
Static jninativemethod gmethods [] ={< br/> {"setdatasource", "(ljava/lang/string;) V", (void *) android_media_mediaplayer_setdatasource }, <br/> {"setdatasource", "(ljava/IO/filedescriptor; JJ) V", (void *) android_media_mediaplayer_setdatasourcefd}, <br/> {"prepare ", "() V", (void *) android_media_mediaplayer_prepare}, <br/> {"prepareasync", "() V", (void *) android_media_mediaplayer_prepareasync }, <br/> {"_ start", "() V", (void *) android_media_mediaplayer_start}, <br/> {"_ Stop", "() V ", (void *) android_media_mediaplayer_stop}, <br/> {"getvideowidth", "() I", (void *) handle}, <br/> {"getvideoheight ", "() I", (void *) android_media_mediaplayer_getvideoheight}, <br/> {"seekto", "(I) V", (void *) android_media_mediaplayer_seekto }, <br/> {"_ pause", "() V", (void *) android_media_mediaplayer_pause}, <br/> {"isplaying", "() Z ", (void *) android_media_mediaplayer_isplaying}, <br/> {"getcurrentposition", "() I", (void *) android_media_mediaplayer_getcurrentposition}, <br/> {"getduration ", "() I", (void *) android_media_mediaplayer_getduration}, <br/> {"_ release", "() V", (void *) android_media_mediaplayer_release }, <br/> {"_ RESET", "() V", (void *) android_media_mediaplayer_reset}, <br/> {"setaudiostreamtype", "(I) V ", (void *) Authorization },< br/> {"setlooping", "(z) V", (void *) android_media_mediaplayer_setlooping}, <br/> {"setvolume ", "(FF) V", (void *) android_media_mediaplayer_setvolume}, <br/> {"getframeat", "(I) landroid/graphics/bitmap;", (void *) android_media_mediaplayer_getframeat}, <br/> {"native_setup", "(ljava/lang/object;) V", (void *) identifier}, <br/> {"native_finalize ", "() V", (void *) android_media_mediaplayer_native_finalize}, <br/>}The first member of jninativemethod is a string that represents the name of the Java local call method. This name is called in the Java program. The second member is also a string, indicates the parameters and return values of the local call method in Java. The Third Member is the C language function corresponding to the local call method in Java.
The android_media_mediaplayer_reset function is implemented as follows:
Static void android_media_mediaplayer_reset (jnienv * ENV, jobject thiz) <br/>{< br/> sp <mediaplayer> MP = getmediaplayer (ENV, thiz ); <br/> If (MP = NULL) {<br/> jnithrowexception (ENV, "Java/lang/illegalstateexception", null); <br/> return; <br/>}< br/> process_media_player_call (ENV, thiz, MP-> Reset (), null, null); <br/>}In the call to android_media_mediaplayer_reset, A mediaplayer pointer is obtained, which is called to implement actual functions.
Register_android_media_mediaplayer is used to register gmethods as a class "android/Media/mediaplayer". Its implementation is as follows.
Static int register_android_media_mediaplayer (jnienv * env) <br/>{< br/> jclass clazz; <br/> clazz = env-> findclass ("android/Media/mediaplayer "); <br/> //...... <br/> return androidruntime: registernativemethods (ENV, "android/Media/mediaplayer", gmethods, nelem (gmethods); <br/>}"Android/Media/mediaplayer" corresponds to Android. Media. mediaplayer in Java.
 
3.3 mediaplayer core library libmedia. So
The libs/Media/mediaplayer. cpp file is used to implement the interfaces provided by mediaplayer. H. An important part is as follows:
Const sp <imediaplayerservice> & mediaplayer: getmediaplayerservice () <br/>{< br/> mutex: autolock _ L (mservicelock); <br/> If (mmediaplayerservice. get () = 0) {<br/> sp <iservicemanager> Sm = defaultservicemanager (); <br/> sp <ibinder> binder; <br/> do {<br/> binder = Sm-> getservice (string16 ("media. player "); <br/> If (binder! = 0) <br/> break; <br/> logw ("mediaplayerservice not published, waiting... "); <br/> usleep (500000); // 0.5 s <br/>}while (true); <br/> If (mdeathnotifier = NULL) {<br/> mdeathnotifier = new deathnotifier (); <br/>}< br/> binder-> linktodeath (mdeathnotifier ); <br/> mmediaplayerservice = interface_cast <imediaplayerservice> (binder); <br/>}< br/> loge_if (mmediaplayerservice = 0, "No mediaplayerservic E !? "); <Br/> return mmediaplayerservice; <br/>}The most important one is binder = Sm-> getservice (string16 ("media. player "). This call is used to obtain a file named" media. player service, the type of the returned value of this call is ibinder, which is converted to imediaplayerservice according to the implementation.
A specific function setdatasource is as follows:
Status_t mediaplayer: setdatasource (const char * URL) <br/>{< br/> logv ("setdatasource (% s)", URL ); <br/> status_t err = unknown_error; <br/> If (URL! = NULL) {<br/> const sp <imediaplayerservice> & Service (getmediaplayerservice (); <br/> If (service! = 0) {<br/> sp <imediaplayer> player (Service-> Create (getpid (), this, URL); <br/> err = setdatasource (player ); <br/>}< br/> return err; <br/>}In the setdatasource function, call getmediaplayerservice to obtain an imediaplayerservice, and obtain an imediaplayer pointer from imediaplayerservice. the pointer is used to perform specific operations.
The implementation of other functions is similar to that of setdatasource.
Some other files in libmedia. So have the same names as header files. They are:
Libs/Media/imediaplayerclient. cpp
Libs/Media/imediaplayer. cpp
Libs/Media/imediaplayerservice. cpp
To implement the specific functions of the binder, you also need to implement a bpxxx class in these classes. For example, the implementation of imediaplayerclient. cpp is as follows: l
Class bpmediaplayerclient: Public bpinterface <imediaplayerclient> <br/>{< br/> Public: <br/> bpmediaplayerclient (const sp <ibinder> & impl) <br/>: bpinterface <imediaplayerclient> (impl) {}< br/> virtual void y (int msg, int ext1, int ext2) <br/>{< br/> parcel data, reply; <br/> data. writeinterfacetoken (imediaplayerclient: getinterfacedescriptor (); <br/> data. writeint32 (MSG); <br/> data. writeint32 (ext1); <br/> data. writeint32 (ext2); <br/> remote ()-> transact (Y, Data, & reply, ibinder: flag_oneway); <br/>}< br/> };You also need to implement the definition macro implement_meta_interface, which will be expanded to generate several functions:
Implement_meta_interface (mediaplayerclient, "android. Hardware. imediaplayerclient ");
The above implementation is based on the binder framework. You only need to implement it according to the template. The bpxxx class is the proxy class, And the bnxxx class is the native class ). The agent functions communicate with the ontransact functions of the local class.

3.4 Media Service libmediaservice. So
Mediaplayerservice. h and mediaplayerservice. cpp in the frameworks/base/media \ libmediaplayerservice directory are used to implement
Servers/Media/services. mediaplayerservice inherits the implementation of bnmediaplayerservice. In this class, the class client is defined. mediaplayerservice: client inherits bnmediaplayer.
Class mediaplayerservice: Public bnmediaplayerservice <br/>{< br/> class client: Public bnmediaplayer <br/>}Mediaplayerservice has the following static function instantiate:
Void mediaplayerservice: instantiate () {<br/> defaultservicemanager ()-> addservice (<br/> string16 ("media. player "), new mediaplayerservice (); <br/>}In the instantiate function, call the addservice function of iservicemanager and add a service named "media. Player" to it.
The service named "media. Player" is the same name as the service called getservice in mediaplayer. cpp. Therefore, you can call addservice to add services in mediaplayer. cpp by name "media. player. This is the role of using binder to implement inter-process communication (IPC). In fact, this mediaplayerservice class runs in the service, while mediaplayer. CPP calls the function to run in the application, and the two are not a process. However, mediaplayer. cpp calls the mediaplayerservice function just like a process call.
The createplayer function in mediaplayerservice. cpp is as follows:
Static sp <mediaplayerbase> createplayer (player_type playertype, void * cookie, notify_callback_f notifyfunc) <br/>{< br/> sp <mediaplayerbase> P; <br/> switch (playertype) {<br/> case pv_player: <br/> logv ("create pvplayer"); <br/> P = new pvplayer (); <br/> break; <br/> case sonivox_player: <br/> logv ("create midifile"); <br/> P = new midifile (); <br/> break; <br/> case vorbis_player: <br/> logv (" Create vorbisplayer "); <br/> P = new vorbisplayer (); <br/> break; <br/>}< br/> //...... <Br/> return P; <br/>}Here, different players are created based on the playertype: In most cases, the type is pv_player, and a new pvplayer () is called to create a pvplayer, then convert the pointer to mediaplayerbase for use. For Mini files of the sonivox_player type, a midifile will be created. For Ogg Vorbis format, a vorbisplayer will be created.
(Ogg vobis is an audio compression format, which is similar to MP3 music formats. It is completely free, open, and without patent restrictions .)
It is worth noting that the pvplayer, midifile, and vorbisplayer classes inherit mediaplayerinterface, and mediaplayerinterface inherits mediaplayerbase, so the three have the same interface type. Each constructor is called only when the constructor is created. After the constructor is created, mediaplayerbase controls them only through the mediaplayerbase interface.
In the frameworks/base/Media/libmediaplayerservice directory, midifile. h and midifile. cpp implement midifile, vorbisplayer. H, and vorbisplayer. cpp to implement a vorbisplayer.

3.5 Implementation of opencoreplayer libopencoreplayer. So
Opencore player is implemented in external/opencore/. This implementation is based on opencore player. The specific implementation file is playerdriver. cpp. Two classes are implemented: playerdriver and pvplayer. Pvplayer calls the playerdriver function to implement specific functions.