Android Video Playback----from the Framework to Driver

I'll analyze the process of the local video playback in Android 4.1 on PandaBoard. Hardware acceleration for HD video playback also would be discussed.

1. Video File

A video file usually contains the Part:file Header and Coded video Frames. So there is steps for video Playback:parse file header and Decode video frames. The media player must implement the functionalities. A media player consist of a media player engine and media codec plugin. The former is used to parse the file header, and the latter are used to decode video frames.

A general video file

2. Architecture Overview

The diagram shows the layers from media Player application to DSP Driver/firmware. Please keep the your mind.

3. Multimedia Framework Evolution

The media player engine in Android is evolving. There is many articles on Android media framework which is based on the different Android version. If you don ' t know about the media framework evolution, your may is confused by the articles. Both of the engine and the directory structure are changed.

• Engine

Opencore is the player engine in Android 1.6. Starefright and Nuplayer replaced Opencore gradually. Stagefight is used to play the local media file. Nuplayer can play the stream online.

• Directory

In 4.1 jb:the native codes of Media Framework,
Include Libeffects, Libmedia, Libmediaplayerservice,
Libstagefright, MediaServer,
is moved from
Aosp/frameworks/base/media
To
Aosp/frameworks/av/media

4. Jellybean MM Architecture

I got the diagram from the Internet. It shows the structure of the main classes/files in Jellybean ' s media framework.

5. Local Video Playback Process

The diagram shows a simple and clear local video playback process in JAVA API layer. In next sections, I'll introduce the details under the APIs.

6. Select Media Player Engine

Mediaplayer::setdatasource () is a key function which are used to

1. Get the proper media player engine for the specified media file type
2. Create Mediaextractor to parse the metadata of the media file

I got the sequence diagram from the Internet.

We can follow the diagram to trace the code. Let's see how the correct player is selected.

The rightest slot in the diagram is Mediaplayerservice. It ' s an important service in Android. Mediaplayerservice is added to Service Manger by Media Server which are started in Android boot process.

7. Load Media Codec Plugin

The correct media Player engine is created and it can not decode the video frames. We need to load the Media Codec Plugin which would be a connected with Media Player Engine to decode the video frames.

There is plugins:hardware and software. AOSP implements a soft OMX plugin, while the chipset vendor usually implements a hardware OMX plugin. Let's see how the Hardware OMX plugin is created and added. The code for the both steps depends on the hardware platform. I give the analysis on PandaBoard.

1. Create OMX Plugin.

The vendor ' s hardware OMX plugin is loaded dynamically from. so file.

In OMX plugin initialization function, a table of the supported media formats is created.

2. Add OMX Plugin

Now we can add the plugin into the plugins list of media Player engine. Omxmaster keeps the plugins list. The media player engine can get the pointer to Omxmaster. So Media Player Engine was connected with Media Codec.

8. Match and Init a Codec

9. TI OpenMAX IL Architecture

Load DSP Bridge Driver

One. Boot kernel with DSP

(To be continued)