Stagefright OMX Summary

Since Stagefright and OpenMAX run on two different processes, the communication between them has to be processed by binder, this summary does not consider the audio this piece, assuming the video is MP4 encapsulated AVC encoded file.

1. Detects whether the queue is empty, waits for an event to insert
2. Gets the first event in the queue
3. The delay operation is calculated after the delay time required by the event
4. When the event is removed from the queue, the event
event Scheduler is dispatched through the process of continuous looping, which in turn changes the scheduling process depending on the particular situation. , there are currently several event events:
1. Onvideoevent
2. Onstreamdone
3. Onbufferingupdate
4. Oncheckaudiostatus
5. Onprepareasyncevent

The Stagefright player class is Awesomeplayer, which has several members (excluding the audio section):
1. Mvideosource (decoding video)
2. Mvideotrack (read video data from multimedia files)
3. Mvideorenderer (format conversion for decoded video, Android using RGB565)
4. Misurface (re-drawing layer)
5. Mqueue (Event queue)

The abstract flow of the Stagefright runtime is as follows:

The following is an MP4 file (AVC code) to analyze the abstract workflow of Awesomeplayer (excluding the audio section)
1) Set the absolute path of Muri to Xxxx.mp4
2) Start Mqueue, which creates a thread that runs Threadentry and is named Timedeventqueue, which is the event scheduler
3) Open the file specified by Muri, xxxx. The head of the MP4 file is (.... Ftypisom ...), and Mpeg4extractor is chosen as the separator
4) Use Mpeg4extractor to separate the audio and video tracks of the MP4 and return the Mpeg4source type of video track to Mvideotrack
5) Select the decoder according to the encoding type in the Mvideotrack, the AVC encoding type will select Avcdecoder (assuming that no OMX is used) and return to Mvideosource, and set the Msource in Mvideosource to Mvideotrack
6) Insert onvideoevent into queue and start decoding playback
7) Read the parsed video buffer by Mvideosource object
8) If the parsed buffer has not reached the time of the AV timestamp synchronization, postpone to the next round of Operation
9) Mvideorenderer is empty, it is initialized and Mvideorenderer is set to Awesomelocalrenderer if no OMX is used
10) Convert parsed video buffer into RGB565 format by Mvideorenderer object and send to display module for image drawing
11) Re-insert the Onvideoevent into the event scheduler to cycle

OMX IL, as the aggregation layer of the underlying decoding components, provides a unified interface for the upper-layer multimedia framework, and in Android2.2 Stagefright, Stagefright uses the OMX IL implementation in Opencore, using the OMX The IL framework requires the mvideosource to be set to the Omxcodec class, and the external interface of the OMX IL layer has the following main types:
1) Stagefright initializes the OMX framework with Omx_masterinit and loads the component
2) Stagefright uses Omx_mastergethandle to match component in OMX, and the match succeeds to return Omx_handletype for communication between Omxcodec and component
3) Omxcodec use Omx_sendcommand to set the component status, operate the component port
4) Omxcodec command execution results with EventHandler notification Omxcodec
5) Omxcodec use Omx_getparameter and Omx_setparameter to get and set the attribute parameters of the component
6) Omxcodec use Omx_usebuffer to set buffer for Omxcodec allocated by Compoment
7) Omxcodec uses Omx_emptythisbuffer to pass the uncompressed buffer to component for decoding
8) Omxcodec uses Omx_fillthisbuffer to pass empty bffer to component for storing decoded frames
9) compoment use Emptybufferdone notification OMXCODEC completed Inputport buffer read
compoment use Fillbufferdone notification Omxcodec completed Outputport buffer fill
The process of initialization is as follows:

OMX component data is primarily interactive via port, where port is divided into input and output, and port is encoded and decoded by sharing buffer with OMXCODEC, such as:

The processing of buffer is mainly driven by the following 4 commands:
? Omxcodec using Omx_emptythisbuffer to pass an uncompressed buffer to component,component receives the command reads data from input port buffer and assembles it into frames for decoding. The Emptybufferdone notification is called when the data in the buffer is read Omxcodec
? Compoment uses Emptybufferdone to notify Omxcodec that a read of Inputport buffer has been completed, Omxcodec reads the new video buffer to input via mvideotrack after receiving the command Port in buffer, and call Omx_emptythisbuffer notification component
? Omxcodec uses Omx_fillthisbuffer to pass an empty bffer to component for storing the decoded frame, and component copies the decoded frame data to the buffer after receiving the command. Then call Fillbufferdone notification Omxcodec
? Compoment uses Fillbufferdone to notify Omxcodec that a fill of outputport buffer has been completed, omxcodec the decoded frame to misurface for image drawing after the command is received, and uses omx_ after the drawing is finished! Fillthisbuffer notifies component that there is an empty buffer to fill
The abstract diagram is as follows:

The decoding in OMX il is divided into two parts, with the example of AVC decoding:
1) Assemble the buffer of input port into a frame using assemblepartialframes
2) The frame is passed to the AVCDECODER_OMX for decoding and outputting to buffer in output port
Such as:

Assuming that there are 2 buffer in input port buffer, buffer_1 and buffer_2 respectively, and that the data carried by these two buffer can constitute 1 frames, assemblepartialframes first applies a piece of memory area Tmp_ buffer_1, copy the buffer_1 valid data to tmp_buffer_1, and then apply for a memory area tmp_buffer_2, the first step after the application to copy the tmp_buffer_1 data to the first half of itself, the second step will Buffer_ 2 of valid data is copied to the second half to be combined into 1 frames.
After the combination is complete, the buffer of tmp_buffer_2 and output port is passed to AVCDECODER_OMX for decoding, and the decoded frame data is AVCDECODER_OMX copied into output port buffer.

Stagefright OMX Summary