Android Multimedia Framework Summary (IX) Data processing of stagefright framework and its process to Omxcodec

Reprint please the head source link and the tail two-dimensional code together reprint, this article from countercurrent Fish: http://blog.csdn.net/hejjunlin/article/details/52532085

Unconsciously to the Nineth article, feel there are many many did not write, the road long its repair far, I will go up and down and quest, the last part of the main introduction of the Stagefright framework and Awesomeplayer Data parser, finally we said, involving the parse and decode parts, will be introduced in this article , look at today's agenda:

• Two graphs looking at data trends
• Prepare process in Awesomeplayer
• Awesomeplayer initialization of the audio and video decoder process
• The decode process of stagefright
• Stagefright Process Data Processing
• The process of data from source to final decoding

Two graphs looking at data trends

Everything comes from the mediasource, and everything goes from the mediasource.
Audio:

Video:

Prepare process in Awesomeplayer

First we begin to look at the execution of the Awesomeplayer prepare:

The above code summarizes: The prepare procedure calls the prepareasync_l function, executes the new awesomeevent in prepareasync_l, The result of Awesomeplayer call Onprepareasyncevent is returned with the Awesomeevent construct as the parameter.
Then analyze the process of awesomeevent: Start Mqueue as event handler

The above new awesomeevent executes the onprepareasyncevent function, so let's see what the function does.

The above code summarizes: AV (audio and video) will be processed, so the Awesomeplayer::initvideodecoder and Awesomeplayer::initaudiodecoder () function.
This article is from the countercurrent fish: http://blog.csdn.net/hejjunlin/article/details/52532085

Initializing the audio and video decoder process

Let's take a look at the Initvideodecoder, which is the initialization of the video decoder:

Then look at the initialization audio decoder and look at the declarations of several variables:

Then look at the following code:

Summarize the above code:
Stagefright Call Awesomeplayer's prepare, Awesomeplayer calls its own prepareasync to initialize the audio and video decoder, both methods will Omxcodec::create, Next look at the process.

The decode process of stagefright

The two mediasource obtained by the "encapsulation of data Flow" is actually two omxcodec. Awesomeplayer and Maudioplayer all get data from the MediaSource to play. Awesomeplayer gets the raw video data that ultimately needs to be rendered, and Maudioplayer reads the raw audio data that is ultimately needed to be played. In other words, the data read from the OMXCODEC is already raw data.
Omxcodec is how to convert the data source after the parse, decode two steps into the original data. Starting with the Omxcodec::create construction method, look at its code below:

The above code summarizes: corresponding parameter analysis:

• Iomx &OMX refers to an instance of a Omxnodeinstance object.
• MetaData &meta This parameter is obtained by Mediasource.getformat. The main member of this object is a keyedvector (uint32_t, Typed_data) Mitems, which contains some name-value pairs representing the MediaSource format information.
• BOOL Createencoder Indicates whether the OMXCODEC is encoded or decoded.
• MediaSource &source is a mediaextractor (data resolver).
• Char *matchcomponentname specifies a codec used to generate this omxcodec.
  First use Findmatchingcodecs to find the corresponding codec, find the current IOMX allocation node and register the event listener: Omx->allocatenode (COMPONENTNAME, Observer, &node). Finally, wrap the IOMX into a omxcodec:

This will get the OMXCODEC.

• Awesomeplayer get this omxcodec, and then look at Initvideodecoder/initaudiodecoder, here to see Initaudiodecoder method, is to put maudiosource = Momxsource, assigns the value, and then invokes Maudiosource->start () to initialize. Omxcodec initialization is done in two main things:
  
  • Send the Start command to OpenMAX. Momx->sendcommand (Mnode, Omx_commandstateset, Omx_stateidle)
  • Call Allocatebuffers () to allocate two buffers, stored in vector mportbuffers[2], for input and output, respectively.
• It is then called in the current Initxxxdecoder method (Maudiosource->start ()/mvideosource->start ())

After the subclass VideoSource and Audiosource that triggered the MediaSource call the Start () method, the inside of it begins to fetch data from the data source and parse it until the buffer is full and stops. In Awesomeplayer, you can call MediaSource's Read method for reading the decoded data.

• For Mvideosource, the data read: Mvideosource->read (&mvideobuffer, &options) is presented to the display module for rendering,mvideorenderer-> Render (Mvideobuffer);

• For Maudiosource, the Maudiosource is encapsulated with Maudioplayer, and then the Maudioplayer is responsible for reading the data and playback control.

• Awesomeplayer call Omxcode to read ES data and decode the processing

• Omxcodec call MediaSource's read function to get the audio and video data
• Omxcodec call Android Iomx interface, is actually stagefrightdecode in the OMX implementation

This article is from the countercurrent fish: http://blog.csdn.net/hejjunlin/article/details/52532085

This process is the prepare process, with the emphasis on decoding the exile into buffer.
Next, when the Java layer calls the Start method, through Mediaplayerservice, and then to Stagefrightplayer, referring to Awesomeplayer, so that the Awesomeplayer play method, see the code:

• When Awesomeplayer calls play, the data is read through Mvideosource->read (&mvideobuffer, &options). Mvideosource->read (&mvideobuffer, &options) is specifically called Omxcodec.read to read the data. The omxcodec.read is mainly divided into two steps to achieve the data read:
• (1) by calling Draininputbuffers () to populate Mportbuffers[kportindexinput], this step completes the parse. The DEMUX data is read from the data source to the input buffer by OpenMAX as input to the OpenMAX.
• (2) Fill mportbuffers[kportindexoutput by Filloutputbuffers (), this step completes the decode. The data in the input buffer is decoded by the OpenMAX and then the video data that can be displayed after decoding is output to the output buffer.
  Awesomeplayer uses Mvideorenderer->render (Mvideobuffer) to render data that has been processed by parse and decode. A mvideorenderer is actually a package of Iomxrenderer Awesomeremoterenderer:

Stagefright Process Data Processing

• Audioplayer is a member of Awesomeplayer, Audioplayer drives the acquisition of data through callback, and Awesomeplayer is driven by videoevent. There is a common denominator, that is, data acquisition is abstracted into Msource->read () to complete, and read inside the parse and decode tied together. Stagefright AV Synchronization section, audio is completely callback drive data flow, note that the video section in the onvideoevent will get audio timestamp, is the traditional AV timestamp to do synchronization.

• Awesomeplayer's video mainly has the following members:

  • Mvideosource (decoding video)
  • Mvideotrack (reading video data from a multimedia file)
  • Mvideorenderer (format conversion for decoded video, Android using RGB565)
  • Misurface (re-drawing layer)
  • Mqueue (Event queue)

• The audio process for the Stagefright runtime is as follows:

  • First set the path of the Muri
  • Start Mqueue, create a thread to run Threadentry (named Timedeventqueue, which is the event scheduler)
  • Opens the header of the file specified by Muri, and a different separator (such as Mpeg4extractor) is selected based on the type
  • Use Mpeg4extractor to separate the audio and video tracks of the MP4 and return the Mpeg4source type of video track to Mvideotrack
  • Selecting the decoder based on the encoding type in Mvideotrack, the AVC encoding type selects Avcdecoder and returns to Mvideosource and sets Msource in Mvideosource to Mvideotrack
  • Insert Onvideoevent into queue and start decoding playback
  • Read the parsed video buffer by Mvideosource object

If the parsed buffer has not reached the time of the AV timestamp synchronization, postpone to the next round of operation

1, Mvideorenderer is empty, then initialize (if you do not use OMX will set Mvideorenderer to Awesomelocalrenderer)
2, through the Mvideorenderer object will parse the good video buffer into the RGB565 format, and sends the display module to carry on the image drawing
3. Re-insert the Onvideoevent into the event scheduler to cycle

This article is from the countercurrent fish: http://blog.csdn.net/hejjunlin/article/details/52532085

Stagefright data from source to final decoded process

Can be compared with the "Android Multimedia Framework Summary (eight) Stagefright framework of the Awesomeplayer and data parser," the figure, not posted here,

• to set DataSource, the data source can have two URIs and FD. URIs can be http://,rtsp://and so on. FD is a local file descriptor, can be over FD, can find the corresponding file. The
• is generated by DataSource Mediaextractor. Implemented by SP extractor = Mediaextractor::create (DataSource). Mediaextractor::create (DataSource) creates different data-reading objects based on different data content.
• generates audio data streams (maudiotrack) and video data Streams (mvideotrack) by calling Setvideosource by Mediaextractor decomposition.
• onprepareasyncevent () If DataSource is a URL, fetch the data based on the address and start buffering until Mvideotrack and Maudiotrack are obtained. Mvideotrack and Maudiotrack generate Mvideosource and Maudiosource by calling Initvideodecoder () and Initaudiodecoder () to both audio and video decoders. Then invoke the postbufferingevent_l () commit event to turn on buffering. The execution function for the
• data buffer is onbufferingupdate (). When the buffer has enough data to play, the call play_l () starts playing. The key in play_l () is to call the postvideoevent_l () and commit the mvideoevent. This event executes when the function onvideoevent () is called. This function decodes the video by calling Mvideosource->read (&mvideobuffer, &options). Audio decoding is implemented via Maudioplayer. The
• Video decoder decodes the data from a frame after decoding by Mvideosource->read, puts it into Mvideobuffer, and finally passes Mvideorenderer->render (mvideobuffer) The video data is sent to the display module. When a pause or stop is required, call Cancelplayerevents to commit the event to stop decoding, and optionally continue buffering the data.

The first time to get blog update reminders, as well as more Android dry, source code Analysis , Welcome to follow my public number, sweep the bottom QR code or long press to identify two-dimensional code, you can pay attention to.

If you feel good, easy to praise, but also to the author's affirmation, can also share this public number to you more people, original not easy

Android Multimedia Framework Summary (IX) Data processing of stagefright framework and its process to Omxcodec