Android Audio Video In depth 18 ffmpeg play video, have sound (with source download)

Project Address
Https://github.com/979451341/AudioVideoStudyCodeTwo/tree/master/FFmpegv%E6%92%AD%E6%94%BE%E8%A7%86%E9%A2%91%E6 %9c%89%e5%a3%b0%e9%9f%b3%ef%bc%8c%e6%9a%82%e5%81%9c%ef%bc%8c%e9%87%8a%e6%94%be%e3%80%81%e5%bf%ab%e8%bf%9b%e3% 80%81%e9%80%80%e5%90%8e

This project is written by the Great God of book 2012LC, play No problem is that other functions are a bit too much ...
Alas, I have not been able to write a good player,

Back to the Chase

First of all, this code is the producer and consumer model, the generator is constantly decoding MP4 will be a frame of data to the consumer, the consumer is audio playback class and video playback class, it is said that the generator, a consumer two, all through the pthread open thread, through mutual exclusion lock and conditional information to maintain the relationship chain

1. Producer-output one frame of data

The start is to initialize the various components and test whether the video file can be opened and get video-related information for later code preparation work

void Init () {
LOGE ("Turn on Decode thread")
1. Registering components
Av_register_all ();
Avformat_network_init ();
Encapsulating the format context
Pformatctx = Avformat_alloc_context ();

//2.打开输入视频文件if (avformat_open_input(&pFormatCtx, inputPath, NULL, NULL) != 0) {    LOGE("%s", "打开输入视频文件失败");}//3.获取视频信息if (avformat_find_stream_info(pFormatCtx, NULL) < 0) {    LOGE("%s", "获取视频信息失败");}//得到播放总时间if (pFormatCtx->duration != AV_NOPTS_VALUE) {    duration = pFormatCtx->duration;//微秒}

}

Initialize the audio class and video class, and give the Surfaceview to the video class

ffmpegVideo = new FFmpegVideo;ffmpegMusic = new FFmpegMusic;ffmpegVideo->setPlayCall(call_video_play);

Turn on the creator thread

Pthread_create (&p_tid, NULL, begin, NULL);//Opening BEGIN thread

Get the video stream and audio stream from the videos, copy the respective XXX contexts to two consumer classes, and the stream in which location and time unit to two consumer classes

Find video stream and audio stream for (int i = 0; i < pformatctx->nb_streams; ++i) {//get xxxavcodeccontext *avcodeccontext = Pformat    ctx->streams[i]->codec;    Avcodec *avcodec = Avcodec_find_decoder (avcodeccontext->codec_id);    Copy a xxx, avcodeccontext *codeccontext = Avcodec_alloc_context3 (AVCODEC);    Avcodec_copy_context (Codeccontext, Avcodeccontext);    if (Avcodec_open2 (Codeccontext, Avcodec, NULL) < 0) {LOGE ("Open failed") continue; }//If it is a video stream if (Pformatctx->streams[i]->codec->codec_type = = Avmedia_type_video) {ffmpegvideo->in        Dex = i;        Ffmpegvideo->setavcodeccontext (Codeccontext);        Ffmpegvideo->time_base = pformatctx->streams[i]->time_base;                                             if (window) {anativewindow_setbuffersgeometry (window, Ffmpegvideo->codec->width, Ffmpegvideo->codec->height, window_format_rgba_8888        ); }}//if it is audioStream else if (Pformatctx->streams[i]->codec->codec_type = = Avmedia_type_audio) {Ffmpegmusic->index = i        ;        Ffmpegmusic->setavcodeccontext (Codeccontext);    Ffmpegmusic->time_base = pformatctx->streams[i]->time_base; }}

Open two consumer-class threads

ffmpegVideo->setFFmepegMusic(ffmpegMusic);ffmpegMusic->play();ffmpegVideo->play();

Then start a frame-by-frame decoding the data to two consumer classes to store the data vector, if the data in the vector there is no play, continue to play

while (isPlay) {    //    ret = av_read_frame(pFormatCtx, packet);    if (ret == 0) {        if (ffmpegVideo && ffmpegVideo->isPlay && packet->stream_index == ffmpegVideo->index           ) {            //将视频packet压入队列            ffmpegVideo->put(packet);        } else if (ffmpegMusic && ffmpegMusic->isPlay &&                   packet->stream_index == ffmpegMusic->index) {            ffmpegMusic->put(packet);        }        av_packet_unref(packet);    } else if (ret == AVERROR_EOF) {        // 读完了        //读取完毕 但是不一定播放完毕        while (isPlay) {            if (ffmpegVideo->queue.empty() && ffmpegMusic->queue.empty()) {                break;            }            // LOGE("等待播放完成");            av_usleep(10000);        }    }}

When it's done, stop two consumer-class threads and release resources

isPlay = 0;if (ffmpegMusic && ffmpegMusic->isPlay) {    ffmpegMusic->stop();}if (ffmpegVideo && ffmpegVideo->isPlay) {    ffmpegVideo->stop();}//释放av_free_packet(packet);avformat_free_context(pFormatCtx);pthread_exit(0);

2. Consumer-Audio class

Open Thread

  pthread_create(&playId, NULL, MusicPlay, this);//开启begin线程

It just comes down to configuring OpenSL es to play audio, and the source of this data is the piece of code that determines

(*bqPlayerBufferQueue)->RegisterCallback(bqPlayerBufferQueue, bqPlayerCallback, this);

Let's look at Bqplayercallback, the data is from the GETPCM function.

FFmpegMusic *musicplay = (FFmpegMusic *) context;int datasize = getPcm(musicplay);if(datasize>0){    //第一针所需要时间采样字节/采样率    double time = datasize/(44100*2*2);    //    musicplay->clock=time+musicplay->clock;    LOGE("当前一帧声音时间%f   播放时间%f",time,musicplay->clock);    (*bq)->Enqueue(bq,musicplay->out_buffer,datasize);    LOGE("播放 %d ",musicplay->queue.size());}

Then in this GETPCM function, the Get function is done to get a frame of data.

Agrs->get (Avpacket);

If there is data in the vector, it takes the data out of the vector, and if not, waits for the producer to pass the condition variable.

The packet popup queue
int Ffmpegmusic::get (Avpacket avpacket) {
LOGE ("Remove queue")
Pthread_mutex_lock (&mutex);
while (Isplay) {
LOGE ("Remove the xxxxxx")
if (!queue.empty () &&ispause) {
LOGE ("Ispause%d", ispause);
If there's data in the queue, you can take it.
if (Av_packet_ref (Avpacket,queue.front ())) {
Break
}
Take a hit, pop the queue, destroy packet
Avpacket Packet2 = Queue.front ();
Queue.erase (Queue.begin ());
Av_free (Packet2);
Break
} else{
LOGE ("Audio execution Wait")
LOGE ("Ispause%d", ispause);
Pthread_cond_wait (&cond,&mutex);

    }}pthread_mutex_unlock(&mutex);return 0;

}

Note that the obtained data is avpacket, we need to decode him to Avframe.

    if (avPacket->pts != AV_NOPTS_VALUE) {        agrs->clock = av_q2d(agrs->time_base) * avPacket->pts;    }    //            解码  mp3   编码格式frame----pcm   frame    LOGE("解码")    avcodec_decode_audio4(agrs->codec, avFrame, &gotframe, avPacket);    if (gotframe) {        swr_convert(agrs->swrContext, &agrs->out_buffer, 44100 * 2, (const uint8_t **) avFrame->data, avFrame->nb_samples);

Size of the buffer
Size = Av_samples_get_buffer_size (NULL, AGRS->OUT_CHANNER_NB, Avframe->nb_samples,
AV_SAMPLE_FMT_S16, 1);
Break
}

Back to OpenSL ES callback function, take the data and press the data into the player to let him play

    //第一针所需要时间采样字节/采样率    double time = datasize/(44100*2*2);    //    musicplay->clock=time+musicplay->clock;    LOGE("当前一帧声音时间%f   播放时间%f",time,musicplay->clock);    (*bq)->Enqueue(bq,musicplay->out_buffer,datasize);    LOGE("播放 %d ",musicplay->queue.size());

3. Consumer-Video class

This is a very similar process, I'm going to say

Open Thread

//Apply Avframeavframe *frame = Av_frame_alloc ();//Assign a avframe struct, avframe struct is typically used to store raw data, pointing to the decoded original frame Avframe *rgb_ frame = Av_frame_alloc ();//Assign a avframe struct, point to the frame converted to RGB avpacket *packet = (Avpacket *) Av_mallocz (sizeof (avpacket)); /output file//file *FP = fopen (OutputPath, "WB");//Buffer uint8_t *out_buffer= (uint8_t *) Av_mallocz (avpicture_get_size (av_pix_ Fmt_rgba, Ffmpegvideo->codec->width,ffmpegvideo->cod Ec->height));//associated with the buffer, set Rgb_frame buffer Avpicture_fill ((avpicture *) Rgb_frame,out_buffer,av_pix_fmt_rgba, Ffmpegvideo->codec->width,ffmpegvideo->codec->height); LOGE ("Convert to RGBA format") Ffmpegvideo->swscontext = Sws_getcontext (ffmpegvideo->codec->width,ffmpegvideo-> CODEC->HEIGHT,FFMPEGVIDEO->CODEC->PIX_FMT, Ffmpegvideo->codec->width , Ffmpegvideo->codec->height,av_pix_fmt_rgba, sws_bicubic,null,null,null); 

Get a frame of data

Ffmpegvideo->get (packet);

And then get the data from the vector.

Adjust the playback speed of video and audio

    diff = ffmpegVideo->clock - audio_clock;

To delay the acceleration beyond a reasonable range
Sync_threshold = (Delay > 0.01 0.01:delay);

    if (fabs(diff) < 10) {        if (diff <= -sync_threshold) {            delay = 0;        } else if (diff >=sync_threshold) {            delay = 2 * delay;        }    }    start_time += delay;    actual_delay=start_time-av_gettime()/1000000.0;    if (actual_delay < 0.01) {        actual_delay = 0.01;    }    av_usleep(actual_delay*1000000.0+6000);

Play Video

Video_call (Rgb_frame);

Freeing resources and exiting threads

LOGE("free packet");av_free(packet);LOGE("free packet ok");LOGE("free packet");av_frame_free(&frame);av_frame_free(&rgb_frame);sws_freeContext(ffmpegVideo->swsContext);size_t size = ffmpegVideo->queue.size();for (int i = 0; i < size; ++i) {    AVPacket *pkt = ffmpegVideo->queue.front();    av_free(pkt);    ffmpegVideo->queue.erase(ffmpegVideo->queue.begin());}LOGE("VIDEO EXIT");pthread_exit(0);

End, later Ah, as far as possible to write a player, to the kind of suspension of the card

Android Audio Video In depth 18 ffmpeg play video, have sound (with source download)