Design of real-time monitoring, live streaming, streaming media and video website development

first of all, I admire eguid, whether it is technology or sharing, thanks to his series of Bowen on my first learning process help, so clearly indicated the reprint address: http://blog.csdn.net/eguid_1/article/details/51725970

one, the local push to send the end1, Local: The use of JAVACV (Android and Java Platform recommended JAVACV), FFmpeg, OpenCV or JMF can be very convenient access to the local camera streaming media

JAVACV Series articles:

JAVACV Development in detail 1: Call the native camera video

JAVACV Development of the 2: the implementation of the converter, push the local camera video to the streaming media server and the camera recording video function (based on javacv-ffmpeg, JAVACV-OPENCV)

JAVACV Development 3: The receiver implementation, recording streaming media server rtsp/rtmp video files (based on javacv-ffmpeg)

JAVACV Development of the 4: the implementation of the converter (also can be used as a local receiver, push converter, add pictures and text watermark, video image frame Save), implementation rtsp/rtmp/local file forward to rtmp streaming media server (based on javacv-ffmpeg)

JAVACV Development Detailed 5: Recording audio (recording microphone) to local file/streaming media server (based on Javax.sound, javacv-ffmpeg)

JAVACV Development Detailed 6: Local Audio (microphone devices) and video (camera) crawl, mix and push (record) to the server (local)

JAVACV Development of the 7: make the audio conversion more simple, the realization of the general audio coding format conversion, resampling and other audio parameters of the conversion function (Pcm16le encoded WAV mp3 as An example)

Supplementary article:

Audio and Video codec problem: How to JAVACV fast preprocessing and decoding (based on javacv-ffmpeg)

Audio and Video codec problem: 16/24/32 bit audio byte[] convert to small end short[],int[], take byte[to short[] as an example

Add image watermark or text watermark to picture (also support video image frame watermark)
2, monitoring (third party camera): through the device SDK or RTSP live stream to obtain streaming media source
second, the flow endLive: Via FFmpeg (recommended), live555 will receive RTSP or bytecode stream and turn into FLV format to rtmp streaming media server (streaming media server must be built first) HLS principle-Ditto Note: rtmp only supports streaming video in FLV format FFmpeg Service Realization Way Example Please refer to: http://blog.csdn.net/eguid_1/article/details/51777716 http://blog.csdn.net/eguid_1/article/ details/51787646
can also refer to JAVACV Converter implementation: JAVACV Development of the 4: the converter implementation, implementation rtsp/rtmp/local file forwarding to the RTMP server Java Encapsulation ffmpeg command, support native ffmpeg all commands, Realize ffmpeg process processing and multithreading output control (open, close, query), rtsp/rtmp push flow, pull flowthird, streaming media serverThe current mainstream streaming media server are: Fms,nginx-rtmp,red5 (Java), FLAZR Local video: Directly through the streaming media server decoding and push video streaming streaming: through the open Udp/rtp/rtsp/rtmp/hls and so on streaming media services, Get the streaming video from ffmpeg/live555 and post it to the Rtmp/hls live stream and push it (you can save it on the live side) rtmp and HLS are the mainstream streaming media protocols in the Web world. The use of RTP or RTSP protocols is generally monitored. Streaming Media protocol selection: RTMP based on TCP protocol, RTMP can maintain a delay of about 3 seconds. HLS is based on the HTTP protocol, so the real time is very poor, want to use HLS to keep real-time, do not think, HLS delay is basically more than 10 seconds. Real-time requirements are especially high, some streaming media protocols based on UDP protocol are recommended. The difference between TCP and UDP two streaming media protocols is that TCP enforces synchronization, and UDP is a matter of sending data out of the way. So the final solution is: strong synchronization But real-time requirements are not high use based on TCP protocol, strong real-time weak synchronization of UDP. Add: Nginx-rtmp Streaming Media Server build Example: http://blog.csdn.net/eguid_1/article/details/51749830nginx-rtmp configuration directive verbose meaning and usage: http://blog.csdn.net/eguid_1/article/details/51821297 Four, the playback end (end of the stream)Live: Via Flex (Flash) player or third party player (Videojs,ckplayer,videolan etc...) The streaming media source that invokes the streaming media server decodes and plays, and if you do not need to be compatible with the low version of IE, you can use the HTML5 websocket player, Videojs is the FLASH/HTML5 dual-core player.
Video: Via HTML with player, Flex (Flash) player or third party player (Videojs,ckplayer,videolan ...) To play Videojs/ckplayer player two development support rtmp live, HLS live and normal video playback: http://blog.csdn.net/eguid_1/article/details/51898912
Generally use the VideoLAN player as a test tool for testing audio and video streaming release status supplement: 1, if the use of Nginx server, it provides RTMP modules can be released rtmp live, video and HLS, Nginx can integrate ffmpeg into the side stream media post-processing (add watermark, etc.). 2, Java can invoke FFmpeg, through the way of JNI there are two ways: 2.1, javaCV1.2 support through Javacpp call FFMPEG,JAVACV currently integrates 8 kinds of streaming media processing framework, is an indispensable powerful streaming media processing tool for Android and Java EE Platform 2.2, JAVAAV (currently the latest 0.7,release 0.5) provides support for Java invoke FFmpeg, currently stopped updating
Add: Why there is no streaming media framework based on native Java (or the language of a GC), because of the GC, which is the Java-proud automatic garbage collection mechanism (really Chengye, Shenya). As you know, live broadcast (as the name suggests, real-time video forwarding), this real-time project will produce a large number of objects, which will lead to two kinds of conditions: 1, the production of a large number of objects after the memory resources are not released in time, so the virtual machine memory overflow. 2, the production of a large number of objects will result in GC full load operation of resource recovery, can seriously occupy the system resources, resulting in the system running sluggish, affecting the performance of the system and real-time and so on.