Real-time monitoring, live streaming, streaming media, video website development Program Design Brief

First, Local:

 Using JAVACV (Android and Java Platform recommended JAVACV), FFmpeg, OpenCV or JMF can easily get to the local camera streaming media

JAVACV Series articles:

JAVACV Development 1: Call native webcam video

JAVACV Development in Detail 2: The implementation of the converter, push the local camera video to the streaming media server and camera recording video function (based on Javacv-ffmpeg , JAVACV-OPENCV)

JAVACV Development in detail 3: The receiver implementation, recording the streaming media server rtsp/rtmp video files (based on javacv-ffmpeg)

JAVACV Development in detail 4: The converter implementation (also as a local receiver, the flow converter, add images and text watermark, video image frame preservation), to achieve rtsp/rtmp/local file forwarding to rtmp streaming media server (based on javacv-ffmpeg)

JAVACV Development 5: Record audio (record microphone) to local file/streaming media server (based on Javax.sound, javacv-ffmpeg)

JAVACV Development 6: Local Audio (microphone device) and video (webcam) crawl, mix and push (record) to server (local)

JAVACV Development 7: Make audio conversion easier, to achieve universal Audio encoding format conversion, resampling and other audio parameters conversion function (Pcm16le encoded WAV MP3 as an example)

Supplementary article:

Audio and Video codec problem: How to quickly JAVACV and Decode codec (based on javacv-ffmpeg)

Audio and Video codec problem: 16/24/32 bit audio byte[] convert to small end order short[],int[], take byte[] to short[) as an example

To add a picture watermark or text watermark to the image (also support the video image frame watermark)
2, monitoring (third-party camera): through the device SDK or RTSP live stream to get the streaming media source
Second, the converter end

Live: Via FFmpeg (recommended), live555 will receive RTSP or byte stream and go to FLV format to publish to rtmp streaming media server (streaming media server must be built first) HLS principle Note: rtmp only supports video streaming in FLV format For examples of ffmpeg service implementations, please refer to: http://blog.csdn.net/eguid_1/article/details/51777716
http://blog.csdn.net/eguid_1/article/details/51787646

can also refer to the JAVACV Converter implementation: JAVACV Development of the 4: the converter implementation, implementation rtsp/rtmp/local file forwarding to the RTMP server Java Package ffmpeg command, support native ffmpeg all commands, Realize ffmpeg multi-process processing and multi-threaded output control (open, close, query), rtsp/rtmp push stream, pull stream
Third, streaming media server

The current mainstream streaming media server is: Fms,nginx-rtmp,red5 (Java), FLAZR Local video: Directly through the streaming media server decoding and push video streaming stream: by opening Udp/rtp/rtsp/rtmp/hls and so on streaming media services, Get the real-time video stream from ffmpeg/live555 and post it to the Rtmp/hls live stream and push it (can be saved with live streaming). Rtmp and HLS are two of the most popular streaming protocols in the web domain. The use of RTP or RTSP protocol is generally monitored. Streaming Media Protocol selection: RTMP is based on the TCP protocol and RTMP can maintain a delay of about 3 seconds. HLS is based on the HTTP protocol, so the real-time is particularly poor, want to use HLS to stay in real-time, don't think, HLS delay basically more than 10 seconds. The real-time requirement is very high, we recommend using some streaming media protocols based on UDP protocol. The difference between TCP and UDP two streaming media protocol is that TCP forces synchronization, and UDP is the data sent out. So the final solution is: Strong synchronization But the real-time requirements are not high with the TCP protocol-based, strong real-time weak synchronization of UDP. Add: Nginx-rtmp Streaming Media Server build Example: http://blog.csdn.net/eguid_1/article/details/51749830

nginx-rtmp 

Configuration Directives detailed meanings and usage: http://blog.csdn.net/eguid_1/article/details/51821297 

Fourth. Playback End (end of stream)

Live: Via Flex (Flash) player or third party player (Videojs,ckplayer,videolan etc... Call streaming media server to decode and play the streaming media source, if you do not need to be compatible with the low version of IE, you can use the HTML5 websocket player, Videojs is FLASH/HTML5 dual-core player.
Video: Via HTML comes with player, Flex (Flash) player or third-party player (Videojs,ckplayer,videolan ... ) Play Videojs/ckplayer player two times development support for RTMP Live, HLS live and normal video playback: http://blog.csdn.net/eguid_1/article/details/51898912
Generally use the VideoLAN player as a test tool for testing the audio and video stream release status supplement: 

1, if the use of Nginx server, it provides the RTMP module can be published rtmp live, recording and HLS, Nginx can integrate ffmpeg into the side stream media post-processing (watermark, etc.). 

2, Java can be called ffmpeg, through the way of JNI there are two ways: 

2.1, javaCV

1.2 support through Javacpp call FFMPEG,JAVACV currently integrates 8 kinds of streaming media processing framework, which is indispensable for Android and Java EE platform, powerful streaming media processing weapon 

2.2, JAVAAV (current latest 0.7,release latest 0.5) provides support for Java call FFmpeg, currently stopped updating
Add: Why there is no streaming framework based on native Java (or the language that comes with GC) because of the GC, which is the Java-proud automatic garbage collection mechanism (really analyticals, reminder). As you know, live streaming (as the name implies, real-time video forwarding), this real-time project will produce a large number of objects, which will result in two situations: 

1, the memory resources occupied after generating a large number of objects are not released in time, so the virtual machine memory overflow. 

2, a large number of objects resulting in GC full-load operation for resource recovery, will seriously occupy the system resources, resulting in sluggish system operation, impact on system performance and real-time and so on.