Streaming opencv videos over the network

Http://mjpg-streamer.svn.sourceforge.net/viewvc
Table of contents:

1. Introduction
2. Design of the System
3. Implementation of the server-side
   1. Frame grabber
   2. Stream server
4. Implementation of the client-side
   1. Stream Client
   2. Video Player
5. Compiling
6. Experiments & Gallery
7. Summary
8. Resources

1. Introduction

One of the most asked questions by opencv users is, "How can I stream videos over the network so I can process it on different computer? "Opencv does not provide such a function by nature (or so I thought), so we have to write custom code to accomplish this task.

If you have experiences with network programming before, this shocould be quite easy. just like when you send text files over the network, the same apply for this one. only this time we need to convert the converted ed data into opencv's iplimage format.

In this tutorial, I will explain to you the system I built to stream opencv videos over TCP/IP network. keep in mind that there are always ways to achieve this. not to mention the freely available video streaming Library such as FFMPEG
And VLC
. This is not about "this one is better", it is just about sharing the knowledge.

2. Design of the System

The system follows the client-server model. the computer that has the video input acts as the server. it waits for a client to connect and stream the videos once the connection has established. the divisor is shown below.

Fig 1. Streaming opencv videos over the network.

The diveabove shows several clients connect to the server and receive the streaming video simultaneously. However, to keep things simple, I made it that the server only accepts one client at a time.

If we look deeper into the server side, it shoshould consists from two parts. one who read the video input in a loop, and one who waits for the client and send the video frames. it is impossible to have both parts as a single block of code, since they have to runSimultaneously at the same time
. To overcome this, we have to write a multi-threaded
Program.

The same also apply for the client side.

But another problem occurs, windows and Unix-like systems have different way for handling with threads. while it is possible to write a code that compile and runs on both systems (using C Preprocessor), it doesn't necessarily. let's just use Unix and throw windows away.

In addition, I use Berkeley sockets
That is widely available on Unix-like systems for the networking code.

In summary, to make this as simplest as possible we keep these things in mind:

• The operating system is Unix-like. Therefore we're re using POSIX Threads
  And Berkeley sockets
  . (If you use Windows, install cygwin
  First ).
• Only one client connected at a time.
• The clientKnows
  The width and the height of the expected frame.
• The client has es grayscaled (Single Channel) frames.

3. Implementation of the server-side

The server side is the computer that has the video input to be streamed. and like I mentioned before, it consists of two parts. one who read the video input in a loop, and the other waits for the client to connect and send the video frames.

Fig 2. stream server digoal.

In the dimo-above, we see two threads running on the server side:Frame Grabber
AndStream Server
. The input is taken from a webcam, but you can use other resources too, like an AVI file.

Frame Grabber
Grabs a frame from the webcam and stores it to a global variableimg
.Stream Server
Waits for a client to connect. Once the connection has established, it sendsimg
To the client repeatedly whenever a newer versionimg
Is available.

The full listing of the server side is in stream_server.c
. Next we'll see the detail of both threads.

3. A. frame grabber

This is the main thread of the server-side. Its just like the usual code to display video from webcam
. Below is the code snippet from stream_server.c
.