Realization of image acquisition of USB camera based on Video4linux __linux

Linux itself with the use of ov511 chip camera, and the most widely used in the market is the use of SMIC produced by the zc301 chip camera, I will focus on the two series of cameras to introduce respectively.
(Note: All the development is done on the Hua Heng hharm-2410-edu, ov511 camera is using the mesh webeye3000,zc301 camera is the ANC S888).

One drive load

1.1 ov511 Drive

1. Static loading
(1) Make menuconfig in the kernel directory of ARM Linux.
(2) First (*) Select multimedia device-> for Linux. Loading the Video4linux module to provide the programming interface for the video capture device;
(3) then select support for USB and USB camera ov511 support in the USB support-> directory (*). This enables the support of USB digital cameras with OV511 interface chips to be added to the kernel.
(4) Save configuration exit.
(5) Make Dep;make zimage a kernel with a ov511 drive is generated at this time under/tftpboot.

2. Dynamic loading
(1) Make menuconfig in the kernel directory of ARM Linux.
(2) First <*> Select Multimedia device-> for Linux.
(3) Then in the USB support-> directory <*> select support for USB and <M> Select USB Camera ov511 support.
(4) Save exit.
(5) Make Dep;make zimage;make modules then generate OV511.O under/driver/usb, and the generated zimage is automatically placed under/tftpboot.
(6) Then start the board with the new kernel Insmod OV511.O can be loaded successfully.

Dynamic methods are much simpler to test than static methods. You do not need to download the entire kernel, just through NFS, load the driver to test. After the test is successful, it can be compiled into the kernel.


Problems with module loading:
1. A difference between insmod and modprobe the latter does not look up a module in the current directory, it only looks in the default directory under/lib/modules, because the program is just a system utility routine, not an interactive tool. You can add them to the default directory set by specifying their own directories in/etc/modules.conf.
2. If you insert the module OV511.O, the following message appears: Ov511.o:unresolved symbol video********, and other modules videodev.o not added.
3. Error occurred: Ov511.o:couldn ' t find the kernel version this modules is compiled for. This is an attempt to insert a target file that is not a loaded module. Because in the kernel configuration phase, the ov511 module is statically added to the kernel, although it looks exactly the same as the file name of the OV511.O module, but cannot be joined with the Insmod command.
4. If ov511.o:unresolved symbol video******** appears, then <M> Select Video for Linux, start the system with the newly generated kernel, and then Insmod Videodev.o,insmod OV511.O is fine.

1.2 zc301 Drive

The camera driver is from http://mxhaard.free.fr/under the embeded environment, there are special patch, I use is usb-2.4.31le06.patch.
(1) Put it under the/HHARM9-EDU/KERNEL/DRIVER/USB, decompression, patching. You will see the Spca5xx folder under this directory. There may be some errors, my error is in the makefile and config.in files, according to its prompts, make the appropriate changes. Patch will be written to the Makefile.rej and Config.in.rej files, the contents of these two files added to the Makefile and config.in on the line.
(2) Compile the kernel and enter the/hharm9-edu/kernel,make menuconfig. I use the ov511-driven approach described above to dynamically load. (M) Select the Spca5xx
(3) Make DEP, make zimage, make modules. will be generated in the/hharm9-edu/kernel/driver/usb/spca5xx.
SPCA5XX.O,SPCADECODER.O,SPCA_CORE.O. This is what we want to drive.
(4) Start with the new kernel, insmod these three. o Files (can not load SPCADECODER.O), the camera loaded successfully.

However, this Le driver has many problems, such as running to the format of the image (RGB565 or RGB24) error, said not to support this parameter. The reason is: (Excerpt from driver homepage http://mxhaard.free.fr/spca5le.html)
The Spca5xx-le design is very different to the SPCA5XX full package (the LE version is much worse than the driver and complete edition).
The memory in use are the most smaller as possible (the Le version drives to minimize memory usage)
The Spcadecoder is reduce and only raw JPEG webcam are used. (The drive module only supports output from the original JPEG format).
There is also a way to download the latest driver spca5xx-20060402.tar.gz from http://mxhaard.free.fr/download.html. This can be independently compiled, no need to put into the Linux kernel, compiled to generate a SPCA5XX.O, not three. o do drive. Because this drive is for 2.6, there will be many errors in compiling, modify cflags can be. Huaheng's group has already compiled a good driver for everyone to download.


Problems with module loading:

1. Error Opening V4L interface when running./servfox.
I test that is not loaded driver. Although the drive is selected in the kernel (M), it is manually added after startup. Insmod.
2. Insmod spcadecoder.o error occurred: Spcadecoder.o:couldn ' t find the kernel version this modules is compiled for. If you insmod SPCA5XX.O succeed, you don't need to insmod other modules.
3. Insmod VIDEO.O said can ' t find the kernel version the modules is compiled for.
This is because the video-for Linux is typically compiled directly into the kernel. Does not need to be loaded.

Two Video4linux programming

2.1 Video4linux Introduction
Video4linux is a common programming interface for television capture cards and USB-port cameras that are now commonplace in the marketplace. It also provides the decoding and vertical blanking data interface of radio communication and text TV broadcast. This article mainly aims at the USB camera device file/dev/video0, carries on the design of the video image acquisition aspect.

2.2 Video4linux Programming Guide

1. The process of video programming
(1) Open the video device:
(2) Reading device information
(3) Change device current settings (can not do)
(4) video capture, two ways:
A. Memory mapping
B. Read directly from the device
(5) Processing of the collected video
(6) Turn off the video device.

Defined data structures and usage functions
struct _V4L_STRUCT
{
int FD;
struct video_capability capability;
struct Video_buffer buffer;
struct Video_window window;
struct Video_channel channel[8];
struct video_picture picture;
struct Video_mmap mmap;
struct VIDEO_MBUF mbuf;
unsigned char *map;
};
typedef struct _V4L_STRUCT V4l_device;
extern int V4l_open (char *, v4l_device *);
extern int V4l_close (V4l_device *);
extern int v4l_get_capability (V4l_device *);
extern int V4l_set_norm (v4l_device *, int);
extern int V4l_get_picture (V4l_device *);
extern int V4l_grab_init (v4l_device *, int, int);
extern int V4l_grab_frame (v4l_device *, int);
extern int V4l_grab_sync (V4l_device *);
extern int V4l_mmap_init (V4l_device *);
extern int V4l_get_mbuf (V4l_device *);
extern int V4l_get_picture (V4l_device *);
extern int V4l_grab_picture (V4l_device *, unsigned int);
extern int V4l_set_buffer (V4l_device *);
extern int V4l_get_buffer (V4l_device *);
extern int V4l_switch_channel (v4l_device *, int);

Data structure supported by 3.video4linux and its use
(1) video_capability contains the basic information of the device (device name, supported maximum minimum resolution
Rate, signal source information, etc.)
NAME[32] Device Name
MaxWidth
MaxHeight
MinWidth
MinHeight
Number of channels signal sources
Type whether it can be capture, color or black and white, whether it can be cropped and so on. Values such as
Vid_type_capture, etc.

(2) Various properties of the image acquired by Video_picture equipment
Brightness 0~65535
Hue
Colour
Contrast
Whiteness
Depth 8 16 24 32
Palette Video_palette_rgb24 | video_palette_rgb565|
Video_palette_jpeg| Video_palette_rgb32

(3) Video_channel about the properties of each signal source
Number of Channel signal source
Name
Tuners
Type VIDEO_TYPE_TV | Ideo_type_camera
Norm format pal| nstc| secam| AUTO

(4) Video_window contains information about capture area
x x coordinates in Windows.
Y y coordinates in Windows.
Width the width of the image capture.
The height the height of the image capture.
Chromakey A Host order RGB32 value for the chroma key.
Flags Additional capture Flags.
Clips A List of clipping rectangles. (Set only)
Clipcount the number of clipping rectangles. (Set only)
(5) video_mbuf information of frames mapped using mmap
Size per frame
Frames the maximum number of frames supported
Offsets the offset of each frame relative to the base address
(6) Video_mmap for mmap

4. Introduction to Key steps

"Note" To connect multiple cameras. The method is as follows: Buy a USB hub and connect to the USB host on the Development Board. Cat/proc/devices can know that the major of video capture device is 81, then ls–l/dev see Video0 's secondary device number is 0. Two cameras of course, two device number, so mknod/dev/video1 C 81 1, if 4, on Mknod/dev/video2 c 2,MKNOD/DEV/VIDEO3 C 81 3. by analogy.

(1) Open Video:
int V4l_open (char *dev, V4l_device *VD)
{
if (!dev)
dev = "/dev/video0";
if (VD->fd = open (Dev, o_rdwr)) < 0) {
Perror ("V4l_open:");
return-1;
}
if (v4l_get_capability (VD))
return-1;
if (V4l_get_picture (VD))
Retu rn-1;
return 0;
}

(2) Read the information in video_capability
int v4l_get_capability (V4l_device *vd)
{
if (IOCTL (VD-&GT;FD, Vidiocgcap, & (vd->capability)) < 0) {
Perror ("v4l_get_capability:");
return-1;
}
return 0;
}
Can read vd->capability components after success

(3) Read the information in Video_picture
int v4l_get_picture (V4l_device *vd)
{
if (IOCTL (VD-&GT;FD, Vidiocgpict, & (vd->picture)) < 0) {
Perror ("v4l_get_picture:");
return-1;
}
return 0;
}
Properties of the image can be read after success

(4) Change the value of the component in the Video_picture (can not do)
Assign a new value to the component before calling Vidiocspict
Vd->picture.colour = 65535;
if (IOCTL (VD-&GT;FD, Vidiocspict, & (vd->picture)) < 0)
{
Perror ("Vidiocspict");
return-1;
}

(5) Initialization of channel (can not be done)
The information in vd->capability must be done first.
int V4l_get_channels (V4l_device *vd)
{
int i;
for (i = 0; i < VD->capability.channels; i++) {
VD->channel[i].channel = i;
if (IOCTL (VD-&GT;FD, Vidiocgchan, & (vd->channel[i)) < 0) {
Perror ("V4l_get_channel:");
return-1;
}
}
return 0;
}

(6) Turn off the equipment
int V4l_close (V4l_device *vd)
{
Close (VD-&GT;FD);
return 0;
}


Emphasis: Two methods of intercepting images

One, use mmap (memory mapping) way to intercept video
Mmap () system calls enable shared memory to be implemented by mapping the same common file between processes. After the normal file is mapped to the process address space, the process can access the file as normal memory, without having to call read (), write (), and so on. Two different processes A, B shared memory means that the same physical memory is mapped to process A, B's respective process address space. Process A can instantly see Process b update of the data in shared memory, and vice versa. One obvious benefit of using shared memory communications is that it is efficient because processes can read and write directly to memory without requiring copies of any data

(1) Set the properties of the picture
(2) Initialize the VIDEO_MBUF to get the information of the mapped buffer
IOCTL (VD-&GT;FD, Vidiocgmbuf, & (VD-&GT;MBUF))
(3) You can modify the current settings for Video_mmap and frame states
(4) Binding mmap with Video_mbuf
void* mmap (void * addr, size_t len, int prot, int flags, int fd, off_t offset)
Len: The number of bytes mapped to the calling process address space, starting with the offset bytes at the beginning of the mapped file
Prot: Specifies the access rights of shared memory prot_read (readable), prot_write (writable), Prot_exec (executable)
Flags:map_shared map_private must be selected, MAP_ fixed does not recommend the use of
ADDR: Total memory of the starting address, generally set 0, indicated by the system allocation
The Mmap () return value is the starting address of the system's actual allocation

int V4l_mmap_init (V4l_device *vd)
{
if (V4l_get_mbuf (VD) < 0)
return-1;
if (VD->map = mmap (0, Vd->mbuf.size, prot_read| Prot_write,
map_shared, VD-&GT;FD, 0)) < 0) {
Perror ("v4 l_mmap_init:mmap");
return-1;
}
return 0;
}

(5) Mmap way to do video interception vidiocmcapture

IOCTL (VD-&GT;FD, Vidiocmcapture, & (Vd->mmap));
If the call succeeds, the beginning of a frame of interception, non-blocking, whether the interception is left to Vidiocsync to judge

(6) Call Vidiocsync wait for the end of a frame intercept
if (IOCTL (VD-&GT;FD, Vidiocsync, &frame) < 0)
{
Perror ("V4l_sync:vidiocsync");
return-1;
}
If successful, it indicates that a frame interception has been completed. You can start vidiocmcapture next time.
Frame is the ordinal number of the frame that is currently being intercepted.


About double buffering ************
Video_bmuf bmuf.frames = 2; one frame is processed to capture another frame
int frame; Which frame is currently being collected
int framestat[2]; The state of the frame does not begin to collect | The address of the waiting for the end frame is obtained by Vd->map + vd->mbuf.offsets[vd->frame].
After the collection work, call Munmap to cancel the binding munmap (Vd->map, vd->mbuf.size) in the actual application can also use the buffer queue and other means.


Second, video interception method: direct Reading equipment
With regard to buffer size, the attributes of the image must be set by the user beforehand
Call Read ();
int read (the file descriptor to access; The pointer to the information to read and write; the number of characters that should be read or written); The return value is the number of characters that are actually read and write
int Len;
unsigned char
*vd->map=
(unsigned char *) malloc (vd◊capability.maxwidth*vd◊capability.maxheight);
Len = Read (Vd◊fd,vd◊vd->map,
VD◊CAPABILITY.MAXWIDTH*VD◊CAPABILITY.MAXHEIGHT*3);

2.3 Programming Instance (Mouse_capture)
Whether ov511 or zc301 camera, they collect the same way, but the data collected are different, ov511 is the RGB bit flow, and zc301 is the JPEG encoded bit stream. Mouse_capture is based on a servfox adapted from the zc301 camera to get a JPEG image,
A small program used to test whether a camera is loaded successfully. This allows you to test whether the camera is normal without cat/dev/video0>1.jpg. As soon as the cat command runs, a stream of JPEG streams is collected. But the collected pictures can only display the data between the first JPEG header and the JPEG tail. Mouse_capture only gets a full JPEG. You can download the reference from (http://www.hhcn.com/cgi-bin/topic.cgi?forum=1&topic=247&start=144&show=0).

the function of the main function is described as follows:
static int getvideopict (struct Vdin *VD);//Get Picture property information.
static int setvideopict (struct Vdin *VD);//Set picture properties.
static int isspcachip (const char *bridgename);//test chip type
static int Getstreamid (const char *bridgename); To test the format of output data
static int getdepth (int format);//Get color depth.
void Exit_fatal (char *messages);//error display.
int Init_videoin (struct Vdin *vd,char *device,int width,int);//Initialize the device.
int convertframe (unsigned char *dst,unsigned char *src, int width,int height, int formatin, int size);//Put the data in the shared buffer into a variable, Notifies the system that a frame has been obtained.
int V4lgrab (struct Vdin *vd,char *filename);//collect pictures from camera.
int close_v4l (struct Vdin *VD);//Turn off the camera.
int get_jpegsize (unsigned char *buf, int insize);//get JPEG picture size.


Three-instance Program

3.1 LCD Real-time display of images collected from ov511

Reference HHARM9-EDU/APPLICATIONS/USBCAM2LCD. The BMP bit flow from the camera is displayed directly in the framebuffer. This program image collection using read way, note that because LCD screen display is 16bits RGB picture, so, ov511 output of the picture format should be 16bits RGB picture data, Macro Video_palette_ RGB565 is defined as the 16bits RGB data picture. And the Linux ov511 drive with the image acquisition is 32 bits, so that the collected pictures on the LCD is the snow point. So you need to modify the driver. In the kernet/driver/usb/directory has the ov511 chip drive ov511.c, the drive Ov51x_set_default_params function is sets the chip default output picture The format, in this function
for (i = 0; i < ov511_numframes; i++)
{
Ov511->frame[i].width = ov511->maxwidth;
Ov511->frame[i].height = ov511->maxheight;
Ov511->frame[i].bytes_read = 0;
if (Force_palette)
Ov511->frame[i].format = Force_palette;
Else
Ov511->frame[i].format = Video_palette_rgb24;
Ov511->frame[i].depth = Ov511_get_depth (Ov511->frame[i].format);
}

Some of the statements are mainly set ov511 default output picture format, where MaxWidth and maxheight set the maximum width and height of the picture. The IfElse statement sets the format of the picture, making the following modifications:
for (i = 0; i < ov511_numframes; i++)
{
Ov511->frame[i].width = ov511->maxwidth;
Ov511->frame[i].height = ov511->maxheight;
Ov511->frame[i].bytes_read = 0;
Ov511->frame[i].format = video_palette_rgb565;
Ov511->frame[i].depth = Ov511_get_depth (Ov511->frame[i].format);
}
If you want, you can also change the default output size of the picture.


3.2 LCD Real-time display of images collected from zc301
Programming idea: The picture that collects from the camera is stored in the local folder, the display is realized by Minigui loading JPEG.
Specific process:
1. Download the jpegsrc-6b JPEG library from the Internet and cross-compile.
(1)./configure–enable-static–enable-shared–prefix=.libs
(2) Modify the makefile and change the compiler to a cross compiler. For example: I changed into/OPT/HOST/ARMV4L/BIN/ARMV4L-UNKNOWN-LINUX-GCC
(3) After make is generated in the. Libs directory for ARM LIBJPEG.A, libjpeg.la,libjpeg.so,libjpeg.so.62,libjpeg.so.62.0.0. Copy these files to the system library file directory, and mine is in/usr/lib.

2. The JPEG header is FF D8 FF DB because it looks at the bits stream of the pictures collected from zc301. While in the Minigui library file Libminigui source file src/mybmp/jpeg.c, load_jpg and check_jpg were tested in header EXIF and JFIF in two formats of JPEG images. The two corresponding binary systems are FF D8 FF E1 and FF D8 FF E0 respectively. So we minigui by judging that this is the wrong JPEG format without loading, so it cannot be displayed. In fact, through testing, in the source code to remove these two judgments can be properly loaded.

3. Cross-compile Minigui

(1) Compile library:./configure--host=arm-unknown-linux--enable-jpgsupport=yes
--enable-pngsupport=no--enable-gifsupport=no--disable-lite
--prefix=/hharm9-edu/applications/minigui-free/nfsroot
--enable-smdk2410ial=yes
Make
Make install

(2) When compiling the instance program, add the support of the JPEG library, that is, add-ljpeg to the makefile. The library files and executables generated in Nfsroot are moved to the corresponding directory in ramdisk.image.gz. (Refer to Huaheng's 2410 development manual).


3. Preparation of Minigui Program

Programming tips, the way I take is a moment of non-stop collection from the camera to the image stored in the/tmp/1.jpg, in the Minigui through the LoadBitmap function to load the picture. And the picture is not automatically updated after loading, can not automatically change according to 1.jpg changes. So I set a timer in the program.

Every 100ms refresh screen, basically realize real-time update. Another problem is that the refresh will fill the desktop with a background color, causing the screen to flicker heavily. So think of the way to use the MSG_ERASEBKGND, using the previous picture as a refresh screen when the fill background picture. This ensures the continuity of the image on the LCD.
The Minigui program is as follows: Some of the custom functions are the same as those in mouse_capture, except that they are only variable to collect multiple pieces. You can change it yourself:). You can also ask me for the source code.
#include <minigui/common.h>
#include <minigui/minigui.h>
#include <minigui/gdi.h>
#include <minigui/window.h>
#include <minigui/control.h>
#include "spcav4l.h"
#define Idtimer 100
Static BITMAP bmp;
static int Loadbmpwinproc (HWND hwnd, int message, WPARAM WPARAM,
LPARAM LPARAM)
{
HDC HDC;
RECT rc={0,0,240,320};
Switch (message) {
Case Msg_create:
SetTimer (hwnd,idtimer,100);
return 0;
Case MSG_ERASEBKGND:
{
RECT rctemp;
if (LoadBitmap (hdc_screen,&bmp, "/tmp/1.jpg"))
{
printf ("Load wrong!/n");
return-1;
}
GetClientRect (HWnd, &rctemp);
HDC = BeginPaint (HWND);
Fillboxwithbitmap (hdc, Rctemp.left, Rctemp.top, Rctemp.right-rctemp.left,
Rctemp.bottom-rctemp.top, &bmp);
EndPaint (HWND, HDC);
return 0;
}
Case Msg_timer:
InvalidateRect (hwnd,&rc,true);
return 0;
Case Msg_close:
Unloadbitmap (&bmp);
Destroymainwindow (HWND);
PostQuitMessage (HWND);
return 0;
}
Return Defaultmainwinproc (hWnd, message, WParam, LParam);
}
int miniguimain (int argc, const char* argv[])
{
MSG msg;
HWND Hmainwnd;
Mainwincreate Createinfo;
Char videodevice[] = "/dev/video0";
Char jpegfile[] = "/tmp/1.jpg";
int grabmethod = 0;
int format = Video_palette_jpeg;
int width = 240;
int height = 320;
int i;
#ifdef _lite_version
Setdesktoprect (0, 0, 1024, 768);
#endif
Createinfo.dwstyle = ws_visible | Ws_border | ws_caption;
Createinfo.dwexstyle = Ws_ex_none;
Createinfo.spcaption = "Load and display a bitmap";
Createinfo.hmenu = 0;
Createinfo.hcursor = getsystemcursor (0);
Createinfo.hicon = 0;
Createinfo.mainwindowproc = Loadbmpwinproc;
CREATEINFO.LX = 0;
Createinfo.ty = 0;
Createinfo.rx = 240;
Createinfo.by = 320;
Createinfo.ibkcolor = Pixel_lightwhite;
Createinfo.dwadddata = 0;
createinfo.hhosting = Hwnd_desktop;
Hmainwnd = Createmainwindow (&createinfo);
if (Hmainwnd = = hwnd_invalid)
return-1;
ShowWindow (Hmainwnd, SW_SHOWNORMAL);
memset (&videoin, 0, sizeof (struct vdin));
if (Init_videoin (&videoin, videodevice, width, height, format,grabmethod) = = 0)
{
printf ("Init is ok!/n");
}
else printf ("Init is wrong!/n");
while (GetMessage (&msg, Hmainwnd)) {
TranslateMessage (&AMP;MSG);
V4lgrab (&videoin, jpegfile);
DispatchMessage (&AMP;MSG);
}
CLOSE_V4L (&videoin);
Mainwindowthreadcleanup (Hmainwnd);
return 0;
}
#ifndef _lite_version
#include <minigui/dti.c>
#endif
First write to here, hehe, hope to be helpful to you. If you find the question in the process of reading
Questions, welcome to communicate with me.

Reference Documents
1. HHARM2410 camera Debug record Hua Heng Technology
2. Video device programming based on Video4linux Lingzhi_shi APR 7 2004
3. "Video4linux Programming" Alan Cox
4. "Video streaming discussion" Chen Junhong
5. "Video4linux Kernel API Reference"