The JPEG format of embedded Linux based on framebuffer device is displayed on the local LCD screen

In the Linux based video surveillance acquisition system, the camera captured a frame of video image data are generally hardware automatically compressed into the JPEG format, and then saved to the camera device buffer. If you want to display the collected JPEG format on the local LCD screen, Since our Linux system does not transplant any GUI systems, consider the following: 1. Unzip JPEG format to bitmap format, i.e. JPEG decoding. 2. The decoded bitmap format is exported to the local LCD screen. The Linux system is implemented by writing a frame buffer (framebuffer). 3. Framebuffer is equivalent to providing a unified interface for LCD devices, the control of framebuffer will be reflected in the LCD display equipment. If the Linux kernel was not found to support the model of the local LCD screen driver, then we have to write the LCD screen driver, and then choose the static join the kernel or in the form of modules to join the kernel dynamic loading. For the above three points, we solve each:





1. JPEG decoding





Let's take a look at the coding process for the JPEG standard: The original frame of uncompressed image can be regarded as RGB (red-green-blue) color space on a set of vectors, but in the RGB space is not conducive to data compression, so in order to compress the image first to facilitate the compression of the YUV space ( The reason is that human eyes are more sensitive to brightness differences than color changes, and the YUV space of a base vector y is brightness), this step is called color space conversion. Next, you can reduce the composition of U (hue) and V (saturation) in YUV space, which is to remove some color information without decreasing the brightness information , who called the eyes of the human eye on the brightness is better than the sensitivity of color. This is called reduced sampling. The next step is to map the image from the color space to the frequency space, the transformation methods can be used: discrete cosine transform, Fourier transform, sine transform and so on. The most widely used is the discrete cosine transform (DCT). This step is lossless and is intended to be rounded to remove the compressed matrix after the high-frequency amount is removed in the next step called quantization. Quantization is the problem of encoding the matrix. In view of the distribution of this matrix, the "Z" Glyph of sequential scan choreography, and then carry out RLE stroke coding, a large number of continuous repetitive data compression. Finally, the Huffman code is used. To understand the detailed procedures, you can view the JPEG standard. And decoding is the inverse process of the above coding. Unless you want to implement JPEG encoding and decoding functions yourself, we can do without having to examine these processes, but directly using the JPEG codec libraries that others have already implemented. Under the Linux platform, there is a libjpeg library, which is written entirely in C language, According to its license agreement, it is free to use, not the GPL agreement, it can be used for commercial purposes. A problem with the 6b version of Libjpeg is the decoding interface, which only accepts file sources. Open the function of the source jpeg_stdio_src (j_decompress_ptr cinfo, file *infile) Requires decoding the source infile is a file. And we want to decode the data that comes directly from the mapped memory. To decode the memory stream, you need to modify the Libjpeg source code, you can refer to here: http://my.unix-center.net/~Simon_fu/?p=565 Currently Libjpeg's latest version of 8c has solved this interface problem, and it has increased support for decoding the memory stream. By calling a function





JPEG_MEM_SRC (&cinfo, Fdmem, st.st_size);





You can enter the JPEG format data saved in memory as the source. So we use the Libjpeg 8c to decode this version. The main functions used are:





Initialize JPEG Decompression object:





/* init JPEG Decompress object Error Handler * *


Cinfo.err = Jpeg_std_error (&jerr);


Jpeg_create_decompress (&cinfo);





Binding JPEG Decompression object to input stream:





/* Bind JPEG Decompress object to infile * *


#if read_file//read from a JPEG file


JPEG_STDIO_SRC (&cinfo, infile);


#elif Read_mem//read from memory into the JPEG format


JPEG_MEM_SRC (&cinfo, Fdmem, st.st_size);


#endif





To read the JPEG header information:





/* Read JPEG header * *


Jpeg_read_header (&cinfo, TRUE);





Decompression process:





* Decompress process/*


Jpeg_start_decompress (&cinfo);





After you call this function, you can get the following parameters for the JPEG image:





Width of output_width//image


Output_height//height of image


Output_components//Number of bytes per pixel





We use the method of outputting a line of pixels per pixel to the screen, and according to the above parameters we can determine the buffer to allocate one line of information:





Buffer = (unsigned char *) malloc (Cinfo.output_width *


cinfo.output_components);





A total of output_height lines need to be scanned.





Read one line of scanned data and output to the LCD screen:





y = 0;


while (Cinfo.output_scanline < cinfo.output_height) {


Jpeg_read_scanlines (&cinfo, &buffer, 1);


if (fb_depth = = 16) {//If the display device color depth is 16 bits


...


else if (fb_depth = 24) {//If the display device color depth is 24 bits


...


else if (fb_depth = 32) {//If the display device color depth is 32 bits


...


}


y++;


}





End JPEG decoding:





/* Finish decompress, destroy decompress object * *


Jpeg_finish_decompress (&cinfo);


Jpeg_destroy_decompress (&cinfo);





To release a buffer:





/* Release Memory Buffer * *


Free (buffer);





2. Output bitmap to LCD screen





The main steps to write a screen directly through Framebuffer are:





To open the framebuffer device:





/* Open framebuffer device * *


Fbdev = Fb_open ("/dev/fb0");





Get framebuffer Device Parameters:





/* Get Status of framebuffer device * *


Fb_stat (Fbdev, &fb_width, &fb_height, &fb_depth);





To map framebuffer devices to shared memory:





screensize = fb_width * Fb_height * FB_DEPTH/8;


Fbmem = Fb_mmap (Fbdev, screensize);





directly to map to the memory of the write operation, the LCD screen refreshing refresh will be reflected to the screen up.





y = 0;


while (Cinfo.output_scanline < cinfo.output_height) {


Jpeg_read_scanlines (&cinfo, &buffer, 1);


if (fb_depth = = 16) {


unsigned short color;





for (x = 0; x < Cinfo.output_width + +) {


color =


rgb888torgb565 (BUFFER[X * 3],


BUFFER[X * 3 + 1], Buffer[x * 3 + 2]);


Fb_pixel (Fbmem, Fb_width, Fb_height, x, y, color);


}


else if (fb_depth = 24) {


memcpy ((unsigned char *) Fbmem + y * fb_width * 3,


Buffer, Cinfo.output_width * cinfo.output_components);


else if (fb_depth = 32) {


memcpy ((unsigned char *) Fbmem + y * fb_width * 4,


Buffer, Cinfo.output_width * cinfo.output_components);


for (x = 0; x < Cinfo.output_width + +) {


* (Fbmem + y * fb_width * 4 + x * 4) = (unsigned char) buffer[x * 3 + 2];


* (Fbmem + y * fb_width * 4 + x * 4 + 1) = (unsigned char) buffer[x * 3 + 1];


* (Fbmem + y * fb_width * 4 + x * 4 + 2) = (unsigned char) buffer[x * 3 + 0];


* (Fbmem + y * fb_width * 4 + x * 4 + 3) = (unsigned char) 0;


}


}


y++; Next Scanline


}





Uninstall the portion of memory that maps framebuffer:





/* Unmap framebuffer ' s shared memory * *


Fb_munmap (Fbmem, screensize);





Turn off the framebuffer device:





Close (Fbdev);





Based on the above two points, you can write a test program, without opening the X-window graphics system, the local JPEG file directly to the screen.





#include <stdio.h>


#include <string.h>


#include <stdlib.h>


#include <unistd.h>


#include <sys/ioctl.h>


#include <sys/types.h>


#include <sys/stat.h>


#include <errno.h>


#include <fcntl.h>


#include <sys/mman.h>


#include <linux/fb.h>


#include "Jpeglib.h"


#include "Jerror.h"





#define FB_DEV "/dev/fb0"


#define __FNC__ __function__





#define DEBUG 0


#define DEBUG_PRINTF 0


#define BYREAD 0


#define BYMEM 1





/* Function Deciaration */





void usage (char *msg);


unsigned short rgb888torgb565 (unsigned char red,


unsigned char green, unsigned char blue);


int Fb_open (char *fb_device);


int fb_close (int fd);


int fb_stat (int fd, unsigned int *width, unsigned int *height, unsigned int * depth);


void *fb_mmap (int fd, unsigned int screensize);


void *fd_mmap (int fd, unsigned int filesize);


int Fb_munmap (void *start, size_t length);


int Fb_pixel (void *fbmem, int width, int height,


int x, int y, unsigned short color);





#if (Debug)


void Draw (unsigned char *fbp,


struct Fb_var_screeninfo vinfo,


struct Fb_fix_screeninfo finfo);


#endif





/* Function Implementation */





int main (int argc, char **argv)


{


struct Jpeg_decompress_struct cinfo;


struct Jpeg_error_mgr jerr;


#if (Byread)


FILE *infile;


#endif


int FD;


unsigned char *buffer;


struct STAT st;





int Fbdev;


Char *fb_device;


unsigned char *fbmem;


unsigned char *fdmem;


unsigned int screensize;


unsigned int fb_width;


unsigned int fb_height;


unsigned int fb_depth;


register unsigned int x;


register unsigned int y;





/* Check auguments * *


if (argc!= 2) {


Usage ("Insuffient auguments");


Exit (-1);


}





/* Open framebuffer device * *


if ((Fb_device = getenv ("framebuffer")) = = NULL)


Fb_device = Fb_dev;


Fbdev = Fb_open (Fb_device);





/* Get Status of framebuffer device * *


Fb_stat (Fbdev, &fb_width, &fb_height, &fb_depth);





/* Map framebuffer device to shared memory * *


screensize = fb_width * Fb_height * FB_DEPTH/8;


Fbmem = Fb_mmap (Fbdev, screensize);





#if (Byread)


/* Open input JPEG file */


if ((infile = fopen (argv[1], rb)) = = NULL) {


fprintf (stderr, "open%s Failedn", argv[1]);


Exit (-1);


}


#endif





if (fd = open (argv[1], o_rdonly) < 0) {


Perror ("open");


Exit (-1);


}





if (Fstat (FD, &st) < 0) {


Perror ("Fstat");


Exit (-1);


}





Fdmem = Fd_mmap (fd, st.st_size);





/* init JPEG Decompress object Error Handler * *


Cinfo.err = Jpeg_std_error (&jerr);


Jpeg_create_decompress (&cinfo);





/* Bind JPEG Decompress object to infile * *


#if (Byread)


JPEG_STDIO_SRC (&cinfo, infile);


#endif





#if (BYMEM)


JPEG_MEM_SRC (&cinfo, Fdmem, st.st_size);


#endif





/* Read JPEG header * *


Jpeg_read_header (&cinfo, TRUE);





* Decompress process/*


Jpeg_start_decompress (&cinfo);


if ((Cinfo.output_width > Fb_width) | |


(Cinfo.output_height > Fb_height)) {


printf ("Too large JPEG file, can ' t Displayn");


#if (0)


return-1;


#endif


}





Buffer = (unsigned char *) malloc (Cinfo.output_width *


cinfo.output_components);





struct Fb_fix_screeninfo fb_finfo;


struct Fb_var_screeninfo fb_vinfo;





if (IOCTL (Fbdev, Fbioget_fscreeninfo, &fb_finfo)) {


Perror (__fnc__);


return-1;


}





if (IOCTL (Fbdev, Fbioget_vscreeninfo, &fb_vinfo)) {


Perror (__fnc__);


return-1;


}





#if (Debug)


Draw (Fbmem, Fb_vinfo, Fb_finfo);


#endif


y = 0;


while (Cinfo.output_scanline < cinfo.output_height) {


Jpeg_read_scanlines (&cinfo, &buffer, 1);


if (fb_depth = = 16) {


unsigned short color;





for (x = 0; x < Cinfo.output_width + +) {


color =


rgb888torgb565 (BUFFER[X * 3],


BUFFER[X * 3 + 1], Buffer[x * 3 + 2]);


Fb_pixel (Fbmem, Fb_width, Fb_height, x, y, color);


}


else if (fb_depth = 24) {


memcpy ((unsigned char *) Fbmem + y * fb_width * 3,


Buffer, Cinfo.output_width * cinfo.output_components);


else if (fb_depth = 32) {


memcpy ((unsigned char *) Fbmem + y * fb_width * 4,


Buffer, Cinfo.output_width * cinfo.output_components);


for (x = 0; x < Cinfo.output_width + +) {


* (Fbmem + y * fb_width * 4 + x * 4) = (unsigned char) buffer[x * 3 + 2];


* (Fbmem + y * fb_width * 4 + x * 4 + 1) = (unsigned char) buffer[x * 3 + 1];


* (Fbmem + y * fb_width * 4 + x * 4 + 2) = (unsigned char) buffer[x * 3 + 0];


* (Fbmem + y * fb_width * 4 + x * 4 + 3) = (unsigned char) 0;


}


}


y++; Next Scanline


}





/* Finish decompress, destroy decompress object * *


Jpeg_finish_decompress (&cinfo);


Jpeg_destroy_decompress (&cinfo);





/* Release Memory Buffer * *


Free (buffer);





#if (Byread)


/* Close JPEG inputing file */


Fclose (infile);


#endif





/* Unmap framebuffer ' s shared memory * *


Fb_munmap (Fbmem, screensize);





#if (BYMEM)


Munmap (Fdmem, (size_t) st.st_size);


Close (FD);


#endif





/* Close framebuffer device * *


Fb_close (Fbdev);





return 0;


}





void usage (char *msg)


{


fprintf (stderr, "%SN", msg);


printf ("USAGE:FV some-jpeg-file.jpgn");


}





/* Open framebuffer device.


* Return positive file descriptor if success,


* Else return-1


*/


int Fb_open (char *fb_device)


{


int FD;





if (fd = open (Fb_device, O_RDWR)) < 0) {


Perror (__fnc__);


return-1;


}


return FD;


}





int fb_close (int fd)


{


Return (Close (FD));


}





/* Get framebuffer ' s width, height, and depth.


* Return 0 if success, else return-1.


*/


int fb_stat (int fd, unsigned int *width, unsigned int *height, unsigned int * depth)


{


struct Fb_fix_screeninfo fb_finfo;


struct Fb_var_screeninfo fb_vinfo;





if (IOCTL (FD, Fbioget_fscreeninfo, &fb_finfo)) {


Perror (__fnc__);


return-1;


}





if (IOCTL (FD, Fbioget_vscreeninfo, &fb_vinfo)) {


Perror (__fnc__);


return-1;


}





*width = Fb_vinfo.xres;


*height = Fb_vinfo.yres;


*depth = Fb_vinfo.bits_per_pixel;





return 0;


}





/* Map shared memory to framebuffer device.


* Return maped Memory if success


* Else return-1, as mmap dose


*/


void *fb_mmap (int fd, unsigned int screensize)


{


caddr_t Fbmem;





if (Fbmem = mmap (0, screensize, Prot_read | Prot_write,


map_shared, FD, 0) = = map_failed) {


Perror (__func__);


return (void *) (-1);


}





return fbmem;


}





/* Map shared Memmory to a opened file */


void *fd_mmap (int fd, unsigned int filesize)


{


caddr_t Fdmem;





if (Fdmem = mmap (0, FileSize, Prot_read,


map_shared, FD, 0) = = map_failed) {


Perror (__func__);


return (void *) (-1);


}





return fdmem;


}





/* unmap map memory for framebuffer device * *


int Fb_munmap (void *start, size_t length)


{


Return (Munmap (start, length));


}





/* Convert 24bit RGB888 to 16bit RGB565 color format */


unsigned short rgb888torgb565 (unsigned char red,


unsigned char green, unsigned char blue)


{


unsigned short B = (blue >> 3) & 0x001f;


unsigned short G = ((green >> 2) << 5) & 0x07e0;


unsigned short R = ((Red >> 3) << one) & 0xf800;





return (unsigned short) (R | G | B);


}





/* Display a pixel on the framebuffer device.


* Fbmem is the starting memory of Framebuffer,


* Width and height are dimension of framebuffer,


* Width and height are dimension of framebuffer,


* x and Y are the coordinates to display,


* Color is the pixel ' s color value.


* Return 0 if success, otherwise return-1.


*/


int Fb_pixel (void *fbmem, int width, int height,


int x, int y, unsigned short color)


{


if ((x > width) | | (Y > Height))


return-1;





unsigned short *DST = (unsigned short *) Fbmem + y * width + x);





*DST = color;


return 0;


}





3. LCD Driver





We use a piece of Donghua 3.5-inch digital screen, the model is WXCAT35-TG3. The following driver is written by Vedon teacher in class, as follows:





#include <linux/module.h>


#include <linux/kernel.h>


#include <linux/errno.h>


#include <linux/string.h>


#include <linux/mm.h>


#include <linux/slab.h>


#include <linux/delay.h>


#include <linux/interrupt.h>


#include <linux/fb.h>


#include <linux/init.h>


#include <linux/ioport.h>


#include <linux/dma-mapping.h>





#include <asm/uaccess.h>


#include <asm/system.h>


#include <asm/irq.h>


#include <asm/setup.h>





* WXCAT35-TG3 * *


struct S3c_lcd_regs {


unsigned long lcdcon1;


unsigned long lcdcon2;


unsigned long lcdcon3;


unsigned long lcdcon4;


unsigned long lcdcon5;


unsigned long lcdsaddr1;


unsigned long LCDSADDR2;


unsigned long LCDSADDR3;


unsigned long redlut;


unsigned long greenlut;


unsigned long bluelut;


unsigned long reserved[9];


unsigned long dithmode;


unsigned long tpal;


unsigned long lcdintpnd;


unsigned long lcdsrcpnd;


unsigned long Lcdintmsk;


unsigned long Lpcsel;


};





Static U32 colregs[16];


static struct Fb_info *s3c_fb_info;


Static dma_addr_t S3c_fb_handle;


static unsigned long fb_va;





* FROM PXAFB.C * *


static inline unsigned int chan_to_field (unsigned int chan, struct Fb_bitfield)


{


Chan &= 0xFFFF;


Chan >>= 16-bf->length;


Return Chan << bf->offset;


}





static int S3cfb_setcolreg (unsigned regno,


Unsigned red, unsigned green, unsigned blue,


Unsigned transp, struct fb_info *info)


{


unsigned int val;





/* DPRINTK ("setcol:regno=%d, Rgb=%d,%d,%dn", Regno, red, green, blue); */





/* True-colour, use Pseuo-palette * *





if (Regno < 16) {


U32 *pal = s3c_fb_info->pseudo_palette;





val = Chan_to_field (red, &s3c_fb_info->var.red);


Val |= Chan_to_field (Green, &s3c_fb_info->var.green);


Val |= Chan_to_field (Blue, &s3c_fb_info->var.blue);





Pal[regno] = val;


}





return 0;


}





static struct Fb_ops S3cfb_ops = {


. Owner = This_module,


. Fb_check_var = Clps7111fb_check_var,


. Fb_set_par = Clps7111fb_set_par,


. Fb_setcolreg = Clps7111fb_setcolreg,


. Fb_blank = Clps7111fb_blank,





. Fb_setcolreg = S3cfb_setcolreg,


. Fb_fillrect = Cfb_fillrect,


. Fb_copyarea = Cfb_copyarea,


. Fb_imageblit = Cfb_imageblit,


};





struct S3c_lcd_regs *s3c_lcd_regs;


Static volatile unsigned long *gpccon;


Static volatile unsigned long *gpdcon;


Static volatile unsigned long *gpgcon;





int s3c_lcd_init (void)


{


extern int DEBUG_LCD;


/* 1. Assign a FB_INFO structure body * *


S3c_fb_info = Framebuffer_alloc (0, NULL);


PRINTK ("%s%dn", __function__, __line__);





/* 2. Set FB_INFO structure Body * *


/*


2.1 Set the fixed information


2.2 Setting variable Information


2.3 Set Action function


*/





/* 24BPP (bits per pixel), which will use 4 bytes, which waste 1 bytes */


strcpy (s3c_fb_info->fix.id, "wxcat35-tg3");


S3c_fb_info->fix.smem_start//Frame buffer ' s physical address


S3c_fb_info->fix.smem_len = 320*240*32/8;


S3c_fb_info->fix.type = Fb_type_packed_pixels;


S3c_fb_info->fix.visual = Fb_visual_truecolor;


S3c_fb_info->fix.line_length = 320 * 4;





S3c_fb_info->var.xres = 320;


S3c_fb_info->var.yres = 240;


S3c_fb_info->var.xres_virtual = 320;


S3c_fb_info->var.yres_virtual = 240;


S3c_fb_info->var.bits_per_pixel = 32;





S3c_fb_info->var.red.offset = 16;


S3c_fb_info->var.red.length = 8;





S3c_fb_info->var.green.offset = 8;


S3c_fb_info->var.green.length = 8;





S3c_fb_info->var.blue.offset = 0;


S3c_fb_info->var.blue.length = 8;





S3c_fb_info->var.activate = fb_activate;





S3c_fb_info->fbops = &s3cfb_ops;


S3c_fb_info->pseudo_palette = COLREGs;





/* 3. Hardware-related Operations * *


/* Configure GPIO * *


Gpccon = Ioremap (0x56000020, 4);


Gpdcon = Ioremap (0x56000030, 4);


Gpgcon = Ioremap (0x56000060, 4);


*gpccon = 0XAAAAAAAA;


*gpdcon = 0XAAAAAAAA;


*gpgcon |= (3<<8); /* GPG4 use as Lcd_pwren * *


PRINTK ("%s%dn", __function__, __line__);





S3c_lcd_regs = Ioremap (0x4d000000, sizeof (struct s3c_lcd_regs));





/*


* VCLK = HCLK/[(clkval+1) x2] = 100m/[(clkval+1) x2] = 6.4


* Clkval = 6.8 = 7


* TFT LCD Panel


* 24BPP


*/


S3c_lcd_regs->lcdcon1 = (7<<8) | (0<<7) | (3<<5) | (0x0d<<1) | (0<<0);


PRINTK ("%s%dn", __function__, __line__);





* VBPD: After the electron gun receives the vsync signal, "How long" can jump back to the 1th line


* Vbpd=14, lcd:tvb=15


* lineval=239, LCD: 240 lines


* vfpd=11, LCD:TVF=12//Send the last line of data, after a long time before the issue of VSync


* vspw=2, lcd:tvp=3//VSync width


*/


S3c_lcd_regs->lcdcon2 = (14<<24) | (239<<14) | (11<<6) | (2<<0);





* HBPD: After the electron gun receives the hsync signal, "How long" can jump back to the 1th column


* hbpd=37, lcd:thb=38


* horval=319, LCD: 320 lines


* hfpd=19, LCD:THF=20//Send last pixel data, it is too long to send out HSync


* hspw=29, lcd:thp=30//VSync width


*/


S3c_lcd_regs->lcdcon3 = (37<<19) | (319<<8) | (19<<0);


S3c_lcd_regs->lcdcon4 = 29;





/* Bit10: In the VCLK rise along the fetch data


* Bit9:vsync Low Level effective


* Bit8:hsync Low Level effective


* Bit5:pwren Low Level effective


*/


S3c_lcd_regs->lcdcon5 = (1<<10) | (1<<9) | (1<<8) | (1<<5) | (0<<3);





/* Allocate Frame buffer * *


Fb_va = (unsigned long) dma_alloc_writecombine (NULL, S3c_fb_info->fix.smem_len, &s3c_fb_handle, Gfp_kernel);





PRINTK ("Fb_va = 0x%x, PA = 0x%xn", Fb_va, S3c_fb_handle);


S3c_fb_info->fix.smem_start = S3c_fb_handle;


S3c_fb_info->screen_base = Fb_va;





* * Tell the framebuffer address to the LCD controller * *


S3C_LCD_REGS->LCDSADDR1 = (S3c_fb_info->fix.smem_start >> 1);


S3C_LCD_REGS->LCDSADDR2 = ((s3c_fb_info->fix.smem_start+320*240*4) >> 1) & 0x1fffff;


S3C_LCD_REGS->LCDSADDR3 = 320*4/2;





/* Enable LCD */


S3c_lcd_regs->lcdcon1 |= (1<<0);





/* 4. Register_framebuffer *


PRINTK ("%s%dn", __function__, __line__);


DEBUG_LCD = 1;


Register_framebuffer (S3c_fb_info);


PRINTK ("%s%dn", __function__, __line__);





return 0;


}





void S3c_lcd_exit (void)


{


Unregister_framebuffer (S3c_fb_info);


Dma_free_writecombine (NULL, S3c_fb_info->fix.smem_len, Fb_va, S3c_fb_handle);


Iounmap (S3c_lcd_regs);


Iounmap (Gpccon);


Iounmap (Gpdcon);


Iounmap (Gpgcon);


Framebuffer_release (S3c_fb_info);


}





Module_init (S3c_lcd_init);


Module_exit (S3c_lcd_exit);





Module_license ("GPL");





It is then added to the kernel to start in a statically loaded mode. Finally, you can read the memory JPEG data output to the LCD screen in this part of the integration to the Mjpg-stream or servfox, it realized the local display of the acquisition image.