HLS Image Processing Series--build DDR image processing pathway in Zedboard

ZYNQ The chip contains a rich feature of the dual-core ARM cortex-a9 processing subsystem (processing System,ps) and Xilinx 28nm programmable logic (Programmable LOGIC,PL). In addition to the core, PS includes on-chip memory, external memory interface and a large number of peripheral connection interfaces.

With arm, we can do the embedded operating system related tasks, shape interface, user input, network, DDR3 control, etc., because arm itself has a rich peripheral interface, and support multi-level pipeline, handling these transactions, However, for large computational applications, because arm itself is a typical serial processor, not suitable for big data, high-real-time processing tasks. FPGA just make up this point, the use of programmable logic can achieve parallel processing, as long as the logical resources enough, we can use the space-time strategy, so that multiple computing units at the same time, can greatly shorten processing time.

Image processing with ZYNQ has architectural advantages, so it is necessary for a set of ZYNQ image paths to be used for ZYNQ video-related developers. This post describes a camera +hls image processing +DDR storage +VGA display image path. This channel is I and another colleague to achieve, I am responsible for the camera FPGA driver, HLS image processing IP Implementation and the system later optimization. Because of the company project, it is not possible to provide engineering documents, only to provide the framework and ideas.

The system framework is as follows:

Key modules introduced:

Camera: Using ov7725 camera, frame rate 60fps, resolution 640*480. For a detailed description of the camera timing, please refer to my Blog "tutorial – Implementing an Image path in Zedboard (Block RAM version)"

Post-packaged camera IP such as:

VDMA: In this design, the VDMA's position is similar to the usual design of DMA. However, unlike DMA, the VDMA provides a highly-stored access module specifically for video streaming data. It receives video data through the Axi-stream protocol, while control signals (such as the size of the frame buffer, the opening and closing of the DMA function, and some other settings) are accessed through the Axi-lite from other interfaces.

The VDMA has two DMA paths, and the s2mm path maps the input Axi4-stream data stream to the specified frame buffer, while mm2s, instead, outputs the framebuffer as a axi4-stream type of data flow.

VDMA's Configuration Interface

You can see some configuration information for the data frame, as well as the settings for the number of framebuffers. The number of framebuffers is at least 3, because if there are fewer than 3, there is a conflict between the write DDR and the read DDR, causing the image to have a fault phenomenon. In this design, the framebuffers is set to 3.

video in to Axi4-stream with Axi4-stream to videoout: Because the protocol used by the camera's incoming signal is not axi4-stream, and the VDMA needs to receive the Axi4-stream type of data, Therefore, a dedicated IP core is required to perform the conversion between the two.

These two IP cores are:

Video in to Axi4-stream, which converts the visual data to the Axi4-stream type. In this project, video data from the camera module, only need the camera to provide four signals: line synchronization, field synchronization, 24bit pixels, data effective signal.

Axi4-stream to video out, the IP converts axi4-stream type data into a visual data. Video data is a VGA signal, including line synchronization, field synchronization, data valid signals, and pixels. The module has a timing input, in this system, from the video timing control IP. The video Timing control module provides a 640*480p VGA timing signal to the Axi4-stream to Video out IP.

For video in to Axi4-stream IP with Axi4-stream to Video out IP:

These two IP cores are in a bridge position in this design and are used to connect external IO and VDMA. The video data inside these two modules is axi4-stream type, eliminating the address and therefore faster and easier to handle. In general, the HLS hardware image processing module will be placed between the two IP cores.

HLS image Processing Module : In this design, a simple Sobel image processing module is built, which is generated and packaged into IP by HLS tools. About HLS tools do not introduce, you can refer to my blog "HLS image Processing Series-preface" to get a preliminary understanding of HLS. The IP input and output video streams generated by HLS are compliant with the Axi4-stream protocol.

Then, attach the Vivado project:

In the SDK project, the main is the initialization of HLS IP and the VDMA initialization. Only after initialization can the entire image path run correctly. HLS initialization of the source code just a few lines, but also tortured me for a long time, because the need to shut down the interruption, for a long period did not find this problem. The main source of initialization is as follows:

/************config HLS ip********/void configurehlsip (ximgprocess_top *imgprocess) {Imgprocess->control_bus_ baseaddress = Xpar_imgprocess_top_0_s_axi_control_bus_baseaddr;imgprocess->isready = XIL_COMPONENT_IS_READY; Ximgprocess_top_enableautorestart (imgprocess); Ximgprocess_top_setrows (imgprocess, 480); Ximgprocess_top_setcols (imgprocess,640); Ximgprocess_top_interruptdisable (imgprocess, 0xFFFFFFFF); Ximgprocess_top_interruptglobaldisable (imgprocess); Ximgprocess_top_start (imgprocess);}    int main () {init_platform (); Usleep (100000);    Print ("Hello world\n\r"); Configurehlsip (&imgprocess);//Mm2sxil_out32 (xpar_axi_vdma_0_baseaddr + 0x00, 0x008b);//enable Run, Circular_ Park, Genlocken, Genlocksrcxil_out32 (xpar_axi_vdma_0_baseaddr + 0x5C, 0x01000000);//Start address of the 1st frame (3 fram Es in all) xil_out32 (xpar_axi_vdma_0_baseaddr + 0x60, 0x02000000);//Start address of the 2nd frame (3 frames in all) xil_out (Xpar_axi_vdma_0_baseaddr + 0x64, 0x03000000);//Start address of the 3rd FRAME (3 frames in all) Xil_out32 (xpar_axi_vdma_0_baseaddr + 0x58, 0x0780);//Stride Numberxil_out32 (xpar_axi_vdma_0_ Baseaddr + 0x54, 0x0780);//number of bytes per line (640 x 3) Xil_out32 (xpar_axi_vdma_0_baseaddr + 0x50, 0x01e0);//number of lines per frame (480)//s2mmxil_out32 (xpar_axi_vdma_0_baseaddr + 0x30, 0x108b);//enable Run, Circular_parkxil_out32 ( Xpar_axi_vdma_0_baseaddr + 0xAC, 0x01000000);//Start address of the 1st frame (3 frames in all) Xil_out32 (xpar_axi_vdma_0_b Aseaddr + 0xb0, 0x02000000);//Start address of the 2nd frame (3 frames in all) Xil_out32 (xpar_axi_vdma_0_baseaddr + 0xb4, 0 x03000000);//Start address of the 3rd frame (3 frames in all) Xil_out32 (xpar_axi_vdma_0_baseaddr + 0xa8, 0x0780);//Stride Numberxil_out32 (xpar_axi_vdma_0_baseaddr + 0xa4, 0x0780);//number of bytes per line (640 x 3) Xil_out32 (xpar_axi_vdma_0_ Baseaddr + 0xA0, 0x01e0);//number of lines per frame (480) return 0;}

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

HLS Image Processing Series--build DDR image processing pathway in Zedboard