Implementation of image acquisition and Display Based on niosⅱ

Implementation of image acquisition and Display Based on niosⅱ

[Date:]

Source: Electronic Technology Application Author: Luo Jun, Wu ksong, Liao Honghua

[Font:Large Medium Small]

　

With the development of large-scale integrated circuit design technology, the improvement of manufacturing technology and the increase of the number of logical doors on a single chip, the design of embedded systems becomes increasingly complex. Integrating the entire System into a Chip, that is, the on-Chip System SoC (System on Chip) technology is a hot spot in the design of embedded systems. The FPGA-based SoC system designed on the FPGA platform provided by Altera has many advantages such as short development cycle, low cost, and reconfigurability.
1 System Design
There are two solutions to achieve high-speed image data acquisition and Display System Based on niosⅱ. One is to use software to control the PIO (Peripheral I/O) port on the niosⅱ processor to simulate the timing of the acquisition module, so as to flexibly store image data, and can be stored into the serial address of the SDRAM according to the image Macro Block (such as 8x8 pixel MCU block). The second solution is to add a FIFO buffer, and the image data is stored directly by the DMA controller, this scheme can save the software overhead of niosⅱ and complete image acquisition more efficiently.
The FPGA used in this system is the EP1C6Q240C8 of Altera. It has 5980 LC (logic gate unit) and 20 M4K RAM blocks. An 11-bit deep DMA controller designed in this system requires 280 Logic Cells and 151 LC Registers, which accounts for 7.2% of FPGA's overall system resources and consumes less resources. The system has designed two DMA controllers, one for the frontend acquisition module channel and the other for the backend display module channel.
Due to the many types of CMOS and CCD solid image sensors, some sensors (such as OV7620) output the YCrCb () format, however, the image Data produced by most CMOS and CCD image sensors is in Raw Data (Bayer) format. Therefore, we need to perform CFA Interpolation on the image data first. To save software overhead, this system uses the simplest CFA interpolation algorithm (Bilinear), compared with interpolation functions such as high-order B-spline and cubic-spline, the image quality does not differ much. However, Bilinear algorithms are much simpler and consume less software resources. After interpolation, the image needs to be further cropped into the resolution (QVGA) that can be displayed on the LCD screen. The RGB component is only 6 digits in height.
2 Hardware Design
2.1 hardware circuit diagram of the system
System Circuit Diagram 1.

 

As shown in figure 1, the Controller of the system's acquisition, storage, and display modules is completed on the FPGA EP1C6Q240C8.
2.2 Circuit Design in FPGA
Three controllers need to be designed in FPGA: CMOS acquisition controller, SDRAM controller, and tft LCD controller. Altera Corporation provides the SDRAM Controller. You must correctly configure the time parameters of the SDRAM controller based on the specific SDRAM device to operate normally and stably. The SDRAM model used in the system is HY57V561620T-H, and its configuration parameters are shown in table 1.

OV2610 and tft LCD controllers 2 and 3 are shown. The two controllers use hardware description language to write user logic, create the Alvalon Slave interface, and directly connect to the master port of the DMA controller. Control the data collection and display by using the Controller words written by nioⅱ CPU data and the program bus.

 

When the image data of OV2610 is displayed on a tft LCD screen, two DMA controllers need to be designed, one for image data collection and the other for image display. Because the interface of the OV2610 image data is in the form of data streams, the FIFO storage unit is used for caching. Compared with the dual-port RAM, the address generator is not required, which reduces the logic integrated wiring inside the FPGA. The specific design includes two parts: the first part uses the hardware description language to design the time series generator of the CMOS acquisition and LCD modules; the second part is in the niosⅱ IDE environment, write the DMA initialization and control program in C language.
Both the CMOS acquisition timing module and the tft LCD timing generation module must be designed in strict accordance with the time sequence of the CMOS image sensor and tft LCD. In the design of CMOS acquisition controller and LCD controller, the key is to design the logic of the sequence generation module in good time. At the same time, the data collected and displayed must be stored in the FIFO with limited resources. In EP1C6Q240C8, resources are limited because there are only 20 M4K storage units. In the CMOS acquisition controller, 11-bit 2 kb FIFO storage unit is occupied; QVGA resolution (320 × 240 ), the LCD module must design a 9-bit 512B FIFO storage unit for video memory to meet the requirements.
A hardware description language is used to design a controller for the acquisition timing module and the LCD timing module. The Controller waveform 4 for LCD timing generation simulated in Quartus Ⅱ Simulator is shown in.

 

The time sequence of CMOS Image Acquisition is similar to that of TFT liquid crystal.
On the systems platform, you need to design an interface between the CMOS sensor and aveon, store the data to the SDRAM, and mount the data to the aveon bus. Both the CMOS acquisition timing module and the tft LCD timing generation module are designed as aveon slave devices. The address, clk, reset, chipselect, read, readdata, begintransfer, and endofpacket pins are required.
Clk, reset, chipselect, address, write, writedata, begintransfer, and endofpacket must be used to design the interfaces between TFT_ LCD and aveon. In the SOPCbuilder4.2 environment, you can create user-defined components. Here, the HDL file is used to describe the behavior of the controller.
Data transmission is performed according to the bus interface time sequence of aveon bus in slave mode. From the Transmission sequence shown in figure 5, we can see that the data transmission in the stream mode is similar to that in the FIFO mode. Therefore, it can be seamlessly connected on the interfaces of the FIFO and the aveon bus. Here, the aveon bus reads the first-in-first-out memory (FIFO) into the master port of DMA or niosⅱ. The clock of the internal BUS is used, and the system uses 80 MHz. That is to say, the maximum bandwidth transmitted in stream mode can reach 160 Mbps, which can meet the requirements of 3 million pixels and 30 frames/s for Image Collection, which is larger than the bandwidth collected in PIO mode. The key of the image acquisition system is the storage of SDRAM. In the design of the system, if more Master ports are mounted to the SDRAM controller, the time consumed by the arbitration circuit of the SDRAM will increase, resulting in a decrease in the data storage speed of the SDRAM. When too many primary ports are connected to the SDRAM Controller, image data transmission becomes a bottleneck.

 

3 System Software Design
3.1 NiosII IDE software environment
To run the system hardware correctly, you also need to correctly configure the two-way DMA control register in the software environment of the niosⅱ IDE, and design the source address and destination address of the DMA.
Due to the hardware abstraction layer (HAL) system library driver provided by Altera for the niosⅱ processor users, users are allowed to use HAL application interface (API) functions to access peripherals, therefore, you can easily operate the underlying hardware.
In this system design, the HAL layer uses the drivers provided by niosⅱ IDE. The altera_avalon_dma_regs.h file defines the control registers of each DMA and the macro definitions of each bit control unit, provides a standard header file for programming. Although the altera_avalon_dma.c file also provides some standard DMA call functions, it does not significantly improve the system speed, calling the HAL macro can achieve DMA initialization faster and subsequent image data interpolation. The DMA controller registers are all 32-bit, with a total of 8 registers (including the three newly added registers compared with the Nios ), including status registers, source address registers, target address registers, length registers, and control registers.
3.2 software implementation process
The software process 6 that realizes CMOS Image Acquisition to liquid crystal display is shown.

The key is to initialize DMA throughout the system. The initialization process is 7.

 

The source code for DMA Initialization is as follows:
Np_dma * dma_cmos_con;
Dma_cmos_con = (np_dma *) DMA_0_BASE;
Np_dma * pdma = (np_dma *) dma_cmos_con;
Alt_irq_register (DMA_0_IRQ, (void *) pdma, (void *) isr_dma );
Pdma-> np_dmacontrol = 0;
Pdma-> np_dmastatus = 0;
Pdma-> np_dmalength = 1600*1200;
Pdma-> np_dmareadaddress = (int) Sdram_address;
Pdma-> np_dmawriteaddress = (int) na_cmos_cont;
Pdma-> np_dmacontrol =
Np_dmacontrol_go_mask |
Np_dmacontrol_ I _en_mask |
Np_dmacontrol_byte_mask |
Np_dmacontrol_reen_mask |
Np_dmacontrol_rcon_mask |
Np_dmacontrol_leen_mask;
In the function for registering the DMA device, niosⅱ calls the sys_dev_init function to initialize the dma hal layer and specifies the function entry address for the interrupt service program, other statements are used to correctly configure the DMA register.
3.3 Data Sorting
The original image data is in the Bayer mode, as shown in table 2. The CFA technology needs to be converted to the BT666 data format for LCD display of TFT. Because the resolution of tft LCD pixels is QVGA (320 × 240), and the image pixel collected from the image sensor is SVGA (1600 × 1200), you need to crop the original image. The image is displayed in a 16:1 Extraction mode. The processing of these data can be flexibly processed in the software environment of the niosⅱ IDE.

 

Figure 8 is the final debugging of a 2 million-pixel CMOS image. It simulates the controller of the CF Card by using the PIO kernel in the environment of niosⅱ and following the CompactFlash Card interface protocol, at the same time, in the environment of niosⅱ IDE, RAW image data is sorted through interpolation and restored, and written to CompactFlash Card according to the FAT16 file system.

 

From the images obtained from the acquisition experiment, we can see that the image quality is good and the image signal-to-noise ratio is high. Therefore, the image acquisition and display controllers developed on the system can enhance the flexibility and adaptability of the system. At the same time, some organizations provide public IP cores and apply them to self-development systems, which can shorten the design and development cycle and is also a low-cost approach. This new 'soft' hardware design concept will be widely used in intelligent measurement, automatic control, portable instrumentation and other fields.
References
[1] Xu ningyi, Zhou zucheng. aveon bus and an example of system architecture of the system [J]. semiconductor technology, 2003, (2 ).
[2] Party A, Chen zewen, Peng chenglian. User-Defined logic in the design of the system. Computer Engineering, 2004, (17 ).
[3] ALTERA Coperation. aveon video input module application note 372 altera (Version 1.0) [Z]. 12.
[4] ALTERA Copertion. Nio DMA, Data Sheet (Version 1.1) [Z]. Private 1.
[5] ALTERA Coperation. Nio software development tutoria (Version 1.2) [Z]. Computers 5.