CN1731861A - A kind of method that realizes the multiple image sensor information processing - Google Patents

Abstract

The invention relates to a method for realizing multi-image sensor information processing by utilizing the reconfigurable characteristic of field programmable gate array. Connect multiple image sensor output terminals to a field programmable gate array device; use the reconfigurable feature of the field programmable gate array device to partition and store the configuration files of each type of image sensor in a non-volatile external memory; According to the order from the outside, the single-chip microcomputer downloads the corresponding configuration file in the memory to the field programmable gate array device. The invention utilizes the on-line reconfigurable characteristic of the field programmable gate array, and uses a field programmable gate array device to realize an input method for multi-image sensor image information processing. It solves the problem that a field programmable gate array can only process one type of image sensor information, and to complete multi-image sensor information processing, it is necessary to use a multi-chip field programmable gate array. Programmable gate array completes multi-algorithm sharing.

Description

A method for realizing multi-image sensor information processing

Technical field:

The invention belongs to the application technical field of field programmable gate arrays, and relates to a method for realizing multi-image sensor information processing by using the reconfigurable characteristics of field programmable gate arrays.

Background technique:

The block diagram of the multi-image sensor input system is shown in Figure 1: it includes the first image sensor 1-1, the Nth image sensor 1-2, the first interface conversion device 1-3, and the Nth interface conversion device 1- 4. The first field programmable gate array 1-5, the Nth field programmable gate array 1-6, the image processing unit 1-7, the configuration timing generator 1-8, and the configuration data memory 1-9.

In the actual image processing system, the image processing system is often required to support the information processing of multiple image sensors, such as traditional analog cameras, digital cameras, and infrared cameras. The data formats of these image sensors are different, and the number of pixels is different. The frame frequency is different, so the interface between it and the following devices is different, and the algorithm for processing it is also different. Usually, it is connected to the field programmable gate array, such as the video signal from the analog camera through the analog/ The digital transformation is sent to the field programmable gate array, and the differential signal from the digital camera is sent to the field programmable gate array after being converted by a special interface chip. The field programmable gate array decodes according to different input signal formats, and converts After it is processed, it is sent to the image processing unit for processing.

Field Programmable Gate Array uses static random access memory (SRAM) technology inside, and its configuration data is stored in static random access memory. Since the data of SRAM is volatile, these data must be stored in non-volatile memory such as erasable read-only memory or flash memory other than field programmable gate array devices. It is downloaded to the static random access memory unit of the field programmable gate array, and the system can work normally only when the data configuration is correct.

After power-up, it automatically loads the configuration field programmable gate array, which is necessary for its application. Generally, it is configured by a dedicated configuration device or controlled by a single-chip microcomputer. When multiple FPGA devices require different configuration data, these devices can be connected in a daisy chain, and a combined bit stream can be used to program the devices individually. It can also be controlled by a single-chip microcomputer to load configurations for each device separately.

If a single-chip ultra-large-capacity field programmable gate array device is used to complete the complex image algorithm of multi-image sensor information, it is not conducive to the stability of the system. Difficulty of welding process. Usually, for complex image algorithms, a field programmable gate array is used to process a type of image sensor information. To complete the information processing of multiple image sensors, it is necessary to use multiple field programmable gate arrays, which will inevitably increase the system cost.

Invention content:

In order to solve the problem that a field programmable gate array in the background technology can only process one type of image sensor signals, and to complete multi-image sensor processing, a multi-chip field programmable gate array will be used. The purpose of the present invention is to provide a A method for realizing information processing of multiple image sensors by using a single-chip field programmable gate array device.

As shown in Figure 2: Since the configuration data of the field programmable gate array determines its internal interconnection and logic functions, changing these data also changes the logic functions of the device. The data can be reconfigured during the working process of the system, so as to realize in-circuit reconfiguration. In the method used in the present invention, multiple image sensor output terminals are connected to a field programmable gate array device through an interface conversion circuit; using the reconfigurable characteristics of the field programmable gate array device, the above-mentioned multiple image sensors Each type of image sensor adopts its own configuration file; the above-mentioned multiple configuration files are partitioned and stored in the non-volatile external memory; then the single-chip microcomputer selects and reads the non-volatile external The configuration file corresponding to each type of image sensor in the memory, and then download the configuration file of the corresponding image sensor type to the field programmable gate array device, then the present invention utilizes a single-chip field programmable gate array device to realize multiple Image sensor image information processing.

The advantages of the present invention are: because the present invention utilizes the online reconfigurable characteristic of the field programmable gate array, the input method for image information processing of multi-image sensors is realized by using a field programmable gate array device. It solves the problem that a field programmable gate array in the background technology can only process one type of image sensor information, and multi-chip field programmable gate arrays must be used to complete the information processing of multiple image sensors. The invention reduces the system cost; Single-chip field programmable gate array completes multi-algorithm sharing, that is, uses the same hardware structure to realize multiple system functions, making it multi-functional hardware. The invention can be applied in the technical field of image processing.

Description of drawings:

Fig. 1 is the functional block diagram of the information processing system of the background technology multi-image sensor

Fig. 2 is the functional block diagram of the information processing system of the multi-image sensor of the present invention

Fig. 3 is the schematic circuit diagram of an embodiment of the present invention

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS: The present invention will be further described below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to these embodiments.

The embodiment of the present invention is realized according to the block diagram shown in Figure 2, including the first image sensor 2-1, the Nth image sensor 2-2, the first interface conversion device 2-3, and the Nth interface conversion device Device 2-4, Field Programmable Gate Array 2-5, Image Processing Unit 2-6, Configuration Timing Generator 2-7, Configuration Data Storage 2-8.

Image sensor 2-1 and image sensor 2-2 can adopt infrared camera digital output port, infrared camera analog output port, digital camera video output port or analog camera video output port, etc.; interface conversion device 2-3, interface conversion Devices 2-4 can use level conversion devices, A/D conversion devices or digital camera dedicated interface chips, etc.; field programmable gate arrays 2-5 can use Xilinx’s Virtex series devices, and can also use Altera’s devices; image processing Unit 2-6 adopts TMS320C6000 series digital signal processor; configuration timing generator 2-7 uses programmable logic device; configuration data memory 2-8 adopts non-volatile flash memory or erasable read-only memory.

Fig. 3 is a specific embodiment of the present invention, and the first image sensor 2-1 adopts the infrared camera digital output port; The second image sensor 2-2 adopts the analog output port of the infrared camera; The third image sensor 2-9 Adopt color 3CCD (charge-coupled device) camera, the first interface conversion device 2-3 adopts DS9LV048 level conversion device; the second interface conversion device 2-4 adopts AD9200A/D conversion device; the third interface conversion device 2- 10 uses three pieces of AD9200A/D conversion devices; field programmable gate array 2-5 uses XCV300; image processing unit 2-6 uses TMS320C6203 digital signal processor; configuration timing generator 2-7 uses programmable logic device XC95108; configuration data Memory 2-8 adopts 29LV800B flash memory.

The connection relationship of the embodiment is as follows: the differential signal of the digital output port of the infrared camera is connected to the input/output pin IO[15:0] of the XCV300 after being converted by the DS9LV048. The analog signal of the analog output port of the infrared camera is connected to the XCV300 input/output pin IO[25:16] after AD9200 performs analog/digital conversion. The red, green, and blue analog signals output by the color 3CCD (charge-coupled device) camera are respectively connected to XCV300 input/output pins IO[84:58] after analog/digital conversion by their own AD9200. The data receiving end DR of the TMS320C6203 multi-function serial port receives the commands of the used image sensor type from the outside. Pin IO[53:46] and 8 input/output pins IO[7:0] of XC95108 are connected, address bus EA[21:2] of TMS320C6203, address bus A[18:0] of 29LV800, 20 of XCV300 The input/output pins IO[45:26] and the 20 input/output pins A[27:8] of XC95108 are connected, the output strobe AOE of TMS320C6203, the output strobe OE of 29LV800, the input/output pin of XCV300 Pin IO54 and XC95108 input/output pin I/O29 are connected, TMS320C6203 write effective AWE, 29LV800 write effective WE, XCV300 input/output pin IO55 and XC95108 input/output pin I/O30 are connected, TMS320C6203 Read valid ARE, XCV300 input/output pin IO56 and XC95108 input/output pin I/O31 are connected, TMS320C6203 external memory space 1 chip select signal CE1 is connected to XC95108 input/output pin I/O28, TMS320C6203 external The chip selection signal CE2 of memory space 2 is connected with the chip selection signal CE of 29LV800, and the chip selection signal CE3 of TMS320C6203 external memory space 3 is connected with the input/output pin I/O57 of XCV300. The five configuration pins CCLK, DONE, INIT, PROG and DIN of XCV300 are respectively connected to the input/output pins I/O32, I/O33, I/O34, I/O35 and I/O36 of XC95108, and the mode control of XCV300 Connect pins M0, M1, M2 to high level to set XCV300 to serial slave mode.

This embodiment only adopts the above-mentioned infrared camera digital output port information, analog output port information and color 3CCD (charge-coupled device) camera information as the input information of the three image sensors. The high-performance digital signal processing chip TMS320C6203 is very suitable for image processing. It has a powerful external memory interface and can directly interface with almost all types of memory (including flash memory). In this example, a piece of TMS320C6203 is used as an image processing unit to read The data in the field programmable gate array is taken for image processing, and it is also used as a configuration single-chip microcomputer to cooperate with the programmable logic device XC95144 to control the download process of the field programmable gate array configuration file. A piece of 29LV800 (flash memory) is used as the configuration memory of the field programmable gate array, and the field programmable gate array adopts XCV300 device of Xilinx Company. There are five download modes for configuration files: serial master mode (master serial), serial slave mode (slave serial), parallel master mode (master selectMAP), parallel slave mode (slave selectMAP), and JTAG mode. Set by the mode selection pins M0, M1, M2, this embodiment is set to the serial slave mode, the working process of the serial slave mode is as follows: when PROG generates a falling edge pulse, the configuration process is started, and at the rising edge of CCLK, the data is shifted into target chip. During the configuration process, the system needs real-time monitoring. Once an error occurs, INIT will be pulled low. After the system recognizes this signal, the configuration process will be restarted immediately. After the configuration data are all correctly moved into the target chip, the DONE signal jumps to high and enters the user working mode. Using TMS320C6203 to configure the on-site programming gate array, in essence, TMS320C6203 reads the configuration data in the flash memory, and then writes it into the programmable logic device XC95108, and XC95108 simulates the serial slave mode configuration sequence to complete the configuration Downloads for Field Programmable Gate Arrays.

In addition to the above-mentioned embodiments, the first image sensor 2-1, the Nth image sensor 2-2, the first interface conversion device 2-3, the Nth interface conversion device 2-4, the field programmable gate array 2-5. The image processing unit 2-6, the configuration timing generator 2-7 and the configuration data storage 2-8 may also use other components to implement other embodiments.

Claims (1)

1. A method for realizing multi-image sensor information processing, characterized in that: a plurality of image sensor output terminals are connected to a field programmable gate array device through an interface conversion circuit; Features, each type of image sensor in the above-mentioned multiple image sensors adopts its own configuration file; the above-mentioned multiple configuration files are stored in non-volatile external memory partitions; , to choose to read the configuration file corresponding to each type of image sensor in the non-volatile external memory, and then download the configuration file of the corresponding image sensor type to the field programmable gate array device, then use the single-chip field programmable gate The array device realizes the processing of image information of multiple image sensors.

Images