CN203813894U - Interface sharing panoramic digital image sensor - Google Patents

Abstract

The utility model provides an interface sharing panoramic digital image sensor, which is composed of M identical CMOS sensors, a CPLD/FPGA and a crystal clock source. The sensors are arranged in a regular M-polygonal shape, a system clock signal line, a configuration signal line and a sleep signal line are respectively connected and are driven by the CPLD/FPGA, a frame synchronization signal, a row enable signal and a pixel clock signal of any CMOS sensor are adopted to replace other CMOS sensors to be connected with the CPLD/FPGA, a data line of each CMOS sensor is respectively connected to the CPLD/FPGA, the CPLD/FPGA reads data of the sensors in parallel, and outputs a panoramic image data series comprising frame synchronization, pixel clock and associated clock signals via a non-standard image interface in a time division multiplexing manner. The interface sharing panoramic digital image sensor can solve the problem that the pins of multi-image sensors exceed the access capacity of an embedded type processor, and is especially suitable for panoramic image monitoring applications.

Description

The panoramic digital image transducer that a kind of interface is shared

Technical field:

What the present invention relates to is the shared panoramic digital image transducer of a kind of interface, particularly relates to a kind of panoramic digital image transducer that obtains the linear realtime graphic of 360 ° of scenes by M cmos sensor interface Share interlinkage.

Background technology:

For obtaining the realtime graphic of panorama, people have done a lot of research, and major way has special lens formula panoramic imagery (as used the banded imaging of the cylindrical plane projection as shown in the preposition panoramic imagery of non-half spherical reflector as shown in fish-eye hyper-hemispherical staring imaging, patent [1], patent [2]) and general lens type panoramic imagery (the multi-lens array panoramic imagery as shown in patent [3], single-lens scan-type panoramic imagery) two large classes at present.Wherein special lens formula panoramic imaging device needs special Optical devices, the required cost of the relative common lens of this device is higher, and imaging results mostly is nonlinear images, therefore need could obtain and approach linear image compared with complex calculations, real-time and image quality (geometric distortion) are had to certain influence, obtain high-definition picture also very difficult; The required cost of single-lens scan-type panoramic imagery is minimum, but its required mechanical structure has reduced the disguise of camera lens, and the scan period is grown and makes to follow the tracks of fast target difficulty; Multi-lens array panoramic imagery can obtain linear image and cost mainly on sensor chip, relatively low, but because transducer used is more, multiple transducers have just become the key technical problem of a needs solution with control chip interconnection.And this problem does not all provide solution in patent [1] [2] [3] [4].

Pin demand in the time that transducer increases between transducer and control chip (as ARM, FPGA, DSP) will increase, and in view of using USB as interconnection agreement in patent [4], needs the equipment such as usb communication chip, hub, and its cost is relatively high.Consider in the time that each image sensor signal is synchronous, the synchronizing signal of multiple imageing sensors (pixel clock signal, row enable signal, frame synchronizing signal) can be used completely from the signal of one of them transducer and replace; The data of multiple imageing sensors also can be controlled, in the mode of time division multiplexing high-speed transfer, 8 parallel-by-bit data nonstandardized technique signal forms be passed to follow-up picture processing chip with CPLD.Therefore use interface signal can construct a kind of panoramic digital image transducer with non-standard image interface by sharing.The art of this patent can design and reach the object that reduces cost for simplifying omnidirectional imaging system, and then promotes omnidirectional imaging system technological progress.

List of references

[1] Zhu Qidan, malleable iron just, Shen Jianyong, Wang Huiyong, Zhang Zhi, Han Rui, Cai Chengtao, Wang Lihui, Li Peng, horse great achievement. explosion-proof high-resolution full view visual monitoring system, patent of invention ZL200710072677,2008 07 month 16 (authorizing day).

[2] Zhou Xiangdong, Huang Zhi, Bai Jian. a kind of panoramic imaging device and method, patent of invention ZL201310257672, October 16 in 2013 (authorizing day).

[3] Zhuan Yueting, Yao Cheng, Wu Fei. Separated real-time panorama, patent of invention ZL200720112359, on September 10th, 2008 (authorizing day).

[4] Xu Tao, Yang Yang, Chen Li, Jin Weiwei, Cen Zhaofeng, Li Xiaotong. a kind of multi-camera real-time omnidirectional imaging system, patent of invention ZL201010164788, on September 8th, 2010 (authorizing day).

Summary of the invention

The object of the present invention is to provide a kind of panoramic digital image transducer that obtains the linear realtime graphic of 360 ° of scenes by M cmos sensor interface Share interlinkage.

The object of the present invention is achieved like this:

1. according to the angle of image FOV of imageing sensor, choose the cmos sensor of M same model by formula (1), make it to cover 360 ° of panoramic imagery scenes.

In formula, η gets 0.95, the coefficient of efficiency of imageing sensor angle of image after expression deduction adjacent image lap; Symbol " " represent the computing that rounds up.

2. the M of a selection cmos sensor is placed to form by positive M limit shape and pick up the camera lens of panorama image information, and make separated time (optical axis line) in its lens imaging angle (FOV) intersect at positive M Bian Xing center.As a reference (without loss of generality, only for ease of the laying method of transducer is described), while providing M=8, the installation site vertical view of cmos sensor as shown in Figure 1.

3. M cmos sensor pressed to interface sharing mode and be connected with piece of CPLD/FPGA, composition covers the panoramic digital image sensor circuit of 360 ° of imaging scenes.Interface Share interlinkage relation between cmos sensor and CPLD/FPGA is shown in Fig. 2, specifically can be described below:

(1) the system clock line (VXCLK) of M cmos sensor, SCCB configuration control line (SIO_C, SIO_D), dormancy line (PWDN) are linked together separately, be connected with the I/O lead-foot-line of CPLD/FPGA respectively again, drive signal to be provided by CPLD/FPGA;

(2) in M cmos sensor, appoint and get a cmos sensor, its frame synchronizing signal line (VSYNC), enforcement energy holding wire (HREF), pixel clock signal line (PCLK) are connected respectively on the I/O lead-foot-line of CPLD/FPGA, replace the signal of the same name on an other M-1 cmos sensor;

(3) by the data port line of each cmos sensor (Camera_data k[7..0], k=1,2 ..., M) be all connected on the I/O lead-foot-line of CPLD/FPGA, and guarantee that the PCB layout of bearing this connection task designs by isometric line;

(4) data of panoramic digital image transducer are via nonstandardized technique image interface (also referred to as the nonstandardized technique interface) output of CPLD/FPGA, and its pin function defines as shown in Figure 2.

(5) CPLD/FPGA internal circuit is responsible for the yuv format data that produce clock signal of system, synchronously read in M cmos sensor, generate nonstandardized technique interface auxiliary signal (channel associated clock signal Trans_clk, row enable signal Trans_href, frame synchronizing signal Trans_vsync, pixel clock signal Trans_pclk), by time division multiplexing mode, the data of the M synchronously a reading in cmos sensor are delivered to nonstandardized technique interface (eight parallel-by-bit image data interface Trans_data[7..0]) with higher transmission rate and export.CPLD/FPGA internal circuit function realizes by packing hardware description language program in machine code into, and corresponding circuit theory as shown in Figure 3.

In Fig. 3, VXCLK is the clock signal of system of giving cmos sensor, is obtained through frequency division by 200MHz crystal clock source; CTL[0..N] be the control signal of variable connector, wherein m is the number of cmos sensor, symbol " " the representative computing that rounds up; M the cmos sensor data that state machine, cycle counter, variable connector complete jointly to synchronously reading in are carried out time division multiplexing processing and are given the task that nonstandardized technique interface is exported; Shift counter is for generating channel associated clock signal Trans_clk, row enable signal Trans_href, frame synchronizing signal Trans_vsync and the pixel clock signal Trans_pclk of nonstandardized technique interface.

(6) when nonstandardized technique image interface is expert at enable signal effectively rising edge is appearred in (high level) and channel associated clock signal, log-on data (by time division multiplexing mode) transmission, can stop when invalidating signal (low level) until exercise.When work, signal sequence relation as shown in Figure 4, Figure 5.

In Fig. 4, C1, C2 ..., CM is the view data from the yuv format of M cmos sensor.Because yuv format view data is expressed by two bytes on each pixel, so through the view data Trans_data[7..0 of nonstandardized technique interface output] transfer rate equal the twice of pixel clock speed and number of sensors product, or equal with road clock rate.

4. the view data of nonstandardized technique interface output allows to receive, process, splice with the signal sequence that the intelligent devices such as FPGA, DSP, ARM provide by Fig. 4, Fig. 5 the panoramic picture that becomes 360 ° of scenes of covering.

Key feature of the present invention is: shared and the minimum interconnection line of multiple cmos sensors is connected to CPLD/FPGA realizes panoramic picture sensor design by interface; By the internal circuit function of hardware description language design CPLD/FPGA with the configuration register of each cmos sensor is configured; View data from each cmos sensor is arranged in to 8 parallel-by-bit data sequences with time division multiplexing mode, use with exercising energy, frame synchronization, pixel clock, outwards carrying high speed image data with 8 parallel-by-bit nonstandardized technique image interfaces of road clock, for building omnidirectional imaging system.

Essence of the present invention is: utilize interface technology of sharing to reduce to greatest extent the interconnection line between many cmos sensors and CPLD/FPGA; Simplify panoramic picture transducer hardware designs and reduce the cost of its Project Realization; Promote technological progress and the development of obtaining panoramic picture based on many cmos sensors partial image data by splicing.Panoramic digital image transducer of the present invention, also has that the image linearity is good, resolution is high, hardware external connection is few, is easy to process intelligent chip interfaces with image and output data are convenient to the feature of processing in real time.

Specific works process of the present invention and principle are:

1, utilize formula (1) to choose the cmos sensor of M same model;

2, M cmos sensor placed by the method shown in Fig. 1;

3, the clock signal of system line (VXCLK) of the cmos sensor of M same model, configuration signal line (SIO_C, SIO_D), sleep signal line (PWDN) are connected to together by the unified driving signal that provides of CPLD/FPGA;

4, utilize same model cmos sensor, under with homophase system clock (VXCLK) driving frequently, its row enable signal (HREF), frame synchronizing signal (VSYNC), pixel clock signal (PCLK), view data output (camera_data1[7..0], camera_dataM[7..0]) all can keep synchronous characteristic, appoint the row enable signal (HREF) of getting a cmos sensor, frame synchronizing signal (VSYNC), pixel clock signal (PCLK) is connected to the signal of the same name of upper other M-1 of replacement of CPLD/FPGA cmos sensor, meanwhile, the data port line of M cmos sensor is received respectively to CPLD/FPGA upper, line is by isometric line design, to guarantee that signal transmission delay is identical.

5, nonstandardized technique digital picture interface is made up of the 8 parallel-by-bit data-interfaces of CPLD/FPGA, 1 row enable signal, 1 frame synchronizing signal, 1 pixel clock signal, 1 channel associated clock signal (frequency equals pixel clock frequency and number of sensors product 2 times).

6, CPLD/FPGA controls the opening and closing to M cmos sensor IMAQ with PWDN.

7, distribute M time slot, allow 8 parallel-by-bit data-interface timesharing on CPLD/FPGA transmit the data of each cmos sensor, each time slot width is 1/ (2M) times of pixel clock signal cycle, the view data of a byte yuv format of each slot transmission.

8, each signal sequence of nonstandardized technique image interface output meets the sequential relationship that Fig. 4 and Fig. 5 provide.

Beneficial effect of the present invention is:

1. a kind of panoramic digital image transducer based on local linear Image Mosaics principle and interface technology of sharing is provided, solve multiple image sensor and process when intelligent chip is connected pin number is required to too high problem with image, can promote the technological progress of panoramic video monitoring system image capture device.

2. can be used for developing periscope, crossroad overall view monitoring equipment; With realize the imaging system of wide-angle image monitoring by mechanical underprop compared with, have noiselessness, hidden and can high speed tracked mobile target etc. advantage.

Brief description of the drawings

Fig. 1 is that 8 cmos sensors lay position vertical view;

Fig. 2 is interconnected relationship and the nonstandardized technique image interface of cmos sensor and CPLD/FPGA;

Fig. 3 is CPLD/FPGA internal circuit principle;

Signal sequence graph of a relation when Fig. 4 criteria of right and wrong image interface output a line view data;

The timing diagram of Fig. 5 criteria of right and wrong image interface view data, enable signal, synchronizing signal;

Fig. 6 is the electrical schematic diagram of one embodiment of the present of invention.

The specific embodiment of the present invention is described in detail in aforesaid summary of the invention.Be specifically described below in conjunction with embodiment.

The function of the present embodiment is mainly made up of jointly 8 cmos sensor U1-U8 (OV9650), programmable digital logic chip U9 (EPM1270GT144C3), clock source U10.

U1-U8 is responsible for gathering image sensing data, and form is YUV signal/or the rgb signal of 8 parallel-by-bits, 2 byte representations for each pixel; The data port pin of enforcement energy, frame synchronization, pixel clock and the U1-U8 of U1 is connected with the I/O line of U9 respectively; After 8 SCCB control pins of U1-U8,8 PWDN pin, 8 system clock pin are connected in parallel separately, then be connected with the I/O line of U9; U9 is CPLD/FPGA chip, and its function realizes by programming by VHDL language, and program is downloaded the JTAG mode that uses; U10 is responsible for U9 system clock is provided.U9 processes intelligent chip (as DSP, FPGA, ARM) by nonstandardized technique image interface and external image and is connected.

The design parameter that the present embodiment adopts is:

1.8 OV9650 cmos sensors are placed (seeing Fig. 1) by octagon mode, optical axis meets at octagon center, the visual angle of each OV9650 is 62 °, and wherein 8 OV9650CMOS transducers carry out pin with EPM1270GT144C3 by interface sharing mode and are connected.

The frequency of the input clock GCLK1 of 2.U9 is 200MHz, is provided by U10; This clock is direct as using with road clock after buffering in U9; Pixel clock frequency is 12.5MHz.

3. the image resolution ratio of single-sensor is configured to SXGA (1280 × 1024), and frame per second is 15fps.Can realize maximum panoramic picture resolution and be about 9728 × 1024.

4. the output of the data of panoramic digital image transducer adopts nonstandardized technique data-interface, comprises 8 parallel-by-bit data-signals, 1 row enable signal, 1 frame synchronizing signal, 1 pixel clock signal, 1 channel associated clock signal.

5.U9 adopts the EPM1270GT144C3 chip of altera corp, supply power voltage 3.3V.

More than be described as a kind of embodiment of the present invention, can carry out respective change according to technical scheme of the present invention.

Claims (4)

1. the shared panoramic digital image transducer of interface, is characterized in that: be made up of M identical cmos sensor, a slice programmable digital logic chip (CPLD/FPGA), a clock source chip.The data port of each cmos sensor is directly connected with the I/O line of CPLD/FPGA, SIO_C is connected with the I/O line of CPLD/FPGA after being connected in parallel separately with SIO_D line, PWDN line, system clock line again, appoint enforcement energy line, frame synchronization line, the pixel clock line of getting a cmos sensor to be connected with the I/O line of CPLD/FPGA, the output pin of crystal clock source chip is connected with the global clock line of CPLD/FPGA, the internal circuit of CPLD/FPGA by the function shown in Fig. 3 in patent specification of the present invention by the VHDL language realization of programming.

2. the shared panoramic digital image transducer of a kind of interface according to claim 1, is characterized in that specifically comprising the steps:

(1) according to the angle of image FOV of imageing sensor, choose the cmos sensor of M same model by formula (1), make it to cover 360 ° of panoramic imagery scenes.

In formula, 77 get 0.95, the coefficient of efficiency of imageing sensor angle of image after expression deduction adjacent image lap; Symbol represent to round up computing.

(2) M the cmos sensor of selecting placed to formation by positive M limit shape and pick up the camera lens of panorama image information, and make separated time (optical axis line) in its lens imaging angle (FOV) intersect at positive M Bian Xing center.

(3) M cmos sensor linking together by interface sharing mode and piece of CPLD/FPGA, that is: 1) the clock signal of system line (VXCLK) of M cmos sensor, configuration control signal line (SIO_C, SIO_D), sleep signal line (PWDN) are linked together separately, be connected with the I/O lead-foot-line of CPLD/FPGA respectively again, drive signal to be provided by CPLD/FPGA; 2) in M cmos sensor, appoint and get a cmos sensor, its frame synchronizing signal line (VSYNC), enforcement energy holding wire (HREF), pixel clock signal line (PCLK) are connected respectively on the I/O lead-foot-line of CPLD/FPGA, replace the signal of the same name on an other M-1 cmos sensor; 3) the data port line of each cmos sensor (8) is connected on the I/O lead-foot-line of CPLD/FPGA, and guarantees that the PCB layout of bearing this connection task designs by isometric line; 4) the nonstandardized technique image interface of CPLD/FPGA is by 8 parallel-by-bit data-signal Trans_data[7..0], 1 row enable signal Trans_href, 1 frame synchronizing signal Trans_vsync, 1 pixel clock signal Trans_pclk, 1 channel associated clock signal Trans_clk (frequency equals pixel clock frequency and number of sensors product 2 times) form; 5) CPLD/FPGA controls the opening and closing to M cmos sensor IMAQ with PWDN; 6) distribute M time slot, allow 8 parallel-by-bit data-interface timesharing on CPLD/FPGA transmit the data of each cmos sensor, each time slot width is 1/ (2M) times of pixel clock signal cycle, the view data of a byte yuv format of each slot transmission.

3. the shared panoramic digital image transducer of a kind of interface according to claim 1, is characterized in that with CPLD/FPGA, the internal register of each cmos sensor being configured, and makes it meet the job requirement of output yuv format view data; CPLD/FPGA also produces and drives the system clock of each cmos sensor work and the signal of nonstandardized technique image interface work simultaneously.

4. the shared panoramic digital image transducer of a kind of interface according to claim 1, when it is characterized in that being expert at enable signal by nonstandardized technique image interface effectively rising edge appearring in (high level) and channel associated clock signal, log-on data (by time division multiplexing mode) transmission, can invalidating signal stop transmission when (low level) until exercise, and each signal sequence of image interface output meets the sequential relationship shown in Fig. 4 and Fig. 5 in patent specification of the present invention.