JP2007079708A - Image processor and processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and an image processing method for highly precisely correcting the distortion of an image, and for quickly performing image processing. <P>SOLUTION: An image processor 1 includes a distortion correcting part 2 for correcting the distortion of image data acquired by an optical lens. The distortion correcting part 2 includes an interpolation relation generating part 12 for performing an interpolation arithmetic operation by using interpolation relations corresponding to density data stored in an input line buffer 8. The density data generated by the interpolation arithmetic operation are obtained as density data at coordinates after correction. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly, to an image processing apparatus and an image processing method for correcting image distortion.

  In general, since a camera such as a digital camera forms an image of an object using an optical lens and records the image, it is known that the obtained image has aberration. This aberration includes spherical aberration and coma aberration, which are aberrations related to image sharpness, and distortion aberration, which is an aberration that distorts the obtained image. There are a pincushion type and a barrel type in distortion, and both are caused because the magnification of the optical lens is not constant depending on the image height.

  As an image processing technique for correcting such distortion, image data is divided into a plurality of blocks, data is acquired for each block and stored in a frame buffer, thereby reducing bus collisions and bus transfer. A technique for improving efficiency is disclosed (for example, see Patent Document 1).

  In addition, a technique for reducing the image processing time by dividing the image data stored in the frame buffer into a plurality of blocks and correcting the image data in the same block using the same correction amount is disclosed. (For example, refer to Patent Document 2).

  Also, a technique for correcting image distortion by reading out image data stored in a random access memory such as an SDRAM in an order based on the coordinates of the original image data is disclosed (although different from the stored order) ( For example, see Patent Document 3).

Furthermore, a technique for reducing the required frame buffer capacity and performing high-speed image processing by correcting one of the one-dimensional image data for two-dimensional image data has been disclosed. (For example, see Patent Document 4).
JP-A-2005-45514 JP 11-25238 A JP-A-10-224695 JP-A-7-37079

  However, as in Japanese Patent Application Laid-Open No. H10-26097, the image processing unit that acquires image data and stores it in the frame buffer and then performs image processing needs to store one image in the frame buffer. And the problem that the image data storage time cannot be reduced from the image processing time.

  Further, in the means for storing image data in a frame buffer and performing image processing by dividing it into a plurality of blocks as in Patent Document 2, there is a problem that it takes time to start image processing, and the capacity of the frame buffer. And the storage time of image data is required.

  Further, in the means for reading out image data in the order based on the coordinates of the corrected image data from the random access memory as in Patent Document 3, since the data is read out discontinuously from the memory, the efficiency of image processing by hardware is reduced. Therefore, there is a problem that the image processing time becomes long.

  Further, as disclosed in Patent Document 4, the means for correcting one-dimensional image data in one of two-dimensional image data does not correct the other one-dimensional image data. Therefore, there is a problem that high-precision images cannot be obtained.

  The present invention has been made in view of the above points, and has an object to provide an image processing apparatus and an image processing method capable of correcting image distortion with high accuracy and performing image processing at high speed. To do.

In order to solve such a problem, the invention described in claim 1
In an image processing apparatus provided with a buffer that stores image data captured through a lens, and a distortion correction unit that generates distortion correction image data in which distortion due to the lens is corrected from the image data.
The distortion correction unit is an image processing apparatus that generates the distortion-corrected image data when the image data necessary for generating the distortion-corrected image data is stored in a buffer.

The invention according to claim 2 is the image processing apparatus according to claim 1,
The distortion correction unit includes a distortion correction table storing a correspondence relationship between the distortion correction data and image data necessary for generating the distortion correction data, and the image data required for generating the distortion correction image data. Is stored in the buffer, the image data is generated using the correspondence.

  According to a third aspect of the present invention, in the image processing apparatus according to the second aspect, the distortion correction table includes the pixels constituting the distortion correction data and the pixels necessary for generating the pixels constituting the distortion correction data. A correspondence relationship with image data is stored.

  According to a fourth aspect of the present invention, in the image processing apparatus according to the third aspect, the distortion correction unit stores the image data necessary for generating pixels constituting the distortion correction data when the image data is stored. A plurality of pixels constituting the distortion correction data that can be generated from the image data thus generated are generated.

  According to a fifth aspect of the present invention, in the image processing apparatus according to the fourth aspect, the distortion correction unit includes a plurality of arithmetic units that generate pixels constituting the distortion correction data.

  According to a sixth aspect of the present invention, in the image processing apparatus according to any one of the first to fifth aspects, the buffer is an input line buffer that stores the image data in units of lines. And

  A seventh aspect of the present invention is the image processing apparatus according to any one of the first to sixth aspects, further comprising an output line buffer for storing the distortion-corrected image data in line units. To do.

The invention according to claim 8 provides:
In an image processing method for generating distortion-corrected image data obtained by performing image processing on image data captured through a lens and correcting distortion caused by the lens,
Storing the image data in a buffer;
Determining whether image data necessary for generating the distortion-corrected image data is stored in the buffer;
When it is determined that image data necessary for generating the distortion-corrected image data is stored, the distortion-corrected image data is generated from the stored image data.

  According to a ninth aspect of the present invention, in the image processing method according to the eighth aspect, it is determined whether or not the image data necessary for generating the pixels constituting the distortion correction data is stored in the buffer. Features.

  According to the first aspect of the present invention, when image data that can generate distortion-corrected image data is stored, distortion-corrected image data can be generated. There is an effect that it is possible to shorten the time until the process ends.

  According to the second aspect of the invention, since the correspondence between the image data and the distortion correction data is stored in the distortion correction table in advance, the calculation time of the correspondence between the image data and the distortion correction data is reduced. Thus, distortion correction data can be generated, and the image processing time can be shortened.

  According to the third aspect of the present invention, since the distortion correction data is associated with the image data on a pixel basis, the distortion correction data can be generated on a pixel basis, and image processing is performed after the image is acquired. There is an effect that it is possible to shorten the time until the start.

  According to the fourth aspect of the invention, a plurality of pixels of the distortion correction image data can be generated using the image data for generating one pixel of the distortion correction data, thereby reducing the image processing time. There is an effect that can.

  According to the fifth aspect of the present invention, since a plurality of calculation units that generate distortion correction data in units of pixels are provided, distortion correction data for a plurality of pixels can be generated at the same time, and image processing time can be reduced. The effect which can be done is produced.

  According to the invention described in claim 6, since the image data can be stored in units of lines, the time for storing the image data is reduced, and the time from the acquisition of the image data to the start of the image processing is reduced. There is an effect that can be shortened.

  According to the seventh aspect of the present invention, by storing the distortion-corrected image data in the output line buffer, the order of the pixels in which the image data is calculated and the order of the pixels output from the distortion correction unit can be changed. Therefore, there is an effect that the distortion correction image data can be output from the distortion correction unit in an appropriate order.

  According to the eighth aspect of the invention, when image data that can generate distortion-corrected image data is stored, distortion-corrected image data can be generated. There is an effect that it is possible to shorten the time until the process ends.

  According to the ninth aspect of the invention, a plurality of pixels of the distortion correction image data can be generated by using the image data for generating one pixel of the distortion correction data, thereby reducing the image processing time. There is an effect that can.

  Embodiments of an image processing apparatus according to the present invention will be described below with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

  As shown in FIG. 1, the image processing apparatus 1 performs a distortion correction unit 2 that corrects distortion of image data acquired by an optical lens, and performs other image processing such as color correction on the distortion-corrected image data. A post-stage image processing unit 3 is provided.

The distortion correction unit 2 is connected to an imaging device 4 that captures an image and generates image data. The imaging device 4 is provided with an imaging lens 5 made of glass, plastic, or the like, and an image is captured using the imaging lens 5. On the optical path of the imaging lens 5, a plurality of imaging elements 6 as light receiving elements are arranged corresponding to the pixels, and image data is generated according to the amount of light received by each imaging element 6 and output to the distortion correction unit 2. It is supposed to be. This image data is generated in units of pixels, and is composed of coordinate data indicating pixel coordinates and density data indicating pixel density.
As the imaging element 6, a CMOS (Complementary Metal Oxide Semiconductor), a CCD (Charge Coupled Device), a MOS (Metal Oxide Semiconductor), or the like can be used.

  The distortion correction unit 2 is provided with a control unit 7 that controls the entire distortion correction unit 2, and is connected to each unit constituting the distortion correction unit 2.

  Further, the distortion correction unit 2 is provided with an input line buffer 8 for temporarily storing density data among the image data output from the imaging device 4. The input line buffer 8 can store at least density values for the number of lines necessary to generate an interpolation relationship described later. The number of lines varies depending on the interpolation method of density data to be used. In the present embodiment, a linear interpolation method is used in which linear interpolation is performed using density data of four pixels existing around a pixel on which density data interpolation is performed. Therefore, the input line buffer 8 can store at least the density data of pixels for two lines.

  In addition, the distortion correction unit 2 is provided with an input counter 9 for counting when storing image data in the input line buffer 8, corresponding to the input line buffer 8. The input counter 9 includes an input x counter 10 that counts the x position of the input line buffer 8 and an input y counter 11 that counts the y position of the input line buffer. By using these counters, individual density data can be stored at an appropriate position in the input line buffer 8.

  Connected to the input line buffer 8 is an interpolation relationship generation unit 12 that performs an interpolation operation by using an interpolation relationship corresponding to density data stored in the input line buffer 8. The density data generated by this interpolation calculation is the density data at the corrected coordinates and becomes distortion corrected image data.

Connected to the interpolating relation generation unit 12 is a distortion correction table 13 in which interpolating relations used when performing interpolation calculations are stored in advance. As shown in FIG. 2, this interpolation relationship is the relationship between the density data of the pixel to be interpolated and the density data of the four pixels existing around the pixel. In this embodiment, the following equation (1) is obtained. Use to calculate.
a (x, y) = p1 (n + 1, m + 1) (1-α) (1-β) + p2 (n + 2, m + 1) α (1-β) + p3 (n + 1, m + 2) (1-α) β + p4 (n + 2, m + 2) αβ (1)
here,
a (x, y) is the density data at coordinates (x, y) p1 (n + 1, m + 1) is the density data at coordinates (n + 1, m + 1) p2 (n + 2, m + 1) is the density at coordinates (n + 2, m + 1) Data p3 (n + 1, m + 2) is density data at coordinates (n + 1, m + 2) p4 (n + 2, m + 2) is density data at coordinates (n + 2, m + 2) α is the difference between x and (n + 1), x Directional weighting β is the difference between y and (m + 1), and is the weighting in the y direction.
In the distortion correction table 13, the arithmetic expression used for the pixel to be interpolated and the coordinates and weighting of the density data to be used are calculated in advance and stored in association with each other. In addition, the distortion correction table 13 stores coordinates in which coordinates to be interpolated (hereinafter referred to as coordinates before distortion correction) and coordinates after distortion correction are associated with each other.

The interpolation relation generation unit 12 is provided with an interpolation calculation unit 14 that performs interpolation calculation using density data stored in the input line buffer 8 and calculation formulas and weights stored in the distortion correction table 13. The interpolation calculation unit 14 is provided corresponding to the maximum number of pixels before distortion correction that can be calculated from the density data of the surrounding four pixels. In the present embodiment, three interpolation calculation units 14 are provided. The number of the interpolation calculation units 14 differs depending on distortion caused by the lens or the imaging system and the linear interpolation method used for interpolation, and is calculated in advance based on coordinates before correction, coordinates after correction, and coordinates of pixels to be acquired. It is.
In the present embodiment, as shown in FIG. 3, pixels originally positioned at a1 (n, m), b1 (n + 1, m), and c1 (n, m + 1) (hereinafter referred to as corrected pixels) And image data obtained at the positions a2, b2, and c2 (hereinafter referred to as uncorrected pixels) due to the distortion of the imaging system, and the image data acquired by the imaging device 4 is Suppose that the image data are located at the intersections of the grids, for example, p1 (n + 1, m + 1), p2 (n + 2, m + 1), p3 (n + 1, m + 2), and p4 (n + 2, m + 2). Thereby, the pixels before distortion correction that can be calculated by the surrounding four pixels p1, p2, p3, and p4 are three pixels a2, b2, and c2. Note that p1, p2, p3, and p4 are values indicating the respective density data.

  The interpolation relation generation unit 12 is provided with an interpolation counter 15 that manages the coordinates before the distortion correction of the density data calculated by the interpolation calculation unit corresponding to the interpolation relation generation unit 12. The interpolation counter 15 includes an interpolation x counter 16 that manages the x coordinate of the density data calculated by the interpolation calculation unit, and an interpolation y counter 17 that manages the y coordinate of the pixel that is also subjected to the interpolation calculation. These counters associate the coordinates before distortion correction with the coordinates after distortion correction.

  An output line buffer 18 for temporarily storing density data after the interpolation calculation is connected to the interpolation calculation unit 14. The output line buffer 18 can store density data having the same number of lines as the input line buffer 8 at a minimum.

  The output line buffer 18 is provided with an output counter 19 for counting when the distortion correction data is stored in the output line buffer 18 corresponding to the output line buffer 18. The output side counter includes an output x counter 20 that counts the x position of the output line buffer 18 and an output y counter 21 that counts the y position of the output line buffer 18. By using these counters, individual distortion correction density data can be stored at an appropriate position in the output line buffer 18, and the distortion correction density data is efficiently output from the distortion correction unit 2. It can be stored as possible.

  Next, the operation of the image processing apparatus according to this embodiment will be described.

  When an image is picked up using the image pickup lens 5 of the image pickup device 4, the image pickup element 6 on the optical path of the image pickup lens 5 converts the image data into image data according to the amount of received light, so that image data is generated.

  When the image data is generated by the imaging device 4, the control unit 7 of the distortion correction unit 2 temporarily stores density data constituting the image data in the input line buffer 8. Then, the control unit 7 calculates pixels before distortion correction that can be interpolated from density data stored in the input line buffer 8. In addition, the control unit 7 causes the interpolation relation generation unit 12 to output the interpolation calculation formula and weighting used for the calculated pixel from the distortion correction table 13. At the same time, the control unit 7 causes the interpolation counter 15 to manage the coordinates before correction of the density data calculated by the interpolation relation generation unit.

Specifically, when density data p1, p2, p3, and p4 are stored in the input line buffer 8, the control unit 7 determines that the pixels before distortion correction that can be interpolated using these density data are a2, b2, c2 is calculated, and the interpolation calculation formula and weighting of each pixel are output from the distortion correction table 13 to the interpolation calculation unit 14 to perform the interpolation calculation to generate distortion correction density data. For example, the interpolation calculation of a2 is performed using the following calculation formula (2), and the distortion correction density data of the pixel after distortion correction is calculated.
a2 = (1-α1) (1-β1) p1 + α1 (1-β1) p2 + (1-α1) β1p3 + α1β1p4 (2)
here,
a2 is the density data p1 of the coordinate a2, p1 is the density data of the coordinate p1, p2 is the density data of the coordinate p2, p3 is the density data of the coordinate p3, p4 is the density data of the coordinate p4 α1 and β2 are weights of a2. .

  Similarly, each interpolation calculation unit 14 performs the interpolation calculation of b2 and c2, and calculates the distortion correction density data of the pixel after the distortion correction of b2 and c2.

  The control unit 7 temporarily stores the density data of the pixel after distortion correction calculated by the interpolation calculation in the output line buffer 18. Then, the control unit 7 outputs the density data stored in the output line buffer 18 to the subsequent image processing unit 3, and further performs image processing such as color correction.

As described above, according to this embodiment, the density data is stored in the input line buffer 8 without using the frame buffer. It is possible to shorten the time from the input of density data to the distortion correction unit 2 to the time of interpolation calculation, and the time from the image capture to the end of image processing.
Further, a pixel that can be interpolated is calculated from the density data stored in the input line buffer 8, and the distortion correction density data after correction is calculated immediately from the density data before correction by the interpolation calculation unit 14. Thus, the time for calculating the distortion correction density data can be shortened, and the time required for image processing can be further shortened.

  In the present embodiment, the example in which the interpolation calculation unit 14 is provided corresponding to the maximum number of pixels before distortion correction that can be calculated at one time is shown, but the present invention is not limited to this, and one interpolation calculation is performed. The configuration may be such that the interpolation calculation is sequentially performed in the unit.

  In this embodiment, the output line buffer 18 is provided. However, the present invention is not limited to this. The output line buffer 18 is not provided, and the interpolation calculation unit is directly connected to the subsequent image processing unit 3. It may be a simple configuration.

It is a figure which shows schematic structure of the image processing apparatus in this embodiment. It is a figure which shows the relationship between the pixel which performs the interpolation calculation in this embodiment, and a surrounding pixel. It is a figure which shows the correspondence of the pixel before distortion correction in this embodiment, and the pixel after distortion correction.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Distortion correction | amendment part 3 Back | latter stage image processing part 4 Imaging device 5 Imaging lens 7 Control part 8 Input line buffer 9 Input counter 12 Interpolation relation production | generation part 13 Distortion correction table 14 Interpolation calculating part 15 Interpolation counter 18 Output line buffer 19 Output counter

Claims (9)

In an image processing apparatus provided with a buffer that stores image data captured through a lens, and a distortion correction unit that generates distortion correction image data in which distortion due to the lens is corrected from the image data.
The image processing apparatus, wherein the distortion correction unit generates the distortion-corrected image data that can be generated from the stored image data.   The distortion correction unit includes a distortion correction table that stores a correspondence relationship between the image data and the distortion corrected image data that can be generated from the image data, and the correspondence relationship when the image data is stored in a buffer. The image processing apparatus according to claim 1, wherein the distortion-corrected image data is generated using an image.   The distortion correction table stores a correspondence relationship between pixels constituting the image data and pixels constituting the distortion correction image data that can be generated from the pixels constituting the image data. Item 3. The image processing apparatus according to Item 2.   The image processing apparatus according to claim 3, wherein the distortion correction unit generates a plurality of the distortion correction image data from the image data stored in the buffer.   The image processing apparatus according to claim 4, wherein the distortion correction unit includes a plurality of calculation units that generate pixels constituting the distortion correction image data.   6. The image processing apparatus according to claim 1, wherein the buffer is an input line buffer that stores the image data in units of lines.   The image processing apparatus according to claim 1, further comprising an output line buffer that stores the distortion-corrected image data in units of lines. In an image processing method for generating distortion-corrected image data obtained by performing image processing on image data captured through a lens and correcting distortion caused by the lens,
Storing the image data in a buffer;
Determining whether the distortion-corrected image data that can be generated from the stored image data exists,
An image processing method comprising: generating the distortion-corrected image data from the stored image data when it is determined that the generatable distortion-corrected image data exists.   The image processing method according to claim 8, wherein whether or not the distortion correction data that can be generated exists is determined in units of pixels.

Images