[go: up one dir, main page]

EP2101295B1 - Procédé de traitement d'images, dispositif de traitement d'images et dispositif de capture d'images - Google Patents

Procédé de traitement d'images, dispositif de traitement d'images et dispositif de capture d'images Download PDF

Info

Publication number
EP2101295B1
EP2101295B1 EP09154620A EP09154620A EP2101295B1 EP 2101295 B1 EP2101295 B1 EP 2101295B1 EP 09154620 A EP09154620 A EP 09154620A EP 09154620 A EP09154620 A EP 09154620A EP 2101295 B1 EP2101295 B1 EP 2101295B1
Authority
EP
European Patent Office
Prior art keywords
chromatic aberration
magnification chromatic
coordinate transformation
image
correction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP09154620A
Other languages
German (de)
English (en)
Other versions
EP2101295A1 (fr
Inventor
Ryosuke Kasahara
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ricoh Co Ltd
Original Assignee
Ricoh Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ricoh Co Ltd filed Critical Ricoh Co Ltd
Publication of EP2101295A1 publication Critical patent/EP2101295A1/fr
Application granted granted Critical
Publication of EP2101295B1 publication Critical patent/EP2101295B1/fr
Ceased legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
    • H04N25/611—Correction of chromatic aberration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
    • H04N25/615—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
    • H04N25/6153—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF] for colour signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements

Definitions

  • the present invention relates to an image processing method, an image processing device, and an image capturing device.
  • a memory with a large capacity and a small latency at the time of random access such as an SRAM, or a multi-port memory is needed for each RGB color component.
  • SRAM static random access memory
  • a large capacity SRAM or a memory having plural ports is very expensive, and accordingly, the price of the device has been very high, particularly, in the case of a high resolution which requires a large capacity.
  • the method for applying a common coordinate transformation to each RGB color component to correct only the distortion requires a large capacity memory and it may be possible to use a DRAM or the like in which the price of one chip or component is low but is not available if the magnification chromatic aberration is hardly ignored at a wide view angle.
  • the magnification chromatic aberration is different among the color components. Furthermore, the magnification chromatic aberration is smaller than the distortion. Therefore, it is preferable to correct the magnification chromatic aberration and the distortion separately.
  • an independent correction circuit for each color component is commonly needed for correction of a magnification chromatic aberration and it may be necessary to correct the magnification chromatic aberration at the minimum circuit scale for cost reduction.
  • EP 1 473 671 A describes an image processing device for correcting aberrations caused by a capturing lens for an image signal to which light passing through the capturing lens is electrically converted.
  • the image processing device includes a correction vector calculation unit for calculating a correction vector in accordance with coordinates of each pixel of each color component contained in the image signal on the basis of at least data on chromatic aberration and distortion aberration caused by the capturing lens; a conversion ratio calculation unit for receiving at least the image signal, capturing conditions, and the correction vector, and for calculating a conversion ratio for scaling the color component in accordance with the coordinates of the pixel; and a resolution conversion unit for scaling the color component in accordance with the coordinates of the pixel on the basis of the conversion ratio.
  • the invention is defined by the subject-matter of the independent claims.
  • the dependent claims are directed to advantageous embodiments.
  • an image processing method configured to process an image captured by using an optical system with a wide view angle and a large magnification chromatic aberration, wherein a magnification chromatic aberration correction is conducted by conducting no coordinate transformation for an image of a particular color component while conducting a coordinate transformation for only an image of a color component except the particular color component.
  • an image processing device configured to process an image captured by using an optical system with a wide view angle and a magnification chromatic aberration, including a magnification chromatic aberration correction device configured to conduct a magnification chromatic aberration correction by conducting no coordinate transformation for an image of a particular color component while conducting a coordinate transformation for only an image of a color component except the particular color component.
  • an image capturing device including an optical system with a wide view angle, and at least, a large magnification chromatic aberration, an image capturing element configured to read an image captured via the optical system, and the image processing device as described above.
  • An embodiment of the present invention relates to an image processing method, image processing device, and image capturing device for processing an image captured by using an optical system with a wide view angle and a large magnification chromatic aberration, and in particular, relates to a technique for correcting a magnification chromatic aberration of a captured image.
  • the inventor conceived of the idea of providing an image processing method and image processing device capable of correcting a magnification chromatic aberration at a minimum circuit scale without degrading a correction performance.
  • the inventor also conceived of the idea of installing an image processing device for conducting a magnification chromatic aberration correction on a small circuit scale to provide a low-cost image capturing device.
  • an image is captured by using an optical system with a wide view angle and a large magnification chromatic aberration, and a magnification chromatic aberration correction is conducted by conducting no coordinate transformation for an image of a particular color component and conducting a coordinate transformation for only an image of a color component except the particular color component.
  • the particular color component may be green (G) and color components except that may be red (R) and blue (B).
  • G green
  • R red
  • B blue
  • a correction quantity may be obtained based on a coordinate of an image of the particular color component and a coordinate transformation may be conducted for an image of a color component except the particular color component based on the correction quantity.
  • a coordinate transformation may be conducted for an image of a color component except the particular color component based on the correction quantity.
  • an image of a color component except the particular color component may be adjusted by further multiplying a predetermined gain by a coordinate value. Thereby, it may be possible to adjust an asymmetric shift of a magnification chromatic aberration.
  • a magnification chromatic aberration correction and a distortion correction may be conducted simultaneously by further obtaining a correction quantity of a distortion commonly for each color component and each adding the correction quantity of a distortion to a coordinate value to an image of each color component, and a distortion correction may be separately conducted commonly for an image of each color component after conducting a magnification chromatic aberration correction.
  • the embodiment is an image capturing device for capturing an image of an object using an optical system with a wide view angle and a large magnification chromatic aberration and distortion, wherein an image processing system has a configuration for correcting not only a magnification chromatic aberration but also a distortion, and however, one feature of the embodiment is a magnification chromatic aberration correction and wherein it is only necessary for the image to be an image captured by using an optical system with, at least, a wide view angle and a large magnification chromatic aberration.
  • the color components of an image are red (R), green (G) and blue (B) in accordance with three additive primary colors, and however, may be yellow (Y), magenta (M), and cyan (C) in accordance with three subtractive primary colors.
  • FIG. 1 is a block diagram illustrating the function of an image processing system in an image capturing device on which one embodiment of the present invention is applied.
  • the image capturing device also includes a manipulation part, an image storage part, an image display part, and the like, which are omitted in FIG. 1 .
  • the image capturing device is used as, for example, an on-vehicle camera, and of course, the application is not limited thereto.
  • a control part 100 provides a control a signal (clock, horizontal/vertical synchronizing signals, or the like) necessary for each part of the device so as to control the operation of the each part in a manner of a pipeline.
  • An image capturing element 110 is composed of, for example, a CCD, CMOS sensor, or the like, for converting an optical image captured by using an optical system with a wide view angle and a large magnification chromatic aberration and distortion (not illustrated in the figure) into an electric signal (image data).
  • the image capturing element 110 is provided with a Bayer pattern color filter and Bayer pattern RGB image data are sequentially output based on coordinate values (x, y) provided from the control part 100.
  • the control part 100 also sequentially provides the coordinate values (x, y) to a subsequent stage with a predetermined time deviation which are provided to the image capturing element 110.
  • the coordinate values (x, y) may be generated inside the image capturing element 110 while clock and horizontal/vertical synchronizing signals are inputs, and may be sequentially provided to the subsequent stage from the image capturing element 110.
  • An A/D converter 120 converts Bayer pattern RGB image data as an analog signal output from the image capturing element 110 into a digital signal to send it to a Bayer complementation part 130.
  • the digital signal is composed of, for example, 8 bits for each of RGB.
  • an AGC circuit is generally provided but omitted herein.
  • the Bayer complementation part 130 receives an input of Bayer pattern RGB image data which have been converted into a digital signal, then generates image data at all the coordinate positions (pixel data) independently for each of RGB colors by means of a linear complementation, and sends them to a magnification chromatic aberration complementation part 140.
  • FIGS. 2A, 2B, and 2C illustrate Bayer pattern color filters.
  • B 2 , B 4 , B 6 , B 8 , and B 0 are the same as the case of the above-mentioned R 2 , R 4 , R 6 , R 8 , and R 0 and therefore omitted.
  • an image capturing element having Bayer pattern color filters has been described in the present example, an image capturing element having another color filter pattern such as an CMYG pattern or an RGB + Ir (infrared) pattern is, of course, also effective.
  • a memory with an even lower latency or a 4-port RAM may be needed for a magnification chromatic aberration correction in an image capturing element having four color filter patterns, as compared with a three-color type such as an RGB type.
  • the magnification chromatic aberration correction part 140 receives an input of Bayer-complemented R, G, and B image data, then independently applies a coordinate transformation (magnification chromatic aberration coordinate transformation) to each of RGB color components by means of a predetermined polynomial or the like, and outputs RGB image data with the corrected magnification chromatic aberration.
  • the magnification chromatic aberration correction part 140 will be described in detail below, and it is possible to use a memory with a low capacity and a low latency or a memory having a low capacity and plural ports (such as an SRAM) for the coordinate transformation for a magnification chromatic aberration correction. Furthermore, it is possible to reduce a circuit scale of a coordinate transformation operation part by application of the example.
  • the MTF correction part 150 receives an input of the RGB image data with the corrected magnification chromatic aberration and applies an MTF correction by using an FIR filter so as to output RGB image data with corrected MTF (enhanced high frequencies).
  • FIG. 3 is a schematic diagram illustrating the configuration of the MTF correction part 150.
  • An FIR filter (5 x 5 filter) 154 receives an input of only a brightness signal Y among the YCbCr and conducts a predetermined MTF correction. Filtering (MTF correction) of only Y signal is conducted whereby it is possible to obtain a high quality image with suppressed increase of color noise.
  • FIG. 4 illustrates one example of FIR filter coefficients.
  • a distortion correction part 160 receives an input of the RGB data with corrected magnification chromatic aberration and corrected MTF, then applies a common coordinate transformation (distortion coordinate transformation) to each of the RGB color components by means of a predetermined polynomial or the like, and outputs RGB image data with a corrected distortion.
  • a common coordinate transformation disortion coordinate transformation
  • For the coordinate transformation at the distortion correction part 160 it is possible to use a memory with a high latency (such as a DRAM) because only 1 port is needed although the memory capacity (at most, for 1 image) is larger than the memory for magnification chromatic aberration correction.
  • the distortion correction part 160 will be described in detail below.
  • a gamma correction part 170 receives an input of the RGB image data output from the distortion correction part 160, then applies a predetermined gamma correction operation by using a look-up table for each of RGB or the like, and outputs gamma-corrected RGB image data.
  • the output image data of the gamma correction part 170 are sent to a display part (not illustrated in the figure) and displayed on a monitor.
  • magnification chromatic aberration correction part 140 and the distortion correction part 160 will be described in detail below. First, the principles of a magnification chromatic aberration correction and distortion correction will be described.
  • image data (pixel data) at the position (pixel) denoted by "1" in the upper right part of a screen are shifted from an original position due to a distortion and further each of RGB color components is shifted differently due to a magnification chromatic aberration so that the positions of RGB imaged by an image capturing element practically are positions "2" (R), "3” (G), and "4" (B), respectively.
  • FIG. 6 schematically illustrates a method for correcting a magnification chromatic aberration and distortion simultaneously. That is, it is possible to correct the magnification chromatic aberration and the distortion simultaneously by copying, that is, conducting a coordinate transformation of, image data of each of the RGB color components at the positions (pixels) "2" (R), “3” (G), and "4" (B) onto the position (pixel) "1" which is the original position.
  • this method may require a memory with a large capacity for each of RGB color components and a low latency or a multi-port memory.
  • a high-speed 6-line memory may be needed for each of RGB for coordinate transformation.
  • a coordinate transformation operation circuit or a coordinate transformation cable (LUT) may be needed for each of RGB.
  • FIGS 7A and 7B schematically illustrate a method for correcting a magnification chromatic aberration ( FIG. 7A ) and distortion ( FIG. 7B ) separately.
  • the magnification chromatic aberrations of color components are different from one another but the shift caused thereby is small.
  • a shift caused by the distortion is large but is equal for each color component.
  • coordinate transformation is first applied to image data of each of RGB color components to correct magnification chromatic aberrations and subsequently coordinate transformation is applied to all the RGB data with corrected magnification chromatic aberration to correct distortion.
  • the color components of R and B are shifted nearly symmetrically centered on G due to a magnification chromatic aberration. Therefore, the magnification chromatic aberration is attained by only conducting coordinate transformation of color components R and B and copying them onto the position of a G component. Thereby, a coordinate transformation operation circuit or coordinate transformation table (LUT) for magnification chromatic aberration correction is needed for only the RB color components and it is possible to reduce a circuit scale whereby it is possible to further reduce the cost totally.
  • LUT coordinate transformation operation circuit or coordinate transformation table
  • FIG. 7A schematically illustrates a magnification chromatic aberration correction, wherein coordinate transformation is applied to image data (pixel data) of RB color components at the positions (pixels) "2" (R) and “4" (B) and they are copied onto the position (pixel) "3" (G) of the G component.
  • the magnification chromatic aberrations are corrected by this operation.
  • FIG. 7B schematically illustrates a distortion correction, wherein a coordinate transformation is applied to all of the image data of the RGB color components at the position (pixel) "3” in which the magnification chromatic aberrations have been corrected and they are copied onto the position (pixel) "1" which is the original position.
  • the distortion is corrected by this operation.
  • a 3-line memory for each of RGB is sufficient for a high-speed memory for a magnification chromatic aberration correction.
  • a 5-line memory is needed for a distortion correction
  • a low-speed memory common to RGB is allowed and it is possible to attain cost reduction totally as compared with the case of FIG. 6 .
  • a coordinate transformation operation circuit or coordinate transformation table for a magnification chromatic aberration correction is needed for only RB color components, and it is possible to attain further cost reduction totally.
  • a distortion intended herein means a lens distortion with respect to an intended projection manner, wherein the intended projection manner may be, for example, a projection manner of obtaining an image looked down from above a camera and includes a projection manner of enlarging and displaying a part.
  • FIG. 8 is a diagram illustrating the configuration of one example of the magnification chromatic aberration correction part 140.
  • a coordinate transformation memories for magnification chromatic aberration correction (line buffers) 142 whose RGB color components correspond to 142(R), 142(G), and 142(B), respectively, a magnification chromatic aberration correction-coordinate transformation operation part 144 for calculating a coordinate transformation for magnification chromatic aberration correction with respect to each of RGB (only R and B in practice) in accordance with a predetermined coordinate transformation formula, and a coordinate transformation coefficient table for holding a coefficient used in the coordinate transformation formula.
  • a memory with a small capacity and 3 ports for RGB or a memory with a low latency is needed as a line buffer.
  • the maximum shift caused by a magnification chromatic aberration is supposed to be 20 lines to a Y direction and each of the coordinate transformation memories 142(R), 142(G), and 142 (B) is composed of an SRAM for 20 lines.
  • the size in X directions is determined depending on the resolution, and for example, the size in X directions is 640 dots for the resolution VGA (640 x 480).
  • the color depth for each of RGB is 8 bits, reading from and writing in any of the coordinate transformation memories 142(R), 142(G) and 142(B) is conducted in an 8-bit unit.
  • RGB image data of a captured image having a magnification chromatic aberration and a distortion are written in the coordinate transformation memories 142(R), 142(G), and 142 (B), respectively, sequentially from the first line, in accordance with the coordinate values (x, y). Then, as each of image data of 20 lines is written, the image data are erased sequentially from the first line, and instead, image data of subsequent lines are newly written.
  • RGB image data of 20 lines at most which are required for a magnification chromatic aberration correction coordinate transformation are sequentially stored in the coordinate transformation memories 142(R), 142(G), and 142(B), respectively.
  • coordinate values (x, y) indicate a position for reading out 1 frame of a captured image.
  • the coordinate transformation memories 142(R), 142(G), and 142(B) are line buffers of 20 lines and writing lines are changed cyclically, it may be impossible to use the coordinate values (x, y) for writing addresses of the coordinate transformation memories 142(R), 142 (G), and 142(B) without a change. Therefore, it may be necessary to convert the coordinate values (x, y) into real addresses of the coordinate transformation memories 142(R), 142(G), and 142(B), wherein such a configuration is omitted in FIG. 8 .
  • This also applies to the relation between coordinate values (X, Y) after transformation in a reading operation as described below and the read-out addresses of the coordinate transformation memory 142 (R) , 142 (G), or 3.42 (B).
  • the magnification chromatic aberration correction-coordinate transformation calculation part 144 receives inputs of coordinate values (x, y) which are coordinates before coordinate transformation, then calculates coordinates transformed for magnification chromatic aberration correction for each of RGB in accordance with a predetermined coordinate transformation formula such as a polynomial, and outputs coordinate values (X, Y) which are coordinates after coordinate transformation for each of RGB. As illustrated in FIG. 7A , a coordinate transformation for magnification chromatic aberration correction is only applied to the R and B color components so as to copy them at the position of the G component.
  • the magnification chromatic aberration correction-coordinate transformation operation part 144 outputs input coordinate values (x, y) for G component as coordinate values (X, Y) after transformation without a change and transforms input coordinate values (x, y) for each of R and B color components into coordinate values (X, Y) using a predetermined coordinate transformation formula so as to output the coordinate values (X, Y) after transformation. These are repeated for each set of coordinate values (x, y).
  • the center of a screen is an origin
  • Y y + b 1 + b 2 ⁇ abs y + b 3 ⁇ abs x + b 4 ⁇ x 2 ⁇ y
  • abs ( ) indicates an absolute value
  • a (1) - a(4) and b (1) - b(4) are coordinate transformation coefficients.
  • the coordinate transformation coefficients have already been held in the coordinate transformation coefficient table 146.
  • the coordinate transformation memories 142(R), 142 (G), and 142(B) sequentially read out RGB image data, respectively, based on the coordinate values (X, Y) output from the magnification chromatic aberration correction-coordinate transformation operation part 144 (in practice, values provided by address conversion of the coordinate values (X, Y)) in parallel with (in practice, at a predetermined delayed time after) the above-mentioned writing operation.
  • G component image data at the same position as that at the time of writing are read out from the coordinate transformation memory 142(G).
  • RB color component image data at the positions with predetermined positional shifts from those at the time of writing that is, the positions shifted by magnification chromatic aberrations are read out from the coordinate transformation memories 142(R) and 142 (B).
  • RGB image data with a corrected magnification chromatic aberrations are output from the coordinate transformation memories 142(R), 142(G), and 142(B). That is, RGB image data on the coordinate values (X, Y) after coordinate transformation are output based on RGB image data on the coordinate values (x, y) before coordinate transformation.
  • FIGS. 9 to 13 illustrate some examples of the configuration of the magnification chromatic aberration correction-coordinate transformation operation part 144. Additionally, although the basis is a G color component herein, the basis of R or B is also allowed.
  • FIG. 9 illustrates a specific example in which only for R and B color components, input coordinate values (x, y) are transformed by coordinate transformation operation parts 1441 and 1442, respectively, using a set of formulas (13) or the like so as to output coordinate values for R (X, Y) and coordinate values for B (X, Y) after coordinate transformation, while no coordinate transformation is applied to a G color component and input coordinate values (x, y) are output as coordinate values (X, Y) after coordinate transformation without a change. Because the coordinate transformation operation parts are needed for only the R and B color components, a small circuit scale is allowed.
  • FIG. 10 illustrates a specific example in which correction quantities (shift quantities) for coordinate values (x, y) are obtained by a coordinate transformation operation part 1443 and the values provided by subtracting the correction quantities from the coordinate values (x, y) in a subtraction part 1444 are coordinate values for B (X, Y) while the values provided by adding the correction quantities to the coordinate values (x, y) in an addition part 1445 are coordinate values for R (X, Y).
  • correction quantities (shift quantities) for coordinate values (x, y) are obtained by a coordinate transformation operation part 1443 and the values provided by subtracting the correction quantities from the coordinate values (x, y) in a subtraction part 1444 are coordinate values for B (X, Y) while the values provided by adding the correction quantities to the coordinate values (x, y) in an addition part 1445 are coordinate values for R (X, Y).
  • FIG. 10 illustrates a specific example in which correction quantities (shift quantities) for coordinate values (x, y) are obtained by a coordinate transformation operation part 1443
  • FIG. 11 illustrates that the correction quantity for R is further adjusted by a gain circuit 1446 with respect to the difference between the symmetrical positional shifts. Additionally, the gain circuit may be provided for the B side. According to the specific example illustrated in FIG. 10 or FIG. 11 , only one coordinate transformation operation part is needed whereby it is possible to further reduce the circuit scale.
  • the above configurations may be used similarly for a sensor which has an Ir pixel in addition to the R, G, and B pixels and is capable of capturing an infrared region and the like.
  • a gain circuit for Ir 1461 and an addition part 1462 are added to the configuration illustrated in FIG. 11 whereby it is possible to calculate coordinate values for B by one coordinate transformation operation part 1443 and coordinate values for R and Ir via gains, and therefore, only one coordinate transformation operation part is needed on the condition that three coordinate transformation operation parts were needed conventionally, whereby it is possible to further reduce a circuit scale.
  • FIG. 13 and FIG. 14 illustrate examples of a configuration for conducting a magnification chromatic aberration correction and a distortion correction simultaneously. In these cases, a distortion correction part 160 as described below is not needed.
  • FIG. 13 is a specific example in which a correction quantity (shift quantity) for a distortion common to RGB is obtained for coordinate values (x, y) by a coordinate transformation operation part 1447 using a predetermined polynomial or the like and the correction quantity is added to the coordinate values (x, y) in an addition part 1448 while outputs of the addition part 1448 are coordinate values for G (X, Y) and one of inputs of a subtraction part 1444 and addition part 1445.
  • the operations of the coordinate transformation operation part 1443, subtraction part 1444, and addition part 1445 are similar to those illustrated in FIG. 10 .
  • the coordinate values for each of RGB (X, Y) indicate coordinates after a coordinate transformation with respect to both of correction quantities of a magnification chromatic aberration and distortion.
  • FIG. 14 is a specific example in which a correction quantity for a distortion common to RGB is similarly obtained by a coordinate transformation operation part 1447 using a predetermined polynomial or the like, and the values provided by adding an output of an addition part 1445 to an output of the coordinate transformation operation part 1447 in an addition part 1449 are coordinate values for R (X, Y), while the values provided by adding input coordinate values (x, y) to an output of the coordinate transformation operation part 1447 in an addition part 1450 are coordinate values for G (X, Y) and the values provided by adding an output of a subtraction part 1444 to an output of the coordinate transformation operation part 1447 in an addition part 1451 are coordinate values for B (X, Y).
  • the coordinate values for each of RGB (X, Y) indicate coordinates after a coordinate transformation with respect to both of correction quantities of a magnification chromatic aberration and distortion, similarly to the case of FIG. 13 .
  • a look-up-table (LUT) in which the correspondence between input coordinate vales (x, y) and output coordinate vales (X, Y) for each of R and B color components is stored may be provided instead of the coordinate transformation operation parts 1441 and 1442 in FIG. 9 and the coordinate values (X, Y) after coordinate transformation corresponding to the coordinate values (x, y) before coordinate transformation may be directly obtained by using the LUT.
  • an LUT in which the correspondence between input coordinate values (x, y) and a correction quantity is stored may be provided instead of the coordinate transformation operation parts 1443 and 1447 in FIG. 10 to FIG. 14 and the correction quantity corresponding to the coordinate values (x, y) may be directly obtained by using the LUT.
  • FIG. 15 is a diagram illustrating one example of the configuration of a distortion correction part 160.
  • the distortion correction part 160 is not needed. In FIG. 13 or FIG. 14 is used in the magnification chromatic aberration correction part 140, the distortion correction part 160 is not needed.
  • an RGB combination part 161 for combining three image data for RGB into a set of data
  • a coordinate transformation memory for a distortion correction common to the RGB image data 162
  • an RGB separation part 163 for separating the combined RGB image data into the original color components thereof
  • a distortion correction-coordinate transformation operation part 164 for calculating a coordinate transformation for a distortion correction with respect to the combined RGB image data in accordance with a predetermined coordinate transformation formula
  • a coordinate transformation coefficient table 165 for holding a coefficient(s) used in the coordinate transformation formula.
  • a buffer memory for storing image data on a screen at most is needed for a distortion correction operation. Meanwhile, the shift is common to all the RGB color components and only one buffer memory with a total bit width of RGB image data is allowed.
  • the resolution is supposed to VGA (640 x 480) and the bit number (color depth) of each of RGB image data is supposed to 8 bits, while the coordinate transformation memory 162 is supposed to be composed of a DRAM in which each of writing and reading 640 x 480 dots is conducted in a 24-bit unit.
  • An RGB combining part 161 sequentially receives an input of each of RGB image data (each 8 bits) with a corrected magnification chromatic aberration and combines them into a set of image data (24 bits) for an output.
  • the combined RGB image data are written into the coordinate transformation memory 162 sequentially from the first line in accordance with the coordinate values (x, y).
  • the distortion correction-coordinate transformation operation part 164 receives an input of coordinate values (x, y) before coordinate transformation, then calculates a coordinate transformation for distortion correction common to RGB inn accordance with a predetermined coordinate transformation formula such as a polynomial, and outputs coordinate values (X, Y) after coordinate transformation.
  • the coordinate transformation formula may be represented by the same formula (13) as that of a magnification chromatic aberration correction described above.
  • the used coordinate transformation coefficients are, of course, different.
  • the coordinate transformation coefficients have already been held in the coordinate transformation coefficient table 165.
  • the coordinate transformation formula for distortion correction may be, for example, the same formula (13) as that for a magnification chromatic aberration, and if the calculation result in the magnification chromatic aberration correction (which may be held in a memory or the like) are employer for the terms x 2 , y 2 , abs(s), and abs(y) used in the formula (13), it is unnecessary to calculate them newly and it is possible to reduce the circuit scale of the operation party 164.
  • the coordinate transformation memory 162 sequentially reads out the combined RGB image data based on the coordinate values (X, Y) output from the distortion correction coordinate transformation part 164 in parallel with (accurately, at a predetermined delayed time after) a writing operation of the a combined RGB image data (24 bits) described above.
  • the RGB separation part 163 separates the combined RGB image data (24 bits) read out from the coordinate transformation memory 162 into the original image data of each color component (8 bit).
  • each of RGB image data with corrected magnification chromatic aberration and distortion is output from the RGB separation part 163. That is, each of RGB image data is copied at the original position (x, y).
  • a look-up-table in which the correspondence between input coordinate values (x, y) and output coordinate values (X, Y) is stored may also be provided for a distortion correction operation and the coordinate values (X, Y) after coordinate transformation corresponding to the coordinate values (x, y) before coordinate transformation may be directly obtained by using the LUT.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)
  • Color Television Image Signal Generators (AREA)

Claims (6)

  1. Procédé de traitement d'image configuré de façon à traiter une image capturée au moyen d'un système optique avec un angle de vue élargi et une aberration chromatique par agrandissement importante ; le procédé comprenant les étapes suivantes :
    une correction d'une aberration chromatique par agrandissement est réalisée en n'exécutant aucune transformation de coordonnées pour une image d'une composante de couleur particulière (G) mais en exécutant une transformation de coordonnées seulement pour une image d'une première composante de couleur (R, B) à l'exception de la composante de couleur particulière (G) ;
    caractérisé en ce que :
    la correction de l'aberration chromatique par agrandissement est réalisée séparément par rapport à une correction d'une déformation ;
    une fois que la correction de l'aberration chromatique par agrandissement a été réalisée, la correction de déformation est exécutée conjointement pour une image comprenant toutes les composantes de couleur (R, G, B) qui sont les composantes de couleur (R, B) qui ont été corrigées au cours de la correction de l'aberration chromatique par agrandissement séparée et la composante de couleur particulière (G),
    dans lequel une quantité de la correction de l'aberration chromatique par agrandissement devant être appliquée sur une image d'une deuxième composante de couleur à l'exception de la composante de couleur particulière est réglée en appliquant sur ladite transformation de coordonnées un gain prédéterminé.
  2. Procédé de traitement d'image selon la revendication 1, dans lequel la composante de couleur particulière est le vert (G) et les composantes de couleur à l'exception de celle-ci sont le rouge (R) et le bleu (B).
  3. Dispositif de traitement d'image configuré de façon à traiter une image capturée au moyen d'un système optique avec un angle de vue élargi et une aberration chromatique par agrandissement importante, le dispositif comprenant :
    un dispositif de correction d'aberration chromatique par agrandissement (140) configuré de façon à réaliser une correction d'une aberration chromatique par agrandissement en n'exécutant aucune transformation de coordonnées pour une image d'une composante de couleur particulière (G) mais en exécutant une transformation de coordonnées seulement pour une image d'une première composante de couleur (R, B) à l'exception de la composante de couleur particulière (G) ;
    caractérisé en ce que :
    le dispositif de correction d'aberration chromatique par agrandissement (140) est configuré de façon à réaliser séparément la correction de l'aberration chromatique par agrandissement par rapport à une correction d'une déformation ;
    un dispositif de correction de déformation (160) configuré de façon à réaliser une correction d'une aberration de déformation conjointement pour une image comprenant toutes les composantes de couleur (R, G, B) qui sont les composantes de couleur (R, G) qui ont été corrigées dans le dispositif de correction d'aberration chromatique par agrandissement séparée (140) et la composante de couleur particulière (G),
    dans lequel le dispositif de correction d'aberration chromatique par agrandissement (140) est configuré de façon à régler une quantité de la correction de l'aberration chromatique par agrandissement devant être appliquée sur une image d'une deuxième composante de couleurs à l'exception de la composante de couleur particulière en appliquant sur ladite transformation de coordonnées un gain prédéterminé.
  4. Dispositif de traitement d'image selon la revendication 3, dans lequel la composante de couleur particulière est le vert (G) et les composantes de couleur à l'exception de celle-ci sont le rouge (R) et le bleu (B).
  5. Dispositif de traitement d'image selon la revendication 3 ou 4, comprenant par ailleurs une table de transformation de coordonnées pour la correction de l'aberration chromatique par agrandissement.
  6. Dispositif de capture d'image comprenant un système optique avec un angle de vue élargi et au moins une aberration chromatique par agrandissement importante, un élément de capture d'image configuré de façon à lire une image capturée par l'intermédiaire du système optique, et un dispositif de traitement d'image selon l'une quelconque des revendications 3 à 5.
EP09154620A 2008-03-10 2009-03-09 Procédé de traitement d'images, dispositif de traitement d'images et dispositif de capture d'images Ceased EP2101295B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2008059704A JP5078148B2 (ja) 2008-03-10 2008-03-10 画像処理装置及び画像撮像装置

Publications (2)

Publication Number Publication Date
EP2101295A1 EP2101295A1 (fr) 2009-09-16
EP2101295B1 true EP2101295B1 (fr) 2011-12-28

Family

ID=40810631

Family Applications (1)

Application Number Title Priority Date Filing Date
EP09154620A Ceased EP2101295B1 (fr) 2008-03-10 2009-03-09 Procédé de traitement d'images, dispositif de traitement d'images et dispositif de capture d'images

Country Status (4)

Country Link
US (1) US8106973B2 (fr)
EP (1) EP2101295B1 (fr)
JP (1) JP5078148B2 (fr)
CN (1) CN101534384B (fr)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5240453B2 (ja) * 2008-02-14 2013-07-17 株式会社リコー 画像処理方法、画像処理装置及び画像撮像装置
JP5078147B2 (ja) * 2008-03-04 2012-11-21 株式会社リコー 画像処理装置及び画像撮像装置
JP5273371B2 (ja) 2008-12-24 2013-08-28 株式会社リコー 画像処理装置及び方法、並びに車載カメラ装置
JP5267794B2 (ja) 2008-12-26 2013-08-21 株式会社リコー 画像処理装置及び車載カメラ装置
WO2011052172A1 (fr) * 2009-10-27 2011-05-05 パナソニック株式会社 Dispositif d'imagerie et dispositif de mesure de distance utilisant ce dispositif
KR101635101B1 (ko) * 2009-11-19 2016-06-30 삼성전자주식회사 렌즈 왜곡과 색수차를 보정하는 장치 및 방법
US8624999B2 (en) 2009-12-01 2014-01-07 Ricoh Company, Ltd. Imaging apparatus
JP5570260B2 (ja) * 2010-03-23 2014-08-13 京セラ株式会社 撮像装置
JP5188651B2 (ja) * 2010-03-31 2013-04-24 キヤノン株式会社 画像処理装置、およびそれを用いた撮像装置
JP5585885B2 (ja) 2011-03-18 2014-09-10 株式会社リコー 画像処理装置及び画像処理方法
JP5704984B2 (ja) 2011-03-22 2015-04-22 キヤノン株式会社 撮像装置
WO2012176355A1 (fr) * 2011-06-23 2012-12-27 パナソニック株式会社 Dispositif d'imagerie
CN102306372A (zh) * 2011-07-12 2012-01-04 浙江大学 一种基于缨帽变换的遥感影像处理方法
US20130321675A1 (en) 2012-05-31 2013-12-05 Apple Inc. Raw scaler with chromatic aberration correction
JP6417666B2 (ja) 2013-05-15 2018-11-07 株式会社リコー 画像処理システム
WO2016088406A1 (fr) * 2014-12-04 2016-06-09 三菱電機株式会社 Dispositif et procédé de traitement d'image, dispositif de capture d'image, programme et support d'enregistrement
JP2017098631A (ja) * 2015-11-18 2017-06-01 オリンパス株式会社 画像合成処理装置
US20200137380A1 (en) * 2018-10-31 2020-04-30 Intel Corporation Multi-plane display image synthesis mechanism

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3549413B2 (ja) 1997-12-04 2004-08-04 富士写真フイルム株式会社 画像処理方法および画像処理装置
DE19855885A1 (de) 1997-12-04 1999-08-05 Fuji Photo Film Co Ltd Bildverarbeitungsverfahren und -vorrichtung
JP2000069343A (ja) * 1998-06-11 2000-03-03 Fuji Photo Film Co Ltd デジタル撮像装置
JP2001086332A (ja) * 1999-09-09 2001-03-30 Fuji Photo Film Co Ltd 画像処理装置
JP2003189169A (ja) * 2001-09-27 2003-07-04 Fuji Photo Film Co Ltd 画像処理装置
US7433089B2 (en) * 2001-09-27 2008-10-07 Fujifilm Corporation Image processor
JP4378994B2 (ja) * 2003-04-30 2009-12-09 ソニー株式会社 画像処理装置、画像処理方法ならびに撮像装置
JP4479457B2 (ja) * 2004-05-27 2010-06-09 ソニー株式会社 画像処理装置、および画像処理方法、並びにコンピュータ・プログラム
JP4543147B2 (ja) * 2004-07-26 2010-09-15 ジーイーオー セミコンダクター インコーポレイテッド パノラマビジョンシステム及び方法
US7576767B2 (en) * 2004-07-26 2009-08-18 Geo Semiconductors Inc. Panoramic vision system and method
JP4596986B2 (ja) 2005-06-07 2010-12-15 オリンパス株式会社 画像撮像装置
JP4553798B2 (ja) 2005-06-07 2010-09-29 オリンパス株式会社 画像撮像装置
JP4596988B2 (ja) 2005-06-07 2010-12-15 オリンパス株式会社 画像撮像装置
JP4596987B2 (ja) 2005-06-07 2010-12-15 オリンパス株式会社 画像撮像装置
JP2007158628A (ja) * 2005-12-02 2007-06-21 Canon Inc 撮像装置及び画像処理方法
JP4709084B2 (ja) * 2006-07-07 2011-06-22 キヤノン株式会社 画像処理装置及び画像処理方法
JP2008059704A (ja) 2006-08-31 2008-03-13 Toshiba Corp 光ディスク記録再生装置及び光ディスク記録再生方法

Also Published As

Publication number Publication date
US20090231472A1 (en) 2009-09-17
CN101534384B (zh) 2014-10-08
CN101534384A (zh) 2009-09-16
JP5078148B2 (ja) 2012-11-21
JP2009218802A (ja) 2009-09-24
US8106973B2 (en) 2012-01-31
EP2101295A1 (fr) 2009-09-16

Similar Documents

Publication Publication Date Title
EP2101295B1 (fr) Procédé de traitement d'images, dispositif de traitement d'images et dispositif de capture d'images
JP5267794B2 (ja) 画像処理装置及び車載カメラ装置
EP2104362B1 (fr) Appareil d'imagerie
JP5008139B2 (ja) 画像撮像装置
JP5078147B2 (ja) 画像処理装置及び画像撮像装置
JP4911628B2 (ja) 画像処理方法、画像処理装置及び画像撮像装置
JP5008578B2 (ja) 画像処理方法、画像処理装置及び画像撮像装置
KR101265000B1 (ko) 화상 처리 방법 및 장치, 및 차량 탑재 카메라 장치
JP5062846B2 (ja) 画像撮像装置
JP2009218909A (ja) 画像撮像装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20090309

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA RS

17Q First examination report despatched

Effective date: 20091023

AKX Designation fees paid

Designated state(s): DE FR GB

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602009004313

Country of ref document: DE

Effective date: 20120308

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20121001

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602009004313

Country of ref document: DE

Effective date: 20121001

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20180321

Year of fee payment: 10

Ref country code: DE

Payment date: 20180322

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20180330

Year of fee payment: 10

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602009004313

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20190309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191001

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190331