JP2011228855A - Image processor, image noise determination method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of determining a noise amount based on an average luminance of an image signal inputted from an imaging device without acquiring data relating to the noise amount from the imaging device.
[Solution]
The image processing device 20 picks up an image and corrects the brightness of the image obtained by the image pickup from the image pickup device 10. The image signal input control unit 21 inputs the image signal after the brightness correction, and the brightness in the image pickup device. Based on the storage unit 25 that stores an algorithm for determining the amount of gain to be corrected, the average luminance of the image signal input to the image signal input control unit 21, and the algorithm stored in the storage unit 25, it is applied to the image signal. A gain amount estimator 22 that estimates the gain amount for luminance correction, and a noise amount determiner 23 that determines whether or not there is a lot of noise based on the gain amount estimated by the gain amount estimator 22.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus used in an in-vehicle camera system, and more particularly to a method for determining noise in an image signal-processed by an imaging apparatus.

  The in-vehicle camera system includes an imaging device that captures an image of the periphery of the vehicle and the inside of the vehicle, and an image processing device that receives an image signal transmitted from the imaging device and performs image processing according to the application.

  The imaging apparatus includes a lens that collects light, an imaging element that photoelectrically converts light collected by the lens to obtain an electrical signal, and a signal processing unit that performs processing of the electrical signal. The signal processing unit adjusts the captured image based on the setting value, and the setting range of the setting value has an upper limit and a lower limit.

  One of the signal processing performed by the imaging apparatus is luminance control. The brightness control is a function for adjusting the amount of the brightness level so as to keep the output brightness constant. When the image is dark, the brightness level is increased so that the target brightness level is set, and when the image is bright, the brightness level is decreased so as to be the set brightness level.

  Since the setting range of the setting value of the luminance level has an upper limit, even if the luminance level is increased to the upper limit, the target luminance level may not be reached. If the set value is increased in order to reach the target luminance level, the luminance level increases. However, since the noise included in the image signal is amplified at the same rate, an image with much noise is obtained.

  In an in-vehicle camera system, an image processing device performs obstacle detection on a road, white line detection, and the like based on an image captured by an imaging device. Therefore, if the in-vehicle camera system performs image processing based on a noisy image, it may cause an obstacle or a white line to be erroneously recognized.

  Patent Document 1 discloses a method for measuring noise in an image signal. In the method described in this document, a plurality of local regions are provided in an image signal, and a noise level is detected based on luminance variations of the unsaturated local regions. Further, although not related to image noise, Patent Document 2 discloses a method of acquiring control information by transmitting and receiving control data of an imaging device through communication between the imaging device and the image processing device.

JP 2003-331283 A JP 2006-121524 A

  In an environment where an in-vehicle camera system is actually used, it is difficult to detect noise directly from an image signal because the brightness of a subject is almost constant. Therefore, it is difficult to apply the method of Patent Document 1 for detecting noise based on luminance variation to an in-vehicle camera system.

  Also, in order to employ the method of Patent Document 2 that transmits and receives control parameter information and the like of the imaging apparatus using communication, it is necessary to add dedicated communication lines to both the imaging apparatus and the image processing apparatus. Therefore, the in-vehicle camera system has problems such as an increase in cost and an increase in the size of the apparatus.

  Therefore, in view of the above background, the present invention provides an image processing device capable of determining the amount of noise based on the average luminance of the image signal input from the imaging device without acquiring data relating to the amount of noise from the imaging device. For the purpose.

  An image processing apparatus according to the present invention includes an image signal input unit that inputs an image signal after luminance correction from an imaging apparatus that captures an image and performs luminance correction on an image obtained by the imaging, and the imaging apparatus. A storage unit storing an algorithm for determining a gain amount for luminance correction, an average luminance of the image signal input to the image signal input unit, and an algorithm stored in the storage unit are applied to the image signal. A gain amount estimator that estimates the amount of gain for luminance correction, and a noise amount determiner that determines whether or not there is a lot of noise based on the gain amount estimated by the gain amount estimator. The noise amount determination unit may determine that there is a lot of noise when the gain amount estimated by the gain amount estimation unit is a maximum settable gain amount.

  With this configuration, the amount of noise can be determined based on the average luminance of the image signal input from the imaging device without acquiring data relating to the amount of noise from the imaging device. Since it is not necessary to change the imaging device in order to determine the amount of noise in the image processing apparatus, it is possible to perform noise determination while suppressing the cost of the entire system including the image processing apparatus.

  The image processing apparatus of the present invention has a configuration in which image processing based on the image signal is not performed when the noise amount determination unit determines that there is a lot of noise. With this configuration, since processing based on a noisy image is not performed, the accuracy of image processing can be improved.

  An in-vehicle camera system of the present invention includes an imaging device that images the periphery of a vehicle and the image processing device, and the imaging device includes a lens and an imaging element that photoelectrically converts light input through the lens, A gain control unit that determines a gain amount according to the luminance of the image signal obtained by photoelectric conversion, a luminance correction unit that corrects the luminance of the image signal with the gain amount determined by the gain control unit, and the luminance correction And an output unit for outputting an image signal. The gain control unit includes an average luminance calculation unit that calculates an average luminance of the image signal, and a gain amount adjustment unit that adjusts the gain amount so that the average luminance of the image signal becomes a target value based on the average luminance. The gain amount set by the gain amount adjustment unit has an upper limit. With this configuration, the imaging apparatus can determine an appropriate gain amount based on the average luminance of the image.

  In the in-vehicle camera system according to the present invention, the average luminance calculation unit divides the image into a plurality of regions, obtains an average luminance of each divided region, and a region where the average luminance is a first threshold value or more In the case where there is a region where is less than or equal to the second threshold, the average luminance of the region excluding the region is calculated. By this configuration, an area with an extremely high average luminance (area where the average luminance is equal to or higher than the first threshold) or an area where the average luminance is extremely low (an area where the average luminance is equal to or lower than the second threshold) is excluded from the average luminance calculation target. Thus, an appropriate average brightness for determining the gain amount can be obtained.

  The image noise determination method of the present invention includes a step of inputting the image signal after luminance correction from an imaging device that captures an image and performs luminance correction on the image obtained by imaging, and the imaging from the storage unit A step of reading an algorithm for determining a gain amount of luminance correction in the apparatus, and a gain amount of luminance correction applied to the image signal based on an average luminance of the input image signal and an algorithm read from the storage unit An estimation step, and a step of determining whether or not there is a lot of noise based on the estimated gain amount.

  The program of the present invention captures an image and outputs the image signal from the imaging device to a computer in order to determine the amount of noise of the image signal acquired from the imaging device that performs luminance correction on the image obtained by imaging. Based on the step of inputting, the step of reading out an algorithm for determining the gain amount of luminance correction in the imaging device from the storage unit, and the average luminance of the input image signal and the algorithm read out from the storage unit A step of estimating a luminance correction gain amount applied to the signal and a step of determining whether or not there is a lot of noise based on the estimated gain amount are executed.

  With these configurations, the amount of noise can be determined based on the average luminance of the image signal input from the imaging device without acquiring data regarding the amount of noise from the imaging device. Various configurations of the image processing apparatus of the present invention described above can be applied to the image noise determination method and program of the present invention.

  The present invention provides an image signal input from an imaging device without adding a special communication line to the imaging device or performing a significant hardware configuration change to acquire data on the amount of noise from the imaging device. It has an excellent effect that the amount of noise can be determined based on the average luminance.

The figure which shows the structure of the vehicle-mounted camera system of this Embodiment (A) The figure which shows the example which divided the imaged image into areas, (b) The figure which shows the example of the image containing the area | region where average brightness is larger than a predetermined threshold value The figure which shows the change of the average brightness | luminance of the captured image after brightness correction The figure which shows operation | movement of the image processing apparatus of this Embodiment.

Hereinafter, an in-vehicle camera system according to an embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing a configuration of the in-vehicle camera system 1. The in-vehicle camera system 1 includes an imaging device 10 and an image processing device 20. The imaging device 10 includes an imaging element 12 that receives and photoelectrically converts light that has entered through a lens 11, a gain control unit 13 that determines a luminance correction gain amount based on the luminance of an image signal obtained by photoelectric conversion, A luminance correction unit 14 that performs luminance correction with a gain amount determined by the gain control unit 13 and an image signal output control unit 15 that outputs an image signal after luminance correction are provided.

  The gain control unit 13 determines the gain amount based on the average luminance of the image. The gain control unit 13 includes an average luminance calculation unit 16 that calculates the average luminance of the image, and a gain amount adjustment unit 17 that adjusts the gain amount based on the average luminance. The gain amount adjustment unit 17 adjusts the gain amount so that the average luminance of the image becomes the target luminance. However, the gain amount has an upper limit. Even if the gain amount of the upper limit value is used, the average luminance of the image may not reach the target value. In this case, the gain control unit 13 uses the set upper limit value as the gain amount.

  The average luminance calculation unit 16 may obtain the average luminance excluding a portion where intense light such as sunlight or a headlight of another vehicle is reflected. Hereinafter, this process will be described in detail. The average luminance calculation unit 16 first divides the image into a plurality of regions. FIG. 2A is a diagram illustrating an example in which a captured image is equally divided into 8 × 6 regions. The average luminance calculation unit 16 calculates average luminance in each of the 48 areas. The gain control unit 13 determines whether there is a region where the average luminance is a predetermined threshold value or more.

  In FIG. 2B, the four shaded areas indicate areas having an average luminance equal to or higher than a predetermined threshold. In this case, the average luminance calculation unit 16 obtains the average luminance of 44 areas excluding the four shaded areas, and determines the gain amount using the obtained average luminance.

  By excluding the area where the average luminance is equal to or higher than the predetermined threshold from the area for calculating the average luminance, the influence of the area where the luminance is extremely increased by intense light or the like is excluded, and an appropriate gain is obtained. The amount can be determined. As a result, it is possible to adjust the image signal to ensure the brightness necessary for image processing in an area other than an extremely high brightness area, and to perform appropriate brightness control even in a real environment where various subjects exist. is there.

  Next, the image processing apparatus 20 will be described. The image processing device 20 includes an image signal input control unit 21 that inputs an image signal transmitted from the imaging device 10, and a gain amount estimation unit that estimates a gain amount of luminance correction performed by the imaging device 10 from the received image signal. 22, a noise amount determination unit 23 that determines the amount of noise included in the image signal from the gain amount obtained by the gain amount estimation unit 22, and a driving support application 24 that processes the captured image and performs driving support. ing. The driving support application 24 is an application for supporting safe driving by, for example, detecting an obstacle or a white line from a captured image and providing it to the driver. The information provided for driving assistance may be various things as well as obstacles and white lines.

  The gain amount estimation unit 22 is connected to the storage unit 25. The storage unit 25 stores an algorithm for determining the gain amount by the gain control unit 13 of the imaging apparatus 10. The gain amount estimation unit 22 estimates the gain amount from the average luminance of the image signal using the algorithm read from the storage unit 25.

  FIG. 3 is a diagram illustrating a change in average luminance of a captured image after luminance correction. Normally, the average brightness of the captured image after brightness correction is controlled to a target value by gain control. However, if the illuminance falls below a predetermined threshold, even if the upper limit value of the gain amount is used, it cannot be corrected to the target value. Therefore, when the image signal does not reach the target average luminance, it is determined that the upper limit value is used as the gain amount. When the image signal reaches the target average luminance, it is determined that a value equal to or smaller than the upper limit value is used as the gain amount.

  As shown in FIG. 3, when the illuminance is lower than a predetermined threshold, the average luminance after luminance correction is lowered. When the gain amount is increased, noise included in the image signal is also amplified at the same rate, so that an image with much noise is obtained. In the present embodiment, the noise amount determination unit 23 determines the noise amount with reference to a place where the average luminance level decreases with the upper limit value of the gain amount as a threshold value. Specifically, the noise amount determination unit 23 determines whether there is a lot of noise based on the gain amount estimated by the gain amount estimation unit 22. Specifically, when an upper limit value is used as the gain amount, it is determined that there is a lot of noise, and when a value smaller than the upper limit value is used, it is determined that there is little noise.

  The image processing apparatus 20 described above causes a computer to execute a program having modules that realize the functions of the image signal input control unit 21, the gain amount estimation unit 22, the noise amount determination unit 23, and the driving support application 24. May be realized. Such a program is also included in the scope of the present invention.

  FIG. 4 is a diagram illustrating the operation of the image processing apparatus 20 according to the present embodiment. When the image signal transmitted from the imaging device 10 is input (S10), the image processing device 20 passes the input image signal from the image signal input control unit 21 to the gain amount estimation unit 22. The gain amount estimation unit 22 reads out an algorithm for determining the gain amount by the gain control unit 13 from the storage unit 25, and obtains the average luminance of the image signal using the read algorithm (S12). Specifically, when the gain control unit 13 determines the average luminance of the entire image, the gain amount estimation unit 22 also calculates the average luminance of the entire image, and the gain control unit 13 has a luminance greater than a predetermined threshold. When the average luminance is obtained by excluding the area, the gain amount estimation unit 22 also obtains the average luminance by excluding an area having a luminance greater than a predetermined threshold.

  From the relationship shown in FIG. 3, it can be determined whether the gain amount has reached the upper limit value based on whether the average luminance of the captured image has reached the target luminance. When the average luminance level of the captured image matches the target luminance, the gain amount estimation unit 22 does not have the upper limit value, and when the average luminance level is lower than the target luminance, the gain amount has the upper limit value. It is estimated that there is (S14).

  The noise amount determination unit 23 determines whether or not the estimated gain amount is an upper limit value (S16). If the gain amount is the upper limit value (YES in S16), it is determined that there is a lot of noise, and the image processing apparatus 20 stops image processing based on the image (S18). When the gain amount is not the upper limit value (NO in S16), it is determined that there is little noise, and the image processing apparatus 20 continues the image processing based on the image by the driving support application 24 (S20).

  As a result, the image processing device 20 does not process all the image signals input from the imaging device 10, but performs image processing only on high-quality image signals suitable for image processing. The in-vehicle camera system 1 can perform appropriate control based on image processing.

  As mentioned above, although the image processing apparatus and the vehicle-mounted camera system of the present invention have been described in detail with reference to the embodiments, the present invention is not limited to the above-described embodiments. In the above-described embodiment, the average luminance calculation unit 16 has been described as an example in which the average luminance of the image is obtained by excluding an extremely high luminance region, but the average of the image is excluded by excluding an extremely low luminance region. The luminance may be obtained.

  As described above, the present invention has an excellent effect that the noise amount can be determined based on the average luminance of the image signal input from the imaging device without acquiring the data regarding the noise amount from the imaging device. It is useful as an image processing apparatus used in an in-vehicle camera system.

DESCRIPTION OF SYMBOLS 1 Car-mounted camera system 10 Imaging device 11 Lens 12 Image sensor 13 Gain control part 14 Brightness correction part 15 Image signal output control part 16 Average brightness calculation part 17 Gain amount adjustment part 20 Image processing apparatus 21 Image signal input control part 22 Gain amount estimation Unit 23 noise determination unit 24 driving support application 25 storage unit

Claims (8)

An image signal input unit that inputs an image signal after the luminance correction from an imaging device that picks up an image and performs luminance correction on the image obtained by the imaging;
A storage unit storing an algorithm for determining a gain amount of luminance correction in the imaging device;
A gain amount estimation unit that estimates a gain amount of luminance correction applied to the image signal based on an average luminance of the image signal input to the image signal input unit and an algorithm stored in the storage unit;
A noise amount determination unit that determines whether or not there is a lot of noise based on the gain amount estimated by the gain amount estimation unit;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the noise amount determination unit determines that there is a lot of noise when the gain amount estimated by the gain amount estimation unit is a maximum settable gain amount.   The image processing apparatus according to claim 2, wherein when the noise amount determination unit determines that there is a lot of noise, image processing based on the image signal is not performed. An imaging device for imaging the periphery of the vehicle;
An image processing device according to any one of claims 1 to 3,
With
The imaging device
A lens,
An image sensor that photoelectrically converts light input through the lens;
A gain control unit that determines a gain amount according to the luminance of an image signal obtained by photoelectric conversion;
A luminance correction unit that corrects the luminance of the image signal with the gain amount determined by the gain control unit;
An output unit for outputting a luminance-corrected image signal;
An in-vehicle camera system. The gain controller is
An average luminance calculator for calculating the average luminance of the image signal;
A gain amount adjusting unit that adjusts the gain amount so that the average luminance of the image signal becomes a target value based on the average luminance;
With
The in-vehicle camera system according to claim 4, wherein the gain amount set by the gain amount adjustment unit has an upper limit. The average luminance calculation unit
When the image is divided into a plurality of regions, the average luminance of each divided region is obtained, and there is a region where the average luminance is equal to or higher than the first threshold or a region where the average luminance is equal to or lower than the second threshold The in-vehicle camera system according to claim 5, wherein an average luminance of an area excluding the area is calculated. Inputting an image signal after the brightness correction from an imaging device that picks up an image and performs brightness correction on the image obtained by the imaging;
Reading out an algorithm for determining a gain amount of luminance correction in the imaging device from the storage unit;
Estimating a gain correction gain amount applied to the image signal based on an average luminance of the input image signal and an algorithm read from the storage unit;
Determining whether there is a lot of noise based on the estimated gain amount;
An image noise determination method comprising: In order to determine the amount of noise of an image signal acquired from an imaging device that captures an image and performs luminance correction on the image obtained by imaging,
Inputting an image signal from the imaging device;
Reading out an algorithm for determining a gain amount of luminance correction in the imaging device from the storage unit;
Estimating a gain correction gain amount applied to the image signal based on an average luminance of the input image signal and an algorithm read from the storage unit;
Determining whether there is a lot of noise based on the estimated gain amount;
A program that executes