JP2019029784A - Infrared light cutoff filter control means in imaging system - Google Patents

Abstract

To provide an imaging device that can determine whether illuminance is high due to the lighting of an infrared illuminator, whether illuminance is high due to the brightness of the outside, and whether switching to the day mode is performed when the illuminance is high during the night mode in a configuration in which the imaging device cannot control the infrared illuminator.SOLUTION: An imaging device 100 having an infrared light cutoff filter insertable into and removable from an optical path includes first imaging means for performing imaging by inserting an infrared light cutoff filter, second imaging means for performing imaging by inserting or removing an infrared light cutoff filter, brightness analyzing means for analyzing the brightness of an image captured by each of the first and second imaging means for each region, and ambient light detection means for detecting the infrared light component of ambient light from the brightness information obtained by the brightness analyzing means. Switching is performed between the first imaging means and the second imaging means according to the detection result of the ambient light detection means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging system, and more particularly to a method for changing a photographing wavelength band by infrared light cut filter control or the like in a surveillance camera.

  Since a color image can obtain color information as well as information on shape and brightness, identification of a photographing object is remarkably superior to a monochrome image. For this purpose, color images are usually taken in cameras for surveillance purposes. However, imaging devices widely used in cameras for monitoring purposes have sensitivity not only to light in the visible range (λ = about 400 nm to about 700 nm) but also to light in the infrared range (λ = about 700 nm or more). Therefore, if nothing is done, the color balance of the photographed color image will be out of order, and the color of the subject cannot be reproduced correctly. Therefore, when photographing a color image, an infrared light cut filter for cutting infrared light is arranged in the incident light path of the photographing light, and the wavelength band of the light incident on the image sensor is selected.

  By the way, when shooting very dark subjects such as outdoors at night or indoors where there is no light, the amount of incident light is insufficient, so the image signal output from the image sensor is amplified and the subject is clearly captured. This technique is generally used.

  However, when the color image signal is greatly amplified, the S / N of the color signal is deteriorated and the image becomes noisy. On the contrary, the contour and shape of the subject are unclear. Also, under low illuminance, the color information is relatively low and the value of the color image is relatively low. Therefore, when shooting under low illuminance, in order to capture the subject more clearly and brightly, the infrared light cut filter is removed from the optical path, the infrared light is taken in, and the color information that deteriorates the S / N is discarded and black and white. Output an image.

  There is also known a technique (hereinafter referred to as “auto day / night”) in which the above infrared light cut filter is automatically inserted / removed in accordance with subject illumination. This is because when the subject illuminance decreases, the infrared light cut filter is automatically removed and switched to the black and white mode (hereinafter referred to as night mode), and when the subject illuminance increases, the infrared light cut filter is inserted into the color mode. (Hereinafter referred to as day mode). The object illuminance is detected based on, for example, an image signal output of an image sensor or an output of a brightness sensor in which an image sensor having sensitivity only to visible light is provided separately from the photographing optical system.

  In Patent Document 1, a reference light amount for returning from the night mode to the day mode is determined according to the light amount immediately after switching from the day mode to the night mode. Disclosed is a technology that reduces the influence of the environment by setting the reference light amount to a value corresponding to the imaging environment light at the time of switching, and performs auto day / night switching from night mode to day mode with a configuration without the brightness sensor. Has been.

JP 2004-120202 A

  There is a case in which an infrared light illuminator is arranged in the vicinity of a monitoring camera as photographing auxiliary light under low illumination. A general infrared light illuminator is turned on when the external illuminance (by visible light) is low, and turned off when the illuminance is high. In the night mode, when the illuminance is high, it is necessary to determine whether to switch to the day mode by determining whether the infrared light illuminator is bright or the outside is bright.

  In the prior art disclosed in Patent Document 1 described above, in a configuration in which the imaging apparatus cannot control the infrared illuminator, there is a problem that the above determination is incorrect and the mode cannot be switched appropriately.

  It is assumed that the infrared light illuminator is turned on immediately after switching from the day mode to the night mode, but this does not hold in a configuration in which the imaging device cannot control the infrared light illuminator, and the mode is switched from the day mode to the night mode. Immediately after that, infrared light irradiation is not always performed. When the infrared illuminator is turned on after a lapse of a certain time after transitioning to the night mode, it is erroneously recognized that the outside is bright at the lighting timing, and the mode is switched to the day mode. Since the judgment is wrong (the outside world is still dark), hunting occurs again to return to the night mode. When hunting occurs, there is a large change in the image, and an increase in time until convergence in an automatic system function (AE, AWB, AF) of image quality control generally mounted in a camera function for surveillance purposes, an error such as a moving object detection function, etc. There is a problem of adverse effects such as an increase in detection and distribution video data.

  In addition, if a strong light (such as a car headlight) other than the infrared light illuminator is irradiated immediately after switching from the day mode to the night mode, it becomes difficult to return to the day mode and the outside world is bright. Since it will remain in the night mode more than necessary, the display period of the monochrome image is extended and the visibility of the object to be photographed is lowered.

  Moreover, even if infrared light irradiation is made immediately after switching from the day mode to the night mode, the infrared light illuminator does not always change from the extinguished state to the maximum intensity lit state. In the case of an infrared light LED whose emission intensity can be changed stepwise and smoothly by PWM control, the problem of hunting occurs in the same way as described above.

In order to solve the above problems, an imaging apparatus according to the present invention provides:
An image pickup apparatus having an infrared light cut filter that can be inserted into and removed from an optical path, wherein the first image pickup means for picking up an image by inserting the infrared light cut filter, and removing the infrared light cut filter A second imaging unit that captures images, a brightness analysis unit that analyzes the brightness of each of the images captured by each of the second imaging units, and brightness information obtained by the brightness analysis unit. And an ambient light detection unit that detects an infrared light component, and the first imaging unit and the second imaging unit are switched according to a detection result of the ambient light detection unit.

  According to the imaging apparatus of the present invention, even in a configuration in which the imaging system cannot control the infrared light illuminator, the night mode and the day mode can be performed at an appropriate timing without hunting by using the spatial brightness distribution information in the captured image. Auto day and night can be realized.

The figure explaining the whole structure of an imaging system 7 is a flowchart of learning processing executed by the imaging system according to the first embodiment. Flowchart of execution processing executed by the imaging system according to the first embodiment Flow chart of night mode processing in FIG. Flowchart of learning completion processing in FIG. Flowchart of learning incomplete processing in FIG. Flow chart of day mode processing in FIG. Arrangement example of imaging device and infrared illumination device at ceiling installation Example of image taken in day mode in FIG. Example of night mode image in FIG. Arrangement example of imaging device and infrared illumination device at the time of wall installation Example of night mode image in FIG. Example of night mode image at the time of high installation in FIG. Example of night mode image taken at high altitude in FIG. Example of night mode image including disturbance light in FIG. Example of color filter output image with low sensitivity to infrared light in FIG. Example of color filter output image with high sensitivity to infrared light in FIG.

  Hereinafter, the best embodiment of the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[overall structure]
First, the overall configuration of an imaging system including an infrared light cut filter control unit will be described with reference to FIG.

  The imaging system including the infrared light cut filter control unit in the present embodiment includes an imaging device 100, a display unit 101, an input unit 102, and an infrared light illumination device 103. The imaging apparatus 100 includes an imaging lens 100-1, an imaging element 100-2, an imaging element control unit 100-3, a video signal processing unit 100-4, a video signal distribution unit 100-5, a photometry unit 100-6, and a motor unit 100. -7, CPU 100-8, I / F unit 100-9, infrared light cut filter control unit 100-10, and storage unit 100-11. The infrared light illumination device 103 includes an infrared light LED 103-1, an infrared light LED control unit 103-2, a CPU 103-3, a storage unit 103-4, a visible light sensor 103-5, and a visible light sensor control unit 103-6. Have

  The imaging lens 100-1 is an optical device for imaging a subject. The imaging element 100-2 receives incident light that has passed through the imaging lens 100-1 and generates an image signal. The image sensor control unit 100-3 controls the amplification amount and the exposure time of the image signal of the image sensor 100-2. The video signal processing unit 100-4 performs signal processing on the video such as color component gain adjustment (color is changed to monochrome) on the generated image signal. The video signal distribution unit 100-5 is a distribution unit that converts an image signal into a specific image compression format (for example, JPEG, H.264) and transmits the image to the outside.

  The photometric unit 100-6 performs photometry on the generated image signal in the area designated by the CPU 100-8. The motor unit 100-7 is a filter driving device that switches an infrared light cut filter. The CPU 100-8 is an arithmetic processing unit that gives instructions to surrounding blocks based on the control flow. The I / F unit 100-9 is a notification unit that transmits an instruction received from the input unit 102 outside the imaging apparatus 100 to the CPU 100-8. The infrared light cut filter control unit 100-10 performs insertion / extraction control of the infrared light cut filter according to an insertion / extraction instruction from the CPU 100-8. The storage unit 100-11 stores the learning result. The infrared light LED 103-1 is an LED lamp that emits light having an infrared wavelength.

  The CPU 103-3 is an arithmetic processing unit that gives control instructions for the visible light sensor and the infrared light LED. The memory | storage part 103-4 memorize | stores the table of the light emission intensity | strength of infrared light LED set according to the output value of the visible light sensor set beforehand. The visible light sensor 103-5 is a brightness detection sensor that detects the brightness of light in the visible region. The visible light sensor control unit 103-6 controls the visible light sensor. The infrared illumination device 103 does not require a means for communicating with the imaging device 100. Therefore, the imaging apparatus 100 cannot acquire the output of the visible light sensor 103-5 and control the infrared light LED 103-1.

[Embodiment 1]
Embodiment 1 is an embodiment showing a method for controlling an infrared light cut filter from information on brightness difference for each region of an image captured in an imaging system. In the present embodiment, an embodiment in which an omnidirectional lens is used as the imaging lens 100-1 will be described. The processing procedure is divided into a learning processing phase and an execution processing phase. When the imaging apparatus is activated, an execution process is usually performed. When there is an instruction for learning processing from the input unit 102, learning processing is performed. First, a specific execution procedure for the learning process phase will be described below according to the flowchart of FIG.

<Step S101>
In step S101, switching to the day mode is performed. To switch to the day mode, the CPU 100-8 instructs the infrared light cut filter control unit 100-10 to insert an infrared light cut filter and instructs the video signal processing unit to colorize an image. The infrared light cut filter control unit 100-10 performs drive control of the motor unit 100-7 according to the insertion instruction, and inserts the infrared light cut filter. The video signal processing unit performs colorization of the image according to the colorization instruction.

<Step S102>
In step S102, the brightness of the image is measured by the photometry unit 100-6 in the state switched to the day mode state in step S101. If the brightness is high, the outside world is still bright, so it is determined that the timing is inappropriate for executing the learning process, and the learning process ends.

<Step S103>
In step S103, when the brightness is low in step S102, switching to the night mode is performed. To switch to the night mode, the CPU 100-8 instructs the infrared light cut filter control unit 100-10 to remove the infrared light cut filter and instructs the video signal processing unit 100-4 to make an image monochrome. The infrared light cut filter control unit 100-10 performs drive control of the motor unit 100-7 according to the removal instruction, and removes the infrared light cut filter. The video signal processing unit 100-4 performs monochrome conversion of the image according to the monochrome conversion instruction.

<Step S104>
In step S104, the infrared light irradiation region is estimated in a state where the state is switched to the night mode state in step S103. The brightness of each image area is measured, the presence or absence of a brightness difference is detected, and the presence or absence of infrared light irradiation and its area are estimated.

  The estimation method will be described below with reference to FIGS. First, each figure will be described. FIG. 8 is an arrangement example of the imaging device 801 and the infrared light illumination devices 802 and 803 when installed on the ceiling. FIG. 9 is a photographed image example in which no brightness difference occurs in the image 901 in FIG. FIG. 10 is an example of a night mode photographed image when the brightness difference in FIG. 8 occurs. The image region 1001 is a region having a relatively high illuminance by infrared light irradiation, and the image region 1002 has a relatively illuminance by infrared light irradiation. The low region, the image region 1003, is a region that is not irradiated with infrared light and has extremely low illuminance.

  As shown in FIG. 10, when the infrared LED is turned on and the infrared light irradiates a part within the angle of view, the brightness distribution varies in the captured image. On the other hand, when the infrared LED is turned off as shown in FIG. 9, a difference in brightness is unlikely to occur in the image. The photometry unit 100-6 acquires the brightness difference for each region, thereby detecting the presence or absence of the brightness difference and estimating the presence or absence of the infrared light irradiation and the region. It should be noted that the detection range of the area may be narrowed instead of setting the front area in the image as the detection target using the installation direction information of the imaging device and the infrared light illumination device. In the case of the ceiling installation as shown in FIG. 8, the detection range of the region may be narrowed by using the infrared light irradiation at the center of the image as shown in FIG. The case of wall installation will be described below with reference to FIGS. First, each figure will be described.

  FIG. 11 is an arrangement example of the imaging device 1101 and the infrared light illumination device 1102 at the time of ceiling installation. FIG. 12 is an example of a night mode image when the brightness difference in FIG. 11 occurs. The image region 1201 is a region with relatively high illuminance by infrared light irradiation, and the image region 1202 is relatively illuminance by infrared light irradiation. The low image area, the image area 1203, is not irradiated with infrared light, and is an area with extremely low illuminance.

  In the case of wall installation as shown in FIG. 11, the detection range of the region may be narrowed by using infrared light irradiation at the lower part of the image as shown in FIG. Note that the installation height information of the imaging device and the infrared light illuminating device may be used to limit the detection range of the region instead of setting the front region in the image as a detection target.

  FIG. 10 is an example of a photographed image when the installation position is higher than that in FIG. 13. The image region 1301 is a region having a relatively high illuminance by infrared light irradiation, and the image region 1302 is a relatively illuminance by infrared light irradiation. The low image area, the image area 1303, is an area that is not irradiated with infrared light and has extremely low illuminance.

  FIG. 14 is an example of a photographed image when the installation position is low compared to FIG. 12. The image region 1401 is a region with relatively high illuminance by infrared light irradiation, and the image region 1402 is relatively illuminance by infrared light irradiation. The low image area, the image area 1403 is an area that is not irradiated with infrared light and has extremely low illuminance. The detection range of the region may be limited using the fact that the irradiation range changes according to the height of installation. In an environment where ambient light is radiated, the spectral sensitivity of each color filter of the image sensor is used, and each image of a color filter that is highly sensitive to infrared light and a color filter that is sensitive to infrared light. May be used to increase the accuracy of identification of the infrared light irradiation region.

  FIG. 15 is an example of a night mode photographed image including disturbance light in FIG. A region 1501 is an infrared light irradiation region, a region 1502 is a disturbance light region, a region 1503 is a region with extremely low illuminance without infrared light irradiation and no disturbance light, and a region 1504 is a region where infrared light irradiation and disturbance light overlap. .

  FIG. 16 is an image of a color filter having low sensitivity to infrared light, and FIG. 17 is an image of a color filter having high sensitivity to infrared light. Regions 1602 and 1702 are regions with extremely low illuminance. A region 1601 corresponds to the regions 1502 and 1504 and can be identified as ambient light. A region 1701 corresponds to the regions 1501 and 1504 and can be identified as infrared illumination light. By using the region 1501 and the region 1503 for brightness difference calculation, the influence of ambient light may be reduced. If there is a moving area in the image, for example, if an object passes by the imaging device, the brightness distribution changes and becomes a disturbance, so the moving area is removed from the infrared light irradiation area, and the influence of moving objects is reduced. It may be reduced. Note that the accuracy of identification of the infrared light irradiation region may be increased from a focus shift due to axial chromatic aberration.

<Step S105>
In step S105, if it is determined that there is no difference in brightness as a result of the estimation in step S104, the infrared LED may not be lit yet, or the infrared light may be radiated uniformly throughout the field angle. Therefore, it is determined that the timing is inappropriate for executing the learning process, and the learning process is terminated.

<Step S106>
In step S103, a lightness difference threshold value is calculated from the infrared light irradiation region estimated in step S104 and the lightness difference during infrared light lighting, and stored in the storage unit 100-11.

  By performing this flow, information for distinguishing between the infrared light irradiation region and the non-irradiation region is stored in the storage unit 100-11, and the control accuracy in the execution processing phase can be improved. Note that the learning process is performed not only when the user instructs the external input unit 102 but also when the output of the video signal output unit 100-5 is stopped and no video is presented to the user on the display unit 101. May be.

  Next, regarding the execution processing phase, a specific execution procedure will be described below according to the flowchart of FIG.

<Step S201>
In step S201, it is determined whether auto day / night is valid or invalid. If invalid, the execution process ends.

<Step S202>
In step S202, it is determined whether or not the current day / night mode is the night mode when auto day / night is valid in step S201.

<Step S203>
In step S203, if the night mode is determined in step S202, the night mode process is executed.

<Step S204>
In step S204, if the night mode is not determined in step S202, a day mode process is executed.

  By implementing this flow, it is possible to realize auto day / night switching between the night mode and the day mode.

  Next, a specific execution procedure for the night mode processing in the execution processing phase will be described below according to the flowchart of FIG.

<Step S301>
In step S301, it is determined whether learning is correctly completed in the learning process phase.

<Step S302>
In step S302, when learning is completed in step S301, a learning process is executed.

<Step S303>
In step S303, if learning is not completed in step S301, an unlearned process is executed.

  Next, a specific execution procedure for the learning process of the day mode process in the execution process phase will be described below according to the flowchart of FIG.

<Step S401>
In step S401, a photometric area is set for the infrared light irradiation area and the non-irradiation area held in the storage unit 100-11.

<Step S402>
In step S402, the brightness difference between the irradiated area measured in step S401 and the non-irradiated area is calculated.

<Step S403>
In step S403, the brightness difference calculated in step S402 is determined to be larger or smaller than the brightness difference threshold held in the storage unit 100-11. If the brightness difference is large, it is determined that the infrared illumination is turned on, and the process is terminated while the night mode is not switched to the day mode.

<Step S404>
In step S404, when it is determined in step S403 that the brightness difference is small, it is determined that the infrared illumination is off, and the mode is switched to the day mode. Switching to the day mode is the same process as step S101, and since it overlaps, detailed description is omitted.

  By implementing this flowchart, even in a configuration where the imaging device cannot control the infrared light illuminator, from the night mode to the day mode at an appropriate timing without hunting by using the spatial brightness distribution information in the captured image. Can be switched.

  Next, a specific execution procedure for the unlearned process of the night mode process in the execution process phase will be described below according to the flowchart of FIG.

<Step S501>
In step S501, if the day / night mode is hunting, the threshold is updated.

<Step S502>
In step S502, photometry is performed to obtain brightness.

<Step S503>
In step S503, it is determined whether the brightness measured in step S502 is higher than a brightness threshold. If it is low, it is determined that the external environment is still dark, and the processing is terminated while the night mode is not switched to the day mode.

<Step S504>
In step S504, if it is determined in step S503 that it is higher than the lightness threshold, it is determined whether the time threshold has been exceeded. If not, the process is terminated in the night mode without switching to the day mode in order to suppress hunting.

<Step S505>
In step S505, when the time threshold is exceeded in step S505, the mode is switched to the day mode. Switching to the day mode is the same process as step S101, and since it overlaps, detailed description is omitted.

  By implementing this flowchart, even in a configuration where the imaging system cannot control the infrared light illuminator, compared with the state where learning is completed, the hunting is suppressed in exchange for the switching response, and the day mode is changed from the night mode. Can be switched to.

  Next, a specific execution procedure for the day mode processing in the execution processing phase will be described below according to the flowchart of FIG.

<Step S601>
In step S601, if the day / night mode is hunting, the threshold is updated.

<Step S602>
In step S602, photometry is performed to obtain brightness.

<Step S603>
In step S603, it is determined whether the brightness measured in step S602 is lower than the brightness threshold. If it is high, it is determined that the outside world is still bright, and the process is terminated while the day mode is not switched to the night mode.

<Step S604>
In step S604, if it is determined in step S603 that it is higher than the brightness threshold, it is determined whether the time threshold has been exceeded. If not exceeded, the process is terminated in the day mode without switching to the night mode in order to suppress hunting.

<Step S605>
In step S605, when the time threshold is exceeded in step S605, switching to the night mode is performed. Switching to the night mode is the same processing as step S103, and since it overlaps, detailed description is omitted.

  By executing this flowchart, the day mode can be switched to the night mode.

  As described above, by configuring as in the present embodiment, it is possible to appropriately change the mode from the night mode to the day mode even in a configuration in which the imaging device cannot control the infrared light illumination device. Further, in addition to the above, in a case where a plurality of cameras capture a close area, it is possible to appropriately change the mode from the night mode to the day mode even for a camera in which no lighting device is installed. Therefore, it is not necessary to install an illumination device using infrared light in all the cameras, and not only the cost of introduction but also the problem of image quality due to excessive illumination by a plurality of illuminations is eliminated.

100 imaging device, 100-1 imaging lens, 100-2 imaging device,
100-3 image sensor control unit, 100-4 video signal processing unit,
100-5 video signal distribution unit, 100-6 photometry unit, 100-7 motor unit,
100-8 CPU, 100-9 I / F part,
100-10 infrared light cut filter control unit, 100-11 storage unit,
101 display unit, 102 input unit, 103 infrared light illumination device,
103-1 infrared LED, 103-2 infrared LED controller,
103-3 CPU, 103-4 storage unit, 103-5 visible light sensor,
103-6 Visible Light Sensor Control Unit

Claims (12)

An imaging device having an infrared light cut filter that can be inserted into and removed from an optical path,
First imaging means for imaging by inserting the infrared light cut filter;
A second imaging means for performing imaging by removing the infrared light cut filter;
Brightness analysis means for analyzing the brightness of an image captured by each of the second imaging means for each region;
Ambient light detection means for detecting an infrared light component of ambient light from the brightness information obtained by the brightness analysis means;
Have
An image pickup apparatus that switches between the first image pickup means and the second image pickup means in accordance with a detection result of the ambient light detection means. The ambient light detection means further includes learning means for detecting ambient light, and the brightness analysis result of the second imaging means learned in advance and the brightness analysis result of the second imaging means during normal operation are obtained. The imaging apparatus according to claim 1, wherein the infrared light component of the ambient light is detected by comparison. The imaging apparatus according to claim 2, wherein the learning unit further has an input unit and can instruct execution timing of learning. The imaging apparatus according to claim 2, wherein the learning unit further includes an imaging signal distribution unit and can perform learning when the distribution is stopped. The ambient light detection means has a first color filter that is highly sensitive to light in the infrared region and a second color filter that is highly sensitive to light in the infrared region, respectively, and an output signal from each color filter. The imaging apparatus according to claim 1, wherein an infrared light component of ambient light is detected from the difference between the two. The imaging apparatus according to claim 1, wherein the ambient light detection unit increases the accuracy of detecting an infrared light component by excluding a moving subject region. The imaging apparatus according to claim 1, wherein the ambient light detection unit enhances the accuracy of detecting the infrared light component by excluding the disturbance light region. The imaging apparatus according to claim 1, wherein the ambient light detected by the ambient light detection unit is an infrared illumination device that is controlled independently of the imaging apparatus. 2. The ambient light detection means has installation information of the infrared light illumination device and the imaging device, and detects an infrared light component of ambient light from the infrared light irradiation angle and angle of view. The imaging device described in 1. The imaging apparatus according to claim 1, wherein when learning by the learning unit is not completed, hunting is suppressed by increasing brightness threshold hysteresis. The imaging apparatus according to claim 1, wherein the time threshold hysteresis is increased when learning by the learning unit is not completed. The ambient light detection means includes a focus focusing degree detection means, and increases the accuracy of identification of the infrared light irradiation area from the shift of the focus surface due to axial chromatic aberration due to the infrared light irradiated from the infrared light illumination device. The imaging apparatus according to claim 1.