JP2016220002A - Imaging apparatus, method for controlling the same, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve an imaging apparatus capable of automatically photographing an optimum image, according to a time zone and a subject, while preventing halation caused by a high-luminance subject with an inexpensive and simple configuration.SOLUTION: The imaging apparatus includes: imaging means for capturing an image; first filter means for cutting the infrared light component of subject image light guided to the imaging means; second filter means for cutting the visible light component of the subject image light guided to the imaging means; subject detection means for detecting a predetermined subject from the image captured by the imaging means; and control means for determining whether or not the luminance of the subject detected by the subject detection means is larger than a predetermined threshold and for performing control, so as to insert/remove the first filter means and the second filter means into/from an optical path for guiding the subject image light to the imaging means, according to a determination result.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an imaging apparatus used for a surveillance camera or the like.

  When night photography is performed with a conventional surveillance camera, visible light such as a vehicle headlight may fall within the field of view of the surveillance camera. The outline of the car body, the license plate, and the driver's seat of the vehicle are basically dark areas because they are not illuminated by headlights. However, since the headlight is a very bright light source, if it falls within the angle of view of the camera, a halation phenomenon may occur in which a part of the image is whitened due to the influence.

  When halation occurs, the license plate or driver's face is covered with the whiteout area due to the headlights and cannot be recognized. An image cannot be obtained. In a time zone such as dawn or evening, a color image can be acquired by amplifying the image signal read from the image sensor. However, when there is a vehicle with a headlight turned on, whiteout occurs due to the headlight as in night photography, or control is performed so that whiteout does not occur, and other than the headlight portion becomes dark. Thus, when a high-luminance subject exists in the image, it is necessary to change the setting to an optimum value according to the time zone and the subject.

  In order to cope with such a problem, Japanese Patent Application Laid-Open No. H10-228561 describes a technique for performing photographing at a plurality of shutter speeds when a sudden luminance change occurs in a part of the screen. Japanese Patent Application Laid-Open No. 2004-228561 describes a technique in which two optical sensors for color and monochrome are provided in one optical lens and individual shutter control is performed according to the subject. In addition, there is a high dynamic range (HDR) technique that captures images of bright and dark subjects with exposure adjusted and combines them to obtain an image with a wide dynamic range.

JP 2009-302763 A JP 2010-250710 A

  However, in Patent Document 1, halation cannot be prevented only by changing the shutter speed, and a clear image cannot be obtained in a dark region. Moreover, although the said patent document 2 obtains a desired image, since several image sensors are required, it is disadvantageous in terms of cost. Also, the configuration of the HDR technology is complicated, and the memory and the like are increased, which is similarly disadvantageous in terms of cost.

  The present invention has been made in view of the above problems, and an object thereof is to realize an imaging apparatus that can automatically capture an optimal image according to a time zone and a subject while preventing whiteout due to a high-luminance subject with a low-cost and simple configuration. It is to be.

  In order to solve the above-described problems and achieve the object, an imaging apparatus of the present invention includes an imaging unit that captures an image, and a first filter that cuts an infrared light component of subject image light guided to the imaging unit. Means, second filter means for cutting a visible light component of subject image light guided to the imaging means, subject detection means for detecting a predetermined subject from an image taken by the imaging means, and subject detection It is determined whether the luminance of the subject detected by the means is greater than a predetermined threshold, and the first filter means and the second filter are arranged on the optical path for guiding the subject image light to the imaging means according to the determination result. Control means for controlling the filter means to be inserted and removed.

  According to the present invention, it is possible to realize an imaging apparatus that can automatically capture an optimal image according to a time zone and a subject while preventing whiteout due to a high-luminance subject with a low-cost and simple configuration.

The block diagram which shows the apparatus structure of embodiment which concerns on this invention. The block diagram which shows the structure of the signal processing unit of this embodiment. The figure explaining the high-intensity subject detection method by this embodiment. FIG. 6 is a diagram for explaining a movement detection method for a high-luminance subject according to the present embodiment. 5 is a flowchart illustrating filter insertion / extraction control according to the first embodiment. 9 is a flowchart showing filter insertion / extraction control according to the second embodiment. 9 is a flowchart showing filter insertion / extraction control according to the third embodiment.

  Hereinafter, embodiments for carrying out the present invention will be described in detail. The embodiment described below is an example for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment. Moreover, you may comprise combining suitably one part of each embodiment mentioned later.

  Hereinafter, an embodiment in which the imaging apparatus of the present invention is applied to, for example, a digital video camera (hereinafter referred to as a digital camera) that captures a still image or a moving image will be described, but the present invention can also be applied to other surveillance cameras.

  <Apparatus Configuration> With reference to FIGS. 1 and 2, an outline of the configuration and functions of a digital camera according to an embodiment of the present invention will be described.

  In FIG. 1, the digital camera 100 includes a lens unit 101, an image sensor 201, a signal processing unit 301, and an infrared illumination 401.

  The subject image light formed by the lens unit 101 is converted into an electrical signal by the image sensor 201, and then subjected to signal processing by the signal processing unit 301, which is sequentially output to the outside through a network or the like as a moving image or a still image. Is done.

  The lens unit 101 includes a zoom lens 102, a focus lens 103, a diaphragm 104, a filter 105, and the like as an optical system, and is controlled based on a control signal from the signal processing unit 301 so that an optimal image can be obtained according to the subject. Is done. The filter 105 is provided on the optical path between the lens unit 101 and the image sensor 201 and is detachable on the optical axis 106 of the lens unit 101. The filter 105 includes an infrared cut filter 111 and a visible light cut filter 112. The infrared cut filter 111 mainly cuts the infrared light component of the light that passes through the lens unit 101 and enters the image sensor 201 and transmits the visible light component. The visible light cut filter 112 mainly cuts the visible light component of the light incident on the image sensor 201 through the lens unit 101 and transmits the infrared light component. These filters 111 and 112 can independently control the insertion / extraction of the lens unit 101 on the optical axis 106 based on a control signal from the signal processing unit 301. And when one filter is in the insertion position with respect to the optical axis 106, it is controlled exclusively so that the other filter is in the retracted position. Further, it is possible to control so that none of the filters is located on the optical axis 106.

  The image sensor 201 includes a semiconductor integrated circuit chip in which pixels including photoelectric conversion elements such as a CCD and a CMOS are arranged in a matrix. The image sensor 201 has high sensitivity mainly to visible light (wavelength 380 to 780 nanometers), and high sensitivity to any one of red (R), green (G), and blue (B) for each pixel. However, it has some sensitivity to infrared rays (780 nanometers or more). Therefore, for a subject that is bright in infrared rays such as a time zone in which sunlight is present and a place illuminated by infrared illumination, the subject can be clearly imaged even when the visible light cut filter 112 is inserted. The digital camera 100 of the present embodiment further includes an infrared illumination 401 for projecting infrared rays, and an infrared image of the subject can be acquired by irradiating the infrared illumination 401 toward the subject and performing imaging.

  <Configuration of Signal Processing Unit> Next, the internal configuration of the signal processing unit 301 will be described with reference to FIG.

  In FIG. 2, an image signal obtained by capturing the subject image light by the image sensor 201 is subjected to image processing by the image processing unit 211 of the signal processing unit 301, and a JPEG format still image file or H.264 image is processed. A moving image file encoded by H.264 or the like is generated. The image processing performed by the image processing unit 211 includes, for example, processing such as exposure control, white balance control, and gamma correction, and the signal processing unit 301 performs image processing so that an optimal image is generated according to the subject. The unit 211 is controlled. In addition, the signal processing unit 301 controls the zoom lens 102, the focus lens 103, the diaphragm 104, and the filter 105 of the lens unit 101 by the lens unit control unit 212 so that an optimal image is generated by the image processing unit 211. . The signal processing unit 301 simultaneously controls the gain and shutter speed of the image sensor 201 by the imaging control unit 213. For example, when the subject is bright, control is performed so that the aperture 104 is stopped, the shutter speed is shortened, and the sensor gain is decreased. In this case, the infrared cut filter 111 is controlled to a position inserted on the optical axis 106 of the lens unit 101, and the infrared component of the subject image light is cut to improve the color reproducibility of the captured image. When the subject becomes dark, control is performed so that the aperture 104 is opened, the shutter speed is increased, and the sensor gain is increased. Further, a brighter image can be obtained by retracting the infrared cut filter 111 from the optical axis 106 of the lens unit 101 and imaging light including infrared rays. In this case, since the color reproducibility of the image deteriorates when infrared rays are incident on the image sensor 201, the white balance control by the image processing unit 211 is usually changed to output a monochrome image instead of a color image. The signal processing unit 301 determines the insertion / extraction timing of the infrared cut filter 111 by estimating the illuminance of the subject using the control results of the lens unit 101, the image sensor 201, and the image processing unit 211.

  The signal processing unit 301 further includes an area luminance calculation unit 214 and a determination unit 215. The region luminance calculation unit 214 divides the image generated by the image processing unit 211 into a plurality of regions having a predetermined size, and calculates a luminance average value for each of the divided regions. The determination unit 215 compares the luminance average value for each region calculated by the region luminance calculation unit 214 with a reference value L0 (for example, 250), and if it is larger than the reference value L0 set as a predetermined threshold, It is determined that there is a high brightness subject. Thereby, it is possible to detect whether or not a high brightness subject such as a headlight of a vehicle exists in the image. Detailed operation will be described later.

  The signal processing unit 301 further includes a subject detection unit 216. The subject detection unit 216 performs subject detection processing on the image data generated by the image processing unit 211 by a known method such as pattern matching to extract feature amounts, and detects a predetermined subject existing in the image. . The predetermined subject is, for example, the face of a person such as a vehicle license plate, a vehicle driver, or a passerby. The detection results by the subject detection unit 216 are stored as additional information related to the feature amount of the subject in association with the image file or stored in the memory 219 inside the signal processing unit 301.

  The memory 219 may be a recording medium such as a hard disk drive or a memory card that can be built in or externally attached to the digital camera. Further, the memory 219 may be configured to store a moving image or a still image that is output externally.

  The signal processing unit 301 further includes a communication control unit 217. The communication control unit 217 outputs the image data generated by the image processing unit 211 to the outside via a wireless or wired network. In this case, the luminance average value as the calculation result by the region luminance calculation unit 214, the determination result by the determination unit 215, or the feature amount of the subject as the detection result by the subject detection unit 216 may be output as control information. Furthermore, as the control status of each block of the digital camera, the insertion / extraction state of the filter 105 may be output as control information. Note that control signals for each block of the digital camera may be received from an external device such as a personal computer via the communication control unit 217.

  The signal processing unit 301 further includes a communication control unit 217. The communication control unit 217 outputs the image data generated by the image processing unit 211 to the outside via a wireless or wired network. In this case, the feature amount of the subject as a detection result by the subject detection unit 216 may be output simultaneously.

  The signal processing unit 301 further includes an illumination control unit 218. The illumination control unit 218 performs turning on / off of the infrared illumination 401 and dimming control. When the infrared illumination 401 is turned on, the lens unit control unit 212 retracts the infrared cut filter 111 from the optical axis 106. The signal processing unit 301 is connected to the above-described units and the memory 219 via the bus 220, and controls various units by exchanging various data.

  Note that one or more of the units shown in FIG. 2 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a processor such as a CPU or MPU executing software. Good. Further, it may be realized by a combination of software and hardware. Therefore, in the following, even if different functional blocks are described as the operation subject, the same hardware can be realized as the subject.

  <Region Luminance Calculation Unit and Determination Unit> Next, processing by the region luminance calculation unit 214 and the determination unit 215 of the signal processing unit 301 will be described with reference to FIG.

  Each pixel value of the image data generated by the image processing unit 211 includes light amount information received by each pixel as luminance information. FIG. 3A shows an image of the subject generated by the image processing unit 211 when the vehicle V, which is one of the subjects, turns on the headlight. For this image, the area luminance calculation unit 214 first divides the image into a plurality of areas in an N × M matrix as shown in FIG. 3B, and calculates the average value of the luminance from the pixel values in each area. To do. N and M are positive integer values, respectively, and are selected so that the divided areas are square or rectangular. Although the values of N and M vary depending on the size of the image, it is desirable to select such that one side of the divided area is about several tens to several hundreds of pixels. The values of N and M vary depending on the angle of view of the camera, and when shooting at a wide angle, the subject often appears small in the screen, so a smaller value is preferable. On the other hand, when shooting on the telephoto side, the subject is often reflected in a large size, so a larger value is preferable.

  For each area divided as described above, the average luminance value at the i-th position in the horizontal direction and the j-th position in the vertical direction (i, j) is expressed as Lij (i = 1, 2,... N, j = 1, 2,. M). Thereafter, the determination unit 215 compares the luminance average value Lij of each region with a predetermined reference value L0. When the luminance value of the compared region is brighter than the reference value L0, the position (i, It is determined that a high brightness subject exists in j). As the reference value L0, a value when the luminance value of a pixel in the image is saturated or a value close thereto is used, and when the luminance value is 8 bits, a decimal value of about 200 to 255 is used. Is preferred.

  When the determination unit 215 determines that a high brightness subject exists, the signal processing unit 301 controls the lens unit control unit 212 to insert the visible light cut filter 112 on the optical axis 106 of the lens unit 101. In this way, it is possible to obtain an image of only the infrared component by cutting the visible light component of the subject image light. In this embodiment, the headlight of the vehicle V that is assumed as a high-luminance subject has recently been used in many cases that emits a lot of visible light, such as an LED or an HID lamp, in terms of power consumption and light emission efficiency. It has become. Such a light source is not as bright as visible light in the infrared wavelength region, so that whiteout of an image hardly occurs. In such a situation, when the infrared cut filter 111 is inserted on the optical axis 106 of the lens unit 101, the infrared wavelength region of the subject image light is not incident on the image sensor 201. For this reason, the infrared cut filter 111 is retracted from the optical axis 106 almost simultaneously with the insertion timing of the visible light cut filter 112.

  Note that if the visible light cut filter 112 is frequently inserted and removed, the brightness and white balance change each time, and the appearance of the monitoring image deteriorates. Therefore, control may be performed so that the subject detection unit 216 inserts and removes only when a predetermined subject is detected.

  FIGS. 4A1 and 4A2 illustrate a case where a light source such as a streetlight G exists in the captured image. FIGS. 4B1 and 4B2 illustrate FIGS. 4A1 and 4A2. 2 illustrates a method of determining a high-intensity subject from the images. In the images of FIGS. 4 (a1) and (a2), assuming that the determination unit 215 determines that only the headlights of the vehicles V1 and V2 are high-luminance subjects, a plurality of images are continuously captured, The position information of the subject detected from the image is stored in the memory 219. Then, by detecting whether or not the position information of the subject stored in the memory 219 has changed, for example, in successive images, the vehicle V1 in FIG. 4 (a1) and the vehicle V2 in FIG. 4 (a2) Thus, it can be determined that the high brightness subject is moving.

  Next, the operation of the digital camera 100 when the visible light cut filter 112 is inserted on the optical axis 106 of the lens unit 101 will be described.

  When the visible light cut filter 112 is inserted on the optical axis 106 of the lens unit 101, only the infrared component of the subject image light is picked up, so that it is generally dark. Therefore, it is desirable to brighten the subject image by turning on the infrared illumination 401.

  When the visible light cut filter 112 is inserted, no visible light component is incident on the image sensor 201. Therefore, the brightness of the subject in the visible light wavelength region is unknown, and it is determined whether or not a high-luminance subject exists. There is a possibility that it will not be possible. Therefore, in the present embodiment, when a predetermined time (several seconds to several hundred seconds) elapses after the visible light cut filter 112 is inserted, the visible light cut filter 112 is retracted again, and the visible light component is transferred to the image sensor 201. Control to enter. Further, the visible light cut filter 112 may be retracted when a predetermined time elapses after the subject detection unit 216 no longer detects the predetermined subject. A sensor capable of detecting the illuminance of the subject, such as an illuminance sensor, may be used.

  <Description of Operation> Next, filter insertion / extraction control in the monitoring camera mode of the digital camera 100 of the present embodiment will be described with reference to FIGS. In the digital camera 100 of the present embodiment, the image processing unit 211 generates JPEG format image data of 1920 × 1080 pixels and sequentially outputs the data from the communication control unit 217.

  5 is realized by reading out a program stored in the memory 219 and executing each part of the signal processing unit 301. 5 is executed when the digital camera 100 is set to the surveillance camera mode, an image is captured, and is output to the outside. The same applies to FIGS. 6 and 7 described later.

  In step S <b> 501, the region luminance calculation unit 214 acquires the image data generated by the image processing unit 211.

  In step S502, the region luminance calculation unit 214 divides the image data obtained from the image processing unit 211 into regions of 60 × 60 pixels and calculates an average luminance value of each region. Further, the area luminance calculation unit 214 stores the average luminance value of each area in the memory 219.

  In step S503, the determination unit 215 compares the luminance average value for each region calculated by the region luminance calculation unit 214 in step S502 with a reference value L0 that is a predetermined threshold value. As a result of the comparison, if there is even one region larger than the reference value L0, the process proceeds to step S504, and if not, the process returns to step S501.

  In step S504, the determination unit 215 stores, in the memory 219, the position of the area where the average luminance value is larger than the reference value L0 in step S503 as position information where the high-luminance subject exists.

  In step S <b> 505, when the infrared cut filter 111 exists on the optical axis 106 of the lens unit 101, the lens unit controller 212 retracts the visible light cut filter 112 from the optical axis 106 of the lens unit 101. Insert on top.

  In steps S506 and S507, the lens unit controller 212 waits until a predetermined time (for example, 30 seconds) elapses, and then retracts the visible light cut filter 112 from the optical axis 106 of the lens unit 101 again. Return to S501.

  As described above, by controlling the insertion / extraction of the infrared cut filter and the visible light cut filter, it is possible to automatically shoot an optimal image according to the time zone and the subject while preventing the whiteout due to the high brightness subject. For example, when shooting at night with a surveillance camera or in-vehicle camera, visible light components contained in light such as headlights are attenuated by a visible light cut filter, so the effect of headlights is suppressed and the license plate is imaged. Be able to recognize above.

  Second Embodiment Next, filter insertion / extraction control in the surveillance camera mode of the digital camera 100 according to the second embodiment will be described with reference to FIGS.

  The configuration of the digital camera 100 that realizes the present embodiment is the same as that of FIG. 1 and FIG. 2 of the first embodiment. The image processing unit 211 generates JPEG format image data of 1920 × 1080 pixels and continuously from the communication control unit 217. And output externally.

  Note that steps S601 to S604 in FIG. 6 are the same as steps S501 to S504 in FIG.

  In step S605, the determination unit 215 determines whether the position information of the high-intensity subject is stored in the memory 219 for two or more consecutive images. If the information is stored, the process proceeds to step S606. Returns to step S601.

  In step S <b> 606, the determination unit 215 compares the position information of the high brightness subject in two or more images stored in the memory 219. If the position information of the high-intensity subject of the compared images is adjacent within a predetermined distance, it is determined that the high-intensity subject has moved and the process proceeds to step S607, and it is not determined that it has moved. In this case, the process returns to step S601.

  Steps S607 to S609 perform the same processing as steps S505 to S506 in FIG.

  As described above, according to the present embodiment, in addition to the effects of the first embodiment, filter control can be performed only on a moving high-luminance subject.

  [Third Embodiment] Next, filter insertion / extraction control in the monitoring camera mode of the digital camera 100 according to the third embodiment will be described with reference to FIGS. 1, 2, and 7. FIG.

  The configuration of the digital camera 100 that realizes the present embodiment is also the same as that in FIGS. 1 and 2 of the first embodiment. The image processing unit 211 generates JPEG format image data of 1920 × 1080 pixels, and continuously from the communication control unit 217. And output externally. In addition, the subject detection unit 216 detects a vehicle license plate as a predetermined subject.

  Steps S701 to S705 in FIG. 7 are the same as steps S501 to S505 in FIG.

  In step S706, the subject detection unit 216 determines whether or not a vehicle license plate has been detected as a predetermined subject from the image generated by the image processing unit 211. If detected, the process proceeds to step S707 and is not detected. In this case, the process proceeds to step S708.

  In step S707, the lens unit control unit 212 waits until a predetermined time (for example, 30 seconds) elapses.

  In step S708, the lens unit control unit 212 retracts the visible light cut filter 112 from the optical axis 106 of the lens unit 101 again, and returns to step S701.

  As described above, according to the present embodiment, in addition to the effects of the first embodiment, the visible light cut filter is continuously inserted while a predetermined high-luminance subject is being detected, and when the detection is not detected. Can be evacuated.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 101 ... Lens unit, 111 ... Infrared cut filter, 112 ... Visible light cut filter, 201 ... Image sensor, 211 ... Image processing part, 212 ... Lens unit control part, 214 ... Area brightness | luminance calculation part, 215 ... Determination part, 216 ... Subject detection unit, 219, memory, 301, signal processing unit

Claims (13)

An imaging means for capturing an image;
First filter means for cutting infrared light components of the subject image light guided to the imaging means;
Second filter means for cutting a visible light component of the subject image light guided to the imaging means;
Subject detection means for detecting a predetermined subject from the image captured by the imaging means;
It is determined whether the brightness of the subject detected by the subject detection means is greater than a predetermined threshold, and the first filter means and the first filter means on the optical path that guides the subject image light to the imaging means according to the determination result Control means for controlling the second filter means to be inserted and removed.   When the detected brightness of the subject is greater than the predetermined threshold, the control means retracts the first filter means from the optical path and inserts the second filter means into the optical path. The imaging apparatus according to claim 1, wherein the imaging apparatus is controlled. A calculation unit that divides an image captured by the imaging unit into a plurality of regions and calculates an average value of luminance for each of the divided regions;
The imaging apparatus according to claim 1, further comprising a determination unit that determines whether or not the average value of the luminance is greater than a predetermined reference value. Storage means for storing position information of an area where the average value of the luminance is larger than the predetermined reference value;
The control means emits light from the first filter means when two or more images are adjacent to each other in position information of areas determined by the determination means to have an average luminance value larger than a predetermined reference value. 4. The imaging apparatus according to claim 3, wherein control is performed so that the second filter means is inserted into the optical path by retracting from the path.   2. The control unit according to claim 1, wherein the second filter unit is controlled to be retracted from the optical path after a predetermined time has elapsed since the second filter unit was inserted into the optical path. 5. The imaging apparatus according to any one of items 4 to 4.   5. The control unit according to claim 1, wherein when the predetermined object is no longer detected by the object detection unit, the control unit controls the second filter unit to retreat from the optical path. The imaging device according to item.   The imaging device according to claim 6, wherein the subject detection unit detects a license plate of a vehicle as the predetermined subject.   The imaging apparatus according to claim 6, wherein the subject detection unit detects a human face as the predetermined subject.   The first filter means and the second filter means are provided so as to be insertable / removable on an optical path between the imaging means and an optical system for forming a subject image. The imaging device according to any one of 8. A communication unit for outputting an image captured by the imaging unit to an external device;
The imaging apparatus according to claim 1, wherein the communication unit outputs control information together with an image. An imaging unit that captures an image, a first filter unit that cuts an infrared light component of the subject image light guided to the imaging unit, and a visible light component of the subject image light guided to the imaging unit A second filter means, and a method for controlling the imaging apparatus,
Detecting a predetermined subject from an image captured by the imaging means;
It is determined whether the detected luminance of the subject is greater than a predetermined threshold value, and the first filter unit and the second filter are arranged on an optical path for guiding subject image light to the imaging unit according to the determination result. And a step of controlling to insert / remove the filter means.   The program for making a computer perform the control method of the imaging device of Claim 11.   A computer-readable storage medium storing a program for causing a computer to execute the control method for an imaging apparatus according to claim 11.

Images