JP2019129339A - Photographed image display system and electronic mirror system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a "photographed image display system and an electronic mirror system" which suppresses the appearance of two light sources as one light source without reducing the visibility of the displayed image as much as possible. SOLUTION: A high-luminance region 301 in which high-luminance pixels are collectively present is present in an image (a) taken with a gain maxG, and the high in the image (b) taken with a gain midG (midG <maxG). When the images of the two light sources appear separately in the area 302 corresponding to the brightness area 301, the image portion of the high brightness area 301 of the image taken with the gain maxG is the area 302 in the image taken with the gain midG. The image (d) replaced by the image portion of is displayed. [Selection diagram] Fig. 3

Description

  The present invention relates to a captured image display system that displays an image captured by a camera.

  As a captured image display system that displays an image captured by a camera, an electronic mirror system that displays an image captured by a camera that captures the rear of an automobile on a display disposed above a front windshield instead of a rearview mirror. It is known (for example, patent documents 1 and 2).

  Further, as a technology in a photographed image display system, in a system for photographing an image including headlights of a car, when an area of a pixel where a luminance value is saturated exists in an image photographed by a camera, the gain of the camera is reduced. Or by increasing the shutter speed of the camera so that the headlight image is blurred by saturation and two headlights appear as one light source (for example, Patent Document 3). ).

  Also, as a technique in the photographed image display system, when a camera whose gain is adjustable for each pixel is used as a camera and a region of pixels where a luminance value is saturated is generated in an image photographed by a standard gain, There is also known a technique for suppressing the appearance of two headlights as one light source by performing imaging while reducing the gain only for pixels in the area of the pixel where the luminance value is saturated (for example, Patent Document 3).

Unexamined-Japanese-Patent No. 2016-60411 JP 2007-214749 A Japanese Patent Application Publication No. 2006-339994

  When there is a pixel area with saturated luminance in the image captured by the camera, the two headlights can be used as one light source by reducing the camera gain or increasing the camera shutter speed. According to the technique for suppressing the appearance, the entire image becomes dark, so the dark part of the region to be photographed may be blackened and may not be visible.

  Further, according to the above-described technique in which the gain is reduced with respect to pixels in the area of the pixel where the luminance value is saturated, it is necessary to use a special camera whose gain can be adjusted for each pixel.

Further, in any of the techniques, the gain may be lowered even when the area of the pixel where the luminance value is saturated is not the one by a plurality of light sources but the one by a single light source. In such a case, the brightness of the display image is reduced without merit and the visibility of the display image is degraded. Therefore, the present invention relates to a photographed image display system for displaying an image photographed by a camera. It is an object of the present invention to suppress the appearance of two light sources as one light source without reducing the visibility of a display image as much as possible.

  In order to achieve the above object, the present invention provides a plurality of shooting settings which are different from each other as a shooting setting value, the gain of the camera × the exposure time of the camera in a captured image display system displaying an image captured by a camera A high brightness area which is an area in which a high brightness image appears in a first image which is a shooting control means which causes the camera to shoot an image by a value and a first image which is an image shot with a first shooting setting value A plurality of corresponding areas which are areas corresponding to the high brightness area in the second image which is an image photographed with a small photographing setting value which is different from the first photographing setting value; Using the image portion of the high brightness area of the first image and the image portion of the corresponding area of the second image, when the separated high brightness image of the The part is several separated high Modified to represent the image of degrees is the first image after the correction that a corrected image display means for displaying on said display.

  Here, in such a photographed image display system, the photographing control means causes the camera to photograph an image with three or more different photographing setting values, and the first photographing setting value is a maximum photographing setting value. In the modified image display means, there is a high brightness area, which is an area in which a high brightness image appears, in the first image which is an image shot at the first shooting setting value. A plurality of separated high-brightness images in a corresponding area, which is an area corresponding to the high-brightness area in the image shot with the shooting setting value, for any of the shooting setting values different from the first shooting setting value In the first image, an image captured with the largest shooting setting value among images in which a plurality of separated high-brightness images appear in the corresponding area is displayed as the second image. The image portion of the high brightness area is Using the image portion of the corresponding region of the second image, the image portion of the high luminance region is corrected so as to represent a plurality of separated high luminance images, and the large image after correction is displayed on the display It may be displayed.

  Here, the above-described captured image display system replaces, in the corrected image display means, the image portion of the high brightness area of the first image with the image portion of the corresponding area of the second image in the correction. May be configured to

  Alternatively, in the photographed image display system, the correction image display unit combines the correction with the image portion of the corresponding area of the second image with the image portion of the high luminance area of the first image. May be configured to

  Alternatively, in the photographed image display system, in the corrected image display means, the correction may be performed by using the image portion of the high brightness area of the first image and the image portion of the corresponding area of the second image. You may comprise by masking so that the high-intensity pixel part of a part may appear the pixel color of a said 1st image.

  In addition, the present invention also provides an electronic mirror system including the above-described captured image display system. Here, in the electronic mirror system, the camera is a camera that captures the rear, left rear, or right rear of the automobile mounted on the automobile, and the display is disposed at a position that is visible to the driver of the automobile. ing.

  According to the above-described captured image display system or electronic mirror system, the camera captures an image with a plurality of different capture setting values, with the gain of the camera × the exposure time of the camera as the capture setting value. In the first image, which is an image captured at the shooting setting value of, whether the high brightness area which is the area where the high brightness image appears is due to a single light source or a plurality of light sources are blurred due to saturation. Whether or not it appears in one light source state, in a corresponding area which is an area corresponding to the high brightness area in the second image which is an image photographed at a photographing setting value smaller than the first photographing setting value, It discriminate | determines from whether the several separated high-intensity image has appeared.

  When the high-luminance region in the first image is identified as a single light source that is blurred by saturation, the plurality of high-luminance regions in the first image are separated. Using the second image in which the high-brightness image appears, and displaying the image portion of the high-brightness region of the first image so that the image portion represents a plurality of separated high-brightness images Do.

  Therefore, the image portion of the high-luminance region in which the plurality of light sources in the first image appear as a single light source due to saturation can be corrected to represent the images of the plurality of separated light sources, and It is also possible to reduce the brightness of the image portion of the high luminance region by a plurality of single light sources of one image without merit and to deteriorate the visibility of the display image.

  As described above, according to the present invention, in a captured image display system that displays an image captured by a camera, two light sources appear as one light source without reducing the visibility of the displayed image as much as possible. Can be suppressed.

It is a block diagram which shows the structure of the electronic mirror system which concerns on embodiment of this invention. It is a flowchart which shows the nighttime display process which concerns on this invention. It is a figure which shows the process example of the night display process which concerns on this invention. It is a figure which shows the process example of the night display process which concerns on this invention.

Hereinafter, embodiments of the present invention will be described.
FIG. 1a shows the configuration of the electronic mirror system according to the present embodiment.
The electronic mirror system is a system mounted on an automobile, and includes a camera 1, a display control device 2, and a display 3 as shown in the figure.
Here, for example, as shown in FIG. 1b, the camera 1 is arranged at the rear of the automobile and photographs the appearance behind the automobile. Further, as shown in FIG. 1c, the display 3 is arranged at a position above the front windshield, that is, a position where a room mirror is traditionally arranged with the display surface facing rearward.

  Next, night display processing performed by the display control device 2 at night in such an electronic mirror system will be described. It is to be noted that whether the current night is or not is detected according to the overall brightness of the image captured by the camera 1, the illuminance detected by the separately provided illuminance sensor, the turning on of the light of the automobile, and the like.

FIG. 2 shows the procedure of the nighttime display process.
First, constants used in the nighttime display process will be described.
In this nighttime display process, MinG, MidG, and MaxG are used as constants representing the gain of the camera 1. However, MinG <MidG <MaxG.
Now, as shown in the figure, in the nighttime display process, the display control device 2 first sets the gains of MaxG, MidG, and MinG in the camera 1 sequentially, and captures an image with each gain (step 202).

  In step 202, it is determined whether or not there is a high luminance area that is an area of a predetermined size or more in which high luminance pixels having a luminance value equal to or higher than a predetermined threshold ThL are gathered in the image photographed with the gain MaxG. Examine (step 204). Here, the threshold value ThL is set to a value close to a saturation value of luminance or a saturation value less than the saturation value of luminance. In addition, the predetermined size is, for example, a size including two points in the photographed image in which two points separated by 1.3 m and which are located at a distance of 250 m from the camera 1 are captured.

  Then, if there is no high brightness area in the image captured with the gain MaxG (step 204), the image captured with the gain MaxG is displayed on the display 3 (step 206), and the process returns to step 202.

  On the other hand, if there is a high luminance area in the image captured with the gain MaxG (step 204), the following processing is performed for each of the existing high luminance areas (steps 208, 218, and 220).

  That is, in the image captured with gain MidG in step 202, a plurality of relatively high luminance pixel areas divided by a relatively low luminance pixel area at the same position and the same size area as the high luminance area. Are separated, that is, whether they are separated into a plurality of light source images (step 210).

  If the images of the light sources are separated, the image captured with the gain MaxG is corrected using the image portion of the region in the image captured with the gain MidG (step 212). End the process.

  Here, this modification replaces the image portion of the high brightness region in the image shot with the gain MaxG with the image portion of the same position and the same size as the high brightness region in the image shot with the gain midG. By doing. However, in this modification, the image portion of the high brightness region in the image shot with the gain MaxG is synthesized with the image portion of the same position and the same size as the high brightness region in the image shot with the gain midG. Or correcting the image portion of the high brightness area in the image shot with the gain MaxG using the image portion of the same position and size as the high brightness area in the image shot with the gain midG as a mask. It may be performed by the like.

  On the other hand, in the image photographed with the gain MidG, if the region having the same position and the same size as the high-intensity region is not separated into a plurality of light source images (step 210), the image photographed with the gain MinG in step 202 It is checked whether or not a region having the same position and size as the high-intensity region is separated into a plurality of light source images (step 214).

  If the images of the light sources are separated, the image captured with the gain MaxG is corrected using the image portion of the region in the image captured with the gain MinG (step 216). End the process.

Here, the correction of the image photographed with the gain MaxG using the image portion of the area in the image photographed with the gain MinG is performed in the same manner as step 212.
On the other hand, if an area photographed at a gain MinG and having the same position and size as the high brightness area is not separated into images of a plurality of light sources (step 214), processing for the high brightness area is performed as it is finish.

  Then, if the processing in steps 210 to 216 is performed for each high luminance region detected in step 204, an image photographed with the gain MaxG is displayed on the display 3 (step 222). Return. However, the image captured with the gain MaxG displayed in step 222 is an image captured with the gain MaxG after the correction when the correction in step 212 or step 216 is performed.

The night display process performed by the display control device 2 at night has been described above.
An example of such nighttime display processing is shown in FIG.
Here, FIGS. 3a, 3b, and 3c represent images taken when there is a following vehicle at night, FIG. 3a is an image taken with gain maxG, FIG. 3b is an image taken with gain midG, FIG. 3c is an image photographed with a gain minG.

  As shown in the figure, in this example, in the image photographed with the gain maxG in FIG. 3A, there is a high luminance region 301 that appears as an image of one light source in which two headlights of the following vehicle are continuous due to saturation.

  On the other hand, as shown in FIG. 3b, in the region 302 having the same position and the same size as the high brightness region 301 in FIG. It appears as separated. Also, as shown in FIG. 3c, two headlights of the following vehicle are images of two light sources in the same position and the same size region 303 of the high brightness region 301 of FIG. 3a in the image captured with gain minG. It appears as separated.

  In such a night display process, in such a case, the image portion of the high luminance region 301 of the image photographed with the gain maxG is converted into two regions 302 and 303 having the same position and the same size as the high luminance region 301. The image of the area 302 of the same position and the same size as the high luminance area 301 in the image photographed with the larger gain midG among the image photographed with the gain midG and the image photographed with the gain minG in which the image appears separately The image shown in FIG. 3d corrected by replacing with a part is displayed.

  As described above, in the nighttime display process, the correction of the high brightness area 301 of the image taken with gain maxG was taken with gain midG / minG in the image portion of the high brightness area 301 of the image taken with gain maxG. You may make it perform by synthesize | combining the image part of the area | region 302/303 of the same position and the same size as the said high-intensity area | region 301 in an image by a predetermined ratio. That is, for example, in the high brightness area 301 of the image photographed with gain maxG shown in FIG. 4a1, the image portion of the area 302 in the image photographed with gain midG shown in FIG. 4a2 has a predetermined transparency as shown in FIG. The correction may be performed by superimposing the display.

Further, as described above, in the nighttime display process, the correction of the high brightness area 301 of the image photographed with the gain maxG is an area of the same position and the same size as the high brightness area 301 in the image photographed with the gain midG / minG. The image portion 302/303 may be used as a mask to mask the image portion of the high luminance region 301 of the image captured with the gain maxG.
That is, in this case, for example, using the image portion of the region 302 in the image photographed with the gain midG shown in FIG. 4b2 as a mask of the image portion of the high luminance region 301 of the image photographed with the gain maxG in FIG. Then, the image portion of the high luminance region 301 of the image photographed with the gain maxG is masked with the mask. Also, in this case, as the brightness value of the pixel in the image portion is larger, the image portion of the region 302 in the image shot with gain midG has higher brightness of the image shot with gain maxG in FIG. The color of the pixel in the region 301 is displayed in a large ratio, and the smaller the luminance value of the pixel in the image portion, the black is used as a mask for displaying in a larger ratio.

The embodiments of the present invention have been described above.
As described above, in the present embodiment, whether a high luminance region in an image captured with gain MaxG is caused by a single light source or a plurality of light sources are blurred by saturation and appear as one light source. It is identified whether or not a plurality of separated high-brightness images appear in an area corresponding to a high-brightness area in an image captured with gain MidG or gain MinG.

  Of the high-brightness region 301 in the image shot with the gain MaxG, only a high-brightness region in which a plurality of light sources can be identified as appearing as one light source due to saturation is separated into a plurality of separated high-light regions. Using the image portion of the region corresponding to the high luminance region in the image captured with the gain MidG or the gain MinG in which the luminance image appears, the image portion of the high luminance region is separated into a plurality of separated high luminance Correct and display the image.

  Therefore, the image portion of the high luminance area where a plurality of light sources in the image photographed with the gain MaxG are blurred and appear as one light source can be corrected to represent a plurality of separated light source images. In addition, the brightness of the image portion of the high luminance area by the single light source in the image photographed with the gain MaxG is reduced without merit, and the visibility of the display image is not deteriorated.

  In addition, since the correction is performed using an image captured with the gain MidG preferentially over an image captured with the gain MinG, the reduction in the brightness of the high luminance region due to the correction can be minimized.

  By the way, in the above embodiment, in the night display process, a plurality of images are taken with different gains of the camera 1, but this is the exposure time (shutter speed) of the camera 1 or the exposure time of the camera 1 × A plurality of images may be taken with different gains.

  That is, for example, in night-time display processing, an image is taken with different exposure times of the camera 1 in three stages, and instead of an image taken with a gain MaxG in the above embodiment, the exposure time used for shooting is maximized. Use an image instead of the image taken with the gain MidG, use the image with the second longest exposure time used for shooting, and replace the image taken with the gain MinG with the image with the smallest exposure time used during shooting. Use it.

  Or, for example, in the night display process, the exposure time x gain of the camera 1 was taken in three stages, and the image was taken with the gain MaxG in the above embodiment. Instead of the image taken with gain MidG using the image with the largest gain, the exposure time used for shooting × the image with the second longest gain is used, and the image used for shooting instead of the image taken with gain MinG An image with the minimum time × gain is used.

  Moreover, although the above embodiment demonstrated the electronic mirror system which displays the image image | photographed with the camera 1 on the display 3 arrange | positioned in the position above a front windshield, the above night display processing is image | photographed with the camera 1. FIG. The image including the light source with the camera 1, such as an electronic mirror system that displays the left back / right back image on the display 3 arranged on the left / right, that is, an electronic mirror system that replaces the traditional side mirror Thus, the present invention can be similarly applied to any other captured image display system.

  1 ... camera, 2 ... display control device, 3 ... display.

Claims (6)

A captured image display system for displaying an image captured by a camera on a display,
Shooting control means for causing the camera to take an image with a plurality of mutually different shooting setting values, wherein the camera gain × the exposure time of the camera is set as a shooting setting value;
In the first image which is an image photographed at the first photographing setting value, there is a high luminance region which is a region where a high luminance image appears, and the first photographing value different from the first photographing setting value When a plurality of separated high-brightness images appear in a corresponding area, which is an area corresponding to the high-brightness area in the second image, which is an image shot with a small shooting setting value. An image portion of the high brightness region of the first image and an image portion of the corresponding region of the second image, wherein the image portion of the high brightness region represents a plurality of separated high brightness images And a corrected image display means for displaying the corrected first image on the display. The photographed image display system according to claim 1,
The shooting control means causes the camera to take an image with three or more different shooting setting values,
The first shooting setting value is a maximum shooting setting value;
In the first image which is an image photographed at the first photographing setting value, the corrected image display means has a high luminance region which is a region where a high luminance image appears, and the first photographing For any of the shooting setting values different from the setting value, a plurality of separated high-brightness images appear in the corresponding region which is a region corresponding to the high-brightness region in the image shot with the shooting setting value. In this case, the image photographed with the largest photographing setting value among the images in which a plurality of separated high-brightness images appear in the corresponding area is taken as the second image, the high-brightness area of the first image The image portion of the second image is corrected using the image portion of the corresponding region of the second image so that the image portion of the high brightness region represents a plurality of separated high brightness images, and Photographing characterized by displaying an image on the display Image display system. The photographed image display system according to claim 1 or 2, wherein
The corrected image display means performs the correction by replacing an image portion of the high brightness area of the first image with an image portion of the corresponding area of the second image. system. The photographed image display system according to claim 1 or 2, wherein
The corrected image display means performs the correction by combining the image portion of the corresponding region of the second image with the image portion of the high brightness region of the first image. system. The photographed image display system according to claim 1 or 2, wherein
The correction image display means may perform the correction on the image portion of the high brightness area of the first image, the image portion of the corresponding area of the second image, and the high brightness pixel portion of the image portion. A photographed image display system, wherein masking is performed so that a pixel color of a first image appears. An electronic mirror system comprising the captured image display system according to claim 1, 2, 3, 4 or 5.
The camera is a camera that captures the rear or left rear or right rear of a car mounted on a car,
The electronic mirror system according to claim 1, wherein the display is disposed at a position that is visible to a driver of the automobile.