JP2013009041A - Vehicle photographing display control system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform appropriate switching corresponding to a travel scene of a vehicle in a photographing display control system for switching a calculation method of luminance by using signals of visible light and near infrared light.SOLUTION: The photographing display control system includes: a camera 2 including a signal processing section 23 for calculating the luminance of respective pixels on the basis of intensity of visible light and near infrared light which an imaging element 22 receives and outputting RAW data showing the intensity of the visible light and the near infrared light; and an image recognition section 8 for displaying the image based on the luminance of the respective pixels, which the signal processing section 23 calculates. The image recognition section 8 searches an object which is previously set as an object which strongly reflects or discharges the near infrared light on the basis of the RAW data. When the object which strongly reflects or discharges the near infrared light cannot be detected as a result of search, the luminance of the respective pixels is calculated by a first calculation method. When the object has been detected, the luminance of the respective pixels is calculated by a second calculation method with which contribution of the intensity of the near infrared light to the luminance becomes lower than the first method.

Description

  The present invention relates to an imaging display control system for a vehicle.

  Conventionally, an imaging display control system for a vehicle includes an imaging unit having a plurality of near-infrared light imaging elements that receive near-infrared light and a plurality of visible light imaging elements that receive visible light, Using this imaging unit, the visible light intensity of each pixel and the near-infrared light intensity of each pixel are acquired, and the luminance of each pixel obtained based on the visible light intensity of each pixel is determined according to the near-infrared light intensity. A technique for providing an image that is easy for the user to see even in a dark environment such as at night is known.

  In such a vehicle display control system, luminance calculation using visible light and near infrared light signals is performed when photographing a place where the near infrared light is strong and photographing a weak place. Techniques for switching methods are known (see, for example, Patent Documents 1 and 2).

JP 2010-98358 A JP 2008-289000 A

  However, the technique as described above merely switches the calculation method of the luminance depending on whether the environment around the vehicle is an environment where the near-infrared light is strong or weak (more typically day or night). However, even at night, depending on the driving scene of the vehicle, there is a subject that strongly reflects or emits near-infrared light, so that the calculation method switching method is not always appropriate.

  In view of the above points, the present invention enables an appropriate switching according to a traveling scene of a vehicle in an imaging display control system capable of switching a luminance calculation method using visible light and near infrared light signals. For the purpose.

  In order to achieve the above object, the invention according to claim 1 is directed to an image sensor (22) that receives visible light and near infrared light, and the intensity of visible light and near infrared light received by the image sensor (22). Signal processing means (23) for calculating the luminance of each pixel based on the output, outputs the luminance of each pixel calculated by the signal processing means (23), and visible light received by the imaging device (22) And a camera (2) that outputs the intensity of near-infrared light, and display control means (8) that causes the display (9) to display an image based on the luminance of each pixel output from the camera (2). The display control means (8) is a subject preset as a subject that strongly reflects or emits near-infrared light based on the intensities of the visible light and near-infrared light output by the camera (2). Search by image recognition If no object that strongly reflects or emits near-infrared light is found as a result of the search, the first calculation method is used based on the intensities of visible light and near-infrared light received by the image sensor (22). When the luminance of each pixel is calculated and a subject that strongly reflects or emits near infrared light is found as a result of the search, based on the intensity of visible light and near infrared light received by the imaging device (22), Controlling the signal processing means (23) so as to calculate the luminance of each pixel by a second calculation method in which the contribution of the intensity of near-infrared light to the luminance is lower than that of the first method. This is a vehicle image display control system.

  In this way, by setting in advance a subject that strongly reflects or emits near-infrared light, and by changing the luminance calculation method when such a subject is found or not found, It is possible to perform appropriate switching according to the traveling scene of the vehicle. When a subject that strongly reflects or emits near-infrared light is found, the contribution of the near-infrared light intensity is reduced during the luminance calculation, so that the user can view the subject on the display (9). The possibility of being too bright and difficult to see decreases.

  According to a second aspect of the present invention, in the vehicle photographing display control system according to the first aspect, the preset subject includes a road sign. Thus, by setting the road sign as a subject that strongly reflects or emits near infrared light in advance, the possibility that the road sign becomes too bright and difficult to see when the vehicle is traveling is reduced.

  According to a third aspect of the present invention, in the vehicle photographing display control system according to the first or second aspect, the preset subject is a traffic light, a vehicle headlamp, a brake lamp, or a blinker. It is characterized by including at least one.

  Thus, by setting at least one of a traffic light, a vehicle headlight, a brake lamp, and a turn signal as a subject that strongly reflects or emits near-infrared light in advance, Alternatively, it is possible to reduce the halation of the headlamp, the brake lamp, or the turn signal (the color signal is saturated and the whiteout occurs).

  According to a fourth aspect of the present invention, in the vehicular photographing display control system according to any one of the first to third aspects, the signal processing means (23) is further configured to receive light by the imaging element (22). The color component of each pixel is calculated based on the intensity of the visible light, and the display control means (8) displays an image based on the luminance and color component of each pixel calculated by the signal processing means (23). (9), and further, the display control means (8) searches for a subject that strongly reflects or emits near-infrared light by image recognition based on the video signal received from the camera (2), As a result of the search, if an object that strongly reflects or emits near-infrared light is not found, the absolute value of the color component of each pixel is increased for the same visible light intensity. To do.

  By doing in this way, the user can confirm the color of the subject more clearly on the display (9). For example, it is easy to check characters inside the subject.

  According to a fifth aspect of the present invention, in the vehicle photographing display control system according to any one of the first to fourth aspects, a vehicle on which the vehicle photographing display control system is mounted includes the vehicle. There is an IR light projecting unit (7) that projects near-infrared light in the front, and the display control means (8) is able to find the object that strongly reflects or emits near-infrared light as a result of the search. When the number of pixels of the subject is greater than or equal to a threshold value, the signal processing means (23) is controlled to calculate the luminance of each pixel by the first method.

  This is a portion where the near-infrared light is projected by the IR light projecting unit (7) is a part away from the vehicle to some extent, and the light from the IR light projecting unit (7) is in the vicinity of the vehicle. Since the road sign may not reflect near-infrared light strongly, the overall visibility is improved compared to the second method for calculating the luminance for such road sign. It is because there is a high possibility that it is better to let them.

  In addition, the code | symbol in the bracket | parenthesis in the said and the claim shows the correspondence of the term described in the claim, and the concrete thing etc. which illustrate the said term described in embodiment mentioned later. .

It is a block diagram of the imaging | photography display control system 1 for vehicles. It is a flowchart of the process which an image recognition part performs. It is a figure which shows an example of the state in which the subjects 51-56 with strong IR entered the night vision camera. . It is a figure which shows an example of the image displayed on a display, when IR coefficient is a default value. It is a figure which shows an example of the image displayed on a display when IR coefficient is a limiting value.

  Hereinafter, an embodiment of the present invention will be described. FIG. 1 shows a configuration of a vehicle shooting display control system 1 according to the present embodiment. This vehicle display control system 1 is a system that is mounted on a vehicle and displays a captured image outside the vehicle to passengers (drivers, etc.) in the passenger compartment. The night vision camera 2 and near-infrared light in front of the vehicle. An IR light projecting unit 7, an image recognizing unit 8, a display 9, and the like.

  The night vision camera 2 is a camera that receives visible light and near-infrared light, and outputs video signals of luminance and color components based on the received visible light and near-infrared light to the image recognition unit 8. The night vision camera 2 is attached to the vehicle so as to photograph the front as viewed from the vehicle, and includes a lens 21, an image sensor 22, a signal processing unit 23, a memory 24, and a D / A conversion unit 25. In the night vision camera 2, visible light and near-infrared light incident on the image pickup device 22 from the lens 21 are received by the image pickup device 22, and the image pickup device 22 receives signals of the intensity of the received visible light and near-infrared light. Output to the signal processing unit 23.

  The image sensor 22 has a plurality of pixels arranged in a matrix. In this embodiment, as in the case of Patent Document 1, the pixel arrangement is an R + I pixel having sensitivity to the wavelength (R) in the red region and the wavelength (I) in the near infrared region, the wavelength (G) in the green region, and the near region. G + I pixel sensitive to wavelength (I) in infrared region, B + I pixel sensitive to wavelength (B) in blue region and wavelength (I) in near infrared region, and wavelength (I) in near infrared region The I pixels that are sensitive only to each other are arranged so as to form a rectangular vertex with a set of four, and a plurality of such sets are arranged vertically and horizontally. You may have.

  The signal processing unit 23 (corresponding to an example of signal processing means) acquires the intensity of visible light and near-infrared light received by the image sensor 22 at a constant period (for example, 60 times per second), and each time it acquires it. Based on the acquired intensity, the luminance and color components of each pixel are calculated, and the calculated luminance and color components of each pixel are output to the D / A converter 25. When calculating luminance and color components, the memory 24 is used as a work area.

  A method by which the signal processing unit 23 calculates the luminance and color components of each pixel will be described below. First, the signal processing unit 23 acquires the intensities of visible light (R, G, B) and near-infrared light (I) received by the image sensor 22 from the image sensor 22 and, for each pixel, based on the acquired intensity. , R intensity, G intensity, B intensity, and I intensity are calculated.

  More specifically, a method for calculating the intensity of I for each pixel is as follows. For the I pixel, the intensity of I received at the pixel is used as it is, and for the other pixels, the intensity of I received at a plurality of I pixels is used to calculate the I intensity at the other pixel by interpolation calculation. calculate.

  The calculation method of the R intensity of each pixel is as follows. For the R + I pixel, the result obtained by subtracting the I intensity of the R + I pixel calculated as described above from the sum of the R intensity and the I intensity (R + I) received at the R + I pixel is used. For other pixels, the R intensity calculated for a plurality of R + I pixels is used to calculate the R intensity of the other pixels by interpolation calculation. As a calculation method of the intensity of G and B of each pixel, a calculation method in which R is replaced with G and B by this method may be used.

After calculating the intensity of R (denoted as R), intensity of G (denoted as G), intensity of B (denoted as B), and intensity of I of each pixel, Luminance Y and color components U and V are calculated. The calculation formula is
(1) Y = 0.2 · R ′ + 0.4 · G ′ + 0.1B ′ + 0.3 · α · I
(2) U = −0.15 · (R / α) −0.35 · (G / α) + 0.5 · (B / α)
(3) V = 0.5 · (R / α) −0.35 · (G / α) −0.15 · (B / α)
These three expressions are used. However, R ′ = R + I, G ′ = G + I, and B ′ = B + I. As described above, the luminance and color components of the present embodiment are the luminance and color components in the YUV format, but other types of luminance and color components may be adopted.

  Note that the IR coefficient α in the above equation is a parameter that has a default value of 1 but changes according to a command from the image recognition unit 8. When the IR coefficient α is 1, the luminance Y is set to the value of the luminance Y that is normally calculated from the intensities of R, G, and B (that is, 0.2 · R + 0.4 · G + 0.1 · B). It is a value obtained by adding an increase amount (1.0 · I) that increases in proportion to the light intensity I. That is, the normal increase amount of the luminance of the pixel representing the imaging target is higher as the imaging target strongly reflects or emits near infrared light. By doing so, it is possible to provide an image that is easy for the user to see even in a dark environment such as at night.

  The color components U and V of each pixel are determined independently of the near-infrared light intensity I of the pixel. However, as another example, the color components with the near-infrared light intensity I taken into account. U and V may be corrected.

  The D / A conversion unit 25 converts the luminance Y and color components U and V of each pixel sequentially calculated and output by the D / A conversion unit 25 into analog signals and outputs the analog signals to the image recognition unit 8.

  Further, the image sensor 22 outputs the received visible light and near-infrared light intensity signals not only to the signal processing unit 23 but also to the image recognition unit 8 outside the night vision camera 2. Therefore, the luminance Y and the color components U and V of each pixel calculated by the signal processing unit 23 are output from the night vision camera 2 to the image recognition unit 8 and visible light of a plurality of pixels received by the image sensor 22 is output. In addition, near infrared light intensity data (hereinafter referred to as RAW data) of a plurality of pixels is also output.

  The image recognition unit 8 (corresponding to an example of display control means) is a device that is configured separately from the night vision camera 2, and is calculated and output by the signal processing unit 23, and the D / A conversion unit 25 is analog. Images based on the luminance Y and the color components U and V of each pixel converted into signals are sequentially displayed on the display 9.

  Each of the image recognition unit 8 and the signal processing unit 23 may be configured as an integrated circuit having a dedicated circuit configuration, or as a microcomputer that realizes desired processing by executing a program. May be.

  The display 9 is installed in the vehicle interior so that a passenger such as a driver can visually recognize the display screen of the display 9. Therefore, even when the vehicle is moving, the occupant can view on the display 9 an image of the scenery in front of the vehicle that changes every moment.

  Whether or not to display an image based on the output of the night vision camera 2 on the display 9 is determined by the image recognition unit 8 in accordance with, for example, an occupant's operation on a switch (not shown) in the vehicle.

  Hereinafter, the operation of the vehicle photographing display control system 1 when the image recognition unit 8 displays an image based on the output of the night vision camera 2 on the display 9 while the vehicle is traveling at a reference speed or higher at night. explain. In this case, the image recognizing unit 8 operates the IR projector 7 so that the IR projector 7 projects near-infrared light in front of the vehicle.

  At this time, the image recognition unit 8 displays an image based on the signal output from the D / A conversion unit 25 on the display 9, and the signal processing unit 23 performs from the RAW data received from the night vision camera 2. With the method using the same interpolation calculation, the intensity of R, the intensity of G, the intensity of B, and the intensity of I are calculated for every pixel for every pixel. The resulting R intensity (simply referred to as R), G intensity (simply referred to as G), B intensity (simply referred to as B), and I (simply referred to as I) are obtained. Based on this, an object that is preset as a subject that strongly reflects or emits near-infrared light is searched by image recognition, and the IR coefficient α described above is determined depending on whether such a subject is found or not found as a result of the search. The signal processing unit 23 is controlled to be switched.

  Due to such an operation, the image recognition unit 8 executes the processing shown in FIG. In this processing, the image recognizing unit 8 first acquires RAW data from the night vision camera 2 in step 110, and takes a photographic image composed of the acquired RAW data (R, G, B, For a subject that strongly reflects or emits near-infrared light (hereinafter also referred to as a subject with a strong IR), using a known image recognition technique (specifically, pattern matching). . At this time, feature information of an image of a subject with strong IR is used for comparison with the above-described captured image in pattern matching. This feature information is stored in advance in a storage medium (for example, ROM, flash memory) of the image recognition unit 8. Read and use what is stored.

  Examples of subjects that store characteristic information in advance include various road signs, traffic lights, vehicle headlamps, brake lamps, turn signals, and the like. The road sign tends to strongly reflect near infrared light such as near infrared light projected from the IR light projecting unit 7. Traffic lights and vehicle headlamps tend to emit strong near-infrared light.

  The feature information of various road signs may be created from images obtained by photographing the road signs from the front in advance. The feature information created in this way includes not only the feature of the shape of the road sign but also the feature of the characters described on the road sign. However, at night, the characters in the signs may not be distinguished from the photographed image composed of RAW data, so the feature information of various road signs is the same as that road sign but a solid object is photographed from the front. You may use what was created from the done image. The characteristic information of the traffic light, the vehicle headlight, the brake lamp, and the blinker is created from an image obtained by photographing the traffic light or the headlight from the front in advance.

  Then, as a result of the search, the image recognition unit 8 determines whether or not a strong IR subject has been found in the captured image composed of the RAW data received from the night vision camera 2. Proceed to

  In step 120, a command for setting the IR coefficient to 1 which is a default value is output to the signal processing unit 23 of the night vision camera 2. As a result, the signal processing unit 23 captures images of all the pixels by a method (corresponding to an example of the first method) using the above equations (1), (2), and (3) with the IR coefficient α being 1. Based on the intensities of visible light and near-infrared light received by the element 22, the luminance Y and color components U and V of each pixel are calculated, and the image recognition unit 8 continues with step 130 in the signal processing unit 23. An image is displayed on the display 9 based on the signal of the luminance Y and the color components U and V calculated and D / A converted by the D / A converter 25.

  Subsequently, in step 140, it is determined whether or not to end the operation of displaying an image based on the output of the night vision camera 2 on the display 9 (for example, based on the operation of the occupant). If it is determined that the process is not completed, the process returns to step 110.

  As described above, while a subject with strong IR is not found in the captured image composed of the RAW data, the processing of steps 110, 120, 130, and 140 is repeated in this order. Image display on the display 9 based on the luminance Y and the color components U and V of each pixel calculated by the method using the above formulas (1), (2), and (3) with α being 1 continues.

  Then, as shown in FIG. 3, road signs 51 and 53, a traffic light 52, a vehicle headlamp 54, a vehicle brake lamp 55, a vehicle turn signal 56, and other strong IR subjects are within the shooting range of the night vision camera 2. When entering, the image recognition unit 8 finds a subject with strong IR as a result of the search in step 130, and as a result, advances the process to step 125.

  In step 125, a command for setting the IR coefficient to 0.02 which is the limit value is output to the signal processing unit 23 of the night vision camera 2. Thereby, the signal processing unit 23 uses a method (corresponding to an example of the second method) using the above equations (1), (2), and (3) with an IR coefficient α of 0.02 for all pixels. The luminance Y and the color components U and V of each pixel are calculated based on the intensities of the visible light and near infrared light received by the image sensor 22, and the image recognition unit 8 subsequently proceeds to step 130 in the signal processing unit. The image is displayed on the display 9 based on the luminance Y and the color components U and V signals calculated at 23 and D / A converted by the D / A converter 25.

  Here, the difference between the case where the brightness etc. is calculated with the IR coefficient α as the default value 1 and the case where the brightness etc. is calculated with the IR coefficient as the limit value 0.02 will be described with reference to FIGS. .

  In the calculation formula (1) for the luminance Y, when the IR coefficient α is 0.02, the contribution of the intensity of near-infrared light to the luminance is lower than when the IR coefficient α is 1. ing. That is, it increases in proportion to the intensity I of near-infrared light with respect to the value of luminance Y normally calculated from the intensities of R, G, and B (that is, 0.2 · R + 0.4 · G + 0.1 · B). The amount of increase (ie, (0.7 + 0.3 · α) · I) is smaller when α is 0.02 than when it is 1.

  In addition, in the calculation formula (2) for the color component U and the calculation formula (3) for the color component V, as the value of α increases, the absolute values of U and V increase with respect to the same R, G, and B values. It is supposed to be. Therefore, when α is 0.02, the absolute values of the color components U and V of each pixel are larger for the same visible light intensity R, G, and B than when α is 1.

  For example, with respect to pixels on a temporary stop road sign, the intensity of R ′, G ′, B ′, I is R ′ = 255, G ′ = 250, B ′ = 250, I = 245, and the IR coefficient α Is assumed to be 1. Note that the range of values that R, G, B, R ′, G ′, B ′, I, Y, U, and V can take is 0 to 255. In this case, the R, G, and B values are R = 10, G = 5, B = 5, and I = 245, and when this is applied to the above equations (1), (2), and (3) with α = 1. , Luminance Y = 249.5, color component U = −0.75, color component V = 2.5, and almost no color is displayed. As shown in FIG. Above, only the white shape of the inverted triangle will be displayed.

  On the other hand, when α = 0.02 for the same pixel and applied to the above equations (1), (2), (3), luminance Y = 117.47, color component U = -37.5, and color component V = 125. As shown in FIG. 4, the luminance “Y” decreases and the red color is emphasized, so that the character “stop” in the road sign becomes visible on the display 9.

  As described above, while a strong IR subject is found in the captured image of the night vision camera 2, the processing of steps 110, 125, 130, and 140 is repeated in this order, so that the signal processing unit 23 has the IR coefficient. The image display on the display 9 continues based on the luminance Y and the color components U and V of each pixel calculated by the method using the above formulas (1), (2), and (3) with α set to 0.02.

  As described above, a subject that strongly reflects or emits near-infrared light is set in advance, and the luminance calculation method is changed depending on whether such a subject is found or not. Thus, it is possible to perform appropriate switching according to the traveling scene of the vehicle. If a subject that strongly reflects or emits near-infrared light is found, the contribution of the near-infrared high intensity is reduced in the luminance calculation, so that the user is too bright on the display 9. The possibility of becoming difficult to see decreases.

  In addition, if a subject that strongly reflects or emits near infrared light is not found as a result of the search, the absolute value of the color component of each pixel is increased for the same visible light intensity. By doing so, the user can confirm the color of the subject more clearly on the display (9). For example, it is easy to check characters inside the subject.

  In order to search for a subject with strong IR, a photographed image composed of RAW data is used because the photographed image is converted from the intensity of R, G, B, I to luminance Y, color components U, V. Since the data has not yet undergone conversion (conversion using equations (1), (2), and (3)), the luminance is saturated by the conversion, resulting in problems such as difficulty in image recognition. This is because the possibility is low.

  In addition, by setting a road sign in advance as a subject that strongly reflects or emits near-infrared light, the possibility that the road sign becomes too bright and difficult to see when the vehicle is traveling is reduced.

  In addition, traffic lights, vehicle headlights, brake lamps, turn signals, etc. are set in advance as subjects that strongly reflect or emit near-infrared light, so that traffic lights, headlights, brake lights, It is possible to reduce blinker halation (saturation of the color signal and whiteout).

(Other embodiments)
As mentioned above, although embodiment of this invention was described, the scope of the present invention is not limited only to the said embodiment, The various form which can implement | achieve the function of each invention specific matter of this invention is included. It is. For example, the following forms are also acceptable.

  (A) In the above embodiment, the image recognizing unit 8 uses the above formulas (1), (2), and (3) in step 125 in FIG. Thus, a command is output to calculate the luminance Y and the color components U and V of each pixel based on the intensities of visible light and near infrared light received by the image sensor 22.

  However, this need not always be the case, and the image recognizing unit 8 may include a portion (for example, a found IR) including a pixel in which a subject with a strong IR found is captured in step 125 of FIG. (A portion consisting only of pixels in which a strong subject is captured, a portion including only pixels found in a subject having a strong IR and its vicinity), and the IR coefficient α is set to 0.02 for the target pixel. Based on the intensity of visible light and near-infrared light received by the image sensor 22 by the method using (1), (2), and (3), the luminance Y and color components U and V of each pixel are calculated, Based on the intensities of visible light and near-infrared light received by the image sensor 22 by using the above formulas (1), (2), and (3) with the IR coefficient α as default 1 for the pixels other than the target pixel. The brightness of each pixel A command may be output so as to calculate the degree Y and the color components U and V.

  By doing so, it is possible to improve the visibility of the image based on the night vision camera 2 at night and also reduce the possibility that a subject with strong IR will be too bright and difficult to see.

  (B) In the above embodiment, the image recognizing unit 8 always applies to all the pixels to the signal processing unit 23 of the night vision camera 2 when there is a subject with strong IR in the captured image composed of the RAW data. Although the command is output to set the IR coefficient to the limit value, this need not always be the case.

  For example, the image recognition unit 8 finds only one subject as a subject with strong IR by image recognition, calculates the number of pixels that the one subject occupies in the captured image composed of RAW data, and the calculated number of pixels is If the threshold value is equal to or greater than the predetermined threshold, the signal processing unit 23 may be instructed to set the IR coefficient to the default value for all pixels, as in the case where a subject with strong IR is not found.

  By doing in this way, a subject with strong IR approaches from the front while the vehicle is traveling, and the subject with strong IR (here, a road sign) is captured in RAW data by image recognition. After that, the display 9 is displayed with the luminance and color components calculated using the IR coefficient as the limit value for the subject, but the subject further approaches the vehicle and the number of pixels is equal to or greater than the threshold value. Thus, the IR coefficients of all the pixels return to the default values.

  This is because the position where the near infrared light is projected by the IR projector 7 is a part far away from the vehicle to some extent, and the light from the IR projector 7 does not hit the vicinity of the vehicle. Since the road sign may not reflect near infrared light strongly, it is more likely to improve the overall visibility than to lower the IR coefficient for such a road sign. It is.

  Therefore, the threshold value differs depending on the size that can be seen from the night vision camera 2 when the IR light projecting unit 7 is not applied to each subject that is preset as a subject that strongly reflects or emits near infrared light. A value may be set, or roughly the same value may be set.

  (C) In the above embodiment, as the IR coefficient value, the default value is 1 and the limit value is 0.02. However, the present invention is not limited to this value, and the limit value is a positive value smaller than the default value. It only has to be a number.

  (D) In the above embodiment, a subject with strong IR found by pattern recognition from a captured image composed of RAW data may be a road sign, a traffic light, or a headlamp. The limit value commanded to the signal processing unit 23 is 0.02 which is the same whether it is a brake lamp or a blinker. However, the limit value commanded to the signal processing unit 23 may be changed according to the type of the subject with the strong IR found.

  For example, when only one subject with strong IR is found and the found subject is a road sign, the first limit value (for example, 0.02) is commanded to the signal processing unit 23 as the value of the IR coefficient, If the found subject is any one of a traffic light, a headlamp, a brake lamp, and a turn signal, the IR coefficient value is a second limit value that is greater than the first limit value and smaller than the default value. (For example, 0.1) may be instructed to the signal processing unit 23. This is because the traffic light and the headlight do not want to read the characters in the traffic light and the headlight, and the brightness should be suppressed to such an extent that the traffic light and the headlight are prevented from expanding in the display 9. .

  Further, for example, only one subject having a strong IR is found in the photographed image, and when the found subject is a traffic light, a headlamp, a brake lamp, or a blinker, the IR coefficient A different value may be commanded to the signal processing unit 23 as the value of.

  (E) In the above embodiment, the headlamp and the traffic light are distinguished and recognized by pattern matching. However, the intensity of R, G, Based on the intensity, the headlamp and the traffic light may be distinguished. Specifically, a subject that is a headlight or a traffic light is found by pattern matching, and the found subject's R / G (the value obtained by dividing the intensity of R by the intensity of G) and B / G (the intensity of B is the value of G (Value divided by intensity) is calculated, and if any of R / G and B / G exceeds the threshold, it is determined as a traffic light.

  Specifically, in the case of a red signal, for example, the intensities of R, G, and B are R = 100, G = 20, and B = 20. At this time, reference values R / G, B for white balance control / G is R / G = 5 and B / G = 1, respectively.

  In the case of a green signal, for example, the intensities of R, G, and B are R = 20, G = 100, and B = 20. At this time, the reference values R / G and B / G for white balance control are R / G = 0.2 and B / G = 0.2, respectively.

  On the other hand, in the case of a headlamp, for example, the intensities of R, G, and B are R = 100, G = 100, and B = 100. At this time, the reference values R / G and B / G for white balance control are R / G = 1 and B / G = 1, respectively.

Therefore, when the values of R / G and B / G are both near 1 (for example, 0.9 or more and 1.1 or less), the light source (headlight) that is not colored is shining, and R / G , B / G is much larger than 1 (for example, threshold 3 or more), the light source (signal) colored red or blue is shining, and both R / G and B / G are If it is much smaller than 1 (for example, 0.3 or less as a threshold), it can be determined that the light source (signal) colored green is shining.
(F) In the above embodiment, the signal processing unit 23 is incorporated in the night vision camera 2 as a part of the night vision camera 2, and the image recognition unit 8 is a separate component from the night vision camera 2. It is configured. However, this need not always be the case. For example, the function of the signal processing unit 23 may be incorporated in the image recognition unit 8.

DESCRIPTION OF SYMBOLS 1 Vehicle imaging | photography display control system 2 Night vision camera 7 IR light projection part 8 Image recognition part 9 Display 21 Lens 22 Image pick-up element 23 Signal processing part 24 Memory 25 D / A conversion part 51-56 IR strong subject

Claims (5)

An image sensor (22) that receives visible light and near infrared light, and a signal processing means (23) that calculates the luminance of each pixel based on the intensity of visible light and near infrared light received by the image sensor (22). A camera (2) that outputs the luminance of each pixel calculated by the signal processing means (23) and outputs the intensity of visible light and near-infrared light received by the imaging device (22);
Display control means (8) for causing the display (9) to display an image based on the luminance of each pixel output by the camera (2),
The display control means (8) images a subject set in advance as a subject that strongly reflects or emits near infrared light based on the intensities of the visible light and near infrared light output from the camera (2). If a subject that strongly reflects or emits near-infrared light is not found as a result of the search, a first search is performed based on the intensities of visible light and near-infrared light received by the image sensor (22). If the subject that strongly reflects or emits near-infrared light is found as a result of the search, the intensity of visible light and near-infrared light received by the image sensor (22) is calculated. The signal processing means (23) is configured to calculate the luminance of each pixel by a second calculation method in which the contribution of the intensity of near infrared light to the luminance is lower than that of the first method. Characterized by controlling Vehicle for taking a picture display control system that.   The vehicle photographing display control system according to claim 1, wherein the preset subject includes a road sign.   The vehicle photographing display control system according to claim 1 or 2, wherein the preset subject includes at least one of a traffic light, a vehicle headlamp, a brake lamp, and a winker. The signal processing means (23) further calculates a color component of each pixel based on the intensity of visible light received by the image sensor (22),
The display control means (8) causes the display (9) to display an image based on the luminance and color components of the pixels calculated by the signal processing means (23),
Further, the display control means (8) searches for a subject that strongly reflects or emits near-infrared light by image recognition based on the video signal received from the camera (2), and the result of the search is near-infrared light. The absolute value of the color component of each pixel is increased for the same visible light intensity when an object that strongly reflects or emits light is not found. The imaging | photography display control system for vehicles as described in any one. The vehicle equipped with the vehicle image display control system has an IR projector (7) that projects near-infrared light in front of the vehicle,
Even when a subject that strongly reflects or emits near-infrared light is found as a result of the search, the display control means (8) uses the first method if the number of pixels of the found subject is equal to or greater than a threshold value. The vehicle image display control system according to any one of claims 1 to 4, wherein the signal processing means (23) is controlled so as to calculate the luminance of each pixel.

Images