JP5291524B2 - Vehicle periphery monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle circumference monitoring device which prevents a driver from feeling a reporting operation to be cumbersome and performs a reporting operation at an appropriate timing. <P>SOLUTION: A vehicle circumference monitoring device 10 includes: infrared cameras 12L, 12R; an object detecting unit 31 which detects an object existing in an area outside a vehicle based on infrared images outputted from the infrared cameras 12L, 12R; a reporting control unit 36 which performs a reporting operation to the driver when the object detected by the object detecting unit 31 lies in a predetermined area for reporting in real space outside the vehicle; an external illumination intensity acquisition unit 32 which detects or estimates a change of an illumination intensity outside the vehicle; and an area extension unit 35 which extends an area for reporting during a predetermined time period if the change of an illumination intensity acquired by the external illumination intensity acquisition unit 32 is greater than or equal to a predetermined amount. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a vehicle periphery monitoring device.

  Conventionally, for example, when a warning area is set beyond the light distribution area of the headlamp, and the position of the target detected by the infrared camera is within the warning area, the voice and display are used. 2. Description of the Related Art A vehicle periphery monitoring device that notifies a driver is known (see, for example, Patent Document 1).

Japanese Patent No. 3919975

By the way, it is determined that the object existing in the light distribution area of the headlamp can be visually recognized by the driver as in the vehicle periphery monitoring device according to the above-described conventional technology, and the object is excessive. It is possible to prevent the driver from feeling bothered by the notification operation by prohibiting the user from being alerted.
However, simply disabling notification to the object existing in the light distribution area of the headlamps causes a problem that the driver cannot perform the notification operation at an appropriate timing when it is difficult to visually recognize the object. Occurs.

  The present invention has been made in view of the above circumstances, and provides a vehicle periphery monitoring device capable of performing a notification operation at an appropriate timing while preventing a driver from feeling troublesome in the notification operation. For the purpose.

In order to solve the above problems and achieve the object, the vehicle periphery monitoring apparatus according to the first aspect of the present invention is mounted on a vehicle and outputs infrared imaging means that outputs an infrared image obtained by imaging in the infrared region ( For example, the infrared camera 12L, 12R in the embodiment and the object detection means for detecting the object existing outside the vehicle based on the infrared image output from the infrared imaging means (for example, in the embodiment) An object detection unit 31) and a vehicle headlamp in which the object detected by the object detection means is a predetermined area (for example, a notification target area in the embodiment) in real space outside the vehicle. A vehicle periphery monitoring device including notification means (for example, notification control unit 36 in the embodiment) for notifying a driver when the vehicle is present in a region beyond the light distribution region , and for detecting a change in illuminance outside the vehicle. Detection or When the illuminance change acquired by the external illuminance acquisition means (for example, the external illuminance acquisition unit 32 in the embodiment) acquired by estimation and the external illuminance acquisition means is greater than or equal to a predetermined change, over a predetermined period. And a region expansion means (for example, a region expansion unit 35 in the embodiment) for expanding the predetermined region into a light distribution region of the headlamp of the vehicle .

  Furthermore, in the vehicle periphery monitoring apparatus according to the second aspect of the present invention, the external illuminance acquisition means detects a oncoming vehicle approaching the host vehicle and a heat source having a luminance equal to or higher than a predetermined luminance on the infrared image. The illuminance change greater than or equal to the predetermined change is acquired in at least any one of a case where the appearance or disappearance is detected and a case where a time zone of sunset or sunrise is detected.

  According to the vehicle periphery monitoring device according to the first aspect of the present invention, the predetermined area for determining whether or not to notify the driver when the illuminance change outside the vehicle is equal to or greater than the predetermined change is set to the predetermined period. It extends over. As a result, even if the driver's ability to visually recognize the outside world is temporarily reduced due to a change in illuminance outside the vehicle, the notification operation can be performed at an appropriate timing, and the driver is bothered by the notification operation. Can also be prevented.

  According to the vehicle periphery monitoring apparatus according to the second aspect of the present invention, it is possible to acquire an illuminance change that is equal to or greater than a predetermined change without the need to directly detect an illuminance change outside the vehicle.

It is a block diagram which shows the structure of the vehicle periphery monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows the vehicle carrying the vehicle periphery monitoring apparatus shown in FIG. It is a figure which shows the example of the alerting | reporting area | region of the vehicle periphery monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the alerting | reporting area | region of the vehicle periphery monitoring apparatus which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the vehicle periphery monitoring apparatus shown in FIG. It is a flowchart which shows the alerting | reporting determination process shown in FIG. It is a figure which shows an example of the relative position of the own vehicle and target object which concern on embodiment of this invention. It is a flowchart which shows the area | region determination process shown in FIG. It is a flowchart which shows the area | region determination process shown in FIG. It is a flowchart which shows the area | region determination process shown in FIG. It is a flowchart which shows the area | region determination process shown in FIG.

Hereinafter, a vehicle periphery monitoring device according to an embodiment of the present invention will be described with reference to the accompanying drawings.
For example, as shown in FIG. 1, the vehicle periphery monitoring device 10 according to the present embodiment is provided for a processing unit 11 including a CPU (central processing unit) that controls the vehicle periphery monitoring device 10 and a predetermined imaging region outside the vehicle. Two infrared cameras 12L and 12R that can detect infrared rays, a yaw rate sensor 13 that detects the yaw rate of the host vehicle, a vehicle speed sensor 14 that detects a traveling speed (vehicle speed) of the host vehicle, and whether or not the driver performs a brake operation. A brake sensor 15 for detecting the illuminance, an illuminance sensor 16 for detecting the illuminance outside the vehicle, a vehicle interior camera 17 for imaging the eyes of the driver, a speaker 18, a display device 19, and a navigation device 20. Has been.

The processing unit 11 of the vehicle periphery monitoring apparatus 10 includes, for example, an infrared image around the own vehicle obtained by imaging with the two infrared cameras 12L and 12R, and the traveling state of the own vehicle detected by the sensors 13 to 16 or the own vehicle. From the detection signal related to the traveling environment of the vehicle and the signal related to the state of the driver obtained by the imaging of the in-vehicle camera 17, an object such as an object and a pedestrian existing in front of the traveling direction of the host vehicle are detected and detected. When it is determined that the contact between the target object and the host vehicle may occur, an alarm is output from the speaker 18 or the display device 19.
The display device 19 is a display device integrated with instruments for displaying various travel state quantities of the host vehicle, for example, or a HUD (Head Up) that displays various information at a position that does not obstruct the driver's front view on the front window. Display) 19a and the like.

  The processing unit 11 includes an A / D conversion circuit (not shown) that converts an input analog signal into a digital signal, an image memory (not shown) that stores a digitized image signal, and a CPU ( A central processing unit (not shown), a RAM (Random Access Memory) (not shown) that is used by the CPU to store various data during execution of arithmetic processing, and programs, tables, maps, etc. that the CPU executes A ROM (Read Only Memory) (not shown) for storing, and an output circuit (not shown) for outputting a drive signal of the speaker 18 and a display signal of the display device 19 are provided, and the infrared cameras 12L and 12R and each sensor are provided. Signals output from 13 to 16 and the vehicle interior camera 17 are converted to digital signals and input to the CPU.

For example, as shown in FIG. 2, the infrared cameras 12L and 12R are arranged at positions substantially symmetrical with respect to the center in the vehicle width direction at the front portion of the host vehicle P, and the two infrared cameras 12L and 12R. The optical axes are parallel to each other, and the height from both road surfaces is fixed. The infrared cameras 12L and 12R have a characteristic that the higher the temperature of the object, the higher the output signal level (that is, the luminance increases).
Further, the HUD 19a of the display device 19 is provided such that a display screen is displayed at a position that does not obstruct the driver's front view of the front window of the host vehicle P.

The navigation device 20 includes, for example, a display unit 21, a position positioning unit 22, and a navigation processing unit 23.
The position positioning unit 22 receives a positioning signal such as a GPS (Global Positioning System) signal for measuring the position of the vehicle using, for example, an artificial satellite, and calculates the current position of the host vehicle based on the positioning signal.
The navigation processing unit 23 calculates the current position of the host vehicle by, for example, a calculation process of autonomous navigation based on detection signals such as the yaw rate and the vehicle speed of the host vehicle output from the sensors 13 and 14.

In addition, the navigation processing unit 23 includes map display data for displaying a map on the display unit 21 and road coordinate data necessary for map matching processing based on the current position of the vehicle. Further, the navigation processing unit 23 is a line connecting each node with data necessary for processing such as route search and route guidance, for example, a node consisting of latitude and longitude at a predetermined position such as an intersection and a branch point. Road data consisting of a certain link is provided as map data. Various types of information are added to the nodes and links.
Then, the navigation processing unit 23 performs map matching on the road data based on information on the current position of the host vehicle obtained from either or both of the positioning signal and the autonomous navigation calculation processing, thereby detecting the position. Correct the result.
In addition, the navigation processing unit 23 executes processing such as route search and route guidance of the own vehicle in response to an input operation by the operator, for example, and displays, for example, route information to the destination and various additional information along with the road data. In addition to outputting to the unit 21, various voice messages are output from the speaker 18.

  For example, as illustrated in FIG. 1, the processing unit 11 includes an object detection unit 31, an external illuminance acquisition unit 32, an operation detection unit 33, a pupil state detection unit 34, a region expansion unit 35, and a notification control unit 36. And is configured.

The object detection unit 31 detects an object existing outside the vehicle based on the infrared images output from the infrared cameras 12L and 12R.
The external illuminance acquisition unit 32 detects a change in illuminance outside the vehicle based on the detection signal output from the illuminance sensor 16, or estimates the occurrence of a change in illuminance outside the vehicle by a predetermined estimation operation.
The predetermined estimation operation includes, for example, a case where an oncoming vehicle approaching the host vehicle is detected by the object detection unit 31, and the appearance of a heat source having a luminance equal to or higher than a predetermined luminance on the infrared images output from the infrared cameras 12L and 12R. Alternatively, it is estimated that an illuminance change equal to or more than a predetermined change has occurred in at least one of the case where the disappearance is detected and the case where the time zone of sunset or sunrise is detected.

The operation detection unit 33 detects whether or not the driver has operated the display device 19 and whether or not the navigation device 20 has been operated.
The pupil state detection unit 34 is based on the captured image output from the vehicle interior camera 17 that images the driver's eyes, and whether or not a change in the driver's pupil state (for example, the size of the pupil is greater than or equal to a predetermined value). Or the like).

For example, the area expanding unit 35 performs a predetermined operation detected by the operation detecting unit 33 for a predetermined time or more when the illuminance change outside the vehicle acquired by the external illuminance acquiring unit 32 is a predetermined change or more. The predetermined notification target area is expanded over a predetermined period in at least one of the case where the change in the pupil state detected by the pupil state detection unit 34 is equal to or greater than the predetermined change.
In the normal time, the notification target area is farther than a predetermined light distribution area LA of the headlight of the own vehicle in a predetermined imaging area SA of the infrared cameras 12L and 12R, for example, as shown in FIG. Area (normal area) WA. When the area expansion unit 35 has instructed the area expansion of the notification target area, the notification target area is a normal one of the predetermined imaging areas SA of the infrared cameras 12L and 12R, as shown in FIG. 3B, for example. The extended area EA is composed of the area WA and the area α extended from the normal area WA to the light distribution area LA at least toward the host vehicle P.

  Note that the region α that is expanded from the normal region WA to the light distribution region LA when the region is expanded has a predetermined width xa corresponding to the vehicle width of the host vehicle P, for example, as shown in FIG. It may be formed to extend toward. For example, as shown in FIG. 4B, the predetermined width xa corresponding to the vehicle width of the host vehicle P may be extended to an appropriate extension width xb and extended toward the host vehicle P. Good.

  The notification control unit 36 notifies the driver at least by the display device 19 when the target detected by the target detection unit 31 is present in a predetermined notification target area in the real space outside the vehicle.

  The vehicle periphery monitoring device 10 according to the present embodiment has the above-described configuration. Next, the operation of the vehicle periphery monitoring device 10, particularly the notification target region setting process, and other objects such as other vehicles and pedestrians are processed. The notification operation will be described with reference to the accompanying drawings.

First, for example, in step S1 shown in FIG. 5, the processing unit 11 acquires an infrared image that is an output signal of the infrared cameras 12L and 12R.
Next, in step S2, the acquired infrared image is A / D converted.
Next, in step S3, a grayscale image including halftone information is acquired and stored in the image memory. In this embodiment, the right image is obtained by the infrared camera 12R, and the left image is obtained by the infrared camera 12L. In addition, since the horizontal position of the same object on the display screen is shifted in the right image and the left image, the distance from the host vehicle to the object can be calculated based on this shift (that is, parallax). it can.

Next, in step S4, the right image obtained by the infrared camera 12R is used as a reference image, and binarization processing of this image signal, that is, a region brighter than the predetermined luminance threshold ITH is set to “1” (white), and a dark region Is set to “0” (black).
In addition, the process of the following step S4-step S9 is performed about the reference | standard image (for example, right image) obtained by the binarization process.
Next, in step S5, image data obtained by performing binarization processing on the infrared image is converted into run-length data. In the run-length data, an area that has become white by binarization processing is displayed as a line at the pixel level, and each line has a width of one pixel in the y direction and an appropriate number of pixels in the x direction. Is set to

Next, in step S6, the object is labeled in the image data converted into run-length data.
Next, in step S7, the object is extracted according to the labeling of the object. Here, among the lines in the run-length data, when the lines including the equivalent x-direction coordinates are adjacent in the y direction, the adjacent lines are regarded as constituting a single object.
Next, in step S8, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle are calculated.

Here, the area S is calculated by integrating the lengths of the run length data in the same object. The coordinates of the center of gravity G are calculated by dividing the value obtained by multiplying the values obtained by multiplying the length of the run-length data by the coordinates of the run-length data in the same object by the area S. Further, the aspect ratio ASPECT is calculated as the ratio Dy / Dx between the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object.
Note that the coordinates of the center of gravity G may be replaced with the center of gravity of a circumscribed rectangle.

Next, the process of step S9 and step S10 and the process of step S11-step S13 are performed in parallel.
First, in step S9, tracking of a target object between times, that is, recognition of the same target object at every sampling period is performed. In this inter-time tracking, k is a time obtained by discretizing time t as an analog quantity with a sampling period. For example, when objects A and B are extracted at time k, the objects C and D extracted at time (k + 1) are extracted. And the objects A and B are determined. If it is determined that the objects A and B and the objects C and D are the same, the labels of the objects C and D are changed to the labels of the objects A and B, respectively. Then, the position coordinates (for example, the center of gravity, etc.) of each recognized object are stored in an appropriate memory as time series position data.

  Next, in step S10, the vehicle speed VCAR detected by the vehicle speed sensor 14 and the yaw rate YR detected by the yaw rate sensor 13 are acquired, and the yaw rate YR is integrated over time to calculate the turning angle θr of the host vehicle.

On the other hand, in steps S11 to S13 executed in parallel with the processes in steps S9 and S10, a distance z between the object and the host vehicle is calculated. Since the process of step S11 requires a longer calculation time than the processes of step S9 and step S10, the cycle is longer than that of steps S9 and S10 (for example, about three times the execution cycle of steps S1 to S10). Etc.).
First, in step S11, one object is selected from a plurality of objects tracked on image data obtained by binarizing a reference image (for example, the right image). The entire region surrounding the object with a circumscribed rectangle is extracted from the reference image (for example, the right image) as the search image R1.

  Next, in step S12, a search area for searching for an image (corresponding image) R2 corresponding to the search image R1 from an image (for example, the left image) corresponding to the reference image (for example, the right image) is set and correlated. An operation is executed to extract the corresponding image R2. Here, for example, a search area is set in the left image according to each vertex coordinate of the search image R1, and the luminance difference sum C (a, b) indicating the level of correlation with the search image R1 in the search area. And a region where the luminance difference total value C (a, b) is minimum is extracted as the corresponding image R2. This correlation calculation is performed not on the image data obtained by the binarization process but on the gray scale image. In addition, when past position data for the same object exists, the search area can be narrowed based on the past position data.

  Next, in step S13, the gravity center position of the search image R1, the gravity center position of the corresponding image R2, and the parallax Δd in units of the number of pixels are calculated, and further, the distance between the host vehicle and the object, that is, the infrared camera 12L. , 12R distance between the lens and the object (object distance) z (m), for example, as shown in the following formula (1), the camera base length, that is, between the center positions of the image sensors of the infrared cameras 12L, 12R Calculated based on the horizontal distance D (m), the camera focal length, that is, the focal length f (m) of each lens of the infrared cameras 12L and 12R, the pixel pitch p (m / pixel), and the parallax Δd (pixel). To do.

In step S14, after the calculation of the turning angle θr in step S10 and the calculation of the distance z in step S13, for example, as shown in the following formula (2), the object on the image data is calculated. The coordinates (x, y) and the distance z are converted into real space coordinates (X, Y, Z).
Here, the real space coordinates (X, Y, Z), for example, as shown in FIG. 2, are set with the origin O as the midpoint position of the attachment positions of the infrared cameras 12L, 12R in the front part of the host vehicle P, The coordinates on the image data are set with the center of the image data as the origin and the horizontal direction as the x direction and the vertical direction as the y direction. Also, the coordinates (xc, yc) are the coordinates (x, y) on the reference image (for example, the right image) based on the relative positional relationship between the mounting position of the infrared camera 2R and the origin O in the real space. This is a coordinate obtained by converting the origin O of the real space and the center of the image data into coordinates on a virtual image obtained.

  Next, in step S15, turning angle correction is performed to correct a positional shift on the image due to the turning of the host vehicle. This turning angle correction is performed when the host vehicle turns, for example, to the left by the turning angle θr during the period from time k to time (k + 1), on the image data obtained by the infrared cameras 12L and 12R. Is a process for correcting the shift in the x direction by Δx. For example, as shown in the following formula (3), the corrected coordinates (Xr, Yr, Zr) obtained by correcting the real space coordinates (X, Y, Z) ) Are newly set as real space coordinates (X, Y, Z).

Next, in step S16, from the real space position data forming N (for example, about N = 10) time-series data after the turning angle correction obtained within the monitoring period of the predetermined time ΔT for the same object, An approximate straight line LMV corresponding to the relative movement vector between the object and the host vehicle is calculated.
In this step S16, the latest position coordinate P (0) and the position coordinate P (N-1) before (N-1) samples (that is, before the predetermined time ΔT) are set to positions on the approximate straight line LMV. The corrected position coordinates Pv (0) and Pv (N-1) are calculated.
Thereby, a relative movement vector is obtained as a vector from the position coordinates Pv (N−1) to Pv (0).
In this way, by calculating an approximate straight line that approximates the relative movement trajectory of the object with respect to the host vehicle from a plurality of (for example, N) real space position data within the monitoring period of the predetermined time ΔT, a relative movement vector is obtained. It is possible to reduce the influence of the position detection error and accurately predict the possibility of occurrence of contact between the host vehicle and the object.

Next, in step S17, a notification determination process described later is executed to set whether or not the object is a notification target.
Next, in step S18, it is determined whether there is a notification target.
If this determination is “NO”, the flow proceeds to return.
On the other hand, if the determination is “YES”, the flow proceeds to step S19.
Next, in step S19, for example, based on a detection result signal output from the brake sensor 15, an alarm output is determined by an alarm output determination process depending on whether or not the driver of the host vehicle is operating a brake. Determine if necessary.
If this determination is “NO”, the flow proceeds to return.
On the other hand, if the determination is “YES”, the flow proceeds to step S20.

In step S20, for example, an audio alarm such as a sound output from the speaker 18, a visual alarm such as a display output from the display unit 21 or the display device 19 of the navigation device, or a predetermined alarm is applied to the seat belt, for example. A tactile alarm is generated by causing the driver to apply a tactile force that can be perceived tactilely or to generate vibration that can be perceived tactilely by the driver (steering vibration) on the steering wheel. To do.
Next, in step S21, the display device 19 outputs image data obtained by, for example, the infrared camera 2R, displays the object to be notified as an emphasized video, and proceeds to return.

Hereinafter, the notification determination process in step S17 described above will be described with reference to the accompanying drawings.
First, for example, in step S31 shown in FIG. 6, it is determined whether or not there is a possibility of contact between the object and the host vehicle. Hereinafter, as an example of this determination process, for example, as shown in FIG. 7, there is a case where there is an object Q that moves at a speed Vp in a direction substantially orthogonal to the traveling direction (for example, the Z direction) of the host vehicle P. explain. When the object Q approaches the distance Zv (0) from the distance Zv (N−1) along the direction parallel to the traveling direction of the host vehicle P during the predetermined time ΔT, The relative speed Vs is calculated, and it is assumed that the host vehicle P and the object Q move within a predetermined ground height H while maintaining the relative speed Vs. It is determined whether or not. The margin time Ts is a time for determining the possibility of occurrence of contact at a timing earlier than the predicted contact occurrence time by a predetermined time Ts, and is set to about 2 to 5 seconds, for example. Further, the predetermined ground height H is set to a value that is about twice the vehicle height of the host vehicle P, for example.
If this determination is “NO”, the flow proceeds to step S 32, and in this step S 32, it is determined that the object is not a notification target, and the flow proceeds to return.
On the other hand, if the determination is “YES”, the flow proceeds to step S33.

Next, in step S33, an area determination process described later is executed.
Next, in step S34, it is determined whether or not an area extension of the notification target area is instructed.
If this determination is “NO”, the flow proceeds to step S 35, and in this step S 35, the notification target area is set as the normal area WA and the process proceeds to step S 37.
On the other hand, if this determination result is "YES", the process proceeds to step S36, and in this step S36, the notification target area is expanded to the extended area EA, and the process proceeds to step S37.
Next, in step S37, it is determined whether or not the object is present in the notification target area.
If this determination is “NO”, the flow proceeds to step S 32 described above.
On the other hand, if this determination is “YES”, the flow proceeds to step S 38, and in this step S 38, the object is determined to be a notification target and the flow proceeds to return.

Hereinafter, the region determination processing in step S33 described above will be described with reference to the accompanying drawings.
This area determination process is a process of setting whether or not to output an area extension instruction for instructing to extend the notification target area from the normal area WA to the extended area EA over a predetermined period.
The area expansion instruction is output when the driver's visual recognition ability temporarily decreases due to, for example, the driver's pupil size temporarily decreasing. For example, when the illuminance change outside the vehicle acquired by the external illuminance acquisition unit 32 is greater than or equal to a predetermined change, when the predetermined operation detected by the operation detection unit 33 is performed for a predetermined time or more, and the pupil state An area expansion instruction is output in at least one of the cases where the change in the state of the pupil detected by the detection unit 34 is greater than or equal to a predetermined change.

Hereinafter, assuming that the illuminance change outside the vehicle acquired by the external illuminance acquisition unit 32 is greater than or equal to a predetermined change, the occurrence of the illuminance change outside the vehicle is estimated by a predetermined estimation operation, and the region expansion is performed according to the estimation result A process for setting whether to output an instruction will be described.
First, in step S41 shown in FIG. 8, for example, a predetermined time after the oncoming vehicle detected by the object detection unit 31 based on the infrared images output from the infrared cameras 12L and 12R disappears due to the passing of the own vehicle. It is determined whether it is within.
When the determination result is “YES”, for example, it is determined that the pupil size of the driver of the own vehicle is temporarily reduced by the headlamp of the oncoming vehicle, and the process proceeds to step S42. In S42, an area expansion instruction is output and the process proceeds to return.
On the other hand, if this determination is “NO”, the flow proceeds to step S 43.

Next, in step S43, it is determined whether or not the object detected by the object detection unit 31 is a vehicle based on the infrared images output from the infrared cameras 12L and 12R.
If the determination result is “NO”, the region expansion instruction is not output and the process proceeds to return.
On the other hand, if this determination is “YES”, the flow proceeds to step S44.
Next, in step S44, while calculating the relative movement vector of the target object (vehicle) with respect to the own vehicle, a predetermined pattern matching process (for example, a pattern matching process based on the position of the headlamp) is performed.
Next, in step S45, based on the relative movement vector and the pattern matching calculation result, it is determined whether or not the object (vehicle) is an oncoming vehicle.
If the determination result is “NO”, the region expansion instruction is not output and the process proceeds to return.
On the other hand, if this determination is “YES”, for example, it is determined that the pupil size of the driver of the own vehicle has temporarily decreased due to the headlamp of the oncoming vehicle, and the process proceeds to step S42. In step S42, an area expansion instruction is output and the process proceeds to return.

Further, for example, in step 51 shown in FIG. 9, a predetermined time has elapsed after the object detected by the object detection unit 31 based on the infrared images output from the infrared cameras 12L and 12R disappears due to, for example, blinking of a light. It is determined whether it is within the range.
If the determination result is “YES”, for example, it is determined that the size of the pupil of the driver of the host vehicle has temporarily decreased due to an object such as a traffic light or a lighting object on the road, and the process proceeds to step S52. In step S52, an area expansion instruction is output, and the process proceeds to return.
On the other hand, if this determination is “NO”, the flow proceeds to step S 43.

Next, in step 53, whether or not the object detected by the object detection unit 31 based on the infrared images output from the infrared cameras 12L and 12R has appeared within a predetermined time since, for example, lighting has occurred. Determine.
If the determination result is “YES”, for example, it is determined that the size of the pupil of the driver of the host vehicle has temporarily decreased due to an object such as a traffic light or a lighting object on the road, and the process proceeds to step S52. In step S52, an area expansion instruction is output, and the process proceeds to return.
On the other hand, if this determination is “NO”, the flow proceeds to step S54.

Next, in step S54, it is determined whether or not the object detected by the object detection unit 31 is a person other than a person based on the infrared images output from the infrared cameras 12L and 12R.
If the determination result is “NO”, the region expansion instruction is not output and the process proceeds to return.
On the other hand, if this determination is “YES”, the flow proceeds to step S55.
Next, in step S55, when the object detected by the object detection unit 31 based on the infrared images output from the infrared cameras 12L and 12R is, for example, a light from a traffic light or a lighting object on the road, It is determined whether or not the size is a predetermined value or more.
Note that whether or not the target object is, for example, a traffic light on a road or a lighting object is a characteristic that a person (pedestrian) cannot, for example, has a plurality of heating elements spaced at predetermined intervals along the vertical direction. It is determined by pattern matching processing or the like of whether or not they are arranged.
If the determination result is “NO”, the region expansion instruction is not output and the process proceeds to return.
On the other hand, if this determination is “YES”, it is determined that the pupil size of the driver of the vehicle has temporarily decreased due to, for example, a traffic light on the road or a lighting object, and the process proceeds to step S52. In step S52, an area expansion instruction is output, and the process proceeds to return.

Hereinafter, a process of outputting an area expansion instruction when a predetermined operation detected by the operation detection unit 33 is performed for a predetermined time or more will be described.
For example, in step 61 shown in FIG. 10, an operation state by a driver of a device related to various displays such as the display device 19 and the navigation device 20 is detected.
Next, in step S62, it is determined whether or not there is an operation input of the navigation device 20 by the driver.
When this determination result is “YES”, it is determined that the size of the pupil of the driver of the own vehicle is temporarily reduced by visually observing the display unit 21 of the navigation device 20, for example. The process proceeds to step S63. In step S63, an area extension instruction is output, and the process proceeds to return.
On the other hand, if this determination is “NO”, the flow proceeds to step S64.
Next, in step S64, it is directly or indirectly determined whether or not the screen viewing by the driver has continued for a predetermined time or more. In the indirect determination, for example, by determining that a predetermined operation (for example, route guidance of the navigation device 20) of various devices such as the navigation device 20 is being performed, a screen of a predetermined time or more by the driver. It is determined that execution of predetermined operations of various devices is instructed by an operation input accompanied by visual inspection.
If the determination result is “YES”, it is determined that the size of the pupil of the driver of the host vehicle is temporarily reduced by viewing the screen for a predetermined time or longer, and the process proceeds to step S63. In, an area extension instruction is output and the process proceeds to return.
On the other hand, if the determination result is “NO”, the area expansion instruction is not output and the process proceeds to return.

Hereinafter, a process of outputting a region expansion instruction when the change in the state of the pupil detected by the pupil state detection unit 34 is equal to or greater than a predetermined change will be described.
For example, in step 71 shown in FIG. 11, the state of the driver's pupil (for example, the size of the pupil) is detected based on the captured image output from the vehicle interior camera 17 that images the driver's eyes.
Next, in step S72, it is determined whether or not the size of the pupil is equal to or smaller than a predetermined value.
If the determination result is “NO”, the region expansion instruction is not output and the process proceeds to return.
On the other hand, if this determination is “YES”, the flow proceeds to step S73.
Next, in step S73, it is determined whether or not the state in which the pupil size is a predetermined value or more has continued for a predetermined time or more.
If the determination result is “NO”, the region expansion instruction is not output and the process proceeds to return.
On the other hand, if this determination is “YES”, the flow proceeds to step S 74, and in this step S 74, an area expansion instruction is output and the flow proceeds to return.

As described above, according to the vehicle periphery monitoring device 10 according to the present embodiment, when the illuminance change outside the vehicle is greater than or equal to the predetermined change, and the operation of the in-vehicle display device by the driver is performed for a predetermined time or more. The notification target area is expanded over a predetermined period in at least one of the case where the change in the state of the driver's pupil is greater than or equal to the predetermined change. As a result, even if the external visibility of the driver is temporarily reduced due to changes in illuminance outside the vehicle or screen viewing over a predetermined time by the driver or contraction of the driver's pupil, The notification operation can be performed at an appropriate timing, and the driver can be prevented from feeling bothered by the notification operation.
In addition, since the external illuminance acquisition unit 32 estimates the occurrence of illuminance change outside the vehicle by a predetermined estimation operation, the external illuminance acquisition unit 32 acquires illuminance change that exceeds the predetermined change without the need to directly detect the illuminance change outside the vehicle. can do.

  In the above-described embodiment, for example, when the driver's visual ability temporarily decreases due to, for example, the driver's pupil size being temporarily reduced, the notification target region is Although it is assumed that the vehicle is extended from the normal area WA to the extended area EA over a predetermined period, the present invention is not limited to this. Further, when an area expansion instruction is output, a process for advancing the notification timing of various alarms May be executed.

  In the above-described embodiment, for example, in the notification determination process illustrated in FIG. 6, the determination process for the possibility of collision in step S <b> 31 may be omitted.

10 vehicle periphery monitoring device 12L, 12RL infrared camera (infrared imaging means)
31 Object detection unit (object detection means)
32 External illuminance acquisition unit (external illuminance acquisition means)
33 Operation detection unit (operation detection means)
34 Pupil state detection unit (pupil state detection means)
35 Area expansion unit (area expansion means)
36 Notification control unit (notification means)

Claims (2)

An infrared imaging means mounted on the vehicle and outputting an infrared image obtained by imaging in the infrared region;
Object detection means for detecting an object existing outside the vehicle based on the infrared image output from the infrared imaging means;
Informing means for informing the driver when the object detected by the object detecting means exists in an area beyond the light distribution area of the headlight of the vehicle, which is a predetermined area in real space outside the vehicle. A vehicle periphery monitoring device comprising:
An external illuminance acquisition means for detecting or estimating an illuminance change outside the vehicle;
A region expansion unit that expands the predetermined region into a light distribution region of the headlight of the vehicle over a predetermined period when the illuminance change acquired by the external illuminance acquisition unit is greater than or equal to a predetermined change. The vehicle periphery monitoring apparatus characterized by the above-mentioned. The external illuminance acquisition means includes
A case where an oncoming vehicle approaching the host vehicle is detected, a case where an appearance or disappearance of a heat source having a luminance equal to or higher than a predetermined luminance is detected on the infrared image, and a case where a time zone of sunset or sunrise is detected 2. The vehicle periphery monitoring device according to claim 1, wherein the illuminance change equal to or greater than the predetermined change is acquired in at least one of the cases.

Images