JP2009116539A - Peripheral monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a peripheral monitoring device, which performs peripheral monitoring and monitoring result report considering the state of the circumference. <P>SOLUTION: The peripheral monitoring device includes a circumferential state detection part 12 which detects a circumferential state of a vehicle by calculating a distance to an object on the side of a vehicle mounted with the own device; and a pedestrian detection part 13 which detects a pedestrian present in front of the vehicle based on a forward photographic image of the vehicle mounted with the own device. The pedestrian detection part 13 raises the pedestrian detection accuracy when the distance calculation result by the circumferential state detection part 12 is a predetermined threshold or less. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technology for supporting driving of an automobile, and in particular, surrounding monitoring for notifying a driver of a surrounding situation when traveling on a narrow road using information acquired from a camera or the like attached to a vehicle body. Relates to the device.

  2. Description of the Related Art In recent years, various techniques have been proposed in which a video attached to a vehicle body is used to capture a video outside the vehicle and the resulting image is used to assist driving of the automobile.

  For example, in the technique described in Patent Document 1 below, an image captured by a camera built in a door mirror is displayed on a display panel in front of the driver's seat so that the driver can The state can be recognized. Further, when the door mirror is pulled out (opened), the side state can be recognized.

  Patent Document 2 listed below describes a technique for performing parking assistance using an image of a situation outside the vehicle.

  Further, Patent Document 3 below provides an image that is easy for the driver to grasp by photographing a blind spot from the driver, converting the captured image as appropriate, and displaying it on a panel in front of the driver's seat. The technology to support driving is described.

JP 2003-320898 A JP 2004-148901 A JP 2004-140771 A

  When driving a car, the points that the driver should be aware of change according to changes in the surrounding conditions, such as the width of the road, the traffic volume, and the number of bicycles and pedestrians. Therefore, even when driving assistance is performed by monitoring the surroundings, it is desirable to change the content of driving assistance according to the surrounding situation. In particular, when traveling on a narrow road, it is necessary to pay attention to pedestrians while paying attention to the positional relationship between the host vehicle and other vehicles and the positional relationship between the host vehicle and surrounding buildings. That is, since there is a higher risk of distracting attention to pedestrians than usual, it is considered that a driving support mechanism that considers such cases is useful for preventing accidents. However, in the conventional driving assistance technology represented by the above Patent Documents 1 to 3, processing in consideration of the surrounding situation has not been performed.

  The present invention has been made in order to solve the above-described problems caused by the prior art, and provides a peripheral monitoring device that contributes to accident prevention by performing peripheral monitoring and monitoring result notification in consideration of surrounding conditions. Objective.

  In order to solve the above-described problems and achieve the object, the periphery monitoring device according to the present invention is a surrounding that detects the surrounding state of the vehicle by calculating the distance to the object on the side of the vehicle on which the device is mounted. A pedestrian detection unit that detects a pedestrian existing in front of the vehicle based on a captured image in front of the vehicle on which the device is mounted, and the pedestrian detection unit includes the ambient state When the distance calculation result by the detection means is equal to or less than a predetermined threshold value α, the pedestrian detection accuracy is increased.

  According to the present invention, the distance to the object on the side is monitored, and if the risk is determined to be high as a result of the monitoring, the pedestrian recognition accuracy is increased. There is an effect that a pedestrian can be detected.

  Exemplary embodiments of a periphery monitoring device according to the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration example of a periphery monitoring device according to the present invention. As shown in FIG. 1, the periphery monitoring device 1 is connected to a surrounding state notification system 50, a vehicle control system 60, an in-vehicle notification system 70, and a navigation device 80.

  The surrounding situation notification system 50 is a group of devices that notify the surroundings monitoring apparatus 1 of the surrounding situation of the vehicle, and includes a camera 51 for photographing the surroundings of the vehicle, an object detector 52 for detecting objects around the vehicle, and the vehicle. It includes a photodetector 53 for detecting ambient brightness. There are a plurality of cameras 51. The object detection unit 52 is, for example, a millimeter wave radar, an ultrasonic sensor, a radio wave radar, a laser radar, or the like.

  The vehicle control system 60 is a group of devices that control the operation of the vehicle, an engine control mechanism 61 that controls the operation of the engine based on the accelerator operation, a braking mechanism 62 that brakes the vehicle based on the operation of the brake pedal, and a handle. A steering mechanism 63 that controls the steering angle of the vehicle based on the operation, a mirror control mechanism 64 that controls the opening and closing of the door mirror, and the like are included.

  The in-vehicle notification system 70 is a group of devices that notify a passenger of the host vehicle, and includes a speaker 71 that performs voice notification, a display 71 that performs display notification, and the like. This in-vehicle notification system 70 can be shared by various in-vehicle devices such as a navigation device and an in-vehicle audio device in addition to the periphery monitoring device 1.

  The periphery monitoring device 1 includes a state detection unit 10, a vehicle control unit 20, a monitoring result notification unit 30 corresponding to a notification unit, and a setting state storage unit 41.

  The state detection unit 10 detects various states based on information acquired from the surrounding state notification system 50, and includes a traveling state detection unit 11 that detects a traveling state of the host vehicle, a road width, and ambient brightness. And the like, and a pedestrian detection unit 13 that determines the presence or absence of a pedestrian.

  The vehicle control unit 20 gives an operation instruction to each part of the connected vehicle control system 60, a mirror control unit 21 that gives an instruction to open and close the door mirror to the connected vehicle control system 60, and a brake ( And a braking mechanism control unit 22 for instructing control of the braking device). The mirror control unit 21 performs processing as mirror control instruction means, and the brake mechanism control unit 22 performs processing as deceleration instruction means.

  The monitoring result notification unit 30 notifies the driver of the state detection result (monitoring result) by the state detection unit 10 by controlling each part of the in-vehicle notification system 70, and instructs the driver to notify the monitoring result by display. And a voice control unit 32 that gives an instruction to notify the monitoring result by voice.

  The setting state storage unit 41 stores settings for a monitoring result notification method and the like.

  FIG. 2 is a diagram for explaining a monitoring range by the camera 51 and the object detector 52, and shows a case where an ultrasonic sensor is used as the object detector 52 (using radio wave radar, laser data, etc.). May be good). As shown in FIG. 2, the vehicle of this embodiment includes a plurality of cameras 51 for photographing a plurality of front directions (front, left front, right front) and rear (rear front), and a right side and a left side. And a plurality of ultrasonic sensors (object detector 52) for detecting the object. It is desirable that the object detector 52 be attached at a position close to the front of the vehicle so that the side state can be detected quickly during normal travel (a state of moving forward).

  First, basic functions of the periphery monitoring device 1 will be briefly described. The periphery monitoring device 1 has a function of displaying an image captured by the camera 51 on a display panel (display 71) installed in front of the driver's seat. Specifically, the display control unit 31 of the monitoring result notification unit 30 acquires a captured image by the camera 51 via the state detection unit 10, processes the acquired image data as necessary, and displays the acquired image data on the display 71. Which direction the captured image is displayed (a plurality of captured images can be displayed) is appropriately changed according to the driver's instruction and the surrounding monitoring result.

  The periphery monitoring device 1 has a function of determining whether or not there is a pedestrian (person) in the vicinity (within the image capturing range of the camera 51) by analyzing an image captured by the camera 51. The process of analyzing the image and determining whether there is a pedestrian is executed by the pedestrian detection unit 13 of the state detection unit 10. However, the direction opposite to the traveling direction of the vehicle is excluded from the determination process. For example, when the vehicle is moving forward, the determination process for the rear is not executed. In addition, when the vehicle is stopped, determination processing is executed for all directions. In the present embodiment, it is assumed that the determination process for determining whether or not there is a pedestrian is continuously executed every predetermined period.

  In addition, the periphery monitoring device 1 has a function of determining whether the host vehicle is traveling in an intersection by analyzing an image captured by the camera 51. The process of determining whether or not the vehicle is traveling within an intersection is executed by the traveling state detection unit 11 of the state detection unit 10. Instead of the determination by image analysis, the determination may be made using GPS or the like that the navigation apparatus has.

  Next, the periphery monitoring operation by the periphery monitoring device 1 of the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing an example of the periphery monitoring operation.

  The state detection unit 10 of the periphery monitoring device 1 monitors whether or not the vehicle is traveling on a narrow road (whether there is a high risk of the vehicle coming into contact with a side object) (step S1). Specifically, based on the information acquired from the object detector 52, the surrounding state detection unit 12 calculates the distance to the side object, and the calculation result is equal to or less than a predetermined value (α). Judge that you are driving on a narrow road.

  The ambient state detection unit 12 calculates the distance to the object on the right side and the distance to the object on the left side. If at least one of the calculation results is equal to or less than a predetermined value (α), the narrow state is detected. Judge that the vehicle is running. That is, even if the vehicle is traveling on a sufficiently wide road, if the distance between one of the side surfaces and the object is short, there is a high risk of contact, and therefore it is considered that the vehicle is traveling on a narrow road. Therefore, in the following description, the phrase “running on a narrow road” indicates a case where “the distance between any side surface of the vehicle and the object is equal to or less than a predetermined value (α)”.

  As a result of monitoring, when it is determined that the vehicle is not traveling on a narrow road (the risk of contact is low) (No in step S1), monitoring is continued. On the other hand, when it is determined that the vehicle is traveling on a narrow road (step S1, Yes), the state detection unit 10 notifies the vehicle control unit 20 to that effect, and in the vehicle control unit 20, the mirror control unit 21 is a mirror. Control is started (step S2).

  Here, the mirror control operation will be described with reference to FIG. FIG. 4 is a flowchart showing an example of the mirror control operation. When the notification that the vehicle is traveling on a narrow road is received from the surrounding state detection unit 12, the mirror control unit 21 of the vehicle control unit 20 checks whether or not the door mirror may collide with a side object ( Step S21). For example, the distance to the side object (the distance to the right side object and the distance to the left side object) calculated by the surrounding state detection unit 12 is acquired, and at least one of them is a predetermined threshold value (β However, if β <α) or less, it is determined as “possible”.

  If it is determined that there is a possibility (step S21, Yes), the door mirror is closed (stored) (steps S22, S23, S24). That is, when the vehicle is traveling with the door mirror open until then, the mirror control unit 21 first stores the door mirror by instructing the mirror control mechanism 64 to close the door mirror (step S22). Then, the mirror control unit 21 notifies the monitoring result notification unit 30 that the door mirror has been stored. In the monitoring result notification unit 30, the display control unit 31 displays the rear video on the monitor (step S23). Specifically, the display control unit 31 acquires an image captured by the camera 51 via the state detection unit 10 and controls the display 71 to display the image. FIG. 5 is a diagram showing a display example on the monitor (display 71) when the rear image is displayed. In this example, in addition to the right front image and the left front image, the rear image is displayed. It is displayed.

  Then, the monitoring result notification unit 30 informs by voice that the voice control unit 32 has shifted to the state in which the door mirror is stored via the speaker 72 (step S24). On the other hand, when the vehicle is traveling with the door mirror stored, the door mirror is not controlled.

  If it is determined that there is no possibility in step S21 (step S21, No), the vehicle travels with the door mirror opened (steps S25, S26, S27). That is, when the vehicle is traveling with the door mirror opened until then, the mirror control unit 21 does not control the door mirror. In contrast, when the vehicle is traveling with the door mirror closed, the door mirror is first opened (step S25), and the display on the monitor is changed to the normal display (step S26). Note that “normal display” refers to display of a navigation screen if, for example, the navigation device is in use. Then, it is notified by voice that the door mirror has been opened (step S27).

  Returning to the description of FIG. 3, after notifying the mirror control unit 21 that the vehicle is traveling on a narrow road, the state detection unit 10 confirms whether or not the vehicle is traveling at an intersection (step S3). Specifically, it is determined whether or not the traveling state detection unit 11 is traveling at an intersection based on an image acquired from the camera 51. Or you may judge an intersection position based on the information acquired from the navigation apparatus.

  When the vehicle is not traveling at the intersection (step S3, No), the pedestrian recognition accuracy in the front direction is increased from the pedestrian recognition accuracy when the vehicle is not traveling on a narrow road (step S4). Here, “traveling an intersection” means a state where the vehicle is traveling between a point where it has been determined that the vehicle has entered the intersection before the intersection is reached and a point where it has been determined that the vehicle has passed the intersection. Further, “pedestrian recognition accuracy” refers to whether or not there is a pedestrian (person) within the shooting range of the camera 51 based on the image data acquired from the camera 51 by the pedestrian detection unit 13 described above. Recognition accuracy in the processing (hereinafter referred to as pedestrian detection processing). “Increase recognition accuracy” means, for example, that pedestrian detection processing is performed using more detailed image data compared to a case where the vehicle is not traveling on a narrow road (a state where the vehicle is not traveling on a narrow road). If the detection process is performed by thinning out the pixels at, stop the thinning and perform the detection process, etc.) "," Increase the frequency of execution of the pedestrian detection process that is continuously executed every predetermined period (execution Shorten the cycle) ”. Note that both using more detailed image data and increasing the execution frequency may be executed, or only one of them may be executed.

  After step S4 is executed, the process proceeds to step S1, and step S1 and subsequent processing are repeated. Although not shown in FIG. 3, if it is determined that the vehicle is not traveling on a narrow road at step S1 (step S1, No), and the vehicle has been traveling with the door mirror stored therein until then, Then, the door mirror opening process (corresponding to steps S25 to S27 shown in FIG. 4) is executed.

  If it is determined that the vehicle is traveling at the intersection in step S3 (step S3, Yes), the pedestrian recognition accuracy of the front, right front, and left front in the pedestrian detection process executed by the pedestrian detection unit 13 is reduced to a narrow road. Is higher than the pedestrian recognition accuracy when not running (step S5).

  Further, the display control unit 31 causes the display 71 to display a left-right forward video (a video taken by the left front camera and the right front camera shown in FIG. 2) (step S6). FIG. 6 is a diagram showing a display example on the display 71 in the case where the left front and right front images are simultaneously displayed. The right front image is displayed on the right half of the display screen, and the left front direction is displayed on the left half. The example in the case of displaying the image of is shown.

  Furthermore, the periphery monitoring device 1 checks whether or not the vehicle turns at the intersection (whether to perform a right turn or a left turn) (step S7). Specifically, the traveling state detection unit 11 determines whether or not the vehicle is turning based on, for example, the state of a direction indicator or steering (both not shown).

  When the vehicle goes straight without turning (No at Step S7), the process proceeds to Step S1, and Step S1 and subsequent processes are repeated.

  On the other hand, in the case of turning (step S7, Yes), the display of the turning direction is emphasized (step S8). As a method of emphasis, increase the display ratio on the display 71 in the direction of the turn (change the direction that was displayed at the same ratio in both directions to 6: 4 or 7: 3 when going straight), and reverse the direction of the turn It is possible to make the display of gray.

  Thereafter, it is monitored whether the right turn or the left turn is finished (step S9), and when finished (step S9, Yes), the pedestrian recognition accuracy and the display on the display 71 are initialized (step S10). Thereafter, step S1 and subsequent processing are repeated.

  Subsequently, the pedestrian detection process executed by the periphery monitoring device 1 in parallel with the periphery monitoring operation shown in FIG. 3 and the operation when it is detected will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of the operation of the periphery monitoring device 1 when a pedestrian is detected.

  The pedestrian detection unit 13 of the periphery monitoring device 1 monitors whether there is a pedestrian in the vicinity based on the image acquired from the camera 51 for each predetermined period (step S31). Specifically, the image captured by the camera 51 is analyzed, and it is determined whether or not a pedestrian exists within the capturing range of the camera 51. At this time, if the vehicle is moving forward, the images taken by the front direction camera, the left front direction camera, and the right front direction camera (see FIG. 2) are analyzed to determine the presence or absence of a pedestrian in these three directions. .

  When a pedestrian is detected as a result of monitoring (step S31, Yes), the pedestrian detection unit 13 notifies the monitoring result notification unit 30 to that effect.

  When the notification that the pedestrian has been detected is received, in the monitoring result notification unit 30, the voice control unit 32 is connected to the direction corresponding to the position of the pedestrian via the speaker 72 (the direction in which the pedestrian is detected, the front The voice from the left front direction or the right front direction) is notified (step S32). As described above, the driver can easily recognize in which direction the pedestrian exists in a short time by performing the notification by the voice from the same direction as the position where the pedestrian exists.

  Next, the ambient state detection unit 13 determines whether or not the vehicle periphery is dark (whether the light detection result is equal to or less than a predetermined threshold) based on the light detection result by the light detector 53 (step S33). And when it is judged that it is dark (step S33, Yes), that is notified to the monitoring result notification part 30.

  When the notification that the vehicle periphery is dark is received, in the monitoring result notification unit 30, the display control unit 31 causes the display 71 to display the video captured by the camera 51 and the recognition result (step S34), and then proceeds to step S38. Transition. Here, “recognition result display” refers to a display in which a pedestrian existing in a video is emphasized by being surrounded by a frame. In displaying the recognition result, highlighting may be performed by enlarging and displaying the pedestrian (including the pedestrian's surroundings). Further, when control is performed so that a normal screen (for example, a navigation screen) is displayed over the video, the normal screen and the recognition result display may overlap. In this case, the display control unit 31 performs control so that the entire recognition screen is displayed as shown in FIG. When the normal screen and the recognition result display overlap, the normal screen may not be displayed.

  On the other hand, if it is determined in step S33 that the image is not dark (No in step S33), the monitoring result notification unit 30 is notified of this.

  In the monitoring result notification unit 30, the display control unit 31 confirms the setting state of the forward screen display function stored in the setting state storage unit 41. If the setting state is “ON” (Yes in step S35), The image and the recognition result are displayed on the display 71 (step S36). And it changes to step S38. On the other hand, when the setting state is “OFF” (step S35, No), the normal screen display on the display 71 is continued (step S37), and the process proceeds to step S38. That is, when the setting state is “OFF”, the fact that a pedestrian has been detected is notified only by voice.

  Next, the pedestrian detection unit 13 determines whether or not there is a possibility of colliding with the pedestrian detected in step S31 based on the image acquired from the camera 51 (step S38). If there is no possibility of collision (No at Step S38), the process is terminated. On the other hand, when there is a possibility of collision (step S38, Yes), the vehicle control unit 20 is notified that there is a possibility of collision with a pedestrian, and the vehicle control unit 20 performs brake control. Do. That is, in the vehicle control unit 20, the brake mechanism control unit 22 instructs the brake mechanism 62 of the vehicle control system to perform brake control (step S39). As a result, the brake control for avoiding the collision is performed and the vehicle is decelerated.

  As described above, in this embodiment, whether the vehicle is traveling on a narrow road or whether it is traveling in an intersection, etc., and the pedestrian recognition accuracy is changed according to the monitoring result. A pedestrian can be detected with recognition accuracy corresponding to the degree of danger, and can contribute to accident prevention. In addition, since the method of notifying the monitoring result is changed according to the surrounding situation and the running state, the monitoring result can be notified to the driver by an optimum method according to the situation.

  In addition, when the distance between the side of the vehicle and the object on the side is short, the door mirror is automatically retracted, and the image taken by the camera behind the vehicle is displayed on the display panel in front of the driver seat. As a result, the rear view can be secured while preventing the door mirror from contacting the side object.

  In addition, when a pedestrian is detected, the detection result is notified by sound output from the detected direction (the direction in which the pedestrian is present), so the driver can easily determine the position of the pedestrian. Can be recognized. In addition, when the surroundings are dark, the detection result is notified by highlighting on the display panel in addition to the voice notification, so it is more reliable than when the driver directly looks outside the vehicle. The position of the pedestrian can be recognized.

  Furthermore, when there is a possibility of collision with a pedestrian, the brake control is performed, so that an accident can be prevented.

  Note that the configurations and operations shown in the above embodiments are merely examples, and do not limit the present invention. Further, the present invention can be implemented by appropriately changing the configuration and operation shown in the above embodiments.

  As described above, the periphery monitoring device according to the present invention is useful for driving support of an automobile, and particularly in a state where there is a high risk that the side surface of the vehicle is in contact with a side object, such as when driving on a narrow road. Suitable for assisting driving.

It is a figure which shows the structural example of the periphery monitoring apparatus concerning this invention. It is a figure for demonstrating the monitoring range by a camera and an object detector. It is a flowchart which shows an example of the periphery monitoring operation | movement by the periphery monitoring apparatus concerning this invention. It is a flowchart which shows an example of the mirror control operation | movement by the periphery monitoring apparatus concerning this invention. It is a figure which shows the example of a display on a monitor in the case of displaying a back image. It is a figure which shows the example of a display on a monitor in the case of displaying the image of the left front and the right front. It is a flowchart which shows an example of operation | movement when the periphery monitoring apparatus concerning this invention detects a pedestrian. It is a figure which shows the example of a display on a monitor in the case of displaying a pedestrian recognition result.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Perimeter monitoring apparatus 10 State detection part 11 Running state detection part 12 Ambient state detection part 13 Pedestrian detection part 20 Vehicle control part 21 Mirror control part 22 Braking mechanism control part 30 Monitoring result notification part 31 Display control part 32 Voice control part 41 Setting state storage unit 50 Peripheral situation notification system 51 Camera 52 Object detector 53 Photo detector 60 Vehicle control system 61 Engine control mechanism 62 Braking mechanism 63 Steering mechanism 64 Mirror control mechanism 70 In-car notification system 71 Display 72 Speaker 80 Navigation device

Claims (6)

Ambient state detection means for detecting the ambient state of the vehicle by calculating the distance to the object on the side of the vehicle on which the device is mounted;
Pedestrian detection means for detecting a pedestrian existing in front of the vehicle based on a photographed image in front of the vehicle on which the device is mounted;
With
The pedestrian detection unit increases the pedestrian detection accuracy when a calculation result by the ambient state detection unit is equal to or less than a predetermined threshold value α. further,
A driving state detecting means for determining whether or not the vehicle is driving at an intersection based on a photographed image in front of the vehicle;
With
The pedestrian detection means increases the pedestrian detection accuracy when the calculation result by the ambient state detection means is equal to or less than a predetermined threshold value α and the determination result by the running state detection means indicates that an intersection is running. The periphery monitoring device according to claim 1. further,
A notification means for notifying a driver of a photographed image in front of the vehicle and a detection result by the pedestrian detection means;
With
When it is determined that the traveling state detection unit is traveling at the intersection, the steering operation state and the direction indicator state acquired from the vehicle on which the device is mounted further determines whether to perform a right turn or a left turn. Based on at least one of
The said notification means emphasizes and displays the picked-up image of the direction which is going to bend, when it is judged by the said driving | running | working state detection means to perform a right turn or a left turn. Perimeter monitoring device. When the pedestrian detection means detects a pedestrian,
The perimeter monitoring apparatus according to claim 3, wherein the notification unit performs notification that a pedestrian has been detected by voice from a direction in which the pedestrian exists. further,
When the calculation result by the surrounding state detection means is less than or equal to a predetermined threshold value β (β <α), the vehicle on which the device is mounted is instructed to close the door mirror, while the calculation result is Mirror control instruction means for instructing to open the door mirror when the threshold β is exceeded,
The periphery monitoring device according to any one of claims 1 to 4, further comprising: further,
Deceleration instruction means for instructing deceleration to the vehicle on which the device is mounted;
With
When the pedestrian detection means detects a pedestrian, the pedestrian detection means determines whether there is a possibility of colliding with the detected pedestrian based on a photographed image in front of the vehicle,
The periphery monitoring device according to claim 1, wherein the deceleration instruction unit issues a deceleration instruction when the pedestrian detection unit determines that there is a possibility of a collision. .

Images