WO2014034065A1 - Moving body warning device and moving body warning method - Google Patents

Description

Moving body warning device and moving body warning method Cross-reference of related applications

This disclosure is based on Japanese Application No. 2012-192593 filed on August 31, 2012, the contents of which are incorporated herein by reference.

The present disclosure relates to a technique for detecting a moving object existing in front of a vehicle and warning a driver.

A technology has been proposed in which a moving body such as a pedestrian or a bicycle is detected from an image taken in front of a vehicle and a driver is warned that a moving body exists. This technology not only prevents the driver from overlooking pedestrians and the like, but also prevents the occurrence of accidents, and can reduce the burden on the driver. It has been. However, when there are many moving bodies, there is a possibility that the driver will be burdened if the warning of the existence of all these moving bodies is issued.

Therefore, instead of warning all moving bodies in the same way, an attempt is made to warn according to the danger level of the moving bodies. As one of such attempts, a technique has been proposed in which a driver's line of sight is detected and a moving body located at a position deviated from the line of sight is emphasized to display a warning to the driver (Patent Document 1).

However, the proposed technique described above has a problem that it is still difficult to reduce the burden on the driver when there are many moving objects. This is because the driver's line of sight is constantly moving, and the moving body that is highlighted and displayed in the proposed method changes one after another, which makes the driver feel annoying. Also, if a moving object is highlighted and displayed at a position outside the driver's line of sight, the driver's line of sight is naturally guided there, and attention to the part that the driver should really pay attention to is displayed. There is also a risk of scraping.

JP 2009-40107 A

This disclosure is intended to provide a moving body warning device and a moving body warning method capable of warning a driver without burden even when a large number of moving bodies exist.

In the first aspect of the present disclosure, a moving body warning device that warns a driver of a vehicle that a moving body exists in front of the vehicle captures a front image that is an image of the situation in front of the vehicle. A range that the driver is viewing in a range corresponding to the front image in the situation in front and a moving body detection device that detects the moving body from the front image A visual recognition range detection device for detecting a visual recognition range; a determination device that detects the moving object that is continuously present in the visual recognition range for a predetermined reference time; A warning device that warns the driver of the presence of the moving body in a different manner between the unrecognized moving body that is not a recognized moving body and the recognized mobile body;

In the above case, even when there are a plurality of moving bodies in the front image, the recognized moving body and the unrecognized moving body can be distinguished from the warning mode, so the burden on the driver can be reduced. . Further, since the warning is given to the unrecognized moving body in a mode different from that of the already-recognized moving body, it is possible to make the driver surely recognize the presence of the unrecognized moving body.

In the second aspect of the present disclosure, a moving body warning method for warning a driver of a vehicle that a moving body is present in front of the vehicle captures a front image that is an image of the situation ahead of the vehicle. And detecting the moving body from the front image, detecting a visual recognition range which is a range visually recognized by the driver in a range corresponding to the front image in the front situation, and Detecting the moving body that is continuously present within a predetermined reference time within a range, determining the moving body as an already-recognized moving body, and the unrecognized moving body that is not the already-recognized moving body; Warning the driver of the presence of the moving body is provided in a manner different from that of the recognized moving body.

In the above case, even when there are a plurality of moving bodies in the front image, the recognized moving body and the unrecognized moving body can be distinguished from the warning mode, so the burden on the driver can be reduced. . Further, since the warning is given to the unrecognized moving body in a mode different from that of the already-recognized moving body, it is possible to make the driver surely recognize the presence of the unrecognized moving body.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing
FIG. 1 is an explanatory diagram of a vehicle equipped with the moving body warning device of the present embodiment, FIG. 2 is a block diagram of the mobile warning device of the present embodiment. FIG. 3 is a flowchart of a mobile object warning process for warning a mobile object, FIG. 4 is an explanatory diagram illustrating a state in which the moving object is detected from the front image. FIG. 5 is an explanatory diagram showing a method for determining whether or not a moving object exists in the viewing range. FIG. 6 is a list diagram showing an outline of attributes to be given to the moving object. FIG. 7 is a flowchart of attribute assignment processing for assigning an attribute to a mobile object. FIG. 8 is a list illustrating various warning modes of the moving object, FIG. 9 is an explanatory diagram illustrating the manner in which warning modes for a plurality of moving objects change, FIG. 10 is a flowchart of attribute assignment processing according to a modification.

Hereinafter, examples will be described in order to clarify the contents of the disclosure of the present application described above.
A. Device configuration :
FIG. 1 shows a general configuration of a vehicle 1 on which a moving body warning device 10 of the present embodiment is mounted. As shown in the figure, the moving body warning device 10 of the present embodiment is mounted on a front camera 12 that captures the situation in front of the vehicle 1 and a dashboard (or steering column) of the vehicle 1 so that the driver can view the vehicle from the front. A driver camera 14 that captures an image of the face, a display 16 that displays the situation in front of the vehicle on the screen and displays a warning to the driver of the presence of a moving object (pedestrian, bicycle, car, etc.); A control unit 18 for controlling the operation is provided. Note that "warning display" is a warning that is displayed by highlighting the marker image in some way, such as overlaying the marker image, displaying the contour with emphasis, or displaying the relevant part brighter than the surroundings. This is the display form.

Further, in this embodiment, an image of the front situation (front image) taken by the front camera 12 is displayed on the display 16, and the presence of a moving object is displayed on the display 16 as a warning. However, the presence of the moving body may be displayed as a warning by projecting a warning image on the windshield over the driver's field of view from the dashboard of the vehicle 1 or the like. Alternatively, an audible warning may be given by generating a warning sound or sound that informs the driver of the presence of the moving body.

FIG. 2 shows an internal configuration of the control unit 18 of the moving object warning device 10. As illustrated, the control unit 18 includes a moving body detection unit 18a, a determination unit 18b, a visual range detection unit 18c, and a warning unit 18d. The moving body detection unit 18a, the determination unit 18b, the visual recognition range detection unit 18c, and the warning unit 18d are conceptually classified from the inside of the control unit 18 focusing on functions, and each is physically separated. Need not exist. Also, these can be realized in hardware by an integrated circuit called a so-called LSI, or can be realized in software by a computer program. Of course, a part may be realized by hardware and the remaining part may be realized by software.

When the moving body detection unit 18a receives the front image captured by the front camera 12, the moving body detection unit 18a detects a moving body such as a pedestrian, a bicycle, or a car. The viewing range detection unit 18c detects a range (viewing range) that the driver is viewing from the driver's face image captured by the driver camera 14. The determination unit 18b determines whether or not the driver recognizes the moving object for each of the moving objects detected by the moving object detection unit 18a based on the visual field range detected by the visual range detection unit 18c. . Then, the warning unit 18d displays a warning on the moving body on the screen of the display 16 based on the determination result in the determination unit 18b. At this time, by displaying a warning by the method described below, even if a plurality of moving bodies are detected from the front image, it is possible to warn the moving bodies without burdening the driver. Become.

In the present embodiment, the front camera 12 that captures the front image corresponds to the “imaging device” in the present disclosure, and the moving body detection unit 18a that detects the moving body based on the front image includes the “movement” in the present disclosure. Corresponds to “body detection device”. Further, the viewing range detection unit 18c of the present embodiment corresponds to the “viewing range detection device” in the present disclosure, and the determination unit 18b corresponds to the “determination device” in the present disclosure. In this embodiment, the warning unit 18d and the display 16 correspond to the “warning device” in the present disclosure.
B. Moving object warning processing:
FIG. 3 shows a flowchart of processing (moving body warning processing) in which the moving body warning device 10 according to the present embodiment detects a moving body existing in front of the vehicle 1 and warns the driver. This process is executed by the control unit 18 of the mobile object warning device 10.

When the moving body warning process is started, first, a front image photographed by the front camera 12 is acquired (S100). The front camera 12 has a so-called moving image shooting function, and takes a front image at a rate of 30 frames per second and outputs it to the control unit 18. In S100, image data for one frame output from the front camera 12 is acquired and stored in a memory (not shown) in the control unit 18.

Next, a moving object is detected from the front image (S102). The moving object can be detected by a method for obtaining a difference between images or a method for obtaining a correlation between images. That is, since the front camera 12 captures the front image at a rate of 30 frames per second, the image of a certain frame is almost the same as the image of the previous frame, but only the part where the moving object is shown Will be a different image. Therefore, if a difference is taken between two images, a moving object shown in the image can be detected. In addition, the moving body is shown in a certain size in the front image, and one moving body moves in substantially the same direction. Therefore, strong correlation will appear if image correlation is taken in the part where the moving object exists. By using this property, information such as the position, size, moving direction, moving speed, etc. of the moving object can be obtained. it can. In S102, the moving object existing in the front image is detected by using any one of the image difference and the correlation, or by combining these.

FIG. 4 illustrates a state in which a moving object is detected from the front image. In the illustrated example, three moving bodies A to C are detected.

Subsequently, the visible range of the driver is detected (S104). Here, the “viewing range” refers to a range within the field of view of the driver of the vehicle 1 and in which the driver can correctly recognize the object. That is, a human has a field of view close to 180 degrees, but can only recognize such things as “something” or “something moved” at the edge of the field of view. An object can be correctly recognized only within a predetermined solid angle range with a so-called line of sight as a central axis. The viewing range refers to a range of a predetermined solid angle with this line of sight as the central axis.

In the present embodiment, it is assumed that the left and right angle (Yaw angle) of the driver's face is detected from the driver's image taken from the front using the driver camera 14, and the face direction is the direction of the line of sight. To detect the viewing range.

FIG. 5 conceptually shows how the driver's visible range is detected. In the illustrated example, the driver is directed rightward by a Yaw angle with respect to the front front indicated by a one-dot chain line in the drawing. Accordingly, assuming that the face direction is the direction of the line of sight, the range of the angle α on each side is set as the viewing range. Therefore, in the illustrated example, only the moving body C among the moving bodies A to C exists within the viewing range.

In this embodiment, for the sake of simplicity, description will be made assuming that only the face direction in the left-right direction (Yaw angle direction) is detected, but the face direction in the up-down direction (Pitch angle direction) may also be detected. . In this case, assuming that the direction defined by the Yaw angle and the Pitch angle is the direction of the line of sight, within the range of the angle α vertically or horizontally, or within the range of the solid angle 2α with the direction of the line of sight as the central axis. It is good also as a visual recognition range.

In the present embodiment, it is assumed that the face direction represents the direction of the line of sight, but the direction of the line of sight may be detected more directly by detecting the position of the driver's pupil.

As described above, when the moving object in the front image and the driver's visible range are detected, an attribute assigning process for assigning an attribute to the detected moving object is started (S200). Although details of the attribute assignment process will be described later, in this process, various attributes are assigned to each detected moving object. For example, in the example shown in FIG. 4, three moving objects A to C are detected, but various attributes are assigned to each moving object.

Thereafter, a warning display mode is determined for each moving body according to the attached attribute, and the result is output to the display 16 to display the warning on the moving body for the driver (S106).

Then, it is determined whether or not the driving of the vehicle 1 is finished (S108). If the driving is not finished (S108: no), the process returns to S100 to acquire a new front image, and then the series of processes described above are performed. repeat. As described above, since the front camera 12 acquires a front image at a rate of 30 frames per second, the moving object warning process of FIG. 3 also executes the above-described series of processes at a rate of 30 times per second. As a result, the attribute of the moving object in the front image is updated every frame, and the manner in which the moving object is displayed as a warning on the display 16 also changes every moment.
C. Attribute assignment processing:
In the following, the details of the attribute assigning process (S200) for assigning attributes to a mobile object will be described. As preparation, an outline of attributes provided to the mobile object by the control unit 18 of this embodiment will be described in advance.

FIG. 6 shows a list of attributes assigned to the moving object. Of these seven attributes, attribute 1 and attribute 2 are attributes that are assigned when a moving object is detected in the front image, and the remaining five attributes (attributes 3 to 7) are attribute attributes. This attribute is given by the grant process.

Attribute 1 is an attribute related to the position and size of the moving object on the front image. In the present embodiment, the detected moving body is surrounded by a rectangular frame, and the coordinate values of two points existing on the diagonal of the rectangular frame are used as the attribute 1. For example, for the moving object A shown in FIG. 4, the coordinate values on the images of the points a1 and a2 are the attribute 1.

Attribute 2 is an attribute that represents the moving direction of the moving object. In the present embodiment, the attribute 2 can take a value of 0 to 7, and “0” of the attribute 2 indicates that it moves rightward on the image. Further, “1” indicates a movement in the lower right direction, and “2” indicates a movement in the lower direction. Further, “3” indicates that the movement is in the lower left direction, “4” indicates that the movement is in the left direction, “5” indicates that the movement is in the upper left direction, “6” indicates that the movement is in the upward direction, and “7” indicates that the movement is in the upper right direction.

In the present embodiment, the result of roughly classifying the moving direction of the moving body into eight as described above is used as attribute 2. However, the present invention is not limited to this, and a vector representing the moving direction and moving speed of the moving body is used. It can also be used as attribute 2.

Attribute 3 is a flag (viewing range flag) indicating whether or not the moving body is within the driver's viewing range (see FIG. 5). The viewing range flag ON indicates that the moving body is within the viewing range.

Attribute 4 is an attribute that indicates how many frames the moving body within the viewing range is continuously within the viewing range. If a certain moving body is continuously within the viewing range over a certain number of frames, it can be considered that the driver has recognized.

Attribute 5 is an attribute indicating the number of times recognized by the driver. This attribute 5 counts the number of times of recognition in response to the fact that the driver may recognize the moving body once recognized many times.

Attribute 6 is an attribute indicating the number of frames that have passed without the moving body in the front image being recognized even once by the driver. If the predetermined number of frames has passed without being recognized by the driver even once, the moving body is considered to be overlooked by the driver.

Attribute 7 is a flag (oversight flag) indicating that the moving body is overlooked by the driver. The oversight flag ON indicates that the moving body is overlooked.

When the moving object is detected for the first time, attribute 3 (viewing range flag) and attribute 7 (missing flag) are set to OFF, and attributes 4 to 6 are set to “0”. However, thereafter, the image of the frame is supplied from the front camera 12 and is updated each time the attribute assigning process described below is executed. Note that attribute 1 and attribute 2 are updated not by the attribute assigning process but by the process of detecting the moving object in the moving object warning process (S102 in FIG. 3).

FIG. 7 shows a flowchart of attribute assignment processing. Prior to this process, in the moving body warning process of FIG. 3, the moving body of the front image and the driver's visual recognition range have already been detected.

In the attribute assignment process (S200), first, one arbitrary moving body is selected from the detected moving bodies (S202). Then, it is determined whether or not the selected moving object is within the driver's visual recognition range (S204).

Whether or not the moving body is within the viewing range can be determined based on the attribute 1 of the moving body and the driver's viewing range that has been detected in advance. That is, from the attribute 1 of the moving body, it is possible to determine how much the moving body is located at the Yaw angle from the front in front of the driver. Then, if the calculated Yaw angle of the moving body is within the range of α on both sides of the Yaw angle of the driver's line of sight, it is determined that the moving body is within the driver's viewing range. (See FIG. 5).

As a result, when it is determined that the moving body is within the viewing range (S204: yes), the viewing range flag of attribute 3 is turned ON (S206), and "1" is added to the value of attribute 4 (S208). . As described above with reference to FIG. 6, the attribute 4 indicates the number of frames in which the moving object is continuously present in the viewing range. Here, in the currently processed frame, in response to the determination that the moving body is within the viewing range (S204: yes), the number of frames that are continuously within the viewing range (that is, attribute 4) is set. Add “1”.

On the other hand, when it is determined that the moving body does not exist within the viewing range (S204: no), the viewing range flag of attribute 3 is turned off (S210), and the value of attribute 4 is initially set to “0”. (S212). As a result, the attribute 4 is not the cumulative value of the number of frames in which the moving object is present in the viewing range, but the number of frames that are continuously present in the viewing range.

If “1” is added to attribute 4 (S208), it is determined whether or not the value of attribute 4 has reached a predetermined first threshold value TH1 (S214). Here, the first threshold value TH1 is a threshold value used for determining whether or not the driver has recognized the moving body. In other words, the driver cannot recognize that the moving body is present in the viewable range, but the moving body continues to stay in the viewable range for a certain period of time. If so, it may be considered that the driver has recognized the moving body. This is because, if the moving object exists only within the viewing range, it may be possible that the line of sight happens to pass over the moving object, whereas the moving object remains within the viewing range for a certain period of time. The fact that the vehicle is staying is because it is considered that the driver intentionally directs his / her line of sight toward the moving body.

Therefore, when the value of attribute 4 (the number of frames in which the moving body is continuously present within the driver's visual recognition range) reaches a certain value (first threshold value TH1), the driver intends the moving body. Can be determined (that is, recognized by the driver).

As described above, in the present embodiment, 30 frames can be converted to time at a rate of 1 second, so the number of frames continuously present in the viewing range is continuously present in the viewing range. Synonymous with time. Therefore, in the present embodiment, the first threshold value TH1 corresponds to the “reference time” in the present disclosure.

As a result, if the value of attribute 4 has reached the first threshold value TH1 (S214: yes), “1” is added to the value of attribute 5 (S216). As described above with reference to FIG. 6, the attribute 5 represents the number of times that the moving body is recognized by the driver. In addition, since the value of the attribute 5 of the present embodiment represents the number of times recognized by the driver, it corresponds to the “determination number” in the present disclosure.

Then, after initializing the value of attribute 4 to “0” (S218), the oversight flag of attribute 7 is set to OFF (S220). As described above with reference to FIG. 6, the oversight flag is a flag indicating that the moving body is overlooked by the driver. Therefore, if “1” is added to the value of attribute 5 (the number of times the moving body has been recognized by the driver) in S216, it will be unreasonable if the oversight flag is set to ON. Therefore, the oversight flag of attribute 7 is set to OFF (S220).

On the other hand, when the value of attribute 4 (the number of frames in which the moving body is continuously present in the viewing range) does not reach the first threshold value TH1 (S214: no), or the moving body is within the viewing range in the first place. If the attribute 3 viewing range flag and the attribute 4 value are initialized (S210, S212), the attribute 5 value (number of times recognized by the driver) is “0”. Is determined (S222).

As a result, if the value of attribute 5 is “0” (S222: yes), the moving body has not been recognized by the driver yet, so the value of attribute 6 is “1”. Are added (S224). As described above with reference to FIG. 6, the value of the attribute 6 indicates the number of frames that have passed without being recognized by the driver even once after being detected as a moving body.

Then, it is determined whether or not the value of attribute 6 has reached a predetermined second threshold value TH2 (S226). The second threshold TH2 is set to a value larger than the first threshold TH1 described above. As a result, if the attribute 6 has reached the second threshold value TH2 (S226: yes), it is considered that the mobile body has passed a long time without being recognized even once by the driver. Is set to ON (S228). On the other hand, when the attribute 6 has not reached the second threshold value TH2 (S226: no), the oversight flag of the attribute 7 is kept OFF. In addition, since the attribute 6 and the attribute 7 are attributes related to a state in which the moving body has not been recognized by the driver even once, when the value of the attribute 5 is not “0” (S222: no), The processing relating to attribute 6 and attribute 7 (S224 to S228) is not necessary.

It should be noted that the number of frames that have passed without the mobile body being recognized by the driver once is synonymous with the time that has passed without being recognized by the driver once. Therefore, in the present embodiment, the second threshold value TH2 corresponds to the “missing time” in the present disclosure. In addition, the moving object in which the oversight flag is set to ON corresponds to the “overlooked moving object” in the present disclosure.

When the attributes 3 to 7 of the selected moving object are set as described above, it is determined whether all the moving objects detected from the front image have been selected (S230). As a result, when there is still a moving body that has not been selected (S230: no), the process returns to S202 to select one new moving body and then repeat the series of processes described above. On the other hand, if it is determined that all the moving objects have been selected (S230: yes), the attribute assignment process shown in FIG. 7 is terminated and the process returns to the moving object warning process of FIG.
D. Moving object warning display:
As shown in FIG. 3, after returning from the attribute assignment process (S200) in the moving object warning process, the display mode is determined after the warning display mode is determined for each moving object according to the attribute given to the moving object. A warning is displayed on the screen (S106 in FIG. 3).

FIG. 8 shows a list of warning display modes set according to the attributes of the moving object. As shown in FIG. 8A, in this embodiment, the warning display mode is set based on the value of attribute 5 (number of times recognized by the driver) and the setting of the oversight flag of attribute 7. Yes. FIG. 8B conceptually shows how the warning display changes according to the attribute setting.

In all moving objects, the value of attribute 5 immediately after detection is set to “0”, and the oversight flag of attribute 7 is set to OFF. In this state (attribute 5 = 0), since it is considered that the driver has not yet recognized, the mobile object is surrounded by a red rectangular frame (red frame) and a warning is displayed. In FIG. 8B, the red frame is displayed as a thick solid line frame.

Thereafter, when the number of frames continuously staying within the driver's visual recognition range (first threshold value TH1) or more, “1” is added to the value of attribute 5 (see S216 in FIG. 7). Accordingly, since it can be considered that the mobile object having the attribute 5 value of “1” is recognized once by the driver, it is surrounded by a thin blue line rectangular frame (blue frame) and displayed as a warning. Compared with the red frame in the case of the attribute 5 value “0”, the color of the frame line has been changed from red to blue, and the thickness of the frame line has become thinner. The blue frame is a warning display that is less noticeable than the red frame.

After that, “1” is added to the value of attribute 5 again, and the moving object having the value of attribute 5 “2” is recognized twice by the driver. Therefore, such a moving body is displayed with a warning by being surrounded by a thin dashed blue frame (dashed blue frame). Compared with the blue frame, the broken blue frame is a less noticeable warning as the frame line is changed to a broken line.

Furthermore, a mobile object in which “1” is added to the value of attribute 5 and the value of attribute 5 is “3” is recognized by the driver three times. Since such a moving body is considered to be sufficiently recognized by the driver, there is no need to display a warning. Therefore, no frame display (warning display) is performed for a moving object having an attribute 5 value of “3” or more.

As described above, in this embodiment, since the moving object having the attribute 5 value of “1” or more is considered to be a moving object already recognized by the driver (recognized moving object), the blue frame (dashed blue frame) The warning display mode is not so conspicuous as well as the warning display is not performed.

On the other hand, when the number of frames (second threshold value TH2) or more passes without being recognized by the driver even once (that is, the value of attribute 5 remains “0”), the oversight flag of attribute 7 is turned on. (See S228 of FIG. 7). For such a moving body, it is considered that the warning display of the extent surrounded by the red frame is insufficient, so the red frame is blinked to make the warning display more conspicuous.

As described above, in this embodiment, the moving object having the attribute 5 value of “0”, including the moving object with the oversight flag ON, is not yet recognized by the driver (unrecognized). Therefore, the warning is displayed in a more conspicuous manner than the recognized mobile object.

And since the mode of warning display is different between the recognized mobile body and the unrecognized mobile body, even if there are a large number of mobile bodies, there is no burden on the driver, but the existence of the mobile body Can be reliably warned. This point will be described below.

FIG. 9 shows the manner in which the warning display mode changes for a plurality of moving bodies detected from the front image. FIG. 9A illustrates an image displayed on the screen of the display 16 when a moving object is detected for the first time from the front image. In the illustrated example, ten moving bodies from the moving bodies a to j are detected. In addition, since these mobile objects are detected for the first time (attribute 5 = 0), all of them are surrounded by a red frame and displayed as a warning.

FIG. 9B shows an image after a predetermined time (for example, 1 to 2 seconds) has elapsed from the image of FIG. 9A. During this predetermined time, several moving bodies (here, moving bodies b, c, d, g, h) are recognized by the driver, and the value of attribute 5 is set to “1”. The display mode is changed from a red frame to a blue frame.

FIG. 9 (c) shows an image after a predetermined time has elapsed from the image of FIG. 9 (b). In the image of FIG. 9 (c), several more moving bodies (here, moving bodies e and i) are recognized by the driver and the warning display mode is changed from the red frame to the blue frame. Among the mobile bodies that have already been recognized, there are mobile bodies (here, mobile bodies b, c, g, and h) that have been recognized again and whose warning display mode has been changed from a blue frame to a dashed blue frame. ing. On the other hand, there are also moving bodies (here, moving bodies a and f) that are not yet recognized by the driver at this stage.

FIG. 9 (d) shows an image after a predetermined time has elapsed from the image of FIG. 9 (c). In the image of FIG. 9D, the moving body that has not yet been recognized (here, the moving body f) is recognized by the driver, and the warning display mode is changed from the red frame to the blue frame. In addition, a mobile body (here, mobile bodies d, e, i, j) in which a warning display mode is changed from a blue frame to a broken blue frame is recognized again among mobile bodies that have already been recognized. There are also mobile objects (in this case, mobile objects b, c, g, and h) that are no longer displaying warnings. However, there is still a moving body (moving body a in this case) that is not yet recognized by the driver at this stage, and the warning flag of attribute 7 is set to ON for such a moving body. The display mode is changed from a red frame to a flashing red frame.

Thus, in the moving body warning device 10 of the present embodiment, even when there are a plurality of moving bodies in the front image (see FIG. 9A), the moving body recognized by the driver (recognized moving body). With respect to, the burden on the driver can be reduced by changing the manner of warning display. On the other hand, since the moving body that is not recognized by the driver (unrecognized moving body) is more noticeable than the already recognized moving body, the driver can be surely warned of the presence of the moving body. It becomes possible.

Further, as illustrated in FIGS. 8 and 9, the burden on the driver can be further reduced by changing the warning display mode for the already-recognized moving body according to the value of the attribute 5. Similarly, for an unrecognized moving body, if the mode of warning display is changed according to the setting of the oversight flag of attribute 7, it is possible to more reliably warn the driver of the presence of the moving body.
E. Modified example:
In the above-described embodiment, the number of times the moving body has been recognized by the driver (value of attribute 5) has been described as increasing but not decreasing. However, the value of attribute 5 may be subtracted if the state that is not recognized by the driver continues for a long time after being recognized by the driver.

FIG. 10 shows a flowchart of such a modification attribute assignment process. The attribute assignment process (S300) of the modification is a process executed instead of the attribute assignment process (S200) of the present embodiment described above with reference to FIG.

The processing content of the attribute assignment process (S300) of the modification is substantially the same as the attribute assignment process (S200) of the present embodiment, but the following points are greatly different. That is, in the attribute assignment process (S200) of the present embodiment, the value of attribute 6 is counted and the oversight flag of attribute 7 is set only while the moving object is not recognized once (during attribute 5 = 0). Had gone. On the other hand, in the attribute assignment process (S300) of the modified example, the value of attribute 6 is counted even when the value of attribute 5 is “1” or more, and the value of attribute 6 reaches the second threshold value TH2. Subtracts the value of attribute 5. Hereinafter, the attribute assignment process (S300) of the modification will be briefly described with the difference as a center.

Also in the attribute assignment process (S300) of the modified example, first, one arbitrary moving body is selected from the detected moving bodies (S302), and the selected moving body is within the driver's viewing range. It is determined whether or not there is (S304).

As a result, when it is determined that the moving body is within the viewing range (S304: yes), the viewing range flag of attribute 3 is turned ON (S306), and the value of attribute 4 (continuously exists within the viewing range). “1” is added to the number of frames) (S308). On the other hand, when it is determined that the moving object does not exist within the viewing range (S304: no), the viewing range flag of attribute 3 is turned off (S310), and the value of attribute 4 is initialized to “0” (S312). ).

When “1” is added to attribute 4 (S308), it is determined whether or not the value of attribute 4 has reached a predetermined first threshold value TH1 (S314), and if it has reached the first threshold value TH1. (S314: yes) Since it can be determined that the vehicle has been recognized by the driver, "1" is added to the value of attribute 5 (the number of times the driver has recognized) (S316). Thereafter, after the value of attribute 4 is initialized to “0” (S318), the oversight flag of attribute 7 is set to OFF (S320).

On the other hand, when the value of attribute 4 (the number of frames continuously present in the viewing range) does not reach the first threshold value TH1 (S314: no), or the moving object is not in the viewing range. When the viewing range flag of attribute 3 and the value of attribute 4 are initialized (S310, S312), the following process is executed in the attribute assignment process of the modified example.

First, “1” is added to the value of attribute 6 (S322). The attribute 6 in the modified example represents the number of frames that have passed without the moving body being recognized by the driver. Therefore, in the modification, even after the moving body is recognized by the driver, the number of frames is counted by the attribute 6 if the state in which the moving body is not recognized thereafter continues.

Then, it is determined whether or not the value of attribute 6 has reached the second threshold value TH2 (S324). As a result, if the value of attribute 6 has reached the second threshold value TH2 (S324: yes), the value of attribute 6 is initialized to “0” (S326), and then the value of attribute 5 (recognized by the driver). It is determined whether or not the number of times is “0” (S328). As a result, if the value of attribute 5 is “0” (S328: yes), the oversight flag of attribute 7 is set to ON as in the attribute assignment process of the above-described embodiment (S330).

On the other hand, if the value of attribute 5 is not “0” (S328: no), “1” is subtracted from the value of attribute 5 (S332). Therefore, in the attribute assignment process of the modified example, even if the moving object having the attribute 5 value of “1” is continued to be recognized by the driver for the number of frames corresponding to the second threshold value TH2, the value of the attribute 5 Returns to “0”. Similarly, if the value of attribute 5 is “2”, it returns to “1”, and if the value of attribute 5 is “3”, it returns to “2”.

If the value of attribute 6 has not reached the second threshold value TH2 (S324: no), the above-described processing for subtracting the value of attribute 5 or setting the oversight flag of attribute 7 (S326) To S330) become unnecessary.

In the modification described above, the second threshold value for determining whether to subtract the value of attribute 5 is the same as the threshold value for determining whether to set the oversight flag for attribute 7 to ON. It has been described that the threshold value TH2 is used. Of course, a threshold different from the second threshold TH2 may be used as a threshold for determining whether to subtract the value of attribute 5.

Also, the second threshold value TH2 used for determining whether or not to subtract the value of attribute 5 corresponds to “threshold time” in the present disclosure.

As described above, after setting the attribute 3 to the attribute 7, it is determined whether or not all the moving objects detected from the front image have been selected (S334). As a result, if there is still a moving body that has not been selected (S334: no), the process returns to S302 to select one new moving body and then repeat the series of processes described above. On the other hand, if it is determined that all the moving objects have been selected (S334: yes), the attribute assignment process of the modification shown in FIG. 10 is terminated and the process returns to the moving object warning process of FIG.

In the attribute assignment process of the modified example described above, even if the moving body is recognized by the driver and the mode of the warning display is changed to a mode that is not conspicuous, if the state that is not recognized thereafter continues, the warning display of the mode that is easy to stand out Returned. For this reason, when there is a mobile body that has not been recognized for a long time among the already recognized mobile bodies, it is possible to warn the presence of such a mobile body and call the driver's attention.

This disclosure includes the following aspects.

In the first aspect of the present disclosure, a moving body warning device that warns a driver of a vehicle that a moving body exists in front of the vehicle captures a front image that is an image of the situation in front of the vehicle. A range that the driver is viewing in a range corresponding to the front image in the situation in front and a moving body detection device that detects the moving body from the front image A visual recognition range detection device for detecting a visual recognition range; a determination device that detects the moving object that is continuously present in the visual recognition range for a predetermined reference time; A warning device that warns the driver of the presence of the moving body in a different manner between the unrecognized moving body that is not a recognized moving body and the recognized mobile body;

In the above case, even when there are a plurality of moving bodies in the front image, the recognized moving body and the unrecognized moving body can be distinguished from the warning mode, so the burden on the driver can be reduced. . Further, since the warning is given to the unrecognized moving body in a mode different from that of the already-recognized moving body, it is possible to make the driver surely recognize the presence of the unrecognized moving body.

As an alternative, the visual recognition range detection device may detect the visual recognition range based on an image of the driver taken from the front of the driver. In this case, if a driver's image is taken from the front, the visual recognition range can be easily detected from the driver's face orientation, pupil position, and the like.

As an alternative, the moving body warning device may further include a counting device that counts the number of times that the moving body is determined to be the recognized moving body for each moving body. The warning device warns the driver of the presence of the recognized mobile object in a manner corresponding to the number of determinations. In this case, it is considered that the degree of recognition of the driver is different between a moving body with a large number of times determined as a recognized moving body and a moving body with a small number of times. Therefore, for a recognized mobile object, if the driver is warned in a manner corresponding to the number of times of determination, it is possible to reliably warn of the existence of the mobile object while placing an excessive burden on the driver. Become.

As an alternative, the warning device may stop the warning to the driver for the already-recognized moving body whose number of determinations has reached a predetermined number. In this case, it is considered that the moving body that has been determined many times as the already recognized moving body is sufficiently recognized by the driver. Therefore, if the warning to the driver is stopped for the already-recognized moving body that has reached the predetermined number of times, the burden on the driver is reduced without causing the driver to overlook the presence of the moving body. be able to.

As an alternative, for the moving body that is not determined again as the recognized moving body after a predetermined threshold time elapses after the counting apparatus is determined as the already recognized moving body, The number of determinations may be reduced. In this case, for a moving body that continues to be unrecognized by the driver for a long time, it may be possible to review the warning mode even if it is an already recognized moving body. Therefore, in the moving body warning device according to the present disclosure that warns in a manner according to the number of determinations, the driver can be warned in a more appropriate manner by reducing the number of determinations as described above.

As an alternative, the determination device may detect the moving body in which the state of the unrecognized moving body continues for a predetermined oversight time and determine the moving body as an overlooked moving body. The warning device warns the driver of the presence of the moving body in a manner that is different from the unrecognized moving body and the recognized mobile body. In this case, since the state of the unrecognized moving body continues over the predetermined oversight time, it is desirable that the overlooking moving body warns the driver more clearly. Therefore, the warning can be clearly given to the driver if the warning is given in a different manner from the unrecognized moving object that is not the overlooking moving object and the recognized moving object.

In the second aspect of the present disclosure, a moving body warning method for warning a driver of a vehicle that a moving body is present in front of the vehicle captures a front image that is an image of the situation ahead of the vehicle. And detecting the moving body from the front image, detecting a visual recognition range which is a range visually recognized by the driver in a range corresponding to the front image in the front situation, and Detecting the moving body that is continuously present within a predetermined reference time within a range, determining the moving body as an already-recognized moving body, and the unrecognized moving body that is not the already-recognized moving body; Warning the driver of the presence of the moving body is provided in a manner different from that of the recognized moving body.

In the above case, even when there are a plurality of moving bodies in the front image, the recognized moving body and the unrecognized moving body can be distinguished from the warning mode, so the burden on the driver can be reduced. . Further, since the warning is given to the unrecognized moving body in a mode different from that of the already-recognized moving body, it is possible to make the driver surely recognize the presence of the unrecognized moving body.

Although the present disclosure has been described based on the embodiments, it is understood that the present disclosure is not limited to the embodiments and structures. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims (7)

A moving body warning device that warns a driver of a vehicle that a moving body exists in front of the vehicle,
A photographing device (12) for photographing a front image which is an image obtained by photographing a situation in front of the vehicle;
A moving body detection device (18a) for detecting the moving body from the front image;
A visual range detection device (18c) that detects a visual range that is visible to the driver in a range corresponding to the front image in the front situation;
A determination device (18b) that detects the moving body that is continuously present in the visual recognition range for a predetermined reference time and determines the moving body as an already recognized moving body;
A moving body comprising: a warning device (18d) that warns the driver of the presence of the moving body in a different manner between the unrecognized moving body that is the moving body that is not the recognized moving body and the recognized moving body. Warning device. The moving body warning device according to claim 1, wherein the visual range detection device (18c) detects the visual range based on an image of the driver taken from the front of the driver. A counting device (18) for counting the number of times that the moving body is determined to be the recognized moving body for each moving body;
The moving body warning device according to claim 1 or 2, wherein the warning device (18d) warns the driver of the presence of the recognized moving body in a manner corresponding to the number of times of determination. The moving device warning device according to claim 3, wherein the warning device (18d) stops a warning to the driver for the already recognized moving body that has reached the predetermined number of times of determination. The counting device (18), for the moving body that is not determined as the recognized moving body again after a predetermined threshold time elapses after it is determined as the recognized moving body, The moving body warning device according to claim 3 or 4, wherein the number of determinations is reduced. The determination device (18b) detects the moving body in which the state of the unrecognized moving body continues for a predetermined oversight time, determines that the moving body is an overlooked moving body,
The warning device (18d) warns the driver of the presence of the moving body in a manner different from the unrecognized moving body and the recognized moving body with respect to the overlooked moving body. The moving body warning device according to claim 5. A moving object warning method for warning a driver of a vehicle that a moving object exists in front of the vehicle,
Taking a front image, which is an image of the situation ahead of the vehicle,
Detecting the moving body from the front image;
In a range corresponding to the front image in the front situation, a visual range that is a range visually recognized by the driver is detected.
Detecting the moving body that is continuously present in the visual recognition range for a predetermined reference time, and determining the moving body as an already recognized moving body;
A moving body warning method comprising: notifying the driver of the presence of the moving body in a different mode between the unrecognized moving body that is the moving body that is not the recognized mobile body and the recognized mobile body.

Images