JP2019175372A - Danger prediction device, method for predicting dangers, and program - Google Patents

Abstract

The present invention provides a driver with information for preventing a traffic accident between a vehicle and a vulnerable person in a rear part and a side part which is a blind spot of a driver of the vehicle. A distance data generation unit generates distance data. The reflection data display unit 311 generates first image data indicating a spatio-temporal distance change of the reflection object 21 based on input data including spatio-temporal data in which distance data is accumulated in the direction of the time axis. The reflection data of the reflection object 21 is displayed on the first image data. The first identification unit 312 identifies the reflection data U into moving object reflection data and fixed object reflection data. The mobile data generation unit 313 generates mobile data. The imaging unit 102 captures an image around the vehicle 10. The second image data generator 122 generates second image data. The second identification unit 103 identifies the moving object 31 approaching the vehicle 10 among the moving objects included in the second image data. [Selection diagram] FIG.

Description

  The present invention relates to a risk prediction device, a risk prediction method, and a program.

  While the number of fatalities caused by commercial trucks is decreasing, the number of fatalities caused by large commercial vehicles is decreasing. In addition, the ratio of “people” to fatal accidents involving large commercial vehicles accounted for nearly 40%, of which nearly 70% occurred during the crossing of people, and about 90% of the causes of the accident were driving. Attributed to the

Accidents involving large commercial vehicles tend to have more social losses than passenger car accidents. This is because, in addition to the misery of the accident itself, the driver loses its position, and the operator is forced to manage due to the increase in non-life insurance premiums, and the social corporate responsibility is strictly pursued.
However, among the accidents involving large business vehicles, the accident involving the business large vehicle turning left is an accident that occurs in a region that is difficult for the driver to see. For this reason, it is difficult to completely eradicate the entrainment accidents, no matter how careful the driver is. In particular, in a case where a bicycle, a motorcycle, or the like enters the side of a large business vehicle from behind, it is extremely difficult for the driver to visually recognize them, and a serious accident may occur. Under these circumstances, manufacturers who manufacture large commercial vehicles are conventionally required to introduce driver assistance systems that can detect people in the side of the vehicle and the area behind the vehicle when turning left or right. It has been.

  However, the development of driver support systems has not progressed in each manufacturer that manufactures large commercial vehicles. The main factors are: (1) the market size of large business vehicles is as small as about 1/128 of the ordinary automobile market; and (2) the size of commercial large vehicles varies depending on the loading capacity. Is difficult to share, and cannot be supplied in small lots / multiple varieties, and (3) it is difficult to flexibly support models as it cannot invest a large amount of development costs as in the ordinary automobile market. There is. In other words, the market for business large vehicles requiring these supply conditions is unsuitable for large companies that are good at mass production. This is the main reason why technological development does not progress.

  Conventional technologies in the field of driver assistance systems include (1) an all-around monitoring technology and (2) a front obstacle detection technology. Of these, the all-around monitoring technique (1) is merely a display function and cannot detect an obstacle. In addition, because the front obstacle detection technology (2) is a technology specialized in detecting front obstacles, it is diverted to the side and rear obstacle detection technology required for large commercial vehicles. It is difficult to do.

  In order for a driver of a large business vehicle to predict the danger, the type of the object (person, bicycle, motorcycle, etc.) must be recognized at an early stage, and based on the position and direction of movement of the object, It is necessary to estimate the possibility of collision with a large vehicle. Although it is possible to recognize the type of an object from the image of a commonly used in-vehicle camera, there is a possibility that the object will collide with a large business vehicle based on the position, moving direction, etc. of the object Is difficult to estimate.

  For example, Patent Document 1 discloses a vehicle obstacle that can recognize an obstacle even when a sensor that measures the relative position or relative speed of an object around the host vehicle and a monocular camera are used. An object recognition device has been proposed.

JP 2008-276689 A

  However, the conventional techniques including the technique proposed in Patent Document 1 have at least the following problems (1) to (3). That is, (1) when radar reflected waves from both an approaching object and a separating object are detected, it may not be possible to distinguish both. Specifically, for example, when a radar is installed in the rear side of the vehicle, the radar reflected wave from a safety object such as a utility pole or tree existing on the road, and the radar reflected wave from a dangerous object such as an approaching motorcycle If they pass each other, the radar reflected waves cross and cannot be distinguished. In addition, (2) If the approaching object suddenly stops or the approaching object suddenly accelerates, the approaching object cannot follow the approaching object. Specifically, for example, if a two-wheeled vehicle running in parallel with the vehicle behind it is overtaken by rapid acceleration, it is difficult to obtain the behavior (speed change) of the two-wheeled vehicle in the case of one-dimensional radar only. Become. Furthermore, (3) it is difficult to detect the radar reflected wave of “human”. Specifically, since the shape of the human body is complicated, the level of the radar reflected wave of “human” is data having a large variation. For this reason, the shape of the radar reflected wave is chopped and cannot be detected as an effective reflector.

  The present invention has been made in view of such a situation. In the rear part and the side part, which are blind spots of the driver of the vehicle, the approaching object and the separation object are accurately discriminated, and the approaching object rapidly An object is to provide a driver with information to prevent a traffic accident between a vehicle and a vulnerable person by accurately responding to a speed change and accurately discriminating a “person”.

In order to achieve the above object, a risk prediction apparatus according to an aspect of the present invention includes:
A danger prediction device mounted on a vehicle,
Distance data generating means for measuring a distance to a reflector existing in a measurable area of the surrounding area of the vehicle, and generating distance data based on a result of the measurement;
Based on input data including spatio-temporal data in which the distance data is accumulated in the direction of the time axis, first image data indicating a change in the spatio-temporal distance of the reflector is generated, and the generated first image Reflection data display means for displaying the reflection data of the reflector on the data;
First identification means for identifying one or more of the reflection data displayed on the first image data into moving object reflection data indicating a moving object and fixed object reflection data indicating a fixed body, respectively;
Moving body data generating means for generating moving body data including a moving speed, acceleration, and moving direction of the moving body indicated by the moving body reflection data;
Imaging means for imaging the periphery of the vehicle;
Second image data generating means for generating second image data based on the captured image;
Second identification for identifying a moving body approaching the vehicle among the moving bodies included in the second image data, based on the information including at least the generated second image data and the moving body data. Means,
Is provided.

  According to the present invention, it is possible to accurately discriminate between an approaching object and a separation object at a rear part and a side part which are blind spots of the driver of the vehicle.

Further, in the risk prediction device according to one aspect of the present invention, the first identification unit is configured to perform the one or more of the reflection data, the mobile object reflection data indicating the mobile object, and the fixed body by a spatial filtering process. It is preferable that each can be distinguished from reflection data.
According to the present invention, it is possible to identify and display moving object reflection data approaching the vehicle from one or more reflection data.

In the risk prediction device of one embodiment of the present invention, it is preferable that the spatial filtering process is a process using a plurality of Gabor filters.
According to the present invention, the inclination (angle) of a line segment in spatiotemporal data can be detected.

In the risk prediction apparatus according to one aspect of the present invention, it is preferable that the second identification unit further specifies a horizontal position of the moving body.
In the risk prediction apparatus according to an aspect of the present invention, it is preferable that the second identification unit further specifies the type of the identified moving object.
According to these inventions, it is possible to ensure the accuracy of the second image data, which is a prerequisite for the risk prediction to be performed with high accuracy.

In the risk prediction apparatus according to one aspect of the present invention, it is preferable that the type includes at least a human.
According to the present invention, it is possible to accurately respond to a sudden speed change of the approaching object (moving body 31) and accurately determine “person”.

Moreover, in the risk prediction device of one aspect of the present invention,
Of the measurable area, a dangerous area setting means for preliminarily setting as a dangerous area an area where a risk of a traffic accident between the vehicle and the moving body is high,
Based on the spatio-temporal pattern, the movement trajectory of the vehicle and the movement trajectory of the moving body are evaluated to determine whether or not the moving body may enter the dangerous area, and may enter. When it is determined that there is a risk prediction means for presenting the fact to the driver as a risk prediction;
It is preferable to further comprise.
According to the present invention, it is possible to provide the driver with information for preventing a traffic accident between the vehicle and the traffic weak.

In the risk prediction apparatus according to the aspect of the present invention, the second identification unit adjusts the measurable region and the second image data according to an installation environment of the distance data generation unit and the imaging unit. It is preferable to do.
According to the present invention, it is possible to ensure the accuracy of the second image data, which is a premise for the risk prediction to be performed with high accuracy.

  The risk prediction method and program of one aspect of the present invention are a method and program corresponding to the above-described risk prediction apparatus of one aspect of the present invention.

  According to the present invention, an approaching object and a separation object are accurately discriminated in a rear part and a side part which are blind spots of the driver of the vehicle, and a rapid response to a sudden speed change of the approaching object is accurately performed. By accurately determining the information, it is possible to provide the driver with information for preventing a traffic accident between the vehicle and the traffic weak.

It is an image figure which shows the structure of the danger prediction apparatus which concerns on this invention. It is a block diagram which shows the hardware constitutions of the danger prediction apparatus of FIG. It is a functional block diagram which shows an example of a functional structure for implement | achieving a risk prediction process among the functional structures of the danger prediction apparatus of FIG. It is a figure which shows the maximum area | region which a radar can measure which is decided by the maximum distance which the radar mounted in a vehicle can measure, and the maximum angle. It is a flowchart which shows the flow of a process in the case of performing a spatiotemporal analysis using a Gabor filter. It is a figure which shows the result of the spatiotemporal analysis using a Gabor filter. It is a figure which shows the flow of a process which cuts out the image containing the area | region which shows the range of the same distance as the distance to the moving body which exists in image data. It is a figure which shows a mode that a moving body is recognized using the tracking algorithm by a support vector machine (SVM) space. It is a figure which shows the method of calculating the angle of a moving body from the horizontal direction on an arc, and specifying the relative position of a vehicle and a moving body. It is a figure which shows the method of making a radar and a camera respond | correspond to an installation environment. It is a figure which shows the specific example of a two-dimensional map. It is a figure which shows the specific example of danger prediction. It is a flowchart which shows the flow of a series of processes which the danger prediction apparatus which has a functional structure of FIG. 3 performs.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an image diagram showing a configuration of a risk prediction apparatus 1 according to the present invention.

  As shown in FIG. 1, the danger prediction device 1 is a device that is mounted on a vehicle 10 driven by a driver D. The danger prediction device 1 detects an object existing in a surrounding space including at least the rear and side of the vehicle 10, extracts only the moving body 31, and the extracted moving body 31 is a traffic weak person 70. Judge whether there is. Note that “at least” means that other directions may be included. That is, not only the rear and side of the vehicle 10 but also the front, upper, lower, etc. of the vehicle 10 may be included. When the risk prediction device 1 determines that the moving body 31 is the weak traffic person 70, the image information indicating the positional relationship between the vehicle 10 and the weak traffic person 70 (hereinafter “ A two-dimensional map (hereinafter referred to as “two-dimensional map”) on which a “time-space pattern” is displayed and information on a predicted future danger (hereinafter referred to as “danger prediction”) are presented.

Here, “vehicle” refers to a vehicle in accordance with the Road Traffic Act (for example, an ordinary vehicle, a motorbike, a light vehicle, etc.), a military vehicle such as a tank, an armored vehicle, or a self-propelled gun, a large vehicle such as a truck or a trolley bus. It means vehicles such as construction vehicles that are self-propelled among construction machinery, agricultural vehicles that are self-propelled among agricultural machinery, and industrial vehicles that are self-propelled among industrial machinery.
“Moving object” refers to an object that moves in space. The “traffic weak person” refers to a pedestrian, a bicycle, a motorcycle, and the like among moving objects, and mainly an object smaller in size than the vehicle 10.

  As shown in FIG. 1, the danger prediction apparatus 1 in this embodiment is mainly mounted on a large vehicle 10 such as a truck. In the rear part and the side part of the large vehicle 10, there is a wide area Q (hereinafter referred to as “dead area”) Q that becomes a blind spot for the driver D. For this reason, how the driver D recognizes the blind spot area Q during driving is a very important matter for safe driving.

Since the danger prediction device 1 accurately determines the state of the blind spot area Q using a method described later, for example, even if the moving body 31 approaching the vehicle 10 and the fixed body 40 moving away from the vehicle 10 pass each other, These can be accurately determined. Even if a sudden speed change occurs in the moving body 31 approaching the vehicle 10, the danger prediction device 1 can respond to the speed change. Furthermore, the danger prediction device 1 can accurately determine “person” having a large variation in the level of radar reflected waves.
Such a risk prediction device 1 can present a two-dimensional map and a risk prediction to the driver D. Thus, the driver D can know at a suitable timing that there is a possibility that a traffic accident may occur with the traffic weak person 70. As a result, it is possible to provide the driver D with information for preventing a traffic accident between the vehicle 10 and the traffic weak person 70 that may occur in the blind spot area Q of the vehicle 10.

  FIG. 2 is a block diagram showing a hardware configuration of the danger prediction apparatus 1 of FIG.

  The risk prediction apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, and an input unit. 17, a storage unit 18, a communication unit 19, and a drive 20.

The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 to the RAM 13.
The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.

The output unit 16 includes various liquid crystal displays and outputs various information.
The input unit 17 is composed of various hardware leads and the like, and inputs various information.
The storage unit 18 is configured by a DRAM (Dynamic Random Access Memory) or the like, and stores various data.
The communication unit 19 controls communication with other devices via the network N including the Internet.

  The drive 20 is provided as necessary. A removable medium 30 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 20. The program read from the removable medium 30 by the drive 20 is installed in the storage unit 18 as necessary. The removable medium 30 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

Next, the functional configuration of the risk prediction apparatus 1 having such a hardware configuration will be described with reference to FIGS.
FIG. 3 is a functional block diagram illustrating an example of a functional configuration for realizing the risk prediction process among the functional configurations of the risk prediction device 1 of FIG. 1.
FIG. 4 shows a radar determined by a maximum distance (hereinafter referred to as “measurable distance”) L that can be measured by the radar 120 mounted on the vehicle 10 and a maximum angle (hereinafter referred to as “measurable angle”) K. 120 is a diagram showing a maximum area 120 that can be measured (hereinafter referred to as “measurable area”) M. FIG. FIG. 4A is a diagram showing a measurable region M when the vehicle 10 is viewed from a plane. FIG. 4B is a diagram showing a measurable region M when the vehicle 10 is viewed from the side.
FIG. 5 is a flowchart showing the flow of processing when a spatiotemporal analysis is performed using a Gabor filter.
FIG. 6 is a diagram illustrating a result of spatiotemporal analysis using a Gabor filter. FIG. 6A is a diagram showing a scene in which a motorcycle overtakes the vehicle 10 decelerating. FIG. 6B is a graph in which the horizontal axis represents time and the vertical axis represents the reflection level. 6C is a diagram showing a result of spatio-temporal analysis of the scene shown in FIG. 6A based on the graph of FIG. 6B. The vertical axis represents time T, and the horizontal axis represents distance P. ing.
FIG. 7 is a diagram showing a flow of processing for cutting out an image S including an area indicating the same distance range as the distance P to the moving object existing in the image data.
FIG. 8 is a diagram illustrating a state in which the moving object 31 is recognized using a tracking algorithm based on a support vector machine (SVM) space.
FIG. 9 is a diagram illustrating a method of calculating the angle of the moving body from the horizontal direction in the arc-shaped line segment W and specifying the relative position between the vehicle and the moving body.
FIG. 10 is a diagram illustrating a technique for causing the radar 120 and the camera 121 to correspond to the installation environment.
FIG. 11 is a diagram illustrating a specific example of a two-dimensional map.
FIG. 12 is a diagram illustrating a specific example of risk prediction.
FIG. 13 is a flowchart showing a flow of a series of processes executed by the risk prediction apparatus 1 having the functional configuration of FIG.

  As shown in FIG. 3, when the risk prediction process is executed in the CPU 11 (FIG. 2) of the risk prediction device 1, the sensor unit 101, the imaging unit 102, the image processing unit 103, and a two-dimensional map generation presentation The unit 104 and the risk prediction unit 105 function. In one area of the storage unit 18 (FIG. 2), a distance DB 401, an image DB 402, a host vehicle DB 403, and a space-time DB 404 are provided.

(Sensor part)
The sensor unit 101 is configured to include a distance data generation unit 111, a host vehicle data generation unit 112, and a spatiotemporal analysis unit 113.

  The distance data generation unit 111 measures the distance P to the reflector 21 existing in the measurable region M in the region around the vehicle 10 and information based on the measurement result (hereinafter referred to as “distance data”). Is generated. The distance data generated by the distance data generation unit 111 is stored and managed in the distance DB 401. The specific method by which the distance data generation unit 111 measures the distance P to the reflector 21 is not particularly limited. A microwave radar using a frequency band of 24 GHz band, a millimeter wave radar using a frequency band of 60 GHz band and 76 GHz band, an infrared sensor, or the like can be used. In this embodiment, a microwave radar (hereinafter simply referred to as “radar”) 120 is employed, and the distance P to the reflector 21 is measured by a frequency-modulated continuous wave (FMCW / Frequency Modulated Continuous Wave radar). To generate distance data. By using the radar 120, it is possible to measure an accurate distance in the depth direction that cannot be measured by image data based on an image captured by a camera 121 described later.

  The measurable distance L and the measurable angle K of the radar 120 are not particularly limited, but the measurable distance L is preferably 25 m or more, and the measurable angle K is preferably 45 ° or more. For example, it is assumed that the radar 120 having a measurable distance L of 25 m and a measurable angle K of 45 ° is mounted at three locations of the right front end portion, the left front end portion, and the rear end central portion of the vehicle 10. To do. In this case, even if the total length A of the vehicle 10 is 12 m, which is the same as the maximum length of an ordinary car defined by the Road Transport Vehicle Law, for example, 5 m from the right side of the vehicle 10 to the right and the left side of the vehicle 10 An area indicated by 5 m leftward from the side and 13 m backward from the rear end of the vehicle 10 (measurable distance L (25 m) −total length A (12 m) of the vehicle 10) is a measurable area M of the radar 120. It will be included in a part.

The sensitivity of the radar 120 is not particularly limited, but it is preferable that the radar 120 can detect even if the cross-sectional area of reflection of a pedestrian (human body) included in the weak traffic person 70 is 0.1 m ² or less. If the sensitivity of the radar 120 reaches this level, it becomes possible to discriminate a “person” that is difficult to detect as an effective reflector because of its complicated shape.

  The own vehicle data generation unit 112 includes an acceleration sensor, calculates the speed, acceleration, and moving direction of the vehicle 10 that becomes the own vehicle, and uses information (hereinafter referred to as “own vehicle data”) based on the calculation result. Generate. The own vehicle data generated by the own vehicle data generation unit 112 is stored and managed in the own vehicle DB 403. In addition, the detection method of the acceleration by the own vehicle data generation part 112 is not specifically limited. For example, a technique using a MEMS (Micro Electro Mechanical System) technique may be used.

  The spatiotemporal analysis unit 113 performs spatiotemporal analysis. The spatiotemporal analysis refers to a series of processes executed by the reflection data display unit 311, the first identification unit 312, and the moving body data generation unit 313 that constitute the spatiotemporal analysis unit 113.

  The reflection data display unit 311 is based on input data including information (hereinafter referred to as “spatio-temporal data”) obtained by accumulating the distance data generated by the distance data generation unit 111 in the direction of the time axis. First image data indicating a change in distance in space and time is generated, and the reflection data of the reflecting object 21 is displayed on the generated first image data. The input data can include the vehicle data generated by the vehicle data generation unit 112.

  The first identification unit 312 identifies one or more reflection data U displayed on the first image data as moving object reflection data indicating a moving object and fixed object reflection data indicating a fixed object, respectively. . Specifically, the first identification unit 312 performs one or two or more reflection data U on the moving object reflection data indicating the moving object 31 and the fixed object reflection data indicating the fixed object 40 by spatial filtering. Identify. That is, in the first identification unit 312, as shown in FIG. 6B, the curve γ indicating the reflecting object 21 in the graph B having the horizontal axis as time T and the vertical axis as the reflection data U, and the horizontal axis as time T. And a process of arranging a plurality in the direction of time T in the graph C having the distance P as the vertical axis is performed. And the process which connects the point used as each peak of the some curve (gamma) arranged on the graph C is performed. When the shape of the line segment drawn by this processing is right-up, it is identified as fixed body reflection data, and when it is right-down, it is identified as mobile body reflection data. In this way, the mobile object reflection data α approaching the vehicle 10 can be identified and displayed from one or more reflection data U.

  The moving body data generation unit 313 generates moving body data including the moving speed, acceleration, and moving direction of the moving body 31 indicated by the moving body reflection data.

  As a result of the spatiotemporal analysis by the spatiotemporal analysis unit 113, fixed body reflection data such as utility poles and guardrails are removed from the input data, and only mobile body reflection data having different relative velocities are extracted. Thus, for example, as shown in FIG. 6C, the moving body reflection data indicating the moving body 31 (bike) approaching the vehicle 10 from among the plurality of reflecting bodies 21 is a line segment α having a downward slope. Is displayed. At this time, the line segment β that rises to the right is indicated by the moving object reflection data indicating the fixed body 40 (electric pole) that is separated from the vehicle 10. In the present embodiment, the mobile body 31 approaching the vehicle 10 is identified and displayed by indicating the difference between the fixed body reflection data and the mobile body reflection data by the color difference. However, the present invention is not limited to this, and the identification display may be performed using any method such as pattern identification or shape identification.

  The specific method of the spatial filtering process for identifying and displaying the moving object reflection data approaching the vehicle 10 from the plurality of reflection data U with the line segment α descending to the right is not particularly limited. A process using a Gabor filter, which is one of the spatial filtering processes used in the process, is employed. The Gabor filter has a property of enhancing a specific inclination (angle). In the present embodiment, using this property, as shown in FIG. 5, a total of 18 stages of Gabor filters are used in the range where the value θ indicating the inclination (angle) is 0 ° to 180 °. That is, a total of 18 stages of Gabor filters having different values of inclination (angle) θ every 10 °, from filter 0 for 0 ° ≦ θ <10 ° to filter 17 for 170 ° ≦ θ <180 °. Analysis using a Gabor filter is performed. In this way, by performing a spatiotemporal analysis using 18-stage Gabor filters having different inclinations (angles), it is possible to detect the inclinations (angles) of line segments in the spatiotemporal data.

(Imaging part)
The imaging unit 102 includes a camera 121 and a second image data generation unit 122. The camera 121 images the surroundings of the vehicle 10. The second image data generation unit 122 generates second image data based on the captured image. The generated second image data is used for image processing by the image processing unit 103 described later. That is, since the position of the moving body 31 in the horizontal direction cannot be specified only by the radar 120, the image processing unit 103 described later uses the second image data generated by the imaging unit 102 to use the horizontal position of the moving body 31. Specify the position of the direction.

(Second identification part)
The second identification unit 103 generates second image data based on information including at least the second image data generated by the second image data generation unit 122 and the moving body data generated by the moving body data generation unit 313. Of the mobile bodies 31 included in the vehicle 10 is identified. Note that “at least” means that information other than the moving object data is included in the information used to identify the moving object 31 approaching the vehicle 10 among the moving objects 31 included in the second image data. Means good. For example, the vehicle data may be included in this information.
The specific method by which the second identification unit 103 identifies the moving body 31 approaching the vehicle 10 among the moving bodies 31 included in the second image data is not particularly limited. For example, the moving body 31 included in the image data may be identified by learning HOG (Histograms of Oriented Gradients) feature amounts by SVM (Support Vector Machine) which is one of machine learning methods. Thereby, it is possible to determine whether or not the mobile body 31 that is identified and displayed is the weak traffic person 70.

  Further, the second identification unit 103 specifies the horizontal position of the identified mobile object 31 and specifies the type of the mobile object 31. The type of the moving body 31 specified by the second identification unit 103 includes at least a human. Note that “at least” means that the type of the mobile body 31 may include a person other than a human. That is, the moving body 31 may include any object that can move in the area around the vehicle 10. For example, bicycles and motorcycles driven by humans, and animals such as dogs and cats are all examples of the moving body 31.

Here, when the size of the moving body 31 is the same, the moving body 31 that is located closer to the camera 121 is displayed larger in the image data than the moving body 31 that is located far from the camera 121. .
For this reason, the second identification unit 103 adjusts the scale of the moving body 31 on the second image data. Specifically, as shown in FIG. 7, the second identification unit 103 cuts out an image S including a region indicating the same distance range as the distance P to the moving body 31 existing in the second image data. The cut-out image S is reduced to the assumed “person” size. The number n (n is an integer value of 1 or more) of the reduction scale when reducing the image S is not particularly limited, but is 12 in this embodiment. Thereby, the scale of the mobile bodies 31 existing at positions where the distance P from the camera 121 is different can be adjusted. As a result, the second identification unit 103 can more accurately identify the moving object 31 on the second image data.

  When the entire moving object 31 is not displayed in the second image data, the second identifying unit 103 identifies the moving object 31 using a tracking algorithm based on a support vector machine (SVM) space. For example, FIG. 8 shows a state in which the moving body 31 approaches the vehicle 10 in the order of FIGS. When the distance between the moving body 31 and the vehicle 10 is 5 m (FIG. 8A), and when the distance between the moving body 31 and the vehicle 10 is 3 m (FIG. 8B), the second image data. The entire moving body 31 is displayed on the screen. For this reason, the image processing unit 103 identifies the moving object 31 included in the second image data by learning the HOG feature value by SVM. 8A and 8B, a frame line R surrounding the mobile body 31 is a frame line for identifying and displaying the mobile body 31 by learning the HOG feature value by SVM. On the other hand, when the distance between the moving body 31 and the vehicle 10 is 2 m (FIG. 8C) and when the distance between the moving body 31 and the vehicle 10 is 1 m (FIG. 8D), the moving body A part of 31 is not displayed in the second image data (that is, the moving body 31 is cut off from the second image data). For this reason, the image processing unit 103 identifies the moving body 31 included in the second image data by using a tracking algorithm based on a support vector machine (SVM) space. A frame V surrounding the moving body 31 shown in FIGS. 8C and 8D is a frame line for identifying and displaying the moving body 31 by learning the HOG feature value by SVM. As a result, as shown in FIGS. 8A to 8D, it is possible to smoothly recognize the moving body 31 in which only the part is displayed without being entirely displayed in the second image data. .

  As described above, the second identification unit 103 performs the processing by cutting out the image S including the area indicating the same distance range as the distance P to the moving body 31 existing in the second image data. The range in which image processing for identifying the body 31 is performed can be limited. That is, the image processing unit 103 can perform image processing in the second image data in which the size of the moving body 31 that is the target of image processing and the position where image processing is performed are limited. Thereby, the calculation amount accompanying image processing can be reduced. As a result, it is possible to reduce the calculation cost that tends to increase as the resolution of the camera 121 increases.

  The second identification unit 103 limits the range of image processing for identifying the moving object 31 present in the second image data, and as shown in FIG. 9, the second identification unit 103 extends to the moving object 31 present in the second image data. A region indicating the same distance range as the distance P is displayed as an arc-shaped line segment W. Then, the angle of the moving body 31 is calculated based on the horizontal direction of the frame line R displayed superimposed on the line segment W. Thereby, the relative position of the own vehicle (vehicle 10) and the mobile body 31 can be specified.

  By the way, when the radar 120 and the camera 121 are mounted on both sides of the vehicle 10, the specific installation location is the arm portion of the side mirror. However, the position of the arm portion of the side mirror varies greatly depending on the type of the vehicle 10. For example, when the vehicle 10 is a large truck, the arm portion of the side mirror is disposed at a height of about 180 cm from the ground. Further, when the vehicle 10 is a medium truck, the arm portion of the side mirror is disposed at a height of about 155 cm from the ground. When the vehicle 10 is a trailer, the arm portion of the side mirror is disposed at a height of about 200 cm from the ground. For this reason, the radar 120 and the camera 121 mounted on the vehicle 10 need to be able to cope with the difference in height and angle (hereinafter referred to as “installation environment”) from the ground of the place where the vehicle is actually installed.

  The image processing unit 103 adjusts the measurable area M including the measurable distance L and the position of the camera 121 on the second image data according to the installation environment of the radar 120 and the camera 121. Thereby, the radar 120 and the camera 121 can easily cope with the difference in the installation environment. Specifically, as shown in FIG. 10 (A), the image processing unit 103 performs the operation according to the installation environment (the height a and the angle b from the ground) of the radar 120 and the camera 121 installed on the arm. Calibration is executed for the angle of the image displayed as the two-image data, the distance P to the reflector 21, the vertical length d and the horizontal length e of the frame lines R and V surrounding the reflector 21, and the like. For example, as illustrated in FIG. 10B, the image processing unit 103 uses the second image data to indicate a distance P from the vehicle 10 (that is, the radar 120 and the camera 121) and a relative angle with the vehicle 10. Calibration may be executed by displaying the line segment Y in a superimposed manner. Accordingly, it is possible to ensure the accuracy of the second image data, which is a prerequisite for risk prediction performed by the risk prediction unit 105 described later with high accuracy.

(2D map generation and presentation unit)
The two-dimensional map generation and presentation unit 104 generates and generates a two-dimensional map F on which a spatio-temporal pattern indicating the positional relationship between the vehicle 10 and the moving body 31 is displayed for the driver D who drives the vehicle 10. A two-dimensional map F is presented to the driver D. As shown in FIG. 11, the two-dimensional map F generated by the two-dimensional map generation / presentation unit 104 includes a vertical axis representing the longitudinal direction of the vehicle 10 and a distance in the vertical direction (horizontal direction) with respect to the longitudinal direction of the vehicle 10. Is a two-dimensional map composed of a horizontal axis. In the two-dimensional map F, a spatio-temporal pattern in which the vehicle 10 is arranged in the center is displayed, and a movement trajectory based on the past movement history of the traffic weak 70 (the moving body 31) is displayed. When the two-dimensional map F is output from the monitor (output unit 16) mounted on the driver's seat of the vehicle 10, the driver D who drives the vehicle 10 has a traffic accident with the traffic weak 70 (the moving body 31). It is possible to intuitively recognize that there is a possibility of occurrence of
Although the specific configuration of the two-dimensional map F is not particularly limited, the two-dimensional map F according to the present embodiment has a size of horizontal (W) 480 pixels × vertical (H) 480 pixels (pixels) and a resolution of horizontal (W). 50 mm × length (H) 50 mm (0.104 mm / pixel (pixel)), and the movement trajectory holding period is 4 seconds. Accordingly, the two-dimensional map generation / presentation unit 104 provides the driver D with information for preventing a traffic accident between the vehicle 10 and the traffic weak 70 (the moving body 31) that may occur in the blind spot area Q of the vehicle 10. be able to. As a result, the driver D can know at a suitable timing that a traffic accident may occur with the traffic weak 70 (the mobile body 31).

(Danger Prediction Department)
The danger prediction unit 105 presets, as the danger area X, an area in the measurable area M where the risk of a traffic accident between the vehicle 10 and the moving body 31 is high. The danger prediction unit 105 evaluates the movement locus of the vehicle 10 and the movement locus of the moving body 31 based on the spatiotemporal pattern displayed on the two-dimensional map F, and the moving body 31 enters the danger area X. When it is determined whether or not there is a possibility of entering the vehicle, it is indicated to the driver D as a risk prediction when it is determined that there is a risk of entering. For example, in FIG. 12, the relative speed between the vehicle 10 and the moving body 31 (bike) existing in the measurable area M of the vehicle 10 is 25 km / h, and after 0.8 seconds, the moving body 31 (bike) is in the dangerous area. An example of risk prediction that there is a risk of entering X is shown.
In this example, the danger prediction unit 105 evaluates the movement trajectory of the vehicle 10 and the movement trajectory of the moving body 31 (bike) existing in the measurable region M in the latest 1 second (1.0 S), and the moving body 31 ( The motorcycle) determines that there is a risk of entering the dangerous area X that is 5.6 m away 0.8 seconds from now, and presents this to the driver D as a risk prediction. The time until the driver D receives the warning and operates the steering wheel is normally about 0.5 seconds. For this reason, if the timing of presenting the danger prediction is set to 0.8 seconds before entering the danger area X, the driver D can cope with the margin.

  Although the specific method of presenting the risk prediction to the driver D is not particularly limited, in this embodiment, the monitor (output unit 16) mounted in the driver's seat of the vehicle 10 and the engine operation control are electrically controlled. The danger prediction is presented by outputting to the vehicle engine control unit (ECU). In this case, the vehicle engine control unit (ECU) may perform vehicle control, rudder control, alarm output, and the like based on the presented risk prediction.

  The specific method used when the risk prediction unit 105 sets the risk area X in advance is not particularly limited. For example, regression analysis by machine learning may be performed based on the spatiotemporal pattern stored in the spatiotemporal DB 404. In this case, the risk predicting unit 105 quantifies the risk of a traffic accident between the vehicle 10 and the moving body 31, and calculates the risk level at any location within the measurable region M in multiple stages (for example, in descending order of risk level). It may be set in three stages (high, medium and low).

Next, a flow of a series of processes executed by the risk prediction device 1 having the functional configuration of FIG. 3 will be described with reference to FIG.
FIG. 13 is a flowchart for explaining the flow of the risk prediction process executed by the risk prediction device 1 of FIG.

As shown in FIG. 13, the risk prediction apparatus 1 executes the following series of processes.
In step S <b> 1, the distance data generation unit 111 (radar 120) measures the distance P to the reflector 21 existing in the measurable area M in the area around the vehicle 10, and obtains distance data based on the measurement result. Generate.

In step S <b> 2, the reflection data display unit 311 changes the distance change in the spatio-temporal of the reflector 21 based on the input data including the spatiotemporal data in which the distance data generated in step S <b> 1 is accumulated in the direction of the time axis. First image data to be shown is generated, and reflection data of the reflecting object 21 is displayed on the generated first image data.
In step S <b> 3, the first identification unit 312 displays one or more reflection data U displayed on the first image data, the moving object reflection data indicating the moving object 31, and the fixed object reflection data indicating the fixed object. And identify each.
In step S4, the moving body data generation unit 313 generates moving body data including the moving speed, acceleration, and moving direction of the moving body 31 indicated by the moving body reflection data.

In step S5, the imaging unit 102 images the surroundings of the vehicle.
In step S6, the second image data generation unit 122 generates second image data based on the captured image.

In step S <b> 7, the two-dimensional map generation / presentation unit 104 generates a two-dimensional map F on which a spatio-temporal pattern indicating the positional relationship between the vehicle 10 and the moving body 31 is displayed for the driver D driving the vehicle 10. The generated two-dimensional map F is presented to the driver D.
In step S <b> 8, the danger prediction unit 105 presets an area in the measurable area M where the risk of a traffic accident between the vehicle 10 and the moving body 31 is high as the danger area X.
In step S <b> 9, the danger prediction unit 105 evaluates the movement locus of the vehicle 10 and the movement locus of the moving body 31 based on the spatiotemporal pattern, and is there a possibility that the moving body 31 may enter the danger area X? If it is determined that there is a risk of entering, the driver D is informed of this as a risk prediction. Thus, the process ends.

  By executing the series of processes as described above, the moving body 31 and the fixed body 40 are accurately discriminated in the blind spot region Q of the driver D of the vehicle 10, and the moving body 31 can be accurately detected for a rapid speed change. In addition, the “person” is accurately identified. As a result, it is possible to provide the driver D with information for preventing a traffic accident between the vehicle 10 and the traffic weak 70 (the moving body 31).

  Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, and the like within the scope that can achieve the object of the present invention are included in the present invention. It is.

  For example, although the vehicle 10 in the present embodiment is a large vehicle such as a truck, this is merely an example and may be a normal vehicle or the like.

  Each hardware configuration shown in FIG. 2 is merely an example for achieving the object of the present invention, and is not particularly limited.

  Further, the functional block diagram shown in FIG. 3 is merely an example, and is not particularly limited. That is, it suffices if the risk prediction apparatus has a function capable of executing the above-described series of processes as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG. .

Also, the location of the functional block is not limited to the location shown in FIG. 3, and may be arbitrary.
One functional block may be constituted by hardware alone or in combination with software alone.

When the processing of each functional block is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or personal computer other than a server.

  A recording medium including such a program is not only constituted by a removable medium distributed separately from the apparatus main body in order to provide the program to each user, but also provided to each user in a state of being incorporated in the apparatus main body in advance. It is composed of a provided recording medium or the like.

  In the present specification, the steps for describing the program recorded on the recording medium are not limited to the processing performed in time series according to the order, but may be performed in parallel or individually even if not necessarily performed in time series. The process to be executed is also included.

In summary, the risk prediction apparatus (for example, the risk prediction apparatus 1 in FIG. 3) to which the present invention is applied only needs to have the following configuration, and can take various embodiments.
That is, the danger prediction device to which the present invention is applied is
A risk prediction device mounted on a vehicle (for example, the vehicle 10 in FIG. 1),
Measure the distance (for example, distance P in FIG. 10) to the reflector (for example, reflector 21 in FIG. 10) present in the measurable area (for example, measurable area M in FIG. 4) of the surrounding area of the vehicle, Distance data generating means (for example, the distance data generating unit 111 in FIG. 3) for generating distance data based on the measurement result;
Based on input data including spatio-temporal data in which the distance data is accumulated in the direction of the time axis, first image data indicating a change in the spatio-temporal distance of the reflector is generated, and the generated first image Reflection data display means (for example, the reflection data display unit 311 in FIG. 3) for displaying the reflection data of the reflector on the data;
First identifying means for identifying one or more of the reflection data displayed on the first image data into moving object reflection data indicating a moving object and fixed object reflection data indicating a fixed object, respectively (for example, The first identification unit 312) of FIG.
Mobile object data generation means (for example, a mobile object data generation unit 313 in FIG. 3) for generating mobile object data including the moving speed, acceleration, and moving direction of the moving object indicated by the moving object reflection data;
Imaging means for imaging the surroundings of the vehicle (for example, the imaging unit 102 in FIG. 3);
Second image data generation means (for example, the second image data generation unit 122 in FIG. 3) for generating second image data based on the captured image;
Second identification for identifying a moving body approaching the vehicle among the moving bodies included in the second image data, based on the information including at least the generated second image data and the moving body data. Means (for example, the second identification unit 103 in FIG. 3);
Is provided.
As a result, an approaching object (for example, the moving body 31) and a separation object (for example, a fixed object) are formed at the rear and side portions that are blind spots of the driver (for example, the driver D of FIG. 1) of the vehicle (for example, the vehicle 10 in FIG. 1). It is possible to accurately determine the body 40).

Further, in the risk prediction device according to one aspect of the present invention, the first identification unit is configured to perform the one or more of the reflection data, the mobile object reflection data indicating the mobile object, and the fixed body by a spatial filtering process. It is preferable that each can be distinguished from reflection data.
According to the present invention, it is possible to identify and display moving object reflection data approaching the vehicle from one or more reflection data.

In the risk prediction device of one embodiment of the present invention, it is preferable that the spatial filtering process is a process using a plurality of Gabor filters.
According to the present invention, the inclination (angle) of a line segment in spatiotemporal data can be detected.

In the risk prediction apparatus according to one aspect of the present invention, it is preferable that the second identification unit further specifies a horizontal position of the moving body.
In the risk prediction apparatus according to an aspect of the present invention, it is preferable that the second identification unit further specifies the type of the identified moving object.
According to the present invention, it is possible to ensure the accuracy of the second image data, which is a prerequisite for the risk prediction to be accurately performed.

In the risk prediction apparatus according to one aspect of the present invention, it is preferable that the type includes at least a human.
According to the present invention, it is possible to accurately respond to a sudden speed change of the approaching object (moving body 31) and accurately determine “person”.

Moreover, in the risk prediction device of one aspect of the present invention,
Of the measurable area, a dangerous area setting means for preliminarily setting as a dangerous area an area where a risk of a traffic accident between the vehicle and the moving body is high,
Based on the spatio-temporal pattern, the movement trajectory of the vehicle and the movement trajectory of the moving body are evaluated to determine whether or not the moving body may enter the dangerous area, and may enter. When it is determined that there is a risk prediction means for presenting the fact to the driver as a risk prediction;
It is preferable to further comprise.
According to the present invention, information for preventing a traffic accident between the vehicle 10 and the weak traffic person 70 (the moving body 31) can be provided to the driver D.

In the risk prediction apparatus according to the aspect of the present invention, the second identification unit adjusts the measurable region and the second image data according to an installation environment of the distance data generation unit and the imaging unit. It is preferable to do.
According to the present invention, it is possible to ensure the accuracy of the second image data, which is a premise for the risk prediction to be performed with high accuracy.

1: Danger prediction device 10: Vehicle 11: CPU
12: ROM
13: RAM
14: Bus 15: Input / output interface 16: Display unit 17: Input unit 18: Storage unit 19: Communication unit 20: Drive 21: Reflected object 30: Removable media 31: Mobile body 40: Fixed body 70: Traffic weak person 101: Sensor Unit 102: imaging unit 103: second identification unit 104: two-dimensional map generation unit 105: danger prediction unit 111: distance data generation unit 112: own vehicle data generation unit 113: spatio-temporal analysis unit 120: radar 121: camera 122: 2nd image data generation part 311: Reflection data display part 312: 1st identification part 313: Mobile body data generation part 401: Distance DB
402: Image DB
403: Own car DB
404: Spatio-temporal DB
A: Total length of vehicle B: Graph C: Graph D: Driver F: Two-dimensional map K: Measurable angle L: Measurable distance M: Measurable area n: Number of steps P: Distance Q: Blind spot area R: Border line S : Image T: Time U: Reflection level V: Frame line W: Line segment X: Dangerous area Y: Line segment α: Line segment β: Line segment γ: Curve

Claims (11)

A danger prediction device mounted on a vehicle,
Distance data generating means for measuring a distance to a reflector existing in a measurable area of the surrounding area of the vehicle, and generating distance data based on a result of the measurement;
Based on the input data including the spatio-temporal data in which the generated distance data is accumulated in the direction of the time axis, first image data indicating a change in the spatio-temporal distance of the reflector is generated, and the generated first Reflection data display means for displaying the reflection data of the reflector on the image data;
First identification means for identifying one or more of the reflection data displayed on the first image data into moving object reflection data indicating a moving object and fixed object reflection data indicating a fixed body, respectively;
Moving body data generating means for generating moving body data including a moving speed, acceleration, and moving direction of the moving body indicated by the moving body reflection data;
Imaging means for imaging the periphery of the vehicle;
Second image data generating means for generating second image data based on the captured image;
Second identification for identifying a moving body approaching the vehicle among the moving bodies included in the second image data, based on the information including at least the generated second image data and the moving body data. Means,
A risk prediction device comprising: The first identification means includes
The spatial filtering process identifies the one or more reflection data as moving object reflection data indicating the moving object and fixed object reflection data indicating the fixed body, respectively.
The danger prediction device according to claim 1. The spatial filtering process is a process using a plurality of Gabor filters.
The danger prediction device according to claim 2. The second identification means further includes
Identifying the horizontal position of the identified mobile object;
The danger prediction apparatus of any one of Claims 1-3. The second identification means further includes
Identify the type of the identified moving object;
The danger prediction apparatus of any one of Claims 1-4. The type includes at least a human,
The danger prediction device according to claim 5. 2D map generation / presentation means for generating a 2D map on which a spatiotemporal pattern indicating a positional relationship between the vehicle and the moving body is displayed, and presenting the 2D map to a driver driving the vehicle. In addition,
The danger prediction device according to any one of claims 1 to 6. Of the measurable area, a dangerous area setting means for preliminarily setting as a dangerous area an area where a risk of a traffic accident between the vehicle and the moving body is high,
Based on the spatio-temporal pattern, the movement trajectory of the vehicle and the movement trajectory of the moving body are evaluated to determine whether or not the moving body may enter the dangerous area, and may enter. When it is determined that there is a risk prediction means for presenting the fact to the driver as a risk prediction;
Further comprising
The risk prediction apparatus according to claim 7. The second identification unit adjusts the measurable area and the second image data according to an installation environment of the distance data generation unit and the imaging unit.
The danger prediction device according to any one of claims 1 to 8. A method used for risk prediction in a vehicle,
A distance data generation step of measuring a distance to a reflector existing in a measurable area of the area around the vehicle, and generating distance data based on the measurement result;
Based on input data including spatio-temporal data in which the distance data is accumulated in the direction of the time axis, first image data indicating a change in the spatio-temporal distance of the reflector is generated, and the generated first image A reflection data display step for displaying the reflection data of the reflector on the data;
A first identification step for identifying one or more of the reflection data displayed on the first image data into moving object reflection data indicating a moving object and fixed object reflection data indicating a fixed body;
A moving object data generating step for generating moving object data including moving speed, acceleration, and moving direction of the moving object indicated by the moving object reflection data;
An imaging step of imaging the surroundings of the vehicle;
A second image data generation step of generating second image data based on the captured image;
Second identification for identifying a moving body approaching the vehicle among the moving bodies included in the second image data, based on the information including at least the generated second image data and the moving body data. Steps,
Risk prediction method including In the computer that controls the danger prediction device,
A distance data generation step of measuring a distance to a reflector existing in a measurable area of the surrounding area of the vehicle, and generating distance data based on a result of the measurement;
Based on input data including spatio-temporal data in which the distance data is accumulated in the direction of the time axis, first image data indicating a change in the spatio-temporal distance of the reflector is generated, and the generated first image A reflection data display step for displaying the reflection data of the reflector on the data;
A first identification step for identifying one or more of the reflection data displayed on the first image data into moving object reflection data indicating a moving object and fixed object reflection data indicating a fixed body;
A moving object data generating step for generating moving object data including moving speed, acceleration, and moving direction of the moving object indicated by the moving object reflection data;
An imaging step of imaging the surroundings of the vehicle;
A second image data generation step of generating second image data based on the captured image;
Second identification for identifying a moving body approaching the vehicle among the moving bodies included in the second image data, based on the information including at least the generated second image data and the moving body data. Steps,
A program that executes control processing including