JP2011227571A - Information processing method, information processing program, and information processing apparatus - Google Patents

Abstract

An object of the present invention is to provide a technique for providing a method for efficiently extracting dangerous driving from a driving image.
In order to solve the above problems, a dangerous driving analysis apparatus 100 includes a peripheral information DB 128 that stores peripheral information around the host vehicle as driving information related to the host vehicle, and the peripheral information is stored in the host vehicle. A dangerous driving extraction means for determining whether or not a safe driving standard indicating that the vehicle is safely operated is satisfied, and extracting a dangerous driving that does not satisfy the safe driving standard from the peripheral information DB 128; Including.
[Selection] Figure 4

Description

  The present invention relates to an information processing method, an information processing program, and an information processing apparatus for extracting dangerous driving of a host vehicle.

  Promoting safety and reducing the number of accidents in the automobile society has become a major issue, and various countermeasures have been taken. For example, as one of the countermeasures, there is a method in which a driver learns when an accident is likely to occur using a video. Specifically, for example, as in the system described in Patent Document 1, an accident video is acquired by a drive recorder mounted on a vehicle, and the accident video is reproduced to be used for traffic safety education.

  When a drive recorder detects an impact in a collision accident or a driver suddenly operates a sudden brake and sudden steering operation, the drive recorder detects the dangerous driving based on changes in acceleration, etc. Records video for a predetermined period. In addition, there is a type of drive recorder that always records video regardless of changes in acceleration.

Japanese Unexamined Patent Publication No. 2007-011148

  However, in the drive recorder that records video based on the above acceleration change, the driver cannot perceive a dangerous state, so there is no danger of acceleration change, such as when sudden braking and sudden steering operation are not performed. Driving images cannot be recorded. That is, in the drive recorder, when the driver does not recognize the dangerous driving, the video of the dangerous driving cannot be recorded.

  For example, as shown in FIG. 41, it is assumed that the host vehicle 300 is traveling on a lane 600 and the other vehicle 500 a is traveling on a lane 601 adjacent to the lane 600. The driver of the host vehicle 300 recognizes that the other vehicle 500b is traveling in front of the host vehicle 300. However, the driver of the host vehicle 300 is not aware that the other vehicle 500a is traveling at a position adjacent to the right rear of the host vehicle 300. If the host vehicle 300 changes to a lane 601 as it is, the host vehicle 300 and the other vehicle 500a collide. In such a case, the driver of the host vehicle 300 does not recognize the presence of the other vehicle 500a, and therefore does not perform a sudden brake or a sudden steering operation in order to avoid a collision with the other vehicle 500a. At this time, if the other vehicle does not avoid, it collides, and the drive recorder records a dangerous driving image by the impact. On the other hand, when the other vehicle takes an avoidance action and avoids a collision, the drive recorder cannot record such a dangerous driving. Also, a dangerous driving in which the driver of the host vehicle 300 changes the lane of the host vehicle 300 to the front of the other vehicle 500a without performing a sudden brake and a sudden steering operation is not recorded.

  Here, it has also been analyzed that the number of vehicle accidents caused by a driver's lack of awareness of dangerous driving is 70% or more of the total number of vehicle accidents. Therefore, it is effective for traffic safety education to learn when dangerous driving that is not recognized by the driver occurs.

  In addition, a drive recorder that constantly records images records all images including dangerous driving, but it is necessary for humans to analyze and select dangerous driving images from all images, which requires a lot of labor. However, it is difficult to implement realistically.

  Further, in order to extract dangerous driving that is not recognized by the driver, as shown in FIG. 41, side and rear information different from the front recorded by a normal drive recorder may be required. When trying to deal with such side and rear images, the amount of images to be confirmed further increases, making implementation more difficult.

  Then, it aims at providing the information processing method, the information processing program, and information processing apparatus for performing the assistance which extracts dangerous driving efficiently from driving information.

  In order to solve the above-mentioned problem, it is determined whether or not a safe driving standard indicating that the driving of the host vehicle is safely performed from driving information related to the host vehicle, and the safe driving standard is determined. Provided is an information processing method including a dangerous driving extraction step for extracting a dangerous driving that is not satisfied.

  An information processing method, an information processing program, and an information processing apparatus for efficiently extracting dangerous driving from driving information can be provided.

An example of the connection relation of a dangerous driving analysis device, an information acquisition device, and a driving education terminal according to the first embodiment. An example of the block diagram which shows the hardware constitutions of a dangerous driving | running analysis apparatus, an information acquisition apparatus, and the terminal for driving education. Explanatory drawing which shows the attachment position and imaging | photography range of a periphery information acquisition apparatus. An example of a block diagram showing functional composition of each device concerning the example of the 1st embodiment. An example of peripheral information in the peripheral information DB. The schematic diagram which shows the three-dimensional projection surface on which a periphery image | video is projected. Explanatory drawing which shows an example of the extraction method of a target object An example of relative information in the relative information DB. Explanatory drawing which shows an example of the calculation method of the relative distance L. FIG. An explanatory view showing an example of dangerous driving extraction processing. An example of dangerous driving information in the dangerous driving information DB. An example of the relative information for demonstrating the specific example of a dangerous driving | running | working extraction process. An example of driving | running | working information and operation information of driving | running | working / operation information DB. An example of the positional information of positional information DB. An example of dangerous driving pattern DB. Explanatory drawing which shows the relationship between the own vehicle and another vehicle. An example of the display image | video based on the viewpoint from the other vehicle behind with respect to the own vehicle. The flowchart which shows an example of the flow of the whole process which the dangerous driving | running analysis apparatus concerning 1st Embodiment performs. The flowchart which shows an example of the flow of the target object extraction process concerning the example of 1st Embodiment. The flowchart which shows an example of the flow of the relative information calculation process concerning 1st Embodiment. The flowchart (1) which shows an example of the flow of the dangerous driving | running | working extraction process concerning 1st Embodiment. The flowchart (2) which shows an example of the flow of the dangerous driving | running | working extraction process concerning 1st Embodiment. The flowchart which shows an example of the flow of the recognition determination extraction process concerning the example of 1st Embodiment. The flowchart which shows an example of the flow of the dangerous driving classification process concerning 1st Embodiment. An explanatory view showing other examples of dangerous driving extraction processing. An explanatory view showing other examples of dangerous driving extraction processing. FIG. 10 is an example of a block diagram showing a connection relationship and hardware configuration of each device according to Modification 2. Explanatory drawing which shows the attachment position of a gaze detection apparatus. Explanatory drawing which shows the attachment position of a gaze detection apparatus. Explanatory drawing which shows an example of the area | region which can be confirmed with a mirror. The perspective view which shows the relationship between the own vehicle and another vehicle. An example of a block diagram showing functional composition of each device concerning modification 2. Explanatory drawing which shows an example of the calculation method of the gaze origin P and a gaze vector. An example of line-of-sight data in the line-of-sight data DB. An example of relative information in the relative information DB. Explanatory drawing which shows an example of the calculation method of angle (DELTA) (theta) which a line-of-sight vector and an object vector form. Explanatory drawing which shows an example of the calculation method of angle | corner (DELTA) (theta) which a mirror line-of-sight vector and an object vector form. An example of a vehicle provided with an obstacle detection sensor. An example of the block diagram which shows the function structure of each apparatus concerning 2nd Embodiment. An example of the block diagram which shows the hardware constitutions of the dangerous driving | running analysis apparatus based on 3rd Embodiment. An example of the block diagram which shows the function structure of the dangerous driving | running analysis apparatus concerning 3rd Embodiment. The schematic diagram explaining the case where the image | video of a dangerous driving is not recorded.

<First embodiment>
The dangerous driving analysis apparatus 100 according to the first embodiment determines whether or not the driving information related to the own vehicle satisfies a safe driving standard indicating that the driving of the own vehicle is performed safely, and performs safe driving. Dangerous driving that does not meet the criteria is extracted from the driving information.

  Note that the driving information related to the own vehicle includes at least a video around the own vehicle. In addition, the driving information related to the host vehicle may include a relative distance and a relative speed between an object existing around the host vehicle and the host vehicle, a speed and an acceleration of the host vehicle, and the like. In addition, driving information related to the host vehicle is acquired not only when the driver of the host vehicle recognizes dangerous driving such as when sudden braking and sudden steering operation is performed, but also when the host vehicle is in a driving state. Memorized.

  The object is an obstacle existing around the host vehicle 300. Specifically, the object is an obstacle that the driver should recognize when driving the host vehicle 300. The objects include vehicles such as cars, bicycles, and motorcycles, people, animals, and other objects that can interfere with traveling.

  First, the relationship between the dangerous driving analysis apparatus 100 of the first embodiment, the information acquisition apparatus 200 that acquires various types of information, and the driving education terminal 250 and the respective hardware configurations will be described.

(1) Relationship between Dangerous Driving Analysis Device, Information Acquisition Device, and Driving Education Terminal FIG. 1 is an example of a connection relationship between a dangerous driving analysis device, an information acquisition device, and a driving education terminal according to the first embodiment. . FIG. 2 is an example of a block diagram illustrating a hardware configuration of the dangerous driving analysis device, the information acquisition device, and the driving education terminal.

  As described above, the dangerous driving analysis device 100 extracts dangerous driving from driving information related to the host vehicle 300. The information acquisition device 200 acquires various information that is the basis of driving information related to the host vehicle 300. Examples of the various information include peripheral information around the host vehicle 300, travel information of the host vehicle 300, operation information of the host vehicle 300 by the driver, position information of the host vehicle 300, and the like. The driving education terminal 250 is a terminal for a driver who is a subject of safe driving education to view dangerous driving extracted by the dangerous driving analysis device 100.

  The dangerous driving analysis device 100, the information acquisition device 200, and the driving education terminal 250 are connected so that various types of information can be transmitted and received. Examples of the connection method include interfaces such as SCSI (Small Computer System Interface) and USB (Universal Serial Bus), and networks such as the Internet.

(2) Hardware Configuration (2-1) Dangerous Driving Analysis Device The dangerous driving analysis device 100 includes, for example, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an input / output A device I / F 104 and a communication I / F (InterFace) 108 are included. These are connected to each other via a bus 109.

  The input / output device I / F 104 is connected to input / output devices such as a display 105, a mouse 106, and a keyboard 107.

  The ROM 102 stores various control programs related to various controls described later performed by the dangerous driving analysis apparatus 100.

  The RAM 103 temporarily stores various information such as various control programs in the ROM 102 and peripheral information around the host vehicle 300 acquired from the information acquisition device 200. The RAM 103 temporarily stores information such as various flags in accordance with the execution of various control programs.

  The CPU 101 develops various control programs stored in the ROM 102 in the RAM 103 and performs various controls described later.

  The communication I / F 108 performs communication such as transmission / reception of commands or data between the information acquisition device 200 and the driving education terminal 250 based on the control of the CPU 101.

  The bus 109 includes, for example, a PCI (Peripheral Component Interconnect) bus, an ISA (Industrial Standard Architecture) bus, and the like, and connects the above configurations to each other.

(2-2) Information Acquisition Device The information acquisition device 200 includes, for example, a CPU 201, a ROM 202, a RAM 203, an input / output device I / F 204, and a communication I / F 209. These are connected to each other via a bus 214.

(A) Input / output device I / F
The input / output device I / F 204 is connected to a peripheral information acquisition device 205, a travel information acquisition device 206, an operation information acquisition device 207, a position information acquisition device 208, and the like. Information detected by each of the acquisition devices 205 to 208 is output to the RAM 203, the CPU 201, the communication I / F 209, and the like via the input / output device I / F 204.

(B) Peripheral information acquisition device The peripheral information acquisition device 205 acquires peripheral information including one or more objects existing around the host vehicle 300. The peripheral information refers to, for example, peripheral video around the host vehicle 300, target information such as the position and size of the target around the host vehicle 300, and the like. In the present embodiment example, it is assumed that the peripheral information acquisition device 205 acquires a peripheral video as peripheral information. For example, the peripheral information acquisition device 205 includes an imaging device such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera, and acquires peripheral video around the host vehicle 300.

  FIG. 3 is an explanatory diagram showing the attachment position and photographing range of the peripheral information acquisition device. For example, as shown in FIG. 3, the peripheral information acquisition device 205 includes four cameras, for example, a front camera 205a, a right camera 205b, a left camera 205c, and a rear camera 205d. The front camera 205 a is attached to the center of the front bumper of the vehicle 300 and photographs the front of the vehicle 300. The rear camera 205d is attached to the center of the rear bumper of the vehicle 300 and photographs the rear of the vehicle 300. The right camera 205 b is attached to the center of the right side surface of the vehicle 300 and photographs the right side of the vehicle 300. The left camera 205 c is attached to the center of the left side surface of the vehicle 300 and photographs the left side of the vehicle 300.

  Each of the cameras 205a to 205d is, for example, a camera using an ultra-wide angle lens having a field angle of 180 degrees. Therefore, as shown in FIG. 3, the front camera 205 a captures the front area 210 of the vehicle 300, the right camera 205 b captures the right area 211 of the vehicle 300, and the left camera 205 c captures the left area of the vehicle 300. The rear camera 205 d captures the rear region 213 of the vehicle 300. The shooting areas of the cameras 205a to 205d are configured to overlap with areas shot by adjacent cameras.

  The camera characteristics such as the mounting position and mounting angle of each of the cameras 205a to 205d, and other camera lens distortion correction values and the focal length can be adapted to a spatial coordinate system with the center point O of the vehicle 300 as the origin. Correction, so-called calibration is performed. By executing this calibration, the images captured by the cameras 205a to 205d can be incorporated into a spatial coordinate system having the center point O of the vehicle 300 as the origin. Note that the spatial coordinate system is represented by X, Y, and Z coordinates. For example, the center point O is defined by the position of the center of the host vehicle 300, which is the half of the vehicle width of the host vehicle 300 and the half of the vehicle length. (X, Y, Z) = (0, 0, 0). Y is the forward direction of the vehicle, X is the direction crossing the forward direction, and Z is the height direction.

  As shown in the figure, each of the cameras 205a to 205d is preferably attached to the center of the front surface, right side surface, left side surface, and rear surface of the vehicle 300. However, it suffices if the shooting area of each camera 205a to 205d partially overlaps the shooting area of the adjacent camera, and the mounting position of each camera 205a to 205d is not particularly limited. For example, the right camera 205 b and the left camera 205 c can be attached to the left and right door mirrors of the vehicle 300. Further, the number of cameras is not limited to four as long as the shooting areas of each camera partially overlap and it is possible to capture a 360 ° range around the vehicle 300.

  In addition, each of the cameras 205a to 205d captures 30 frames per second, for example. Video data captured by the peripheral information acquisition device 205 including the cameras 205a to 205d is stored in the RAM 203 via the input / output device I / F 204.

  As described above, the dangerous driving analysis apparatus 100 can acquire the surrounding image of the entire periphery of the vehicle 300 by the image processing unit 121 described later by capturing the images with the cameras 205a to 205d. Therefore, since the dangerous driving analysis apparatus 100 can extract the object all around the vehicle 300, the object can be extracted even in a blind spot that is difficult for the driver of the vehicle 300 to visually recognize.

(C) Travel Information Acquisition Device The travel information acquisition device 206 acquires travel information such as the speed and acceleration of the host vehicle 300. The speed and acceleration include, for example, data on the forward direction (hereinafter referred to as Y direction) of the host vehicle and the direction intersecting the forward direction (hereinafter referred to as X direction). The travel information acquisition device 206 calculates the speed and acceleration from, for example, but not limited to, the number of rotations of the axle supporting the tire within a predetermined time, the diameter of the tire, and the like. In addition, the travel information acquisition device 206 calculates the speed and acceleration based on the time interval of sensor data output when a plurality of sensors provided on the tire of the host vehicle 300 come into contact with the road and the sensor installation interval. Also good.

(D) Operation Information Acquisition Device The operation information acquisition device 207 acquires operation information indicating what operation the driver of the host vehicle 300 has performed. The operation information of the host vehicle is information including the presence / absence of a brake operation, the steering angle, the presence / absence of a turn indicator, the presence / absence of an accelerator operation, and the like.

  For example, the operation information acquisition device 207 detects a signal corresponding to the operation of the brake and the accelerator of the host vehicle 300, and detects the presence or absence of the brake operation and the accelerator operation. Further, the operation information acquisition device 207 detects the steering angle of the handle based on the rotation angle of the rotation axis of the handle, for example. Further, the operation information acquisition device 207 detects, for example, a signal from a direction indicator, and detects whether the direction indicator is operated in the left or right direction.

(E) Location Information Acquisition Device The location information acquisition device 208 acquires location information of the host vehicle 300 on the map. For example, the position information acquisition device 208 receives radio waves from GPS (Global Position System) hygiene and acquires position information on a map based on information included in the radio waves.

(F) ROM, RAM, communication I / F
The ROM 202 stores various control programs executed by the information acquisition device 200.

  The RAM 203 temporarily stores various control programs, various flags, and various types of information received from the acquisition devices 205 to 208 in the ROM 202.

  The communication I / F 209 transmits and receives data such as peripheral information, travel information, operation information, position information, and various commands to and from the dangerous driving analysis apparatus 100 based on the control of the CPU 201.

(G) CPU
The CPU 201 develops various control programs stored in the ROM 202 in the RAM 203 and performs various controls. For example, the CPU 201 controls acquisition devices such as the peripheral information acquisition device 205 and the travel information acquisition device 206 by executing various control programs, and starts acquisition of various information such as peripheral images.

(2-3) Driving Education Terminal The driving education terminal 250 is a terminal used by a user who receives safe driving education, and particularly dangerous driving can be viewed on this terminal.

  The driving education terminal 250 includes, for example, a CPU 251, a ROM 252, a RAM 253, an input / output device I / F 254, and a communication I / F 258. These are connected to each other via a bus 259. The input / output device I / F 254 is connected to input / output devices such as a display 255, a mouse 256, and a keyboard 257. The input / output device I / F 254 receives a display instruction of a desired dangerous driving from the user via the mouse 256 and the keyboard 257. A speaker or the like for outputting sound may be connected to the input / output device I / F 254.

  The CPU 251 expands various control programs stored in the ROM 252 to the RAM 253, acquires data related to dangerous driving from the dangerous driving analysis device 100, and outputs the data to the display 255 or the like.

  Other configurations are substantially the same as those of the dangerous driving analysis apparatus 100, and thus the description thereof is omitted.

(3) Functional Configuration Next, functional configurations of the dangerous driving analysis device 100, the information acquisition device 200, and the driving education terminal 250 will be described.

  FIG. 4 is an example of a block diagram illustrating a functional configuration of each device according to the first embodiment. Note that the connection lines of the respective functional units shown in FIG. 4 show an example of the data flow, and do not describe all the data flows.

  First, the functional configuration of the information acquisition apparatus 200 will be described.

(3-1) Information Acquisition Device When the hardware configurations of the information acquisition device 200 cooperate with each other to execute the program, the information acquisition device 200 functions as each functional unit described later.

  The functional units of the information acquisition device 200 include, for example, a peripheral information acquisition unit 220, a travel information acquisition unit 221, an operation information acquisition unit 222, a position information acquisition unit 223, a transmission / reception unit 224, an acquisition data DB 225, and the like.

(3-1-1) Each acquisition unit, acquisition data DB
The peripheral information acquisition unit 220 acquires the peripheral video captured by the peripheral information acquisition device 205 including the front camera 205a, the right camera 205b, the left camera 205c, and the rear camera 205d shown in FIG. 3 and stores it in the acquisition data DB 225. .

  The travel information acquisition unit 221 receives travel information such as speed and acceleration acquired by the travel information acquisition device 206 and stores it in the acquisition data DB 225.

  The operation information acquisition unit 222 receives the operation information acquired by the operation information acquisition device 207 and stores it in the acquisition data DB 225.

  The position information acquisition unit 223 receives the position information of the host vehicle 300 on the map acquired by the position information acquisition device 208 and stores it in the acquisition data DB 225.

(3-1-2) Transmission / Reception Unit The transmission / reception unit 224 of the information acquisition device 200 transmits / receives various commands and various data in the acquisition data DB 225 to / from the transmission / reception unit 120 of the dangerous driving analysis device 100.

(3-2) Dangerous Driving Analysis Device When each hardware configuration of the dangerous driving analysis device 100 executes a program in cooperation with each other, the dangerous driving analysis device 100 functions as each functional unit described later.

  The functional units of the dangerous driving analysis device 100 include, for example, a transmission / reception unit 120, a video processing unit 121, an object extraction unit 122, a relative information calculation unit 123, a dangerous driving extraction unit 124, a recognition determination unit 125, and a dangerous driving classification unit 126. , A display video generation unit 127 and the like are included. Furthermore, in order to store various information, the dangerous driving analysis apparatus 100 includes a peripheral information DB 128, a relative information DB 129, a dangerous driving information DB 130, a driving / operation information DB 131, a position information DB 132, a dangerous driving pattern DB 133, various correspondence tables DB 134, and the like. including.

(3-2-1) Transmission / Reception Unit The transmission / reception unit 120 of the dangerous driving analysis device 100 transmits / receives various data, various commands, and the like to / from the transmission / reception unit 224 of the information acquisition device 200.

(3-2-2) Peripheral information DB
The peripheral information DB 128 acquires and stores a peripheral video around the host vehicle from the information acquisition device 200 as peripheral information including objects around the host vehicle. The peripheral video includes video captured by the peripheral information acquisition device 205 including the front camera 205a, the right camera 205b, the left camera 205c, and the rear camera 205d.

  FIG. 5 is an example of peripheral information in the peripheral information DB. For example, the peripheral information DB 128 stores a frame number and video data of each camera 205 for each frame. The video data includes a front video shot by the front camera 205a, a right video shot by the right camera 205b, a left video shot by the left camera 205c, and a rear video shot by the rear camera 205d.

(3-2-3) Video Processing Unit The video processing unit 121 obtains video data captured by each camera 205 from the peripheral information DB 128 and synthesizes the video data onto a three-dimensional projection plane 400 as shown in FIG. Generate peripheral video. FIG. 6 is a schematic diagram showing a three-dimensional projection plane onto which the peripheral video is projected.

  Specifically, first, the video processing unit 121 acquires a correspondence relationship between each pixel of each of the cameras 205a to 205d and each coordinate of the three-dimensional projection plane 400 from various correspondence table DBs 134 described later. Next, the video processing unit 121 projects the video data of each of the cameras 205a to 205d on the three-dimensional projection surface 400 based on the correspondence relationship of the coordinates, and generates a peripheral video.

(3-2-4) Object Extraction Unit The object extraction unit 122 extracts one or more objects existing around the host vehicle 300 from the peripheral video generated by the video processing unit 121.

  FIG. 7 is an explanatory diagram illustrating an example of an object extraction method. In FIG. 7, the host vehicle 300 is traveling on the lane 600, and the other vehicle 500 that is the object is traveling on the lane 601 adjacent to the left. First, the object extraction unit 122 extracts edges based on, for example, the luminance contrast ratio in the surrounding video, and detects the lane display lines 602a to 602d. Next, the object extraction unit 122 detects the vanishing point D from the intersection of the lane display lines 602a to 602d, and determines the object search range based on the vanishing point D. For example, the search range can be determined as a range surrounded by the vanishing point D and the lane display lines 602a to 602d or a predetermined range including the range.

  The object extraction unit 122 extracts object candidates around the host vehicle 300 within the search range. An object candidate is a candidate that can be an object. Next, the target object extraction unit 122 compares the target object candidate with the pattern data storing various characteristics of the target object stored in advance, and determines whether or not the target object candidate is the target object. decide. For example, when the object candidate matches the vehicle pattern data, the object extraction unit 122 determines the object candidate as the object, and further determines the object type indicating what the object is. . Examples of the object type include passenger cars, trucks / buses, bicycles / bikes, and pedestrians. In the example of FIG. 7, the object extraction unit 122 extracts another vehicle 500 as the object. On the other hand, the object extraction unit 122 determines that the object candidate is not an object when the object candidate does not match the pattern data of the object.

  In each frame, the object extraction unit 122 sets the object flag to “1” when the object is extracted, and sets the object flag to “0” when the object is not extracted.

  Further, the object extraction unit 122 assigns an object number to each object in order to identify each extracted object, and acquires the relative position Q of the object with respect to the host vehicle 300. Examples of the relative position Q include coordinates of a portion of the target object that is closest to the host vehicle 300. When the other vehicle 500 is traveling left front relative to the host vehicle 300 as shown in FIG. 7, the object extraction unit 122 acquires the right corner point Q of the other vehicle 500 as a relative position. On the other hand, when the other vehicle 500 is traveling right rear with respect to the host vehicle 300, the object extraction unit 122 acquires the left corner point of the other vehicle 500 as a relative position.

  Here, the relative position Q is defined by a spatial coordinate system having an arbitrary center point O of the vehicle 300 as an origin. The object extraction unit 122 only needs to be able to grasp the position of the object, and the relative position of the object is not limited to the relative position Q described above.

  The object extraction unit 122 stores the object flag, the object number, the object type, and the relative position Q in the relative information DB 129.

(3-2-5) Relative information calculation unit, relative information DB
The relative information DB 129 stores the object flag, the object number, the object type, and the relative position Q acquired from the object extracting unit 122 for each frame and each object. In addition, the relative information DB 129 stores the relative distance L, the relative speed V, TTC (Time To Collision), and the like calculated by the relative information calculation unit 123 for each frame and each object.

  FIG. 8 is an example of relative information in the relative information DB. The relative information DB 129 stores, for example, a frame number, an object flag, an object number, an object type, a relative position Q, a relative distance L, a relative speed V, and a TTC. TTC is the expected time until the host vehicle 300 and each object collide. The relative distance L, the relative speed V, and the TTC have data values in the X direction and the Y direction, respectively.

  Since the object approaches or leaves the host vehicle 300, the number of objects increases or decreases. For example, the number of objects in the frame with frame number = 1 in FIG. 8 is “N”, and the number of objects in the frame with frame number = 2 is “M”, which is different. Further, the objects having the object numbers “1” and “2” existed in the frames having the frame numbers = 1 and 2, but do not exist in the frame having the frame numbers = 1.

  The relative information calculation unit 123 calculates relative information such as a relative distance L, a relative speed V, and TTC between the host vehicle 300 and one or more objects. The relative information calculation method will be described again with reference to FIG. First, the relative information calculation unit 123 reads the relative position Q of the object from the relative information DB 129. Further, the relative information calculation unit 123 calculates a relative distance Lx ′ in the X direction and a relative distance Ly ′ in the Y direction based on the distance between the center point O of the host vehicle 300 and the relative position Q of the target object. Referring to the relative information DB 129 of FIG. 8, when the frame number = 1 and the object number = 2, the relative position Q is (X_21, Y_21, Z_21). Since the coordinates of the center point O are (0, 0, 0), the relative distance Lx ′ in the X direction is calculated as Lx ′ = X_21. Similarly, the relative distance Ly ′ in the Y direction is calculated as Ly ′ = Y_21. Here, the relative distance Lx ′ and the relative distance Ly ′ in the Y direction are distances including half of the vehicle width of the host vehicle 300 and half of the vehicle length. Therefore, the relative information calculation unit 123 calculates the relative distance Lx by subtracting Lxcar, which is half the vehicle width in the X direction, of the host vehicle 300 from the relative distance Lx ′. Similarly, the relative information calculation unit 123 calculates the relative distance Ly by subtracting Lycar, which is half the vehicle length of the host vehicle 300 in the Y direction, from the relative distance Ly ′.

  As a more specific method for calculating the relative distance L, an example of a method for calculating the relative distance in consideration of the installation height of the camera 205, the focal length, and the like will be described below. FIG. 9 is an explanatory diagram illustrating an example of a method for calculating the relative distance L. It is assumed that the camera 205 that captures a peripheral image of the host vehicle 300 is provided on the vehicle body above the center point O of the host vehicle 300. The height of the camera 205 from the ground is H, the focal length of the lens of the camera 205 is f, and the coordinates of the vanishing point D are (XD, YD, ZD). Here, if the relative position Q of the other vehicle 500 as the object is (XQ, YQ, ZQ), the relative information calculation unit 123 calculates the relative distance Lx ′ and the relative distance based on the following equations (1) and (2). The distance Ly ′ is calculated.

Relative distance Ly ′ = f × H / | YQ−YD | (1)
Relative distance Lx ′ = Ly ′ × f / | XQ−XD | (2)
The relative information calculation unit 123 calculates the relative distance Lx and the relative distance Ly in the same manner as described above based on the relative distance Lx ′ and the relative distance Ly ′, and Lxcar and Lycar, and stores them in the relative information DB 129. Thereby, it is possible to calculate the relative distance Lx and the relative distance Ly in the current frame of interest.

  Next, in order to calculate the relative speed V, the relative information calculation unit 123 acquires the relative distance Ly and the relative distance Lx of the target object in the previous frame immediately before the current frame. That is, the relative information calculation unit 123 calculates the difference in the relative distance L between the previous frame and the current frame for the same object number. Next, the relative information calculation unit 123 calculates the relative velocity Vy in the Y direction based on the difference in the relative distance Ly between the current frame and the previous frame and the time between frames. Similarly, the relative information calculation unit 123 calculates the relative velocity Vx in the X direction based on the difference in the relative distance Lx between the current frame and the previous frame and the time between frames.

  Furthermore, the relative information calculation unit 123 calculates TTC required until the target object and the host vehicle 300 collide based on the relative distance L and the relative speed V. For example, assuming that the object and the host vehicle 300 move at a constant speed, the TTC can be calculated based on the following equation (3).

TTC = relative distance / relative speed (3)
In order to calculate TTC for each of the X direction and the Y direction, first, the relative information calculation unit 123 acquires the relative distances Lx and Ly and the relative velocities Vx and Vy from the relative information DB 129. The relative information calculation unit 123 calculates the TTCx in the X direction and the TTCy in the Y direction based on the equation (3).

  The relative information calculation unit 123 calculates the TTC based on the equation (3) on the assumption that the object and the host vehicle 300 move at a constant speed, but calculates the TTC further considering the relative acceleration and the like. You may do it.

(3-2-6) Dangerous driving extraction unit, dangerous driving information DB
The dangerous driving extraction unit 124 extracts the dangerous driving that does not satisfy the safe driving standard from the driving information regarding the host vehicle 300. Note that the driving information related to the host vehicle 300 includes at least peripheral information stored in the peripheral information DB 128, for example. In addition, the driving information related to the host vehicle 300 may include various relative information in the relative information DB 129 and various information in the travel / operation information DB 131.

  Here, the safe driving standards include, for example, the following standards.

  (I) Relative distance reference values Lxs and Lys (Ls: Length standard) for the relative distances Lx and Ly between the subject vehicle 300 and the object.

  (Ii) TTC reference values TTCxs and TTCys (TTCs: Time To Collision standard) for TTCx and TTCy.

  (Iii) TTC change rate reference values DTTCxs and DTTCys (DTTCs: Deviation rate of Time To Collision standard) with respect to the change rates of TTCx and TTCy per predetermined unit time.

  Next, the dangerous driving extraction process according to the present embodiment will be described.

(A) Overview of Hazardous Driving Extraction Process First, an overview of the dangerous driving extraction process will be described with reference to FIG. FIG. 10 is an explanatory diagram illustrating an example of the dangerous driving extraction process.

  Here, the peripheral information includes video data for each frame as shown in FIG. 5 as an example of the peripheral information DB 128, and is configured along the time axis. FIG. 10 shows the time axis of this peripheral information.

  First, the dangerous driving extraction unit 124 refers to the relative information DB 129 and extracts a frame section in which the object exists around the host vehicle 300 on the time axis of the peripheral information based on the object flag = 1. Hereinafter, a frame section in which an object exists around the host vehicle 300 is referred to as an “object-existing section”, and a frame section in which the object does not exist around the host vehicle 300 is referred to as an “no-object section”. As a result of the processing, as shown in FIG. 10, the “object existence section” defined by the start frame A and the end frame A is extracted on the time axis of the peripheral information.

  Next, the dangerous driving extraction unit 124 refers to the relative information DB 129 and compares the relative distances Lx and Ly with the above-mentioned reference (i) Lxs and Lys for each target existing in the “target presence section”. To do. As a result, the dangerous driving extraction unit 124 extracts the “approaching section” from the “object existence section” where the relative distance Lx is smaller than Lxs or the relative distance Ly is smaller than Lys. Here, in FIG. 10, there is only one target object in the “target existence section”, and the “approach section” between the target object and the host vehicle 300 is defined by the start frame B and the end frame B.

  Here, the dangerous driving extraction unit 124 starts the first frame for the “approaching section” of the plurality of objects existing in the “object-containing section” among the “object-containing section”, The section defined by the end frame that ends last is extracted as a “dangerous driving candidate section”. In other words, the dangerous driving extraction unit 124 extracts, as one “hazardous driving candidate section”, a section obtained by removing a section that does not correspond to the “approaching section” of any target object from one “target existing section”. Further, the dangerous driving extraction unit 124 assigns a “dangerous driving candidate number” to each “dangerous driving candidate section”. This will be described in detail later with reference to FIG. In the case of FIG. 10, since only one target exists in the “target presence section”, one “approach section” corresponds to one “dangerous driving candidate section”.

  Further, the dangerous driving extraction unit 124 refers to the relative information DB 129 of FIG. 8 and calculates the minimum value of TTCx and TTCy and / or the maximum value of the change rate of TTCx and TTCy for the “approach section” of each object. calculate. Next, the dangerous driving extraction unit 124 determines the calculation result based on the criteria (ii) and (iii). That is, the dangerous driving extraction unit 124 determines whether or not the condition that the minimum value of TTCx is smaller than TTCxs and / or the condition that the minimum value of TTCy is smaller than TTCys is satisfied. Similarly, the dangerous driving extraction unit 124 determines whether or not the condition that the maximum value of the change rate of TTCx is larger than DTTCxs and / or the condition that the maximum value of the change rate of TTCy is larger than DTTCys is satisfied. Determine.

  If it is determined that at least one of the conditions is met, the dangerous driving extraction unit 124 determines that the driving is dangerous driving, and sets the dangerous determination flag for the “dangerous driving candidate section” to “1” (danger determination flag = 1). On the other hand, if it is determined that none of the conditions is satisfied, the dangerous driving extraction unit 124 determines that the driving is not dangerous driving, and sets the dangerous determination flag for the “dangerous driving candidate section” to “0” (danger determination flag). = 0). By increasing the number of combinations of conditions, it is possible to extract “dangerous driving candidate sections” with a higher degree of danger.

  Finally, a “dangerous driving candidate section” having a danger determination flag of “1” can be extracted as a dangerous driving section.

  Here, when the relative distances Lx and Ly are smaller than Lxs and Lys, the distance between the host vehicle 300 and the target is close, and the risk of the host vehicle 300 colliding with the target is high. In addition, when the minimum values of TTCx and TTCy are smaller than TTCxs and TTCys, the distance between the host vehicle 300 and the target object is short, and there is a high risk that the host vehicle 300 will eventually collide with the target object. Further, when the maximum value of the rate of change of TTCx and TTCy is larger than DTTCxs and DTTCys, the distance between the host vehicle 300 and the object is rapidly approaching, and the risk of collision is high. Therefore, when a section satisfying such a condition is extracted as a section for dangerous driving, it is possible to effectively learn under what circumstances dangerous driving occurs and to perform safe driving education efficiently.

  Further, in the dangerous driving extraction process, a condition that the relative distance L between the host vehicle 300 and the target object is smaller than the relative distance reference value Ls is indispensable as a condition for extracting dangerous driving information. . For example, even if the minimum value of TTC is smaller than TTCs due to sudden acceleration, if the host vehicle 300 and the target object are far away from each other, the risk that the host vehicle 300 and the target object collide is Low. Therefore, a section in which the relative distance L between the host vehicle 300 and the object is Ls or more is excluded from the dangerous driving section. Thereby, only a section with high risk of collision can be extracted as a section of dangerous driving.

  In the dangerous driving extraction process, the dangerous driving extracting unit 124 first extracts an “approaching section” having a relative distance L smaller than Ls from the “object existence section”, and then extracts the reference (ii) and The determination based on (iii) is performed. In this way, by first extracting the “approach section”, the section having a high risk of collision is narrowed down, and the data to be determined based on the above criteria (ii) and (iii) is narrowed down. Therefore, the calculation related to the extraction of dangerous driving can be performed efficiently.

(B) Example of Dangerous Driving Information DB The dangerous driving extraction unit 124 stores the above extraction results and determination results in the dangerous driving information DB 130. FIG. 11 is an example of dangerous driving information in the dangerous driving information DB.

  The dangerous driving information DB 130 stores, for each “dangerous driving candidate section”, a “dangerous driving candidate number”, a frame range of the “dangerous driving candidate section”, a dangerous driving pattern, presence / absence of dangerous recognition, an object number, an object type, a danger The determination flag, the frame range of the “approach section”, the minimum value of TTC, the maximum value of the change rate of TTC (DTTC), and the like are stored.

  Each “dangerous driving candidate section” is assigned a “dangerous driving candidate number” and is defined by a start frame and an end frame. The object number and the object type are information for identifying objects existing in each “dangerous driving candidate section”. The risk determination flag, the frame range of the “approach section”, the minimum value of TTC, the maximum value of change rate of TTC (DTTC), and the like are stored for each object. The dangerous driving pattern and the presence or absence of danger recognition will be described later.

(C) Specific Example of Dangerous Driving Extraction Process Next, a specific example of the dangerous driving extraction process will be described with reference to FIG. FIG. 12 is an example of relative information for explaining a specific example of the dangerous driving extraction process.

  In FIG. 12, it is assumed that the object flag = 1 in the frames with the frame numbers = 1 to 7 and the object flag = 0 in the frame with the frame number = 8 (not shown). Therefore, the dangerous driving extraction unit 124 refers to the relative information in FIG. 12 and first extracts the continuous frames with the frame numbers = 1 to 7 as the “object existence section”.

  Next, the dangerous driving extraction unit 124 compares the relative distances Lx and Ly with Lxs and Lys for each object existing in each frame in the “object existence section”, Extract the “approach section” of the object.

  For example, a passenger car that is an object with object number = 1 will be described. In FIG. 12, the relative distances Lx and Ly smaller than Lxs and Lys are shaded with respect to the object with the object number = 1. In the frame with frame number = 1, the relative distance Lx_1 (1) is greater than or equal to Lxs, and the relative distance Ly_1 (1) is greater than or equal to Lys. On the other hand, in the frame with frame number = 2, the relative distance Lx_1 (2) is greater than or equal to Lxs, and the relative distance Ly_1 (2) is less than Lys. Therefore, the “approach section 1” of the object with the object number = 1 starts from the frame with the frame number = 2. In the frame with frame number = 3, the relative distance Ly_1 (3) is smaller than Lys. In the frame with frame number = 4, the relative distance Lx_1 (4) is smaller than Lxs, and the relative distance Ly_1 (4) is smaller than Lys. Further, in the frame of frame number = 5, the relative distance Lx_1 (5) is smaller than Lxs, and the relative distance Ly_1 (5) is smaller than Lys. However, in the frames with frame numbers = 6 and 7, the relative distances Lx and Ly are greater than or equal to Lxs and Lys. Therefore, “approach section 1” ends with the frame of frame number = 5. Therefore, the dangerous driving extraction unit 124 extracts frame sections from frame numbers = 2 to 5 as “approach section 1” of the object having the object number = 1.

  Next, the track / bus that is the object with the object number = 2 will be described. In FIG. 12, with respect to the object with the object number = 2, the relative distances Lx and Ly smaller than Lxs and Lys are shaded with sand patterns. By the same procedure as described above, the dangerous driving extracting unit 124 extracts frame sections from frame numbers 3 to 6 as “approach section 2” of the object having the object number = 2.

  Here, in the “approach section 1” and “approach section 2” of the two objects, the “approach section” starts first in the frame of the “approach section 1” with the frame number = 2. On the other hand, the “approaching section” ends last in the frame of “approaching section 2” with frame number = 6. Therefore, the dangerous driving extraction unit 124 extracts the sections with frame numbers = 2 to 6 as “dangerous driving candidate sections”. However, in the case of the relative information in FIG. 12, it is assumed that only “approach section 1” and “approach section 2” are extracted as “approach section”.

  Further, the dangerous driving extraction unit 124 calculates the minimum value of TTCx and TTCy and the maximum value of the rate of change of TTCx and TTCy in the “approach section 1” for the object with the object number = 1. Similarly, the dangerous driving extraction unit 124 calculates the minimum value of TTCx and TTCy and the maximum value of the rate of change of TTCx and TTCy in the “approach zone 2” for the object with the object number = 2.

  When the dangerous driving extraction unit 124 compares the calculation result with each reference value and determines that it is dangerous driving, the dangerous driving extraction unit 124 sets the dangerous determination flag for the “dangerous driving candidate section” to “1” (danger determination flag = 1).

(3-2-7) Travel / operation information DB
The travel / operation information DB 131 acquires travel information of the host vehicle 300 from the travel information acquisition device 206, and acquires and stores operation information of the host vehicle 300 from the operation information acquisition device 207.

  FIG. 13 is an example of travel information and operation information in the travel / operation information DB. The travel information includes the speed and acceleration of the host vehicle 300 in the X direction and the Y direction. The operation information includes the presence / absence of a brake operation, the steering angle θs, the presence / absence of a direction indicator, the presence / absence of an accelerator operation, etc. in the host vehicle 300. The time when the brake operation is detected is “1”, and the time when the brake operation is not detected is “0”. The steering angle is 0 ° when the steering wheel is not rotated. In addition, it stores a direction instruction to the left or right according to the operation of the direction indicator. The time when the accelerator operation is detected is “1”, and the time when the accelerator operation is not detected is “0”.

(3-2-8) Recognition Determination Unit The recognition determination unit 125 determines whether or not the driver recognizes that the driving of the host vehicle 300 is dangerous driving.

  First, the recognition determination unit 125 refers to the dangerous driving information DB 130 illustrated in FIG. 11 and reads the “approach section” of the object with the risk determination flag = 1 for each “dangerous driving candidate”. For example, in the dangerous driving information DB 130 of FIG. 11, the recognition determining unit 125 sets the start frame ns <b> 1 </ b> _ <b> 1 of the “approach zone” in the record of “dangerous driving candidate number” = 1, object number = 1, and dangerous determination flag = 1. Read end frame ne1_1.

  Next, the recognition determination unit 125 refers to the travel / operation information DB 131 for each frame in the “approach zone” and determines whether or not the driver recognizes dangerous driving. When the danger determination flag = 0, it is not dangerous driving, and it is not necessary to determine whether or not dangerous driving is recognized.

  Specifically, for example, the following determination is performed.

  For example, the recognition determination unit 125 refers to the acceleration αy in the forward direction (Y direction) of the host vehicle 300 for a predetermined frame. When determining that the acceleration αy is a negative acceleration and the magnitude of the acceleration αy is greater than a predetermined value, the recognition determination unit 125 determines that the driver recognizes the dangerous driving. Here, the magnitude of the negative acceleration αy being larger than the predetermined value means that the rate of decrease of the speed Vy is larger than the predetermined value, for example, when the driver performs a sudden braking operation. That is, since the driver recognizes dangerous driving and spontaneously performs a sudden braking operation, it can be determined that the rate of decrease in the speed of the host vehicle 300 is greater than a predetermined value. A similar determination can be made for the acceleration αx in the X direction.

  Further, the recognition determination unit 125 may determine that the driver recognizes the dangerous driving when it is determined that the brake operation of the host vehicle 300 is performed with reference to the travel / operation information DB 131.

  The recognition determination unit 125 may determine whether or not the dangerous driving is recognized based on both the brake operation and the acceleration with reference to the travel / operation information DB 131. That is, the recognition determination unit 125 may determine that dangerous driving is recognized when the driver performs a braking operation and the magnitude of the negative acceleration is greater than a predetermined value. Here, the phenomenon that the magnitude of the negative acceleration becomes larger than a predetermined value may occur not only when the brake operation is performed, but also when the host vehicle 300 travels on a steep uphill, for example. By reliably detecting the operation performed by the driver based on the brake operation, it is possible to reliably detect the presence or absence of recognition for dangerous driving.

  Further, the recognition determination unit 125 may determine that the driver recognizes the dangerous driving when the driving / operation information DB 131 is referred to and the change rate of the steering angle is determined to be large. The case where the rate of change of the steering angle is large may be a case where the driver suddenly operates the steering wheel to avoid a collision with an object, and it can be assumed that the driver recognizes the dangerous driving.

  In addition, if the recognition determination unit 125 determines that the host vehicle 300 is traveling in a direction in which the object does not exist, the recognition determination unit 125 determines that the driver recognizes dangerous driving. For example, the recognition determination unit 125 refers to the relative information DB of FIG. 8, acquires the relative position Q of the target object in the “approach section” with the above-described risk determination flag = 1, and the target object is relative to the host vehicle 300. Know where it is. In addition, the recognition determination unit 125 acquires the speed, acceleration, steering angle, and the like of the host vehicle 300 in the “approach section” from the travel / operation information DB 131 of FIG. It is determined in which direction the vehicle 300 is traveling. Finally, when the recognition determination unit 125 determines that the host vehicle 300 is traveling in a direction away from the object, the recognition determination unit 125 determines that the driver recognizes dangerous driving.

  The recognition determination unit 125 stores the determination result in the dangerous driving information DB of FIG. In the case of FIG. 11, a determination result is stored for each “dangerous driving candidate number” in the item of dangerous recognition. For example, “Yes” is recognized when dangerous driving is recognized, and “No” is recognized when dangerous driving is not recognized. “None” is stored.

  In addition, the recognition determination part 125 may determine not only the presence or absence of recognition that it is dangerous driving, but the recognition degree which shows how much dangerous driving is recognized. For example, the recognition determination unit 125 determines the degree of recognition based on a state in which acceleration increases instantaneously or a braking operation is performed instantaneously. A correspondence table of the rate of change of acceleration and the time required for the brake operation and the degree of recognition is prepared in advance, and the recognition determination unit 125 required the rate of change of acceleration and the brake operation based on these correspondence tables. The degree of recognition is determined according to time.

(3-2-9) Location information DB
The position information DB 132 acquires the position information of the host vehicle 300 on the map from the position information acquisition device 208 and stores it.

  FIG. 14 is an example of position information in the position information DB. For example, in the global coordinates defined on the road map, the position of the host vehicle 300 is specified by the coordinates of Xg and Yg. Based on these position information and road map information described later, it is possible to grasp where the host vehicle 300 is located such as an intersection or a straight road.

(3-2-10) Dangerous driving pattern DB
FIG. 15 is an example of a dangerous driving pattern DB. The dangerous driving pattern DB 133 stores a dangerous driving pattern for each dangerous driving pattern ID. The dangerous driving pattern is a definition for classifying the dangerous driving, for example, a classification for classifying what kind of object the dangerous driving has occurred at what place.

(3-2-11) Dangerous Driving Classification Unit The dangerous driving classification unit 126 classifies the dangerous driving pattern for each “dangerous driving candidate” in the dangerous driving information DB 130.

  First, the dangerous driving classification unit 126 reads out the position information of the host vehicle 300 in the position information DB 132 and the road map information in the various correspondence tables DB 134 for the frame range for each “dangerous driving candidate”. In addition, the dangerous driving classification unit 126 identifies which position on the map the host vehicle 300 is traveling on such as an intersection or a straight road based on the position information and the road map information.

  Next, the dangerous driving classification unit 126 reads the traveling information and the operation information of the host vehicle from the traveling / operation information DB 131 for the frame range corresponding to each “dangerous driving candidate”. The dangerous driving classification unit 126 determines a traveling state such as whether the own vehicle is in a straight traveling state, a right turn / left turn state, or a stopped state based on the traveling information and operation information of the own vehicle. For example, when the direction indicator is “right”, the steering angle θs indicates the movement of the steering wheel in the right direction, and the speed V is not “0”, the dangerous driving classification unit 126 determines that the host vehicle 300 Is determined to be turning right.

  Next, the dangerous driving classification unit 126 reads out the object type of the danger determination flag = 1 for each “dangerous driving candidate” from the dangerous driving information DB 130. The dangerous driving classification unit 126 refers to the dangerous driving pattern DB 133 based on the position and traveling state on the map determined as described above and the object type of the danger determination flag = 1, and sets each “dangerous driving candidate”. Identify dangerous driving patterns.

  For example, in the dangerous driving information DB 130 of FIG. 11, the object type for which “dangerous driving candidate number” = 1 and the danger determination flag = 1 is a passenger car. At this time, according to the travel / operation information DB 131, it is assumed that the host vehicle 300 is going to turn right at the intersection. The dangerous driving classification unit 126 selects the most suitable dangerous driving pattern based on the combination of “turn right at the intersection” and “passenger car”. Here, “Meeting with a passenger car when turning right at the intersection” is selected and stored as a dangerous driving pattern of “dangerous driving candidate number” = 1 in the dangerous driving information DB 130.

(3-2-12) Display Video Generation Unit The display video generation unit 127 generates a display image of dangerous driving according to an instruction from the driving education terminal 250.

  For example, upon receiving an instruction “display dangerous driving” from the driving education terminal 250, the display video generation unit 127 selects “dangerous driving candidate” with the risk determination flag = 1 from the dangerous driving information DB 130, The frame range of the selected “dangerous driving candidate” is specified. When the danger determination flag = 0, it is determined that the driving is not dangerous.

  For example, “dangerous driving candidate number” of “dangerous driving candidate number” = 1 is dangerous determination flag = 1 for the object type = passenger car. Therefore, the display video generation unit 127 selects a record with “dangerous driving candidate number” being 1, and acquires the start frame ns1 and the end frame ne1 as the frame range. Next, the display video generation unit 127 reads the corresponding frame from the peripheral information DB 128 and generates a display video.

  In addition, it is assumed that the display video generation unit 127 receives a display instruction specifying a dangerous driving pattern from the driving education terminal 250. The display image generation unit 127 selects the “dangerous driving candidate” with the dangerous determination flag = 1 from the dangerous driving information DB 130 and the designated dangerous driving pattern, and displays the corresponding frame with reference to the peripheral information DB 128. Generate video.

  Further, it is assumed that the display video generation unit 127 receives an instruction “display unrecognized dangerous driving” from the driving education terminal 250. The display video generation unit 127 selects the “dangerous driving candidate” with the danger determination flag = 1 and the danger recognition = “none” from the dangerous driving information DB 130, and displays the corresponding frame with reference to the surrounding information DB 128. Generate video.

  The display video generation unit 127 may generate a display image of the dangerous driving not only including the data in the peripheral information DB 128 but also including data such as the travel / operation information DB 131 and the position information DB 132, for example. As a result, it is possible to grasp not only the dangerous driving image, but also at what position the dangerous driving occurred and what kind of running state and operating state at that time.

  The display video generation unit 127 outputs the display video generated in this way to the driving education terminal 250.

  The peripheral information in the peripheral information DB 128 is peripheral information around the entire 360 ° of the host vehicle, for example. Therefore, the display video generation unit 127 can generate a display image of dangerous driving based on the surrounding information of the entire periphery. Therefore, a viewer of dangerous driving can watch dangerous driving over the entire periphery of the host vehicle 300. For example, even if the cause of dangerous driving occurs in a blind spot that is difficult to visually recognize while driving the host vehicle 300, a viewer of dangerous driving can look back on such dangerous driving with a display image.

  In addition, the display video generation unit 127 can generate a display video based on the viewpoint of the driver of the host vehicle 300, but the display video is generated based on the viewpoint from the other vehicle 500 behind the host vehicle 300. It can also be generated. FIG. 16 is an explanatory diagram showing the relationship between the host vehicle and the other vehicle, and FIG. 17 is an example of a display image based on the viewpoint from the other vehicle behind the host vehicle.

  Referring to FIG. 16, other vehicles 500 a and 500 b are traveling diagonally forward and rearward with respect to the host vehicle 300. For example, the display image generation unit 127 calculates the data of the peripheral information DB 128 based on the viewpoint and the line-of-sight direction of the other vehicle 500a behind to generate a display image of dangerous driving. Thereby, the display image shown in FIG. 17 based on the viewpoint from the other vehicle 500a behind the host vehicle 300 is generated. Safe driving education can be effectively performed by objectively viewing the dangerous driving of the host vehicle 300 from the viewpoint of another vehicle. Note that data calculation processing based on a predetermined viewpoint and line-of-sight direction is performed using, for example, affine transformation.

(3-2-13) Various correspondence table DB
Various correspondence table DB134 has memorize | stored all the data which dangerous driving | running analysis apparatus 100 uses for a calculation and determination. One example will be described below.

  For example, the various correspondence table DB 134 stores a correspondence relationship between each pixel of each of the cameras 205a to 205d and each coordinate of the three-dimensional projection plane 400.

  The various correspondence table DB 134 stores pattern data that stores various characteristics of the object.

  The various correspondence tables DB 134 stores various reference values such as safe driving standards.

  The various correspondence tables DB 134 is a road map in which global coordinate position information on the map is associated with information indicating the situation where the vehicle 300 is located such as an intersection, a straight road, a three-lane road, and a T-junction. I remember information.

(3-3) Driving Education Terminal Functional units of the driving education terminal 250 in FIG. 4 include, for example, a transmission / reception unit 270 and a display control unit 271. The driving education terminal 250 receives a display instruction of a desired dangerous driving from the viewer via the mouse 256 and the keyboard 257. The transmission / reception unit 270 outputs a display instruction for a desired dangerous driving to the display video generation unit 127 of the dangerous driving analysis device 100. The transmission / reception unit 270 receives a display video based on a desired dangerous driving from the display video generation unit 127, and the display control unit 271 displays the display video on the display 255.

(4) Process Flow The process flow executed by the dangerous driving analysis apparatus 100 according to the first embodiment will be described below. First, the flow of the entire process will be described, and then the flow of each process constituting the overall process will be described.

(4-1) Overall Processing FIG. 18 is a flowchart illustrating an example of the flow of overall processing executed by the dangerous driving analysis apparatus according to the first embodiment. The entire process is executed for each frame.

  Steps S <b> 1 and S <b> 2: The dangerous driving analysis device 100 acquires and reads data from the information acquisition device 200 for each frame. Thereby, data is taken into each DB of the dangerous driving analysis device 100.

  Step S3: The dangerous driving analysis apparatus 100 performs an object extraction process in order to extract objects around the host vehicle 300.

  Step S4: The dangerous driving analysis apparatus 100 executes a relative information calculation process in order to calculate relative information for each object.

  Step S5: The dangerous driving analyzing apparatus 100 proceeds to step S6 when the processes of steps S1 to S3 are completed for all the frames. Otherwise, the process returns to step S1.

  Step S6: The dangerous driving analysis device 100 executes dangerous driving extraction processing in order to extract dangerous driving from the driving information.

  Step S7: The dangerous driving analysis apparatus 100 executes a recognition determination process to determine whether or not dangerous driving is recognized.

  Step S8: The dangerous driving analysis device 100 executes dangerous driving classification processing to classify dangerous driving.

  Below, each process which comprises the whole process is demonstrated.

(4-2) Object Extraction Processing FIG. 19 is a flowchart illustrating an example of the flow of object extraction processing according to the first embodiment.

  Step S3a: The video processing unit 121 projects the video data of each of the cameras 205a to 205d on the three-dimensional projection plane 400 for the current frame to generate a peripheral video.

  Steps S3b, S3c: The object extraction unit 122 extracts edges in the peripheral video generated by the video processing unit 121 (S3b), and detects the lane display lines 602a to 602d (S3c).

  Steps S3d, S3e: Next, the object extraction unit 122 detects the vanishing point D from the intersection of the lane display lines 602a to 602d (S3d), and determines the search range of the object based on the vanishing point D (S3e). ).

  Step S3f: The object extraction unit 122 extracts object candidates from the search range.

  Steps S3g, S3h: The object extraction unit 122 performs pattern matching on the object candidates (S3g), extracts the objects, and determines the object type (S3h).

  Step S3i: If an object is extracted, the process proceeds to step S3j. If an object is not extracted, the process proceeds to step S3m.

  Step S3j: The object extraction unit 122 confirms the identity with the object one frame before and assigns an object number. For example, the identity is confirmed as follows. The object extraction unit 122 compares the images of the previous frame and the current frame, and determines that objects with high similarity are the same object between frames. In addition, the object extraction unit 122 predicts the position in the current frame based on the speed, distance, traveling direction, and the like for the object detected one frame before. The object extraction unit 122 determines that the object existing at the predicted position in the current frame is the same as the object one frame before.

  Step S3k: The target object extraction unit 122 acquires the relative position Q of each target object with respect to the host vehicle 300.

  Step S31: The object extraction unit 122 sets the object flag to “1” because the object exists in the current frame.

  Step S3m: The object extraction unit 122 sets the object flag to “0” because there is no object in the current frame.

(4-3) Relative Information Calculation Processing FIG. 20 is a flowchart illustrating an example of the flow of relative information calculation processing according to the first embodiment.

  Steps S4a and S4b: The relative information calculation unit 123 acquires the first object number n in the current frame (S4a), and acquires the relative position Q of the corresponding object from the relative information DB 129 (S4b).

  Steps S4c, S4d: The relative information calculation unit 123 calculates the relative distance Ly in the Y direction based on the distance between the center point O of the host vehicle 300 and the relative position Q of the object (S4c), and the relative information in the X direction. The distance Lx is calculated (S4d).

  Step S4e: Next, the relative information calculation unit 123 acquires the relative distance Ly and the relative distance Lx of the object in the previous frame.

  Steps S4f and S4g: The relative information calculation unit 123 calculates the relative velocity Vy in the Y direction and the relative velocity Vx in the X direction based on the difference in the relative distance L between the current frame and the previous frame and the time between frames. To do.

  Steps S4h, S4i: The relative information calculation unit 123 calculates the TTCx in the X direction and the TTCy in the Y direction based on the relative distances Lx and Ly and the relative velocities Vx and Vy.

  Steps S4j, S4k: If the relative information calculation unit 123 has calculated the relative information for all the objects in the current frame, the process ends. Otherwise, the next object is selected and the process returns to step S4a. For example, the relative information calculation unit 123 calculates the next object number n = n + 1, and if there is an object with the next object number n = n + 1, the process starts from step S4a. When there is no target object with the next target number n = n + 1, but there are still other target objects, the relative information calculation unit 123 further increases the target number n = n + 1 until the next target object exists. Is added. In addition, the relative information calculation unit 123 ends the process when the target object currently focused on is the final target object in the current frame.

(4-4) Dangerous Driving Extraction Processing FIGS. 21A and 21B are flowcharts illustrating an example of the flow of dangerous driving extraction processing according to the first embodiment.

  Steps S6a and 6b: The dangerous driving extraction unit 124 reads the relative information DB 129 of the current frame i (i = i + 1).

  Step S6c: The dangerous driving extraction unit 124 determines whether or not the object flag of the current frame is “1”. If the object flag = 1, the process proceeds to step S6d. If the object flag = 0, the process proceeds to step S6q.

  Step S6d: The dangerous driving extracting unit 124 determines whether or not the object flag is “1” for the previous frame. If the object flag = 1, the process proceeds to step S6f. If the object flag = 0, the process proceeds to step S6e.

  Step S6e: Since the object flag = 0 in the previous frame and the object flag = 1 from the current frame, the dangerous driving extraction unit 124 sets the current frame as the start frame of the “object existence section”. .

  Step S6f: Since the object flag = 1 in the previous frame and the current frame, the dangerous driving extracting unit 124 sets the current frame as a provisional end frame of the “object existence section”. By these processes, “object existence section” in which frames with the object flag = 1 continue is extracted. Note that the end frame of the “object existence section” is updated until the next frame with the object flag = 1 appears, and when the next frame with the object flag = 0 appears, the end frame of the “object existence section” is determined. To do.

  Step S6g: The dangerous driving extracting unit 124 acquires the first object number n in the current frame, and pays attention to the object having the object number n.

  Step S6h: The dangerous driving extracting unit 124 determines whether or not the relative distance Lx of the target object is smaller than Lxs or whether the relative distance Ly of the target object is smaller than Lys in the current frame. judge. Here, the dangerous driving extraction unit 124 proceeds to step S6i if any of the above conditions is satisfied, and proceeds to step S6n otherwise.

  Step S6i: Further, the dangerous driving extraction unit 124 determines whether or not the relative distance Lx of the target object is smaller than Lxs or whether the relative distance Ly of the target object is smaller than Lys in the previous frame. Determine whether. If any one of the conditions is satisfied, the process proceeds to step S6k. Otherwise, the process proceeds to step S6j.

  Step S6j: Any of the above conditions is not satisfied in the previous frame, and any of the above conditions is satisfied from the current frame. That is, the distance between the host vehicle and the object is approaching from the current frame. Therefore, the dangerous driving extraction unit 124 sets the current frame as the start frame of the “approach zone”.

  Step S6k: Any of the above conditions is satisfied in the previous frame and the current frame. That is, the distance between the host vehicle and the object is close in both the front frame and the current frame. Therefore, the dangerous driving extraction unit 124 sets the current frame as a provisional end frame of the “approach section”. By these processes, an “approach section” having a short relative distance between the host vehicle 300 and the target object is extracted from the “target section”. Note that the provisional end frame of the “approach section” continues to be updated while the condition is satisfied in step S6i.

  Step S61: The dangerous driving extraction unit 124 refers to the relative information DB 129 and calculates the minimum values of TTCx and TTCy in the “approach section” of each object.

  Step S6m: The dangerous driving extraction unit 124 refers to the relative information DB 129, and calculates the maximum value of the rate of change of TTCx and TTCy in the “approach section” of each object.

  Step S6n: The dangerous driving extraction unit 124 performs the processes of steps S6h to S6m for all objects in the current frame. Therefore, the dangerous driving extraction unit 124 proceeds to step S6o when the object number n is not the last object number, and proceeds to step 6q when the object number n is the last object number.

  Steps S6o, S6p: The dangerous driving extraction unit 124 calculates the next object number n = n + 1, and if there is an object with the next object number n = n + 1, the process starts from step S6h. When the next object number n = n + 1 does not exist, the dangerous driving extraction unit 124 further adds the object number n = n + 1 until the object exists.

  Step S6q: The dangerous driving extraction unit 124 performs the processing of Steps S6b to S6p for all the frames, and thus returns to Step S6b when the current frame is not the last frame.

  Step S6r: The “object existence section” and the “approach section” are extracted by the processing described above. The dangerous driving extraction unit 124 extracts the “dangerous driving candidate section” based on the minimum value and the maximum value of the “approaching section” in each “target existence section”. For example, it is assumed that the relative distance between the host vehicle 300 and a plurality of objects is short, and one “object existence section” includes a plurality of “approach sections”. The minimum value of the “approach section” is a start frame in which the “approach section” starts first among a plurality of “approach sections”. The maximum value of the “approach section” is an end frame in which the “approach section” ends at the end of a plurality of “approach sections”. The “dangerous driving candidate section” is defined by these start frame and end frame.

  Step S6s: The dangerous driving extraction unit 124 uses the frame range of the “dangerous driving candidate section”, the object number, the frame range of the “approaching section”, the TTC minimum value, the TTC change rate (DTTC) maximum value, and the like. It is stored in the driving information DB 130 and a “dangerous driving candidate number” is assigned to each “dangerous driving candidate section”.

  Step S6t: Next, the dangerous driving extraction unit 124 refers to the dangerous driving information DB 130 and reads data of “dangerous driving candidate number” = k.

  Step S6u: The dangerous driving extraction unit 124 sets a danger determination flag for the relevant “dangerous driving candidate section” based on the minimum values of TTCx and TTCy and the maximum change rate of TTCx and TTCy of each target object. .

  Step S6v: If “dangerous driving candidate number” = k is not the last number, the process proceeds to step S6W, and if it is the last number, the process is terminated.

  Step S6w: The dangerous driving extraction unit 124 calculates the next “dangerous driving candidate number” = k + 1, and starts the process from step S6t.

(4-5) Recognition determination extraction process FIG. 22 is a flowchart illustrating an example of the flow of the recognition determination extraction process according to the first embodiment.

  Steps S7a, S7b: The recognition determination unit 125 refers to the dangerous driving information DB 130 for each “dangerous driving candidate”.

  Step S7c: Further, the recognition determination unit 125 reads out the start frame and the end frame of the “approach section” of the target with the risk determination flag = 1 for the “dangerous driving candidate” of interest. Next, the recognition determination unit 125 reads travel information and operation information in the travel / operation information DB 131 for each frame in the “approach section”.

  Step S7d: Next, the recognition determination unit 125 determines whether or not the driver recognizes dangerous driving based on the traveling information and the operation information.

  Step S7e: If “dangerous driving candidate number” = k is not the last number, the process proceeds to step S7b, and if it is the last number, the process is terminated.

(4-6) Dangerous Driving Classification Processing FIG. 23 is a flowchart illustrating an example of the flow of dangerous driving classification processing according to the first embodiment.

  Steps S8a, S8b: The dangerous driving classification unit 126 performs dangerous driving classification processing for each “dangerous driving candidate” (“dangerous driving candidate number” = k + 1).

  Step S8c: The dangerous driving classification unit 126 reads the position information of the host vehicle 300 in the position information DB 132 for the frame range for the “dangerous driving candidate” of interest. Further, the dangerous driving classification unit 126 reads road map information in the various correspondence table DB 134, and specifies the position of the host vehicle 300 on the map based on the position information.

  Step S8d: Next, the dangerous driving classification unit 126 reads the driving information and operation information of the host vehicle from the driving / operation information DB 131 for the frame range for the “dangerous driving candidate” of interest, and the driving state of the host vehicle 300 Judging.

  Step S8e: The dangerous driving classification unit 126 reads out the object type of the danger determination flag = 1 for the “dangerous driving candidate” of interest.

  Step S8f: The dangerous driving classification unit 126 identifies the dangerous driving pattern of each “dangerous driving candidate” based on the object type, the position on the map, and the driving state.

  Step S8g: If “dangerous driving candidate number” = k is not the last number, the process proceeds to step S8b, and if it is the last number, the process is terminated.

(5) Effects The dangerous driving analysis apparatus 100 extracts dangerous driving from driving information of the host vehicle 300 based on an objective safe driving standard. Therefore, not only dangerous driving recognized by the driver of the host vehicle 300 but also dangerous driving not recognized by the driver of the host vehicle 300 can be extracted.

  Further, if the dangerous driving analysis apparatus 100 is used, it is possible to extract dangerous driving efficiently, objectively, and quickly without depending on human hands. By letting the driver view the dangerous driving image and the like extracted in this way, it is possible to effectively learn under what circumstances dangerous driving occurs and to efficiently perform safe driving education.

  Further, the dangerous driving analysis apparatus 100 determines whether or not the driver recognizes dangerous driving. Therefore, as shown in FIG. 11, not only “dangerous driving candidates” that do not satisfy the safe driving standard can be extracted, but also dangerous driving that the driver cannot recognize is extracted from “dangerous driving candidates”. Can do. Safe driving education can be effectively performed by having the driver watch dangerous driving that cannot be recognized and showing the driver what kind of dangerous driving has not been recognized.

  Further, the dangerous driving analysis apparatus 100 classifies the extracted “dangerous driving candidates” according to a predetermined classification item. Therefore, desired data can be immediately searched from “dangerous driving candidates”. Further, by classifying “dangerous driving candidates” according to the above classification items and investigating the tendency, it is possible to analyze under what circumstances the driver is likely to fall into dangerous driving.

(6) Modification (6-1) Modification 1
In the first embodiment described above, the dangerous driving extraction unit 124 extracts the “object existence section” from the peripheral information, and further extracts the “approach section” of each object from the “object existence section”. Further, the dangerous driving extraction unit 124 defines a “dangerous driving candidate section” based on the “approaching section” of each object. Next, the dangerous driving extraction unit 124 sets a danger determination flag based on the minimum value of TTCx and TTCy and / or the maximum value of the change rate of TTCx and TTCy in each “approach section” of each object. . Then, the dangerous driving extraction unit 124 extracts the “dangerous driving candidate section” having the danger determination flag “1” as the dangerous driving section.

  However, various methods of extracting the dangerous driving section are conceivable. For example, the extraction can be performed by the following method.

  Here, the safe driving criteria include, for example, the criteria (iv) and (v) in addition to the criteria (i) to (iii) described above.

  (I) Relative distance reference values Lxs and Lys.

  (Ii) TTC reference values TTCxs and TTCys.

  (Iii) TTC change rate reference values DTTCxs and DTTCys.

  (Iv) Acceleration reference values αxs and αys (αs: α standard) for the acceleration α.

  (V) Steering angle change rate reference value (θsDs: θs Deviation rate standard) with respect to the steering angle θs change rate per predetermined unit time.

(A) Example of Extraction Method Like the first embodiment, the dangerous driving extraction unit 124 first extracts the “approach section” of each object from the “object-containing section” based on the criterion (i), Based on the “approaching section”, the “dangerous driving candidate section” is defined. After that, the dangerous driving extraction unit 124 extracts “dangerous driving candidate sections” that satisfy at least one of the criteria (ii) to (v) from the “dangerous driving candidate sections” as dangerous driving sections.

  For example, when using the reference (iv), the dangerous driving extraction unit 124 calculates the maximum values of the accelerations αx and αy of each object in each “approach section” of each object included in the “dangerous driving candidate section”. To do. For example, it is assumed that the “dangerous driving candidate section” includes “approach section 1” of the first object and “approach section 2” of the second object. The dangerous driving extraction unit 124 calculates the maximum values of the accelerations α1x and α1y of the first object in the “approach zone 1”. Similarly, the dangerous driving extraction unit 124 calculates the maximum values of the accelerations α2x and α2y of the second object in the “approach zone 2”. Next, the dangerous driving extraction unit 124 determines whether the maximum value of the acceleration α1x and / or α2x is larger than αxs and / or whether the maximum value of the accelerations α1y and α2y is larger than αys. When the condition is satisfied, the dangerous driving extraction unit 124 extracts “dangerous driving candidate sections” including “approaching section 1” and “approaching section 2” as sections of the dangerous driving.

  Further, when the reference (v) is used, the dangerous driving extraction unit 124 calculates the maximum value of the steering angle change rate of each target object in each “approach section” of each target object included in the “dangerous driving candidate section”. To do. Next, when the maximum value of the steering angle change rate in each “approaching section” of each object is larger than θsDs, the dangerous driving extracting unit 124 selects a “dangerous driving candidate section” including the “approaching section”. Extract as dangerous driving section.

(B) Example of Extraction Method FIG. 24 is an explanatory diagram illustrating another example of the dangerous driving extraction process. In FIG. 24, first, each “approach section” of each object is extracted from the “object-presented section” based on the criterion (i), and “dangerous driving candidate section” is defined based on the “approach section”. Thereafter, further dangerous driving sections are extracted from the “dangerous driving candidate sections” based on the determinations based on the criteria (ii) to (v). Specifically, the following processing is performed.

  First, the dangerous driving extracting unit 124 refers to the relative information DB 129 in the same manner as described above, and extracts the frame section in which the target object exists around the host vehicle 300 on the time axis of the peripheral information based on the target flag = 1. To do. As shown in FIG. 24, “object existence section” defined by the start frame A and the end frame A is extracted on the time axis of the peripheral information.

  Next, similarly to the above, the dangerous driving extraction unit 124, based on the above-mentioned criterion (i), sets the “approach section” whose relative distance Lx is smaller than Lxs or whose relative distance Ly is smaller than Lys as “target”. Extract from “Property section”. Further, the dangerous driving extraction unit 124 defines a “dangerous driving candidate section” based on the “approaching section”. Here, in FIG. 24, there is only one object in the “object existence section”, and the “approach section” of the object is defined by the start frame B and the end frame B. Therefore, the “dangerous driving candidate section” is defined by the start frame B and the end frame B.

  Further, the dangerous driving extraction unit 124 performs determination based on at least one of the above-described standards (ii) to (v) for each object existing in each frame in the “dangerous driving candidate section”. For example, as illustrated in FIG. 24, the dangerous driving extraction unit 124 may have a condition that TTCx is smaller than TTCxs and / or TTCy is smaller than TTCys for each object in each frame in the “dangerous driving candidate section”. It is determined whether or not the condition of “small” is satisfied.

  The dangerous driving extraction unit 124 extracts a section determined to satisfy at least one of the conditions as a dangerous driving section, and sets a danger determination flag to “1” (danger determination flag = 1). In the case of FIG. 24, the dangerous driving section is defined by a start frame C and an end frame C. On the other hand, if the dangerous driving extraction unit 124 determines that none of the conditions is satisfied, the dangerous driving extracting unit 124 determines that the “object existence section” is not a dangerous driving section, and sets the risk determination flag to “0”. (Danger determination flag = 0).

(C) Example of Extraction Method FIG. 25 is an explanatory diagram illustrating another example of the dangerous driving extraction process.

  FIG. 25 is different from FIG. 24 in the order of determination. That is, in the “object existence section”, the section is first extracted by the determination based on the criteria (ii) to (v), and the “approach section” is extracted from the extracted section by the determination based on the reference (i). The This “approach section” is a dangerous driving section. Specifically, the following processing is performed.

  First, the dangerous driving extraction unit 124 refers to the relative information DB 129 as described above, and extracts the “target existence section” defined by the start frame A and the end frame A.

  Next, the dangerous driving extraction unit 124 makes a determination based on at least one of the above-mentioned criteria (ii) to (v) for each object existing in each frame in the “object-containing section”. Thereby, the dangerous driving extraction unit 124 extracts a section defined by the start frame C and the end frame C as a section determined to satisfy at least one of the criteria. In this example, this section can be a “dangerous driving candidate section”.

  Next, the dangerous driving extraction unit 124 performs a determination based on the criterion (i) for each object existing in each frame in the “dangerous driving candidate section”. That is, the dangerous driving extraction unit 124 determines whether the relative distance Lx of each object is smaller than Lxs or whether the relative distance Ly is smaller than Lys. If it is determined that the condition is satisfied, the dangerous driving extraction unit 124 extracts the “approach section” defined by the start frame B and the end frame B as a dangerous driving section, and sets the risk determination flag to “1”. Set (danger determination flag = 1). On the other hand, when it is determined that the condition is not satisfied, the dangerous driving extraction unit 124 determines that the “object existence section” is not a dangerous driving section, and sets the risk determination flag to “0” ( Risk determination flag = 0).

(D) An example of extraction method In (a) to (d) of the first embodiment and the first modification, in addition to the determination based on the reference (i), the reference (ii) to (v) A dangerous driving section is extracted based on the determination. However, the dangerous driving extraction unit 124 may extract a dangerous driving section by determination based on only one of the above criteria (i) to (v). Further, the dangerous driving extraction unit 124 may extract a dangerous driving section based on a combination of two or more of the criteria (i) to (v).

  When the relative distances Lx and Ly are smaller than Lxs and Lys, the distance between the host vehicle 300 and the target object is short, and the risk of the host vehicle 300 colliding with the target object is high. In addition, when TTCx and TTCy are smaller than TTCxs and TTCys, the distance between the host vehicle 300 and the target object is short, and there is a high risk that the host vehicle 300 will eventually collide with the target object. Further, when the rate of change of TTCx and TTCy is larger than DTTCxs and DTTCys, the distance between the host vehicle 300 and the object is rapidly approaching, and the risk of collision is high. When the acceleration of the host vehicle 300 is large, the host vehicle 300 is rapidly accelerating or decelerating. When the rate of change of the steering angle is large, the host vehicle 300 is meandering. It can be determined that the host vehicle is not traveling safely. Therefore, when a section satisfying such a condition is extracted as a section for dangerous driving, it is possible to effectively learn under what circumstances dangerous driving occurs and to perform safe driving education efficiently.

  The safe driving standard is not limited to the above, and any standard that can extract dangerous driving from driving information can be included in the safe driving standard.

(6-2) Modification 2
In the above embodiment example, the recognition determination unit 125 determines whether or not the driver recognizes dangerous driving based on the traveling information and operation information of the host vehicle 300. However, the recognition determination unit 125 may determine whether or not dangerous driving is recognized based on the driver's line of sight. Hereinafter, a configuration different from that of the first embodiment will be described.

(6-2-1) Hardware Configuration FIG. 26 is an example of a block diagram showing a connection relationship and hardware configuration of each device according to Modification 2. As shown in the hardware configuration of FIG. 26, the information acquisition apparatus 200 further includes a line-of-sight detection device 215 as compared with the first embodiment. The line-of-sight detection device 215 detects line-of-sight information such as the driver's face, eyeball, and iris.

  27 and 28 are explanatory diagrams showing the attachment position of the line-of-sight detection device. The line-of-sight detection device 215 includes an imaging device such as a CCD camera, a CMOS camera, or an infrared camera that can acquire driver's line-of-sight information.

  The line-of-sight detection device 215 is provided on the dashboard 301 of the vehicle 300, for example, as shown in FIGS. At this time, the line-of-sight detection device 215 can detect the face and eyes of the driver from the front, and can capture the face and eyes without being blocked by the handle 302, for example, a dash near the handle 302. It is mounted on the board 301 at a predetermined angle. However, as long as the driver's face and eyes can be detected, the attachment position and the attachment angle are not limited.

  Note that the characteristics of the line-of-sight detection device such as the mounting position and the mounting angle of the line-of-sight detection device 215 are corrected, so-called calibrated, so as to be compatible with the spatial coordinate system with the center point O of the vehicle 300 as the origin.

  The line-of-sight detection device 215 captures, for example, 30 video frames per second, and the captured video data is stored in the RAM 203 via the input / output device I / F 204.

  The line of sight 150 can be detected based on the driver's face, eyeball, iris, and other images detected by the line-of-sight detection device 215. When the driver's line of sight 150 is detected, the direction of the line of sight 150 indicates which direction the driver is viewing. For example, if the direction of the line of sight 150 is forward, it can be estimated that the driver has visually recognized the front. Further, if the direction of the line of sight 150 is toward the mirror 303, it can be assumed that the rear side and the rear side of the vehicle 300 are visually recognized via the mirror 303.

  As shown in FIG. 28, the mirror 303 provided in the vehicle 300 includes door mirrors 303L and 303R provided near the left and right doors of the vehicle 300, a rearview mirror 303B provided in the vehicle 300, and a hood of the vehicle 300. And a fender mirror. FIG. 29 is an explanatory diagram illustrating an example of a region that can be confirmed by a mirror. The driver of the vehicle 300 can visually recognize the left mirror region 304L, the right mirror region 304R, and the rearview mirror region 304B with the left door mirror 303L, the right door mirror 303R, and the rearview mirror 303B, respectively.

(6-2-2) Outline of Processing Whether the driver of the host vehicle 300 has visually recognized an object that is a target of dangerous driving based on the line of sight 150 acquired by the line-of-sight detection device 215. Judge whether or not. The outline of the process performed by the dangerous driving analysis apparatus 100 will be described with reference to FIG.

  FIG. 30 is a perspective view showing the relationship between the host vehicle and another vehicle. The other vehicle 500 is traveling on the lane 601 adjacent to the left with respect to the lane 600 on which the host vehicle 300 travels. Further, since the line-of-sight vector 150a of the driver of the host vehicle 300 is directed toward the other vehicle 500, the dangerous driving analysis apparatus 100 can determine that the driver of the host vehicle 300 is viewing the other vehicle 500. Specifically, for example, the dangerous driving analysis apparatus 100 sets a line-of-sight space 151 centered on the line of sight 150 of the driver of the host vehicle 300, and at least one area of the other vehicle 500 that is the target object in the line-of-sight space 151. It is judged whether it is visually recognizing based on whether or not is included. Although more specifically described later, the dangerous driving analysis apparatus 100 makes a determination based on an angle Δθ formed by the line-of-sight vector 150a and the object vector 160a from the host vehicle 300 to the other vehicle 500. Here, the line-of-sight space 151 refers to a space formed by a set of line-of-sight space lines 151a whose starting point is the line-of-sight origin P and whose angle θa formed with the line-of-sight vector 150a is equal to or smaller than a predetermined threshold θth.

(6-2-3) Functional Configuration Next, the functional configuration of the dangerous driving analysis apparatus 100 will be described.

  FIG. 31 is an example of a block diagram illustrating a functional configuration of each device according to the second modification. In addition to the functional configuration of the first embodiment, a line-of-sight detection unit 226 of the information acquisition device 200, a line-of-sight data DB 135 of the dangerous driving analysis device 100, and a line-of-sight determination unit 136 are provided. Each functional configuration will be described below. Configurations other than those described below are the same as those in the first embodiment.

(A) Gaze Detection Unit of Information Acquisition Device The gaze detection unit 226 of the information acquisition device 200 detects the gaze origin P and the gaze 150 based on the driver's face, eyeball, iris, and other images detected by the gaze detection device 215. A line-of-sight vector 150a indicating the direction is calculated.

  FIG. 32 is an explanatory diagram showing an example of a method of calculating the line-of-sight origin P and the line-of-sight vector. For example, the line-of-sight detection unit 226 calculates facial feature points based on images such as faces, eyeballs, and irises, and compares the feature points with driver feature values stored in advance. Next, the line-of-sight detection unit 226 extracts the orientation of the face based on the comparison result and the image of the face, eyeball, iris, etc., and the center of the left eyeball 152L and the right eyeball 152R shown in FIG. The position is detected as the line-of-sight origin P. Furthermore, the line-of-sight detection unit 226 calculates the center position of the iris 153a, that is, the center position of the pupil 153b. Finally, the line-of-sight detection unit 226 calculates the line-of-sight vector 150a based on the line-of-sight origin P and the center position of the pupil 153b. Since the driver can change the head forward / backward / left / right and up / down, the position of the line-of-sight origin P with respect to the center point O of the spatial coordinate system is changed according to the position and orientation of the head.

  The line-of-sight vector 150a can be defined by coordinates in a spatial coordinate system with an arbitrary center point O of the vehicle 300 as the origin. In addition, the line-of-sight vector 150a is an angle formed by a pitch angle 156a that is an angle formed between the line-of-sight vector 150a and the XY plane and a line-of-sight vector 150a and the YZ plane, as shown in FIGS. And azimuth angle 156b.

  The line-of-sight detection unit 226 stores the line-of-sight origin P and the line-of-sight vector 150a in the acquisition data DB 225.

(B) Line-of-sight data DB of the dangerous driving analysis device
The line-of-sight data DB 135 acquires the line-of-sight origin P and the line-of-sight vector 150a of the driver of the host vehicle from the information acquisition apparatus 200 and stores them. The line-of-sight data DB 135 also stores the presence / absence of visual recognition of the mirror 303, the mirror line-of-sight origin R, the mirror line-of-sight vector 155a, and the like determined by the line-of-sight determination unit 136.

  The mirror line-of-sight origin R refers to the coordinates of the intersection of the line-of-sight vector 150a and the mirror surface area of the mirror 303. Further, the line of sight 150 from the driver is reflected by the mirror 303 to become a mirror line of sight 155. The mirror line-of-sight vector 155a is a vector indicating the direction of the mirror line-of-sight 155. The mirror line-of-sight origin R and the mirror line-of-sight vector 155a are defined by a spatial coordinate system centered on an arbitrary center point O of the host vehicle 300.

  FIG. 33 is an example of line-of-sight data in the line-of-sight data DB. The line-of-sight data DB 135 stores a frame number, a line-of-sight origin P, a line-of-sight vector 150a, mirror viewing Yes / NO, a mirror line-of-sight origin R, and a mirror line-of-sight vector 155a for each frame. For example, the line-of-sight data DB 135 stores “NO” indicating that the line-of-sight origin P (Xv0, Yv0, Zv0), the line-of-sight vector Visual_1, and the mirror 305 are not visually recognized in the record of frame number = 1. In the line-of-sight data DB 135, in the record of frame number = 6, the line-of-sight origin P (Xv1, Yv1, Zv1), the line-of-sight vector Visual_6, “YES” indicating that the mirror 303 is visually recognized, and the mirror line-of-sight origin R (Xm1) , Ym1, Zm1), and the mirror line-of-sight vector Vmirror_6.

(D) Relative information calculation unit of dangerous driving analysis device, relative information DB
The relative information calculation unit 123 further calculates an object vector in addition to the relative distance L, the relative speed V, and TTC. The object vector is a vector indicating the direction from the line-of-sight origin P or the mirror line-of-sight origin R to the object.

  The relative information calculation unit 123 acquires the line-of-sight origin P or the mirror line-of-sight origin R from the line-of-sight data DB 135. Specifically, the relative information calculation unit 123 acquires the line-of-sight origin P if the result of mirror viewing in the line-of-sight data DB 135 is “NO”, and determines the mirror line-of-sight origin R if the result of mirror viewing is “YES”. get. In addition, the relative information calculation unit 123 acquires the relative position Q of the target object for which the target object vector is calculated from the relative information DB 129. Next, the relative information calculation unit 123 calculates an object vector 160a indicating the direction 160 from the line-of-sight origin P or the mirror line-of-sight origin R to the object.

  For the object with frame number = 1 and object number = 2, the object vector 160a is calculated as follows. The relative information calculation unit 123 acquires the line-of-sight origin P (Xv0, Yv0, Zv0) because the result of the mirror viewing of the frame number = 1 is “NO” in the line-of-sight data DB 135. Further, the relative information calculation unit 123 acquires the relative position Q (X_21, Y_21, Z_21) from the relative information DB 129. Next, the relative information calculation unit 123 calculates the object vector Vobject_21 based on the line-of-sight origin P and the relative position Q.

  Further, the object vector 160a is calculated as follows for the object of frame number = 6 and object number = 2. The relative information calculation unit 123 acquires the mirror line-of-sight origin R (Xm1, Ym1, Zm1) because the result of the mirror viewing of the frame number = 6 is “YES” in the line-of-sight data DB 135. Here, it is assumed that the relative position Q of the object is (X_26, Y_26, Z_26). The relative information calculation unit 123 calculates the object vector Vobject_26 based on the mirror line-of-sight origin R and the relative position Q.

  The object vector 160a may be defined by coordinates in the space coordinate system, or may be defined by an angle formed by the object vector 160a, the XY plane, and the YZ plane.

  The relative information calculation unit 123 stores the calculation result in the relative information DB 129.

  FIG. 34 is an example of relative information in the relative information DB. The relative information DB 129 stores, for example, a frame number, an object number, an object type, a relative position Q, a relative distance L, a relative speed V, an object vector 160a, and an angle formed later.

(E) Line of sight determination unit of dangerous driving analysis device The line of sight determination unit 136 includes a line of sight vector 150a or a mirror line of sight vector 155a, and an object vector 160a so that the recognition determination unit 125 can determine whether or not dangerous driving is recognized. The angle formed by, is calculated.

  First, a method for calculating the mirror line-of-sight vector will be described.

  The line-of-sight determination unit 136 determines whether or not the line-of-sight vector 150a is on the mirror 303 based on the line-of-sight origin P and the line-of-sight vector 150a of the line-of-sight data DB 135. Here, the line-of-sight determination unit 136 grasps the area of each mirror surface such as the left and right door mirrors 303L and 303R and the rearview mirror 303B provided in the host vehicle 300 by, for example, acquiring from the information acquisition device 200. And The region of the mirror surface is a region of the reflecting surface that reflects incident light, and is defined by a set of coordinates based on a spatial coordinate system, for example. The line-of-sight determination unit 136 determines whether or not the line-of-sight vector 150a is on the mirror 303 based on whether or not the line-of-sight vector 150a extending from the line-of-sight origin P intersects with the region of the mirror surface.

  When the line-of-sight vector 150a and the mirror surface region intersect, the line-of-sight determination unit 136 sets the intersection point as the mirror line-of-sight origin R. Further, the line-of-sight determination unit 136 obtains the mirror line-of-sight vector 155a by reflecting the line-of-sight vector 150a from the mirror line-of-sight origin R with the mirror 303. The line-of-sight determination unit 136 stores the presence / absence of viewing of the mirror 303, the mirror line-of-sight origin R, and the mirror line-of-sight vector 155a in the line-of-sight data DB 135 as shown in FIG. When the position is changed by finely adjusting each mirror 303 or the like, the line-of-sight determination unit 136 acquires the area of the mirror surface after the change from the information acquisition apparatus 200, for example.

  Next, the line-of-sight determination unit 136 calculates an angle formed by the line-of-sight vector 150a or the mirror line-of-sight vector 155a and the object vector 160a. The calculation method will be described below with reference to FIGS. 35 and 36.

  FIG. 35 is an explanatory diagram showing an example of a method for calculating the angle Δθ formed by the line-of-sight vector and the object vector. The host vehicle 300 is traveling on the lane 600, and the other vehicle 500 that is the object is traveling on the lane 601 adjacent to the left of the lane 600, and is located on the left front side with respect to the host vehicle 300. Yes. The line-of-sight determination unit 136 reads the line-of-sight origin P and the line-of-sight vector 150a when referring to the line-of-sight data DB 135 and determining that the mirror view is NO in the determination target frame. In addition, the line-of-sight determination unit 136 reads the object vector 160a for the determination target frame and the determination target object with reference to the relative information DB 129. Next, the line-of-sight determination unit 136 calculates an angle Δθ formed by the driver's line-of-sight vector 150a starting from the line-of-sight origin P and the object vector 160a starting from the line-of-sight origin P. Here, the line-of-sight determination unit 136 calculates an angle ΔθH formed by the line-of-sight vector 150a and the object vector 160a in the XY plane, as shown in FIG. Further, the line-of-sight determination unit 136 calculates an angle ΔθV formed by the line-of-sight vector 150a and the object vector 160a in the YZ plane, as shown in FIG.

  FIG. 36 is an explanatory diagram showing an example of a method for calculating the angle Δθ formed by the mirror line-of-sight vector and the object vector. The own vehicle 300 is traveling on the lane 600, and the other vehicle 500 that is the object is traveling on the lane 603 adjacent to the right of the lane 600, and is located on the right rear side with respect to the own vehicle 300. Yes. The line-of-sight determination unit 136 refers to the line-of-sight data DB 135 and reads the mirror line-of-sight origin R and the mirror line-of-sight vector 155a when determining that the mirror viewing is YES in the determination target frame. For example, it is assumed that the driver is viewing the right door mirror 303R of the host vehicle 300. Therefore, as shown in FIG. 36, the line-of-sight vector 150a starting from the line-of-sight origin P is reflected by the mirror 303R and becomes the mirror line-of-sight vector 155a starting from the mirror line-of-sight origin R. In addition, the line-of-sight determination unit 136 reads the object vector 160a for the determination target frame and the determination target object with reference to the relative information DB 129. The line-of-sight determination unit 136 calculates an angle Δθ formed by the mirror line-of-sight vector 155a starting from the mirror line-of-sight origin R and the object vector 160a starting from the mirror line-of-sight origin R. That is, the line-of-sight determination unit 136, as shown in FIGS. 6A and 6B, shows the angle ΔθH formed by the mirror line-of-sight vector 155a and the object vector 160a in the XY plane, and the mirror line-of-sight in the YZ plane. An angle ΔθV formed by the vector 155a and the object vector 160a is calculated.

  The line-of-sight determination unit 136 stores the formed angles ΔθH and ΔθV in the relative information DB 129 as determination results.

(F) Recognition determination unit of dangerous driving analysis device The recognition determination unit 125 determines whether or not the driver recognizes that driving of the host vehicle 300 is dangerous driving. Here, the recognition determination unit 125 determines whether or not the driver of the host vehicle 300 recognizes an object that is a target of dangerous driving.

  For example, the recognition determination unit 125 determines the presence or absence of recognition based on the positional relationship between the line-of-sight space 151 illustrated in FIG. 30 and the other vehicle 500 that is the target. For example, the recognition determination unit 125 determines whether or not the object is located in the center of the line-of-sight space 151. If the object is located in the center of the line-of-sight space, the line of sight 150 and the object intersect. It is determined that the object is recognized. In addition, the recognition determination unit 125 determines what percentage of the object occupies the line-of-sight space 151, and the presence or absence of the object recognition according to the degree to which the object area occupies the line-of-sight space 151. May be determined. The determination method will be described in more detail.

  First, similarly to the first embodiment, the recognition determination unit 125 refers to the dangerous driving information DB 130 illustrated in FIG. 11, and for each “dangerous driving candidate”, the “approach section” of the object with the risk determination flag = 1. Is read. For example, in the dangerous driving information DB 130 of FIG. 11, the recognition determining unit 125 sets the start frame ns <b> 1 </ b> _ <b> 1 of the “approach zone” in the record of “dangerous driving candidate number” = 1, object number = 1, and dangerous determination flag = 1. Read end frame ne1_1.

  Next, the recognition determination unit 125 refers to the relative information DB 129 of FIG. 34 for each frame in the “approach section” and determines whether or not the driver recognizes dangerous driving. First, the recognition determination unit 125 reads the angles ΔθH and ΔθV formed for each frame in the “approach section” of the target object.

  Next, the recognition determination unit 125 compares predetermined threshold values θHth and θVth for determining whether or not dangerous driving is recognized with the formed angles ΔθH and ΔθV, and the driver visually recognizes the object. Judge whether or not. The threshold θth includes a predetermined threshold θHth for determining an angle ΔθH formed in the XY plane and a predetermined threshold θVth for determining an angle ΔθV formed in the YZ plane. Therefore, the recognition determination unit 125 determines whether or not the formed angle ΔθH is equal to or smaller than the predetermined threshold θHth, and further determines whether or not the formed angle ΔθV is equal to or smaller than the predetermined threshold θVth. When the formed angle ΔθH is equal to or smaller than the predetermined threshold value θHth and the formed angle ΔθV is equal to or smaller than the predetermined threshold value θVth, the recognition determining unit 125 determines that the driver of the host vehicle 300 has visually recognized the object. It is determined that he was aware of dangerous driving. On the other hand, when the formed angle ΔθH is larger than the predetermined threshold θHth or the formed angle ΔθV is larger than the predetermined threshold θVth, the recognition determining unit 125 determines that the driver has not visually recognized the object, and is dangerous. It is determined that driving was not recognized.

  In the above description, the recognition determination unit 125 determines the presence / absence of recognition based on both the formed angles ΔθH and ΔθV, but for example, the presence / absence of recognition based on the angle formed by one of the angles ΔθH and ΔθV. Can also be determined.

  In addition, the recognition determination unit 125 may calculate the visual recognition time for the object and determine whether or not the object is recognized based on the calculated visual recognition time. For example, the recognition determination unit 125 determines that the object is recognized when the viewing time is equal to or greater than a predetermined value. On the other hand, the recognition determination unit 125 determines that the object is not recognized when the viewing time is less than the predetermined value.

  The recognition determination unit 125 stores the determination result in the dangerous driving information DB as in FIG. 11 described above.

  In addition, although the recognition determination part 125 determines only the presence or absence of recognition of dangerous driving, you may determine the recognition degree which shows how much the driver has recognized dangerous driving. Thereby, dangerous driving according to the degree of recognition can be extracted. For example, the recognition determination unit 125 may diagnose the degree of recognition according to the size of the angle formed by the line-of-sight vector 150a or the mirror line-of-sight vector 155a and the object vector 160a. For example, the degree of recognition is determined higher as the angle formed is smaller. Moreover, the recognition determination part 125 may diagnose a recognition degree according to the length of visual recognition time. For example, the recognition degree is determined to be higher as the viewing time is longer.

(6-3) Modification 3
In the above embodiment example, the dangerous driving analysis apparatus 100 extracts one or more objects existing around the host vehicle 300 from the surrounding video generated by the video processing unit 121. However, the dangerous driving analysis apparatus 100 may detect an object using an obstacle detection sensor attached to the host vehicle 300, for example. The obstacle detection sensor detects a distance from the obstacle, and can be constituted by an optical sensor, an ultrasonic sensor, or the like.

  FIG. 37 is an example of a vehicle provided with an obstacle detection sensor. For example, the obstacle detection sensors 216a to 216h are embedded at three locations on the front bumper of the host vehicle 300, both side surfaces of the vehicle body, three locations on the rear bumper, and the like. The object extraction unit 122 of the dangerous driving analysis apparatus 100 detects an object around the host vehicle 300 based on the sensor signals detected by these obstacle detection sensors, and determines the relative position Q of the object to the host vehicle 300 and the like. get. Further, the relative information calculation unit 123 calculates the relative distance L, the relative speed V, and the like based on the sensor signal.

  The dangerous driving analysis apparatus 100 can also detect a target and calculate the relative distance L and the relative speed V using a stereo camera with a twin lens.

  Further, the dangerous driving analysis apparatus 100 may detect the object based on communication between the host vehicle 300 and the object. Examples of the communication include inter-vehicle communication, which is communication between vehicles.

<Second Embodiment>
FIG. 38 is an example of a block diagram illustrating a functional configuration of each device according to the second embodiment.

  The dangerous driving analysis device 100 only needs to be able to extract dangerous driving, and the dangerous driving classification unit 126 and the like of the first embodiment are not essential components. The dangerous driving analysis apparatus 100 according to the second embodiment does not include the dangerous driving classification unit 126 and the dangerous driving pattern DB 133 according to the first embodiment. Other configurations are the same as those of the first embodiment, and modifications of the first embodiment can also be applied to the second embodiment.

<Third Embodiment>
The dangerous driving analysis device 100 according to the first embodiment obtains peripheral information around the host vehicle, travel information, operation information, and position information of the host vehicle 300 from an external information acquisition device 200. On the other hand, the dangerous driving analysis apparatus 100 of the third embodiment obtains such information by itself. Hereinafter, differences from the first embodiment will be described.

  The configuration of the dangerous driving analysis apparatus 100 of the third embodiment will be described below. FIG. 39 is an example of a block diagram illustrating a hardware configuration of the dangerous driving analysis apparatus according to the third embodiment.

  The dangerous driving analysis device 170 includes, for example, a CPU 101, a ROM 102, a RAM 103, an input / output device I / F 104, and a communication I / F 108. These are connected to each other via a bus 109.

  The input / output device I / F 104 is connected to input / output devices such as the display 105, mouse 106, keyboard 107, peripheral information acquisition device 205, travel information acquisition device 206, operation information acquisition device 207, and position information acquisition device 208. .

  Next, the functional configuration of the dangerous driving analysis apparatus 100 will be described. FIG. 40 is an example of a block diagram illustrating a functional configuration of the dangerous driving analysis apparatus according to the third embodiment. The dangerous driving analysis apparatus 100 according to the third embodiment includes, in addition to the dangerous driving analysis apparatus 100 according to the first embodiment, a surrounding information acquisition unit 220, a travel information acquisition unit 221, an operation information acquisition unit 222, and position information. An acquisition unit 223 is provided. Further, since the dangerous driving analysis apparatus 100 according to the third embodiment does not require transmission / reception of data, commands, and the like with the information acquisition apparatus 200, the transmission / reception units 120 and 224 and the acquisition data DB 225 are omitted. The processing of each function is the same as in the first embodiment.

  Other configurations are the same as those of the first embodiment. Also, the third embodiment can be applied to the second embodiment.

<Other embodiment examples>
Further, a computer program that causes a computer to execute the above-described method and a computer-readable recording medium that records the program are included in the scope of the present invention. Here, as a computer-readable recording medium, for example, a flexible disk, a hard disk, a CD-ROM (Compact Disc-Read Only Memory), an MO (Magneto Optical disk), a DVD, a DVD-ROM, a DVD-RAM (DVD-RAM). Random Access Memory), BD (Blue-ray Disc), USB memory, semiconductor memory, and the like. The computer program is not limited to the one recorded on the recording medium, and may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, or the like.

  Regarding the above embodiment and other embodiments, the following additional notes are disclosed.

<Appendix>
(Appendix 1)
An acquisition step of acquiring driving information including surrounding information of the own vehicle, a detecting step of detecting surrounding objects obtained from the surrounding information, and a safe driving standard from the relative relationship between the own vehicle and the objects are satisfied. A calculation step for calculating a determination value for determining whether or not, a determination step for comparing the determination value with a reference value for the calculation to determine dangerous driving that does not satisfy the safe driving criterion, and the determination step And a dangerous driving extraction step of extracting the peripheral information of the time determined as dangerous driving in (1).

  The driving information related to the host vehicle includes at least a video around the host vehicle. In addition, the driving information related to the host vehicle may include various types of information such as the relative distance and relative speed between an object existing around the host vehicle and the host vehicle, and the speed and acceleration of the host vehicle. In addition, the driving information related to the host vehicle is stored not only when the driver of the host vehicle recognizes dangerous driving such as when sudden braking and sudden steering operation is performed, but also when the host vehicle is in a driving state. The

  The information processing method extracts dangerous driving based on objective safe driving standards from the driving information of the own vehicle. Therefore, not only dangerous driving recognized by the driver of the host vehicle but also dangerous driving not recognized by the driver of the host vehicle can be extracted. Further, if the information processing method is used, it is possible to extract dangerous driving efficiently, objectively, and quickly without depending on human hands. By letting the driver view the dangerous driving image and the like extracted in this way, it is possible to effectively learn under what circumstances dangerous driving occurs and to efficiently perform safe driving education.

(Appendix 2)
The safe driving standards include
Relative distance reference value relative to the relative distance between the host vehicle and the object, and TTC (Time to collision) which is a predicted collision time until the host vehicle and each object existing around the host vehicle collide with each other. TTC reference value, TTC change rate reference value with respect to the change rate of the TTC per predetermined unit time, acceleration reference value with respect to the acceleration of the host vehicle, and / or steering angle change rate reference with respect to the steering angle change rate per predetermined unit time Value,
In the dangerous driving extraction step,
A condition that the relative distance is smaller than the relative distance reference value; a condition that the TTC is smaller than the TTC reference value; a condition that the rate of change of the TTC is larger than the TTC change rate reference value; At least one condition is satisfied under the condition that the magnitude of the acceleration of the host vehicle is greater than the acceleration reference value and the condition that the steering angle change rate of the host vehicle is greater than the steering angle change rate reference value. The information processing method according to attachment 1, wherein the driving information is extracted as information on the dangerous driving.

  When the relative distance is smaller than the relative distance reference value, the distance between the host vehicle and the target is close, and the risk of the host vehicle colliding with the target is high. Further, when TTC is smaller than the TTC reference value, the distance between the host vehicle and the object is short, and there is a high risk that the host vehicle will eventually collide with the object. When the TTC change rate is larger than the TTC change rate reference value, the distance between the host vehicle and the object is rapidly approaching, and the risk of collision is high. When the acceleration of the host vehicle is large, the host vehicle suddenly accelerates or decelerates. When the rate of change of the steering angle is large, the host vehicle is meandering. You can judge that you are not driving safely. Therefore, in the information processing method, such a driving state is extracted from driving information as dangerous driving. More specifically, since the driving information is configured along the time axis, the section in the driving state as described above is extracted from the driving information as the dangerous driving section.

  As a method for extracting an object, first, peripheral information including one or more objects existing around the host vehicle is acquired in the peripheral information acquisition step, and then, the one or more objects are acquired from the peripheral information. The method of extracting a thing is mentioned.

  More specifically, the peripheral information acquisition unit that acquires the peripheral information includes one or a plurality of imaging devices that capture the periphery of the host vehicle. The image processing means generates a surrounding image of the host vehicle from the image taken by the imaging device, and the object extracting means extracts the one or more objects from the surrounding image.

  As another method, the object extracting means obtains a detection result from an object sensor that detects one or more objects existing around the host vehicle, and the one or more objects are detected based on the detection result. The method of extracting a thing is mentioned.

  As yet another method, the object extracting means transmits object information including the position, speed, acceleration and / or traveling direction of the object transmitted from one or more objects existing around the host vehicle. There is a method of acquiring and extracting the one or more objects based on the object information.

(Appendix 3)
In the dangerous driving extraction step,
When the condition that the relative distance is smaller than the relative distance reference value is satisfied and the TTC is smaller than the TTC reference value, the change rate of the TTC is larger than the TTC change rate reference value. The condition that the magnitude of the acceleration of the host vehicle is greater than the acceleration reference value, and the condition that the steering angle change rate of the host vehicle is greater than the steering angle change rate reference value. The information processing method according to appendix 2, wherein the driving information that satisfies a condition is extracted as information on the dangerous driving.

  In the above configuration, the condition that the relative distance between the host vehicle and the target object is smaller than the relative distance reference value is essential as a condition for extracting information on dangerous driving. For example, even if TTC becomes smaller than the TTC reference value due to sudden acceleration, if the host vehicle and the object are far away from each other, the risk of collision between the host vehicle and the object is low. Therefore, in the driving information configured along the time axis, a section in which the relative distance between the host vehicle and the object is equal to or greater than the relative distance reference value is excluded from the dangerous driving section. Thereby, only a section with high risk of collision can be extracted as a section of dangerous driving.

(Appendix 4)
The dangerous driving extraction step includes:
A first step of extracting each approach section that satisfies a condition that the relative distance is smaller than the relative distance reference value from the driving information on the time axis;
A second step of calculating a minimum value of the TTC, a maximum value of the rate of change of the TTC, a maximum value of the acceleration, and / or a maximum value of the rate of change of the steering angle based on driving information of each approach section; ,
Among each approaching section, a condition that the minimum value of the TTC is smaller than the TTC reference value, a condition that the maximum value of the change rate of the TTC is larger than the TTC change rate reference value, and the magnitude of the acceleration An approach section that satisfies at least one condition under the condition that the maximum value of the steering angle is greater than the acceleration reference value and the condition that the maximum value of the steering angle change rate is greater than the steering angle change rate reference value. A third step of extracting and extracting driving information of the extracted approaching section as information on the dangerous driving;
The information processing method according to attachment 2, further comprising:

  First, an approach section whose relative distance is smaller than the relative distance reference value is extracted from the driving information. Next, of the extracted approaching sections, an approaching section that satisfies the above conditions is extracted as a dangerous driving section. Therefore, the same effect as the above supplementary note 3 is achieved.

(Appendix 5)
A recognition determination step of determining whether or not the driver of the host vehicle recognizes that the driving of the host vehicle is dangerous driving;
In the dangerous driving extraction step, the dangerous driving determined to be not recognized by the driver of the host vehicle in the recognition determining step is further extracted from the dangerous driving that does not satisfy the safe driving standard. The information processing method in any one of -4.

  As a result, it is possible to extract dangerous driving that the driver cannot recognize among dangerous driving that does not satisfy the safe driving standard. Safe driving education can be effectively performed by having the driver watch dangerous driving that cannot be recognized and showing the driver what kind of dangerous driving has not been recognized.

  In the recognition determination step, not only the presence / absence of recognition of dangerous driving, but also the degree of recognition indicating how much dangerous driving is recognized may be determined.

(Appendix 6)
In the recognition determination step, when it is determined that the negative acceleration of the host vehicle is greater than a predetermined value, the driver of the host vehicle recognizes that the driving of the host vehicle is dangerous driving. The information processing method according to attachment 5, wherein the information is determined.

  That the magnitude of the negative acceleration of the host vehicle is greater than a predetermined value means that the rate of decrease in speed is greater than a predetermined value, for example, when the driver performs a sudden braking operation. That is, since the driver recognizes dangerous driving and spontaneously performs a sudden braking operation, it can be determined that the rate of decrease in the speed of the host vehicle has become greater than a predetermined value.

(Appendix 7)
In the recognition determination step, when it is further determined that the braking operation of the host vehicle is performed, it is determined that the driver of the host vehicle recognizes that the driving of the host vehicle is a dangerous driving. Information processing method described in 1.

  In the recognition determination step, presence / absence of recognition for dangerous driving may be determined based on both the brake operation and the acceleration. That is, in the recognition determination step, it is determined that dangerous driving is recognized when the driver performs a braking operation and the magnitude of the negative acceleration is greater than a predetermined value. Here, the phenomenon that the magnitude of the negative acceleration becomes larger than a predetermined value may occur not only when the brake operation is performed, but also when the host vehicle travels on a steep uphill, for example. By reliably detecting the operation performed by the driver based on the brake operation, it is possible to reliably detect the presence or absence of recognition for dangerous driving.

  In addition, when the rate of change of the steering angle is large, a case where the driver suddenly operates the steering wheel in order to avoid a collision with an object can be considered, and it can be determined that dangerous driving is recognized.

(Appendix 8)
The dangerous driving extracted in the dangerous driving extraction step is a classification including position information of the own vehicle, traveling information of the own vehicle, operation information of the own vehicle, and information on an object existing around the own vehicle. The information processing method according to any one of appendices 1 to 7, further including a dangerous driving classification step of classifying by item.

  The position information of the own vehicle is information for specifying where the own vehicle is, and is information indicating a situation where the own vehicle is located such as an intersection, a straight road, a three-lane road, and a T-junction. is there. The traveling information of the host vehicle is, for example, the speed and acceleration of the host vehicle. The operation information of the host vehicle is information including the presence / absence of a brake operation, the steering angle, the presence / absence of a turn indicator, the presence / absence of an accelerator operation, and the like. The information related to the object existing around the host vehicle is information for specifying the object, for example, the object is a truck, a person, or a motorcycle. By classifying the extracted dangerous driving according to these classification items, it is possible to immediately retrieve desired data from the dangerous driving. Further, by classifying dangerous driving according to the above classification items and investigating the tendency, it is possible to analyze under what circumstances the driver is likely to fall into dangerous driving.

(Appendix 9)
The driving information is generated based on peripheral information around the host vehicle,
The information processing method according to claim 1, further comprising a display video generation step of generating a display video for displaying a dangerous driving extracted from the driving information based on the peripheral information.

  The peripheral information around the host vehicle is peripheral information around the entire 360 ° of the host vehicle, for example. In the display image generation step, a display image of dangerous driving can be generated based on the surrounding information of the entire periphery. Therefore, the viewer of dangerous driving can watch dangerous driving over the entire periphery of the host vehicle. For example, even if a cause of dangerous driving occurs in a blind spot that is difficult to visually recognize while driving the host vehicle, a viewer of dangerous driving can look back on such dangerous driving with a display image.

(Appendix 10)
The display image generation step receives a viewpoint and a gaze direction with the viewpoint as an origin, and generates a display image of the dangerous driving from the peripheral information based on the viewpoint and the gaze direction. Information processing method.

  In the display video generation step, if a viewpoint and a line-of-sight direction are designated, a display image of dangerous driving based on various viewpoints can be generated. For example, if a viewpoint is set for the vehicle behind the host vehicle and a line-of-sight direction from the vehicle behind the host vehicle is set, the display image generation step generates a display image of the dangerous driving of the host vehicle from the viewpoint of the rear vehicle. can do. Therefore, the driver of the own vehicle who performed dangerous driving can look back at dangerous driving of the own vehicle seen from other vehicles objectively, and can perform safe driving education effectively.

(Appendix 11)
Acquisition means for acquiring driving information including surrounding information of the own vehicle, detecting means for detecting surrounding objects obtained from the surrounding information, and whether or not a safe driving standard is satisfied from a relative relationship between the own vehicle and the objects A computing means for computing a judgment value for judging whether the judgment value is compared with a reference value for the computation, a judgment means for judging a dangerous driving that does not satisfy the safe driving standard, and a dangerous driving by the judgment means An information processing program for causing a computer to function as dangerous driving extraction means for extracting the peripheral information of the time determined to be.

(Appendix 12)
Satisfy safe driving standards from the relative relationship between the acquisition means for acquiring driving information including the surrounding information of the own vehicle, the detecting means for detecting surrounding objects obtained from the surrounding information, and the own vehicle and the objects A calculation means for calculating a determination value for determining whether or not, a determination means for comparing the determination value with a reference value for the calculation to determine dangerous driving that does not satisfy the safe driving standard, and the determination means And an unsafe driving extraction means for extracting the peripheral information of the time determined as unsafe driving.

100: Dangerous driving analysis device
121: Video processing unit
122: Object extraction unit
123: Relative information calculation unit
124: Dangerous driving extraction unit
125: Recognition determination unit
126: Dangerous driving classification section
127: Display video generation unit
128: Peripheral information DB
129: Relative information DB
130: Dangerous driving information DB
131: Driving / operation information DB
132: Location information DB
133: Dangerous driving pattern DB
134: Various correspondence table DB
135: Gaze data DB
136: Gaze determination unit
150a: eye vector
150: Line of sight
151: Gaze space
155a: Mirror line of sight vector 155: Mirror line of sight
160a: object vector
200: Information acquisition device
250: Terminal for driving education
300: own vehicle 400: three-dimensional projection plane
500: Other vehicle

Claims (10)

An acquisition step of acquiring driving information including surrounding information of the own vehicle;
A detection step of detecting a surrounding object obtained from the surrounding information;
A calculation step of calculating a determination value for determining whether or not a safe driving standard is satisfied from a relative relationship between the host vehicle and the object;
A determination step of comparing the determination value with a reference value for the calculation to determine dangerous driving that does not satisfy the safe driving criterion;
A dangerous driving extraction step of extracting the peripheral information of the time determined to be dangerous driving in the determination step. The safe driving standards include
Relative distance reference value relative to the relative distance between the host vehicle and the object, and TTC (Time to collision) which is a predicted collision time until the host vehicle and each object existing around the host vehicle collide with each other. TTC reference value, TTC change rate reference value for the change rate of the TTC per unit time, acceleration reference value for the acceleration of the host vehicle and / or steering angle change rate reference value for the steering angle change rate per unit time are included. ,
In the dangerous driving extraction step,
A condition that the relative distance is smaller than the relative distance reference value; a condition that the TTC is smaller than the TTC reference value; a condition that the rate of change of the TTC is larger than the TTC change rate reference value; At least one condition is satisfied under the condition that the magnitude of the acceleration of the host vehicle is greater than the acceleration reference value and the condition that the steering angle change rate of the host vehicle is greater than the steering angle change rate reference value. The information processing method according to claim 1, wherein peripheral information about the dangerous driving is extracted from the driving information. In the dangerous driving extraction step,
When the condition that the relative distance is smaller than the relative distance reference value is satisfied and the TTC is smaller than the TTC reference value, the change rate of the TTC is larger than the TTC change rate reference value. The condition that the magnitude of the acceleration of the host vehicle is greater than the acceleration reference value, and the condition that the steering angle change rate of the host vehicle is greater than the steering angle change rate reference value. The information processing method according to claim 2, wherein the driving information that satisfies a condition is extracted as information on the dangerous driving. The dangerous driving extraction step includes:
A first step of extracting each approach section that satisfies a condition that the relative distance is smaller than the relative distance reference value from the driving information on the time axis;
A second step of calculating a minimum value of the TTC, a maximum value of the rate of change of the TTC, a maximum value of the acceleration, and / or a maximum value of the rate of change of the steering angle based on driving information of each approach section; ,
Among each approaching section, a condition that the minimum value of the TTC is smaller than the TTC reference value, a condition that the maximum value of the change rate of the TTC is larger than the TTC change rate reference value, and the magnitude of the acceleration An approach section that satisfies at least one condition under the condition that the maximum value of the steering angle is greater than the acceleration reference value and the condition that the maximum value of the steering angle change rate is greater than the steering angle change rate reference value. A third step of extracting and extracting driving information of the extracted approaching section as information on the dangerous driving;
The information processing method according to claim 2, further comprising: A recognition determination step of determining whether or not the driver of the host vehicle recognizes that the driving of the host vehicle is dangerous driving;
The dangerous driving extraction step further extracts dangerous driving determined as not recognized by the driver of the host vehicle in the recognition determination step from dangerous driving that does not satisfy the safe driving standard. The information processing method in any one of 1-4.   The dangerous driving extracted in the dangerous driving extraction step is a classification including position information of the own vehicle, traveling information of the own vehicle, operation information of the own vehicle, and information on an object existing around the own vehicle. The information processing method according to claim 1, further comprising a dangerous driving classification step of classifying by item. The driving information is generated based on peripheral information around the host vehicle,
The information processing method according to claim 1, further comprising a display video generation step of generating a display video for displaying a dangerous driving extracted from the driving information based on the peripheral information.   The display video generation step receives a viewpoint and a gaze direction with the viewpoint as an origin, and generates a display video of the dangerous driving from the peripheral information based on the viewpoint and the gaze direction. Information processing method. Acquisition means for acquiring driving information including surrounding information of the host vehicle;
Detecting means for detecting a surrounding object obtained from the surrounding information;
A calculation means for calculating a determination value for determining whether or not a safe driving standard is satisfied from a relative relationship between the host vehicle and the object;
The determination value is compared with a reference value for the calculation, and determination means for determining dangerous driving that does not satisfy the safe driving standard, and the surrounding information of the time determined as dangerous driving by the determination means are extracted. An information processing program that causes a computer to function as dangerous driving extraction means. Obtaining means for obtaining driving information including surrounding information of the own vehicle;
Detecting means for detecting a surrounding object obtained from the surrounding information;
A calculation means for calculating a determination value for determining whether or not a safe driving standard is satisfied from a relative relationship between the host vehicle and the object;
A determination unit that compares the determination value with a reference value for the calculation to determine dangerous driving that does not satisfy the safe driving standard;
An information processing apparatus comprising: dangerous driving extraction means for extracting the peripheral information at a time determined as dangerous driving by the determination means.