JP2018151693A - Driving support device and driving support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a driving support device and a driving support method capable of minimizing damage to a driver. A driving support device according to an embodiment includes a detection unit, a determination unit, and an information generation unit. The detection unit detects the behavior of the passenger with respect to the driver of the vehicle based on the captured image of the inside of the vehicle. The determination unit determines whether or not the behavior detected by the detection unit is a damaging behavior that causes damage to the driver. The information generation unit generates damage occurrence information indicating the occurrence of the damage behavior when the determination unit determines that the damage behavior is performed. [Selection diagram] Fig. 2

Description

  The present invention relates to a driving support device and a driving support method.

  2. Description of the Related Art Conventionally, for example, a camera system that stores a captured image obtained by capturing an image inside a vehicle is known. Further, for example, there is a driving support device that detects a driver's behavior such as looking aside or falling asleep based on such a captured image, and warns the driver according to the detected behavior (see, for example, Patent Document 1).

JP 2016-110449 A

  However, the prior art warns the driver that the danger is imminent based on the driver's own behavior, and for example, no consideration is given to the case where there is a passenger. That is, the conventional apparatus cannot respond to a warning or the like when the driver is damaged due to, for example, the passenger's behavior in the vehicle, and there is a risk that the driver's damage may be increased.

  The present invention has been made in view of the above, and an object of the present invention is to provide a driving support device and a driving support method capable of minimizing damage to the driver.

  In order to solve the above-described problems and achieve the object, the driving support apparatus according to the present invention includes a detection unit, a determination unit, and an information generation unit. The detection unit detects a passenger's behavior with respect to the driver of the vehicle based on a captured image obtained by capturing an image of the interior of the vehicle. The determination unit determines whether or not the behavior detected by the detection unit is a damage behavior that causes damage to the driver. An information generation part produces | generates the damage occurrence information which shows generation | occurrence | production of the said damage action, when it determines with the said damage action by the said determination part.

  According to the present invention, damage to the driver can be minimized.

FIG. 1A is a diagram illustrating an outline of a driving support method according to the embodiment. FIG. 1B is a diagram illustrating an overview of the driving support method according to the embodiment. FIG. 2 is a block diagram illustrating a configuration of the driving support apparatus according to the embodiment. FIG. 3 is a diagram illustrating body feature points. FIG. 4A is an explanatory diagram of behavior pattern information. FIG. 4B is an explanatory diagram of audio pattern information. FIG. 5 is a diagram illustrating processing contents of the determination unit. FIG. 6 is a diagram illustrating processing contents of the determination unit. FIG. 7 is a flowchart illustrating a processing procedure of processing executed by the driving support apparatus according to the embodiment.

  Hereinafter, embodiments of a driving support apparatus and a driving support method disclosed in the present application will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by this embodiment.

  First, the outline | summary of the driving assistance method which concerns on embodiment is demonstrated using FIG. 1A and FIG. 1B. 1A and 1B are diagrams illustrating an outline of a driving support method according to an embodiment. FIG. 1A shows a vehicle C equipped with the driving support device 1 according to the embodiment.

  The mounting target of the driving support device 1 is suitable for a vehicle C on which a customer is frequently placed, for example, a taxi or the like, that is, there is a relatively high possibility that a passenger is present in the vehicle. In FIG. 1, the vehicle C is shown in the form of an ordinary passenger car. However, the present invention is not limited to this, and the vehicle C may be a large vehicle such as a bus or a truck.

  FIG. 1A shows an imaging apparatus 10 that is a camera that is provided, for example, in the front of the vehicle and that captures the interior of the vehicle C. Further, FIG. 1B shows a captured image P captured by the imaging device 10, and the captured image P includes the driver 100 and the passenger 110 of the vehicle C.

  The driving support method according to the embodiment detects the behavior of the passenger 110 with respect to the driver 100, and when the detected behavior is a damaging behavior that causes damage to the driver 100, the damage occurrence information indicating the occurrence of the damaging behavior is displayed. Is to be generated.

  Here, a conventional driving support method will be described. The conventional driving support method warns the driver that the danger is imminent by detecting the driver's own behavior such as looking aside or falling asleep.

  However, conventionally, for example, when there is a passenger in the vehicle, no consideration has been given to the danger to the driver due to the action in the vehicle caused by the passenger. For example, when the passenger holds the driver's arm for a long time, the driver himself is injured or the driving operation is hindered. That is, conventionally, when the driver is damaged by the passenger's behavior in the vehicle, it is not possible to take a warning or the like, and the driver's damage may be increased.

  Therefore, in the driving support method according to the embodiment, for example, when the passenger 110 takes an action of causing harm to the driver 100, the passenger 110 is warned. Specifically, as illustrated in FIG. 1B, the driving support device 1 according to the embodiment shows the behavior of the passenger 110 with respect to the driver 100 of the vehicle C based on the captured image P obtained by capturing the interior of the vehicle C. Detect (step S1).

  Subsequently, the driving support apparatus 1 according to the embodiment determines whether or not the detected action is a damaging action that causes damage to the driver 100 (step S2). And the driving assistance apparatus 1 which concerns on embodiment produces the damage generation | occurrence | production information which shows generation | occurrence | production of damage action, when the action of the passenger 110 is damage action (step S3).

  For example, as illustrated in FIG. 1B, the driving assistance device 1 determines that an action in which the passenger 110 grabs the arm of the driver 100 is a damage action. And the driving assistance device 1 outputs a buzzer sound in the vehicle as damage occurrence information, for example, and warns the passenger 110.

  Thereby, for example, the passenger's dangerous state is improved by stopping the damage behavior such that the passenger 110 who hears the buzzer sound releases the arm of the driver 100. Alternatively, if the dangerous state is not improved even by the buzzer sound, the driver 100 is protected early by reporting to related organizations such as the police.

  Therefore, according to the driving support apparatus 1 according to the embodiment, the damage to the driver 100 can be minimized by detecting a damage action that causes damage to the driver 100 and performing, for example, a warning or a notification. .

  The driving support device 1 according to the embodiment is not limited to the case of generating damage occurrence information for warning by a buzzer sound or reporting to related organizations such as the police, but records the state in the vehicle at the time of the damage action. The damage occurrence information may be generated as described above, but this point will be described later.

  Note that the driving support apparatus 1 according to the embodiment determines a contact action that the passenger 110 touches the driver 100, for example, grasps an arm as a damage action, and details of this point will be described later with reference to FIG. .

  Moreover, although the driving assistance apparatus 1 which concerns on embodiment determines damage action based on the objective action performed with respect to the fixtures of the vehicle C, such as a seat sheet and a window glass, and the vibration of the vehicle C accompanying an objective action, it takes, Details of the points will be described later with reference to FIG.

  In addition, the driving support device 1 according to the embodiment determines that the behavior is damaging when, for example, the passenger carries a tool such as a knife or a handgun, which will be described later.

  Next, with reference to FIG. 2, the structure of the driving assistance apparatus 1 which concerns on embodiment is demonstrated in detail. FIG. 2 is a block diagram illustrating a configuration of the driving support device 1 according to the embodiment.

  As illustrated in FIG. 2, the driving support device 1 according to the embodiment includes an imaging device 10, a microphone 11, a position acquisition unit 12, a vehicle speed sensor 13, an acceleration sensor 14, a warning device 20, and a notification device 30. And connected to. First, a configuration other than the driving support device 1 will be described.

  The imaging device 10 is an in-vehicle camera including, for example, a lens such as a fisheye lens and an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS).

  The imaging device 10 is provided, for example, at a position (see FIG. 1) for imaging from the front to the rear in the vehicle, captures a captured image P including the driver 100 and the passenger 110 in the vehicle, and outputs the captured image P to the driving support device 1 To do.

  The imaging device 10 is not limited to being provided in front of the vehicle, and may be on the side of the vehicle, and may be provided at a position where the driver 100 and the passenger 110 are included in the captured image P. . The imaging device 10 may be provided at a plurality of positions in the vehicle. In such a case, the plurality of captured images P are output to the driving support device 1.

  The microphone 11 is mounted on the vehicle C and collects sound generated in the vehicle and outputs the sound to the driving support device 1. The position acquisition unit 12 detects the current position of the vehicle C based on, for example, a signal from a GPS (Global Positioning System) satellite, and outputs the current position information to the driving support device 1 as information on the current position.

  The vehicle speed sensor 13 generates a pulse signal corresponding to the number of rotations of the axle in the vehicle C, detects the current traveling speed of the vehicle C based on the pulse signal, and outputs it to the driving support device 1.

  The acceleration sensor 14 detects the acceleration applied to the vehicle C and outputs it to the driving support device 1. As the acceleration sensor 14, for example, a triaxial acceleration sensor that detects acceleration in three axial directions orthogonal to each other can be used.

  As the acceleration sensor 14, for example, an acceleration sensor provided in an airbag ECU (Electronic Control Unit) can be used. However, the present invention is not limited to this, and an acceleration sensor mounted in another electronic device such as a drive recorder is used. It may be used.

  The warning device 20 is a device that emits sound in and out of the vehicle C by controlling, for example, a buzzer, a horn, and the like provided in the vehicle C based on the damage occurrence information output from the driving support device 1. For example, the warning device 20 can give a warning to the passenger 110 by emitting a buzzer sound in the vehicle, or can sound a horn to inform the surroundings of the vehicle C of the occurrence of the damaging behavior.

  Note that the warning device 20 is not limited to a warning using sounds such as a buzzer or a horn, but can also give a warning using light by controlling, for example, a light that illuminates the outside of the vehicle C, a room light, or the like.

  The notification device 30 includes a communication module used for a telematics service such as DCM (Data Communication Module), for example, and is a device that makes an emergency call to related organizations such as the police based on damage occurrence information output from the driving support device 1. is there.

  Specifically, the reporting device 30 transmits damage occurrence information to a server device (not shown) via a communication network such as a mobile phone network or the Internet network, and the server device reports to related organizations based on the damage occurrence information.

  In such a case, it is preferable to include information on the current position of the vehicle C acquired by the position acquisition unit 12 in the damage occurrence information. Thereby, since the police etc. which received the emergency call can reach the spot quickly, the driver 100 can be protected early.

  Next, the driving assistance device 1 according to the embodiment will be described. The driving support device 1 includes a control unit 2 and a storage unit 3. The control unit 2 includes a detection unit 21, a vibration detection unit 22, a determination unit 23, and an information generation unit 24. The storage unit 3 stores image information 31 and damage behavior information 32.

  Here, the driving support device 1 is, for example, a computer having various functions such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), an input / output port, and the like. including.

  The CPU of the computer functions as the detection unit 21, the vibration detection unit 22, the determination unit 23, and the information generation unit 24 of the control unit 2 by reading and executing a program stored in the ROM, for example.

  In addition, at least one or all of the detection unit 21, the vibration detection unit 22, the determination unit 23, and the information generation unit 24 of the control unit 2 are configured as hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). It can also be configured with wear.

  The storage unit 3 corresponds to, for example, a RAM or an HDD. The RAM and HDD can store image information 31, damage behavior information 32, information on various programs, and the like. The driving support device 1 may acquire the above-described program and various types of information via another computer or a portable recording medium connected via a wired or wireless network.

  The image information 31 is information including a time-series captured image P periodically captured by the imaging device 10. In addition, the storage unit 3 stores, as the damage occurrence information, the captured image P captured at the time of the occurrence of the damage of the driver 100 in the image information 31 in accordance with an instruction from the information generation unit 24 described later. The method for saving the image information 31 in the damage occurrence information may be either a trigger type or a constant recording type, but this point will be described later.

  The control unit 2 detects the behavior of the passenger 110 with respect to the driver 100, and generates damage occurrence information indicating the occurrence of the damage behavior when the detected behavior is damage behavior that causes damage to the driver 100.

  The detection unit 21 detects the behavior of the passenger 110 based on various information such as the imaging device 10, the microphone 11, the position acquisition unit 12, and the vehicle speed sensor 13.

  For example, the detection unit 21 detects the behavior of the passenger 110 with respect to the driver 100 of the vehicle C based on the captured image P obtained by imaging the interior of the vehicle C. Here, a specific detection process of the detection unit 21 will be described with reference to FIG.

  FIG. 3 is a view showing feature points of the passenger 110's body. Note that FIG. 3 only shows an example of human feature points, and the position and number of feature points can be arbitrarily set.

  As shown in FIG. 3, the detection unit 21 detects feature points j1 to j20 of each part of the passenger 110 body present in the vehicle, and determines the posture and orientation of the passenger 110 based on the feature points j1 to j20. By analyzing, various actions of the passenger 110 are detected.

  Specifically, in FIG. 3, 20 feature points j1 to j20 that define the behavior of the passenger 110 are shown as an example. Note that the feature points j1 to j4 correspond to the head, the shoulder center, the turning portion, and the hip center in order. Further, the feature points j5 to j8 correspond to the right hand tip, the right wrist, the right elbow, and the shoulder right, respectively, in order.

  Also, the feature points j9 to j12 correspond to the right hip, the right knee, the right ankle, and the right foot, respectively, in order. Further, the feature points j13 to j16 correspond to the left hand tip, the left wrist, the left elbow, and the left shoulder part in order. Also, the feature points j17 to j20 correspond to the left hip, the left knee, the left ankle, and the left foot, respectively, in order. Each feature point j1 to j20 is expressed as a coordinate position on the XY plane, for example.

  That is, the detection unit 21 analyzes the posture and orientation of the passenger 110 by grasping the positional relationship between the feature points j1 to j20 when the passenger 110 moves the body.

  Note that the detection unit 21 preferably predefines the body feature points of the driver 100 before detecting the feature points j1 to j20 of the passenger 110. For example, the detection unit 21 performs feature point detection processing immediately after the driving support device 1 is started, that is, without the passenger 110, and detects the feature point of the driver 100 in advance. Thereby, it is possible to determine whether the detected feature point belongs to the driver 100 or the passenger 110.

  The vibration detection unit 22 detects vibration generated in the vehicle C based on the detection value of the acceleration sensor 14, for example. For example, the vibration detection unit 22 detects the vibration of the vehicle C accompanying the action of the passenger 110 detected by the detection unit 21. Specifically, the vibration detection unit 22 detects the vibration of the vehicle C accompanying the action of the passenger 110 by associating the detection time of the detection unit 21 with the detection value of the acceleration sensor 14.

  In addition, when the vehicle C is traveling, if it is difficult to distinguish the vibration of the vehicle C due to the action of the passenger 110 and the vibration of the vehicle C due to traveling, the vibration will occur only when the vehicle C is stopped. The vibration may be detected by the detection unit 22.

  The determination unit 23 determines whether or not the behavior of the passenger 110 detected by the detection unit 21 is a damaging behavior that causes damage to the driver 100. Specifically, the determination unit 23 determines the presence / absence of the damage behavior based on the damage behavior information 32 stored in the storage unit 3.

  The damage behavior information 32 is information including the behavior pattern information 32a and the voice pattern information 32b, and is information serving as a criterion for determining the damage behavior by the determination unit 23. Here, the damage behavior information 32 will be described with reference to FIGS. 4A and 4B.

  FIG. 4A is an explanatory diagram of the behavior pattern information 32a. FIG. 4B is an explanatory diagram of the audio pattern information 32b. First, the behavior pattern information 32a will be described with reference to FIG. 4A. As shown in FIG. 4A, information regarding various patterns of damage behavior is registered in advance in the behavior pattern information 32a.

  The behavior pattern information 32a is information including “behavior type” and “feature point image”, for example. “Behavior type” is information indicating the type of damage behavior. The “feature point image” is image information indicating a position pattern of the feature points j1 to j20 of the passenger 110 detected by the detection unit 21. The “feature point image” may be a still image or a moving image.

  For example, FIG. 4A shows “feature point images” in “gripping” when the passenger 110 grabs the arm 100 of the driver 100 and “ruling” when the passenger 110 hits the driver 100. Note that the “feature point image” pattern shown in FIG. 4A is merely a representative example. In fact, for example, “feature” “feature point images” include various “feature” pattern “features”. Point images "are included.

  For example, the “feature point image” of “grab” is a moving image in which the feature point j5 of the right hand of the passenger 110 (or the feature point j13 of the left hand) (see FIG. 3) remains at the position of the driver 100 for a predetermined time. Is shown.

  Further, for example, the “feature point image” of “speak” is a moving image in which the feature point j5 (or the feature point j13 of the left hand) of the passenger 110 reaches the position of the driver 100 at a predetermined moving speed. Show. Note that the moving speed of the feature point j5 can be calculated from the magnitude of the vector of the feature point j5 extracted from the captured image P, for example.

  That is, the determination unit 23 determines whether or not the action performed by the passenger 110 is a damaging action by comparing the feature points j1 to j20 detected by the detection unit 21 with the action pattern information 32a. . And the determination part 23 determines with it being a damage action, when the action which the passenger 110 performed matches with the action pattern information 32a, ie, it corresponds with the pattern of damage action.

  As described above, the determination unit 23 can accurately determine the damaging behavior of the passenger 110 by performing pattern matching based on the body feature points of the passenger 110.

  Note that the determination unit 23 only needs to detect some feature points of the “feature point image” even if the detection unit 21 does not detect all (20 in FIG. 3) feature points j1 to j20. That is, the determination unit 23 may determine that the behavior is a damage action when some of the feature points of the passenger 110 detected by the detection unit 21 match some of the feature points of the “feature point image”. Good.

  Further, since the passenger is sitting on the seat, the lower body hardly moves even when performing the damage behavior, so the “feature point image” may be data only for the upper body. Thereby, the memory capacity can be reduced.

  Furthermore, the determination unit 23 may determine the damage behavior using the voice pattern information 32b illustrated in FIG. 4B in consideration of the utterance content of the passenger 110. As shown in FIG. 4B, the voice pattern information 32b is information including “voice type” and “character information”, for example.

  As shown in FIG. 4B, the voice pattern information 32b is information in which risk keywords that are issued before the passenger 110 inflicts harm to the driver 100 or issued to threaten the driver 100 are registered in advance. is there. The voice pattern information 32b is information in which “voice type” and “character information” are associated with each other, for example.

  The determination unit 23 determines that the voice is a damaging action when a predetermined risk keyword is included in the voice indicating the utterance content of the passenger 110 detected by the detection unit 21 from the microphone 11.

  Specifically, the determination unit 23 converts the voice detected by the detection unit 21 into voiced text information converted into text, and the voice text information that matches the “character information” of the voice pattern information 32b. If there is such a voice, the voice is determined as a damage behavior.

  In other words, the determination unit 23 detects the voice that is uttered before the physical harm such as the danger keyword, and gives a warning by the warning device 20, for example, to determine the damage behavior that the passenger 110 is about to perform. Can be prevented.

  For example, when the number of characters in the “character information” is relatively small as “oi” shown in FIG. 4B, the determination unit 23 analyzes the speech detected by the detection unit 21, for example, to suppress the speech. It is preferable to detect and determine the damage behavior.

  Moreover, although the determination part 23 detected the damage action based on the passenger's 110 feature points j1-j20, it is not limited to this, A damage action is considered in addition to the passenger 110 and the driver's 100 feature point. May be detected. This point will be described with reference to FIG.

  FIG. 5 is a diagram illustrating the determination contents of the determination unit 23. FIG. 5 shows a captured image P obtained by capturing a situation where the passenger 110 holds the driver 100. FIG. 5 shows feature points aj15 and aj16 of the driver 100 detected from the captured image P and a feature point j5 of the passenger 110.

  In FIG. 5, other feature points of the driver 100 including the feature points aj15 and aj16 on the left elbow and shoulder left of the driver 100 are shown by white circles from the viewpoint of easy viewing. Similarly, with respect to the feature points of the passenger 110, other feature points including the right-hand feature point j5 are indicated by black circles.

  As illustrated in FIG. 5, when the behavior of the passenger 110 detected by the detection unit 21 is a contact behavior that touches the driver 100, the determination unit 23 determines the behavior as a damage behavior.

  Specifically, the determination unit 23 determines whether the behavior detected by the detection unit 21 is based on the positional relationship between the feature points of the driver 100 (for example, feature points aj15 and aj16) and the feature points j1 to j20 of the passenger 110. It is determined whether or not it is a contact action.

  For example, as illustrated in FIG. 5, the determination unit 23 determines that the feature point j5 on the right hand of the passenger 110 overlaps the line connecting the feature points aj15 and aj16 on the left elbow and shoulder left of the driver 100. It is determined that it is a contact action.

  Thus, by using the passenger 110 directly touching the driver 100 as a trigger, it is possible to prevent erroneous detection of occurrence of damage behavior and to reliably detect damage behavior.

  Note that when the passenger pays the fare to the driver, money may be handed over to the driver, so it is desirable not to determine that this is a contact action. To do so, for example, if the passenger's hand touches the driver's hand while the vehicle is stopped, it is determined that the contact action is an act of paying the fare, and not a damage action. do it. Or, when the contact action is judged by the image, the voice is also taken into account, and a series of conversations related to payment of the fare (for example, “○○ yen.” “△△ yen please change.” “Yes, change If it is ")", it is sufficient not to determine that it is a damaging action.

  Note that the determination unit 23 is not limited to the case where the presence / absence of the contact action is determined based on the positional relationship between the feature points on the two-dimensional plane. For example, the depth of the captured image P calculated based on the attachment position of the imaging device 10 You may determine the presence or absence of a contact action based on the positional relationship of the feature point in the three-dimensional space which considered the direction. Thereby, the determination part 23 can determine a contact action more correctly.

  Moreover, although the determination part 23 determined the action which the passenger 110 touches the driver 100 directly as a damage action, it is not limited to this, Based on the action which the passenger 110 performed indirectly with respect to the driver 100 The presence or absence of damage behavior may be determined. This point will be described with reference to FIG.

  FIG. 6 is a diagram illustrating the determination contents of the determination unit 23. FIG. 6 shows a scene in which the passenger 110 seated on the rear seat kicks the seat on which the driver 100 is seated. As shown in FIG. 6, when the passenger 110 performs an action of kicking the seat relatively strongly (the driver 100 is surprised), the vehicle C often vibrates with the action.

  The determination unit 23 determines the presence / absence of the damage behavior by paying attention to the vibration. Specifically, as shown in FIG. 6, the determination unit 23 is an objective action that the action of the passenger 110 detected by the detection part 21 performs on the equipment of the vehicle C, and the vibration detection part 22 If the detected value of the detected vibration is equal to or greater than a predetermined value, this action is determined as a damage action.

  Then, for example, by recording the state of the objective action, when the equipment of the vehicle C is damaged, the recorded captured image P can be used as useful evidence when claiming damages.

  The objective action on the equipment of the vehicle C may be, for example, an action of hitting a door or window glass of the vehicle C other than the action of kicking the seat on which the driver 100 is seated. Any action that leads to damage to the equipment of the vehicle C may be used.

  Moreover, although the determination part 23 determined as a damage action in the case of the action which directly harms a person or an object like the action with respect to the driver | operator 100 or the equipment of the vehicle C, for example, the passenger 110 The damage behavior may be determined based on the possessed tool.

  Specifically, when the behavior of the passenger 110 detected by the detection unit 21 is possession of a tool for causing damage to the driver 100, the determination unit 23 determines that the behavior is a damage behavior. To do.

  For example, the detection unit 21 detects possession of a sharp metal member such as a blade based on the captured image P. More specifically, the detection unit 21 detects possession of the blade by detecting reflection of light with a metal member. In addition, the detection part 21 can also detect a blade by pattern matching, for example.

  The detection unit 21 is not limited to a sharp metal member tool, and may detect possession of a tool that does not have a blade such as a blunt tool or a tool that causes damage by flying an object to a target such as a gun.

  Thus, by detecting a tool that can cause damage to the driver 100, the driver 100 can be prevented from being injured by the tool.

  Further, the determination unit 23 may determine the presence / absence of the damage behavior in consideration of the traveling speed of the vehicle C. Specifically, the determination unit 23 sets the detection sensitivity of the detection unit 21 to be high at the right or left hand of the passenger 110 when the traveling speed of the vehicle C is zero, that is, when the vehicle C is stopped.

  As a result, for example, when the passenger 110 refuses to pay the fare and performs an action that causes harm to the driver 100 in a taxi or the like, such action can be detected quickly and notified to the driver 100. Therefore, the driver 100 can easily escape from the passenger 110 who is trying to cause harm.

  Further, for example, the determination unit 23 may determine the damage behavior taking into account the biological information of the driver 100. Specifically, the determination unit 23 learns biological information such as a normal pulse and body temperature for each driver 100, for example, and the biological information becomes an abnormal value due to the behavior of the passenger 110. In addition, such an action is determined as a damage action.

  This is because the degree of damage caused by the behavior of the passenger 110 is different for each driver 100. That is, the determination unit 23 can perform appropriate determination processing according to each of the drivers 100 by learning in advance the normal biological information of the driver 100 in advance.

  The information generation unit 24 generates damage occurrence information indicating the occurrence of damage behavior when the determination unit 23 determines that the behavior of the passenger 110 is damage behavior. For example, the information generation unit 24 generates warning information for warning the passenger 110 as damage occurrence information.

  Specifically, the information generation unit 24 generates warning information for emitting sound or light to the passenger 110 and outputs the warning information to the warning device 20. Note that the sound generated by the warning device 20 is not limited to a single sound such as a beep sound, but may be a sound.

  As a result, it is possible to stop the damaging behavior performed by the passenger 110, or to conceal the passenger 110 with sound or light and give the driver 100 a chance to escape.

  For example, if the passenger 110 continues the damage action even if the warning device 20 gives a warning, the information generation unit 24 outputs the damage occurrence information to the notification device 30 and sends it to the related organizations such as the police. An emergency call may be made.

  Further, the information generation unit 24 may generate the damage occurrence information by recording the state in the vehicle at the time of the damage action. Specifically, the information generation unit 24 causes the storage unit 3 to store the captured image P for a predetermined period including the time when the damage action occurred as damage occurrence information. Note that the recording method may be either a trigger type or a constant recording type.

  For example, in the case of the trigger type, the time-series captured image P is stored endlessly in a volatile memory such as a RAM. When the determination unit 23 determines that there has been a damaging action (trigger), the information generation unit 24 extracts the captured images P for a predetermined period before and after the current time from the volatile memory and saves them in a nonvolatile memory such as an HDD. To do. Thereby, the captured image P captured in the period before and after the damage occurrence time is stored.

  Further, for example, in the case of the continuous recording type, the time-series captured images P are stored in a nonvolatile memory such as an HDD, and when the memory capacity is full, the captured images P at the oldest time are sequentially deleted and new The captured image P at the time is overwritten and saved.

  The information generation unit 24 performs a setting for prohibiting overwriting of the captured image P captured in a period before and after the damage occurrence time when the determination unit 23 determines that there is a damage behavior. As a result, the captured image P for which overwriting is prohibited is stored by being left without being overwritten and deleted when the captured image P at a new time is stored.

  Thus, for example, by recording the situation at the time of occurrence of the damaging behavior, it can be submitted as useful evidence when the driver 100 is injured and requests a reward or the like.

  Next, a processing procedure of processing executed by the driving support device 1 according to the embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating a processing procedure of processing executed by the driving support device 1 according to the embodiment.

  As illustrated in FIG. 7, first, the detection unit 21 detects the behavior of the passenger 110 with respect to the driver 100 of the vehicle C based on the captured image P obtained by imaging the interior of the vehicle C (step S101).

  Subsequently, the determination unit 23 determines whether or not the behavior detected by the detection unit 21 is a contact behavior that touches the driver 100 (step S102). When the behavior detected by the detection unit 21 is a contact behavior that touches the driver 100 (step S102, Yes), the determination unit 23 determines that the behavior is a damage behavior (step S103).

  Subsequently, when the determination unit 23 determines that the behavior is a damage behavior, the information generation unit 24 generates damage occurrence information indicating the occurrence of the damage behavior (step S104), and ends the process.

  On the other hand, in step S102, when the behavior detected by the detection unit 21 is not a contact behavior that touches the driver 100 (No in step S102), the determination unit 23 performs the objective that the behavior performs on the equipment of the vehicle C. It is determined whether or not it is an action (step S105).

  Subsequently, when the behavior detected by the detection unit 21 is an objective behavior to be performed on the equipment of the vehicle C (step S105, Yes), the determination unit 23 detects the vibration detected by the vibration detection unit 22. Is determined to be greater than or equal to a predetermined value (step S106).

  If the detection value of the vibration detected by the vibration detection unit 22 is equal to or greater than the predetermined value (Yes at Step S106), the determination unit 23 proceeds to Step S103.

  On the other hand, in step S105, the determination unit 23 determines that the behavior detected by the detection unit 21 is not the objective behavior performed on the equipment of the vehicle C (No in step S105), and the behavior is directed to the driver 100. It is determined whether or not he / she possesses a tool for causing damage (step S107).

  If the behavior detected by the detection unit 21 is possession of a tool for causing damage to the driver 100 (step S107, Yes), the determination unit 23 proceeds to step S103.

  On the other hand, in step S106, the determination part 23 transfers a process to step S101, when the detected value of the vibration detected by the vibration detection part 22 is less than predetermined value (step S106, No).

  In step S107, if the action detected by the detection unit 21 is not possession of a tool for causing damage to the driver 100 (No in step S107), the determination unit 23 proceeds to step S101. Transition.

  As described above, the driving support device 1 according to the embodiment includes the detection unit 21, the determination unit 23, and the information generation unit 24. The detection unit 21 detects the behavior of the passenger 110 with respect to the driver 100 of the vehicle C based on the captured image P obtained by capturing the interior of the vehicle C. The determination unit 23 determines whether or not the behavior detected by the detection unit 21 is a damaging behavior that causes damage to the driver 100. The information generation unit 24 generates damage occurrence information indicating the occurrence of the damage behavior when the determination unit 23 determines that the action is damage behavior. As a result, for example, by warning the passenger 110 or reporting to the police, damage to the driver 100 can be minimized.

  In the above-described embodiment, the case where the passenger 110 is present in the rear seat has been described. However, for example, the passenger 110 may be present in the passenger seat. Further, the number of passengers 110 is not limited to one, but may be plural.

  Further effects and modifications can be easily derived by those skilled in the art. Thus, the broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

DESCRIPTION OF SYMBOLS 1 Driving assistance device 2 Control part 3 Memory | storage part 10 Imaging device 11 Microphone 12 Position acquisition part 13 Vehicle speed sensor 14 Acceleration sensor 20 Warning device 21 Detection part 22 Vibration detection part 23 Determination part 24 Information generation part 30 Information apparatus 31 Image information 32 Damage Action information 100 Driver 110 Passenger C Vehicle P Captured image

Claims (9)

A detection unit for detecting a passenger's behavior with respect to a driver of the vehicle based on a captured image obtained by capturing an image of the interior of the vehicle;
A determination unit that determines whether the behavior detected by the detection unit is a damaging behavior that damages the driver;
A driving support apparatus comprising: an information generation unit that generates damage occurrence information indicating occurrence of the damage behavior when the determination unit determines that the damage behavior is the damage behavior. The detector is
Detecting the behavior by detecting a feature point indicating the passenger's body from the captured image;
The determination unit
The driving support device according to claim 1, wherein when the pattern of the feature points indicating the behavior matches a predetermined pattern of the damage behavior, it is determined that the behavior is the damage behavior. The determination unit
The driving support device according to claim 1 or 2, wherein when the behavior detected by the detection unit is a contact behavior touching the driver, the behavior is determined as the damage behavior. A vibration detection unit that detects vibrations of the vehicle accompanying the action;
The determination unit
When the action is an objective action to be performed on the equipment of the vehicle and the detected value of the vibration detected by the vibration detection unit is a predetermined value or more, the action is determined as the damage action. The driving support apparatus according to any one of claims 1 to 3, wherein The detector is
Detecting possession of a tool for causing damage to the driver as the action,
The determination unit
The driving support device according to any one of claims 1 to 4, wherein possession of the tool is determined as the damage behavior. The detector is
Detecting voice indicating the passenger's utterance content as the action,
The determination unit
6. The voice according to claim 1, wherein the voice is determined to be the damaging behavior when a predetermined risk keyword is included in the voice detected by the detection unit. Driving assistance device. A storage unit that stores the time-series captured images;
The information generator is
The driving support device according to any one of claims 1 to 6, wherein the captured image of a predetermined period including a time when the damage behavior occurs is stored in the storage unit as the damage occurrence information. The information generator is
The driving support device according to any one of claims 1 to 7, wherein warning information for warning the passenger is generated as the damage occurrence information. A detection step of detecting a passenger's behavior with respect to the driver of the vehicle based on a captured image obtained by capturing an image of the interior of the vehicle;
A determination step of determining whether the behavior detected by the detection step is a damaging behavior that damages the driver;
An information generation step of generating damage occurrence information indicating the occurrence of the damage behavior when the damage determination is determined by the determination step.

Images