JP2009098900A - Aside look status determination device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a looking aside status determination device which can determine more accurately whether a driver looks aside. <P>SOLUTION: The looking aside status determination device 100 includes: an observation amount acquisition means 10 for acquiring at least one piece of information out of information on line of sight, information on operation and information on traveling condition of a driver as an observation amount; a looking aside status determination means 12 for determining a looking aside status based on a stochastic relation between the looking aside status and the observation amount and the observation amount acquired by the observation amount acquisition measure 10. The looking aside status determination means 12 computes a looking aside probability of the driver and determines that the driver is looking aside when the computed probability exceeds a threshold. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an armpit state determination device that determines a driver's armpit state, and in particular, acquires at least one of information related to the driver's line of sight, information related to driving operation, information related to driving state, and the like as an observation amount, and is known. The present invention relates to a side-by-side state determination device that determines a current side-by-side state based on a stochastic relationship between a side-by-side state and a known observation amount and a current observation amount.

  Conventionally, a driver who determines whether or not the driver is looking aside based on the frequency according to the direction of the gaze direction of the driver within a predetermined time in the past and the frequency according to the angular velocity of the steering angular velocity within a predetermined period in the past. A state detection device is known (for example, refer to Patent Document 1).

  This driver state detection device detects the vehicle position when there is another vehicle in the adjacent lane and the driver is trying to maintain the lane, and the driver is driving without looking aside. Steering operation is relatively frequent and steering angular velocity is relatively large due to a strong awareness of keeping constant, while steering operation is relatively slow when the driver is driving while looking aside. If the precondition that the steering angular velocity becomes relatively small and the direction frequency of the driver's line of sight in the past predetermined time becomes a predetermined state, the steering angular velocity in the past predetermined period is set. It is determined whether the driver is looking aside based on the frequency according to the steering angular velocity.

As described above, the driver state detection device determines whether or not a specific precondition set in advance based on the statistical value of the steering angular velocity in a predetermined period is satisfied, so that a predetermined time period in the past can be satisfied. When the frequency according to the direction of the driver's line of sight is in a predetermined state, it is determined with high accuracy whether or not the driver is driving aside.
JP 2003-80969 A Yoshihiro Suda, Yoshitoshi Takahashi, Masaaki Onuki, “Universal Driving Simulator for Research”, Automotive Technology, Vol.59, No.7, (2005), pp.83-88, Automotive Engineering Society Okakane, Yusuke Kanno, Yoichi Sato, “Real-time Head Posture Estimation with Automatic Construction of Head Deformation Model”, IPSJ Transactions on Computer Vision and Image Media, Vol. 47, No. SIG10 (CVIM15), pp.185-194

  However, since the driver state detection device described in Patent Literature 1 defines specific preconditions regarding observable data (steering angular velocity) in order to determine the presence or absence of looking aside, it does not apply to such specific preconditions. The steering operation cannot be handled, and it is determined that the driver is driving aside while the driver is not looking aside. Conversely, the driver is driving aside. However, it is determined that the driver is not driving aside.

  In view of the above-described points, an object of the present invention is to provide a look-ahead state determination device that can more accurately determine whether or not a driver is looking aside.

  In order to achieve the above-described object, the aside look state determination device according to the first aspect of the invention acquires information relating to the driver's line of sight and at least one of information relating to the driving operation or information relating to the driving state as an observation amount. An observation amount acquisition unit, and a stochastic state determination unit that determines a state of looking aside based on a stochastic relationship between the observation state and the observation amount and the observation amount acquired by the observation amount acquisition unit. .

  Further, the second invention is a side-by-side state determination device according to the first invention, wherein the side-by-side state determination unit calculates a probability that the driver is looking aside, and the calculated probability exceeds a threshold value. In this case, it is determined that the driver is looking aside.

  Further, the third invention is a side-by-side state determination device according to the second invention, wherein the side-by-side state determination unit calculates a probability that the driver is looking aside using a Bayesian network. To do.

  Further, the fourth invention is an armpit state determination device according to the second or third invention, wherein the armpit state determination means learns a transition model and an observation model in advance, and the transition model and the observation model And a probability that the driver is looking aside based on the observation amount acquired by the observation amount acquisition means.

  A fifth aspect of the invention is a side-viewing state determination device according to any one of the first to fourth aspects of the invention, wherein the information related to the driving operation is information related to a steering angle, an accelerator depression amount, a brake depression amount, or a blinker operation. It is characterized by including.

  According to a sixth aspect of the present invention, there is provided an armpit state determination apparatus according to any one of the first to fourth aspects of the present invention, wherein the information related to the traveling state includes a vehicle speed or acceleration.

  The seventh invention is an armpit state determination device according to any one of the first to fourth inventions, wherein the information on the running state is a distance between the vehicle and the center line, or the vehicle and the center line. Including the angle between.

  An eighth aspect of the invention is an armpit state determination device according to any one of the first to fourth aspects of the invention, wherein the information related to the running state includes information related to the surrounding environment.

  According to a ninth aspect of the present invention, there is provided a side-viewing state determination device according to any one of the first to fourth aspects, wherein the information relating to the driver's line of sight includes a head inclination angle.

  With the above-described means, the present invention can provide an aside look state determination device that can more accurately determine whether or not the driver is looking aside.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of an aside look state determination device according to the present invention. A look aside state determination device 100 receives a control unit 1, devices 20 to 28 for acquiring various observation amounts, and driver input. This is an in-vehicle device that includes an input device 29 and a sound output device 30 that outputs various sounds.

  Here, the “observation amount” is an amount that can be observed as a numerical value. For example, the “observation amount” is classified into the categories “driving operation”, “vehicle state”, or “driver information” as shown in FIG. “Steering angle”, “accelerator depression amount”, “brake depression amount” and “direction indicator setting” in category, “speed”, “acceleration”, “distance between vehicle and centerline” in “vehicle state” category And “the angle of the vehicle with respect to the center line”, and “the head yaw angle (which refers to the rotation angle around the vertical axis of the driver's head”) in the “driver information” category.

  “Observed amount” includes accelerator depression speed, brake depression speed, steering angular speed, vehicle position (latitude, longitude, altitude), road type (straight road, curved road, near intersection, etc.), and surroundings. Environment (there are busy streets, scenic spots, or highways that are difficult to induce side effects), the presence or absence of pedestrians, the speed and acceleration of other vehicles, and the distance between your vehicle and other vehicles , Head pitch angle (refers to the rotation angle around the axis of the driver's head extending in the vehicle lateral direction), head roll angle (refers to the rotation angle of the driver's head around the axis extending in the vehicle longitudinal direction) Etc.), and any amount that can affect the look-ahead state may be included.

  The “observed quantity” may be a continuous value (analog data), but is a discrete value obtained by dividing the continuous value into a plurality of stages (digital data. For example, when the observed quantity is “speed”, (5, 10 and 15 are classified at every 5 km / h) and a plurality of indices (for example, “1” is set as an index of a straight road in a road type, and “2” is set as an index of a curved road in a road type. Or “.” May be adopted.

  Note that the aside look state determination apparatus 100 preferably handles various observation amounts by dividing them into 12 discrete values. This is because if the number of steps is small, it is difficult to extract the characteristics of various observation quantities, while if the number of steps is large, mixing of noise (abnormal values) cannot be ignored.

  The looking-ahead state determination device 100, for example, continuously acquires various observation amounts related to the driver's head yaw angle and the looking-aside state, and various types when the head yaw angle becomes a predetermined angle or more. Based on the instantaneous values and statistical values of observations (mean values, maximum values, minimum values, variances, intermediate values, etc.) and predetermined transition models and predetermined observation models, Express with conditional probability.

Here, the “transition model” is a conditional probability distribution of each state when two or more states transition to each other. For example, the time t + 1 when the state is not an aside state at the time t (X _t = OFF). Is a conditional probability distribution that results in an aside look state (X _{t + 1} = ON), and is expressed as P (X _{t + 1} = ON | X _t = OFF).

In addition, the “observation model” is a conditional probability distribution of the observation amount. For example, the observation amount Z _t ^(k) becomes a predetermined value z in a state of looking aside (X _t = ON) at time t. A conditional probability distribution, expressed as P (Z _t ^(k) = z | X _t = ON). Z ^(k) represents the value of the k-th observed amount of N types of observed amounts.

  Moreover, the aside look state determination apparatus 100 prepares, for example, a transition model and an observation model in advance, and the transition model and the observation model use, for example, a driving simulator (see, for example, Non-Patent Document 1). Learned in advance for each driver.

  FIG. 3 is a schematic diagram illustrating a configuration example of a driving simulator. FIG. 3A illustrates a case where a subject (driver) DR is operated by various operation units (not shown, an accelerator pedal, a brake pedal, a steering, etc.) of the vehicle model VC. It is shown that the user can experience the driving of the vehicle in a pseudo manner while viewing the video on the screen SC linked to the screen SC.

  FIG. 3A also shows that the driving simulator DS includes the driver photographing camera 21 in the front of the driver's seat and a verification camera MC for verifying the driving operation of the subject DR in the upper rear of the driver's seat. It shows that.

  FIG. 3A shows that the driving simulator DS includes projectors P1 and P2 for displaying a target image TG that induces the driver DR to look aside, on the left and right of the rear seat. It is assumed that the target image TG cannot be visually recognized unless the head is shaken (the head yaw angle is not equal to or larger than the predetermined angle φ).

  In this way, the driving simulator DS uses the conventional technique (see, for example, Non-Patent Document 2) from the image acquired by the driver photographing camera 21, and is considered to best reflect the look-ahead state. Make the corners detectable.

  The driving simulator DS randomly determines the display frequency of the target image TG, the display time, the display position of the target image TG, the type of the target image TG, and the like. For example, the display frequency is 2.2 times per minute and the display time is set. The projection image PG including the target image TG is projected onto the left and right wall surfaces W1 and W2 using the left and right projectors P1 and P2, respectively, while the target image TG is set to 50 types for 5 seconds each time.

  FIG. 3B shows an example of a projection image PG projected on the right wall surface W2, and shows a state in which the projection image PG includes a target image TG in the shape of a pedestrian in the left side area.

  In addition to the pedestrian, the driving simulator DS may use, as the target image TG, an image that attracts the subject's DR, such as a road sign or a four-character idiom. In order to naturally look aside, it may be notified by voice whether the target image TG is displayed in the left or right direction.

  Further, the driving simulator DS defines a state in which the head yaw angle of the subject DR is φ (for example, ± 10 °) or more and the target image TG is displayed as a state in which a side look is occurring. Then, the subject DR is caused to repeatedly perform the pseudo driving a predetermined number of times (this action is called supervised learning).

  It should be noted that the state in which a side look is occurring may be defined using an observation amount other than the head yaw angle, and the state in which no side look is occurring may be defined using an arbitrary observation amount. (For example, even if the head yaw angle is equal to or greater than φ, a state in which the steering angle is equal to or greater than a predetermined angle is defined as a state in which a side look has not occurred.)

  The driving simulator DS learns (generates) the transition model and the observation model based on the various observation amounts acquired according to the above-described method.

  The transition model and the observation model are derived based on the number of frames (referred to as a predetermined time unit, for example, a frame in terms of a shooting unit (frame / second) of the driver shooting camera 21). Is done.

  Specifically, the transition model is represented by the equation (1), and the observation model is represented by the equation (2).

なお、ｆｒａｍｅ｛ｙ｝は、条件ｙを満たすフレームの数を意味する。

Note that frame {y} means the number of frames that satisfy the condition y.

  Here, with reference to FIG. 1 again, the components of the looking-aside state determination apparatus 100 will be described.

  The control unit 1 is a computer including a CPU (Central Processing Unit) and a storage unit 15 such as a RAM (Random Access Memory) and a ROM (Read Only Memory). While storing the program corresponding to each of the learning means 11 and the looking-aside state determination means 12 in the storage unit 15, the CPU is caused to execute processing corresponding to each means.

  The storage unit 15 stores, for example, the transition model 150 and the observation model 151 learned using the driving simulator DS for each driver, and the transition model 150 and the observation model 151 corresponding to each driver driving the vehicle. Can be used selectively by the aside state determination apparatus 100.

  The road photography camera 20 is a camera for recognizing lane markings, road markings, and the like installed on the road. Take a picture.

  The driver photographing camera 21 is a camera for photographing the driver, and is installed, for example, in an instrument panel or a dashboard, and photographs the driver's head. The driver photographing camera 21 preferably has the same arrangement and the same type as those used in the driving simulator DS. This is because the transition model 150 and the observation model 151 obtained by the driving simulator DS can be used effectively.

  The steering angle sensor 22 is a sensor for measuring the rotation angle of the steering shaft related to the steering angle of the wheel. For example, the steering shaft reads a magnetic resistance by a magnet embedded in the steering shaft by an MR (Magnetic Resistance) element. And the detection result is output to the control unit 1.

  The accelerator depression amount sensor 23 is a sensor for detecting the accelerator depression amount. For example, the accelerator depression amount sensor 23 detects the depression amount of the accelerator pedal using a potentiometer using a variable resistor, and outputs the detection result to the control unit 1.

  The brake depression amount sensor 24 is a means for detecting the depression amount of the brake, like the accelerator depression amount sensor 23.

  The direction indicator 25 is a device for notifying the traveling direction of the host vehicle to the outside. For example, the blinker lamp blinks to notify the traveling direction of the host vehicle to the outside, and an electric signal indicating the state is sent. Output to the control unit 1.

  The speed sensor 26 is a sensor that measures the speed of the vehicle. For example, the MR element reads a change in magnetic field caused by a magnet attached to each wheel and rotates with each wheel as a magnetic resistance, and this is a pulse signal proportional to the rotation speed. As a result, the rotational speed of the wheel and the speed of the vehicle are detected, and the detection result is output to the control unit 1.

  The acceleration sensor 27 is a sensor for detecting the acceleration of the vehicle. For example, the acceleration sensor 27 detects the acceleration in the three axial directions of the vehicle in the front-rear direction, the vertical direction, and the left-right direction using a semiconductor strain gauge. Output to 1.

  The navigation device 28 guides the vehicle while showing the route to the destination based on the vehicle position information acquired by the GPS (Global Positioning System) function and the map information stored in the hard disk or DVD. It is a device, and the location of the vehicle (latitude, longitude, altitude), the type of road on which the vehicle travels (straight roads, curved roads, intersections, etc.), or the surrounding environment (a busy street that tends to induce side effects) , Etc.), information relating to a scenic spot or a highway that is difficult to induce aside is output to the control unit 1.

  The input device 29 is a device for receiving an input from the driver. For example, the input device 29 is a steering switch disposed on the steering wheel, and the driver provides feedback as to whether or not the determination result by the aside look determination device 100 is correct. The input information by the driver is output to the control unit 1.

  The voice output device 30 is a device for outputting a voice message, a warning, or the like. For example, the voice output device 30 is an in-vehicle speaker or an alarm, and when the driver determines that the driver is looking aside, In response to a control signal sent from the control unit 1, a warning message or an alarm sound is output.

  Next, various units included in the control unit 1 will be described.

  The observation amount acquisition means 10 is a means for acquiring various observation amounts, and stores the acquired observation amounts in the storage unit 15 while continuously acquiring outputs from the devices 20 to 28 for acquiring various observation amounts. To do. Note that the observation amount acquisition unit 10 may delete or discard the observation amount after reflecting the acquired latest observation amount in the statistical value.

  Moreover, the observation amount acquisition means 10 may be able to use all of the observation amounts acquired during the entire period from the start of operation to the current time, or can use only the observation amounts acquired in the past predetermined period. You may do it.

  The observation amount acquisition means 10 receives, for example, a road image output from the road photographing camera 20 and performs image processing such as binarization processing and edge detection processing to extract road markings, lane boundary lines, and the like. To do.

  Furthermore, the observation amount acquisition means 10 calculates the distance between the own vehicle and the lane boundary line from the relationship between the extracted lane boundary line and the known camera position, or forms the own vehicle and the lane boundary line. Calculate the angle to perform.

  The observation amount acquisition unit 10 receives, for example, an image of the driver's head output from the driver photographing camera 21 and uses a known head posture estimation method (see, for example, Non-Patent Document 2). The posture of the head is detected, and the yaw angle, pitch angle, and roll angle of the driver's head are derived.

  Further, the observation amount acquisition means 10 receives an image of the driver's face output from the driver photographing camera 21, and performs image processing such as binarization processing and edge detection processing to thereby make eyes, nose, The driver's line-of-sight direction may be estimated by extracting the position of the mouth or extracting the position of the pupil in the eye.

  The looking-aside state learning means 11 is a means for learning the transition model 150 and the observation model 151. For example, after the aside-looking state determination device 100 determines that the looking-aside state is once aside and outputs a warning from the voice output device 30 When the fact that the driver is not looking aside is detected afterward through the input device 29, the transition model 150 and the observation model 151 reflect that the determination result is incorrect.

  Further, the side-by-side state learning means 11 updates the transition model 150 and the observation model 151 for each driver after identifying the driver by a driver identification means such as a fingerprint authentication device, an iris authentication device, or a password. To do. This is because the transition model 150 and the observation model 151 are prepared while incorporating the features of each driver, thereby further improving the accuracy of the side-by-side determination by the side-by-side state determination unit 12 described later.

  In this way, the looking-aside state learning unit 11 can continuously improve the accuracy of looking aside by the looking-ahead state determination device 100.

  The looking-aside state determining means 12 is a means for determining whether or not the person is looking aside. For example, using a dynamic Bayesian network (hereinafter referred to as “DBN”), various observations obtained up to the present time are used. Based on the quantity, the probability of looking aside at the current time is derived, and when the derived probability is equal to or greater than a predetermined value ξ (for example, 0.8), it is determined that the person is looking aside.

  Note that the predetermined value ξ is, for example, an evaluation of the aside look determination accuracy based on the leave-one-out method for a plurality of simulation results in the driving simulator DS, and a TP (True Positive: true positive rate). FP (False Positive: a false positive rate, and a ratio to determine a non-aware state as an aside), FN (False Negative: a false negative rate) It is determined in consideration of the values of the ratio of determining that the aside is not looking aside) and TN (True Negative: the probability of determining that the aside is not aside).

  The DBN is a hidden state variable (a variable that cannot be directly acquired from the history of observation amounts, for example, a state of looking aside) and an observation variable (a variable that can be acquired directly from the history of observation amounts). For example, when there is a stochastic relationship with the head yaw angle, velocity, acceleration, etc.), this is a technique for describing the relationship using a directed graph.

  The DBN calculates the likelihood of the value of the hidden state variable at that time from the history of the state of the observed variable up to a certain time by learning quantitatively assuming this relationship as a steady state.

  In the present embodiment, it is assumed that the hidden state variable (armpit state) follows a first-order Markov process, and the hidden state variable (armpit state) at each time affects the observed variable at each time.

  FIG. 4 is a conceptual diagram showing the DBN network topology. Transitions of hidden state variables according to the first-order Markov process are represented by broken-line arrows, and hidden state variables at each time (side-viewed state) are observed variables at each time. The influence is represented by solid arrows.

Here, X _t and Zt (Zt = {Z _t ⁽¹⁾ ,..., Z _t ^(N) }) are respectively the value of the state variable (side-view state) at time t and the observed variable at time t ( head yaw angle, velocity, indicates the value of the acceleration, etc.), X _t takes a value of either ON (with which) or OFF (not looking aside and the inattentive), N is the observed variables The number of types. Each observation variable Z _t ^(k) has a discrete value obtained by discretizing a continuous quantity, and the discrete value of each observation variable is divided into, for example, 12 stages.

The expression (3) is based on the information Z _{1: t} related to all observation amounts from the time when the observation amount acquisition is started until the time _{t, and the} a posteriori probability P (X _t | Z _{1: t} of the side-view state X _{t at} the time t. The DBN derives the probability of looking aside when each observation amount has a predetermined value using the equation (3). The “posterior probability” is a kind of conditional probability, and in the present embodiment, represents the likelihood of looking aside, taking into account the already obtained observation amount.

ここで、αは、Ｐ（Ｚ_ｔ＋１）を示す定数であり、Ｐ（Ｚ_ｔ＋１ ^（ｋ）｜Ｘ_ｔ＋１）は、記憶部１５に記憶された観測モデル１５１を用いて決定され、Ｐ（Ｘ_ｔ＋１｜ｘ_ｔ）は、記憶部１５に記憶された遷移モデル１５０を用いて決定される。

Here, α is a constant indicating P (Z _{t + 1} ), and P (Z _{t + 1} ^(k) | X _{t + 1} ) is determined using the observation model 151 stored in the storage unit 15, and P (X _{t + 1} | X _t ) is determined using the transition model 150 stored in the storage unit 15.

Therefore, the a posteriori probability P (X _t = ON | Z _{1: t} ) at the current time is P (x _t | Z _{1: t} ^(k) based on the observation amount Z _{1: t} observed so far. ) Is derived.

The looking-aside state determination unit 12 uses the DBN as described above, and the a posteriori probability P (X _t = ON | Z _{1: t} ) of the looking-aside at the current time and a predetermined value ξ (for example, 0.8). And the posterior probability P (X _t = ON | Z _{1: t} ) is equal to or greater than a predetermined value ξ, the current state (in this case, the state where the head yaw angle is equal to or greater than φ). Is determined to be an aside.

  Next, with reference to FIG. 5, a description will be given of a process in which the aside look determination device 100 alerts the driver's aside (hereinafter referred to as “aside look alerting process”). FIG. 5 is a flowchart showing the flow of the aside look alerting process, and the aside look state determination device 100 repeatedly executes the aside look alerting process while the vehicle is traveling at a predetermined speed or higher.

  First, the control unit 1 of the side-by-side state determination apparatus 100 uses the side-by-side state determination unit 12 to calculate the a posteriori probability of looking aside at the current time using the DBN (step S1).

  Thereafter, the control unit 1 derives the yaw angle of the driver's head by receiving the image output from the driver photographing camera 21 by the observation amount acquisition means 10, and the yaw angle of the head and the predetermined angle φ. Are compared (step S2).

  When the yaw angle of the head is less than the predetermined angle φ (NO in step S2), the control unit 1 once ends the aside look alerting process. This is because it is not necessary to call attention if the yaw angle of the head is less than the predetermined angle φ.

  When the head yaw angle is equal to or larger than the predetermined angle φ (YES in step S2), the control unit 1 compares the calculated posterior probability with the predetermined value ξ by the look-ahead state determination unit 12 (step S3). If the calculated posterior probability is less than or equal to the predetermined value ξ (NO in step S3), the aside look alerting process is temporarily terminated. This is because if the calculated posterior probability is low, it is not necessary to call attention, and this is to prevent an alarm from being output when no warning is required.

  When the calculated posterior probability is greater than the predetermined value ξ (YES in step S3), the control unit 1 outputs a control signal to the audio output device 30 to output an alarm, and alerts the driver to stop looking aside. Above (step S4), the aside look alerting process is terminated.

  In the aside look alerting process shown in FIG. 5, the a posteriori probability of looking aside at the current time may be calculated using DBN only when the head yaw angle is equal to or greater than the predetermined angle φ.

  When the head yaw angle derived by the observation amount acquisition means 10 is treated as a discrete value and the predetermined angle φ is included in the middle of one section (for example, when the section is set in increments of ± 4 °) And the predetermined angle φ (± 10 °) is included in the range of ± 8 ° to ± 12 °.) At this time, the head yaw angle ± 8 ° and the head This is because the yaw angle ± 10 ° is treated as the same discrete value. This means that it may be determined that the head is looking aside even if the head yaw angle is less than the predetermined angle φ.

  With the above-described configuration, the aside look state determination device 100 uses the stochastic relationship between the various observation amounts and the look aside state to calculate the aside look probability of the aside state that cannot be observed directly, while the driver looks aside. Therefore, it is possible to more accurately determine the presence or absence of looking aside.

  Moreover, since the armpit state determination apparatus 100 determines the armpit state using a stochastic relationship between various observation amounts and the armpit state, between the various observation amounts and the armpit state that cannot find a law property at first glance. Can be flexibly dealt with, and the presence or absence of looking aside can be determined more accurately.

  Further, the armpit state determination apparatus 100 is more suitable for a safety check at the time of turning left or right or when changing lanes. It is possible to accurately discriminate, and it is possible to prevent an alarm from being output (alarm output based on an erroneous determination) when the line of sight deviates from the front although not looking aside.

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-described embodiment, the stochastic relationship between the observation amount output from the devices 20 to 28 and the aside state is described in DBN, and it is determined whether the aside is aside by calculating the a posteriori probability of the aside. However, the side-by-side state determination device according to the present invention further provides another observation amount such as the inter-vehicle distance output by the radar sensor and the type of lane output by the road-to-vehicle communication device (such as a traveling lane or an overtaking lane). It is also possible to determine whether or not the person is looking aside while inputting.

  Further, the side-by-side state determination device according to the present invention may determine whether or not a side-by-side is using another probability model such as a neural network.

It is a block diagram which shows the structural example of the looking-aside state determination apparatus which concerns on this invention. It is an example of the classification table of the observation amount which a look-aside state determination apparatus acquires. It is the schematic which shows the structural example of a driving simulator. It is a conceptual diagram which shows the network topology of DBN. It is a flowchart which shows the flow of an armpit attention alert process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Control part 10 Observed amount acquisition means 11 Aside look state learning means 12 Aside look state determination means 15 Memory | storage part 20 Camera for road photography 21 Camera for driver photography 22 Steering angle sensor 23 Acceleration depression amount sensor 24 Brake depression amount sensor 25 Direction indicator 26 Speed sensor 27 Acceleration sensor 28 Navigation device 29 Input device 30 Audio output device 100 Aside look state determination device 150 Transition model 151 Observation model DR Subject DS Driving simulator MC Verification camera P1, P2 Projector PG Projection image SC Screen TG Target image VC Vehicle Model W1, W2 Wall surface

Claims (9)

An observation amount acquisition means for acquiring, as an observation amount, information relating to the driver's line of sight and information relating to driving operation or information relating to the driving state;
A side-by-side state determination unit that determines a side-by-side state based on the stochastic relationship between the side-by-side state and the observation amount and the observation amount acquired by the observation amount acquisition unit;
A device for determining a state of looking aside. The aside state determination means calculates the probability that the driver is looking aside, and determines that the driver is looking aside when the calculated probability exceeds a threshold,
The side-by-side state determination apparatus according to claim 1. The aside look determination means calculates the probability that the driver is looking aside using a Bayesian network.
The side-by-side state determination apparatus according to claim 2. The armpit state determination means learns the transition model and the observation model in advance, and the probability that the driver is looking aside based on the transition model and the observation model and the observation amount acquired by the observation amount acquisition means. calculate,
The side-view state determination apparatus according to claim 2 or 3, The information related to the driving operation includes information related to a steering angle, an accelerator depression amount, a brake depression amount, or a winker operation.
The aside look state determination apparatus according to any one of claims 1 to 4. The information on the running state includes vehicle speed or acceleration,
The aside look state determination apparatus according to any one of claims 1 to 4. The information on the running state includes a distance between the vehicle and the center line or an angle between the vehicle and the center line.
The aside look state determination apparatus according to any one of claims 1 to 4. The information on the running state includes information on the surrounding environment.
The aside look state determination apparatus according to any one of claims 1 to 4. The information on the driver's line of sight includes the tilt angle of the head,
The aside look state determination apparatus according to any one of claims 1 to 4.

Images