JP2018144534A - Driving assist system, driving assist method and driving assist program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to confirm whether or not a driving operator is operating an abnormal system and whether or not a normal system is being operated correctly, and to enable better safe driving support. SOLUTION: A driver's voice, a gesture (gesture), information indicating the state of an operation target device and data for comparison in advance, the driver's voice, a gesture (gesture), and a state of the device are stored in a database. Driving support equipped with means including a function to compare the information indicating the above information and confirm the safety of whether or not the abnormal system is being operated and whether or not the normal system is being operated correctly based on the comparison result of the information. system. [Selection diagram] Fig. 1

Description

The present invention relates to a driving support system, a driving support method, and a driving support program.
More specifically, for example, a server (computer) having a function for judging safety of driving operation is provided, and a voice of a railway driver or a bus driver (hereinafter collectively referred to as a driver) and / or surroundings Recognize and analyze gestures and movements (gestures) such as voice recognition functions that recognize sounds and abnormal sounds of railway and bus facilities and output voice recognition information, and driver gestures, and obtain motion pattern analysis result information Using a combination of information including the motion gesture function to be output and information indicating the status of the operation target device at that time, the driver's driving operation content, that is, whether or not the driver is performing a safe driving operation (non-safe driving operation ) And output the information indicating the result. For example, the information indicating the result can reduce or improve the driving error of the driver, improve the accuracy of the driving operation, Supported by the system, method, and program.

  As a technique for supporting safe driving of a vehicle, for example, there is a technique described in Japanese Patent Application Laid-Open No. 2014-92804 (Patent Document 1). In this publication, “determination information for determining whether or not the driving operation in the vehicle during the determination period is ideal is acquired, and the vehicle at the guidance timing during the determination period is obtained based on the determination information. It is determined whether or not the driving operation of the vehicle is ideal, and when the driving operation of the vehicle at the guidance timing is ideal, guidance indicating that the driving operation is ideal at the guidance timing is performed. There is a description (see abstract).

JP 2014-92804 A

  The above-described conventional technology is, for example, a driving support system that compares an ideal acceleration curve of vehicle speed with actual speed, and, based on the result, prompts an operator to pay attention to driving operation with respect to the vehicle speed. According to such a technique, it is possible to “provide a guideline for continuously setting a driving operation in an ideal state”. However, the driver's voice, gestures, and movements that are not directly related to the operation of the machine including the operation target device that is operated when driving a vehicle such as a railroad or a bus are judged, and attention is paid to the safety of the driving operation. Is not taken into account. Therefore, it has not been possible to confirm whether the driving operator is operating an abnormal system or whether the normal system is operating correctly.

  Therefore, in the present invention, focusing on the fact that the driver's shout and the driver's gestures and movements that are not directly related to the operation of the machine are also useful for assisting safe driving, the driver does not operate the abnormal system. It is possible to confirm whether or not the normal system is operating correctly, and to enable more detailed safe driving support.

  In order to solve the above-described problems, the present invention provides, for example, audio information including a driver's voice, video information including motions of gestures and gestures (gestures) such as gestures, and information indicating a state of an operation target device. Each information is collected / recognized / analyzed, and the driver's voice recognition information, motion pattern analysis result information, and operation target device information registered and stored in the recording means such as a database as the safety judgment information in advance. Compared with the information indicating the status, it is confirmed from the comparison result of the information whether or not the driver is operating the abnormal system and whether the normal system is operating correctly. Means having a function of outputting the indicated information is provided.

One of the typical driving support systems, methods and programs of the present invention is:
Driving support provided with a server that receives information indicating the device status of the driving operation target device and the voice information and video information of the driver of the driving operation target device from the communication terminal on the driver's seat side of the moving object via the network A system,
Information acquisition means for acquiring information indicating the state of the audio information, the video information, and the operation target device;
Voice recognition means for recognizing voice information acquired by the information acquisition means and outputting voice recognition information;
Analyzing the video information acquired by the information acquisition means, and outputting motion analysis result information of the driver;
Storage means for registering speech recognition information, motion pattern analysis result information, information indicating the state of the target device, and information indicating a determination result as comparison information in advance,
The operation of the user who operates the operation target device is performed using the voice recognition information of the voice recognition unit, the motion pattern analysis result information of the motion analysis unit, and the information indicating the state of the operation target device of the data acquisition unit. Safety judgment processing means for judging safety of safety or not, and outputting information indicating a result of the judgment;
With
The safety determination processing means includes
Information indicating the state of the operation target device in the data acquisition unit, voice recognition information in the voice recognition unit, and motion pattern analysis result information in the motion analysis unit are set as input information, and the input information is previously stored in the storage unit as comparison information. Compared with information indicating the status of the operation target device registered as, voice recognition information, motion pattern analysis result information,
From the comparison result of the comparison, information indicating a state of the operation target device in the storage unit, voice recognition information, and information indicating a determination result registered corresponding to the motion pattern analysis result information are output. .

  According to the present invention, it is possible to confirm whether or not the driver is operating an abnormal system and whether or not the normal system is operating correctly, and better safe driving support is possible.

For example, the effects described below can be expected.
(1) Judging the details of operations that are linked to the voice and movement of railway drivers and bus drivers, and judging safety.
(2) Safety or non-safety (insufficient utterances, unclear utterances, insufficient pointing recognition, finger detection, etc.) by detecting a state of reduced safety due to the lack of basic actions such as voice only and gesture gestures. It is possible to reflect detailed results including over-representation) in safe driving.
Problems, configurations, and effects other than those described above will become apparent from the following description of embodiments.

FIG. 1 is an overall configuration diagram of a driving support system according to the present invention. FIG. 2 is a flowchart of the speech recognition process in the present invention. FIG. 3 is a flowchart of the motion analysis software in the present invention. FIG. 4 is a flowchart of the safety determination process in the present invention. FIG. 5 shows an embodiment of the driving support system according to the present invention. FIG. 6 is a diagram showing a data configuration example of the database in FIG.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

FIG. 1 is an overall configuration diagram of a driving support system according to the present invention.
The driving support system includes, for example, a voice collector M1 installed in a driver's seat of a moving body such as a railroad or a bus, and has a function of collecting voice of the driver U1 (user) and outputting voice information. A gesture detector C1 having a function of detecting a driver U1 (user) gesture and movement and outputting motion information of the driver, including a voice collecting device M and a gesture detector C1 A device status information acquisition unit L1 having a function of acquiring information indicating an operation target device A1 that is sometimes an operation target, information indicating the device status of the operation target device A1, for example, information indicating that the current state is “when departure” or “when traveling”; A communication terminal T1 having a communication function, a safety determination server S1 having a safety determination function, and a network N1 are included.

  The sound collector M1 of the sound collector M has, for example, at least a function of collecting voices (screams) produced by railway drivers, bus drivers, and passengers (including instructors and passengers). It consists of one or more microphones. In this example, only the voices of the driver, driver, and passengers are stored in the center, but other than voices, for example, detecting abnormal sounds of railway equipment such as pantographs, tracks, vehicles, etc. good.

  The gesture gesture detector C1 of the sensing device includes, for example, one or more motion sensors, a video camera (hereinafter referred to as a camera) for photographing the driver, and video (including image) information captured by the camera. A gesture sensing unit having a function of sensing movements (gestures) such as mouth movements, eye gaze, gesture gestures, and the like.

  The gesture detection unit has a gesture analysis function for analyzing the detected gesture, and is configured by, for example, a technology having an existing posture / gesture recognition function and face recognition function. Posture / gesture recognition techniques and techniques having face recognition functions are already known. For example, a technique for recognizing a human gesture / flow line is described in Japanese Patent Application Laid-Open No. 2004-303014, and detailed description thereof is omitted. By using posture and gesture recognition technology, it is possible to identify multiple people and analyze multiple gestures.

  The operation target device A1 is a machine including a target device operated by an operator. For example, a steering wheel, a brake, and a driving instrument that are directly related to the driving operation are not directly related to the driving operation, but a control device, a measuring instrument, and the like are required.

  The device state information acquisition unit L1 is connected to or embedded in the operation target device A1, and has a function of converting (characterizing) the state of the operation target device A1 into an electric signal and transmitting the electric signal to the communication terminal T1.

  The device state of the operation target device A1 is, for example, information indicating that the operation target device is traveling (during traveling), stopped (during departure), obstruction progress, relay progress, and the like, and the operation of the railway main control device. This is information (data) indicating the status, the status of the bus handle, and the like.

  Note that either a wired connection or a wireless connection may be used between the device state information acquisition unit L1 and the communication terminal T1. Further, in this example, the device is described as a device independent of the communication terminal T1, but the device state information acquisition unit L1 is implemented inside the communication terminal T1 as long as the state of the operation target device A1 can be acquired. May be.

  The communication terminal T1 receives information (data) collected, detected, and acquired by each unit of the sound collector M1, the gesture detection unit of the gesture gesture detector C1, and the device state information acquisition unit L1, and via the network N1. Information indicating the safety determination result transmitted to the safety determination server S1 and transmitted from the safety determination server S1 side, for example, “insufficient utterance”, “unspoken utterance”, “safety”, “pointing” It has a function of receiving information on character strings indicating “insufficient confirmation”, “excessive pointing”, “none”, etc., and displaying the information on the display unit.

The safety determination server S1 is composed of, for example, a computer, and includes voice information (voice data), gesture detection information (video data) transmitted from the communication terminal T1, and information indicating the device status of the target device (when departing / running) A communication device TR including a communication unit having a function of receiving information (data) such as information and transmitting information indicating a determination result, a voice recognition software unit P1 storing voice recognition software, and a motion analysis software are stored. An arithmetic device P including a motion analysis software unit P2, a safety determination processing unit P3 having a function of determining safety, a database (DB) D1 for registering each information, which is used in advance for comparison, and a storage device D including Have
The communication device TR, the arithmetic device P (speech recognition software unit P1, motion analysis software unit P2, safety determination processing unit P3), and the storage device D constitute a computer system.

  Each information registered in the database D1 is a voice recognition information for comparison (for example, a voice recognition character string or a phoneme) used for safety judgment in the safety judgment processing unit P3, a motion pattern analysis. Result information (for example, motion pattern string / motion analysis result character string) and information (data) indicating the device status (starting / running) of the target device. Moreover, it is information (for example, safety, lack of utterance, etc.) indicating what kind of determination result is obtained from the combination of each information (see FIG. 6).

The voice recognition software of the voice recognition software unit P1 recognizes voice as voice data transmitted from the communication terminal T1 via the network N1, and outputs the voice at that time as voice recognition information as a voice recognition result.
For example, the recognized speech data is converted into characters (characterized), a speech recognition character string is generated, and the speech recognition character string (speech recognition information) is output as a speech recognition result. The speech recognition character string of the speech recognition result is a value input to the safety determination processing unit P3. The processing procedure by the voice recognition software will be described later.

The motion analysis software of the motion analysis software part P2 performs motion analysis using the video data transmitted from the communication terminal T1 via the network N1 as an input value, and outputs the motion at that time as motion pattern analysis result information. To do.
For example, the driver's gesture and movement are analyzed, converted as character information (characterized), a motion pattern analysis result character string is generated, and the motion analysis result character string is output as a motion analysis result. The motion pattern analysis result character string of the motion analysis result is a value input to the safety determination processing unit P3. The processing procedure by the motion analysis software will be described later.

The safety determination processing unit P3 receives the network N1 from the speech recognition result (for example, speech recognition character string) in the speech recognition software, the motion pattern analysis result (for example, motion analysis result character string) in the motion analysis software, and the communication terminal T1. The device state (for example, character string) of the operation target device A1 transmitted through the server is used as an input value (input information), the database D1 is searched, compared with each information registered in the database, and the comparison result is obtained. , Whether or not the driving operation is safe by the driver, for example, “safety, non-safety (insufficient utterance, unclear utterance, insufficient pointing confirmation, excessive pointing)”, and information indicating the determination result ( For example, a character string) is output.
That is, the safety determination processing unit P3 is a speech recognition character string, motion pattern analysis result character string, operation, and the like, in which the results of the voice recognition software unit P1 and the motion analysis software unit P2 are registered in the database D1 of the storage device D in advance. When there is a character string that matches the database D1 as a result of the comparison with a character string indicating the state of the target device, the determination result character string registered corresponding to the character string is extracted.
The safety determination process is performed according to a program stored inside the arithmetic device P. The safety determination result is transmitted to the communication terminal T1 via the communication device and notified (reported) to the driver side. The safety judgment result may be registered in the storage device D so that it can be read and confirmed when necessary. A processing procedure in the safety determination processing unit P3 will be described later.

  The database D1 of the storage device D includes, for example, a speech recognition character string for comparison with a speech recognition result (speech recognition character) in the speech recognition unit, and a motion analysis result (motion pattern analysis result character string) in the motion analysis unit. A series of data (see FIG. 6) in which a motion pattern analysis result character string for comparison and a character string of a determination result corresponding to the voice recognition character string and the motion pattern analysis result character string are registered in time series are stored. Yes. When the determination process is performed by the safety determination processing unit P3, a search is performed in response to a request from the safety determination processing unit P3, and the result is output. For example, if the voice recognition character string is “None”, the motion character string is “Right forearm: Torso side, Upper right arm: Horizontal”, and the device status is “At departure”, the judgment result is “Insufficient utterance” Is output.

  FIG. 2 is a flowchart of the voice recognition software (P1). The operation based on the flowchart of FIG. 2 is as follows.

  First, in step S201, voice data transmitted from the communication terminal T1 via the network N1 is acquired as an input value.

  Next, the voice data acquired by the voice recognition process in step S201 is voice-recognized in step S202.

  In step S203, speech output processing is performed, and speech recognition information (speech recognition character string) is output as an output value to be input to the safety determination processing unit P3 as the speech recognition result.

After the series of processing in steps S201 to S203 is completed, it is determined in step S204 whether there is input of subsequent audio data.
Here, if there is input of subsequent audio data (YES), the process returns to the process of acquiring audio data in step S201.
If there is no subsequent voice data input (NO), the voice recognition software (P1) process is terminated.

  The processing content of the speech recognition software (P1) may use a mechanism of a generally existing speech recognition system.

  FIG. 3 is a flowchart of the motion analysis software (P2). The operation based on the flowchart of FIG. 3 is as follows.

  First, in step S301, video data transmitted from the communication terminal T1 via the network N1 is acquired as an input value.

  Next, the video data acquired by the motion analysis processing in step S301 is subjected to motion analysis processing in step S302.

  In step S303, a motion output process is performed, and the result is output as a motion pattern analysis result (motion pattern analysis result character string) as an output value to be input to the safety determination processing unit (P3).

After the series of processing in steps S301 to S303 is completed, it is determined in step S304 whether there is input of subsequent video data.
If there is subsequent video data input (YES), the process returns to the video data acquisition process in step S301.
If there is no subsequent video data input (NO), the processing of the motion analysis software (P2) is terminated.

  The processing contents of the motion analysis software (P2) may use a mechanism of a generally existing motion analysis system.

  FIG. 4 is a flowchart of the safety determination processing unit (P3). The operation based on the flowchart of FIG. 4 is as follows.

  First, in step S401, speech recognition information (speech recognition character string) that is a speech recognition result from the speech recognition software (P1) is acquired as an input value.

  Next, in step S402, motion pattern analysis result information (motion pattern analysis result character string), which is a motion pattern analysis result, is acquired as an input value from the motion analysis software (P2).

  Thereafter, in step S403, information indicating the device state (starting or traveling) of the operation target device A1 transmitted from the communication terminal T1 via the network N1 is acquired as an input value.

  When the input values are obtained, in step S404, the registration information for comparison registered in advance in the table 60 of the database D1 with information indicating the voice recognition result, the motion pattern analysis result, and the device state of the operation target device A1 as arguments. Compare / compare.

  Next, in step S405, a determination result corresponding to the collation / comparison is acquired from the database D1, and information indicating the determination result (insufficient utterance, unclear utterance, safety, insufficient pointing recognition, excessive pointing recognition) Any). The safety determination result is transmitted to the communication terminal T1 via the communication device TR. As a result, the driver can know the precautions regarding the safety of the driving operation.

After the series of processing in steps S401 to S405 is completed, it is determined in step S406 whether there is a subsequent determination processing.
If there is a subsequent determination process (YES), the process returns to the voice recognition result acquisition process in step S401. When there is no subsequent determination process (NO), the process of the safety determination process part (P3) is complete | finished.

FIG. 5 is a diagram for explaining an example of an embodiment of the present invention.
In FIG. 5, assuming that the information indicating the device state of the operation target device A1 is “at the time of departure”, four patterns of utterance contents (“departure progress”, “voice” by the driver (U1 to U4) as values of the input 501 None ”,“ Departure Progress ”,“ Aiueo ”), and voice recognition processing 502, motion analysis processing 503, and safety determination processing 504 at that time.

The speech recognition character string processed by the speech recognition software (P1)
For example, as shown in FIG.
"Departure progress" is recognized, "No recognition voice", "Departure progress" is recognized, and "Aiueo" is recognized.

The motion pattern analysis result character string processed by the motion analysis software (P2) is, for example, as shown in FIG.
The motion pattern 1 changes from “right forearm: trunk, upper right arm: horizontal” to “right forearm: horizontal, upper right arm: horizontal” in motion pattern 2.
The motion pattern 1 changes from “right forearm: trunk, upper right arm: horizontal” to “right forearm: horizontal, upper right arm: horizontal” in motion pattern 2.
Changed from “right forearm: horizontal, upper right arm: horizontal” in motion pattern 1 to “right forearm: horizontal, upper right arm: horizontal” in motion pattern 2,
This shows that the motion pattern 1 has changed from “right forearm: trunk side, upper right arm: horizontal” to “right right arm: horizontal, upper right arm: horizontal” in motion pattern 2.

The character string of the safety judgment result processed by the safety judgment processing unit P3 is
For example, as shown in FIG.
"Safety", "Insufficient utterance", "Insufficient pointing confirmation", "Unspoken utterance", etc. corresponding to speech recognition processing and motion analysis processing are shown.

That is, each process of the safety determination server S1 with respect to the input 501 in FIG. 5 is as follows.
When the input 501 is “departure progress” (first pattern), the voice recognition software unit P1 receives the voice data of the voice data (departure progress), recognizes that the voice data is “departure progress”, and the result Is output as a character string of "departure progress"
The motion analysis software part P2 receives the video data of the driver, and the driver has the motion pattern 2 from “right forearm: horizontal, upper right arm: horizontal” to “right forearm: horizontal, upper right arm: horizontal”. Is recognized (moved), and the result is output as a character string as shown in the figure.
The safety determination processing unit P3 receives each processing result in the voice recognition software unit P1 and the motion analysis software unit P2, determines whether or not the operation by the driver is safe, and the result is “safe”. Is output as a string.

When the input 501 is “no voice” (second pattern), the voice recognition software unit P1 receives voice data (no voice) of the voice data, recognizes that the voice data is “no voice”, and the result Is output as a character string of “No recognized speech”
The motion analysis software part P2 receives the video data of the driver, and the driver has the motion pattern 2 from “right forearm: horizontal, upper right arm: horizontal” to “right forearm: horizontal, upper right arm: horizontal”. Is recognized (moved), and the result is output as a character string as shown in the figure.
The safety determination processing unit P3 receives each processing result in the voice recognition software unit P1 and the motion analysis software unit P2, determines whether or not the operation by the driver is safe, and as a result, “speaking is insufficient”. Output as a character string to that effect.

When the input 501 is “departure progress” (third pattern), the voice recognition software unit P1 receives the voice data of the voice data (departure progress), recognizes that the voice data is “departure progress”, and the result Is output as a character string of "departure progress"
The motion analysis software part P2 receives the driver's video data, and the driver changes the motion pattern 1 from “right forearm: horizontal, upper right arm: horizontal” to motion pattern 2 from “right forearm: horizontal, upper right arm: horizontal”. Recognize that there is no change in movement, that is, motion pattern, and output the result as a character string as shown in the figure,
The safety determination processing unit P3 receives each processing result in the voice recognition software unit P1 and the motion analysis software unit P2, determines whether or not the operation by the driver is safe, and results in “insufficient pointing recognition”. Is output as a character string.

When the input 501 is “Aiueo” (fourth pattern), the voice recognition software unit P1 receives the voice data (Aiueo) of voice data, recognizes that the voice data is “Aiueo”, and obtains the result as “ Aiueo "is output as a character string,
The motion analysis software part P2 receives the video data of the driver, and the driver has the motion pattern 2 from “right forearm: horizontal, upper right arm: horizontal” to “right forearm: horizontal, upper right arm: horizontal”. The result is output as a character string as shown in the figure,
The safety determination processing unit P3 receives the processing results in the voice recognition software unit P1 and the motion analysis software unit P2, determines whether or not the operation by the driver is safe, and the result is “unintentional utterance”. Output as a character string to that effect.

  FIG. 6 is a table 60 showing a configuration example of registration information (data) registered in the database D1 in FIG.

  In FIG. 6, the speech recognition information (speech recognition character string) 601 for comparison with the speech recognition information (speech recognition character string) processed by the speech recognition software (P1), the motion pattern processed by the motion analysis software (P2). Motion pattern analysis result information (motion pattern analysis result character strings of a plurality of motion patterns 1 to n) 602 for comparison with the motion pattern analysis result information (motion pattern analysis result character string) shown, and the device state of the operation target device A1 The example of the table 60 which stores the information (character string) 603 which shows the state of operation object apparatus A1 for comparing with information (character string), and the information (character string) 604 which shows a determination result is shown.

  The voice recognition character string 601 is, for example, a character string such as “none”, “unknown”, “departure progress”, “departure warning”, “departure warning”, “first blockage progress”, “relay progress”, and the like.

  The motion pattern analysis result character string 602 in the motion patterns 1 and 2 is a character string such as “right forearm: trunk side, upper right arm horizontal”, “right forearm: horizontal, upper right arm horizontal”, and the like.

The character string 603 indicating the device status of the operation target device is, for example, a character string such as “when departing” and “when traveling”.
The character string 604 indicating the determination result is, for example, a character string such as “insufficient utterance”, “unintelligible utterance”, “safety”, “insufficient pointing recognition”, “none”, “excessive pointing”.
The motion patterns 3 to n indicate that there is no change in the motion pattern thereafter.

In FIG. 6, it has the row | line | column structure for demonstrating the embodiment of FIG. 5 simply.
Although the speech recognition character sequence is one, a speech recognition character sequence corresponding to each of the motion pattern sequences may be included. Similarly, the device status column of the operation target device A1 is one column, but an attribute column may be included in each of the motion pattern columns.

  According to the embodiment described above, it is possible to detect a state in which safety is lowered due to a lack of basic operations such as improvement of safety linked with voice and movement, only voice, and only gesture gesture.

In addition, this invention is not limited to the Example mentioned above, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described.
Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them, for example, with an integrated circuit. Further, each of the above-described configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Programs, tables, and information (data) that realize each function can be placed in a memory, a recording device such as a hard disk or an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

U1-U4 ... Driver / User (Driver, Driver, Passenger)
M ··· Voice sound collection device (voice sound collection means)
M1 ... Voice collector (microphone)
C ... Gesture gesture detection device (gesture gesture detection means)
C1 ... Gesture gesture detector (motion sensor, camera)
A ... Machine A1 related to driving operation ... Operation target device L1 ... Device state information acquisition unit T1 ... Communication terminal N1 ... Network S1 ... Safety judgment server (computer)
P ··· arithmetic device P1 ··· voice recognition software part (voice recognition means)
P2 ... Motion analysis software part (motion analysis means)
P3 ... Safety judgment processing part (safety judgment processing means)
D .... Storage device (storage means)
D1 ... Database TR ... Communication device

Claims (5)

A driving support system including a server that receives information about a device state of a driving operation target device and a user including an operator of the driving operation target device from a communication terminal on a driver's seat side of a mobile object via a network. And
The server
Data acquisition means for acquiring user voice data, gesture data, and data indicating the state of the operation target device;
Voice recognition means for recognizing user voice data acquired by the data acquisition means and outputting voice recognition information;
Analyzing the video data indicating the user's motion acquired by the data acquisition means, and outputting motion information;
Storage means for registering speech recognition information, motion analysis information, information indicating the state of the target device, and information indicating a determination result as information for comparison beforehand
The operation of the user who operates the operation target device is performed using the voice recognition information of the voice recognition unit, the motion pattern analysis result information of the motion analysis unit, and the information indicating the state of the operation target device of the data acquisition unit. Safety judgment processing means for judging safety of safety or not, and outputting information indicating a result of the judgment;
With
The safety determination processing means includes
Information indicating the state of the operation target device in the data acquisition unit, voice recognition information in the voice recognition unit, and motion pattern analysis result information in the motion analysis unit are set as input values, and the input values are stored in the storage unit in advance for comparison information. Compare the information indicating the status of the operation target device registered as, voice recognition information, motion pattern analysis result information,
A driving support system, wherein information indicating a state of an operation target device in the storage unit, voice recognition information, and information indicating a determination result corresponding to the motion pattern analysis result information are output from the comparison result of the comparison. In the driving assistance system according to claim 1,
The data acquisition means is a communication unit that transmits and receives voice data of a microphone, gesture data of a camera, and data of a device state character string indicating a state of an operation target device, transmitted from the communication terminal,
A database for registering a device state character string, a voice recognition character string, a motion pattern analysis result character string, and a character string indicating a determination result, which are registered in advance as comparison information by the storage unit;
The voice recognition means includes voice recognition software, recognizes the voice data, and outputs a voice recognition character string;
The motion analysis means includes a motion analysis software, analyzes the gesture data, and outputs a motion pattern analysis result character string,
The safety judgment processing means is a voice recognition character string in the voice recognition software unit, a motion pattern analysis result character string in the motion analysis software unit, a character string indicating a device state in the communication unit, and a voice recognition character string in the database Safety that determines whether the operation of the user who operates the operation target device is safe based on the motion pattern analysis result character string and the character string indicating the device state, and outputs the determination result as a character string A sex determination processing unit,
The character string indicating the determination result includes safety, insufficient utterance, unclear utterance, insufficient pointing confirmation, excessive pointing confirmation, and the like, and is transmitted to the communication terminal via the communication unit. Driving support system. A server that determines the safety of operation of the device to be operated from the communication terminal on the driver's seat side of the mobile object from the device state of the device to be operated and information related to the user including the operator of the device to be operated A driving support method in the provided driving support system,
The server
A data acquisition step for acquiring user voice data, gesture data, and data indicating the state of the operation target device;
A voice recognition step of recognizing user voice data acquired by the data acquisition means and outputting voice recognition information;
Analyzing the video data indicating the user's motion acquired by the data acquisition means, and outputting the motion information;
Registering speech recognition information, motion analysis information, information indicating the state of the target device, and information indicating a determination result as information for comparison in advance;
The operation of the user who operates the operation target device is performed using the voice recognition information of the voice recognition unit, the motion pattern analysis result information of the motion analysis unit, and the information indicating the state of the operation target device of the data acquisition unit. A safety determination processing step for determining safety of whether or not it is safe and outputting information indicating a result of the determination;
With
The safety determination processing step includes:
Obtaining a voice of a person including a user of the operation target device as voice data, recognizing the voice, and outputting the recognized voice recognition information as a voice recognition result;
Acquiring a video of a person including a user of the operation target device as video data, performing a motion analysis process on the video, and outputting a motion character string of a motion pattern as a motion analysis result;
The voice recognition result, the motion analysis result, the state of the operation target device, and the voice recognition character string, the motion character string, the character string indicating the state of the target device, and the character indicating the determination result registered in the database as comparison data in advance. A column as an input value and comparing them,
Receiving a comparison result of the step of performing the comparison process, determining safety of operation of the operation target device, and outputting the determination result as a safety determination result. In a computer system including a safety judgment server,
A function of acquiring the voice of a person including the user of the operation target device as voice data, recognizing the voice, and outputting the recognized voice recognition character string as a voice recognition result;
A function of acquiring a video of a person including a user of the operation target device as video data, performing a motion analysis process on the video, and outputting a motion character string of a motion pattern as a motion analysis result;
The voice recognition result, the motion analysis result, the state of the operation target device, the voice recognition character string registered in the database as comparison data in advance, the motion character string, the character string indicating the state of the target device, and the determination result are shown. A function that compares a character string as an input value and
A function of receiving the comparison result of the step of performing the comparison process, determining the safety of operation of the operation target device, and outputting the determination result as a safety determination result;
Driving support program to realize. A driving support program that causes a computer to function as the driving support system according to claim 1.

Images