JP2018120531A - Gesture user interface system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a gesture user interface system capable of suppressing occurrence of malfunction.SOLUTION: Body movement performed within recognition space Ao set within space where a user exists is detected, as well as the user is notified of that effect, when the body movement is performed in space outside the recognition space Ao. Thereby, when the body movement is performed in the space outside the recognition space Ao, a user who has performed the body movement can understand that fact. Specifically, it is possible to urge the user "to perform the body movement again within the recognition space Ao". Consequently, occurrence of malfunction can be suppressed.SELECTED DRAWING: Figure 6

Description

  The present application relates to a gesture user interface system that detects a user's physical motion and recognizes the user's intention.

For example, in the invention described in Patent Document 1, an occupant's body motion (gesture) is detected, and the operation of the vehicle-mounted device is controlled using the detected result.
In other words, in the invention described in Patent Document 1, a user such as an occupant does not directly operate an operation panel, a switch, or the like, but the user performs a body motion such as a gesture of shaking, so that the on-vehicle equipment is Operate.

JP2015-133109A

  For example, when the user performs a gesture that moves the fingertip, if the position of the moving fingertip is out of the recognizable space area, (a) the vehicle-mounted device does not operate, or (b) the vehicle is mounted Malfunctions such as devices operating differently from the user's intentions occur.

  A recognizable spatial region (hereinafter also referred to as a recognition space) refers to a spatial region that is unlikely to be erroneously detected by the detection device. In other words, when the user performs physical movements in the recognition space, the possibility of malfunctions also decreases.

  In view of the above points, the present application provides a gesture user interface system capable of suppressing the occurrence of malfunction.

  In the present application, in a gesture user interface system that detects a user's physical motion and recognizes the user's intention, the physical motion detected in the recognition space (Ao) set in the space where the user exists is detected. And a notification device (10, 13) for notifying the user of a physical motion in a space outside the recognition space (Ao).

  Thereby, when a body motion is performed in a space outside the recognition space (Ao), the user who has performed the body motion can understand that fact. That is, it can be urged to the user that “the body motion needs to be executed again in the recognition space (Ao)”. As a result, the occurrence of malfunction can be suppressed.

In addition, you may comprise this application as follows.
That is, the notification device (10, 13) recognizes the movement position detected by the movement position detection unit (10, 11) and the movement position detection unit (10, 11) that detects the position where the body movement is realized. It is desirable to have a notification pattern determination unit (10) that determines a notification pattern for notification using a separation distance from the space (Ao).

  Thereby, it can be told to the user “in which space it is appropriate to perform the next physical motion”. As a result, since the possibility that the body movements after the next time will be executed in the recognition space (Ao) increases, the occurrence of malfunction can be suppressed.

  The seat (S) is a seat (S) that can be seated by the user, and is provided with a vibration device (13) that applies vibration to the seated person. The notification device (10, 13) includes the vibration device (13). ) May be vibrated to notify the user. In the case of notifying by vibration, it is desirable that the vibration device (13) is configured by the first vibrator (13A) and the second vibrator (13B) disposed at different locations. Thereby, it may be possible to make the user understand the content of the notification with certainty.

  A user position detection unit (12) for detecting a position where the user exists, and a recognition space setting for setting a recognition space (Ao) using the user's presence position detected by the user position detection unit (12). It is desirable to provide a part (10). Thereby, the recognition space (Ao) can be set at a position preferable for the user.

  Incidentally, the reference numerals in the above parentheses are examples showing the correspondence with the specific configurations described in the embodiments described later, and the present invention is limited to the specific configurations indicated by the reference numerals in the parentheses. It is not something.

It is a figure which shows the structure of the gesture user interface system which concerns on embodiment of this invention. It is a conceptual diagram about the recognition space Ao etc. in the gesture user interface system which concerns on embodiment of this invention. It is a figure which shows an example of the alerting vibration pattern in the gesture user interface system which concerns on embodiment of this invention. It is a figure which shows an example of the alerting vibration pattern in the gesture user interface system which concerns on embodiment of this invention. It is a figure which shows an example of the alerting vibration pattern in the gesture user interface system which concerns on embodiment of this invention. It is a figure which shows an example of the control flow of the gesture user interface system which concerns on embodiment of this invention.

  The “embodiment of the invention” described below shows an example of an embodiment belonging to the technical scope of the present invention. In other words, the invention specific items described in the claims are not limited to the specific configurations and structures shown in the following embodiments.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, the arrow etc. which show the direction attached | subjected to each figure are described in order to make it easy to understand the relationship between each figure. The present invention is not limited to the directions given in the drawings.

  At least one member or part described with at least a reference numeral is provided, except where “one” or the like is omitted. That is, when there is no notice such as “one”, two or more members may be provided.

(First embodiment)
1. Outline of Vehicle Gesture User Interface System In this embodiment, a gesture user interface system is applied to a vehicle such as an automobile. In the present embodiment, as shown in FIG. 1, a gesture user interface system 1 is applied to a rear seat S of an automobile.

  In other words, in the present embodiment, an occupant's intention is recognized via a gesture user interface system, and a device mounted on a vehicle (hereinafter referred to as a mounted device) is operated.

  The on-board equipment is various equipment that operates on the vehicle. Specifically, for example, vehicle interior devices such as a vehicle air conditioner, a vehicle audio device, and a car navigation system, or outdoor devices such as a back door opening and closing device and a sliding door opening and closing device.

2. Outline of the gesture user interface system (see Fig. 1)
2.1 Configuration of Gesture User Interface System The gesture user interface system 1 includes at least a motion detection device and a notification device. The motion detection device is a device for detecting a physical motion performed by the user in the recognition space Ao.

  The recognition space Ao is “an area set in advance in a space where a user exists”. In other words, the recognition space Ao is an area where the motion detection device can recognize (understand) the intention of the body motion. Note that the recognition space Ao according to the present embodiment is set to a space where a person seated on the rear seat S is highly likely to move his hand (for example, a fingertip).

  The motion detection apparatus according to the present embodiment is configured to include an imaging sensor 11, a control device 10, and the like. The imaging sensor 11 is configured by a camera or the like that can convert body motion in a space such as an imaging element into an electrical signal.

  The control device 10 receives the image data output from the image sensor 11 and analyzes the image data to recognize the intention of the body movement. The control device 10 is configured by a microcomputer having a CPU, a ROM, a RAM, and the like. Software for analyzing image data is stored in advance in a nonvolatile storage unit such as a ROM.

  The control device 10 according to the present embodiment controls the entire gesture user interface system 1 and also serves as a part of the motion detection device. That is, software for controlling the operation of the gesture user interface system 1 is stored in advance in a nonvolatile storage unit such as a ROM.

  The notification device is a device for notifying the user of a physical movement in a space outside the recognition space Ao. The notification device according to the present embodiment includes a vibration device 13 and a control device 10.

  The vibration device 13 is a device that imparts vibration to a seated person seated on the seat S. When the control device 10 determines that “the body motion of the seated person cannot be recognized”, the control device 10 regards it as “the body motion has been performed outside the recognition space Ao” and vibrates the vibration device 13.

  As shown in FIG. 1, the vibration device 13 according to the present embodiment includes a first vibrator 13 </ b> A and a second vibrator 13 </ b> B disposed at different locations. The first vibrator 13A is disposed on the seat back and applies vibration to the back of the seated person. The second vibrator 13B is disposed on the armrest and applies vibration to the arm of the seated person.

2.2 General Operation of Gesture User Interface System In the present embodiment, as shown in FIG. 2, a first recognition space A1 and a second recognition space A2 are set outside the recognition space Ao. The recognition space Ao is a space that has a probability that the control device 10 can recognize the intention of body movement (hereinafter referred to as a recognition rate) that is greater than or equal to a predetermined value.

  The first recognition space A1 is a space in which the control device 10 can recognize that a body motion has been executed, and is a space with a lower recognition rate than the recognition space Ao. The second recognition space A2 is a space in which the control device 10 can recognize that a body motion has been executed, and is a space with a lower recognition rate than the first recognition space A1.

  That is, the recognition rate decreases as the distance from the recognition space Ao increases. Outside the second recognition space A2, the control device 10 cannot recognize whether or not a body motion has been executed. Then, when the body movement is recognized in the first recognition space A1 or the second recognition space A2, the control device 10 regards that “the body movement has been performed in a space outside the recognition space Ao”.

  In the present embodiment, even when the control device 10 cannot recognize the presence / absence of a physical motion within a preset time after the operation of the gesture user interface system 1 is started, the control device 10 is “in a space outside the recognition space Ao”. It is considered that physical movement has been made.

  In other words, the motion detection device also has a motion position detection function for detecting the position where the body motion is realized. And the control apparatus 10 performs the alerting | reporting of the determined alerting | reporting pattern, after determining the alerting | reporting pattern at the time of alerting | reporting using the separation distance of the detected movement position of the body motion and the recognition space Ao.

  Specifically, when recognizing the body movement in the second recognition space A2, the control device 10 executes a notification operation of the first vibration pattern shown in FIG. 3, for example. When the control device 10 recognizes the body motion in the first recognition space A1, for example, the control device 10 performs a notification operation of the second vibration pattern shown in FIG.

  When the control apparatus 10 according to the present embodiment recognizes the body movement in the recognition space Ao, for example, the control apparatus 10 performs a notification operation of the third vibration pattern illustrated in FIG. That is, the third vibration pattern is a notification pattern indicating that “a body motion has been performed in the recognition space Ao”.

  When the control device 10 cannot recognize the presence / absence of a physical motion within a preset time after the operation of the gesture user interface system 1 is started, the control device 10 performs a notification operation of the first vibration pattern.

  In each vibration pattern according to the present embodiment, the greater the distance between the recognition space Ao and the movement position, the shorter the vibration period of each vibrator 13A, 13B, and the combined vibration period of both vibrators 13A, 13B. Changes to be shorter.

  The vibration period of the first vibrator 13A refers to twice the time T1 in FIGS. The vibration period of the second vibrator 13B means twice the time T2 in FIGS. The combined vibration period refers to “T1 + T2−Ta”.

  The seat position sensor 12 illustrated in FIG. 1 is an example of a user position detection unit that detects a position where a user exists. That is, the seat S is slidable relative to the vehicle body. The seat position sensor 12 detects the position of the seat S with respect to the vehicle body, and outputs the detection result to the control device 10.

  The control device 10 sets the recognition space Ao using the detection result of the sheet position sensor 12. That is, when the position of the seat S with respect to the vehicle body is determined, the space for performing the body motion of the seated person who is the user is generally determined. Therefore, in the present embodiment, the positional relationship between “the position of the seat S” and “the space in which the seated person performs physical movements” is stored in advance in a nonvolatile storage unit such as a ROM.

  Then, the control device 10 sets the recognition space Ao using the positional relationship and the detection result of the sheet position sensor 12. That is, for example, when the position of the seat position sensor 12 changes in the vehicle front-rear direction, the position of the recognition space Ao also changes in the vehicle front-rear direction accordingly.

  Note that the positional relationship between the “position of the seat S” and the “space where the seated person moves” is optimized by trial and error such as a test. Therefore, the positional relationship is content that can be appropriately changed depending on the vehicle type, the destination of the vehicle, and the like.

2.3 Details of Operation of Gesture User Interface System The control flow shown in FIG. 6 is an example of the above-described general operation, and the control flow is executed by the control device 10. Note that software for executing the control flow is stored in advance in a nonvolatile storage unit such as a ROM.

  The control flow is started when the on-board equipment is ready for operation (for example, when an engine start switch is turned on), and is stopped when the on-board equipment becomes inoperable.

  When the on-board device is ready for operation, the control device 10 determines whether or not the gesture user interface system 1 is activated (S1). In addition, the control apparatus 10 which concerns on this embodiment makes the said determination using states, such as a start switch (not shown) of a gesture user interface system.

  That is, when the control device 10 determines that the activation switch is turned on, the control device 10 determines that the gesture user interface system 1 is activated, that is, “gesture ON (on)”. .

  When the control device 10 determines that “gesture is ON” (S1: YES), after reading the detection result of the sheet position sensor 12 (S3), the control device 10 uses the detection result to recognize the recognition space. Ao is set (S5).

  Note that when the setting of the recognition space Ao is completed, the control device 10 according to the present embodiment outputs, for example, a preset vibration pattern via the vibration device 13 to the user to recognize the “recognition space Ao. Is set "(S7).

  After the above notification, the control device 10 acquires the motion position of the body motion (in this embodiment, the fingertip position coordinates of the seated person) using the imaging sensor 11 (S9), and then the motion position is in any space. It is determined whether it is located at Ao to A2 (S11).

  When the control device 10 determines that the operation position is located in any of the spaces Ao to A2 (S11: YES), the control device 10 determines and recognizes the separation distance of the operation position (S13), and then performs the separation. A predetermined vibration pattern is determined for the distance (S15), and the determined vibration pattern is output to the vibration device 13 (S17).

  Next, the control device 10 recognizes and judges the meaning content of the body motion, that is, the intention of the user through the body motion (S19), and if the meaning content of the body motion can be recognized (S19: YES), A notification to that effect (vibration pattern shown in FIG. 5) is output, and a command signal indicating the meaning content of the body motion is output to the mounted device that is the target of the body motion (S21).

  Specifically, for example, when the control device 10 determines that “the body motion is the meaning of changing the set temperature of the vehicle air conditioner”, the control device 10 controls the vehicle air conditioner. A command signal to that effect is output to the unit.

  And the said control part operates a vehicle air conditioner according to a command signal. In addition, when the control apparatus 10 judges that the meaning content of a body motion cannot be recognized (S19: NO), the control apparatus 10 performs S3.

  After outputting the command signal, the control device 10 stops the notification operation by the vibration device 13 (S23), and then determines whether or not the body motion is continuously executed (S25). When the control device 10 determines that the body motion is continuously executed (S25: YES), a command signal indicating the meaning content of the body motion is output to the mounted device that is the target of the body motion (S27). ).

  Next, the control device 10 determines in which space Ao to A2 the movement position of the body movement is located (S29). When the control device 10 determines that the operation position is located in any of the spaces Ao to A2 (S29: YES), the control device 10 executes S25. When the control device 10 determines that the operation position is not located in any of the spaces Ao to A2 (S29: NO), the control device 10 executes S1.

3. Features of Gesture User Interface System According to the Present Embodiment In the present embodiment, the body motion performed in the recognition space Ao set in the space where the user exists is detected and the body motion is performed in the space outside the recognition space Ao. If this happens, the user is notified of this.

  Thereby, when a body motion is performed in a space outside the recognition space Ao, the user who has performed the body motion can understand that fact. That is, the user can be prompted to “execute physical movement again in the recognition space Ao”. As a result, the occurrence of malfunction can be suppressed.

In the present embodiment, a notification pattern at the time of notification, that is, a vibration pattern, is determined using the separation distance between the detected motion position and the recognition space Ao.
Thereby, it can be told to the user “in which space it is appropriate to perform the next physical motion”. As a result, the possibility that the next and subsequent body movements are executed in the recognition space Ao is increased, so that the occurrence of malfunction can be suppressed.

  In the present embodiment, the vibration device 13 is configured by the first vibrator 13A and the second vibrator 13B that are disposed at different locations. Thereby, it may be possible to make the user understand the content of the notification with certainty.

  In this embodiment, the position where the user exists is detected, and the recognition space Ao is set using the detected position of the user. Thereby, the recognition space Ao can be set at a position preferable for the user.

(Other embodiments)
In each vibration pattern according to the above-described embodiment, the greater the separation distance between the recognition space Ao and the operation position, the shorter the vibration period of each vibrator 13A, 13B, and the combined vibration period of both vibrators 13A, 13B. Changed to be shorter. However, the invention disclosed in this specification is not limited to this.

  In the above-described embodiment, when the activation switch of the gesture user interface system is turned on, it is determined that “gesture is ON”. However, the invention disclosed in this specification is not limited to this.

  That is, for example, when the activation switch is configured by a voice input device, a configuration in which “gesture ON (on)” is determined based on the state of the voice input device may be employed.

  In the above-described embodiment, the notification device is configured by the vibration device 13. However, the invention disclosed in this specification is not limited to this. That is, for example, a notification device that notifies by voice or video may be used.

  The vibration device 13 according to the above-described embodiment includes the first vibrator 13A and the second vibrator 13B. However, the invention disclosed in this specification is not limited to this. That is, for example, it may be configured by one vibrator or three or more vibrators.

  In the above-described embodiment, the position where the user exists is detected by the sheet position sensor 12. However, the invention disclosed in this specification is not limited to this. That is, for example, the position where the user exists may be detected using a wide-angle camera or the like.

  In the above-described embodiment, the recognition space Ao is set using the detected location of the user. However, the invention disclosed in this specification is not limited to this. That is, for example, the recognition space Ao may be set as a fixed space. In FIG. 2, each space is shown in a spherical shape, but the invention disclosed in the present specification is not limited to this.

  In the above-described embodiment, the gesture user interface system according to the present invention is applied to a passenger car. However, the application of the present invention is not limited to this, and can also be applied to seats used for vehicles such as railway vehicles, ships and aircraft, and stationary seats used for theaters, homes and the like.

  Furthermore, the present invention is not limited to the above-described embodiment as long as it matches the gist of the invention described in the claims. Therefore, you may combine at least 2 embodiment among several embodiment mentioned above.

DESCRIPTION OF SYMBOLS 1 ... Gesture user interface system 10 ... Control apparatus 11 ... Imaging sensor 12 ... Sheet position sensor 13 ... Vibration apparatus

Claims (6)

In a gesture user interface system that detects a user's physical motion and recognizes the user's intention,
A motion detection device for detecting the physical motion performed in a recognition space set in a space where a user exists;
A gesture user interface system comprising: a notification device that notifies a user when the body motion is performed in a space outside the recognition space. The notification device
A motion position detection unit that detects a position where the body motion is realized, and a separation pattern between the motion position detected by the motion position detection unit and the recognition space is used to determine a notification pattern for notification. The gesture user interface system according to claim 1, wherein the gesture user interface system is configured to include a notification pattern determination unit. A seat that can be seated by a user, comprising a seat provided with a vibration device for applying vibration to the seated person,
The gesture user interface system according to claim 1, wherein the notification device vibrates the vibration device to notify a user.   The gesture user interface system according to claim 3, wherein the vibration device includes a first vibrator and a second vibrator disposed at different locations. A user position detection unit for detecting a position where the user exists;
The gesture user interface system according to any one of claims 1 to 4, further comprising: a recognition space setting unit that sets the recognition space using a user location detected by the user position detection unit. Mounted on the vehicle and operating on the vehicle;
A vehicle having the gesture user interface system according to any one of claims 1 to 5, and a control unit that operates the on-board equipment using the intention of the passenger recognized by the gesture user interface system. Control system.