JP2010134057A - Head-mounted display - Google Patents

Abstract

An object of the present invention is to provide a head mounted display having excellent operability with respect to an operation for instructing an operation of the head mounted display.
In the portion constituting the face of the user wearing the head mounted display, the state of the eyebrows, cheeks and mouth that are in a specific state when the user consciously moves is detected ( S202, S206, S208), on condition that any of the eyebrows, cheeks, and mouth is in a specific state, the head is displayed so that the operation associated with the detected state is executed on the head mounted display. The mount display is controlled (S210).
[Selection] Figure 6

Description

  The present invention relates to a head-mounted display that presents a content image indicated by content data to a user's eyes so that the user can recognize the content image.

A technology related to a head-mounted display has been proposed in which a content image indicated by content data is presented to a user's eyes so that the user can recognize the content image. For example, in order to improve the operability of the head mounted display, the gaze direction, focal length, pupil state, fundus pattern, eyelid movement of the user (user) wearing the head mounted display is detected by a sensor. A technique has been proposed in which a user's intention or situation is determined from information acquired by a sensor, and based on the determination result, a control is performed on display operation in a display unit that displays an image (content image) ( For example, see Patent Document 1).
JP 2008-65169 A

  With regard to the operation of the head-mounted display, the operation that instructs the user's intended operation is not performed via the user interface that the user directly operates through the hand etc., but the state of the pupil of the user's body, the fundus pattern, the eyelid A technique that can be performed based on movement is a useful technique because it can reduce the operation burden on the user. For example, in a production site of a product, a work instruction is presented to a user who wears the head-mounted display, and when the user assembles a product, both hands of the user who is working are It is used for product assembly work and it is difficult to operate the head mounted display.

  In such a situation, the above technique can reduce the burden of operation on the head mounted display by the user wearing the head mounted display. However, in this case, it is important that an operation that matches the user's intention is accurately executed in the head mounted display. For example, maintaining a state in which the line-of-sight direction is artificially fixed in a certain direction is a painful operation for humans, and usually the direction of the user's line-of-sight is constantly changing. Furthermore, depending on various stimuli from the outside world, the line of sight may be changed without intention to operate. Therefore, it is difficult to distinguish whether the eye direction is detected by the method of detecting the direction of the line of sight, and erroneous detection occurs. The movement of the eyelids is the same, and the eyelids are usually unintentionally blinked or the eyelids are closed without intention to operate due to fatigue or the like, and erroneous detection is likely to occur. Also, it is difficult to intentionally change the state of the pupil and the fundus pattern.

  The present invention relates to an operation for instructing an operation of a head mounted display, and an object thereof is to provide a highly reliable head mounted display excellent in operability.

  The head mounted display of the present invention made in view of the above conventional problems has almost no unintentional movement in the portion constituting the face of the user wearing the head mounted display, and the user is conscious. Detects the state of a specific part that becomes a specific state by moving, and on the condition that the state of the specific part is a specific state, the operation associated with the detected state is executed in the head mounted display Is to be controlled.

  A first problem-solving means that reflects the present invention is a head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, and allows the user to recognize the content image. Control means for controlling the operation of the head-mounted display, and detection means for detecting that the state of a specific part of the user's face is a predetermined state, wherein the detection means comprises the user The eyebrow state of the user is detected to be a predetermined state, and the control means changes to the predetermined state on condition that the detection means detects that the user's eyebrow state is the predetermined state. The head mounted display is characterized in that the associated operation is controlled.

  The second problem-solving means reflecting the present invention is a head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, and allows the user to recognize the content image. Control means for controlling the operation of the head-mounted display, and detection means for detecting that the state of a specific part of the user's face is a predetermined state, wherein the detection means comprises the user The cheek state is detected to be in a predetermined state, and the control means changes to the predetermined state on condition that the detection means detects that the user's cheek state is in the predetermined state. The head mounted display is characterized in that the associated operation is controlled.

  The third problem-solving means reflecting the present invention is a head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, and allows the user to recognize the content image. Control means for controlling the operation of the head-mounted display, and detection means for detecting that the state of a specific part of the user's face is a predetermined state, wherein the detection means comprises the user The state of the mouth of the user is detected to be a predetermined state, and the control means changes to the predetermined state on condition that the detection means detects that the state of the mouth of the user is the predetermined state. The head mounted display is characterized in that the associated operation is controlled.

  According to the head-mounted display of the first to third problem solving means, unless the user wearing the head-mounted display is not particularly conscious, a large state change is unlikely to occur, based on the state of the eyebrows, cheeks or mouth. Since the operation of the mount display can be controlled, the operability of the head mounted display can be improved while suppressing malfunction.

  In the first to third problem solving means, the “head-mounted display operation” includes, for example, content data selection, playback / stop for content data executed when presented to the user's eyes, content image This includes processing related to presentation to the user's eyes, supply or interruption of power to the head mounted display, and other various operations realized in the head mounted display. Further, the head-mounted display is not necessarily a single device, and may be configured by two devices, for example. For example, the control means may be configured as another device and connected to the head mounted display with a predetermined signal cable.

  The fourth problem solving means is any one of the first to third problem solving means, and the detection means can capture a specific image of a specific part of the user's face. An imaging means and an analysis means for analyzing the specific image captured by the imaging means, and according to an analysis result of the analysis means, a state of a specific part of the user's face is in a predetermined state It is characterized by detecting something. According to this, the state of the specific part of a user's face can be detected appropriately.

  The fifth problem solving means is any one of the first to third problem solving means, and the detecting means can irradiate a specific part of the user's face with light. A light-emitting element; and a light-detecting element that detects reflected light reflected by the light irradiated on a specific part of the user's face, and the intensity of the reflected light detected by the light-detecting element is a predetermined reference When the value deviates, it is detected that the state of the specific part of the user's face is a predetermined state. According to this, the state of the specific part of a user's face can be detected appropriately.

  The sixth problem solving means is any one of the first to fifth problem solving means, and the control means performs the first operation and the second operation as the operation of the head mounted display. And when the state of the specific part of the user's face is the first state, the detection means detects the first state as the predetermined state, and specifies the specific state of the user's face. When the state of the portion is the second state, the second state is detected as the predetermined state, and the control means detects that the detection means is the first state, on the condition that the first state is detected. Controlling the first action associated with the first state, and controlling the second action associated with the second state on condition that the detection means detects the second state. It is characterized by. According to this, a plurality of actions can be assigned to the state of a specific part of the user's face, and the plurality of actions can be controlled.

  According to the present invention, an erroneous operation due to an unconscious operation is unlikely to occur with respect to an operation for instructing an operation of the head mounted display, and a reliable operation can be performed according to the user's intention without using a hand. A mount display can be obtained.

  Embodiments for implementing the above problem solving means reflecting the present invention will be described below in detail with reference to the drawings. In addition, the said subject solution means is not limited to the structure described below, Various structures can be employ | adopted in the same technical idea. For example, in the following description, a head mounted display configured by connecting a head mounted display main body and a control box that provides a content image to the head mounted display main body will be described as an example. It can also be configured as a device. In the following description, the head-mounted display body is simply referred to as a head-mounted display (hereinafter referred to as “HMD”).

(Overview of head mounted display)
FIG. 1 shows the configuration of the HMD, more specifically, FIG. 1A shows a top view of the HMD, FIG. 1B shows a front view of the HMD, and FIG. 1C shows a left side view of the HMD. FIG. 2 shows a state in which the user wears the HMD.

  The HMD 100 includes temples 104A and 104B each having a modern 102A and 102B that hits a user's ear at one end, armor 106A and 106B coupled to the other end of the temples 104A and 104B via trillions 112A and 112B, A skeleton portion is formed by a front frame 108 that connects 106A and 106B, and a nose pad 110 that is attached to the center portion of the front frame 108 and contacts the user's nose. The temples 104A and 104B can be folded at the trillions 112A and 112B formed on the armatures 106A and 106B. The structure of the skeleton part of the HMD 100 is the same as, for example, normal glasses, and the HMD 100 is supported on the user's face by the modern 102A and 102B and the nose pad 110 when worn on the user. (See FIG. 2). In FIG. 1B, drawing of the moderns 102A and 102B and the temples 104A and 104B is omitted.

  The image presentation device 114 is attached to the skeleton part of the HMD 100 via an attachment part 122 provided (provided) near the armor 106A. The image presentation device 114 is disposed at a position that is substantially the same height as the left eye 118 of the user wearing the HMD 100 in a state where the image presentation device 114 is attached to the vicinity of the armor 106A via the attachment portion 122. The image presentation device 114 is connected to a control box 200 (see FIG. 2; see FIG. 3 to be described later in detail) and a predetermined signal cable 250 (see FIG. 2). Although details will be described later, the control box 200 executes rendering processing on the content data stored in a predetermined area, controls an input / output interface (hereinafter referred to as “I / F”) included in the own device, and The content image signal including the content image obtained by the rendering process (reproduction process) is output to the image presentation device 114 via the signal cable 250. The image presentation device 114 receives the content image signal output from the control box 200 via the input / output I / F that is not depicted in FIGS. 1 and 2 and acquires it. Then, the content image based on the content image signal is optically emitted toward the half mirror 116.

  The content image (light beam) emitted from the image presentation device 114 is reflected by the half mirror 116 and is incident on the left eye 118 of the user, in other words, presented (projected) so as to be visible. Thereby, the user recognizes the content image. Here, in FIG. 1A, reference numeral 120a indicates a light beam related to the content image emitted from the image presentation device 114, and reference numeral 120b indicates a light beam that reflects the half mirror 116 and enters the left eye 118 of the user. Show. The image presentation device 114 scans the light rays 120a and 120b corresponding to the acquired content image signal in a two-dimensional direction, guides the scanned light rays 120a and 120b to the user's left eye 118, and displays the content image on the retina. In addition to a retinal scanning type display that forms a liquid crystal display, a liquid crystal display, an organic EL (Organic Electroluminescence) display, and other devices may be used.

  An eyebrow sensor 214 is attached to the upper surface of the image presentation device 114, and a cheek sensor 216 is attached to the lower surface. In addition, a stay 124 (not shown in FIG. 1; see FIG. 2) with a mouth sensor 218 attached to the tip is attached to the temple 104A. The eyebrow sensor 214, cheek sensor 216, and mouth sensor 218 will be described in “Control Box Configuration” described below.

(Control box configuration)
FIG. 3 shows functional blocks showing the configuration of the control box. The control box 200 is used by being attached to the user's waist or the like, for example. The control box 200 includes various signals among the CPU 202 that controls the device itself, a ROM 204 that stores various programs, a RAM 206 as a work area, a storage unit 208 that stores content data 2082 and a table 2084, and the HMD 100. And an input / output I / F 210 that transmits / receives and an operation unit 212 that is operated by a user and receives an instruction from the user. Further, the control box 200 includes an eyebrow sensor 214 attached to the upper surface of the image presentation device 114, a cheek sensor 216 attached to the lower surface, and a mouth sensor 218 attached to the tip of the stay 124 (see FIG. 2). , Is connected.

  Here, the storage unit 208 is configured by, for example, a hard disk. The content data 2082 stored in the storage unit 208 includes, for example, a work instruction describing a method for assembling a predetermined product (hereinafter, the case where the content data is a work instruction will be described as an example). To do.) The table 2084 associates each state of the eyebrows, cheeks, and mouth detected by the eyebrow sensor 214, cheek sensor 216, and mouth sensor 218 attached to the HMD 100 with each action realized in the HMD 100 and the control box 200. Registered table. The operation unit 212 includes, for example, a key, and receives instructions for starting and stopping the reproduction of content data.

  The eyebrow sensor 214 detects the state (movement) of the left eyebrow of the user, and the cheek sensor 216 detects the state (movement) of the user's cheek. The mouth sensor 218 detects the state (shape) of the user's mouth, more specifically, the user's lips. As the eyebrow sensor 214, the cheek sensor 216, and the mouth sensor 218, for example, an image sensor configured by a CCD (Charge Coupled Devices) or a photoelectric sensor configured by a light emitting element and a light detecting element can be used. A detection signal including a state detected by each sensor 214, 216, 218 (in the case of an image sensor, indicated by an image indicating a state such as eyebrows, and in the case of a photoelectric sensor, indicated by the intensity of reflected light) is controlled. The data is input to the box 200 and stored on the RAM 206.

  The CPU 202 acquires a content image by executing a program for reproducing (rendering) the content data 2082 stored in the ROM 204 on the RAM 206. Then, a program for controlling the input / output I / F 210 stored in the ROM 204 is executed on the RAM 206, and a content image signal including a content image is output from the input / output I / F 210 to the HMD 100. The CPU 202 detects an analysis program (for example, a program for pattern matching) stored in the ROM 204 by the sensors 214, 216, and 218, and uses the detection signal and the table 2084 stored in the RAM 206. By executing on the RAM 206, the state of the user's eyebrows, cheeks and mouth is analyzed. Further, the CPU 202 executes a program for controlling the HMD 100 stored in the ROM 204 on the RAM 206, and controls the operation of the HMD 100 instructed by the operation unit 212 and the operation of the HMD 100 based on the analysis result. . Therefore, the CPU 202 uses the various data such as the content data 2082, the table 2084, and the detection signal, and executes various programs stored in the ROM 204 on the RAM 206, whereby various functional units (for example, a control unit and an analysis unit) are used. Composed.

  FIG. 4 is a diagram conceptually showing the registration state of the table 2084. First, a standard face image, which is a face image of the user in the standard state, is registered in the table 2084 (see FIG. 4A). The operation of the HMD 100 is not associated with the standard face image. In other words, the standard face image is associated with continuing the operation in progress on the HMD 100, for example, continuously executing the reproduction operation.

  FIG. 4B is a diagram illustrating the correspondence between the state of the eyebrows and the operation realized by the HMD 100 and the control box 200. The action “next” is associated with the face image indicating the state of “raised eyebrows”. For example, in a state where a predetermined content image is presented to the user's left eye 118, when the user raises his / her eyebrows, the content image next to the content image being presented is presented. More specifically, when a user who has finished the work in the first process raises the eyebrows in the state where the content image including the work instruction in the first process is presented, the work instruction related to the second process is given. A content image as content is presented.

  Further, the action “return to the previous” is associated with the face image indicating the state of “closed eyebrows”. Specifically, in the state where the content image including the work instruction relating to the second step is presented, when the user brings his / her eyebrows, the content image including the work instruction relating to the first step is presented. In the HMD 100 shown in FIGS. 1 and 2, the eyebrow sensor 214 is disposed so as to face the user's left eyebrow. Therefore, whether the eyebrow is raised or not is determined by the state of the left eyebrow. Judgment based on.

  FIG. 4C is a diagram illustrating the correspondence between the cheek state and the operations realized by the HMD 100 and the control box 200. The action “decision” is associated with a face image indicating the state of “raised cheek”. Further, the operation “cancel” is associated with the face image indicating the state of “dented cheek”. For example, a content image containing the work instruction of the first step, a content image containing the work instruction of the second step, and a content image containing the work instruction of the third step are sequentially arranged at predetermined intervals. If the user raises the cheek while the content image containing the work instruction of the second step is being presented in the state of being displayed while being switched, the work instruction of the second step is made the content. When the content image is determined to be a content image that is continuously presented and the cheek is recessed after the determination, this determination is canceled. Note that the cheek sensor 216 is also disposed opposite the user's left cheek, like the eyebrow sensor 214, so whether or not the cheek is swollen is determined based on the state of the left cheek. The

  FIG. 4D is a diagram illustrating correspondence between the mouth state (shape) and the operation realized by the HMD 100 and the control box 200. The “brightness adjustment” of the presented content image is associated with the face image indicating the mouth state when “e” is uttered. In addition, “brightness up” is associated with an image indicating the state of the mouth when “a” is uttered. Furthermore, “brightness down” is associated with the state of the mouth when “o” is generated. More specifically, if the user's left eye 118 is presented with a content image that contains the work instruction of the first step, it will become bright when the user speaks “e”. A setting screen for adjustment is presented. When the setting screen is presented, if the user speaks “A”, the brightness increases by one level, while if the user speaks “O”, the brightness increases. Go down one step. Note that if the user makes “a” or “o” into the shape of a mouth that utters two times in succession, the brightness increases or decreases by two levels.

(Processing executed by the control box)
FIG. 5 shows a flow of main processing executed in the control box 200. This process is started when the CPU 202 executes a program stored in the ROM 204 on the RAM 206 on condition that the HMD 100 and the control box 200 are powered on. In the execution of this process, the detection signal detected by the eyebrow sensor 214, cheek sensor 216, and mouth sensor 218 and input to the control box 200, the content data 2082 stored in the storage unit 208, and the table 2084 are displayed at a predetermined timing. Used.

  The CPU 202 that has started the process first initializes the components of the control box 200 (S100), initializes the sensors 214, 216, and 218 (S102), and moves the process to S104. In step S <b> 104, the CPU 202 determines whether the user has input an instruction to start playback of the content data 2082 via the operation unit 212. As a result of the determination, if a reproduction start instruction is not input (S104: No), the process waits until a reproduction start instruction is input. On the other hand, when an instruction to start reproduction is input (S104: Yes), the CPU 202 starts content image signal output processing (S106). Here, the content image output processing reads content data 2082 from the storage unit 208 onto the RAM 206, renders the content data 2082, and outputs a content image signal including the content image obtained by the rendering to the input / output I / F 210. This is a process of controlling and outputting to the HMD 100.

  The CPU 202 that has started the content image signal output process in S106 determines whether or not a face motion trigger has been detected from any one of the sensors 214, 216, and 218 (S108). S108 will be described in detail. For example, the CPU 202 determines the state of the user's eyebrow (left eyebrow) included in the detection signal input from the eyebrow sensor 214 to the control box 200 and the state of the left eyebrow of the standard face image (see FIG. 4A). , A pattern matching process is executed to determine whether or not matching can be achieved between the two. Further, the CPU 202 determines between the state of the user's cheek (left cheek) included in the detection signal input from the cheek sensor 216 and the state of the upper left cheek of the standard face image (see FIG. 4A). Then, pattern matching processing is executed to determine whether matching can be obtained between the two. Furthermore, the CPU 202 determines between the mouth state (shape) of the user included in the detection signal input from the mouth sensor 218 and the mouth state (shape) of the standard face image (see FIG. 4A). Then, pattern matching processing is executed to determine whether matching can be obtained between the two.

  When matching is obtained in S108, in other words, when the state of the user's eyebrows, cheeks, and mouth matches the state of the left eyebrow indicated by the standard image, the CPU 202 has not input a face motion trigger. (S108: No), the process proceeds to S112. On the other hand, if any one of the eyebrows, cheeks, and mouth states of the user does not match the state of the left eyebrow indicated by the standard image, the CPU 202 determines that a face motion trigger has been input (S108: Yes), a state determination process is executed (S110).

  FIG. 6 shows a flow of the state determination process executed in S110. The CPU 202 that has started the state determination process determines whether or not the face motion trigger detected in S108 is based on a change in the state of the eyebrows (S200). Continuing the description of the determination in S200, the CPU 202 determines in S108 that the state of the left eyebrow of the user matches the state of the left eyebrow indicated by the standard image is not based on a change in the state of the eyebrow. (S200: No), the process proceeds to S204, and if both do not match, it is determined that the face motion trigger is based on a change in the state of the eyebrows (S200: Yes), and the process proceeds to S202.

  In S202, the CPU 202 executes the pattern matching process again to determine what the state of the user's left eyebrow included in the detection signal input to the control box 200 by the eyebrow sensor 214 is. . The determination in S202 will be specifically described. The CPU 202 determines that the eyebrow state of the user included in the detection signal input by the eyebrow sensor 214 is the eyebrow state registered in the table 2084 (see FIG. 4B). Which of the following is true is determined. As a result of the determination, for example, when the state of the eyebrows of the user included in the detection signal matches (matches) with the image indicating the raised state, the CPU 202 specifies the operation “next”. Similarly, when the state of the eyebrows of the user included in the detection signal matches the image indicating the close state, the CPU 202 specifies the operation “return to the previous”. After executing S202, the CPU 202 shifts the process to S210.

  In S204, the CPU 202 determines whether or not the face motion trigger detected in S108 is based on a change in cheek condition (S204). Continuing the description of the determination in S204, the CPU 202 determines in S108 that the user's left cheek state and the left cheek state indicated by the standard image are not based on a change in cheek state. However, if the process proceeds to S208 and the two do not match, it is determined that the face motion trigger is based on a change in cheek condition (S204: Yes), and the process proceeds to S206. .

  In step S <b> 206, the CPU 202 executes pattern matching processing to determine what the state of the user's left cheek included in the detection signal input to the control box 200 by the cheek sensor 216 is. The determination in S206 will be specifically described. The CPU 202 determines that the state of the cheek of the user included in the detection signal input by the cheek sensor 216 is the state of the cheek registered in the table 2084 (see FIG. 4C). Which of the following is true is determined. As a result of the determination, for example, when the state of the user's cheek included in the detection signal matches (matches) the image indicating the raised state, the CPU 202 specifies the operation “decision”. Similarly, when the state of the user's cheek included in the detection signal matches the image indicating the depressed state, the CPU 202 specifies the operation “cancel”. After executing S206, the CPU 202 shifts the processing to S210.

  In S <b> 208, the CPU 202 determines that the face motion trigger detected in S <b> 108 is based on a change in the mouth state (shape), and the user included in the detection signal input to the control box 200 by the mouth sensor 218. A pattern matching process is executed to determine whether the state of the mouth (lips) is “E”, “A”, or “O”. Continuing the description of the determination in S208, the CPU 202 determines that the state (shape) of the user's mouth included in the detection signal input by the mouth sensor 218 is the state of the mouth registered in the table 2084 (FIG. 4D). It is determined whether it corresponds to (see). As a result of the determination, for example, when the state of the mouth of the user included in the detection signal matches (matches) the image when “e” is uttered, the CPU 202 specifies the operation “brightness adjustment”. Similarly, when the state of the user's mouth included in the detection signal matches the image when “a” is uttered, the CPU 202 identifies the operation “brightness up” and when “o” is uttered If the image matches the image, the CPU 202 identifies the operation “brightness down”. After executing S206, the CPU 202 shifts the processing to S210.

  In S210, the CPU 202 performs control so that the operation specified in S202, S206, or S208 is executed in the content image output process being executed. For example, in the state where the content image including the work instruction related to the first process is presented, when the operation “proceed to the next” is specified in S202, the CPU 202 includes the content including the work instruction related to the second process. The rendering process is executed so that the image is presented, and the content image signal including the content image including the work instruction related to the second step is output from the input / output I / F 210 to the HMD 100. A content image based on a content image including work instructions relating to the two steps is optically emitted toward the half mirror 116. After executing S210, the CPU 202 ends the state determination process and proceeds to S112 (see FIG. 5).

  Returning to FIG. 5, in step S <b> 112, the CPU 202 determines whether the user has finished playing the content data 2082 via the operation unit 212, in other words, whether the content data has been processed to the end. . As a result of the determination, if the reproduction has not ended (S112: No), the process proceeds to S108. On the other hand, if the reproduction has ended (S112: Yes), the CPU 202 executes an end process of the content image signal output process. (S114), the main process is terminated.

  In the above description, the eyebrow sensor 214, the cheek sensor 216, and the mouth sensor 218 are described based on an example in which the image sensor is used. However, as described above, each of the sensors 214, 216, and 218 can be a photoelectric sensor. In this case, the intensity of the reflected light is registered in the table 2084 in association with each operation of the HMD 100, and the CPU 202 inputs the intensity registered in the table 2084 and the sensors 214, 216, 218 to the HMD 100. Based on the intensity of the reflected light, the state of the user's eyebrows, cheeks and mouth is determined. FIG. 7 conceptually shows the registration state in the table 2084 when the photoelectric sensor is employed, taking the “eyebrow state” as an example. In FIG. 7, “hatched rectangle” indicates a user's eyebrow, and “◯” indicates a “light spot” from the light emitting element.

  For example, the intensity (reference value) of the reflected light when the user's face is in the standard state is “brow: 80 ± 15 (65 to 95)” (see FIG. 7), “cheek: 120 ± 15 ( 105-135) ”and“ mouth: 100 ± 15 (85-115) ”, when the table 2084 is registered, the intensity of the reflected light from the eyebrow sensor 214, cheek sensor 216, and mouth sensor 218 is“ When detection signals including “brow: 82”, “cheek: 118”, and “mouth: 100” are respectively input to the control box 200, the CPU 202 detects a face motion trigger in S108 (see FIG. 5). It is determined that it has not been done (S108: No).

  Further, in the table 2084, for example, “110 ± 14 (96 ≦ intensity ≦ 124)” deviating from the reference value “80 ± 15 (65 to 95)” of the amount of change as the amount of change in the state of “raised eyebrows”. When the detection signal including that the intensity of the reflected light is “eyebrow: 115” is input from the eyebrow sensor 214 in the state registered as (see FIG. 7), the CPU 202 determines that S108 (see FIG. 5) and The determination in S200 (see FIG. 6) is affirmed (S108, S200: Yes), and the operation “next” is specified in S202.

  Furthermore, in the table 2084, for example, “50 ± 14 (36 ≦ strength ≦ 64)” that deviates from the reference value is registered as the amount of change in the “eyebrows” state (see FIG. 7). When the detection signal including that the intensity of the reflected light is “brow: 45” is input from the eyebrow sensor 214, the CPU 202 affirms the determination of S108 (see FIG. 5) and S200 (see FIG. 6) (S108). , S200: Yes), the operation “enlargement” is specified in S202.

  In the above description, the state of a specific part of the user's face alone included in the detection signal input to the control box 200 from each of the eyebrow sensor 214, the cheek sensor 216, and the mouth sensor 218, and the operation of the HMD 100. The description is based on the associated configuration. In addition to such a configuration, the following configuration may be adopted. For example, the operation of the HMD 100, for example, “fast forward” is set to a combination of a state where the eyebrows are raised and a state where the cheeks are raised. It can also be set as the structure matched. In this case, the CPU 202 determines that a face motion trigger has been input based on detection signals input from the eyebrow sensor 214 and cheek sensor 216 (see S108 in FIG. 5: Yes), and executes state determination processing (FIG. 5). (See FIG. 6 for details). In the state determination process, the CPU 202 makes the determination in S200 of FIG. 6 and, after affirming the determination (see S200: Yes in FIG. 6), again performs the same determination as S204 in FIG. If it is determined that the state is “raised the eyebrows while raising the cheeks”, the action “fast forward” is specified.

  Furthermore, in the above description, for example, the eyebrow sensor 214 is configured to detect the state (movement) of one eyebrow, specifically, the left eyebrow, but is configured to detect the state of the left and right eyebrows, The combination of the state of the right and left eyebrows, specifically, the state in which the left eyebrow is raised and the right eyebrow is lowered, and the operation of the HMD 100, for example, “reduction” may be associated. In this case, the CPU 202 determines that a face motion trigger has been input based on the detection signals input from the left and right eyebrow sensors 214 (see S108 in FIG. 5: Yes), and executes state determination processing (see FIG. 5). S110 (Refer to FIG. 6 for details). In the state determination process, the CPU 202 performs the determination in S200 of FIG. If the CPU 202 affirms the determination regarding the state of the right and left eyebrows (see S200 in FIG. 6: Yes), the CPU 202 determines that the left eyebrow is raised and the right eyebrow is lowered. Specify “reduction”.

  Further, in the above description, among the parts constituting the face of the user wearing the HMD 100, the specific parts that are brought into a specific state by the user's conscious movement, particularly the eyebrows, cheeks and mouth Limited. For example, it can also be set as the structure which detects the state of a forehead or a jaw.

  Further, operations realized by the HMD 100 and the control box 200 include “next”, “enlarge”, “return to previous”, “decision”, “cancel”, “brightness adjustment”, “brightness up”, and Although “brightness down” has been described as an example, other operations can be registered in the table 2084. For example, in a state where a plurality of content data 2082 is stored in the HDD 208, an operation related to “selection / confirmation of content data 2082 to be played back” or an operation related to “start or stop playback” of content data (in the above description, an operation An example of input via the unit 212. Regarding the start of reproduction (see S104 in FIG. 5) may be registered in the table 2084 in association with the state of the eyebrows, cheeks, or mouth. Further, an operation related to “power supply or power supply cutoff” to the HMD 100 or the control box 200 may be registered in association with a predetermined state such as an eyebrow.

(Advantageous effects based on the configuration of the embodiment)
According to the configuration of the HMD 100 and the control box 200 of the above embodiment, the state of the eyebrows, cheeks and mouth does not change unless the user wearing the HMD 100 moves consciously, and the HMD 100 and the control box 200 The movement to be realized is registered in association with each other, and the state of each part is sensed. When the state of the eyebrows, cheeks and mouth is the registered state, the movement associated with the state is executed. Adopted the configuration.

  According to this, a hands-free operation can be realized while preventing an operation instruction related to an operation that is not intended by the user from being input to the control box 200. Specifically, for example, in a state where a user is viewing a content image including a work instruction related to a first process for assembling a predetermined product, the user can use the HMD 100 or his / her hands while working. By operating the eyebrows without operating the control box 200, it is possible to visually recognize and recognize the work instruction relating to the second step.

(A) thru | or (c) is a figure which shows the external appearance of HMD in embodiment of this invention. The figure which shows the state which mounted | wore with HMD in embodiment of this invention. The figure which shows the functional block of the control box connected to HMD in embodiment of this invention (A) thru | or (d) is a figure which shows notionally the registration state of the table in embodiment of this invention. The figure which shows the flow of the main process in embodiment of this invention. The figure which shows the flow of the state determination process in embodiment of this invention The figure which shows notionally the registration state (registration state regarding the state of eyebrows) of the other table in embodiment of this invention.

Explanation of symbols

100 Head Mount Display (HMD)
116 Image presentation device 120a, 120b Ray (content image)
200 Control box 202 CPU
204 ROM
206 RAM
208 storage unit 2082 content data 2084 table 214 eyebrow sensor 216 cheek sensor 218 mouth sensor

Claims (6)

A head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, and allows the user to recognize the content image,
Control means for controlling the operation of the head mounted display;
Detecting means for detecting that the state of the specific part of the user's face is a predetermined state,
The detection means detects that the state of the user's eyebrows is a predetermined state,
The control means controls the operation associated with the predetermined state on condition that the detection means detects that the state of the eyebrows of the user is the predetermined state. Head mounted display. A head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, and allows the user to recognize the content image,
Control means for controlling the operation of the head mounted display;
Detecting means for detecting that the state of the specific part of the user's face is a predetermined state,
The detecting means detects that the state of the cheek of the user is a predetermined state,
The control unit controls the operation associated with the predetermined state on condition that the detection unit detects that the state of the cheek of the user is the predetermined state. Head mounted display. A head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, and allows the user to recognize the content image,
Control means for controlling the operation of the head mounted display;
Detecting means for detecting that the state of the specific part of the user's face is a predetermined state,
The detecting means detects that the mouth of the user is in a predetermined state;
The control unit controls the operation associated with the predetermined state on condition that the detection unit detects that the state of the mouth of the user is the predetermined state. Head mounted display. The detection means includes
Imaging means capable of imaging a specific image of a specific part of the user's face;
Analyzing means for analyzing the specific image picked up by the image pickup means,
4. The apparatus according to claim 1, wherein a state of a specific part of the user's face is a predetermined state according to an analysis result of the analysis unit. 5. Head mounted display. The detection means includes
A light emitting element capable of irradiating a specific part of the user's face;
A light detecting element for detecting reflected light reflected by the light irradiated on a specific part of the user's face,
When the intensity of the reflected light detected by the light detection element deviates from a predetermined reference value, it is detected that the state of a specific part of the user's face is a predetermined state. The head mounted display of any one of Claim 1 thru | or 3. The control means controls the first operation and the second operation as the operation of the head mounted display,
When the state of the specific part of the user's face is the first state, the detecting unit detects the first state as the predetermined state, and the state of the specific part of the user's face is In the second state, the second state is detected as the predetermined state,
The control unit controls the first operation associated with the first state on the condition that the detection unit detects that the first state is in the first state, and the detection unit is in the second state. 6. The head mounted display according to claim 1, wherein the second operation associated with the second state is controlled on the condition that it is detected.

Images