WO2025204110A1 - Eyeglasses-type device, information processing method, and program - Google Patents

Abstract

This eyeglasses-type device comprises: a front section; a temple section that is connected to the front section; an end tip section that is connected to the temple section; a first contact section that comes into contact with the nasal root or glabella of a user and supports the front section; a second contact section that has a connection section connected to the end tip section, comes into contact with the chin of the user, and supports the end tip section; and at least one of a first sensor and a second sensor, wherein the first sensor is disposed in the first contact section and acquires a first biological signal of the user from the nasal root or glabella of the user, and the second sensor is disposed in the second contact section and acquires a second biological signal of the user from the chin of the user.

Description

Glasses-type device, information processing method, and program

This disclosure relates to an eyeglass-type device, an information processing method, and a program.

Patent Document 1 discloses electronic eyeglasses according to background art. The frames of the electronic eyeglasses are equipped with sensors that perform sensing operations. The left and right temples of the electronic eyeglasses are connected to each other by a connection means that crosses the back of the user's head.

The electronic eyeglasses disclosed in Patent Document 1 have a connection means that crosses the back of the user's head, which gives the user the feeling that their head is being squeezed, causing a great mental strain.

Patent No. 6046005

The present disclosure aims to provide an eyeglass-type device, information processing method, and program that can reduce misalignment of the eyeglass-type device without using a connection means that crosses the back of the user's head.

A glasses-type device according to one aspect of the present disclosure comprises a front section, temple sections connected to the front section, end pieces connected to the temple sections, a first contact section that contacts the bridge of the user's nose or between the eyebrows to support the front section, a second contact section that has a connection section connected to the end pieces and contacts the area under the user's chin to support the end pieces, and at least one of a first sensor that is disposed in the first contact section and acquires a first biosignal of the user from the bridge of the user's nose or between the eyebrows, and a second sensor that is disposed in the second contact section and acquires a second biosignal of the user from the area under the user's chin.

FIG. 1 is a front view showing a simplified configuration example of AR glasses according to an embodiment. FIG. 1 is a side view showing a simplified configuration example of AR glasses according to an embodiment. FIG. 2 is a simplified diagram illustrating a first configuration example of a connection portion. FIG. 10 is a simplified diagram illustrating a second configuration example of the connection portion. FIG. 1 is a diagram showing a simplified functional configuration of AR glasses. FIG. 2 is a simplified diagram showing the functional configuration of a processing unit. 10 is a flowchart showing a process executed by a processing unit. FIG. 10 is a diagram showing a simplified functional configuration of AR glasses according to a first modified example. FIG. 10 is a diagram showing a simplified functional configuration of a processing unit according to a first modified example. FIG. 10 is a diagram showing an image of another user's face captured by an external camera. FIG. 10 is a side view showing an example of how the mask is worn. FIG. 10 is a diagram showing an example of displaying the faces of other users on the display unit. FIG. 10 is a diagram showing a simplified functional configuration of a processing unit according to a second modified example. FIG. 10 is a diagram showing a simplified configuration example of a connection portion according to a third modified example. FIG. 10 is a simplified diagram showing a first configuration example of AR glasses according to a fourth modified example. FIG. 10 is a simplified diagram showing a second configuration example of AR glasses according to a fourth modified example. FIG. 10 is a simplified diagram showing a third configuration example of AR glasses according to a fourth modified example. FIG. 10 is a simplified diagram showing a fourth configuration example of AR glasses according to a fourth modified example. FIG. 10 is a simplified diagram showing a fifth configuration example of AR glasses according to a fourth modified example.

(Findings that form the basis of this disclosure)
Glasses-type devices such as AR (Augmented Reality) glasses or VR (Virtual Reality) glasses are becoming commercially available.

General eyeglass-type devices are designed to support the glasses with the temples and nose pads, which can cause the glasses to slip off easily.

As with the electronic eyeglasses in the background art, misalignment of the eyeglasses can be reduced by connecting the left and right temples with a connection means that crosses the back of the user's head.

However, a structure that uses a connection means that crosses the back of the user's head can give the user the feeling that their head is being squeezed, which places a significant mental strain on the user.

In order to solve this problem, the inventor discovered that by supporting the front section by contacting the bridge of the nose or between the eyebrows and supporting the temple sections by contacting the area under the chin, it is possible to reduce misalignment of the eyeglass-type device without using a connection means that crosses the back of the head, and this led to the present disclosure.

Next, each aspect of this disclosure will be described.

A glasses-type device according to a first aspect of the present disclosure comprises a front section, temple sections connected to the front section, end pieces connected to the temple sections, a first contact section that contacts the bridge of the user's nose or between the eyebrows to support the front section, a second contact section that has a connection section connected to the end pieces and contacts the area under the user's chin to support the end pieces, and at least one of a first sensor that is disposed in the first contact section and acquires a first biosignal of the user from the bridge of the user's nose or between the eyebrows, and a second sensor that is disposed in the second contact section and acquires a second biosignal of the user from the area under the user's chin.

According to the first aspect, misalignment of the eyeglass-type device can be reduced without using a connection means that crosses the back of the user's head.

The eyeglass-type device according to the second aspect of the present disclosure may be the first aspect, but may also include both the first sensor and the second sensor.

According to the second aspect, the user's intention to operate the object to be operated can be estimated with high accuracy.

In the eyeglass-type device according to the third aspect of the present disclosure, in the first or second aspect, the connection portion biases the end piece in a direction that pushes up the temple portion, using the end piece as a fulcrum, and the first contact portion preferably has an inter-brow pad that contacts the user's between the eyebrows from below.

According to the third aspect, the effect of reducing misalignment of the eyeglass-type device can be improved.

In the eyeglass-type device according to the fourth aspect of the present disclosure, in the third aspect, the connection portion may have a biasing member that biases the end piece in a direction that linearly attracts the end piece toward the second contact portion.

According to the fourth aspect, a biasing force can be applied in a direction that pushes up the temple portion, with the end piece serving as a fulcrum.

In the eyeglass-type device according to the fifth aspect of the present disclosure, in the third aspect, the connection portion may have a biasing member that biases the end piece in a direction that rotates the end piece backward relative to the second contact portion.

According to the fifth aspect, a biasing force can be applied in a direction that pushes up the temple portion, with the end piece acting as a fulcrum.

In the eyeglass-type device according to the sixth aspect of the present disclosure, in the first or second aspect, the connection portion may bias the end piece in a direction that presses down the temple portion using the end piece as a fulcrum, and the first contact portion may have a nose bridge pad that contacts the bridge of the user's nose from above.

According to the sixth aspect, the effect of reducing misalignment of the eyeglass-type device can be improved.

A glasses-type device according to a seventh aspect of the present disclosure is the sixth aspect, wherein the connection portion has a biasing member that biases the end piece in a direction that rotates the end piece forward relative to the second contact portion.

According to the seventh aspect, a biasing force can be applied in a direction that pushes down the temple portion, with the end piece serving as a fulcrum.

In the second aspect of the eyeglass-type device according to the eighth aspect of the present disclosure, the second biological signal may include myoelectric potential of the user's platysma muscle.

According to the eighth aspect, the user's intention to operate the object to be operated can be estimated with high accuracy.

The eyeglass-type device according to the ninth aspect of the present disclosure may be the eighth aspect, further comprising a movement estimation unit that estimates the movement of the user's mouth, tongue, or throat based on the second biological signal.

According to the ninth aspect, the movement of the user's mouth, tongue, or throat can be estimated with high accuracy based on the second biological signal.

The glasses-type device according to a tenth aspect of the present disclosure may be the ninth aspect, further comprising an intention estimation unit that estimates the user's intention to operate an object to be operated, based on movement information indicating the movement of the user's mouth, tongue, or throat estimated by the movement estimation unit.

According to the tenth aspect, the user's intention to operate an object to be operated can be estimated with high accuracy based on movement information indicating the movement of the user's mouth, tongue, or throat.

The glasses-type device according to the eleventh aspect of the present disclosure may be the tenth aspect, further comprising a gaze detection unit that detects the direction of the user's gaze, and an object identification unit that identifies the object based on gaze information indicating the direction of the user's gaze detected by the gaze detection unit.

According to the eleventh aspect, the object to be operated can be identified with high accuracy based on gaze information indicating the direction of the user's gaze.

In any one of the ninth to eleventh aspects, the eyeglass-type device according to the twelfth aspect of the present disclosure may further include a transmitter that transmits movement information indicating the movement of the user's mouth, tongue, or throat estimated by the movement estimation unit.

According to the twelfth aspect, the movement information indicating the movement of the user's mouth, tongue, or throat transmitted by the transmission unit can be utilized by other eyeglass-type devices or any information processing device, etc.

The eyeglass-type device according to the thirteenth aspect of the present disclosure may further include a receiving unit that receives, from another transmitting unit provided in the other eyeglass-type device, other movement information indicating the movement of the mouth, tongue, or throat of another user wearing the other eyeglass-type device; an image creating unit that, when it is detected that the other user is wearing a mask, creates a facial expression image of the area around the mouth of the other user based on the other movement information received by the receiving unit; and a display control unit that displays the facial expression image created by the image creating unit in accordance with the position of the other user's face.

According to the thirteenth aspect, an image of the facial expression around the mouth of another user wearing a mask can be created and displayed, thereby facilitating communication between users.

In the eyeglass-type device according to a fourteenth aspect of the present disclosure, in any one of the first to thirteenth aspects, the front section preferably has a display section that displays information, and further includes a misplacement detection section that detects misplacement of the second contact section, and a display control section that, when the misplacement detection section detects misplacement of the second contact section, displays notification information on the display section that notifies the user of the misplacement.

According to the fourteenth aspect, malfunctions or reduced accuracy caused by incorrect attachment of the second contact portion can be prevented.

In the eyeglass-type device according to a fifteenth aspect of the present disclosure, in any one of the first to fourteenth aspects, the first biological signal may include at least one of the user's electromyography, electrooculography, electroencephalography, changes in muscle shape, changes in muscle hardness, sounds generated by the living body, skin temperature, skin conductivity, respiration, sweating, heart rate, and blood pressure.

According to the fifteenth aspect, the user's intention to operate the object to be operated can be estimated with high accuracy.

An information processing method according to a sixteenth aspect of the present disclosure is an information processing method for an eyeglass-type device comprising a front section, temple sections connected to the front section, end pieces connected to the temple sections, a first contact section that contacts the bridge of the user's nose or between the eyebrows to support the front section, a second contact section having a connection section connected to the end pieces and that contacts under the user's chin to support the end pieces, a first sensor that is disposed in the first contact section and acquires a first biometric signal of the user from the bridge of the user's nose or between the eyebrows, and a second sensor that is disposed in the second contact section and acquires a second biometric signal of the user from under the user's chin. The information processing method is executed by an information processing device mounted on the eyeglass-type device, and acquires the first biometric signal from the first sensor and the second biometric signal from the second sensor, and estimates the user's operational intention with respect to an object to be operated based on the acquired first biometric signal and second biometric signal, and performs an operation on the object corresponding to the estimated operational intention.

According to the sixteenth aspect, misalignment of the eyeglass-type device can be reduced, thereby suppressing errors in acquiring the first biological signal and the second biological signal. As a result, the user's intention to operate the object to be operated can be estimated with high accuracy.

A seventeenth aspect of the present disclosure provides a program for causing an information processing device mounted on an eyeglass-type device to execute processing in the eyeglass-type device, the program comprising: a front section; temple sections connected to the front section; end sections connected to the temple sections; a first contact section that contacts the bridge of the user's nose or between the eyebrows to support the front section; a second contact section having a connection section connected to the end sections and that contacts the user's under-chin to support the end sections; a first sensor disposed in the first contact section to acquire a first biometric signal of the user from the bridge of the user's nose or between the eyebrows; and a second sensor disposed in the second contact section to acquire a second biometric signal of the user from under the user's chin, the program acquiring the first biometric signal from the first sensor, acquiring the second biometric signal from the second sensor, estimating the user's operational intention with respect to an object to be operated based on the acquired first and second biometric signals, and performing an operation on the object corresponding to the estimated operational intention.

According to the seventeenth aspect, misalignment of the eyeglass-type device can be reduced, thereby suppressing errors in acquiring the first biological signal and the second biological signal. As a result, the user's intention to operate the object to be operated can be estimated with high accuracy.

The present disclosure can also be realized as a program that causes a computer to execute each of the characteristic components included in such a method or apparatus, or as a system operated by this program. It goes without saying that such a computer program can also be distributed on a computer-readable, non-transitory recording medium such as a CD-ROM, or via a communications network such as the Internet.

(Embodiments of the present disclosure)
Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Elements with the same reference numerals in different drawings indicate the same or corresponding elements. Furthermore, the components, the layout positions of the components, the connection forms, the order of operations, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. The present disclosure is limited only by the claims. Therefore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept of the present disclosure are not necessarily required to achieve the objectives of the present disclosure, but are described as constituting more preferred forms.

In this embodiment, an example will be described in which AR glasses are used as the target of application of this disclosure, but the target of application of this disclosure is not limited to this. This disclosure is widely applicable to eyeglass-type devices worn by a user that use cross-reality technology such as AR, VR, or MR. Furthermore, this disclosure is widely applicable to eyeglass-type devices in general that acquire biosignals, even if they do not use cross-reality technology.

FIGS. 1 and 2 are respectively a front view and a side view showing a simplified example of the configuration of AR glasses 1 according to an embodiment of the present disclosure, worn by a user.

The AR glasses 1 comprise a front portion 11, temple portions 12, and end pieces 13. The front portion 11 has left and right rims 111 to which lenses 113 are fixed, and a bridge 112 connecting the inner ends of the left and right rims 111. The front ends of the temple portions 12 are connected to the outer ends of the rims 111. The rear ends of the temple portions 12 are connected to the upper ends of the end pieces 13. The end pieces 13 have a curved structure that makes them easy to wear over the user's ears.

The AR glasses 1 also have a first contact portion 21 and a second contact portion 22. The first contact portion 21 comes into contact with the user's eyebrows and supports the front portion 11. The second contact portion 22 comes into contact with the user's chin and supports the temple portion 13.

The first contact portion 21 has an glabellar pad 211 and an electrode 212. The glabellar pad 211 is a pad that contacts the skin surface between the user's eyebrows from below, following the slope of the space between the eyebrows. The glabellar pad 211 is connected to the bridge 112 or rim 111 via a pad arm (not shown). The electrode 212 is arranged on the glabellar pad 211. Multiple electrodes 212 may be arranged on one glabellar pad 211. The electrode 212 is included in the first sensor 33, which will be described later. The first sensor 33 acquires a first biological signal D5 from between the user's eyebrows via the electrode 212. The first biological signal D5 includes at least one of the following: electromyography (EMG) of the user's proximal muscle, electrooculography (EOG), electroencephalogram (EEG), changes in muscle shape, changes in muscle hardness, sounds generated by the living body, skin temperature, skin conductivity, respiration, sweating, heart rate, and blood pressure.

The second contact portion 22 has a subchin pad 221, an electrode 222, an arm 223, and a connection portion 224. The subchin pad 221 is a pad that contacts the skin surface under the user's chin from below, following the slope of the area under the chin. The subchin pad 221 is connected to the lower end of the arm 223. The arm 223 is made of, for example, resin. The upper end of the arm 223 is connected to the lower end of the connection portion 224. The upper end of the connection portion 224 is connected to the lower end of the end piece 13. The electrode 222 is disposed on the subchin pad 221. Multiple electrodes 222 may be disposed on one subchin pad 221. The electrode 222 is included in the second sensor 34, which will be described later. The second sensor 34 acquires a second biosignal D6 from under the user's chin via the electrode 222. The second biosignal D6 includes the myoelectric potential of the user's platysma muscle.

Figure 3 is a simplified diagram showing a first example configuration of the connection portion 224. The connection portion 224 has a biasing member 224A that uses a coil spring or the like. The biasing member 224A biases the end piece 13 in a direction that linearly attracts the lower end of the end piece 13 toward the second contact portion 22 (the direction indicated by arrow Y1). As a result, the connection portion 224 biases the end piece 13 in a direction that pushes up the temple portion 12 (the direction indicated by arrow Y2) with the end piece 13 as a fulcrum, resulting in the glabella pad 211 coming into close contact with the user's eyebrows.

Figure 4 is a simplified diagram showing a second example configuration of the connection part 224. The connection part 224 has a biasing member 224B that uses a spiral spring or the like. The biasing member 224B biases the end piece 13 in a direction that rotates the lower end of the end piece 13 backward relative to the second contact part 22 (the direction indicated by arrow Y3). As a result, the connection part 224 biases the end piece 13 in a direction that pushes up the temple part 12 (the direction indicated by arrow Y2) with the end piece 13 as a fulcrum, resulting in the glabella pad 211 coming into close contact with the user's eyebrows.

Figure 5 is a simplified diagram showing the functional configuration of the AR glasses 1. The AR glasses 1 include a display unit 31, a processing unit 32, a first sensor 33, a second sensor 34, a memory unit 35, and an internal camera 36. The display unit 31 is configured using an LCD display or an organic EL display, etc., and the inner surface of the lens 113 serves as the display surface of the display unit 31. The processing unit 32 is configured using a processor (information processing device) such as a CPU. The first sensor 33 acquires a first biosignal D5 from between the user's eyebrows via an electrode 212. The second sensor 34 acquires a second biosignal D6 from under the user's chin via an electrode 222. The memory unit 35 is configured using a semiconductor memory, etc. The internal camera 36 is configured using an optical system and a CMOS image sensor, etc. The AR glasses 1 may include either the first sensor 33 or the second sensor 34, or both.

The memory unit 35 stores a program 41 and graphic data 42. The graphic data 42 includes image data of objects for eye-gaze operation, such as icons, displayed on the display unit 31. The memory unit 35 includes a computer-readable non-volatile storage medium. The program 41 is stored in this storage medium.

The internal camera 36 is positioned on the inside side of the rim 111 (toward the user's face). The internal camera 36 captures an image of the eyes of the user wearing the AR glasses 1 and outputs the image data D1.

FIG. 6 is a simplified diagram showing the functional configuration of the processing unit 32. The processing unit 32 has a gaze detection unit 51, a display control unit 52, an object identification unit 53, a movement estimation unit 54, an intention estimation unit 55, and a gaze operation execution unit 56, which are functions realized by the processor executing the program 41 read from the storage unit 35.

Figure 7 is a flowchart showing the processing performed by the processing unit 32.

First, in step S01, the gaze detection unit 51 detects the direction of the user's gaze based on image data D1 acquired from the internal camera 36. The gaze detection unit 51 inputs data D2 indicating gaze information including the direction of the user's gaze to the object identification unit 53.

Next, in step S02, the object identification unit 53 identifies the object that is the target of the user's gaze operation based on the data D2 input from the gaze detection unit 51 and the data D3 input from the display control unit 52. The data D3 includes the position coordinates of the object displayed on the display unit 31. Multiple objects may be displayed simultaneously on the display unit 31. The object identification unit 53 inputs data D4 indicating the identified object to the gaze operation execution unit 56.

Next, in step S03, the movement estimation unit 54 inputs the second bio-signal D6 acquired from the second sensor 34 into an estimation model such as a machine-learned neural network, thereby estimating the movement of the user's mouth, tongue, or throat based on the second bio-signal D6. The movement estimation unit 54 inputs data D7 indicating movement information including the estimated movement of the user's mouth, tongue, or throat to the intention estimation unit 55.

Next, in step S04, the intention estimation unit 55 estimates the user's intention in the gaze operation based on the first biometric signal D5 acquired from the first sensor 33 and the data D7 input from the movement estimation unit 54. The user's intention includes, for example, the intention to select an object to be operated. The intention estimation unit 55 estimates the user's intention based on the first biometric signal D5 and the data D7 by inputting the first biometric signal D5 and the data D7 into an estimation model such as a machine-learned neural network. The intention estimation unit 55 inputs data D8 indicating the estimated user's intention to the gaze operation execution unit 56.

Next, in step S05, the gaze operation execution unit 56 determines whether the user intends to perform a gaze operation based on the data D8 input from the intention estimation unit 55.

If the user does not intend to perform gaze control (step S05: NO), the process returns to step S01.

If the user intends to perform a gaze operation (step S05: YES), then in step S06, the gaze operation execution unit 56 performs a gaze operation corresponding to the intention indicated by the data D8 input from the intention estimation unit 55 on the object indicated by the data D4 input from the object identification unit 53.

According to this embodiment, the first contact portion 21 contacts the user's forehead and supports the front portion 11, while the second contact portion 22 contacts the area under the user's chin and supports the temple portion 13. This reduces misalignment of the AR glasses 1 without using a connection means that crosses the back of the user's head.

Furthermore, according to this embodiment, the user's operational intention with respect to the object to be operated can be estimated with high accuracy based on the first biological signal D5 and the second biological signal D6.

Furthermore, according to this embodiment, the connection portion 224 biases the end piece 13 in a direction that pushes up the temple portion 12, with the end piece 13 as a fulcrum, so that the glabella pad 211 comes into close contact with the user's forehead, thereby improving the effectiveness of reducing misalignment of the AR glasses 1.

Furthermore, the biasing member 224A shown in Figure 3 can appropriately apply a biasing force in the direction of pushing up the temple portion 12 with the end piece 13 as a fulcrum.

Furthermore, the biasing member 224B shown in Figure 4 can appropriately apply a biasing force in the direction of pushing up the temple portion 12 with the end piece 13 as a fulcrum.

Furthermore, according to this embodiment, the first biological signal D5 includes at least one of the electromyography of the user's proximal muscle, electrooculography, electroencephalogram, changes in muscle shape, changes in muscle hardness, sounds generated by the living body, skin temperature, skin conductivity, respiration, sweating, heart rate, and blood pressure, making it possible to estimate the user's operational intention with respect to the object being operated with high accuracy.

Furthermore, according to this embodiment, the second biological signal D6 includes myoelectric potential of the user's platysma muscle, making it possible to estimate the user's operational intention with respect to the object to be operated with high accuracy.

Furthermore, the movement estimation unit 54 according to this embodiment can estimate the movement of the user's mouth, tongue, or throat with high accuracy based on the second biological signal D6.

Furthermore, the intention estimation unit 55 according to this embodiment can estimate with high accuracy the user's intention to operate the object to be operated, based on movement information (data D7) indicating the movement of the user's mouth, tongue, or throat.

Furthermore, the object identification unit 53 according to this embodiment can identify the object to be operated with high accuracy based on line-of-sight information (data D2) indicating the direction of the user's line of sight.

Various variations of this disclosure are described below. The variations described below can be applied in any combination.

(First Modification)
8 is a simplified diagram showing the functional configuration of the AR glasses 1 according to the first modified example. The AR glasses 1 further include an external camera 37 in addition to the configuration shown in FIG.

The external camera 37 is positioned on the external side of the rim 111 (the side opposite the user's face). The external camera 37 captures an image corresponding to the field of view of the user wearing the AR glasses 1 and outputs the image data D10.

Figure 9 is a simplified diagram showing the functional configuration of the processing unit 32 according to the first modified example. The processing unit 32 further includes a transmitting unit 57, a receiving unit 58, and an image creating unit 59 in addition to the configuration shown in Figure 6.

The transmission unit 57 transmits movement information (data D7) indicating the movement of the user's mouth, tongue, or throat estimated by the movement estimation unit 54 to the other AR glasses 1A. The other AR glasses 1A are, for example, AR glasses worn by another user who is a face-to-face conversation partner with the user wearing the AR glasses 1.

The receiving unit 58 receives other movement information (data D7A) indicating the movement of the mouth, tongue, or throat of another user wearing another pair of AR glasses 1A from another transmitting unit 57A provided in the other pair of AR glasses 1A. The receiving unit 58 inputs the data D7A to the image creating unit 59.

The image creation unit 59 creates image data D11 of the facial expression image of the other user based on image data D10 acquired from the external camera 37 and data D7A input from the receiving unit 58. The image data D10 includes an image of the other user's face captured by the external camera 37. When the image creation unit 59 detects that the other user is wearing a mask 60 based on the image data D10, it creates an image of the facial expression around the mouth of the other user based on other movement information (data D7A) received by the receiving unit 58. The image creation unit 59 inputs the image data D11 of the created facial expression image to the display control unit 52.

Figure 10 shows an image of the face of another user captured by the external camera 37. The other user is wearing a mask 60, and the other user's nose and mouth are hidden by the mask 60.

Figure 11 is a side view showing an example of how the mask 60 is worn. Both left and right ends of the mask 60 are fixed to the arms 223 of the second contact portion 22 using any fastener such as a magnet, button, or string. This allows the mask 60 to be worn with a stronger holding force than when the mask strings are hooked onto the ears. Furthermore, the upper end of the mask 60 may be fixed to the lower end of the rim 111 using any of the above fasteners.

Referring to FIG. 9, the display control unit 52 superimposes and displays an image of the facial expression around the mouth of another user, created by the image creation unit 59 based on the image data D11, on the display surface of the display unit 31, in accordance with the position of the face of the other user.

Figure 12 is a diagram showing an example of the display of another user's face on the display unit 31. An image of the facial expression around the mouth of the other user is displayed on the display surface of the display unit 31, as if seen through a mask 60.

The AR glasses 1 may further include an image projection unit, which may use the mask 60 worn by the other user as a screen and project an image of the other user's facial expression (image data D11) onto the screen. In this case, the material of the mask 60 may be a retroreflective material.

According to this modified example, the movement information (data D7) indicating the movement of the user's mouth, tongue, or throat transmitted by the transmission unit 57 can be used by other AR glasses 1A or any information processing device, etc.

Furthermore, according to this modified example, it is possible to create and display an image of the facial expression around the mouth of another user wearing the mask 60, thereby facilitating communication between users.

The above-mentioned optional information processing device may also be a management device that manages the user's health. The management device receives images (image data D10) captured by the external camera 37 from the AR glasses 1 and, based on the captured images, determines whether the user is currently eating and what they are chewing. Based on data D7 received from the AR glasses 1, the management device measures the user's jaw movement while chewing, the number of times they swallow, or the timing of swallowing. Data D7 may also include measurements taken by a sensor that can measure the degree to which the user's jaw is opened and closed. Because users who chew less frequently are at higher health risk, the management device issues a warning to such users and encourages them to improve their chewing behavior. Health insurance premiums may be increased or decreased in real time depending on the number of chews. Furthermore, users who have abnormalities in their chewing behavior, such as asymmetry, may have dental or oral diseases. The management device issues a warning to such users and encourages them to visit a hospital. Furthermore, in a care facility or the like, if the management device detects a user's aspiration based on data D7, it may output an alarm or send an emergency call to the staff in charge.

Furthermore, the arbitrary information processing device may be an information terminal carried by a language learning instructor, such as an English conversation instructor. The instructor may be a real instructor who is face-to-face with the user, a remote instructor who can communicate with the user, or a virtual instructor. A user wearing AR glasses 1 can have a real conversation with the instructor displayed on the display unit 31. At that time, data D7 indicating the user's jaw or tongue movements is sent from the AR glasses 1 to the information terminal carried by the instructor. The instructor determines whether the user's jaw or tongue movements are correct based on the received data D7, and provides feedback on the correct pronunciation method to the AR glasses 1 worn by the user. This makes it possible to provide instruction based on jaw and tongue movements in addition to instruction based on differences in pronunciation due to the ear, which is expected to promote language learning.

(Second Modification)
13 is a simplified diagram showing the functional configuration of the processing unit 32 according to the second modified example. The processing unit 32 further includes a mis-attachment detection unit 71 in addition to the configuration shown in FIG.

The mis-wearing detection unit 71 detects that the second contact unit 22 has been worn incorrectly by the user, for example, when the detection value of the electrode 222 is an abnormal value. An example of mis-wearing is when the second contact unit 22 is worn turned toward the back of the user's head. When the mis-wearing detection unit 71 detects that the second contact unit 22 has been worn incorrectly, it inputs mis-wearing detection information D20 to the display control unit 52. When the display control unit 52 receives detection information D20 from the mis-wearing detection unit 71, it displays notification information such as an image or text message on the display unit 31 to notify the user of the mis-wearing. The manner in which the mis-wearing is notified is not limited to a display, and may also be the output of a warning sound or voice message, etc.

This modified example makes it possible to prevent malfunction of the AR glasses 1 or a decrease in estimation accuracy in gaze control due to incorrect attachment of the second contact portion 22.

As a simple measure to prevent incorrect attachment of the second contact portion 22, a message such as "Please attach under the chin" may be written on the surface of the subchin pad 221.

Furthermore, to allow the user to easily wear the second contact portion 22, the connection portion 224 may be configured to be stretchable or have a controllable tensile strength. For example, ease of wearing may be ensured by weakening the tensile strength before wearing, and the degree of adhesion of the subchin pad 221 may be increased by strengthening the tensile strength after sensing that the pad has been worn. A shape memory alloy that reacts to the user's body temperature may be used to variably control the tensile strength, or an actuator such as a motor may be used.

Furthermore, even if the second contact portion 22 is correctly attached, if the detection values of the electrodes 212, 222 change significantly due to misalignment of the AR glasses 1, notification information may be output to prompt the user to reattach the AR glasses 1. To detect misalignment of the AR glasses 1, not only the detection values of the electrodes 212, 222 but also detection information of the user's eye position by the internal camera 36 may be used.

(Third Modification)
FIG. 14 is a simplified diagram showing an example configuration of the connection unit 224 according to the third modified example. The first contact unit 21 has a nasal root pad 213 and an electrode 214. The nasal root pad 213 is a pad that contacts the skin surface of the user's nasal root from above, following the slope of the nasal root. The nasal root pad 213 is connected to the bridge 112 or the rim 111 via a pad arm (not shown). The electrode 214 is disposed on the nasal root pad 213. Multiple electrodes 214 may be disposed on one nasal root pad 213. The electrode 214 is included in the first sensor 33. The first sensor 33 acquires a first biological signal D5 from the user's nasal root via the electrode 214.

The connection portion 224 has a biasing member 224C that uses a spiral spring or the like. The biasing member 224C biases the end piece 13 in a direction that rotates the lower end of the end piece 13 forward relative to the second contact portion 22 (the direction indicated by arrow Y4). As a result, the connection portion 224 biases the end piece 13 in a direction that presses down on the temple portion 12 (the direction indicated by arrow Y5) with the end piece 13 as a fulcrum, resulting in the nose bridge pad 213 coming into close contact with the bridge of the user's nose.

In this modified example, the first contact portion 21 contacts the bridge of the user's nose to support the front portion 11, and the second contact portion 22 contacts the area under the user's chin to support the temple portion 13. This reduces misalignment of the AR glasses 1 without using a connection means that crosses the back of the user's head.

Furthermore, according to this modified example, the connection portion 224 biases the end piece 13 in a direction that presses down the temple portion 12, with the end piece 13 as a fulcrum, so that the nose bridge pad 213 adheres closely to the bridge of the user's nose, thereby improving the effect of reducing misalignment of the AR glasses 1.

Furthermore, the biasing member 224C shown in Figure 14 can appropriately apply a biasing force in a direction that presses down the temple portion 12 with the end piece 13 as a fulcrum.

(Fourth Modification)
15 is a simplified diagram showing a first configuration example of the AR glasses 1 according to the fourth modification. The AR glasses 1 may be monocle style glasses worn on only one of the left or right eye.

Figure 16 is a simplified diagram showing a second configuration example of AR glasses 1 relating to the fourth modified example. The AR glasses 1 may have a connection part 225 that connects the temple part 12 and the arm 223 forward of the position of the user's ear. The connection part 225 may simply be a string. When the connection part 225 pulls the temple part 12 downward as indicated by arrow Y6, the connection part 224 urges the front part 11 and temple part 12 horizontally backward as indicated by arrow Y7. As a result, the nasal root pad 213 comes into close contact with the root of the user's nose. Note that the nasal root pad 213 may be replaced with an inter-brow pad 211.

Figure 17 is a simplified diagram showing a third configuration example of AR glasses 1 relating to the fourth modified example. The connection portion 225 has a biasing member 226 that uses a coil spring or the like. The biasing member 226 biases the temple portion 12 in a direction that linearly attracts the temple portion 12 toward the connection portion 225 (the direction indicated by arrow Y6).

Figure 18 is a simplified diagram showing a fourth configuration example of AR glasses 1 relating to the fourth modified example. The second contact portion 22 has a string 227 made of a stretchable material such as rubber, instead of the arm 223 and connection portion 224. Because the string 227 itself is stretchable, the second contact portion 22 fits better to the user's jawline. This allows the subchin pad 221 to fit securely and closely under the user's chin, and the pressure is distributed, reducing the burden on the user.

Figure 19 is a simplified diagram showing a fifth configuration example of AR glasses 1 relating to the fourth modified example. The second contact portion 22 has been omitted from the configuration shown in Figure 14. Furthermore, instead of electrode 222, the temple portion 12 has electrode 121, and conductive paste 122 is applied from electrode 121 to under the user's chin. The myoelectric signal generated under the user's chin is measured by electrode 121 via conductive paste 122.

The present disclosure is widely applicable to eyeglass-type devices worn by a user, such as AR glasses, VR glasses, smart glasses, or head-mounted displays.

Claims (17)

The front part and
a temple portion connected to the front portion;
an end piece connected to the temple piece;
a first contact portion that contacts the base of the nose or the space between the eyebrows of the user to support the front portion;
a second contact portion having a connection portion connected to the end cap portion and contacting a portion under the user's chin to support the end cap portion;
At least one of a first sensor that is disposed at the first contact portion and acquires a first biosignal of the user from the root of the nose or between the eyebrows of the user, and a second sensor that is disposed at the second contact portion and acquires a second biosignal of the user from under the chin of the user;
An eyeglass-type device comprising: The arrangement includes both the first sensor and the second sensor.
The eyeglass-type device according to claim 1 . the connection portion biases the end piece in a direction pushing up the temple portion with the end piece as a fulcrum,
The first contact portion has an inter-eyebrow pad that contacts the area between the eyebrows of the user from below.
The eyeglass-type device according to claim 1 . The connection portion has a biasing member that biases the end piece in a direction that linearly attracts the end piece to the second contact portion.
The eyeglass-type device according to claim 3 . The connection portion has a biasing member that biases the end cap in a direction that rotates the end cap rearward relative to the second contact portion.
The eyeglass-type device according to claim 3 . the connection portion biases the end piece in a direction that presses down the temple portion with the end piece as a fulcrum,
The first contact portion has a nose bridge pad that contacts the user's nose bridge from above.
The eyeglass-type device according to claim 1 . The connection portion has a biasing member that biases the end cap in a direction that rotates the end cap forward relative to the second contact portion.
The eyeglass-type device according to claim 6 . The second biological signal includes a myoelectric potential of the user's platysma muscle.
The eyeglass-type device according to claim 2 . Further comprising a movement estimation unit that estimates a movement of the mouth, tongue, or throat of the user based on the second biological signal.
The eyeglass-type device according to claim 8 . an intention estimation unit that estimates an operation intention of the user with respect to an object to be operated, based on movement information indicating a movement of the user's mouth, tongue, or throat estimated by the movement estimation unit;
The eyeglass-type device according to claim 9 . a gaze detection unit that detects the direction of the user's gaze;
an object identification unit that identifies the object based on gaze information indicating a direction of the user's gaze detected by the gaze detection unit;
Further provided with
The eyeglass-type device according to claim 10. a transmitter for transmitting movement information indicating a movement of the user's mouth, tongue, or throat estimated by the movement estimation unit;
The eyeglass-type device according to claim 9 . a receiving unit that receives other motion information indicating a motion of a mouth, tongue, or throat of another user wearing another eyeglass-type device from another transmitting unit included in the other eyeglass-type device;
an image creation unit that, when it is detected that the other user is wearing a mask, creates an image of a facial expression around the mouth of the other user based on the other action information received by the receiving unit;
a display control unit that displays the facial expression image created by the image creation unit in accordance with the position of the face of the other user;
Further provided with
The eyeglass-type device according to claim 12. the front portion has a display portion for displaying information,
a mis-attachment detection unit that detects mis-attachment of the second contact portion;
a display control unit that, when the mis-attachment detection unit detects that the second contact portion is mis-attached, displays, on the display unit, notification information that notifies the mis-attachment;
Further provided with
The eyeglass-type device according to claim 1 . The first biological signal includes at least one of electromyography, electrooculography, electroencephalography, changes in muscle shape, changes in muscle hardness, sounds generated by a living body, skin temperature, skin conductivity, respiration, sweating, heart rate, and blood pressure of the user.
The eyeglass-type device according to claim 1 . In an eyeglass-type device comprising a front portion, temple portions connected to the front portion, end pieces connected to the temple portions, a first contact portion that contacts the bridge of the user's nose or between the eyebrows to support the front portion, a second contact portion having a connection portion connected to the end pieces and that contacts under the user's chin to support the end pieces, a first sensor that is disposed in the first contact portion and acquires a first biosignal of the user from the bridge of the user's nose or between the eyebrows, and a second sensor that is disposed in the second contact portion and acquires a second biosignal of the user from under the user's chin, an information processing method executed by an information processing device mounted on the eyeglass-type device,
acquiring the first biological signal from the first sensor;
acquiring the second biological signal from the second sensor;
estimating a user's intention to operate an object to be operated based on the acquired first biological signal and the acquired second biological signal;
performing an operation corresponding to the estimated operation intention on the object;
Information processing methods. a first contact portion that contacts the bridge of the user's nose or between the eyebrows to support the front portion; a second contact portion that has a connection portion connected to the eyebrows and contacts the area under the user's chin to support the eyebrows; a first sensor that is disposed in the first contact portion and acquires a first biosignal of the user from the bridge of the user's nose or between the eyebrows; and a second sensor that is disposed in the second contact portion and acquires a second biosignal of the user from the area under the user's chin, the program causing an information processing device mounted on the eyebrow device to execute processing,
The process comprises:
acquiring the first biological signal from the first sensor;
acquiring the second biological signal from the second sensor;
estimating a user's intention to operate an object to be operated based on the acquired first biological signal and the acquired second biological signal;
performing an operation corresponding to the estimated operation intention on the object;
program.