TWI746169B - Augmented reality eyeglasses having structured light detecting function - Google Patents

Abstract

Augmented reality eyeglasses having a structured light detecting function which include a laser projector, an eyeglass lens, at least one first diffractive optical element (DOE) film, an invisible light camera, and a second DOE film are provided. The laser projector is configured to emit at least one invisible beam and an image beam. The at least one first DOE film is disposed on the eyeglass lens. The first DOE film is configured to diffract the invisible beam into a structured beam. The structured beam is transmitted to an object to be detected, so as to form a light pattern on the object to be detected. The invisible light camera is configured to photograph the light pattern on the object to be detected. The second DOE film is disposed on the eyeglass lens, and is configured to make the image beam travel to an eye.

Description

Augmented reality glasses with structured light detection function

The present invention relates to an augmented reality display, and particularly relates to an augmented reality glasses with structured light detection function.

With the advancement of display technology, virtual reality (virtual reality) display technology and augmented reality (augmented reality) display technology have gradually become popular, and they have been fully researched and developed. The virtual reality display technology allows users to be immersed in the virtual world displayed on the display, and can display three-dimensional images. The augmented reality display technology allows users to see not only the images of the virtual world, but also the objects in the real world, and even enables the images of the virtual world to interact with the objects in the real world.

When the display provides an image of the virtual world (ie, virtual image) to the user's eyes, if the system can know the position of the eye and the angle of rotation, it can provide the corresponding virtual image and have a better display effect. However, when an eye tracker is to be added to an augmented reality display device, there are disadvantages that the number of components is too large and the system is too complicated.

The present invention provides an augmented reality glasses with structured light detection function, which integrates a light source of structured light into a laser projector for displaying images, so that it can have a simpler structure and a smaller number of components.

An embodiment of the present invention provides an augmented reality glasses with structured light detection function, suitable for wearing in front of an eye. The augmented reality glasses with structured light detection function include a laser projector, a spectacle lens, at least one first diffractive optical element (DOE) film, an invisible light camera and a second diffraction Optical element film. The laser projector is used for emitting at least one invisible beam and an image beam, and the glasses lens is arranged on the path of the invisible beam and the image beam. The at least one first diffractive optical element film is disposed on the glasses lens and is located on the path of the invisible light beam. The first diffractive optical element is used for diffracting the invisible light beam into a structured light beam, wherein the structured light beam is transmitted to an object under test to form a light pattern on the object under test. The invisible light camera is used to photograph the light pattern on the object to be measured. The second diffractive optical element film is disposed on the glasses lens and located on the path of the image light beam, and the second diffractive optical element film is used for transmitting the image light beam to the eyes.

In the augmented reality glasses with structured light detection function of the embodiment of the present invention, in addition to the image beam, the laser projector also emits an invisible beam, and the invisible beam passes through the first diffractive optical element. The diffraction effect forms a structured light beam, which is used to detect the object under test. That is to say, the embodiment of the present invention integrates the structured light source into the laser projector for displaying images, so the AR glasses with structured light detection function can have a simpler structure and a smaller number. Component, and achieve the function of displaying images and detecting the object under test at the same time.

FIG. 1 is a schematic diagram of the optical path of AR glasses with structured light detection function according to an embodiment of the present invention, FIG. 2 is a schematic diagram of the optical path of the laser projector in FIG. 1, and FIG. 3 is the structured light beam of FIG. 1 Schematic diagram of the glazing pattern formed on the eyes. 1 to 3, the augmented reality glasses 100 with structured light detection function of this embodiment is suitable for being worn in front of an eye 50, and the augmented reality glasses 100 with structured light detection function includes a The laser projector 200, a spectacle lens 110, at least one first diffractive optical element film 120 (in FIG. 1 is an example of a first diffractive optical element film 120), an invisible light camera 130, and a second Diffraction optical element film 140. The laser projector 200 is used to emit at least one invisible light beam 202 (in FIG. 1 is an invisible light beam 202 as an example) and an image light beam 204. The glasses lens 110 is arranged on the invisible light beam 202 and the image light beam 204. On the path. The first diffractive optical element film 120 is disposed on the spectacle lens 110 and is located on the path of the invisible light beam 202. The first diffractive optical element 120 is used to diffract the invisible light beam 202 into a structured light beam 203. The diffraction is, for example, a reflection type diffraction. The structured light beam 203 is transmitted to an object under test to form a structure on the object Light pattern. In this embodiment, the object to be measured is the eye 50, and the structured light beam 203 forms a light pattern 201 on the eye 50.

The invisible light camera 130 is used to photograph the light pattern 201 on the object to be measured. The second diffractive optical element film 140 is disposed on the spectacle lens 110 and is located on the path of the image light beam 204. The second diffractive optical element film 140 is used to transmit the image light beam 204 to the eye 50, such as diffracted to the eye 50 , So that the eye 50 can see the image frame to be displayed by the laser projector 200, which is presented as a virtual image located in front of the eye 50. The diffraction of the image light beam 204 by the second diffractive optical element film 140 is, for example, reflection type diffraction. In addition, in this embodiment, the first diffractive optical element film 120 and the second diffractive optical element film 140 are disposed on the surface 112 of the spectacle lens 110 facing the eye 50. In addition, the second diffractive optical element film 140 may be a general diffractive optical element film or a holographic optical element (HOE) film.

In this embodiment, the laser projector includes an infrared laser light source 210, a plurality of laser light sources 220 of different colors, a light combining module 230, and a scanning mirror 240. The infrared laser light source 210 is used to emit an infrared light beam 202', and the laser light sources 220 of different colors are used to emit multiple light beams of different colors. In this embodiment, the laser light sources 220 of different colors include a red laser light source 222, a green laser light source 224, and a blue laser light source 226, respectively emitting a red light beam 221, a green light beam 223, and a blue light beam 225. In this embodiment, the infrared laser light source 210 and the laser light sources 220 of different colors are both laser diodes, and the light beams emitted by them are laser light beams.

The light combining module 230 is arranged on the path of the infrared light beam 202' and these different color light beams (for example, the red light beam 221, the green light beam 223, and the blue light beam 225) to combine the path of the infrared light beam 202' and these different color light beams . The scanning mirror 240 is arranged on the path of the infrared light beam 202' from the light combining module 230 and the light beams of different colors, wherein the scanning mirror 240 is adapted to rotate so that the infrared light beam 202' is formed to irradiate the first diffractive optical element film The invisible light beam 202 on the 120, and the light beams of different colors form the image light beam 204 scanned on the second diffractive optical element film 140. In addition, in this embodiment, the invisible light camera 130 is, for example, an infrared light camera.

FIG. 4 shows the scanning paths and positions of the red light beam, the green light beam, the blue light beam, and the infrared light beam of FIG. 2 on the glasses lens. Please refer to Figure 1, Figure 2 and Figure 4, by the rotation of the scanning mirror 240, when the scanning mirror 240 is rotated to an appropriate angle, the infrared laser light source 210 can emit an infrared beam 202' at this time, but these different colors The laser light source 220 does not emit the red light beam 221, the green light beam 223, and the blue light beam 225. At this time, the infrared light beam 202' is the invisible light beam 202 irradiated on the first diffractive optical element film 120, and is diffracted in the first A light spot I1 is formed on the optical element film 120, and then the first diffractive optical element film 120 diffracts the infrared light beam 202' into a structured light beam 203. In addition, during most of the other time, the scanning mirror 240 continuously rotates at other angles. At this time, the laser light sources 220 of different colors can emit a red light beam 221, a green light beam 223, and a blue light beam 225, while the infrared light The light source 210 does not emit the infrared light beam 202', the red light beam 221, the green light beam 223, and the blue light beam 225 can respectively form a red light scanning path I2, a green light scanning path I3 and a blue light scanning on the second diffractive optical element film 140 Path I4. In addition, with the continuous rotation of the scanning mirror 240, the respective intensities of the red light beam 221, the green light beam 223, and the blue light beam 225 can be continuously changed, so that the color and brightness of these scanning paths can be changed, and the second diffraction The optical element film 140 then diffracts the red light beam 221, the green light beam 223 and the blue light beam 225 to the eye 50, so that the eye 50 can see the color image.

In addition, the eye 50 can not only see the color image frame, but also can see the outside scene through the glasses lens 110, so as to achieve the effect of augmented reality. The spectacle lens 110 is, for example, a myopia spectacle lens, a hyperopia spectacle lens, a presbyopia spectacle lens, or a plano spectacle lens.

In this embodiment, the light combining module 230 may include multiple dichroic mirrors or multiple dichroic prisms. For example, the light combining module 230 includes a dichroic mirror 232, a dichroic mirror 234, and a dichroic mirror 236, wherein the dichroic mirror 232 is adapted to allow the infrared light beam 202' to penetrate and be transmitted to the dichroic mirror 234, and the color separation The mirror 232 is adapted to reflect the blue light beam 225 to the dichroic mirror 234. The dichroic mirror 234 is adapted to allow the infrared light beam 202' and the blue light beam 225 to pass through to the dichroic mirror 236, and the dichroic mirror 234 is adapted to reflect the green light beam 223 to the dichroic mirror 236. The dichroic mirror 236 is suitable for allowing the infrared light beam 202', the blue light beam 225 and the green light beam 223 to pass through to the scanning mirror 240, and the dichroic mirror 236 is suitable for reflecting the red light beam 221 to the scanning mirror 240. In this way, the light combining module 230 can combine the paths of the red light beam 221, the green light beam 223, the blue light beam 225, and the infrared light beam 202'.

In this embodiment, the augmented reality glasses 100 with structured light detection function further includes a glasses frame 150, and the laser projector 200, the glasses lens 110 and the invisible light camera 130 are arranged on the glasses frame 150, wherein the laser The projector 200 may be arranged on the temples of the glasses frame 150, and the invisible light camera 130 may be arranged near the center of the glasses frame 150 near the nose pads. In addition, in the present embodiment, the augmented reality glasses 100 with structured light detection function further includes a processor 160, which is electrically connected to the invisible light camera 130, and is used to respond to the light captured by the invisible light camera 130. The pattern 201 (shown in FIG. 3) calculates the position of the object to be measured (that is, the eye 50 in this embodiment), for example, calculates the position of the eye 50 and its gaze direction. Since the light pattern 201 is deformed or shifted along with the concave-convex curved surface of the eye 50, the processor 160 can calculate the position of each position on the eye in the three-dimensional space based on these deformations or shifts. The processor 160 may also be configured on the glasses frame 150, for example, on the temples of the glasses frame 150.

In an embodiment, the processor 160 is, for example, a central processing unit (CPU), a microprocessor (microprocessor), a digital signal processor (DSP), a programmable controller, and a programmable controller. A logic device (programmable logic device, PLD) or other similar devices or a combination of these devices is not limited by the present invention. In addition, in one embodiment, the functions of the processor 160 may be implemented as multiple program codes. These codes are stored in a memory, and the processor 160 executes these codes. Alternatively, in an embodiment, each function of the processor 160 may be implemented as one or more circuits. The present invention does not limit the use of software or hardware to implement the functions of the processor 160.

In the augmented reality glasses 100 with structured light detection function of this embodiment, the laser projector 200 not only emits the image light beam 204, but also emits an invisible light beam 202, and the invisible light beam 202 passes through the first diffraction The diffraction effect of the optical element 120 forms a structured light beam 203 which is used to detect the object under test. That is to say, this embodiment integrates the light source of the structured light 203 into the laser projector 200 for displaying images, that is, integrates the light source of the eye tracker into the laser projector 200, thus having The AR glasses 100 with structured light detection function can have a simpler structure and a smaller number of components, and achieve the functions of displaying images and detecting the object under test at the same time.

5 is a three-dimensional schematic diagram of an embodiment of the first diffractive optical element film in FIG. 1, and FIG. 6 is a three-dimensional schematic diagram of another embodiment of the first diffractive optical element film in FIG. 1. 1 and 5, the first diffractive optical element film 120 may have a plurality of microstructures 122, such as strip-shaped protrusions in FIG. The microstructures 122 can be arranged along the horizontal direction of FIG. 1, and the light pattern generated in this way is, for example, a stripe-shaped light pattern. Please refer to FIGS. 1 and 6 again. In another embodiment, the first diffractive optical element film 120a of FIG. 6 may be used to replace the first diffractive optical element film 120 of FIG. 5. The first diffractive optical element film 120a of FIG. 6 has a plurality of microstructures 122a arranged in two dimensions. These microstructures 122a are, for example, dot-shaped protrusions, and the light pattern generated in this way is, for example, like the dots arranged in an array in FIG.状光图201。 Like light pattern 201. The first diffractive optical element film 120 of FIG. 5 and the first diffractive optical element film 120a of FIG. 6 are, for example, diffraction gratings.

FIG. 7 is a schematic diagram of the optical path of the augmented reality glasses with structured light detection function according to another embodiment of the present invention. FIG. 8 is a schematic front view of the glasses lens in FIG. It is a schematic diagram of the optical path of the laser projector in FIG. 7. Please refer to FIGS. 7-9, the AR glasses 100b with structured light detection function of this embodiment is similar to the AR glasses 100 with structured light detection function of FIG. 1, and the main difference between the two As described below. The AR glasses 100b with structured light detection function of this embodiment includes two first diffractive optical element films 120b respectively disposed on both sides of the second diffractive optical element film 140, for example, located in the second diffractive optical element film. The first diffractive optical element film 124 b on the right side of the optical element film 140 and the first diffractive optical element film 126 b on the left side of the second diffractive optical element film 140. In addition, when the scanning mirror 240 of the laser projector 200 rotates to two different angles at different times, the infrared laser light source 210 emits an infrared beam 202', and the infrared light beam 202' is emitted at two different angles at two different times. The scanning mirror 240 respectively reflects the infrared light beam 202 ′ to different directions to form two invisible light beams 202 b transmitted in different directions, for example, the invisible light beam 2021 and the invisible light beam 2022. Wherein, the invisible light beam 2021 is irradiated on the first diffractive optical element film 124b to form a structured light beam 203, and the invisible light beam 2022 is irradiated on the first diffractive optical element film 126b to form another structured light beam 203, and Both structured light beams 203 are delivered to the eye 50 to form two light patterns on the eye. The two light patterns can cover more angles of the eye 50, so that the processor 160 can be more accurate in calculating the position of the eye 50 and its gaze direction.

FIG. 10 is a schematic front view of the spectacle lens in the augmented reality spectacles with structured light detection function according to another embodiment when viewed from the direction of the line of sight of the eye. Please refer to FIG. 7, FIG. 8, and FIG. 10. The AR glasses with structured light detection function of the embodiment of FIG. 10 are similar to the AR glasses with structured light detection function 100b of FIG. The difference is that the first diffractive optical element film 124b and the first diffractive optical element film 126b in the embodiment of FIG. 10 are respectively arranged on the upper and lower sides of the second diffractive optical element film 140, and the invisible light beam The 2021 and invisible light beams 2022 are respectively irradiated on the first diffractive optical element film 124b and the first diffractive optical element film 126b.

FIG. 11 is a schematic front view of the spectacle lens in the augmented reality glasses with structured light detection function as viewed from the line of sight of the eye according to another embodiment. Please refer to FIG. 7, FIG. 8, and FIG. 11. The AR glasses with structured light detection function of the embodiment of FIG. 11 are similar to the AR glasses with structured light detection function 100b of FIG. The difference is that the AR glasses with structured light detection function in the embodiment of FIG. 11 have four first diffractive optical element films 120c respectively disposed around the second diffractive optical element film 140, For example, the first diffractive optical element films 124c, 126c, 128c, and 129c are respectively arranged on the right, left, upper, and lower sides of the second diffractive optical element film 140, and the scanning mirrors of the laser projector are in four different The time is rotated to four different angles to reflect the infrared beams to four different directions to form four invisible beams that irradiate the first diffractive optical element films 124c, 126c, 128c, and 129c, respectively. The first diffractive optical element films 124c, 126c, 128c, and 129c diffract the four invisible light beams into four structured light beams, and the four structured light beams are all transmitted to the eye to form four light patterns on the eye. The four light patterns can cover more angles of the eye 50, so that the processor 160 is more accurate in calculating the position of the eye 50 and its gaze direction.

12 is a schematic diagram of the optical path of the augmented reality glasses with structured light detection function according to still another embodiment of the present invention. Please refer to FIG. 12, the AR glasses 100d with structured light detection function of this embodiment are similar to the AR glasses 100 with structured light detection function of FIG. 1, and the difference between the two is as follows. In the augmented reality glasses 100d with structured light detection function of this embodiment, the object to be measured is a foreign object 60, and the spectacle lens 110 is located between the foreign object 60 and the eye 50. In addition, the first diffractive optical element film 120d diffracts the invisible light beam 202 toward the outside into a structured light beam 203, and this diffraction is, for example, a transmission type diffraction. The structured light beam 203 is transmitted to the external object 60 to form a light pattern on the external object 60. By capturing the light pattern by the invisible light camera 130, the processor 160 can calculate the position of the external object 60. In this embodiment, there may be a plurality of invisible light cameras 130, such as two, which are respectively arranged at the center and one side of the glasses frame 150. However, the present invention does not limit the number of invisible light cameras 130. In another embodiment, the number of the invisible light camera 130 may also be one.

In summary, in the AR glasses with structured light detection function of the embodiment of the present invention, the laser projector not only emits the image beam, but also emits an invisible beam, and the invisible beam passes through the first The diffraction effect of the diffractive optical element forms a structured light beam, which is used to detect the object under test. That is to say, the embodiment of the present invention integrates the structured light source into the laser projector for displaying images, so the AR glasses with structured light detection function can have a simpler structure and a smaller number. Component, and achieve the function of displaying images and detecting the object under test at the same time.

50: eyes 60: Foreign objects 100, 100b, 100d: AR glasses with structured light detection function 110: glasses lens 112: Surface 120, 120a, 120b, 120c, 120d, 124b, 124c, 126b, 126c, 128c, 129c: first diffractive optical element film 122, 122a: Microstructure 130: Invisible light camera 140: The second diffraction optical element film 150: glasses frame 160: processor 200: Laser projector 201: Light Pattern 202, 202b, 2021, 2022: invisible beam 202’: Infrared beam 203: structured beam 204: image beam 210: Infrared laser light source 220: Laser light source 221: Red beam 222: Red laser light source 223: Green beam 224: Green laser light source 225: Blue beam 226: Blue laser light source 230: Heguang Module 232, 234, 236: dichroic mirror 240: Scanning mirror I1: light spot I2: Red light scan path I3: Green light scan path I4: Blu-ray scan path

FIG. 1 is a schematic diagram of the optical path of an augmented reality glasses with structured light detection function according to an embodiment of the present invention. Fig. 2 is a schematic diagram of the optical path of the laser projector in Fig. 1. Fig. 3 is a schematic diagram of the structured beam of Fig. 1 forming a light pattern on the eye. FIG. 4 shows the scanning paths and positions of the red light beam, the green light beam, the blue light beam, and the infrared light beam of FIG. 2 on the glasses lens. FIG. 5 is a three-dimensional schematic diagram of an embodiment of the first diffractive optical element film in FIG. 1. FIG. 6 is a three-dimensional schematic diagram of another embodiment of the first diffractive optical element in FIG. 1. FIG. 7 is a schematic diagram of the optical path of the augmented reality glasses with structured light detection function according to another embodiment of the present invention. Fig. 8 is a schematic front view of the spectacle lens in Fig. 7 as viewed from the direction of the line of sight of the eye. FIG. 9 is a schematic diagram of the optical path of the laser projector in FIG. 7. FIG. 10 is a schematic front view of the spectacle lens in the augmented reality spectacles with structured light detection function according to another embodiment when viewed from the direction of the line of sight of the eye. FIG. 11 is a schematic front view of the spectacle lens in the augmented reality glasses with structured light detection function as viewed from the line of sight of the eye according to another embodiment. 12 is a schematic diagram of the optical path of the augmented reality glasses with structured light detection function according to still another embodiment of the present invention.

50: eyes

100: AR glasses with structured light detection function

110: glasses lens

112: Surface

120: The first diffractive optical element film

130: Invisible light camera

140: The second diffraction optical element film

150: glasses frame

160: processor

200: Laser projector

202: Invisible beam

203: structured beam

204: image beam

Claims (9)

An augmented reality glasses with structured light detection function, suitable for wearing in front of an eye, the augmented reality glasses with structured light detection function includes: a laser projector for emitting at least one Seeing light beam and an image light beam; a spectacle lens arranged on the path of the invisible light beam and the image light beam; at least one first diffractive optical element film arranged on the spectacle lens and located in the path of the invisible light beam Above, the first diffractive optical element is used to diffract the invisible light beam into a structured light beam, wherein the structured light beam is transmitted to an object to be measured to form a light pattern on the object to be measured; an invisible light camera, Used to photograph the light pattern on the object to be measured; and a second diffractive optical element film disposed on the glasses lens and located on the path of the image beam, the second diffractive optical element film The image light beam is transmitted to the eye, wherein the at least one first diffractive optical element film includes two first diffractive optical element films respectively disposed on both sides of the second diffractive optical element film, and the at least one non-conforming optical element film The visible light beam includes two invisible light beams respectively irradiating the two first diffractive optical element films. The augmented reality glasses with structured light detection function according to claim 1, wherein the object to be tested is the eye. The augmented reality glasses with structured light detection function according to claim 1, wherein the object to be tested is an external object, and the glasses lens is located between the external object and the eye. The augmented reality glasses with structured light detection function according to claim 1, wherein the first diffractive optical element film and the second diffractive optical element film are arranged on the surface of the spectacle lens facing the eye . The augmented reality glasses with structured light detection function according to claim 1, wherein the laser projector includes: an infrared laser light source for emitting an infrared light beam; a plurality of laser light sources of different colors , Used to emit multiple light beams of different colors; a light combining module arranged on the path of the infrared light beam and the light beams of different colors to combine the path of the infrared light beam and the light beams of different colors; and a The scanning mirror is arranged on the path of the infrared beam from the light combining module and the beams of different colors, wherein the scanning mirror is adapted to rotate so that the infrared beam is formed to irradiate the at least one first diffractive optics The at least one invisible light beam on the element film and the light beams of different colors form the image light beam scanned on the second diffractive optical element film. The augmented reality glasses with structured light detection function according to claim 1, wherein the at least one first diffractive optical element film is four first diffractive optical element films respectively arranged around the second diffractive optical element film A diffractive optical element film, and the at least one invisible light beam is four invisible light beams respectively irradiating the four first diffractive optical element films. The augmented reality glasses with structured light detection function according to claim 1, wherein the second diffractive optical element film is a holographic optical element film. The augmented reality glasses with structured light detection function according to claim 1, further comprising a processor, which is electrically connected to the invisible light camera, and used for according to the light pattern captured by the invisible light camera Calculate the position of the object to be measured. The augmented reality glasses with structured light detection function according to claim 1, further comprising a spectacle frame, wherein the laser projector and the spectacle lens are arranged on the spectacle frame.

Images