JP2006195665A - Information processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a portable information processing apparatus that is easy to carry and has excellent operability.
A mobile computer 1 includes a virtual keyboard 10, a computer 20, and a wearable display 30. The virtual keyboard 10 projects an image representing a keyboard including input keys for inputting information onto a desk or the like, and detects the position of a human fingertip, thereby extracting the input key corresponding to the fingertip position. To do. The computer 20 executes a predetermined process based on the information indicated by the input key transmitted from the virtual keyboard 10, and transmits screen information indicating a screen including information based on the executed process to the wearable display 30. The wearable display 30 irradiates light onto the image display unit 31 that displays a screen based on the screen information transmitted from the computer 20, irradiates reflected light onto the retina of the human eyeball, and projects the screen onto the retina.
[Selection] Figure 1

Description

  The present invention relates to an information processing apparatus, and more particularly to a portable information processing apparatus that is easy to carry.

  2. Description of the Related Art Portable computers such as notebook personal computers (hereinafter referred to as portable personal computers) are provided with built-in large-capacity storage devices and high processing speeds. However, from the viewpoint of easy viewing of the screen and operability, there is a limit to reducing the display screen and the keyboard. Therefore, the size of the portable personal computer needs to be secured to some extent, and it may require labor to carry or the work place may be limited.

  A mobile PDA (Personal Digital Assistant) has a smaller size and improved portability, but has a drawback that a display screen and a keyboard are small and workability is poor. Further, when a portable personal computer and a mobile PDA are used in a vehicle seat, a person in the next seat may see the screen and lack confidentiality.

  A portable display device has a wearable display such as a head-mounted display that is worn on the head, that is, a display device that is worn on the human body. Is also expensive.

  As a first conventional technology of a head mounted display, there is a head-mounted display device that enables switching between a computer display image and a keyboard. This head-mounted display device detects the line of sight and controls the ratio of the video display part and the external light transmission part on the display screen to make it easier to see the keyboard displayed on the external light transmission part (For example, see Patent Document 1).

  As a second conventional technique for a head-mounted display, there is an input device for physically handicapped persons that can input information without using limbs. This input device detects the line of sight on the display screen of the head-mounted display, thereby identifying the information displayed on the display screen and enabling the input of information (see, for example, Patent Document 2). .

  As a third conventional technology for a head mounted display, there is a head mounted display for displaying a video signal transmitted through a human body as an image. This head-mounted display displays a video signal as an image when a person touches an electrode that outputs a video signal from a video playback device that explains an exhibit such as a museum (see, for example, Patent Document 3). ).

JP-A-8-328512 JP-A-11-73273 JP 2000-196975 A

  However, the first prior art inputs information using a keyboard, and there is a problem that it is not easy to carry the keyboard.

  The second conventional technique is to input information by detecting a gazing point without using a keyboard, and the input device can be miniaturized, but the speed of inputting information is higher when using a keyboard. There is a problem that it is slower than that.

  The third prior art detects a video signal and starts displaying the video signal, and has a problem that processing based on input information cannot be performed.

  An object of the present invention is to provide a portable information processing apparatus that is easy to carry and has excellent operability.

The present invention includes a first projection unit that projects an image representing a keyboard including an input key for inputting information onto a predetermined projection plane;
Input key detection means for detecting an input key selected from among the input keys represented by the image projected by the first projection means;
Processing means for executing predetermined processing based on information indicated by the input key detected by the input key detection means;
Display means for displaying a screen including information based on processing executed by the processing means;
An information processing apparatus comprising: a second projection unit that projects a screen displayed on the display unit onto a retina of a human eyeball.

  According to the present invention, an image representing a keyboard including an input key for inputting information is projected onto a predetermined projection plane by the first projection means, and the first projection means is projected by the input key detection means. The input key selected from the input keys represented by the image projected by is detected, and the processing means performs a predetermined process based on the information indicated by the input key detected by the input key detection means. When executed, the display means displays a screen including information based on the processing executed by the processing means, and the second projection means projects the screen displayed on the display means onto the retina of the human eyeball. .

  In this way, the keyboard can be provided as an image and the screen can be projected directly onto the retina.

  According to the present invention, the second projection means scans the screen displayed on the display means by changing the direction of the light beam output from the light output means that outputs the light beam and the light output means. And a refracting means for refracting a beam of light scanned by the scanning means in a direction to irradiate the retina.

  According to the present invention, the second projecting means outputs the light beam by the light output means, and changes the direction of the light beam output from the light output means by the scanning means, thereby causing the display means to The displayed image is scanned, and the light beam scanned by the scanning unit is refracted by the refraction unit in a direction in which the beam is irradiated on the retina. Therefore, the screen displayed on the display means can be projected on the retina.

  According to the present invention, the second projection means outputs a light output means for irradiating the display means, and collects the light emitted to the display means at the position of the eyeball lens to the retina. Condensing means for projecting an image is included.

  According to the present invention, the second projection means outputs light for irradiating the display means by the light output means, and the light emitted to the display means by the light collecting means at the position of the eyeball lens. The light is condensed and an image is projected onto the retina. Therefore, the screen displayed on the display means can be projected on the retina.

  In the invention, it is preferable that the second projecting unit projects an image onto one of the retina of the right eyeball and the retina of the left eyeball.

  According to the present invention, since the image is projected onto one of the retina of the right eyeball and the retina of the left eyeball by the second projecting means, the screen is projected only on the retina of one eyeball. Can do.

  In the invention, it is preferable that the second projecting unit projects an image on the retina of the right eyeball and the retina of the left eyeball.

  According to the present invention, since the image is projected onto the retina of the right eyeball and the retina of the left eyeball by the second projection means, the screen can be projected onto the retina of both eyeballs.

  Further, the present invention is characterized in that the display means transmits light incident from a surface opposite to the surface facing the eyeball among the surfaces constituting the display means.

  According to the present invention, since the light incident from the surface opposite to the surface facing the eyeball is transmitted by the display means among the surfaces constituting the display means, the image of the scene entering the field of view is displayed on the retina. Can be formed.

  Further, the present invention is characterized in that the display means does not transmit light incident from a surface opposite to the surface facing the eyeball among the surfaces constituting the display means.

  According to the present invention, since the light incident from the surface opposite to the surface facing the eyeball is not transmitted by the display means among the surfaces constituting the display means, the image of the scene to enter the field of view is on the retina. Not formed.

  In the invention, it is preferable that the light output means includes red, blue, and green light emitting diodes.

  According to the present invention, since the light output means includes red, blue, and green light emitting diodes, a color image can be projected on the retina.

  In the invention, it is preferable that the light output means includes a monochromatic light emitting diode or a monochromatic laser diode.

  According to the present invention, the light output means includes a monochromatic light-emitting diode or monochromatic laser diode, so that a monochromatic image can be projected onto the retina.

In the invention, it is preferable that the single color is orange.
According to the present invention, since the single color is orange, an orange image can be projected on the retina.

  According to the present invention, the input key detection means is a second light output means for outputting light, and a light detection for detecting the reflected light reflected from the human fingertip by the light output from the second light output means. And extraction means for extracting an input key corresponding to the position of the fingertip based on the reflected light detected by the light detection means.

  According to the present invention, the input key detection means outputs light from the second light output means, and the light output from the second light output means is reflected by the human fingertip by the light detection means. The reflected light is detected, and the extraction unit extracts the input key corresponding to the fingertip based on the reflected light detected by the light detection unit, so that the input key selected by the fingertip can be detected.

The present invention also provides a line-of-sight detection means for detecting a line-of-sight direction
And a deterring unit that deters projection of the image by the second projecting unit when the gaze direction detected by the gaze detecting unit is the direction of the keyboard of the image projected by the first projecting unit. It is characterized by.

  According to the present invention, when the gaze direction is detected by the gaze detection means, and the gaze direction detected by the gaze detection means by the suppression means is the direction of the keyboard of the image projected by the first projection means. Since the projection of the image by the second projection means is suppressed, the display of the screen can be suppressed when inputting information by looking at the keyboard image.

Further, the present invention projects an image representing a keyboard including an input key for inputting information onto a predetermined projection plane, and the input key selected from the input keys represented by the projected image And an input device for inputting information indicated by the detected input key;
A processing device that executes a predetermined process based on information input by the input device;
An information processing apparatus comprising: an output device that projects a screen including information based on processing executed by the processing device onto a retina of a human eyeball.

  According to the present invention, an image representing a keyboard including an input key for inputting information is projected on a predetermined projection plane by the input device, and the input key is represented by the projected image. The selected input key is detected, information indicated by the detected input key is input, the processing device performs a predetermined process based on the information input by the input device, and the output device A screen including information based on processing executed by the processing device is projected onto the retina of the human eyeball.

  In this way, the keyboard can be provided as an image and the screen can be projected directly onto the retina.

  Further, the present invention is characterized in that the communication between the input device and the processing device and the communication between the processing device and the output device are wireless wireless communication.

  According to the present invention, the communication between the input device and the processing device and the communication between the processing device and the output device are wireless communication by wireless, so there is no need to connect the devices with a cable.

  Further, the invention is characterized in that the wireless communication is wireless communication using light or radio waves.

  According to the present invention, since the wireless communication is wireless communication using light or radio waves, a commercially available communication device can be used.

  Further, the present invention is characterized in that the communication between the input device and the processing device and the communication between the processing device and the output device are wired communication using optical fibers.

  According to the present invention, the communication between the input device and the processing device and the communication between the processing device and the output device are wired communication using optical fiber, so that a large amount of information can be transmitted and received. it can.

According to the present invention, the input device includes a light source for projecting an image representing the keyboard, and a part of light output from the light source is transmitted to the output device via the optical fiber,
The output device uses light transmitted through the optical fiber as a light source for projecting the screen.

  According to the present invention, a part of light output from the light source representing the keyboard by the input device is transmitted to the output device via an optical fiber, and the transmitted light is displayed on the screen by the output device. Since it is used as a light source for projection, the light source can be shared.

In the invention, the output device includes a light source for projecting the screen, and a part of the light output from the light source is transmitted to the input device via the optical fiber,
The input device uses light transmitted through the optical fiber as a light source for projecting an image representing the keyboard.

  According to the present invention, a part of the light output from the light source for projecting the screen by the output device is transmitted to the input device via an optical fiber, and the transmitted light is transmitted by the input device. Since it is used as a light source for projecting an image representing a keyboard, the light source can be shared.

Further, the present invention projects an image representing a keyboard including an input key for inputting information onto a predetermined projection plane, and the input key selected from the input keys represented by the projected image An input processing device that inputs information indicated by the detected input key and executes a predetermined process based on the input information;
An information processing apparatus comprising: an output device that projects a screen including information based on processing executed by the input processing device onto a retina of a human eyeball.

  According to the present invention, an image representing a keyboard including an input key for inputting information is projected on a predetermined projection plane by the input processing device, and the input key is represented by the projected image. The input key selected from the input key is detected, information indicated by the detected input key is input, a predetermined process is executed based on the input information, and is executed by the input processing device by the output device. A screen including information based on the processing is projected onto the retina of the human eyeball.

  In this way, the keyboard can be provided as an image and the screen can be projected directly onto the retina.

Further, the present invention projects an image representing a keyboard including an input key for inputting information onto a predetermined projection plane, and the input key selected from the input keys represented by the projected image And an input / output device for inputting information indicated by the detected input key;
A processing device that executes a predetermined process based on information input by the input / output device,
The input / output device is an information processing device that further projects a screen including information based on processing executed by the processing device onto a retina of a human eyeball.

  According to the present invention, an image representing a keyboard including an input key for inputting information is projected on a predetermined projection plane by the input / output device, and the input key is represented by the projected image. The input key selected from the input key is detected, information indicated by the detected input key is input, and a predetermined process is executed by the processing device based on the information input by the input / output device. The device further projects a screen including information based on the processing executed by the processing device onto the retina of the human eyeball.

  In this way, the keyboard can be provided as an image and the screen can be projected directly onto the retina.

Further, in the present invention, the input / output device includes a light source for projecting an image representing the keyboard,
The light source is also used as a light source for projecting the screen onto a retina.

  According to the present invention, the light source for projecting the image representing the keyboard is also used as the light source for projecting the screen, so that the light source can be shared.

The present invention also includes a display means for displaying an image representing a keyboard including input keys for inputting information,
Projection means for projecting the image displayed on the display means onto the retina of the human eyeball;
An input key detecting means for detecting an input key selected from among the input keys represented by the image projected by the projecting means;
Processing means for executing a predetermined process based on information indicated by the input key detected by the input key detection means,
The display means is an information processing apparatus characterized by combining and displaying a screen including information based on processing executed by the processing means with an image representing the keyboard.

  According to the present invention, the display means displays an image representing a keyboard including input keys for inputting information, and the projection means projects the image displayed on the display means onto the retina of a human eyeball. An input key selected from the input keys represented by the image projected by the projection unit is detected by the input key detection unit, and the input key detected by the input key detection unit is indicated by the processing unit. A predetermined process is executed based on the information, and a screen including information based on the process executed by the processing unit is combined with the image representing the keyboard and displayed by the display unit.

  In this way, an image showing a keyboard and a screen can be projected directly onto the retina.

  According to the present invention, since the keyboard is provided as an image and the screen is directly projected onto the retina, the keyboard as the object and the display as the object are not required, and the information processing apparatus can be reduced in size and weight. it can. Therefore, the information processing apparatus can be a portable information processing apparatus that is easy to carry and has excellent operability.

  Further, according to the present invention, the screen displayed on the display means can be projected onto the retina, and since the lens function is not used, the lens remains in a state of looking far, and the eyes are tired. Hateful. Furthermore, a low vision person with weak vision can also recognize an image.

  Further, according to the present invention, the screen displayed on the display means can be projected onto the retina, and since the lens function is not used, the lens remains in a state of looking far, and the eyes are tired. Hateful. Furthermore, a low vision person with weak vision can also recognize an image.

  Further, according to the present invention, since an image is projected only on the retina of one eyeball, it is possible to operate while confirming the surrounding situation with the other eye, and immediately respond when there is a change in the surrounding situation. can do.

Further, according to the present invention, since images are projected on the retinas of both eyeballs, for example, one of two types of images using parallax between the left eye and the right eye is placed on the retina of the left eyeball. 3D that can be viewed stereoscopically by projecting and projecting the other image onto the retina of the right eyeball
(3-Dimension) images can be provided.

  Further, according to the present invention, an image of a scene entering the field of view is formed on the retina, so that the image of the scene entering the field of view and the screen can be recognized.

  Further, according to the present invention, since an image of a scene that should enter the field of view is not formed on the retina, only the screen can be recognized, and a clear image that is not affected by external light can be seen.

  Further, according to the present invention, since a color image can be projected on the retina, a color screen similar to a display screen of a notebook personal computer or the like can be provided.

  Further, according to the present invention, since a monochromatic image can be projected on the retina, it is possible to select and display a color that can be easily distinguished from the color of the image entering the field of view.

  Further, according to the present invention, since an orange image is projected on the retina, for example, when an image on a monitor is projected on a human retina, it can be recognized as a monitor with a smaller amount of light than other colors.

  In addition, according to the present invention, since the input key selected by the fingertip can be detected, it is possible to input information with the same fingertip operation as when operating the real keyboard.

  Further, according to the present invention, when the information is input by looking at the image on the keyboard, the display of the screen is suppressed, so that it is possible to prevent an image unrelated to the information input from entering the field of view.

  Further, according to the present invention, since the keyboard is provided as an image and the screen is directly projected onto the retina, the keyboard as the object and the display as the object become unnecessary, and the information processing apparatus can be reduced in size and weight. Can do. Therefore, the information processing apparatus can be a portable information processing apparatus that is easy to carry and has excellent operability.

  Further, according to the present invention, since it is not necessary to connect the apparatuses with a cable, it is possible to reduce factors that restrict human movement and to reduce a sense of restraint at the time of wearing.

  In addition, according to the present invention, since communication between transmissions is wireless communication using light or radio waves, a commercially available communication device can be used, and a smaller and less expensive communication device can be selected.

  Further, according to the present invention, a large amount of information can be transmitted / received through an optical fiber, so that a high-resolution image can be displayed.

  Further, according to the present invention, since the light source can be shared by using the optical fiber, it is possible to reduce the size, the weight, and the price.

  Further, according to the present invention, since the light source can be shared by using the optical fiber, it is possible to reduce the size, the weight, and the price.

  Further, according to the present invention, since the keyboard is provided as an image and the screen is directly projected onto the retina, the keyboard as the object and the display as the object become unnecessary, and the information processing apparatus can be reduced in size and weight. Can do. Therefore, the information processing apparatus can be a portable information processing apparatus that is easy to carry and has excellent operability. Furthermore, since a transmission / reception unit for communication via a connection cable is not required, it is possible to further reduce the size and weight.

  Further, according to the present invention, since the keyboard is provided as an image and the screen is directly projected onto the retina, the keyboard as the object and the display as the object become unnecessary, and the information processing apparatus can be reduced in size and weight. Can do. Therefore, the information processing apparatus can be a portable information processing apparatus that is easy to carry and has excellent operability. Furthermore, since the input device and the output device are integrated, further downsizing, lighter weight, and lower price can be achieved.

  In addition, according to the present invention, since the light source is shared, it is possible to further reduce the size, the weight, and the price.

  In addition, according to the present invention, since an image showing a keyboard and a screen are projected directly onto the retina, a keyboard as an object and a display as an object become unnecessary, and the information processing apparatus can be reduced in size and weight. . Therefore, the information processing apparatus can be a portable information processing apparatus that is easy to carry and has excellent operability. Furthermore, since the screen and the keyboard are projected on the retina, the amount of line-of-sight movement can be reduced.

  FIG. 1 is a diagram showing a schematic configuration of a mobile computer 1 according to the first embodiment of the present invention. The information processing apparatus, for example, the mobile computer 1 includes a virtual keyboard 10, a computer 20, a wearable display 30, a connection cable A50, and a connection cable B51. The mobile computer 1 is a portable information processing apparatus that processes information based on input information and outputs the processed information.

  The virtual keyboard 10 is an input device for inputting information, and includes a light irradiation unit 11 and a light detection unit 12. The virtual keyboard 10 first projects an image representing a keyboard including input keys for inputting information onto a predetermined projection surface, for example, on a desk or the like. Next, by detecting the position of a human fingertip that designates the input key represented by the projected image, the input key corresponding to the position of the fingertip is extracted, and input key information indicating the extracted input key is stored in the computer. 20 to send.

  The light irradiation unit 11 is realized using, for example, a light source such as a laser diode, outputs a laser beam for projecting a keyboard as an image, and projects an image of the keyboard on a desk or the like. In FIG. 1, a keyboard image projected on a desk or the like, that is, a virtual keyboard 19 is displayed. The virtual keyboard 19 includes input keys corresponding to a real keyboard, for example, and is displayed as a part of numeric input keys of numbers “1” to “6” in FIG.

  The light detection unit 12 is realized including, for example, an infrared light emitting element and an infrared sensor. The infrared sensor detects reflected light in which infrared light output from the infrared light emitting element is reflected by a human fingertip. The virtual keyboard 10 extracts the input key corresponding to the position of the fingertip based on the reflected light detected by the light detection unit 12, and transmits the input key information indicated by the extracted input key to the computer 20.

  The computer 20 is a processing device that processes information, and executes predetermined processing based on input key information transmitted from the virtual keyboard 10. Next, screen information indicating a screen including information based on the executed process is transmitted to the wearable display 30.

The computer 20 is connected to the virtual keyboard 10 by a connection cable A50, and is further connected to the wearable display 30 by a connection cable B51. These connections are wired connections for performing wired communication by wire. The connection cable A50 and the connection cable B51 are signal lines using electrical signals, such as USB (
Universal serial bus) interface cable or optical fiber using optical signal. When an optical fiber is used, a large amount of information can be transmitted and received, so that a high-resolution image can be displayed.

  The wearable display 30 is an output device that outputs information, and is, for example, a head-mounted display that is worn on the head. Wearable display 30 includes an image display unit 31 realized by an image display element such as a liquid crystal display. The wearable display 30 projects the screen displayed on the image display unit 31 onto the retina by irradiating the screen displayed on the image display unit 31 with light and irradiating the reflected light onto the retina of the human eyeball. Thus, since the screen represented by the screen information transmitted from the computer 20 is displayed on the image display unit 31, a screen including information based on the processing executed by the computer 20 is projected onto the retina.

  FIG. 2 is a block diagram of the mobile computer 1 shown in FIG. The virtual keyboard 10 includes a light irradiation unit 11, a light detection unit 12, a keyboard image generation unit 13, an input key extraction unit 14, and a first transmission unit 15. The light irradiation unit 11 and the light detection unit 12 are the light irradiation unit 11 and the light detection unit 12 illustrated in FIG. 1, respectively.

  The keyboard image generation unit 13 generates image data representing a keyboard including input keys for inputting information, and provides the generated data to the light irradiation unit 11. The light irradiation unit 11 outputs light modulated based on the provided data, and projects a keyboard image on a desk or the like.

  The light detection unit 12 detects position information indicating the direction and distance of the fingertip by infrared reflected light reflected by the human fingertip, and provides the detected position information to the input key extraction unit 14. The input key extraction unit 14 extracts the input key selected by the fingertip based on the position information provided from the light detection unit 12, and inputs the input key information indicating the extracted input key to the second transmission unit 15. provide. The second transmission unit 15 transmits the input key information provided from the input key extraction unit 14 to the computer 20.

  The computer 20 includes a processing unit 21, a first receiving unit 22, and a second transmitting unit 23. The processing unit 21 includes, for example, a processing device such as a CPU (Central Processing Unit) that processes information and a storage device such as a semiconductor memory that stores information.

  The processing unit 21 executes a predetermined process, for example, a process determined by a program stored in the storage device, based on the input key information received by the first receiving unit 22. When the processing unit 21 provides screen information indicating a screen including information based on the executed process to the second transmission unit 23, the second transmission unit 23 wears the screen information provided from the processing unit 21 into a wearable. Send to display 30.

  The wearable display 30 includes an image display unit 31, a second reception unit 32, a display control unit 33, an optical system control unit 34, and an optical unit 35. The image display unit 31 is the image display unit 31 shown in FIG. The second receiving unit 32 receives the screen information transmitted from the computer 20 and provides it to the display control unit 33.

  The display control unit 33 converts the screen information provided from the second receiving unit 32 into a video signal and outputs the video signal to the image display unit 31. The image display unit 31 is realized by an image display element such as a liquid crystal display, for example, and displays a screen based on the video signal converted by the display control unit 33.

  The optical system control unit 34 controls a movable unit such as a scan mirror in order to scan the screen displayed on the image display unit 31. The optical unit 35 includes a light source, a scan mirror, and a lens, and projects the screen displayed on the image display unit 31 onto the retina of the human eyeball. Details of the optical unit 35 will be described later.

  The first projecting unit is, for example, the light irradiating unit 11 and the keyboard image generating unit 13, the input key detecting unit is, for example, the photodetecting unit 12, the input key extracting unit 14, and the first transmitting unit is, for example, the first transmitting unit. 15. The first receiving unit is, for example, the first receiving unit 22, the processing unit is, for example, the processing unit 21, the second transmitting unit is, for example, the second transmitting unit 23, and the second receiving unit is, for example, the first receiving unit. The second receiving unit 32 and the display unit are, for example, the display control unit 33 and the image display unit 31, and the projection unit or the second projection unit is, for example, the optical system control unit 34 and the optical unit 35.

  In the above-described embodiment, the input device, for example, the virtual keyboard 10, the processing device, for example, the computer 20, and the output device, for example, the wearable display 30, are independent devices, and the devices are connected with cables or the like. Although configured, it may be configured as one device. In this case, the input key information from the input key extraction unit 14 is directly input to the processing unit 21, and the screen information from the processing unit 21 is directly input to the display control unit 33, and the first transmission unit 15, the first transmission unit 15 This can be realized by deleting the reception unit 22, the second transmission unit 23, and the second reception unit 32. The connection cable A50 and the connection cable B51 are also unnecessary.

  FIG. 3 is a diagram for explaining the light detection unit 12 illustrated in FIG. 1. The light detection unit 12 includes, for example, an infrared light emitting element and an infrared sensor. Infrared rays irradiated from the infrared light emitting element irradiate at least a region of the image of the virtual keyboard 19 projected from the light irradiation unit 11.

  The infrared sensor detects the direction and distance of the fingertip by infrared reflected light reflected by the human fingertip. Position information indicating the direction and distance of the detected fingertip is provided to the input key extraction unit 14. The input key extraction unit 14 extracts the input key selected with the fingertip based on the provided position information.

  The second light output means is, for example, an infrared light emitting element, the light detection means is, for example, an infrared sensor, and the extraction means is, for example, the input key extraction unit 14.

  In this way, since the input key selected by the fingertip can be detected, information can be input by the same fingertip operation as when operating the real keyboard.

  FIG. 4 is a diagram illustrating a first example of the configuration of the optical unit 35 illustrated in FIG. 2. The optical unit 35 shown in FIG. 4 uses a retinal projection scan method. The optical unit 35 includes an image display light source 42, a scan mirror 43, a first lens 44, and a second lens 45. In the image display element 41 realized by a liquid crystal display or the like, for example, an image display surface of the image display element 41 is parallel to both lenses between a first lens 44 and a second lens 45 arranged in parallel. It is arranged to become.

  The image display light source 42 is a light source that outputs light for projecting an image on the retina. If, for example, red, blue and green light emitting diodes are used as the image display light source 42, a color image can be projected onto the retina. Therefore, a color screen similar to a display screen of a notebook personal computer or the like can be provided.

  If, for example, a single color light emitting diode or a single color laser diode is used as the image display light source 42, a single color image can be projected onto the retina. Therefore, it is possible to select and display a color that can be easily distinguished from the color of the image entering the field of view. In the case of a single color, it is desirable to use an orange color. The orange color can be recognized as a monitor with a smaller amount of light than other colors when a screen, that is, an image of the monitor is projected onto a human retina.

  The scan mirror 43 is a mirror for reflecting the light output from the image display light source 42 in the direction of the first lens 44. The first lens 44 is a convex lens for refracting light passing through the first lens 44 so as to travel in the direction of the image display element 41. The scan mirror 43 changes the direction of the reflected light so as to scan the screen displayed on the image display element 41 by changing the angle with respect to the image display light source 42.

  The light that has scanned the image display element 41 passes through the image display element 41. The light transmitted through the image display element 41 is modulated by the screen displayed on the image display element 41 and proceeds in the direction of the second lens 45. The second lens is a convex lens for refracting the light that has passed through the second lens so as to travel in the direction of the pupil 73 of the eyeball 71. The light modulated by the image display element 41 is refracted by the second lens, passes through the crystalline lens 72 at the position of the pupil 73, reaches the retina 74, and projects the screen onto the retina.

  In the example shown in FIG. 4, the image display element 41, the first lens 44, and the second lens 45 are configured independently, but may be configured to be in close contact without changing the order of arrangement. . Since there is no distance between the lenses, the wearable display 30 can be reduced in size.

  The light output means is, for example, the image display light source 42, the scanning means is, for example, the scan mirror 43, and the refracting means and the light collecting means are, for example, the second lens.

  Since this retinal projection scanning method can project an image on the retina without using the function of the lens of the lens, the lens is in a state of looking far, and the eyes are not tired. Furthermore, a low vision person with weak vision can also recognize an image.

  FIG. 5 is a diagram in which the wearable display 30 shown in FIG. 1 is mounted on the head. A one-eye wearable display 30 is mounted on the head so that the image display unit 31 is positioned on the right eye. The image display unit 31 may be positioned in the left eye.

  In this way, by configuring the image display unit 31 to be attached to only one eye, it is possible to operate while confirming the surrounding situation with the other eye, so immediately when there is a change in the surrounding situation. It can correspond to.

  FIG. 6 is a diagram illustrating a second example of the configuration of the optical unit 35 illustrated in FIG. 2. The optical unit 35 is used in a binocular wearable display 30 including a left-eye optical unit and a right-eye optical unit. The left-eye optical unit includes a scan mirror 43a and a diffraction mirror 46a. The optical unit for the right eye includes a scan mirror 43b and a diffraction mirror 46b.

  The scan mirror 43a reflects the light output from the light source, and controls the direction of the reflected light by changing the angle with respect to the light output from the light source. The diffraction mirror 46 a reflects the light reflected by the scan mirror 43 a toward the image display element 41. The diffraction mirror 46a has a lens effect by utilizing, for example, light diffraction.

  The light reflected by the image display element 41a is modulated by the screen displayed on the image display element 41, passes through the pupil of the left eyeball 71a, and irradiates the retina. The image display element 41a is, for example, a reflective liquid crystal display. The optical unit for the right eye is symmetrical with the optical unit for the left eye, and a description thereof is omitted.

  Of the two types of images using the parallax between the left eye and the right eye, one image is projected onto the retina of the left eyeball, and the other image is projected onto the retina of the right eyeball, so that it is viewed stereoscopically. 3D (3-Dimension) images can be provided.

  FIG. 7 is a diagram illustrating a third example of the configuration of the optical unit 35 illustrated in FIG. 2. The third example uses Maxwell's view in which parallel rays modulated by the screen of the image display element 41 are condensed at the position of the eyeball lens and an image is projected onto the retina. As the light source, a point light source 47 such as a light emitting diode is used. The configuration excluding the light source and the scan mirror is the same as the configuration shown in FIG.

  The first lens 44 refracts the light output from the point light source 47 and irradiates the image display element 41 as a parallel light beam 48. The light transmitted through the image display element 41 is refracted by the second lens, collected at the position of the pupil 73, passes through the crystalline lens 72, reaches the retina 74, and projects an image on the retina. The light output means in the third example is a point light source 47 such as a light emitting diode.

  As described above, when Maxwell vision is used, an image can be projected onto the retina without using the lens function as a lens, so that the lens is in a state of looking far and the eyes are less likely to get tired. Furthermore, a low vision person with weak vision can also recognize an image.

  FIG. 8 is a diagram illustrating a fourth example of the configuration of the optical unit 35 illustrated in FIG. 2. The fourth example is a method other than the retinal projection scanning method, in which light is converted into a light beam that is thin parallel light, and an image is projected onto the retina. For example, an image on the image display element 41 is passed through a pinhole. Parallel light is used.

  The light beam 48a, which is parallel light transmitted through the image display element 41, is refracted by the condenser lens 45, that is, the second lens, in the direction of condensing at the center of the pupil 73 of the eyeball 71, and the light beam that has passed through the crystalline lens 72 is reflected. The retina is irradiated and a projection image A75 is projected.

  FIG. 9 is a diagram for explaining the operation principle of the transmissive image display element 41. For example, if the light modulated by the display image 77 in the screen displayed on the image display element 41 uses Maxwell's vision, the light is condensed at the position of the crystalline lens 72 and then reaches the retina. Projection image A75 is projected on top. The scenery image 78 by the outside light, that is, the light of the scene image entering the field of view, passes through the image display element 41 and is condensed on the retina by the crystalline lens 72 of the eyeball 71 to form the projection image B76.

  The lens 72 works as a lens when viewing a landscape image 78 by external light. However, since the light modulated by the display image 77 is condensed at the position of the pupil, the lens 72 functions as a lens. The projection image A75 is projected on the retina without being affected by this. Therefore, it is possible to recognize an image and a screen of a scene that enters the field of view.

  FIG. 10 is a diagram for explaining the operation principle of the non-transmissive image display element 41b. If the light modulated by the display image 77 in the screen displayed on the image display element 41b is, for example, using Maxwell's view, the light is condensed at the position of the crystalline lens 72 and then reaches the retina. Projection image A75 is projected on top. However, since the landscape image 78 by external light is reflected by the image display element 41b, the projection image B76 shown in FIG. 9 is not formed on the retina. Therefore, only the screen can be recognized, and a clear image that is not affected by external light can be seen.

  The configuration of the optical unit 35 is not limited to the configuration in the above-described embodiment, and may be any configuration that can project the screen displayed on the image display element 41 onto the retina.

  Thus, since the keyboard is provided as an image and the screen is directly projected onto the retina, a keyboard as an object and a display as an object become unnecessary, and the information processing apparatus can be reduced in size and weight. Therefore, the information processing apparatus can be a portable information processing apparatus that is easy to carry and has excellent operability.

  In the embodiment described above, the image display element 41 always displays a screen. However, when the virtual keyboard 19 is viewed, an image based on the screen displayed on the image display element 41 is not necessary. The display of the screen on 41 may be suppressed. For example, when the line-of-sight direction of the human eye is detected and the line-of-sight direction is the direction of the keyboard, the optical system control unit 34 suppresses display of the screen on the image display element 41.

  The detection of the line-of-sight direction can be realized, for example, by using the detection means described in Japanese Patent Laid-Open No. 8-328512. The line-of-sight detection means is, for example, detection means, and the suppression means is, for example, the optical system control unit 34.

  As described above, when inputting information by looking at the image of the keyboard, the display of the screen is suppressed, so that it is possible to prevent an image unrelated to information input from entering the field of view.

  FIG. 11 is a diagram showing a schematic configuration of a mobile computer 1 according to the second embodiment of the present invention. The mobile computer 1 includes an input processing device 60, a wearable display 30, and a connection cable B51. The input processing device 60 is obtained by integrating the virtual keyboard 10 and the computer 20 shown in FIG.

  The connection cable A50 shown in FIG. 1 is not necessary, and the transmission / reception unit for communication via the connection cable A50, that is, the first transmission unit 16 and the first reception unit 22 shown in FIG. . The transmission unit is, for example, the second transmission unit 23 illustrated in FIG. 2, and the reception unit is, for example, the second reception unit 32 illustrated in FIG.

  As described above, since a transmission / reception unit for communication via a connection cable is not required, it is possible to reduce the size and weight.

  FIG. 12 is a diagram showing a schematic configuration of a mobile computer 1 according to the third embodiment of the present invention. In the mobile computer 1 shown in FIG. 1, the connection between the computer 20 and the virtual keyboard 10 and the connection between the computer 20 and the wearable display 30 are performed by wired connection, but are performed by wireless connection that performs wireless communication by radio. Also good. That is, a wireless communication device may be used instead of the connection cable A50 and the connection cable B51. Wireless communication uses light or radio waves.

  In the example shown in FIG. 12, the computer 20 and the wearable display 30 are connected by wireless connection. The wearable display 30 includes a communication unit 36. The communication unit 36 is a communication device including an antenna and a transmission / reception circuit, for example, when communication is performed using radio waves. For example, an infrared light emitting element and an infrared detection element are used when communication is performed using light. Consists of including. The computer 20 is also configured to include a communication unit similar to the communication unit 36.

  In this way, it is not necessary to connect the computer 20 and the wearable display 30 attached to the head with a cable, the factor that restrains human movement can be reduced, and the sense of restraint at the time of wearing can be reduced. . In particular, in the case of wireless communication using light or radio waves, a commercially available communication device can be used, so a smaller and less expensive communication device can be selected.

  FIG. 13 is a diagram showing a schematic configuration of a mobile computer 1 according to the fourth embodiment of the present invention. The mobile computer 1 includes an input / output device 61, a computer 20, and a connection cable C52. The connection cable C52 is a cable for transmitting and receiving information transmitted and received by the connection cable A50 and the connection cable B51. For example, the connection cable C52 is an interface cable using USB or an optical fiber using an optical signal.

  The input / output device 61 is a device in which the virtual keyboard 10 and the wearable display 30 are integrated. The input / output device 61 includes a light irradiation unit 11, a light detection unit 12, a keyboard image generation unit 13, an input key extraction unit 14, and a first transmission unit 15 that configure the virtual keyboard 10 illustrated in FIG. 2 includes an image display unit 31, a second receiving unit 32, a display control unit 33, an optical system control unit 34, and an optical unit 35 that constitute the wearable display 30 shown in FIG.

  The light output unit 49 is, for example, the second lens 45 illustrated in FIG. 4 and is included in the optical unit 35. The light output from the light output unit 49 is applied to the eyes of the user 80, and an image is projected on the retina of the eyes of the user 80.

  Thus, since the keyboard is provided as an image and the screen is directly projected onto the retina, a keyboard as an object and a display as an object become unnecessary, and the information processing apparatus can be reduced in size and weight. Therefore, the information processing apparatus can be a portable information processing apparatus that is easy to carry and has excellent operability. Furthermore, since the input device and the output device are integrated, further downsizing, lighter weight, and lower price can be achieved.

  In the fourth embodiment, the light source used for the light irradiation unit 11 of the virtual keyboard 10 and the light source used for the optical unit 35 of the wearable display 30 use different light sources. You may use the light of a light source as a light source of the light irradiation part 11, and a light source of the optical part 35. FIG. Since the light source can be shared, it is possible to reduce the size, the weight, and the price.

  Furthermore, in the first embodiment shown in FIG. 1 and the second embodiment shown in FIG. 11, when the connection between the devices is an optical fiber, one light source is connected via the optical fiber. Light can be provided to other devices to serve as a light source for other devices. For example, the light from the light source of the light irradiation unit 11 of the virtual keyboard 10 is sent to the wearable display 30 via the connection cable A50 and the connection cable B51 realized by optical fibers, and used as the light source of the optical unit 35. Or conversely, the light of the light source of the optical part 35 of the wearable display 30 is sent to the virtual keyboard 10 via the connection cable B51 and the connection cable A50 realized by optical fibers, and used as the light source of the light irradiation part 11.

  Thus, since the light source can be shared, it is possible to reduce the size, the weight, and the price.

  FIG. 14 is a diagram showing a schematic configuration of a mobile computer 1 according to the fifth embodiment of the present invention. The mobile computer 1 includes an input processing device 62, a wearable display 30, and a connection cable B51. The input processing device 62 is obtained by removing the light irradiation unit 11 from the configuration of the input processing device 60 shown in FIG. The keyboard image is once synthesized and displayed on the screen displayed on the image display element 41 of the wearable display 30, and the keyboard image displayed on the screen is projected onto the retina.

  FIG. 15 is a block diagram of the mobile computer 1 shown in FIG. The input processing device 62 includes the light detection unit 12, the keyboard image generation unit 13, the input key extraction unit 14, the processing unit 21, and the second transmission unit 23, and includes the input processing device 60 illustrated in FIG. The light irradiation part 11 is remove | excluded from the structure. The wearable display 30 includes an image display unit 31, a second reception unit 32, a display control unit 33, an optical system control unit 34, and an optical unit 35, and has the same configuration as the wearable display 30 in FIG.

  The input key extraction unit 14 extracts the input key selected by each fingertip based on the position information of each fingertip detected by the light detection unit 12. The input key extraction unit 14 aligns the position of the fingertip detected by the light detection unit 12 with the reference position of the input key included in the keyboard. For example, when the position of each fingertip is first detected, The index finger and the index finger of the right hand are associated with the input key of the letter “F” and the input key of the letter “J”, respectively.

  The keyboard image generation unit 13 combines identification information for identifying the input key extracted by the input key extraction unit 14, for example, a circle with a keyboard image representing the keyboard, and keyboard image information indicating the combined keyboard image. And provided to the second transmitter 23. The second transmission unit 23 transmits the keyboard image information provided from the keyboard image generation unit 13 to the wearable display 30.

  The second receiving unit 32 of the wearable display 30 provides the display control unit 33 with the keyboard image information and the screen information transmitted from the input processing device 62. The display control unit 33 combines the screen information provided from the second reception unit 32 with the screen information provided from the second reception unit 32. Further, the synthesized image is converted into a video signal and provided to the image display unit 31. The functions of the other components are the same as the functions of the components of the mobile computer 1 shown in FIG.

  In this way, an image of the keyboard showing the fingertip position identification information is displayed on the screen of the wearable display 30 attached to the head. Therefore, the user can work while looking at the screen projected on the retina and the keyboard image, so that the amount of movement of the line of sight can be reduced. Furthermore, since the light irradiation unit 11 for projecting the virtual keyboard 19 on a desk or the like is not necessary, the size and weight can be further reduced.

  In the embodiment described above, the input key is extracted by detecting the position of the human fingertip by the light detection unit 12, but the input key may be specified by detecting the line-of-sight direction with respect to the image of the keyboard. . The detection of the line-of-sight direction and the confirmation of the input information can be realized, for example, by using the line-of-sight detection means and the input confirmation means described in JP-A-11-73273.

  The light detection unit 12 for detecting the position of the fingertip is not necessary, and the input processing device 62 shown in FIG. 14 can be further reduced in size and weight. Furthermore, although the input speed is inferior to the input using the keyboard, a place for moving the finger of the hand is unnecessary, and the mobile computer 1 can be used without a desk or the like.

1 is a diagram showing a schematic configuration of a mobile computer 1 according to a first embodiment of the present invention. It is a block diagram of the mobile computer 1 shown in FIG. It is a figure for demonstrating the photon detection part 12 shown in FIG. It is a figure which shows the 1st example of a structure of the optical part 35 shown in FIG. FIG. 2 is a diagram in which the wearable display 30 shown in FIG. 1 is mounted on the head. It is a figure which shows the 2nd example of a structure of the optical part 35 shown in FIG. It is a figure which shows the 3rd example of a structure of the optical part 35 shown in FIG. It is a figure which shows the 4th example of a structure of the optical part 35 shown in FIG. FIG. 5 is a diagram for explaining an operation principle of a transmissive image display element 41. It is a figure for demonstrating the principle of operation of the non-transmissive image display element 41b. It is a figure which shows the schematic structure of the mobile computer 1 which is the 2nd Embodiment of this invention. It is a figure which shows the schematic structure of the mobile computer 1 which is the 3rd Embodiment of this invention. It is a figure which shows the schematic structure of the mobile computer 1 which is the 4th Embodiment of this invention. It is a figure which shows the schematic structure of the mobile computer 1 which is the 5th Embodiment of this invention. It is a block diagram of the mobile computer 1 shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Mobile computer 10 Virtual keyboard 11 Light irradiation part 12 Light detection part 13 Keyboard image generation part 14 Input key extraction part 15, 23 Transmission part 19 Virtual keyboard 20 Wearable display 21 Processing part 22, 32 Reception part 30 Computer 31 Image display part 33 Display Control Unit 34 Optical System Control Unit 35 Optical Unit 36 Communication Unit 41 Image Display Element 42 Image Display Light Source 43 Scan Mirror 44, 45 Lens 46 Diffraction Mirror 47 Point Light Source 49 Light Output Unit 50-52 Connection Cable 60, 62 Input Processing Device 61 I / O device

Claims (22)

First projection means for projecting an image representing a keyboard including an input key for inputting information onto a predetermined projection plane;
Input key detection means for detecting an input key selected from among the input keys represented by the image projected by the first projection means;
Processing means for executing predetermined processing based on information indicated by the input key detected by the input key detection means;
Display means for displaying a screen including information based on processing executed by the processing means;
An information processing apparatus comprising: a second projecting unit that projects a screen displayed on the display unit onto a retina of a human eyeball.   The second projecting means includes a light output means for outputting a beam of light, a scanning means for scanning a screen displayed on the display means by changing the direction of the beam of light output from the light output means, and 2. The information processing apparatus according to claim 1, further comprising a refracting unit that refracts the beam beam scanned by the scanning unit in a direction in which the beam is irradiated onto the retina.   The second projecting means is a light output means for outputting light for irradiating the display means, and a light collecting means for projecting an image on the retina by condensing the light emitted to the display means at the position of the eyeball lens. The information processing apparatus according to claim 1, further comprising an optical unit.   The information processing apparatus according to any one of claims 1 to 3, wherein the second projecting unit projects an image onto one of the retina of the right eyeball and the retina of the left eyeball. .   The information processing apparatus according to claim 1, wherein the second projecting unit projects an image on a retina of a right eyeball and a retina of a left eyeball.   The said display means permeate | transmits the light which injects from the surface on the opposite side of the surface which faces an eyeball among the surfaces which comprise the display means, The Claim 1 characterized by the above-mentioned. Information processing device.   The said display means does not permeate | transmit the light which injects from the surface on the opposite side of the surface which opposes an eyeball among the surfaces which comprise the display means, The Claim 1 characterized by the above-mentioned. Information processing device.   The information processing apparatus according to claim 1, wherein the light output unit includes red, blue, and green light emitting diodes.   The information processing apparatus according to claim 1, wherein the light output unit includes a monochromatic light emitting diode or a monochromatic laser diode.   The information processing apparatus according to claim 9, wherein the single color is orange.   The input key detection means includes a second light output means for outputting light, a light detection means for detecting reflected light reflected by a human fingertip, and the light output from the second light output means. The information processing apparatus according to claim 1, further comprising an extraction unit that extracts an input key corresponding to the position of the fingertip based on the reflected light detected by the detection unit. Eye-gaze detecting means for detecting the eye-gaze direction;
And a deterring unit that deters projection of the image by the second projecting unit when the gaze direction detected by the gaze detecting unit is the direction of the keyboard of the image projected by the first projecting unit. The information processing apparatus according to claim 1, wherein: An image representing a keyboard including an input key for inputting information is projected on a predetermined projection plane, an input key selected from the input keys represented by the projected image is detected, and An input device for inputting information indicated by the detected input key;
A processing device that executes a predetermined process based on information input by the input device;
An information processing apparatus, comprising: an output device that projects a screen including information based on processing executed by the processing apparatus onto a retina of a human eyeball.   14. The information processing apparatus according to claim 13, wherein communication between the input device and the processing device and communication between the processing device and the output device are wireless communication.   The information processing apparatus according to claim 14, wherein the wireless communication is wireless communication using light or radio waves.   The information processing apparatus according to claim 13, wherein communication between the input device and the processing device and communication between the processing device and the output device are wired communication using an optical fiber. The input device includes a light source for projecting an image representing the keyboard, and transmits a part of the light output from the light source to the output device via the optical fiber,
The information processing apparatus according to claim 16, wherein the output device uses light transmitted through the optical fiber as a light source for projecting the screen. The output device includes a light source for projecting the screen, and transmits a part of the light output from the light source to the input device via the optical fiber,
The information processing apparatus according to claim 16, wherein the input device uses light transmitted through the optical fiber as a light source for projecting an image representing the keyboard. An image representing a keyboard including an input key for inputting information is projected on a predetermined projection plane, an input key selected from the input keys represented by the projected image is detected, and An input processing device that inputs information indicated by the detected input key and executes a predetermined process based on the input information;
An information processing apparatus comprising: an output device that projects a screen including information based on processing executed by the input processing device onto a retina of a human eyeball. An image representing a keyboard including an input key for inputting information is projected on a predetermined projection plane, an input key selected from the input keys represented by the projected image is detected, and An input / output device for inputting information indicated by the detected input key;
A processing device that executes a predetermined process based on information input by the input / output device,
The input / output device further projects a screen including information based on processing executed by the processing device onto a retina of a human eyeball. The input / output device includes a light source for projecting an image representing the keyboard;
The information processing apparatus according to claim 20, wherein the light source is also used as a light source for projecting the screen onto a retina. Display means for displaying an image representing a keyboard including input keys for inputting information;
Projection means for projecting the image displayed on the display means onto the retina of the human eyeball;
An input key detecting means for detecting an input key selected from among the input keys represented by the image projected by the projecting means;
Processing means for executing a predetermined process based on information indicated by the input key detected by the input key detection means,
The information processing apparatus, wherein the display means combines and displays a screen including information based on processing executed by the processing means with an image representing the keyboard.

Images