JP2010213214A - Head-mounted display - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a head-mounted display with which new original contents are easily created on the model of contents displayed by a display section. <P>SOLUTION: A head-mounted display includes a display means in which image light corresponding to content information is made incident to eyes of a user to allow the user to watch an image corresponding to the image light. The head-mounted display includes: an imaging means; a storage means for storing the content information; and a storage control means for storing in the storage means data of an image captured by the imaging means in association with the content information, in accordance with association information included in the content information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a head mounted display.

  2. Description of the Related Art Conventionally, a head-mounted display (hereinafter referred to as “HMD: Head Mount Display”) that includes display means for causing image light according to content information to enter a user's eyes and allowing the user to visually recognize an image according to the image light. )It has been known.

  Various types of usage can be considered for such an HMD. For example, it is possible to use such a mode as carrying a trip, displaying content such as a travel guide on the display means, and enjoying the trip according to the guidance.

  On the other hand, as a traveler's feelings, it is natural to keep a record of the trip with photos and videos, especially for the sights and tourist spots introduced in the travel guide with images etc. It's natural to want to leave a photo of yourself.

  Therefore, if an image can be taken at the places visited according to the travel guide, an original travel album simulating the travel guide can be created.

  The camera may be carried separately by the user, but many HMDs including, for example, Japanese Patent Application Laid-Open No. H10-28400, have been provided with a small camera that captures an image in the user's line-of-sight direction.

  Since the image captured by such a small camera can be displayed in front of the eyes by the display means of the HMD, the user can check the image or video image immediately after imaging on the spot. Therefore, it is easy to grasp the image of the travel album to be created and it is convenient.

JP 2006-139124 A

  However, even if an HMD equipped with a camera is used, when it comes to actually creating a travel album, images and video images captured by itself, images displayed in a travel guide displayed as content, It's not easy to associate

  In other words, it is necessary to check whether or not the contents of images and video images taken by the user are related to the image contents of the travel guide. However, the task of pasting them in order on a worksheet is time consuming even if dedicated application software is used.

  Therefore, it is desired from the market to provide an HMD that allows a travel guide serving as content to be handled as a template as it is and anyone can easily create a travel album or the like.

  Such an HMD is useful not only for creating a travel album, but also for creating a report or the like by pasting captured images of a predetermined place or specified object in a predetermined order, for example. It is considered that

  In order to achieve the above object, the invention described in claim 1 includes display means for causing image light according to content information to enter the user's eyes and allowing the user to visually recognize the image according to the image light. In the head-mounted display, the imaging unit, the storage unit that stores the content information, and the image data captured by the imaging unit according to the association information included in the content information are stored in the storage unit in association with the content information. And a storage control means.

  Further, in the invention according to claim 2, in the head mounted display according to claim 1, condition presence / absence determining means for determining whether or not the association information includes a condition of an imaging position by the imaging means, Detection means for detecting information for specifying an imaging position by the imaging means, and condition satisfaction determination means for determining whether the imaging position by the imaging means satisfies a condition of the imaging position based on a detection result by the detection means And the storage control unit stores the data of the image captured when the condition satisfaction determination unit determines that the condition is satisfied in the storage unit in association with the content information. .

  According to a third aspect of the present invention, in the head mounted display according to the second aspect, the detection means includes position detection means for detecting information on the latitude and longitude of the current position as information for specifying the imaging position. It is characterized by including.

  According to a fourth aspect of the present invention, in the head mounted display according to the second aspect, the detection means includes a tag reader that detects wireless tag information as information for specifying the imaging position.

  The invention according to claim 5 is the head-mounted display according to claim 2, wherein the detection unit displays the image when the image captured by the imaging unit includes a predetermined image. It is detected as information for specifying an imaging position.

  The invention according to claim 6 is the head mounted display according to any one of claims 1 to 5, wherein the imaging means images the visual field range of the user, and further the direction of the user's line of sight A line-of-sight direction detecting means for detecting the image, and an image pickup control means for causing the image pickup means to pick up an image of the visual field range of the user according to a detection result by the line-of-sight direction detecting means.

  The invention according to claim 7 is the head-mounted display according to any one of claims 1 to 6, wherein an image captured by the imaging unit and associated with the content information is included in the content information. Display control means for combining with the corresponding image and displaying it by the display means is provided.

  The invention according to claim 8 is the head-mounted display according to any one of claims 1 to 7, wherein the display means causes the user to visually recognize an image according to the image light so as to overlap the outside scene. It is a see-through type.

  According to the present invention, the data of the image captured by the imaging unit can be stored in association with the content information of the content displayed by the display unit. Therefore, for example, a new model using the displayed content as a template It is possible to easily create original content.

It is explanatory drawing which shows the use condition of a head mounted display (HMD). It is a schematic block diagram of HMD. It is explanatory drawing which shows the specific external appearance structure of HMD. It is the block diagram which showed the electrical structure and optical structure of HMD. It is a block diagram which shows the structure of the control part of HMD. It is explanatory drawing which shows an example of the usage pattern which concerns on 1st Embodiment of HMD. It is explanatory drawing which shows an example of the usage pattern which concerns on 1st Embodiment of HMD. It is explanatory drawing which shows an example of the usage form of HMD. It is explanatory drawing which shows the modification of the usage form of HMD. It is explanatory drawing which shows the flow of the main control process of HMD. It is explanatory drawing which shows the flow of the subroutine of the main control processing. It is explanatory drawing which shows the flow of the process which concerns on 2nd Embodiment of HMD.

  Hereinafter, a head mounted display (hereinafter referred to as “HMD: Head Mount Display”) according to a first embodiment will be described with reference to the drawings. In the present embodiment, the HMD is a retinal scanning type in which image light generated based on an image signal is two-dimensionally scanned, and the scanned image light is projected onto the user's eye to form an image on the retina. However, the present invention is not limited to this, and the image light corresponding to the content information is incident on the user's eyes together with the external light, and the image corresponding to the content information is displayed so as to be superimposed on the outside scene. The present invention can be applied to all devices having a configuration including means.

[1. Overview of HMD]
As shown in FIG. 1A, the HMD according to the present embodiment halves the image light 500 corresponding to the content information in a state where a part of the external light 400 reaches at least one of the eyes 101 of the user 100. A see-through type display unit 1 is provided that is reflected by the mirror 10 and projected onto the eyes 101 of the user 100 so that the user 100 can visually recognize the content image 200 superimposed on the outside scene.

  Further, a small CCD camera 2 capable of capturing an image of the visual field range of the user 100 is attached to the housing 13 (see FIG. 3) of the display unit 1. As will be described in detail later, in the present embodiment, the CCD camera 2 can perform imaging in the manual imaging mode and the automatic imaging mode.

  Then, as shown in FIG. 1B, the user 100 wearing the HMD on the head 102 is caused to visually recognize a travel guide as a content image 200 that is a display image. You can enjoy traveling according to the guide. Then, a desired photograph can be taken manually or automatically by the CCD camera 2 without taking out the camera separately at the destination.

  The travel guide, which is the content, is an itinerary in which travel distance and travel time are minimized as much as possible for each travel destination, and a plurality of tourist spots as destinations are shown in FIG. As shown, a predetermined content image 200 is displayed.

  The content image 200 includes, together with the representative image column 201, a text information column 202 including historical or social significance of the land and useful travel information regarding the land. Is displayed in a state of being superimposed on the outside scene in the field of view.

  The content information for displaying the content image 200 can be downloaded from a predetermined Internet site, or read from a recording medium such as a CD-ROM or DVD-ROM, a personal computer, or the like.

  As shown in FIG. 2, the HMD according to the present embodiment further includes a control unit 30 that controls the entire HMD.

  The control unit 30 includes storage means such as a ROM for storing content information, and stores image data captured by the CCD camera 2 in accordance with the association information included in the content information in the storage means in association with the content information. It functions as a storage control means.

  Therefore, the data of the image captured by the imaging unit can be stored in association with the content information of the content displayed by the display unit. For example, new original content using the displayed content as a template can be easily obtained. It becomes possible to create.

  Further, the control unit 30 is a condition presence / absence determining unit that determines whether or not the association information includes a condition of an imaging position by the CCD camera 2, a detecting unit that detects information specifying the imaging position by the CCD camera 2, Also, it functions as condition satisfaction determination means for determining whether the imaging position by the CCD camera 2 satisfies the condition of the imaging position based on the detection result by the detection means.

  Then, the control unit 30 stores data of an image captured when it is determined that a predetermined condition is satisfied, in the storage unit in association with the content information.

  In order to cause the control unit 30 to function as a detection unit that detects information for specifying the imaging position of the CCD camera 2, as will be described in detail later, the configuration includes a position detection unit such as a GPS, Longitude information can be detected as information specifying the imaging position.

  Alternatively, in order to cause the control unit 30 to function as a detection unit, as will be described in detail later, for example, an IC tag 830 is installed at a predetermined position on a predetermined sightseeing route (see FIG. 7), and the IC tag 830 A tag reader that detects tag information stored in the camera as information for specifying the imaging position may be provided.

  Alternatively, when the control unit 30 determines that a predetermined image is included in the image captured by the CCD camera 2, the image may be detected as information for specifying the imaging position.

  Thus, according to this embodiment, the user 100 can enjoy a comfortable trip by wearing the HMD according to this embodiment, displaying a travel guide as content, and following the guide. In addition, when the image is taken by the CCD camera 2 and the image pickup position satisfies the above-described conditions, it is stored in association with the content information.

  Thus, since the image data captured by the CCD camera 2 can be stored in association with the content information corresponding to the content (travel guide) displayed by the display unit 1, the travel guide is stored after the storage. It is possible to easily create new and original travel guides and commemorative travel albums at any time.

  The CCD camera 2 captures the field of view of the user 100, and the control unit 30 of the HMD according to the present embodiment detects the direction of the line of sight of the user 100 in cooperation with the CCD camera 96 described later. In addition to functioning as a line-of-sight direction detection unit, the CCD camera 2 also functions as an imaging control unit that causes the CCD camera 2 to image the visual field range of the user 100 according to the result of functioning as the line-of-sight direction detection unit.

  That is, as the operation mode of the CCD camera 2, in addition to the manual imaging mode in which the user 100 manually captures images, when the user 100 looks at a predetermined direction, the control unit 30 detects the direction as the line-of-sight direction, and the line-of-sight direction Is maintained for a predetermined time (for example, 3 seconds), there is an automatic imaging mode in which the shutter of the CCD camera 2 is automatically turned off for imaging.

  When the CCD camera 2 functions as a video camera, in the automatic imaging mode, an image when the line-of-sight direction is maintained for a predetermined time (for example, 3 seconds) during the monitor operation as the video camera is displayed. It is captured as a still image.

  As will be described in detail later, the control unit 30 is composed of a computer including a CPU 40, a program ROM 41, a flash R0M 42, a RAM 43, and the like, and the CPU 40 functions as the above means according to a control program stored in the program ROM 41. The control program stored in the program ROM 41 may be recorded on a recording medium such as a CD-R or DVD-R, for example, and may be read from a recording medium drive (not shown) into the flash R0M42. .

[2. Specific configuration including electrical configuration and optical configuration of HMD]
The outline of the HMD according to the present embodiment has been described above, and the specific configuration of the HMD will be described in detail below with reference to FIGS.

  As shown in FIG. 3, the HMD is connected to the housing 13 attached to the mounting portion 11 attached to the head 102 of the user 100 and the connection cable 52 including the optical fiber cable 50 and the transmission cable 51. A control unit 3 connected to the housing 13 is provided. The display unit 1 is composed of the projection unit 1 a housed in the housing 13 and the light source unit 1 b housed in the control unit 3. The mounting portion 11 is made of a support member having a substantially glasses shape, and can be easily mounted as if it were glasses.

  The housing 13 that houses the projection unit 1a is formed in a substantially L shape in plan view so as to cover the left eye portion and the left temple portion of the user 100 when the mounting portion 11 is mounted on the head 102 of the user 100. The CCD camera 2 is disposed on the upper surface 13 a side of the housing 13, and is connected to the control unit 30 of the control unit 3 via the transmission cable 51. Although not shown, the CCD camera 2 includes a shutter button, and in the manual imaging mode, an image can be taken by operating the shutter button. The control unit 3 may be provided with an operation button for imaging, and manual imaging may be performed by pressing the imaging button.

  Further, as shown in the figure, the half mirror 10 is attached to the front end side of the housing 13 so as to be positioned in front of the left eye 101, and the external light 400 is transmitted through the half mirror 10 to allow the user 100. The light is incident on the left eye 101, and the image light 500 is reflected by the half mirror 10 and is incident on the eye 101 of the user 100 (see FIG. 1A). Of course, the external light 400 is also incident on the right eye 101. Therefore, the user 100 can also see the outside world while viewing the content image 200 even when the right eye 101 is closed when the eyes 101 and 101 are opened.

  The control unit 3 can be carried by the user 100 in a clothes pocket or the like, and a power switch 7 and a power lamp 8 are provided on the surface of the case. Further, the control unit 30 housed and disposed in the control unit 3 together with the light source unit 1b transmits content information, image information, and the like via a wireless communication device 820 (see FIG. 6) via a wireless communication I / F 39. Can be obtained from outside. And the acquired image information etc. are transmitted to the image signal supply circuit 6 of the light source unit 1b. Note that content information and image information can be acquired by connecting to an external device such as a computer by wire.

  The image signal supply circuit 6 of the light source unit 1b modulates the intensity of the acquired image information in units of pixels to form image light 500, and transmits the image light 500 to the projection unit 1a via the optical fiber cable 50. In the projection unit 1a, the transmitted image light 500 can be scanned so that the user 100 can visually recognize the content image 200.

  4 is a block diagram showing the electrical configuration and optical configuration of the HMD. FIG. 5 is a block diagram showing the configuration of the control unit. As shown in FIG. 4, the HMD is provided in the mounting unit 11. The display unit 1 includes a projection unit 1 a and a light source unit 1 b disposed in the control unit 3, and a CCD camera 2 that can capture the field of view of the user 100. Moreover, in this embodiment, the control part 30 is provided in the control unit 3, and the GPS receiving part 610 is provided in this control part 30 so that it may mention later (FIG. 5). The GPS receiving unit 610 can receive a signal from a GPS satellite 600 (see FIG. 6), and the control unit 30 having the GPS receiving unit 610 cooperates with the GPS satellite 600 to determine an imaging position by the CCD camera 2. It functions as detection means for detecting the information to be specified.

  The control unit 30 disposed in the control unit 3 can control the overall operation of the HMD, and the light source unit 1b reads out image information in pixel units from the image signal S supplied from the control unit 30, Based on the read image information for each pixel, a luminous flux whose intensity is modulated for each color of R (red), G (green), and B (blue) is generated and emitted. Although details will be described later with reference to FIG. 5, the control unit 30 can be connected to the peripheral device 5, the wireless communication device 820, and the IC tag 830 as illustrated.

  The light source unit 1b is not provided in the control unit 3, but can be provided in the housing 13 attached to the mounting portion 11 similarly to the projection unit 1a. In the present embodiment, the display unit 1 will be described by taking, as an example, a retinal scanning type display unit that projects two-dimensionally scanned laser light into the eye 101 of the user 100 and projects an image on the retina 101b. For example, a liquid crystal display unit can also be used. Examples of the liquid crystal display unit include a transmissive liquid crystal panel that irradiates light from a light source to the transmissive liquid crystal panel and makes the light transmitted through the liquid crystal panel enter the user's eye as image light, or a reflective liquid crystal display. There is a reflective type in which light from a light source is irradiated onto a panel and light reflected from the liquid crystal panel is incident on a user's eye as image light.

(Light source unit 1b)
The light source unit 1b is provided with an image signal supply circuit 6 that generates a signal or the like as an element for synthesizing an image. When image data supplied from a computer (not shown) or the like is input to the control unit 30, the control unit 30 generates an image signal S based on the image data and sends it to the image signal supply circuit 6. Based on the image signal S, the image signal supply circuit 6 generates each signal that is an element for forming the content image 200 in units of pixels. That is, the image signal supply circuit 6 generates and outputs an R (red) image signal 60r, a G (green) image signal 60g, and a B (blue) image signal 60b. The image signal supply circuit 6 outputs a horizontal drive signal 61 used in the horizontal scanning unit 80 and a vertical drive signal 62 used in the vertical scanning unit 90, respectively.

  The light source unit 1b has an intensity based on the image signals 60r, 60g, and 60b of the R image signal 60r, the G image signal 60g, and the B image signal 60b output from the image signal supply circuit 6 in units of pixels. An R laser driver 66 and a G laser driver 67 for driving the R laser 63, the G laser 64, and the B laser 65, respectively, so as to emit image light 500 that is modulated laser light (also referred to as “light beam”). , B laser driver 68 is provided. Each laser 63, 64, 65 can be configured as, for example, a semiconductor laser or a solid-state laser with a harmonic generation mechanism. If a semiconductor laser is used, the drive current can be directly modulated to modulate the intensity of the laser beam. However, if a solid-state laser is used, each laser is equipped with an external modulator, and the intensity of the laser beam is modulated. Need to do.

  Furthermore, the light source unit 1b combines the collimated laser light and collimated optical systems 71, 72, 73 provided so as to collimate the laser light emitted from the lasers 63, 64, 65 into parallel light. There are provided dichroic mirrors 74, 75, and 76, and a coupling optical system 77 that guides the combined laser light to the optical fiber cable 50.

  Therefore, the laser beams emitted from the lasers 63, 64, 65 are collimated by the collimating optical systems 71, 72, 73, respectively, and then enter the dichroic mirrors 74, 75, 76. Thereafter, each of the laser beams is selectively reflected and transmitted with respect to the wavelength by these dichroic mirrors 74, 75, and 76. The three primary color laser beams incident on these three dichroic mirrors 74, 75, and 76 are reflected or transmitted in a wavelength selective manner, reach the coupling optical system 77, and are collected and output to the optical fiber cable 50. The

(Projection unit 1a)
The projection unit 1a located between the light source unit 1b and the eye 101 of the user 100 includes a collimating optical system 79 that collimates laser light generated by the light source unit 1b and emitted through the optical fiber cable 50. The laser beam collimated by the collimating optical system 79 is scanned horizontally in the horizontal direction for image display, and the laser beam scanned in the horizontal direction by the horizontal scanning unit 80 is scanned in the vertical direction. The vertical scanning unit 90, the first relay optical system 85 provided between the horizontal scanning unit 80 and the vertical scanning unit 90, and the laser beam thus scanned in the horizontal and vertical directions are emitted to the pupil 101a. A second relay optical system 95 is provided.

  The horizontal scanning unit 80 and the vertical scanning unit 90 scan in the horizontal direction and the vertical direction to scan the laser beam incident from the optical fiber cable 50 as an image so that the laser beam can be projected onto the retina 101b of the user 100. The horizontal scanning unit 80 includes a resonance type deflection element 81 having a deflection surface for scanning the laser beam in the horizontal direction, and resonating the deflection element 81 to shake the deflection surface of the deflection element 81. A horizontal scanning drive circuit 82 that generates a drive signal to be moved based on the horizontal drive signal 61 is provided.

  On the other hand, the vertical scanning unit 90 vertically drives a non-resonance type deflection element 91 having a deflection surface for scanning the laser beam in the vertical direction and a drive signal for swinging the deflection surface of the deflection element 91 in a non-resonance state. And a vertical scanning drive circuit 92 that is generated based on the signal 62, and for each frame of an image to be displayed, a laser beam for forming an image is vertically directed from the first horizontal scanning line toward the last horizontal scanning line. Scan to. Here, the “horizontal scanning line” means one scanning in the horizontal direction by the horizontal scanning unit 80.

  Here, galvanometer mirrors are used as the deflection elements 81 and 91. However, as long as the deflection surface (reflection surface) can be swung or rotated so as to scan the laser beam, piezoelectric driving, electromagnetic Any driving method such as driving or electrostatic driving may be used. In this embodiment, a resonance type deflection element is used for the horizontal scanning unit 80, and a non-resonance type deflection element is used for the vertical scanning unit 90. However, the present invention is not limited to this. A resonance type deflection element may be used, or both may be non-resonance type deflection elements.

  Further, the first relay optical system 85 that relays the laser light between the horizontal scanning unit 80 and the vertical scanning unit 90 converts the laser light scanned in the horizontal direction by the deflection surface of the deflection element 81 into the deflection surface of the deflection element 91. To converge. Then, the laser light is scanned in the vertical direction by the deflection surface of the deflection element 91, and the front of the eye 101 is passed through the second relay optical system 95 in which two lenses 95a and 95b having positive refractive power are arranged in series. Is reflected by the half mirror 10 positioned on the user 101 and enters the pupil 101a of the user 100, and the content image 200 corresponding to the image signal S is projected onto the retina 101b. As a result, the user 100 recognizes the laser light, which is the image light 500 incident on the pupil 101a, as the content image 200 (see FIG. 1).

  In the second relay optical system 95, the respective laser beams are made substantially parallel to each other by the lens 95a and converted into convergent laser beams. Then, the laser beams are converted into substantially parallel laser beams by the lens 95b, and the center lines of these laser beams are converted so as to converge on the pupil 101a of the user 100.

  Further, a half mirror 97 is provided between the second relay optical system 95 and the half mirror 10, and the light reflected by the half mirror 97 on the left eye 101 of the user 100 and its periphery is sent to the CCD camera 96. Incident light is incident. The control unit 30 detects the line-of-sight direction of the user 100 based on the iris position of the left eye 101 of the user 100 from the image captured by the CCD camera 96.

(Control unit 30)
The control unit 30 functions as the above-described determination unit, detection unit, storage control unit, condition satisfaction unit, and imaging control unit by executing predetermined processing described later according to a control program stored therein. In addition, the display unit 1 functions as a display control unit that controls the entire display. Furthermore, in the present embodiment, in cooperation with the CCD camera 96 disposed in the projection unit 1a, it also functions as a line-of-sight direction detection unit that detects the line-of-sight direction of the user 100.

  As shown in FIG. 5, the control unit 30 includes controllers 31, 32, 34, and 36, video random access memories (VRAMs) 33, 35, and 37, peripheral device interfaces (“I / F” in the figure). (Hereinafter also referred to as “I / F”) 38) and a communication I / F 39.

  The main controller 31 includes a CPU (Central Processing Unit) 40, a program ROM (Read Only Memory) 41 that is a nonvolatile memory, a flash ROM (flash memory) 42, and a RAM (Random Access Memory) 43. These are respectively connected to a data communication bus, and various types of information are transmitted and received through the data communication bus.

  The CPU 40 is an arithmetic processing unit that operates various units included in the HMD as the main controller 31 to execute various functions provided in the HMD by executing a control program stored in the program ROM 41. The flash ROM 42 stores image data output from the CCD cameras 2 and 96 and image data supplied from other devices.

  The HMD controller 32 controls the display unit 1 in response to a request from the main controller 31 and supplies the display unit 1 with an image signal S based on the image data stored in the HMD VRAM 33 by the main controller 31. When the image signal S is input from the HMD controller 32, the display unit 1 generates and scans laser light of each color whose intensity is modulated based on the image signal S, and emits it to the eye 101 of the user 100. An image corresponding to the image signal S is projected onto the retina 101b. As a result, the main controller 31 performs control to display an image.

  The camera controller 34 controls a CCD camera 96 that is an imaging means disposed in the display unit 1, and the camera VRAM 35 temporarily stores image data output from the CCD camera 96. The main controller 31 controls the CCD camera 96 via the camera controller 34 in order to recognize the line-of-sight direction of the user 100, causes the eyes of the user 100 and its surroundings to be captured by the CCD camera 96, and outputs from the CCD camera 96. The acquired image data is acquired via the camera VRAM 35. As a result, the main controller 31 can acquire an image captured by the CCD camera 96 and recognizes the direction of the line of sight of the user 100 by analyzing the image.

  The camera controller 36 can control the CCD camera 2 that is an image pickup means in two systems of manual imaging mode and automatic imaging mode, and the camera VRAM 37 temporarily stores image data output from the CCD camera 2.

  In the case of the automatic imaging mode, the main controller 31 controls the CCD camera 2 via the camera controller 34, and images the range including at least the display area of the image by the display unit 1 in the visual field range of the user 100 by the CCD camera 2. The image data output from the CCD camera 2 is acquired via the camera VRAM 37. Note that image processing can be performed at high speed by separately configuring an image processing unit in the main controller 31 in order to process images captured by the CCD cameras 2 and 96.

  The peripheral device I / F 38 is an interface for connecting the peripheral device 5 such as the power switch 7 (see FIG. 3) and lamps (for example, the power lamp 8 (see FIG. 3)) to the control unit 3. The main controller 31 receives operation information from the switches such as the power switch 7 from the peripheral device I / F 38 and performs processing according to the operation information, and also via the peripheral device I / F 38 according to the processing status. Supply lamp lighting information to lamps.

  The GPS receiver 610 can receive a signal from the GPS satellite 600 as described above, and each time the signal is received, the main controller 31 calculates the current position from the satellite orbit information included in the signal. Thus, the control unit 30 having the GPS receiving unit 610 functions as a detection unit that detects information for specifying the imaging position of the CCD camera 2.

  The communication I / F 39 controls the control unit 3 and the wireless communication device 820, the IC tag 830, and further an external device such as a personal computer so that they can communicate wirelessly. For example, the main controller 31 requests the wireless communication device 820 to supply the content information via the communication I / F 39, and the image signal S based on the image data included in the content information supplied via the communication I / F 39. Is supplied to the display unit 1.

  Further, the main controller 31 reads tag information included in the IC tag 830 via the communication I / F 39, and detects position information included in the tag information as information for specifying an imaging position by the CCD camera 2. Can do. That is, the main controller 31 and the communication I / F 39 of the control unit 30 serving as detection means function as a tag reader that detects information specifying the imaging position by the CCD camera 2 from the IC tag 830.

  In the above description, the CCD cameras 2 and 96 are used as the imaging means. However, the present invention is not limited to this, and a CMOS camera or the like may be used. In this embodiment, the GPS receiver 610 is housed in the controller 30 of the control unit 3. However, the GPS receiver 610 is housed in the housing 13 housing the projection unit 1a, and is connected to the controller 30 by a transmission cable or the like. It is good also as a structure to connect.

[3. Example of HMD use]
In the present embodiment, as a usage example of the HMD, a travel guide is displayed as a content image 200 on the display unit 1, and the user 100 enjoys a trip according to the travel guide while viewing the content image 200 and is attached to the HMD. It is assumed that a still image is captured using the CCD camera 2 and the user 100 creates a travel album using the captured image after the travel is completed. Here, a still image is described as an example of a captured image, but a moving image may be used instead of the still image, or a still image and a moving image may be mixed.

  In the present embodiment, the image data captured by the CCD camera 2 according to the association information included in the content information serving as a travel guide is stored in the flash ROM 42 in association with the content information. An electronic travel album using a guide as a model can be easily created, and can be displayed on the display unit 1 of the HMD as needed.

  As described above, when the created electronic travel album is displayed on the display unit 1 of the HMD, the control unit 30 functions as a display control unit, and is an image captured by the CCD camera 2 and associated with the content information. Are combined with an image corresponding to the content information and displayed on the display unit 1.

  The created electronic travel album can be displayed not only on the HMD but also on other monitors. It is also possible to print out the data of an electronic travel album on paper and make it a specification like an actual travel album.

  Here, “the data of the image captured by the CCD camera 2 in accordance with the association information included in the content information is stored in the flash ROM 42 in association with the content information” means the following.

  That is, the content information includes image information for generating the content image 200 and association information, and the image information includes the location information (position information) of the tourist spot that is the destination and the content image 200. While the image information and text information in the image and the information indicating the respective screen display positions are included, the association information includes information indicating the “imaging position condition” by the CCD camera 2 and imaging by the CCD camera 2. Link information of the image with the content image 200 is included.

  In addition, the album content image 300 of the travel electronic album (see FIG. 8) created using the travel guide (content information) as a template includes a representative image column 201 (see FIG. 8) in the content image 200 displayed as the travel guide. 1 (b)) is provided with an image column 301 in which a captured image is displayed.

  When creating the travel electronic album based on the content information, the control unit 30 displays the captured image so that the image captured by the user 100 with the CCD camera 2 is displayed in the image column 301 of the album content image 300. Is stored in the flash ROM 42 in association with the content information.

  More specifically, as the content image 200 displayed as a travel guide, a plurality of content images 200A, 200B,... Corresponding to a plurality of sightseeing spots A, B,. When the captured image captured at the tourist spot A is associated with the content image 200A, the captured image is captured when the content image 200A is displayed and the “imaging position condition” by the CCD camera 2 as the association information is satisfied. A unique file name such as file 200A01 or file 200A02 is attached to the image data and stored in the content information folder.

  When displaying the album content image 300A of the travel electronic album corresponding to the content image 200A, the captured image data is obtained from the content information folder because the captured image data has a unique file name. And displayed in the image column 301.

  Alternatively, when the content image 200A is displayed and the “imaging position condition” by the CCD camera 2 is satisfied as the association information, the captured image data corresponds to the content images 200A, 200B,. Are sequentially stored in the selected folder.

  In this case, when the album content image 300A of the travel electronic album corresponding to the content image 200A is displayed, captured image data is acquired from the folder corresponding to each of the content images 200A, 200B,. Will be displayed.

  As described above, the control unit 30 stores the data of the image captured by the CCD camera 2 in association with the content information. At this time, the control unit 30 stores the “imaging position of the imaging position” by the CCD camera 2 in the association information. It is determined whether or not “condition” is included, information for specifying the imaging position by the CCD camera 2 is detected, and the imaging position by the CCD camera 2 is determined based on the detection result. The image data captured when it is determined that the condition is satisfied is stored in the flash ROM 42 in association with the content information.

  As described above, the control unit 30 includes the GPS reception unit 610 and functions as a detection unit that detects information for specifying the imaging position of the CCD camera 2. That is, since the control unit 30 includes a so-called GPS function as a position detection unit, it can detect the current position of the CCD camera 2, that is, information on the current latitude and longitude of the HMD. Then, when the content information includes “imaging position condition” by the CCD camera 2 as association information, the detection result of the position detection unit of the image captured when the detection result satisfies the “imaging position condition”. The control unit 30 stores the data in the flash ROM 42 in association with the content information as described above, that is, in association with the position of the representative image column 201 of the content image 200 serving as a travel guide.

  Further, as described above, the control unit 30 detects the tag information stored in the IC tag 830 (see FIG. 7) installed at a predetermined place such as a sightseeing spot as information for specifying the imaging position (mainly). Even when the tag information detected by the tag reader satisfies the “imaging position condition” included in the content information, the control unit 30 converts the image data into the content information. And stored in the flash ROM 42.

(Specific usage example of HMD)
Here, a specific usage example of the HMD according to the present embodiment will be described with reference to FIGS. 6 to 8 are explanatory diagrams illustrating an example of the usage state of the HMD according to the present embodiment, and FIGS. 9 and 10 are explanatory diagrams illustrating a flow of control processing executed by the control unit 30. FIG.

  As shown in FIG. 6, a user 100 who has gone on a trip using the HMD first has a travel guide as content information at a predetermined trip start location (here, the nearest station 810 of the trip destination). Data is downloaded and stored in the flash ROM 42. The data of the travel guide includes information on itineraries assembled from the station 810 as a starting point.

  That is, as shown in FIG. 6, a wireless communication device 820 is installed at a station 810, and this wireless communication device 820 is connected to a computer (not shown), and this computer connects a predetermined travel site via the Internet 800. It can be connected to the server device 700 to be operated. That is, in the present embodiment, the user 100 obtains content information serving as a travel guide for travel from the server device 700 via the Internet 800 at the station 810.

  For example, in the itinerary, first go to the park and visit the □□ monument and □□ garden, then move to the sea by bus etc. While walking along the highway, you can visit sake breweries, etc., move on foot as it is, and go to the □□ tower, which is a symbol of the city center of modern buildings as a new city center. Here, the △ park and □□ monument correspond to the above-mentioned tourist spots A and B, and the content image 200 corresponding to the △ park and □□ monument is displayed in the travel guide as content information, respectively. Will be.

  Further, as shown in the figure, the GPS satellite 600 and the GPS receiving unit 610 (see FIG. 5) provided in the HMD can specify the position of the user 100 during the trip based on the latitude and longitude information. The position of the CCD camera 2 provided in the HMD to which is attached is also specified.

  Thus, the user 100 who wears the HMD and acquires the travel guide as the content information starts the travel according to the pre-built itinerary starting from the station 810.

  As shown in FIG. 1B, the content image 200 displayed based on the content information that is a travel guide is composed of an image showing a destination or an image of a famous tourist spot existing at the destination. A text information field 202 in which travel information about the destination and travel information about the destination is posted is included together with the representative image field 201, and the user 100 views the destination information by viewing the content image 200. It can move reliably while obtaining.

  In addition to the content image 200, the content information acquired at the station 810 includes association information with an image captured by the user 100 using the CCD camera 2. In this association information, information indicating an optimal position (imaging position area) for imaging an imaging object suitable for imaging at the tourist spot introduced in the content image 200 with the CCD camera 2 is “the imaging position condition”. May be included.

  On the other hand, the control unit 30 of the HMD constantly monitors the position where the CCD camera 2 exists, that is, the position of the user 100 wearing the HMD while acquiring information on latitude and longitude by GPS, and the information is the content. When it is included in “information indicating an optimum position (imaging position area) for imaging with the CCD camera 2” as the association information included in the information, that is, when the imaging position satisfies the “imaging position condition” The image data captured by the CCD camera 2 is stored in the flash ROM 42 in association with the content information.

  The control unit 30 includes a tag reader including a main controller 31 and a communication I / F 39. As shown in FIG. 7, even when tag information is acquired from the IC tag 830 appropriately provided at various sightseeing spots introduced in the content image 200, as long as the tag information can be acquired in the imaging position area, Assume that the conditions of the imaging position are satisfied. Therefore, image data captured by the CCD camera 2 in the imaging position area is stored in the flash ROM 42 in association with the content information. In the example shown in FIG. 7, the user 100 observes a □□ monument and a □□ garden in the park, and a guide board provided with an IC tag 830 is installed in the vicinity of the monument and the garden. In addition, an imaging position area is formed in the vicinity of the guide plate.

  As described above, when the imaging position satisfies the conditions of the imaging position (when the CCD camera 2 exists in the imaging position area), the HMD according to the present embodiment allows the user 100 to manually operate the CCD camera 2. In addition to storing the captured image in the flash ROM 42 in association with the content information, data of the image captured by the CCD camera 2 and captured automatically can be stored in the flash ROM 42 in association with the content information.

  That is, the CCD camera 2 images the visual field range of the user 100, and further functions as a visual line direction detection unit by the control unit 30 and the CCD camera 96 as described above. The direction of the line of sight of the user 100 can be detected. The control unit 30 functions as an imaging control unit, and causes the CCD camera 2 to image the visual field range of the user 100 according to the detection result of the line-of-sight direction.

  Thus, since automatic imaging can be performed, for example, if the user 100 is in an area that satisfies the conditions of the imaging position, that is, in the imaging position area, the user 100 looks at the direction of the desired imaging target. It is possible to take an image automatically only with this. Therefore, there is no inconvenience of searching for a place that will be a photo opportunity and releasing the shutter, as in the case of ordinary camera photography.

  The data of the automatically captured image is stored in the flash ROM 42 in association with content information indicating the display position of the representative image column 201 in the content image 200 of the travel guide.

  For example, it is assumed that the user travels according to the content information (travel guide) displayed on the display unit 1 and travels a sightseeing spot previously set in the itinerary, and is automatically imaged by the CCD camera 2 at many places during that time. Since the captured image data is stored in the flash ROM 42 in association with the content information, the user 100 can create new content, such as a travel electronic album, at any time later using the image data. Can do.

  Even in the case of an image captured manually, if the captured position is within the area satisfying the “imaging position condition”, it is assumed that the association condition is satisfied, and this is also content. The information is stored in association with the information (see step S109 in FIG. 10). Therefore, the user 100 can also use an image captured manually to create a travel electronic album.

  The album content image 300 that is a travel electronic album shown in FIG. 8 is a model of content used as a travel guide (see FIG. 1B), and is an image corresponding to the representative image column 201 of the travel guide. In the column 301, images captured by the user 100 at the tourist spots displayed in the content image 200 are displayed, and a comment column 302 created by the user 100 is displayed corresponding to the text information column 202 of the travel guide. ing.

Further, the album content image 300 shown in FIG. 8 is an image in which images corresponding to the content image 200 provided for each tourist spot of the travel guide are continuously displayed in a band shape. The corresponding album content image 300 may be displayed individually. As shown in FIG. 8, the album content image 300 displayed in a band shape can be scroll-displayed by a predetermined operation.
Note that the comment field 302 in the album content image 300 may not be newly generated by the user 100 but may use the travel information described in the text information field 202 of the travel guide as it is.

  Further, it is desirable that the image column 301 can display not only one image but also all images captured in association with the tourist spot. For example, even if the size of the image column 301 corresponds to one image, the captured images are sequentially displayed on the image column 301 by a predetermined operation.

  Further, what is shown in FIG. 9 is a modification of the album content image 300, and the image captured by the user 100 is not directly displayed on the album content image 300. That is, the image of the representative image column 201 of the travel guide content image as a template is displayed as it is, while the image taken at the tourist spot displayed as the image of the representative image column 201 below the text information column 202. A link information column 303 including link information linked to is formed. In this case, an image captured by the user 100 can be displayed by following the link destination address displayed in the link information column 303.

  In this way, a travel electronic album can also be created by a method of linking an image captured by the user 100 to the content image 200 representing a tourist spot without changing the appearance of the travel guide.

  The above-described album content image 300 created by the user 100 can be displayed using an HMD, or can be appropriately displayed on a monitor via a personal computer, for example. It is also possible to output paper from a printer and make it look like an actual album. Note that when the album content image 300 has a belt-like shape as in the present embodiment, for example, a scroll unit or the like is appropriately provided in the control unit 3 in order to display in the HMD.

  Hereinafter, the flow of HMD control processing centered on the control unit 30 will be described with reference to FIGS. 10 and 11. FIG. 10 shows a flow of the main control process of the HMD, and FIG. 11 shows a flow of a subroutine of the main control process. In FIG. 10, the power switch 7 of the HMD has already been turned on, and the initial setting process after the power switch 7 is turned on has been completed.

  As shown in FIG. 10, first, content information of a destination as a travel destination is downloaded (step S101). That is, in the present embodiment W, content information serving as a travel guide is downloaded from the server device 700 via the Internet 800 (see FIG. 6) at the station 810 serving as a travel start location, and the main controller 31 of the control unit 30. The CPU 40 stores this in the flash ROM 42. This “destination content information” includes content information (travel guide information) having association information.

  Next, the CPU 40 determines whether or not content (travel guide) having association information is being displayed (step S102). That is, when the content image 200 based on the downloaded content information is displayed on the display unit 1 and the content information of the displayed content image 200 is captured by the user 100 using the CCD camera 2. It is determined whether or not the association information for the image to be included is included.

  The content information having this association information is started to be displayed in step S106 described later. Immediately after loading the destination content information, the determination in step S102 is No, and the CPU 40 proceeds to step S103. Transfer.

  On the other hand, if it is determined in step S102 that the content image 200 including the association information is being displayed (association information is included in the content information) (step S102: Yes), the CPU 40 proceeds to step S108. Transfer.

  In step S103, the CPU 40 determines whether or not the user is at a tourist spot that is the destination. That is, information on the current position of the HMD worn by the user 100, that is, information on longitude and latitude is obtained by the GPS function, and position information (including content information) of a tourist spot that is a destination indicated in the travel guide It is determined whether or not the user 100 is at the destination (sightseeing spot).

  As a method for determining whether or not the user is at the destination (tourist spot), in addition to the GPS function, as described above, when the RFID tag information is acquired from the IC tag 830 provided at an appropriate place of the tourist spot as described above. However, the CPU 40 can determine that it is at the destination. That is, the CPU 40 determines that the user is at the destination if the location information of the sightseeing spot included in the content information matches the tag information read from the IC tag 830.

  If it is determined in step S103 that the user is at the destination (step S103: Yes), the CPU 40 has association information such as “information indicating an optimum position (imaging position area) for imaging with the CCD camera 2”. The display of the content image 200 on the display unit 1 is started (step S106), and then the process proceeds to step S112.

  On the other hand, if it is determined in step S103 that the user has not yet arrived at the destination (step S103: No), the CPU 40 has received an instruction to display the content image 200 having the association information on the display unit 1 from the user 100, for example. If there is a display instruction (step S104: Yes), the process proceeds to step S106 in the same manner as when it is determined that the user is at the destination, and then the process proceeds to step S112. . Note that an operation unit (not shown) is provided in the control unit 3. For example, when a predetermined operation is performed on the operation unit, the CPU 40 is instructed to display the content image 200 having association information on the display unit 1. judge. When only one piece of content information having association information is included in the “destination content information”, the CPU 40 determines that the content image 200 having the association information has been triggered to be displayed on the display unit 1. The process may be moved to step S104.

  As a result of the processing from step S104 to step S106, if the user 100 is not at the destination but will arrive soon, for example, the user 100 determines that the “optimum position for imaging with the CCD camera 2 (imaging position area It is possible to go to the destination while displaying the content image 200 having the association information such as “information indicating ()”.

  In step S104, when there is no instruction to display the content image 200 having the association information on the display unit 1 (step S104: No), the CPU 40 moves the process to step S105. In step S105, it is determined whether there is an instruction to display a normal content image. This normal content image is, for example, content information such as a movie or music promotion, and does not have association information like the content information of the destination. The normal content image is downloaded from a predetermined Internet site and stored in advance in the flash ROM 42, for example.

  If there is a display instruction (step S105: Yes), the CPU 40 starts displaying a normal content image (step S107), and then moves the process to step S112. On the other hand, if there is no display instruction (step S105: No), the process returns to step S103.

  In step S102, when the content image having the association information is being displayed (when the association information is included in the content information), it is determined whether or not the user is at the destination in the process of step S108. When it determines with not being in the destination (step S108: No), CPU40 moves a process to step S112.

  On the other hand, when it is determined that the user is at the destination (step S108: Yes), the imaging process is executed (step S109).

  Explaining this imaging process, as shown in FIG. 11, the CPU 40 first determines whether or not a manual imaging start trigger of the CCD camera 2 has been detected (step S201).

  When the CPU 40 detects the manual imaging start trigger (step S201: Yes), the CPU 40 releases the shutter of the CCD camera 2 and images (step S202). Then, the CPU 40 attaches information indicating the imaging position to the captured image data and temporarily stores it in the RAM 43. Here, the manual imaging start trigger is a trigger for performing imaging in the manual imaging mode, and is a signal that detects a pressing operation of a shutter button (not shown) provided in the CCD camera 2.

  When the manual imaging start trigger is not detected in step S201 (step S201: No), the CPU 40 determines whether or not the automatic imaging start condition is satisfied in step S203. That is, the association information corresponding to the content image 200 currently displayed on the display unit 1 includes “information indicating an optimum position (imaging position area) for imaging with the CCD camera 2” (that is, “imaging position condition”). ”) Is included, and it is determined whether or not the automatic imaging mode has been entered based on whether or not the information matches the current HMD position information (position information where the user 100 is present). The acquisition of the current HMD position information is the same process as the “determination process of whether or not the vehicle is at the destination” described in the process of step S103.

  When the automatic image capturing condition is satisfied (step S203: Yes), when the automatic image capturing condition is satisfied, that is, the CPU 40 shifts to the automatic image capturing mode, detects the line-of-sight direction of the user 100, and the user 100 determines the predetermined direction. When the direction is looked at for a predetermined time (for example, 3 seconds), the shutter is released and the image is taken (step S204). The CPU 40 temporarily stores the captured image in the RAM 43. The “predetermined direction” is a direction within the imaging range of the CCD camera 2, and the focus position of the CCD camera 2 is determined according to the detected line-of-sight direction of the user 100 and focus adjustment (focus adjustment) is performed. May be.

  After the process of step S204, or when it is determined in step S203 that the conditions for starting automatic imaging are not satisfied, the CPU 40 ends the imaging process and moves the process to step S110.

  Returning to FIG. When the imaging process is finished, the CPU 40 determines whether or not there is a captured image in step S110.

  If there is no captured image (step S110: No), the process proceeds to step S112. On the other hand, if it is determined that there is a captured image (step S110: Yes), that is, in the imaging process of step S109 described above (FIG. 11). If there is a captured image, the CPU 40 moves the process to step S111.

  In step S <b> 111, the CPU 40 stores (saves) the image data captured in step S <b> 109 in association with the content information of the content currently displayed on the display unit 1 in the flash ROM 42. For example, a unique file name is given to the content information so that the captured image to be displayed in the image field 301 corresponding to which representative image field 201 in the content image 200 of the travel guide as the content can be identified. The data is stored in the flash ROM 42 in association with each other, and then the process proceeds to step S112.

  In step S112, the CPU 40 determines whether or not the display of the content (travel guide) has ended, and if it has not ended (step S112: No), the process returns to step S102, and the processes from step S102 to step S111 are performed. Will be repeated. On the other hand, if it has been completed (step S112: Yes), this main control process is terminated.

  If it is determined in step S110 that there is an image captured in step S109 (step S110: Yes), the image captured in step S109 is used as content (travel guide) before the processing in step S111 described above. You may make it display on the display part 1 linked | related. By doing in this way, the user 100 can confirm the image captured manually or automatically. At that time, the user 100 may select whether or not to store (save) the image captured in step S109. For example, the CPU 40 displays information for selecting whether or not to save the image captured in step S109 on the display unit 1, and according to the operation of the user 100 to a selection button (not shown) provided in the control unit 3. It is determined whether to move to step S111 or delete the image captured in step S109. In this way, if there is an image that seems to be an imaging failure, the image can be erased by a predetermined operation.

  By the way, although a still image has been described as an example of a captured image, when a moving image is automatically captured as a captured image, the timing of the start and end of the imaging is, for example, that the imaging position is “captured by the CCD camera 2. It is conceivable that a predetermined time has elapsed from when it matches the “information indicating the optimal position (imaging position area)”, or that the imaging is continued while remaining in this imaging position area.

  Further, in the embodiment described above, when determining whether or not the “imaging position condition” by the CCD camera 2 included in the content information is satisfied, the information for specifying the imaging position by the CCD camera 2 is set to GPS. Although the latitude and longitude information of the current position acquired by the function or the tag information detected from the IC tag 830 is used, the determination can be made as follows.

  For example, in the determination processing in step S103 in FIG. 10, step S109, or step S204 in FIG. 11, the control unit 30 displays the image when the predetermined image is included in the image captured by the CCD camera 2. Then, it is detected as information specifying the imaging position, and it is determined whether or not this information is satisfied as information (information specifying the imaging position) related to the image in the representative image column 201 of the content. For example, when the image is manually captured, it is determined later whether the captured image satisfies the condition of the imaging position included in the association information, or when the CCD camera 2 is used as a video camera. Is particularly useful.

  Thus, by storing the image data captured by the CCD camera 2 in accordance with the association information included in the content information displayed by the display unit 1 as a travel guide in the flash ROM 42 in association with the content information, for example, travel After finishing the above, the user 100 can easily create a travel electronic album having the mode shown in FIGS.

  The destination content information may include content information other than the content information having the association information. For example, in addition to directions to the destination, content information (route guide content information) that displays information such as the introduction of a land that is simply a passing point or an explanation of how to move to the destination, etc. In this case, the CPU 40 displays the route guide content information when the content information of the destination is downloaded or when the user 100 performs a predetermined operation on an operation unit (not shown). be able to.

[5. Second Embodiment]
In the above-described first embodiment, the case where the HMD is used for travel has been described. However, the HMD can be used for factory inspection, for example.

  That is, when inspecting the operating state of a predetermined factory, the content information of the inspection mode in which information such as the production line to be visited in advance and the imaging target to be left in the photograph is recorded is created, and the HMD stored in the flash ROM 42 is stored. It is installed and a factory inspection is performed.

  On the HMD side, an imaging target or the like is automatically detected according to related information included in the content information, and this is imaged and stored in association with the content information. After the inspection is completed, an inspection report having a predetermined format associated with the content information in the inspection mode can be easily created using the stored captured image.

  FIG. 12 is an explanatory diagram showing a flow of processing of the control unit 30 in the inspection mode in which the HMD is used for factory inspection. As shown in the figure, the CPU 40 first loads content information corresponding to the destination in the inspection mode (such as a factory to be inspected). That is, the inspection mode content information is read from a personal computer or the like and stored in the flash ROM 42 (step S301). The content information in the inspection mode includes content information for each production line to be imaged, and the content information for each production line includes the image and description of the production line to be imaged, and the above purpose. Association information is included as well as this content information of the ground.

  When the storage of the content information in the inspection mode in the flash ROM 42 is completed, the CPU 40 causes the display unit 1 to display an image of the content of the production line to be imaged (hereinafter also referred to as “content image to be imaged”). It is determined whether or not there is (step S302).

  The content image to be imaged starts to be displayed in step S306 to be described later. Immediately after loading the content information corresponding to the destination in the inspection mode, the determination in step S302 is No, and the CPU 40 performs the process. Move to step S303.

  On the other hand, when it is determined in step S302 that the content image to be imaged is being displayed (step S302: Yes), the CPU 40 moves the process to step S308.

  In step S <b> 303, the CPU 40 determines whether or not it is in the production line that is the imaging target. That is, whether to acquire information on the current position of the HMD worn by the user 100, that is, information on longitude and latitude by the GPS function, or detect RFID tag information from an IC tag installed on the production line Then, it is determined whether or not the user 100 is in the production line to be imaged by comparing the information and the position information of the production line to be included in the content.

  When it is determined in step S303 that the image is on the production line to be imaged (step S303: Yes), the CPU 40 starts displaying the content image including the image of the production line to be imaged on the display unit 1 (step S303). Next, the process proceeds to step S314.

  On the other hand, if it is determined that the user has not yet come to the production line (step S303: No), the CPU 40 determines whether there is an instruction to display the content image to be imaged from the user 100 on the display unit 1, for example. (Step S304) When there is a display instruction (Step S304: Yes), the process proceeds to Step S306 in the same manner as when it is determined that the production line is to be imaged, and then the process proceeds to Step S314. Note that an operation unit (not shown) is provided in the control unit 3. For example, when a predetermined operation is performed on the operation unit, the CPU 40 determines that there is an instruction to display the content image to be captured on the display unit 1. .

  If the process from step S304 to step S306, for example, is not in the production line to be imaged but is about to arrive soon, the user 100 must select a production line that must be imaged using the CCD camera 2. It is possible to go to the production line of the imaging target while displaying the content image of the imaging target having the information to be displayed.

  In step S304, if there is no instruction to display the content image in the inspection mode on the display unit 1 (step S304: No), the CPU 40 moves the process to step S305. In step S305, it is determined whether there is an instruction to display a normal content image. The normal content image is, for example, content information such as inspection material data or a general inspection manual, and does not have association information like the content information for each production line to be imaged. The normal content image is downloaded from a computer in the company and stored in advance in the flash ROM 42, for example.

  If there is a display instruction (step S305: Yes), the CPU 40 starts displaying a normal content image (step S307), and then moves the process to step S314. On the other hand, if there is no display instruction (step S305: No), the process returns to step S303.

  In step S302, when the content image to be imaged is being displayed, the process of step S308 determines whether or not the manufacturing line is the imaging target and determines that the image is not yet on the manufacturing line. If it is determined (step S308: No), the CPU 40 moves the process to step S314.

  On the other hand, when it is determined that the manufacturing line is the imaging target (step S308: Yes), the CPU 40 determines whether the imaging start condition is satisfied (step S309). That is, the CPU 40 detects the line-of-sight direction of the user 100, and when the user 100 determines that the user 100 has looked at the predetermined direction for a predetermined time (for example, 3 seconds) (step S309: Yes), the line to be imaged by the CCD camera 2 is determined. An image is taken (step S310). When the imaging start condition is not satisfied (step S309: No), the CPU 40 moves the process to step S314. The “predetermined direction” is a direction within the imaging range of the CCD camera 2, and the focus position of the CCD camera 2 is determined according to the detected line-of-sight direction of the user 100 and focus adjustment (focus adjustment) is performed. May be.

  After step S310, the CPU 40 stores data of the captured image in the flash ROM 42 in association with the content information of the inspection mode in which the captured image is being displayed (step S311).

  Next, the CPU 40 determines whether the imaging of the imaging target has been completed (step S312). If it is determined that the imaging has not been completed yet (step S312: No), it is determined that the imaging has been completed (step S312). : Yes), the display of the content image to be imaged is terminated (step S313), and the process proceeds to step S314.

  In step S314, the CPU 40 determines whether or not the inspection mode has ended. If the inspection mode has not ended (step S314: No), the process returns to step S302, and the processes from step S302 to step S313 are repeated. On the other hand, when it has been completed (step S314: Yes), this inspection mode process is terminated.

  Although the present invention has been described through the above-described embodiments, the following effects can be expected according to this embodiment.

  (1) An HMD (display device) including a display unit 1 (display unit) that causes image light 500 according to content information to enter the eye 101 of the user 100 and causes the user 100 to visually recognize the content image 200 according to the image light 500. In the head mounted display), the CCD camera 2 (imaging means), the flash ROM 42 (storage means) that stores the content information, and the image data captured by the CCD camera 2 according to the association information included in the content information Since the control unit 30 (storage control unit) that stores the content information in the flash ROM 42 is provided, for example, new original content can be easily created using the displayed content image 200 as a template. Is possible. That is, if the template content is a travel guide or the like, a new album such as a travel album, or if the template content is any kind of inspection content, an inspection report or the like as new content, Each can be easily created.

  (2) The control unit 30 detects condition presence / absence determining means for determining whether or not the association information includes a condition for the imaging position by the CCD camera 2 and information for specifying the imaging position by the CCD camera 2 Functioning as a condition satisfaction determining unit that determines whether the imaging position of the CCD camera 2 satisfies a condition of the imaging position based on a detection result by the detection unit, and the control unit 30 Since the image data captured when it is determined that the imaging position condition is satisfied is stored in the flash ROM 42 in association with the content information, a necessary image is automatically displayed without the need for the user 100 to think about each time. This saves the user 100 trouble.

  (3) Since the control unit 30 includes the GPS receiving unit 610 (position detecting unit) that detects the latitude and longitude information of the current position as information for specifying the imaging position, the control unit 30 functions as a detecting unit. The imaging position by the CCD camera 2 can be obtained accurately.

  (4) Since the control unit 30 includes a tag reader that detects the RFID tag information as information for specifying the imaging position, the imaging position by the CCD camera 2 can be obtained accurately when functioning as a detection unit. .

  (5) Since the control unit 30 detects the image as information for specifying the imaging position when the image captured by the CCD camera 2 includes a predetermined image, the image is manually captured, for example. In this case, it is useful when it is determined later whether or not the captured image satisfies the condition of the imaging position included in the association information, or when the CCD camera 2 is used as a video camera.

  (6) The CCD camera 2 captures the field of view of the user 100, and further includes a CCD camera 96 that forms a gaze direction detecting means for detecting the gaze direction of the user 100 together with the control unit 30, and the gaze direction of the gaze direction. According to the detection result, the control unit 30 has a function as an imaging control unit that causes the CCD camera 2 to capture the field of view of the user 100, so that a desired target can be easily and automatically imaged. it can.

  (7) The control unit 30 functions as a display control unit that combines the image captured by the CCD camera 2 and associated with the content information with the image corresponding to the content information and displays the image on the display unit 1. Therefore, the user 100 can easily create a travel electronic album, for example, after the trip is completed, and an inspection report after the inspection is completed.

  (8) Since the display unit 1 is a see-through type that allows the user 100 to visually recognize the content image 200 corresponding to the image light 500 on the outside scene, the display unit 1 can move without any trouble even when the HMD is mounted. Can do. Therefore, there is no problem in traveling or inspecting while wearing the HMD.

  As mentioned above, although this invention has been demonstrated through each embodiment, this invention is not limited to these. Moreover, each effect mentioned above only enumerated the most suitable effect which arises from this invention, and the effect by this invention is not limited to what was described in this Embodiment.

DESCRIPTION OF SYMBOLS 1 Display part 2 CCD camera 3 Control unit 30 Control part 31 Main controller 40 CPU
42 Flash ROM
96 CCD camera 100 User 200 Content image 400 External light 500 Image light

Claims (8)

In a head-mounted display including display means for causing image light according to content information to enter the user's eyes and allowing the user to visually recognize an image according to the image light.
Imaging means;
Storage means for storing the content information;
And a storage control unit for storing data of an image captured by the imaging unit in association with the content information in the storage unit in accordance with association information included in the content information. Condition presence / absence determining means for determining whether or not the association information includes a condition of an imaging position by the imaging means;
Detecting means for detecting information for specifying an imaging position by the imaging means;
Condition satisfaction determination means for determining whether or not an imaging position by the imaging means satisfies a condition of the imaging position based on a detection result by the detection means,
The storage control unit stores data of an image captured when the condition satisfaction determination unit determines that the condition is satisfied in association with the content information in the storage unit. Head mounted display.   3. The head mounted display according to claim 2, wherein the detecting means includes position detecting means for detecting information on latitude and longitude of a current position as information for specifying the imaging position.   The head mounted display according to claim 2, wherein the detection unit includes a tag reader that detects wireless tag information as information for specifying the imaging position.   The head mount according to claim 2, wherein the detection unit detects the image as information for specifying the imaging position when a predetermined image is included in the image captured by the imaging unit. display. The imaging means is for imaging a user's visual field range, and
Gaze direction detecting means for detecting the gaze direction of the user;
The imaging control unit according to claim 1, further comprising: an imaging control unit that causes the imaging unit to capture an image of a visual field range of the user according to a detection result by the line-of-sight direction detection unit. Head mounted display.   7. A display control unit that combines an image captured by the imaging unit and associated with the content information with an image according to the content information and displayed by the display unit. The head mounted display of any one of these.   The head mounted display according to any one of claims 1 to 7, wherein the display means is a see-through type that allows a user to visually recognize an image corresponding to the image light while being superimposed on an outside scene.

Images