JP2018165066A - Head-mounted display device and control method thereof - Google Patents

Abstract

A head-mounted display device capable of controlling the operation of an unmanned aerial vehicle is provided. A head-mounted display device (100) includes a GPS information acquisition unit that acquires GPS information, a communication unit (500) that can communicate with the unmanned aircraft using wireless communication, a control unit (10), Prepare. The control unit controls the field of view visible by the display unit 20 when the unmanned aircraft is navigating in an automatic navigation mode moving toward a position indicated by the GPS information acquired by the GPS information acquisition unit. When it is determined that the unmanned aircraft has entered the field of view, the unmanned aircraft is stopped from flying in the automatic navigation mode, and the automatic Said drone is operated in a particular flight mode different from the flight mode. [Selection drawing] Fig. 10

Description

  The present invention relates to a head-mounted display device and a method for controlling the head-mounted display device.

  In recent years, development of an unmanned aerial vehicle capable of remote control or autonomous flight, called a drone, is in progress. A conventional unmanned aerial vehicle is equipped with a GPS receiving unit that receives GPS radio waves and acquires GPS information, as described in Patent Document 1, for example. An unmanned aerial vehicle can estimate the attitude and position of the aircraft from GPS information. An unmanned aerial vehicle can be operated by other devices such as a smartphone and a tablet in addition to a dedicated remote control device.

JP 2008-304260 A

  In an unmanned aerial vehicle, it is conceivable to automatically navigate to a destination by using GPS information. However, since the position information included in the GPS information has a relatively large error of several meters to several tens of meters, the conventional unmanned aircraft may not be able to navigate accurately to the destination. For example, in mountainous areas, mountainous areas, valleys of buildings, etc., the positional information has a large error, and has reached a point far away from the destination. Therefore, there has been a demand for a technology capable of accurately navigating to a destination in an apparatus for maneuvering an unmanned aerial vehicle. Such a problem is not limited to unmanned aerial vehicles but is a problem common to vehicles (that is, unmanned aircraft) on which various people who are remotely operated or automatically operated do not board.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms.

(1) According to one aspect of the present invention, a head-mounted display device capable of performing control related to the operation of the drone is provided. The head-mounted display device includes a GPS information acquisition unit that acquires GPS information, a communication unit that can perform communication using wireless communication with the drone, and a control unit. The control unit has a field of view that is visible by the display unit when the drone is navigating in an automatic navigation mode that moves toward the position indicated by the GPS information acquired by the GPS information acquisition unit. Determining whether or not the drone has entered, and when it is determined that the drone has entered the field of view, causing the drone to stop traveling in the automatic navigation mode, The drone is caused to navigate in a specific navigation mode different from the navigation mode. According to this form of the head-mounted display device, when the drone enters the field of view visible by the display unit while navigating in the automatic navigation mode, the navigation in the automatic navigation mode is stopped. The drone navigates in a specific navigation mode different from the automatic navigation mode. For this reason, according to the head-mounted display device of this embodiment, navigation in the automatic navigation mode using low-accuracy GPS information can be canceled, so that it is possible to navigate to the destination with high accuracy. .

(2) The head-mounted display device further includes a camera that captures the field of view, and the control unit is prepared in advance to acquire a captured image of the camera and identify the drone. It may be determined whether or not the drone has entered the field of view by performing pattern matching between a target image and the captured image. According to the head-mounted display device of this form, it is possible to determine with high accuracy whether or not the drone has entered the field of view visible by the display unit.

(3) The head-mounted display device further includes a radio wave receiver that receives radio waves transmitted from the drone, and the specific navigation mode is received by the radio wave receiver. The mode may be a mode in which the intensity of the radio wave is measured and a navigation route is determined from the measured change in the intensity. Since the navigation mode based on the intensity of the radio wave enables navigation with high accuracy, according to the display device of this embodiment, navigation to the destination can be performed with higher accuracy.

(4) In the head-mounted display device, the drone includes a first inertial sensor that detects movement of the airframe, and obtains the position of the airframe by sequentially integrating the detection values of the first inertial sensor. The specific navigation mode may be a mode in which the head-mounted display device acquires the position of the aircraft obtained by the unmanned aerial vehicle and determines a navigation route from the acquired position of the aircraft. According to the display device of this embodiment, navigation to the destination can be performed with high accuracy.

(5) The head-mounted display device includes a second inertial sensor that detects movement, and in the specific navigation mode, the head-mounted display device sequentially detects a detection value of the second inertial sensor. The mode may be a mode in which the position of the head-mounted display device is obtained by integrating and a navigation route is determined from the obtained position of the head-mounted display device and the obtained position of the aircraft. According to the head-mounted display device of this aspect, the feedback to the head-mounted display device can be performed with higher accuracy.

(6) In the head-mounted display device, the head-mounted display device further includes an operation unit that is operated by a user and directs the movement of the drone, and the specific navigation mode includes the head-mounted display device, In this mode, a navigation route may be determined according to an instruction to the operation unit. According to the head-mounted display device of this aspect, the feedback to the head-mounted display device can be performed with high accuracy.

(7) In the head-mounted display device, the display unit is a display unit capable of seeing through the outside world, and the control unit displays an operation screen for the operation unit on the display unit, When it is determined that the drone has entered the field of view, the position of the drone on the display unit may be estimated, and the operation screen may be moved to a position away from the estimated position. According to the head-mounted display device of this aspect, since the drone is not hidden by the operation screen in the operator's field of view, the operability of returning the drone can be improved.

(8) In the head-mounted display device, the GPS information acquisition unit may include a GPS receiving unit that receives the GPS information, and the automatic navigation mode may be returned to the head-mounted display device. . According to the head-mounted display device of this aspect, it is possible to perform the return to the head-mounted display device with high accuracy.

(9) In the head-mounted display device, the GPS information acquisition unit may receive the GPS information from a GPS receiving unit provided at a position away from the head-mounted display device. According to the head-mounted display device of this embodiment, when navigating in the automatic navigation mode toward the position where the GPS receiving unit is provided, the navigation is switched to the specific navigation mode and is high up to the destination. It is possible to navigate with accuracy.

  The present invention can also be realized in various forms other than the head-mounted display device. For example, the present invention can be realized by a method for controlling the head-mounted display device, a computer program for realizing the function of each component included in the head-mounted display device, a recording medium on which the computer program is recorded, and the like.

It is explanatory drawing which shows schematic structure of the unmanned aircraft system provided with the head mounted display apparatus in 1st Embodiment of this invention. It is a principal part top view which shows the structure of the optical system with which an image display part is provided. It is a figure which shows the principal part structure of the image display part seen from the user. It is a figure for demonstrating the angle of view of a camera. It is a block diagram which shows the structure of HMD functionally. It is a block diagram which shows the structure of a control apparatus functionally. It is explanatory drawing which shows an example of the augmented reality display by HMD. It is a block diagram which shows the electrical structure of an unmanned aerial vehicle and the electrical structure of a remote control device. It is a flowchart which shows a feedback process. It is a figure for demonstrating the visual field visually recognizable through the image display part by the user. It is a figure for demonstrating Wi-Fi navigation mode. It is a flowchart which shows the feedback process performed with HMD of 2nd Embodiment. It is a flowchart which shows the feedback process performed with HMD of 3rd Embodiment. It is explanatory drawing which shows the operation screen in 3rd Embodiment. It is explanatory drawing which shows the operation screen in 4th Embodiment. It is a principal part top view which shows the structure of the optical system with which the image display part of a modification is provided.

A. First embodiment:
A-1. Overall configuration:
FIG. 1 is an explanatory diagram illustrating a schematic configuration of an unmanned aerial vehicle system including a head-mounted display device according to the first embodiment of the present invention. The unmanned aircraft system includes a head-mounted display device 100, an unmanned aircraft 400, and a remote control device 500.

  The unmanned aerial vehicle 400 is equipped with a GPS receiver, an IMU sensor, and the like, and can fly while grasping the attitude and position of the aircraft. Unmanned aerial vehicle 400 is a so-called drone. Remote control device 500 is a device that remotely controls unmanned aerial vehicle 400. The unmanned aircraft 400 is remotely controlled by the remote control device 500, but autonomous flight is also possible. The head-mounted display device 100 is connected to the remote control device 500 via the USB cable 350. The head-mounted display device 100 can maneuver the unmanned aerial vehicle 400 via the remote control device 500. First, the configuration of the head-mounted display device 100 will be described in detail.

A-2. Configuration of head mounted display device:
The head-mounted display device 100 is a display device that is mounted on a user's head, and is also referred to as a head mounted display (HMD). The HMD 100 is a see-through type (transmission type) head-mounted display device in which an image floats in an external environment visually recognized through a glass. The HMD 100 includes an image display unit 20 that allows a user to visually recognize an image, and a control device (controller) 10 that controls the image display unit 20.

  The image display unit 20 is a wearing body that is worn on the user's head, and has a glasses shape in the present embodiment. The image display unit 20 corresponds to a “display unit”. The image display unit 20 includes a right display unit 22, a left display unit 24, a right light guide plate 26, and a left light guide plate on a support having a right holding unit 21, a left holding unit 23, and a front frame 27. 28.

  Each of the right holding unit 21 and the left holding unit 23 extends rearward from both end portions of the front frame 27 and holds the image display unit 20 on the user's head like a temple of glasses. Here, of both ends of the front frame 27, an end located on the right side of the user in the mounted state of the image display unit 20 is defined as an end ER, and an end located on the left side of the user is defined as an end EL. To do. The right holding unit 21 extends from the end ER of the front frame 27 to a position corresponding to the right side of the user when the image display unit 20 is worn. The left holding part 23 is provided to extend from the end EL of the front frame 27 to a position corresponding to the left side of the user when the image display part 20 is worn.

  The right light guide plate 26 and the left light guide plate 28 are provided on the front frame 27. The right light guide plate 26 is positioned in front of the right eye of the user when the image display unit 20 is mounted, and causes the right eye to visually recognize the image. The left light guide plate 28 is positioned in front of the left eye of the user when the image display unit 20 is mounted, and causes the left eye to visually recognize the image.

  The front frame 27 has a shape in which one end of the right light guide plate 26 and one end of the left light guide plate 28 are connected to each other. This connection position corresponds to the position between the eyebrows of the user when the image display unit 20 is mounted. The front frame 27 may be provided with a nose pad portion that comes into contact with the user's nose when the image display unit 20 is mounted at a connection position between the right light guide plate 26 and the left light guide plate 28. In this case, the image display unit 20 can be held on the user's head by the nose pad, the right holding unit 21 and the left holding unit 23. Further, a belt that is in contact with the back of the user's head when the image display unit 20 is mounted may be connected to the right holding unit 21 and the left holding unit 23. In this case, the image display unit 20 can be firmly held on the user's head by the belt.

  The right display unit 22 displays an image by the right light guide plate 26. The right display unit 22 is provided in the right holding unit 21 and is located in the vicinity of the right side of the user when the image display unit 20 is worn. The left display unit 24 displays an image by the left light guide plate 28. The left display unit 24 is provided in the left holding unit 23 and is located in the vicinity of the user's left head when the image display unit 20 is worn. The right display unit 22 and the left display unit 24 are also collectively referred to as “display driving unit”.

  The right light guide plate 26 and the left light guide plate 28 of the present embodiment are optical units (for example, prisms) formed of a light transmissive resin or the like, and use image light output from the right display unit 22 and the left display unit 24. Lead to the eyes of the person. A dimming plate may be provided on the surfaces of the right light guide plate 26 and the left light guide plate 28. The light control plate is a thin plate-like optical element having different transmittance depending on the wavelength region of light, and functions as a so-called wavelength filter. For example, the light control plate is disposed so as to cover the surface of the front frame 27 (the surface opposite to the surface facing the user's eyes). By appropriately selecting the optical characteristics of the light control plate, the transmittance of light in an arbitrary wavelength region such as visible light, infrared light, and ultraviolet light can be adjusted. The amount of external light incident on the light plate 28 and transmitted through the right light guide plate 26 and the left light guide plate 28 can be adjusted.

  The image display unit 20 guides the image light generated by the right display unit 22 and the left display unit 24 to the right light guide plate 26 and the left light guide plate 28, respectively, and an image (augmented reality (AR) image) is generated by the image light. The user visually recognizes (this is also referred to as “display an image”). When external light enters the user's eyes through the right light guide plate 26 and the left light guide plate 28 from the front of the user, image light constituting the image and external light are incident on the user's eyes. To do. For this reason, the visibility of the image for the user is affected by the intensity of external light.

  For this reason, for example, by attaching a light control plate to the front frame 27 and appropriately selecting or adjusting the optical characteristics of the light control plate, it is possible to adjust the ease of visual recognition of the image. In a typical example, it is possible to select a dimming plate having a light transmittance that allows a user wearing the HMD 100 to visually recognize at least the outside scenery. Use of the light control plate can be expected to protect the right light guide plate 26 and the left light guide plate 28, and to suppress damage to the right light guide plate 26 and the left light guide plate 28, adhesion of dirt, and the like. The light control plate may be detachable from the front frame 27 or each of the right light guide plate 26 and the left light guide plate 28. In addition, a plurality of types of light control plates may be exchanged and removable, or the light control plates may be omitted.

  The camera 61 is disposed on the front frame 27 of the image display unit 20. The camera 61 is provided on the front surface of the front frame 27 at a position that does not block outside light that passes through the right light guide plate 26 and the left light guide plate 28. In the example of FIG. 1, the camera 61 is disposed on the end ER side of the front frame 27. The camera 61 may be disposed on the end EL side of the front frame 27 or may be disposed at a connection portion between the right light guide plate 26 and the left light guide plate 28.

  The camera 61 is a digital camera that includes an imaging element such as a CCD or CMOS, an imaging lens, and the like. The camera 61 of this embodiment is a monocular camera, but a stereo camera may be adopted. The camera 61 images at least a part of the outside world (real space) in the front side direction of the HMD 100, in other words, in the viewing direction that the user visually recognizes when the image display unit 20 is mounted. In other words, the camera 61 images a range or direction that overlaps the user's field of view, and images a direction that the user visually recognizes. The angle of view of the camera 61 can be set as appropriate. In the present embodiment, the width of the angle of view of the camera 61 is set so as to match the entire field of view of the user that can be viewed through the right light guide plate 26 and the left light guide plate 28. The camera 61 performs imaging according to the control of the control function unit 150 (FIG. 6), and outputs the obtained imaging data to the control function unit 150.

  The HMD 100 may include a distance measuring sensor that detects a distance to a measurement object positioned in a preset measurement direction. The distance measuring sensor can be disposed, for example, at a connection portion between the right light guide plate 26 and the left light guide plate 28 of the front frame 27. The measurement direction of the distance measuring sensor can be the front side direction of the MD 100 (the direction overlapping the imaging direction of the camera 61). The distance measuring sensor can be composed of, for example, a light emitting unit such as an LED or a laser diode, and a light receiving unit that receives reflected light that is reflected from the light to be measured by the light source. In this case, the distance is obtained by triangular distance measurement processing or distance measurement processing based on a time difference. The distance measuring sensor may be configured by, for example, a transmission unit that emits ultrasonic waves and a reception unit that receives ultrasonic waves reflected by the measurement object. In this case, the distance is obtained by distance measurement processing based on the time difference. The distance measuring sensor is controlled by the control function unit 150 (FIG. 6) similarly to the camera 61, and outputs the detection result to the control function unit 150.

  FIG. 2 is a principal plan view showing the configuration of the optical system provided in the image display unit 20. For convenience of explanation, FIG. 2 shows a user's right eye RE and left eye LE. As shown in FIG. 2, the right display unit 22 and the left display unit 24 are configured symmetrically.

  As a configuration for allowing the right eye RE to visually recognize an image (AR image), the right display unit 22 includes an OLED (Organic Light Emitting Diode) unit 221 and a right optical system 251. The OLED unit 221 emits image light. The right optical system 251 includes a lens group and the like, and guides the image light L emitted from the OLED unit 221 to the right light guide plate 26.

  The OLED unit 221 includes an OLED panel 223 and an OLED drive circuit 225 that drives the OLED panel 223. The OLED panel 223 is a self-luminous display panel configured by light emitting elements that emit light by organic electroluminescence and emit color lights of R (red), G (green), and B (blue). In the OLED panel 223, a plurality of pixels each having a unit including one R, G, and B element as one pixel are arranged in a matrix.

  The OLED drive circuit 225 performs selection and energization of the light emitting elements included in the OLED panel 223 under the control of the control function unit 150 (FIG. 6), and causes the light emitting elements to emit light. The OLED drive circuit 225 is fixed to the back surface of the OLED panel 223, that is, the back side of the light emitting surface by bonding or the like. The OLED drive circuit 225 may be configured by a semiconductor device that drives the OLED panel 223, for example, and may be mounted on a substrate that is fixed to the back surface of the OLED panel 223. A temperature sensor 217 (FIG. 5) described later is mounted on this board. Note that the OLED panel 223 may employ a configuration in which light emitting elements that emit white light are arranged in a matrix, and color filters corresponding to the colors R, G, and B are stacked. Further, an OLED panel 223 having a WRGB configuration including a light emitting element that emits W (white) light in addition to the light emitting elements that respectively emit R, G, and B color light may be employed.

  The right optical system 251 includes a collimator lens that converts the image light L emitted from the OLED panel 223 into a light beam in a parallel state. The image light L converted into a parallel light beam by the collimator lens enters the right light guide plate 26. A plurality of reflecting surfaces that reflect the image light L are formed in an optical path that guides light inside the right light guide plate 26. The image light L is guided to the right eye RE side through a plurality of reflections inside the right light guide plate 26. A half mirror 261 (reflection surface) located in front of the right eye RE is formed on the right light guide plate 26. The image light L is reflected by the half mirror 261 and then emitted from the right light guide plate 26 to the right eye RE. The image light L forms an image with the retina of the right eye RE, thereby allowing the user to visually recognize the image.

  As a configuration for causing the left eye LE to visually recognize an image (AR image), the left display unit 24 includes an OLED unit 241 and a left optical system 252. The OLED unit 241 emits image light. The left optical system 252 includes a lens group and the like, and guides the image light L emitted from the OLED unit 241 to the left light guide plate 28. The OLED unit 241 includes an OLED panel 243 and an OLED drive circuit 245 that drives the OLED panel 243. Details of each part are the same as those of the OLED unit 221, the OLED panel 223, and the OLED drive circuit 225. A temperature sensor 239 (FIG. 5) is mounted on the substrate fixed to the back surface of the OLED panel 243. The details of the left optical system 252 are the same as those of the right optical system 251.

  According to the configuration described above, the HMD 100 can function as a see-through display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 enter the right eye RE of the user. The image light L reflected by the half mirror 281 and the external light OL transmitted through the left light guide plate 28 enter the left eye LE of the user. As described above, the HMD 100 causes the image light L of the image processed inside and the external light OL to overlap and enter the user's eyes. As a result, for the user, the outside world (real world) can be seen through the right light guide plate 26 and the left light guide plate 28, and an image (AR image) by the image light L is visually recognized so as to overlap the outside world.

  The half mirror 261 and the half mirror 281 function as an “image extraction unit” that reflects image light output from the right display unit 22 and the left display unit 24 and extracts an image. Further, the right optical system 251 and the right light guide plate 26 are collectively referred to as a “right light guide”, and the left optical system 252 and the left light guide plate 28 are also collectively referred to as a “left light guide”. The configurations of the right light guide unit and the left light guide unit are not limited to the above-described examples, and any method can be used as long as an image is formed in front of the user's eyes using image light. For example, a diffraction grating or a transflective film may be used for the right light guide and the left light guide.

  In FIG. 1, the control device 10 and the image display unit 20 are connected by a connection cable 40. The connection cable 40 is detachably connected to a connector provided at the lower part of the control device 10, and is connected to various circuits inside the image display unit 20 from the tip AL of the left holding unit 23. The connection cable 40 includes a metal cable or an optical fiber cable that transmits digital data. The connection cable 40 may further include a metal cable that transmits analog data. A connector 46 is provided in the middle of the connection cable 40.

  The connector 46 is a jack for connecting a stereo mini-plug, and the connector 46 and the control device 10 are connected by a line for transmitting an analog audio signal, for example. In the example of the present embodiment shown in FIG. 1, a right earphone 32 and a left earphone 34 constituting stereo headphones and a headset 30 having a microphone 63 are connected to the connector 46.

  For example, as shown in FIG. 1, the microphone 63 is arranged so that the sound collection unit of the microphone 63 faces the line of sight of the user. The microphone 63 collects sound and outputs a sound signal to the sound interface 182 (FIG. 5). The microphone 63 may be a monaural microphone or a stereo microphone, and may be a directional microphone or an omnidirectional microphone.

  The control device 10 is a device for controlling the HMD 100 (particularly the image display unit 20). The control device 10 includes a lighting unit 12, a touch pad 14, a direction key 16, a determination key 17, and a power switch 18. The lighting unit 12 notifies the operation state of the HMD 100 (for example, ON / OFF of the power supply) by its light emission mode. For example, an LED (Light Emitting Diode) can be used as the lighting unit 12.

  The touch pad 14 detects a contact operation on the operation surface of the touch pad 14 and outputs a signal corresponding to the detected content. As the touch pad 14, various touch pads such as an electrostatic type, a pressure detection type, and an optical type can be adopted. The direction key 16 detects a pressing operation on a key corresponding to the up / down / left / right direction, and outputs a signal corresponding to the detected content. The determination key 17 detects a pressing operation and outputs a signal for determining the content operated in the control device 10. The power switch 18 switches the power state of the HMD 100 by detecting a slide operation of the switch.

  FIG. 3 is a diagram illustrating a main configuration of the image display unit 20 as viewed from the user. In FIG. 3, the connection cable 40, the right earphone 32, and the left earphone 34 are not shown. In the state of FIG. 3, the back sides of the right light guide plate 26 and the left light guide plate 28 can be visually recognized, the half mirror 261 for irradiating the right eye RE with image light, and the image light for irradiating the left eye LE with image light. The half mirror 281 can be visually recognized as a substantially rectangular area. The user sees the outside through the entire left and right light guide plates 26 and 28 including the half mirrors 261 and 281, and visually recognizes a rectangular display image at the position of the half mirrors 261 and 281.

  FIG. 4 is a diagram for explaining the angle of view of the camera 61. In FIG. 4, the camera 61 and the right eye RE and left eye LE of the user are schematically shown in plan view, and the angle of view (imaging range) of the camera 61 is indicated by λ. In addition, the angle of view λ of the camera 61 extends in the horizontal direction as shown in the figure, and also extends in the vertical direction in the same manner as a general digital camera.

  As described above, the camera 61 is arranged at the right end of the image display unit 20 and images the direction of the user's line of sight (that is, the front of the user). For this reason, the optical axis of the camera 61 is a direction including the line-of-sight directions of the right eye RE and the left eye LE. The outside world that can be visually recognized by the user wearing the HMD 100 is not limited to infinity. For example, when the user gazes at the object OB with both eyes, the user's line of sight is directed toward the object OB as indicated by reference numerals RD and LD in the figure. In this case, the distance from the user to the object OB is often about 30 cm to 10 m, and more often 1 m to 4 m. Therefore, for the HMD 100, an upper limit and a lower limit of the distance from the user to the object OB during normal use may be determined. This standard may be obtained in advance and preset in the HMD 100, or may be set by the user. The optical axis and the angle of view of the camera 61 are set so that the object OB is included in the angle of view when the distance to the object OB during normal use corresponds to the set upper and lower limits. It is preferred that

  In general, a human viewing angle is approximately 200 degrees in the horizontal direction and approximately 125 degrees in the vertical direction. Among them, the effective visual field with excellent information receiving ability is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction. A stable gaze field in which a gaze point that a person gazes at appears quickly and stably is set to 60 to 90 degrees in the horizontal direction and about 45 to 70 degrees in the vertical direction. In this case, when the gazing point is the object OB (FIG. 4), the effective visual field is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction around the lines of sight RD and LD. Further, the stable focus is about 60 to 90 degrees in the horizontal direction and about 45 to 70 degrees in the vertical direction. The actual field of view that the user permeates through the image display unit 20 and permeates through the right light guide plate 26 and the left light guide plate 28 is referred to as a real field of view (FOV). The real field of view is narrower than the viewing angle and stable focus field, but wider than the effective field of view.

  The angle of view λ of the camera 61 of the present embodiment is set so as to be able to image a range that matches the user's field of view. The angle of view λ of the camera 61 may be configured to be able to image at least a range wider than the effective visual field of the user. It is good also as a structure set so that imaging of the range wider than a real visual field is possible. The angle of view λ of the camera 61 may be set so as to be able to image a range wider than the user's stable visual field, or as a configuration set to be able to image a range that matches the viewing angle of both eyes of the user. Also good. For this reason, the camera 61 may include a so-called wide-angle lens as an imaging lens so that a wide angle of view can be captured. The wide-angle lens may include a lens called an ultra-wide-angle lens or a quasi-wide-angle lens. The camera 61 may include a single focus lens, a zoom lens, or a lens group including a plurality of lenses.

  FIG. 5 is a block diagram showing an electrical configuration of the HMD 100. The control device 10 includes a main processor 140 that controls the HMD 100 by executing a program, a storage unit, an input / output unit, sensors, an interface, and a power supply unit 130. The storage unit, input / output unit, sensors, interface, and power supply unit 130 are connected to the main processor 140. The main processor 140 is mounted on the controller board 120 built in the control device 10.

  The storage unit includes a memory 118 and a nonvolatile storage unit 121. The memory 118 constitutes a work area for temporarily storing a computer program executed by the main processor 140 and data to be processed. The nonvolatile storage unit 121 is configured by a flash memory or an eMMC (embedded Multi Media Card). The nonvolatile storage unit 121 stores a computer program executed by the main processor 140 and various data processed by the main processor 140. In the present embodiment, these storage units are mounted on the controller board 120.

  The input / output unit includes a touch pad 14 and an operation unit 110. The operation unit 110 includes a direction key 16, a determination key 17, and a power switch 18 provided in the control device 10. The main processor 140 controls these input / output units and acquires signals output from the input / output units.

  The sensors include a six-axis sensor 111, a magnetic sensor 113, and a GPS (Global Positioning System) receiver 115. The 6-axis sensor 111 is a motion sensor (inertial sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 111 may employ an IMU (Inertial Measurement Unit) in which these sensors are modularized. The magnetic sensor 113 is, for example, a triaxial geomagnetic sensor.

  The GPS receiver 115 receives a radio wave transmitted from a GPS satellite using a GPS antenna (not shown), and acquires GPS information by analyzing the received radio wave using a GPS circuit (not shown). The GPS information includes position information including latitude, longitude, and altitude, and shooting time and date information.

  These sensors (six-axis sensor 111, magnetic sensor 113, GPS receiver 115) output detection values to the main processor 140 according to a sampling frequency designated in advance. The timing at which each sensor outputs a detection value may be in accordance with an instruction from the main processor 140.

  The interface includes a wireless communication unit 117, an audio codec 180, an external connector 184, an external memory interface 186, a USB (Universal Serial Bus) connector 188, a sensor hub 192, an FPGA 194, and an interface 196. ing. These function as an interface with the outside. The wireless communication unit 117 performs wireless communication between the HMD 100 and an external device. The wireless communication unit 117 includes an antenna, an RF circuit, a baseband circuit, a communication control circuit, and the like (not shown), or is configured as a device in which these are integrated. The wireless communication unit 117 performs wireless communication complying with a standard such as a wireless LAN including Bluetooth (registered trademark) and Wi-Fi (registered trademark), for example.

  The audio codec 180 is connected to the audio interface 182 and encodes / decodes an audio signal input / output via the audio interface 182. The audio interface 182 is an interface for inputting and outputting audio signals. The audio codec 180 may include an A / D converter that converts analog audio signals to digital audio data, and a D / A converter that performs the reverse conversion. The HMD 100 of the present embodiment outputs audio from the right earphone 32 (FIG. 1) and the left earphone 34 and collects sound by the microphone 63. The audio codec 180 converts the digital audio data output from the main processor 140 into an analog audio signal, and outputs the analog audio signal via the audio interface 182. The audio codec 180 converts an analog audio signal input to the audio interface 182 into digital audio data and outputs the digital audio data to the main processor 140.

  The external connector 184 is a connector for connecting an external device (for example, a personal computer, a smart phone, a game device, etc.) that communicates with the main processor 140 to the main processor 140. The external device connected to the external connector 184 can be a content supply source, and can be used for debugging a computer program executed by the main processor 140 and collecting an operation log of the HMD 100. The external connector 184 can employ various modes. As the external connector 184, for example, an interface corresponding to a wired connection such as a USB interface, a micro USB interface, a memory card interface, or an interface corresponding to a wireless connection such as a wireless LAN interface or a Bluetooth interface can be adopted.

  The external memory interface 186 is an interface to which a portable memory device can be connected. The external memory interface 186 includes, for example, a memory card slot in which a card-type recording medium is mounted and data is read and written, and an interface circuit. The size, shape, standard, etc. of the card type recording medium can be selected as appropriate. The USB connector 188 is an interface that can connect a memory device, a smartphone, a personal computer, or the like that conforms to the USB standard.

  The USB connector 188 includes, for example, a connector conforming to the USB standard and an interface circuit. The size, shape, USB standard version, etc. of the USB connector 188 can be selected as appropriate.

  The sensor hub 192 and the FPGA 194 are connected to the image display unit 20 via an interface (I / F) 196. The sensor hub 192 acquires detection values of various sensors included in the image display unit 20 and outputs them to the main processor 140. The FPGA 194 executes processing of data transmitted and received between the main processor 140 and each unit of the image display unit 20 and transmission via the interface 196. The interface 196 is connected to each of the right display unit 22 and the left display unit 24 of the image display unit 20. In the example of the present embodiment, the connection cable 40 (FIG. 1) is connected to the left holding unit 23, and the wiring connected to the connection cable 40 is laid inside the image display unit 20, and the right display unit 22 and the left display unit 24 Are connected to the interface 196 of the control device 10.

  Further, the HMD 100 includes a vibrator 19. The vibrator 19 includes a motor (not shown), an eccentric rotor, and the like, and generates vibrations under the control of the main processor 140. The HMD 100 causes the vibrator 19 to generate a vibration with a predetermined vibration pattern when, for example, an operation on the operation unit 110 is detected or when the power of the HMD 100 is turned on / off.

  The power supply unit 130 includes a battery 132 and a power supply control circuit 134. The power supply unit 130 supplies power for operating the control device 10. The battery 132 is a rechargeable battery. The power control circuit 134 detects the remaining capacity of the battery 132 and controls the charging of the OS 143. The power supply control circuit 134 is connected to the main processor 140 and outputs a detected value of the remaining capacity of the battery 132 and a detected value of the voltage of the battery 132 to the main processor 140. Note that power may be supplied from the control device 10 to the image display unit 20 based on the power supplied by the power supply unit 130. The power supply state from the power supply unit 130 to each unit of the control device 10 and the image display unit 20 may be configured to be controllable by the main processor 140.

  The right display unit 22 includes a display unit substrate 210, an OLED unit 221, a camera 61, an illuminance sensor 65, an LED indicator 67, and a temperature sensor 217. An interface (I / F) 211 connected to the interface 196, a receiving unit (Rx) 213, and an EEPROM (Electrically Erasable Programmable Read-Only Memory) 215 are mounted on the display unit substrate 210. The receiving unit 213 receives data input from the control device 10 via the interface 211. When receiving image data of an image to be displayed on the OLED unit 221, the receiving unit 213 outputs the received image data to the OLED drive circuit 225 (FIG. 2).

  The EEPROM 215 stores various data in a form that can be read by the main processor 140. The EEPROM 215 stores, for example, data on the light emission characteristics and display characteristics of the OLED units 221 and 241 of the image display unit 20, data on the sensor characteristics of the right display unit 22 or the left display unit 24, and the like. Specifically, for example, parameters relating to gamma correction of the OLED units 221 and 241 and data for compensating detection values of temperature sensors 217 and 239 described later are stored. These data are generated by the factory inspection of the HMD 100 and are written in the EEPROM 215. After shipment, the main processor 140 reads the data in the EEPROM 215 and uses it for various processes.

  The camera 61 executes imaging in accordance with a signal input via the interface 211 and outputs captured image data or a signal representing the imaging result to the control device 10. As shown in FIG. 1, the illuminance sensor 65 is provided at an end ER of the front frame 27 and is arranged to receive external light from the front of the user wearing the image display unit 20. The illuminance sensor 65 outputs a detection value corresponding to the amount of received light (received light intensity). As shown in FIG. 1, the LED indicator 67 is disposed near the camera 61 at the end ER of the front frame 27. The LED indicator 67 is lit during execution of imaging by the camera 61 to notify that imaging is in progress.

  The temperature sensor 217 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature. The temperature sensor 217 is mounted on the back side of the OLED panel 223 (FIG. 3). For example, the temperature sensor 217 may be mounted on the same substrate as the OLED drive circuit 225. With this configuration, the temperature sensor 217 mainly detects the temperature of the OLED panel 223. The temperature sensor 217 may be incorporated in the OLED panel 223 or the OLED drive circuit 225. For example, when the OLED panel 223 is mounted as an Si-OLED as an integrated circuit on an integrated semiconductor chip together with the OLED drive circuit 225, the temperature sensor 217 may be mounted on the semiconductor chip.

  The left display unit 24 includes a display unit substrate 230, an OLED unit 241, and a temperature sensor 239. On the display unit substrate 230, an interface (I / F) 231 connected to the interface 196, a receiving unit (Rx) 233, a six-axis sensor 235, and a magnetic sensor 237 are mounted. The receiving unit 233 receives data input from the control device 10 via the interface 231. When receiving image data of an image to be displayed on the OLED unit 241, the receiving unit 233 outputs the received image data to the OLED drive circuit 245 (FIG. 2).

  The 6-axis sensor 235 is a motion sensor (inertia sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 235 may employ an IMU in which the above sensors are modularized. The magnetic sensor 237 is, for example, a triaxial geomagnetic sensor. Since the 6-axis sensor 235 and the magnetic sensor 237 are provided in the image display unit 20, when the image display unit 20 is mounted on the user's head, the movement of the user's head is detected. The direction of the image display unit 20, that is, the field of view of the user is specified from the detected head movement.

  The temperature sensor 239 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature. The temperature sensor 239 is mounted on the back side of the OLED panel 243 (FIG. 3). The temperature sensor 239 may be mounted on the same substrate as the OLED drive circuit 245, for example. With this configuration, the temperature sensor 239 mainly detects the temperature of the OLED panel 243. The temperature sensor 239 may be incorporated in the OLED panel 243 or the OLED drive circuit 245. Details are the same as those of the temperature sensor 217.

  The camera 61, the illuminance sensor 65, and the temperature sensor 217 of the right display unit 22, and the 6-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 of the left display unit 24 are connected to the sensor hub 192 of the control device 10. The sensor hub 192 sets and initializes the sampling period of each sensor according to the control of the main processor 140. The sensor hub 192 executes energization to each sensor, transmission of control data, acquisition of a detection value, and the like in accordance with the sampling period of each sensor. The sensor hub 192 outputs detection values of the sensors included in the right display unit 22 and the left display unit 24 to the main processor 140 at a preset timing. The sensor hub 192 may have a cache function that temporarily holds the detection value of each sensor. The sensor hub 192 may be provided with a conversion function (for example, a conversion function to a unified format) of a signal format or a data format of a detection value of each sensor.

  The FPGA 194 turns on or off the LED indicator 67 by starting and stopping energization of the LED indicator 67 according to the control of the main processor 140.

  FIG. 6 is a block diagram functionally showing the configuration of the control device 10. Functionally, the control device 10 includes a storage function unit 122 and a control function unit 150. The storage function unit 122 is a logical storage unit configured by the nonvolatile storage unit 121 (FIG. 5). The storage function unit 122 may be configured to use the EEPROM 215 or the memory 118 in combination with the nonvolatile storage unit 121 instead of the configuration using only the storage function unit 122. The control function unit 150 is configured when the main processor 140 executes a computer program, that is, when hardware and software cooperate.

  The storage function unit 122 stores various data used for processing in the control function unit 150. Specifically, the storage function unit 122 of the present embodiment stores setting data 123 and content data 124. The setting data 123 includes various setting values related to the operation of the HMD 100. For example, the setting data 123 includes parameters, determinants, arithmetic expressions, LUTs (Look Up Tables), and the like when the control function unit 150 controls the HMD 100.

  The content data 124 includes content data (image data, video data, audio data, etc.) including images and video displayed by the image display unit 20 under the control of the control function unit 150. The content data 124 may include interactive content data. Bi-directional content refers to a user's operation acquired by the operation unit 110, processing corresponding to the acquired operation content is executed by the control function unit 150, and content corresponding to the processing content is stored in the image display unit 20. Means the type of content to display. In this case, the content data may include image data of a menu screen for acquiring a user's operation, data defining a process corresponding to an item included in the menu screen, and the like. Video data is video data indicating a video.

  The control function unit 150 executes various processes using the data stored in the storage function unit 122, whereby the OS 143, the image processing unit 145, the display control unit 147, the imaging control unit 149, and the input / output control unit 151. The functions as the communication control unit 153 and the unmanned aircraft control unit 155 are executed. In the present embodiment, each functional unit other than the OS 143 is configured as a computer program executed on the OS 143.

  The image processing unit 145 generates a signal to be transmitted to the right display unit 22 and the left display unit 24 based on image / video image data displayed by the image display unit 20. The signal generated by the image processing unit 145 may be a vertical synchronization signal, a horizontal synchronization signal, a clock signal, an analog image signal, or the like. The image processing unit 145 may be configured by hardware (for example, DSP (Digital Signal Processor)) different from the main processor 140 in addition to the configuration realized by the main processor 140 executing a computer program.

  Note that the image processing unit 145 may execute resolution conversion processing, image adjustment processing, 2D / 3D conversion processing, and the like as necessary. The resolution conversion process is a process for converting the resolution of the image data into a resolution suitable for the right display unit 22 and the left display unit 24. The image adjustment processing includes processing for adjusting the brightness and saturation of image data, gamma correction, and the like. The 2D / 3D conversion process is a process of generating 2D image data from 3D image data or generating 3D image data from 2D image data. When these processes are executed, the image processing unit 145 generates a signal for displaying an image based on the processed image data, and transmits the signal to the image display unit 20 via the connection cable 40.

  The display control unit 147 generates a control signal for controlling the right display unit 22 and the left display unit 24, and controls the generation and emission of image light by each of the right display unit 22 and the left display unit 24 based on the control signal. To do. Specifically, the display control unit 147 controls the OLED drive circuits 225 and 245 to cause the OLED panels 223 and 243 to display an image. The display control unit 147 controls the timing at which the OLED drive circuits 225 and 245 draw on the OLED panels 223 and 243 and controls the luminance of the OLED panels 223 and 243 based on the signal output from the image processing unit 145.

  The imaging control unit 149 controls the camera 61 to execute imaging, generates captured image data, and temporarily stores it in the storage function unit 122. When the camera 61 is configured as a camera unit including a circuit that generates captured image data, the imaging control unit 149 acquires captured image data from the camera 61 and temporarily stores the captured image data in the storage function unit 122.

  The input / output control unit 151 appropriately controls the touch pad 14 (FIG. 1), the direction key 16, and the determination key 17, and acquires an input command therefrom. The acquired command is output to the OS 143 or a computer program operating on the OS 143 together with the OS 143. The OS 143 or the computer program operating on the OS 143 moves the cursor displayed on the screen of the image display unit 20 based on these input commands. The communication control unit 153 controls the wireless communication unit 117 to perform wireless communication with an external device.

  The unmanned aerial vehicle control unit 155 can control the unmanned aerial vehicle 400 from the HMD 100 side by directly or indirectly transmitting and receiving signals to and from the unmanned aircraft 400. Details of the unmanned aircraft control unit 155 will be described later.

  FIG. 7 is an explanatory diagram illustrating an example of augmented reality display by the HMD 100. FIG. 7 illustrates the user's field of view VT. As described above, the image light guided to both eyes of the user of the HMD 100 forms an image on the retina of the user, so that the user visually recognizes the image AI as augmented reality (AR). In the example of FIG. 7, the image AI is a menu screen of the OS of the HMD 100. The menu screen includes, for example, icon ICs for starting application programs “message”, “phone”, “camera”, “browser”, and “unmanned aerial vehicle”. Further, the right and left light guide plates 26 and 28 transmit light from the outside world SC, so that the user visually recognizes the outside world SC. As described above, the user of the HMD according to the present embodiment can view the image AI so that the portion of the field of view VT where the image VI is displayed overlaps the outside SC. Further, only the outside SC can be seen in the portion of the field of view VT where the image AI is not displayed.

A-3. Configuration of unmanned aerial vehicle and remote control device:
As shown in FIG. 1, the unmanned aerial vehicle 400 includes four propellers 410 (two are not shown) and a camera 420. The unmanned aerial vehicle 400 can fly freely and stably in the air by the four propellers 410, and the ground can be taken from the sky by the camera 420.

  FIG. 8 is a block diagram showing an electrical configuration of unmanned aerial vehicle 400 and an electrical configuration of remote control device 500. The unmanned aerial vehicle 400 includes four motors 430, four ESCs (Electronic Speed Controllers) 435, a GPS receiver 440, a six-axis sensor 450, a wireless communication unit 460, a Wi-Fi communication unit 470, and a battery. 480 and a control unit 490.

  Each motor 430 is connected to each propeller 410 and drives each propeller 410. Each ESC is connected to each motor 430 and controls the number of revolutions of each motor 430 according to an input signal.

  The GPS receiver 440 receives radio waves transmitted from GPS satellites using a GPS antenna (not shown), and acquires GPS information by analyzing the received radio waves using a GPS circuit (not shown). The GPS information includes position information including latitude, longitude, and altitude, and shooting time and date information. From the GPS information, the unmanned aerial vehicle 400 can know the current position (latitude, longitude, altitude) of the aircraft (= unmanned aircraft 400).

  The 6-axis sensor 450 is a motion sensor (inertial sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 450 may employ an IMU (Inertial Measurement Unit) in which these sensors are modularized. The 6-axis sensor 450 allows the unmanned aircraft 400 to know the attitude and speed of the aircraft.

  Radio communication unit 460 performs radio communication with radio communication unit 540 of remote control device 500. The Wi-Fi communication unit 470 performs wireless communication conforming to a wireless LAN standard including Wi-Fi (registered trademark). In the present embodiment, in a Wi-Fi navigation mode, which will be described later, the Wi-Fi communication unit 470 exchanges Wi-Fi radio waves with the wireless communication unit 117 provided in the control device 10 of the HMD 100.

  The battery 480 is a power source for the entire unmanned aircraft 400, and a lithium polymer battery is employed in this embodiment.

  To the control unit 490, a camera 420, each ESC 435, a GPS receiver 440, a 6-axis sensor 450, a wireless communication unit 460, a Wi-Fi communication unit 470, and a battery 480 are connected. The control unit 490 is configured by a microcomputer including a main processor and a storage unit, and executes various functions by reading and executing programs in the storage unit. Specifically, control unit 490 performs wireless communication with remote control device 500 via wireless communication unit 460, and individually controls the rotation speed of each motor 430 in accordance with a control command from remote control device 500. As a result, the control unit 490 can raise, lower, advance, reverse, move left, move right, and hover according to a command from the remote control device 500. In addition, the control unit 490 can perform shooting by the camera 420 in accordance with a command from the remote control device 500, and displays a captured image on the display 510.

  Control unit 490 can further move the aircraft in automatic navigation mode M1. The automatic navigation mode M1 is a mode for automatically navigating the aircraft to the destination by setting position information indicating the destination. Since the unmanned aircraft 400 can know the current position of the aircraft by the GPS receiver 440 as described above, the control unit 490 always determines the direction and distance from the current position to the destination position in the automatic navigation mode M1. By calculating and controlling the rotation speed of each motor 430 according to the direction and distance, the aircraft can be automatically moved to the destination.

  As one of the functions using the automatic navigation mode M1, for example, when communication between the unmanned aircraft 400 and the remote control device 500 is interrupted, there is a function of returning the unmanned aircraft 400 to the takeoff point. In the automatic navigation mode M1, the control unit 490 can return the aircraft to the takeoff point by setting the position information included in the GPS information of the takeoff point stored in advance as the destination.

  As shown in FIG. 1, the remote control device 500 includes a display 510 and a joystick type operation unit 520. As shown in FIG. 8, the remote control device 500 includes a GPS receiver 530, a wireless communication unit 540, a USB connector 550, and a control unit 590.

  The GPS receiver 530 receives radio waves transmitted from GPS satellites using a GPS antenna (not shown), and acquires GPS information by analyzing the received radio waves using a GPS circuit (not shown). The GPS information includes position information including latitude, longitude, and altitude, and shooting time and date information. From the GPS information, the remote controller 500 can know the current position (latitude, longitude, altitude) of the remote controller 500.

  The wireless communication unit 540 performs wireless communication with the wireless communication unit 460 of the unmanned aircraft 400. The USB connector 550 includes, for example, a connector conforming to the USB standard and an interface circuit. The USB connector 550 is an interface that can connect a memory device, a smartphone, a personal computer, or the like that conforms to the USB standard. In the present embodiment, the USB connector 550 is connected to the USB connector 188 provided in the control device 10 of the HMD 100 via the USB cable 350. The USB connector 188 provided in the control device 10 corresponds to “a communication unit capable of performing communication using wireless communication with the unmanned aircraft 400”. Note that the connection between the remote control device 500 and the control device 10 is not limited to the USB connection, but may be another wired connection. Furthermore, the configuration may be such that the remote control device 500 and the control device 10 are connected wirelessly instead of wired.

  A display 510, an operation unit 520, a GPS receiver 530, a wireless communication unit 540, and a USB connector 550 are connected to the control unit 590, respectively. The control unit 590 is configured by a microcomputer including a main processor and a storage unit, and executes various functions by reading and executing programs in the storage unit. Specifically, the control unit 590 transmits an operation command from the operation unit 520 to the unmanned aircraft 400, thereby causing the unmanned aircraft 400 to rise, descend, advance, reverse, left move, right move, and hover. The flight form is commanded. Control unit 590 instructs unmanned aircraft 400 to take a picture with camera 420.

A-4. Return processing by HMD:
As described above with reference to FIG. 6, the control device 10 provided in the HMD 100 includes the unmanned aircraft control unit 155 that controls the unmanned aircraft 400 as one of the control function units 150. The unmanned aerial vehicle control unit 155 displays an operation screen (for example, the operation screen CB in FIG. 14) obtained by implementing the operation unit 520 of the remote control device 500 in software on the image display unit 20, and this operation screen is displayed by the user of the HMD 100. By transmitting the operation command input to the remote control device 500, the unmanned aircraft 400 is operated from the HMD 100 side using the functions of the remote control device 500. Unmanned aerial vehicle control unit 155 may have all the functions of remote control device 500 or only a part of all the functions of remote control device 500. The unmanned aerial vehicle control unit 155 transmits a shooting command to the remote control device 500 when receiving the shooting command from the operation screen, thereby using the functions of the remote control device 500 to perform the unmanned aircraft from the HMD 100 side. 400 is instructed to shoot by the camera 420.

  In addition, a software button called “return to HMD” is prepared on the operation screen in which the operation unit 520 is realized as software. When this software button is operated by the direction key 16 (FIG. 1) and the enter key 17 (FIG. 1), the unmanned aerial vehicle control unit 155 executes a feedback process for returning the unmanned aircraft 400 to the HMD 100 ( Feedback processor 155a).

  FIG. 9 is a flowchart showing the feedback process. This feedback processing is processing by the feedback processing unit 155a shown in FIG. 6 and is executed by the main processor 140 of the HMD 100. In the application program of the “unmanned aircraft” that is activated by the direction key 16 (FIG. 1) and the decision key 17 (FIG. 1) indicated by the icon IC of “unmanned aircraft” on the menu screen illustrated in FIG. When the “Return to HMD” software button is operated, the feedback process is started.

  When the process is started, the main processor 140 of the HMD 100 first acquires GPS information from the GPS receiver 115 (step S110). The processing of the main processor 140 and step S110 corresponds to a “GPS information acquisition unit”.

  Next, the main processor 140 instructs the unmanned aircraft 400 to navigate in the automatic navigation mode M1 (FIG. 8) in which the position information (latitude, longitude, altitude) included in the acquired GPS information is set as the destination. (Step S120). This instruction is given to the unmanned aircraft 400 via the remote control device 500 by being given to the remote control device 500. That is, in step S120, an instruction to return the unmanned aircraft 400 to the current position of the control device 10 of the HMD 100 is given to the unmanned aircraft 400 via the remote control device 500.

  Subsequently, the main processor 140 activates the camera 61 to photograph the outside world in the user's line of sight (step S130). The photographed external world corresponds to a visual field that can be visually recognized through the image display unit 20 by the user.

  Subsequently, the main processor 140 determines whether or not the unmanned aircraft 400 is included in the image captured by the camera 61 (step S140). This determination is performed when an image showing the shape characteristics of the unmanned aircraft 400 is prepared as a target image, and an image portion having a high degree of coincidence with the target image is detected from the captured images by pattern matching. It is determined that the unmanned aircraft 400 is included in the obtained image.

  It should be noted that a marker may be pasted on the unmanned aerial vehicle 400 in advance and the marker may be used as the target image instead of the configuration in which the image showing the shape characteristics of the unmanned aircraft 400 is the target image. The marker is a two-dimensional marker and serves as a mark for designating a target image for pattern matching.

  FIG. 10 is a diagram for explaining a field of view that can be visually recognized by the user through the image display unit 20. In the figure, for example, a range indicated by a two-dot chain line is a visual field WA that can be visually recognized through the image display unit 20. By the determination process in step S140, it is determined whether or not the unmanned aircraft 400 has entered the field of view WA.

  Returning to FIG. 9, when it is determined in step S140 that the unmanned aircraft 400 is not included in the captured image, that is, when it is determined that the unmanned aircraft 400 has not entered the field of view WA, The main processor 140 determines whether or not the unmanned aircraft 400 has arrived at the destination set in step S120 (step S150). Whether or not the vehicle has arrived at the destination is known by the remote control device 500 communicating with the unmanned aircraft 400, and the remote control device 500 is inquired to make a determination in step S150.

  If it is determined in step S150 that the destination has not arrived, that is, if it is determined that the unmanned aircraft 400 has not entered the field of view WA and has not arrived at the destination, the main The processor 140 returns the process to step S110 to continue the return in the automatic navigation mode M1. As illustrated in FIG. 10, the unmanned aircraft 400 navigates in the automatic navigation mode M1 when flying outside the field of view WA.

  The main processor 140 repeatedly executes the processing from step S110 to step S140, and when it is determined in step S140 that the unmanned aircraft 400 is included in the photographed image, that is, the unmanned in the field of view WA. When it is determined that the aircraft 400 has entered, the unmanned aircraft 400 is stopped from traveling in the automatic navigation mode and instructed to travel in the Wi-Fi navigation mode (step S160).

  FIG. 11 is a diagram for explaining the Wi-Fi navigation mode. In the Wi-Fi navigation mode M2 (see FIG. 6), the Wi-Fi radio waves B1 and B2 are exchanged between the control device 10 of the HMD 100 and the unmanned aircraft 400. Define 400 navigation routes. Specifically, the navigation route is determined in a direction in which the Wi-Fi radio wave intensity increases. The main processor 140 measures the Wi-Fi radio wave intensity from the unmanned aerial vehicle 400 in the Wi-Fi navigation mode M2, estimates the direction in which the radio wave intensity increases from the change in the radio wave intensity, and the radio wave intensity increases. The direction is a navigation route, and the flight form is instructed to the unmanned aircraft 400 via the remote control device 500 so as to fly along this navigation route. The flight form is the movement of the aircraft such as ascending, descending, moving forward, moving backward, moving left, moving right, and hovering.

  Instead of the configuration in which the unmanned aircraft 400 is instructed to fly through the remote control device 500, the control device 10 may directly instruct the unmanned aircraft 400 from the wireless communication unit 117. Further, the process of estimating the direction in which the radio field strength is increased and obtaining the direction in which the radio field intensity is increased as the navigation route is not performed on the control device 10 side, but performed by the control unit 490 of the unmanned aircraft 400, The control unit 490 may be configured to control the flight form of the unmanned aircraft 400 according to the navigation route.

  After execution of step S160 in FIG. 9, the main processor 140 determines whether or not the unmanned aircraft 400 has arrived at the position of the control device 10 of the HMD 100 (step S170), and in the Wi-Fi navigation mode M2 until it arrives. Continue to perform the return. As illustrated in FIG. 10, the unmanned aerial vehicle 400 navigates in the Wi-Fi navigation mode M2 when flying in the field of view WA. However, immediately before arrival, the unmanned aircraft 400 recognizes an obstacle (here, the HMD 100 or the user) with the camera 420 so that the unmanned aircraft 400 does not collide with the HMD 100 or the user, and sets a predetermined distance. Release and automatically land.

  When it is determined in step S170 or step S150 that it has arrived, the main processor 140 ends this feedback processing.

A-5. About effect of embodiment:
According to the HMD 100 of the first embodiment configured as described above, when the unmanned aircraft 400 is traveling toward the control device 10 in the automatic navigation mode M1, the field of view WA is visible through the image display unit 20. When the unmanned aircraft 400 enters the vehicle, the automatic navigation mode M1 is stopped, and the unmanned aircraft 400 is navigated in the Wi-Fi navigation mode M2 in which the navigation route is determined from the change in the Wi-Fi radio wave intensity. For this reason, according to the HMD 100, it is possible to cancel the navigation in the automatic navigation mode M1 using GPS information with low accuracy, and therefore it is possible to return to the HMD 100 with high accuracy. In particular, in the present embodiment, the Wi-Fi navigation mode M2 that is switched from the automatic navigation mode M1 determines the navigation route from the change in the intensity of the Wi-Fi radio wave, and enables navigation with high accuracy. For this reason, the feedback to the HMD 100 can be performed with higher accuracy.

  As a modification of the first embodiment, instead of the Wi-Fi navigation mode M2, the navigation route is changed depending on the intensity of radio waves other than Wi-Fi, for example, wireless LAN, Bluetooth (registered trademark), iBeacon (registered trademark), etc. It is good also as a mode which determines.

B. Second embodiment:
FIG. 12 is a flowchart illustrating feedback processing executed by the HMD of the second embodiment. The feedback processing executed in the HMD of the second embodiment is different from the feedback processing (FIG. 9) in the first embodiment in the processing of step S260 corresponding to step S160, and the processing of other steps is the same. It is. The hardware configuration of the HMD of the second embodiment is the same as the hardware configuration of the HMD of the first embodiment. In the following description, the same parts as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment.

  In step S260 in the return process of FIG. 12, the main processor 140 stops the navigation in the automatic navigation mode and instructs the unmanned aircraft 400 to navigate in the dead reckoning navigation mode. That is, when the unmanned aerial vehicle 400 enters the field of view WA that can be visually recognized through the image display unit 20, the feedback processing of the first embodiment switches from the automatic navigation mode M1 to the Wi-Fi navigation mode M2. However, instead of this, in the feedback processing of the second embodiment, the automatic navigation mode M1 is switched to the dead reckoning navigation mode.

  The unmanned aerial vehicle 400 is provided with a six-axis sensor 450. From the detection signal of the 6-axis sensor 450, the attitude and speed of the unmanned aircraft 400 can be known. Therefore, the detection signal value (= detection value) of the 6-axis sensor 450 when the unmanned aerial vehicle 400 is in the state before takeoff is set as an initial value, and the detection value of the 6-axis sensor 450 is sequentially added to the initial value ( By adding up), it is possible to obtain a relative position and posture (position and posture of the unmanned aerial vehicle 400) based on the position corresponding to the initial value. This method of obtaining the symmetric position and orientation is called “Dead-Reckoning”.

  On the other hand, the HMD control device of the second embodiment is also provided with a six-axis sensor 111 (FIG. 5), as in the first embodiment. For this reason, by adopting the dead reckoning technique, it is possible to obtain the relative position and posture (position and posture of the HMD control device) based on the position and posture corresponding to the initial values. Before executing the return processing, the position and posture corresponding to the initial value of the unmanned aircraft 400 are calibrated to the position corresponding to the initial value of the control device of the HMD, and then the unmanned aircraft 400 side, By performing dead reckoning on both the control device side, the current position and posture of the unmanned aerial vehicle 400 with respect to the current position and posture of the control device of the HMD can be obtained.

  In the dead reckoning navigation mode executed in step S260, the main processor of the control device obtains the current position and posture of the unmanned aircraft 400 with respect to the current position and posture of the control device of the HMD, as described above. A navigation route is obtained from the obtained position and posture, and the flight form is instructed to the unmanned aircraft 400 via the remote control device 500 so as to fly along this navigation route. The flight form is the movement of the aircraft such as ascending, descending, moving forward, moving backward, moving left, moving right, and hovering.

  Instead of the configuration in which the unmanned aircraft 400 is instructed to fly through the remote control device 500, the control device may directly instruct the unmanned aircraft 400 from the wireless communication unit 117 (FIG. 5). . Further, the control device 10 performs processing for obtaining the current position and posture of the unmanned aerial vehicle 400 with respect to the current position and posture of the HMD control device and obtaining a navigation route from the obtained position and posture. Alternatively, the control unit 490 of the unmanned aircraft 400 may be configured so that the control unit 490 controls the flight form of the unmanned aircraft 400 according to the obtained navigation route.

  According to the dead reckoning navigation mode, the current position and posture of the unmanned aerial vehicle 400 with respect to the current position and posture of the control device of the HMD are obtained with high accuracy using detection signals of the six-axis sensors 111 and 450. be able to. For this reason, according to the HMD of the second embodiment, the unmanned aircraft 400 can be returned to the HMD with high accuracy, like the HMD 100 of the first embodiment.

C. Third embodiment:
FIG. 13 is a flowchart illustrating feedback processing executed by the HMD of the third embodiment. The feedback process executed by the HMD of the third embodiment is different from the feedback process (FIG. 9) in the first embodiment in the process of step S260 corresponding to step S160, and the other steps are the same. It is. The hardware configuration of the HMD of the third embodiment is the same as the hardware configuration of the HMD of the first embodiment. In the following description, the same parts as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment.

  In step S160 in the return process of FIG. 13, the main processor 140 stops the navigation in the automatic navigation mode and instructs the unmanned aircraft 400 to navigate in the manual navigation mode. That is, when the unmanned aerial vehicle 400 enters the field of view WA that can be visually recognized through the image display unit 20, the feedback processing of the first embodiment switches from the automatic navigation mode M1 to the Wi-Fi navigation mode M2. However, instead of this, in the feedback processing of the third embodiment, switching from the automatic navigation mode M1 to the manual navigation mode is performed.

  FIG. 14 is an explanatory diagram showing an operation screen CB in which the operation unit 520 of the remote control device 500 (FIG. 1) is realized in software. The operation screen CB is displayed by the image display unit 20. The operator performs an input operation on the operation screen CB using the direction key 16 (FIG. 1) and the enter key 17 (FIG. 1). In the manual navigation mode, the main processor 140 controls the unmanned aircraft 400 from the HMD 100 side using the functions of the remote control device 500 by transmitting the operation command input to the operation screen CB to the remote control device 500. .

  In addition, it replaces with the structure which sends the operation command input into the operation screen to the unmanned aircraft 400 via the remote control device 500, and the control device sends the unmanned aircraft 400 directly from the wireless communication unit 117 (FIG. 5). Also good.

  According to the HMD of the third embodiment configured as described above, when the unmanned aircraft 400 is sailing toward the control device 10 in the automatic navigation mode M1, the inside of the field of view WA that can be visually recognized through the image display unit 20. When the unmanned aerial vehicle 400 enters, the automatic navigation mode M1 using the GPS information is stopped, and the unmanned aircraft 400 is navigated in the manual navigation mode. For this reason, according to the HMD of the third embodiment, it is possible to cancel the navigation in the automatic navigation mode M1 using low-accuracy GPS information, so that the high accuracy up to the HMD 100 is achieved as in the first and second embodiments. It is possible to return with.

  As a modification of the third embodiment, the six-axis sensors 111 and 450 may be replaced with an inertial sensor having only a three-axis acceleration sensor.

D. Fourth embodiment:
FIG. 15 is an explanatory diagram illustrating an operation screen CBX displayed in the HMD of the fourth embodiment. The operation screen CBX corresponds to the operation screen CB (FIG. 14) in the third embodiment. The operation screen CBX has the same shape and function as the operation screen CB in the third embodiment, except that the display position by the image display unit 20 is different. Other software configurations and hardware configurations are the same as those of the HMD of the third embodiment.

  In FIG. 15, the broken line indicates the display position of the operation screen CB of the third embodiment. In the third embodiment, when the unmanned aerial vehicle 400 enters the view VT, the position of the unmanned aircraft 400 in the view VT may overlap with the operation screen CB. On the other hand, in the HMD of the fourth embodiment, the position of the operation screen CBX is changed so that the position is away from the position of the unmanned aircraft 400 in the field of view VT. Specifically, in step S140 of FIG. 9, a position where the unmanned aircraft 400 is detected by pattern matching is obtained, the position of the unmanned aircraft 400 in the field of view VT is estimated from the position, and a position away from the estimated position Move the operation screen CBX. As a result, the position of the unmanned aircraft 400 in the field of view VT and its surroundings can pass through the outside world, and the user can confirm the unmanned aircraft 400 reliably.

  Therefore, in the HMD of the fourth embodiment, it is possible to feed back to the HMD 100 with high accuracy, as in the third embodiment. Particularly in the HMD of the fourth embodiment, since the unmanned aircraft 400 is not hidden by the operation screen CBX in the field of view VT, the operability of manually returning the unmanned aircraft 400 can be improved. Note that the configuration for changing the display position of the operation screen CB so that the position of the unmanned aircraft 400 and the operation screen CB do not overlap in this embodiment is also adopted in the first embodiment and the second embodiment. May be.

E. Variation:
The present invention is not limited to the first to fourth embodiments and their modifications, and can be carried out in various modes without departing from the gist thereof. For example, the following modifications are possible. Is also possible.

・ Modification 1:
In each embodiment and modification, the unmanned aerial vehicle system includes the HMD 100, the unmanned aircraft 400, and the remote control device 500. On the other hand, as a modification, the unmanned aerial vehicle system may include the HMD 100 and the unmanned aircraft 400 without including the remote control device 500. In this case, all functions of the remote control device 500 are configured to be realized by the HMD 100 by software.

Modification 2
In each embodiment and modification, the GPS information that is the destination in the automatic navigation mode is acquired by the GPS receiver 115 provided in the control device 10 of the HMD 100. On the other hand, as a modification, a GPS receiver may be provided in the image display unit 20 of the HMD 100, and GPS information may be acquired from the GPS receiver. Further, the GPS information acquired by the GPS receiver 530 provided in the remote control device 500 may be acquired from the remote control device 500.

・ Modification 3:
In each embodiment and modification, the GPS receiver 115 is provided in the control device 10 of the HMD 100, and the automatic navigation mode is configured to return the unmanned aircraft 400 to the current position of the control device 10 of the HMD 100. On the other hand, as a modification, a GPS receiver may be provided at a position away from the head-mounted display device, and the unmanned aircraft 400 may be navigated to the position of the GPS receiver. For example, a GPS receiver may be arranged at a predetermined position in a predetermined facility, and the position may be set as a destination in the automatic navigation mode. In this case, the unmanned aerial vehicle navigates to a predetermined facility in the automatic navigation mode, and the user of the HMD goes to this facility and operates in a specific navigation mode different from the automatic navigation mode. It is possible to navigate unmanned aircraft.

-Modification 4:
In each embodiment and the modified example, whether the unmanned aircraft 400 has entered the field of view WA using the captured image of the camera 61 on the assumption that the captured image of the camera 61 matches the field of view WA visible by the image display unit 20. It was judged whether or not. On the other hand, as a modified example, the orientation in which the image display unit 20 faces is obtained from the 6-axis sensor 235 and the magnetic sensor 237 provided in the image display unit 20, and this orientation can be visually recognized by the image display unit 20. Assuming that the field of view WA can be specified, it is possible to determine whether or not the unmanned aircraft 400 has entered the field of view WA from this direction and the position indicated by the GPS information acquired by the GPS receiver 440 of the unmanned aircraft 400. Good. Even with this configuration, the same effects as those of the embodiments and the modified examples can be obtained.

  Furthermore, as another modification, the distance to the unmanned aircraft is detected by the distance measuring sensor described above, and it is determined that the unmanned aircraft 400 has entered the field of view WA when the detected distance is equal to or less than a predetermined value. May be. For humans, binocular parallax is the most dominant up to about 10 m, and the depth sensitivity is the highest. For this reason, the predetermined value is set to 11 m to 9 m, for example, and when the distance to the unmanned aircraft is equal to or less than the predetermined value, it is determined that the unmanned aircraft 400 has entered the field of view WA. Even with this configuration, the same effects as those of the embodiments and the modified examples can be obtained. The predetermined value may be 10 m, for example.

  Furthermore, as another modified example, as a configuration for determining whether or not the unmanned aircraft 400 has entered the field of view WA using the captured image of the camera 420 on the unmanned aircraft 400 side instead of the captured image of the camera 61. Also good. Specifically, it is configured to determine whether or not the unmanned aircraft 400 has entered the field of view WA based on whether or not a user wearing the HMD is included in the captured image of the camera 420 on the unmanned aircraft 400 side. . Even with this configuration, the same effects as those of the embodiments and the modified examples can be obtained. In addition, a captured image of the camera 420 displayed on the display 510 of the remote control device 500 (see FIG. 8) is acquired from the remote control device 500, and whether the unmanned aircraft 400 has entered the field of view WA using the captured image. It may be configured to determine whether or not.

-Modification 5:
In each embodiment and the modification, when the unmanned aircraft 400 enters the field of view WA during the navigation in the automatic navigation mode, the automatic navigation mode is switched to the Wi-Fi navigation mode. On the other hand, as a modification, after switching to the Wi-Fi navigation mode, when the unmanned aircraft 400 turns in a direction away from the user and the unmanned aircraft 400 moves outside the field of view WA, the Wi-Fi The navigation mode may be switched to the automatic navigation mode. Furthermore, the present invention can be applied even when the user repeatedly enters and leaves the field of view, such as when the user enters the field of view again after leaving the user's field of view.

Modification 6:
In the second embodiment, the position and orientation of the control device of the HMD are obtained based on the detection signal of the 6-axis sensor 111 (FIG. 5). On the other hand, as a modification, in addition to the detection signal of the six-axis sensor 111, a configuration that obtains the position and orientation of the HMD control device with higher accuracy by using a GPS detection signal and a captured image of the camera. Also good. In another HMD equipped with a camera and a communication device, the position and posture of the user's HMD (HMD to which the present invention is applied) can be observed, and the observed position and posture are communicated from the other HMD. By receiving, the position and posture of the user's HMD may be obtained.

Modification 7:
In the dead reckoning navigation mode in the second embodiment, the value of the detection signal (= detection value) of the six-axis sensor 450 when the unmanned aircraft 400 is in the state before takeoff is set as the initial value. On the other hand, as a modification, the position and orientation when the unmanned aircraft 400 is in the state before takeoff may be obtained from the captured image of the camera 61 on the HMD side, and the obtained position and orientation may be used as initial values. . The camera 61 may be a stereo camera. The camera 61 may be a monocular camera, and the position and orientation may be obtained by calculating the distance from two or more captured images based on the principle of triangulation, and may be used as the initial value. Furthermore, it can be changed to when the unmanned aerial vehicle 400 is in a state before take-off, and when it is in a predetermined state after take-off. Specifically, after the unmanned aircraft takes off, it may be hovered at a predetermined height, and the position and orientation at that time may be set as initial values.

-Modification 8:
In each embodiment and modification, an unmanned aerial vehicle capable of remote control or autonomous flight such as a drone has been described as an example. However, the present invention is not limited thereto, and is not limited to this. An unmanned ground vehicle, an unmanned surface ship, an unmanned surface boat, The present invention can also be applied to unmanned aerial vehicles such as unmanned surface ships, unmanned submarines, unmanned submersibles, and unmanned spacecraft.

-Modification 9:
In each embodiment and modification, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware. It may be.

Modification 10:
In the said embodiment, it illustrated about the structure of HMD. However, the configuration of the HMD can be arbitrarily determined without departing from the gist of the present invention, and for example, addition / deletion / conversion of components can be performed.

  In the above embodiment, the so-called transmission type HMD 100 in which the right light guide plate 26 and the left light guide plate 28 transmit external light has been described. However, the present invention can also be applied to, for example, a so-called non-transmissive HMD 100 that displays an image without passing through the outside world. Further, the non-transmissive HMD 100 may be configured to capture the outside world with a camera and display the captured image on the display unit. In these HMDs 100, in addition to AR (Augmented Reality) display that displays an image superimposed on the real space described in the above embodiment, MR (Mixed Reality) that displays a combination of the captured real space image and a virtual image. Display or VR (Virtual Reality) display for displaying a virtual space can also be performed.

  In the above embodiment, the functional units of the control device 10 and the image display unit 20 have been described, but these can be arbitrarily changed. For example, the following aspects may be adopted. A mode in which the storage function unit 122 and the control function unit 150 are mounted on the control device 10 and only the display function is mounted on the image display unit 20. A mode in which the storage function unit 122 and the control function unit 150 are mounted on both the control device 10 and the image display unit 20. The aspect which integrated the control apparatus 10 and the image display part 20. FIG. In this case, for example, the image display unit 20 includes all the components of the control device 10 and is configured as a glasses-type wearable computer. A mode in which a smartphone or a portable game device is used instead of the control device 10. The aspect which connected the control apparatus 10 and the image display part 20 by radio | wireless communication, and arranged the connection cable 40. FIG. In this case, for example, power supply to the control device 10 and the image display unit 20 may be performed wirelessly.

Modification 11:
In the said embodiment, it illustrated about the structure of the control apparatus. However, the configuration of the control device can be arbitrarily determined without departing from the gist of the present invention, and for example, addition / deletion / conversion of components can be performed.

  In the above embodiment, an example of the input unit provided in the control device 10 has been described. However, the control device 10 may be configured by omitting some of the illustrated input means, and may include other input means not described above. For example, the control device 10 may include an operation stick, a keyboard, a mouse, and the like. For example, the control device 10 may include an input unit that interprets a command associated with the movement of the user's body. The user's body movement or the like can be acquired by, for example, line-of-sight detection for detecting line-of-sight, gesture detection for detecting hand movement, foot switch for detecting foot movement, or the like. The line-of-sight detection can be realized by, for example, a camera that images the inside of the image display unit 20. Gesture detection can be realized, for example, by analyzing an image taken with the camera 61 over time.

  In the above embodiment, the control function unit 150 is operated by the main processor 140 executing the computer program in the storage function unit 122. However, the control function unit 150 can employ various configurations. For example, the computer program can be stored in the nonvolatile storage unit 121, the EEPROM 215, the memory 118, and other external storage devices (such as a USB memory inserted in various interfaces) instead of or together with the storage function unit 122. Or an external device such as a server connected via a network). Further, each function of the control function unit 150 may be realized using an ASIC (Application Specific Integrated Circuit) designed to realize the function.

Modification 12:
In the said embodiment, it illustrated about the structure of the image display part. However, the configuration of the image display unit can be arbitrarily determined without departing from the gist of the present invention, and for example, addition / deletion / conversion of components can be performed.

  FIG. 16 is a plan view of a principal part showing a configuration of an optical system provided in an image display unit according to a modification. In the image display unit of the modified example, an OLED unit 221a corresponding to the user's right eye RE and an OLED unit 241a corresponding to the left eye LE are provided. The OLED unit 221a corresponding to the right eye RE includes an OLED panel 223a that develops white color and an OLED drive circuit 225 that drives the OLED panel 223a to emit light. A modulation element 227 (modulation device) is arranged between the OLED panel 223a and the right optical system 251. The modulation element 227 is configured by, for example, a transmissive liquid crystal panel, and generates image light L by modulating light emitted from the OLED panel 223a. The image light L that has been transmitted through the modulation element 227 and modulated is guided to the right eye RE by the right light guide plate 26.

  The OLED unit 241a corresponding to the left eye LE includes an OLED panel 243a that emits white light and an OLED drive circuit 245 that drives the OLED panel 243a to emit light. A modulation element 247 (modulation device) is arranged between the OLED panel 243a and the left optical system 252. The modulation element 247 is constituted by, for example, a transmissive liquid crystal panel, and generates image light L by modulating light emitted from the OLED panel 243a. The image light L that has been modulated through the modulation element 247 is guided to the left eye LE by the left light guide plate 28. The modulation elements 227 and 247 are connected to a liquid crystal driver circuit (not shown). The liquid crystal driver circuit (modulation device driving unit) is mounted on a substrate disposed in the vicinity of the modulation elements 227 and 247, for example.

  According to the image display unit of the modified example, the right display unit 22 and the left display unit 24 respectively include OLED panels 223a and 243a as light source units and image light including a plurality of color lights by modulating light emitted from the light source units. Are provided as modulation elements 227 and 247 that output a video signal. Note that the modulation device that modulates the light emitted from the OLED panels 223a and 243a is not limited to a configuration in which a transmissive liquid crystal panel is employed. For example, a reflective liquid crystal panel may be used instead of the transmissive liquid crystal panel, a digital micromirror device may be used, or a laser retinal projection type HMD 100 may be used.

  In the above embodiment, the spectacle-type image display unit 20 has been described, but the aspect of the image display unit 20 can be arbitrarily changed. For example, the image display unit 20 may be mounted like a hat, or may be built in a body protector such as a helmet. Further, the image display unit 20 may be configured as a HUD (Head Up Display) mounted on a vehicle such as an automobile or an airplane, or other transportation means.

  In the above embodiment, a configuration in which virtual images are formed by the half mirrors 261 and 281 on a part of the right light guide plate 26 and the left light guide plate 28 as an optical system that guides image light to the user's eyes has been exemplified. However, this configuration can be arbitrarily changed. For example, a virtual image may be formed in a region that occupies the entire surface (or most) of the right light guide plate 26 and the left light guide plate 28. In this case, the image may be reduced by an operation for changing the display position of the image. The optical element of the present invention is not limited to the right light guide plate 26 and the left light guide plate 28 having the half mirrors 261 and 281, but is an optical component (for example, a diffraction grating, a prism, or the like) that makes image light incident on the user's eyes. Any form can be adopted as long as holography or the like is used.

  The present invention is not limited to the above-described embodiments, examples, and modifications, and can be realized with various configurations without departing from the spirit thereof. For example, the technical features in the embodiments, examples, and modifications corresponding to the technical features in each embodiment described in the summary section of the invention are to solve some or all of the above-described problems, or In order to achieve part or all of the above-described effects, replacement or combination can be performed as appropriate. Further, if the technical feature is not described as essential in the present specification, it can be deleted as appropriate.

    DESCRIPTION OF SYMBOLS 10 ... Control apparatus, 12 ... Lighting part, 14 ... Touchpad, 16 ... Direction key, 17 ... Decision key, 18 ... Power switch, 19 ... Vibrator, 20 ... Image display part, 21 ... Right holding part, 22 ... Right display Unit: 23 ... Left holding unit, 24 ... Left display unit, 26 ... Right light guide plate, 27 ... Front frame, 28 ... Left light guide plate, 30 ... Headset, 32 ... Right earphone, 34 ... Left earphone, 40 ... Connection Cable, 46 ... Connector, 61 ... Camera, 63 ... Microphone, 65 ... Illuminance sensor, 67 ... LED indicator, 100 ... Head-mounted display device (HMD), 110 ... Operation unit, 111 ... 6-axis sensor, 113 ... Magnetic Sensor, 115 ... GPS receiver, 117 ... Wireless communication unit, 118 ... Memory, 120 ... Controller board, 121 ... Non-volatile storage unit, 122 ... Memory Function unit, 123: Setting data, 124: Content data, 130: Power supply unit, 132 ... Battery, 134 ... Power supply control circuit, 140 ... Main processor, 145 ... Image processing unit, 147 ... Display control unit, 149 ... Imaging control unit , 150: control function unit, 151: input / output control unit, 153 ... communication control unit, 155 ... unmanned aircraft control unit, 155a ... feedback processing unit, 180 ... voice codec, 182 ... voice interface, 184 ... external connector, 186 ... External memory interface, 188 ... USB connector, 192 ... sensor hub, 196 ... interface, 210 ... display unit board, 211 ... interface, 213 ... receiver, 215 ... EEPROM, 217 ... temperature sensor, 221,221a ... OLED unit, 223 223a ... LED panel, 225 ... OLED drive circuit, 227 ... modulation element, 230 ... display unit substrate, 231 ... interface, 233 ... receiver, 235 ... 6-axis sensor, 237 ... magnetic sensor, 239 ... temperature sensor, 241,241a ... OLED Unit, 243, 243a ... OLED panel, 245 ... OLED drive circuit, 247 ... modulation element, 251 ... right optical system, 252 ... left optical system, 261 ... half mirror, 281 ... half mirror, 400 ... unmanned aircraft, 410 ... propeller , 420 ... Camera, 430 ... Motor, 440 ... GPS receiver, 450 ... 6-axis sensor, 460 ... Wireless communication unit, 470 ... Wi-Fi communication unit, 480 ... Battery, 490 ... Control unit, 500 ... Remote control device, 510 ... Display, 520 ... operation unit, 530 ... GPS receiver -Bar, 540 ... Wireless communication unit, 550 ... USB connector, 590 ... Control unit, WA ... External

Claims (10)

A head-mounted display device capable of performing control related to the operation of the drone,
A display that can visually recognize the outside world;
A GPS information acquisition unit for acquiring GPS information;
A communication unit capable of performing communication using wireless communication with the drone; and
A control unit;
With
The controller is
When the drone is navigating in an automatic navigation mode that moves toward the position indicated by the GPS information acquired by the GPS information acquisition unit, the drone is within the field of view visible by the display unit. Determine whether or not
When it is determined that the drone has entered the field of view, the drone is stopped in the automatic navigation mode, and the drone is operated in a specific navigation mode different from the automatic navigation mode. A head-mounted display device. The head-mounted display device according to claim 1, further comprising:
A camera for imaging the field of view;
The controller is
A determination is made as to whether or not the drone has entered the field of view by acquiring a captured image of the camera and pattern-matching a target image prepared in advance to identify the drone and the captured image. A head mounted display device. The head-mounted display device according to claim 1 or 2, further comprising:
A radio wave receiver for receiving radio waves sent from the drone,
The specific navigation mode is:
A head-mounted display device that is a mode in which the intensity of a radio wave received by the radio wave receiver is measured and a navigation route is determined from the measured change in the intensity. The head-mounted display device according to claim 1 or 2,
The drone includes a first inertial sensor that detects the movement of the aircraft, and obtains the position of the aircraft by sequentially integrating the detection values of the first inertial sensor,
The specific navigation mode is:
The head-mounted display device is a mode in which the head-mounted display device acquires a position of the airframe obtained by the drone and determines a navigation route from the acquired position of the airframe. The head-mounted display device according to claim 4, further comprising:
A second inertial sensor for detecting movement;
The specific navigation mode is:
The head-mounted display device obtains the position of the head-mounted display device by sequentially integrating the detection values of the second inertial sensor, and obtains the obtained position of the head-mounted display device. A head-mounted display device which is a mode for determining a navigation route from the position of the aircraft. The head-mounted display device according to claim 1 or 2, further comprising:
Operated by a user, having an operation unit for instructing the movement of the drone,
The specific navigation mode is:
The head-mounted display device, wherein the head-mounted display device is in a mode for determining a navigation route in accordance with an instruction to the operation unit. The head-mounted display device according to claim 6,
The display unit is a display unit that is visible through the outside world,
The controller is
An operation screen for the operation unit is displayed on the display unit,
A head-mounted device that estimates the position of the drone on the display unit and moves the operation screen to a position away from the estimated position when it is determined that the drone has entered the field of view. Type display device. The head-mounted display device according to any one of claims 1 to 7,
The GPS information acquisition unit includes a GPS reception unit that receives the GPS information,
The automatic navigation mode is a head-mounted display device that returns to the head-mounted display device. The head-mounted display device according to any one of claims 1 to 7,
The GPS information acquisition unit
A head-mounted display device that receives the GPS information from a GPS receiver provided at a position distant from the head-mounted display device. A method for controlling a head-mounted display device capable of performing control related to the operation of the drone,
Whether the drone has entered the field of view visible by the head-mounted display device when the drone is navigating in the automatic navigation mode that moves toward the position indicated by the GPS information. And
When it is determined that the drone has entered the field of view, the drone is stopped in the automatic navigation mode, and the drone is operated in a specific navigation mode different from the automatic navigation mode. A method for controlling a head-mounted display device.

Images