JP2000348198A - Display control device - Google Patents

Abstract

(57) [Problem] To provide a display control device that can be used as a pseudo display system without restriction of operation conditions. SOLUTION: A tilt of a head-mounted display 5 is detected, and a correction angle of the opposite polarity of the tilt angle is given to an image generator to cancel each other, thereby holding a pseudo image in the horizontal direction. At that time, the invisible light emitters 10a to 10h serving as a reference are incorporated on the screen side, and the infrared camera 7a is incorporated as a measurement system on the observer side so that the tilt measurement system is not added with the multi-screen.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display control device having a head-mounted display, which resembles a pseudo-telescope used in, for example, an air traffic control simulator.

[0002]

2. Description of the Related Art A simulation display system represented by an air traffic control simulator includes a simulation display of a visual field image on a large screen, a simulation display of a telescope, and the like. The head-mounted display, which resembles a pseudo-telescope, has an image that passes through the head-mounted display if the unit is tilted and appropriate image correction processing is not performed.
In general, it becomes oblique with the unit and does not match the actual large-screen visual field image.

Conventionally, various systems for simulating an image or the like on a screen have been known. FIG.
1 is a system configuration example of a simulation display system disclosed in Japanese Utility Model Laid-Open No. 3-77976. The system (large screen display system) shown in FIG. 1 includes a screen image generator 101 of this large screen display system, a projector 102 for projecting an image on a projection screen 103, a head mounted display image generator 104, A head-mounted display 105, an invisible light emitter 106, and an infrared camera 10 for imaging the projection surface of the screen 103
7. Invisible light coordinate calculator 108 projected on screen 103, and central processing unit 1 controlling this system
09.

Next, the operation of the conventional simulation display system will be described. This simulation display system consists of a part that simulates the image of the entire field of view of the observer,
Magnifiers including binoculars are collectively referred to as "telescopes." The central processing unit 109 in FIG. 12 controls the entire system, and includes, for example,
A drawing instruction is generated to the screen image generator 101 and transmitted. Screen image generator 101
Processes this drawing instruction, and gives R,
G and B video signals and horizontal and vertical synchronization signals are transmitted.

The projector 102 performs “electric-light conversion” for converting an input electric signal into an optical signal, and projects an image to be displayed on a screen 103. By these operations, an image of the entire field of view of the observer 119 is displayed on the screen 103.

The head mounted display 10
Reference numeral 5 denotes a so-called pseudo-telescope, and a uniaxial invisible light emitter 106 integrally formed with the head-mounted display 105 irradiates the screen 103 with invisible light as an indication of the line of sight of the observer 119. . The infrared camera 107 captures the invisible light reflected by the screen 103, and the coordinate calculator 108 calculates the projection coordinates of the invisible light, which is the line of sight, on the screen surface from the signal captured by the infrared camera 107. Then, it is replaced with the coordinates displayed on the head mounted display 105 and sent to the central processing unit 109.

The central processing unit 109 has a head mount
A drawing instruction indicating display coordinates and a display range is generated and transmitted to the display image generator 104. Therefore, the image generator 104 for the head mounted display is used.
Processes this drawing instruction, and sends out R, G, B video signals and horizontal and vertical synchronization signals to the head mounted display 105. As a result, the head mounted display 105 performs display based on these R, G, B video signals and horizontal and vertical synchronization signals. By the above operation,
An image viewed through the telescope is displayed on the head mounted display 105.

[0008]

However, according to the display control method in the above-described conventional simulated display system, the image of the head-mounted display 105 as a simulated telescope is linked to the screen direction of the observer 119 even if it is interlocked with the line of sight. (The direction of rotation of the head-mounted display 105 in a plane parallel to the screen surface) is not detected. Therefore, when the observer tilts his or her head, the image on the head-mounted display 105 is tilted together with the head-mounted display 105 as it is.

Therefore, when a scene such as an external display is displayed on the screen surface, the observer tilts his / her head, and the ground included in the scene is displayed on the screen looking through the head-mounted display 105. There is a problem that the image is rotated (tilted), and the image looks different from an image obtained when looking through an actual telescope.

Further, when the conventional head-mounted display 105 observes an aircraft in a field of view without a background, such as a mountain, the recognition of the horizontal flight, ascending flight, and descending flight is erroneous. For this reason, there is also a problem that it is necessary to limit the operating conditions of the system.

Furthermore, the conventional system shown in FIG. 12 has a single screen. However, when the screen is multi-screened, it is necessary to install an infrared camera 107 for each screen, which increases the cost. Can lead to problems.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and has as its object the purpose of viewing through an image on a head-mounted display even when a pseudo telescope rotates in a plane parallel to a screen surface. The horizontal axis does not fluctuate,
An object of the present invention is to provide a display control device which always coincides with the horizontal axis of a screen image and which can be used as a pseudo display system without restricting operation conditions.

[0013] Another object of the present invention is to provide a horizontal system viewed through an image of a head-mounted display even in a display system environment with only a head-mounted display in a narrow space where a large system cannot be installed. An object of the present invention is to provide a display control device in which the axis and the horizontal axis of the screen image always coincide.

[0014]

In order to achieve the above object, a first aspect of the present invention is to form a whole or a part of a visual field image projected on one or more screens by one or more projectors. In a display control device for displaying a pseudo image on a pseudo telescope, an instruction means for optically instructing a corner of each of the screens, a detection means for detecting the instruction, and a first image processing method for the visual field image based on the detection result. Means for calculating the coordinates of the pseudo-image, means for calculating the second coordinates of the pseudo image based on the detection result, inclination between the coordinate systems indicated by the first and second coordinates, and Means for calculating center coordinates, correction means for performing drawing correction of the pseudo image based on the inclination and center coordinates, and means for sending the corrected pseudo image to the projector The provided, said correction means, regardless of the slope of the pseudo telescope, to provide a display control device for drawing corrected to the horizontal direction of the field of view image and the pseudo-image after the correction is always match.

[0015] Preferably, the indicating means is an invisible light emitter that emits light with a natural period. Preferably, the detection means and the pseudo telescope have an integral structure.

According to a second aspect of the present invention, there is provided a display control device for displaying a whole or a part of a visual field image projected on a screen by a projector as a pseudo image on a pseudo telescope.
Means for projecting an invisible light beam onto the screen, detection means for detecting the reflected light of the invisible light beam from the screen, means for determining the center coordinates of the invisible light beam from the detection result, and based on the center coordinates Means for calculating the coordinates of the pseudo image and the inclination of the invisible light beam; correction means for performing drawing correction of the pseudo image based on the inclination and coordinates; and sending the corrected pseudo image to the projector. And a display control device that performs drawing correction so that the horizontal direction of the visual field image always coincides with the corrected pseudo image regardless of the inclination of the pseudo telescope.

Preferably, the invisible light is a four-axis invisible light that emits light with a natural period. Further, the line-of-sight direction of the pseudo telescope is the same as the projection direction of the invisible light.

According to a third aspect of the present invention, there is provided a display control apparatus for displaying a whole or a part of a visual field image projected on a screen by a projector as a pseudo image on a pseudo telescope,
Means for replacing the coordinates of the pseudo image with a coordinate system for the visual field image; angle detecting means for obtaining three-axis angle information of the pseudo telescope based on the replacement result; Means for calculating a coordinate value indicating the line of sight direction and a rotation angle, correction means for performing drawing correction of the pseudo image based on the coordinate value and the rotation angle, and means for sending the corrected pseudo image to the projector A display control device that performs drawing correction so that the horizontal direction of the visual field image always coincides with the corrected pseudo image regardless of the inclination of the pseudo telescope.

Preferably, the angle detecting means and the pseudo telescope have an integral structure.

Preferably, the display control device according to the third invention further includes a light projecting means for projecting a uniaxial invisible light beam, and a detecting device for detecting the reflected light of the invisible light beam from the screen. Means, and means for calculating the line-of-sight coordinates of the pseudo telescope from the detection result, wherein the correction means performs drawing correction of the pseudo image based on the line-of-sight coordinates and the rotation angle.

Preferably, the angle detecting means, the light projecting means, and the pseudo telescope have an integral structure.

According to a fourth aspect of the present invention, there is provided a display control device for displaying a pseudo image on a built-in display, a detecting means for detecting angles of the display control device in three axial directions, and a detection means for detecting predetermined angles in a space based on the detection result. Means for calculating the display center coordinates of the pseudo image with respect to the plane and the rotation angle of the display control device; correction means for performing drawing correction of the pseudo image based on the display center coordinates and the rotation angle; Means for sending an image to the display.

Preferably, the angles in the three axial directions are a roll angle, a pitch angle, and a yaw angle.

According to a fifth aspect of the present invention, there is provided a display control device for displaying a pseudo image on a built-in display, an instruction means for instructing the display control device to tilt and move up and down, and a conversion for converting the instruction result into video information. Means, means for calculating the inclination and rotation angle of the display control device based on the video information, correction means for performing drawing correction of the pseudo image based on the inclination and rotation angle, and Means for sending a pseudo image to the display device.

Preferably, the indicating means is a reference scale comprising a plurality of radial straight lines and concentric circles. The indicating means and the display control device have an integral structure, and the converting means is located above the central axis of the concentric circle.

Preferably, the indicating means is a scale composed of a plurality of horizontal lines and vertical lines covering all directions in a cylindrical shape. Further, the conversion means and the display control device have an integral structure, and the conversion means is located near the center of the cylinder.

Preferably, the display control device according to the fifth aspect of the present invention is mounted on the head of an observer, and interlocks with the movement of the head.

According to a sixth aspect of the present invention, in a display control device for displaying a pseudo image on a built-in display, a means for detecting a line of sight of an observer and a movement of the line of sight in a predetermined display area of the display are detected. A display control device that includes: a correction unit that performs drawing correction of the pseudo image based on the movement of the line of sight; and a unit that sends the corrected pseudo image to the display.

[0029]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings. Embodiment 1 FIG. First, Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram showing the overall configuration of the display control system according to the present embodiment. Note that the system described here is an example in which an observer's visual field image is displayed on a three-screen multi-screen.

In the display control system according to the present embodiment, the left screen 3, the center screen 3a, the right screen 3b, and the projectors 2 for these screens 3, 3a, 3b simulate the visual field image of the observer 19. 2
a, 2b and the image generators 1, 1a, 1b for the projectors 2, 2a, 2b.

The part simulating the telescope is integrated with the head-mounted display 5 as a pseudo-telescope, the invisible light emitters 10a to 10h for indicating the four corners of the screens 3 to 3b, and the head-mounted display 5. An infrared camera 7a, a coordinate calculator 8a for calculating a display coordinate and a rotation angle of a screen coordinate system of the head mounted display 5, and an image generator 4 for a head mounted display.

Here, the "screen coordinate system" is a display coordinate system on a screen of a large screen display system for displaying the outside world. The coordinate system in the imaging plane of the infrared camera is defined as “camera coordinate system”. Note that the central processing unit 9 includes a portion that simulates the observer's visual field image described above,
It is a component common to the part simulating the telescope, and operates to control the entire system.

Next, the operation of the display control system according to the present embodiment will be described. This system functionally consists of a visual field image simulation unit and a telescope simulation unit.
The description starts with the visual field image simulation unit.

The visual field image simulating unit in the present system allows the observer to visually recognize the entire visual field area (for example, an external scene). Therefore, the central processing unit 9
A drawing instruction for displaying a desired image is generated for each screen of the three-screen multi-screen composed of the above-described screens 3, 3a, 3b, and is sent to the screen image generators 1, 1a, 1b. I do.

Screen image generators 1, 1a, 1b
Processes the above-mentioned drawing instructions, draws and updates display buffers (not shown) incorporated in each of these generators, and outputs R, G, B video signals and horizontal and vertical synchronizing signals for performing raster display. It is sent to the projectors 2, 2a, 2b. Then, the projectors 2, 2a, and 2b connect these R, G, B
The format of the video signal and the horizontal and vertical synchronizing signals is converted from an electrical signal to an optical signal, and an image is projected on the screen 3, 3a, 3b based on the signal. As a result, the screens 3, 3a, and 3b can be visually recognized as projected predetermined images.

Next, the telescope simulator will be described. This telescope simulating unit realizes a pseudo telescope by treating the head mounted display 5 as a telescope,
Here, it has a function of enlarging and displaying an arbitrary part of the image displayed on the screens 3, 3a, 3b.

Multi-screen by screens 3, 3a, 3b
As shown in FIG. 1, the screen incorporates invisible light emitters 10a to 10h indicating the corners of each screen. These invisible light emitters 10a-
For example, by assigning a unique code such as a blinking cycle to 10h, the invisible light is
Make it possible to recognize from which point.

The observer 19 mounts the head mounted display 5 integrally formed with the infrared camera 7a for detecting invisible light on his / her own head, and a pseudo telescope is provided in a region on the screen where one wishes to watch. Head mount
Turn the display 5. This infrared camera 7a converts an incident light signal into an electric signal. In this operation, the invisible light emitted from a corner of the screen is obtained in an operation of pointing the camera to an area to be watched, and is converted into an electric signal. Convert to Then, the infrared camera 7a outputs the converted signal as a video signal to the coordinate calculator 8a together with a camera control synchronization signal.

The coordinate calculator 8a generates the raster coordinates of the camera system from the synchronization signal and sets the raster coordinates at the time of detecting the invisible light signal as the screen corner coordinates.
Here, the telescope image is displayed on the head-mounted display 5
Is displayed in the “center coordinates” of the video to be drawn.
And that "tilt" is needed. Therefore, the coordinate calculator 8
“a” calculates the drawing center coordinates and the inclination of the display target screen from the screen corner coordinates obtained in the camera coordinate system.

Hereinafter, the operation for calculating the drawing center coordinates and the inclination of the display target screen will be described in detail. FIG. 2 is a diagram illustrating the concept of coordinate calculation according to the present embodiment. Here, for ease of explanation, the projection plane is a screen 3
a only. Further, in the same figure, a range indicated by reference numeral 20 is an imaging range of the infrared camera 7a.

FIGS. 2A and 2B show the relationship between the above-described screen coordinate system and the camera coordinate system. FIG. 2A shows that the infrared camera 7a is tilted with respect to the screen 3a. Is 0 degrees, and the camera and the screen have the same center with each other.
The case where there is a difference in inclination between them is shown. Also, (c) of the figure is a state when viewed through arrows a and b in (a) and (b), and expresses a relative position of the coordinate system.

The screen coordinate system is a coordinate system of an image projected from the projector, and is independent of the movement of the head mounted display integrated with the infrared camera. Further, the camera coordinate system is a coordinate system indicating an imaging range of the camera, and is a coordinate system independent of the screen coordinate system as described above.

Therefore, an example of calculating the drawing center coordinates and the inclination of the display target screen will be described below. FIG. 3 is a flowchart illustrating a process for calculating the drawing center coordinates and the inclination of the display target screen. In step S1 of FIG.
Of the corners 10c, 10e, 10f, and 10d in the camera coordinate system are calculated based on the synchronization signal of the infrared camera 7a. In the following step S2, the inclination θ of both coordinate systems (see (c) of FIG. 2) is calculated from the coordinate values of the corner calculated in step S1.

In step S3, the base point of the camera coordinate system (in FIG. 2C, the upper left corner (0,
0)) X offset value X _of the Y offset value Y _of from
Is calculated. These have already been calculated as corner coordinate values. Then, in step S4, the screen coordinates corresponding to the center position of the imaging range 20 are calculated from the base point coordinates, the offset value, and the inclination.

Here, the camera coordinate system is (0, 0) to (m, n) as shown in the imaging range 20, and is a known value. The screen coordinate system has already been calculated as the value of the camera coordinate system at each corner. In step S5,
From these, the ratio between the two coordinate systems is calculated. That is, “drawing center coordinates” and “tilt” located at the center of the camera coordinate system
Are the “drawing center coordinates” and the “rotation angle” to be obtained.

The coordinates and the rotation angle calculated by the coordinate calculator 8a are sent to the central processing unit 9. Therefore, the central processing unit 9 updates the drawing instruction from the display coordinates of the screen coordinate system and the corrected rotation angle (the detected rotation angle is an angle of opposite polarity), and sends the drawing instruction to the head-mounted display image generator 4. Send out. The value of the magnification of the telescope can be made variable by obtaining information from a head-mounted display unit (not shown).

As described above, according to the present embodiment, the external image viewed through the head-mounted display 5 indicates the line of sight of the head-mounted display 5 and the horizontal axis to be displayed is the head axis. By operating in conjunction with the rotation of the mount display 5,
The axis can be made to coincide with the horizontal axis of the large screen display screen, and the image on the head mounted display 5 can be maintained in a normal state according to the inclination of the observer's head.

Further, in this embodiment, since the electric enlargement processing is performed as the pseudo telescope function, the object to be drawn can secure a high definition display and incorporate a zoom function as needed.

Further, according to this embodiment, in order to detect the line of sight of the head-mounted display 5,
By incorporating the infrared camera 7a on the head-mounted display side and incorporating an invisible light emitter having a unique code (signal) on the screen side, for example, a three-screen multi-screen, Irrespective of this, it is not necessary to add equipment for detecting the line-of-sight direction and the rotation angle of the head-mounted display 5.

Embodiment 2 Hereinafter, Embodiment 2 of the present invention will be described. In the first embodiment, an invisible light emitter is mounted on the screen side, and an infrared camera is mounted on the head-mounted display side in order to determine the line of sight of the head-mounted display 5 which is a pseudo telescope. However, the present invention is not limited to this configuration. When the screen has one or several screens and there is no angle for each screen, the infrared camera may be fixed with four axes of invisible light rays. .

FIG. 4 is a block diagram showing the overall configuration of the display control system according to the present embodiment. In the figure, the same components as those of the system according to the first embodiment shown in FIG. 1 are denoted by the same reference numerals. The system shown in FIG. 4 includes a head-mounted display 5 serving as a pseudo telescope, a 4-axis invisible light emitter 6a integrally formed with the head-mounted display 5, a screen 3, a fixed infrared camera 7, a head-mounted display Display 5
And a display coordinate calculator 8 for obtaining a screen coordinate system in the line of sight direction. Here, the four-axis invisible light (indicated by reference numeral 41 in FIG. 4) is coded in, for example, a blink cycle or the like.

Next, the operation of the system according to this embodiment will be described. The four-axis invisible light emitter 6a is integrally formed on the screen 3 in the line of sight of the head-mounted display 5 with the four-axis invisible light emitter 6a.
Four invisible lights 41 from a are projected and they are reflected from the screen as four lights. Also,
The infrared camera 7 has its imaging range adjusted in advance so as to match the screen 3.

For this reason, the infrared camera 7 detects four invisible lights 42 on the screen, converts the invisible lights into electric signals, and sends the converted signals to the display coordinate calculator 8 as video signals. . The display coordinate calculator 8 replaces four invisible light centers with a screen coordinate system based on the input video signal. The display coordinate calculator 8 calculates the rotation angle of the head-mounted display 5 from the imaging coordinate system and the inclinations of the four invisible lights.

The central processing unit 9 sends an updated drawing instruction to the head mounted display image generator 4 based on the center coordinates and the corrected rotation angle obtained by the above operation. Therefore, the head mounted display image generator 4 processes the update drawing instruction, and displays the processed image on the head mounted display 5 as a pseudo-telescope image. The magnification of the telescope can be made variable by obtaining information from a head-mounted display unit (not shown).

As described above, according to the present embodiment, four axes of invisible light directed in the same direction as the line of sight of the head mounted display 5 are projected on the screen, and the four invisible lights are projected. By calculating the rotation angle of the head-mounted display from the inclination of the imaging coordinate system and the inclination of the imaging coordinate system, regardless of the rotation angle of the head-mounted display 5 used by the observer, the horizontal angle of the external image viewed through the head-mounted display 5 The axis can always be kept constant.

Also, in the present embodiment, similarly to the first embodiment, since the electric enlargement process is performed as the pseudo telescope function, the object to be drawn is to secure a high-definition display and to zoom if necessary. Features can be included.

Embodiment 3 Hereinafter, Embodiment 3 of the present invention will be described. In the first and second embodiments, the invisible light emitter and the infrared camera are used for detecting the line of sight and the rotation angle of the head-mounted display serving as the pseudo telescope. However, the present invention is not limited to this. For example, an angle sensor may be used instead of the invisible light emitter and the infrared camera.

FIG. 5 is a block diagram showing the configuration of the display control system according to the present embodiment. In the figure, the same components as those of the system according to the first embodiment shown in FIG. 1 are denoted by the same reference numerals. As shown in FIG. 5, the system includes a head-mounted display 5 serving as a pseudo telescope, an angle sensor 11 integrated with the head-mounted display, and an angle sensor interface 12.

Next, the operation of the display control system according to the present embodiment will be described. Here, the coordinate system of the screen 3 and the angle sensor /
The interface 12 is matched so that the angle components can be converted into coordinate values. Also, the angle sensor 1
Reference numeral 1 sequentially updates the three-axis angle information following the movement of the head-mounted display 5. From this angle information, the angle sensor interface 12 obtains coordinate values indicating the line-of-sight direction of the head-mounted display 5 and the rotation angle of the head-mounted display 5 in a plane parallel to the surface of the screen 3, and places them in the center. It is sent to the processing device 9.

The central processing unit 9 sends an updated drawing instruction to the head mounted display image generator 4 based on the coordinate values and the corrected rotation angle. And the image generator 4 for the head mounted display is
The updated drawing instruction is processed, and the image obtained by the processing is displayed on the head-mounted display 5 as a pseudo-telescope image. The magnification of the telescope is
The information can be changed by obtaining information from a head-mounted display unit (not shown).

As described above, according to the present embodiment, following the movement of the head-mounted display 5, the line-of-sight of the head-mounted display 5 is determined based on the sequentially updated three-axis angle information. The coordinate value indicating the direction and the rotation angle are obtained, and the image for the head-mounted display is updated and displayed based on them, so that the head-mounted display 5 has a simplified system configuration.
Regardless of the rotation angle, the horizontal axis of the external image viewed through the head-mounted display 5 can always be kept constant.

Also, in this embodiment, similarly to the above-described embodiments, since an electric enlargement process is performed as a pseudo-telescope function, the object to be drawn is to secure a high-definition display and to perform zooming if necessary. It is possible to incorporate functions.

Further, even if the screen 3 is multiplied,
By previously matching the coordinate system of these screens with the head-mounted display 5, it is not necessary to newly add equipment for detecting the line-of-sight direction and the rotation angle of the head-mounted display 5. .

Embodiment 4 Hereinafter, Embodiment 4 of the present invention will be described. In the system according to the third embodiment, the movement of the head-mounted display 5 and the tracking accuracy of the screen displayed on the head-mounted display depend on the resolution of the angle sensor 11 used. Therefore, as Embodiment 4, a configuration for improving the accuracy of the angle sensor alone when the accuracy is insufficient is proposed.

FIG. 6 is a block diagram showing the configuration of the display control system according to the present embodiment. As shown in the figure, the system according to the present embodiment includes a uniaxial invisible light emitter 6, an infrared camera 7 for detecting invisible light 61 projected from the invisible light emitter 6 onto the screen 3, and a coordinate calculation. The device 8, angle sensor 11a, and angle sensor interface 12 are incorporated to achieve high-accuracy tracking.

In FIG. 6, the head-mounted display 5 is a pseudo telescope, and the uniaxial invisible light emitter 6 and the angle sensor 11a are integrated with the head-mounted display 5. Note that the angle sensor interface 12 calculates display coordinates in the screen coordinate system in the line-of-sight direction of the head-mounted display 5.

Next, the operation of the system according to the present embodiment will be described. The angle sensor 11a detects an angle in a plane parallel to the screen surface in conjunction with the movement of the head mounted display 5, and outputs the detection result to the angle sensor.
Send it to the interface 12. The angle sensor interface 12 determines the rotation angle of the head mounted display 5 and sends it to the central processing unit 9.

The reflection of the invisible light 61 from the uniaxial invisible light emitter 6 from the screen surface is captured by the infrared camera 7, and the coordinate calculator 8 calculates information based on the reflected light. The line-of-sight coordinates of the head-mounted display 5 are calculated, and the result is sent to the central processing unit 9.

The central processing unit 9 sends an updated drawing instruction to the head-mounted display image generator 4 based on the coordinates and the corrected rotation angle obtained in this way.
Therefore, the image generator 4 for head mounted display
Processes the update drawing instruction, and displays the image on the head-mounted display 5 as a pseudo telescope image. Note that, also here, the magnification of the telescope can be made variable by obtaining information from a head-mounted display unit (not shown).

As described above, according to the present embodiment, the rotation angle of the head mounted display 5 detected by the angle sensor interface 12 and the infrared camera 7
Light 61 from the uniaxial invisible light emitter 6 captured by
Of the head-mounted display 5 is calculated based on the reflected light from the screen surface, and the updated drawing instruction is sent to the head-mounted display image generator 4 from these coordinates and the corrected rotation angle. By sending
Regardless of the rotation angle of the head-mounted display 5, the horizontal axis of the external image viewed through the head-mounted display 5 can always be kept constant, and as a result, high-precision tracking can be realized.

Also, in this embodiment, since the electric enlargement process is performed as the pseudo telescope function, the object to be drawn can secure a high definition display and incorporate a zoom function as necessary. .

Embodiment 5 Hereinafter, a fifth embodiment of the present invention will be described. When the simulation display system is only a head-mounted display, the gaze direction and the inclination of the observer's head in a plane perpendicular to the gaze direction are not detected (for example, when a scene is displayed as an external display). Sometimes, the image viewed through the head-mounted display is exactly the same regardless of the direction of the observer, regardless of head movement.

FIG. 7 is a block diagram showing a configuration of the display system according to the present embodiment. The system shown in the figure has a head mounted display 5 for displaying an external world,
The angle sensor interface 12 that includes the angle sensor 11b, the angle sensor interface 12, and the like and functions as a coordinate calculator is assumed to have a display center coordinate of a screen coordinate system in a line-of-sight direction of the head-mounted display 5 and a screen. In this case, the rotation angle of the head-mounted display with respect to the screen is calculated.

Here, the angle sensor 11b is a three-axis angle sensor, and detects the roll angle, pitch angle, and yaw angle (indicated by reference numeral 71 in FIG. 7) from the movement of the observer's 19 head. The angle sensor updates the angle information in accordance with the movement of the head, and sends the updated information to the angle sensor interface 12.

The coordinates and the corrected rotation angle obtained in this way are sent to the central processing unit 9, where the drawing instruction is updated. The updated drawing instruction is sent to the central processing unit 9
From the head-mounted display 5. As a result, in response to the drawing instruction, the head mounted display 5 updates the display as an external image according to the movement of the observer's head.

As described above, according to the present embodiment, the coordinates of the display center in the line of sight of the head-mounted display 5 and the angle information of the movement of the observer's head in the three axes are used. By giving a drawing instruction to the head mounted display, in a narrow space etc.
Even when a display system is constructed using only a head-mounted display, an external image according to the movement of the observer's head can be displayed on the head-mounted display.

Embodiment 6 FIG. Hereinafter, Embodiment 6 of the present invention will be described. In the system according to the fifth embodiment described above, the display on the head-mounted display is made to correspond to the movement of the head using the three-axis angle sensor.
Here, we propose a display system that can obtain the same effect as moving the head without moving the head.

FIG. 8 is a block diagram showing the configuration of the display system according to the present embodiment. As shown in the figure, the system includes a head-mounted display 5, a line-of-sight sensor 13, a line-of-sight sensor interface 14, and a central processing unit 9.

FIG. 9 shows the display surface of the head-mounted display according to the present embodiment.
5, screen movement designation prohibited area 16, screen movement designation area 17
Consists of

Next, the operation of the present system will be briefly described. The movement of the eye 81 (specifically, the movement of the line of sight) of the observer 19 wearing the head-mounted display 5 on the head is always detected by the line-of-sight sensor 13. Then, when the observer moves his / her gaze in the direction of observation in the display area on the display 5a, the gaze sensor 13
Sends the movement to the gaze sensor interface 14 as data.

When the line-of-sight sensor interface 14 detects the line-of-sight position, it sends detection information to the central processing unit 9. The central processing unit 9 updates the drawing instruction based on the information,
The update result is sent to the head mounted display image generator 4. Then, the image generator 4 updates the drawing according to the updated drawing instruction, and sends it to the display 5a. As a result, the display of the external image is updated on the head-mounted display 5 according to the movement of the line of sight of the observer.

As described above, according to the present embodiment, the line-of-sight sensor 13 provided in the head-mounted display 5
By detecting the movement of the observer's line of sight and updating the drawing in accordance with the movement, the head mounted display 5 alone can be used to display an external image corresponding to the movement of the observer's line of sight even in a small space. Display and update can be performed.

Embodiment 7 FIG. Hereinafter, a seventh embodiment of the present invention will be described. FIG. 10 is a diagram showing a configuration of the display system according to the present embodiment. Here, a television camera is installed above the observer's head, and at the same time, the observer wears a head-mounted display on which a plate displaying a reference scale is mounted. And
The external display displayed on the head-mounted display is linked to the movement of the head.

More specifically, a television camera 7b is installed above the place where the observer 19 is located, for example, by suspending it from the ceiling in accordance with the position of the head. The head-mounted display 5 is mounted on the head. This head mounted display 5
The upper surface of itself (position where it can be seen more clearly than TV camera 7b)
As shown in FIG. 10, a reference graduation plate 18 provided with reference graduations formed of a plurality of radial straight lines and concentric circles.

A rotation angle calculator 8b is connected to the television camera 7b, and thereby calculates a rotation angle indicating the front direction of the head and the inclination of the head mounted display 5. Then, the output is sent to the central processing unit 9.
The image generator 4 for the head mounted display
Is, for example, the same as that in Embodiment 6 above.

Next, the operation of the display system according to the present embodiment will be described. As described above, the observer 19
Since the head-mounted display 5 is mounted on the head, the movement of the observer's own head appears as a tilt and a vertical movement of the reference scale plate 18 on the head-mounted display.

That is, when the observer moves his / her head, the scale displacement of the scale of the reference scale plate 18 and the azimuth displacement are generated with respect to the television camera 7b located above the scale. These displacements are captured by the camera and sent to the rotation angle calculator 8b as video information. The rotation angle calculator 8b calculates the direction and inclination of the front of the observer's head based on the information, and sends the calculation result to the central processing unit 9.

The central processing unit 9 updates the drawing instruction of the head-mounted display image generator 4 according to the above calculation result. Head mounted display 5
Performs video display according to the drawing instruction updated in this manner, so that the external display to the observer is linked to the movement of the head.

As described above, according to the present embodiment, the observer wears the head mounted display on which the plate on which the reference scale is displayed is mounted on his or her own head, and further, the upper part of the head mounted display. By capturing the displacement of the reference scale and the azimuth displacement of the reference scale with the TV camera installed in the, and calculating the direction and inclination of the front of the observer's head, even in a narrow space, etc., in conjunction with the movement of the head, The external display displayed on the head-mounted display can be updated.

Embodiment 8 FIG. Hereinafter, an eighth embodiment of the present invention will be described. FIG. 11 is a diagram showing a configuration of the display system according to the present embodiment. As shown in the figure,
Here, the observer enters the cylinder displaying the reference scale inside, and the television camera fixed to the head mounted display under the situation where the television camera is installed above the observer's head. Read the scale. Then, the position of the front of the observer is detected, and the external display displayed on the head-mounted display is linked with the movement of the observer's head.

More specifically, the television camera 7a fixed to the head-mounted display 5 always has a reference scale 18b on the inner surface of a cylinder 18a located on the front surface thereof.
When the observer 19 moves his or her head, the movement is determined by the vertical displacement and the horizontal displacement of the scale, and a reference line (a specific line among a plurality of reference scales 18b is set as a reference line). )). Note that the reference scale 18b is a scale composed of a plurality of horizontal lines and vertical lines that covers the observer 360 degrees in a cylindrical shape.

The displacement information is sent from the television camera 7a to the coordinate calculator 8a connected to the television camera 7a. The coordinate calculator 8 calculates the displacement information based on the displacement information from the front side of the observer's head and the head mounted display. 5 is detected. That is, the coordinate calculator 8 a sends information indicating the front direction of the head of the observer 19 to the central processing unit 9.

The central processing unit 9 creates drawing update information based on the direction information from the coordinate calculator 8a and sends it to the head mount display image generator 4 to thereby generate the head mount display 5 Update the drawing instruction for. As a result, the external display displayed on the head-mounted display 5 is linked to the movement of the observer's head.

As described above, according to the present embodiment, the observer enters the cylinder on which the reference scale is displayed, and at the same time, the observer puts the head mounted display of the television camera on the head. By mounting the observer's head as the displacement of these scales in the vertical and horizontal directions, it can be linked to the movement of the head even in a narrow space where a large system cannot be installed. Thus, the external display displayed on the head-mounted display can be updated.

[0095]

As described above, according to the first aspect of the present invention, the whole or a part of the visual field image projected on one or more screens by one or more projectors is simulated as a simulated image. In a display control device for displaying on a telescope, instructing means for optically instructing each corner of the screen, detecting means for detecting the instruction, and obtaining first coordinates of the visual field image based on the detection result. Means, means for obtaining second coordinates for the pseudo image based on the detection result, calculation of the inclination between the coordinate systems indicated by the first and second coordinates, and calculation of the center coordinates of the second coordinates Means for correcting the drawing of the pseudo image based on the inclination and the center coordinates, and means for sending the corrected pseudo image to the projector. , Regardless of the slope of the pseudo telescope, the image of the pseudo telescope can be maintained in the normal state, as a result, can hold levelness of external display constant. Further, it is possible to suppress an increase in hardware due to the number of screens.

Since the instructing means is an invisible light emitter that emits light with a specific period, it is easy to identify individual screens and distinguish them from other screens. In addition, since the detection means and the pseudo telescope have an integral structure, the line of sight of the pseudo telescope can be reliably detected.

According to the second aspect of the present invention, in the display control device for displaying the whole or a part of the visual field image projected on the screen from the projector as a pseudo image on the pseudo telescope, the means for projecting an invisible light beam on the screen When,
Detecting means for detecting the reflected light of the invisible light from the screen, means for obtaining the center coordinates of the invisible light from the detection result, and coordinates of the pseudo image and the invisible light based on the center coordinates. Means for calculating a tilt, correction means for performing drawing correction of the pseudo image based on the tilt and the coordinates, and means for sending the corrected pseudo image to the projector. Irrespective of the tilt of the telescope, by performing drawing correction so that the horizontal direction of the visual field image and the corrected pseudo image always coincide, the horizontal axis of the external image viewed through the pseudo telescope can always be kept constant.

Since the invisible light beam is a four-axis invisible light beam that emits light with a natural period, the coordinates of the pseudo image can be reliably obtained based on the center coordinates.

Further, since the direction of the line of sight of the pseudo telescope and the direction of projection of the invisible light beam are the same, it is easy to calculate the corrected rotation angle of the pseudo telescope.

According to the third aspect of the present invention, in a display control device for displaying a whole or a part of a visual field image projected on a screen by a projector as a pseudo image on a pseudo telescope, the coordinates of the pseudo image are set with respect to the visual field image. Means for replacing the coordinate system with the coordinate system described above, angle detecting means for obtaining angle information of the three axes of the pseudo telescope based on the result of the replacement, and a coordinate value and a rotation angle indicating the line of sight of the pseudo telescope based on the angle information. Calculating means for performing drawing correction of the pseudo image based on the coordinate values and the rotation angle, and means for sending the corrected pseudo image to the projector, whereby the tilt of the pseudo telescope is increased. Irrespective of the above, drawing correction can be performed so that the horizontal axis of the visual field image always coincides with the horizontal axis of the corrected pseudo image.

Since the angle detecting means and the pseudo telescope have an integral structure, the line of sight of the pseudo telescope can be reliably detected.

The display control device according to the third aspect of the present invention further comprises: a light projecting means for projecting a uniaxial invisible light beam;
Detecting means for detecting the reflected light of the invisible light from the screen, and means for calculating the line-of-sight coordinates of the pseudo-telescope from the detection result, based on the line-of-sight coordinates and rotation angle, the While holding the horizontal axis normally,
It is possible to perform a drawing correction of a pseudo image having a high-precision tracking performance.

Since the angle detecting means, the light projecting means, and the pseudo telescope have an integral structure, the line of sight of the pseudo telescope can be reliably detected.

According to the fourth aspect of the present invention, in a display control device for displaying a pseudo image on a built-in display, a detecting means for detecting the angle of the display control device in the three-axis direction, Means for determining the display center coordinates of the pseudo image with respect to the predetermined plane and the rotation angle of the display control device; correction means for performing drawing correction of the pseudo image based on the display center coordinates and the rotation angle; Means for sending the pseudo-image to the display device, the external image can be displayed according to the movement of the display control device itself.

Since the angles in the three axial directions are the roll angle, the pitch angle, and the yaw angle, the movement of the display control device can be reliably obtained.

According to the fifth aspect, in a display control device for displaying a pseudo image on a built-in display, an instruction means for instructing the display control device to tilt and move up and down, and converting the instruction result into video information Converting means for calculating the tilt and rotation angle of the display control device based on the video information; correcting means for performing drawing correction of the pseudo image based on the tilt and rotation angle; Means for sending the subsequent pseudo image to the display device, so that a pseudo image that is faithful to the movement of the display control device itself can be displayed.

Since the indicating means is a reference scale composed of a plurality of radial straight lines and concentric circles, the movement of the display control device can be grasped as an interval displacement and an azimuth displacement of the reference scale.

Further, since the indicating means and the display control device are integrated, and the conversion means is located above the central axis of the concentric circle, the deformation of the reference scale on the image is controlled by the display control device. It can be determined as a movement.

Further, since the indicating means is a scale composed of a plurality of horizontal lines and vertical lines covering all directions in a cylindrical shape, it is possible to immediately determine the deformation of the scale on the image as the movement of the display control device. .

Further, since the conversion means and the display control device have an integral structure, and the conversion means is located near the center of the cylinder, no matter which direction the display control device faces, Movement can be grasped reliably.

Further, since the display control device according to the fifth aspect of the present invention is mounted on the head of the observer and linked with the movement of the head, it is possible to update the external display corresponding to the movement of the head. Becomes

According to the sixth aspect of the present invention, there is provided a display control device for displaying a pseudo image on a built-in display, means for detecting a line of sight of an observer, and movement of the line of sight in a predetermined display area of the display. And a correction unit that performs drawing correction of the pseudo image based on the movement of the line of sight,
The provision of the means for sending the corrected pseudo image to the display device makes it possible to display and update an external image according to the movement of the line of sight instead of the movement of the observer's head.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an overall configuration of a display control system according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a concept of coordinate calculation according to the first embodiment.

FIG. 3 is a flowchart showing a process of calculating drawing center coordinates and a tilt of a display target screen according to the first embodiment;

FIG. 4 is a block diagram showing an overall configuration of a display control system according to Embodiment 2 of the present invention.

FIG. 5 is a block diagram showing a configuration of a display control system according to Embodiment 3 of the present invention.

FIG. 6 is a block diagram showing a configuration of a display control system according to Embodiment 4 of the present invention.

FIG. 7 is a block diagram showing a configuration of a display system according to Embodiment 5 of the present invention.

FIG. 8 is a block diagram showing a configuration of a display system according to Embodiment 6 of the present invention.

FIG. 9 is a diagram showing a display surface of a head mounted display according to a sixth embodiment.

FIG. 10 is a diagram showing a configuration of a display system according to Embodiment 7 of the present invention.

FIG. 11 is a diagram showing a configuration of a display system according to Embodiment 8 of the present invention.

FIG. 12 is a block diagram illustrating a system configuration example of a conventional simulation display system.

[Explanation of symbols]

1, 1a, 1b image generator, 2, 2a, 2b projector, 3 left screen, 3a central screen, 3
b ... Right screen, 4 ... Image generator for head mounted display, 5 ... Head mounted display,
6, 6a, 10a to 10h: invisible light emitter, 7, 7
a: infrared camera, 7b: television camera, 8, 8a: coordinate calculator, 8b: rotation angle calculator, 9: central processing unit, 1
Reference numerals 1, 11a: angle sensor, 12: angle sensor interface, 13: line-of-sight sensor, 14: line-of-sight sensor interface, 15: outermost periphery of visual field, 16: screen movement designation prohibited area, 17: screen movement designation area, 18: reference Scale plate, 18a: cylinder, 18b: reference scale, 19: observer,
20 ... imaging range

Claims (19)

[Claims] 1. A display control device for displaying a whole or a part of a view image projected on one or more screens by one or more projectors as a pseudo image on a pseudo telescope, comprising: Instruction means for optically instructing; detecting means for detecting the instruction; means for obtaining first coordinates for the visual field image based on the detection result; and Means for obtaining coordinates of 2; inclination between the coordinate systems indicated by the first and second coordinates;
Means for calculating the center coordinates of the second coordinates; correction means for performing drawing correction of the pseudo image based on the inclination and the center coordinates; and means for sending the corrected pseudo image to the projector. The correction means, regardless of the inclination of the pseudo telescope,
A display control device that performs drawing correction so that the horizontal direction of the visual field image always matches the horizontal direction of the corrected pseudo image. 2. The apparatus according to claim 1, wherein the indicating means is an invisible light emitter that emits light with a natural period.
The display control device according to the above. 3. The display control device according to claim 1, wherein said detection means and said pseudo telescope have an integral structure. 4. A display control device for displaying a whole or a part of a visual field image projected on a screen by a projector as a pseudo image on a pseudo telescope, comprising: means for projecting an invisible light beam on the screen; Detecting means for detecting the reflected light from the screen; means for obtaining the center coordinates of the invisible light from the detection result; means for obtaining the coordinates of the pseudo image and the inclination of the invisible light based on the center coordinates Correction means for performing drawing correction of the pseudo image based on the inclination and coordinates, and means for sending the corrected pseudo image to the projector, wherein the correction means adjusts the inclination of the pseudo telescope. regardless of,
A display control device that performs drawing correction so that the horizontal direction of the visual field image always matches the horizontal direction of the corrected pseudo image. 5. The display control device according to claim 4, wherein the invisible light beam is a four-axis invisible light beam that emits light with a natural period. 6. The visual direction of the pseudo-telescope and a projection direction of the invisible light beam are the same.
The display control device according to the above. 7. A display control device for displaying a whole or a part of a visual field image projected on a screen by a projector as a pseudo image on a pseudo telescope, wherein coordinates of the pseudo image are replaced with a coordinate system for the visual field image. Means, angle detecting means for obtaining three-axis angle information of the pseudo telescope based on the replacement result, means for obtaining a coordinate value indicating a line of sight of the pseudo telescope and a rotation angle from the angle information, and the coordinates Correction means for performing drawing correction of the pseudo image based on the value and the rotation angle, and means for sending the corrected pseudo image to the projector, wherein the correction means is provided regardless of the inclination of the pseudo telescope. ,
A display control device that performs drawing correction so that the horizontal direction of the visual field image always matches the horizontal direction of the corrected pseudo image. 8. The display control device according to claim 7, wherein said angle detecting means and said pseudo telescope have an integral structure. 9. A light projecting means for projecting a uniaxial invisible light beam, a detecting means for detecting the reflected light of the invisible light beam from the screen, and calculating a line-of-sight coordinate of the pseudo telescope from the detection result. 8. The display control device according to claim 7, further comprising: a correction unit configured to perform the drawing correction of the pseudo image based on the line-of-sight coordinates and the rotation angle. 10. The display control device according to claim 9, wherein the angle detecting means, the light projecting means, and the pseudo telescope have an integral structure. 11. A display control device for displaying a pseudo image on a built-in display, detecting means for detecting angles in three axial directions of the display control device, and detecting the pseudo image with respect to a predetermined plane in a space based on the detection result. Means for determining the display center coordinates of the image and the rotation angle of the display control device; correction means for performing drawing correction of the pseudo image based on the display center coordinates and the rotation angle; and displaying the corrected pseudo image on the display A display control device for sending to a display device. 12. The apparatus according to claim 1, wherein the angles in the three axial directions are a roll angle, a pitch angle, and a yaw angle.
2. The display control device according to 1. 13. A display control device for displaying a pseudo image on a built-in display, an instruction means for instructing the display control device to tilt and move up and down, a conversion means for converting the instruction result into video information, Means for calculating the tilt and rotation angle of the display control device based on the video information; correction means for performing drawing correction of the pseudo image based on the tilt and rotation angle; and A display control device for transmitting the data to a display device. 14. The display control device according to claim 13, wherein said indicating means is a reference scale composed of a plurality of radial straight lines and concentric circles. 15. The display control device according to claim 14, wherein the indicating means and the display control device have an integral structure, and the conversion means is located above a central axis of the concentric circle. 16. The display control device according to claim 13, wherein said indicating means is a scale composed of a plurality of horizontal lines and vertical lines covering all directions in a cylindrical shape. 17. The display control device according to claim 16, wherein said conversion means and said display control device have an integral structure, and said conversion means is located near the center of said cylinder. 18. The display control device according to claim 11, wherein the display control device is mounted on a head of an observer and interlocks with movement of the head. 19. A display control device for displaying a pseudo image on a built-in display, wherein: means for detecting a line of sight of an observer; means for detecting movement of the line of sight in a predetermined display area of the display; A display control device comprising: a correction unit that performs drawing correction of the pseudo image based on the movement of the image; and a unit that sends the corrected pseudo image to the display.