JP2009146140A - Three-dimensional image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display device which displays three-dimensional images by which a user (viewer) can recognize an object instinctively and realistically. <P>SOLUTION: The image display device (100) which shows three-dimensional images of an object includes a vision point detecting means (21) that detects a virtual vision point of the viewer, a perspective arithmetic processing means (23) which generates the three-dimensional images of the object which can be viewed when the object is viewed from such virtual vision point based on the detected virtual vision point, and a means (15) for displaying the generated three-dimensional images on a display device. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an apparatus that displays a three-dimensional image, and more particularly, to an apparatus that displays a three-dimensional image according to a viewer's viewpoint position.

  In the world of computer graphics (CG), display of a three-dimensional (3D) image is a general technique used as a means for expressing a virtual world. For example, three-dimensional images are displayed in various applications in a practical field such as CAD (Computer Aided Design) and a car navigation system, and in an entertainment field such as a movie, animation, and a computer game.

  Conventionally, when displaying a three-dimensional image in CAD or a game, the user changes the viewpoint position by operating a keypad, a mouse, or the like, so that the three-dimensional image of the object viewed from the viewpoint position is displayed. In the 3D image display, the viewpoint position is one of the important factors, and various techniques for changing the image according to the viewpoint position have been devised (see, for example, Patent Document 1).

JP 2000-076481 A

  On the other hand, in the world of computer games, an interface has been developed that allows users to operate intuitively by tilting or vibrating a device that incorporates a sensor without operating a keypad as in the past. ing. Such an interface is taken in as an interface for a mobile phone game, for example, and allows a user to operate the game by inclining or vibrating the mobile phone and proceeding with the game.

  In a game using the interface as described above, the user can operate without using a keypad or a mouse, and can perform an operation closer to a human sense. In the future, such games are expected to become increasingly popular. Accordingly, there is a demand for a display technique that allows the user to recognize an object more intuitively and realistically for a three-dimensional image displayed on the screen.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an image display device that displays a three-dimensional image that allows a user to recognize an object intuitively and realistically.

  A three-dimensional image display device according to the present invention is a device that displays a three-dimensional image of an object, based on viewpoint position detection means for detecting a viewer's virtual viewpoint position, and the detected virtual viewpoint position, A parsing calculation processing unit that generates a three-dimensional image of the object that is visible when the object is viewed from the virtual viewpoint position, and a unit that displays the generated three-dimensional image on a display device.

The image display program according to the present invention is a computer-readable program that performs control to display a three-dimensional image of an object, and causes the computer to execute the following procedure.
-Procedure for receiving information on the viewer's position via the input means-Procedure for detecting the viewer's virtual viewpoint position by the calculation means based on the received information on the viewer's position-Based on the detected viewpoint position Procedure for generating a three-dimensional image of an object that can be seen when the object is viewed from the viewpoint position by the calculation means-Procedure for displaying the generated three-dimensional image on the display means

  According to the present invention, a virtual viewpoint position of a user (viewer) is detected, and a three-dimensional image that can be seen when viewed from the viewpoint position is displayed in real time, so that the user actually has an object on the side of the display surface. The object can be recognized more intuitively and realistically.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

(First embodiment)
1. Configuration of Image Display Device FIG. 1 shows the configuration of the image display device according to the first embodiment of the present invention. In FIG. 1, the image display device 100 detects and detects the viewer (user) and an image capturing device 10 that captures an image in the vicinity thereof, and the viewer's viewpoint position (virtual viewpoint position) from the captured image. The image processing apparatus 11 which produces | generates the three-dimensional image (henceforth a "3D image") which should be displayed based on a viewpoint position, the display apparatus 14 which displays a 3D image, and the operation part 19 which a viewer operates are provided. . In the following description, the horizontal direction of the display surface of the display device 14 is the x direction, the vertical direction is the y direction, and the depth direction (the direction toward the eyelid of the screen when viewed from the viewer side) is the z direction.

  The image processing device 11 includes an interface 13 that receives image information from the imaging device 10, a control unit 15 that performs generation and display processing of an image to be displayed, and data that stores programs executed by the control unit 15, image data, and the like. And a storage unit 17. The control unit 15 is constituted by a CPU, for example, and realizes various functions to be described later in cooperation with a predetermined control program in the data storage unit 17. The control unit 15 detects the viewer's viewpoint position (virtual viewpoint position) from the captured image including the viewer (user) and images in the vicinity thereof, and updates the viewpoint position calculation processing unit 21 to the latest one, A parsing calculation processing unit 23 for obtaining data of a 3D image to be displayed. The operation unit 19 is a keypad or a keyboard including operation buttons for a viewer to perform an operation, but may be a mouse.

  The data storage unit 17 includes data storage means such as a ROM, a hard disk, and an optical disk. The data storage unit 17 stores a control program 31 executed by the control unit 11, coordinate data 32 (details will be described later) of a 3D object to be displayed, and a coordinate conversion table 33 (details will be described later).

  The imaging device 10 includes a lens and an imaging device that converts an optical signal incident through the lens into an electrical signal, and outputs captured image information to the image processing device 11 as an electrical signal. The imaging device 10 transmits image information to the image processing device at predetermined time intervals. The display device 14 includes a liquid crystal display (LCD) and the like, and displays an image generated by the image processing device 11. The image display apparatus 100 according to the present embodiment is an information processing device such as a mobile phone with a camera or a personal computer (PC) to which a camera and a display are connected.

2. 3D image display according to viewpoint position The image display apparatus 100 according to the present embodiment has a function of displaying a three-dimensional image on the display apparatus 14, and in particular, a viewer's viewpoint position (virtual position from an image obtained by capturing the viewer. (Viewpoint position) is detected in real time, and a three-dimensional image predicted to be seen when viewed from the viewpoint position is displayed in real time. Hereinafter, the display of such a three-dimensional image will be specifically described with reference to FIG.

  For example, when it is determined that the viewer's viewpoint position is at the front position (hereinafter referred to as “reference position”) with respect to the center of the display surface of the display device 14, an object (display target) is displayed on the display device 14. An image that is visible when 50 is viewed from the front is displayed (see FIG. 2A). Further, when it is determined that the viewer's viewpoint is slightly on the left side with respect to the front position (reference position), an image of an object that is normally expected to be viewed at the viewpoint position is displayed. That is, as shown in FIG. 2B, the image is displayed so that the right side surface of the object 50 can be seen. Similarly, when it is determined that the viewpoint position is at a position shifted to the upper left with respect to the front position, an image is displayed so that the right side surface and the upper surface of the object 50 can be seen as shown in FIG. As described above, the image display apparatus according to the present embodiment displays a three-dimensional image corresponding to the viewpoint position of the viewer, whereby the viewer can view the viewer in the depth direction of the display surface of the display device 14 (the viewer sandwiching the display surface) The object can be visually recognized as if the object actually exists on the other side.

  Here, with reference to FIG.3 and FIG.4, it demonstrates more concretely about the three-dimensional image displayed according to a viewer's viewpoint position.

  As shown in FIG. 3, the viewpoint coordinate plane 200 is set at a position separated by a predetermined distance (d) parallel to the display surface 210 of the display device 14, and the coordinate system is set on the viewpoint coordinate plane 200. Using the coordinates of the coordinate system, the viewer's viewpoint position on the viewpoint coordinate plane 200 is represented. Then, the shape of the image of the object displayed on the display surface 210 is changed according to the viewpoint position on the viewpoint coordinate plane 200.

  For example, in FIG. 4, the viewpoint position Va is the center position (reference position) of the viewpoint coordinate plane 200. In this case, the viewer faces the display plane 210 of the display apparatus. Is displayed from the front. The viewpoint position Vb is a position shifted by −Δx from the viewpoint position Va in the x direction. In this case, the viewer views the screen of the display device 14 from a position slightly shifted to the left side with respect to the center of the display surface 210. Become. For this reason, the display device 14 displays an image B when the object is viewed from the left slightly with respect to the front. Further, the viewpoint position Vc is a position that is shifted by −Δx in the x direction and −Δy in the y direction from the viewpoint position Va. In this case, the viewer is moved from a position slightly shifted to the upper left with respect to the center of the display surface 210. The screen of the display device 14 is seen. For this reason, the image C when the object is viewed from the upper left side with respect to the front is displayed on the display device 14.

  The image display device 100 stores the object coordinate data 32 and the coordinate conversion table 33 in the data storage unit 17 in order to display the 3D image viewed from each viewpoint position as described above.

  In particular, in order to display a 3D image corresponding to the viewpoint position, the image display apparatus 100 uses, as object coordinate data 32, the three-dimensional coordinate data of the image seen when viewed from each viewpoint for each of a plurality of viewpoint positions. Storing. That is, the object coordinate data includes object coordinate data (three-dimensional coordinate data of each vertex constituting the object) seen when viewed from a certain viewpoint. For example, the object coordinate data for the viewpoint position Vb shown in FIG. 4 includes the coordinate data of each vertex constituting the object that looks like the image B. It goes without saying that the coordinate data of each vertex constituting the object that looks like the image B is different from the coordinate data of each vertex that constitutes the object that looks like the image A.

  The coordinate conversion table 33 is a table that manages the viewpoint position (coordinates) and the image data (path name + file name) of the object displayed at the viewpoint position in association with each other. FIG. 5A shows a data configuration example of the coordinate conversion table 33. By referring to the coordinate conversion table 33, if the viewpoint position is determined, 3D image data to be displayed at that time can be obtained. For example, when the viewpoint position is V2, the image data to be displayed at that time is “image 0003.DAT”. In addition, instead of the coordinate conversion table 33 using the image coordinate data including the vertex coordinate data of the 3D image as shown in FIG. 5A, an image composed of pixel data such as a bitmap or TIFF format. It is also possible to use a display image table 34 that associates data with viewpoint positions. (See FIG. 5 (b)).

3. Operation of Image Display Device The operation of the image display device 100 will be described with reference to the flowchart of FIG. Note that the following processing is executed until there is a display termination request (instruction) by the viewer (S11). First, an image of the viewer and its vicinity is captured by the imaging device 10 (S12). Information on the captured image is sent to the parse calculation processing unit 23 of the control unit 15. Note that during the operation of the image display apparatus 10, the lens of the imaging apparatus 10 is directed toward the viewer or the vicinity thereof.

  The control unit 15 determines whether or not a reset request from the viewer has been received (S13). The reset request is transmitted from the operation unit 19 to the control unit 15 when the viewer presses the reset button on the operation unit 19.

  When the reset request is received, the control unit 15 resets the viewpoint position to the reference position (S14). In the present embodiment, the reference position of the viewpoint position is the center position of the viewpoint coordinate plane 200, but is not limited to this. When the reset request has not been received, the control unit 15 performs a viewpoint position update process based on the captured image information (S15). Details of the viewpoint position update processing will be described later.

  Thereafter, the parse calculation processing unit 23 performs a parse calculation process for generating or acquiring 3D image data of an object to be newly displayed based on the set viewpoint position (S16). Details of the parsing calculation process will be described later. And the control part 15 transmits 3D image data to the display apparatus 14, and displays a 3D image (S17).

  Details of the viewpoint position update processing (S15) will be described with reference to the flowchart of FIG. In the viewpoint position update process, when the viewpoint position calculation processing unit 21 receives data of the captured image from the imaging apparatus 10 (S21), the whole captured image part or a part of the characteristic part of the image captured this time is compared with the image captured this time. Then, the moving amount of the image is detected based on the difference between the images (S22). As an example of such a detection technique, for example, there is a motion track technique (GestureTek EyeMobaile) of GestureTek Inc. For example, when movement of the captured image from position A to position A ′ is detected as shown in FIG. 8A, the movement amounts Δx and Δy are obtained (see FIG. 8B). When the movement amounts of the captured image 220 are Δx and Δy, the movement amounts on the viewpoint coordinate plane 200 are −Δx and Δy. Therefore, a new viewpoint Va ′ is obtained by shifting the previous viewpoint Va by −Δx in the x direction and Δy in the y direction (see FIG. 8C). Then, a new viewpoint position is set based on the movement amount obtained as described above (S23). As described above, the viewpoint position can be updated based on the captured image.

  Next, details of the parsing calculation process (S16) will be described with reference to the flowchart of FIG. In the parsing calculation process, first, the coordinate conversion table 33 is referred to, and the coordinate data of the object corresponding to the viewpoint position set in step S14 or S15 is acquired (S31). Then, 3D image data is generated based on the acquired coordinate data of the object (S32).

  In the above example, coordinate data of an object viewed from a plurality of viewpoints is prepared in advance and managed by the coordinate conversion table 33 for one object. However, the coordinate conversion table 33 is not necessarily used. In the parse calculation processing unit 23, the 3D coordinate data of each vertex constituting the object can also be obtained by calculation based on the coordinates of the viewpoint position at that time. Hereinafter, a method for calculating the 3D coordinate data will be described.

  Assume that the 3D image display of the object 50 as shown in FIG. FIG. 11A shows an object 50 that is displayed when the viewpoint position is V. FIG. As shown in the figure, in addition to the front image of the object 50, the 3D image is expressed by displaying the right side image and the upper surface image together. In this case, how to obtain the coordinates of the vertices P, Q, and R constituting the 3D image will be described below.

FIG. 11B is a diagram illustrating the positional relationship between the vertices when the object 50 is viewed from the left side, that is, from the B-C line side in the 3D space. FIG. 11C illustrates the positional relationship between the vertices when the object 50 is viewed from directly above, that is, from the AB line side in the 3D space. From the relationship of FIG. 11, the x coordinate of the vertex P of the object 50 when viewed from the viewpoint V is the straight line connecting the point P and the point V and the line segment AB on the z-x plane of FIG. The x coordinate (P′x) of the intersection coordinate P ′ is obtained. Further, the y coordinate of the vertex P of the object 50 when viewed from the viewpoint V is the intersection coordinate P ′ of the straight line connecting the point P and the point V and the line segment BC on the yz plane of FIG. Y coordinate (P'y). Specifically, it is obtained by the following formula.
P'x = (Pz ・ Vx-Px ・ Vz) / (Pz-Vz) (1a)
P'y = (Pz ・ Vy-Py ・ Vz) / (Pz-Vz) (1b)

  Here, the original coordinates of the vertex P of the object 50 are (Px, Py, Pz), and the coordinates of the viewpoint V are (Vx, Vy, Vz). The other vertices Q and R can be obtained in the same manner. As described above, based on the coordinates of the viewpoint position V, each vertex of the 3D image obtained when viewed from the viewpoint position can be calculated.

  As described above, according to the image display device of the present embodiment, the virtual viewpoint position of the viewer is estimated based on the image obtained by capturing the viewer, and the 3D image to be displayed is displayed in real time according to the viewpoint position of the viewer. By changing the image, it is possible to display an image that is actually seen from the viewpoint position. Thereby, the viewer can recognize the object as if the object actually exists in the depth direction of the display surface, and can easily and accurately grasp the shape of the entire object.

  In the present embodiment, the viewpoint position is determined based on the change amount of the image captured by the imaging device 10, but the imaging device is not essential. Instead of the imaging device, an acceleration sensor that detects the movement of the display surface of the display device 14 or a mechanical mechanism such as a leveler or a gear may be used. By detecting the change in the angle of the display surface of the display device 14 based on the output from these sensors or the like, the change amount (Δθ) in the x direction and the change amount (Δφ) in the y direction of the viewpoint position can be detected. The viewpoint position can be specified. A plurality of imaging devices 10 may be provided. Thereby, the movement amount of the viewpoint position in the depth direction (z direction) can also be detected based on the captured image. In this case, the viewpoint coordinates are managed three-dimensionally in the coordinate conversion table 33, or the vertex coordinates of the object are calculated using the above formulas (1a) and (1b) to display the new viewpoint position. A power 3D image can be obtained.

(Second Embodiment)
In the first embodiment, the captured image is used to estimate the viewer's viewpoint position. In the present embodiment, an example will be described in which a position detection device that detects a standing position of a viewer with a sensor in order to estimate a viewpoint position.

  FIG. 12 is a diagram illustrating a configuration of an image display device according to the second embodiment. The image display apparatus 101 of the present embodiment is different from the configuration of the image processing apparatus of the first embodiment in that a position detection device 12 is provided instead of the imaging device 10.

  The position detection device 12 detects the standing position of the viewer using a sensor. As the sensor, an infrared sensor, an electrostatic induction switch, a mechanical switch, or the like can be used. FIG. 13A shows an example of the configuration of the position detection device 12. In the position detection device 12 shown in the figure, a plurality of sensors 12b for detecting the position of the viewer are arranged one-dimensionally in the x direction, and a position detection signal indicating the position of the viewer is output. The viewpoint position calculation processing unit 21 updates the viewpoint position based on the position detection signal from the position detection device 12. Subsequent processing is the same as that described in the first embodiment.

  FIG. 13B shows an application example of the image display apparatus 101 of the present embodiment. The display device 14 is arranged in a shop show window, and the position detection device 12 is embedded in a road or a wall in front of the store. When the position of the passerby (viewer) is detected by the position detection device 12, the image processing device 11 calculates a viewpoint position based on the detected position, and displays an image obtained when observing from the viewpoint position. To display. Accordingly, for example, when the passerby moves from position A to position A ′ in FIG. 13B, the 3D image displayed on the display device 14 changes as shown in FIG. 13C. By changing the image as the passerby moves in this manner, the passerby can recognize the object as if it existed in the depth direction (z direction) of the screen of the display device 14.

  In the above example, the position detection device 12 detects the position in the x direction, but may detect the position in the y direction or the z direction. In the position detection device 12, a plurality of sensors 12b may be two-dimensionally arranged. Or you may comprise the position detection apparatus 12 so that a viewer's position can be detected three-dimensionally. In any case, a 3D image corresponding to each of a plurality of viewpoint positions is prepared in advance, or the coordinates of the 3D image viewed from each viewpoint position are calculated each time the viewpoint position is updated. Thus, it is possible to display a 3D image corresponding to the viewpoint position.

  In addition, when the output value of each sensor 12b takes three or more values, each sensor 12b is installed so that the detection ranges overlap between adjacent sensors, and the value detected by each sensor 12b is averaged. The position of the viewer may be detected based on the above.

(Third embodiment)
In the above-described embodiment, the case of 3D display of a still image has been described. However, in this embodiment, the case of 3D display when an object has movement will be described. The configuration of the image display apparatus in this embodiment is the same as that shown in FIG. 1 or FIG.

  The parse calculation processing unit 23 in the image display apparatus according to the present embodiment generates an image corresponding to the viewpoint position detected at the time when displaying each image displayed in time series constituting the moving image. By such a method, even in the case of a moving image, it is possible to display a 3D image viewed from the viewpoint position of the viewer, as in the case of a still image.

  FIG. 14 is a diagram illustrating images displayed in time series when an object that rotates clockwise around an axis parallel to the z axis in the xy plane is viewed from the reference position. That is, a moving object moving image can be realized by sequentially displaying images (still images) of objects # 0 to # 11 at a predetermined frame rate.

  The 3D display when viewed from the viewpoint position shifted to the upper right side from the reference position is an image when viewed from the viewpoint position shifted to the upper right side from the reference position as shown in FIG. This can be realized by displaying.

  Various tools have been conventionally known as tools for drawing moving images of 3D images. For example, “RenderWare” is known. Further, “OpenGL”, “DirectX” and the like are known as drawing libraries. When a 3D image is drawn using these tools, the latest viewpoint position information is given as a camera position parameter, so that an image corresponding to the viewpoint position can be drawn.

  Since the present invention can display a 3D image that can be observed from the viewpoint position based on the viewpoint position of the viewer, the viewer can visually recognize the 3D image at a more natural interval. Therefore, the present invention can be applied to various technical fields that perform virtual 3D display such as displays used in various computer games, events, showrooms, exhibitions, and the like.

The figure which shows the structure of the image display apparatus of the 1st Embodiment of this invention. The figure explaining the example of a display of the three-dimensional image displayed according to a viewpoint position by the image display apparatus of embodiment of this invention. Illustration for explaining the viewpoint coordinate plane The figure explaining the three-dimensional image displayed according to a viewer's viewpoint position (A) The figure explaining the coordinate conversion table, (b) The figure explaining the display image table The flowchart which shows operation | movement of the image display apparatus of 1st Embodiment. Flow chart of viewpoint position update processing Diagram explaining how to detect the movement of the viewpoint Flow chart of parse calculation processing Illustration for explaining how to calculate the vertex coordinates of an object after moving the viewpoint Illustration for explaining how to calculate the vertex coordinates of an object after moving the viewpoint The figure which shows the structure of the image display apparatus of the 2nd Embodiment of this invention. (A) The figure which showed the example of a position detection apparatus, (b) The figure for demonstrating the movement of a viewer's standing position, (c) The example of the 3D image which changes according to a viewer's standing position was demonstrated. Figure The figure explaining the image of each flame | frame which comprises the moving image of an object displayed when the virtual viewpoint position is a reference | standard position by the image display apparatus of the 3rd Embodiment of this invention. The image of each frame constituting the moving image of the object displayed when the virtual viewpoint position is shifted to the upper right direction from the reference viewpoint position by the image display device according to the third embodiment of the present invention has been described. Figure

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Image processing apparatus 13 Interface 14 Display apparatus 15 Control part 19 Operation part 17 Data storage part 32 Object coordinate data 33 Coordinate conversion table 50 Object 100, 101 Image display apparatus 200 Viewpoint coordinate plane 210 Display surface of display apparatus

Claims (5)

An image display device that displays a three-dimensional image of an object,
Viewpoint position detecting means for detecting the virtual viewpoint position of the viewer;
Based on the detected virtual viewpoint position, a parse calculation processing means for generating a three-dimensional image of the object seen when the object is viewed from the virtual viewpoint position;
A three-dimensional image display device comprising: means for displaying the generated three-dimensional image on a display device. Storage means for storing a coordinate conversion table for managing coordinates of vertices constituting the object for a plurality of virtual viewpoint positions;
The parse calculation processing means refers to the coordinate conversion table, reads the coordinates of each vertex of the object according to the detected virtual viewpoint position, and generates a three-dimensional image of the object based on the read coordinates.
The three-dimensional image display apparatus according to claim 1. It further comprises imaging means for taking an image,
The viewpoint position detection unit receives an image of a viewer captured by the imaging unit or the vicinity thereof, and detects the virtual viewpoint position based on the movement of the whole or a part of the image. 3. The three-dimensional image display device according to 1. It further comprises detection means for detecting the standing position of the viewer,
The three-dimensional image display device according to claim 1, wherein the viewpoint position detection unit receives information on the standing position of the viewer from the detection unit and detects the virtual viewpoint position based on the standing position of the viewer. A computer-readable three-dimensional image display program for performing control to display a three-dimensional image of an object,
A procedure for receiving information about the position of the viewer via the input means;
A procedure for detecting the virtual viewpoint position of the viewer by the calculation means (21) based on the received information on the position of the viewer;
Based on the detected viewpoint position, a procedure for generating a three-dimensional image of the object that is seen when the object is viewed from the viewpoint position by the calculation means;
A three-dimensional image display program for causing a computer to execute a procedure for causing a display unit to display a generated three-dimensional image.

Images