JPH11232486A - 3D video playback device and method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To reduce the load at the time of moving image reproduction when displaying moving image data obtained by expanding compressed image data as being in a virtual 3D space with a 3D object. SOLUTION: A three-dimensional object 10 and a display area 11 in a virtual three-dimensional space are displayed on a display screen 9 of a display 7 as viewed from a visual field.
1 displays moving image data obtained by expanding the compressed moving image data. The three-dimensional object 10 and the display area 11 may partially overlap with each other depending on the position in the three-dimensional space. When it is in a distant place, this part is defined as an invisible area 11a,
The compression moving image data is not expanded in the invisible area 11a. As a result, unnecessary extension processing can be omitted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the reproduction of moving picture data, and more particularly to a three-dimensional moving picture reproducing apparatus and method for displaying a moving picture of moving picture data in a designated area in a virtual three-dimensional space. .

[0002]

2. Description of the Related Art As three-dimensional graphics for displaying a three-dimensional object as a two-dimensional image, in particular, for example, VRML (Virtual Reality Modeling Language) is known as an interactive three-dimensional graphics technology. In this VRML, 3 on the WWW (World Wide Web)
The user exchanges dimensional data and changes the viewpoint according to the operation of the user in a virtual three-dimensional space in the display screen (that is, moving an object in this three-dimensional space,
By changing the direction in which the object is viewed by rotating the object, it is possible to realize a deformed display of various objects in the space.

On the other hand, as a technique for compressing moving image data, for example, the MPEG (Moving Picture Experts Group) standard is known. This standard digitizes audio-visual information that has been treated as an analog signal, and achieves compression of the information amount.
tal Versatile Disc), as well as satellite broadcasting.

A technique is known in which these interactive three-dimensional graphics are combined with the reproduction of compressed moving image data to display such graphics and moving images based on the compressed moving image data on the same display screen such as a personal computer. Have been.

[0005]

In the case where the three-dimensional graphics and the moving image based on the compressed moving image data are combined and displayed on a common display screen as described above, all of the conventional techniques decompress the compressed moving image data. Processing is performed to obtain decompressed moving image data, which is mapped to a designated display area in a three-dimensional space and displayed as a moving image. When decompressing compressed moving image data,
The position of the display area in the three-dimensional space is not reflected.

In general, as the compression ratio is increased without deteriorating the image quality, the load of the decompression process increases, and as a result, a device for realizing the decompression process requires a high processing capability. However, in a conventional apparatus for reproducing compressed moving image data in a three-dimensional space, no attempt has been made to reduce the load of the decompression processing, and therefore, a heavy load is imposed on a CPU and the like used in such an apparatus. Will be.

SUMMARY OF THE INVENTION An object of the present invention is to solve such a problem and to efficiently perform expansion processing of compressed moving image data.
It is an object of the present invention to provide a three-dimensional moving image reproducing apparatus and method capable of reducing the load.

[0008]

To achieve the above object, the present invention provides a geometry calculating means for performing coordinate conversion, lighting calculation, and the like of three-dimensional object data,
Rendering means for converting the data created by the geometry calculation means into two-dimensional display data, and converting one or more moving image data into one or more data in a three-dimensional space such as the surface of the three-dimensional object. In a three-dimensional moving image reproducing apparatus arranged and displayed in a designated display area, an invisible part of a display area of moving image data is determined, and only a visible part of the compressed moving image data is expanded in accordance with the determination result.

[0009]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing one embodiment of a three-dimensional moving picture reproducing apparatus and method according to the present invention.
Is a storage means, 3 is an input means, 4 is a memory, 5 is a three-dimensional drawing processing means, 6 is a display driving means, 7 is a display, 8 is a CPU, 9 is a display screen, 10 is a three-dimensional object, and 11 is a display area. It is.

In FIG. 1, a CPU 8 controls the entire three-dimensional moving image reproducing apparatus such as the three-dimensional drawing processing means 5 and the input means 3 according to a program stored in the memory 4. The storage unit 1 stores compressed moving image data, and the storage unit 2 stores three-dimensional data including information on the position coordinates and colors of the three-dimensional object.

The data stored in the storage means 1 and 2 is written into the memory 4 as required, and the two-dimensional image data as viewed from the viewpoint positioned in the three-dimensional space by the three-dimensional drawing processing means 5 Is displayed on the display screen 9 of the display 7 by the display driving means 6. Here, the display screen 9 displays an image in a visual field viewed from a set viewpoint in a three-dimensional space. Hereinafter, the display screen 9 is also referred to as a visual field screen.

On the display screen 9, a three-dimensional object 10 based on the three-dimensional data read from the storage means 2 is displayed, a display area 11 is set, and the display area 11 is read from the storage means. A moving image based on the compressed moving image data is displayed. For this reason, the display area 11 can be regarded as an object called a display panel for displaying a moving image.

The input means 3 is composed of a mouse, a keyboard, or the like, and starts or ends the processing of the three-dimensional drawing processing means 5, moves the viewpoint in the three-dimensional space, and displays the visual field range (displayed on the display screen 9 in the three-dimensional space). Data input is performed for setting a range, a light source position, and designating the position and orientation of the display area 11 in a three-dimensional space.

Here, it is assumed that the compressed moving image data in the storage means 1 and the three-dimensional data in the storage means 2 are read as data on a storage medium directly connected to the three-dimensional moving image reproducing apparatus. However, the present invention is not limited to this, and it is also possible to adopt a configuration in which the data is transmitted from another data source to the three-dimensional moving image reproducing apparatus through a network. Although the three-dimensional drawing processing means 5 is configured as a dedicated circuit, it may be realized by a combination of a general-purpose CPU, software executed thereon, a processor dedicated to three-dimensional drawing, a processor dedicated to moving image expansion, and the like. It is possible.

In this embodiment, under the control of the CPU 8, the three-dimensional drawing processing means 5 decompresses the compressed moving image data from the storage means 1, and converts the three-dimensional data from the storage means 2 into two-dimensional data. Convert to image data, combine each,
On the display screen 9 of the display 7, as shown, a moving image is displayed in the display area 11 together with the three-dimensional object 10. In the three-dimensional drawing processing means 5, the three-dimensional object 10 and the display area 11 are further displayed. By determining the degree of overlap with the display area 11
In this case, a part that is invisible from the viewpoint and overlaps with the three-dimensional object 10 is detected as an invisible part, and the part included in the invisible part of the compressed image data is not subjected to the expansion processing.

As a result, the three-dimensional drawing processing means 5 can reduce the processing amount for the compressed image data decompression processing, and the load is reduced. As a result, when the three-dimensional drawing processing means 5 is also used for processing other than the three-dimensional moving image reproduction processing, the occupancy of the decompression processing of the compressed image data can be reduced, and other processing is also performed. This can be performed with a margin, and even when the three-dimensional rendering processing means 5 is dedicated to the expansion processing of the compressed image data, the expansion processing can be performed quickly with limited capability.

FIG. 2 shows the three-dimensional drawing processing means 5 in FIG.
FIG. 2 is a block diagram showing a specific example, wherein 12 is a geometry calculation means, 13 is a rendering means, and portions corresponding to those in FIG.

In FIG. 1, the three-dimensional drawing processing means 5 comprises:
It comprises a geometry calculation means 12 and a rendering means 13.

The geometry calculation means 12 converts the three-dimensional data of the object to be displayed from the storage means 2 into data consisting of a set of closed plane figures called polygons.
In accordance with the input data of the input means 3 (FIG. 1), deformation, rotation, enlargement, and reduction are performed on each polygon in accordance with the movement of a viewpoint or an object positioned in a three-dimensional space. The calculation of light and shadow generated according to the position, the position, and the shape of the object is performed for each polygon.

Here, the most basic polygon shape is a triangle, and each polygon can be described by coordinates on the display screen 9 (FIG. 1) of three vertices constituting the triangle. Therefore, the range surrounded by these three vertices is the display range of this polygon on the display screen 9, and the three-dimensional object 1
A set of the display ranges of the respective polygons constituting 0 (FIG. 1) is the display range of the three-dimensional object on the display screen 9.

The geometry calculating means 12 initializes the position coordinates of the display area 11 in the three-dimensional space (accordingly, the display screen 9) based on the input data from the input means 3. The display area 11 is also subjected to a process of three-dimensionally changing its position, orientation, and the like according to the movement of the viewpoint positioned in the three-dimensional space.

The rendering means 13 includes the three-dimensional object and the display area 11 created by the geometry means 201.
Is converted into two-dimensional display data, and the compressed moving image data from the storage means 1 is subjected to decompression processing to display data in the display area 11.
The two-dimensional display data obtained in this way is displayed on the display 7 (FIG. 1) by the display driving means 6.

FIG. 3 shows the rendering means 13 in FIG.
2 is a block diagram showing one specific example, wherein 14 is an invisible judging unit, 15 is a moving image decompressing unit, and 16 is a mapping unit, and portions corresponding to those in FIG.

In the figure, the rendering means 13
It is composed of an invisibility judging means 14, a moving picture expanding means 15, and a mapping means 16.

The invisibility judging means 14 calculates the coordinates of the display area 11 and the coordinates of the object in the three-dimensional space calculated by the geometry calculating means 12 (FIG. 2) and the moving image included in the compressed moving image data in the storage means 1. Data such as a display size is acquired, and an invisible portion hidden by the three-dimensional object in the moving image based on the moving image data is determined from these data. The invisible portion determination process will be described later in detail with reference to FIG.

The moving image decompression means 15 reads compressed moving image data in a range included in the set display area 11 of the moving image and displayed from the storage means 1, and a pixel set as a compression unit (in the MPEG standard) , Macroblock), the compressed moving image data is expanded. When the expansion processing is performed, this pixel set is invisible.
It is determined whether or not it is within the area determined to be an invisible part in step 4, and if it is in the invisible part, its expansion is not performed, and the compressed moving image data is expanded only for a pixel set that is a visible part. is there. The expansion process of the compressed moving image will be described later in detail with reference to FIG.

The mapping means 16 includes a moving image decompression means 15
Is attached to the display area 11 in the three-dimensional space calculated by the geometry calculation means 12 (FIG. 2). As a method for realizing such a mapping means 16, texture mapping is generally known, and is described in, for example, "3D Computer Graphics" Shokodo (1986), and detailed description is omitted here. .

FIG. 4 is a view for explaining the data structure of one playback frame of the compressed moving image data stored in the storage means 1.

The moving image data is compressed and decompressed for each reproduction frame. As shown in FIG. 4A, this reproduced frame 401 is composed of n × m sets of pixels as compression and decompression units.
In the case of the EG standard, it is composed of a macro block) 402. Here, the pixel set 402 in the i-th row and the j-th column in the reproduction frame 401 is referred to as a pixel set (i, j).

The two-dimensional moving image data is arranged in the order along the connected pattern 403 in units of pixel sets (ie, from left to right in the arrangement order of the pixel sets, and from top to bottom of this array) for each playback frame 401. 4B. One-dimensional compressed moving image data 4 in which header information 405 required at the time of decompression is added to the head as shown in FIG.
04 is formed. FIG. 4C shows the header information 405.
The height (height of the moving image) H and the width (height of the moving image) when a moving image obtained by expanding the compressed image data is displayed on the display screen 9 (FIG. 1). Width) W
And data such as the height h and the width w of the pixel set at that time. These pieces of information are used by the invisibility determination unit 14 and the moving image expansion unit 15 in FIG.

The connection pattern 403 of the compressed moving image data 404 can be changed according to moving image data and characteristics of the moving image reproducing apparatus.

FIG. 5 is a flowchart showing a processing procedure of the invisible determining means 14 in FIG.

The determination as to whether or not the object is invisible is performed by referring to FIG.
For each reproduction frame 401 shown in FIG.
Is performed as a unit.

4 and 5, first, the pixel set 4
02 is performed (step 501).
Here, from the header information 405 of the compressed moving image data 404 and the calculation result of the geometry calculation unit 12 (FIG. 2), the height H and width W of the playback frame 401 of the moving image data and the height h and width w of the pixel set 402 are determined. (See FIG. 6A. These are initial values, and the height H and width W of the reproduction frame 401 are the same as those on the display 7 when the image is displayed as it is.
The size may be such that it is displayed on the entire display screen 9, or may be smaller. Further, as described above, when the reproduction frame 401 is composed of n × m pixel sets 402, h = H / n, w = W / m) and the display area 11 in the three-dimensional space. And the three-dimensional coordinates of the pixel set 402 in the display area 11 are calculated using these values. In this case, the three-dimensional coordinates are coordinates of one specific pixel of the pixel set (for example, a pixel at the center of the pixel set). The coordinate conversion processing 501 will be described later in detail with reference to FIG.

Next, the depth information of the pixel set 402 is calculated (step 502). Here, the viewpoint 3
The three-dimensional coordinates of the pixel set 402 calculated in step 501 are obtained using the dimensional coordinates and the visual field range, and the depth value from the viewpoint of each pixel set 402 included in the visual field range is calculated using these values. .

The next depth information comparison process (step 50)
In 3), the three-dimensional coordinates of the object calculated by the geometry calculation means 12 are obtained, and the depth value is calculated.
For a portion that appears to overlap with the pixel set 402 on the visual field screen (the display screen 9 of the display 7 in FIG. 1), the depth value of the object and the depth value of the pixel set 402 are compared.

As a result of comparing these depth values, when the pixel set 402 of the moving image in the display area 11 overlaps with the three-dimensional object and becomes invisible, the invisible flag corresponding to this pixel set is set to 1. Then, an invisible state is indicated (step 504). After setting the invisible flag to 1 as described above, or in the display area 11
If the pixel set 402 of the moving image in the image is not made invisible by the three-dimensional object, it is determined whether or not the comparison of the depth information has been performed for all the objects in the three-dimensional space (step 505). If there is an object that has not been performed, the processing from step 503 is repeated for that object.

The pixel set 402 in step 501 of FIG.
Will be described with reference to FIG.

FIG. 6A shows a reproduction frame 401 of moving image data composed of a set of pixels, and FIG.
It is the same as that shown by. First, from the header information 405 of the compressed moving image data 404 shown in FIG. 4B, the height H and width W of the reproduced frame 401 of the moving image data, the pixel set 40
2, the height h and the width w of each pixel are obtained, and the position coordinates of each pixel set 402 on the reproduction frame 401 are obtained. At this time, the coordinates of the center pixel of the pixel set 402 are used as the coordinates of the pixel set 402. Of course, other specific pixels may be used, but this specific pixel is a pixel having the same relative position (for example, upper left corner) in the pixel set for all pixel sets.

Next, the position of the pixel set 402 is scaled according to the shape of the display area 11 in the three-dimensional space.
For example, for a display area 11 having a height H ′ and a width W ′ shown in FIG.
The position 2 is scaled W '/ W times vertically and H' / H times horizontally. The position of the image set 402 scaled in this way is determined by setting the reference point on the display area 11 to O.
L is the coordinate position in the coordinate system, which is the reference point O
The coordinates are stored as relative coordinates v _ij from L, and further converted to a reference coordinate system on a three-dimensional space.

FIG. 6C shows the coordinate transformation, where (X _W , Y _W , Z _W ) is a reference coordinate system in a three-dimensional space, and (X _L , Y _L , Z _L). ) Is a coordinate system on the display area 11. Therefore, as described above, the reproduction frame 401
Coordinate system in each pixel group 402 is a display area 11 in the _{(X W, Y W, Z} W) when positioned in the next calculation _{X W = a 11 X L +} a 12 Y L + a 13 Z L Y W = by _{a 21 X L + a 22 Y} L + a 23 Z L ...... (1) Z W = a 31 X L + a 32 Y L + a 33 Z L, the position of each pixel group is represented in the reference coordinate system of the three-dimensional space Will be.

Here, when the display area 11 and the viewpoint in the three-dimensional space are moved by the data input of the input means 3 (FIG. 1), the appearance of the display area 11 from the viewpoint changes. . This means that the coordinate system on the display area 11
_{(X L, Y L, Z} L) the reference coordinate system in three-dimensional space (X _W,
Y _W , Z _W ), move, rotate, or both, according to the above equation.
Conversion coefficients a _{11 to} a ₁₃ , a _{21 to} a ₂₃ , and a _{31 to} a _{33 in} (1)
Changes. Therefore, even if the display area 11 and the viewpoint move in the three-dimensional space in this manner, as described above, the coordinate system ( _XL ,
Since the coordinate position of each pixel set 402 in Y _L , Z _L ) is stored, the coordinate position of each pixel set 402 in the reference coordinate system (X _W , Y _W , Z _W ) in the three-dimensional space can be easily obtained. Can be requested.

FIG. 7 is a diagram showing a data structure of a depth information buffer for storing an invisible flag and a depth value of a pixel set.

In the figure, a depth information buffer 4a is provided in the memory 4 (FIG. 1) and has a structure having an invisible flag and a depth value for each pixel set 402. FIG.
When the depth values of the respective pixel sets 402 are calculated in step 502, the depth values are sequentially stored in the depth information buffer 4a. The data is stored in the buffer 4b.

The pointer buffer 4b stores a view screen (FIG. 1).
Has the same number of elements (indicated by the divided frames) as the number of pixels of the display screen 9) of the display 7, and each element corresponds one-to-one to each pixel on the visual field screen.

Here, a set of pixels in the display area 11 corresponds to one pixel on the visual field screen. That is, when the expanded moving image data is displayed on the visual field screen, that is, the display area 11 on the display screen 9 of the display 7, one pixel is displayed for each pixel group 402 of the moving image data. The pixel to be displayed for each pixel set 402 is the specified pixel such as the center pixel in the pixel set 402. Therefore, for example, the image compression method
Assuming that the MPEG system is used and the pixel set 402 is one macroblock of 16 × 16 pixels, the number of pixels of the moving image displayed in the display area 11 is 1 / (16 × 16) of the original number of pixels.

Now, an arbitrary pixel set 402 in which the invisible flag and the depth value are stored in the depth information buffer 4a is defined as a pixel set (i, j), and these pixel sets (i, j) in the depth information buffer 4a are _Assuming that the address where the information is stored is A _ij , the address A _ij is written in the pointer buffer 4b to an element corresponding to the position of this pixel set (i, j) on the visual field screen.

A depth information buffer 4a is provided for each display area 11 of moving image data existing in the same space for each viewpoint in the three-dimensional space, and correspondingly, the pointer buffer 4b is also provided in each display area 11a. It is provided for each. That is, each element of the pointer buffer 4b is initialized by a pointer having no address (hereinafter, referred to as a null pointer), and a different depth information buffer 4a and a different pointer buffer 4b are used every time the viewpoint moves. Will be.

At step 502 in FIG. 5, the depth value of each pixel set 402 of the moving image data is calculated, and each depth value is stored in the depth information buffer 4a. When the storage address is stored in the pointer buffer 4b, This means that the addresses of the elements within the range corresponding to the display area 11 in the pointer buffer 4b have been written.

Therefore, in step 503 of FIG. 5, an address is read from an element within a range corresponding to the display area 11 of the pointer buffer 4b, and a depth is stored from a storage location of the depth information buffer 4a corresponding to this address. The value is read and compared with the depth value of the polygon of the three-dimensional object at the position corresponding to this element, and it is determined whether or not the pixel set 402 at this time is invisible to the three-dimensional object. If the result of this determination is that the invisible state is set, the invisible flag is set to 1 at the storage location corresponding to this pixel set 402 in the depth information buffer 4a in step 504 of FIG.

FIG. 8 is a flowchart showing one specific example of the processing procedure of step 502 (depth information calculation processing), step 503 (depth information comparison processing) and step 505 in FIG.

In FIG. 7 and FIG.
In the (depth information calculation process), depth information is calculated for each pixel set 402 of the compressed moving image data (step 80).
1), the value is written to the depth information buffer 4a (step 802), the invisible flag is set to 0 (step 803), and the address of the location where the depth information is stored is set on the view screen of the pixel set 402. Is written to the element of the pointer buffer 4b corresponding to the position of (4) (step 804).

Here, regarding the position of the pixel set 402 on the visual field screen, a case where a plurality of pixel sets are at the same position, including a pixel set of another compressed moving image data when a plurality of display areas 11 are set. It is possible. In such a case, the depth value of the pixel set at the address already written in the same element of the pointer buffer 4b is compared with the depth value of the pixel set to be newly written, and the pixel set to be newly written is compared. Only when the depth value is smaller, that is, when the depth value is on the viewpoint side, the corresponding element in the pointer buffer 4b is rewritten.

In the next step 503 (depth information comparison processing), first, a depth value from the viewpoint of the three-dimensional object is calculated (step 805), and whether or not the three-dimensional object overlaps the pixel set on the visual field screen is determined. It is determined whether the corresponding element in the pointer buffer 4b is a null pointer (step 806). If the pointer is not a null pointer, the depth value of the overlapping pixel set is obtained from the depth information buffer 4a (step 807), and the depth value of this pixel set is compared with the depth value of the three-dimensional object calculated in step 805. (Step 808).
The depth value of the pixel set overlapping the three-dimensional object on the visual field screen can be obtained from the storage location of the depth information buffer 4a pointed to by the pointer stored in the pointer buffer 4b.

Based on the processing result of step 808, if the depth value of the pixel set 402 is larger, that is, if this pixel set 402 is far from the viewpoint, it is determined that the pixel set 402 is invisible. , The invisible flag is set to 1 (step 504).

In the next step 505, it is determined whether or not the depth information comparison processing 503 has been performed for all three-dimensional objects in the three-dimensional space (step 81).
0) If there is a three-dimensional object that has not been compared yet, the processing from step 805 is repeated for the three-dimensional object. If the comparison with all the three-dimensional objects has been completed, the Null pointer is assigned to each element of the pointer buffer 4b, and the pointer buffer 4b is cleared (step 811).

The processing in steps 801 to 811 is as follows.
This is performed for each playback frame or each time the viewpoint is moved.

FIG. 9 is a flow chart showing the processing procedure of the moving picture expansion means 15 in FIG. The video decompression process is shown in FIG.
This is performed for each pixel set 402 of the compressed image data 404 shown in FIG.

In FIG. 9, the pixel set 402 to be subjected to moving image decompression processing is referred to as a pixel set (i, j), and the invisible flag for this pixel set (i, j) is set to the invisible flag (i, j).
j), first, from the depth information buffer 4a set by the invisibility determining means 14 (FIG. 3), this pixel set (i,
an invisible flag (i, j) indicating whether or not j) is invisible
Is acquired (step 901). For one piece of moving image data, invisible flags (i, j) exist for each viewpoint in the number of display areas of the moving image data. Then, this pixel set (i,
All invisible flags (i, j) of j) are compared with “1” (step 902), and all invisible flags (i, j) are compared.
If j) is 1, it is determined that this pixel set (i, j) is invisible. The decompression process is skipped (that is, not performed) for the pixel set (i, j), and the pixel set 402 located next to the pixel set (i, j) on the compressed moving image data 404 is extracted. The same moving image decompression process from step 901 is repeated.

However, if each pixel set 402 is variable-length coded and the size of the data is unknown, the above pixel set (i, j) on the compressed moving image data 404
In order to acquire the pixel set 402 located next to, a process of decoding the pixel set (i, j) into fixed-length data is required.

Further, at least one invisible flag (i,
If j) is 0, it is determined that this pixel set (i, j) is visible, and the decompression processing of step 903 is performed on the portion of the compressed moving image data 404 corresponding to this pixel set (i, j). Done.

The decompression process of the compressed moving image data 404 is realized by performing a process reverse to the process performed at the time of compression. For example, video data compressed by discrete cosine transform and quantization can be expanded by inverse quantization and inverse discrete cosine transform.

The moving image decompression means 15 assumes that each playback frame 401 is compressed independently of each other. However, if the playback frames 401 depend on each other, the other The reproduction frame 401 referred to from the reproduction frame 401 is all expanded regardless of whether it is visible or invisible, and only the reproduction frame 401 which refers to another reproduction frame 401 is invisible.
A configuration is also possible in which the decompression process is performed using the processing result of No. 01.

FIG. 10 is a diagram showing a specific example of a playback screen on the display 7 in FIG. 1. Reference numeral 11a denotes an invisible area of a display area of moving image data designated in a three-dimensional space.
Reference numeral 7 denotes an operation panel, and portions corresponding to those in FIG.

In the figure, an operation panel 17 constitutes a part of the input means 3 shown in FIG. 1, and starts and ends the processing of the three-dimensional drawing processing means 5 (FIG. 1), and operates in the three-dimensional space. This is for interactively inputting data for moving a viewpoint, setting a visual field range and a light source position, specifying a display area of moving image data in a three-dimensional space, and the like. The invisible area 17 in the display area 11 of the moving image data is a part that is blocked by the three-dimensional object 10 from the viewpoint, and changes with the movement of the viewpoint, the display area 11, and the three-dimensional object 10. Is always skipped by the invisibility determining unit 301 and the moving image expanding unit 15.

As described above, in this embodiment, expansion and reproduction of compressed moving image data are performed only for a portion that is visible on the visual field screen. This makes it possible to efficiently process 3D moving images in a short time even with a device having a limited processing capability, and is more wasteful than a conventional 3D moving image reproducing device that expands all compressed moving image data. Processing capacity can be reduced by eliminating unnecessary processing, and the CPU 8 (FIG. 1)
Even when shared with other processing other than the two-dimensional moving image display processing,
The occupancy of the three-dimensional moving image display process can be reduced, and other processes can be executed effectively.

[0068]

As described above, according to the present invention,
Since the load at the time of reproducing the moving image data can be reduced, the image quality of the moving image data can be improved without deteriorating the real-time property, and the number of reproducible moving image data can be increased. Further, when the image quality and the number of the moving image data to be reproduced are the same, the required processing capacity can be reduced, and the three-dimensional moving image reproduction can be realized at a low cost.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of a three-dimensional moving picture reproducing apparatus and method according to the present invention.

FIG. 2 is a block diagram showing a specific example of a three-dimensional drawing processing unit in FIG.

FIG. 3 is a block diagram illustrating a specific example of a rendering unit in FIG. 2;

FIG. 4 is a diagram showing a data structure of compressed moving image data stored in a storage unit in FIG. 3;

FIG. 5 is a flowchart illustrating a processing procedure of an invisible determining unit in FIG. 3;

FIG. 6 is a diagram showing a coordinate conversion process by a process of step 501 in FIG. 5;

FIG. 7 is a diagram illustrating a data structure of a depth information buffer that stores an invisible flag and a depth value.

FIG. 8 shows steps 502, 503 and 50 in FIG.
9 is a flowchart showing a specific example of the fifth embodiment.

FIG. 9 is a flowchart illustrating a processing procedure of a moving image processing unit in FIG. 3;

FIG. 10 is a diagram showing a specific example of a playback screen on the display in FIG. 1;

[Explanation of symbols]

 1, storage means 3 input means 4 memory 5 three-dimensional drawing processing means 6 display driving means 7 display 8 CPU 9 display screen (view field screen) 10 three-dimensional object 11 display area of video data 11a invisible area 12 geometry calculation means 13 rendering Means 14 Invisible judgment means 15 Moving picture expansion means 16 Mapping means

Claims (16)

[Claims] 1. Geometry calculation means for performing coordinate conversion, lighting calculation, and the like of three-dimensional object data,
Rendering means for converting the data created by the geometry calculation means into two-dimensional display data, and converting one or more compressed moving image data into one or three-dimensional space such as the surface of the three-dimensional object. In a three-dimensional moving image reproducing apparatus arranged and displayed in a plurality of designated display areas, an invisible judging means for judging an invisible part of a display area of moving image data; A three-dimensional moving image reproducing apparatus, comprising: moving image decompressing means for decompressing only a part. 2. The three-dimensional moving image reproducing apparatus according to claim 1, wherein said moving image decompressing means is capable of decompressing compressed moving image data of any size (height, width). . 3. The three-dimensional moving image reproducing apparatus according to claim 1, wherein said moving image decompressing means is capable of decompressing compressed moving image data compressed and constructed for each pixel set. 4. The three-dimensional moving image reproducing apparatus according to claim 1, wherein the invisible judging means judges an invisible part for each pixel set, and determines whether or not to perform expansion and reproduction display of the pixel set. A three-dimensional moving image reproducing apparatus characterized in that it is possible. 5. The three-dimensional moving image reproducing apparatus according to claim 4, wherein the invisibility determining means compares depth information from a viewpoint of a pixel set of the moving image data with depth information from a viewpoint of the three-dimensional object. A three-dimensional moving image reproducing apparatus capable of determining whether or not the pixel set is invisible. 6. The three-dimensional moving image reproducing apparatus according to claim 5, wherein the invisibility determining means uses the depth value from the viewpoint of the central coordinates of the pixel set as the depth information from the viewpoint of the pixel set. A three-dimensional moving image reproducing apparatus capable of determining whether a set is invisible or not. 7. The three-dimensional moving image reproducing apparatus according to claim 6, wherein the invisible judging unit holds coordinates of a pixel set as relative coordinates from a display area of the moving image data, and moves or moves the display area. 3D moving image reproducing apparatus characterized in that it is possible to determine whether or not the pixel set is invisible by recalculating the absolute coordinates of the pixel set and acquiring the depth value in response to the movement of . 8. The three-dimensional moving image reproducing apparatus according to claim 7, wherein the invisibility judging means includes a depth information buffer capable of storing depth information from a viewpoint in the three-dimensional space for each of the viewpoints. A three-dimensional moving image reproducing apparatus comprising a plurality of moving image data and storing depth information of each pixel set of the moving image data in the depth information buffer, so that the depth information can be compared. 9. A geometry calculation means for performing coordinate conversion, lighting calculation, and the like of three-dimensional object data,
Rendering means for converting the data created by the geometry calculation means into two-dimensional display data; and converting one or more moving image data into one or more designations in a three-dimensional space such as the surface of the three-dimensional object. In the three-dimensional moving image reproducing method of displaying and arranging in the displayed display area, an invisible part of the display area of the moving image data is determined, and only the visible part of the compressed moving image data is expanded according to the determination result. Dimensional video playback system. 10. The three-dimensional moving image reproducing method according to claim 9, wherein compressed moving image data of an arbitrary size (height, width) can be expanded. 11. The three-dimensional moving image reproducing method according to claim 9, wherein compressed moving image data compressed and constructed for each pixel set can be expanded. 12. The three-dimensional moving image reproducing method according to claim 9, wherein the determination of the invisible portion is performed in units of a pixel set, and it is possible to determine whether to expand and display the pixel set. 3D moving image playback method. 13. The three-dimensional moving image reproduction method according to claim 12, wherein the pixel set is invisible by comparing depth information from the viewpoint of the pixel set of the moving image data with depth information from the viewpoint of the three-dimensional object. A three-dimensional moving image reproducing method, wherein it is possible to determine whether or not the moving image is 14. The three-dimensional video playback system according to claim 13, wherein the depth value from the viewpoint of the center coordinates of the pixel set is used as the depth information from the viewpoint of the pixel set to determine whether the pixel set is invisible. A three-dimensional moving image reproducing method, characterized in that it is possible to determine whether or not the moving image is reproduced. 15. The three-dimensional moving image reproducing method according to claim 14, wherein the coordinates of the pixel set are held as relative coordinates from a display area of the moving image data, and the coordinates of the pixel set correspond to the movement of the display area or the movement of the viewpoint. A three-dimensional moving image reproduction method, wherein it is possible to determine whether or not the pixel set is invisible by recalculating the absolute coordinates of the pixel set and obtaining a depth value. 16. The three-dimensional moving image reproduction method according to claim 15, wherein the depth information buffers capable of storing depth information from the viewpoints in the three-dimensional space are provided for each viewpoint in the number of display areas of the moving image data. A depth information of each pixel set is stored in the depth information buffer so that the depth information can be compared.