JP2000023198A - Multi-view image compression encoding apparatus and decompression decoding apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a conventional multi-view (especially two-view) image encoding device and decoding device compatible with a single-view image encoding and decoding system. However, there is a problem to be solved in that the transmission bit rate increases as the number of viewpoints of an image to be transmitted increases. A viewpoint ranking is defined for a plurality of viewpoints when observing a subject (1), and image information and viewpoint information of an image corresponding to an upper viewpoint and viewpoint information of an image corresponding to a lower viewpoint are determined. A predicted value of the image information of the image corresponding to the lower viewpoint is calculated (2, 6-1), and encoded data corresponding to the difference between the predicted value and the true value is transmitted by predictive encoding (7-1). This effectively compresses and encodes multi-view images,
Decompression decoding is performed on the receiving side.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the transmission of a multi-viewpoint image of the same subject photographed by a plurality of cameras and the transmission of a three-dimensional image generated by computer graphics as a multi-viewpoint image (two-dimensional image). More particularly, the present invention relates to a compression encoding apparatus and a decompression decoding apparatus for multi-view images in consideration of compatibility with a conventional single-view image transmission apparatus.

[0002]

2. Description of the Related Art Conventionally, as a compression encoding method for a binocular stereoscopic television signal, M is compatible with a single viewpoint image encoding method.
PEG-2 multi-view profile is standardized [ISO / IEC 13818-2 Amendment 3, WG11 N1366 (1996)
(This is called a first conventional method)].

[0003] Further, a subject having a relatively simple shape such as a face image of a person is photographed by a plurality of cameras, and these photographed images are connected to form a developed image plane image, and distance information, the position of the photographed camera, A method of transmitting information such as orientation and angle of view is being studied [JROhm and K. Mueller:
“Incomplete 3D Representation of Video Objectsfor
Multiview Applications, ”Proc. Picture Coding Sy
mposium (PCS'97), pp. 427-432 (1997) (referred to as the second conventional system)].

Further, with respect to a three-dimensional image generated by computer graphics or the like, a method of approximating a subject with a polyhedron and expressing the coordinates with the coordinates of the vertices of the polyhedron and the pixel values of the surface of the polyhedron has been widely used [VRML2.0 ISO]. / IEC CD14
772 (referred to as the third conventional method)].

[0005]

In the above-mentioned first conventional method, a hierarchical coding method in which a left-eye image and a right-eye image of a binocular stereoscopic television signal is encoded by a base layer and an enhancement layer is encoded. Adopted, the left-eye image can be decoded only from the base layer data, while the right-eye image is decoded from the base layer and enhancement layer data. If this hierarchical decoding method is applied to the encoding of a multi-view image, a multi-view image encoding device and a decoding device compatible with a single-view image encoding and decoding system can be realized. , The image is divided into small blocks, and the blocks are translated and predicted by parallax compensation prediction or motion compensation prediction for each block, and the residual is coded, so that the number of viewpoints of the image to be transmitted increases. As a result, there is a problem that the transmission bit rate increases. Further, since this method is not a method of transmitting all of the parallax or distance information, when combining images at viewpoints not transmitted by interpolation, the parallax or distance of a desired viewpoint is obtained from the received image information on the receiving device side. A device for calculating information is required.

On the other hand, the above-described second and third conventional systems have a problem that they are not compatible with a system for encoding and decoding a single viewpoint image.

[0007] An object of the present invention is to efficiently compress and encode each image data observed from a plurality of viewpoints of three or more and transmit (decompress and decode on the receiving side). Encoded data corresponding to a given viewpoint can be decoded by using a conventional single-view image encoding device, so that encoding of a single-view image and multi-view image compatibility with a decoding system are possible. A compression encoding device and a decompression decoding device are provided.

[0008]

In order to achieve the above object, a multi-view image compression encoding apparatus according to the present invention, when predictively encoding an image corresponding to one viewpoint, uses a plurality of other viewpoints. Among the corresponding images, a viewpoint order memory in which an image corresponding to which viewpoint is used as a reference image is stored, and among a plurality of reference candidate images, at least according to the viewpoint order stored in the viewpoint order memory. A prediction image selection unit for selecting one reference image, image information and viewpoint information of an image corresponding to the reference image selected by the prediction image selection unit, and viewpoint information of an image corresponding to the one viewpoint are used. A viewpoint conversion unit configured to convert image information of an image corresponding to the selected reference image into a predicted value of image information of an image corresponding to the one viewpoint, and an image of an image corresponding to the highest viewpoint A first encoding unit for encoding information and viewpoint information, obtained by subtracting a predicted value of image information obtained by the viewpoint conversion unit from image information (true value) of an image corresponding to the one viewpoint. Image information (difference value)
Encoding the viewpoint information of the image corresponding to the two viewpoints
The highest viewpoint and the one viewpoint obtained by locally decoding the encoded data respectively encoded by the first encoding unit and the second encoding unit. And image information and viewpoint information of images respectively corresponding to the plurality of reference candidate images to be selected are supplied to the predicted image selecting unit.

A multi-view image decompression decoding apparatus according to the present invention further includes a predictive image selecting unit for selecting at least one reference image from a plurality of reference candidate images in accordance with a viewpoint order transmitted from a transmitting side. Using the image information and the viewpoint information of the image corresponding to the reference image selected by the predicted image selection unit and the viewpoint information of the image corresponding to the one viewpoint transmitted from the transmission side, the selected reference A viewpoint conversion unit configured to convert image information of an image corresponding to an image into a predicted value of image information of the image corresponding to the one viewpoint, and A first decoding unit that decodes the image information and the viewpoint information; and a viewpoint conversion unit that converts the image information (difference value) of the image corresponding to the one viewpoint that has been predictively encoded and transmitted from the transmission side. Second decoding for decoding the image information obtained by adding the prediction values of the obtained image information and the viewpoint information of the image corresponding to the one viewpoint coded and transmitted from the transmission side. And at least one of the first
The image information and the viewpoint information of the image respectively corresponding to the highest viewpoint and the one viewpoint decoded by the decoding unit and the second decoding unit are referred to as the plurality of reference candidate images. The apparatus is configured to be supplied to the prediction image selection unit.

The viewpoint conversion unit according to the present invention represents a pixel position and a distance value in a coordinate system with reference to the viewpoint of the reference image as (X, Y) and Z, respectively, and displays the viewpoint of the image to be encoded. When the pixel position and the distance value of the coordinate system with respect to are represented by (X ′, Y ′) and Z ′, respectively,

(Equation 2) Here, M is characterized in that it includes a coordinate converter for performing coordinate conversion according to a row example of 3 rows and 4 columns.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on embodiments of the present invention with reference to the accompanying drawings. As will be described in detail below, the present invention defines viewpoint ranks for three or more viewpoints when observing a subject, and determines the viewpoint order from a pixel value and a viewpoint at a pixel position of an image corresponding to an upper viewpoint. Information on the distance to the subject corresponding to the pixel (hereinafter, referred to as image information), information on the viewpoint position, viewpoint orientation, and angle of view of the image corresponding to the upper viewpoint (hereinafter, referred to as viewpoint information); From the viewpoint information of the image corresponding to the lower viewpoint, a predicted value of the image information of the image corresponding to the lower viewpoint is calculated, and encoded data corresponding to the difference between the predicted value and the true value is transmitted by predictive coding. Thus, the multi-viewpoint image is effectively compressed and encoded, and the receiving side attempts to decompress and decode.

First, the order of viewpoints will be described. FIG.
Is a diagram for explaining this, (a) is a viewpoint (camera)
The arrangement is shown, and in this example, 7
Cameras are shooting the same subject. In the figure, (b) defines the order between these viewpoints (viewpoint order),
Viewpoint D is the highest viewpoint, viewpoints B and F are the next highest viewpoint,
The viewpoints A, C, E, and G indicate the next higher viewpoints. Further, FIG. 3C shows a reference relationship at the time of predictive encoding. For example, A ← B shown first indicates that the image of the viewpoint A can be predictively encoded using the image of the viewpoint B as a reference image. Is shown.

A multi-viewpoint image is compression-coded based on the viewpoint order of the viewpoints described above, image information (pixel position, pixel value, distance value) and viewpoint information (position, direction, angle of view) for each viewpoint. The multi-view image compression encoding apparatus of the present invention will be described below. FIG. 2 is a block diagram showing an embodiment of a multi-view image compression encoding apparatus according to the present invention. In addition, FIG. 2 illustrates only blocks of four viewpoints including the highest viewpoint among the seven viewpoints of FIG. In FIG.
1 is a viewpoint rank memory, 2 is a predicted image selection unit, 3 is a top viewpoint image input unit, 4 is an image / viewpoint information encoding unit, and 5-1.
Is an n-th viewpoint image input unit, 6-1 is a viewpoint conversion unit, 7-1 is an image / viewpoint information prediction encoding unit, and 8-1, 8-2,
Reference numeral 8-3 denotes a coding block including an image input unit, a viewpoint conversion unit, and an image / viewpoint information prediction coding unit corresponding to each viewpoint.

The operation will be described. The view order memory 1 stores reference relation data between the views shown in FIG. 1C, and the stored contents are sent to the receiving side via the transmission path a for use as a reference at the time of decoding. At the same time, it is sent to the predicted image selection unit 2 via the signal path b. The predicted image selecting unit 2 obtains a viewpoint order among the viewpoints shown in FIG. 1B from the transmitted reference relation data, and obtains a signal path e and k−1, k as a locally decoded signal of the encoded signal. −
From the image information and the viewpoint information of the images corresponding to all the viewpoints input to the predicted image selection unit 2 via 2, 2−3, the reference image of the image corresponding to the viewpoint to be encoded corresponds to the viewpoint image to be encoded. Select the image information and viewpoint information to be
-1, g-2, g-3.

Here, the reference image is, for example, based on the reference relationship between the viewpoints shown in FIG. 1C when encoding an image taken by the camera C (see FIG. 1A). The images corresponding to the viewpoints of the camera B higher than the camera C and the camera D higher than the camera C are the reference images.

In the case of this example, the image taken by the camera D of the highest viewpoint (FIG.
(See (a)), and viewpoint information (position, direction, angle of view) and image information (pixel position, pixel value, distance value) of the image corresponding to the highest viewpoint are output to signal paths c and d, respectively. You.

The viewpoint information and image information of the image corresponding to the highest viewpoint output through the signal paths c and d are encoded by the image / viewpoint information encoding unit 4, and the encoded image information and viewpoint information are converted. The data is transmitted to the transmission path f. The image / viewpoint information encoding unit 4 uses the MPE as described in the related art.
It is configured with an encoder that encodes independently without referring to another image like an image for the left eye in the G-2 multi-view profile.

Therefore, the image / viewpoint information encoding unit 4 encodes the pixel value of the image corresponding to the highest viewpoint by the encoder using the above-described conventional single-viewpoint image encoding method. Then, the pixel value of the top viewpoint image can be decoded by the conventional decoding device. The encoding unit 4 includes a local decoder (not shown), decodes the encoded image information and viewpoint information, and supplies the decoded image information and viewpoint information to the predicted image selection unit 2 via a signal path e.

The n-th viewpoint image input unit 5-1 is supplied with an image photographed by a camera of the n-th viewpoint (for example, the camera C in FIG. 1A), and supplies signal paths h and i. The n-th
The viewpoint information and the image information of the image corresponding to the viewpoint are output.

Next, the viewpoint converter 6-1 which plays an important role in the present invention will be described. The configuration of the viewpoint conversion unit 6-1 is shown, for example, in the block diagram of FIG. In FIG. 3, 9-1, 9-2, and 9-3 are coordinate converters, 10-1, and
10-2 and 10-3 are frame memories, 11 is a comparator,
And 12 are selectors.

As described above, the predicted image selecting unit 2
Locally decoded image information and viewpoint information from the image / viewpoint information encoding unit 4 and a plurality of image / viewpoint information prediction encoding units 7-1 (7-2 and below are not shown) are signal lines e and k-
1, k-2 and k-3 respectively.
Of the supplied image information corresponding to each viewpoint, the image information of the reference image of the image corresponding to the n-th viewpoint is
The role of the viewpoint conversion unit 6-1 (6-2 and below is not shown) is to convert the image information corresponding to the n-th viewpoint based on the viewpoint information of the image and the viewpoint information of the image corresponding to the n-th viewpoint. Specifically, the coordinate conversion is performed by the following coordinate transformation.

Now, assuming that the n-th viewpoint is the camera C shown in FIG.
The image information and the viewpoint information supplied to the viewpoint conversion unit 6-1 via the above are the information corresponding to the cameras B and D. In this example, the viewpoint information of the image corresponding to the camera C is also supplied to the viewpoint conversion unit 6-1 via the signal line h.

FIG. 4 is a diagram showing an example of a coordinate system for defining the position and orientation of the viewpoint as a premise for viewpoint conversion. FIG. 5 is a diagram showing positions where points on the subject are projected onto points on the photographing plane. 4, and the position of the viewpoint B (optical center of the camera) O _B as the origin, the direction of the optical axis of the camera Z _B, the direction of the upper camera Y _B, and the direction Z _B and Y
_In a coordinate system in which a direction orthogonal to _B is XB, a point P (X, Y, Z) on a subject is represented by a distance f between an optical center and a photographing plane, as shown in FIG. It is projected on a point (fX / Z, fY / Z) on the photographing surface. Therefore, conversely, the coordinates of the point P (X, Y, Z) can be obtained using the distance information Z between the pixel position on the photographing surface and the point P on the subject.

Next, in FIG. 4, the coordinates of the point P are set to the origin O _C of the viewpoint C, and the directions of the viewpoint C are X _C , Y _C , and Z _C.
When expressed using a coordinate system defined by the following equation, coordinate transformation by a matrix M of 3 rows and 4 columns,

(Equation 3) Can represent the coordinates of the point P, and the points (fX '/ Z', fY '/ Z') projected on the photographing plane of the camera C
Can be calculated.

As described above, the viewpoint to be referred to (viewpoint B
The pixel position (X, Y) and distance value Z of D) and D) are subjected to matrix transformation from a coordinate system determined by the viewpoint information of the referred viewpoint to a coordinate system determined by the viewpoint information of the viewpoint (viewpoint C) to be encoded. In the coordinate system at the viewpoint to be encoded, it is represented by a pixel position (X ′, Y ′) and a distance value Z ′.

In FIG. 3 showing the configuration of the viewpoint converter 6-1, pixel positions and distance values before and after conversion on the input and output sides of the coordinate converters 9-1, 9-2 and 9-3 are described above. Together, they are represented by (X, Y, Z) and (X ', Y', Z '), respectively. The pixel positions X ′ and Y ′ and the distance value Z ′ output by the coordinate conversion are based on the pixel values of the reference image (FIG. 3).
In this case, the data is input to the frame memories 10-1, 10-2, and 10-3 together with R, G, and B as an example, and is rearranged in the scanning order of the pixels. When there are a plurality of reference images, coordinate conversion and rearrangement are performed for each viewpoint to align the positions with the pixel positions observed at the viewpoint to be encoded. Each frame memory 10-1, 10-2, 10-
In the case where a plurality of pixel values R, G, and B output from 3 exist at the same pixel position of the viewpoint to be encoded, the plurality of distance values Z 'are compared by the comparator 11 and the minimum distance value Z ′, That is, the pixel values R, G, and B that are the foremost from the n-th viewpoint.
Is selected by controlling the selector 12 based on the comparison result of (i), and is set as the predicted value of the image corresponding to the n-th viewpoint.

Referring back to FIG. 2, the image / viewpoint information predictive coding unit 7-1 stores the viewpoint information (true value) of the image corresponding to the nth viewpoint via signal paths h, i, and j. , Image information (true value), and image information (predicted value) of an image corresponding to the reference image are supplied. In response to the supply of the true value and the predicted value of the image information and the viewpoint information, the image / view information prediction encoding unit 7-1 encodes the difference encoded data of the true value and the predicted value of the image information (for the viewpoint information, Encoded data of only an image corresponding to n viewpoints) is encoded and output to the transmission line L-1 as a compressed encoded signal.

The image information obtained by expanding and decoding the obtained compressed and coded signal and the compressed / decompressed locally decoded signal of the viewpoint information are sent to the predicted image selecting unit 2 via the signal line k-1, and are selected. This is one of the reference candidate images to be used. This is the same as the above-described image corresponding to the highest viewpoint (sent to the selection unit 2 via the signal path e).

Each of the coding blocks 8-2 and 8-3 (increased as necessary) has the same configuration as the above-described coding block 8-1, and includes image data corresponding to a plurality of viewpoints. Is compression-encoded.

FIG. 6 is a block diagram showing an embodiment of a multi-viewpoint image decompression decoding apparatus according to the present invention. Also,
In FIG. 6, as in FIG. 2, only blocks of four viewpoints including the highest viewpoint are shown. In FIG. 6, 13 is a predicted image selecting unit, 14 is an image / viewpoint information decoding unit, 15 is a top viewpoint image output unit, 16-1 is a viewpoint conversion unit, and 17-1 is an image / viewpoint prediction decoding unit. , 18-1 include an n-th viewpoint image output unit, and 19-1, 19-2, and 19-3 include a viewpoint conversion unit, an image / viewpoint information prediction decoding unit, and an image output unit corresponding to each viewpoint. Decoding block.

The operation will be described. First, the reference relation data between the viewpoints stored in the viewpoint ranking memory 1 (see FIG. 2) on the transmitting side is transmitted to the predicted image selecting unit 13 via the transmission line a. The viewpoint ranking between the viewpoints is obtained in the same manner as on the side. As shown in the figure, the predicted image selection unit 13 includes an image / viewpoint information decoding unit 14 and an image / viewpoint prediction decoding unit 17-
1 (17-2 and below are not shown), the decoded signal (image information and image information of the image corresponding to each viewpoint) is supplied via signal paths r and w-1, w-2, and w-3, respectively. ing.

Based on the above, the predicted image selecting section 13
Among the supplied image information and viewpoint information of the images corresponding to all viewpoints, an image corresponding to the viewpoint to be decoded (for example, an image captured by the camera C of the n-th viewpoint)
Of the reference image (in this case, images captured by the cameras B and D) and the viewpoint information are selected and output to the signal path s-1.

On the other hand, the coded image information and viewpoint information transmitted to the receiving side via the transmission path f and corresponding to the highest viewpoint are supplied to the image / viewpoint information decoding unit 14, and the decoding unit Decodes the encoded data, and outputs the decoded image information and viewpoint information of the image corresponding to the highest viewpoint to the signal path r. This can be extracted from the top viewpoint image output unit 15 as a decoded signal, and is also supplied to the predicted image selection unit 13 as described above.

Next, the decompression decoding of the image for the n-th viewpoint will be described. In FIG. 6, the viewpoint converter 16-1
Performs exactly the same operation as the viewpoint converter 6-1 on the transmitting side (therefore, the circuit configuration is also as shown in FIG. 3). The converter 16-1 is connected via the signal path s-1. Image information and viewpoint information corresponding to the reference image, and viewpoint information of the image corresponding to the n-th viewpoint via a signal path u, respectively, and the n-th viewpoint (as described above, For example, a predicted value of the image information of the image corresponding to the camera C) is obtained, and this is supplied to the image / viewpoint information predictive decoding unit 17-1 via the signal path v.

In FIG. 6, when a picture corresponding to an image at a viewpoint other than the n-th viewpoint is decompressed and decoded using a reference image, a predicted image selecting unit 1
3, the image information and the viewpoint information of the corresponding reference image are converted into the viewpoint converters 16-2 and 16-2 via signal paths s-2 and s-3, respectively.
16-3 (not shown).

The image / viewpoint information predictive decoding unit 17-
1 corresponds to the difference value of the image information of the image corresponding to the n-th viewpoint transmitted through the transmission path L-1 and the compression-coded data of the viewpoint information, and the n-th viewpoint via the signal path v. Image information (prediction value) of an image to be transmitted is supplied, and the prediction decoding unit 17-1 uses the difference value and the prediction value of the image information to generate an n-th viewpoint original image (the first image transmitted from the transmission side). Reconstruct (add) image information of the image corresponding to (n-viewpoint image)
And output. At the same time, the viewpoint information of the image corresponding to the n-th viewpoint is also output as the output of the image / viewpoint prediction decoding unit 17-1 by decompression decoding.

As shown in FIG. 6, the output of the predictive decoding unit 17-1 is supplied to the n-th viewpoint image output unit 18-1 via the signal path w-1, and the image is output from the n-th viewpoint image output unit 18-1. The information and viewpoint information can be extracted, and on the other hand, they are also supplied to the predicted image selection unit 13.

Decoding blocks 19-1, 19-2, 19
-3 have the same configuration, and transmit compressed image data corresponding to a required viewpoint to transmission lines L-1, L-2,
Received via L-3 and decompressed and decoded. The number of decoded blocks can be further added corresponding to the number of encoded blocks on the transmission side.

Finally, it will be explained that the number of decoded blocks can be reduced depending on the decompression decoding device. For example, as shown in FIG. 1 (b), when decoding image information at seven viewpoints defined hierarchically, and when decoding images at all viewpoints simultaneously, a viewpoint converter and a predictive decoder may be used. Are required. On the other hand, when decoding only image information in a certain viewpoint, the viewpoints are defined in a hierarchical manner, so that there are only two viewpoints at the top of any viewpoint, and therefore, up to two decodings are performed. If there is a block, it can be decoded.

In the case of using a method of detecting the position of the viewpoint of the observer and switching the viewpoint position of the image to be reproduced according to the position as the display device of the multi-viewpoint image, the number of viewpoints to be reproduced simultaneously is required. Is one or two corresponding to the left and right eyes, and it is not always necessary to simultaneously decode images at all viewpoints. Therefore, by defining the viewpoints hierarchically in this manner, the number of decoded blocks may be (the number of viewpoints requiring simultaneous decoding) ×
(The number of layers) or less, and the complexity of the decoding device can be reduced.

[0041]

According to the present invention, viewpoint ranking is defined for a plurality of viewpoints, and an image corresponding to a lower viewpoint is predictively encoded using image information and viewpoint information of an image corresponding to an upper viewpoint. , Image data observed from a plurality of viewpoints can be efficiently compression-encoded. Further, by transmitting a distance value for each pixel and performing coordinate conversion by a viewpoint converter, it is possible to improve prediction accuracy. As a result, it is possible to suppress an increase in the amount of transmission data when the number of viewpoints is increased.

Further, the image corresponding to the viewpoint defined at the highest level is independently encoded without referring to the image corresponding to the other viewpoint, thereby providing a conventional single-viewpoint image decoding apparatus. Thus, the pixel value of the image corresponding to the highest viewpoint can be decoded. By defining viewpoint rankings for a plurality of viewpoints and performing hierarchical coding, one decoding block consisting of a viewpoint converter and a predictive decoder required when decoding an image corresponding to one viewpoint is decoded. The number can be less than or equal to the number of layers.

Further, even when the number of layers of the decoded blocks provided in the decoding device is smaller than the number of layers of the coded blocks in the coding device, the number of viewpoints corresponding to the number of layers provided in the coding device can be increased. Can be decoded. For example, if an image corresponding to multiple viewpoints is compressed and transmitted in the viewpoint order shown in FIG. 1B, the image is decoded by a receiver having 0 layers (a conventional single-view decoder). When decoding is performed by the receiver having only the viewpoint D, or by the receiver having one layer, the images corresponding to the viewpoints B, D, and F are decoded. When decoding is performed by the high-grade receiver having two layers, the image corresponding to all viewpoints is decoded. Can be realized.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the order of viewpoints.

FIG. 2 is a block diagram illustrating an embodiment of a multi-view image compression encoding apparatus according to the present invention.

FIG. 3 is a block diagram illustrating a configuration example of a viewpoint conversion unit in FIG. 2;

FIG. 4 is a diagram illustrating an example of a coordinate system that defines the position and orientation of a viewpoint as a premise of viewpoint conversion.

FIG. 5 is a diagram showing a position where a point on a subject is projected on a point on a photographing surface.

FIG. 6 is a block diagram illustrating an embodiment of a multi-view image decompression decoding apparatus according to the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 viewpoint rank memory 2 predicted image selection unit 3 top-level viewpoint image input unit 4 image / viewpoint information encoding unit 5-1 n-th viewpoint image input unit 6-1 viewpoint conversion unit 7-1 image / viewpoint prediction encoding unit 8-1, 8-2, 8-3 Coding block 9-1, 9-2, 9-3 Coordinate converter 10-1, 10-2, 10-3 Frame memory 11 Comparator 12 Selector 13 Predicted image Selection unit 14 Image / viewpoint information decoding unit 15 Top-level viewpoint image output unit 16-1 Viewpoint conversion unit 17-1 Image / viewpoint information prediction decoding unit 18-1 n-th viewpoint image output unit 19-1, 19-2 , 19-3 decoding block

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Miwa Katayama 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Research Institute, Japan (72) Makoto Tadenuma 1-10-11 Kinuta, Setagaya-ku, Tokyo No. Japan Broadcasting Corporation Broadcasting Research Institute (72) Inventor Yuichi Iwadate 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Corporation Research Institute (72) Inventor Yuji Nojiri 1-10 Kinuta, Setagaya-ku, Tokyo No. 11 Japan Broadcasting Corporation Broadcasting Research Institute (72) Inventor Yutaka Tanaka 1-10-11 Kinuta, Setagaya-ku, Tokyo F-term in Japan Broadcasting Corporation Broadcasting Research Institute 5C059 KK00 MA01 MA31 MA32 MA47 MC26 PP04 PP13 RC37 RC38 SS06 UA02 UA05 UA31 5C061 AA29 AB01 AB04 AB08 AB24

Claims (3)

[Claims] 1. When predictive encoding is performed on an image corresponding to one viewpoint, an image corresponding to which viewpoint among images corresponding to a plurality of other viewpoints is used as a reference image is stored. A viewpoint ranking memory 1; and a viewpoint ranking memory 1 among a plurality of reference candidate images.
A predicted image selecting unit 2 for selecting at least one reference image in accordance with the viewpoint order stored in the image processing unit; image information and viewpoint information of an image corresponding to the reference image selected by the predicted image selecting unit 2; A viewpoint conversion unit 6 for converting image information of an image corresponding to the selected reference image into a predicted value of image information of an image corresponding to the one viewpoint, using viewpoint information of the corresponding image; A first encoding unit 4 that encodes image information and viewpoint information of an image corresponding to an upper viewpoint, and a viewpoint conversion unit 6 that obtains the image information (true value) of the image corresponding to the one viewpoint. A second encoding unit that encodes image information (difference value) obtained by subtracting a predicted value of the image information and viewpoint information of an image corresponding to the one viewpoint; Encoding unit 4 and the second The image information and the viewpoint information of the image corresponding to the highest viewpoint and the one viewpoint respectively obtained by locally decoding the encoded data respectively encoded by the encoding unit 7 are selected. A compression encoding apparatus for a multi-view image, which is configured to be supplied to the predicted image selection unit 2 as a reference candidate image. 2. A decoding device for decoding coded data transmitted by the multi-view image compression encoding device according to claim 1, wherein the decoding device transmits the coded data from a plurality of reference candidate images. A predicted image selecting unit 13 for selecting at least one reference image according to a viewpoint order, and image information and viewpoint information of an image corresponding to the reference image selected by the predicted image selecting unit 13 and the 1 transmitted from the transmitting side. A viewpoint conversion unit 16 that converts image information of an image corresponding to the selected reference image into a predicted value of image information of an image corresponding to the one viewpoint, using viewpoint information of an image corresponding to one viewpoint. A first decoding unit 14 that decodes image information and viewpoint information of an image corresponding to the highest viewpoint that has been coded and transmitted from the transmission side, and predictively coded and transmitted from the transmission side. The image information obtained by adding the prediction value of the image information obtained by the viewpoint conversion unit 16 to the image information (difference value) of the image corresponding to the one viewpoint has been encoded and transmitted from the transmission side. At least a second decoding unit 17 for decoding viewpoint information of an image corresponding to the one viewpoint, and decoding by the first decoding unit 14 and the second decoding unit 17, respectively. The top-level viewpoint and the 1
Image information of the image corresponding to each of the three viewpoints and the viewpoint information are supplied to the predicted image selecting unit 6 as the plurality of reference candidate images to be selected. Decompression decoding device. 3. The compression encoding device according to claim 1, or the decompression decoding device according to claim 2, wherein the viewpoint converting unit 6 or 16 determines a pixel position in a coordinate system based on a viewpoint of a reference image. When the distance value is represented by (X, Y) and Z, respectively, and the pixel position and the distance value of the coordinate system with respect to the viewpoint of the image to be encoded are represented by (X ', Y') and Z ', respectively. , [Equation 1] Here, M is a viewpoint conversion unit including a coordinate converter 9 for performing coordinate conversion according to a row example of 3 rows and 4 columns.