JP2001285881A - Digital information conversion apparatus and method, and image information conversion apparatus and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve prediction accuracy when applying class classification adaptive prediction processing to a digital information signal that has undergone encoding / decoding processing. A decoder outputs a decoded image signal and additional information for decoding. Additional information
The information includes signal type information, image format information, image quality information, and motion vectors. A class is generated based on the additional information. A feature amount is extracted from the pixel data of the class tap extracted by the region extracting unit 4. A class code is generated based on the additional information class and the feature amount. Prediction coefficient R
The OM 8 outputs a prediction coefficient set corresponding to the supplied class code to the prediction calculation unit 9. The prediction coefficient is determined in advance by learning and stored. The pixel data of the prediction tap extracted by the prediction tap data generation unit 5 and R
By performing a prediction operation using the prediction coefficient set supplied from the OM 8, a predicted image signal having higher temporal resolution and / or spatial resolution is generated for the output image signal of the decoder 1.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital information conversion apparatus and method for improving the quality of an encoded digital image signal or an encoded digital audio signal after decoding, and an image information conversion. The present invention relates to an apparatus and a conversion method.

[0002]

2. Description of the Related Art MPEG2 (Moving Picture Expert Group phase) is one of the compression coding methods for image signals.
2) is used. In a transmission / reception or recording / reproducing system based on MPEG2, an image signal is subjected to compression / encoding processing based on MPEG2 to be transmitted or recorded, and a received / reproduced image signal is subjected to decompression corresponding to compression / encoding processing based on MPEG2. By decoding, the original image signal is restored.

[0003] In the encoding process by MPEG2, in order to make the encoding process versatile and to improve the efficiency of compression by encoding, together with the encoded image data,
The additional information for the decoding process is transmitted. Additional information
It is inserted into the header in the MPEG2 stream and transmitted to the decoding device.

[0004] In addition to MPEG, the characteristics of image signals obtained by decoding differ greatly depending on the coding / decoding system applied. For example, physical characteristics (frequency characteristics and the like) greatly differ depending on signal types such as a luminance signal, a color difference signal, and three primary color signals. This difference will remain in the decoded signal that has undergone the encoding / decoding process. In general, in image coding / decoding processing, the number of pixels to be coded is often reduced by introducing space-time thinning processing. The characteristics of the spatio-temporal resolution of an image greatly differ depending on the thinning method. Further, even when the difference between the spatio-temporal resolution characteristics is small, the image quality characteristics such as S / N and the amount of coding distortion vary greatly depending on the compression rate (transmission rate) condition in coding.

The applicant of the present application has previously proposed a classification adaptive processing. This is because, in a learning process (offline), a prediction coefficient is obtained for each class using an actual image signal (teacher signal and student signal) and accumulated, and in an actual image conversion process, an input image signal is obtained. , And an output pixel value is obtained by a prediction operation of a prediction coefficient corresponding to the class and a plurality of pixel values of the input image signal. The class is determined according to the distribution and waveform of pixel values in the spatial and temporal vicinity of the pixel to be created. By calculating prediction coefficients using actual image signals, and calculating prediction coefficients for each class, it is possible to create a resolution higher than that of the input image signal as compared with a simple interpolation filter process. Having.

[0006]

By applying the classification adaptive processing to the decoded image signal,
When creating a spatio-temporal resolution, the target image signal has a difference in characteristics as described above. As a result, the prediction accuracy of the classification adaptive processing is reduced, and there is a problem that sufficient resolution creation performance cannot be obtained.

Further, in the classification adaptive processing, the prediction performance can be improved by introducing the motion information of the target image signal into the class. As the motion information, an expression form of detailed motion information such as a motion vector is effective. However, when detecting a motion vector from an image signal that has undergone encoding / decoding processing, the detection accuracy of the motion vector is reduced due to distortion of the decoded image signal, and
There is a problem that a large amount of arithmetic processing is required for motion vector detection.

Accordingly, an object of the present invention is to improve the temporal and / or spatial resolution by performing a classification adaptive process using additional information on a digital information signal that has undergone encoding / decoding. It is an object of the present invention to provide a digital information conversion apparatus and method, and an image information conversion apparatus and method that can be performed in a digital camera.

[0009]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the invention according to claim 1 converts an input digital information signal generated by decoding an encoded digital information signal into time and / or time signals. Alternatively, in a digital information conversion device adapted to convert a signal into a signal having a larger number of samples in space, a class information generating means for generating class information based on additional information for decoding processing, and a predetermined prediction coefficient are stored. A coefficient storage means for outputting a prediction coefficient corresponding to the output of the class information generation means from among the stored prediction coefficients; an area extraction means for extracting an area consisting of a plurality of samples from the input digital information signal; Operation processing means for performing a prediction operation to generate a sample value based on the plurality of samples and the prediction coefficient extracted by the means; and A digital information conversion apparatus characterized by having.

[0010] According to a seventeenth aspect of the present invention, an input digital information signal generated by decoding an encoded digital information signal is converted into a signal having a larger number of samples in time and / or space. In the digital information conversion method, a class information generating step of generating class information based on additional information for decoding processing, storing a predetermined prediction coefficient, and corresponding to the class information from among the stored prediction coefficients. Outputting a prediction coefficient to be extracted, extracting a region composed of a plurality of samples from the input digital information signal, and calculating a prediction based on the plurality of samples extracted in the region extraction step and the prediction coefficient. Generating a sample value by performing the following. That.

According to the first and 17th aspects of the present invention, by using the additional information for the decoding process, the prediction accuracy in the process of generating an output with a large number of samples by the class classification adaptive process can be improved. .

According to a second aspect of the present invention, an input digital information signal generated by decoding an encoded digital information signal is converted into a signal having a larger number of samples in time and / or space. In the digital information converter, a first area extracting means for extracting an area consisting of a plurality of samples from an input digital information signal, and a feature extracting means for extracting a feature based on the sample from the first area extracting means. A class information generating means for generating class information based on the feature amount and the additional information for decoding processing, and storing a predetermined prediction coefficient, and from among the stored prediction coefficients, an output of the class information generating means. Coefficient storage means for outputting a corresponding prediction coefficient, and a second means for extracting a region consisting of a plurality of samples from the input digital information signal. Digital information characterized by comprising: an area extracting means for calculating the number of samples extracted by the second area extracting means and a prediction coefficient to generate a sample value by performing a prediction operation. It is a conversion device.

[0013] The invention according to claim 18 converts an input digital information signal generated by decoding an encoded digital information signal into a signal having a larger number of samples in time and / or space. In the digital information conversion method, a first area extraction step of extracting an area including a plurality of samples from an input digital information signal, and a feature of extracting a feature amount based on the samples extracted in the first area extraction step. Amount extraction step, a class information generation step of generating class information based on the feature amount and the additional information for the decoding process, and storing a predetermined prediction coefficient. Outputting a prediction coefficient corresponding to the information; and outputting a plurality of samples from the input digital information signal. And a step of generating a sample value by performing a prediction operation based on the plurality of samples and the prediction coefficient extracted in the second region extraction step. This is a feature of the digital information conversion method.

According to the second and eighteenth aspects of the present invention, it is possible to perform the classification adaptive processing by using the additional information for decoding together with the characteristic amount of the input digital information signal. As a result, the prediction accuracy in the process of generating an output with a large number of samples can be improved.

According to a fourth aspect of the present invention, an input image signal generated by decoding an encoded digital image signal is converted into an image signal having a higher resolution in time and / or space. In the information conversion device, class information generating means for generating class information based on the additional information for decoding processing, storing a predetermined prediction coefficient, and outputting the class information generating means from among the stored prediction coefficients. A coefficient storage unit that outputs a prediction coefficient corresponding to a region extraction unit that extracts a region including a plurality of pixels from an input image signal; An image processing apparatus for performing an operation to generate a pixel value.

According to a nineteenth aspect of the present invention, an input image signal generated by decoding an encoded digital image signal is converted into an image signal having higher resolution in time and / or space. In the information conversion method, a class information generation step of generating class information based on the additional information for decoding processing, and storing a predetermined prediction coefficient, from among the stored prediction coefficients,
Outputting a prediction coefficient corresponding to the class information;
An area extracting step of extracting an area composed of a plurality of pixels from the input image signal, and a step of performing a prediction operation to generate a pixel value based on the plurality of pixels and the prediction coefficient extracted in the area extracting step; An image information conversion method characterized by comprising:

According to the fourth and nineteenth aspects of the present invention, by using the additional information for the decoding process, it is possible to improve the prediction accuracy in the process of generating a high-resolution output image by the class classification adaptive process. .

According to a fifth aspect of the present invention, there is provided an image processing apparatus for converting an input image signal generated by decoding an encoded digital image signal into an image signal having a higher resolution in time and / or space. In the information conversion device, a first area cutout unit that extracts a region including a plurality of pixels from an input image signal, a feature amount extraction unit that extracts a feature amount based on pixel data from the first region cutout unit, A class information generating means for generating class information based on the feature amount and the additional information for decoding processing, and storing a predetermined prediction coefficient, and corresponding to an output of the class information generating means from the stored prediction coefficients. Coefficient storage means for outputting a prediction coefficient to be performed, second area cutout means for extracting an area consisting of a plurality of pixels from an input image signal,
And a calculation processing means for performing a prediction calculation based on the plurality of pixels extracted by the area extracting means and the prediction coefficient to generate a pixel value.

According to a twentieth aspect of the present invention, an input image signal generated by decoding an encoded digital image signal is converted into an image signal having higher resolution in time and / or space. In the information conversion method, a first area cutout step of extracting a pixel area including a plurality of pixels from an input image signal, and a feature of extracting a feature amount based on the pixel data extracted in the first area cutout step Amount extraction step, a class information generation step of generating class information based on the feature amount and the additional information for the decoding process, and storing a predetermined prediction coefficient. Outputting a prediction coefficient corresponding to the information; and extracting a second area for extracting a pixel area including a plurality of pixels from the input image signal. And a step of performing a prediction operation to generate a pixel value based on the plurality of pixels extracted in the step of extracting the second region and the prediction coefficient, and generating a pixel value. .

According to the fifth and twentieth aspects of the present invention, by using the additional information for decoding processing together with the feature amount of the input digital image signal, the processing for generating an output image with high resolution by the classification adaptive processing is performed. The prediction accuracy can be improved.

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. First, a configuration related to generation of a predicted image signal will be described with reference to FIG. An input bit stream is supplied to the decoder 1. Here, the input bit stream is transmitted and received by a transmitting / receiving system (or a recording / reproducing system,
Hereinafter, the same applies. ), Image data compressed and encoded by MPEG2 and other data such as additional information. The decoder 1 outputs a decoded image signal and additional information for decoding.

The additional information is additional information necessary for the decoding process.
Inserted in the headers of the P layer and the picture layer, the decoder 1 performs decoding processing using the additional information, and separates and outputs the additional information.

The additional information is supplied to the additional information extracting unit 2, and the additional information used for the classification adaptive processing is selectively output from the additional information extracting unit 2. The extracted additional information is supplied to the additional information class generator 3. For example, there is the following as additional information used in the classification adaptive processing.

(1) Signal type information: each component of a component signal (Y, U, V components, Y, Pr, Pb
(2) Image format information: Interlace / progressive identification information, field or frame frequency (time resolution information), image size information of horizontal pixels and vertical lines (spatial resolution) Information) Aspect ratio information such as 4: 3, 16: 9, etc. (3) Picture quality information: Transmission bit rate (compression rate) information (4) Motion vector: Horizontal and vertical motion amount information

The additional information is supplied to the additional information extracting unit 2, and the additional information used for the classification adaptive processing is selectively output from the additional information extracting unit 2. The extracted additional information is supplied to the additional information class generator 3.

The decoded image signal from the decoder 1 is supplied to an area extracting section 4 and a prediction tap data generating section 5.
The region cutout unit 4 extracts a region including a plurality of pixels from the input image signal and supplies pixel data relating to the extracted region to the feature amount extraction unit 6. The feature amount extraction unit 6 generates an ADRC code by performing processing such as 1-bit ADRC on the supplied pixel data, and supplies the generated ADRC code to the class code generation unit 7. The plurality of pixel regions extracted in the region extracting unit 4 are called class taps. A class tap is an area composed of a plurality of pixels existing in a spatial and / or temporal vicinity of a target (target) pixel. As described later, the class is determined for each attention (target) pixel.

ADRC calculates the maximum value and the minimum value of pixel values in a class tap, obtains a dynamic range that is the difference between the maximum value and the minimum value, and requantizes each pixel value according to the dynamic range. It is. 1 bit A
In the case of DRC, the pixel value is converted into one bit depending on whether it is larger or smaller than the average value of a plurality of pixel values in the tap. The ADRC process is a process for making the number of classes representing the level distribution of pixel values relatively small.
Therefore, the present invention is not limited to ADRC, and may use encoding for compressing the number of bits of a pixel value such as vector quantization.

The additional information class generated by the additional information class generator 3 based on the additional information is also supplied to the class code generator 7. The class code generation unit 7
Based on the additional information class and the ADRC code, a class code indicating the result of the class classification is generated, and the class code is supplied to the prediction coefficient ROM 8 as an address. R
The OM 8 outputs a prediction coefficient set corresponding to the supplied class code to the prediction calculation unit 9. The prediction coefficient set is
It is determined in advance by a learning process described later, and for each class,
More specifically, it is stored in the prediction coefficient ROM 8 in a form using a class code as an address. The prediction coefficients may be stored in a memory having a RAM configuration from which the prediction coefficients can be downloaded from the outside.

On the other hand, the prediction tap data generator 5 extracts a predetermined area (prediction tap) including a plurality of pixels from the input image signal, and supplies the extracted prediction tap pixel data to the prediction calculator 9. The prediction tap is an area including a plurality of pixels existing in the spatial and / or temporal vicinity of the target (target) pixel, like the class tap. The prediction operation unit 9 performs a product-sum operation according to the following equation (1) based on the pixel data supplied from the prediction tap data generation unit 5 and the prediction coefficient set supplied from the ROM 8 to obtain a predicted pixel value. And outputs a predicted pixel value.
The prediction tap and the class tap described above may be the same or different.

Y = w ₁ × x ₁ + w ₂ × x ₂ + ‥‥ + w _n × x _n (1) where x ₁ , ‥‥, and x _n are pixel data of the prediction tap, and w ₁ , ‥‥ and w _n are prediction coefficient sets. The prediction operation is not limited to the linear expression represented by the expression (1), but may be a second-order or higher-order expression, or may be nonlinear.

The predicted image signal has a higher spatial resolution than the output image signal of the decoder 1. For example, an image signal is output in which the number of pixels in each of the horizontal direction and the vertical direction is twice that of the original image. The class classification adaptation process is different from the process of interpolating pixels by an average value or the like, and is a process capable of creating a resolution because a prediction coefficient obtained in advance using an actual image signal is used. Further, the present invention can be applied not only to the spatial resolution but also to processing for increasing the temporal resolution.
For example, the present invention can be applied to processing for changing the field frequency from 60 Hz to 120 Hz. Further, processing for increasing the resolution of spatiotemporal (space and time) may be performed.

FIG. 2 shows an example of the arrangement of class taps extracted by the region extracting section 4. A class tap is set by a total of seven pixels including the pixel of interest and a plurality of pixels around the pixel of interest in the decoded image signal. FIG. 3 shows an example of the arrangement of prediction taps output from the prediction tap data generation unit 5. In the decoded image signal, a prediction tap is set by a total of 13 pixels including a target pixel and a plurality of pixels around the target pixel. 2 and 3
, The solid line indicates the first field, and the broken line indicates the second field.
Indicates a field. Further, the illustrated arrangement of taps is merely an example, and various arrangements can be used.

Next, referring to FIG. 4, the class code (prediction coefficient ROM) formed in the class code generation section 7 will be described.
An example of the prediction coefficient stored in the prediction coefficient ROM 8 will be described. In the class information shown in FIG. 4, a signal type class, a format class, a compression ratio (transmission rate) class, and a motion vector class are:
This is a class generated by the additional information class generation unit 3. The signal feature class is a class based on the feature extracted by the feature extractor 6, for example, an ADRC class. In the table of FIG. 4, the leftmost signal type class is the highest order of the address, and the rightmost signal feature class is the lowest order.

The signal type classes are, for example, two types of Y, U, V and Y, Pr, Pb. Prediction coefficients are separately obtained for each signal type.
0 and K1. The format class corresponds to the spatio-temporal resolution characteristic of the image to be processed, and is, for example, of two types.
The class of F1 is defined. For example, for an interlaced image, F0, for a progressive image, F0
One class is assigned. Other examples of image format classes are field or frame frequency, horizontal pixels or vertical lines. As an example, F0,
The larger the F1, F2,... Number, the higher the spatio-temporal resolution.

The compression rate (transmission rate) class is a class based on image quality information, and includes i types of classes R0 to Ri-1.
Is prepared. The higher the compression ratio, the greater the amount of coding distortion. The motion vector class is a class corresponding to the motion vector, and j types are prepared. The compression ratio class and the motion vector class may be individual values. In that case, however, the number of classes increases, so that they are grouped into a plurality of representative values. For example, one representative value may be set for each of a plurality of ranges formed by appropriate threshold values, and a class corresponding to the representative value may be set. In particular,
Three-stage classes of still, small, and large motions may be formed from motion vectors expressing horizontal and vertical motions.

The above four types of classes are classes generated by the additional information class generation unit 3. However, the classes described above are merely examples, and only some of the classes may be used. For example, only the additional information class may be used as a class. Then, a signal feature class (for example, a class based on an ADRC code) generated by the feature extractor 6 is added to the lower side of the above-described four types of classes. As the signal feature class, k kinds are prepared.

As described above, four types of additional information classes and one
A prediction coefficient set is stored in the ROM 8 for each class determined by the type of signal feature amount class. Equation (1) described above
When performing the prediction operation represented by, w ₁ , w ₂ , ‥‥,
There are n sets of prediction coefficients of w _n for each class.

Referring to FIG. 5, another embodiment of the present invention will be described. Parts corresponding to FIG. 1 showing the configuration of one embodiment are denoted by the same reference numerals. Another embodiment is based on the characteristics of the decoded image signal from the decoder 1,
The prediction performance of the classification adaptive processing is improved by changing the data extraction method for class classification and the structure of the prediction tap.

In order to change the class tap structure for extracting the characteristic amount of the decoded image signal based on the additional information extracted by the additional information extracting unit 2, as shown in FIG. The pattern of the class tap to be performed is switched. The feature extraction unit 6 uses ADR
When extracting a waveform and a level distribution as a feature value by C, the size of the region to be subjected to ADRC is changed according to the spatial resolution of the target image. Further, since the signal characteristics vary depending on the type of signal, the class tap structure is changed. Furthermore, it is also possible to change the class tap structure according to the aspect ratio of the image.

The additional information also includes a compression rate (transmission rate information) indicating distortion of an image due to encoding and decoding.
Information on the compression ratio can be extracted from the additional information. It is difficult to detect the amount of coding distortion in an image signal once decoded. When the classification adaptive processing is applied to signals having different encoding distortion amounts, it is difficult to improve the prediction performance. Therefore, the configuration of the class tap is changed according to the compression rate (transmission rate information). Further, by changing the configuration of the class taps based on the motion vector information, it is possible to realize a class tap structure having high spatio-temporal correlation characteristics. For example, in the case of stillness, a class tap is formed in a frame, and when there is movement, a class tap is formed in the previous and next frames in addition to the current frame.

Further, as shown in FIG. 5, the class code formed by the class code generator 7 is supplied to the prediction tap data generator 5 as a control signal. As a result, an optimal prediction tap pattern is set for each class in which additional information is added as shown in FIG. The structure of the prediction tap is changed according to the additional information in the class in the same way as the above-described class tap structure is changed by the additional information, and the prediction tap is changed by changing the prediction tap similarly to the case of the class tap. Performance can be improved.

FIG. 6 shows an example in which the area of the tap (class tap or prediction tap) is changed according to the additional information. FIG. 6 shows an example in which both the spatial resolution and the temporal resolution are created. That is, a new frame T 'is created between the temporally continuous frames (or fields) T0 and T1, and the number of pixels is four times the original number of pixels.

The tap is a spatiotemporal tap obtained by combining spatial taps formed in the images belonging to frames T0 and T1 existing in the decoded image signal. The area of the range including the tap is changed according to the image format information, for example, the spatial resolution information F0, F1, F2. A specific tap structure is configured in any of these regions. The spatial resolution information F0, F1, F2 is included in the class information generated by the class code generation unit 7 as an additional information class. In the example of FIG. 4 described above, 2 of F0 and F1
There are different kinds of classes.

As an example, F0 has the lowest spatial resolution, F1 has an intermediate spatial resolution, and F2 has the highest spatial resolution. As the spatial resolution increases, the area including the tap is gradually enlarged. When the spatial resolution is low, the range in which pixels having a strong correlation exists is narrow, so that the tap region is also narrow. Thereby,
The performance of resolution creation by the class classification adaptive processing can be improved.

As shown in FIG. 4, the position of the region including the tap is changed according to the amount of movement between frames T0 and T1. In the case of a class tap, the position of the area including the tap is changed according to the motion vector in the additional information. In the case of the prediction tap, the position of the region including the tap is changed according to the motion vector class in the class information. By changing the position of the region in this way, it is possible to cut out the pixel of the class tap from the region having a higher correlation or extract the pixel of the prediction tap. Thereby, the prediction accuracy of the classification adaptive processing can be improved.

Next, learning, that is, a process of obtaining a prediction coefficient for each class will be described. Generally, an image signal having the same signal format as an image signal to be predicted by the classification adaptive processing (hereinafter, referred to as a teacher signal) is used for converting the teacher signal into image information conversion intended for the classification adaptive processing. A prediction coefficient is determined by a predetermined calculation process based on an image signal obtained by performing a related process.
In the class classification adaptive processing performed to improve the resolution of an image signal that has undergone encoding / decoding of an image signal according to the MPEG2 standard or the like, learning is performed by, for example, a configuration illustrated in FIG. FIG. 7 shows a configuration for learning prediction coefficient data in another embodiment shown in FIG.

For learning, a teacher signal and an input image signal are used. The teacher signal is a signal having a higher spatiotemporal resolution than the input image signal. The input image signal may be formed by thinning out the teacher signal. The input image signal is encoded by the encoder 21 according to, for example, MPEG2. The output signal of the encoder 21 corresponds to the received signal in the image information conversion device of FIG. An output signal of the encoder 21 is supplied to a decoder 22. The decoded image signal from the decoder 22 is used as a student signal. Also, the decoder 22
Are supplied to the additional information extracting unit 23, and the additional information is extracted.

The extracted additional information is supplied to an additional information class generation unit 24 and an area extraction unit 25. The additional information includes signal type information, image format information, image quality information,
Motion vector and the like.

The decoded image signal from the decoder 22, that is, the student signal, is supplied to the area extracting section 25 and the prediction tap data generating section 26. As in the configuration of FIG. 5, the area cutout unit 25 is controlled by the additional information extracted by the additional information extraction unit 23, and the prediction tap data generation unit 26 generates the additional information of the class generated by the class code generation unit 28. Controlled by class. Thereby, a tap can be set by a plurality of pixels having high temporal and / or spatial correlation. The data of the class tap extracted by the region extracting unit 25 is supplied to the feature amount extracting unit 27, and the feature amount extracting unit 27 extracts a feature amount by processing such as ADRC. This feature amount is supplied to the class code generation unit 28. Class code generator 28
Generates a class code representing the result of the classification based on the additional information class and the ADRC code. The class code is supplied to the normal equation adding unit 29.

On the other hand, the pixel data of the prediction tap extracted by the prediction tap data generation unit 26 is supplied to the normal equation addition unit 29. The normal equation addition unit 29 performs a predetermined operation process based on the output of the prediction tap data generation unit 26 and the teacher signal to generate a normal coefficient that sets a prediction coefficient set corresponding to the class code supplied from the class code generation unit 28 as a solution. Generate equation data. Normal equation adder 2
9 is supplied to the prediction coefficient calculation unit 30.

The prediction coefficient calculation unit 30 performs an operation for solving a normal equation based on the supplied data. The prediction coefficient set calculated by this calculation process is supplied to the memory 31 and stored. Prior to performing the image conversion process related to the prediction estimation, the contents stored in the memory 31 are loaded into the prediction coefficient ROM 8 in FIG.

The normal equation will be described below. In the above equation (1), before learning, the prediction coefficient sets w ₁ , ‥
‥ and w _n are undetermined coefficients. Learning is performed by inputting a plurality of teacher signals for each class. When the number of types of the teacher signal is denoted by m, the following equation (2) is obtained from the equation (1).
Is set.

Y _k = w ₁ × x _k1 + w ₂ × x _k2 + ‥‥ + w _n × x _kn (2) (k = 1, 2, ‥‥, m)

If m> n, the prediction coefficient sets w ₁ , ‥
Since ‥ and w _n are not uniquely determined, the element e _k of the error vector e is defined by the following equation (3), and the prediction coefficient set is set so as to minimize the error vector e defined by the equation (4). To be determined. That is, the so-called minimum 2
A prediction coefficient set is uniquely determined by multiplication.

E _k = y _k − {w ₁ × x _k1 + w ₂ × x _k2 + Δ + w _n × x _kn } (3) (k = 1, 2, Δm)

[0056]

(Equation 1)

[0057] A practical calculation method for obtaining the prediction coefficient set that minimizes the e ² of formula (4), partially differentiated by the prediction coefficient w _{i (i} = 1,2 ‥‥) the e ² ( (5)), each prediction coefficient w _i such that the partial differential value becomes 0 for each value of _i.
Should be determined.

[0058]

(Equation 2)

A specific procedure for determining each prediction coefficient w _i from equation (5) will be described. If X _ji and Y _i are defined as in equations (6) and (7), equation (5) can be written in the form of a determinant of equation (8).

[0060]

(Equation 3)

[0061]

(Equation 4)

[0062]

(Equation 5)

Expression (8) is generally called a normal equation. The prediction coefficient calculation unit 30 calculates a prediction coefficient w _i by performing a calculation process for solving the normal equation (8) according to a general matrix solution such as a sweeping method.

The generation of the prediction coefficients can also be performed by software processing as shown in the flowchart of FIG. The process is started from step S1, and in step S2, a student signal is generated to generate necessary and sufficient learning data for generating a prediction coefficient. In step S3, it is determined whether sufficient learning data necessary to generate a prediction coefficient has been obtained. If it is determined that sufficient learning data has not been obtained yet, the process proceeds to step S4. I do.

In step S4, a class is determined from the feature amount extracted from the student signal and the additional information. In step S5, a normal equation is generated for each class, and the process returns to step S2 to repeat the same processing procedure, thereby generating a normal equation necessary and sufficient to generate a prediction coefficient set.

If it is determined in step S3 that necessary and sufficient learning data has been obtained, the process proceeds to step S6. In step S6, the prediction coefficient sets w ₁ , w ₂ ,.
.., W _n are generated for each class. Then, at step S7, stores the prediction coefficient set w ₁ to w _n for each class generated in the memory, and ends the learning processing in step S8.

The present invention is not limited to the above-described embodiment of the present invention, and various modifications and applications are possible without departing from the gist of the present invention. For example, M
The present invention is applicable not only to the case of using PEG2 but also to the case of using another encoding method such as MPEG4.

[0068]

As described above, according to the present invention, when applying the classification adaptive processing to the decoded image signal in order to increase the time resolution and / or the spatial resolution, the image signal to be processed is obtained. Since the class classification reflecting the difference in the characteristics of the class classification is performed, the prediction accuracy of the class classification adaptive processing can be improved, and sufficient resolution creation performance can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating an example of a pixel arrangement of a class tap.

FIG. 3 is a schematic diagram illustrating an example of a pixel arrangement of a prediction tap.

FIG. 4 is a schematic diagram illustrating an example of a class based on additional information and a feature amount.

FIG. 5 is a block diagram showing a configuration of another embodiment of the present invention.

FIG. 6 is a schematic diagram for explaining another embodiment of the present invention.

FIG. 7 is a block diagram illustrating an example of a configuration related to a prediction coefficient learning process when performing a class classification adaptive process.

FIG. 8 is a flowchart showing processing when the learning processing is performed by software.

[Explanation of symbols]

1,22 ... decoder, 2,23 ... additional information extraction unit, 3,24 ... additional information class generation unit, 4,25
..Area extraction unit, 5, 26: prediction tap data generation unit, 6, 27: feature amount extraction unit, 7, 28: class code generation unit, 8: prediction coefficient ROM, 9.・ ・
Prediction calculation unit

 ────────────────────────────────────────────────── ─── Continuing from the front page (72) Inventor Satoshi Teshigawara 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo F-term in Sony Corporation (reference) 5B057 CA01 CA08 CA12 CA18 CA19 CB01 CB08 CB12 CB16 CD05 CG05 CG05 CH01 CH11 5C059 KK01 LB11 MA00 MA04 MA05 NN01 RB09 RC11 RC12 RC16 TA30 TC00 TD13 UA05 UA38 5J064 AA01 BA13 BA15 BB03 BB12 BC01 BC02 BC26 BD01

Claims (24)

[Claims] 1. A digital information conversion device for converting an input digital information signal generated by decoding an encoded digital information signal into a signal having a larger number of samples in time and / or space. A class information generating means for generating class information based on the additional information for decoding processing, and storing a predetermined prediction coefficient, and corresponding to an output of the class information generating means from the stored prediction coefficients. Coefficient storage means for outputting a prediction coefficient to be extracted, area extraction means for extracting an area consisting of a plurality of samples from the input digital information signal, and a plurality of samples extracted by the area extraction means and the prediction coefficient. And a calculation processing means for performing a prediction calculation to generate a sample value. Apparatus. 2. A digital information conversion apparatus for converting an input digital information signal generated by decoding an encoded digital information signal into a signal having a larger number of samples in time and / or space. First area cutout means for extracting a region consisting of a plurality of samples from an input digital information signal; feature quantity extraction means for extracting a feature quantity based on the samples from the first area cutout means; And class information generating means for generating class information based on the additional information for decoding processing, and storing predetermined prediction coefficients, and corresponding to the output of the class information generating means from the stored prediction coefficients. Coefficient storage means for outputting prediction coefficients to be extracted, and extracting an area consisting of a plurality of samples from the input digital information signal And a calculation processing means for performing a prediction calculation based on the plurality of samples extracted by the second area extraction means and the prediction coefficient to generate a sample value. Digital information converter characterized by the following. 3. The prediction coefficient according to claim 1, wherein the prediction coefficient is generated in advance using a teacher signal corresponding to a digital information signal having a larger number of samples and a student signal corresponding to the input digital information signal. A digital information converter characterized by the above-mentioned. 4. An image information conversion apparatus for converting an input image signal generated by decoding an encoded digital image signal into an image signal having higher resolution in time and / or space. Class information generating means for generating class information based on the additional information for decoding processing, storing a predetermined prediction coefficient, and corresponding to an output of the class information generating means from the stored prediction coefficients Coefficient storage means for outputting a prediction coefficient; area extraction means for extracting an area consisting of a plurality of pixels from the input image signal; and prediction based on the plurality of pixels extracted by the area extraction means and the prediction coefficient. An image processing apparatus for performing an operation to generate a pixel value. 5. An image information conversion apparatus for converting an input image signal generated by decoding an encoded digital image signal into an image signal having a higher resolution in time and / or space. First area cutout means for extracting an area consisting of a plurality of pixels from an input image signal; feature quantity extraction means for extracting a feature quantity based on pixel data from the first area cutout means; Class information generating means for generating class information based on the additional information for decoding processing, storing a predetermined prediction coefficient, and corresponding to an output of the class information generating means from the stored prediction coefficients Coefficient storage means for outputting a prediction coefficient; second area cutout means for extracting an area consisting of a plurality of pixels from the input image signal; An image information conversion device, comprising: an arithmetic processing unit that performs a prediction operation based on a plurality of pixels extracted by the extraction unit and the prediction coefficient to generate a pixel value. 6. The image information conversion device according to claim 4, wherein the additional information is information indicating a type of the image signal to be processed. 7. The image information conversion device according to claim 4, wherein the additional information is format information of an image signal to be processed. 8. The image information conversion device according to claim 4, wherein the additional information is image quality information. 9. The image information conversion device according to claim 4, wherein the additional information is motion vector information. 10. The position of an area to be extracted by at least one of the first area extracting means and the second area extracting means in response to motion vector information included in the additional information according to claim 4 or 5. An image information conversion device characterized by being performed. 11. The method according to claim 4, wherein at least one of the first area cutout means and the second area cutout means has a size of an area cut out in response to image format information included in the additional information. An image information conversion device characterized by being changed. 12. The image information conversion device according to claim 11, wherein the image format information is time and / or spatial resolution information of the image signal. 13. The image information conversion device according to claim 11, wherein the image format information is aspect information of the image signal. 14. The method according to claim 4, wherein the prediction coefficient is generated in advance using a teacher signal corresponding to a higher resolution image signal and a student signal corresponding to the input image signal. Image information conversion device. 15. A learning class information generating means for generating class information based on additional information for decoding processing attached to the student signal according to claim 14, wherein Learning area extracting means for extracting an area consisting of pixels of the above, generating data for solving a normal equation based on the teacher signal, the output of the class information generating means, and the output of the area extracting means An image information conversion apparatus which is determined by a normal equation calculating means to perform the predetermined calculation processing based on the output of the normal equation calculating means, thereby to calculate the prediction coefficient. . 16. The learning method according to claim 14, wherein the prediction coefficient is a first area extracting means for learning for extracting an area composed of a plurality of pixels from an input image signal, and pixel data from the first area extracting means. Learning feature extraction means for extracting a feature based on the class information; and learning class information generation means for generating class information based on the feature and the additional information for decoding associated with the student signal. A second area extracting means for learning for extracting an area composed of a plurality of pixels from the student signal; a teacher signal; an output of the class information generating means; and an output of the second area extracting means. Calculating a prediction coefficient by performing a predetermined calculation process based on an output of the normal equation calculation means; Picture information converting apparatus characterized by being determined by the prediction coefficient determining means for. 17. A digital information conversion method for converting an input digital information signal generated by decoding an encoded digital information signal into a signal having a larger number of samples in time and / or space. A class information generating step of generating class information based on the additional information for decoding processing; storing a predetermined prediction coefficient; and, from among the stored prediction coefficients, a prediction coefficient corresponding to the class information. Outputting an area; extracting an area consisting of a plurality of samples from the input digital information signal; and performing a prediction operation based on the plurality of samples extracted in the area extraction step and the prediction coefficient. Generating a sample value by performing a digital information conversion Law. 18. A digital information conversion method for converting an input digital information signal generated by decoding an encoded digital information signal into a signal having a larger number of samples in time and / or space. A first area extracting step of extracting an area consisting of a plurality of samples from the input digital information signal; and a feature extracting step of extracting a feature based on the samples extracted in the first area extracting step. A class information generation step of generating class information based on the feature amount and the additional information for decoding processing; storing a predetermined prediction coefficient; and, among the stored prediction coefficients, the class information Outputting a prediction coefficient corresponding to the input digital information signal from a plurality of samples. A second area extraction step of extracting an area to be extracted, and a step of performing a prediction operation to generate a sample value based on the plurality of samples extracted in the second area extraction step and the prediction coefficient. A digital information conversion method characterized by comprising: 19. An image information conversion method for converting an input image signal generated by decoding an encoded digital image signal into an image signal having a higher resolution in time and / or space. A class information generation step of generating class information based on the additional information for decoding processing; and storing a predetermined prediction coefficient; and, among the stored prediction coefficients, a prediction coefficient corresponding to the class information. Outputting, an area extracting step of extracting an area composed of a plurality of pixels from the input image signal, and performing a prediction operation based on the plurality of pixels extracted in the area extracting step and the prediction coefficient. Generating a pixel value by using the image information conversion method. 20. An image information conversion method for converting an input image signal generated by decoding an encoded digital image signal into an image signal having higher resolution in time and / or space. A first region extracting step of extracting a pixel region including a plurality of pixels from an input image signal; and a feature amount extracting step of extracting a feature amount based on the pixel data extracted in the first region extracting step. A class information generation step of generating class information based on the feature amount and the additional information for decoding processing; storing a predetermined prediction coefficient; and, among the stored prediction coefficients, the class information Outputting a prediction coefficient corresponding to the following, and extracting a pixel area including a plurality of pixels from the input image signal. And a step of performing a prediction operation to generate a pixel value based on the plurality of pixels extracted in the step of extracting the second area and the prediction coefficient. . 21. The image information conversion method according to claim 19, wherein the additional information is information indicating a type of a processing target image signal. 22. The image information conversion method according to claim 19, wherein the additional information is format information of an image signal to be processed. 23. The image information conversion method according to claim 19, wherein the additional information is image quality information. 24. The image information conversion method according to claim 19, wherein the additional information is motion vector information.