JP2000348020A - Data processing device, data processing method, and medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To effectively remove noise contained in a moving image. SOLUTION: A noise adding circuit 62 generates student data as a student by superimposing noise on teacher data as a teacher for learning a prediction coefficient, and a class tap generating circuit 64 generates the student data as a student. Then, data around the focused student data of interest is extracted as a class tap. Then, the classifying circuit 66 classifies the student data of interest based on the stationarity of the class tap, and outputs a corresponding class code. Further, the arithmetic circuit 71 uses the teacher data and the student data,
A prediction coefficient for performing linear primary prediction is obtained for each class code. Then, linear primary prediction is performed using the prediction coefficient and the input data, and data obtained by removing noise from the input data is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing apparatus, a data processing method, and a medium, and more particularly, to a method for removing noise contained in data of a moving image or the like more effectively. The present invention relates to a data processing device, a data processing method, and a medium that perform processing.

[0002]

2. Description of the Related Art For example, data such as image data and audio data transmitted or reproduced generally includes noise that fluctuates with time. As a method for removing noise included in data, Conventionally, an average of the entire input data (hereinafter, appropriately referred to as a total average) and a method of calculating a moving average as a local average of the input data are known.

[0003]

However, the method of calculating the total average is based on the degree of noise included in the data,
That is, it is effective when the S / N (Signal / Noise) of the data is constant, but when the S / N of the data fluctuates, the data with a poor S / N is changed to data with a good S / N. In some cases, it may be difficult to effectively remove noise.

In the method of calculating a moving average, an average of data at a position close in time to input data is obtained, and the processing result is affected by a change in S / N of the data. That is, the S / N of the processing result is improved for a portion having a good S / N of the data, but the S / N of the processing result is also deteriorated for a portion having a bad S / N.

[0005] The present invention has been made in view of such circumstances, and not only when the degree of noise included in data is constant but also when it varies over time,
The noise can be effectively removed.

[0006]

According to a first aspect of the present invention, there is provided a data processing apparatus comprising: an extracting unit configured to extract, from input data, peripheral data around an input data of interest; Classifying means for classifying the input data of interest based on the input data and outputting a corresponding class code; and prediction means for predicting output data of the input data of interest using a predetermined prediction coefficient corresponding to the class code. It is characterized by the following.

[0007] The prediction means can linearly predict output data linearly using a prediction coefficient.

[0008] The data processing apparatus may further include storage means for storing prediction coefficients for each class code.

The data processing apparatus may further include an estimating unit for estimating the amount of noise included in the input data. In this case, the classifying unit includes a routine for determining a constant of peripheral data based on the amount of noise. The input data of interest can be classified based on the stationarity.

The classifying means determines the variance of the surrounding data, determines the continuity of the surrounding data based on the variance, and determines the input data of interest based on the continuity. When further providing estimation means for estimating the amount of noise included in the data, the classifying means compares the variance of the surrounding data with the amount of noise to determine the continuity of the surrounding data. , The input data of interest can be classified into classes.

The input data can be image data.

[0012] The extracting means can extract, from the image data, peripheral pixels spatially or temporally surrounding the target pixel of interest.

According to a ninth aspect of the present invention, there is provided a data processing method comprising the steps of: extracting, from input data, peripheral data around the target input data of interest; Is classified into classes and a corresponding class code is output, and a prediction step of predicting output data for the input data of interest using a predetermined prediction coefficient corresponding to the class code.

According to a tenth aspect of the present invention, there is provided a program for causing a computer to execute a medium based on an extraction step of extracting, from input data, peripheral data around a target input data of interest, and a continuity of the peripheral data. Classifying the target input data and outputting a corresponding class code; and a prediction step of predicting output data for the target input data using a predetermined prediction coefficient corresponding to the class code. Features.

[0015] The data processing apparatus according to the eleventh aspect is a generating means for generating student data to be a student from teacher data to be a teacher for learning a prediction coefficient, and a target student to be focused on from the student data. Extraction means for extracting peripheral data around the data, class classification means for classifying the student data of interest based on the stationarity of the peripheral data and outputting a corresponding class code, and using teacher data and student data. And calculating means for calculating a prediction coefficient for each class code.

The calculating means can determine a prediction coefficient for obtaining teacher data by linear primary prediction using student data.

The generating means can generate student data by adding noise to the teacher data.

The data processing apparatus may further include an estimating means for estimating the amount of noise included in the student data. In this case, the classifying means determines the continuity of the surrounding data based on the amount of noise. It is possible to make a determination and classify the student data of interest based on the stationarity.

The class classification means can determine the variance of the peripheral data, determine the continuity of the peripheral data based on the variance, and classify the student data of interest based on the continuity.

When an estimating means for estimating the amount of noise included in the student data is further provided, the class classifying means includes:
By comparing the variance of the surrounding data and the noise amount, the continuity of the surrounding data can be determined, and the student data of interest can be classified based on the continuity.

The teacher data and the student data can be image data.

The extracting means can extract, from the image data as the student data, peripheral pixels spatially or temporally surrounding the target pixel of interest.

According to a nineteenth aspect of the present invention, in the data processing method, a generating step of generating student data to be a student from teacher data to be a teacher for learning a prediction coefficient; An extraction step of extracting peripheral data around the data, a class classification step of classifying the student data of interest based on the stationarity of the peripheral data, and outputting a corresponding class code, and using teacher data and student data. And for each class code,
Calculating a prediction coefficient.

According to a twentieth aspect of the present invention, there is provided a computer-readable storage medium storing a program for causing a computer to execute, based on a generation step of generating student data to be a student from teacher data to be a teacher for learning a prediction coefficient, An extraction step of extracting peripheral data around the target student data of interest, a class classification step of classifying the target student data based on the stationarity of the peripheral data, and outputting a corresponding class code; And calculating a prediction coefficient for each class code using the student data.

According to a twenty-first aspect of the present invention, there is provided a data processing apparatus, comprising: first extraction means for extracting, from input data, first peripheral data around the target input data of interest;
First class classification means for classifying the input data of interest based on the continuity of the peripheral data of the input data and outputting a corresponding class code, and using a predetermined prediction coefficient corresponding to the class code, Prediction means for predicting the output data for the target, generation means for generating student data to be a student from teacher data to be a teacher for learning of prediction coefficients, and Second extraction means for extracting certain second peripheral data, and second class classification means for classifying the student data of interest based on the stationarity of the second peripheral data and outputting a corresponding class code. Computing means for calculating a prediction coefficient for each class code using teacher data and student data.

[0026] In the data processing device according to the first aspect, the data processing method according to the ninth aspect, and the medium according to the tenth aspect, the input data and the surroundings in the vicinity of the target input data of interest. Data is extracted, the input data of interest is classified based on the continuity of the surrounding data, and a corresponding class code is output. Then, output data for the target input data is predicted using a predetermined prediction coefficient corresponding to the class code.

[0027] A data processing apparatus according to claim 11, a data processing method according to claim 19, and a data processing apparatus according to claim 20.
In the medium described in (1), student data to be a student is generated from teacher data to be a teacher for learning a prediction coefficient, and peripheral data around the noted student data of interest is extracted from the student data. Is done. Then, based on the continuity of the surrounding data, the student data of interest is classified into classes, and corresponding class codes are output. Using the teacher data and the student data, for each class code,
A prediction coefficient is determined.

In the data processing apparatus according to the twenty-first aspect, first peripheral data around the target input data of interest is extracted from the input data, and based on the stationarity of the first peripheral data. Thus, the input data of interest is classified into classes, and the corresponding class codes are output. Then, output data for the target input data is predicted using a predetermined prediction coefficient corresponding to the class code.
On the other hand, student data to be a student is generated from teacher data to be a teacher for learning a prediction coefficient, and second peripheral data around the noted student data of interest is extracted from the student data. . Then, based on the stationarity of the second peripheral data, the student data of interest is classified and a corresponding class code is output, and a prediction coefficient is calculated for each class code using the teacher data and the student data. Can be

[0029]

FIG. 1 shows an example of the configuration of an embodiment of a noise removing apparatus to which the present invention is applied.

In this image processing apparatus, an input image (input data) input thereto is subjected to a classification adaptive process, whereby noise contained in the input image is removed (reduced). ) (Output data) (hereinafter, appropriately referred to as an original image) is predicted.

Here, the class classification adaptive processing includes a class classification processing and an adaptive processing. The class classification processing classifies data into classes based on their properties, and performs an adaptive processing for each class. The adaptive processing is based on the following method.

That is, in the adaptive processing, for example, a prediction value of a pixel of an original image is obtained by a linear combination of a pixel constituting the input image (hereinafter, appropriately referred to as an input pixel) and a predetermined prediction coefficient. An image from which noise contained in the input image has been removed can be obtained.

Specifically, for example, the original image is
Data as well as noise superimposed on the original image
Pixels that make up the original image, using the input image as student data
(Hereinafter, appropriately referred to as an original pixel) a predicted value E of a pixel value y
[Y] is converted to some input pixels (images constituting the input image).
Element value x₁, X_Two,... And a predetermined predictor
Number w ₁, W_Two,... Defined by a linear combination of
Let's consider finding it by the following coupling model. In this case,
The measured value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +... (1) In order to generalize the equation (1), a matrix W composed of a set of prediction coefficients w, A matrix X consisting of a set and a matrix Y ′ consisting of a set of predicted values E [y] are

[0035]

(Equation 1) Defines the following observation equation.

XW = Y '(2) The component x _{ij of the} matrix X is a set of student data on the i-th row (a set of student data used for predicting teacher data on the i-th row). Represents the j-th student data in the matrix, and the component w _{j of the} matrix W represents a prediction coefficient for calculating a product with the j-th student data in the set of the student data. The component E _[ y _i] of the matrix Y ′ represents the predicted value of the teacher data y _{i in} the _i- th row.

Then, the least squares method is applied to the observation equation of Expression (2) to obtain a predicted value E [y] close to the pixel value y of the original pixel.
Think about asking. In this case, a matrix Y consisting of a set of true pixel values y of the original pixels serving as teacher data and a matrix E consisting of a set of residuals e of predicted values E [y] with respect to the pixel values y of the original pixels are given by:

[0038]

(Equation 2) From equation (2), the following residual equation is established.

XW = Y + E (3) In this case, a predicted value E [y] close to the pixel value y of the original pixel is obtained.
Prediction coefficient w _iIs the square error

[0040]

(Equation 3) Can be obtained by minimizing.

Therefore, when the above-mentioned squared error is differentiated by the prediction coefficient w _i , the result becomes 0, that is, the prediction coefficient w _i that satisfies the following equation is the prediction value E [y] close to the pixel value y of the original pixel Is the optimum value.

[0042]

(Equation 4) (4) Then, first, the following equation is established by differentiating equation (3) with the prediction coefficient w _i .

[0043]

(Equation 5) (5) From the expressions (4) and (5), the expression (6) is obtained.

[0044]

(Equation 6) (6) Further, the student data x in the residual equation of the equation (3),
Considering the relationship between the prediction coefficient w, the teacher data y, and the residual e, the following normal equation can be obtained from Expression (6).

[0045]

(Equation 7) (7) Each of the equations constituting the normal equation of the equation (7) is represented by student data x
By preparing a certain number of training data y and teacher data y, the same number as the number of prediction coefficients w to be obtained can be obtained. Therefore, solving equation (7) (provides equation (7) In order to solve, in Expression (7), a matrix composed of coefficients related to the prediction coefficient w needs to be regular),
An optimal prediction coefficient w can be obtained. In solving equation (7), for example, the sweeping method (Gauss
s-Jordan elimination) can be used.

As described above, the optimum prediction coefficient w is obtained, and further, using the prediction coefficient w, a prediction value E [y] close to the pixel value y of the original pixel is obtained by the equation (1). This is the adaptive processing.

The adaptive processing differs from, for example, interpolation processing in that components not included in the input image but included in the original image are reproduced. That is, the adaptive processing is the same as the interpolation processing using the so-called interpolation filter as far as only the equation (1) is viewed, but the prediction coefficient w corresponding to the tap coefficient of the interpolation filter is obtained by using the teacher data y. Therefore, the components included in the original image can be reproduced. That is, for example, high S / N
Image can be obtained. From this, it can be said that the adaptive processing has a so-called image creation (resolution imagination) action. Therefore, in addition to obtaining an original image from which noise has been removed from an input image, for example, a low-resolution or standard-resolution image is converted. It can also be used when converting to a high-resolution image.

The noise removing apparatus shown in FIG. 1 removes noise contained in an input image (predicts an image without noise (original image)) by the above-described class classification adaptive processing, and removes the noise. It outputs the later image.

That is, the noise elimination device shown in FIG. 1 comprises a frame memory 1, a class tap generation circuit 2, a prediction tap generation circuit 3, a class classification circuit 4, a coefficient RAM (Random Access Memory).
ry) 45, and a prediction operation circuit 6, into which an input image to be subjected to noise removal is input.

The frame memory 1 is adapted to temporarily store an input image input to the noise elimination device, for example, in frame units. In this embodiment, the frame memory 1 can store a plurality of frames of input images.

The class tap generation circuit 2 performs an input pixel for obtaining a predicted value of an original pixel by a class classification adaptive process, that is, an input pixel for which noise is to be removed (hereinafter, referred to as an input pixel).
Pixels around the pixel of interest are appropriately extracted from the input image stored in the frame memory 1 and output to the classifying circuit 4 as a class tap used for classifying the pixel of interest. ing.

That is, for example, as shown in FIG. 2, the class tap generating circuit 2 generates (A) a target pixel and (B) four frames (past frames) temporally preceding the frame of the target pixel. , Four pixels at the same spatial position as the target pixel, and (C) four frames (future frames) temporally subsequent to the frame of the target pixel.
4 pixels at the same spatial position as the pixel of interest, (D) 4 pixels to the left of the pixel of interest in the same frame as the frame of the pixel of interest, (E) of the same frame as the frame of the pixel of interest
Four pixels to the right of the pixel of interest, (F) Four pixels above the pixel of interest in the same frame as the frame of the pixel of interest, (G)
A total of 25 pixels, that is, 4 pixels below the pixel of interest in the same frame as the frame of the pixel of interest, are extracted from the input image stored in the frame memory 1, and this is used as a class tap for the pixel of interest, 4 is output.

The prediction tap generation circuit 3 includes a prediction operation circuit 6
, An input pixel used for obtaining a predicted value of an original pixel for a target pixel is extracted from an input image stored in the frame memory 1 and supplied to a prediction operation circuit 6 as a prediction tap. That is, the prediction tap generation circuit 3 extracts, from the input image stored in the frame memory 1, for example, as shown in FIG. 2, the same input pixel as the class tap extracted by the class tap generation circuit 2,
It is extracted as a prediction tap and supplied to the prediction calculation circuit 6.

Here, in order to simplify the explanation,
Although the class tap and the prediction tap are configured from the same input pixel, the class tap and the prediction tap are
It is not necessary to constitute from the same input pixel, but can be constituted from different input pixels.

The class classification circuit 4 classifies the pixel of interest based on the stationarity of the class tap from the class tap generation circuit 2, and assigns a class code corresponding to the obtained class to the coefficient RAM 5 in an address. It has been made to give as. That is, in the classifying circuit 4,
In addition to the class tap being supplied from the class tap generation circuit 2, an estimated value (estimated noise amount) of the noise amount included in the pixel of interest is also supplied from the noise amount estimation circuit 7. Determines the continuity of the pixel values of the input pixels constituting the class tap by using the estimated noise amount, and determines the class code corresponding to the continuity by a coefficient RAM.
5 is output.

The coefficient RAM 5 stores prediction coefficients for each class obtained by performing a learning process described later. When a class code is supplied from the class classification circuit 4, it is stored at an address corresponding to the class code. The read prediction coefficient is read and supplied to the prediction calculation circuit 6.

The prediction operation circuit 6 supplies the prediction coefficients w, w ₂ ,
, And the prediction taps x ₁ , x ₂ ,... From the prediction tap generation circuit 3, perform the calculation shown in Expression (1) to predict the original pixel y with respect to the target pixel x. The value E [y] is obtained and output as a pixel value of a pixel obtained by removing noise from the target pixel x.

The noise amount estimating circuit 7 includes the frame memory 1
Based on the input image stored in, the amount of noise included in the pixel of interest is estimated, and the estimated amount of noise obtained as
The data is supplied to the classification circuit 4.

Next, referring to the flowchart of FIG.
The noise removal processing performed in the noise removal device of FIG. 1 will be described.

The input image (moving image) to be subjected to the noise removal processing is sequentially supplied to the frame memory 1 in frame units, and the input image supplied in such a frame unit is sequentially stored in the frame memory 1. Will be done. In the present embodiment, since the class tap and the prediction tap described in FIG. 2 need to be configured, the frame memory 1 stores at least 9 frames of the input image (the frame of the pixel of interest and the frames before and after the frame). (4 frames each, 9 frames in total).

Then, in step S1, the noise amount estimating circuit 7 uses the input image stored in the frame memory 41 to process the current frame (the frame having the pixel of interest) (hereinafter, appropriately The amount of noise included in the frame of interest is estimated, and the estimated amount of noise obtained as a result is output to the classification circuit 4.

Thereafter, in step S2, the class tap generation circuit 2 or the prediction tap generation circuit 3 sets the input pixel of the frame of interest for which a prediction value is to be obtained by adaptive processing as the pixel of interest, and sets the input pixels around the pixel of interest as the target pixel. It reads from the frame memory 1 and configures the class tap or the prediction tap shown in FIG. The class tap or the prediction tap is supplied to the classification circuit 4 or the prediction operation circuit 6, respectively.

When the class tapping circuit 4 receives the class tap from the class tap generating circuit 2, the classifying circuit 4 determines the stationarity of the class tap and the variance of the class tap and the estimated noise amount from the noise amount estimating circuit 7 in step S 3. And the class of the pixel of interest is determined based on the stationarity. The class code corresponding to this class is the coefficient RA
The coefficient RAM 5 is given as an address, and the coefficient RAM 5 reads the prediction coefficient stored in the address corresponding to the class code from the class classification circuit 4 in step S 4, and supplies the prediction coefficient to the prediction calculation circuit 6.

In the prediction operation circuit 6, in step S5, the prediction tap from the prediction tap generation circuit 3 and the coefficient RA
By performing the operation (linear primary prediction) shown in Expression (1) using the prediction coefficient from M5, the original pixel for the pixel of interest x (in the state where noise is completely removed from the pixel of interest x) The predicted value E [y] of the pixel (y) is obtained, and the process proceeds to step S6. In step S6, the prediction coefficient calculation circuit 6 outputs the predicted value E [y] of the original pixel y with respect to the target pixel x obtained in step S5 as the pixel value of a pixel from which noise has been removed from the target pixel x, and step S7. Proceed to.

In step S7, it is determined whether or not all the input pixels constituting the frame of interest stored in the frame memory 1 have been processed as the pixel of interest. If it is determined that the processing has not been performed, the process proceeds to step S2. Returning, the input pixel which has not been set as a target pixel is newly set as a target pixel, and thereafter, the same processing is repeated.

If it is determined in step S7 that all the input pixels constituting the frame of interest have been processed as the pixel of interest, the process proceeds to step S8, where the next frame to be processed (the current frame of interest) is stored in the frame memory 1. It is determined whether or not the next frame (the frame next to the current frame) is stored. In step S8,
If it is determined that the frame to be processed next is stored in the frame memory 1, the process returns to step S1, and the same process is repeated with the new frame as the target frame.

On the other hand, if it is determined in step S8 that the frame to be processed next is not stored in the frame memory 1, the noise removal processing ends.

Next, FIG. 4 shows the noise amount estimation circuit 7 shown in FIG.
2 shows a configuration example.

Now, in the t-th frame, if an input pixel at a certain spatial position is represented as x (t), the input pixel x (t) stored in the frame memory 1 of FIG. The variance calculation unit 19 and the motion vector detection circuit 29 are configured to be input.

The delay circuit 11 delays the input pixel x (t) inputted thereto by a time corresponding to one frame, and sets the input pixel x (t-1) as the input pixel x (t-1).
And a motion vector detecting circuit 29. The delay circuit 12 delays the input pixel x (t-1) from the delay circuit 11 by a time corresponding to one frame, and supplies the input pixel x (t-2) to the delay circuit 13 and the variance calculator 19 as the input pixel x (t-2). It has become. The delay circuit 13 delays the input pixel x (t−2) from the delay circuit 12 by a time corresponding to one frame, and supplies the input pixel x (t−3) to the delay circuit 14 and the variance calculator 19 as the input pixel x (t−3). It has become. The delay circuit 14 receives the input pixel x from the delay circuit 13
(T-3) is delayed by a time corresponding to one frame, and supplied to the variance calculator 19 as an input pixel x (t-4).

Each of the delay circuits 15 to 18 controls the motion vector output from the motion vector detection circuit 29.
The same delay processing as in the case of the delay circuits 11 to 14 is performed. Therefore, for example, if the motion vector of the input pixel x (t) is represented as v (t), the delay circuits 15 to 18 will be able to use the motion vectors v (t−1) to v (t−1).
(T-4) are output. This motion vector v
(T-1) to v (t-4) are the stillness determination units 23 to 2
6, respectively.

The variance calculator 19 calculates the input pixels x (t) to x (t−4) supplied thereto, that is, the input pixels x (x) at the same spatial position in the past 5 frames including the frame of interest. The variances of t) to x (t-4) are calculated and supplied to the variance accumulation memory 20. The dispersion accumulation memory 20 is controlled by a memory controller 28
The variance supplied from the variance calculator 19 is integrated. The variance value frame average calculator 21 calculates the average value of the variance integrated in the variance integration memory 20 (divides the integrated value of the variance by the number of the integrated variances), and divides the input pixel x ( The estimated value (estimated noise amount) of the noise amount included in each input pixel of the frame (t) of frame t) is output.

The stillness judging unit 22 calculates a motion vector v (t) based on the motion vector v (t) supplied from the motion vector detecting circuit 29.
It is determined whether or not the input pixel x (t) belongs to a stationary portion, and the determination result is sent to the continuous stationary position detection unit 2.
7. Stillness determination units 22 to 2
6, the delay circuits 15 to 18 as well as the stillness determination unit 22.
Motion vectors v (t-1) to v (t-
4), input pixels x (t-1) to x (t-
It is determined whether or not 4) is for a stationary portion, and the determination result is sent to the continuous stationary position detector 27.
To be supplied.

The continuous stationary position detecting section 27 includes a stationary determining section 2
Based on the determination results from 2 to 26, the position (spatial position) of the pixel of the stationary portion is detected in consecutive frames. That is, the continuous stationary position detecting unit 27 detects the input pixel x (t) at a position where the determination results from the stationary determining units 22 to 26 are all stationary, and stores that position in the memory. The data is supplied to the controller 28.

The memory controller 28 includes the distributed computing unit 1
9, the variance accumulation memory 20 is controlled so that only the variance output from the pixel at the stationary position, which is supplied from the continuity pixel detection unit 27, is integrated. I have. Further, the memory controller 2
8 is supplied with a frame reset signal indicating that supply of input pixels of a new frame is started. When the memory controller 28 receives the frame reset signal, the memory controller 28 calculates a variance value frame average. Part 2
1 causes the average value of the integrated value of the variance stored in the variance accumulation memory 20 to be calculated, and resets the storage value of the variance accumulation memory 20 to 0.

The motion vector detection circuit 29 calculates the input pixel x
The motion vector v (t) of the input pixel x (t) is detected from the frame (t) and the frame one frame before the frame supplied from the delay circuit 11, and the delay circuit 1
5 and the stillness determination unit 22.

In the noise amount estimating circuit 7 configured as described above, only the stationary portion of the input pixels constituting the frame (frame of interest) that is the target of the processing (noise removal processing) is performed. , A noise amount estimating process of estimating the noise amount of each input pixel constituting the frame of interest is performed.

That is, the noise amount estimating circuit 7 converts a predetermined input pixel constituting the frame of interest into a pixel x to be processed.
As (t), the pixel x (t) to be processed and the input pixel x (t−1) of the past four frames at the same position as the pixel x (t)
The variance is calculated using the five pixels x (t−4) to x (t−4) and the five pixels x (t) to x (t−4).
4) Each of the pixels at a stationary portion (still pixel)
Is determined.

More specifically, the delay circuit 11 and the variance calculator 19 store the input pixel x (t) constituting the frame of interest.
Are sequentially supplied, and in each of the delay circuits 11 to 14, the input pixel input thereto is delayed by a time corresponding to one frame, and is supplied to the dispersion calculation unit 19. That is, thereby, the input pixels x (t) to x (t−4) are supplied to the variance calculation unit 19. In the variance calculation unit 19, the variance of the input pixels x (t) to x (t−4) is obtained and supplied to the variance accumulation memory 20.

On the other hand, the motion vector detection circuit 29 is supplied with input pixels forming the frame of interest and input pixels of the frame immediately before the frame, and uses these input pixels to process the pixel x to be processed of the frame of interest. A motion vector v (t) of (t) is detected. This motion vector v (t)
Is supplied to the delay circuit 15 and the stillness determination unit 22.

In each of the delay circuits 15 to 18, the motion vector input thereto is delayed by a time corresponding to one frame and supplied to the stillness determination units 23 to 26. Accordingly, the motion vectors v (t) to v (t-t) of the input pixels x (t) to x (t-4) are provided to the stillness determination units 22 to 26.
4) are supplied respectively. Then, in the stillness determination units 22 to 26, based on the motion vectors v (t) to v (t-4) input thereto, the input pixels x (t-1) to x (t-4) are stationary. It is determined whether or not the part is a part where the part is located, and the result of the determination is supplied to the continuous stationary position detection unit 27.

In the continuous stationary position detecting section 27, based on the stationary determination results in the stationary determining sections 22 to 26, respectively,
It is detected whether or not the position of the processing target pixel x (t) is a stationary portion in consecutive frames (here, from the (t-4) th frame to the tth frame). This detection result is supplied to the memory controller 28.

When the memory controller 28 receives from the continuous still position detecting section 27 a detection result indicating that the position of the processing target pixel x (t) is not a stationary portion in a continuous frame, That is, the processing target pixel x (t)
If any of x (t) to x (t−4) at the same position as above has a motion, the variance accumulation memory 20 is instructed to discard the variance from the variance calculation unit 19. . Therefore, in this case, in the dispersion accumulation memory 20,
The variance from the variance calculation unit 19, that is, the variance obtained from pixels having motion, is discarded without being integrated.

On the other hand, the memory controller 28 receives from the continuous still position detecting section 27 a detection result indicating that the position of the pixel x (t) to be processed is a stationary portion in a continuous frame. In other words, the processing target pixel x
If any of x (t) to x (t−4) at the same position as (t) has no motion, the variance from the variance calculator 19 is integrated in the variance accumulation memory 20. Command. Therefore, in this case, the dispersion accumulation memory 20
Then, the variance from the variance calculation unit 19, that is, the variance obtained from only the stationary pixels, is added to the variance already stored therein, and the integrated value is newly stored.

The above processing is performed with all the input pixels of the frame of interest as the pixels to be processed. Thereafter, the variance value frame average calculation section 21 calculates the amount of noise contained in each input pixel constituting the frame of interest. (Estimated).

That is, when the processing for all the input pixels of the target frame is completed, a frame reset signal is input to the memory controller 28. When the memory controller 28 receives the frame reset signal, the memory controller 28 causes the variance value frame average calculation unit 21 to calculate the average value of the variance integrated value stored in the variance integration memory 20 and sets the stored value of the variance integration memory 20 to 0. Reset to.

The variance value frame average calculator 21 calculates the average value of the integrated value of the variance stored in the variance accumulation memory 20 under the control of the memory controller 28.
As described above, the variance accumulation memory 20 accumulates only the variance obtained from the stationary pixels and stores the integrated value. Pixels (here,
The average value of the variance obtained from the pixels that are stationary for five consecutive frames is obtained. This average value is output as an estimated value (estimated noise amount) of the noise amount included in each input pixel of the frame of interest (the t-th frame).

Hereinafter, the same processing is repeated with the next frame as the frame of interest.

As described above, in the noise amount estimation circuit 7,
From the frame of interest, five pixels at the same spatial position that remain stationary for four frames before are detected, and the average value of the variance of such five pixels in the frame of interest is calculated for each input pixel of the frame of interest. Is estimated to be the amount of noise included in. Therefore, in this case, since the moving pixel is not used for estimating the noise amount, the effect of the motion of the image is
It can be prevented from being reflected in the estimated noise amount. That is, it is possible to estimate a noise amount (noise amount closer to a true noise amount) that is hardly affected by the motion of the image.

Further, conventionally, for a moving image,
A method is known in which pixels included in spatially identical positions arranged in the time direction are averaged to remove noise contained therein, but in this case, the motion of each pixel is determined and the motion of the pixel is determined. It is necessary to average only the missing pixels. Therefore, if the motion determination is incorrect, that is, if a moving pixel is mistaken for a non-moving pixel, the moving pixel is included in the target to be averaged, and the average value is directly used as the noise removal result. Therefore, an error in the motion determination greatly affects the noise removal result, and in the worst case, the processing is broken (instead, the image is destroyed).

On the other hand, in the noise elimination device of FIG. 1, the noise amount estimating circuit 7 determines whether or not a certain input pixel is stationary based on a motion vector detected for the input pixel. By taking the average of the variances of the five pixels at the same position determined to be stationary, the estimated noise amount is obtained. Therefore, if any one of the five pixels in the temporal direction at the same spatial position has a motion, it is determined that the input pixel has no motion, and the 5 Even if the variances of the pixels are integrated, the influence on the estimated noise amount is small. Then, after the target pixel is classified based on the estimated noise amount, adaptive processing is performed on the target pixel to obtain a noise removal result from the target pixel. However, the influence on the noise removal result is small (it can be said that there is almost no effect compared with the above-mentioned conventional method).

In the above-described case, the variance is calculated from the five pixels at the same position, which are still from the frame of interest up to four frames before, but the pixels for which the variance is to be calculated are: The present invention is not limited to five pixels from the frame of interest to four frames before.

Further, the noise amount estimating circuit 7 shown in FIG. 4 determines the same estimated noise amount for input pixels constituting each frame, assuming that each frame contains a fixed amount of noise. But if possible,
The estimated noise amount may be obtained for each input pixel.

Next, FIG. 5 shows a configuration example of the class classification circuit 4 of FIG.

The class tap for the pixel of interest output from the class tap generation circuit 2 is calculated by the variance calculation units 31 to 3
The estimated noise amount for the pixel of interest, which is output from the noise amount estimating circuit 7, is supplied to threshold processing units 37 to 42.

Each of the variance calculation units 31 to 36 receives only the input pixels constituting the class tap of the target pixel located in a predetermined direction and calculates the variance.

That is, in the class tap shown in FIG. 2, the target pixel (A) and four pixels (B) at the same spatial position as the target pixel in each of the four frames temporally preceding the frame of the target pixel. ), The five pixels that are temporally preceding (−t direction) with the target pixel as the starting point,
The target pixel of the tap in the −t direction, the target pixel (A), and the four pixels (C) at the same spatial position as the target pixel in each of the four frames temporally subsequent to the frame of the target pixel In the direction that follows in time starting from (+ t direction)
, A tap in the + t direction and a target pixel (A)
Tapping in the left direction (−h direction) with the target pixel as the starting point, and tapping in the −h direction, 4 pixels (D) to the left of the target pixel in the same frame as the frame of the target pixel;
Of the pixel of interest (A) and the four pixels (E) on the right of the pixel of interest in the same frame as the frame of the pixel of interest, five pixels in the right direction (+ h direction) starting from the pixel of interest are 5 of the tap, the target pixel (A), and the four pixels (F) above the target pixel in the same frame as the target pixel frame.
The pixels are set in a downward direction (−v) with the target pixel as a starting point, which is a tap in the + v direction, the target pixel (A), and four pixels (G) below the target pixel in the same frame as the target pixel frame.
Assuming that the five pixels in the (direction) are referred to as taps in the −v direction, the variance calculators 31 to 36 respectively indicate taps in the + t direction, taps in the −t direction, taps in the + h direction, and taps in the −h direction. Taps, taps in the + v direction, and taps in the -v direction are received, and the variance is calculated for each. This + t direction tap, -t direction tap, + h
Direction tap, -h direction tap, + v direction tap,
The variance of taps in the −v direction is supplied to the threshold processing units 37 to 42, respectively.

The threshold processing unit 37 determines the magnitude relationship between the variance of the tap in the + t direction and the estimated noise amount for the pixel of interest, and outputs a 1-bit code corresponding to the determination result to the shifter 48. You. That is, the variance of the tap in the + t direction is sufficiently larger than the estimated noise amount of the target pixel, and therefore, (the pixel value of) the input pixel in the temporally preceding direction (−t direction) starting from the target pixel. Is small, the threshold processing unit 37
Is a code representing that effect, of 0 or 1,
For example, 1 is output.

Further, the variance of the tap in the + t direction is not sufficiently larger than the estimated noise amount of the target pixel, and therefore, the input pixel in the temporally preceding direction (−t direction) starting from the target pixel. If the continuity is large, the threshold processing unit 37 sets the code
Alternatively, for example, 0 of 1 is output.

In the threshold processing sections 38 to 42, as in the case of the threshold processing section 37, taps in the -t direction,
A magnitude relationship between each of the variances of the taps in the + h direction, the taps in the −h direction, the taps in the + v direction, and the taps in the −v direction and the estimated noise amount for the target pixel is determined.
The one-bit code corresponding to the result of the determination is
3 to 47 are output.

In the shifter 48, the 1 from the threshold processing section 37
Bit code is shifted right by one bit (LSB (Least
Shift from the Significant Bit) to the MSB (Most Significant Bit) direction and output to the arithmetic unit 43. In the arithmetic unit 43, the 1-bit code output from the threshold processing unit 43 is added to the output of the shifter 48, and the result is supplied to the shifter 49. In the shifter 49, the output of the arithmetic unit 43 is shifted right by one bit and output.

The arithmetic units 44 to 47 include a threshold processing unit 39
42 to 42 and the outputs of shifters 49 to 52 are supplied, and the outputs of arithmetic units 44 to 46 are supplied to shifters 50 to 52, respectively. I have. And the arithmetic unit 44
In steps 47 to 47, the same addition as in the arithmetic unit 43 is performed, and in the shifters 50 to 52, the shifter 4
The 1-bit right shift similar to that in the cases of 8 and 49 is performed, whereby the arithmetic unit 47 outputs the continuity in each of the + t direction, the −t direction, the + h direction, the −h direction, the + v direction, and the −v direction. A 6-bit code in which the code to be displayed is sequentially arranged from the MSB is output. This 6-bit code is
It is supplied to the coefficient RAM 5 in FIG. 1 as the class code (classification result) of the pixel of interest.

As described above, the class classification circuit 4 classifies the pixel of interest based on the stationarity of the class tap in each direction, and outputs a class code as a result of the classification. Therefore, according to this class code, the target pixel is classified according to its stationarity.

FIG. 6 shows an example of the configuration of an embodiment of a learning apparatus for obtaining a prediction coefficient for each class stored in the coefficient RAM 5 of FIG.

An original image serving as teacher data y is supplied to the frame memory 61 in, for example, a frame unit.
It is designed to be stored temporarily. Noise addition circuit 62
Reads the original image stored as the teacher data y in the prediction coefficient learning stored in the frame memory 61, and superimposes the noise on the original pixels constituting the original image, so that the noise as the student data is obtained. (Including the following)
A noise image). This noise image is supplied to the frame memory 63.

The frame memory 63 includes a noise adding circuit 6
2 is temporarily stored.

Note that the frame memories 61 and 63
It has the same configuration as the frame memory 1 in FIG.

The class tap generation circuit 64 or the prediction tap generation circuit 65 uses the pixels constituting the noise image stored in the frame memory 63 (hereinafter, appropriately referred to as noise pixels) and uses the class tap generation circuit 2 or the In the same manner as the prediction tap generation circuit 3, a class tap or a prediction tap is formed for the pixel of interest, and supplied to the class classification circuit 66 or the addition circuit 67, respectively.

The classifying circuit 66 has the same configuration as the classifying circuit 4 shown in FIG.
Class tap generation circuit 6 using the estimated noise amount from
Based on the continuity of the class tap from No. 4, the target pixel is classified into classes, and the corresponding class codes are given to the prediction tap memory 68 and the teacher data memory 70 as addresses.

The addition circuit 67 reads the storage value of the address corresponding to the class code output from the class classification circuit 66 from the prediction tap memory 68, and forms the storage value and the prediction tap from the prediction tap generation circuit 65. By adding the noise pixel, an operation corresponding to a summation (Σ) which is a multiplier of the prediction coefficient w on the left side of the normal equation of Expression (7) is performed. Then, the addition circuit 67
Is configured to store the calculation result in an address overwriting the address corresponding to the class code output from the class classification circuit 66.

The prediction tap memory 68 reads out the stored value of the address corresponding to the class output from the classifying circuit 66 and supplies it to the adding circuit 67, and stores the output value of the adding circuit 67 at that address. Has become.

The adder circuit 69 outputs the target pixel x of the original pixels constituting the original image stored in the frame memory 61.
Is read out as teacher data y, the stored value of the address corresponding to the class code output from the classifying circuit 66 is read out from the teacher data memory 70, and the stored value and the teacher data (read out from the frame memory 61) are read. By adding the original pixel) y, the summation (Σ) on the right side of the normal equation of Expression (7) is obtained.
An operation corresponding to is performed. Then, the adding circuit 69 stores the result of the calculation in an overwritten form at the address corresponding to the class code output from the class classification circuit 66.

Note that, to be precise, the adders 67 and 69
Then, the multiplication in Expression (7) is also performed. The right side of the equation (7) includes multiplication of the teacher data y and the noise pixel x. Therefore, the multiplication performed by the addition circuit 69 includes not only the teacher data y but also the teacher data y. The noise pixel x is required, and is read from the frame memory 63 in the addition circuit 69.

The teacher data memory 70 reads the storage value of the address corresponding to the class code output from the class classification circuit 66, supplies the read value to the addition circuit 69, and stores the output value of the addition circuit 69 at that address. It has become.

The arithmetic circuit 71 sequentially reads out the storage values stored at the addresses corresponding to the respective class codes from the prediction tap memory 68 or the teacher data memory 70, and expresses each of the class codes by the equation (7). By setting up the normal equation and solving it, a prediction coefficient for each class is obtained. That is, the arithmetic circuit 7
1 is a prediction tap memory 68 or a teacher data memory 7
From the stored value stored in the address corresponding to each class code of 0, a normal equation of equation (7) is set and solved to obtain a prediction coefficient for each class.

The noise amount estimating circuit 72 is configured similarly to the noise amount estimating circuit 7 shown in FIG.
The amount of noise included in each noise pixel of the noise image stored in 1 is estimated, and the estimated amount of noise obtained as a result is supplied to the class tap generation circuit 64.

Next, referring to the flowchart of FIG.
A learning process for obtaining a prediction coefficient for each class, which is performed in the learning device of FIG. 6, will be described.

An original image (moving image) as teacher data is supplied to the learning device in units of frames, and the original images are sequentially stored in the frame memory 61.

In step S11, the noise adding circuit 62 reads the original image stored in the frame memory 61 and adds noise to generate a noise image. This noise image is supplied to and stored in the frame memory 63.

Thereafter, in step S12, the noise amount estimating circuit 72 sets the noise image of the predetermined frame stored in the frame memory 63 as the target frame, and calculates the noise amount of the target frame as the noise amount described in FIG. The estimation is performed in the same manner as in the estimation circuit 7. And
The estimated noise amount obtained as a result is supplied to the class tap generation circuit 64.

Here, since the noise included in the frame of interest is added by the noise adding circuit 62,
In the learning device, an accurate value can be obtained, and such an accurate value can be supplied to the class tap generating circuit 64. However, in the noise removing device in FIG. Since the noise amount is estimated at,
Such an accurate value cannot always be obtained.
Therefore, in the learning device, in an environment that matches the environment in which the noise removing device is used as much as possible, in order to obtain a prediction coefficient, the noise included in the frame of interest is subjected to noise amount estimation configured similarly to the noise amount estimation circuit 7. The estimation is performed in the circuit 72.

In step S12, when the noise amount of each noise pixel of the frame of interest is estimated, step S1
In 3, the class tap generation circuit 64 or the prediction tap generation circuit 65 reads a certain noise pixel of the frame of interest as a pixel of interest, reads noise pixels around the noise pixel from the frame memory 63, and reads the class tap shown in FIG. Alternatively, each prediction tap is configured. The class tap or the prediction tap is supplied to the classifying circuit 66 or the adding circuit 67, respectively.

In step S14, the class classification circuit 66 uses the estimated noise amount from the noise amount estimation circuit 72 as in the case of the class classification circuit 4 described with reference to FIG. Are classified into classes based on the continuity of, and a class code as a result of the classification is given to the prediction tap memory 68 and the teacher data memory 70 as an address.

Then, the process proceeds to a step S15, where prediction taps or teacher data are added.

That is, in step S 15, the prediction tap memory 68 reads the stored value of the address corresponding to the class code output from the class classification circuit 66 and supplies the read value to the addition circuit 67. The addition circuit 67 includes a storage value supplied from the prediction tap memory 68 and a prediction tap generation circuit 65.
Using the noise pixels that constitute the prediction taps supplied from, a calculation corresponding to a summation (Σ) that is a multiplier of a prediction coefficient on the left side of the normal equation of Expression (7) is performed. Then, the adding circuit 67 calculates the calculation result as
The data is overwritten and stored in the address of the prediction tap memory 68 corresponding to the class code output from the class classification circuit 66.

Further, at step S 15, the teacher data memory 70 reads out the stored value of the address corresponding to the class code output from the classifying circuit 66 and supplies it to the adding circuit 69. The addition circuit 69 includes a frame memory 61
Among the original pixels constituting the original image stored in the frame memory 63, the original pixel corresponding to the pixel of interest is read out as teacher data, and the teacher data among the noise pixels constituting the noise image stored in the frame memory 63 is read out. Then, using the read pixel and the stored value supplied from the teacher data memory 70, an operation corresponding to the summation (Σ) on the right side of the normal equation of Expression (7) is performed. Then, the adding circuit 69 calculates the calculation result as
The data is overwritten and stored in the address of the teacher data memory 70 corresponding to the class code output from the class classification circuit 66.

Thereafter, the flow advances to step S16 to determine whether or not all the noise pixels constituting the frame of interest stored in the frame memory 63 have been processed as the pixel of interest. Returning to step S13, the same process is repeated hereafter with a noise pixel that has not been set as the target pixel as a new target pixel.

On the other hand, if it is determined in step S16 that all the noise pixels forming the frame of interest have been processed as the pixel of interest, the process proceeds to step S17.
It is determined whether or not the original image to be processed next is stored in the frame memory 61. If it is determined in step S17 that the original image to be processed next is stored in the frame memory 61, the process returns to step S11, and the process returns to step S11 for the next original image to be processed.
The processing of 11 and below is repeated.

In step S17, when it is determined that the original image to be processed next is not stored in the frame memory 61, that is, the processing is performed on all the original images prepared for learning in advance. If so, the process proceeds to step S18, where the arithmetic circuit 71 sequentially reads out the storage values stored at the addresses corresponding to the respective class codes from the prediction tap memory 68 or the teacher data memory 70, respectively, and expresses the values in the equation (7). The prediction coefficients for each class are obtained by setting up the normal equations and solving them. Further, in step S19, the arithmetic circuit 71 outputs the obtained prediction coefficient for each class, and ends the processing.

In the above-described prediction coefficient learning process, there may be a case where a class in which the number of normal equations necessary for obtaining the prediction coefficient cannot be obtained may occur. For example, it is possible to output a default prediction coefficient.

As described above, the target pixel is classified into classes based on the variance of the class taps for the target pixel, and the prediction coefficient for each class is obtained. A prediction coefficient suitable for removing noise of a pixel is obtained, and as a result, by performing a classification adaptive process using such a prediction coefficient, particularly for a moving image, the noise is effectively removed. It is possible to do.

Next, the above-described series of processing can be performed by hardware or software. When a series of processing is performed by software, a program constituting the software is
It is installed in a computer incorporated in a noise elimination device or a learning device as dedicated hardware, or a general-purpose computer that performs various processes by installing various programs.

With reference to FIG. 8, a description will now be given of a medium used to install a program for executing the above-described series of processing in a computer and to make the computer executable.

As shown in FIG. 8A, the program
It can be provided to a user in a state where it is installed in advance on a hard disk 102 as a recording medium built in the computer 101.

Alternatively, the program may be implemented as shown in FIG.
As shown in the figure, the floppy disk 111, CD-ROM (Com
pact Disc Read Only Memory) 112, MO (Magneto opti)
cal) Disc 113, DVD (Digital Versatile Disc) 1
14, temporarily or permanently stored in a recording medium such as a magnetic disk 115 or a semiconductor memory 116, and can be provided as package software.

Further, as shown in FIG. 8C, the program is transmitted from a download site 121 to a computer 12 via an artificial satellite 122 for digital satellite broadcasting.
3 or wirelessly to a computer 123 via a network 111 such as a LAN (Local Area Network) or the Internet.
In this case, the data can be stored in a built-in hard disk or the like.

The medium in the present specification means a broad concept including all these media.

[0138] In this specification, steps for describing a program provided by a medium need not necessarily be processed in chronological order according to the order described in the flowchart, but may be performed in parallel or individually. (For example, parallel processing or object processing)
Is also included.

In the class classification application process, learning is performed to obtain a prediction coefficient for each class using teacher data and student data, and linear primary prediction using the prediction coefficient and the input data is performed to obtain a prediction coefficient from the input data. Since a prediction value of teacher data for the input data is obtained, a prediction coefficient for obtaining a desired prediction value can be obtained by teacher data and student data used for learning. That is, for example, a prediction coefficient for improving the resolution can be obtained by using a high-resolution image as teacher data and using an image with a reduced resolution of the image as student data. Further, for example, while using an image in which edges are emphasized as teacher data,
By using an image in which the edge is blurred as the student data, a prediction coefficient that emphasizes the edge can be obtained. Therefore, as described above, the present invention is applicable not only to the case where noise is removed from an input image, but also to the case where the resolution of the input image is improved, the case where edges are emphasized, the case where waveform equalization is performed, and the like. .

In the present embodiment, a moving image is subjected to the class classification application processing. However, in addition to a moving image, a still image, a sound, and a signal reproduced from a recording medium (RF)
(RadioFrequency) signal) and the like.

Furthermore, in the present embodiment, the variance of the class tap in each direction is compared with the estimated noise amount, and the target pixel is classified based on the comparison result. Classification is performed by comparing the variance in each direction of the class tap with their average value and performing the variance in each direction of the class tap based on the comparison result.
nge Coding) processing, and can be performed based on the ADRC result. Here, in the ADRC processing, for example, for a certain set of data, the maximum value MAX and the minimum value MIN of the data constituting the set are detected, and DR
= MAX-MIN is the local dynamic range of the set,
Based on the dynamic range DR, data forming the set is requantized to K bits. That is, the minimum value MIN is subtracted from each data in the set, and the subtracted value is DR / 2 ^K
Is divided (quantized) by

As described above, when the class of the target pixel is classified without comparing the variance of the class tap with the estimated noise amount, the variance of the class tap includes the class tap. Since the noise component is included in addition to the stationary component of the pixel itself, the classification is performed with some influence of the noise.

Further, in this embodiment, the noise elimination device and the learning device for learning the prediction coefficient for each class used in the noise elimination device are configured as separate devices. It is also possible to configure integrally with the learning device. Then, in this case, it is possible to make the learning device perform learning in real time, and update the prediction coefficient used in the noise removal device in real time.

In this embodiment, the coefficient RAM 5 stores
Although the prediction coefficient for each class is stored in advance, the prediction coefficient may be supplied to, for example, a noise removal device together with the input image.

Further, the pixels constituting the class tap are:
It is not limited to the pixels having the positional relationship as shown in FIG.

In the present embodiment, the class tap has six directions (+ t direction, -t direction, + h direction,
Although the variance in the −h direction, the + v direction, and the −v direction) is obtained to perform the classification, the variance in an oblique direction may be obtained and used for the classification. Further, the classification can be performed by calculating the variance of the pixels on the curved line instead of the pixels on the straight line extending in a certain direction.

Further, in the present embodiment, a linear linear expression is used in the adaptive processing.
It is also possible to use an equation of a second or higher order.

[0148]

According to the data processing apparatus according to the first aspect, the data processing method according to the ninth aspect, and the medium according to the tenth aspect, the input data and the vicinity of the target input data of interest are determined. Is extracted, and the input data of interest is classified into classes based on the stationarity of the peripheral data, and the corresponding class code is output. Then, output data for the target input data is predicted using a predetermined prediction coefficient corresponding to the class code. Therefore, for example, noise can be effectively removed from the input data.

A data processing apparatus according to claim 11, a data processing method according to claim 19, and a twentieth aspect.
According to the medium described in (1), student data to be a student is generated from teacher data to be a teacher for learning a prediction coefficient, and from the student data, peripheral data around the noted student data of interest is obtained. Is extracted. Then, based on the continuity of the surrounding data, the student data of interest is classified into classes, and corresponding class codes are output. Using the teacher data and the student data, for each class code,
A prediction coefficient is determined. So, for example, from the data:
It is possible to obtain a prediction coefficient from which noise can be effectively removed.

According to the data processing apparatus of the twenty-first aspect, the first peripheral data around the target input data of interest is extracted from the input data, and the continuity of the first peripheral data is determined. Based on this, the input data of interest is classified into classes, and the corresponding class codes are output. Then, output data for the target input data is predicted using a predetermined prediction coefficient corresponding to the class code.
On the other hand, student data to be a student is generated from teacher data to be a teacher for learning a prediction coefficient, and second peripheral data around the noted student data of interest is extracted from the student data. . Then, based on the stationarity of the second peripheral data, the student data of interest is classified and a corresponding class code is output, and a prediction coefficient is calculated for each class code using the teacher data and the student data. Can be Therefore, for example, it is possible to obtain a prediction coefficient that can effectively remove noise from data, and it is possible to effectively remove noise from data using the prediction coefficient. .

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of an embodiment of a noise removal device to which the present invention has been applied.

FIG. 2 is a diagram illustrating a configuration example of a class tap.

FIG. 3 is a flowchart illustrating a noise removal process performed by the noise removal device of FIG. 1;

FIG. 4 is a block diagram illustrating a configuration example of a noise amount estimation circuit 7 in FIG. 1;

FIG. 5 is a block diagram illustrating a configuration example of a class classification circuit 4 of FIG. 1;

FIG. 6 is a block diagram illustrating a configuration example of an embodiment of a learning device to which the present invention has been applied.

FIG. 7 is a flowchart illustrating a learning process performed by the learning device in FIG. 6;

FIG. 8 is a diagram for explaining a medium to which the present invention is applied.

[Explanation of symbols]

1 frame memory, 2 class tap generation circuit,
3 prediction tap generation circuit, 4 class classification circuit, 5
Coefficient RAM, 6 prediction operation circuit, 7 noise amount estimation circuit, 11 to 18 delay circuit, 19 variance calculation unit, 20 variance accumulation memory, 21 variance value frame average calculation unit, 22 to 26 stillness determination unit, 27 continuous still position detection Part, 28 memory controller, 2
9 motion vector detection circuit, 31 to 36 variance calculation unit, 37 to 42 threshold processing unit, 43 to 47 arithmetic unit, 48 to 52 shifter, 61 frame memory, 62 noise addition circuit, 63 frame memory,
64 class tap generation circuit, 65 prediction tap generation circuit, 66 class classification circuit, 67 addition circuit,
68 prediction tap memory, 69 addition circuit, 70
Teacher data memory, 71 arithmetic circuit, 72 noise amount estimation circuit, 101 computer, 102 hard disk, 103 semiconductor memory, 111 floppy disk, 112 CD-ROM, 113 MO disk, 114 DVD, 115 magnetic disk, 11
6 semiconductor memory, 121 download site,
122 satellites, 123 computers, 131 networks

Continued on the front page F term (reference) 5B056 BB02 BB62 BB64 FF01 FF02 FF06 GG03 HH03 5B057 CE02 CH07 5C021 PA42 PA52 PA56 PA66 PA72 PA76 PA78 PA83 RA01 RA06 RA16 RB06 RB07 SA24 YA01

Claims (21)

[Claims] 1. A data processing device for processing input data and predicting output data corresponding to the input data, comprising: extracting, from the input data, peripheral data around the target input data of interest. Means, based on the continuity of the peripheral data, classifies the input data of interest, class classification means that outputs a corresponding class code, using a predetermined prediction coefficient corresponding to the class code,
A prediction unit for predicting output data for the input data of interest. 2. The data processing apparatus according to claim 1, wherein the prediction unit performs linear primary prediction on the output data using the prediction coefficient. 3. The data processing apparatus according to claim 1, further comprising a storage unit that stores the prediction coefficient for each of the class codes. 4. The method according to claim 1, further comprising estimating means for estimating an amount of noise included in the input data, wherein the classifying means determines continuity of the surrounding data based on the amount of noise, and hand,
2. The data processing apparatus according to claim 1, wherein the target input data is classified. 5. The classifying means obtains a variance of the peripheral data, determines continuity of the peripheral data based on the variance, and classifies the input data of interest based on the continuity. The data processing device according to claim 1, wherein: 6. The method according to claim 1, further comprising estimating means for estimating an amount of noise included in the input data, wherein the classifying means compares a variance of the surrounding data with the amount of noise to determine a steady state of the surrounding data. 6. The data processing apparatus according to claim 5, wherein the data is determined based on the stationarity, and the input data of interest is classified based on the stationarity. 7. The data processing apparatus according to claim 1, wherein the input data is image data. 8. The image processing apparatus according to claim 1, wherein the extracting unit includes:
The data processing apparatus according to claim 7, wherein a peripheral pixel spatially or temporally peripheral to the target pixel of interest is extracted. 9. A data processing method for processing input data and predicting output data corresponding to the input data, comprising: extracting, from the input data, peripheral data around the target input data of interest. Based on the stationarity of the peripheral data, based on the continuity of the surrounding data, a class classification step of classifying the input data of interest and outputting a corresponding class code, using a predetermined prediction coefficient corresponding to the class code,
A prediction step of predicting output data for the input data of interest. 10. A medium for causing a computer to execute a program for processing input data and performing data processing for predicting output data for the input data, the medium comprising: An extraction step of extracting peripheral data around the data; a class classification step of classifying the input data of interest based on the stationarity of the peripheral data and outputting a corresponding class code; Using a predetermined prediction coefficient
A medium for causing the computer to execute a program, comprising: a prediction step of predicting output data with respect to the target input data. 11. A data processing device for processing input data and learning a prediction coefficient used for predicting output data corresponding to the input data, comprising: Generating means for generating student data to be a student; extracting means for extracting, from the student data, peripheral data around the target student data of interest; and, based on the stationarity of the peripheral data, the target student A data classifying unit that classifies data and outputs a corresponding class code; and a calculating unit that calculates the prediction coefficient for each class code using the teacher data and the student data. Processing equipment. 12. The data processing method according to claim 11, wherein said calculating means obtains said prediction coefficient for obtaining said teacher data by linear primary prediction using said student data. apparatus. 13. The generating means according to claim 1, wherein
The data processing apparatus according to claim 11, wherein the student data is generated by adding noise. 14. The method further comprising estimating means for estimating an amount of noise included in the student data, wherein the class classification means determines the continuity of the surrounding data based on the amount of noise, and based on the continuity. hand,
The data processing apparatus according to claim 11, wherein the attention student data is classified. 15. The class classification means obtains a variance of the peripheral data, determines continuity of the peripheral data based on the variance, and classifies the student data of interest based on the continuity. The method of claim 11, wherein
A data processing device according to claim 1. 16. The method according to claim 16, further comprising estimating means for estimating a noise amount included in the student data, wherein the classifying means compares a variance of the peripheral data with the noise amount to thereby determine a steady state of the peripheral data. 16. The data processing apparatus according to claim 15, wherein gender is determined, and the student data of interest is classified based on the stationarity. 17. The teacher data and student data are:
The data processing device according to claim 11, wherein the data is image data. 18. The image processing apparatus according to claim 18, wherein the extracting unit is configured to extract a pixel of interest from the image data as the student data.
18. The data processing apparatus according to claim 17, wherein peripheral pixels that are spatially or temporally peripheral are extracted. 19. A data processing method for processing input data and learning a prediction coefficient used for predicting output data for the input data, comprising: A generating step of generating student data to be a student; an extracting step of extracting peripheral data around the noted student data of interest from the student data; and A classifying step of classifying data and outputting a corresponding class code; and a calculating step of obtaining the prediction coefficient for each class code using the teacher data and the student data. Processing method. 20. A medium for causing a computer to execute a program for processing input data and performing a data process of learning a prediction coefficient used for predicting output data for the input data, the program comprising: A generating step of generating student data to be a student from teacher data to be a teacher for learning; an extracting step of extracting peripheral data around the noted student data of interest from the student data; A class classification step of classifying the noted student data based on data continuity and outputting a corresponding class code; and calculating the prediction coefficient for each class code using the teacher data and the student data. A medium for causing the computer to execute a program, comprising: a calculating step. 21. A data processing apparatus comprising: a first device that processes input data and predicts output data for the input data; and a second device that learns a prediction coefficient used to predict the output data. An apparatus, wherein the first apparatus is configured to: extract, from the input data, first peripheral data around the target input data of interest; A first class classification unit that classifies the input data of interest based on the stationarity and outputs a corresponding class code; and a predetermined prediction coefficient corresponding to the class code.
Prediction means for predicting output data with respect to the noted input data, wherein the second device generates student data to be a student from teacher data to be a teacher for learning the prediction coefficient; Second extracting means for extracting, from the student data, second peripheral data around the target student data of interest; and classifying the target student data based on the stationarity of the second peripheral data. Data comprising: a second class classifying unit that classifies and outputs a corresponding class code; and a calculating unit that calculates the prediction coefficient for each class code using the teacher data and the student data. Processing equipment.