CN1788499A - Video quality evaluation device, video quality evaluation method, video quality evaluation program, video matching device, video matching method, and video matching program - Google Patents

Abstract

A subjective quality estimation unit (11) receives a reference video signal (RI) which is a video signal not to be degraded and a degraded video signal (PI) obtained by degrading the reference video signal, calculates a video signal feature amount of both signals, and estimates the subjective quality of the degraded video signal from the difference between the calculated video signal feature amounts of both signals. A feature value calculation unit (12) calculates a video signal feature value of a reference video signal. A correction information storage unit (13) stores correction information for correcting subjective quality in association with the video signal feature amount. And a correction calculation unit (14) that acquires correction information corresponding to the video signal feature of the reference video signal input from the feature calculation unit (12) from the correction information storage unit (13) and transfers the correction information to the correction unit (15). And a correction unit (15) for correcting the subjective quality of the degraded video signal estimated by the subjective quality estimation unit (11) on the basis of the forwarded correction information.

Description

Image quality evaluation device, image quality evaluation method, and image quality evaluation program, and image matching device, image matching method, and image matching program

technical field

The present invention relates to a video quality evaluation device, a video quality evaluation method, and a video quality evaluation that estimate subjective quality from measurement of physical feature quantities of video signals instead of a subjective quality evaluation test in which people view actual video. program.

In addition, the present invention also relates to the determination of spatial and temporal positions between a reference video signal and a degraded video signal whose quality has deteriorated through encoding or network transmission in the video quality evaluation device, video quality evaluation method, and video quality evaluation program. The matching image matching device, image matching method and image matching program.

Background technique

Generally, the quality of image information will deteriorate through certain processing, such as encoding or network transmission. Regarding such a deteriorated video, the degree of deterioration perceived by people after actually seeing it is called subjective quality.

Conventionally, as a method of evaluating this subjective quality, a subjective quality evaluation test based on what people actually see of a video has been performed. However, for people to evaluate the quality of the video after actually seeing the video, there is a problem that it takes a lot of time and effort to evaluate the quality of the video. In addition, people who are proficient in image quality evaluation and beginners often have different evaluations.

Therefore, as a method of estimating the subjective quality of video by physical measurement, there is the difference between the physical feature quantity of the reference video signal and the degraded degraded video signal, or based on the deterioration amount obtained only from the physical characteristic quantity of the degraded video signal. A method for accurately assessing the degree of deterioration in quality.

However, if the subjective quality of the video is limited, the subjective evaluation value can sometimes be estimated with high accuracy (for example, refer to ANSI T1.801.03-1996, "Digital Transport of One-Way Video Signal Parameters for Objective Performance Assessment"; Okamoto , Takahashi, "Discussion on Applicable Laws of Objective Image Quality Evaluation Technology", Shingaku University, Sep.2002; Okamoto, Kurita, Takahashi "Discussion on Improving the Performance of Image Quality Objective Evaluation", Shingaku University , Mar.2003), but for any image, because it greatly depends on the nature of the image, even if the degree of deterioration is the same, the quality of the result judged will be different.

For this reason, in the case of seeking the subjective quality of an image, there are still many subjective evaluation tests in which people actually see the image and evaluate its quality. Such a technical situation continues.

In such a situation, an objective evaluation method has been proposed that focuses on the edge area of the image, applies a Sobel filter to the image signal, and calculates the amount of deterioration (see PCT Pub. No.: WO99/45715).

However, sufficient evaluation accuracy to replace actual subjective evaluation cannot be obtained only with such feature quantities.

On the other hand, in the method of estimating the subjective quality of video, it is necessary to match the spatial and temporal positions of the reference video signal and the degraded video signal as a prerequisite for comparing the physical feature quantities of the reference video signal and the degraded degraded video signal. That is, it is necessary to match the deviation in the time direction or the deviation in the spatial position between the reference video signal and the degraded video signal.

Originally, such a matching process was performed manually. Soon, however, a scheme for automatically performing the matching process was proposed. For example, in US Pat. No. 5,446,492, delays in time matching processing can be dealt with as pre-processing for objective evaluation.

According to the technology of this document, as in television broadcasting, if the target is a signal of the same size and frame rate, once synchronization is obtained, good synchronization can be obtained thereafter.

However, in video distribution or communication services via IP networks such as the Internet that have become popular in recent years, since a personal computer (PC) terminal receives video, the received video has various sizes and aspect ratios. In addition, there is a case where information concentrated to some extent such as data packets is lost. Therefore, it is difficult to obtain spatial position correspondence between the reference video signal and the degraded video signal. On the other hand, due to fluctuations in IP packet arrival intervals or occurrence of packet loss, phenomena such as timing deviation of video display on the time axis or fluctuations in video display timing or freezes occur that did not exist in the past. Therefore, there is no method of achieving such temporally offset matching between the reference video signal and the degraded video signal.

In the method of estimating the subjective quality of video, if the spatial and temporal position matching of the reference video signal and the degraded video signal cannot be sufficiently obtained, it will be impossible to estimate the subjective quality of video, and the accuracy will be low even if it is possible.

In addition, in the aforementioned PCT Pub. No.: WO99/45715, as a document describing an example of temporal matching processing, ITU-T Contribution COM-12-29 "Draft new recommendation on multimedia communication delay, synchronization, and framerate measurement”, December 1997.

Contents of the invention

The present invention is made in view of the above-mentioned prior art, and an object of the present invention is to provide a video quality evaluation device, a video quality evaluation method, and a video quality evaluation program capable of estimating a unified subjective quality of an arbitrary video with high accuracy.

Another object of the present invention is to provide an image matching device, an image matching method, and an image matching device that can reliably obtain their spatial and temporal position matching when comparing physical feature quantities of a reference image signal and a degraded image signal in estimating subjective quality. program.

In order to achieve the above objects, the gist of the first aspect of the invention is an image quality evaluation device that includes: inputting a reference image signal that is an undegraded image signal and a deteriorated image signal after the reference image signal has been deteriorated, and calculating an image of the two signals. a signal feature value, a subjective quality estimating unit for estimating the subjective quality of the degraded video signal based on the calculated difference between the video signal feature values of the two signals; an information correction information storage unit; input the video signal feature value of the reference video signal, obtain correction information corresponding to the input video signal feature value from the correction information storage unit, and correct the The subjective quality correction part of the estimated subjective quality.

The gist of the invention of the second aspect is that in the invention of the first aspect, the subjective quality estimating unit uses the spatial information indicating the video state in each frame of the video contained in the video signal and the video information between frames. The subjective quality is estimated from a difference between the degraded video signal and the reference video signal of at least any one of the changed temporal information.

The gist of the third aspect of the invention is that, in the second aspect of the invention, the subjective quality estimating unit calculates, as a difference in the video signal feature value between the degraded video signal and the reference video signal, the At least one of an edge power amount (E) of a deterioration amount in each frame of a video included in the video signal and a mobile power amount (M) indicating a deterioration amount between frames.

The gist of the fourth aspect of the invention is that, in the second aspect of the invention, the subjective quality estimating unit, based on at least any one of spatial information and temporal information specified in ITU-R Recommendation P.910, degraded video signal and The difference between the reference video signals estimates the subjective quality.

The gist of the fifth aspect of the invention is that, in the first aspect of the invention, the correction information storage unit, as the correction information for correcting the subjective quality, is related to the image in each frame of the image included in the image signal. The spatial information of the state and the temporal information indicating the change of the image between frames are stored in association with correction coefficients.

The gist of the sixth aspect of the invention is that, in the first aspect of the invention, the subjective quality estimating unit includes: inputting the reference video signal and the degraded video signal, and generating a reference video from the reference video signal; a calibration information generation unit for calibration information of a temporal deviation of a degraded video frame from the degraded video signal, and a calibration information of a spatial deviation between the reference video frame and the degraded video frame; After the spatial offset and the temporal offset are determined, a spatial feature calculation unit that calculates respective spatial feature quantities based on the spatial information representing the image states of the reference image frame and the degraded image frame; based on the calibration information, After the spatial deviation and the temporal deviation are eliminated, for the reference video frame and the degraded video frame, the temporal feature value calculation of calculating the respective temporal feature values based on the time information representing the image change between the respective frames and an estimating unit for estimating the subjective quality of the degraded video signal based on the spatial feature value and the temporal feature value.

In addition, in order to achieve the above object, the seventh aspect of the invention is a method for evaluating video quality, which includes storing a degraded reference video signal for correcting a non-degraded video signal in association with the feature value of the video signal. A video quality evaluation method in a correction information storage unit for correction information of subjective quality of which a degraded video signal is estimated, and a video quality evaluation device for estimating the subjective quality of the degraded video signal; characterized in that the reference video signal and For the degraded video signal, the video signal feature values of the two signals are calculated, and the subjective quality of the degraded video signal is estimated based on the difference between the calculated video signal feature values of the two signals, and obtained and calculated from the correction information storage unit. Correction information corresponding to the video signal feature quantity of the reference video signal, and correcting the estimated subjective quality according to the obtained correction information.

In addition, in order to achieve the above-mentioned object, the invention of the eighth aspect is a video quality evaluation program characterized in that a computer is operated as a unit for inputting a reference video signal which is an undegraded video signal and a degraded reference video signal. The degraded image signal, calculate the image signal characteristic quantity of two signals, according to the difference of calculated image signal characteristic quantity of two signals, estimate the subjective quality estimation unit of the subjective quality of described deteriorated image signal; Correspond to the image signal characteristic quantity a correction information storage unit that stores correction information for correcting the subjective quality; obtaining correction information corresponding to the calculated image signal feature quantity of the reference image signal from the correction information storage unit, and correcting the Subjective quality correction unit for the estimated subjective quality.

In addition, in order to achieve the above object, the ninth aspect of the invention is a video quality evaluation device characterized by comprising: inputting a reference video signal which is an undegraded video signal and a degraded video signal obtained by degrading the reference video signal; Calibration information generation of calibration information regarding a temporal deviation between a reference video frame from the reference video signal and a degraded video frame from the degraded video signal, and a spatial deviation between the reference video frame and the degraded video frame according to the calibration information, after eliminating the spatial deviation and the temporal deviation, calculating the spatial characteristics of the respective spatial characteristic quantities according to the spatial information representing the image state of the reference image frame and the deteriorated image frame An amount calculation unit; after eliminating the spatial deviation and the temporal deviation based on the calibration information, for the reference video frame and the degraded video frame, calculate according to the time information representing the video change between the respective frames a temporal feature calculation unit for each temporal feature; and a subjective quality estimation unit for estimating the subjective quality of the degraded video signal based on the spatial feature and the temporal feature.

The gist of the tenth aspect of the invention is that, in the ninth aspect of the invention, further comprising: converting the file format of the degraded video included in the degraded video signal into a reference file format included in the corresponding reference video signal. A format conversion unit that outputs information related to the unified file format along with the file format of the video, and a correction coefficient that stores a coefficient for estimating the subjective quality of the degraded video signal corresponding to the information related to the file format a storage unit; the subjective quality estimating unit obtains, from the correction coefficient storage unit, the coefficient corresponding to the information associated with the unified document format input from the format conversion unit, and based on the spatial feature value, the The subjective quality of the degraded video signal is estimated based on the temporal feature value and the acquired coefficient.

The gist of the eleventh aspect of the invention is that, in the tenth aspect of the invention, the format conversion unit outputs, as information related to the unified file format, the signal format of the degraded video signal, obtained by the degraded At least one of the amount of information of the degraded video transmitted from the video signal and the encoding method of the degraded video signal, the correction coefficient storage unit corresponding to the signal format of the degraded video signal and transmitted from the degraded video signal The optimal coefficient is stored in at least any one of the amount of information of the degraded video and the encoding method of the degraded video signal.

The gist of the invention of the twelfth aspect is that, in the ninth or tenth aspect of the invention, the spatial feature value calculation unit quantifies the intra-frame luminance value based on the reference video signal and the degraded video signal An index of deterioration of sharply changing boundaries is calculated as the spatial feature quantity.

The gist of the invention of the thirteenth aspect is that, in the invention of the twelfth aspect, the spatial feature value calculating unit calculates the value specified in ANSIT1.801.03-1995 based on the reference video signal and the degraded video signal. The amount of marginal power is calculated as the spatial feature amount.

The gist of the fourteenth aspect of the invention is that, in the ninth, tenth, or twelfth aspect of the invention, the spatial feature value calculation unit compares the reference video frame and quantifies the value corresponding to the reference video frame. An index of occurrence of a boundary in which the luminance value of the degraded video frame changes sharply in the horizontal and vertical directions is calculated as the spatial feature value.

The gist of the fifteenth aspect of the invention is that, in the ninth, tenth, twelfth, or fourteenth aspect of the invention, the spatial feature value calculation unit calculates a value of at least 1 pixel in each frame. The difference between the Temporal Information value specified in ITU-R Recommendation P.910 of the reference image in the block of the set and the Temporal Information value specified in ITU-R Recommendation P.910 of the deteriorated image, and the amount of change between frames of the image is taken as The temporal feature quantity is calculated.

The gist of the sixteenth aspect of the invention is that, in the ninth or tenth aspect of the invention, there is further provided a correction information storage unit that stores correction information for correcting the subjective quality in association with the spatial feature value and the temporal feature value; Inputting the spatial feature value and the temporal feature value of the reference video signal, obtaining correction information corresponding to the input spatial feature value and temporal feature value from the correction information storage unit, and correcting the estimation based on the obtained correction information. The subjective quality correction part of the subjective quality.

In addition, in order to achieve the above object, the gist of the seventeenth aspect of the invention is a video quality evaluation method characterized by inputting a reference video signal which is an undegraded video signal and a degraded video signal obtained by degrading the reference video signal; generating calibration information regarding a temporal offset of a reference image frame from the reference image signal and a degraded image frame from the degraded image signal, and a spatial offset between the reference image frame and the degraded image frame; according to the The calibration information, after eliminating the spatial deviation and the temporal deviation, calculate respective spatial feature quantities according to the spatial information representing the image state of the reference image frame and the deteriorated image frame; according to the calibration information, After the spatial deviation and the temporal deviation are eliminated, for the reference image frame and the deteriorated image frame, calculate respective temporal feature quantities according to time information representing image changes between respective frames; according to the spatial The feature quantity and the temporal feature quantity estimate the subjective quality of the degraded video signal.

In addition, in order to achieve the above object, the gist of the eighteenth aspect of the invention is a video quality evaluation method characterized in that a reference video signal which is an undegraded video signal and a degraded video signal degraded by the reference video signal are input; Converting the file format of the degraded image included in the degraded image signal to the file format of the reference image included in the corresponding reference image signal; generating information associated with the unified file format; generating information about the reference image from the reference image Calibration of a temporal offset between a reference image frame of a signal and a degraded image from a degraded image whose file format has been converted from said degraded image signal, and a spatial offset between said reference image frame and said degraded image frame information; according to the calibration information, after the spatial deviation and the temporal deviation are eliminated, the respective spatial feature quantities are calculated according to the spatial information representing the image state of the reference image frame and the deteriorated image frame; according to the The calibration information, after eliminating the spatial deviation and the temporal deviation, for the reference image frame and the deteriorated image frame, calculate respective time feature values according to time information representing image changes between respective frames; The subjective quality of the degraded video signal is estimated based on the spatial feature value, the temporal feature value, and a coefficient for subjective quality estimation corresponding to information associated with the unified file format.

In addition, in order to achieve the above-mentioned object, the invention of the nineteenth aspect is a video quality evaluation program which causes a computer to operate as a means for inputting a reference video signal which is an undegraded video signal and a degradation result of the degradation of the reference video signal. A video signal, generating information about a temporal deviation between a reference video frame from the reference video signal and a degraded video frame from the degraded video signal, and a spatial deviation between the reference video frame and the degraded video frame A calibration information generation unit for calibration information; according to the calibration information, after eliminating the spatial deviation and the temporal deviation, calculate respective A spatial feature calculation unit for a spatial feature; according to the calibration information, after eliminating the spatial deviation and the temporal deviation, for the reference image frame and the deteriorated image frame, according to the respective inter-frame a temporal feature value calculation unit for calculating respective temporal feature values based on temporal information of video changes; and a subjective quality estimation unit for estimating subjective quality of the degraded video signal based on the spatial feature value and the temporal feature value.

The gist of the invention of the twentieth aspect is that, in the invention of the nineteenth aspect, the computer is further operated as a unit for converting the file format of the degraded video included in the degraded video signal into the corresponding A format conversion unit for outputting information associated with the unified file format while simultaneously outputting the file format of the reference image included in the reference image signal; and making it correspond to the information associated with the file format and storing a correction coefficient storage unit for estimating a coefficient of the subjective quality of the degraded video signal; the subjective quality estimation unit acquires from the correction coefficient storage unit a file format corresponding to the unified file format input from the format conversion unit The coefficients of the associated information are used to estimate the subjective quality of the degraded video signal based on the spatial feature value, the temporal feature value, and the acquired coefficients.

In addition, in order to achieve the above object, the invention of the twenty-first aspect is a video matching device characterized by comprising: inputting a reference video signal which is an undegraded video signal and a degraded video signal obtained by degrading the reference video signal, a format conversion unit for converting the file format of the degraded image included in the degraded image signal into the file format of the reference image included in the corresponding reference image signal; matching the reference image included in the reference image signal a display timing matching unit for the number of video frames and the degraded video frames included in the degraded video signal and display timing; A synchronization/position matching unit that obtains a frame deviation between the reference image frame and the degraded image frame or a frozen state of the degraded image, and obtains a correspondence between frames and a correspondence between pixels.

The gist of the twenty-second aspect of the invention is that, in the twenty-first aspect of the invention, the format conversion unit converts at least one of the data format, size, and aspect ratio of the degraded video to match the reference The images match.

The gist of the twenty-third aspect of the invention is that, in the twenty-first aspect of the invention, the display timing matching unit, when the frame rates of the reference video frame and the degraded video frame are different, Or delete the degraded image frame, so that the frame rates of the reference image frame and the degraded image frame match.

The gist of the twenty-fourth aspect of the invention is that, in the twenty-first aspect of the invention, the display timing matching unit aligns the display timing of the degraded video frame with the display timing of the reference video frame.

The gist of the twenty-fifth aspect of the invention is that, in the twenty-first aspect of the invention, the display timing matching unit matches the reference image frame with the display timing of the reference image frame if the display timing is not constant. The display timing of both of the degraded image frames is set at a predetermined time interval.

The gist of the twenty-sixth aspect of the invention is that, in the twenty-first aspect of the invention, the synchronization/position matching unit, for the target frame of the reference video frame and the degraded video frame, and several frames before and after them and performing a macro-synchronization process for determining a temporal correspondence between the reference video frame and the degraded video frame in which the deviation of the feature values is the smallest by comparing the transition of each feature value of all frames or specific regions thereof.

The gist of the twenty-seventh aspect of the invention is that, in the twenty-sixth aspect of the invention, the synchronization/position matching unit, for the target frame of the reference video frame and the degraded video frame, and several frames before and after them , while shifting the time correspondence and pixel correspondence between the reference image frame and the degraded image frame, comparing the respective feature quantities of all frames or specific regions thereof, and performing the process of determining that the difference of the feature quantities becomes the smallest Micro-synchronization/position matching processing of the time correspondence between the reference image frame and the degraded image frame and the pixel correspondence.

The gist of the twenty-eighth aspect of the invention is that, in the twenty-seventh aspect of the invention, the synchronization and position matching unit initially executes the macro synchronization and the micro synchronization and position matching.

The gist of the twenty-ninth aspect of the invention is that, in the twenty-eighth aspect of the invention, the synchronization/position matching unit counts the number of frames when the degraded video frame is frozen. The duration of the frozen state.

The gist of the invention of the thirtieth aspect is that, in the invention of the twenty-ninth aspect, the synchronization/position matching unit, for the target frame of the reference video frame and the degraded video frame, and several frames before and after them, The respective feature values of all frames are derived, and when the feature values of the reference video frame change over time but the feature values of the degraded video frames do not change over time, it is determined that the degraded video frame is in a frozen state.

The gist of the thirty-first aspect of the invention is that, in the twenty-eighth aspect of the invention, the synchronization/position matching unit cannot synchronize with the reference image frame when the degraded image frame is frozen. In the case of , execute the macro synchronization process again.

The gist of the thirty-second aspect of the invention is that, in the twenty-eighth aspect of the invention, the synchronization/position matching unit outputs the number of frame deviations when the degraded video frame falls into a frame deviation state.

The gist of the thirty-third aspect of the invention is that, in the twenty-first aspect of the invention, the reference image signal and the degraded image signal are input from the synchronization/position matching unit, and the degraded image matching the brightness and color information with the reference image, and returning the deteriorated image whose brightness and color information have matched to the brightness/color correction unit of the synchronization/position matching unit.

In addition, in order to achieve the above object, the thirty-fourth aspect of the invention is a video matching method characterized by inputting a reference video signal which is an undegraded video signal and a degraded video signal after the reference video signal has been degraded; converting the file format of the degraded image included in the degraded image signal into the file format of the reference image included in the corresponding reference image signal; matching the reference image frame included in the reference image signal with the degraded The number and display timing of the degraded video frames contained in the video signal; and for the target frame of the reference video frame and the degraded video frame and several frames before and after it, monitor the reference video frame and the degraded video frame The frame deviation or the frozen state of the degraded image is obtained, and the matching between the corresponding relationship between the frames and the corresponding relationship between the pixels is obtained.

In addition, in order to achieve the above object, the thirty-fifth aspect of the invention is a video matching program characterized by causing a computer to operate as a means for inputting a reference video signal which is a non-degraded video signal and a degraded reference video signal. After the degraded image signal, a format change unit for converting the file format of the degraded image included in the degraded image signal into the file format of the reference image included in the corresponding reference image signal; matching the reference image a reference video frame included in the signal and a display timing matching unit for the number of degraded video frames included in the degraded video signal and display timing; Synchronization/ Location matching unit.

Furthermore, in order to achieve the above object, the thirty-sixth aspect of the invention is a video quality evaluation device characterized by comprising: a reference video signal (RI) which is an undegraded video signal and a degraded reference video signal. A degraded image signal (PI), a format conversion unit (41) for converting the file format of the degraded image included in the degraded image signal into the file format of the reference image included in the corresponding reference image signal; A display timing matching unit (42) for displaying the number and display timing of a reference video frame included in the reference video signal and a degraded video frame included in the degraded video signal; for the reference video frame and the degraded video frame The target frame and several frames before and after it, monitor the deviation of the frame between the reference image frame and the degraded image frame or the freezing state of the degraded image, and simultaneously obtain the correspondence between frames and the correspondence between pixels a synchronization/position matching section (43) for matching the relationship; A subjective quality estimating unit (11) for estimating the subjective quality of the degraded video signal based on the video signal feature quantity of the signal and the calculated difference between the video signal feature quantity of the two signals.

Description of drawings

FIG. 1 is a block diagram showing the configuration of the first embodiment of the video quality evaluation device of the present invention;

FIG. 2 is a diagram showing correction information stored in the correction information database of FIG. 1;

3 is a graph showing the relationship between the final estimated subjective quality Q after correction based on the correction formula for the primary estimated subjective quality SQ and the actual measured subjective quality;

4 is a block diagram showing the configuration of a second embodiment of the video quality evaluation device of the present invention;

FIG. 5 is a diagram for calculating horizontal and vertical edge quantities used in the spatial feature quantity calculation section of FIG. 4;

6 is a block diagram showing the configuration of a third embodiment of the video quality evaluation device of the present invention;

7 is a diagram representing a plurality of conditions stored in the weighting coefficient database of FIG. 6 and weighting coefficients corresponding to the conditions;

Fig. 8 is a diagram listing standard image data classified into learning data and verification data used in verification according to the embodiment of the present invention;

Fig. 9 is a distribution diagram of SI (spatial information) value and TI (time information) value calculated according to the learning data of Fig. 8 and verification data;

FIG. 10 is a diagram showing estimation results of learning data obtained using a conventional peak-to-SN ratio (PSNR);

FIG. 11 is a diagram showing estimation results of learning data obtained using conventional marginal electric power (Ave_EE);

12 is a diagram showing estimation results of learning data obtained using the video quality assessment devices according to the second and third embodiments of the present invention;

13 is a diagram showing estimation results of verification data obtained using the video quality evaluation devices according to the second and third embodiments of the present invention;

FIG. 14 is a diagram showing estimation results of learning data obtained using only the existing marginal electric power (Ave_EE);

Fig. 15 is a diagram showing the relationship between the minimum value (Min_HV) of the horizontal and vertical margin amount and the subjective evaluation value;

Fig. 16 is a diagram showing the relationship between block average motion electric power (Ave_MEB) and subjective evaluation value;

Fig. 17 is a block diagram showing the configuration of an embodiment of the video matching device of the present invention;

Fig. 18 is a flow chart showing the operation steps of the embodiment of the image matching device of the present invention;

FIG. 19 is a diagram for explaining processing in a display timing matching unit;

FIG. 20 is a diagram for explaining macro time synchronization processing in the synchronization/position matching unit;

FIG. 21 is a diagram for explaining micro-synchronization/position derivation processing in a synchronization/position matching unit.

Detailed ways

Embodiments of the image quality evaluation device, image quality evaluation method, and image quality evaluation program, as well as the image matching device, image matching method, and image matching program of the present invention will be described in detail below with reference to the accompanying drawings. The first to third embodiments are embodiments of the invention of the video quality evaluation device, video quality evaluation method, and video quality evaluation program, and the fourth embodiment is an embodiment of the invention of the video matching device, video matching method, and video matching program.

<First Embodiment>

Each device part of the first embodiment of the video quality evaluation device of the present invention will be described with reference to FIG. 1 . FIG. 1 is a block diagram showing the configuration of a first embodiment of the video quality evaluation device of the present invention.

The video quality assessment device of the first embodiment includes at least a subjective quality estimation unit 11 , a feature value calculation unit 12 , a correction information database 13 , a correction calculation unit 14 , and a correction unit 15 .

The subjective quality estimation unit 11 receives a reference video signal RI and a deteriorated video signal PI as inputs. The reference video signal RI is a video signal before degradation, while the degraded video signal PI is a video signal in which the reference video signal RI has been degraded, for example, by encoding or passing through a network.

Next, the subjective quality estimating unit 11 calculates a difference in video signal feature value as a physical feature value with respect to the reference video signal RI and the degraded video signal PI. The video signal feature value includes, for example, spatial information (SI) indicating the video state of video frames included in the video signal, and time indicating video changes between video frames included in the video signal. Information (TI). Such spatial information SI and temporal information TI include, for example, Spatial Information and Temporal Information specified in Appendix A of "Subjective Video Quality Assessment Methods for Multimedia Applications" of ITU-R Recommendation P.910.

Then, the subjective quality estimation unit 11 quantifies the degradation of the degraded video signal PI from the calculated difference of the video signal feature value, and estimates the subjective quality from the quantified degradation. That is, the subjective quality estimation unit 11 quantifies the degradation of the degraded video signal PI from the reference video signal RI and the degraded video signal PI, and estimates the subjective quality from the quantified degradation. The estimated subjective quality is output from the subjective quality estimation unit 11 as primary estimated subjective quality SQ. The primary estimated subjective quality SQ, for example, if determined by the marginal electric power (E) and the kinetic electric power (M), can generally be expressed by the function shown in the following formula (1):

SQ=F(M, E) ... (1).

This function F is obtained in advance through subjective evaluation experiments. In addition, the amount of edge power (E) and motion power (M) are measured in the objective evaluation scale of video quality specified by ANSI in the United States (ANSIT1.801.03-1996, "Digital Transport of One-Way Video Signal Parameters for Objective Performance Assessment") stipulated.

The feature quantity calculation unit 12 receives a reference video signal RI as input, and calculates a video signal feature quantity FI based on the reference video signal RI. The video signal feature FI also includes, for example, spatial information SI or temporal information TI as described above. The feature amount calculation unit 12, for example, specifies at least one of the spatial information SI and the temporal information TI to quantitatively calculate the feature amount.

The correction information database 13 stores correction information in association with video signal feature values. The video signal feature quantity is, for example, spatial information SI or temporal information TI as described above. The correction information is a correction formula or a correction coefficient for correcting the primary estimated subjective quality SQ output from the subjective quality estimation unit 11 . These correction information will be described in detail later with reference to FIG. 2 . These correction formulas or correction coefficients are determined experimentally in advance and stored in the correction information database 13 . When degradation is added to a video having the video signal feature value FI, characteristics such as how much degradation people perceive subjectively after seeing the video are collectively calculated in advance through a subjective evaluation experiment. A correction formula and a correction coefficient for the video signal feature value FI are calculated based on this characteristic.

That is, the physical feature quantity of the reference video signal RI is collectively calculated in advance through a subjective evaluation test, and the subjective evaluation characteristic when adding deterioration to an image having its characteristics is calculated, and the correspondence between the calculated physical feature quantity and the subjective evaluation characteristic is calculated. relationship, the primary estimated subjective quality SQ derived by the subjective quality estimating unit 11 from the video signal feature quantity of the reference video signal RI is corrected so that it can be converted to a unified objective evaluation value with high precision.

The correction calculation unit 14 receives the video signal feature FI from the feature calculation unit 12 , and extracts a correction formula and correction coefficient corresponding to the video signal feature FI from the correction information database 13 . That is, the correction calculation unit 14 searches the correction information database 13 for a correction formula and correction coefficient corresponding to the video signal feature value FI, and extracts the corresponding correction formula and correction coefficient from the correction information database 13 . Then, the correction calculation unit 14 outputs these correction formulas and correction coefficients as correction information CI.

The correction unit 15 receives an estimated subjective quality SQ from the subjective quality estimation unit 11 and correction information CI from the correction calculation unit 14 . Then, the correction unit 15 substitutes the primary estimated subjective quality SQ into a correction formula with a correction coefficient included in the correction information CI, and outputs the corrected primary estimated subjective quality SQ as the final estimated subjective quality Q. The subjective quality Q is finally estimated, and the primary estimated subjective quality SQ quantitatively expressing the subjective quality SQ of the deteriorated video signal PI is corrected.

Next, correction information corresponding to feature quantities of video signals stored in the correction information database 13 will be described with reference to FIG. 2 . FIG. 2 is a diagram showing correction information stored in the correction information database 13 of FIG. 1 .

As shown in FIG. 2 , in the correction information database 13 , a plurality of correction coefficients are databased in association with each value of the feature value of the video signal. According to FIG. 2, when the video signal feature data (spatial information SI and temporal information TI) of the reference video signal RI is calculated by the feature data calculation unit 12 as (SI, TI)=(75, 30), the corresponding video signal The correction coefficient of the feature quantity is (α, β)=(α2, β2). Actually, the correction calculation unit 14 receives the video signal feature FI from the feature calculation unit 12 , and extracts a correction coefficient corresponding to the video signal feature FI from the database of the correction information database 13 together with a correction formula. Furthermore, when the correction formula corresponding to the feature value of the video signal is also changed, not only the correction coefficient but also the correction formula corresponding to the feature value of the video signal are databased in the correction information database 13 .

Finally, the final estimated subjective quality Q outputted after the correction unit 15 corrects the estimated subjective quality SQ once will be described with reference to FIG. 3 . FIG. 3 is a graph showing the relationship between the corrected final estimated subjective quality Q and the actually measured subjective quality obtained by using a correction formula for the primary estimated subjective quality SQ.

In the first embodiment, the correction formula is the following formula (2).

Final estimated subjective quality Q=α×primary estimated subjective quality SQ+β...(2)

Here, α and β are correction coefficients. FIG. 3 shows graphs obtained by three correction formulas calculated for three types of input video signals input to the video signal evaluation device. The horizontal axis represents the primary estimated subjective quality SQ which is the output of the subjective quality estimation unit 11, and the vertical axis represents the subjective quality evaluation test calculation of the primary estimated subjective quality SQ by people actually seeing the video obtained from the input video signal and evaluating its quality. The subjective quality that comes out. In the figure, circles, squares, and triangles represent subjective quality evaluation points for each input video signal for one estimated subjective quality SQ. On the other hand, the three lines are correction formulas based on the present embodiment respectively corresponding to the three types of input video signals. According to the present embodiment, the primary estimated subjective quality SQ is corrected using such correction formulas corresponding to each video.

In this way, according to the first embodiment, the human visual characteristics for the image are obtained from the physical feature quantities of the reference image, and the correction information for the feature quantities of the reference image is databased, and the physical feature quantities of the reference image and the deteriorated image By weighting the estimated subjective quality derived from the difference between the two values, the subjective quality can be uniformly estimated with the same accuracy as the conventional subjective evaluation method for any video.

In addition, in the above-mentioned embodiment, the structure for calculating the feature quantity of the video signal of the reference video signal RI is logically provided as a single feature quantity calculation unit 12, but such a feature quantity calculation unit 12 may not be specially provided, and it may be used as it is. The video signal feature value of the reference video signal RI derived by the subjective quality estimation unit 11 is used without change.

In addition, the correction calculation unit 14 and the correction unit 15 may be integrated not only physically but also logically. That is, while the correcting unit 15 directly inputs the video signal feature value of the reference video signal RI, the correcting unit 15 may input correction information corresponding to the video signal feature value from the correction information database.

<Second Embodiment>

Each device part of the second embodiment of the video quality evaluation device of the present invention will be described with reference to FIG. 4 . FIG. 4 is a block diagram showing the configuration of a second embodiment of the video quality evaluation device of the present invention.

The video quality assessment device according to the second embodiment includes a calibration information generation unit 21 , a spatial feature calculation unit 22 , a temporal feature calculation unit 23 , and a subjective quality estimation unit 24 as shown in FIG. 4 .

The calibration information generator 21 receives the reference video signal RI and the degraded video signal PI as input, receives the reference video frame from the reference video signal RI, receives the degraded video frame from the degraded video signal PI, and detects the timing of the reference video frame and the degraded video frame. Spatial frame deviations, generating calibration information about these temporal and spatial frame deviations.

The temporal deviation between the reference video frame and the degraded video frame refers to the temporal deviation between the reference video frame and the degraded video frame received by the calibration information generation unit 21 at a certain point in time. For example, when the calibration information generation unit 21 receives a frame of a certain video A in the reference video frame at a certain time, and receives a frame 3 frames earlier than the frame of the video A in the degraded video frame at this time, the calibration information generation The unit 21 detects that the degraded video frame is three frames behind the reference video frame, and generates this information as calibration information.

In addition, the spatial deviation between the reference video frame and the degraded video frame means that the spatial position of the video received by the calibration information generation unit 21 at a certain time is deviated. For example, when the frames of the reference video frame and the degraded video frame do not deviate in time, the calibration information generation unit 21 receives the frame mapped at the center of the center sphere of the video in the reference video frame at a certain time, At this point in time, in the case of a degraded video frame that is mapped to the center of the position sphere that is 1 pixel to the right and 2 pixels above the center of the video, the calibration information generation unit 21 detects that the degraded video frame is 1 pixel to the right and 2 pixels above the reference video frame. The fact that 2 pixels are shifted creates this information as calibration information.

The spatial feature calculation unit 22 receives the reference image signal RI, the degraded image signal PI, and the calibration information, so that the spatial and temporal deviations between the reference image frame and the degraded image frame disappear, and then, based on the reference image signal RI and the deteriorated image The signal PI calculates the spatial feature quantity. In the second embodiment, as the spatial feature quantity, indices of the edge electric power (Ave_EE) and the minimum value (Min_HV) of the horizontal and vertical edge quantities to be described in detail below are used.

Edge power (Ave_EE):

This index is an index for quantifying deterioration (for example, blurring) occurring at a boundary (called an edge) where the luminance value changes rapidly within a frame based on the reference video signal RI and the degraded video signal PI. In the second embodiment, as an example, the deterioration on the edge is quantified by emphasizing the edge from the luminance value of the pixel using a Sobel filter. In addition, the quantified edge power (Ave_EE) here is specified in ANSI TI.801.03-1996, "Digital Transport of One-Way Video Signal Parameters for Objective Performance Assessment".

The amount of marginal electric power (Ave_EE) is quantified by the formula (3).

[mathematical formula 1]

$\mathrm{<span\; class="notranslate">\; Ave.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; EE</span>}<span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; in</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; out</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; in</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

In the formula, SI _μ (m) (μ=in or out) is given by (4).

[mathematical formula 2]

$\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\sqrt{\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

μ=in or out

In addition, here, SI _h-μ (i, j, m), SI _v-μ (i, j, m), respectively represent the Sobel filter located at the position (i, j) of the m-th frame, respectively by ( 5) and (6) are given.

SI _h-μ (i, j, m)={-Y _μ (i-1, j-1, m)-2Y _μ (i, j-1, m)-Y _μ (i+1, j-1 , m)

+Y _μ (i-1, j+1, m)+2Y _μ (i, j+1, m)+Y _μ (i+1, j+1, m)}

... (5)

SI _v-μ (i, j, m)={-Y _μ (i-1, j-1, m)+Y _μ (i+1, j-1, m)-2Y _μ (i-1, j , m)

+2Y _μ (i+1, j, m)-Y _μ (i-1, j+1, m)+Y _μ (i+1, j+1, m)}

... (6)

In the formula, Y _in (i, j, m) represents the brightness value (from 0 to 255) of the pixel at the position (i, j) of the mth frame of the reference image frame, Y _out (i, j, m) Indicates the brightness value (from 0 to 255) of the pixel at position (i, j) of the mth frame of the degraded image frame.

In addition, the results of estimation using only the marginal electric power (Ave_EE) will be described later together with the figure, but there are images in which the subjective evaluation value changes greatly although the objective evaluation value hardly changes, and only the marginal electric power (Ave_EE) is used. The deterioration that occurs in these images cannot be fully captured.

Minimum value of horizontal and vertical margin (Min_HV):

This index is an index for quantifying the degree of horizontal and vertical boundaries (edges) in which luminance values change sharply in the degraded video frame corresponding to the reference video frame compared with the reference video frame. In this second embodiment, as an example, the feature value (Min_HV ), to quantify the extent to which edges occur horizontally and vertically. To this extent, as shown in Fig. 5, the edge volume (HV) of the dots entering the horizontal/vertical area determined by the constant r0 = 20 and Δθ = 0.05236 and the area outside this are obtained from the input image and the degraded image. The ratio of edge volume (HV dashes). The reason for adopting such an index is that video information is often coded in units of blocks, so so-called block distortion is reliably reflected in such coding.

The minimum value (Min_HV) of the horizontal and vertical margins is quantified by formula (7).

[mathematical formula 3]

$\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; HV</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; \{</span>\frac{\mathrm{<span\; class="notranslate">\; HV</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; in</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; HV</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; out</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; HV</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; in</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

In the formula, HVR _μ (m) (μ = in or out), given by (8).

[mathematical formula 4]

$\mathrm{<span\; class="notranslate">\; HV</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; h</span>}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&Delta;\&theta;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}}{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; h</span>}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&Delta;\&theta;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

μ=in or out

In addition, here is the definition represented by the formula (9).

[mathematical formula 5]

$\mathrm{<span\; class="notranslate">\; h</span>}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&Delta;\&theta;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; P</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

μ=in or out

However, as a condition, Expression (10) and Expression (11) are satisfied.

SI _{r_μ} (i, j, m) ≥ r _min > 0 ... (10)

k _n /2-Δθ<SI _{θ_μ} (i,j,m)<k _n /2+Δθ(k=0,1,2,3)

... (11)

Here, SI _{θ_μ} (i, j, m) = tan ^-1 [SI _{v_μ} (i, j, m)/SI _{h_μ} (i, j, m)].

In addition, P in the formula (9) is the number of pixels in the range occupied by the shade in FIG. 5 .

On the other hand, there is a definition represented by the formula (12).

[mathematical formula 6]

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; h</span>}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&Delta;\&theta;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; P</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; S</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

μ=in or out

However, as a condition, Expression (13) and Expression (14) are satisfied.

SI _{r_μ} (i, j, m) ≥ r _min > 0 ... (13)

k _п /2+Δθ<SI _{θ_μ} (i, j, m)<(k+1) _п /2-Δθ(k=0, 1, 2, 3)

... (14)

In addition, P in Expression (12) is the number of pixels satisfying the ranges of Expressions (13) and (14).

The minimum value (Min_HV) of the horizontal and vertical edge quantities is a deterioration that cannot be captured in the edge electric quantity (Ave_EE), specifically, it is a feature quantity that captures edges newly generated due to block distortion. As will be described later with reference to the drawings, this index can sensitively capture its deterioration. This index is based on the index stipulated in ANSI, however, as shown in the formula (7), it is independently improved by using the minimum value so that only the number of occurrences of new edges can be captured with high sensitivity.

However, the temporal feature quantity calculation unit 23 inputs the reference video signal RI, the degraded video signal PI and the calibration information, eliminates the spatial deviation and the temporal deviation between the reference video frame and the degraded video frame, and thereafter, based on the reference video signal RI and the degraded video signal, PI calculates the time feature quantity. In the second embodiment, the block average exercise electric power amount (Ave_MEB) which is an index based on a difference of TI (time information) values to be described in detail below is used as the temporal feature amount. The TI value is the difference in brightness values of pixels between image frames, and is specified in ITU-R Recommendation P.910.

Average motion electric power of the block (Ave_MEB):

This index is an index that derives the difference between the TI values of the reference video frame and the degraded video frame for each block, which is a set of several pixels in the frame, and normalizes the difference based on the TI value of each block of the reference video frame.

That is, the block average motion electric power amount (Ave_MEB) is represented by equation (15).

[mathematical formula 7]

$\mathrm{<span\; class="notranslate">\; Ave.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; MEB</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}}\sqrt{\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; in</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; out</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; in</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 15</span>}<span\; class="notranslate">\; )</span>$

In the formula, TI _b-μ (k, l, m) is represented by the formula (16).

[mathematical formula 8]

$\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 64</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 16</span>}<span\; class="notranslate">\; )</span>$

μ=in or out

The block average motion electric power amount (Ave_MEB) is a characteristic amount that captures the endless deterioration in the edge electric power amount (Ave_EE), specifically, the occurrence of deterioration based on motion for each area. As will be described later with reference to the drawings, this index can sensitively capture its deterioration. This index is a unique measure to increase the sensitivity by deriving the TI value of each block in order to capture the motion of each area, and normalizing it with the value of the reference image.

The subjective quality estimating unit 24 receives the spatial feature calculated by the spatial feature calculating unit 22 and the temporal feature calculated by the temporal feature calculating unit 23 as inputs, and estimates the subjective quality of the degraded video signal PI as the reference video signal RI. subjective evaluation value. This subjective evaluation value (Y) is calculated by the formula (17).

Y=αX ₁ +βX ₂ +γX ₃ +δ ... (17)

In the formula, X ₁ =Ave_EE, X ₂ =Min_HV, X ₃ =Ave_MEB.

This subjective evaluation value is a finally obtained value.

Here, α, β, γ, and δ are calculated in advance in order to determine the correspondence relationship between the temporal feature quantity and the spatial feature quantity and the subjective evaluation value from the relationship with the subjective evaluation value of the degraded image obtained through the subjective evaluation test. The obtained weighting coefficient. In addition, α, β, γ, and δ are determined according to, for example, the signal format of the degraded video signal PI, the information amount (size) of the degraded video transmitted from the degraded video signal PI, and the encoding method of the degraded video signal. Therefore, α, β, γ, and δ can be calculated in advance from the signal format of the degraded video signal PI, the information amount (size) of the degraded video transmitted by the degraded video signal PI, and the encoding method of the degraded video signal.

According to the video quality assessment device of the second embodiment described above, it is possible to collectively estimate the subjective quality of any video. How much the accuracy of the video quality evaluation device of the second embodiment can be improved compared with the conventional video quality evaluation device will be described later in detail with reference to the drawings.

In addition, the video quality evaluation device shown in FIG. 4 described in the second embodiment may be incorporated into the subjective quality estimation unit 11 of the video quality evaluation device shown in FIG. 1, and the video quality evaluation device shown in FIG. quality evaluation device. That is, the video quality evaluation device shown in FIG. 4 inputs the reference video signal RI and the degraded video signal PI, and outputs the value output from the subjective quality estimation unit 24 to the correction unit 15 as the primary estimated subjective quality SQ. The correction unit 15 receives the correction information CI from the correction calculation unit 14, inputs the estimated subjective quality SQ once, and calculates the final estimated subjective quality Q.

<Third Embodiment>

Each device part of the third embodiment of the video quality evaluation device of the present invention will be described with reference to FIG. 6 . Fig. 6 is a block diagram showing the configuration of a third embodiment of the video quality evaluation device of the present invention.

The image quality evaluation device of the third embodiment differs from the image quality evaluation device of the second embodiment in that the reference image signal RI and the deteriorated image signal PI input to the device are input in different file formats. Furthermore, the signal form of the degraded image signal PI, the information amount (size) of the degraded image sent by the degraded image signal PI, and the encoding method of the degraded image signal are unknown. Therefore, the same reference numerals are assigned to the same parts as those in the image quality evaluation device of the second embodiment, and description thereof will be omitted.

The video quality evaluation device of the third embodiment, as shown in FIG. Coefficient database 36 .

The format conversion unit 35 converts the file format of the degraded video file into the file format of the reference video file when the file format of the degraded video file is different from that of the reference video file after inputting the reference video file and the degraded video file. same form. More specifically, for example, when the signal form, color distribution, size, aspect ratio, and encoding method of the degraded video file are different from those of the reference video file, the file format of the degraded video file is converted into a file format similar to that of the reference video file. The form is the same. Then, the format conversion unit 35 outputs the degraded video signal PI from the degraded video file converted into the same file format as the reference video file. The degraded video signal PI from the format conversion unit 35 is output to the calibration information generation unit 21 , the spatial feature calculation unit 22 , and the temporal feature calculation unit 23 .

More specifically, the format conversion unit 35, for example, assumes that the reference image is a VGA-sized YUV format reference image, and the reference image and the degraded image have different color formats. If the degraded image is in the RGB format, then -R BT.601 "STUDIO ENCODING PARAMETERS OF DIGITAL TELEVISION FOR STANDARD 4:3 ANDWIDE-SCREEN 16:9 ASPECT RATIOS" etc. to change the video. Also, when the format conversion unit 35 converts the size or aspect ratio of the reference video and the degraded video to be the same, it may be simply calculated as an integer multiple, but if this is not true, it needs to be converted to an arbitrary size. In this case, transformation to an arbitrary size is performed using a well-known method (see, for example, Muramatsu S. and Kiya H.: "Scale Factor of Resolution Conversion Based on Orthogonal Transforms," IEICE Trans. Fundamentals., E76-A, 7, pp.1150-1153 (July 1993), or Matsumura Shogo, Takaya Hitoshi, "Resolution conversion method for arbitrary rational number magnifications for images with change codes", Proceedings of the Society for Electronics, Information and Communications A, Vol.77-A, No. .3, pp.369-378, March 1994). Also, when the video signal is compressed, it is converted into an uncompressed format in advance.

In addition, due to the characteristics of the image coding or transmission system or the different national standards, the distribution of the color or brightness of the deteriorated image is different. The distribution is standardized. That is, for both the reference image and the degraded image, for each of the luminance, color difference signal, or RGB value, from the distribution of pixel values in a specific frame, statistics such as the maximum value, minimum value, average value, and variance are derived. , to make their distributions the same, either by shifting all the pixel values of the degraded image by, for example, the amount of deviation from the mean, or by using (18) in the ranges used to make the minimum and maximum values of the reference image and the degraded image consistent The transformation shown is transformed to normalize the distribution of the reference image and the degraded image.

[mathematical formula 9]

In the formula, Y: the pixel value of the transformed degraded image

X: The pixel value of the degraded image before transformation

The format conversion unit 35 outputs information related to the file format of the converted degraded video signal PI, that is, the same file format as the reference video signal RI, to the subjective quality estimation unit 34 . The information related to the file format is, for example, the signal format, color distribution, size, aspect ratio, and encoding method of the degraded video file, but in this third embodiment, the subjective quality estimation unit 34 outputs the Signal form, size, and encoding method.

The weighting coefficient database 36 stores the signal format, size, and coding method of the degraded video file as a set of conditions, stores a plurality of these conditions, and sets weighting coefficients α, β, γ, and δ corresponding to each condition in advance. These weighting coefficients are coefficients obtained from the relationship with the subjective evaluation value for the degraded image obtained through the subjective evaluation experiment, and the correspondence relationship with the subjective evaluation value obtained in advance with respect to the temporal feature quantity and the spatial feature quantity. FIG. 7 shows a plurality of conditions stored in the weighting coefficient database 36 and weighting coefficients corresponding to these conditions.

Subjective quality estimation unit 34 obtains weighting coefficients α, β, γ, and δ from weighting coefficient database 36 corresponding to the signal format, size, and encoding method of the degraded video file input from format conversion unit 35, which is similar to that of the second embodiment. The subjective quality estimation unit 24 is different. Others are the same as the subjective mass estimation unit 24 .

In addition, in FIG. 6 , the format conversion unit 35 and the calibration information generation unit 21 are arranged separately, but the format conversion unit 35 and the calibration information generation unit 21 may be incorporated into the image quality evaluation device in FIG. 6 as a single component.

According to the third embodiment described above, even when the file format of the degraded video file is different from that of the reference video file, the subjective quality of the degraded video can be estimated. Furthermore, since a plurality of modes can be supported for the signal format, size, and coding method of the degraded video file, it is possible to estimate the subjective quality of various degraded video files.

Next, the actual results of the estimation accuracy of the image quality values estimated by the image quality evaluation apparatuses of the second and third embodiments will be described with reference to FIGS. 8 to 16 in comparison with conventional image quality evaluation apparatuses.

First, subjective evaluation data used for verification of estimation accuracy and derivation of weighting coefficients stored in the weighting coefficient database 36 will be described. Subjective evaluation data, using the standard images selected by ITU-R shown in Figure 8 (refer to ITU-R BT.802-1, "Test Pictures and Sequences for subjective Assessments of Digital Codecs'Coveying Signals produced According to Recommendation ITU- RBT.601," 1994, and ITU-R BT.1201-2, "Testmaterials to be used in subjective assessment," 2001) 36 kinds of images, as shown in Figure 8, the standard images are divided into images for verification The verification data of the estimation accuracy of the quality evaluation value, and the learning data which are the data previously used for deriving the coefficient.

The subjective evaluation data used to verify the estimation accuracy is selected to reduce the influence due to the imbalance of the characteristics of the reference image. That is, considering the distribution of spatial information (SI) and temporal information (TI) expressed in ITU-TP.910 (refer to ITU-TP.910, "SubjectiveVideo Quality assessment Methods for multimedia applications" Aug.1996.), from The same number of images are selected from the regions A to D in FIG. 9 , so that images with various SI values and TI values can be used as reference images. In addition, for these reference images, MPEG4-based coding degradation is used to make degradation images assigned to 4 levels in the range of 256 kbps to 8 Mbps.

The subjective quality evaluation method uses the DSCQS method often used in the performance evaluation tests of multimedia digital signal codecs such as MPEG verification (refer to ITU-R BT.500-10, "Methodology first subjective assessment of the quality of television pictures" March 2000.), the people who received the test were 24 ordinary men and women from the age of 18 to 40.

It is appropriate for the target estimation accuracy of the image quality evaluation device to be equal to the degree of dispersion of evaluation points in the evaluation. When the degree of dispersion of the subjective evaluation values was derived as a one-sided 95% confidence interval, it was 7.24 in all data. Therefore, as the target estimation accuracy of the video quality evaluation device, the mean square error (RMSE) should not exceed this value.

In FIG. 10 and FIG. 11, respectively, in the conventional image quality evaluation device, generally used, PSNR (Peak Signal Noise Ratio: peak SN ratio) and edge power known as ANSI parameters (Ave_EE ) data, the results estimated for the learning data. FIG. 10 shows the estimation results when PSNR is used, and FIG. 11 shows the estimation results when marginal electric power (Ave_EE) is used. According to Figure 10 and Figure 11, the RMSE of PSNR is 9.57, and the RMSE of marginal power (Ave_EE) is 7.47. Compared with PSNR, marginal power (Ave_EE) shows good characteristics, but RMSE exceeds the target value, and the estimation accuracy is inadequate.

FIG. 12 and FIG. 13 show the video quality evaluation apparatuses of the second and third embodiments using the edge power amount (Ave_EE), the minimum value of the horizontal and vertical edge amount (Min_HV), and the entire Ave_MEB, for the learning data and the verification data. presumptive result. 12 and 13, since the RMSE of the learning data is 6.43 and the RMSE of the verification data is 6.49, sufficient target estimation accuracy can be achieved not only for the learning data but also for the verification data which is non-learning data. Therefore, it is obvious that the image quality evaluation devices of the second and third embodiments have the estimated accuracy that can be used instead of the subjective evaluation quality test used by people to evaluate the quality of an actual image after seeing it, and have reached a practical level.

FIG. 14 is a diagram showing estimation results of a part of learning data obtained using only the marginal electric power amount (Ave_EE). As shown in Figure 14, it is obvious that the inclinations of the evaluation points at the input images [3], [6], [7], [9] are large, and although the objective evaluation value hardly changes, the subjective evaluation value changes greatly. The degradation that occurs in these images cannot be fully captured with the marginal power level (Ave_EE).

FIG. 15 is a diagram showing the relationship between the minimum value (Min_HV) of the horizontal and vertical edge amounts and subjective evaluation values. As shown in FIG. 15 , it can be seen that the deterioration is sensitively captured in the input images [3] and [7] according to the minimum value (Min_HV) of the horizontal and vertical edge amounts.

Fig. 16 is a diagram showing the relationship between block average motion electric power (Ave_MEB) and subjective evaluation values. As shown in FIG. 16 , it can be seen that the deterioration is sensitively captured in the input images [6] and [9] according to the block average motion electric quantity (Ave_MEB).

As described above, according to the second and third embodiments of the present invention, in addition to the parameter capturing the deterioration occurring in the edge region as the physical feature value of the image, by adding two unique parameters that supplement the lack of accuracy, it is possible to use the The technique of maintaining estimation accuracy in a specific image is extended to arbitrary images.

<Fourth Embodiment>

Embodiments of the invention relating to the video matching device, video matching method, and video matching program will be described below.

Fig. 17 is a block diagram showing the configuration of an embodiment of the video matching device of the present invention.

The video matching device of the fourth embodiment includes at least a format conversion unit 41 , a display timing matching unit 42 , a synchronization/position matching unit 43 , a brightness/color correction unit 44 , and a deterioration amount deriving unit 45 .

The format conversion unit 41 matches the file format included in the degraded video signal degraded by encoding or loss in the network with the format of the reference video included in the reference video signal. The display timing matching unit 42 matches the video display timings of the reference video signal and the degraded video signal. The synchronization/position matching unit 43 obtains the time-space matching between the reference video signal and the degraded video signal while acquiring the degradation amount or out-of-synchronization information from the degradation amount derivation unit 45 . In addition, the luminance/color correction unit 44 corrects the difference in luminance or color distribution between the reference video signal and the degraded video signal as necessary.

In addition, in this embodiment, the reference video signal and the degraded video signal include frame rate information or frame display time/capture time information. In addition, if necessary, signal format information is also included. As a result, the video matching device performs non-real-time processing on the reference video and the degraded video while storing the target frame and several frames before and after it in the memory.

Fig. 18 is a flowchart showing the operation procedure of the embodiment of the video matching device of the present invention.

First, the format conversion unit 41 converts the signal format of the degraded video when the signal format, size, and aspect ratio of the reference video and the degraded video are different (step S1). For example, if the data format of the reference image is uncompressed YUV format and the data format of the degraded image is uncompressed RGB format, use Rec.ITU-R BT.601”STUDIO ENCODINGPARAMETERS OG DIGITAL TELEVISION FOR SDANDART 4:3 ANDWIDE- SCREEN 16:9 ASPECT RATIOS" transformation transform degrades the image. In addition, the format conversion unit 41 first converts the degraded video to an uncompressed format when it is in a compressed format. Furthermore, the format conversion unit 41 performs conversion so that the sizes and aspect ratios are different when they are different. For example, sometimes it can simply be calculated as an integer multiple, but in the case where this is not enough, it needs to be converted to an arbitrary size. In this case, a known method is used to perform conversion to an arbitrary size (for example, refer to Muramatsu S. and Kiya H.: "Scale Factor of Resolution Conversion Based on Orthogonal Transforms," IEICE Trans. Fundamentals., E76-A, 7, PP.1150-1153 (July 1993), or Matsumura Shogo, Takaya Hitoshi, "For Change Coding The resolution transformation method of arbitrary rational number magnification of the image "Proceedings of the Electronic Information and Communication Society A, Vol.77-A, No.3, pp.369-378, March 1994.). Thereafter, the format conversion unit 41 transfers the reference video signal and the converted degraded video signal to the display timing matching unit 42 .

Next, the display timing matching unit 42 performs processing such as frame complementation to match the display timing of the degraded video signal format-converted by the format converting unit 41 with the reference video signal (step S2).

Specifically, as shown in FIG. 19(a), when the display interval of the degraded image is constant longer than the display interval of the reference image, the display timing matching unit 42, as shown in the lower part of FIG. 19(b), complements the full frame Supplementary images for degraded images.

In addition, as shown in FIG. 19(b), when the display interval of the degraded video fluctuates, the display timing matching unit 42, as shown in the lower part of FIG. The image preceding the degraded image complements the degraded image. In addition, at this time, instead of supplementing the degraded video with a video preceding the degraded video, it may be supplemented with a temporally close video among the degraded video. That is, for example, in the case of FIG. 19( b ), frame I is supplemented as the second frame of the converted degraded video, but frame II is supposed to be supplemented with temporally close video among the degraded video.

Furthermore, as shown in FIG. 19(c), when the reference video signal itself does not have a constant frame rate, it must be complemented so that the reference video signal and the degraded video signal are displayed at different correct time intervals.

Thereafter, the display timing matching unit 42 transfers the reference video signal and the degraded video signal to the synchronization/position matching unit 43 .

The synchronization/position matching unit 43 assumes three states (1) the evaluation start state, (2) the synchronization state, and (3) the freeze state, and prescribes its operation.

(1) First, in the evaluation start state, the synchronization/position matching unit 43 compares the reference video signal and the degraded video signal for a certain period of time in order to obtain a macro match in the time direction, for the entire frame or a specific region of the signal of the degraded video signal. Changes in feature quantities such as luminance/color difference/average values of RGB values are obtained to obtain frame information with the highest matching performance, thereby deriving the deviation in the time direction of the macro (step S3). Specifically, as shown in FIG. 20 , the feature values are shifted in the time direction, and the shift in the time direction (frame difference) is derived from the condition that the difference between the time-series values is the smallest or the correlation coefficient is the largest.

In addition, the synchronization/position matching unit 43 uses the degradation amount deriving unit 45 to make the degraded image in the reference image for several frames before and after the matched image as shown in FIG. The pixels are moved up, down, left, and right, and the position with the smallest difference is regarded as the position where the best match is obtained, and received as pixel position information (step S4).

Furthermore, the synchronization/position matching unit 43 transfers the matched reference video and degraded video to the brightness/color correction unit 44 for brightness/color correction. The luminance/color correction unit 44 matches the average value, minimum value, and maximum value of the luminance and color information of the degraded image, and their distribution with those of the reference image. For example, when the luminance distributions of the reference image and the degraded image are different, the luminance/color correction unit 44 adjusts the luminance distributions of the reference image and the degraded image based on the respective average values and variance values of the luminance distributions of the reference image and the degraded image. Linear normalization is performed, and information on the conversion formula for this linear normalization is passed to the synchronization/position matching unit 43 as correction information. The synchronization/position matching unit 43 receives the correction information, and performs brightness/color correction processing based on the received correction information (step S5).

Hereinafter, the synchronization/position matching unit 43 performs the processing from step S6 to step S22 shown in FIG. 18 as processing in (2) the synchronization state (3) the freeze state.

When the target frame number i of the reference video is smaller than the final frame number N of the reference video, or the target frame number j of the degraded video is smaller than the final frame number M of the degraded video, the following steps S7 to S22 are executed (step S6 ).

Therefore, first, it is judged whether F1 is 1, that is, whether the reference image and the deteriorated image are in an asynchronous state (step S7). If the reference image and the degraded image are synchronized (F1=0), go to step S8, and if the reference image and degraded video are not synchronized (F1=1), go to step S16.

First, the case where the reference image and the degraded image are synchronized (F1=0) will be described below.

The synchronization/position matching unit 43 transfers to the degradation amount deriving unit 45 the degraded image and the reference image obtained in the above-mentioned evaluation start state, the deviation in the time-space direction, or the luminance/color information corrected. The deterioration amount deriving unit 45 performs micro-synchronization processing (refer to FIG. The frame-to-frame difference between the previous frames of the video and the degraded video is derived as a degradation amount (step S8). The synchronization/position matching unit 43 receives the degradation amounts derived by the degradation amount deriving unit 45, and judges whether the degraded video is frozen based on the inter-frame difference among the degradation amounts (step S9). That is, if the difference with respect to the reference image shows a certain value, and the difference with the degraded image is close to 0, the synchronization/position matching unit 43 judges that the degraded image is frozen.

When the degraded image is in the frozen state, the synchronization/position matching unit 43 sets the frozen state to "1" (count=1) while setting it to the asynchronous state (F1=1) (step S10), and transfers to step S10. S22.

On the other hand, when the degraded video is not in the frozen state, then the synchronization/position matching unit 43 judges whether the degraded amount of the target frame of the degraded video obtained by the micro-synchronization process of the degraded amount deriving unit 45 in step S8 is the smallest (step S11). When the degradation amount of the target frame of the degraded video is the minimum, the synchronization state is set (F2=0) (step S12), and the process proceeds to step S14. On the other hand, when the amount of deterioration of the target frame of the degraded image is not the smallest, it is determined that it is a frame deviation state (frame skipping state, frame returning state), and transfers to step S13, and the frame number with the smallest amount of deterioration is set as the j value, If it is frame skipping, set F2=1, otherwise (frame return state, etc.) set F2=2, and set the number of frame deviations at that time as the Count value.

In step S14, the F2 value and the Count value are output together with the output of the reference image and the deteriorated image. Thereafter, the Count value is reset to 0, and the process moves to step S22.

Next, the case where it is judged in step S7 that the reference video and the degraded video are out of synchronization (F1=1) will be described below.

The synchronization/position matching unit 43, in step S16, similarly to step S8, receives from the degradation amount deriving unit 45 the micro-synchronization process performed on the frame of the corresponding reference video and its several frames before and after it as the object. To obtain the deterioration amount, at the same time, the frame-to-frame difference between the reference image and the previous frame of the deteriorated image is received as the deterioration amount. The synchronization/position matching unit 43, similar to step S9, judges whether or not the degraded video is in a frozen state based on the difference between frames among these degraded amounts (step S17).

When the degraded video is frozen, the synchronization/position matching unit 43 increments the frozen count (count value) by 1 (step S18 ), and proceeds to step S22 .

On the other hand, when the degraded image is not in the frozen state, the synchronization/position matching unit 43 judges that the frozen state is over, performs the same macro time synchronization process as in step S3 (step S19), and outputs the frozen number (count value) (step S20). Thereafter, the F1 value and the count value are reset to 0, and the process proceeds to step S22.

In step S22, add 1 to the target frame number i of the reference video and the target frame number j of the degraded video. When the value of i reaches the final frame number N of the reference video and the value j reaches the final frame number M of the degraded video, the process ends.

In addition, it is assumed that the reference video signal and the degraded video signal have a feature quantity such as the average value of luminance/color difference/RGB values in the entire frame or in a specific area for a certain period of time, and synchronization is not possible for some reason. For example, when the feature quantity in the target frame is separated from the average value for a certain period by a certain threshold value (a constant multiple of the standard deviation such as 3σ), it is considered to enter the evaluation initial state of (1), and from step S3 restarts processing.

The above are the details of the video matching processing according to the embodiment of the present invention.

According to the present embodiment, by performing the format conversion of the video and performing synchronous processing at all times based on the macro-matching process and the micro-matching process, even if the received video is different in size or aspect ratio when the personal computer terminal receives the video, Occasionally, or when the correspondence of the spatial positions of the reference image and the deteriorated image cannot be obtained due to a certain degree of concentrated information loss such as packet loss, or further due to fluctuations in the arrival interval of IP packets or the occurrence of packet loss, in Even when the timing of video display on the time axis deviates, or phenomena such as fluctuations in video display timing or freezes that have never existed before occur, it is possible to appropriately match the time direction and space direction of the reference video and the degraded video.

However, in the video matching method of the present invention, it is assumed that the reference video output from the synchronization/position matching unit 43, the degraded video and its auxiliary information (state in the time direction) are input into the video quality evaluation device (for example, the first In the subjective quality estimation unit 11) of the video quality assessment device of the embodiment, however, in the degradation amount derivation unit 45, the objective evaluation value may be derived instead of the degradation amount, and if the result is output by the synchronization/position matching unit 43, It can also be used as an image quality evaluation device.

In addition, in each embodiment described above, the instructions shown in the processing steps may be executed based on a program as software. The general-purpose computer system stores the program in advance, and by reading the program, the general-purpose computer system can be made to work as an image quality evaluation device and an image matching device. The instructions described in the above-mentioned embodiments are recorded on magnetic disks (floppy disks, hard disks, etc.), optical disks (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD-ROM, etc.) ±RW, etc.), semiconductor memory, or similar recording media. The computer reads the program from the recording medium, and when the CPU executes the instructions described in the program based on the program, the computer operates as the image quality evaluation device and the image matching device of the above-mentioned embodiment.

In addition, the present invention is not limited to the copying of the above-mentioned embodiments, but can be realized by changing the constituent elements within a range that does not deviate from the gist at the stage of implementation. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some structural elements may be deleted from all the structural elements shown in the embodiments. Furthermore, constituent elements related to different embodiments may be appropriately combined.

According to the video quality evaluation device, video quality evaluation method, and video quality evaluation system of the present invention, even if it is an unknown reference video, a unified subjective quality can be estimated with high accuracy for any video.

Specifically, according to the video quality evaluation device, video quality evaluation method, and video quality evaluation system of the present invention, the visual characteristics of people for the video are obtained from the physical feature data of the reference video, and are used as correction information for the reference video feature data. By creating a database and adding weights to the correction information on the estimated subjective quality derived from the difference between the physical feature values of the reference image and the deteriorated image, the subjective quality can be uniformly derived for any image with the same accuracy as the existing subjective evaluation method. quality.

In addition, by adding two original parameters to compensate for the lack of accuracy as physical feature values of images, in addition to the parameters that capture the deterioration that occurs in the edge area, the technology that maintains the estimation accuracy in a specific image can be extended to any image. .

According to the image matching device, image matching method, and image matching program of the present invention, when comparing the physical feature values between the reference image signal and the degraded image signal in the estimation of subjective quality, their spatial and temporal characteristics can be reliably obtained. location match.

Specifically, by performing synchronous matching on the basis of macro-matching processing and micro-matching processing while performing video format conversion, even if the size or aspect ratio of the video received by the personal computer terminal is different, Or when the correspondence of the spatial positions of the reference image and the degraded image cannot be obtained due to a certain degree of concentrated information loss such as packet loss, or further due to fluctuations in the arrival interval of IP packets or the occurrence of packet loss, in the time axis Even when a phenomenon such as a deviation in timing of image display, fluctuation in image display timing, or freezing occurs, it is possible to appropriately match the time direction and space direction of the reference image and the degraded image.

Therefore, according to the video quality evaluation device and method based on such a video matching device and method, it is possible to perform objective evaluation of video quality even in the above-mentioned cases, and the accuracy thereof can be improved.

Claims (36)

1. a video quality evaluation device is characterized in that,

Have:

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) after this reference images signal degradation, calculate the signal of video signal characteristic quantity of two signals, poor according to the signal of video signal characteristic quantity of two signals that calculate inferred the subjective quality of the subjective quality of described deterioration signal of video signal and infers portion (11);

Be used to revise the update information storage part (13) of the update information of described subjective quality with the storage that is mapped of signal of video signal characteristic quantity; With

Import the described signal of video signal characteristic quantity of described reference images signal, obtain and the corresponding update information of having imported of signal of video signal characteristic quantity from described update information storage part (13), subjective quality correction portion (12 according to the described subjective quality of obtaining of inferring out of update information correction, 14,15).

2. video quality evaluation device according to claim 1 is characterized in that,

Described subjective quality is inferred portion (11), according to the spatial information (SI) of the image state in each frame that is illustrated in the image that comprises in the signal of video signal and be illustrated in poor between any one at least deterioration signal of video signal in the temporal information (TI) that the image of interframe changes and reference images signal, infer described subjective quality.

3. video quality evaluation device according to claim 2 is characterized in that,

Described subjective quality is inferred portion (11), poor as the described signal of video signal characteristic quantity between described deterioration signal of video signal and described reference images signal, computational chart be shown in the deterioration amount in each frame of the image that comprises in the signal of video signal edge electric power amount (E), be illustrated at least one side of mobile electric power amount (M) of the deterioration amount of interframe.

4. video quality evaluation device according to claim 2 is characterized in that,

Described subjective quality is inferred portion (11) and is inferred described subjective quality according to the difference between, deterioration signal of video signal and reference images signal any one at least among the spatial information of regulation and the temporal information in P.910 at ITU-R Recommendation.

5. the described video quality evaluation device of claim 1 is characterized in that,

Update information storage part (13), as the update information that is used to revise described subjective quality, with the spatial information (SI) that is illustrated in the image state in each frame of image that comprises in the signal of video signal and be illustrated in temporal information (TI) that the image of interframe changes be mapped the storage correction factor (α, β).

6. video quality evaluation device according to claim 1 is characterized in that,

Described subjective quality is inferred portion (11) and is had:

Import described reference images signal (RI) and described deterioration signal of video signal (PI), generate about from the reference images frame of described reference images signal with from the calibration information generating unit (21) of the calibration information that departs from the space of the departing from of time of the deterioration image frame of described deterioration signal of video signal, described reference images frame and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, calculate the space characteristics amount calculating part (22) of space characteristics amount separately according to the spatial information of the image state of described reference images frame of expression and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, for described reference images frame and described deterioration image frame, calculate the temporal characteristics amount calculating part (23) of temporal characteristics amount separately according to the temporal information of the image variation of representing interframe separately; With

Infer the portion of inferring (24) of the subjective quality of described deterioration signal of video signal according to described space characteristics amount and described temporal characteristics amount.

7. video quality evaluation method, this method is to be pushed the update information (α of the subjective quality of making having with the signal of video signal characteristic quantity deterioration signal of video signal (PI) of storage after being used to revise reference images signal (RI) as the signal of video signal that worsens and worsening that be mapped, update information storage part (13) β), infer the video quality evaluation method in the video quality evaluation device of subjective quality of described deterioration signal of video signal, it is characterized in that

Import described reference images signal and described deterioration signal of video signal;

Calculate the signal of video signal characteristic quantity of two signals;

Poor according to the signal of video signal characteristic quantity of two signals of calculating inferred the described subjective quality of described deterioration signal of video signal;

From described update information storage part (13) obtain the update information corresponding with the signal of video signal characteristic quantity of the described reference images signal of calculating (α, β);

(α β) revises described subjective quality of inferring out according to the update information that obtains.

8. a video quality evaluation program is characterized in that,

Make computer as following cell operation:

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) of this reference images signal degradation, calculate the signal of video signal characteristic quantity of two signals, poor according to the signal of video signal characteristic quantity of two signals that calculate inferred the subjective quality of the subjective quality of described deterioration signal of video signal and infers unit (11);

With the signal of video signal characteristic quantity be mapped the storage be used to revise described subjective quality update information (α, update information memory cell (13) β),

Obtain the update information (α corresponding from described update information memory cell (13) with the signal of video signal characteristic quantity of the described reference images signal of calculating, β), subjective quality amending unit (12,14,15) according to the described subjective quality of obtaining of inferring out of update information correction.

9. a video quality evaluation device is characterized in that,

Have:

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) of this reference images signal degradation, and generation is about from the reference images frame of described reference images signal with from the calibration information generating unit (21) of the calibration information that departs from the space of the departing from of time of the deterioration image frame of described deterioration signal of video signal, described reference images frame and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, calculate the space characteristics amount calculating part (22) of space characteristics amount separately according to the spatial information of the image state of described reference images frame of expression and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, for described reference images frame and described deterioration image frame, calculate the temporal characteristics amount calculating part (23) of temporal characteristics amount separately according to the temporal information of the image variation of representing interframe separately; With

Infer portion (24) according to the subjective quality that described space characteristics amount and described temporal characteristics amount are inferred the subjective quality of described deterioration signal of video signal.

10. video quality evaluation device according to claim 9 is characterized in that,

Also have:

In the document form of the reference images that comprises in the described reference images signal (RI) that the document form of the deterioration image that comprises in described deterioration signal of video signal (PI) is transformed in correspondence, the formal argument portion (35) that exports the information related with its document form of unifying; With

Make its corresponding information related, store coefficient (α, beta, gamma, the correction factor storage part (36) δ) of the subjective quality that is used to infer described deterioration signal of video signal with described document form;

Described subjective quality is inferred portion (34), obtain described coefficient corresponding and from described correction factor storage part (36) from the related information of the described unified document form of described formal argument portion (35) input, according to described space characteristics amount, described temporal characteristics amount and the described coefficient of obtaining, infer the subjective quality of described deterioration signal of video signal.

11. video quality evaluation device according to claim 10 is characterized in that,

Described formal argument portion (35), as with the described unified related information of document form, in the amount of information of the deterioration image of exporting the signal form of described deterioration signal of video signal, sending by described deterioration signal of video signal, the coded system of described deterioration signal of video signal at least any one, described correction factor storage part, in the amount of information of the signal form of corresponding described deterioration signal of video signal, the deterioration image that sends by described deterioration signal of video signal and the coded system of described deterioration signal of video signal at least any one, coefficient (the α that storage is best, β, γ, δ).

12. according to claim 9 or 10 described video quality evaluation devices, it is characterized in that,

Described space characteristics amount calculating part (22), according to described reference images signal and described deterioration signal of video signal, the index of the deterioration that quantification brightness value in frame border jumpy is taken place is calculated as described space characteristics amount.

13. the described video quality evaluation device of claim 12 is characterized in that,

Described space characteristics amount calculating part (22), according to described reference images signal and described deterioration signal of video signal, the edge electric power amount (Ave_EE) that will stipulate in ANSI T1.801.03-1995 is calculated as described space characteristics amount.

14. claim 9,10 or 12 described video quality evaluation devices is characterized in that,

Described space characteristics amount calculating part (22), more described reference images frame calculates in the index (Min_HV) of the situation that brightness value border jumpy in should the described deterioration image frame of reference images frame is taken place on the horizontal vertical direction quantification as described space characteristics amount.

15. claim 9,10,12 or 14 described video quality evaluation devices is characterized in that,

Described space characteristics amount calculating part (23), according to each as frame in more than or equal to the piece of the set more than 1 pixel in reference images in Temporal Information value that ITU-R Recommendation is prescribed in P.910 with worsen the poor of Temporal Information value that image is prescribed in P.910 at ITU-R Recommendation, the variable quantity (Ave_MEB) of the interframe of image is calculated as described temporal characteristics amount.

16. claim 9 or 10 described video quality evaluation devices is characterized in that,

And then have:

Be used to revise the update information storage part (13) of the update information of described subjective quality with the storage that is mapped of space characteristics amount and temporal characteristics amount; With

Import the described space characteristics amount of described reference images signal (RI) and described temporal characteristics amount, obtain the space characteristics amount of corresponding input and the update information (α of temporal characteristics amount from described update information storage part (13), β), according to the subjective quality correction portion (12 of the described subjective quality of inferring of update information correction obtained, 14,15).

17. a video quality evaluation method is characterized in that,

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) after this reference images signal degradation;

Generation is about from the reference images frame of described reference images signal with from the calibration information that departs from the space of the departing from of time of the deterioration image frame of described deterioration signal of video signal, described reference images frame and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, calculate separately space characteristics amount according to the spatial information of the image state of described reference images frame of expression and described deterioration image frame;

According to described calibration information, having eliminated after described spatial offset and described time departs from, for described reference images frame and described deterioration image frame, the temporal information that changes according to the image of expression interframe is separately calculated temporal characteristics amount separately;

According to described space characteristics amount and described temporal characteristics amount, infer the subjective quality of described deterioration signal of video signal.

18. a video quality evaluation method is characterized in that,

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (RI) of this reference images signal degradation;

The document form of the deterioration image that in described deterioration signal of video signal, comprises, be transformed to the document form of the reference images that in the described reference images signal of correspondence, comprises;

Generate the information related with this document form of unifying;

Generation is about the calibration information that departs from the space between the departing from of the time between the deterioration image frame of the deterioration image that was transformed from the reference images frame of described reference images signal with from the document form of described deterioration signal of video signal, described reference images frame and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, calculate separately space characteristics amount according to the spatial information of the image state of described reference images frame of expression and described deterioration image frame;

According to described calibration information, having eliminated after described spatial offset and described time departs from, for described reference images frame and described deterioration image frame, the temporal information that changes according to the image of expression interframe is separately calculated temporal characteristics amount separately;

According to described space characteristics amount, described temporal characteristics amount with (α, beta, gamma δ), are inferred the subjective quality of described deterioration signal of video signal corresponding to the described document form of unifying the coefficient that subjective quality infers of being used for related, information.

19. a video quality evaluation program is characterized in that,

Make computer as following cell operation:

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) of this reference images signal degradation, and generation is about from the reference images frame of described reference images signal with from the calibration information generation unit (21) of the calibration information that departs from the space between the departing from of the time between the deterioration image frame of described deterioration signal of video signal, described reference images frame and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, calculating the space characteristics amount computing unit (22) of space characteristics amount separately according to the spatial information of the image state of described reference images frame of expression and described deterioration image frame;

According to described calibration information, eliminating after described spatial offset and described time departs from, for described reference images frame and described deterioration image frame, calculate the temporal characteristics amount computing unit (23) of temporal characteristics amount separately according to the temporal information of the image variation of representing interframe separately; With

According to described space characteristics amount and described temporal characteristics amount, infer the subjective quality of the subjective quality of described deterioration signal of video signal and infer unit (24).

20. video quality evaluation program according to claim 19 is characterized in that,

Also make computer as following cell operation:

In the document form of the reference images that comprises in the described reference images signal (RI) that the document form of the deterioration image that comprises is transformed in correspondence, export the formal argument unit (35) of the information related with this document form of unifying in described deterioration signal of video signal (PI); With

The coefficient (α, beta, gamma, the correction factor memory cell (36) δ) that make it be mapped with the information related, store the subjective quality that is used to infer described deterioration signal of video signal with described document form;

Described subjective quality is inferred unit (34), from described correction factor memory cell (36), obtain with the related corresponding described coefficient of information of the described document form of unifying from described formal argument unit (35) input, according to described space characteristics amount, described temporal characteristics amount and the described coefficient of obtaining, infer the subjective quality of described deterioration signal of video signal.

21. an image coalignment is characterized in that,

Have:

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) of this reference images signal degradation, the document form of the deterioration image that in described deterioration signal of video signal, comprises, be transformed to the formal argument portion (41) of the document form of the reference images that in the described reference images signal of correspondence, comprises;

The number of the deterioration image frame that comprises in reference images frame that coupling comprises in described reference images signal and the described deterioration signal of video signal and the Displaying timer matching part (42) of Displaying timer; With

For described reference images frame and described deterioration image frame to several frames before and after picture frame and its, departing from or the frozen state of described deterioration image of the frame of supervision between described reference images frame and described deterioration image frame, and obtain the corresponding relation of interframe and the corresponding relation between pixel coupling synchronously/location matches portion (43).

22. image coalignment according to claim 21 is characterized in that,

Described formal argument portion (41), at least one makes with described reference images and coincide in the data mode of the described deterioration image of conversion, size, the length-width ratio.

23. image coalignment according to claim 21 is characterized in that,

Described Displaying timer matching part (42) in the described reference images frame occasion different with the frame rate of described deterioration image frame, by interpolation or delete described deterioration image frame, coincide the frame rate of described reference images frame and described deterioration image frame.

24. image coalignment according to claim 21 is characterized in that,

Described Displaying timer matching part (42), for the Displaying timer of described reference images frame, the Displaying timer of the described deterioration image frame of harmonizing.

25. image coalignment according to claim 21 is characterized in that,

Described Displaying timer matching part (42) in the dissimilar occasion of the Displaying timer of described reference images frame, is taken as preset time to described reference images frame and described deterioration image frame both sides' Displaying timer at interval.

26. image coalignment according to claim 21 is characterized in that,

Described synchronously/location matches portion (43), for described reference images frame and described deterioration image frame to several frames before and after picture frame and its, the passing of the characteristic quantity separately of whole frames or its specific region, the grand Synchronous Processing of the corresponding relation that departs from the time that becomes minimum described reference images frame and described deterioration image frame of the described characteristic quantity of execution decision.

27. image coalignment according to claim 26 is characterized in that,

Described synchronously/location matches portion (43), for described reference images frame and described deterioration image frame to several frames before and after picture frame and its, move the corresponding time relationship and the pixel corresponding relation of described reference images frame and described deterioration image frame on one side, relatively whole characteristic quantity separately of frames or its specific region on one side, the difference of carrying out the described characteristic quantity of decision become the corresponding relation of time of minimum described reference images frame and described deterioration image frame and pixel corresponding relation little synchronously/location matches handles.

28. image coalignment according to claim 27 is characterized in that,

Described synchronously/location matches portion (43), the initial stage carry out described grand Synchronous Processing and described little synchronously/location matches handles.

29. image coalignment according to claim 28 is characterized in that,

Described synchronously/location matches portion (43), be absorbed in the occasion of frozen state at described deterioration image frame, by frame number being counted the continuation time of measuring described frozen state.

30. image coalignment according to claim 29 is characterized in that,

Described synchronously/location matches portion (43), for described reference images frame and described deterioration image frame to several frames before and after picture frame and its, derive whole frames characteristic quantity separately, change in time and the characteristic quantity of described deterioration image frame when not changing in time at characteristic quantity, be judged to be described deterioration image frame and be absorbed in frozen state for described reference images frame.

31. image coalignment according to claim 28 is characterized in that,

Described synchronously/location matches portion (43), be absorbed in the occasion of frozen state at described deterioration image frame, perhaps can not with the occasion of described reference images frame synchronization, carry out described grand Synchronous Processing once more.

32. image coalignment according to claim 28 is characterized in that,

Described synchronously/location matches portion (43), be absorbed in the occasion of the state that frame departs from described deterioration image frame, output frame depart from number.

33. image coalignment according to claim 21 is characterized in that,

And then have from described synchronously/location matches portion (43) input described reference images signal (RI) and described deterioration signal of video signal (PI), the brightness of described deterioration image and colouring information and described reference images are coincide, the deterioration image that described brightness and colouring information have been coincide turn back to described synchronous/the brightness/color correction portion (44) of location matches portion (43).

34. an image matching method is characterized in that,

Input is as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) after this reference images signal degradation;

The document form of the deterioration image that in described deterioration signal of video signal, comprises, be transformed to the document form (step S1) of the reference images that in the described reference images signal of correspondence, comprises;

The number of the deterioration image frame that comprises in reference images frame that coupling comprises in described reference images signal and the described deterioration signal of video signal and Displaying timer (step S2); With

For described reference images frame and described deterioration image frame to several frames before and after picture frame and its, departing from or the frozen state of described deterioration image of the frame of supervision between described reference images frame and described deterioration image frame, and obtain the coupling (step S3-S22) of the corresponding relation and the corresponding relation between pixel of interframe.

35. a video matching program is characterized in that,

Make computer as following cell operation:

Input is transformed to the variation unit (41) of the document form of the reference images that comprises in the described reference images signal of correspondence as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) after this reference images signal degradation, the document form of the deterioration image that comprises in described deterioration signal of video signal;

The number of reference images frame that coupling comprises in described reference images signal and the deterioration image frame that in described deterioration signal of video signal, comprises and the Displaying timer matching unit (42) of Displaying timer; With

For described reference images frame and described deterioration image frame to several frames before and after picture frame and its, departing from or the frozen state of described deterioration image of the frame of supervision between described reference images frame and described deterioration image frame, obtain simultaneously the corresponding relation of interframe and the corresponding relation between pixel coupling synchronously/location matches unit (43).

36. a video quality evaluation device is characterized in that,

Have:

Input is transformed to the formal argument portion (41) of the document form of the reference images that comprises in the described reference images signal of correspondence as the reference images signal (RI) of the signal of video signal that does not worsen and the deterioration signal of video signal (PI) after this reference images signal degradation, the document form of the deterioration image that comprises in described deterioration signal of video signal;

The number of reference images frame that coupling comprises in described reference images signal and the deterioration image frame that in described deterioration signal of video signal, comprises and the Displaying timer matching part (42) of Displaying timer;

For described reference images frame and described deterioration image frame to several frames before and after picture frame and its, departing from or the frozen state of described deterioration image of the frame of supervision between described reference images frame and described deterioration image frame, obtain simultaneously the corresponding relation of interframe and the corresponding relation between pixel coupling synchronously/location matches portion (43); With

Input by described synchronously/location matches portion (43) obtained coupling described reference images signal (RI) and described deterioration signal of video signal (PI), calculate the signal of video signal characteristic quantity of two signals, the subjective quality of inferring the subjective quality of described deterioration signal of video signal according to the difference of the signal of video signal characteristic quantity of two signals of calculating infers portion (11).

Images