CN1157059C - Movement character combined video quality evaluation method - Google Patents

Abstract

The present invention relates to a video quality evaluating method combining motion characteristics, which belongs to the field of computer digital video processing. The evaluating method is a comprehensive evaluating method combining the motion characteristics of the sequence of a video image, considering the visual characteristics of human eyes and organically combining the time domain characteristics and the space domain characteristics of a video. The method provides a describing method of the intense degree of motion and provides a measuring method of video definition and fluency based on the describing method, and comprehensive evaluating results are given combining the definition with the fluency. The method orderly comprises: video signals are read in from a video capture card by a computer and a video capture interval T is stored; a subprogram of video image compressing processing is carried out; a reconstruct subprogram of the video image is carried out; a subprogram for calculating the intense degree of the motion of a video sequence is carried out; comprehensive evaluating results are calculated after a subprogram for video quality comprehensive evaluation is carried out; the results can be compared with the visual feelings of the human eyes when the video image is watched from a display. Experiments prove that the results comprehensively evaluated are basically consistent with the visual feelings of the human eyes.

Description

A Video Quality Evaluation Method Combining Motion Features

technical field

The invention relates to the field of computer digital video processing, and designs a video quality evaluation method combined with motion features.

Background technique

Video quality evaluation is one of the foundations of image/video information engineering. For example, in video communication, the video information of the subject is transmitted to the receiving end to restore an acceptable video, which requires photoelectric conversion, compression processing, transmission, Recording and other transformation processes, the pros and cons of all these technologies will be included in the evaluation of video quality.

For the image/video information whose final destination is the human eye, its evaluation should be consistent with the subjective feeling of the human eye. The mechanism of human visual processing is very complicated, and so far, it has not been really understood and mastered. However, people have also discovered some visual phenomena, which have affected the evaluation of human research video quality. Specifically, these visual properties have multichannel structures, visual thresholds, and masking.

1. Multi-channel structure and visual threshold

The human visual system is a multi-channel structure that decomposes an input image into different sensory components. Each sensory channel has its own threshold, called the visual threshold. A stimulus in a given channel is invisible to the human eye if its value is below the visual threshold for that channel.

2. Masking effect

When there are multiple stimuli, they will interfere with each other, resulting in changes in the visual threshold, which is called the visual masking effect. For coded and transmitted video images, the original image is the masker, and the coded transmission damage is the target. Actual video images are ever-changing, and the backgrounds of encoded impairment components are also varied, so the visibility of impairment components also varies.

The existence of the visual threshold makes the damage below the threshold undetectable, and the existence of masking increases the visual threshold, thereby increasing the invisible damage. This is an important starting point for studying video quality metrics based on the human visual system. In other words, if you manage to make the damage appear where the human eye can't see it, you also improve the image quality.

Masking comes in various forms. The human eye's sensitivity to impairment decreases in very bright or very dark areas, called contrast masking; the human eye's sensitivity to impairment is lower in areas with large changes in the image spatial domain than in areas with small changes, called texture masking; content For an image block that changes greatly over time, the human eye is less sensitive to its damage, which is motion masking; immediately after the scene is switched, the human eye’s visibility of the damage decreases, which is switch masking.

At present, people's evaluation of video quality is mainly based on the evaluation of still images, which is measured by the average peak signal-to-noise ratio and frame frequency. The frame rate is the number of frames transmitted per second; the average peak signal-to-noise ratio is defined as:

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; PSNR</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; P</span>}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; PSN</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}$

in

${\mathrm{<span\; class="notranslate">\; PSNR</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; log</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}\frac{\mathrm{<span\; class="notranslate">\; 255</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; 255</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; m</span>}}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ,</span>$

f _k (i, j) and f′ _k (i, j) are the gray levels of the original image and the reconstructed image of the kth frame respectively, and M and N are the width and height of the image respectively.

The average peak signal-to-noise ratio can objectively reflect the degree of video distortion, but it does not consider the visual characteristics of the human eye, and the frame rate and average peak signal-to-noise ratio are independent of each other to reflect the characteristics of the video time domain and space domain. The method separates the time-domain characteristics and space-domain characteristics of video images. Therefore, this evaluation result is often inconsistent with people's visual experience.

Contents of the invention

In order to overcome the shortcomings of the current evaluation methods, the present invention designs a video quality evaluation method combined with motion features. This evaluation method takes into account the visual characteristics of the human eye, and considers the spatial domain characteristics and time domain characteristics of the video, and its evaluation results can be consistent with the visual experience of the human eye. Technical idea of the present invention is characterized in that:

1. Two definitions of video clarity and video fluency are proposed, and they are used as two measurement methods for video quality evaluation. The so-called sharpness refers to the clarity of each frame image; fluency refers to the continuous degree of a video.

2. A description method for the intensity of video image motion is proposed. Its description is divided into two parts: (1) the intensity distribution of each image block in a frame of image, we call it the spatial distribution feature of motion, and apply it to the evaluation of video clarity; (2) the distribution of the whole frame of video image The degree of motion intensity, which we call the temporal distribution of motion, is applied to video fluency evaluation.

3. Taking into account the visual characteristics of the human eye. The human eye's perception of video clarity and fluency is affected by the intensity of video image motion. For a certain frame of video image, the part with intense movement often attracts the attention of human eyes and belongs to the area of interest of human eyes. However, when the whole frame of a video moves violently, the stimulation of image clarity to human eyes will be relatively weakened, while the effect of image fluency will be relatively enhanced.

4. Combining the clarity and fluency of the video image organically, a method for calculating the comprehensive evaluation index of video quality is proposed.

Refer to Fig. 1 and Fig. 2 for the technical solution of the present invention. It combines the motion characteristics of the video image and the visual characteristics of the human eye to objectively evaluate the video image processed by the computer. The solution includes a camera (1) converting the image light signal of the acquired video image sequence of the target object into an electrical signal, and the video sequence from the camera is digitized and input into the computer processor by the acquisition card (2), which is characterized in that It also includes the method for calculating the comprehensive evaluation index of video quality which is set in the computer processor (3) by organically combining the clarity and fluency of the video image, and the method includes the following steps in turn:

1), the computer reads in the video signal from the video capture card, and saves the video capture interval T;

2), enter and complete processing such as motion compensation, conversion and encoding to the raw video frame that collects and enters computer, to compress effectively redundant video image compression processing subroutine;

3), enter the video image reconstruction subroutine that the compressed video code stream is decoded and the video sequence image is reconstructed;

4), according to the two definitions of video clarity and video fluency, and use them as two measurement methods for video quality evaluation:

First enter the subroutine for calculating the intensity of motion of the video sequence, divide each frame of video image into 16×16 image blocks, and calculate the intensity of motion of each image block and the intensity of motion of a whole frame of images;

Enter the video quality comprehensive evaluation subroutine again, according to the aforementioned degree of motion intensity, first calculate the clarity and fluency of the video sequence, and then calculate the comprehensive evaluation result;

5) Considering that the human eye's perception of video clarity and fluency is affected by the intensity of the video picture motion, the output of the comprehensive evaluation result of 4) can be compared with the visual experience of the human eye watching from the display.

According to the foregoing invention, calculate the intensity of motion of the video in two steps, the first step: calculate the intensity of motion of each image block, which can be calculated in two ways:

The first one is calculated according to the motion vector of the image block, and its calculation formula is:

${\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{\mathrm{<span\; class="notranslate">\; \&Delta;x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&Delta;y</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$

In the formula, MA _k (i, j) represents the motion intensity of the (i, j)th image block in the kth frame of video image, and (Δx _k , Δy _k ) is the motion vector of the block. The second is calculated by using the pixel gray level difference between two adjacent frames of images, and its calculation formula is:

${\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; EXP</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 255</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}}\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; block</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; )</span>$

In the formula, block(i, j) represents the (i, j)th block in a frame of image, and L _k (m, n) and L _k-1 (m, n) represent the kth frame and the k-1th frame respectively The gray value of the pixel at image (m, n), N _p represents the number of pixels in an image block.

Step 2: Calculate the intensity of motion of a whole frame of video image, the calculation formula is:

${\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; MB</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>$

In the formula, MA _k represents the motion intensity of the kth frame of video image, N _MB represents the number of image blocks in one frame of image, and the meaning of MA _k (i, j) is the same as above.

According to the aforementioned present invention, when calculating the definition of video images, it is assumed that the video sequence input to the computer is collected at a constant high-speed frame rate, and the acquisition interval is T; in general, the definition of video images is calculated. Not all frames are encoded (processed) for output. k represents the image frame number to be coded (processed), and N represents the total number of frames of the coded (processed) image. The time interval between the kth frame and the k-1th frame is FT _k ·T. Based on the peak signal-to-noise ratio of still images, we consider the visual characteristics of the human eye, assign higher weights to the peak signal-to-noise ratio of image blocks with more intense motion, and assign smaller weights to macroblocks with less motion. First calculate the sharpness of a frame of video image, the calculation formula is:

${\mathrm{<span\; class="notranslate">\; SS</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; log</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}\frac{\mathrm{<span\; class="notranslate">\; 255</span>}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; 255</span>}}{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; MB</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; Diff</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$

In the formula, SS _k represents the sharpness of a frame of video image, and the meanings of N _MB and MA _k (i, j) are the same as in 7,

${\mathrm{<span\; class="notranslate">\; Diff</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; block</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; input</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; output</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>$

Among them, L _{k, Input} (m, n) and L _{k, Output} (m, n) are the pixel gray value at (m, n) of the original captured video and the reconstructed video after processing, and then calculate the definition of a video , whose calculation formula is:

$\mathrm{<span\; class="notranslate">\; P.S.</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; FT</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; SS</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>$

In the formula, PS represents the resolution of a section (N frames) of video, and the larger the PS, the clearer the video image is. This calculation formula contains two meanings: on the one hand, the sharpness of image frames with more intense motion plays a relatively weak role in the evaluation of the clarity of the entire video; on the other hand, the image frames that take up a long time in video playback The sharpness of the video plays a relatively strong role in the evaluation of the sharpness of the entire video, which is in line with the visual experience of the human eye.

According to the aforementioned present invention, when calculating the fluency of a section of video, it is considered that the human eye cannot distinguish the detailed information of the image for the image frame with more intense motion. At this time, the clarity of the image is less stimulating to the human eye, and the continuity of motion The stimulation to the human eye becomes greater, so it is necessary to increase the frame rate when the movement is intense. For video frames with moderate motion, human eyes can distinguish the details of the image. At this time, it is necessary to improve the definition of each frame of video image, and a higher frame rate is not required. The video fluency is calculated according to this visual characteristic of the human eye, and the calculation formula is:

$\mathrm{<span\; class="notranslate">\; PT</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; FT</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>$

In the formula, PT represents the smoothness of a section (N frames) of video, and the smaller the PT, the smoother the video.

When calculating the comprehensive evaluation index value of a section of video image according to the aforementioned present invention, its calculation formula is:

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; P.S.</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; PT</span>}}$

In the formula, P represents the quality comprehensive evaluation value of a video image. PS is clarity, PT is fluency, _WS and W _T are the weights of clarity and fluency in the comprehensive evaluation value.

Based on a large number of experiments, we set _WS as 0.55 and W _T as 0.45. The comprehensive evaluation results of such values are basically consistent with the visual perception of human eyes.

Description of drawings

Figure 1 is a block diagram of a video quality comprehensive evaluation system.

Figure 2 is a flow chart of the main program of the video quality comprehensive evaluation system.

Figure 3 is a flow chart of the video compression processing subroutine.

Figure 4 is a flow chart of the video reconstruction subroutine.

Fig. 5 is a flow chart of the subroutine for calculating the intensity of video motion.

Fig. 6 is a flowchart of a subroutine for calculating video comprehensive quality evaluation index values.

Fig. 7 is a distribution diagram of motion intensity of 150 frames of video sequences calculated by the motion vector method, wherein the horizontal axis represents the sequence number of the video frame, and the vertical axis represents the motion intensity of the video frame.

FIG. 8 is a distribution diagram of motion intensity of 150 frames of video sequences calculated by using the absolute difference method, wherein the horizontal axis represents the serial number of the video frame, and the vertical axis represents the motion intensity of the video frame.

FIG. 9 is a distribution diagram of the resolution of each frame of a video sequence in two different examples, wherein the horizontal axis represents the serial number of the video frame, and the vertical axis represents the resolution of the video frame. In the figure [—■—Variable Frame Rate] indicates an example of playing with a variable frame rate, and [—×—Constant Frame Rate] indicates an example of playing with a constant frame rate.

In the figure, 1. camera, 2. video capture card, 3. computer digital video processor, 4. output buffer, 5. video quality evaluation, 6. display, 7. evaluation result.

Table 1: Video frame numbers processed by compression encoding in two different examples, the first group represents playback at a constant frame rate, and the second group represents playback at a variable frame rate.

Detailed ways

In the block diagram of the video quality comprehensive evaluation system in Figure 1, the camera and the video capture card are commercially available, mainly to complete the collection of video sequences, convert the optical image of the target object into an electrical signal image, and input it to the computer, which is convenient for computer processing and transmission. Operation; computer video processing is mainly to compress and encode the input video image sequence; the processed video image is output to the buffer for easy display; the display is the output device of the video sequence, and the human eye can watch the video image sequence through the display. The visual experience of human eyes can be used to compare with the results of comprehensive evaluation of video quality; comprehensive evaluation of video quality is to evaluate the reconstructed video images after computer processing, and output objective evaluation results, which can be compared with the results of video viewed by human eyes from the monitor. The visual perception of the sequence is compared.

Video quality evaluation is mainly realized by software. The following describes the process of video quality evaluation in detail with examples.

We have obtained 150 frames of video sequence images from the camera and video capture card, and saved them in the hard disk of the computer. Firstly, the video is compressed and coded. Compared with the original video, the compressed video code stream must have certain distortion after reconstruction. Now the video sequence reconstructed after compression is comprehensively evaluated according to the method designed by the present invention.

Step 1: Calculate the intensity of motion in each frame of the video. Calculate with two kinds of methods that the present invention proposes to describe the intensity of motion respectively, Fig. 7 and Fig. 8 are respectively the distribution figure of intensity of motion of 150 frame video sequences obtained by two kinds of methods, wherein horizontal axis represents video frame sequence number, vertical axis The axis represents how violent the motion of the video frame is.

It can be seen that Fig. 7 is similar to Fig. 8, which shows that both methods can objectively describe the intensity of motion of the image. And it can be seen that from the 60th frame to the 110th frame, the image moves relatively violently, and the other frames move relatively gently, which is consistent with the actual situation observed by the human eye.

Step 2: Calculate the definition of the video sequence. In order to illustrate the effectiveness of the method designed in the present invention for video evaluation, two groups of examples are designed here for comparison.

First, explain how the human eye perceives video sequence images. When our human eyes are watching a video, the human eye can distinguish the details of the image and is also interested in the details of the image during the time period when the motion of the video is slow. Due to the slow motion, the difference between the front and back frames is not large. Therefore, a lower frame rate is acceptable to the human eye; but for the period of intense motion of the video image, the human eye cannot distinguish the details of the image, and higher requirements are placed on the continuity of the motion, so at this time Increasing the frame rate gives a better visual experience to the human eye. Therefore, if the playback frame rate of the video sequence can be changed according to the intensity of the motion of the picture, the visual experience of the human eye will be better than when the video sequence is played at a constant frame rate.

Due to this fact, the following two sets of examples are designed. For the same video sequence, the limited bandwidth and other conditions are the same in the two groups of instances, but the frame rate is different.

The first group: the frame rate is constant at 10 frames per second;

The second group: The frame rate changes according to the intensity of the movement, but the average frame rate is still 10 frames per second. The reason why the average frame rate is the same as that of the first group is because the requirements for bandwidth are the same.

The frame numbers that are actually encoded in the two groups of examples are shown in Table 1. It can be seen from this that the frame rate of the first group is constant, while the frame rate of the second group changes. During the time period from 60 frames to 110 frames, the frame rate Higher, and the frame rate is lower in other time periods, which is consistent with the intensity of motion of the previously calculated video frames.

From Figure 7 and Figure 8, it can be seen that from the 60th frame to the 110th frame, the image motion is relatively severe, so in the second group of experiments, the number of encoded frames in this time period is relatively large, while in other time periods The number of encoded frames is less, which is in line with the visual characteristics of the human eye.

FIG. 9 shows the statistical results of the definition of video frames in two groups of examples, where the horizontal axis represents the serial number of the video frame, and the vertical axis represents the definition of the video frame. In the figure [—■—Variable Frame Rate] means variable frame rate, and [—×—Constant Frame Rate] means constant frame rate. It can be seen from FIG. 9 that in the second group of examples, the definition is relatively high in the video segment with moderate motion, while the definition is relatively low in the video segment with severe motion. According to the method designed in the present invention, the video sequence clarity of two groups of examples is calculated as:

Group 1: 28.2;

Group Two: 32.5.

The higher the definition of the video sequence, the better the visual experience for people, and the result obtained by using the method designed in the present invention is consistent with the visual experience of human eyes.

Step 3: Calculate the video sequence fluency. Identical with the example of the second step design, according to the method of the present invention's design, two groups of example video sequence fluency are calculated as:

The first group: 10.5;

Group 2: 9.0

The lower the fluency of the video sequence, the better the visual experience of the human eye, and the result of this evaluation method is consistent with the visual experience of the human eye.

Step 4: Calculate the comprehensive evaluation value of the video sequence. According to the comprehensive evaluation formula of the present invention, we take W _S as 0.55, W _T as 0.45, and the video sequence comprehensive evaluation value of two groups of experiments is:

Group 1: 15.6

Group 2: 17.9

The higher the comprehensive evaluation value, the better the visual experience of the video sequence for human eyes. The evaluation method designed by the present invention is consistent with the subjective visual experience of human eyes.

Table 1 The first set of instances Second set of instances 0, 12, 18, 24, 30, 36, 42, 48, 54, 60, 66, 72, 78, 84, 90, 96, 102, 108, 114, 120, 126, 132, 138, 144, 147 0, 4, 6, 20, 26, 38, 50, 56, 64, 68, 72, 76, 78, 82, 86, 90, 96, 100, 104, 110, 114, 122, 126, 138, 144, 148, 149

Claims (5)

1. A method for evaluating video quality in combination with motion features, comprising converting the image light signal of the video image sequence of the acquired object into an electrical signal by the camera (1), and converting the video sequence from the camera by the capture card (2) Digitized and input into the computer processor, it is characterized in that it also includes the method for calculating the comprehensive evaluation index of video quality that is set in the computer processor (3) and combines the clarity and fluency of the video image organically, the method Include the following steps in turn: 1), the computer reads in the video signal from the video capture card, and saves the video capture interval T; 2), enter and complete processing such as motion compensation, conversion and encoding to the raw video frame that collects and enters computer, to compress effectively redundant video image compression processing subroutine; 3), enter the video image reconstruction subroutine that the compressed video code stream is decoded and the video sequence image is reconstructed; 4), according to two kinds of definitions of video clarity and video fluency, and the steps after using it as two measurement methods of video quality evaluation are: First enter the subroutine for calculating the intensity of motion of the video sequence, divide each frame of video image into 16×16 image blocks, and calculate the intensity of motion of each image block and the intensity of motion of a whole frame of images; Enter the video quality comprehensive evaluation subroutine again, according to the aforementioned degree of motion intensity, first calculate the clarity and fluency of the video sequence, and then calculate the comprehensive evaluation result; 5), starting from considering that the human eye's perception of video clarity and fluency is affected by the intensity of the video picture motion, the comprehensive evaluation result output of 4) is used to compare with the visual experience of human eyes watching from the display. 2. A method for evaluating video quality in combination with motion features according to claim 1, characterized in that it calculates the intensity of motion of the video in two steps, the first step: calculating the intensity of motion of each image block, and its calculation can be There are two methods: the first is to calculate according to the motion vector of the image block, and its calculation formula is: ${\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{\mathrm{<span\; class="notranslate">\; \&Delta;x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&Delta;y</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$ In the formula, MA _k (i, j) represents the motion intensity of the (i, j)th image block in the kth frame video image, (Δx _k , Δy _k ) is the motion vector of the block, and the second method uses the adjacent The difference between the pixel gray levels of the two frames of images is calculated, and the calculation formula is: ${\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; EXP</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 255</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}}\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; block</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; )</span>$ In the formula, block(i, j) represents the (i, j)th block in a frame of image, and L _k (m, n) and L _k-1 (m, n) represent the kth frame and the k-1th frame respectively The gray value of the pixel at the image (m, n), N _p represents the number of pixels in an image block; the second step: calculate the degree of motion intensity of a whole frame of video image, its calculation formula is: ${\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; MB</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>$ In the formula, MA _k represents the motion intensity of the kth frame video image, and N _MB represents the number of image blocks in one frame of image. 3. A method for evaluating video quality in combination with motion features according to claim 1, wherein when calculating the definition of video images, it is assumed that the video sequence input to the computer is collected at a constant high-speed frame rate, and the collection interval is T; in general, not all frames are coded (processed) and output; k represents the coded (processed) image frame number, and N represents the total number of frames of the coded (processed) image; kth frame and k-th The interval between one frame is FT _k T; based on the peak signal-to-noise ratio of still images, we consider the visual characteristics of the human eye and assign higher weights to the peak signal-to-noise ratio of image blocks with more intense movement. Macroblocks with smaller motions have smaller weights; first calculate the sharpness of a frame of video image, and the calculation formula is: ${\mathrm{<span\; class="notranslate">\; SS</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; log</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}\frac{\mathrm{<span\; class="notranslate">\; 255</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; 255</span>}}{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; MB</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; Diff</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$ In the formula, SS _k represents the definition of a frame of video image, N _MB represents the number of image blocks in a frame of image, MA _k (i, j) represents the motion of the (i, j)th image block in the kth frame of video image severity. ${\mathrm{<span\; class="notranslate">\; Diff</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; block</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; input</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; output</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>$ Among them, L _{k, Input} (m, n) and L _{k, Output} (m, n) are the pixel gray value at (m, n) of the original captured video and the reconstructed video after processing; then calculate the definition of a video , whose calculation formula is: $\mathrm{<span\; class="notranslate">\; P.S.</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; FT</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; SS</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>$ In the formula, PS represents the definition of a video, the larger the PS is, the clearer the video image is, and MA _k represents the motion intensity of the kth frame video image. 4. A method for evaluating video quality in combination with motion features according to claim 1, wherein when calculating the fluency of a section of video, the video fluency is calculated according to this visual characteristic of the human eye, and its calculation formula is: $\mathrm{<span\; class="notranslate">\; PT</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; MA</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; FT</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>$ In the formula, PT represents the fluency of a section (N frames) of video, the smaller the PT, the smoother the video, MA _k represents the motion intensity of the kth frame video image, and FT _k is the difference between the kth frame and the k-1th frame during encoding. The number of skipped frames between. 5. A video quality evaluation method combined with motion features according to claim 1, wherein when calculating the comprehensive evaluation index value of a section of video image, the calculation formula is: $\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; P.S.</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; PT</span>}}$ In the formula, P represents the comprehensive evaluation value of a video image quality, PS is sharpness, PT is fluency, _WS and W _T are the weights of sharpness and fluency in the comprehensive evaluation value.

Images