CN106650816A - Video quality evaluation method and device - Google Patents

Abstract

The present invention obtains the image to be evaluated from the video to be evaluated, obtains the original image with the same frame number as the image to be evaluated from the original video, and performs structural similarity calculation on the image to be evaluated and the original image to obtain the image to be evaluated The structural similarity index of the image, extracting multiple image salient areas from the original image, and performing motion characteristic calculation on the image salient areas, calculating the motion characteristic factors of the image salient areas, using two adjacent image salient areas Calculate the similarity of the two adjacent salient regions of the image based on the motion characteristic factor, merge the two adjacent salient regions of the image whose similarity is greater than the preset similarity, and combine the pixel points in the merged image The gray value is weighted to obtain the spatio-temporal saliency of the original image, and the product of the spatio-temporal saliency of the original image and the structural similarity index of the image to be evaluated is weighted to obtain the quality index of the video to be evaluated.

Description

Method and device for evaluating video quality

technical field

The invention belongs to the field of video, and in particular relates to a video quality evaluation method and device.

Background technique

For video streaming media on an IP network, the video quality will be reduced during the process of collection, compression, transmission, and broadcast. Usually, at the encoding end, due to the use of different compression formats, some useful information is also deleted while visual redundant information is deleted. At the same time, block effects and boundary blurring will appear in the process of compression encoding. During transmission, due to limited bandwidth, network delays, jitter, etc. will cause packet loss or bit errors in the video stream, resulting in distortion and affecting the quality of video playback. At the receiving end, the same video, due to different decoders and video playback terminals, the user's visual experience is not the same. Therefore, in order to understand the degree of degradation of video quality and improve the user's visual perception by improving the algorithm and transmission mechanism, it is necessary to accurately evaluate the quality of video playback.

Currently, a video quality assessment method (Video Quality Assessment, VQA) adopts an objective quality assessment method. Most objective video quality assessment methods are based on the calculation of the image quality of a single frame, and then take the mean or weighted mean of all frame quality as the quality score of the entire video. It is fast, low in cost and easy to implement, and is widely used in engineering practice.

However, the existing video quality evaluation methods only use the weighted results of the quality evaluation results of single-frame images in the video as the evaluation results of video quality. This evaluation method only takes the quality of single-frame images as the evaluation basis, ignoring The coherence between images is lost, resulting in inaccurate evaluation of video quality.

Contents of the invention

The present invention provides a video quality evaluation method and device, aiming to solve the problem of inaccurate video quality evaluation by existing video quality evaluation methods.

The invention provides a video quality evaluation method, which comprises obtaining an image to be evaluated from a video to be evaluated, obtaining an original image having the same frame number as the image to be evaluated from an original video, and performing a structure on the image to be evaluated and the original image Similarity operation, to obtain the structural similarity index of the image to be evaluated, extract multiple image salient areas from the original image, and perform motion characteristic calculation on the image salient areas, calculate the motion characteristic factor of the image salient area, use Calculate the similarity of the two adjacent image salient regions based on the motion characteristic factors of the two adjacent image salient regions, merge the two adjacent image salient regions whose similarity is greater than the preset similarity, and The gray value of the pixels in the merged image is weighted to obtain the spatiotemporal saliency of the original image, and the product of the spatiotemporal saliency of the original image and the structural similarity index of the image to be evaluated is weighted to obtain the The quality index of the video to be rated.

The present invention provides a video quality evaluation device. The computing module is used to obtain an image to be evaluated from a video to be evaluated, an original image having the same frame number as the image to be evaluated from an original video, and the image to be evaluated and the image to be evaluated Structural similarity calculation is performed on the original image to obtain the structural similarity index of the image to be evaluated, the extraction module is used to extract multiple image salient areas from the original image, and the operation module is also used to perform motion characteristic calculation on the image salient areas, To calculate the motion characteristic factor of the salient region of the image, and calculate the similarity of the two adjacent salient regions of the image using the motion characteristic factors of the two adjacent salient regions of the image, the merging module is used for the similarity greater than Two adjacent image salient areas with preset similarities are merged, and the operation module is also used to carry out weighted operation on the gray value of the pixels in the merged image to obtain the spatiotemporal saliency of the original image, and, for The product of the spatiotemporal saliency of the original image and the structural similarity index of the image to be evaluated is weighted to obtain the quality index of the video to be evaluated.

The present invention provides a method and device for evaluating video quality. The image to be evaluated is obtained from the video to be evaluated, the original image having the same frame number as the image to be evaluated is obtained from the original video, and the image to be evaluated and the original Perform structural similarity calculation on the image to obtain the structural similarity index of the image to be evaluated, extract multiple image salient areas from the original image, and perform motion characteristic calculation on the image salient areas to calculate the motion characteristics of the image salient areas factor, using the motion characteristic factors of two adjacent image salient areas to calculate the similarity of the two adjacent image salient areas, and merging the two adjacent image salient areas whose similarity is greater than the preset similarity , and carry out a weighted operation on the gray value of the pixels in the merged image to obtain the spatiotemporal saliency of the original image, and perform a weighted operation on the product of the spatiotemporal saliency of the original image and the structural similarity index of the image to be evaluated , to get the quality index of the video to be evaluated. Compared with the prior art, the present invention has the beneficial effect that when calculating the quality index of the video to be evaluated, a motion characteristic factor that can reflect the continuity of motion is added, so that a more accurate video evaluation index can be obtained. At the same time, it only needs to calculate the motion characteristic factor of the salient area, which greatly reduces the calculation amount, saves calculation time and computing resources.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only are some embodiments of the present invention.

Fig. 1 is a schematic diagram of the implementation flow of a video quality evaluation method provided by the first embodiment of the present invention;

Fig. 2 is a schematic diagram of the implementation flow of a video quality evaluation method provided by the second embodiment of the present invention;

Fig. 3 is a spatial domain saliency map of an original image provided by the second embodiment of the present invention;

Fig. 4 is the binarized image of the salient area of the image in the spatial saliency map provided by the second embodiment of the present invention;

5 is a schematic structural diagram of a video quality evaluation device provided by a third embodiment of the present invention;

Fig. 6 is a schematic structural diagram of a video quality evaluation device provided by a fourth embodiment of the present invention.

detailed description

In order to make the purpose, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present invention.

The video quality evaluation method provided by the embodiments of the present invention can be applied to various terminal devices, such as terminals such as computers, mobile phones, and tablet computers, and other terminals.

Please refer to FIG. 1. FIG. 1 is a schematic diagram of the implementation flow of a video quality evaluation method provided in the first embodiment of the present invention. The video quality evaluation method shown in FIG. 1 mainly includes the following steps:

S101. Acquire the image to be evaluated from the video to be evaluated, obtain the original image with the same frame number as the image to be evaluated from the original video, and perform structural similarity calculation on the image to be evaluated and the original image to obtain the image to be evaluated Structural similarity index of images.

The video to be evaluated is the video whose quality index is to be evaluated. The video to be evaluated is generally a video that has been collected, compressed, transmitted or played. The image to be evaluated is an image in the video to be evaluated. The original video is the original video of the video to be evaluated. The original image is an image in the original video, and the frame sequence of the image to be evaluated in the video to be evaluated is the same as the frame number of the original image in the original video.

S102. Extract a plurality of image salient areas from the original image, and perform a motion characteristic operation on the image salient areas to calculate a motion characteristic factor of the image salient areas.

The salient area of the image is a salient area of the original image. The motion characteristic factor is used to represent the motion coherence of the salient area of the image.

S103. Using the motion characteristic factors of two adjacent salient areas of images to calculate the similarity of the two adjacent salient areas of images, and merging the two adjacent image salient areas whose similarity is greater than the preset similarity, And a weighted operation is performed on the gray values of the pixels in the merged image to obtain the spatiotemporal saliency of the original image.

The similarity is used to represent the similarity of the motion characteristic factors of the salient regions of the two images.

S104. Perform a weighted operation on the product of the spatiotemporal saliency of the original image and the structural similarity index of the image to be evaluated to obtain a quality index of the video to be evaluated. The spatiotemporal saliency is used to represent the saliency of the original image in space and time. The quality index of the video to be evaluated is used to evaluate the quality of the video to be evaluated.

In the embodiment of the present invention, the image to be evaluated is obtained from the video to be evaluated, the original image having the same frame number as the image to be evaluated is obtained from the original video, and the structural similarity calculation is performed on the image to be evaluated and the original image, Get the structural similarity index of the image to be evaluated, extract multiple image salient areas from the original image, and perform motion characteristic calculation on the image salient areas, calculate the motion characteristic factors of the image salient areas, use two adjacent Calculate the similarity of the two adjacent image salient regions based on the motion characteristic factor of each image salient region, merge the two adjacent image salient regions whose similarity is greater than the preset similarity, and combine the images in the merged image The gray value of each pixel is weighted to obtain the spatio-temporal saliency of the original image, and the product of the spatio-temporal saliency of the original image and the structural similarity index of the image to be evaluated is weighted to obtain the video to be evaluated quality index. Therefore, when calculating the quality index of the video to be evaluated, a more accurate video evaluation index can be obtained by adding a motion characteristic factor that can reflect the continuity of motion. At the same time, it only needs to calculate the motion characteristic factor of the salient area, which greatly reduces the calculation amount, saves calculation time and computing resources.

Please refer to FIG. 2. FIG. 2 is a schematic diagram of the implementation flow of a video quality evaluation method provided by the second embodiment of the present invention, which can be applied to all display image devices with display functions. The video quality evaluation method shown in FIG. 1 mainly Include the following steps:

S201. Acquire the image to be evaluated from the video to be evaluated, obtain the original image with the same frame number as the image to be evaluated from the original video, and perform structural similarity calculation on the image to be evaluated and the original image to obtain the image to be evaluated Structural similarity index of images.

The video to be evaluated is the video whose quality index is to be evaluated. The video to be evaluated is generally a video that has been collected, compressed, transmitted or played. The image to be evaluated is an image in the video to be evaluated. The original video is the original video of the video to be evaluated. The original image is an image in the original video, wherein the frame sequence of the image to be evaluated in the video to be evaluated is the same as the frame number of the original image in the original video, for example, the image to be evaluated is the i-th frame , then the original image is the i'th frame, i=i'.

Further, performing a structural similarity operation on the image to be evaluated and the original image to obtain a structural similarity index of the image to be evaluated, specifically including:

Perform brightness distortion calculation on the average brightness of the image to be evaluated and the average brightness of the original image to obtain the brightness distortion of the image to be evaluated.

Assuming that the image to be evaluated is the i-th frame in the video to be evaluated, the original image is the i'th frame, i=i', and the brightness distortion of the image to be evaluated is l(i), then

Among them, μ _i is the average brightness of the image to be evaluated, μ _i' is the average brightness of the original image, C ₁ =(K ₁ L) ² , L is the transformation range of the gray value of the pixel, L=100 ;K1<<1.

Contrast distortion calculation is performed on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image, and the brightness covariance between the image to be evaluated and the original image, to obtain the contrast distortion of the image to be evaluated.

Suppose the contrast distortion of the image to be evaluated is c(i), then

Wherein, σ _i is the standard deviation of the brightness of the image to be evaluated, σ _i' is the standard deviation of the brightness of the original image, σ _ii' is the brightness covariance of the image to be evaluated and the original image, C ₂ =(K ₂ L) ² , L=100; K2<<1.

The structure distortion calculation is performed on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image, and the brightness covariance between the image to be evaluated and the original image, to obtain the structure distortion degree of the image to be evaluated.

Suppose the structural distortion of the image to be evaluated is s(i), then

Wherein, C ₃ =C ₂ /2.

The brightness distortion, the contrast distortion and the structure distortion are multiplied to obtain the structural similarity index of the image to be evaluated.

Suppose the structural similarity index of the image to be evaluated is SSIM(i), then

SSIM(i) = [l(i)] ^α · [c(i)] ^β · [s(i)] ^γ

Among them, α=β=γ=1, then,

S202. Perform spectrum redundancy calculation on the original image to obtain spectrum redundancy of the original image.

Let the original image be the i'th frame, and the spectral redundancy of the original image is R(i'), then,

R(i')=A(i')-I(i');

Among them, I(i')=A(i')×h _n (i'), A(i') is the logarithmic amplitude spectrum, h _n (i′) is an n×n mean value filter, is the logarithmic spectrum in the frequency domain, and g(i') is the Gaussian filter function.

S203. Perform a spectral redundancy calculation on the spectral redundancy of the original image and the phase spectrum of the original image to obtain a spatial domain saliency map of the original image.

Let S _S (i') be the spatial saliency map of the original image, then,

in, is the inverse Fourier transform, and P(i') is the phase spectrum of the original image.

As shown in FIG. 3 , FIG. 3 shows the spatial domain saliency map of the original image in the embodiment of the present invention, wherein the region with a larger gray value has a higher saliency.

S204. Determine an area composed of pixels whose grayscale values are greater than a preset grayscale value in the spatial saliency map as a salient area of the image.

Let r(f) be the salient region of the image in the spatial saliency map, then,

Among them, threshold is the preset gray value, threshold=E(SS(f))*3, f is the gray value in the spatial saliency map, and E(SS(f)) is the spatial saliency map S _S (i' ), r(f)=1 means that the area is a salient area of the image, and r(f)=0 means that the image is a non-salient area of the image.

For the convenience of explanation, the image salient area in the spatial saliency map shown in Fig. 3 is binarized according to the preset gray level critical value, and Fig. 4 is obtained. The image after value processing, wherein the white area shown in Figure 4 is the location of the salient area of the image.

S205. Extract a moving object in the salient area of the image, and perform a two-norm operation on the relative motion vector and the background motion vector of the moving object respectively, to obtain the relative motion vector strength and the background motion vector strength of the salient area of the image respectively.

Further, extracting the moving target in the salient area of the image specifically includes:

comparing the block in the salient area of the image with the block in the salient area of the reference image;

Selecting the block with the highest frequency of occurrence in the salient area of the image and the salient area of the reference image;

Subtract the most frequently occurring block from the block in the salient area of the image, and take the remaining block in the salient area of the image as the moving target.

The salient area of the reference image is the same area as the salient area of the image in the previous frame image of the original image. Let (x ₀ , y ₀ ) be the coordinates of the moving target in the salient area of the image, then,

(x _o ,y _o )=(x',y')-mode(x',y')

Among them, (x', y') is the coordinate of the block in the salient area of the image. mode(x',y') indicates the block with the highest frequency of occurrence.

Wherein, (x, y) is a block in the salient region of the reference image. a ₁ is the zoom out parameter, a ₂ is the enlargement parameter, a ₃ is the anticlockwise rotation parameter, a ₄ is the clockwise rotation parameter, a ₅ is the horizontal movement parameter, a ₆ is the vertical movement parameter. According to the minimum error function and iterative least squares

with The values of a ₁ , a ₂ , a ₃ , a ₄ , a ₅ and a ₆ can be found.

in, x _i and y _i are the coordinates of the moving object in the salient area of the reference image, and _xi ' and y _i ' represent the coordinates of the moving object in the salient area of the image respectively.

The relative motion vector and the background motion vector of the moving target are respectively subjected to two-norm operations to obtain the relative motion vector strength and the background motion vector strength of the salient area of the image respectively.

Assume represents the motion vector of the moving target (x ₀ , y ₀ ), then,

in, is the absolute motion vector, for the background motion vector, is the relative motion vector, then, the relative motion vector strength v _r and the background motion vector strength v _b of the salient area of the image are respectively:

In practical applications, the absolute motion vector in the salient area of the image Background motion vector and the relative motion vector The values of can be obtained by the global motion estimation method.

S206. Perform a weighting operation on the relative motion vector strength and the background motion vector strength to obtain a motion factor of the salient area of the image.

Let M _v be the motion factor of the salient area of the image, then,

M _v ＝(1-ω _b )v _r +ω _b v _b

Among them, _ωb is the weight of the background motion vector strength in the salient area of the image,

ω _b = ω _1b × ω _2b × ω _3b

Among them, N _b is the number of macroblocks whose background motion vector is not 0 in the salient area of the image, N(s) is the total number of macroblocks in the salient area of the image, v _bx and v _by are the abscissas and On the ordinate, N _kB is the number of macroblocks with non-zero background motion vectors, s _ib is each non-empty dimension in the direction of the background motion vector, N(s _i ) is the number of macroblocks with non-zero motion vectors in each dimension , i≤36.

S207. Perform a division operation between the motion factor of the salient area of the image and the maximum value of the motion factor of the salient area of the image in the original video, and perform a logarithmic operation on the calculated parameters to obtain a motion characteristic factor of the salient area of the image.

Let ω _M be the motion characteristic factor of the salient area of the image, then,

Among them, M _max is the maximum value of the motion factor in the salient area of the image in the original video, α is an adjustment constant and α>1, through nonlinear fitting and numerical analysis, set α=2.

S208. Using the motion characteristic factors of the two adjacent salient regions of the image to calculate the similarity of the two adjacent salient regions of the image, merging the two adjacent notable regions of the image whose similarity is greater than the preset similarity, and The gray value of the pixels in the merged image is weighted to obtain the spatiotemporal saliency of the original image.

Further, the similarity of the two adjacent image salient regions is calculated by using the motion characteristic factors of the two adjacent image salient regions, specifically including:

Among the plurality of image salient regions, two adjacent image salient regions are selected.

The target value is obtained by subtracting the maximum value of the motion characteristic factors of the two image salient regions from the absolute value of the difference between the motion characteristic factors of the two image salient regions.

The target value is divided by the maximum value of the motion characteristic factors of the salient regions of the two images, and the obtained quotient is used as the similarity between the salient regions of the two adjacent images.

Let S _motion (r _i , r _j ) be the similarity of the salient regions of two adjacent images, then

Among them, r _i , r _j represent the salient regions of two adjacent images, with Indicates the motion characteristic factor of the salient regions of two adjacent images, for with The maximum value of , S _motion (r _i , r _j ) is the similarity between the salient regions of adjacent images.

In practical applications, there will be a plurality of similar image salient areas in the image salient area _ri , and then select the area with the fewest pixels separated from the image salient area _ri in the multiple similar image salient areas as the image The salient regions of the image adjacent to the salient region _ri . For example, there are r ₁ , r ₂ and r ₃ in the salient areas of the image that are close to r _i , where r _i is separated from r ₁ , r ₂ and r ₃ by 0, 1 and 2 pixels respectively, then r _i and r ₁ is the salient area of the adjacent image. Wherein, the separated pixel points are the pixel points separated by the closest adjacent regions between two prominent regions of the image.

The salient regions of two adjacent images whose similarity is greater than the preset similarity are merged, and the gray value of the pixel in the merged image is weighted to obtain the spatiotemporal saliency of the original image.

The preset similarity is a preset similarity, which can be set according to the actual situation. The maximum value of the similarity can be set as the preset similarity, or a specific similarity value can be set as the preset similarity.

It should be noted that the image obtained by merging the salient regions of two adjacent images whose similarity is greater than the preset similarity is the spatio-temporal saliency map of the original image, and S(i) is the spatio-temporal saliency map of the original image, then,

S(i)＝{r _x ,r _x ∈r _i ∪r _j }

Among them, {r _x ,r _x ∈r _i ∪r _j } means to merge the salient regions of the image whose similarity is greater than the preset similarity.

Let D _s (i) be the spatiotemporal saliency of the original image, then,

Among them, D _i (n) is the gray value of each pixel in the spatiotemporal saliency map S(i), and N is the total number of pixels in the spatiotemporal saliency map S(i).

In this way, the motion characteristic factor calculated by extracting the salient area of the image in the spatial saliency map of the original image is more in line with the visual characteristics of the human eye, and can accurately reflect the motion characteristics of the video to be evaluated, and obtain a more accurate video to be evaluated. quality index. Moreover, it is only necessary to calculate the motion characteristic factor of the salient area instead of the motion characteristic factor of the entire original image, which can greatly reduce the calculation amount, save calculation time and computing resources.

S209. Perform a weighted operation on the product of the spatiotemporal saliency of the original image and the structural similarity index of the image to be evaluated to obtain a quality index of the video to be evaluated.

When calculating the quality index of the video to be evaluated, the spatiotemporal saliency of each frame of the original image in the original video and the structural similarity index of each frame of the image in the video to be evaluated are weighted to obtain the video to be evaluated quality index.

Let SMW-SSIM be the quality index of the video to be evaluated, then,

Among them, SSIM(i) represents the structural similarity index of the image to be evaluated, and D _S (i) represents the spatiotemporal saliency of the original image.

In the embodiment of the present invention, the image to be evaluated is obtained from the video to be evaluated, the original image having the same frame number as the image to be evaluated is obtained from the original video, and the structural similarity calculation is performed on the image to be evaluated and the original image, Obtain the structural similarity index of the image to be evaluated. Perform spectral redundancy calculation on the original image to obtain the spectral redundancy of the original image, perform spectral redundancy calculation on the spectral redundancy of the original image and the phase spectrum of the original image, obtain the spatial domain saliency map of the original image, and The spatial domain saliency map is binarized to obtain multiple image salient areas, and the moving object is extracted in the image salient area, and the relative motion vector and the background motion vector of the moving object are respectively subjected to two-norm operations to obtain the The relative motion vector strength and the background motion vector strength of the salient area of the image are weighted by the relative motion vector strength and the background motion vector strength to obtain the motion factor of the salient area of the image, and the motion factor of the salient area of the image in the original video is The maximum value of is divided by the motion factor of the salient area of the image, and the calculated parameters are logarithmically calculated to obtain the motion characteristic factor of the salient area of the image, which is calculated by using the motion characteristic factors of two adjacent image salient areas Find out the similarity of the salient regions of the two adjacent images, merge the salient regions of the two adjacent images whose similarity is greater than the preset similarity, and perform a weighted operation on the gray value of the pixels in the merged image, The spatio-temporal saliency of the original image is obtained, and the product of the spatio-temporal saliency of the original image and the structural similarity index of the image to be evaluated is weighted to obtain the quality index of the video to be evaluated. Therefore, when calculating the quality index of the video to be evaluated, a more accurate video evaluation index can be obtained by adding a motion characteristic factor that can reflect the continuity of motion. At the same time, it only needs to calculate the motion characteristic factor of the salient area, which greatly reduces the calculation amount, saves calculation time and computing resources.

Please refer to FIG. 5 . FIG. 5 is a schematic structural diagram of a video quality evaluation device provided by a third embodiment of the present invention. For ease of description, only parts related to the embodiment of the present invention are shown. The video quality evaluation device illustrated in FIG. 5 may be the subject of execution of the video quality evaluation method provided in the embodiments shown in FIGS. 1 and 2 , and may be a control module in the video quality evaluation device. The video quality evaluation device illustrated in FIG. 5 mainly includes: a calculation module 501 , an extraction module 502 and a combination module 503 . The above functional modules are described in detail as follows:

The calculation module 501 is used to obtain the image to be evaluated from the video to be evaluated, obtain an original image having the same frame number as the image to be evaluated from the original video, and perform a structural similarity operation on the image to be evaluated and the original image, Obtain the structural similarity index of the image to be evaluated.

The video to be evaluated is the video whose quality index is to be evaluated. The video to be evaluated is generally a video that has been collected, compressed, transmitted or played. The image to be evaluated is an image in the video to be evaluated. The original video is the original video of the video to be evaluated. The original image is an image in the original video, and the frame sequence of the image to be evaluated in the video to be evaluated is the same as the frame number of the original image in the original video.

The extraction module 502 is configured to extract a plurality of image salient regions from the original image.

The salient area of the image is a salient area of the original image. The motion characteristic factor is used to represent the motion coherence of the salient area of the image.

Further, the calculation module 501 is also used to perform motion characteristic calculation on the salient area of the image to calculate the motion characteristic factor of the salient area of the image; and calculate the relative The similarity between the salient regions of two adjacent images.

The similarity is used to represent the similarity of the motion characteristic factors of the salient regions of the two images.

The merging module 503 is configured to merge two adjacent salient regions of images whose similarity is greater than a preset similarity.

Further, the calculation module 501 is also used to perform weighted calculations on the gray values of the pixels in the merged image to obtain the spatio-temporal saliency of the original image; and, the spatio-temporal saliency of the original image and the image to be evaluated The product of the structural similarity index is weighted to obtain the quality index of the video to be evaluated.

Please refer to the first and second embodiments shown in FIG. 1 and FIG. 2 for details of the embodiments of the present invention, and details are not repeated here.

In the embodiment of the present invention, the calculation module 501 is used to obtain the image to be evaluated from the video to be evaluated, obtain the original image with the same frame number as the image to be evaluated from the original video, and perform the evaluation on the image to be evaluated and the original image Structural similarity operation to obtain the structural similarity index of the image to be evaluated, the extraction module 502 is used to extract a plurality of image salient areas from the original image, and the operation module 501 is also used to perform motion characteristic calculation on the image salient areas to obtain Calculate the motion characteristic factor of the salient area of the image; and, use the motion characteristic factors of the two adjacent salient areas of the image to calculate the similarity of the two adjacent salient areas of the image, and the merging module 503 is used for the similarity greater than Two adjacent image salient areas with preset similarities are merged, and the calculation module 501 is used to perform weighted calculations on the gray values of the pixels in the merged image to obtain the spatiotemporal saliency of the original image; and, for the The product of the spatiotemporal saliency of the original image and the structural similarity index of the image to be evaluated is weighted to obtain the quality index of the video to be evaluated. Therefore, when calculating the quality index of the video to be evaluated, a more accurate video evaluation index can be obtained by adding a motion characteristic factor that can reflect the continuity of motion. At the same time, it only needs to calculate the motion characteristic factor of the salient area, which greatly reduces the calculation amount, saves calculation time and computing resources.

Please refer to FIG. 6 . FIG. 6 is a schematic structural diagram of a video quality evaluation device provided by a fourth embodiment of the present invention. For ease of description, only parts related to the embodiment of the present invention are shown. The video quality evaluation device illustrated in FIG. 6 may be the subject of execution of the video quality evaluation method provided in the embodiments shown in FIGS. 1 and 2 , and may be a control module in the video quality evaluation device. The video quality evaluation device illustrated in Fig. 6 mainly includes: an operation module 601, an extraction module 602 and a combination module 603; wherein, the operation module 601 includes: a distortion operation module 611, an exponential operation module 621, a vector operation module 631, and a motion characteristic operation module 641 . The selection module 651 and the similarity calculation module 661 ; the extraction module 602 includes: a spectrum calculation module 612 , a spectrum redundancy calculation module 622 and a determination module 632 . The above functional modules are described in detail as follows:

The calculation module 601 is used to obtain the image to be evaluated from the video to be evaluated, obtain an original image having the same frame number as the image to be evaluated from the original video, and perform a structural similarity operation on the image to be evaluated and the original image, Obtain the structural similarity index of the image to be evaluated.

The video to be evaluated is the video whose quality index is to be evaluated. The video to be evaluated is generally a video that has been collected, compressed, transmitted or played. The image to be evaluated is an image in the video to be evaluated. The original video is the original video of the video to be evaluated. The original image is an image in the original video, and the frame sequence of the image to be evaluated in the video to be evaluated is the same as the frame number of the original image in the original video.

The extraction module 602 is configured to extract a plurality of image salient regions from the original image.

The salient area of the image is a salient area of the original image. The motion characteristic factor is used to represent the motion coherence of the salient area of the image.

Further, the calculation module 601 is also used to perform motion characteristic calculation on the salient area of the image to calculate the motion characteristic factor of the salient area of the image; and calculate the relative The similarity between the salient regions of two adjacent images.

The similarity is used to represent the similarity of the motion characteristic factors of the salient regions of the two images.

The merging module 603 is configured to merge two adjacent salient regions of the image whose similarity is greater than a preset similarity.

Further, the calculation module 601 is also used to perform weighted calculations on the gray values of pixels in the merged image to obtain the spatio-temporal saliency of the original image; and, the spatio-temporal saliency of the original image and the image to be evaluated The product of the structural similarity index is weighted to obtain the quality index of the video to be evaluated.

Further, the calculation module 601 includes: a distortion calculation module 611 and an exponential calculation module 621 .

The distortion calculation module 611 is configured to perform brightness distortion calculation on the average brightness of the image to be evaluated and the average brightness of the original image to obtain the brightness distortion of the image to be evaluated.

Further, the distortion calculation module 611 is also used to perform a contrast distortion calculation on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image, and the brightness covariance between the image to be evaluated and the original image, and obtain the Contrast distortion of the image to be evaluated.

Further, the distortion calculation module 611 is also used to perform structural distortion calculation on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image, and the brightness covariance between the image to be evaluated and the original image, and obtain the The structural distortion of the image to be evaluated.

The index calculation module 621 is configured to perform multiplication operations on the brightness distortion, the contrast distortion and the structure distortion to obtain the structure similarity index of the image to be evaluated.

Further, the extraction module 602 includes: a spectrum operation module 612 , a spectrum redundancy operation module 622 and a binary operation module 632 .

The spectrum calculation module 612 is configured to perform spectrum redundancy calculation on the original image to obtain the spectrum redundancy of the original image.

The spectral redundancy computing module 622 is configured to perform spectral redundancy computing on the spectral redundancy of the original image and the phase spectrum of the original image to obtain a spatial domain saliency map of the original image.

The determining module 632 is configured to determine an area composed of pixels whose gray value is greater than a preset gray value in the spatial saliency map as the image salient area.

Further, the computing module 601 also includes: a vector computing module 631 and a motion characteristic computing module 641 .

The vector operation module 631 is used to extract a moving object in the salient area of the image, and perform a two-norm operation on the relative motion vector and the background motion vector of the moving object respectively, to obtain the relative motion vector strength and the background motion of the salient area of the image Vector strength.

The motion characteristic calculation module 641 is configured to perform weighted calculations on the relative motion vector strength and the background motion vector strength to obtain the motion factor of the salient area of the image.

The motion characteristic operation module 641 is also used to divide the motion factor of the salient area of the image by the maximum value of the motion factor of the salient area of the image in the original video, and perform logarithmic operation on the calculated parameters to obtain the image The motion characteristic factor of the salient area.

Further, the computing module 601 also includes: a selecting module 651 and a similar computing module 661 .

The selection module 651 is configured to select two adjacent image salient areas among the plurality of image salient areas.

The similarity operation module 661 is used for subtracting the absolute value of the maximum value of the motion characteristic factors of the two image salient regions and the difference between the motion characteristic factors of the two image salient regions to obtain a target value.

The similarity calculation module 661 is further configured to divide the target value by the maximum value of the motion characteristic factors of the two salient regions of the image, and use the calculated quotient as the similarity between the two adjacent salient regions of the image.

Please refer to the first and second embodiments shown in FIG. 1 and FIG. 2 for details of the embodiments of the present invention, and details are not repeated here.

In the embodiment of the present invention, the calculation module 601 is used to obtain the image to be evaluated from the video to be evaluated, obtain the original image with the same frame number as the image to be evaluated from the original video, and perform the evaluation of the image to be evaluated and the original image Structural similarity calculation to obtain the structural similarity index of the image to be evaluated, the extraction module 602 is used to extract a plurality of image salient regions from the original image, and the operation module 601 is also used to perform motion characteristic calculation on the image salient regions to obtain Calculate the motion characteristic factor of the salient area of the image; and, use the motion characteristic factors of the two adjacent salient areas of the image to calculate the similarity of the two adjacent salient areas of the image, and the merging module 603 is used for the similarity greater than Two adjacent image salient areas with preset similarities are merged, and the operation module 601 is used to perform weighted operation on the gray value of the pixels in the merged image to obtain the spatiotemporal saliency of the original image; and, for the The product of the spatiotemporal saliency of the original image and the structural similarity index of the image to be evaluated is weighted to obtain the quality index of the video to be evaluated. The distortion calculation module 611 is used to perform brightness distortion calculation on the brightness average value of the image to be evaluated and the brightness average value of the original image to obtain the brightness distortion degree of the image to be evaluated, and the distortion calculation module 611 is also used to The brightness standard deviation of the evaluation image, the brightness standard deviation of the original image, and the brightness covariance between the image to be evaluated and the original image are subjected to a contrast distortion calculation to obtain the contrast distortion of the image to be evaluated. The distortion calculation module 611 also uses The structural distortion calculation is carried out on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image, and the brightness covariance of the image to be evaluated and the original image to obtain the structural distortion of the image to be evaluated, and the exponential operation Module 621 is used to perform multiplication operation on the brightness distortion, the contrast distortion and the structure distortion to obtain the structural similarity index of the image to be evaluated, and the spectrum calculation module 612 is used to perform spectrum redundancy calculation on the original image, Obtain the spectral redundancy of the original image, and the spectral redundancy calculation module 622 is used to perform spectral redundancy calculation on the spectral redundancy of the original image and the phase spectrum of the original image to obtain the spatial saliency map of the original image, and determine the module 632 It is used to determine the area composed of pixels whose gray value is greater than the preset gray value in the spatial saliency map as the salient area of the image. The vector operation module 631 is used to extract moving objects in the salient area of the image, and respectively The relative motion vector and the background motion vector of the object are subjected to two-norm operation to obtain the relative motion vector strength and the background motion vector strength of the salient area of the image, and the motion characteristic calculation module 641 is used for the relative motion vector strength and the background motion vector strength The weighted operation is performed to obtain the motion factor of the salient area of the image. The motion characteristic calculation module 641 is also used for dividing the motion factor of the image salient area by the maximum value of the motion factor of the image salient area in the original video, and performing logarithmic operation on the calculated parameters to obtain the image salient area The motion characteristic factor of the area. The selection module 651 is used for selecting two adjacent image salient areas among the multiple image salient areas. The similarity operation module 661 is used for subtracting the absolute value of the difference between the maximum value of the motion factors of the two salient regions of the image and the difference between the motion characteristic factors of the two salient regions of the image to obtain a target value. The similarity calculation module 661 is also used to divide the target value by the maximum value of the motion factors of the two salient regions of the image, and use the calculated quotient as the similarity between the two adjacent salient regions of the image. Therefore, when calculating the quality index of the video to be evaluated, a more accurate video evaluation index can be obtained by adding a motion characteristic factor that can reflect the continuity of motion. At the same time, it only needs to calculate the motion characteristic factor of the salient area, which greatly reduces the calculation amount, saves calculation time and computing resources.

In the multiple embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication link shown or discussed may be through some interfaces, and the indirect coupling or communication link of devices or modules may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules.

If the integrated modules are realized in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes.

It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence. Because of the present invention, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

In the foregoing embodiments, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

The above is the description of the video quality evaluation method and device provided by the present invention. For those skilled in the art, according to the idea of the embodiment of the present invention, there will be changes in the specific implementation and application range. In summary, this The content of the description should not be construed as limiting the present invention.

Claims (10)

1. A video quality evaluation method, the method comprising:

acquiring an image to be evaluated from a video to be evaluated, acquiring an original image with the same frame number as the image to be evaluated from an original video, and performing structural similarity operation on the image to be evaluated and the original image to obtain a structural similarity index of the image to be evaluated;

extracting a plurality of image salient regions from the original image, and performing motion characteristic operation on the image salient regions to calculate motion characteristic factors of the image salient regions;

calculating the similarity of the two adjacent image salient regions by utilizing the motion characteristic factors of the two adjacent image salient regions, merging the two adjacent image salient regions with the similarity larger than the preset similarity, and performing weighted operation on gray values of pixel points in the merged image to obtain the space-time saliency of the original image;

and performing weighted operation on the product of the space-time significance of the original image and the structural similarity index of the image to be evaluated to obtain the quality index of the video to be evaluated.

2. The method according to claim 1, wherein the performing the structural similarity operation on the image to be evaluated and the original image to obtain the structural similarity index of the image to be evaluated specifically comprises:

performing brightness distortion degree operation on the brightness average value of the image to be evaluated and the brightness average value of the original image to obtain the brightness distortion degree of the image to be evaluated;

performing contrast distortion degree operation on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image and the brightness covariance of the image to be evaluated and the original image to obtain the contrast distortion degree of the image to be evaluated;

performing structural distortion degree operation on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image and the brightness covariance of the image to be evaluated and the original image to obtain the structural distortion degree of the image to be evaluated;

and multiplying the brightness distortion degree, the contrast distortion degree and the structure distortion degree to obtain a structure similarity index of the image to be evaluated.

3. The method according to claim 1, wherein the extracting a plurality of image salient regions from the original image specifically comprises:

performing frequency spectrum redundancy operation on the original image to obtain frequency spectrum redundancy of the original image;

performing spectrum redundancy operation on the frequency spectrum redundancy of the original image and the phase spectrum of the original image to obtain a space domain saliency map of the original image;

and determining a region composed of pixel points with the gray values larger than a preset gray value in the airspace saliency map as the image saliency region.

4. The method according to claim 1 or 3, wherein the performing the motion characteristic operation on the image salient region to calculate the motion characteristic factor of the image salient region specifically comprises:

extracting a moving target from the image salient region, and respectively performing two-norm operation on a relative motion vector and a background motion vector of the moving target to obtain the relative motion vector intensity and the background motion vector intensity of the image salient region;

carrying out weighting operation on the intensity of the relative motion vector and the intensity of the background motion vector to obtain a motion factor of the image salient region;

and dividing the motion factor of the image salient region by the maximum value of the motion factor of the image salient region in the original video, and carrying out logarithm operation on the calculated parameters to obtain the motion characteristic factor of the image salient region.

5. The method according to claim 1, wherein the calculating the similarity between two adjacent image salient regions by using the motion characteristic factors of the two adjacent image salient regions specifically comprises:

selecting two adjacent image salient regions from the plurality of image salient regions;

subtracting the absolute value of the difference value between the maximum value of the motion characteristic factors of the two image salient regions and the motion characteristic factors of the two image salient regions to obtain a target value;

and dividing the target value by the maximum value of the motion characteristic factors of the two image salient regions, and taking the calculated quotient as the similarity of the two adjacent image salient regions.

6. A video quality evaluation apparatus, characterized in that the apparatus comprises:

the operation module is used for acquiring an image to be evaluated from a video to be evaluated, acquiring an original image with the same frame number as the image to be evaluated from an original video, and performing structural similarity operation on the image to be evaluated and the original image to obtain a structural similarity index of the image to be evaluated;

the extraction module is used for extracting a plurality of image salient regions from the original image;

the operation module is further configured to perform motion characteristic operation on the image salient region to calculate a motion characteristic factor of the image salient region; calculating the similarity of the two adjacent image salient regions by using the motion characteristic factors of the two adjacent image salient regions;

the merging module is used for merging the two adjacent image salient regions with the similarity greater than the preset similarity;

the operation module is further used for performing weighted operation on the gray values of the pixel points in the combined image to obtain the space-time significance of the original image; and performing weighted operation on the product of the space-time significance of the original image and the structural similarity index of the image to be evaluated to obtain the quality index of the video to be evaluated.

7. The apparatus of claim 1, wherein the operation module comprises:

the distortion operation module is used for performing brightness distortion degree operation on the brightness average value of the image to be evaluated and the brightness average value of the original image to obtain the brightness distortion degree of the image to be evaluated;

the distortion operation module is further configured to perform contrast distortion degree operation on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image, and the brightness covariance of the image to be evaluated and the original image to obtain a contrast distortion degree of the image to be evaluated;

the distortion operation module is further configured to perform structural distortion degree operation on the brightness standard deviation of the image to be evaluated, the brightness standard deviation of the original image, and the brightness covariance of the image to be evaluated and the original image to obtain a structural distortion degree of the image to be evaluated;

and the index operation module is used for multiplying the brightness distortion degree, the contrast distortion degree and the structure distortion degree to obtain a structure similarity index of the image to be evaluated.

8. The apparatus of claim 6, wherein the extraction module comprises:

the frequency spectrum operation module is used for carrying out frequency spectrum redundancy operation on the original image to obtain the frequency spectrum redundancy of the original image;

the spectrum redundancy operation module is used for performing spectrum redundancy operation on the spectrum redundancy of the original image and the phase spectrum of the original image to obtain a space domain saliency map of the original image;

and the determining module is used for determining an area formed by pixel points with the gray value larger than a preset gray value in the airspace saliency map as the image saliency area.

9. The apparatus of claim 6 or 7, wherein the arithmetic module further comprises:

the vector operation module is used for extracting a moving target from the image salient region and respectively carrying out two-norm operation on a relative motion vector and a background motion vector of the moving target to obtain the relative motion vector intensity and the background motion vector intensity of the image salient region;

the motion characteristic operation module is used for performing weighted operation on the relative motion vector intensity and the background motion vector intensity to obtain a motion factor of the image salient region;

the motion characteristic operation module is further configured to perform division operation on the motion factor of the image salient region and the maximum value of the motion factor of the image salient region in the original video, and perform logarithm operation on the calculated parameter to obtain the motion characteristic factor of the image salient region.

10. The apparatus of claim 1, wherein the operation module further comprises:

the selection module is used for selecting two adjacent image salient regions from the plurality of image salient regions;

the similarity operation module is used for carrying out subtraction operation on the maximum value of the motion characteristic factors of the two image salient regions and the absolute value of the difference value of the motion characteristic factors of the two image salient regions to obtain a target value;

the similarity operation module is further configured to perform division operation on the target value and the maximum value of the motion characteristic factors of the two image salient regions, and use the calculated quotient as the similarity of the two adjacent image salient regions.