WO2016131270A1 - Error concealment method and apparatus - Google Patents

Abstract

An error concealment method, and the method comprising: providing each error block in a current error coding tree unit (CTU) as a current error block in turn; determining a space-domain left neighboring block and a space-domain upper neighboring block of the current error block, the space-domain left neighboring block and the space-domain upper neighboring block being adjacent to the current error block in space domain; comparing the space-domain correlation and the time-domain correlation when the space-domain left neighboring block and the space-domain upper neighboring block of the current error block are available and neither is an intra-frame prediction block; performing the space-domain interpolation error concealment if the space-domain correlation of the current error block is greater than the predetermined intensity threshold; and performing the time-domain error concealment if the space-domain correlation of the current error block is not greater than the predetermined intensity threshold. The solution has good repairing effect on the damaged HEVC video coding stream by combining the coding characteristic of HEVC video to utilize the correlation of error block data and space-domain time-domain neighboring data.

Description

Method and device for error concealment Technical field

This document refers to, but is not limited to, the field of video, and specifically relates to methods and apparatus for error concealment.

Background technique

The tremendous advances in network technology and video coding technology have made radio, television, and video communications increasingly popular. Inevitable mobile channel fading and network congestion will inevitably cause video data to be damaged during transmission. Error hiding the video at the decoding end is an important method to improve the user experience.

The error concealment technique recovers the damaged image approximately by using the correlation between the correctly decoded video information and the corrupted data after detecting the video error at the decoding end. The error concealment technique can be divided into spatial domain error concealment and time domain error concealment according to the spatial and temporal correlation between available data and error data.

The spatial error concealment mainly utilizes the spatial correlation of the video image, and the data of the damaged region can be reconstructed by interpolating the correct data adjacent in the frame. In the related art, there are two main types of spatial error concealment methods: BI (Bi-linear Interpolation) and DI (Directional Interpolation).

The time domain error concealment mainly utilizes the temporal correlation of the video image, that is, the erroneous data in the current image is estimated using the image data that has been correctly decoded in the reference image. A representative method is BMA (Boundary Matching Algorithm). The basic idea of the algorithm is to extract the motion vector from the correct receiving block around the damaged block as the candidate motion vector of the current damaged block according to the principle of motion consistency between adjacent blocks, and then select the motion-compensated macroblock boundary. The motion difference vector between the pixel and the missing macroblock boundary pixel and the smallest (Sum of Absolute Difference, SAD) is used as the motion vector of the current lost macroblock, and then motion compensation is performed to repair the damaged macroblock.

Because of the correlation between data, the error concealment technique combined with specific coding characteristics has a better effect. Therefore, there is an error concealment technique in the art that combines the above-mentioned spatial error concealment method and time domain error concealment method with the H.264 coding standard, and the error concealment technique is also true. Brought an updated error concealment effect. However, with the emergence of the High Efficiency Video Coding (HEVC) standard, the error concealment method combined with the H264 coding standard is difficult to obtain satisfactory video quality for HEVC video, and the damaged CTU (Coding TreeUnit, coding) in the HEVC standard. The size of the tree unit is much larger than the size of the macroblock in the H.264 standard. In addition, the variable coding unit size, multi-angle intra prediction and advanced motion vector prediction methods and H.264 coding standards in the HEVC standard are larger. the difference. Therefore, it is necessary to find a error concealment technique suitable for HEVC video.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

Embodiments of the present invention provide a method and apparatus for error concealment to solve the technical problem that it is difficult to obtain satisfactory video quality in HEVC video.

The embodiment of the invention provides a method for error concealing, the method comprising:

Each error block in the current error coding tree unit CTU is sequentially taken as the current error block;

Determining a left-space neighboring block of the current error block and a neighboring block on the air domain, wherein the left adjacent block of the spatial domain and the adjacent block on the airspace are adjacent to the error block on the airspace;

When the left adjacent block of the spatial domain of the current error block and the adjacent block on the air domain are available and are not intra prediction blocks, time domain and spatial domain correlation are compared;

If the spatial domain correlation of the current error block is greater than the preset intensity threshold, the spatial domain interpolation error concealment is performed; if the spatial domain correlation of the current error block is not greater than the preset intensity threshold, the time domain error concealment is performed.

Optionally, the method further includes:

When the left adjacent block of the spatial domain and the adjacent block on the air domain are available and both are intra prediction blocks, intra prediction error concealment is performed on the current error block.

Optionally,

The comparison of the time domain and the airspace correlation includes:

When the SADS is less than (SADT1+SADT2)/3, the spatial correlation of the current error block is strong;

When the SADS is greater than or equal to (SADT1+SADT2)/3, the time domain correlation of the current error block is strong;

among them:

Where m is the size of the current error block, i, j is the pixel coordinate of the current error block, n is the image number, and P(n, 0, 0) is the pixel value of the upper left corner of the current error block.

Optionally,

The performing spatial domain interpolation error concealment includes:

Calculating a predicted pixel P'(i,j) of the current error block according to a spatial neighboring block of the current error block;

Where i, j is the pixel coordinate of the current error block; P'(0, 0) is the pixel value of the upper left corner of the current error block.

Optionally,

The performing time domain error concealment includes:

Obtaining a motion vector MV in the left adjacent block of the current error block, the adjacent block on the air domain, the lower left adjacent block in the spatial domain, and the upper right adjacent block in the spatial domain;

When the difference between the same components of any two MVs is greater than a predetermined threshold, the current error block is divided into 4 error sub-blocks; when the difference between the same components of any two MVs is less than or equal to a predetermined threshold The current error block as a whole is regarded as one error sub-block;

Do the following for each error sub-block:

The MV and the zero motion MV available in the spatially left adjacent subblock of the erroneous subblock, the adjacent subblock on the spatial domain, the right adjacent subspace of the spatial domain, the adjacent subspace in the spatial domain, and the adjacent subblock in the time domain are used as the MV of the erroneous sub-block, each candidate recovery block of the erroneous sub-block is obtained according to each MV of the erroneous sub-block;

A candidate restoration block having the smallest boundary matching difference in each candidate restoration block is taken as the restored pixel block of the current error block.

Optionally,

The performing intra prediction error concealment on the current error block includes:

Determining a prediction mode of the left adjacent block of the spatial domain and the adjacent block of the spatial domain as a candidate prediction mode of the current error block;

Obtaining each candidate recovery block of the current erroneous block according to each candidate prediction mode; using a candidate restoration block having the smallest boundary matching difference in each candidate recovery block as the restored pixel block of the current erroneous block.

Optionally,

The step of using each error block in the current error coding tree unit CTU as the current error block in advance includes:

The current error is caused when any of the neighboring CTUs in the airspace of the current erroneous CTU, the neighboring CTUs in the airspace, the left neighboring CTUs in the airspace, the right neighboring CTUs in the airspace, and the neighboring CTUs in the time domain are divided. The CTU is divided into 4 error blocks;

When the adjacent CTU in the airspace of the current erroneous CTU, the adjacent CTU in the airspace, the left neighboring CTU in the airspace, the right neighboring CTU in the airspace, and the neighboring CTU in the time domain are not divided, the current error CTU is taken as a whole. 1 error block.

The embodiment of the invention further provides a device for error concealment, the device comprising:

The configuration module is set to sequentially make each error block in the current error coding tree unit CTU Is the current error block;

An information acquiring module, configured to determine a spatial neighboring block of the current error block and a neighboring block on the airspace, where the left adjacent block of the airspace and the adjacent block on the airspace are adjacent to the error block on the airspace;

a judging module, configured to perform time domain and spatial domain correlation comparison when the left adjacent block of the spatial domain of the current error block and the adjacent block of the air domain are available and are not intra prediction blocks;

a hidden processing module, configured to: if the spatial correlation of the current error block is greater than a preset intensity threshold, performing spatial domain interpolation error concealment; if the spatial correlation of the current error block is not greater than a preset intensity threshold, proceeding Time domain error is hidden.

Optionally,

The hidden module is further configured to perform intra prediction error concealment on the current error block when both the left adjacent block of the spatial domain and the adjacent block on the spatial domain are available and both are intra prediction blocks.

Optionally,

The determining module is configured to perform time domain and spatial domain correlation when the left adjacent block of the spatial error block and the adjacent block on the air domain are available and are not intra prediction blocks. :

When SADS<(SADT1+SADT2)/3, the spatial domain correlation of the current error block is strong;

Where m is the size of the current error block, i, j is the pixel coordinate of the current error block, n is the image number, and P(n, 0, 0) is the pixel value of the upper left corner of the current error block.

Optionally,

The hidden processing module is configured to perform spatial interpolation error concealment by:

Calculating a predicted pixel P′ of the current error block according to a spatial neighboring block of the current error block (i,j);

Where i, j is the pixel coordinate of the current error block; P'(0, 0) is the pixel value of the upper left corner of the current error block.

Optionally,

The hidden processing module is configured to implement time domain error concealment by:

Obtaining a motion vector MV in the left adjacent block of the current error block, the adjacent block on the air domain, the lower left adjacent block in the spatial domain, and the upper right adjacent block in the spatial domain;

When the difference between the same components of any two MVs is greater than a predetermined threshold, the current error block is divided into 4 error sub-blocks; when the difference between any two MV identical components is less than or equal to a predetermined threshold, the current The error block as a whole is an error sub-block;

Do the following for each error sub-block:

The MV and the zero motion MV available in the spatially left adjacent subblock of the erroneous subblock, the adjacent subblock on the spatial domain, the right adjacent subspace of the spatial domain, the adjacent subspace in the spatial domain, and the adjacent subblock in the time domain are used as the MV of the erroneous sub-block, each candidate recovery block of the erroneous sub-block is obtained according to each MV of the erroneous sub-block;

A candidate restoration block having the smallest boundary matching difference in each candidate restoration block is taken as the restored pixel block of the current error block.

Optionally,

The hidden processing module is configured to perform intra prediction error concealment on the current error block by:

Determining a prediction mode of the left adjacent block of the spatial domain and the adjacent block of the spatial domain as a candidate prediction mode of the current error block;

Obtaining each candidate recovery block of the current erroneous block according to each candidate prediction mode; using a candidate restoration block having the smallest boundary matching difference in each candidate recovery block as the restored pixel block of the current erroneous block.

Optionally, the device further includes:

a dividing module, configured to divide any one of a neighboring CTU, an adjacent CTU in the airspace, a left neighboring CTU in the airspace, a right neighboring CTU in the airspace, and a neighboring CTU in the time domain in the airspace of the current erroneous CTU, Dividing the current error CTU into 4 error blocks; when the current error CTU is in the airspace, the adjacent CTU, the airspace adjacent CTU, the airspace left adjacent CTU, the airspace right adjacent CTU, and the time domain neighboring CTU If the division is not performed, the current error CTU as a whole is taken as one error block.

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the above method.

The above scheme combines the coding characteristics of HEVC video, repairs the damaged image by using the correlation between the error block data and the spatial domain adjacent data, and has a good repair effect on the damaged HEVC video code stream.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

BRIEF abstract

1 is a flow chart of a method of error concealment in an embodiment of the present invention;

2 is a schematic diagram showing the position of adjacent blocks of an error block in the embodiment of the present invention;

3 is a block diagram showing a comparison of spatial and temporal correlations in an embodiment of the present invention;

4 is a schematic diagram of spatial domain interpolation in an embodiment of the present invention;

FIG. 5 is a schematic diagram of a pixel used for calculating a boundary matching difference in an embodiment of the present invention; FIG.

FIG. 6 is a schematic structural diagram of an apparatus for error concealment in an embodiment of the present invention.

Embodiments of the invention

Embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

The technical solution of the embodiment of the present invention combines the HEVC video coding feature to utilize the damaged image region. Correlation with decoded image information to repair damaged images to solve the problem that HEVC video is difficult to obtain satisfactory video quality.

The technical solutions of the embodiments of the present invention are further described below with reference to the accompanying drawings.

As shown in FIG. 1 , an embodiment of the present invention provides a method for error concealing, where the method includes:

Step S101: sequentially use each error block in the current error coding tree unit CTU as the current error block;

Step S103: determining a left adjacent block of the airspace of the current error block and a neighboring block on the airspace;

The left adjacent block of the airspace and the adjacent block on the airspace are adjacent to the error block on the airspace;

Step S105: when the spatial neighboring left neighboring block of the current error block and the adjacent blocks on the airspace are available and are not intra prediction blocks, go to step S107;

Step S107: performing time domain and airspace correlation comparison;

Step S109: If the spatial correlation of the current error block is greater than the preset intensity threshold, then go to step S110; if the spatial correlation of the current error block is not greater than the preset intensity threshold, then go to step S111;

Step S110: performing spatial domain interpolation error concealment; and proceeding to step S101 to determine the next current error block.

Step S111: Perform time domain error concealment; go to S101 to determine the next current error block.

It should be noted that the execution flow provided by this embodiment is the same for each current error block, and each error block in an error CTU sequentially performs the operation, that is, steps S101 to S111. After a current error block has finished error concealment processing, it continues to find the next error block as the current error and performs corresponding error concealment processing. When all the error blocks included in an error CTU complete the error concealment processing, the next error CTU is searched for the corresponding error concealment processing until all the error CTUs complete the error concealment processing.

Optionally, the method further includes:

Step S106: When the left adjacent block of the spatial domain and the adjacent block on the air domain are available and both are intra prediction blocks, intra prediction error concealment is performed on the current error block.

Optionally, the method further includes:

Step S100: when the adjacent CTU in the airspace of the current error CTU, the adjacent CTU in the air domain, the left adjacent CTU in the airspace, the right adjacent CTU in the airspace, and the adjacent CTU in the time domain are divided, The current error CTU is divided into 4 error blocks;

When the adjacent CTU in the airspace of the current erroneous CTU, the adjacent CTU in the airspace, the left neighboring CTU in the airspace, the right neighboring CTU in the airspace, and the neighboring CTU in the time domain are not divided, the current error CTU is taken as a whole. 1 error block.

It should be noted that step S107 and step S106 do not limit the sequence. In other embodiments, step S106 may also be performed first. Further, step S110 and step S111 do not limit the order.

The above steps are further explained and explained below:

As shown in FIG. 2, in the present embodiment, for an error block, assuming that the pixel coordinates of the upper left corner of the error block are (x, y) and the size is w pixels, the upper adjacent block of the error block is a pixel (x). , y-1) is the PU (prediction unit); the left adjacent block is the PU where the pixel (x-1, y) is located; the lower adjacent block is the PU where the pixel (x, y+w) is located; The right adjacent block is the PU where the pixel (x+w, y) is located; the lower left adjacent block is the PU where the pixel (x-1, y+w) is located; the upper right adjacent block is the pixel (x+w, y-1) The PU where it is located;

In this embodiment, the time domain neighboring block is a pixel (x+w/2-1, y+w/2-1) in the decoded image that is smaller than the current image POC and has the closest POC (Picture Order Count). ) The PU where it is located.

In steps S105 and S106, when a neighboring block satisfies the following two conditions at the same time, the neighboring block is considered to be usable; when a neighboring block cannot satisfy the following two conditions at the same time, the neighboring block is considered to be unavailable. use:

Condition 1. The adjacent block does not exceed the image boundary, that is, inside the image;

Condition 2, the adjacent block is correctly decoded, or has undergone error concealment processing. The error concealment processing referred to herein means that the error concealment method described in the embodiment of the present invention has been restored.

The comparison of the time domain and the airspace correlation in step S105 includes:

As shown in FIG. 3, the left neighboring block B(n, x-1, y) and the upper neighboring block B(n, x, y-1) with the current error block B(n, x, y) are found, Finding the time domain neighboring block B(n-1, x, y) of the current error block and the left neighboring block B(n-1, x-1, y) of the time domain neighboring block and the upper neighboring block B (n-1, x, y-1). Here, n represents the image number, x represents the horizontal coordinate of the pixel in the upper left corner of the block, and y represents the vertical coordinate of the pixel in the upper left corner of the block.

The searching for the time domain neighboring block of the current error block includes: because the POC of the time domain neighboring block of the current error block is smaller than the POC of the current image, optionally, selecting the time domain neighboring block of the current error block is located. The image sequence number is the image number of the previous frame, that is, n-1; the PU where the pixel coordinates (x, y) of the upper left corner of the current error block is located is the time domain neighboring block of the current error block.

Calculate SAD _S between adjacent blocks B(n,x-1,y) and B(n,x,y-1); then calculate B(n,x-1,y) and B(n-1, respectively , AD-1, y) of SAD _T1 , and B(n, x, y-1) and B(n-1, x, y-1) of SAD _T2 .

When SAD _{S is} less than (SAD _T1 +SAD _T2 )/3, the spatial correlation of the current error block is strong;

When SAD _{S is} greater than or equal to (SAD _T1 +SAD _T2 )/3, the time domain correlation of the current error block is strong;

among them:

Where m is the size of the current error block, i, j is the pixel coordinate of the current error block, n is the image number, and P(n, 0, 0) is the pixel value of the upper left corner of the current error block.

Performing spatial interpolation error concealment in step S109 includes:

As shown in FIG. 4, the predicted pixel P'(i, j) of the current error block is calculated according to the spatial neighboring block of the current error block. Alternatively, a column of pixels and a spatial domain of the left adjacent block boundary of the spatial domain may be utilized. A row of pixels adjacent to the block boundary is interpolated to obtain the predicted pixel of the current erroneous block.

Where i, j is the pixel coordinate of the current error block; P'(0, 0) represents the pixel value of the upper left corner of the current error block.

Performing intra prediction error concealing on the current error block in step S106 includes:

Determining a prediction mode of the left adjacent block of the spatial domain and the adjacent block of the spatial domain as a candidate prediction mode of the current error block;

Obtaining each candidate recovery block of the current erroneous block according to each candidate prediction mode; using a candidate restoration block having the smallest boundary matching difference in each candidate recovery block as the restored pixel block of the current erroneous block.

As shown in FIG. 5, when calculating the boundary matching difference of each candidate recovery block, it is calculated according to a circle of pixels on the inner boundary of the candidate recovery block and neighboring pixels in the adjacent block.

The boundary matching difference can be calculated by the following formula 1:

Equation 1: SAD _E = Σ _{0 ≤} i _{< m} | | P (i, j) - P (i, j - 1) | | + Σ _{i = 0, m - 1} | | P (i, j) - P (i-1,j)||

Where m is the size of the current error block; i, j is the pixel coordinate of the current error block; P(0, 0) is the pixel value of the upper left corner of the current error block.

Performing time domain error concealing in step S110 includes:

Obtaining a motion vector MV in the left adjacent block of the current error block, the adjacent block on the air domain, the lower left adjacent block in the spatial domain, and the upper right adjacent block in the spatial domain;

When the difference between the same components of any two MVs is greater than a predetermined threshold, the current error block is divided into 4 error sub-blocks; when the difference between any two MV identical components is less than or equal to a predetermined threshold, the current The error block as a whole is an error sub-block;

Do the following for each error sub-block:

The MV and the zero motion MV available in the spatially left adjacent subblock of the erroneous subblock, the adjacent subblock on the spatial domain, the right adjacent subspace of the spatial domain, the adjacent subspace in the spatial domain, and the adjacent subblock in the time domain are used as the Error An MV of the block, each candidate recovery block of the erroneous sub-block is obtained according to each MV of the erroneous sub-block;

A candidate recovery block having the smallest boundary matching difference in each candidate recovery block is used as a restored pixel block of the current error block, and the MV corresponding to the restored pixel block is used as the MV of the current error block.

Wherein, calculating the boundary matching difference can be calculated according to the method of the above formula 1. At the same time, in this step, it is required to determine whether the difference between the same components of any two MVs in each MV is greater than a predetermined threshold. For example, the predetermined threshold may be set to 4, and then the current error block is divided into 4 errors when the formula 2 is satisfied. Subblock

Equation 2 is |MV _i (j)-MV _i (jm)|>4.

Where i denotes the horizontal or vertical component of the MV; j denotes the number of available MVs, j = 1, 2, 3; m denotes the number difference of the two MVs, m ∈ [1, j].

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the above method.

As shown in FIG. 6, an embodiment of the present invention further provides a device for error concealment, where the device includes:

The configuration module 11 is configured to sequentially use each error block in the current error coding tree unit CTU as the current error block;

The information obtaining module 12 is configured to determine a spatial neighboring block and a neighboring block on the airspace of the current error block, where the left neighboring block and the adjacent block on the airspace are adjacent to the error block in the airspace;

The determining module 13 is configured to perform time domain and spatial domain correlation comparison when the spatial neighboring left neighboring block of the current error block and the neighboring block on the airspace are both available and are not intra prediction blocks;

The hiding processing module 14 is configured to perform spatial domain interpolation error concealment if the spatial domain correlation of the current error block is greater than a preset intensity threshold; if the spatial domain correlation of the current error block is not greater than a preset intensity threshold, then Time domain error concealment.

Optionally,

The hidden module 14 is further configured to perform intra prediction error concealment on the current error block when both the left adjacent block of the spatial domain and the adjacent block on the spatial domain are available and both are intra prediction blocks.

Optionally,

The judging module 13 is configured to perform time domain and airspace correlation when the spatial neighboring left neighboring block of the current erroneous block and the neighboring block in the airspace are both available and are not intra prediction blocks. Sex:

When SAD _S <(SAD _T1 +SAD _T2 )/3, the spatial correlation of the current error block is strong; wherein:

Where m is the size of the current error block, i, j is the pixel coordinate of the current error block, n is the image number, and P(n, 0, 0) is the pixel value of the upper left corner of the current error block.

Optionally,

The hidden processing module 14 is configured to perform spatial interpolation error concealment by:

Calculating a predicted pixel P'(i,j) of the current error block according to a spatial neighboring block of the current error block;

Where i, j is the pixel coordinate of the current error block; P'(0, 0) is the pixel value of the upper left corner of the current error block.

Optionally,

The hidden processing module 14 is configured to implement time domain error concealment by:

Obtaining the left adjacent block of the airspace of the current error block, the adjacent block on the airspace, and the lower left adjacent block of the airspace And a motion vector MV in the upper right adjacent block of the airspace;

When the difference between the same components of any two MVs is greater than a predetermined threshold, the current error block is divided into 4 error sub-blocks; when the difference between any two MV identical components is less than or equal to a predetermined threshold, the current The error block as a whole is an error sub-block;

Do the following for each error sub-block:

The MV and the zero motion MV available in the spatially left adjacent subblock of the erroneous subblock, the adjacent subblock on the spatial domain, the right adjacent subspace of the spatial domain, the adjacent subspace in the spatial domain, and the adjacent subblock in the time domain are used as the MV of the erroneous sub-block, each candidate recovery block of the erroneous sub-block is obtained according to each MV of the erroneous sub-block;

A candidate restoration block having the smallest boundary matching difference in each candidate restoration block is taken as the restored pixel block of the current error block.

Optionally,

The hiding processing module 14 is configured to perform intra prediction error concealment on the current error block by:

Determining a prediction mode of the left adjacent block of the spatial domain and the adjacent block of the spatial domain as a candidate prediction mode of the current error block;

Obtaining each candidate recovery block of the current erroneous block according to each candidate prediction mode; using a candidate restoration block having the smallest boundary matching difference in each candidate recovery block as the restored pixel block of the current erroneous block.

Optionally, the device further includes:

The dividing module 15 is configured to divide any one of a neighboring CTU, an adjacent CTU in the airspace, a left neighboring CTU in the airspace, a right neighboring CTU in the airspace, and a neighboring CTU in the time domain in the airspace of the current faulty CTU, And dividing the current error CTU into 4 error blocks; adjacent CTUs in the airspace of the current error CTU, adjacent CTUs in the air domain, left adjacent CTUs in the airspace, right adjacent CTUs in the airspace, and time-domain neighboring CTUs If the division is not performed, the current error CTU as a whole is regarded as one error block.

Each of the above is a further detailed description of the present invention in connection with the specific embodiments, and the specific implementation of the invention is not limited to the description.

One of ordinary skill in the art will appreciate that all or a portion of the steps of the above-described embodiments can be implemented using a computer program flow, which can be stored in a computer readable storage medium, such as on a corresponding hardware platform (eg, The system, device, device, device, etc. are executed, and when executed, include one or a combination of the steps of the method embodiments.

Alternatively, all or part of the steps of the above embodiments may also be implemented by using an integrated circuit. These steps may be separately fabricated into individual integrated circuit modules, or multiple modules or steps may be fabricated into a single integrated circuit module. achieve.

The devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.

When each device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium. The above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Industrial applicability

The above technical solution has a good repair effect on the damaged HEVC video code stream.

Claims (15)

A method of error concealment, the method comprising: Each error block in the current error coding tree unit CTU is sequentially taken as the current error block; Determining a left-space neighboring block of the current error block and a neighboring block on the air domain, wherein the left adjacent block of the spatial domain and the adjacent block on the airspace are adjacent to the error block on the airspace; When the left adjacent block of the spatial domain of the current error block and the adjacent block on the air domain are available and are not intra prediction blocks, time domain and spatial domain correlation are compared; If the spatial domain correlation of the current error block is greater than the preset intensity threshold, the spatial domain interpolation error concealment is performed; if the spatial domain correlation of the current error block is not greater than the preset intensity threshold, the time domain error concealment is performed. The method of claim 1 further comprising: When the left adjacent block of the spatial domain and the adjacent block on the air domain are available and both are intra prediction blocks, intra prediction error concealment is performed on the current error block. The method of claim 1 wherein The comparison of the time domain and the airspace correlation includes: When SAD _{S is} less than (SAD _T1 +SAD _T2 )/3, the spatial correlation of the current error block is strong; When SAD _{S is} greater than or equal to (SAD _T1 +SAD _T2 )/3, the time domain correlation of the current error block is strong; among them:

Where m is the size of the current error block, i, j is the pixel coordinate of the current error block, n is the image number, and P(n, 0, 0) is the pixel value of the upper left corner of the current error block. The method of claim 3, wherein The performing spatial domain interpolation error concealment includes: Calculating a predicted pixel P'(i,j) of the current error block according to a spatial neighboring block of the current error block;

Where i, j is the pixel coordinate of the current error block; P'(0, 0) is the pixel value of the upper left corner of the current error block. The method of claim 3, wherein The performing time domain error concealment includes: Obtaining a motion vector MV in the left adjacent block of the current error block, the adjacent block on the air domain, the lower left adjacent block in the spatial domain, and the upper right adjacent block in the spatial domain; When the difference between the same components of any two MVs is greater than a predetermined threshold, the current error block is divided into 4 error sub-blocks; when the difference between any two MV identical components is less than or equal to a predetermined threshold, the current The error block as a whole is an error sub-block; Do the following for each error sub-block: The MV and the zero motion MV available in the spatially left adjacent subblock of the erroneous subblock, the adjacent subblock on the spatial domain, the right adjacent subspace of the spatial domain, the adjacent subspace in the spatial domain, and the adjacent subblock in the time domain are used as the MV of the erroneous sub-block, each candidate recovery block of the erroneous sub-block is obtained according to each MV of the erroneous sub-block; A candidate restoration block having the smallest boundary matching difference in each candidate restoration block is taken as the restored pixel block of the current error block. The method of claim 2, wherein The performing intra prediction error concealment on the current error block includes: Determining a prediction mode of the left adjacent block of the spatial domain and the adjacent block of the spatial domain as a candidate prediction mode of the current error block; Acquiring each candidate recovery block of the current error block according to each candidate prediction mode; using a candidate restoration block having the smallest boundary matching difference in each candidate recovery block as a recovery pixel of the current error block Piece. The method of any one of claims 1 to 6, the method further comprising: And each of the current error coding tree unit CTUs is sequentially used as the current error block, when the current error CTU is in the airspace, the adjacent CTU, the airspace adjacent CTU, the airspace left adjacent CTU, and the airspace right phase If the neighbor CTU and the time domain neighboring CTU are divided, the current error CTU is divided into four error blocks; When the adjacent CTU in the airspace of the current erroneous CTU, the adjacent CTU in the airspace, the left neighboring CTU in the airspace, the right neighboring CTU in the airspace, and the neighboring CTU in the time domain are not divided, the current error CTU is taken as a whole. 1 error block. A device for error concealment, the device comprising: a configuration module, configured to sequentially use each error block in the current error coding tree unit CTU as the current error block; An information acquiring module, configured to determine a spatial neighboring block of the current error block and a neighboring block on the airspace, where the left adjacent block of the airspace and the adjacent block on the airspace are adjacent to the error block on the airspace; a judging module, configured to perform time domain and spatial domain correlation comparison when the left adjacent block of the spatial domain of the current error block and the adjacent block of the air domain are available and are not intra prediction blocks; a hidden processing module, configured to: if the spatial correlation of the current error block is greater than a preset intensity threshold, performing spatial domain interpolation error concealment; if the spatial correlation of the current error block is not greater than a preset intensity threshold, proceeding Time domain error is hidden. The device of claim 8 The hidden module is further configured to perform intra prediction error concealing on the current error block when both the left adjacent block of the spatial domain and the adjacent block on the spatial domain are available and both are intra prediction blocks. The device of claim 8 wherein The determining module is configured to perform time domain and spatial domain correlation when the left adjacent block of the spatial error block and the adjacent block on the air domain are available and are not intra prediction blocks. : When SAD _S <(SAD _T1 +SAD _T2 )/3, the spatial correlation of the current error block is strong; among them:

Where m is the size of the current error block, i, j is the pixel coordinate of the current error block, n is the image number, and P(n, 0, 0) is the pixel value of the upper left corner of the current error block. The device of claim 10, wherein The hidden processing module is configured to perform spatial interpolation error concealment by: Calculating a predicted pixel P'(i,j) of the current error block according to a spatial neighboring block of the current error block;

Where i, j is the pixel coordinate of the current error block; P'(0, 0) is the pixel value of the upper left corner of the current error block. The device of claim 10, wherein The hidden processing module is configured to implement time domain error concealment by: Obtaining a motion vector MV in the left adjacent block of the current error block, the adjacent block on the air domain, the lower left adjacent block in the spatial domain, and the upper right adjacent block in the spatial domain; When the difference between the same components of any two MVs is greater than a predetermined threshold, the current error block is divided into 4 error sub-blocks; when the difference between any two MV identical components is less than or equal to a predetermined threshold, the current The error block as a whole is an error sub-block; Do the following for each error sub-block: The MV and the zero motion MV available in the spatially left adjacent subblock of the erroneous subblock, the adjacent subblock on the spatial domain, the right adjacent subspace of the spatial domain, the adjacent subspace in the spatial domain, and the adjacent subblock in the time domain are used as the MV of the erroneous sub-block, each candidate recovery block of the erroneous sub-block is obtained according to each MV of the erroneous sub-block; A candidate restoration block having the smallest boundary matching difference in each candidate restoration block is taken as the restored pixel block of the current error block. The device according to claim 9, wherein The hidden processing module is configured to perform intra prediction error concealment on the current error block by: Determining a prediction mode of the left adjacent block of the spatial domain and the adjacent block of the spatial domain as a candidate prediction mode of the current error block; Obtaining each candidate recovery block of the current erroneous block according to each candidate prediction mode; using a candidate restoration block having the smallest boundary matching difference in each candidate recovery block as the restored pixel block of the current erroneous block. The apparatus of any one of claims 8 to 13, the apparatus further comprising: a dividing module, configured to divide any one of a neighboring CTU, an adjacent CTU in the airspace, a left neighboring CTU in the airspace, a right neighboring CTU in the airspace, and a neighboring CTU in the time domain in the airspace of the current erroneous CTU, Dividing the current error CTU into 4 error blocks; when the current error CTU is in the airspace, the adjacent CTU, the airspace adjacent CTU, the airspace left adjacent CTU, the airspace right adjacent CTU, and the time domain neighboring CTU If the division is not performed, the current error CTU as a whole is taken as one error block. A computer storage medium having stored therein computer executable instructions for performing the method of any one of claims 1-7.

Images