CN1306826C - Loop filter based on multistage parallel pipeline mode - Google Patents

Abstract

The invention relates to a loop filtering device based on a multi-stage parallel pipeline mode, which divides the process of calculating the boundary strength and boundary threshold of the entire macroblock and the process of filtering the boundary of the entire macroblock into two-stage pipeline operations; The filtering process sequentially filters the data boundaries of each block according to the block-level pipeline method; each row/column data of the block boundary that needs to be filtered is sequentially sent to a multi-stage pipeline vertical/horizontal filter for filtering, and the filtered row/column is obtained Data; while the data of each block of the current macroblock is filtered, the data that has been filtered is written to the external memory. The invention utilizes multi-stage parallel pipelines to complete the loop filtering of each macroblock boundary of the video image, improves the filtering speed, reduces the pressure of accessing the external memory, ensures the real-time performance of the image loop filtering operation in the process of image encoding and decoding, and controls The complexity of the entire hardware structure is reduced, which is beneficial to design and implementation.

Description

Loop filter device based on multi-stage parallel pipeline

technical field

The invention relates to a coding and decoding technology of digital images, in particular to a loop filter in a high-performance real-time video processor, belonging to the technical field of video coding and decoding.

Background technique

In the MPEG of the International Organization for Standardization and the h.26x series of image coding standards of the International Telecommunication Union, block-based motion estimation and discrete cosine variation are used. The problem caused by this coding method is that adjacent pixels in the data block There is a boundary effect between them, that is, a block effect. In order to alleviate the obvious block effect in the process of image encoding and decoding, many boundary filtering methods have been proposed, among which the latest international coding standard MPEG4-part 10/h.264 and the domestic coding standard AVS both use the deblocking loop The deblocking loop filter method significantly improves the subjective quality of the encoded image.

As shown in Figure 1, it is a flow chart of macroblock loop filtering. First, calculate the boundary strength and boundary threshold of each boundary in the macroblock to be loop filtered, and use the calculated boundary strength and threshold to determine the vertical boundary of the macroblock luminance data. Filter, then filter the horizontal boundary of the macroblock luminance data, and then filter the vertical boundary and horizontal boundary of the macroblock chrominance data in sequence, and finally complete the vertical and horizontal boundary filtered brightness and chroma data output.

As shown in FIG. 2 , it is a block vertical/horizontal filtering boundary map, and the boundaries to be filtered by a macroblock are vertical boundary 1 (vertical edge) and horizontal boundary 2 (horizontal edge).

On the one hand, this loop filtering method needs to perform filtering operations on the pixels on each boundary of the block data, which brings a large amount of calculation; on the other hand, in the process of filtering, irregular pixel values need to be access, so if a general hardware design structure is adopted, it will greatly affect the speed of the entire filtering, and cannot fulfill the requirements of real-time image encoding and decoding in practical applications.

The MPEG4-part 10/h.264 and AVS standards are the latest international/domestic video codec standards, and the proposed solutions to the complexity of the above-mentioned loop filter are very limited, and only a few academic papers address This problem was analyzed and discussed, and some implementation methods were proposed, but some implementation methods did not consider the loop filtering process, the reference frame storage process and the motion compensation process at the same time. Overall consideration is not conducive to the design and use in actual situations. Some methods propose an implementation based on instruction words, including instruction storage and instruction decoding processes, which are suitable for the design and implementation of software accelerators, but not suitable for hardware design structures. Therefore, when facing the real-time encoding and decoding processing of high-definition images in actual needs, especially at present, the processing speed of video images is getting higher and higher. These methods are often not practical because of the difficulty in hardware implementation or the high cost, and urgently need to be further improved. Improve.

Contents of the invention

The technical problem to be solved by the present invention is to provide a loop filter device based on a multi-stage parallel pipeline method, which uses a multi-stage parallel pipeline structure to complete the loop filter of each macroblock boundary of a video image, and ensures that the image loop in the image encoding and decoding process The real-time nature of the filtering operation reduces the pressure of accessing external memory, and controls the complexity of the entire hardware structure, which is beneficial to design and implementation.

The technical problem to be solved by the present invention is achieved through the following technical solutions:

A loop filtering device based on a multi-stage parallel pipeline mode, which includes a three-stage pipeline module: a macroblock boundary calculation module, a vertical/horizontal filter module and an output module; the three-stage pipeline modules run simultaneously in a pipeline order;

The macroblock boundary calculation module includes a filter boundary strength calculator, a filter boundary threshold calculator and a first-in-first-out data buffer; the filter boundary strength calculator and the filter boundary threshold calculator extract the required filter macros from the external memory respectively. Boundary data of the block, calculate the boundary strength and threshold value of each block in the whole macroblock, store in the first-in-first-out data buffer; the described first-in-first-out data buffer filters the boundary strength of the macroblock filter and the threshold are input to the vertical/horizontal filtering module;

Described vertical/horizontal filtering module comprises vertical/horizontal filter, aligner, intermediate data register, selector and loop filter controller; Described selector connects external memory, under the control of loop filter controller , the selector extracts the macroblock boundary data that needs to be filtered in the external memory, and stores them in the intermediate data buffer; the intermediate data buffer is connected with an arranger, which arranges the macroblock boundary data in row and column order and Input to the vertical/horizontal filter in turn; the vertical/horizontal filter performs vertical/horizontal boundary filtering on the row and column data from the arranger under the control of the loop filter controller, and the filtered result needs to be filtered again Write the data back to the intermediate data buffer, and write the filtered data to the output module;

The output module includes an output data buffer and an interface for writing external memory. The output data buffer stores the data that has been filtered from the vertical/horizontal filter, and transmits the data to the write external memory interface; the described write external memory interface will The filtered data is written to the external memory.

The whole device is divided into three-stage pipelines to complete the loop filtering operation: the first-stage pipeline calculates the boundary strength and boundary threshold of each filter boundary of the macroblock to be filtered; the second-stage pipeline performs vertical filtering on each filter boundary of the macroblock to be filtered and horizontal filtering; the third-stage pipeline outputs the filtered macroblock data to the external memory. Through the three-stage pipeline operation, each process of the loop filter operation is allocated to different pipeline steps, which not only reduces the complexity of each pipeline operation, but also improves the parallelism of the entire loop filter operation.

In the entire loop filtering device, the horizontal border and vertical border filtering process in the second-stage pipeline operation is also designed as a multi-stage pipeline, so that the use of two parallel pipeline structures can fully meet the requirements of high-definition video in the process of real-time encoding and decoding. requirements for loop filter operation.

The vertical/horizontal filter includes a filter condition determiner module and a filter calculator module;

The filter condition determiner module includes a filter condition determiner and a data buffer, and the filter condition determiner receives the row and column data from the arranger, the boundary strength of the macroblock filter from the first-in-first-out data buffer, and Threshold, calculate the boundary filtering decision condition, determine the boundary pixel point to be filtered, and input the boundary pixel point to the lower pipeline through the data buffer for filtering;

The filter calculator module includes a filter calculator, which performs filter calculation on each pixel and outputs the filtered pixel.

The filter condition determiner is further divided into pipeline modules of two or more stages, and the pipeline modules of two or more stages run simultaneously according to the pipeline sequence.

The filter calculator is further divided into pipeline modules of two or more stages, and the pipeline modules of two or more stages run simultaneously according to the pipeline sequence.

The present invention completes the filtering of the macroblock boundary through the following steps:

Step 1. Under the control of the loop filter controller, the selector extracts the macroblock boundary data that needs to be filtered in the external memory, stores them in the intermediate data buffer, and then arranges the macroblock boundary data in row and column order through the arranger, and sequentially Input to the vertical/horizontal filter; at the same time, the filter boundary strength calculator and the filter boundary threshold calculator respectively extract the boundary data of the macroblock to be filtered from the external memory, calculate the boundary strength and threshold value of each block in the entire macroblock, and store them in the advanced The first-out data buffer, the first-in-first-out data buffer inputs the block boundary strength and threshold to the vertical/horizontal filter in the order of first-in-first-out.

Step 2: The filtering controller uses the block boundary filtering strength and boundary threshold value calculated by the previous operation to filter the entire macroblock boundary, and the loop filtering controller controls the vertical/horizontal filtering process of the entire macroblock boundary according to the block-level pipeline mode; In the filtered result, the data that needs to be filtered again is written back to the intermediate data buffer, and the filtered data is written to the output module.

Step 3, while completing the filtering operation of a block data, read out the row/column data of the next block boundary that needs to be filtered from the intermediate data buffer, and send it to the vertical/horizontal filter successively according to the above method for filtering operation;

Step 4, repeatedly performing the operations of step 2 and step 3 until the filtering of the current entire macroblock boundary data is completed;

Step 5. While filtering the data of each block of the current macroblock, the final data output by the vertical/horizontal filter is written into the output data cache, waiting for the write-to-external-storage interface to perform an operation to write to the external memory.

The present invention uses a multi-stage parallel pipeline structure to complete the loop filtering of each macroblock boundary of the video image, and at the same time takes into account the reference frame storage process and the actual interface problem of the loop filtering in the design of the codec, ensuring that the image in the process of image encoding and decoding is The real-time nature of the loop filter operation reduces the pressure of accessing external memory and controls the complexity of the entire hardware structure, which is beneficial to design and implementation.

Description of drawings

Fig. 1 macroblock loop filtering flowchart;

Figure 2 block vertical and horizontal filtering boundary map;

Fig. 3 is a structural representation of the present invention;

Fig. 4 is a schematic diagram of the horizontal/vertical boundary filter of the two-stage pipeline.

Detailed ways

Below in conjunction with accompanying drawing and specific embodiment the technical scheme of the present invention is further described:

First embodiment:

A kind of loop filtering device based on multi-stage parallel pipeline mode and the steps of realizing the loop filtering algorithm of AVS standard:

As shown in Figure 3, a kind of loop filter device based on multi-stage parallel pipeline mode, it comprises three-stage pipeline module: macroblock boundary calculation module, vertical/horizontal filter module and output module; Three-stage pipeline module simultaneously run;

Described macroblock boundary calculation module comprises filter boundary intensity calculator, filter boundary threshold value calculator and FIFO (first in first out) data register; Described filter boundary intensity calculator and filter boundary threshold value calculator extract respectively from external memory It is necessary to filter the boundary data of the macroblock, calculate the boundary strength and threshold value of each block in the whole macroblock, and store it in the FIFO data buffer; the described FIFO data buffer filters the boundary strength and The threshold is input to the vertical/horizontal filtering module;

Described vertical/horizontal filtering module comprises vertical/horizontal filter, aligner, intermediate data register, selector and loop filter controller; Described selector connects external memory, under the control of loop filter controller , the selector extracts the macroblock boundary data that needs to be filtered in the external memory, and stores them in the intermediate data buffer; the intermediate data buffer is connected with an arranger, which arranges the macroblock boundary data in row and column order and Input to the vertical/horizontal filter in turn; the vertical/horizontal filter performs vertical/horizontal boundary filtering on the row and column data from the arranger under the control of the loop filter controller, and the filtered result needs to be filtered again Write the data back to the intermediate data buffer, and write the filtered data to the output module.

The output module includes an output data buffer and a write external storage interface; the output data buffer stores the filtered data from the vertical/horizontal boundary filter, and transmits the data to the write external storage interface; The write external memory interface described above writes the filtered data to the external memory under the control of the loop filter controller.

The whole device is divided into three-stage pipelines to complete the loop filtering operation: the first-stage pipeline calculates the boundary strength and boundary threshold of each filter boundary of the macroblock to be filtered; the second-stage pipeline performs vertical filtering on each filter boundary of the macroblock to be filtered Filtering and horizontal filtering; the third-stage pipeline outputs the filtered macroblock data to the external memory. Through the three-stage pipeline operation, each process of the loop filter operation is allocated to different pipeline steps, which not only reduces the complexity of each pipeline operation, but also improves the parallelism of the entire loop filter operation.

The present invention realizes the loop filter algorithm of AVS standard through the following steps:

Step 1, calculating the boundary filtering strength and boundary threshold of 8x8 block data in the macroblock;

Step 2, simultaneously read in the data of the left side, the upper side and the current block to be filtered from the external memory, and put these data into the intermediate data buffer;

Step 3: After the block data, boundary filtering strength and boundary threshold of the current macroblock are ready, the data to be filtered and the data to be filtered of the left block pass through the intermediate data buffer, and are sent to the data arranger at the same time, and then sent to the vertical/horizontal in the filter;

Step 4, in the vertical/horizontal filter, the vertical boundary data is filtered according to the pipeline structure, and the filtered data is written back to the intermediate data buffer in turn;

Step 5. After completing the vertical boundary filtering of a block data of the current macroblock, write the data of the left macroblock that can be output into the output data buffer, notify the output interface to write the external memory operation, and start the next block of the current macroblock at the same time. Block boundary vertical filtering;

Step 6, repeatedly perform the operation of step 3, step 4 or step 5 until the vertical filtering of all block boundaries is completed;

Step 7, take out the data waiting for horizontal filtering in the upper side and the current macroblock boundary from the intermediate buffer, and send it to the data arranger;

Step 8, the block data of the current macroblock and the block data to be filtered on the upper side pass through the intermediate data buffer, and are sent to the data arranger at the same time, and then sent to the vertical/horizontal filter in sequence according to the column order;

Step 9: In the vertical/horizontal filter, the horizontal boundary data is filtered according to the pipeline structure, and the filtered data is written back to the intermediate data buffer in turn;

Step 10. After completing the horizontal boundary filtering of a block of the current macroblock, write the data that can be output in the upper macroblock into the output data buffer, notify the output interface to write the external memory operation, and start the next block of the current macroblock at the same time Boundary horizontal filtering;

Step 11, repeatedly perform the operation of step 8, step 9 or step 10 until the horizontal filtering of all block boundaries is completed;

Step 12: Complete the loop filtering of each macroblock in a frame according to the above method and output it to the external memory.

Embodiment two:

A kind of loop filtering device based on multi-stage parallel pipeline mode and the steps of realizing the loop filtering algorithm of MPEG4-part10/h.264 standard:

As shown in Figure 3, a kind of loop filter device based on multi-stage parallel pipeline mode, it comprises three-stage pipeline module: macroblock boundary calculation module, vertical/horizontal filter module and output module; Three-stage pipeline module simultaneously run;

Described macroblock boundary calculation module comprises filter boundary intensity calculator, filter boundary threshold value calculator and FIFO (first in first out) data register; Described filter boundary intensity calculator and filter boundary threshold value calculator extract respectively from external memory It is necessary to filter the boundary data of the macroblock, calculate the boundary strength and threshold value of each block in the whole macroblock, and store it in the FIFO data buffer; the described FIFO data buffer filters the boundary strength and The threshold is input to the vertical/horizontal filtering module;

Described vertical/horizontal filtering module comprises vertical/horizontal filter, aligner, intermediate data register, selector and loop filter controller; Described selector connects external memory, under the control of loop filter controller , the selector extracts the macroblock boundary data that needs to be filtered in the external memory, and stores them in the intermediate data buffer; the intermediate data buffer is connected with an arranger, which arranges the macroblock boundary data in row and column order and Input to the vertical/horizontal filter in turn; the vertical/horizontal filter performs vertical/horizontal boundary filtering on the row and column data from the arranger under the control of the loop filter controller, and the filtered result needs to be filtered again Write the data back to the intermediate data buffer, and write the filtered data to the output module.

The output module includes an output data buffer and a write external storage interface; the output data buffer stores the filtered data from the vertical/horizontal boundary filter, and transmits the data to the write external storage interface; The write external memory interface described above writes the filtered data to the external memory under the control of the loop filter controller.

The whole device is divided into three-stage pipelines to complete the loop filtering operation: the first-stage pipeline calculates the boundary strength and boundary threshold of each filter boundary of the macroblock to be filtered; the second-stage pipeline performs vertical filtering on each filter boundary of the macroblock to be filtered Filtering and horizontal filtering; the third-stage pipeline outputs the filtered macroblock data to the external memory. Through the three-stage pipeline operation, each process of the loop filter operation is allocated to different pipeline steps, which not only reduces the complexity of each pipeline operation, but also improves the parallelism of the entire loop filter operation.

As shown in Figure 4, in the entire loop filtering device, the horizontal boundary and vertical boundary filtering process in the second-stage pipeline operation is also designed as a multi-stage pipeline, so that the use of two parallel pipeline structures can fully meet the high-definition Requirements for loop filtering operation in the process of real-time video encoding and decoding.

Wherein, the vertical/horizontal filter includes a filter condition determiner module and a filter calculator module; the filter condition determiner module includes a filter condition determiner and a data buffer, and the filter condition determiner receives the information sent by the arranger The row and column data, the boundary strength and threshold value of the macroblock filter transmitted from the first-in-first-out data buffer, calculate the boundary filter judgment condition, determine the boundary pixel point to be filtered, and input the boundary pixel point through the data buffer to the lower pipeline for filtering; the filter calculator module includes a filter calculator, which performs filter calculation on each pixel, and outputs the pixel after filter calculation.

The filter condition determiner can be divided into pipeline modules of two or more stages, and the pipeline modules of two or more stages run simultaneously according to the pipeline sequence. The filter calculator can be divided into pipeline modules with two or more stages, and the pipeline modules with two or more stages run simultaneously according to the pipeline sequence.

The present invention realizes the loop filtering algorithm of MPEG4-part 10/h.264 standard by following steps:

Step 1, calculating the boundary filtering strength and the boundary threshold of the 4x4 block data in the macroblock; compared with the AVS standard, the number of blocks to be filtered is large, but the size of the required intermediate buffer will be reduced;

Step 2, simultaneously read in the data of the left side, the upper side and the current block to be filtered from the external memory, and put these data into the intermediate data buffer;

Step 3: After the block data, boundary filtering strength and boundary threshold of the current macroblock are ready, the data to be filtered and the data to be filtered of the left block pass through the intermediate data buffer, and are sent to the data arranger at the same time, and then sent to the vertical/horizontal in the filter;

Step 4, in the vertical/horizontal filter, the vertical boundary data is filtered according to the pipeline structure, and the filtered data is written back to the intermediate data buffer in turn;

Step 5. After completing the vertical boundary filtering of a block data of the current macroblock, write the data of the left macroblock that can be output into the output data buffer, notify the output interface to write the external memory operation, and start the next block of the current macroblock at the same time. Block boundary vertical filtering;

Step 6, repeatedly perform the operation of step 3, step 4 or step 5 until the vertical filtering of all block boundaries is completed;

Step 7, take out the data waiting for horizontal filtering in the upper side and the current macroblock boundary from the intermediate buffer, and send it to the data arranger;

Step 8, the block data of the current macroblock and the block data to be filtered on the upper side pass through the intermediate data buffer, and are sent to the data arranger at the same time, and then sent to the vertical/horizontal filter in sequence according to the column order;

Step 9: In the vertical/horizontal filter, the horizontal boundary data is filtered according to the pipeline structure, and the filtered data is written back to the intermediate data buffer in turn;

Step 10. After completing the horizontal boundary filtering of a block of the current macroblock, write the data that can be output in the upper macroblock into the output data buffer, notify the output interface to write the external memory operation, and start the next block of the current macroblock at the same time Boundary horizontal filtering;

Step 11, repeatedly perform the operation of step 8, step 9 or step 10 until the horizontal filtering of all block boundaries is completed;

Step 12: Complete the loop filtering of each macroblock in a frame according to the above method and output it to the external memory.

The present invention uses a multi-stage parallel pipeline structure to complete the loop filtering of each macroblock boundary of the video image, which ensures the real-time performance of the image loop filtering operation in the process of image encoding and decoding, reduces the pressure of accessing external memory, and controls the entire hardware structure complexity, which facilitates design and implementation.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention without limitation. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements of the technical solutions without departing from the spirit and scope of the technical solutions of the present invention shall be covered by the scope of the claims of the present invention.

Claims (8)

1, a kind of loop filter based on multistage parallel pipeline mode, it comprises macroblock boundaries computing module, vertical/horizontal filtration module and output module, it is characterized in that, described macroblock boundaries computing module and vertical/horizontal filtration module are parallel pipeline structure, move simultaneously by the flowing water order;

Described macroblock boundaries computing module extracts the data boundary that needs filtered macroblock from external memory storage, calculates the boundary intensity and the threshold value of each piece in the whole macro block, and is input to the vertical/horizontal filtration module;

Described vertical/horizontal filtration module comprises vertical/horizontal filter, arrangement machine, intermediate data buffer, selector and loop filtering controller; Described selector connects external memory storage, under the control of loop filtering controller, this selector extracts the macroblock boundaries data that need filtering in the external memory storage, deposit the intermediate data buffer in, by arrangement machine the macroblock boundaries data are pressed the ranks sequence arrangement again, and be input to the vertical/horizontal filter successively; Described vertical/horizontal filter utilizes the boundary intensity and the threshold value of the macro block filtering that the macroblock boundaries computing module transmits under the control of loop filtering controller, raw column data is carried out the vertical/horizontal boundary filtering, with needing the data of filtering once more to write back to the intermediate data buffer among the filtered result, the data of finishing filtering are write output module;

Described output module will be finished the data of filtering and write external memory storage.

2, the loop filter based on multistage parallel pipeline mode according to claim 1, it is characterized in that: described macroblock boundaries computing module comprises filtering boundary strength calculator and filtering boundary threshold calculations device, described filtering boundary strength calculator and filtering boundary threshold calculations device calculate the boundary intensity and the threshold value of each piece in the whole macro block respectively, and are input to the vertical/horizontal filter.

3, the loop filter based on multistage parallel pipeline mode according to claim 2, it is characterized in that: described macroblock boundaries computing module also comprises a data buffer, the boundary intensity and the threshold value of the macro block filtering that described data buffer reception and buffer memory filtering boundary strength calculator and filtering boundary threshold calculations device transmit, and be entered into the vertical/horizontal filter.

4, the loop filter based on multistage parallel pipeline mode according to claim 3 is characterized in that: described data buffer is the first in first out data buffer; Described first in first out data buffer is input to the vertical/horizontal filtration module by the order of first in first out with the boundary intensity and the threshold value of macro block filtering.

5, according to the arbitrary described loop filter based on multistage parallel pipeline mode of claim 1～4, it is characterized in that: described output module comprises the dateout buffer and writes external memory interface; The data of finishing filtering that the storage of described dateout buffer is transmitted from the vertical/horizontal filter, and with transfer of data to writing external memory interface; The described external memory interface of writing will be finished the data of filtering and write external memory storage under the control of loop filtering controller.

6, according to the arbitrary described loop filter based on multistage parallel pipeline mode of claim 1～4, it is characterized in that: described vertical/horizontal filter comprises filtering condition decision device module and filtering calculator modules;

Described filtering condition decision device module comprises filtering condition decision device and data buffer, the boundary intensity and the threshold value of the raw column data that described filtering condition decision device reception arrangement machine transmits, the macro block filtering that the first in first out data buffer transmits, calculate the boundary filtering judgment condition, decision will be carried out the edge pixel point of filtering operation, and edge pixel point is input to down level production line by data buffer will carry out filtering;

Described filtering calculator modules comprises the filtering calculator, and it carries out filtering calculating to each picture element, and with the picture element output after the filtering calculating.

7, the loop filter based on multistage parallel pipeline mode according to claim 6, it is characterized in that: described filtering condition decision device is divided into two-stage or the above pipeline module of two-stage again, and the above pipeline module of two-stage or two-stage moves simultaneously by the flowing water order.

8, the loop filter based on multistage parallel pipeline mode according to claim 6, it is characterized in that: described filtering calculator is divided into two-stage or the above pipeline module of two-stage again, and the above pipeline module of two-stage or two-stage moves simultaneously by the flowing water order.