WO2026000381A1 - Image decoding method, image encoding method, encoder, decoder, and medium - Google Patents

Abstract

Embodiments of the present application relate to the technical field of image encoding and decoding. Disclosed are an image decoding method, an image encoding method, an encoder, a decoder, and a medium. The image encoding method and the image decoding method each comprise: determining an affine model of a current block on the basis of motion information of a reference image of the current block, and a reference point; obtaining a plurality of motion vectors on the basis of the affine model of the current block; and predicting the current block on the basis of the plurality of motion vectors. By means of the solution provided in the embodiments of the present application, the image encoding and decoding efficiency and performance can be improved.

Description

Image decoding, encoding methods, codecs and media Technical Field

This application relates to the field of image encoding and decoding technology, and in particular to an image decoding and encoding method, an encoder/decoder, and a medium.

Background Technology

During inter-frame prediction, the decoder can determine the motion information of sub-blocks in the current block based on the affine model of the current block, and then make predictions about the current block based on this motion information.

In related technologies, affine models are constructed based on the motion information of reconstructed blocks adjacent to the current block in the current image.

Summary of the Invention

This application provides an image decoding and encoding method, a codec, and a medium. The technical solution is as follows:

On one hand, embodiments of this application provide an image decoding method, the method comprising:

Based on the motion information of the reference image of the current block and the reference point, the affine model of the current block is determined;

Multiple motion vectors are obtained based on the affine model of the current block;

The current block is predicted based on the multiple motion vectors.

On the other hand, embodiments of this application provide an image encoding method, the method comprising:

Based on the motion information of the reference image of the current block and the reference point, the affine model of the current block is determined;

Multiple motion vectors are obtained based on the affine model of the current block;

The current block is predicted based on the multiple motion vectors.

On the other hand, embodiments of this application provide a decoding device, the device comprising:

The decoding unit is used to determine the affine model of the current block based on the motion information of the reference image of the current block and the reference point;

The decoding unit is used to obtain multiple motion vectors based on the affine model of the current block;

The decoding unit is used to predict the current block based on the plurality of motion vectors.

On the other hand, embodiments of this application provide an encoding device, the device comprising:

An encoding unit is used to determine the affine model of the current block based on motion information of a reference image of the current block and a reference point;

The encoding unit is used to obtain multiple motion vectors based on the affine model of the current block;

The encoding unit is used to predict the current block based on the plurality of motion vectors.

On the other hand, embodiments of this application provide a decoder, the decoder including a memory and a processor, the memory being used to store a computer program running on the processor; the processor being used to execute the image decoding method as described above when running the computer program.

On the other hand, embodiments of this application provide an encoder, the encoder including a memory and a processor, the memory for storing a computer program running on the processor; the processor for executing the image encoding method as described above when running the computer program.

On the other hand, embodiments of this application provide a non-volatile computer-readable storage medium for storing a bitstream, the bitstream being generated by an image encoding method using an encoder, or the bitstream being decoded by an image decoding method using a decoder, wherein the image encoding method includes the image encoding method described above, and the image decoding method includes the image decoding method described above.

On the other hand, embodiments of this application provide a computer program product, the computer program product including computer instructions, the computer instructions being stored in a computer-readable storage medium, a processor obtaining the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to implement the image decoding method as described above, or the image encoding method as described above.

In this embodiment, since the reference image of the current image may contain motion information that is the same as or similar to that of the current block, the affine model of the current block is determined based on the motion information of the reference image and the reference point. This can provide the codec with more usable affine models, thereby enabling the encoder to determine a more suitable affine model for the current block and use the affine model to predict the current block, which helps to improve the quality and efficiency of image encoding and decoding.

Attached Figure Description

Figure 1 is a schematic diagram illustrating the encoding process at the encoding end in an exemplary embodiment of this application;

Figure 2 is a schematic diagram illustrating the decoding process at the decoding end in an exemplary embodiment of this application;

Figure 3 is a flowchart illustrating an exemplary embodiment of the image decoding method of this application;

Figure 4 is a schematic diagram of control points illustrating an exemplary embodiment of this application;

Figure 5 is a flowchart illustrating the affine model determination process in an exemplary embodiment of this application;

Figure 6 is a schematic diagram of a virtual control point shown in an exemplary embodiment of this application;

Figure 7 is a schematic diagram illustrating the process of determining the motion vector of a reference point in an exemplary embodiment of this application;

Figure 8 is a schematic diagram illustrating the process of determining the motion vector of a reference point in another exemplary embodiment of this application;

Figure 9 is a schematic diagram illustrating offset processing based on a second offset in an exemplary embodiment of this application;

Figure 10 is a schematic diagram illustrating an implementation of the scaling process in an exemplary embodiment of this application;

Figure 11 is a flowchart illustrating the affine model determination process in another exemplary embodiment of this application;

Figure 12 is a flowchart illustrating an exemplary embodiment of the image encoding method of this application;

Figure 13 is a flowchart illustrating the affine model index encoding process in an exemplary embodiment of this application;

Figure 14 is a flowchart illustrating the affine model determination process in another exemplary embodiment of this application;

Figure 15 shows a structural block diagram of an image decoding apparatus provided in an exemplary embodiment of this application;

Figure 16 shows a structural block diagram of an image encoding apparatus provided in an exemplary embodiment of this application;

Figure 17 shows a structural block diagram of a decoder provided in an exemplary embodiment of this application;

Figure 18 shows a structural block diagram of an encoder provided in an exemplary embodiment of this application.

Detailed Implementation

To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

In this article, "multiple" refers to two or more. "And/or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and/or B can represent: A alone, A and B simultaneously, or B alone. The character "/" generally indicates that the preceding and following related objects have an "or" relationship.

Currently, most common video codec standards employ a block-based hybrid coding framework. A single frame in a video is divided into squares of equal size (e.g., 256×256, 128×128, 64×64) called Largest Coding Units (LCUs) or Coding Tree Units (CTUs). These LCUs or CTUs can be further subdivided into rectangular Coding Units (CUs) according to rules. Coding units may also be further divided into Prediction Units (PUs), Transform Units (TUs), etc.

The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-prediction and inter-prediction. Inter-prediction includes motion estimation and motion compensation. Because there is a strong correlation between adjacent pixels in the same frame, intra-prediction can be used in video coding and decoding techniques to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, inter-prediction can be used in video coding and decoding techniques to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

At the encoding end, as shown in Figure 1, the encoder first divides the image into multiple coding blocks. Then, it uses intra-frame or inter-frame prediction (including motion estimation and motion compensation) algorithms to generate a prediction block for the current block. The encoder then subtracts the prediction block from the original block to obtain a residual block. This residual block is then transformed and quantized to obtain quantization coefficients. Finally, entropy coding is used to encode these quantization coefficients into the bitstream. Furthermore, the encoder reconstructs the image based on the inverse quantization/inverse transform results and the prediction block to obtain a reconstructed block. Loop filtering is then applied to the reconstructed block to compensate for distortion information and provide a better reference for subsequent encoding.

At the decoding end, as shown in Figure 2, the decoder uses intra-frame or inter-frame prediction algorithms to predict the prediction block of the current block. Simultaneously, it performs entropy decoding on the bitstream to obtain quantization coefficients, and then performs inverse quantization and inverse transform on these coefficients to obtain residual blocks. The prediction block and residual blocks are then added together to obtain the reconstructed block. For the reconstructed block, the decoder further performs loop filtering, and finally generates the decoded image based on the loop-filtered reconstructed block.

In inter-frame prediction, motion information is used to represent "motion." Basic motion information includes information about the reference picture and information about the motion vector (MV). Based on this motion information, the encoder and decoder can perform unidirectional or bidirectional inter-frame prediction for the current block.

Since motion in the real world includes not only translation but also other forms such as shrinking, enlarging, and rotation, a motion compensation scheme using affines is proposed in order to achieve more precise predictions.

Compared to uniform motion compensation for the current block, affine motion compensation only requires a small number of control points to derive the motion vectors of each sub-block (or even pixel) in the current block, thereby enabling motion compensation based on the motion vectors of each sub-block (or even pixel).

The encoder can construct multiple candidate affine models, select the affine model with the best prediction quality, and write the model index of the selected affine model into the bitstream. The decoder also constructs multiple candidate affine models and determines the model index of the affine model selected by the encoder by parsing the bitstream, then uses the same affine model as the encoder for motion compensation. In related technologies, the encoder and decoder construct affine models based on the motion information of the reconstructed blocks adjacent to the current block in the current image.

However, the motion information of the current block and its neighboring reconstructed blocks may differ significantly. Therefore, building an affine model based solely on the motion information of neighboring reconstructed blocks may result in poor prediction performance, which in turn affects the encoding and decoding quality.

Considering that the reference image of the current image may contain motion information that is the same or similar to that of the current block, in this embodiment of the application, the affine model of the current block is determined based on the motion information of the reference image and the reference point of the current block. This can provide the codec with more available affine models, thereby enabling the encoder to determine an affine model that is more suitable for the current block, which helps to improve the quality and efficiency of image encoding and decoding.

Please refer to Figure 3, which illustrates a flowchart of an image decoding method provided in an exemplary embodiment of this application. The method is applied to a decoder and may include the following steps:

Step 301: Determine the affine model of the current block based on the motion information of the reference image of the current block and the reference point.

In some embodiments, the decoder determines the reference image of the current block in the current image by parsing the bitstream. The bitstream may contain a reference image index of the reference image of the current block, and the decoder may retrieve the reference image indicated by that reference image index from a reference image list.

Optionally, the reference image may be a forward reference image determined from a list of forward-predicted reference images, or a forward reference image and a backward reference image determined from a list of bidirectional-predicted reference images.

In this embodiment, the reference image of the current block stores its own motion information during prediction. This motion information is the motion information of the reference image referenced when a certain block on the reference image is predicted. This motion information may include motion vectors.

In some embodiments, the decoder determines a reference point from the current image and obtains the reference point coordinates and the reference point motion vector, which are used together with the motion information of the reference image to determine the affine model of the current block.

Regarding the representation of the affine model, in one possible design, the affine model can be represented using at least two control points. For example, the affine model can be represented using a 6-parameter model (CPMV0, CPMV1, CPMV2) or a 4-parameter model (CPMV0, CPMV1). Here, CPMVn represents the motion vector of the nth control point; that is, the affine model can be represented using the motion vectors of 3 or 2 control points.

In another possible design, the affine model can be represented as {reference point information + a, b, c, d}, where the reference point information includes the reference point coordinates and the reference point motion vector, and abcd is the rate of change of the motion vector as a function of the coordinates, which is determined based on the motion information of the reference image.

In other possible designs, the affine model can take the form of a combination of the two representations mentioned above.

Step 302: Obtain multiple motion vectors based on the affine model of the current block.

In one possible implementation, the decoder determines the affine model actually used by the current block (consistent with the affine model used at the encoder) from multiple candidate affine models (including affine models built based on a reference image and affine models built based on the current image), thereby using the affine model to obtain multiple motion vectors.

Optionally, multiple motion vectors can be motion vectors of different sub-blocks in the current block, or motion vectors of multiple coordinate positions in the current block.

For example, when the current block is divided into 4×4 sub-blocks, the decoder can use an affine model to determine the motion vectors of each of the 16 sub-blocks. Alternatively, the decoder can use an affine model to determine the motion vector of the sample at each coordinate position in the current block.

Step 303: Predict the current block based on multiple motion vectors.

In one possible implementation, the decoder determines a reference sub-block from a reference image based on the motion vector of the sub-block in the current block, and then determines the predicted value of the sub-block based on the reconstructed value of the reference sub-block.

In another possible implementation, the decoder determines the reference coordinate position corresponding to the current coordinate position from the reference image based on the motion vector of the current coordinate position in the current block, and then determines the predicted value of the current coordinate position based on the reconstructed value of the reference coordinate position.

Optionally, the decoder determines the reference sub-block (reference coordinate position) corresponding to the current sub-block (current coordinate position) from the reference image based on the current sub-block's position (or current coordinate position) and motion vector, and determines the reconstructed sub-block (or reconstructed value of the reference coordinate position) of the reference sub-block as the predicted sub-block (or sample predicted value) of the current sub-block (or reference coordinate position).

Furthermore, based on each predicted sub-block (or sample predicted value) and the difference obtained from parsing the bitstream, the decoder determines the reconstructed sub-block (or sample reconstructed value) of each sub-block (or coordinate position), and then generates the reconstructed block of the current block based on each reconstructed sub-block (or sample reconstructed value), thus completing the decoding of the current block.

In summary, in the embodiments of this application, since the reference image of the current image may contain motion information that is the same as or similar to that of the current block, the affine model of the current block is determined based on the motion information of the reference image and the reference point. This can provide the codec with more usable affine models, thereby enabling the encoder to determine a more suitable affine model for the current block and use the affine model to predict the current block, which helps to improve the quality and efficiency of image encoding and decoding.

In one possible implementation, in affine mode, both the encoder and decoder construct a candidate list containing multiple candidate affine models and determine the affine model used for affine processing of the current block. Step 302 above, the process of determining the motion vector based on the affine model, may include the following sub-steps:

Sub-step 1: Add the affine model of the current block to the candidate list.

In some embodiments, the candidate list is a subblock merge candidate list, which contains multiple candidate affine models. In this embodiment, the candidate list includes affine models constructed based on the motion vectors of neighboring blocks of the current block in the current image, and affine models constructed based on motion information and reference points of a reference image.

It should be noted that the encoding and decoding ends use the same method to construct the candidate list to ensure the consistency of the affine model ultimately adopted by the encoding and decoding ends.

Sub-step 2: Parse the bitstream and determine the affine model index.

During the encoding process, the encoder encodes the affine model index of the affine model used by the current block from the candidate list into the bitstream. Correspondingly, the decoder parses the bitstream to obtain the affine model index corresponding to the current block.

Sub-step 3: Based on the affine model indicated by the affine model index in the candidate list, obtain multiple motion vectors.

Each candidate affine model in the candidate list has its own affine model index. The decoder determines the affine model indicated by the index from the candidate list based on the parsed affine model index, and uses the affine model to obtain multiple motion vectors.

In some embodiments, the decoder determines an affine relation that characterizes the correspondence between coordinate positions and motion vectors at coordinate positions based on the affine model indicated by the affine model index, thereby determining multiple motion vectors based on the affine relation.

Among them, multiple motion vectors can be motion vectors corresponding to multiple sub-blocks in the current block, or motion vectors corresponding to multiple coordinate positions in the current block.

The representation of an affine model:

Representation 1: The affine model of the current block includes the reference point coordinates, the reference point motion vector, the first parameter, the second parameter, the third parameter, and the fourth parameter.

In some embodiments, for a 4-parameter model, the motion vector (x, y) in the current block can be determined using the following formula:

For the 6-parameter model, the motion vector (x, y) in the current block can be determined using the following formula:

in, This is the motion vector of the control point at the top left corner of the current block. This is the motion vector of the control point at the top right corner of the current block. This is the motion vector of the control point at the bottom left corner of the current block. The superscripts h and v represent the horizontal and vertical components of mv. W is the width of the current block, and H is the height of the current block.

Rewriting the above formula, we get:

By determining the values of a, b, c, and d, as well as the motion vector of the control point at the top left corner of the current block, the motion vector of each sub-block or each coordinate position in the current block can be determined.

For the 4-parameter model, For the 6-parameter model

Extending the control point from the top left corner to any point, the above formula can be further rewritten as:

Where (x _base , y _base ) are the coordinates of the reference point. and These are the horizontal and vertical components of the motion vector of the reference point, respectively.

As can be seen from the above formula, after determining the values of a, b, c, and d, as well as the reference point coordinates and reference point motion vector of a reference point, the motion vector of each sub-block or each coordinate position in the current block can be determined.

Therefore, the affine model of the current block can be represented by the reference point coordinates, the reference point motion vector, the first parameter, the second parameter, the third parameter, and the fourth parameter. The first, second, third, and fourth parameters correspond to a, b, c, and d in the above formula, respectively.

In some embodiments, the first parameter represents the rate of change of the horizontal component of the motion vector with respect to the x-axis coordinate, the second parameter represents the rate of change of the horizontal component of the motion vector with respect to the y-axis coordinate, the third parameter represents the rate of change of the vertical component of the motion vector with respect to the x-axis coordinate, and the fourth parameter represents the rate of change of the vertical component of the motion vector with respect to the y-axis coordinate.

In some embodiments, the first parameter, the second parameter, the third parameter, and the fourth parameter are represented by the variables dHorX, dHorY, dVerX, and dVerY, respectively.

Representation 2: The affine model of the current block includes control point motion vectors of at least two control points.

In some embodiments, when there are two control points, as shown in Figure 4, these two control points can be the upper left corner control point 41 and the upper right corner control point 42 of the current block, represented by the affine model as {CPMV0, CPMV1}; when there are three control points, these three control points... Control points can be the top left control point 41, the top right control point 42, and the bottom left control point 43 of the current block, and the affine model is represented as {CPMV0, CPMV1, CPMV2}.

Representation 3: The affine model of the current block includes the reference point motion vector, the first parameter, the second parameter, the third parameter, and the fourth parameter.

In some embodiments, when the reference point is a fixed reference point, the affine model may not include the reference point coordinates. For example, the fixed reference point can be the top-left vertex of the current block (reference point coordinates are (0, 0)). Of course, the fixed reference point can be other vertices of the current block, or non-vertices, and this embodiment does not limit this.

It should be noted that the affine models of the above representations can be converted into each other. For example, you can first construct an affine model of representation 1, and then convert that affine model from representation 1 to representation 2. Or, you can first construct an affine model of representation 3, and then convert that affine model from representation 3 to representation 2.

The process of the affine model described above will be explained below using examples.

Please refer to Figure 5, which illustrates a flowchart of an affine model determination process provided in an exemplary embodiment of this application. This method is applied to a decoder and may include the following steps:

Step 501: Determine the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vector of the reference image of the current block.

Determination of reference image

Unlike determining the first, second, third, and fourth parameters (characterizing the rate of change of the motion vector with coordinates, hereinafter referred to as the rate of change parameter) of the current block based on the neighboring blocks of the current block in the current image, since the motion followed by the current block and its neighboring blocks in the current image may be significantly different, while there may be parts in the reference image that follow the same or similar motion as the current block, in this embodiment, the decoder determines the above-mentioned rate of change parameter based on the reference image of the current block.

Since motion information from a reference image is required in determining the above parameters, in some embodiments, the decoder determines a collocated picture for the current block, and then determines the rate of change parameter based on the motion vector of the collocated picture.

Among them, the co-position reference image is the reference image of the current image, and the co-position reference image has the motion vector when making prediction (the co-position reference image is reconstructed before the current image), that is, the motion vector of the co-position reference image relative to the reference image of the co-position reference image.

In some embodiments, the decoder parses the bitstream to determine the co-location reference image for the current block.

In some embodiments, the number of co-position reference images is at least one, and the co-position reference image is indicated by the encoder through the bitstream. For example, the bitstream contains a co-position reference image index, and the decoder determines the co-position reference image from the reference image list based on the co-position reference image index and obtains the motion information of the co-position reference image.

In some embodiments, when multiple co-reference images exist, the decoder determines at least one affine model for the current block, wherein different affine models are determined based on different co-reference images.

Optionally, the decoder applies this scheme to each of the multiple co-position reference images to determine the affine mode corresponding to each co-position reference image; or, the decoder applies this scheme to a portion of the multiple co-position reference images to determine the affine mode corresponding to the portion of the co-position reference images, wherein the portion of the co-position reference images can be indicated in the bitstream by the encoder.

For ease of explanation, the following embodiments use a single co-position reference image as an example.

Explanation of virtual control point locations

When determining the rate of change parameter based on the current image, at least two control points need to be determined. In this embodiment, when determining the rate of change based on a co-located reference image, it is also necessary to determine at least two points in the co-located reference image of the current block. These at least two points in the co-located reference image can be referred to as virtual control points (VCPs). For ease of representation in the following embodiments, virtual control points are used to refer to at least two points in the co-located reference image.

After identifying at least two points in the co-located reference image, the decoder determines the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vectors of the at least two points in the co-located reference image.

In one possible implementation, the decoder determines virtual control points in the co-located reference image in groups, where each group of virtual control points contains two or three virtual control points.

For each virtual control point group, the decoder determines a set of rate of change parameters based on the motion vectors of the virtual control points in that group. The rate of change parameters calculated for different virtual control point groups may differ; therefore, based on different virtual control point groups, the decoder can subsequently determine multiple affine models for the current block.

Optionally, when the virtual control point group contains 2 virtual control points, the 2 virtual control points are located at the upper left and upper right corners of the formed rectangle; when the virtual control point group contains 3 virtual control points, the 3 virtual control points are located at the upper left, upper right, and lower left corners of the formed rectangle, respectively.

Optionally, the rectangle formed by the virtual control points (which may be called a virtual block) may have the same or different size as the current block; the position of the rectangle formed by the virtual control points in the co-located reference image may have the same or different position as the current block in the current image.

In some embodiments, the virtual control points in the virtual control point group can be located in various directions of the co-position block in the co-position reference image, such as above, below, left, or right.

When constructing an affine model based on the motion information of the current image, motion information can be obtained from the adjacent blocks to the left and above the current block. Since motion information cannot be obtained from the right and bottom of the current block (because the blocks on the right and bottom have not yet been encoded or decoded), in one possible implementation, among the determined n point groups, there exists at least one point group that satisfies the following condition: the points contained in the point group are located in the lower right region of the current block.

Since the motion information of the region to the right and below the co-position block in the reference image can be used when determining the rate of change parameter based on the point set that meets the above conditions (this is the reference information used when constructing the affine model based on the current image), the affine model determined based on this point set can serve as a powerful supplement to the affine model constructed based on the current image.

In an illustrative example, as shown in Figure 6, the current block in the current image corresponds to the same block 62 in the same reference image 61, and other positions represent motion vectors used for merging in merge mode. Taking the determination that a virtual control point group contains 3 virtual control points as an example, the decoder can select virtual control points numbered {9, 8, 7}, {15, 13, 14}, {6 (middle position of the same block), 17, 18} to form a virtual control point group. Among them, the virtual control points in the virtual control point groups {9, 8, 7} and {15, 13, 14} are located at the upper left of the same block, and the virtual control points in the virtual control point groups {6 (middle position of the same block), 17, 18} are located at the lower right of the same block.

Description of virtual control point availability

To ensure that the rate of change parameter can be determined based on the virtual control points (avoiding the determined rate of change parameter being 0), at least two points in the co-located reference image are located in at least two regions of predetermined size.

When two virtual control points are determined, each virtual control point is located in a region of a predetermined size.

When three virtual control points are determined, the three virtual control points are located in three regions with predetermined dimensions, or two of the three virtual control points are located in the same region with predetermined dimensions, and the other virtual control point is located in another region with predetermined dimensions.

In some embodiments, the region with a predetermined size can be a minimum storage unit for motion information. The coordinate positions within the same minimum storage unit for motion information contain the same motion information.

For example, the smallest storage unit is 8×8, meaning that each 8×8 block stores a set of motion information, that is, each coordinate position in the 8×8 block has the same motion information. Of course, the smallest storage unit can use a larger or smaller size, such as 16×16 (lower implementation cost, lower motion vector accuracy), 4×4 (higher implementation cost, higher motion vector accuracy), or even 1×1. This embodiment does not limit this.

In one possible implementation, the decoder determines whether a virtual control point is located in the same motion information minimum storage unit based on the coordinates of the virtual control points in the virtual control point group and the size of the motion information minimum storage unit.

If the virtual control points in a virtual control point group are located in at least two motion information minimum storage units, then the virtual control point group is determined to be available, and the rate of change parameter is determined based on the virtual control point group; if the virtual control points in a virtual control point group are located in the same motion information minimum storage unit, then the virtual control point group is determined to be unavailable, and the rate of change parameter will not be determined based on the virtual control point group.

To illustrate, given that the minimum storage unit for motion information is 4×4, and the coordinates of point A are (x1, y1) and the coordinates of point B are (x2, y2), if floor(x1/4) = floor(x2/4) and floor(y1/4) = floor(y2/4), then the two points are located in the same minimum storage unit for motion information; if floor(x1/4) ≠ floor(x2/4), and/or floor(y1/4) ≠ floor(y2/4), then the two points are located in different minimum storage units for motion information.

Step 502: Determine the reference point coordinates and the reference point motion vector.

The reference point can be located inside the current block or outside the current block.

In some embodiments, when the reference point is located within the current block, the reference point is the top-left, top-right, or bottom-left vertex of the current block.

Specifically, if the height of the current block is H and the width is W, then if the reference point is the top left corner of the current block, the coordinates of the reference point are (0, 0); if the reference point is the top right corner of the current block, the coordinates of the reference point are (W, 0); if the reference point is the bottom left corner of the current block, the coordinates of the reference point are (0, H).

Regarding the method for determining the reference point motion vector, in one possible implementation, if the adjacent blocks of the current block support affine mode, the decoder determines the reference point motion vector based on the affine model of the adjacent blocks.

Optionally, if the current block supports the merge mode and the adjacent blocks support the affine mode, the decoder uses the coordinates of the reference point in the current block relative to the adjacent blocks to substitute the coordinates into the affine model of the adjacent blocks to obtain the reference motion vector of the reference point.

As illustrated in Figure 7, the adjacent blocks of the current block include A0, A1, B0, B1, and B2. The decoder can determine the reference point motion vector of the upper left reference point based on the affine model of the adjacent block B2; or, based on the affine model of the adjacent blocks B0 or B1, determine the reference point motion vector of the upper right reference point; or, based on the affine model of the adjacent blocks A0 or A1, determine the reference point motion vector of the lower left reference point.

In another possible implementation, the decoder can determine the reference point motion vector based on the available motion vectors in the adjacent blocks of the current block.

Optionally, the decoder can determine the reference point motion vector of the reference point in the current block by fitting based on at least two available motion vectors in the adjacent blocks of the current block.

As illustrated in Figure 8, the neighboring blocks of the current block include A0, A1, A2, B0, B1, B2, and B3. The decoder can determine the reference point motion vector of the top-left reference point based on the available motion vectors in neighboring blocks B2, B3, and A2; and based on the available motion vectors in neighboring blocks B0, B1, and A3... The motion vector used is used to determine the reference point motion vector of the upper right reference point; the reference point motion vector of the lower left reference point is determined based on the available motion vectors in adjacent blocks A0 and A1.

In some embodiments, the decoder constructs an affine model of the form {(x, y), MV, a, b, c, d}, where (x, y) are the coordinates of the reference point, MV is the motion vector of the reference point, and a, b, c, d are the first to fourth parameters mentioned above, respectively.

In some embodiments, when the reference point is a fixed reference point, the decoder only needs to determine the reference point motion vector of the reference point, thereby constructing an affine model of the form {MV, a, b, c, d}, where MV is the reference point motion vector, and a, b, c, and d are the first to fourth parameters mentioned above, which will not be elaborated here in this embodiment.

In this embodiment, when the decoder determines the rate of change parameter based on the motion vector of the virtual control point group in the co-located reference image, it determines the virtual control point group from the lower right region of the current block. This allows the calculation of the rate of change parameter to utilize the motion information of the lower right region, which was previously unavailable when constructing the affine model based on the current image. This enables the affine model constructed using this scheme to serve as a powerful supplement to existing affine models.

Furthermore, by detecting whether the virtual control point is located in at least two minimum storage units of motion information (i.e., regions of predetermined size), the decoder ensures that the subsequent motion vector calculation based on the virtual control point yields usable rate of change parameters, which helps improve the quality of the subsequently constructed affine model.

In the above embodiments, the process of determining multiple motion vectors using an affine model that includes reference point coordinates, reference point motion vectors, a first parameter, a second parameter, a third parameter, and a fourth parameter may include:

Step 1: Determine the affine relation based on the affine model of the current block. The affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

In some embodiments, the affine relation can be expressed as:

Where (x _base , y _base ) are the coordinates of the reference point. and These are the horizontal and vertical components of the motion vector of the reference point, respectively, and abcd are the first to fourth parameters, respectively.

Once the affine relation is determined, the decoder substitutes the coordinates into the affine relation to obtain the motion vector at that coordinate position.

Step 2: Based on affine relations, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block.

In some embodiments, if the current block is divided into several sub-blocks, for each coordinate position in the sub-block, the decoder determines the motion vector at each coordinate position based on the affine relation, and determines the average value of the motion vectors at each coordinate position in the same sub-block as the motion vector of that sub-block.

In some embodiments, without sub-block division, the decoder can also determine the motion vector at each coordinate position based on affine relation for any coordinate position in the current block.

Determining at least two points (i.e., virtual control points) in a reference image can be done in at least one of the following ways:

Method 1: Based on the vertex coordinates of the current block in the current image, determine at least two points in the corresponding reference image.

In some embodiments, when the image content changes slowly, the decoder can identify the vertex in the same position as the control point in the current block in the same reference image as the virtual control point.

In some embodiments, the decoder determines two virtual control points from the co-located reference image based on the vertex coordinates of the top-left vertex and the top-right vertex of the current block; or, it determines three virtual control points from the co-located reference image based on the vertex coordinates of the top-left vertex, the top-right vertex, and the bottom-left vertex of the current block.

In some embodiments, the decoder determines a first virtual control point based on the vertex coordinates of the top-left vertex of the current block, a second virtual control point based on the vertex coordinates of the top-left vertex and the width of the current block, and a third virtual control point based on the vertex coordinates of the top-left vertex and the height of the current block.

Method 2: Based on the vertex coordinates of the current block in the current image and the first offset, determine at least two points in the co-located reference image.

The first offset is used to characterize the coordinate offset between the (virtual control point) and the vertex of the current block. Optionally, the first offset includes an offset in the x-axis direction and an offset in the y-axis direction.

Optionally, the first offset is a coordinate offset relative to a specified vertex in the current block. For example, the specified vertex could be the top-left vertex, top-right vertex, bottom-left vertex, etc. of the current block.

In some embodiments, the decoder determines m virtual control points in the channel reference image based on the vertex coordinates of the top-left vertex in the current block and m first offsets.

Schematic representation: Given the top-left corner vertex coordinates (xCb, yCb) of the current block, the decoder determines the coordinates of the first virtual control point as (xCb+x1, yCb+y1) based on the first offset (x1, y1); the second virtual control point as (xCb+x2, yCb+y2) based on the first offset (x2, y2); and the third virtual control point as (xCb+x3, yCb+y3) based on the first offset (x3, y3). The first, second, and third virtual control points are the top-left, top-right, and bottom-right corners of the virtual block, respectively. The bottom left corner vertex.

Method 3: Based on the vertex coordinates of the current block in the current image and the second offset, determine at least two points in the co-located reference image.

The second offset is determined based on the motion vectors of the adjacent blocks of the current block.

For neighboring blocks of the current block, if the co-location reference image of the neighboring block is the same as that of the current block, then the motion vector of the neighboring block may be quite similar to that of the current block. Therefore, in order to select a more suitable virtual control point, in one possible implementation, the decoder determines the motion vector of the neighboring blocks of the current block. If the motion vector of the neighboring block points to the co-location reference image of the current block, the decoder determines a second offset based on the motion vector of the neighboring block.

The adjacent block is the block that was reconstructed before the current block, and the motion vector of the adjacent block is the difference between the coordinates of the adjacent block and the coordinates of the reference block in the co-located reference image.

Furthermore, the decoder offsets the vertex coordinates of the current block based on the second offset to obtain the coordinates of the virtual control points in the co-located reference image. Optionally, the decoder can offset the vertex coordinates of different vertices in the current block based on the second offset to obtain the coordinates of multiple virtual coordinate points.

As illustrated in Figure 9, the current block has neighboring blocks A0, A1, A2, B0, B1, B2, and B3. The motion vector of neighboring block A0 points to the corresponding reference image of the current block, and the motion vector is _mvA0 . If the coordinates of the top-left corner of the current block are (xCb, yCb), then the coordinates in the corresponding reference image are... The point is determined as a virtual coordinate point, where, These are the horizontal and vertical components, respectively.

Method 4: Based on the vertex coordinates of the current block in the current image, the first offset, and the second offset, determine at least two points in the co-located reference image.

In some embodiments, when there is a motion vector of an adjacent block pointing to a co-location reference image of the current block, the decoder uses a first offset and a second offset to offset the vertex coordinates of the current block to obtain the coordinates of the virtual control point in the co-location reference image. When there is no motion vector of an adjacent block pointing to a co-location reference image of the current block, the decoder uses the first offset to offset the vertex coordinates of the current block.

The process of determining the rate of change parameter

As shown in Figure 10, the motion vector of the co-position block (col_cu) on the co-position reference image (col_pic) is the vector between the co-position reference image and the reference image (col_ref) of the co-position block. For the current block (curr_CU), the required motion vector is the vector between the current image (curr_pic) and the reference image (curr_ref) of the current block.

Let td be the point of convergence (POC) distance between col_pic and col_ref, and tb be the POC distance between curr_pic and curr_ref. Assuming the motion in the same block remains constant over time, the scaling factor can be determined based on td and tb. Let the motion vector of the same block be (col_mv_x, col_mv_y), then the temporal motion vector prediction (tmvp_x, tmvp_y) can be derived as follows:

tmvp_x=col_mv_x*tb/td, tmvp_y=col_mv_y*tb/td

Based on the above derivation process, it can be seen that the current image can be obtained by scaling the motion information of the corresponding reference image to obtain the temporal motion information of the current image. Furthermore, based on the motion information of the corresponding reference image, after determining the rate of change parameter corresponding to the corresponding reference image, this rate of change parameter can be scaled to obtain the rate of change parameter corresponding to the current image.

In one possible implementation, the decoder determines the fifth, sixth, seventh, and eighth parameters based on the motion vectors of at least two points (i.e., virtual control points) in the co-located reference image, and then scales the fifth, sixth, seventh, and eighth parameters to obtain the first, second, third, and fourth parameters.

Among them, the fifth to eighth parameters are used to characterize the rate of change of the motion vector of the coordinate position in the virtual block (a block composed of virtual control points) as the coordinates change.

Regarding the method for determining the scaling ratio, in one possible implementation, the decoder determines a first image interval between the current image and its reference image, and a second image interval between co-located reference images and their reference images. Based on the ratio of the first and second image intervals, the fifth, sixth, seventh, and eighth parameters are scaled to obtain the first, second, third, and fourth parameters.

The first image interval is used to characterize the POC distance between the current image and its reference image, and the second image interval is used to characterize the POC distance between the co-located reference image and its reference image.

In some embodiments, a scaling ratio is determined based on a first image interval and a second image interval. A first parameter is obtained by scaling a fifth parameter based on the scaling ratio; a second parameter is obtained by scaling a sixth parameter based on the scaling ratio; a third parameter is obtained by scaling a seventh parameter based on the scaling ratio; and a fourth parameter is obtained by scaling an eighth parameter based on the scaling ratio.

For illustration, if the POC distance between col_pic and col_ref is td, the POC distance between curr_pic and curr_ref is tb, and the rate of change parameters corresponding to the virtual blocks in the co-position reference image are determined to be a’, b’, c’, and d’ based on the motion vector of the virtual control points in the co-position reference image, then the rate of change parameters corresponding to the current block in the current image are a＝a′*tb/td, b＝b′*tb/td, c＝c′*tb/td, and d＝d′*tb/td.

Please refer to Figure 11, which shows a flowchart of an affine model determination process provided by another exemplary embodiment of this application. This method is applied to... A decoder, the method may include the following steps:

Step 1101: Determine the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vector of the reference image of the current block.

The implementation method of this step can be referred to step 501, and will not be repeated here in this embodiment.

Step 1102: Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, determine the control point motion vectors of at least two control points.

Since the motion vector of any coordinate position in the current block can be determined based on the rate of change parameter, the reference point coordinates, and the reference point motion vector, the decoder can determine the control point motion vectors of at least two control points when it is necessary to obtain the affine model of representation form 2.

Optionally, at least two control points belong to the current block. For example, a first control point located at the top-left vertex of the current block, a second control point determined based on the coordinates of the top-left vertex and the width of the current block, and a third control point determined based on the coordinates of the top-left vertex and the height of the current block.

Optionally, the at least two control points must conform to the representation requirements of the affine model. For example, when using a 4-parameter affine model, the at least two control points include the top-left and top-right vertices of the current block; when using a 6-parameter affine model, the at least two control points include the top-left, top-right, and bottom-left vertices of the current block.

In one possible implementation, the decoder determines an affine relation based on the first parameter, the second parameter, the third parameter, the fourth parameter, the reference point coordinates, and the reference point motion vector. The affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

In some embodiments, the affine relation can be expressed as:

Where (x _base , y _base ) are the coordinates of the reference point. and , respectively, are the horizontal and vertical components of the motion vector of the reference point, and abcd are the first to fourth parameters, respectively.

In other embodiments, when the reference point is fixed, the decoder determines the affine relationship based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point motion vector.

After determining the affine relation, the decoder determines the control point motion vector of at least two control points based on the affine relation and the control point coordinates of at least two control points. That is, by substituting the coordinates of the control points into the affine relation, the control point motion vector of the control point can be obtained.

Schematic: When the height of the current block is H and the width is W, the decoder substitutes the control point coordinates (0, 0) of the top-left control point in the current block into the above affine relation to obtain the control point motion vector CPMV0 of the top-left control point; substitutes the control point coordinates (W, 0) of the top-right control point into the above affine relation to obtain the control point motion vector CPMV1 of the top-right control point; and substitutes the control point coordinates (0, H) of the bottom-left control point into the above affine relation to obtain the control point motion vector CPMV2 of the bottom-left control point.

In some embodiments, when a 4-parameter affine model is used, the decoder constructs an affine model of the form {CPMV0, CPMV1}; when a 6-parameter affine model is used, the decoder constructs an affine model of the form {CPMV0, CPMV1, CPMV2}.

In the above embodiments, an affine model is constructed using control point motion vectors based on at least two control points. The process of determining the motion vectors may include:

Step 1: Based on the affine model of the current block, determine the first parameter, the second parameter, the third parameter, and the fourth parameter.

Similar to the determination of the rate of change parameter based on control points in the above embodiments, the decoder determines the first parameter, the second parameter, the third parameter, and the fourth parameter based on the control point motion vector of at least two control points in the affine model.

Since the rate of change parameter has been determined during the construction of the affine model, in one possible implementation, the decoder directly uses the previously determined rate of change parameter.

Step 2: Based on the first parameter, the second parameter, the third parameter, and the fourth parameter, determine the affine relation. The affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

Since the affine model contains the motion vector of the control point at the top left corner of the current block, the decoder can determine the affine relationship based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the motion vector of the control point at the top left corner.

In some embodiments, the determined affine relations are as follows:

in, and The components of the motion vector of the upper left control point in the horizontal and vertical directions.

Step 3: Based on affine relations, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block.

In some embodiments, if the current block is divided into several sub-blocks, for each coordinate position in the sub-block, the decoder determines the motion vector at each coordinate position based on the affine relation, and determines the average value of the motion vectors at each coordinate position in the same sub-block as the motion vector of that sub-block.

In some embodiments, without sub-block partitioning, the decoder can also base its work on any coordinate position within the current block. Affine relations determine the motion vectors at each coordinate position.

The image encoding process is illustrated below using exemplary embodiments.

Please refer to Figure 12, which illustrates a flowchart of an image encoding method provided in an exemplary embodiment of this application. The method is applied to an encoder and may include the following steps:

Step 1201: Determine the affine model of the current block based on the motion information of the reference image of the current block and the reference point.

In some embodiments, the encoder determines a reference image for the current block in the current image. This reference image belongs to a reference picture list.

Optionally, the reference image may be a forward reference image determined from a list of forward-predicted reference images, or a forward reference image and a backward reference image determined from a list of bidirectional-predicted reference images.

In this embodiment, the reference image of the current block stores its own motion information during prediction. This motion information is the motion information of the reference image referenced when a certain block on the reference image is predicted. This motion information may include motion vectors.

In some embodiments, the encoder determines a reference point from the current image and obtains the reference point coordinates and the reference point motion vector, which are used together with the motion information of the reference image to determine the affine model of the current block.

Regarding the representation of the affine model, in one possible design, the affine model can be represented using at least two control points. For example, the affine model can be represented using a 6-parameter model (CPMV0, CPMV1, CPMV2) or a 4-parameter model (CPMV0, CPMV1). Here, CPMVn represents the motion vector of the nth control point; that is, the affine model can be represented using the motion vectors of 3 or 2 control points.

In another possible design, the affine model can be represented as {reference point information + a, b, c, d}. Here, the reference point information includes the reference point coordinates and the reference point motion vector, and abcd represents the rate of change of the motion vector as a function of the coordinates, determined based on the motion information from the reference image.

In other possible designs, the affine model can take the form of a combination of the two representations mentioned above.

Step 1202: Obtain multiple motion vectors based on the affine model of the current block.

In one possible implementation, the encoder constructs multiple candidate affine models (including an affine model constructed based on a reference image and an affine model constructed based on the current image). For each candidate affine model, the encoder uses the affine model to obtain multiple motion vectors.

Optionally, the multiple motion vectors of the current block can be motion vectors of different sub-blocks in the current block, or motion vectors of multiple coordinate positions in the current block.

For example, when the current block is divided into 4×4 sub-blocks, the encoder can use an affine model to determine the motion vectors of each of the 16 sub-blocks. Alternatively, the encoder can use an affine model to determine the motion vector of the sample at each coordinate position in the current block.

Step 1203: Predict the current block based on multiple motion vectors.

In one possible implementation, the encoder determines a reference sub-block from a reference image based on the motion vector of the sub-block in the current block, and then determines the predicted value of the sub-block based on the reconstructed value of the reference sub-block.

In another possible implementation, the encoder determines the reference coordinate position corresponding to the current coordinate position from the reference image based on the motion vector of the current coordinate position in the current block, and then determines the predicted value of the current coordinate position based on the reconstructed value of the reference coordinate position.

Optionally, the encoder determines the reference sub-block (reference coordinate position) corresponding to the current sub-block (current coordinate position) from the reference image based on the position (or current coordinate position) of the current sub-block and the motion vector, and determines the reconstructed sub-block (or reconstructed value of the reference coordinate position) of the reference sub-block as the predicted sub-block (or sample predicted value) of the current sub-block (or reference coordinate position).

In summary, in the embodiments of this application, since the reference image of the current image may contain motion information that is the same as or similar to that of the current block, the affine model of the current block is determined based on the motion information of the reference image and the reference point. This can provide the codec with more usable affine models, thereby enabling the encoder to determine a more suitable affine model for the current block and use the affine model to predict the current block, which helps to improve the quality and efficiency of image encoding and decoding.

In one possible implementation, in affine mode, the encoder constructs a candidate list containing multiple candidate affine models, determines an affine model suitable for the current block, and further performs image prediction coding based on this affine model. As shown in Figure 13, the method may also include the following steps:

Step 1204: Add the affine model of the current block to the candidate list.

In some embodiments, the candidate list is a subblock merge candidate list, which contains multiple candidate affine models. In this embodiment, the candidate list includes affine models constructed based on the motion vectors of neighboring blocks of the current block in the current image, and affine models constructed based on motion information and reference points of a reference image.

It should be noted that the encoding and decoding ends use the same method to construct the candidate list to ensure the consistency of the affine model ultimately adopted by the encoding and decoding ends.

Each affine model in the candidate list has its own affine model index.

Step 1205: Based on the prediction results of each affine model in the candidate list for the current block, determine the encoding cost corresponding to each affine model.

The encoder uses the various affine models in the candidate list to predict the current block, obtaining predictions for the current block using different affine models. Block. The encoder determines the encoding cost when using this affine model based on the predicted block of the current block and the original block of the current block.

In some embodiments, the encoder determines the residual between the predicted block and the original block, and quantizes and encodes the residual to determine the encoding cost based on the quantization and encoding result. This encoding cost can be characterized by rate, distortion, or complexity.

Step 1206: Determine the affine model index based on the encoding cost corresponding to each affine model.

In some embodiments, the encoder determines the encoding cost corresponding to each affine model in the candidate list, determines the most suitable affine model for predicting the current block, and obtains the affine model index of the affine model.

Optionally, the encoder determines the affine model index corresponding to the affine model with the lowest encoding cost.

Step 1207: Encode the affine model index into the bitstream.

In order to instruct the decoder to use the same affine model as the encoder to predict the current block, the encoder encodes the affine model index in the candidate list into the bitstream.

It should be noted that, in addition to encoding the affine model index, the encoding end will use the affine model to predict and determine the residual quantization encoding into the bitstream.

Similar to the decoder, the affine model constructed by the encoder may include the reference point coordinates, reference point motion vector, first parameter, second parameter, third parameter, and fourth parameter; or, the affine model constructed by the encoder may include the reference point motion vector, first parameter, second parameter, third parameter, and fourth parameter; or, the affine model may include the control point motion vectors of at least two control points. The specific representation of the affine model can be found in the decoder's embodiment, which will not be elaborated upon here.

Please refer to Figure 14, which shows a flowchart of an affine model determination process provided in an exemplary embodiment of this application. The method is applied to an encoder and may include the following steps:

Step 1401: Determine the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vector of the reference image of the current block.

Selection of reference image

Unlike determining the first, second, third, and fourth parameters (characterizing the rate of change of the motion vector with coordinates, hereinafter referred to as the rate of change parameters) of the current block based on the neighboring blocks of the current block in the current image, since the motion followed by the current block and its neighboring blocks in the current image may be significantly different, while there may be parts in the reference image that follow the same or similar motion as the current block, in this embodiment, the encoder determines the above-mentioned rate of change parameters based on the reference image of the current block.

Since motion information from a reference image is required in determining the above parameters, in some embodiments, the encoder determines a collocated picture of the current block, and then determines the rate of change parameter based on the motion vector of the collocated picture.

Among them, the co-position reference image is the reference image of the current image, and the co-position reference image has the motion vector when making prediction (the co-position reference image is reconstructed before the current image), that is, the motion vector of the co-position reference image relative to the reference image of the co-position reference image.

In some embodiments, the encoder determines a co-location reference image for the current block. The number of co-location reference images is at least one.

In some embodiments, when multiple co-reference images exist, the encoder determines at least one affine model for the current block, wherein different affine models are determined based on different co-reference images.

Optionally, the encoder applies this scheme to each of the multiple co-position reference images to determine the affine mode corresponding to each co-position reference image; or, the encoder applies this scheme to some of the multiple co-position reference images to determine the affine mode corresponding to some of the co-position reference images.

For ease of explanation, the following embodiments use a single co-position reference image as an example.

Furthermore, after identifying the corresponding reference image, the encoder encodes the image index of the corresponding reference image into the bitstream to instruct the decoder to determine the affine model based on the corresponding reference image consistent with the encoder. This image index encoding is used to characterize the position of the corresponding reference image in the reference image list.

Explanation of virtual control point locations

When determining the rate of change parameter based on the current image, at least two control points need to be determined. In this embodiment, when determining the rate of change based on a co-located reference image, it is also necessary to determine at least two points in the co-located reference image of the current block. These at least two points in the co-located reference image can be referred to as virtual control points (VCPs). For ease of representation in the following embodiments, virtual control points are used to refer to at least two points in the co-located reference image.

After identifying at least two points in the co-located reference image, the encoder determines the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vectors of the at least two points in the co-located reference image.

In one possible implementation, the encoder determines virtual control points in the co-location reference image in groups, wherein each group of virtual control points contains two or three virtual control points.

For each virtual control point group, the encoder determines a set of rate of change parameters based on the motion vectors of the virtual control points in that group. The rate of change parameters calculated for different virtual control point groups may differ; therefore, based on different virtual control point groups, the encoder can subsequently determine multiple affine models for the current block.

Optionally, when the virtual control point group contains two virtual control points, the two virtual control points are located at the upper left and upper right corners of the formed rectangle; When a virtual control point group contains 3 virtual control points, the 3 virtual control points are located at the upper left, upper right, and lower left corners of the rectangle formed, respectively.

Optionally, the rectangle formed by the virtual control points (which may be called a virtual block) may have the same or different size as the current block; the position of the rectangle formed by the virtual control points in the co-located reference image may have the same or different position as the current block in the current image.

In some embodiments, the virtual control points in the virtual control point group can be located in various directions of the co-position block in the co-position reference image, such as above, below, left, or right.

Since when constructing an affine model based on the motion information of the current image, motion information can be obtained from the adjacent blocks to the left and above the current block, but not from the right and below the current block (because the blocks to the right and below have not yet been encoded or decoded), in one possible implementation, among the determined n point groups, there exists at least one point group that satisfies the following condition: the points contained in the point group are located in the lower right region of the current block.

Since the motion information of the region to the right and below the co-position block in the reference image can be used when determining the rate of change parameter based on the point set that meets the above conditions (this is the reference information used when constructing the affine model based on the current image), the affine model determined based on this point set can serve as a powerful supplement to the affine model constructed based on the current image.

In an illustrative example, as shown in Figure 6, the current block in the current image corresponds to the corresponding block 62 in the corresponding reference image 61, and other positions represent motion vectors used for merging in merge mode. Taking the determination that the virtual control point group contains 3 virtual control points as an example, the encoder can select virtual control points numbered {9, 8, 7}, {15, 13, 14}, {6 (middle position of the corresponding block), 17, 18} to form a virtual control point group. Among them, the virtual control points in the virtual control point groups {9, 8, 7} and {15, 13, 14} are located at the upper left of the corresponding block, and the virtual control points in the virtual control point groups {6 (middle position of the corresponding block), 17, 18} are located at the lower right of the corresponding block.

Description of virtual control point availability

To ensure that the rate of change parameter can be determined based on the virtual control points (avoiding the determined rate of change parameter being 0), at least two points in the co-located reference image are located in at least two regions of predetermined size.

When two virtual control points are determined, each virtual control point is located in a region of a predetermined size.

When three virtual control points are determined, the three virtual control points are located in three regions with predetermined dimensions, or two of the three virtual control points are located in the same region with predetermined dimensions, and the other virtual control point is located in another region with predetermined dimensions.

In some embodiments, the region with a predetermined size can be a minimum storage unit for motion information. The coordinate positions within the same minimum storage unit for motion information contain the same motion information.

For example, the smallest storage unit is 8×8, meaning that each 8×8 block stores a set of motion information, that is, each coordinate position in the 8×8 block has the same motion information. Of course, the smallest storage unit can use a larger or smaller size, such as 16×16 (lower implementation cost, lower motion vector accuracy), 4×4 (higher implementation cost, higher motion vector accuracy), or even 1×1. This embodiment does not limit this.

In one possible implementation, the encoder determines whether a virtual control point is located in the same motion information minimum storage unit based on the coordinates of the virtual control points in the virtual control point group and the size of the motion information minimum storage unit.

If the virtual control points in a virtual control point group are located in at least two motion information minimum storage units, then the virtual control point group is determined to be available, and the rate of change parameter is determined based on the virtual control point group; if the virtual control points in a virtual control point group are located in the same motion information minimum storage unit, then the virtual control point group is determined to be unavailable, and the rate of change parameter will not be determined based on the virtual control point group.

To illustrate, given that the minimum storage unit for motion information is 4×4, and the coordinates of point A are (x1, y1) and the coordinates of point B are (x2, y2), if floor(x1/4) = floor(x2/4) and floor(y1/4) = floor(y2/4), then the two points are located in the same minimum storage unit for motion information; if floor(x1/4) ≠ floor(x2/4), and/or floor(y1/4) ≠ floor(y2/4), then the two points are located in different minimum storage units for motion information.

Step 1402: Determine the reference point coordinates and the reference point motion vector.

The reference point of the current block can be located inside the current block or outside the current block.

In some embodiments, when the reference point is located within the current block, the reference point is the top-left, top-right, or bottom-left vertex of the current block.

Specifically, if the height of the current block is H and the width is W, then if the reference point is the top left corner of the current block, the coordinates of the reference point are (0, 0); if the reference point is the top right corner of the current block, the coordinates of the reference point are (W, 0); if the reference point is the bottom left corner of the current block, the coordinates of the reference point are (0, H).

Regarding the method for determining the reference point motion vector, in one possible implementation, if the adjacent blocks of the current block support affine mode, the encoder determines the reference point motion vector based on the affine model of the adjacent blocks.

Optionally, if the current block supports the merging mode and the adjacent blocks support the affine mode, the encoder substitutes the coordinates of the reference point in the current block relative to the adjacent blocks into the affine model of the adjacent blocks to obtain the reference motion vector of the reference point.

As illustrated in Figure 7, the adjacent blocks of the current block include A0, A1, B0, B1, and B2. The encoder can determine the reference point motion vector of the upper left reference point based on the affine model of the adjacent block B2; or, based on the affine model of the adjacent blocks B0 or B1, determine the reference point motion vector of the upper right reference point; or, based on the affine model of the adjacent blocks A0 or A1, determine the reference point motion vector of the lower left reference point.

In another possible implementation, the encoder can determine the reference point motion vector of the reference point based on the available motion vectors in the adjacent blocks of the current block.

Optionally, the encoder can determine the reference point motion vector of the reference point in the current block by fitting based on at least two available motion vectors in the adjacent blocks of the current block.

As illustrated in Figure 8, the neighboring blocks of the current block include A0, A1, A2, B0, B1, B2, and B3. The decoder can determine the reference point motion vector of the upper left reference point based on the available motion vectors in the neighboring blocks B2, B3, and A2; determine the reference point motion vector of the upper right reference point based on the available motion vectors in the neighboring blocks B0 and B1; and determine the reference point motion vector of the lower left reference point based on the available motion vectors in the neighboring blocks A0 and A1.

In some embodiments, the encoder constructs an affine model of the form {(x, y), MV, a, b, c, d}, where (x, y) are the coordinates of the reference point, MV is the motion vector of the reference point, and a, b, c, d are the first to fourth parameters mentioned above, respectively.

In some embodiments, when the reference point is a fixed reference point, the decoder only needs to determine the reference point motion vector of the reference point, thereby constructing an affine model of the form {MV, a, b, c, d}, where MV is the reference point motion vector, and a, b, c, and d are the first to fourth parameters mentioned above, which will not be elaborated here in this embodiment.

In this embodiment, when the encoder determines the rate of change parameter based on the motion vector of the virtual control point group in the co-located reference image, it determines the virtual control point group from the lower right region of the current block. This allows the calculation of the rate of change parameter to utilize the motion information of the lower right region, which was previously unavailable when constructing the affine model based on the current image. This enables the affine model constructed using this scheme to serve as a powerful supplement to existing affine models.

Furthermore, by detecting whether the virtual control point is located in at least two minimum motion information storage units (i.e., regions with a predetermined size), the encoder ensures that the subsequent motion vector calculation based on the virtual control point yields usable rate of change parameters, which helps improve the quality of the subsequently constructed affine model.

In the above embodiments, the process of determining multiple motion vectors using an affine model that includes reference point coordinates, reference point motion vectors, a first parameter, a second parameter, a third parameter, and a fourth parameter may include:

Step 1: Determine the affine relation based on the affine model of the current block. The affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

In some embodiments, the affine relation can be expressed as:

Where (x _base , y _base ) are the coordinates of the reference point. and These are the horizontal and vertical components of the motion vector of the reference point, respectively, and abcd are the first to fourth parameters, respectively.

Once the affine relationship is determined, the encoder substitutes the coordinates into the affine relationship to obtain the motion vector at that coordinate position.

Step 2: Based on affine relations, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block.

In some embodiments, if the current block is divided into several sub-blocks, for each coordinate position in the sub-block, the encoder determines the motion vector at each coordinate position based on the affine relation, and determines the average value of the motion vectors at each coordinate position in the same sub-block as the motion vector of that sub-block.

In some embodiments, without sub-block division, the encoder can also determine the motion vector at each coordinate position based on affine relation for any coordinate position in the current block.

In one possible implementation, affine models with different representations can be converted to each other. When the affine model needs to be represented by at least two control points, after determining the rate of change parameter and reference point of the current block through the above steps, the encoder can determine the control point motion vectors of at least two control points based on the first parameter, second parameter, third parameter, fourth parameter and reference point information, and thus determine the affine model of the current block based on the control point motion vectors of at least two control points.

Optionally, the at least two control points must conform to the representation requirements of the affine model. For example, when using a 4-parameter affine model, the at least two control points include the top-left and top-right vertices of the current block; when using a 6-parameter affine model, the at least two control points include the top-left, top-right, and bottom-left vertices of the current block.

In one possible implementation, the encoder determines an affine relation based on a first parameter, a second parameter, a third parameter, a fourth parameter, and a reference point. The affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

In some embodiments, the affine relation can be expressed as:

Where (x _base , y _base ) are the coordinates of the reference point. and , respectively, are the horizontal and vertical components of the motion vector of the reference point, and abcd are the first to fourth parameters, respectively.

After determining the affine relation, the encoder determines the control point motion vector of at least two control points based on the affine relation and the control point coordinates of at least two control points. That is, by substituting the coordinates of the control points into the affine relation, the control point motion vector of that control point can be obtained.

Schematic, when the height of the current block is H and the width is W, the encoder substitutes the control point coordinates (0, 0) of the upper left control point in the current block into the above affine relation to obtain the control point motion vector CPMV0 of the upper left control point; substitutes the control point coordinates (W, 0) of the upper right control point into the above affine relation to obtain the control point motion vector CPMV1 of the upper right control point; substitutes the control point coordinates (0, H) of the lower left control point into the above affine relation to obtain the control point motion vector CPMV2 of the lower left control point.

In some embodiments, when a 4-parameter affine model is used, the encoder constructs an affine model of the form {CPMV0, CPMV1}; when a 6-parameter affine model is used, the encoder constructs an affine model of the form {CPMV0, CPMV1, CPMV2}.

Accordingly, an affine model is constructed using the motion vectors of control points based on at least two control points. When multiple motion vectors are determined, the encoder determines the first, second, third, and fourth parameters based on the motion vectors of the control points in the affine model of the current block. Based on the first, second, third, and fourth parameters, an affine relationship (used to characterize the correspondence between coordinate positions and motion vectors at coordinate positions) is determined. Thus, based on the affine relationship, the motion vectors of multiple sub-blocks in the current block are determined, or the motion vectors of multiple coordinate positions in the current block are determined.

Since the rate of change parameter has been determined during the construction of the affine model, in one possible implementation, the encoder directly uses the previously determined rate of change parameter.

Furthermore, since the 4-parameter or 6-parameter affine model contains the control point motion vector of the upper left corner control point of the current block, the encoder can determine the affine relationship based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the control point motion vector of the upper left corner control point.

In some embodiments, the determined affine relations are as follows:

in, and The components of the motion vector of the upper left control point in the horizontal and vertical directions.

Virtual control points in a reference image can be determined using at least one of the following methods:

Method 1: Based on the vertex coordinates of the current block in the current image, determine at least two points (virtual control points) in the corresponding reference image.

In this method, the virtual block formed by the virtual control point is in the same position and has the same size as the current block.

Method 2: Based on the vertex coordinates of the current block in the current image and the first offset, determine at least two points (virtual control points) in the coordinate reference image. The first offset is the coordinate offset relative to a specified vertex in the current block.

Method 3: Based on the vertex coordinates of the current block in the current image and the second offset, determine at least two points (virtual control points) in the co-located reference image.

Method 4: Based on the vertex coordinates, first offset, and second offset of the current block in the current image, determine at least two points (virtual control points) in the co-located reference image.

In one possible implementation, the encoder determines the motion vectors of neighboring blocks of the current block. If the motion vectors of the neighboring blocks point to the co-location reference image of the current block, the encoder determines a second offset based on the motion vectors of the neighboring blocks. If the second offset is determined, the encoder uses method 3 or 4 described above to determine the virtual control point in the co-location reference image; if the second offset is not determined, the encoder uses method 1 or 2 described above to determine the virtual control point in the co-location reference image.

The specific method for determining the virtual control point described above can be found in the decoding side implementation example, which will not be repeated here.

The process of determining the rate of change parameter

In one possible implementation, the encoder determines the fifth, sixth, seventh, and eighth parameters based on the motion vectors of at least two points (virtual control points) in the co-located reference image, and then scales the fifth, sixth, seventh, and eighth parameters to obtain the first, second, third, and fourth parameters.

Among them, the fifth to eighth parameters are used to characterize the rate of change of the motion vector of the coordinate position in the virtual block (the block determined by the virtual control points) as the coordinates change.

Regarding the method for determining the scaling ratio, in one possible implementation, the encoder determines a first image interval between the current image and a reference image of the current image, and a second image interval between two co-located reference images. Based on the ratio of the first image interval and the second image interval, the fifth, sixth, seventh, and eighth parameters are scaled to obtain the first, second, third, and fourth parameters.

The first image interval is used to characterize the POC distance between the current image and its reference image, and the second image interval is used to characterize the POC distance between the co-located reference image and its reference image.

In some embodiments, the encoder determines a scaling ratio based on a first image interval and a second image interval. The scaling is applied to a fifth parameter based on the scaling ratio to obtain a first parameter; to a sixth parameter based on the scaling ratio to obtain a second parameter; to a seventh parameter based on the scaling ratio to obtain a third parameter; and to an eighth parameter based on the scaling ratio to obtain a fourth parameter.

Schematic, if the POC distance between col_pic and col_ref is td, and the POC distance between curr_pic and curr_ref is tb, and based on the motion vectors of virtual control points in the co-located reference image, the rate of change parameters corresponding to the virtual blocks in the co-located reference image are determined to be a', b', ... Given c' and d', the rate of change parameters corresponding to the current block in the current image are a = a′*td/td, b = b′*tb/td, c = c′*tb/td, and d = d′*tb/td.

Please refer to Figure 15, which shows a structural block diagram of an image decoding apparatus provided in an exemplary embodiment of this application. The image decoding apparatus includes:

Decoding unit 1501 is used to determine the affine model of the current block based on motion information of the reference image of the current block and reference points;

Decoding unit 1501 is used to obtain multiple motion vectors based on the affine model of the current block;

Decoding unit 1501 is used to predict the current block based on the plurality of motion vectors.

Optionally, the affine model of the current block includes the reference point coordinates, the reference point motion vector, the first parameter, the second parameter, the third parameter, and the fourth parameter.

Optionally, the affine model of the current block includes the reference point motion vector of the reference point, a first parameter, a second parameter, a third parameter, and a fourth parameter.

Optionally, the affine model of the current block includes control point motion vectors for at least two control points.

Optionally, the decoding unit 1501 is used for:

Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined;

Determine the coordinates of the reference point and the motion vector of the reference point.

Optionally, the decoding unit 1501 is used for:

Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined;

Determine the motion vector of the reference point.

Optionally, the decoding unit 1501 is used for:

Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined;

Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, the control point motion vector of at least two control points is determined.

Optionally, the decoding unit 1501 is used for:

Identify at least two points in the co-location reference image of the current block;

The first parameter, the second parameter, the third parameter, and the fourth parameter are determined based on the motion vectors of at least two points in the co-located reference image.

Optionally, the decoding unit 1501 is used for:

Based on the vertex coordinates of the current block in the current image, determine at least two points in the co-located reference image;

Based on the vertex coordinates of the current block in the current image and the first offset, at least two points in the co-located reference image are determined;

Based on the vertex coordinates of the current block in the current image and the second offset, at least two points in the co-located reference image are determined;

Based on the vertex coordinates, first offset, and second offset of the current block in the current image, at least two points in the co-located reference image are determined;

The first offset is used to characterize the coordinate offset from the vertex of the current block, and the second offset is determined based on the motion vector of the adjacent blocks of the current block.

Optionally, the decoding unit 1501 is used for:

Determine the motion vectors of the adjacent blocks of the current block;

When the motion vector of the adjacent block points to the co-position reference image of the current block, the second offset is determined based on the motion vector of the adjacent block.

Optionally, at least two points in the co-position reference image are located in at least two regions of a predetermined size.

Optionally, coordinate positions within the same region of a predetermined size have the same motion information.

Optionally, the co-position reference image contains n point groups, and each point group contains 2 or 3 points, where n is a positive integer.

Optionally, at least one of the n point groups satisfies the following condition: the points in the point group are located in the lower right region of the current block.

Optionally, the decoding unit 1501 is used for:

Parse the bitstream to determine the co-location reference image of the current block.

Optionally, the decoding unit 1501 is used for:

In the presence of multiple co-located reference images, at least one affine model of the current block is determined, wherein different affine models are determined based on different co-located reference images.

Optionally, the decoding unit 1501 is used for:

Based on the motion vectors of at least two points in the co-located reference image, the fifth, sixth, seventh, and eighth parameters are determined.

The fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter.

Optionally, the decoding unit 1501 is used for:

Determine a first image interval between the current image and a reference image of the current image;

Determine a second image interval between the co-position reference image and the reference image of the co-position reference image;

Based on the ratio of the first image interval to the second image interval, the fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter.

Optionally, the decoding unit 1501 is used for:

Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

Based on the affine relation and the coordinates of the control points of at least two control points, determine the motion vectors of the control points of at least two control points.

Optionally, the decoding unit 1501 is used for:

The affine relation is determined based on the affine model of the current block, and the affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position;

Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block.

Optionally, the decoding unit 1501 is used for:

Based on the affine model of the current block, determine the first parameter, the second parameter, the third parameter, and the fourth parameter;

Based on the first parameter, the second parameter, the third parameter, and the fourth parameter, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position;

Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block.

Optionally, the reference point is located within the current block;

or,

The reference point is located outside the current block.

Optionally, the reference point is the top-left, top-right, or bottom-left vertex of the current block.

Optionally, the decoding unit 1501 is used for:

If the neighboring blocks of the current block support affine mode, the reference point motion vector of the reference point is determined based on the affine model of the neighboring blocks.

Optionally, the decoding unit 1501 is used for:

Based on the motion vector of the sub-block in the current block, a reference sub-block corresponding to the sub-block is determined from the reference image; based on the reconstructed value of the reference sub-block, the predicted value of the sub-block is determined;

or,

Based on the motion vector of the current coordinate position in the current block, the reference coordinate position corresponding to the current coordinate position is determined from the reference image; the predicted value of the current coordinate position is determined based on the reconstructed value of the reference coordinate position.

Optionally, the decoding unit 1501 is used for:

Add the affine model of the current block to the candidate list;

Analyze the bitstream to determine the affine model index;

The plurality of motion vectors are obtained based on the affine model indicated by the affine model index in the candidate list.

Please refer to Figure 16, which shows a structural block diagram of an image encoding apparatus provided in an exemplary embodiment of this application. The image encoding apparatus includes:

The encoding unit 1601 is used to determine the affine model of the current block based on the motion information of the reference image of the current block and the reference point;

Encoding unit 1601 is used to obtain multiple motion vectors based on the affine model of the current block;

The encoding unit 1601 is used to predict the current block based on the plurality of motion vectors.

Optionally, the affine model of the current block includes the reference point coordinates, the reference point motion vector, the first parameter, the second parameter, the third parameter, and the fourth parameter.

Optionally, the affine model of the current block includes the reference point motion vector of the reference point, a first parameter, a second parameter, a third parameter, and a fourth parameter.

Optionally, the affine model of the current block includes control point motion vectors for at least two control points.

Optionally, the encoding unit 1601 is configured to: determine the first parameter and the second parameter based on the motion vector of the reference image of the current block. The second parameter, the third parameter, and the fourth parameter;

Determine the coordinates of the reference point and the motion vector of the reference point.

Optionally, encoding unit 1601 is used for:

Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined;

Determine the motion vector of the reference point.

Optionally, encoding unit 1601 is used for:

Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined;

Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, the control point motion vector of at least two control points is determined.

Optionally, encoding unit 1601 is used for:

Identify at least two points in the co-location reference image of the current block;

The first parameter, the second parameter, the third parameter, and the fourth parameter are determined based on the motion vectors of at least two points in the co-located reference image.

Optionally, encoding unit 1601 is used for:

Based on the vertex coordinates of the current block in the current image, determine at least two points in the co-located reference image;

Based on the vertex coordinates of the current block in the current image and the first offset, at least two points in the co-located reference image are determined;

Based on the vertex coordinates of the current block in the current image and the second offset, at least two points in the co-located reference image are determined;

Based on the vertex coordinates, first offset, and second offset of the current block in the current image, at least two points in the co-located reference image are determined;

The first offset is used to characterize the coordinate offset from the vertex of the current block, and the second offset is determined based on the motion vector of the adjacent blocks of the current block.

Optionally, encoding unit 1601 is used for:

Determine the motion vectors of the adjacent blocks of the current block;

When the motion vector of the adjacent block points to the co-position reference image of the current block, the second offset is determined based on the motion vector of the adjacent block.

Optionally, at least two points in the co-position reference image are located in at least two regions of a predetermined size.

Optionally, coordinate positions within the same region of a predetermined size have the same motion information.

Optionally, the co-position reference image contains n point groups, and each point group contains 2 or 3 points, where n is a positive integer.

Optionally, at least one of the n point groups satisfies the following condition: the points in the point group are located in the lower right region of the current block.

Optionally, encoding unit 1601 is used for:

Determine the co-position reference image of the current block;

The image index of the corresponding reference image is encoded into the bitstream.

Optionally, encoding unit 1601 is used for:

In the presence of multiple co-located reference images, at least one affine model of the current block is determined, wherein different affine models are determined based on different co-located reference images.

Optionally, encoding unit 1601 is used for:

Based on the motion vectors of at least two points in the co-located reference image, the fifth, sixth, seventh, and eighth parameters are determined.

The fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter.

Optionally, encoding unit 1601 is used for:

Determine a first image interval between the current image and a reference image of the current image;

Determine a second image interval between the co-position reference image and the reference image of the co-position reference image;

Based on the ratio of the first image interval to the second image interval, the fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter.

Optionally, encoding unit 1601 is used for:

Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

Based on the affine relation and the coordinates of the control points of at least two control points, determine the motion vectors of the control points of at least two control points.

Optionally, encoding unit 1601 is used for:

The affine relation is determined based on the affine model of the current block, and the affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position;

Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block.

Optionally, encoding unit 1601 is used for:

Based on the affine model of the current block, determine the first parameter, the second parameter, the third parameter, and the fourth parameter;

Based on the first parameter, the second parameter, the third parameter, and the fourth parameter, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position.

Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block.

Optionally, the reference point is located within the current block;

or,

The reference point is located outside the current block.

Optionally, the reference point is the top-left, top-right, or bottom-left vertex of the current block.

Optionally, encoding unit 1601 is used for:

If the neighboring blocks of the current block support affine mode, the reference point motion vector of the reference point is determined based on the affine model of the neighboring blocks.

Optionally, encoding unit 1601 is used for:

Based on the motion vector of the sub-block in the current block, a reference sub-block corresponding to the sub-block is determined from the reference image; based on the reconstructed value of the reference sub-block, the predicted value of the sub-block is determined;

or,

Based on the motion vector of the current coordinate position in the current block, the reference coordinate position corresponding to the current coordinate position is determined from the reference image; the predicted value of the current coordinate position is determined based on the reconstructed value of the reference coordinate position.

Optionally, encoding unit 1601 is used for:

Add the affine model of the current block to the candidate list;

Based on the prediction results of each affine model in the candidate list for the current block, the encoding cost corresponding to each affine model is determined.

Determine the affine model index based on the encoding cost corresponding to each affine model;

The affine model index is encoded into the bitstream.

It should be noted that the image decoding process of the above-mentioned decoding unit can refer to the above-mentioned image decoding method embodiment, and the image encoding process of the above-mentioned encoding unit can refer to the above-mentioned image encoding method embodiment. This embodiment will not be described in detail here.

Please refer to Figure 17, which shows a structural block diagram of a decoder provided in an exemplary embodiment of this application. The decoder may include one or more components such as a processor 1701 and a memory 1702. The components are coupled together via a bus system. It is understood that the bus system is used to implement communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

Memory 1702 is used to store computer programs that can run on processor 1701; processor 1701 is used to execute, when running the computer program:

Based on the motion information of the reference image of the current block and the reference point of the current block, an affine model of the current block is constructed.

Based on the affine model of the current block, perform affine motion vector transformation to obtain multiple motion vectors of the current block;

The current block is predicted based on multiple motion vectors of the current block.

It is understood that the memory 1702 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1702 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

The processor 1701 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the aforementioned image decoding method can be completed through integrated logic circuits in the processor 1701 or through software instructions. The processor 1701 can be a general-purpose processor, a digital signal processor (DSP), or an application-specific integrated circuit (ASIC). Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 1702, and processor 1701 reads information from memory 1702 and, in conjunction with its hardware, completes the steps of the above methods. It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor. Optionally, as another embodiment, the processor 1701 is also configured to execute the image decoding method described in any of the foregoing embodiments when running the computer program.

Please refer to Figure 18, which shows a structural block diagram of an encoder provided in an exemplary embodiment of this application. The encoder may include one or more components such as a processor 1801 and a memory 1802. The components are coupled together via a bus system. It is understood that the bus system is used to enable communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

Memory 1802 is used to store computer programs that can run on processor 1801; processor 1801 is used to execute, when running the computer program:

Based on the motion information of the reference image of the current block and the reference point of the current block, an affine model of the current block is constructed.

Based on the affine model of the current block, perform affine motion vector transformation to obtain multiple motion vectors of the current block;

The current block is predicted based on multiple motion vectors of the current block.

It is understood that the memory 1802 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1802 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

The processor 1801 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the aforementioned image encoding method can be completed by the integrated logic circuitry in the hardware of the processor 1801 or by software instructions. The processor 1801 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1802. Processor 1801 reads information from memory 1802 and, in conjunction with its hardware, completes the steps of the above method. It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by the processor. The memory can be implemented in the processor or external to the processor. Optionally, as another embodiment, processor 1801 is also configured to execute the image encoding method described in any of the foregoing embodiments when running the computer program.

This application also provides a non-volatile computer-readable storage medium for storing a bitstream, the bitstream being generated by an image encoding method using an encoder, or the bitstream being decoded by an image decoding method using a decoder, wherein the image encoding method includes the image encoding method described in the above embodiments, and the image decoding method includes the image decoding method described in the above embodiments.

This application also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. A processor retrieves the computer instructions from the computer-readable storage medium and executes them. The computer instructions are executed to implement the image decoding method or image encoding method as described in the above embodiments.

Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims (58)

An image decoding method, characterized in that the method includes: Based on the motion information of the reference image of the current block and the reference point, the affine model of the current block is determined; Multiple motion vectors are obtained based on the affine model of the current block; The current block is predicted based on the multiple motion vectors. According to the method of claim 1, the affine model of the current block includes the reference point coordinates, the reference point motion vector, the first parameter, the second parameter, the third parameter, and the fourth parameter. According to the method of claim 1, the affine model of the current block includes the reference point motion vector of the reference point, a first parameter, a second parameter, a third parameter, and a fourth parameter. The method according to claim 1 is characterized in that the affine model of the current block includes control point motion vectors of at least two control points. The method according to claim 2, characterized in that, determining the affine model of the current block based on the motion information of the reference image of the current block and the reference point includes: Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined; Determine the coordinates of the reference point and the motion vector of the reference point. The method according to claim 3, characterized in that, determining the affine model of the current block based on the motion information of the reference image of the current block and the reference point includes: Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined; Determine the motion vector of the reference point. The method according to claim 4, characterized in that, constructing the affine model of the current block based on the motion information of the reference image of the current block and the reference point, includes: Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined; Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, the control point motion vector of at least two control points is determined. The method according to any one of claims 5 to 7, characterized in that, determining the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vector of the reference image of the current block includes: Identify at least two points in the co-location reference image of the current block; The first parameter, the second parameter, the third parameter, and the fourth parameter are determined based on the motion vectors of at least two points in the co-located reference image. The method according to claim 8, wherein determining at least two points in the co-location reference image of the current block comprises at least one of the following: Based on the vertex coordinates of the current block in the current image, determine at least two points in the co-located reference image; Based on the vertex coordinates of the current block in the current image and the first offset, at least two points in the co-located reference image are determined; Based on the vertex coordinates of the current block in the current image and the second offset, at least two points in the co-located reference image are determined; Based on the vertex coordinates, first offset, and second offset of the current block in the current image, at least two points in the co-located reference image are determined; The first offset is used to characterize the coordinate offset from the vertex of the current block, and the second offset is determined based on the motion vector of the adjacent blocks of the current block. The method according to claim 9, characterized in that the method further comprises: Determine the motion vectors of the adjacent blocks of the current block; When the motion vector of the adjacent block points to the co-position reference image of the current block, the second offset is determined based on the motion vector of the adjacent block. The method according to claim 8, wherein at least two points in the co-position reference image are located in at least two regions of predetermined size. The method according to claim 11 is characterized in that the coordinate positions within the same region having a predetermined size have the same motion information. According to the method of claim 8, the co-position reference image comprises n point groups, and each point group comprises 2 There are one or three points, where n is a positive integer. According to the method of claim 13, at least one of the n point groups satisfies the following condition: the points in the point group are located in the lower right region of the current block. The method according to claim 8, characterized in that the method further comprises: Parse the bitstream to determine the co-location reference image of the current block. The method according to claim 15, wherein determining the affine model of the current block comprises: In the presence of multiple co-located reference images, at least one affine model of the current block is determined, wherein different affine models are determined based on different co-located reference images. The method according to claim 8, characterized in that, determining the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vectors of at least two points in the co-located reference image, includes: Based on the motion vectors of at least two points in the co-located reference image, the fifth, sixth, seventh, and eighth parameters are determined. The fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter. According to the method of claim 17, the scaling of the fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter includes: Determine a first image interval between the current image and a reference image of the current image; Determine a second image interval between the co-position reference image and the reference image of the co-position reference image; Based on the ratio of the first image interval to the second image interval, the fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter. According to the method of claim 7, the step of determining the control point motion vector based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point includes: Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position. Based on the affine relation and the coordinates of the control points of at least two control points, determine the motion vectors of the control points of at least two control points. The method according to claim 2 or 3, characterized in that, the affine model based on the current block yields multiple motion vectors, including: The affine relation is determined based on the affine model of the current block, and the affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position; Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block. The method according to claim 4, characterized in that, obtaining multiple motion vectors based on the affine model of the current block includes: Based on the affine model of the current block, determine the first parameter, the second parameter, the third parameter, and the fourth parameter; Based on the first parameter, the second parameter, the third parameter, and the fourth parameter, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position. Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block. The method according to any one of claims 1 to 21, characterized in that, The reference point is located within the current block; or, The reference point is located outside the current block. The method according to claim 22 is characterized in that the reference point is the upper left corner, upper right corner, or lower left corner of the current block. The method according to claim 22, characterized in that the method comprises: If the neighboring blocks of the current block support affine mode, the reference point motion vector of the reference point is determined based on the affine model of the neighboring blocks. The method according to any one of claims 1 to 24, characterized in that, the prediction of the current block based on the plurality of motion vectors includes: Based on the motion vector of the sub-block in the current block, a reference sub-block corresponding to the sub-block is determined from the reference image; based on the reconstructed value of the reference sub-block, the predicted value of the sub-block is determined; or, Based on the motion vector of the current coordinate position in the current block, the reference coordinate position corresponding to the current coordinate position is determined from the reference image; the predicted value of the current coordinate position is determined based on the reconstructed value of the reference coordinate position. The method according to any one of claims 1 to 25, characterized in that, the affine model based on the current block yields multiple motion vectors, including: Add the affine model of the current block to the candidate list; Analyze the bitstream to determine the affine model index; The plurality of motion vectors are obtained based on the affine model indicated by the affine model index in the candidate list. An image encoding method, characterized in that the method includes: Based on the motion information of the reference image of the current block and the reference point, the affine model of the current block is determined; Multiple motion vectors are obtained based on the affine model of the current block; The current block is predicted based on the multiple motion vectors. The method according to claim 27 is characterized in that the affine model of the current block includes the reference point coordinates, the reference point motion vector, the first parameter, the second parameter, the third parameter, and the fourth parameter. The method according to claim 27 is characterized in that the affine model of the current block includes the reference point motion vector of the reference point, a first parameter, a second parameter, a third parameter, and a fourth parameter. The method according to claim 27 is characterized in that the affine model of the current block includes control point motion vectors of at least two control points. The method according to claim 28, characterized in that, determining the affine model of the current block based on the motion information of the reference image of the current block and the reference point includes: Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined; Determine the coordinates of the reference point and the motion vector of the reference point. The method according to claim 29, characterized in that, determining the affine model of the current block based on the motion information of the reference image of the current block and the reference point includes: Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined; Determine the motion vector of the reference point. The method according to claim 30, characterized in that, determining the affine model of the current block based on the motion information of the reference image of the current block and the reference point includes: Based on the motion vector of the reference image of the current block, the first parameter, the second parameter, the third parameter, and the fourth parameter are determined; Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, the control point motion vector of at least two control points is determined. The method according to any one of claims 31 to 33, characterized in that, determining the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vector of the reference image of the current block includes: Identify at least two points in the co-location reference image of the current block; The first parameter, the second parameter, the third parameter, and the fourth parameter are determined based on the motion vectors of at least two points in the co-located reference image. The method according to claim 34, wherein determining at least two points in the co-location reference image of the current block comprises at least one of the following: Based on the vertex coordinates of the current block in the current image, determine at least two points in the co-located reference image; Based on the vertex coordinates of the current block in the current image and the first offset, at least two points in the co-located reference image are determined; Based on the vertex coordinates of the current block in the current image and the second offset, at least two points in the co-located reference image are determined; Based on the vertex coordinates, first offset, and second offset of the current block in the current image, at least two points in the co-located reference image are determined; The first offset is used to characterize the coordinate offset from the vertex of the current block, and the second offset is determined based on the motion vector of the adjacent blocks of the current block. The method according to claim 35, characterized in that the method further comprises: Determine the motion vectors of the adjacent blocks of the current block; When the motion vector of the adjacent block points to the co-position reference image of the current block, the second offset is determined based on the motion vector of the adjacent block. The method according to claim 34 is characterized in that at least two points in the co-position reference image are located in at least two regions of predetermined size. The method according to claim 37 is characterized in that the coordinate positions within the same region having a predetermined size have the same motion information. The method according to claim 34 is characterized in that the co-position reference image comprises n point groups, and each point group comprises 2 or 3 points, where n is a positive integer. According to the method of claim 39, at least one of the n point groups satisfies the following condition: the points in the point group are located in the lower right region of the current block. The method according to claim 34, characterized in that the method further comprises: Determine the co-position reference image of the current block; The image index of the corresponding reference image is encoded into the bitstream. The method according to claim 41, wherein determining the affine model of the current block includes: In the presence of multiple co-located reference images, at least one affine model of the current block is determined, wherein different affine models are determined based on different co-located reference images. The method according to claim 34, characterized in that, determining the first parameter, the second parameter, the third parameter, and the fourth parameter based on the motion vectors of at least two points in the co-located reference image, comprises: Based on the motion vectors of at least two points in the co-located reference image, the fifth, sixth, seventh, and eighth parameters are determined. The fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter. The method according to claim 43, characterized in that, scaling the fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter includes: Determine a first image interval between the current image and a reference image of the current image; Determine a second image interval between the co-position reference image and a reference image of the co-position reference image; Based on the ratio of the first image interval to the second image interval, the fifth parameter, the sixth parameter, the seventh parameter, and the eighth parameter are scaled to obtain the first parameter, the second parameter, the third parameter, and the fourth parameter. The method according to claim 33, characterized in that, determining the control point motion vector based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, includes: Based on the first parameter, the second parameter, the third parameter, the fourth parameter, and the reference point, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position. Based on the affine relation and the coordinates of the control points of at least two control points, determine the motion vectors of the control points of at least two control points. The method according to claim 28 or 29, characterized in that, obtaining multiple motion vectors based on the affine model of the current block includes: The affine relation is determined based on the affine model of the current block, and the affine relation is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position; Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block. The method according to claim 30, characterized in that, obtaining multiple motion vectors based on the affine model of the current block includes: Based on the affine model of the current block, determine the first parameter, the second parameter, the third parameter, and the fourth parameter; Based on the first parameter, the second parameter, the third parameter, and the fourth parameter, an affine relation is determined, which is used to characterize the correspondence between the coordinate position and the motion vector at the coordinate position. Based on the affine relationship, determine the motion vectors of multiple sub-blocks in the current block, or determine the motion vectors of multiple coordinate positions in the current block. The method according to any one of claims 27 to 47, characterized in that, The reference point is located within the current block; or, The reference point is located outside the current block. The method according to claim 48 is characterized in that the reference point is the upper left corner vertex, upper right corner vertex, or lower left corner vertex of the current block. The method according to claim 48, characterized in that the method comprises: If the neighboring blocks of the current block support affine mode, the reference point motion vector of the reference point is determined based on the affine model of the neighboring blocks. The method according to any one of claims 27 to 50, characterized in that, the prediction of the current block based on the plurality of motion vectors includes: Based on the motion vector of the sub-block in the current block, a reference sub-block corresponding to the sub-block is determined from the reference image; based on the reconstructed value of the reference sub-block, the predicted value of the sub-block is determined; or, Based on the motion vector of the current coordinate position in the current block, the reference coordinate position corresponding to the current coordinate position is determined from the reference image; the predicted value of the current coordinate position is determined based on the reconstructed value of the reference coordinate position. The method according to any one of claims 27 to 51, characterized in that the method further comprises: Add the affine model of the current block to the candidate list; Based on the prediction results of each affine model in the candidate list for the current block, the encoding cost corresponding to each affine model is determined. Determine the affine model index based on the encoding cost corresponding to each affine model; The affine model index is encoded into the bitstream. A decoding device, characterized in that the device comprises: The decoding unit is used to determine the affine model of the current block based on the motion information of the reference image of the current block and the reference point; The decoding unit is used to obtain multiple motion vectors based on the affine model of the current block; The decoding unit is used to predict the current block based on the plurality of motion vectors. An encoding device, characterized in that the device comprises: An encoding unit is used to determine the affine model of the current block based on motion information of a reference image of the current block and a reference point; The encoding unit is used to obtain multiple motion vectors based on the affine model of the current block; The encoding unit is used to predict the current block based on the plurality of motion vectors. A decoder, characterized in that the decoder includes a memory and a processor, the memory being used to store a computer program running on the processor; the processor being used to execute the image decoding method as described in any one of claims 1 to 26 when running the computer program. An encoder, characterized in that the encoder includes a memory and a processor, the memory being used to store a computer program running on the processor; the processor being used to execute the image encoding method as described in any one of claims 27 to 52 when running the computer program. A non-volatile computer-readable storage medium for storing a bitstream, characterized in that the bitstream is generated by an image encoding method using an encoder, or the bitstream is decoded by an image decoding method using a decoder, wherein the image encoding method includes the image encoding method as described in any one of claims 27 to 52, and the image decoding method includes the image decoding method as described in any one of claims 1 to 26. A computer program product, characterized in that the computer program product includes computer instructions stored in a computer-readable storage medium, a processor retrieves the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the image decoding method as described in any one of claims 1 to 26, or the image encoding method as described in any one of claims 27 to 52.