[go: up one dir, main page]

WO2018230493A1 - Dispositif de décodage vidéo, dispositif de codage vidéo, dispositif de génération d'image de prévision et dispositif de dérivation de vecteur de mouvement - Google Patents

Dispositif de décodage vidéo, dispositif de codage vidéo, dispositif de génération d'image de prévision et dispositif de dérivation de vecteur de mouvement Download PDF

Info

Publication number
WO2018230493A1
WO2018230493A1 PCT/JP2018/022196 JP2018022196W WO2018230493A1 WO 2018230493 A1 WO2018230493 A1 WO 2018230493A1 JP 2018022196 W JP2018022196 W JP 2018022196W WO 2018230493 A1 WO2018230493 A1 WO 2018230493A1
Authority
WO
WIPO (PCT)
Prior art keywords
unit
motion vector
prediction
block
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2018/022196
Other languages
English (en)
Japanese (ja)
Inventor
知宏 猪飼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sharp Corp
Original Assignee
Sharp Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sharp Corp filed Critical Sharp Corp
Publication of WO2018230493A1 publication Critical patent/WO2018230493A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • H04N19/433—Hardware specially adapted for motion estimation or compensation characterised by techniques for memory access
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/537—Motion estimation other than block-based

Definitions

  • the present invention relates to a video decoding device, a video encoding device, a predicted image generation device, and a motion vector derivation device.
  • a moving image encoding device that generates encoded data by encoding the moving image, and a moving image that generates a decoded image by decoding the encoded data
  • An image decoding device is used.
  • the moving image encoding method include a method proposed in H.264 / AVC and HEVC (High-Efficiency Video Coding).
  • an image (picture) constituting a moving image is a slice obtained by dividing the image, a coding tree unit (CTU: Coding Tree Unit obtained by dividing the slice). ), A coding unit obtained by dividing a coding tree unit (sometimes called a coding unit (Coding Unit: CU)), and a prediction unit which is a block obtained by dividing a coding unit (PU) and a hierarchical structure composed of conversion units (TU), and encoded / decoded for each CU.
  • CTU Coding Tree Unit obtained by dividing the slice.
  • CU Coding Unit
  • a prediction unit which is a block obtained by dividing a coding unit (PU) and a hierarchical structure composed of conversion units (TU), and encoded / decoded for each CU.
  • a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and the predicted image is generated from the input image (original image).
  • a prediction residual obtained by subtraction (sometimes referred to as “difference image” or “residual image”) is encoded. Examples of methods for generating a predicted image include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).
  • Non-Patent Documents 1 to 3 can be cited as recent moving picture encoding and decoding techniques.
  • Non-Patent Document 1 describes a technique related to BIO (bi-directional optical flow: bi-predictive gradient change) prediction that is corrected using gradient information in motion compensation processing when generating a predicted image.
  • Non-Patent Document 2 describes a technique using affine prediction.
  • Non-Patent Document 3 describes a technique for searching for a motion vector by matching in an encoding device and a decoding device.
  • JVET-F0028 "Joint Video Exploration Team (JVET) of ITU-T SG16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11", 6th Meeting: Hobart, AU, 3 March-7 April 2017 Improved affine motion prediction, JVET-C0062, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 2016-05-17 Video / JVET, "Algorithm Description of Joint Exploration TestModel 1 (JEM 1)", INTERNATIONAL ORGANIZATION FOR STANDARDIZATION ORGANISATION INTERNATIONALE DE NORMALISATION ISO / IEC JTC1 / SC29 / WG11 CODING OF / MOVING PICTCS29 / AND N15790, October 2015, Geneva, CH.
  • the present invention aims to reduce the memory bandwidth.
  • a predicted image generation apparatus is a predicted image generation apparatus that generates a predicted image by performing motion compensation on one or more reference images, Single prediction mode for generating a predicted image with reference to the first reference image, bi-prediction mode for generating a predicted image with reference to the first reference image and the second reference image, and the first reference image
  • a predicted image generation unit that generates a predicted image using at least one of the BIO modes for generating a predicted image with reference to the second reference image and the gradient correction term, wherein the predicted image generation unit includes: When the reference block in at least one of the reference image and the second reference image is outside the screen of the reference image, the generation of the predicted image using the BIO mode is prohibited.
  • a predicted image generation apparatus is a predicted image generation apparatus that generates a predicted image by performing motion compensation on a plurality of reference images.
  • a predicted image generation unit that generates a predicted image using at least a plurality of modes including a BIO mode that generates a predicted image with reference to the reference image, the second reference image, and the gradient correction term.
  • the unit generates a pixel value outside the reading region that is a pixel value outside the reading region with respect to the corresponding block in at least one of the first reference image and the second reference image, and uses the BIO mode to generate a predicted image
  • the generation process of the pixel value outside the reading area along the vertical direction or the horizontal direction by the predicted image generation unit is prohibited.
  • a motion vector deriving device is a motion vector deriving device for deriving a motion vector of each of subblocks constituting a target block.
  • a motion vector deriving unit that calculates a motion vector of each sub-block with reference to motion vectors at a plurality of control points set in a reference block sharing a vertex with the target block, Restricts the range of the difference of each motion vector at the plurality of control points.
  • a motion vector derivation device provides a motion vector derivation that generates a motion vector referred to in order to generate a prediction image used for encoding or decoding a moving image.
  • a first motion vector search unit that searches for a motion vector for each prediction block by matching processing and a motion vector selected by the first motion vector search unit are included in the prediction block.
  • a second motion vector search unit that searches for a motion vector by a matching process for each of the plurality of sub-blocks, and the first motion vector search unit performs an initial vector search for a prediction block;
  • the second motion vector search unit searches for a motion vector by performing a local search.
  • a motion vector is searched by performing a local search after performing an initial vector search for a block, and a local search by the first motion vector search unit and a local search by the second motion vector search unit In at least one of the target searches, a search in an oblique direction is prohibited.
  • the memory bandwidth can be reduced.
  • FIG. 1 It is a figure which shows the hierarchical structure of the data of an encoding stream. It is a figure which shows the pattern of PU division
  • FIG. 3 is a block diagram illustrating a configuration of an AMVP prediction parameter derivation unit according to Embodiment 1.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter encoding part of the image coding apparatus which concerns on Embodiment 1.
  • FIG. It is a block diagram which shows the structure of the inter estimated image generation part of the image decoding apparatus which concerns on Embodiment 1.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter decoding part which concerns on Embodiment 1.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter decoding part which concerns on Embodiment 1.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter decoding part which concerns on Embodiment 1.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter decoding part which concerns on Embodiment 1.
  • 5 is a flowchart illustrating an example of a flow of processing in which a motion compensation unit in the image decoding device according to Embodiment 1 generates a predicted image.
  • 12 is a flowchart illustrating another example of a flow of processing in which the motion compensation unit in the image decoding device according to Embodiment 1 generates a predicted image.
  • FIG. 12 is a flowchart illustrating an example of a flow of processing in which the motion compensation unit in the image decoding device according to the second embodiment determines whether to prohibit BIO padding in the horizontal direction.
  • 10 is a flowchart illustrating an example of a flow of processing for determining whether or not the motion compensation unit in the image decoding device according to the second embodiment prohibits BIO padding in the vertical direction.
  • 12 is a flowchart illustrating an example of a flow of processing for determining whether or not the motion compensation unit in the image decoding device according to the second embodiment prohibits BIO padding in the horizontal direction or the vertical direction. It is the schematic of the corresponding block referred when the motion compensation part in the image decoding apparatus which concerns on Embodiment 2 uses BIO mode.
  • 14 is a flowchart illustrating an operation of motion vector decoding processing of the image decoding apparatus according to the third embodiment.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter encoding part of the image coding apparatus which concerns on Embodiment 3.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter decoding part which concerns on Embodiment 3.
  • FIG. It is a figure which shows the example which derives
  • (A) is a figure for demonstrating bilateral matching (Bilateral (matching) matching).
  • (B) is a figure for demonstrating template matching (Template
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter encoding part of the image coding apparatus which concerns on Embodiment 4.
  • FIG. It is a block diagram which shows the structure of the inter prediction parameter decoding part which concerns on Embodiment 4.
  • FIG. It is a flowchart figure which shows the outline
  • (A) And (b) is a figure for demonstrating a motion search pattern. It is a flowchart figure which shows the outline
  • FIG. 52 is a schematic diagram illustrating a configuration of the image transmission system 1 according to the first embodiment.
  • the image transmission system 1 is a system that transmits a code obtained by encoding an encoding target image, decodes the transmitted code, and displays an image.
  • the image transmission system 1 includes an image encoding device (moving image encoding device) 11, a network 21, an image decoding device (moving image decoding device) 31, and an image display device 41.
  • the image encoding device 11 receives an image T indicating a single layer image or a plurality of layers.
  • a layer is a concept used to distinguish a plurality of pictures when there are one or more pictures constituting a certain time. For example, when the same picture is encoded by a plurality of layers having different image quality or resolution, scalable encoding is performed, and when a picture of a different viewpoint is encoded by a plurality of layers, view scalable encoding is performed.
  • inter-layer prediction, inter-view prediction encoding efficiency is greatly improved. Further, even when prediction is not performed (simultaneous casting), encoded data can be collected.
  • the network 21 transmits the encoded stream Te generated by the image encoding device 11 to the image decoding device 31.
  • the network 21 is the Internet, a wide area network (WAN: Wide Area Network), a small network (LAN: Local Area Network), or a combination thereof.
  • the network 21 is not necessarily limited to a bidirectional communication network, and may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcast and satellite broadcast.
  • the network 21 may be replaced by a storage medium that records an encoded stream Te such as a DVD (Digital Versatile Disc) or a BD (Blue-ray Disc).
  • the image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates one or a plurality of decoded images Td decoded.
  • the image display device 41 displays all or part of one or a plurality of decoded images Td generated by the image decoding device 31.
  • the image display device 41 includes a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Further, in spatial scalable coding and SNR scalable coding, when the image decoding device 31 and the image display device 41 have high processing capability, an enhancement layer image with high image quality is displayed. When the processing capability is lower, the base layer image that does not require higher processing capability and display capability is displayed as the enhancement layer.
  • X? Y: z is a ternary operator that takes y when x is true (non-zero) and takes z when x is false (0).
  • FIG. 1 is a diagram showing a hierarchical structure of data in the encoded stream Te.
  • the encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence.
  • (A) to (f) of FIG. 1 respectively show an encoded video sequence defining a sequence SEQ, an encoded picture defining a picture PICT, an encoded slice defining a slice S, and an encoded slice defining a slice data
  • the encoded video sequence In the encoded video sequence, a set of data referred to by the image decoding device 31 for decoding the sequence SEQ to be processed is defined. As shown in FIG. 1A, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an addition. Includes SEI (Supplemental Enhancement Information). Here, the value indicated after # indicates the layer ID. FIG. 1 shows an example in which encoded data of # 0 and # 1, that is, layer 0 and layer 1, exists, but the type of layer and the number of layers are not dependent on this.
  • SEI Supplemental Enhancement Information
  • the video parameter set VPS is a set of coding parameters common to a plurality of moving images, a plurality of layers included in the moving image, and coding parameters related to individual layers in a moving image composed of a plurality of layers.
  • a set is defined.
  • the sequence parameter set SPS defines a set of encoding parameters that the image decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are defined. A plurality of SPSs may exist. In that case, one of a plurality of SPSs is selected from the PPS.
  • a set of encoding parameters referred to by the image decoding device 31 in order to decode each picture in the target sequence is defined.
  • a reference value (pic_init_qp_minus26) of a quantization width used for picture decoding and a flag (weighted_pred_flag) indicating application of weighted prediction are included.
  • the picture PICT includes slices S0 to S NS-1 (NS is the total number of slices included in the picture PICT).
  • the coded slice In the coded slice, a set of data referred to by the image decoding device 31 for decoding the slice S to be processed is defined. As shown in FIG. 1C, the slice S includes a slice header SH and slice data SDATA.
  • the slice header SH includes an encoding parameter group that is referred to by the image decoding device 31 in order to determine a decoding method of the target slice.
  • Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.
  • I slice that uses only intra prediction at the time of encoding (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.
  • the slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the encoded video sequence.
  • the slice data SDATA includes a coding tree unit (CTU) as shown in FIG.
  • a CTU is a block of a fixed size (for example, 64x64) that constitutes a slice, and is sometimes called a maximum coding unit (LCU: Large Coding Unit).
  • Encoding tree unit As shown in (e) of FIG. 1, a set of data referred to by the image decoding device 31 in order to decode the encoding tree unit to be processed is defined.
  • the coding tree unit is divided by recursive quadtree division.
  • a tree-structured node obtained by recursive quadtree partitioning is referred to as a coding node (CN).
  • An intermediate node of the quadtree is an encoding node, and the encoding tree unit itself is defined as the highest encoding node.
  • the CTU includes a split flag (cu_split_flag), and when cu_split_flag is 1, it is split into four coding nodes CN.
  • the coding node CN is not divided and has one coding unit (CU: Coding Unit) as a node.
  • CU Coding Unit
  • the encoding unit CU is a terminal node of the encoding node and is not further divided.
  • the encoding unit CU is a basic unit of the encoding process.
  • the size of the coding tree unit CTU is 64 ⁇ 64 pixels
  • the size of the coding unit can be any of 64 ⁇ 64 pixels, 32 ⁇ 32 pixels, 16 ⁇ 16 pixels, and 8 ⁇ 8 pixels.
  • the encoding unit As shown in (f) of FIG. 1, a set of data referred to by the image decoding device 31 in order to decode an encoding unit to be processed is defined. Specifically, the encoding unit includes a prediction tree, a conversion tree, and a CU header CUH. In the CU header, a prediction mode, a division method (PU division mode), and the like are defined.
  • prediction information (a reference picture index, a motion vector, etc.) of each prediction unit (PU) obtained by dividing the coding unit into one or a plurality of parts is defined.
  • the prediction unit is one or a plurality of non-overlapping areas constituting the encoding unit.
  • the prediction tree includes one or a plurality of prediction units obtained by the above-described division.
  • a prediction unit obtained by further dividing the prediction unit is referred to as a “sub-block”.
  • the sub block is composed of a plurality of pixels. When the sizes of the prediction unit and the sub-block are equal, the number of sub-blocks in the prediction unit is one.
  • the prediction unit is divided into sub-blocks. For example, when the prediction unit is 8x8 and the sub-block is 4x4, the prediction unit is divided into four sub-blocks that are horizontally divided into two and vertically divided into two.
  • the prediction process may be performed for each prediction unit (sub block).
  • Intra prediction is prediction within the same picture
  • inter prediction refers to prediction processing performed between different pictures (for example, between display times or between layer images).
  • the division method is encoded by the PU division mode (part_mode) of encoded data, and 2Nx2N (the same size as the encoding unit), 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and NxN etc.
  • 2NxN and Nx2N indicate 1: 1 symmetrical division
  • 2NxnU, 2NxnD, nLx2N, and nRx2N indicate 1: 3 or 3: 1 asymmetric division.
  • the PUs included in the CU are expressed as PU0, PU1, PU2, and PU3 in this order.
  • FIG. 2 specifically illustrate the shape of the partition (the position of the boundary of the PU partition) in each PU partition mode.
  • 2A shows a 2Nx2N partition
  • FIGS. 2B, 2C, and 2D show 2NxN, 2NxnU, and 2NxnD partitions (horizontal partitions), respectively.
  • E), (f), and (g) show partitions (vertical partitions) in the case of Nx2N, nLx2N, and nRx2N, respectively, and (h) shows an NxN partition.
  • the horizontal and vertical partitions are collectively referred to as a rectangular partition
  • 2Nx2N and NxN are collectively referred to as a square partition.
  • the encoding unit is divided into one or a plurality of conversion units, and the position and size of each conversion unit are defined.
  • the transform unit is one or a plurality of non-overlapping areas constituting the encoding unit.
  • the conversion tree includes one or a plurality of conversion units obtained by the above division.
  • the division in the conversion tree includes a case where an area having the same size as that of the encoding unit is assigned as a conversion unit, and a case where recursive quadtree division is used, as in the case of the CU division described above.
  • Conversion processing is performed for each conversion unit.
  • the prediction parameter includes prediction list use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and motion vectors mvL0 and mvL1.
  • the prediction list use flags predFlagL0 and predFlagL1 are flags indicating whether or not reference picture lists called L0 list and L1 list are used, respectively, and a reference picture list corresponding to a value of 1 is used.
  • flag indicating whether or not it is XX when “flag indicating whether or not it is XX” is described, when the flag is other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. 1 is treated as true and 0 is treated as false (the same applies hereinafter).
  • flag when “flag indicating whether or not it is XX” is described, when the flag is other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. 1 is treated as true and 0 is treated as false (the same applies hereinafter).
  • other values can be used as true values and false values.
  • Syntax elements for deriving inter prediction parameters included in the encoded data include, for example, PU partition mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, And there is a difference vector mvdLX.
  • the reference picture list is a list including reference pictures stored in the reference picture memory 306.
  • FIG. 3 is a conceptual diagram illustrating an example of a reference picture and a reference picture list.
  • a rectangle is a picture
  • an arrow is a picture reference relationship
  • a horizontal axis is time
  • I, P and B in the rectangle are intra pictures
  • uni-prediction pictures and bi-prediction pictures respectively
  • numbers in the rectangles are decoded Show the order.
  • the decoding order of pictures is I0, P1, B2, B3, and B4
  • the display order is I0, B3, B2, B4, and P1.
  • FIG. 3B shows an example of the reference picture list.
  • the reference picture list is a list representing candidate reference pictures, and one picture (slice) may have one or more reference picture lists.
  • the target picture B3 has two reference picture lists, an L0 list RefPicList0 and an L1 list RefPicList1.
  • the reference pictures when the target picture is B3 are I0, P1, and B2, and the reference picture has these pictures as elements.
  • refIdxLX the reference picture index
  • the figure shows an example in which reference pictures P1 and B2 are referred to by refIdxL0 and refIdxL1.
  • the prediction parameter decoding (encoding) method includes a merge prediction (merge) mode and an AMVP (Adaptive Motion Vector Prediction) mode.
  • the merge flag merge_flag is a flag for identifying these.
  • the merge prediction mode is a mode in which the prediction list usage flag predFlagLX (or inter prediction identifier inter_pred_idc), the reference picture index refIdxLX, and the motion vector mvLX are not included in the encoded data and are derived from the prediction parameters of already processed neighboring PUs.
  • the AMVP mode is a mode in which the inter prediction identifier inter_pred_idc, the reference picture index refIdxLX, and the motion vector mvLX are included in the encoded data.
  • the motion vector mvLX is encoded as a prediction vector index mvp_LX_idx for identifying the prediction vector mvpLX and a difference vector mvdLX.
  • the inter prediction identifier inter_pred_idc is a value indicating the type and number of reference pictures, and takes one of PRED_L0, PRED_L1, and PRED_BI.
  • PRED_L0 and PRED_L1 indicate that reference pictures managed in the reference picture lists of the L0 list and the L1 list are used, respectively, and that one reference picture (first reference picture) is used (single prediction mode).
  • PRED_BI indicates that two reference pictures (first reference picture and second reference picture) are used (bi-prediction BiPred mode), and reference pictures managed in the L0 list and the L1 list are used.
  • the prediction vector index mvp_LX_idx is an index indicating a prediction vector
  • the reference picture index refIdxLX is an index indicating a reference picture managed in the reference picture list.
  • LX is a description method used when L0 prediction and L1 prediction are not distinguished from each other. By replacing LX with L0 or L1, parameters for the L0 list and parameters for the L1 list are distinguished.
  • the merge index merge_idx is an index indicating whether or not any prediction parameter is used as a prediction parameter of a decoding target PU among prediction parameter candidates (merge candidates) derived from a PU for which processing has been completed.
  • the motion vector mvLX indicates a shift amount between blocks on two different pictures.
  • a prediction vector and a difference vector related to the motion vector mvLX are referred to as a prediction vector mvpLX and a difference vector mvdLX, respectively.
  • Inter prediction identifier inter_pred_idc and prediction list use flag predFlagLX The relationship between the inter prediction identifier inter_pred_idc and the prediction list use flags predFlagL0 and predFlagL1 is as follows and can be converted into each other.
  • the flag biPred as to whether it is in the bi-prediction BiPred mode can be derived based on whether both of the two prediction list use flags are 1. For example, it can be derived by the following equation.
  • the flag biPred can also be derived depending on whether or not the inter prediction identifier is a value indicating that two prediction lists (reference pictures) are used. For example, it can be derived by the following equation.
  • FIG. 5 is a block diagram illustrating a configuration of the image decoding device 31 according to the first embodiment.
  • the image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, and an inverse quantization / inverse conversion unit. 311 and an adder 312.
  • the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304.
  • the predicted image generation unit 308 includes an inter predicted image generation unit 309 and an intra predicted image generation unit 310.
  • the entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, and separates and decodes individual codes (syntax elements).
  • the separated codes include prediction information for generating a prediction image, residual information for generating a difference image, and the like.
  • the entropy decoding unit 301 outputs a part of the separated code to the prediction parameter decoding unit 302.
  • Some of the separated codes are, for example, the prediction mode predMode, the PU partition mode part_mode, the merge flag merge_flag, the merge index merge_idx, the inter prediction identifier inter_pred_idc, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx, and the difference vector mvdLX. Control of which code is decoded is performed based on an instruction from the prediction parameter decoding unit 302.
  • the entropy decoding unit 301 outputs the quantized coefficient to the inverse quantization / inverse transform unit 311.
  • this quantized coefficient is obtained by performing DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT (Karyhnen Loeve Transform) in the encoding process.
  • DCT Discrete Cosine Transform
  • DST Discrete Sine Transform
  • KLT Karyhnen Loeve Transform
  • the inter prediction parameter decoding unit 303 decodes the inter prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301.
  • the inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307. Details of the inter prediction parameter decoding unit 303 will be described later.
  • the intra prediction parameter decoding unit 304 refers to the prediction parameter stored in the prediction parameter memory 307 on the basis of the code input from the entropy decoding unit 301 and decodes the intra prediction parameter.
  • the intra prediction parameter is a parameter used in the process of predicting a CU within one picture, for example, an intra prediction mode IntraPredMode.
  • the intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307.
  • the intra prediction parameter decoding unit 304 may derive different intra prediction modes for luminance and color difference.
  • the intra prediction parameter decoding unit 304 decodes the luminance prediction mode IntraPredModeY as the luminance prediction parameter and the color difference prediction mode IntraPredModeC as the color difference prediction parameter.
  • the luminance prediction mode IntraPredModeY is a 35 mode, and corresponds to planar prediction (0), DC prediction (1), and direction prediction (2 to 34).
  • the color difference prediction mode IntraPredModeC uses one of the planar prediction (0), the DC prediction (1), the direction prediction (2 to 34), and the LM mode (35).
  • the intra prediction parameter decoding unit 304 decodes a flag indicating whether IntraPredModeC is the same mode as the luminance mode.
  • the intra prediction parameter decoding unit 304 assigns IntraPredModeY to IntraPredModeC if it indicates that the flag is the same mode as the luminance mode. If the flag indicates that the flag is different from the luminance mode, the intra prediction parameter decoding unit 304 sets the Plane prediction (0), DC prediction (1), direction prediction (2 to 34), and LM mode (35) as IntraPredModeC. May be decrypted.
  • the loop filter 305 applies filters such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adding unit 312.
  • filters such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adding unit 312.
  • the reference picture memory 306 stores the decoded image of the CU generated by the adding unit 312 at a predetermined position for each decoding target picture and CU.
  • the prediction parameter memory 307 stores the prediction parameter in a predetermined position for each decoding target picture and prediction unit (or sub-block, fixed-size block or pixel). Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode predMode separated by the entropy decoding unit 301. .
  • the stored inter prediction parameters include, for example, a prediction list utilization flag predFlagLX (inter prediction identifier inter_pred_idc), a reference picture index refIdxLX, and a motion vector mvLX.
  • the prediction image generation unit 308 receives the prediction mode predMode input from the entropy decoding unit 301 and the prediction parameter from the prediction parameter decoding unit 302. Further, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a PU or sub-block using the input prediction parameter and the read reference picture (reference picture block) according to the prediction mode indicated by the prediction mode predMode.
  • the inter prediction image generation unit 309 uses the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the read reference picture (reference picture block). To generate a prediction image of the PU or sub-block.
  • the inter predicted image generation unit 309 performs the motion vector mvLX with reference to the decoding target PU from the reference picture indicated by the reference picture index refIdxLX.
  • the reference picture block at the position indicated by is read from the reference picture memory 306.
  • the inter prediction image generation unit 309 performs prediction based on the read reference picture block to generate a prediction image of the PU.
  • the inter prediction image generation unit 309 outputs the generated prediction image of the PU to the addition unit 312.
  • the reference picture block is a set of pixels on the reference picture (usually called a block because it is a rectangle), and is an area that is referred to in order to generate a predicted image of a PU or sub-block.
  • the intra predicted image generation unit 310 When the prediction mode predMode indicates the intra prediction mode, the intra predicted image generation unit 310 performs intra prediction using the intra prediction parameter input from the intra prediction parameter decoding unit 304 and the read reference picture. Specifically, the intra predicted image generation unit 310 reads, from the reference picture memory 306, neighboring PUs that are pictures to be decoded and are in a predetermined range from the decoding target PUs among the PUs that have already been decoded.
  • the predetermined range is, for example, one of the left, upper left, upper and upper right adjacent PUs when the decoding target PU sequentially moves in the so-called raster scan order, and varies depending on the intra prediction mode.
  • the raster scan order is an order in which each row is sequentially moved from the left end to the right end in each picture from the upper end to the lower end.
  • the intra prediction image generation unit 310 generates a prediction image of the PU by performing prediction based on the prediction mode indicated by the intra prediction mode IntraPredMode based on the read adjacent PU.
  • the intra predicted image generation unit 310 outputs the generated predicted image of the PU to the adding unit 312.
  • the intra prediction image generation unit 310 When the intra prediction parameter decoding unit 304 derives different intra prediction modes for luminance and chrominance, the intra prediction image generation unit 310 performs planar prediction (0), DC prediction (1) and DC prediction according to the luminance prediction mode IntraPredModeY. A prediction image of the luminance PU is generated by any one of the direction predictions (2 to 34). The intra-prediction image generation unit 310 predicts a color difference PU according to any of the planar prediction (0), DC prediction (1), direction prediction (2 to 34), and LM mode (35) according to the color difference prediction mode IntraPredModeC. Generate an image.
  • the inverse quantization / inverse transform unit 311 inversely quantizes the quantized coefficient input from the entropy decoding unit 301 to obtain a transform coefficient.
  • the inverse quantization / inverse transform unit 311 performs inverse frequency transform such as inverse DCT, inverse DST, and inverse KLT on the obtained transform coefficient, and calculates a residual signal.
  • the inverse quantization / inverse transform unit 311 outputs the calculated residual signal to the adder 312.
  • the addition unit 312 adds the prediction image of the PU input from the inter prediction image generation unit 309 or the intra prediction image generation unit 310 and the residual signal input from the inverse quantization / inverse conversion unit 311 for each pixel, Generate a decoded PU image.
  • the adding unit 312 stores the generated decoded image of the PU in the reference picture memory 306, and outputs a decoded image Td in which the generated decoded image of the PU is integrated for each picture to the outside.
  • FIG. 11 is a block diagram illustrating a configuration of the inter prediction parameter decoding unit 303 according to the first embodiment.
  • the inter prediction parameter decoding unit 303 includes an inter prediction parameter decoding control unit 3031, an AMVP prediction parameter derivation unit 3032, an addition unit 3038, a merge prediction parameter derivation unit 3036, and a sub-block prediction parameter derivation unit 3037.
  • the inter prediction parameter decoding control unit 3031 instructs the entropy decoding unit 301 to decode a code (syntax element) related to inter prediction. Also, the inter prediction parameter decoding control unit 3031 includes codes (syntax elements) included in the encoded data, for example, PU partition mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction Vector index mvp_LX_idx or difference vector mvdLX is extracted.
  • the inter prediction parameter decoding control unit 3031 first extracts a merge flag merge_flag.
  • the inter prediction parameter decoding control unit 3031 expresses that a certain syntax element is to be extracted, it means that the entropy decoding unit 301 is instructed to decode a certain syntax element, and the corresponding syntax element is read from the encoded data. To do.
  • the inter prediction parameter decoding control unit 3031 uses the entropy decoding unit 301 to extract the AMVP prediction parameter from the encoded data.
  • AMVP prediction parameters include an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference vector mvdLX.
  • the AMVP prediction parameter derivation unit 3032 derives a prediction vector mvpLX from the prediction vector index mvp_LX_idx. Details will be described later.
  • the inter prediction parameter decoding control unit 3031 outputs the difference vector mvdLX to the addition unit 3038.
  • the adding unit 3038 adds the prediction vector mvpLX and the difference vector mvdLX to derive a motion vector.
  • the inter prediction parameter decoding control unit 3031 extracts the merge index merge_idx as a prediction parameter related to merge prediction.
  • the inter prediction parameter decoding control unit 3031 outputs the extracted merge index merge_idx to the merge prediction parameter derivation unit 3036 (details will be described later), and outputs the sub-block prediction mode flag subPbMotionFlag to the sub-block prediction parameter derivation unit 3037.
  • the subblock prediction parameter deriving unit 3037 divides the PU into a plurality of subblocks according to the value of the subblock prediction mode flag subPbMotionFlag, and derives a motion vector in units of subblocks.
  • sub-block prediction is used for a method in which a CU is divided into a plurality of partitions (PUs such as 2NxN, Nx2N, and NxN) and the syntax of a prediction parameter is encoded for each partition unit.
  • PUs such as 2NxN, Nx2N, and NxN
  • the syntax of the prediction parameter is encoded for each set. Therefore, motion information of a large number of sub-blocks can be encoded with a small code amount.
  • FIG. 7 is a block diagram illustrating a configuration of the merge prediction parameter deriving unit 3036 according to the first embodiment.
  • the merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361, a merge candidate selection unit 30362, and a merge candidate storage unit 30363.
  • the merge candidate storage unit 30363 stores the merge candidates input from the merge candidate derivation unit 30361.
  • the merge candidate includes a prediction list use flag predFlagLX, a motion vector mvLX, and a reference picture index refIdxLX.
  • an index is assigned to the stored merge candidate according to a predetermined rule.
  • the merge candidate derivation unit 30361 derives a merge candidate using the motion vector of the adjacent PU that has already been decoded and the reference picture index refIdxLX as they are.
  • merge candidates may be derived using affine prediction. This method will be described in detail below.
  • the merge candidate derivation unit 30361 may use affine prediction for a spatial merge candidate derivation process, a temporal merge candidate derivation process, a combined merge candidate derivation process, and a zero merge candidate derivation process described later. Note that affine prediction is performed for each subblock, and the prediction parameters are stored in the prediction parameter memory 307 for each subblock. Alternatively, the affine prediction may be performed for each pixel unit.
  • the merge candidate derivation unit 30361 reads and reads the prediction parameters (prediction list use flag predFlagLX, motion vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule.
  • the predicted parameters are derived as merge candidates.
  • the prediction parameter to be read is a prediction parameter related to each of the PUs within a predetermined range from the decoding target PU (for example, all or part of the PUs in contact with the lower left end, the upper left end, and the upper right end of the decoding target PU, respectively). is there.
  • the merge candidate derivation unit 30361 reads the prediction parameter of the PU in the reference image including the lower right coordinate of the decoding target PU from the prediction parameter memory 307 and sets it as a merge candidate.
  • the reference picture designation method may be, for example, the reference picture index refIdxLX designated in the slice header, or may be designated using the smallest reference picture index refIdxLX of the PU adjacent to the decoding target PU.
  • the merge candidate derivation unit 30361 uses two different derived merge candidate motion vectors and reference picture indexes already derived and stored in the merge candidate storage unit 30363 as the motion vectors of L0 and L1, respectively. Combined merge candidates are derived by combining them.
  • the merge candidate derivation unit 30361 derives a merge candidate in which the reference picture index refIdxLX is 0 and both the X component and the Y component of the motion vector mvLX are 0.
  • the merge candidates derived by the merge candidate deriving unit 30361 are stored in the merge candidate storage unit 30363.
  • the merge candidate selection unit 30362 selects, from the merge candidates stored in the merge candidate storage unit 30363, a merge candidate to which an index corresponding to the merge index merge_idx input from the inter prediction parameter decoding control unit 3031 is assigned. As an inter prediction parameter.
  • the merge candidate selection unit 30362 stores the selected merge candidate in the prediction parameter memory 307 and outputs it to the prediction image generation unit 308.
  • FIG. 8 is a block diagram illustrating a configuration of the AMVP prediction parameter derivation unit 3032 according to the first embodiment.
  • the AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit 3033, a vector candidate selection unit 3034, and a vector candidate storage unit 3035.
  • the vector candidate derivation unit 3033 derives a prediction vector candidate from the already processed PU motion vector mvLX stored in the prediction parameter memory 307 based on the reference picture index refIdx.
  • the vector candidate derivation unit 3033 stores the derived prediction vector candidate in the vector candidate storage unit 3035 prediction vector candidate list mvpListLX [].
  • the vector candidate selection unit 3034 selects the motion vector mvpListLX [mvp_LX_idx] indicated by the prediction vector index mvp_LX_idx from the prediction vector candidates in the prediction vector candidate list mvpListLX [] as the prediction vector mvpLX.
  • the vector candidate selection unit 3034 outputs the selected prediction vector mvpLX to the addition unit 3038.
  • a prediction vector candidate is a PU for which decoding processing has been completed, and is derived by scaling a motion vector of a PU (for example, an adjacent PU) within a predetermined range from the decoding target PU.
  • the adjacent PU includes a PU that is spatially adjacent to the decoding target PU, for example, the left PU and the upper PU, and an area that is temporally adjacent to the decoding target PU, for example, the same position as the decoding target PU. It includes areas obtained from prediction parameters of PUs with different times.
  • the addition unit 3038 adds the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 3032 and the difference vector mvdLX input from the inter prediction parameter decoding control unit 3031 to calculate a motion vector mvLX.
  • the adding unit 3038 outputs the calculated motion vector mvLX to the predicted image generation unit 308 and the prediction parameter memory 307.
  • FIG. 10 is a block diagram illustrating a configuration of the inter predicted image generation unit 309 included in the predicted image generation unit 308 according to the first embodiment.
  • the inter prediction image generation unit 309 includes a motion compensation unit 3091 and a weight prediction unit 3094.
  • the motion compensation unit 3091 receives the reference picture index refIdxLX from the reference picture memory 306 based on the inter prediction parameters (prediction list use flag predFlagLX, reference picture index refIdxLX, and motion vector mvLX) input from the inter prediction parameter decoding unit 303.
  • the reference picture RefX any one of the reference pictures included in the reference picture list specified in (1)
  • an interpolation picture by reading a reference block at a position shifted by the motion vector mvLX starting from the position of the decoding target PU
  • Motion compensated image predSamplesLX is generated.
  • a motion compensation image is generated by applying a filter for generating a pixel at a decimal position called a motion compensation filter.
  • the motion compensation unit 3091 generates a prediction image by referring to the single prediction mode, the bi-prediction mode, the two reference images (first reference image and second reference image), and the gradient correction term.
  • a predicted image is generated using at least one of the BIO mode using prediction.
  • the motion compensation unit 3091 also determines that the reference block in at least one of the first reference image and the second reference image is outside the screen of the reference image (in other words, the reference block in the first reference image is the first block).
  • Generation of a predicted image using the BIO mode is prohibited when the reference image is outside the screen and / or the reference block in the second reference image is outside the second reference image). To do. That is, in the above case, the motion compensation unit 3091 does not perform prediction image generation using the BIO mode. Details of processing executed by the motion compensation unit 3091 will be described later.
  • the reference images referred to by the single prediction mode, the bi-prediction mode, and the BIO mode may be the same or different.
  • the weight prediction unit 3094 generates a prediction image of the PU by multiplying the input motion compensation image predSamplesLX by a weight coefficient.
  • FIG. 12 is a flowchart for explaining the flow of processing in which the motion compensation unit 3091 having a motion compensation function using gradient change (BIO) prediction derives a predicted image.
  • BIO gradient change
  • step S101 When it is determined that the inter prediction parameter decoding unit 303 is not in the bi-prediction mode (No in step S101, that is, uni-prediction UniPred), the process proceeds to step S105, and the motion compensation unit 3091 performs unidirectional motion compensation.
  • the inter prediction parameter decoding unit 303 when it is determined that the inter prediction parameter decoding unit 303 is in the bi-prediction mode (Yes in step S101, BiPred), the inter prediction parameter decoding unit 303 performs the determination shown in step S102.
  • step S102 If the L0 reference image refImgL0 and the L1 reference image refImgL1 (described as “two prediction blocks” in FIG. 12) acquired from the reference picture memory 306 are different reference images (Yes in step S102), the process proceeds to step S103.
  • the motion compensation unit 3091 performs motion compensation using BIO prediction described later.
  • the L0 reference image refImgL0 and the L1 reference image refImgL1 acquired from the reference picture memory 306 are the same reference image (No in step S102)
  • the process proceeds to step S104, and the motion compensation unit 3091 does not apply BIO prediction. Perform motion compensation.
  • FIG. 17 is a block diagram illustrating an example of the configuration of the motion compensation unit 3091.
  • the motion compensation unit 3091 includes a motion compensation gradient unit 30911 and a gradient correction coefficient unit 30912.
  • the motion compensation gradient unit 30911 includes a motion compensation derivation unit 309111 and a gradient derivation unit 309112
  • the gradient correction coefficient unit 30912 includes a gradient product derivation unit 309121 and a gradient correction coefficient derivation unit 309122.
  • the motion compensation unit 3091 applies bi-directional optical flow (bi-predictive gradient change: BIO) prediction that performs motion correction by applying a gradient correction term to a bi-prediction (BiPred) image. I do.
  • BIO bi-directional optical flow
  • This is a gradient correction term derived by Expression (A3).
  • lx0 (first gradient image), ly0 (second gradient image), lx1 (third gradient image), and ly1 (fourth gradient image) are gradient images.
  • the gradient images lx0 and lx1 show the gradient along the horizontal direction (x direction or first direction), and the gradient images ly0 and ly1 show the gradient along the vertical direction (y direction or second direction).
  • U and v are correction weight vectors.
  • the motion compensation unit 3091 generates a prediction image (inter prediction image) with reference to the motion compensation image P0 (first reference image), the motion compensation image P1 (second reference image), and the gradient correction term.
  • the processing flow will be described with reference to FIG. FIG. 18 is a flowchart showing the flow of processing in motion compensation using BIO prediction.
  • the motion compensation unit 3091 performs the following three steps of STEP 111 to 113 to derive a predicted image.
  • the motion compensation deriving unit 309111 reads out the L0 reference image refImgL0 (first reference image) and the L1 reference image refImgL1 (second reference image) to be used as the reference image from the reference picture memory 306, and the motion compensation image.
  • P0 and P1 are derived (S111a in FIG. 18).
  • the gradient deriving unit 309112 derives horizontal gradient images lx0 and lx1 and vertical gradient images ly0 and ly1 for the motion compensated images P0 and P1 derived by the motion compensation deriving unit 309111 (S111b in FIG. 18).
  • refImgL0, refImgL1, P0, P1, lx0, lx1, ly0, and ly1 are two-dimensional reference images refImgL01 [x] [y], refImgL1 [x] [y], and motion compensation image P0 [x] [ y], P1 [x] [y], gradient image lx0 [x] [y], lx1 [x] [y], ly0 [x] [y] and ly1 [x] [y] (x and y are predetermined Or a reference image value refImgL0, refImgL1, motion compensation image value P0, P1, gradient values lx0, lx1, ly0, and ly1.
  • the gradient product deriving unit 309121 derives the gradient product (S112a in FIG. 18).
  • the gradient correction coefficient deriving unit 309122 uses the gradient product derived by the gradient product deriving unit 309121 to derive a correction weight vector (u, v) (gradient correction coefficient, uh and vh in integer arithmetic) (see FIG. 18 S112b).
  • the gradient correction coefficient deriving unit 309122 may derive the correction weight vector (u, v) using Equation 5 in FIG. 13 or may be derived using the least square method as described later.
  • the motion compensation unit 3091 has gradient changes dI / dx (ie, lx) and dI / dy (ie, ly) at a plurality of points. ) And time variation dI / dt, u and v satisfying the condition that minimizes the square of the left side of Equation 5 in FIG. 13 may be derived.
  • the pixel value I of the point I on the target image Cur As shown in FIG. 14, the pixel value I of the point I on the target image Cur, the pixel value I0 of the point 10 (lx0, ly0) on the L0 reference image refImgL0, and the point l1 (lx1, L1 on the L1 reference image refImgL1) This corresponds to the pixel value I1 of ly1).
  • the gradient is (lx0, ly0), and the distance between IO and P0 is (Vx, Vy).
  • the distance (Vx, Vy) correction weight vector (u, v) Can be derived.
  • the motion compensation unit 3091 uses the pixel value I of the point I on the target image Cur, which is the prediction target image, as the pixel value l0 of the point 10 on the L0 reference image refImgL0 and the pixel value l1 of the point 11 on the L1 reference image refImgL1.
  • Average of I (l0 + l1) >> 1
  • Forma A4-1) ⁇ P0 + P1 + (lx0-lx1) * u + (ly0-ly1) * v ⁇ >> 1 (Formula A4-2)
  • l0 P0 + (lx0 * u + ly0 * v)
  • l1 P1-(lx1 * u + ly1 * v) It is.
  • the correction weight vector is (u, v) that minimizes Formula A5 from the assumption that the pixel value does not change.
  • ⁇ l0-l1 ⁇ 2 ⁇ (P0-P1) + (lx0 + lx1) * u + (ly0 + ly1) * v ⁇ 2 (Formula A5)
  • the gradient correction bi-prediction derivation unit 30913 derives the gradient correction term modBIO [] [] using the correction weight vector (u, v) derived in STEP 112 (see Equation A3 and Equation A19 described later). (S113a in FIG. 18). Then, the gradient correction bi-prediction derivation unit 30913 derives the pixel value Pred of the gradient correction bi-prediction image (prediction image, corrected prediction image) using Expression A2 (S113b in FIG. 18).
  • the motion compensation unit 3091 derives (generates) a motion compensation image and a gradient image will be described as an example, but the motion compensation unit 3091 derives a pixel value included in the image, not an image. It may be a configuration. That is, the motion compensation unit 3091 may derive the pixel value of the motion compensation image and the pixel value of the gradient image.
  • the motion compensation derivation unit 309111 derives motion compensation images P0 and P1 (also referred to as basic motion compensation images) based on the L0 reference image refImgL0 and the L1 reference image refImgL1 (S111a in FIG. 18). Further, the gradient deriving unit 309112 derives horizontal gradient images lx0 and lx1 and vertical gradient images ly0 and ly1 for the derived motion compensation images P0 and P1 (S111b in FIG. 18).
  • the motion compensation deriving unit 309111 applies a vertical motion compensation filter (mcFilterVer) to the reference image. Then, the motion compensation images P0 and P1 are derived by further applying a horizontal motion compensation filter (mcFilterHor) to the reference image to which the vertical motion compensation filter is applied.
  • mcFilterVer vertical motion compensation filter
  • mcFilterHor horizontal motion compensation filter
  • the accuracy of the motion vector is assumed to be 1/4 pel accuracy, but the motion vector accuracy is not limited to this, and may be 1/8, 1/16,.
  • the shift value for deriving the integer positions xInt, yInt is log2 (M) and the logical sum for deriving the phases xFrac, yFrac (&
  • the following formula A8 may be used in which the value used for () is M-1.
  • bit Depth of the filter coefficient is, for example, MC_ACCU
  • bitDepthY + MC_ACCU which is a value obtained by adding MC_ACCU to the bit depth bitDepthY of the reference image.
  • a right shift of only MC_ACCU is required as a right shift for adjusting the dynamic range.
  • the two filters are serial processing, it is appropriate to use the intermediate bit depth INTERNAL_BIT_DEPTH higher than bitDepth in the output of the first filter operation and return from INTERNAL_BIT_DEPTH to bitDepth in the output of the second filter operation. It is.
  • the shift values shift1 and shift2 for adjusting the dynamic range of the first and second filters may be set as follows.
  • bitDepthY represents the bit depth of the reference image
  • INTERNAL_BIT_DEPTH represents the intermediate bit depth
  • MC_ACCU represents the accuracy of the motion compensation filter mcFilter. Note that MC_ACCU is not limited to 6, and a value such as 3 to 10 can be used.
  • bitDepth indicates the bit depth of the temporary image temp [] [].
  • the bit depth of the temporary image temp [] [] is a value obtained by subtracting the right shift value shift1 in the filter processing from the sum of the bit depth bitDepthY of the reference image and the accuracy MC_ACCU of the motion compensation filter. This value is referred to herein as the intermediate bit depth INTERNAL_BIT DEPTH.
  • the product sum of the temporary image temp [] [] derived by the compensation filter mcFilterVer is used as the value obtained by adjusting the range by the shift value shift2.
  • the horizontal gradient image lxX is obtained by the gradient derivation unit 309112 applying the horizontal gradient filter gradFilterHor after applying the vertical motion compensation filter mcFilterVer.
  • the gradient deriving unit 309112 applies the horizontal gradient filter gradFilterHor to the temporary image temp [] [] derived by the above equation A10 to derive the horizontal gradient image lxX as in the following equation A12.
  • x 0..BLKW-1
  • y 0..BLKH-1
  • k 0..NTAPS-1
  • offset2 1 ⁇ (shift2-1)
  • lxX is lx0 or lx1.
  • bitDepth (lxX [] []) indicates the bit depth of the filtered image lxX [] [].
  • GRAD_ACCU represents the accuracy of the gradient filter gradFilter (gradFilterHor or gradFilterVer).
  • the gradient derivation unit 309112 applies the vertical gradient filter (gradFilterVer) with reference to the L0 reference image refImgL0 and the L1 reference image refImgL1, and then applies the horizontal motion compensation filter (mcFilterHor) to thereby obtain the vertical gradient image ly0 and Derive ly1.
  • vertical gradient filter gradeFilterVer
  • horizontal motion compensation filter mcFilterHor
  • a vertical gradient image lyX is derived by applying a horizontal motion compensation filter mcFilterHor to this temp [x] [y].
  • the gradient product deriving unit 309121 may derive the correction weight vector (u, v) that minimizes the value obtained by the above equation A5 using the least square method.
  • the gradient correction coefficient deriving unit 309122 is, for example, an integer value obtained by shifting u to the left by shift BIO bits instead of the correction weight vector (u, v) that actually requires calculation of decimal point precision.
  • shiftBIO 5
  • the precision is 1/32.
  • s2 s4
  • s4 may be used instead of s2 in the above formula A15.
  • the gradient correction bi-prediction derivation unit 30913 derives the gradient correction term modBIO by substituting the correction weight vector (u, v) derived in STEP 112 into the equation A3, and the pixel value Pred of the predicted image is calculated using the equation A6. To derive.
  • the slope correction term modBIO may be weakened to 1/2 and used as in the above equation A7.
  • the pixel value Pred of the predicted image is derived using the prediction formula shown in the above formula A2.
  • shiftBIO2 shiftBIO + bitDepth (mcImg)-bitDepth (gradImg) + 1 (Formula A18) It is.
  • bitDepth (mcImg) is the bit depth of the motion compensated image
  • bitDepth (gradImg) is the bit depth of the gradient image
  • FIG. 19 is a diagram illustrating an area where the motion compensation unit 3091 performs BIO padding.
  • FIG. 20 is a diagram illustrating an example when the motion compensation unit 3091 performs off-screen padding.
  • “padding” refers to a process of generating a pixel in an area without a pixel.
  • a method is used in which pixels at the boundary between the reading area and the reference image are copied vertically and horizontally to generate pixels outside the boundary.
  • off-screen padding generating a pixel outside the boundary of the reference image
  • BIO generating a pixel outside the boundary of the reading area (MC reading area) necessary for motion compensation processing
  • N in FIG. 19 indicates the width and height of a corresponding block that is a block at the same position as the encoding target block in the reference image.
  • T is the number of filter taps of the motion compensation filter, and is the number of pixels on which the motion compensation filter is applied to the reference image.
  • the pixel area of (N + T-1) * (N + T-1) indicates an MC reading area necessary for motion compensation.
  • outside of the reference image means the outside of the reference image. Specifically, the region outside the reference image width pic_width and height pic_hight shown in FIG. 20 vertical line areas). Further, the case where the above-described reference block is outside the screen of the reference image can also be expressed as a case where at least a part of the reference block indicated by the motion vector protrudes beyond the boundary of the reference image, as shown in FIG. it can.
  • the width and height of the corresponding block are both N will be described, but the width and height may be different values.
  • the motion compensation unit 3091 is the MC reading area in FIG. N + T-1) * (N + T-1) pixels are read.
  • the motion compensation unit 3091 when generating a gradient image using the BIO mode, the motion compensation unit 3091 obtains an MC reading area (N + T ⁇ 1) * (N +) in order to obtain a weight coefficient (gradient correction coefficient) of the corresponding block. Read pixel of T-1).
  • the diagonal lines in FIG. Partial area pixels are required. For this reason, as shown in FIG.
  • the motion compensation unit 3091 generates padding pixels using the pixels at the boundary of the MC reading area in the portion that protrudes outside the MC reading area, which is the area indicated by the hatched portion of the reference block (BIO Execute padding) to generate a gradient image.
  • the motion compensation unit 3091 performs the BIO padding on the region of the reference block that protrudes outside the reading region. I do.
  • the motion compensation unit 3091 prohibits the BIO mode when a part of the reference block is outside the screen of the reference image. That is, when a part of the reference block is outside the screen of the reference image, the BIO mode is not performed, and thus the motion compensation unit 3091 does not need to perform both the off-screen padding and the BIO padding.
  • FIG. 21 is a flowchart illustrating an example of a flow of processing for determining whether or not the motion compensation unit 3091 executes (permits) generation of a predicted image using the BIO mode.
  • the motion compensation unit 3091 determines whether at least a part of the reference block is outside the screen of the reference image. When the motion compensation unit 3091 determines that at least a part of the reference block is outside the screen (YES in STEP 121), the motion compensation unit 3091 prohibits generation of a predicted image using the BIO mode (STEP 123). In this case, in the processing following FIG. 21, the motion compensation unit 3091 reads the pixels in the MC reading area (N + T-1) * (N + T-1) as shown in FIG. Alternatively, a prediction image is generated in the bi-prediction mode, and the process ends.
  • the motion compensation unit 3091 permits the BIO mode.
  • the motion compensation unit 3091 stores the MC reading area (N + T ⁇ 1) * (N + T ⁇ 1) as shown in FIG. Pixels are read and a motion compensated image is generated (S111a in FIG. 18).
  • the motion compensation unit 3091 reads pixels in the padding area (shaded area) outside the MC reading area, executes S111b to S113b in FIG. 18, generates a predicted image in the BIO mode, and ends the process.
  • the motion compensation unit 3091 determines whether at least a part of the reference block is outside the screen by determining the following (1) to (4). Also good. (1) Whether the upper left and lower left x coordinates of the reference block are less than the x coordinate of the pixel at the left boundary of the reference image. (2) The upper right and lower right x coordinates of the reference block are the right boundary of the reference image.
  • the motion compensation unit 3091 prohibits the BIO mode if at least a part of the reference block indicated by the motion vector protrudes from the boundary of the reference image, but in the present embodiment, the motion compensation unit 3091 is limited to this. Not.
  • FIG. 22 is a flowchart illustrating another example of a flow of processing for determining whether or not the motion compensation unit 3091 executes (permits) prediction image generation using the BIO mode.
  • the motion compensation unit 3091 determines whether or not at least a part of the reference block is outside a certain range from the boundary of the reference image.
  • the motion compensation unit 3091 when a part of the reference block indicated by the motion vector protrudes from the boundary of the reference image, the motion compensation unit 3091 has a length outW that the reference block protrudes from the reference image in the horizontal direction, The length outH protruding in the vertical direction is measured. Subsequently, the motion compensation unit 3091 compares the length outW that protrudes in the horizontal direction with padW that indicates the length in the horizontal direction within a predetermined range. In addition, the motion compensation unit 3091 compares the length outH protruding in the vertical direction with padH indicating the length in a predetermined range in the vertical direction.
  • the motion compensation unit 3091 is: It is determined that at least a part of the reference block is outside a certain range from the boundary of the reference image.
  • the motion compensation unit 3091 determines the following (1) to (4), so that at least a part of the reference block is outside a certain range from the boundary of the reference image. It may be determined whether or not there is. (1) Whether the upper left and lower left x coordinates of the reference block are less than the x coordinate of a pixel outside a certain range from the boundary of the reference image (2) the upper right and lower right x coordinates of the reference block are Whether or not the x coordinate of the pixel outside the certain range from the boundary of the reference image is larger than (3) the y coordinate of the upper left coordinate and the upper right coordinate of the reference block is y of the pixel outside the certain range from the boundary of the reference image Whether or not it is less than the coordinates (4) Whether or not the y coordinate of the lower left coordinate and the lower right coordinate of the reference block is larger than the y coordinate of a pixel outside a certain range from the boundary of the reference image At least a part of the reference block When the motion compensation unit
  • the motion compensation unit 3091 does not execute the BIO mode (STEP 133). In this case, the motion compensation unit 3091 generates a prediction image in the uni-prediction mode or the bi-prediction mode, as in the first specific example, and ends the process.
  • the motion compensation unit 3091 generates a predicted image using the BIO mode in the same manner as in the first specific example, and ends the process.
  • the motion compensation unit 3091 may prohibit the BIO mode when at least a part of the reference block is outside a certain range from the boundary of the reference image. That is, the motion compensation unit 3091 may not execute the BIO mode.
  • the motion compensation unit 3091 when at least a part of the reference block protrudes from the boundary of the reference image but does not protrude outside the fixed range, the motion compensation unit 3091 generates a highly accurate predicted image using the BIO mode. can do. Further, when at least a part of the reference block is outside a certain range from the boundary of the reference image, the motion compensation unit 3091 prohibits the BIO mode. For this reason, if the motion compensation unit 3091 generates a padding area in advance for an out-of-screen area that is outside a certain range from the boundary of the reference image, the motion compensation unit 3091 reduces the padding to one type (when the BIO mode is on, Only). In other words, the motion compensation unit 3091 derives an enlarged reference image in which off-screen padding is performed in advance, so that the BIO is prohibited in advance when it is outside a certain range from the boundary of the reference image. Therefore, BIO processing can be simplified.
  • the motion compensation unit 3091 can refer to an off-screen region that is outside a certain range from the boundary of the reference image. In this case, only one padding ( When the BIO mode is off, only off-screen padding can be limited.
  • the motion compensation unit 3091 determines that the reference block is a reference image when at least one of the following expressions A20-1 to A20-4 is satisfied. It is determined that it falls outside the screen.
  • the following equation A20-1 is an equation used when determining whether or not the left end of the reference block indicated by the motion vector is outside the screen of the reference image.
  • Expression A20-2 is an expression used when determining whether the right end of the reference block is outside the screen of the reference image.
  • Expression A20-3 is an expression used when determining whether or not the upper end of the reference block is outside the screen of the reference image.
  • Expression A20-4 is an expression used when determining whether or not the lower end of the reference block is outside the screen of the reference image.
  • xInt-NTAPS / 2 + 1 ⁇ 0 (Formula A20-1) xInt + BLKW + NTAPS / 2-1> pic_width-1 (Formula A20-2) yInt-NTAPS / 2 + 1 ⁇ 0 (Formula A20-3) yInt + BLKH + NTAPS / 2-1> pic_height-1 (Formula A20-4)
  • xInt and yInt in the equations A20-1 to A20-4 are the upper left block coordinates (xPb, yPb) when the motion vector accuracy is 1 / M pel accuracy, as shown in the above equation A8.
  • NTAPS in Expressions A20-1 to A20-4 indicates the number of filter taps of the motion compensation filter.
  • BLKW and BLKH in the expressions A20-1 to A20-4 indicate the horizontal width and height of the corresponding block, respectively.
  • pic_width and pic_height in the expressions A20-1 to A20-4 indicate the horizontal width and height of the reference image, respectively.
  • NTAPS corresponds to T
  • BLKW and BLKH each correspond to N.
  • the motion compensation unit 3091 may determine whether the reference block is outside a certain range from the boundary of the reference image, instead of determining whether the reference block is outside the screen of the reference image.
  • the motion compensation unit 3091 in the determination that “at least a part of the reference block is out of a certain range from the screen boundary”, the motion compensation unit 3091 satisfies at least one of the following formulas A21-1 to 21-4: In this case, it may be determined that at least a part of the reference block is outside a certain range from the boundary of the reference image.
  • the following expression A21-1 is an expression used when determining whether or not the left end of the reference block indicated by the motion vector is outside a certain range from the boundary of the reference image.
  • Expression A21-2 is an expression used when determining whether or not the right end of the reference block is outside a certain range from the boundary of the reference image.
  • Expression A21-3 is an expression used when determining whether or not the upper end of the reference block is outside a certain range from the boundary of the reference image.
  • Expression A21-4 is an expression used when determining whether or not the lower end of the reference block is outside a certain range from the boundary of the reference image.
  • padW and padH in formulas A21-1 to A21-4 mean padW indicating a predetermined range in the horizontal direction and padH indicating a predetermined range in the vertical direction, respectively. .
  • FIG. 4 is a block diagram illustrating a configuration of the image encoding device 11 according to the first embodiment.
  • the image encoding device 11 includes a predicted image generation unit 101, a subtraction unit 102, a transform / quantization unit 103, an entropy encoding unit 104, an inverse quantization / inverse transform unit 105, an addition unit 106, a loop filter 107, and a prediction parameter memory. 108, a reference picture memory 109, an encoding parameter determination unit 110, and a prediction parameter encoding unit 111.
  • the prediction parameter encoding unit 111 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit 113.
  • the predicted image generation unit 101 generates, for each picture of the image T, a predicted image P of the prediction unit PU for each coding unit CU that is an area obtained by dividing the picture.
  • the predicted image generation unit 101 reads a decoded block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter encoding unit 111.
  • the prediction parameter input from the prediction parameter encoding unit 111 is, for example, a motion vector in the case of inter prediction.
  • the predicted image generation unit 101 reads a block at a position on the reference image indicated by the motion vector with the target PU as a starting point.
  • the prediction parameter is, for example, an intra prediction mode.
  • a pixel value of an adjacent PU used in the intra prediction mode is read from the reference picture memory 109, and a predicted image P of the PU is generated.
  • the predicted image generation unit 101 generates a predicted image P of the PU using one prediction method among a plurality of prediction methods for the read reference picture block.
  • the predicted image generation unit 101 outputs the generated predicted image P of the PU to the subtraction unit 102.
  • FIG. 6 is a block diagram illustrating a configuration of the inter predicted image generation unit 1011 included in the predicted image generation unit 101.
  • the inter prediction image generation unit 1011 includes a motion compensation unit 10111 and a weight prediction unit 10112. Since the motion compensation unit 10111 and the weight prediction unit 10112 have the same configurations as the motion compensation unit 3091 and the weight prediction unit 3094 described above, description thereof is omitted here.
  • the prediction image generation unit 101 generates a prediction image P of the PU based on the pixel value of the reference block read from the reference picture memory, using the parameter input from the prediction parameter encoding unit.
  • the predicted image generated by the predicted image generation unit 101 is output to the subtraction unit 102 and the addition unit 106.
  • the subtraction unit 102 subtracts the signal value of the predicted image P of the PU input from the predicted image generation unit 101 from the pixel value of the corresponding PU of the image T, and generates a residual signal.
  • the subtraction unit 102 outputs the generated residual signal to the transform / quantization unit 103.
  • the transform / quantization unit 103 performs frequency transform on the residual signal input from the subtraction unit 102 and calculates a transform coefficient.
  • the transform / quantization unit 103 quantizes the calculated transform coefficient to obtain a quantized coefficient.
  • the transform / quantization unit 103 outputs the obtained quantization coefficient to the entropy coding unit 104 and the inverse quantization / inverse transform unit 105.
  • the entropy encoding unit 104 receives the quantization coefficient from the transform / quantization unit 103 and receives the encoding parameter from the prediction parameter encoding unit 111.
  • Examples of input encoding parameters include codes such as a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, and a merge index merge_idx.
  • the entropy encoding unit 104 generates an encoded stream Te by entropy encoding the input quantization coefficient and encoding parameter, and outputs the generated encoded stream Te to the outside.
  • the inverse quantization / inverse transform unit 105 inversely quantizes the quantization coefficient input from the transform / quantization unit 103 to obtain a transform coefficient.
  • the inverse quantization / inverse transform unit 105 performs inverse frequency transform on the obtained transform coefficient to calculate a residual signal.
  • the inverse quantization / inverse transform unit 105 outputs the calculated residual signal to the addition unit 106.
  • the addition unit 106 adds the signal value of the prediction image P of the PU input from the prediction image generation unit 101 and the signal value of the residual signal input from the inverse quantization / inverse conversion unit 105 for each pixel, and performs decoding. Generate an image.
  • the adding unit 106 stores the generated decoded image in the reference picture memory 109.
  • the loop filter 107 performs a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) on the decoded image generated by the adding unit 106.
  • SAO sample adaptive offset
  • ALF adaptive loop filter
  • the prediction parameter memory 108 stores the prediction parameter generated by the encoding parameter determination unit 110 at a predetermined position for each encoding target picture and CU.
  • the reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each picture and CU to be encoded.
  • the encoding parameter determination unit 110 selects one set from among a plurality of sets of encoding parameters.
  • the encoding parameter is a parameter to be encoded that is generated in association with the above-described prediction parameter and the prediction parameter.
  • the predicted image generation unit 101 generates a predicted image P of the PU using each of these encoding parameter sets.
  • the encoding parameter determination unit 110 calculates a cost value indicating the amount of information and the encoding error for each of a plurality of sets.
  • the cost value is, for example, the sum of a code amount and a square error multiplied by a coefficient ⁇ .
  • the code amount is the information amount of the encoded stream Te obtained by entropy encoding the quantization error and the encoding parameter.
  • the square error is the sum between pixels regarding the square value of the residual value of the residual signal calculated by the subtracting unit 102.
  • the coefficient ⁇ is a real number larger than a preset zero.
  • the encoding parameter determination unit 110 selects a set of encoding parameters that minimizes the calculated cost value.
  • the entropy encoding unit 104 outputs the selected set of encoding parameters to the outside as the encoded stream Te, and does not output the set of unselected encoding parameters.
  • the encoding parameter determination unit 110 stores the determined encoding parameter in the prediction parameter memory 108.
  • the prediction parameter encoding unit 111 derives a format for encoding from the parameters input from the encoding parameter determination unit 110 and outputs the format to the entropy encoding unit 104. Deriving the format for encoding is, for example, deriving a difference vector from a motion vector and a prediction vector. Also, the prediction parameter encoding unit 111 derives parameters necessary for generating a prediction image from the parameters input from the encoding parameter determination unit 110 and outputs the parameters to the prediction image generation unit 101.
  • the parameter necessary for generating the predicted image is, for example, a motion vector in units of sub-blocks.
  • the inter prediction parameter encoding unit 112 derives an inter prediction parameter such as a difference vector based on the prediction parameter input from the encoding parameter determination unit 110.
  • the inter prediction parameter encoding unit 112 derives parameters necessary for generating a prediction image to be output to the prediction image generating unit 101, and an inter prediction parameter decoding unit 303 (see FIG. 5 and the like) derives inter prediction parameters.
  • Some of the configurations are the same as the configuration to be performed. The configuration of the inter prediction parameter encoding unit 112 will be described later.
  • the intra prediction parameter encoding unit 113 derives a format (for example, MPM_idx, rem_intra_luma_pred_mode) for encoding from the intra prediction mode IntraPredMode input from the encoding parameter determination unit 110.
  • a format for example, MPM_idx, rem_intra_luma_pred_mode
  • the inter prediction parameter encoding unit 112 is a unit corresponding to the inter prediction parameter decoding unit 303 in FIG. 11, and the configuration is shown in FIG. 9.
  • the inter prediction parameter encoding unit 112 includes an inter prediction parameter encoding control unit 1121, an AMVP prediction parameter derivation unit 1122, a subtraction unit 1123, a sub-block prediction parameter derivation unit 1125, and a partition mode derivation unit and a merge flag derivation unit (not shown). , An inter prediction identifier deriving unit, a reference picture index deriving unit, a vector difference deriving unit, and the like.
  • the partition mode deriving unit, the merge flag deriving unit, the inter prediction identifier deriving unit, the reference picture index deriving unit, and the vector difference deriving unit are respectively a PU partition mode part_mode, a merge flag merge_flag, an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, and a difference vector Derives mvdLX.
  • the inter prediction parameter encoding unit 112 outputs the motion vector (mvLX, subMvLX), the reference picture index refIdxLX, the PU partition mode part_mode, the inter prediction identifier inter_pred_idc, or information indicating these to the predicted image generation unit 101.
  • inter prediction parameter encoding unit 112 entropy PU partition mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX and sub-block prediction mode flag subPbMotionFlag.
  • the data is output to the encoding unit 104.
  • the inter prediction parameter encoding control unit 1121 includes a merge index deriving unit 11211 and a vector candidate index deriving unit 11212.
  • the merge index derivation unit 11211 compares the motion vector and reference picture index input from the encoding parameter determination unit 110 with the motion vector and reference picture index of the merge candidate PU read from the prediction parameter memory 108, and performs merge An index merge_idx is derived and output to the entropy encoding unit 104.
  • the merge candidate is a reference PU (for example, a reference PU in contact with the lower left end, upper left end, and upper right end of the encoding target block) within a predetermined range from the encoding target CU to be encoded.
  • the PU has been processed.
  • the vector candidate index deriving unit 11212 derives a prediction vector index mvp_LX_idx.
  • the sub-block prediction parameter derivation unit 1125 When the encoding parameter determination unit 110 determines to use the sub-block prediction mode, the sub-block prediction parameter derivation unit 1125 performs spatial sub-block prediction, temporal sub-block prediction, affine prediction, and matching motion derivation according to the value of subPbMotionFlag.
  • a motion vector and a reference picture index for any sub-block prediction are derived. As described in the description of the image decoding apparatus, the motion vector and the reference picture index are derived by reading out the motion vector such as the adjacent PU and the reference picture block and the reference picture index from the prediction parameter memory 108.
  • the AMVP prediction parameter derivation unit 1122 has the same configuration as the AMVP prediction parameter derivation unit 3032 (see FIG. 11).
  • the motion vector mvLX is input from the encoding parameter determination unit 110 to the AMVP prediction parameter derivation unit 1122.
  • the AMVP prediction parameter derivation unit 1122 derives a prediction vector mvpLX based on the input motion vector mvLX.
  • the AMVP prediction parameter derivation unit 1122 outputs the derived prediction vector mvpLX to the subtraction unit 1123. Note that the reference picture index refIdx and the prediction vector index mvp_LX_idx are output to the entropy encoding unit 104.
  • the subtraction unit 1123 subtracts the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 1122 from the motion vector mvLX input from the coding parameter determination unit 110 to generate a difference vector mvdLX.
  • the difference vector mvdLX is output to the entropy encoding unit 104.
  • the function of the inter prediction image generation unit in the prediction image generation unit of the image decoding device is not limited to the function of the inter prediction image generation unit 309 in the prediction image generation unit 308 of the image decoding device 31 of the image transmission system 1 according to the first embodiment.
  • the inter-predicted image generation unit functions as an inter-predicted image generation unit 309a in the predicted image generation unit 308a of the image decoding device 31a of the image transmission system 1a (not shown) instead of or in addition to the predicted image generation unit 308. You may have the function of.
  • Embodiment 2 will be described with reference to FIGS.
  • members having the same functions as those described in the first embodiment are denoted by the same reference numerals and description thereof is omitted.
  • the image transmission system 1a includes an image encoding device 11a (not shown) and an image decoding device 31a instead of the image encoding device 11 and the image decoding device 31 in the first embodiment.
  • FIG. 23 is a block diagram showing a main configuration of the image decoding device 31a according to the present embodiment. As illustrated in FIG. 23, the image decoding device 31a according to the present embodiment includes a predicted image generation unit 308a instead of the predicted image generation unit 308 in the first embodiment.
  • the predicted image generation unit 308a includes an inter predicted image generation unit 309a instead of the inter predicted image generation unit 309 in the first embodiment. Except for this point, the predicted image generation unit 308a has the same configuration as the predicted image generation unit 308 in the first embodiment.
  • FIG. 24 is a block diagram illustrating a configuration of an inter predicted image generation unit 309a included in the predicted image generation unit 308a according to the second embodiment.
  • the inter prediction image generation unit 309a includes a motion compensation unit 3091a instead of the motion compensation unit 3091 in the first embodiment.
  • the motion compensation unit 3091a generates a pixel value outside the reading region, which is a pixel value outside the MC reading region regarding the corresponding block in at least one of the first reference image and the second reference image.
  • the motion compensation unit 3091a prohibits the generation process of the pixel value outside the MC reading area along the vertical direction or the horizontal direction. That is, the motion compensation unit 3091a prohibits BIO padding in the vertical direction or the horizontal direction when generating a predicted image using the BIO mode.
  • FIG. 25 and 26 are diagrams illustrating regions in which the motion compensation unit 3091a executes BIO padding.
  • FIG. 27 is a diagram illustrating motion vectors in adjacent blocks.
  • the motion compensation unit 3091a when the motion compensation unit 3091a prohibits the generation process of the pixel value outside the MC reading area along the horizontal direction, the motion compensation unit 3091a performs BIO padding only in the vertical direction in the padding region indicated by the diagonal lines in FIG. .
  • BIO padding in the horizontal direction is not performed, when blocks adjacent in the horizontal direction have the same motion vector ((a) in FIG. 27), these blocks are merged to form one corresponding block. Can be considered. Therefore, when the horizontally adjacent blocks have the same motion vector, the motion compensation unit 3091a merges these blocks, regards them as one corresponding block, and refers to the reference block of the corresponding block to obtain a predicted image. It may be generated.
  • the motion compensation unit 3091a when configured to prohibit horizontal BIO padding, is a corresponding block and a block adjacent to the corresponding block in the horizontal direction and having the same motion vector as the corresponding block. And may be processed by merging. Thereby, the entire processing amount can be reduced.
  • the memory bandwidth can be reduced as compared with the case where the motion compensation unit 3091a performs BIO padding separately for each block boundary. That is, the memory bandwidth can be reduced as compared with the case where the motion compensation unit 3091a performs BIO padding on two blocks separately.
  • the motion compensation unit 3091a may prohibit the BIO padding in the vertical direction instead of the configuration for prohibiting the BIO padding in the horizontal direction. In this case, the motion compensation unit 3091a performs BIO padding in the padding area indicated by hatching in FIG.
  • BIO padding in the vertical direction is not performed, when adjacent blocks in the vertical direction have the same motion vector ((b) in FIG. 27), these blocks are merged and regarded as one corresponding block. be able to. Therefore, when the adjacent blocks in the vertical direction have the same motion vector, the motion compensation unit 3091a merges these blocks, regards them as one corresponding block, and refers to the reference block of the corresponding block to obtain a predicted image. It may be generated.
  • the motion compensation unit 3091a when configured to prohibit vertical BIO padding, includes a corresponding block and a neighboring block that is adjacent to the corresponding block in the vertical direction and has the same motion vector as the corresponding block. Merge and process. Thereby, the entire processing amount can be reduced.
  • the motion compensation unit 3091a may include a selection unit 4000 (not shown) that selects whether the direction in which BIO padding is prohibited is the horizontal direction or the vertical direction.
  • the selection unit 4000 sets the direction for prohibiting the BIO padding to the horizontal direction depending on whether the block having the same motion vector is adjacent to the horizontal direction or the vertical direction. Alternatively, it may be selected whether or not the vertical direction is used.
  • FIG. 28 is a flowchart illustrating an example of a processing flow for determining whether to prohibit BIO padding in the horizontal direction when the motion compensation unit 3091a generates a predicted image using the BIO mode.
  • the motion compensation unit 3091a is allowed to execute (permit) both horizontal and vertical BIO padding.
  • the motion compensation unit 3091a determines whether or not the motion vector of the corresponding block is the same as the motion vector of the block adjacent to the corresponding block in the horizontal direction.
  • the process proceeds to STEP 213.
  • the motion compensation unit 3091a determines that the motion vector of the corresponding block is not the same as the motion vector of a block adjacent to the corresponding block in the horizontal direction (No in STEP 212)
  • the motion compensation unit 3091a BIO padding is performed in both directions, and the process ends.
  • the motion compensation unit 3091a prohibits BIO padding in the horizontal direction, executes only BIO padding in the vertical direction, and ends the process.
  • FIG. 29 is a flowchart illustrating an example of a process flow for determining whether to prohibit BIO padding in the vertical direction when the motion compensation unit 3091a generates a predicted image using the BIO mode. Since STEP 221 in FIG. 29 is the same as STEP 211 in FIG. 28, description thereof is omitted.
  • the motion compensation unit 3091a determines whether or not the motion vector of the corresponding block is the same as the motion vector of the block adjacent to the corresponding block in the vertical direction.
  • the process proceeds to STEP 223.
  • the motion compensation unit 3091a determines that the motion vector of the corresponding block and the motion vector of the block adjacent to the corresponding block in the vertical direction are not the same (NO in STEP 222)
  • the motion compensation unit 3091a BIO padding is performed in both directions, and the process ends.
  • the motion compensation unit 3091a prohibits the BIO padding in the vertical direction, executes only the BIO padding in the horizontal direction, and ends the process.
  • the motion compensation unit 3091a determines whether the motion vector of the corresponding block is the same as the motion vector of the block adjacent to the corresponding block in one of the horizontal and vertical directions. Judging. However, in the present embodiment, the present invention is not limited to this, and the motion compensation unit 3091a determines whether or not the motion vectors of blocks adjacent in the horizontal direction are the same, and the motion vector of blocks adjacent in the vertical direction. It may be determined whether or not the same.
  • FIG. 30 is a flowchart illustrating an example of a processing flow for determining whether to prohibit BIO padding in the horizontal direction or the vertical direction when the motion compensation unit 3091a generates a predicted image using the BIO mode.
  • STEP 30 are the same as STEP 211 and STEP 213 in FIG. 28, and thus description thereof is omitted.
  • STEP 234 and STEP 235 in FIG. 30 are the same as STEP 222 and STEP 223 in FIG.
  • the motion compensation unit 3091a determines whether or not the motion vector of the corresponding block is the same as the motion vector of the block adjacent to the corresponding block in the horizontal direction.
  • the motion compensation unit 3091a determines whether or not the motion vectors of the blocks adjacent to the corresponding block are the same in the order of the horizontal direction and the vertical direction, and the BIO padding in the horizontal direction and the vertical direction. It may be determined whether or not to prohibit.
  • the motion compensation unit 3091a prohibits BIO padding.
  • the motion compensation unit 3091a may regard blocks adjacent in the horizontal direction or the vertical direction as one corresponding block and merge them to generate a predicted image of the corresponding block.
  • the motion compensation unit 3091a determines whether or not the motion vectors of the blocks adjacent to the corresponding block are the same in the order of the horizontal direction and the vertical direction. Not. The motion compensation unit 3091a may determine whether or not the motion vectors of the blocks adjacent to the corresponding block are the same in the order of the vertical direction and the horizontal direction.
  • the motion compensation unit 3091a may prohibit the BIO padding in the horizontal direction or the vertical direction described above when referring to the reference block of the corresponding block illustrated in FIG.
  • FIG. 31 is a schematic diagram of a corresponding block used when the motion compensation unit 3091a uses the BIO mode and a block adjacent to the corresponding block in the horizontal direction or the vertical direction.
  • the motion compensation unit 3091a may prohibit horizontal BIO padding when using the BIO mode for a vertically corresponding block. In this case, if the vertically corresponding block and the block adjacent in the horizontal direction of the vertically corresponding block have the same motion vector, the motion compensation unit 3091a merges these blocks and regards it as one corresponding block. A predicted image may be generated from the reference block of the corresponding block. Thus, even when the BIO mode is used for the vertical corresponding block and the block horizontally adjacent to the vertical corresponding block, the motion compensation unit 3091a reduces the overall processing amount in the same manner as in the above example. Can be reduced. In addition, the memory bandwidth can be reduced.
  • the motion compensation unit 3091a uses 4x8 pixels as the minimum unit block size, and when these blocks are merged and the BIO mode is used, processing is performed for each 8x8 pixel unit, and the overall processing amount is large. You do n’t have to. As a result, the memory bandwidth can be suppressed from becoming larger.
  • the motion compensation unit 3091a may prohibit BIO padding in the vertical direction when the BIO mode is used for a horizontally long corresponding block. In this case, the motion compensation unit 3091a merges the horizontally long corresponding block and the block adjacent to the horizontally long corresponding block in the vertical direction into the same motion vector and regards it as one corresponding block.
  • the predicted image may be generated with reference to the reference block of the corresponding block.
  • the motion compensation unit 3091a uses 8x4 pixels as the minimum unit block size, and when these blocks are merged and the BIO mode is used, processing is performed for each 8x8 pixel unit, and the overall processing amount is large. It ’s not necessary. As a result, the memory bandwidth can be suppressed from becoming larger.
  • the motion compensation unit 3091a may refer to a syntax indicating a block division method when determining whether to prohibit horizontal padding or vertical padding.
  • the motion compensation unit 3091a refers to the syntax indicating the block division method, and prohibits horizontal padding when it finally becomes a block of 4x8 pixels, and prohibits vertical padding when it becomes a block of 8x4 pixels. It may be a configuration.
  • the image encoding device 11a includes a predicted image generation unit 101a (not shown) instead of the predicted image generation unit 101 in the first embodiment. Moreover, the predicted image generation unit 101a includes an inter predicted image generation unit 1011a (not shown) instead of the inter predicted image generation unit 1011 in the first embodiment.
  • the inter predicted image generation unit 1011a includes a motion compensation unit 10111a instead of the motion compensation unit 10111 in the first embodiment. Except for this point, the inter prediction image generation unit 1011a has the same configuration as the inter prediction image generation unit 1011 in the first embodiment.
  • motion compensation unit 10111a has the same configuration as the motion compensation unit 3091a described above, description thereof is omitted here.
  • the function of the prediction parameter decoding unit of the image decoding device is not limited to the function of the inter prediction parameter decoding unit 303 in the prediction parameter decoding unit 302 of the image decoding device 31 of the image transmission system 1 according to the first embodiment.
  • the function of the inter prediction parameter decoding unit is the function of the inter prediction parameter decoding unit 303b of the image transmission system 1b (not shown) image decoding device 31b (not shown) instead of or in addition to the inter prediction parameter decoding unit 303. May be provided.
  • Embodiment 3 will be described with reference to FIGS.
  • members having the same functions as those described in the first embodiment are denoted by the same reference numerals and description thereof is omitted.
  • the image transmission system 1b includes an image decoding device 31b (not shown) and an image encoding device 11b (not shown) instead of the image decoding device 31 and the image encoding device 11 in the first embodiment.
  • the image decoding device 31b according to the present embodiment includes a prediction parameter decoding unit 302b (not shown) instead of the prediction parameter decoding unit 302 in the first embodiment.
  • the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303b instead of the inter prediction parameter decoding unit 303 in the first embodiment. Except for this point, the image decoding device 31b has the same configuration as the image decoding device 31 according to the first embodiment.
  • the image encoding device 11b will be described later.
  • FIG. 34 is a block diagram illustrating a configuration of an inter prediction parameter decoding unit 303b included in the prediction parameter decoding unit 302b.
  • the inter prediction parameter decoding unit 302b includes a sub-block prediction parameter deriving unit 3037b instead of the sub-block prediction parameter deriving unit 3037 in the first embodiment. Except for this point, the inter prediction parameter decoding unit 302b has the same configuration as the inter prediction parameter decoding unit 302 in the first embodiment. As shown in FIG.
  • the sub-block prediction parameter derivation unit 3037b performs at least one of the spatio-temporal sub-block prediction unit 30371, the affine prediction unit 30372, and the matching motion derivation unit 30373 that performs sub-block prediction in the sub-block prediction mode.
  • Subblock prediction mode flag a method of deriving a sub-block prediction mode flag subPbMotionFlag indicating whether or not a prediction mode of a certain PU is a sub-block prediction mode in the image decoding device 31b and the image encoding device 11 will be described.
  • the image decoding device 31b and the image encoding device 11 set the sub-block prediction mode flag subPbMotionFlag based on which of spatial sub-block prediction SSUB, temporal sub-block prediction TSUB, affine prediction AFFINE, and matching motion derivation MAT described later is used.
  • the sub-block prediction mode flag subPbMotionFlag may be derived by the following equation.
  • the image decoding device 31b and the image encoding device 11 may be configured to perform some predictions among the spatial sub-block prediction SSUB, temporal sub-block prediction TSUB, affine prediction AFFINE, and matching motion derivation MAT. That is, when the image decoding device 31b and the image encoding device 11 are configured to perform spatial subblock prediction SSUB or affine prediction AFFINE, the subblock prediction mode flag subPbMotionFlag may be derived as follows.
  • the spatio-temporal sub-block prediction unit 30371 calculates the target PU from the motion vector of the PU on the reference image temporally adjacent to the target PU (for example, the immediately preceding picture) or the motion vector of the PU spatially adjacent to the target PU.
  • the motion vector of the sub-block obtained by dividing is derived.
  • (xPb, yPb) is the upper left coordinate of the target PU
  • nPbW, nPbH are the size of the target PU
  • nSbW, nSbH are the sizes of the sub-blocks.
  • the above-described temporal sub-block prediction candidate TSUB and spatial sub-block prediction candidate SSUB are selected as one mode (merge candidate) of the merge mode.
  • the affine prediction unit 30372 derives the affine prediction parameters of the target PU.
  • motion vectors of two or three control points of the target PU are derived as affine prediction parameters.
  • motion vectors mv0_x, mv0_y), (mv1_x, mv1_y), and (mv2_x, mv2_y) are derived.
  • the motion vector of each control point may be derived by predicting from the motion vector of the adjacent PU of the target PU.
  • the motion vector of each control point may be derived from the sum of the control point prediction vector mvpLX and the difference vector mvdLX derived from the encoded data.
  • FIG. 35 shows an example in which the motion vector spMvLX of each sub-block constituting the target PU (nPbW ⁇ nPbH) is derived from the motion vector (mv0_x, mv0_y) of the control point V0 and the motion vector (mv1_x, mv1_y) of V1.
  • the motion vector spMvLX of each subblock is derived as a motion vector for each point located at the center of each subblock, as shown in FIG.
  • xPb and yPb are the upper left coordinates of the target PU
  • nPbW and nPbH are the width and height of the target PU
  • nSbW and nSbH are the width and height of the sub-block.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 uses representative points (here, V0 and V1) of the target block as two or more control points used for affine prediction for deriving candidates. ) To derive the respective motion vectors. Note that a point on the target block or a point near the target block is used as the representative point of the block. In this specification, a block representative point used as a control point for affine prediction is referred to as a “block control point”.
  • (STEP2) Derivation of sub-block vector
  • the affine prediction unit 30372 or AMVP prediction parameter derivation unit 3032 determines the target block from the motion vector of the block control points (control points V0 and V1) that are representative points of the target block derived in STEP1. Is a step of deriving a motion vector of each sub-block included in. From (STEP 1) and (STEP 2), the motion vector mvLX of each sub-block is derived.
  • the motion compensation unit 3091 receives the reference picture from the reference picture memory 306 based on the prediction list use flag predFlagLX, the reference picture index refIdxLX, and the motion vector mvLX input from the inter prediction parameter decoding unit 303. Performs motion compensation in units of sub-blocks to generate motion compensated image predSamplesLX by reading out and filtering a block at a position shifted by motion vector mvLX from the position of the target block on the reference picture specified by index refIdxLX It is a process.
  • (STEP 4) Storage of motion vectors of sub-blocks
  • the motion vector mvLX of each sub-block derived by the AMVP prediction parameter deriving unit 3032 in (STEP 2) is stored in the prediction parameter memory 307.
  • the motion vector mvLX of each sub-block derived by the affine prediction unit 30372 in (STEP 2) is stored in the prediction parameter memory 307.
  • FIG. 37 is a diagram illustrating an example of the position of a prediction unit (reference block) used for deriving a motion vector of a control point in the AMVP mode and the merge mode.
  • the AMVP prediction parameter derivation unit 3032 decodes the motion vector mvLX of two or more control points (for example, V0 and V1, or V0, V1, and V2). Specifically, the prediction vector mvpLX and the difference vector are added for each control point.
  • the AMVP prediction parameter derivation unit 3032 is adjacent to one of the representative points, and blocks A, B, and C (reference blocks) that share the representative point (vertex) with the target block.
  • the AMVP prediction parameter derivation unit 3032 sets one of the motion vectors of the blocks D and E as the prediction vector mvpLX of the representative point V1.
  • the AMVP prediction parameter derivation unit 3032 derives a prediction vector mvpLX of the representative point V2 from any of the motion vectors of the blocks F and G (reference block). May be.
  • control point in STEP1 is not limited to the above.
  • a vertex at the lower right of the target block or points around the target block as described later may be used.
  • the motion vector (mvi_x, mvi_y) of the sub-block coordinates (xi, yj) has four parameters (mv_x, mv_y, ev, rv), that is, the motion vector of the expansion and rotation center (translation vector mv_x, mv_y) and the expansion parameter Using ev and the rotation parameter rv, the following general formula (eq1) can be obtained.
  • mvi_x mv_x + ev * xi-rv * yj
  • mvi_y mv_y + rv * xi + ev * yj (eq1)
  • (mv_x, mv_y) may be (mv0_x, mv0_y).
  • the affine prediction unit 30372 refers to the prediction parameter memory 307 for a prediction unit including blocks A to E as shown in (c) of FIG. 37, and confirms whether or not affine prediction is used.
  • a prediction unit that is first found using affine prediction (here, referred to as reference block A in FIG. 37C) is selected as an adjacent block (merge reference block), and a motion vector is derived.
  • the affine prediction unit 30372 moves the control point (for example, V0, V1) from the motion vector of the two or three points (point v0, point v1, point v2 in (d) of FIG. 37) of the selected merge reference block. Derive a vector.
  • the horizontal width of the block for which the motion vector is predicted is W
  • the height is H
  • adjacent blocks adjacent blocks including the block A in the example in the figure.
  • the width is w and the height is h.
  • affine parameters ev, rv
  • evBW, rvBW evBW
  • evBH, rvBH ev1, rv1, ev2, rv2
  • ev1BW, rv1BW, ev2BH, rv2BH sub block motion vectors
  • the image encoding device 11 (more specifically, the encoding parameter determination unit 110) or the affine prediction unit 30372 limits the range of the difference between the motion vectors at the plurality of derived control points.
  • the encoding parameter determination unit 110 derives a motion vector so that the difference value (dmv_x, dmv_y) between (mv0_x, mv0_y) and (mv1_x, mv1_y) satisfies (Formula XX1), and the prediction parameter code
  • the encoding unit 111 encodes the difference vector.
  • the encoding parameter determination unit 110 sets the motion vector search range so that the motion vectors of the control point V0 and the control point V1 satisfy Expression XX1 in the motion vector search.
  • THMVX and THMVY are predetermined threshold values.
  • the restricted motion vector at the control point V1 is denoted as (mv1_x ′, mv1_y ′).
  • the AMVP prediction parameter derivation unit 3032 derives a prediction vector of the motion vector selected by the image encoding device 11.
  • a motion vector is derived by the addition unit 3038 adding the derived prediction vector and the difference vector.
  • the affine prediction unit 30372 limits the motion vector of the other control point according to the magnitude of the motion vector of one control point of the control point V0 and the control point V1.
  • the affine prediction unit 30372 restricts the motion vector (mv1_x, mv1_y) of the control point V1 before the restriction to the range from the motion vector (mv0_x, mv0_y) of the control point V0 to (rangeW, rangeH).
  • the motion vector (mv1_x ′, mv1_y ′) of the control point V1 after the restriction is expressed as the following equation.
  • mv1_x ' clip3 (mv0_x-rangeW, mv0_x + rangeW, mv1_x)
  • mv1_y ' clip3 (mv0_y-rangeH, mv0_y + rangeH, mv1_y)
  • the predicted image generation unit 101 can read a region including the reference block group at a time and generate a predicted image. As a result, the memory bandwidth can be reduced.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 calculates an affine parameter, and derives a motion vector of each sub-block using the derived affine parameter.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 refers to the motion vectors of two or three control points among the control points V0, V1, and V2, which are representative points on the block, derived in (STEP 1). , (STEP 2), first, affine parameters are calculated. That is, the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 performs any one of the following 4-parameter affine and 6-parameter affine.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 derives affine parameters (ev, rv) from the motion vectors of two control points among the control points V0, V1, and V2.
  • the affine prediction unit 30372 calculates the affine parameters from the two motion vectors of the motion vector (mv0_x, mv0_y) of the control point V0 and the motion vector (mv1_x ′, mv1_y ′) after the restriction of the control point V1.
  • Deriving (ev, rv). 6-parameter affine The affine prediction unit 30372 or AMVP prediction parameter derivation unit 3032 derives affine parameters (ev1, rv1, ev2, rv2) from the three motion vectors of the control points V0, V1, and V2.
  • the affine predictor 30372 determines the magnitude of two of the three motion vectors as the motion vector of a certain control point. Limit according to size. About the method of control, since a clip function is used similarly to the above-mentioned, description is abbreviate
  • the affine prediction unit 30372 includes the motion vector (mv0_x, mv0_y) of the control point V0, the motion vector after restriction of the control point V1 (mv1_x ′, mv1_y ′), and the motion vector after restriction of the control point V2 (mv2_x ′,
  • the affine parameters (ev1, rv1, ev2, rv2) are derived from the three motion vectors mv2_y ′).
  • the conversion process consisting of the translation vector (mv_x, mv_y) and the affine parameters (ev, rv) is referred to as “4-parameter affine”
  • the translation vector (mv_x, mv_y) and the affine parameters (ev1, rv1, ev2, rv2) is referred to as “6-parameter affine”.
  • Affine parameters can be handled as integers instead of decimal numbers.
  • the decimal affine parameters are (ev, rv), (ev1, rv1, ev2, rv2), and the integer affine parameters are (evBW, rvBW), (evBH, rvBH), (ev1BW, rv1BW, ev2BH, rv2BH) It expresses.
  • the integer affine parameter is obtained by multiplying the decimal affine parameter by a constant (constant for integerization) according to the sub-block size (BW, BH). Specifically, the following relationship exists between the affine parameters for decimal numbers and the affine parameters for integers.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 uses the affine parameters (ev, rv), the motion vector (mv0_x, mv0_y) of the control point V0, and the magnitude of the motion vector of the control point V0 at the control point V1.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 has a motion vector (mv2_x ′, mv2_x ′,) limited according to the motion vector of the control point V1 and the magnitude of the motion vector of the control point V1 at the control point V2.
  • mv2_y ') to ev ((mv1_x-mv2_x')-(mv1_y-mv2_y ') ⁇ / 2W
  • rv ⁇ (mv1_x-mv2_x ') + (mv1_y-mv2_y') ⁇ / 2W May be derived.
  • the affine prediction unit 30372 has a motion vector (mv3_x ′, mv3_y ′) limited according to the motion vector (mv0_x, mv0_y) of the control point V0 and the magnitude of the motion vector of the control point V0 at the control point V3.
  • ev (mv3_x '-mv0_x + mv3_y'-mv0_y) / 2W
  • rv (-mv3_x '+ mv0_x + mv3_y'-mv0_y) / 2W May be derived.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 determines the affine parameters according to the motion vector (mv0_x, mv0_y) of the control point V0 and the magnitude of the motion vector of the control point V0 at the control points V1 and V2.
  • the affine parameter when the motion vector of the control point is not limited is derived as an expression obtained by taking “′” from the above expression.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 uses the affine parameters (evBW, rvBW), (evBH, rvBH), the motion vector (mv0_x, mv0_y) of the control point V0, and the control point V0 at the control point V1.
  • evBW ev * BW
  • rvBW rv * BW
  • evBH ev * BH
  • rvBH rv * BH
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 performs motion limited according to the motion vector (mv0_x, mv0_y) of the control point V0 and the motion vector of the control point V0 at the control point V2.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 performs the motion limited according to the motion vector (mv1_x, mv1_y) of the control point V1 and the motion vector of the control point V1 at the control point V2.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 performs motion limited according to the motion vector (mv0_x, mv0_y) of the control point V0 and the motion vector of the control point V0 at the control point V3.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 determines the affine parameters according to the motion vector (mv0_x, mv0_y) of the control point V0 and the magnitude of the motion vector of the control point V0 at the control points V1 and V2.
  • the above-described integer precision affine parameter derived by the AMVP prediction parameter deriving unit 3032 is derived as an expression obtained by taking “′” from the above expression.
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 derives the motion vector of the sub-block using the affine parameter obtained by the above formula.
  • sub block motion vectors (mvij_x, mvij_y) of subblock coordinates (xi, yj) are calculated from the affine parameters (ev, rv) calculated in the AMVP mode and the merge mode, respectively, as follows: Derived from AF4P_float.
  • mvij_x mv_x + ev * xi-rv * yj
  • mvij_y mv_y + rv * xi + ev * yj
  • the motion vector (mv0_x, mv0_y) of the control point V0 may be used as the affine parameters mv_x, mv_y of the translation vector
  • the present invention is not limited to this.
  • a motion vector other than the control point V0 a motion vector at the control point V1, a motion vector at the control point V2, and a motion vector at the control point V3 may be used as translation vectors.
  • the motion vector (mvij_x, mvij_y) of the subblock at the subblock position (i, j) is derived by the following formula AF4P_integer To do.
  • mvij_x mv_x + of_x + evBW * i-rvBH * j
  • mvij_y mv_y + of_y + rvBW * i + evBH * j
  • (Mv0_x, mv0_y) can also be used as the translation vector (mv_x, mv_y).
  • mvij_x mv_x + ev * xi-rv * yj
  • mvij_x mv_x + ((evBW-rvBW) >> 1) + evBW * i-rvBW * j
  • mvij_y mv_y + ((rvBW + evBW) >> 1) + rvBW * i + evBW * j
  • it may be derived only from (evBH, rvBH).
  • FIG. 38 is a diagram illustrating an example in which the control point V0 of the target block (horizontal width W, height H) is located at the upper left vertex and is divided into sub-blocks having a width BW and a height BH.
  • W, H) and (BW, BH) correspond to the aforementioned (nPbW, nPbH) and (nSbW, nSbH).
  • the points of the subblock position (i, j) and the subblock coordinates (xi, yj) are the intersections of the broken line parallel to the x axis and the broken line parallel to the y axis in FIG.
  • the motion vector (mvij_x, mvij_y) of the subblock at the subblock position (i, j) ) Is derived by the following expression AF6P_integer.
  • mvij_x mv_x + ((ev1BW + rv2BH) >> 1) + ev1BW * i + rv2BH * j
  • mvij_y mv_y + ((rv1BW + ev2BH) >> 1) + rv1BW * i + ev2BH * j (expression AF6P_integer) [Modification Example 1 of Subblock Motion Vector Derivation Processing]
  • the affine prediction unit 30372 or the AMVP prediction parameter derivation unit 3032 limits the range of the motion vector difference between the control points.
  • the present embodiment is not limited to this.
  • the size of the subblock may be limited.
  • the first modification and the second modification the case where the affine prediction unit 30372 executes the process will be described.
  • the AMVP prediction parameter derivation unit 3032 may execute the process.
  • the affine prediction unit 30372 when the difference between the motion vectors at the plurality of control points is larger than a predetermined value, the affine prediction unit 30372 has a sub size larger than that when the difference is equal to or smaller than the predetermined value.
  • a block may be set and a motion vector may be derived.
  • FIG. 39 is a flowchart showing an outline of the sub-block size determination flow.
  • the sub-block size determination process by the affine prediction unit 30372 includes the following three steps (Step S321) to (Step S323).
  • Step S321 Determination of Motion Vector Difference
  • the affine prediction unit 30372 determines whether or not the vector difference between control points is large. For example, when the vector difference between the control point V0 and the control point V1 in FIG. 37 satisfies the following equation (eq3), the affine prediction unit 30372 calculates the vector difference between the control point V0 and the control point V1. You may determine with it being large. Similarly, the affine prediction unit 30372 may determine that the vector difference between the control point V0 and the control point V1 is not large when the following equation (eq3) is not satisfied.
  • M in the above expression represents a predetermined threshold value.
  • the magnitude of the difference may be obtained as the sum of the absolute values of the differences as described above, or may be obtained as the sum of squares of the differences.
  • the vector difference between control points is not limited to the form discriminate
  • the vector difference between the control points may be determined by the vector difference between the control point V0 and the control point V2, as shown in the following equation (eq3 ′), or the following equation (eq3 ′′): As shown, it may be determined by the vector difference between the control point V1 and the control point V2.
  • step S322 If the affine prediction unit 30372 determines that the vector difference between the control points is large (Y in step S321), the process proceeds to step S322. If the affine prediction unit 30372 determines that the vector difference between the control points is not large (N in step S321), the process proceeds to step S323.
  • Step S322 Determination to Large Subblock Size
  • the affine prediction unit 30372 determines the size of the subblock to be the size of a large subblock (large subblock size). For example, the affine prediction unit 30372 may determine the size of the sub-block to be a large sub-block size of 8 ⁇ 8. The large sub-block size only needs to be larger than the small sub-block size (small sub-block size) described later. For example, when the small subblock size is set to 4 ⁇ 4, the affine prediction unit 30372 may determine the size of the large subblock to be 8 ⁇ 4 or 4 ⁇ 8.
  • Step S323 Determination to Small Subblock Size
  • the size of the subblock is determined to be a small subblock size.
  • the affine prediction unit 30372 may determine the size of the subblock to be a 4 ⁇ 4 small subblock size.
  • the small sub-block size only needs to be smaller than the large sub-block size. For example, when the large sub-block size is set to a large size of 8 ⁇ 8, the affine prediction unit 30372 may determine the size of the small sub-block as 8 ⁇ 4 or 4 ⁇ 8.
  • the affine prediction unit 30372 sets the combination of the large sub-block size and the small sub-block size (large sub-block size, small sub-block size), for example, (8 ⁇ 8, 4 ⁇ 4), (8 ⁇ 8, ⁇ 8 ⁇ 4) or (8 ⁇ 4, 4 ⁇ 4), etc.
  • the affine prediction unit 30372 ends the sub-block size determination process.
  • the affine prediction unit 30372 determines that the vector difference between the controls is large when the expression (eq3) is satisfied, but the present embodiment is not limited to this. In the present embodiment, the affine prediction unit 30372 may determine that the vector difference between the control points is large when the following equation (eq4) is satisfied, or when the following equation (eq4 ′) is satisfied: It may be determined that the vector difference between the control points is large.
  • the affine prediction unit 30372 may use equation (eq4) instead of equation (eq3) as an equation for determining whether or not the affine parameter is large.
  • the AMVP prediction parameter derivation unit 3032 may determine the size of the sub-block in the same manner as the affine prediction unit 30372.
  • the affine prediction unit 30372 determines the aspect ratio of the sub-block used for deriving the motion vector according to the aspect ratio of the target block that is the target of motion vector prediction, instead of the above example. Also good. That is, the affine prediction unit 30372 may derive a motion vector of a sub-block having an aspect ratio corresponding to the aspect ratio of the target block.
  • the affine prediction unit 30372 may determine that the aspect ratio of the sub-block is horizontally long when the target block is horizontally long, and may determine the aspect ratio of the sub-block vertically when the target block is vertically long.
  • the aspect ratio of the sub-block is horizontally long.
  • the aspect ratio of the vertically long sub-block is vertically long, so that the sub-block having the optimum aspect ratio can be obtained.
  • the affine prediction unit 30372 may determine the aspect ratio of the sub-block to be horizontal.
  • the cache is more likely to hit when the sub-block is horizontally long than vertically. Therefore, according to the above configuration, the memory bandwidth can be made smaller when the horizontal width and height of the sub-block are set equal when the horizontal width and height of the target block are equal.
  • the affine prediction unit 30372 may determine the aspect ratio of the sub-block to be vertically long.
  • AMVP prediction parameter derivation unit 3032 may also determine the aspect ratio of the sub-block in the same manner as the affine prediction unit 30372.
  • the matching motion deriving unit 30373 derives a motion vector spMvLX of a block or sub-block constituting the PU by performing a matching process of bilateral matching or template matching.
  • FIG. 36 is a diagram for explaining (a) bilateral matching and (b) template matching.
  • the matching motion derivation mode is selected as one merge candidate (matching candidate) in the merge mode.
  • the matching motion deriving unit 30373 derives a motion vector by matching regions in a plurality of reference images, assuming that the object moves at a constant velocity.
  • bilateral matching it is assumed that a certain object passes through a certain region of the reference image A, a target PU of the target picture Cur_Pic, and a certain region of the reference image B with constant velocity motion, and between the reference images A and B.
  • the motion vector of the target PU is derived from the matching.
  • template matching it is assumed that the motion vector of the target PU adjacent region and the target PU are equal, and motion is performed by matching motion compensated images between the target PU adjacent region Temp_Cur (template) and the reference block adjacent region Temp_L0 on the reference picture. Derive a vector.
  • (mv0_x, mv0_y) is determined so that the matching cost between Block_A and Block_B is minimized.
  • the (mv0_x, mv0_y) derived in this way is a motion vector assigned to the sub-block.
  • (b) of FIG. 36 is a figure for demonstrating a template matching (Template
  • one reference picture is referred to at a time in order to derive a motion vector of the sub-block Cur_block in the target picture Cur_Pic.
  • the reference block Block_A having the upper left coordinates (xPos, yPos) specified by is specified.
  • (xCur, yCur) is the upper left coordinate of the sub-block Cur_block.
  • a template region Temp_Cur adjacent to the sub-block Cur_block in the target picture Cur_Pic and a template region Temp_L0 adjacent to the Block_A in the reference picture A are set.
  • the template region Temp_Cur is composed of a region adjacent to the upper side of the sub-block Cur_block and a region adjacent to the left side of the sub-block Cur_block.
  • the template area Temp_L0 is composed of an area adjacent to the upper side of Block_A and an area adjacent to the left side of Block_A.
  • the same processing may be performed on the reference image Ref1 different from the reference image Ref0.
  • the reference block Block_A having the upper left coordinates (xPos1, yPos1) specified by is specified, and the template region Temp_L1 adjacent to Block_A in the reference picture A is set.
  • the matching cost between Temp_Cur and TempL1 is minimum (mv1_x, mv1_y), and the motion vector spMvL1 assigned to the sub-block is obtained.
  • template matching may be performed on two reference images Ref0 and Ref1. In this case, the matching of one reference image Ref0 and the matching of one reference image Ref1 described above are sequentially performed.
  • the motion vector decoding process includes a process of decoding syntax elements related to inter prediction (also referred to as motion syntax decoding process) and a process of deriving a motion vector ( Motion vector derivation process).
  • FIG. 32 is a flowchart showing the flow of inter prediction syntax decoding processing performed by the inter prediction parameter decoding control unit 3031. In the following description in the description of FIG. 32, each process is performed by the inter prediction parameter decoding control unit 3031 unless otherwise specified.
  • inter_pred_idc is decoded in step S304.
  • inter_pred_idc is other than PRED_L1 (PRED_L0 or PRED_BI)
  • the reference picture index refIdxL0, the difference vector parameter mvdL0, and the prediction vector index mvp_L0_idx are decoded in steps S305, S306, and S307, respectively.
  • inter_pred_idc is other than PRED_L0 (PRED_L1 or PRED_BI)
  • the reference picture index refIdxL1 is decoded in S308, S309, and S310.
  • a motion vector derivation process is executed.
  • the image encoding device 11b includes a prediction parameter encoding unit 111b (not shown) instead of the prediction parameter encoding unit 111 in the first embodiment. Further, the prediction parameter encoding unit 111b includes an inter prediction parameter encoding unit (motion vector deriving unit) 112b instead of the inter prediction parameter encoding unit 112 in the first embodiment. Except for this point, the image encoding device 11b has the same configuration as the image encoding device 11 according to the first embodiment.
  • FIG. 33 is a block diagram illustrating a configuration of the inter prediction parameter encoding unit 112b in the prediction parameter encoding unit 111b of the image encoding device 11b according to the third embodiment.
  • the inter prediction parameter encoding unit 112b includes a sub-block prediction parameter deriving unit 1125b instead of the sub-block prediction parameter deriving unit 1125 in the first embodiment. Except this point, the inter prediction parameter encoding unit 112b has the same configuration as the inter prediction parameter encoding unit 112 in the first embodiment. As illustrated in FIG.
  • the inter prediction parameter encoding unit 112b includes at least one of a spatiotemporal sub-block prediction unit 11251, an affine prediction unit 11252, and a matching motion derivation unit 11253.
  • the spatio-temporal sub-block prediction unit 11251, the affine prediction unit 11252, and the matching motion derivation unit 11253 have the same configurations as the spatio-temporal sub-block prediction unit 30371, the affine prediction unit 30372, and the matching motion derivation unit 30373, respectively. The description in is omitted.
  • the function of the inter prediction parameter decoding unit in the prediction parameter decoding unit of the image decoding device is not limited to the function of the inter prediction parameter decoding unit 303b in the prediction parameter decoding unit 302b of the image decoding device 31b of the image transmission system 1b according to Embodiment 3. .
  • the function of the inter prediction parameter decoding unit is performed in the prediction parameter decoding unit 302c of the image decoding device 31c of the image transmission system 1c (not shown) according to Embodiment 4 instead of or in addition to the inter prediction parameter decoding unit 303b.
  • the function of the inter prediction parameter decoding unit (motion vector deriving unit) 303c may be provided.
  • Embodiment 4 will be described with reference to FIGS.
  • members having the same functions as those described in the third embodiment are denoted by the same reference numerals and description thereof is omitted.
  • the image transmission system 1c includes an image encoding device (motion vector deriving device) 11c and an image decoding device (motion vector deriving device) 31c instead of the image encoding device 11b and the image decoding device 31b in the third embodiment. .
  • FIG. 41 is a block diagram showing a main configuration of the image decoding device 31c according to the present embodiment. As illustrated in FIG. 41, the image decoding device 31c according to the present embodiment includes a prediction parameter decoding unit 302c instead of the prediction parameter decoding unit 302b according to the third embodiment.
  • the prediction parameter decoding unit 302c includes an inter prediction parameter decoding unit 303c instead of the inter prediction parameter decoding unit 303b in the third embodiment. Except for this point, the prediction parameter decoding unit 302c has the same configuration as the prediction parameter decoding unit 302b in the third embodiment.
  • FIG. 43 is a block diagram illustrating a configuration of an inter prediction parameter decoding unit 303c included in the prediction parameter decoding unit 302c according to the second embodiment.
  • the inter prediction parameter decoding unit 303c includes a subblock prediction parameter derivation unit 3037c instead of the subblock prediction parameter derivation unit 3037b in the third embodiment.
  • the sub-block prediction parameter derivation unit 3037c includes a matching motion derivation unit (first motion vector search unit, second motion vector search unit) 30373c instead of the matching motion derivation unit 30373 in the third embodiment.
  • the matching motion derivation unit 30373c includes a first motion vector search unit 303731 and a second motion vector search unit 303732.
  • the first motion vector search unit 303731 searches for a motion vector for each prediction block by matching processing.
  • the first motion vector search unit 303731 searches for a motion vector by performing a local search after performing an initial vector search for a prediction block.
  • the second motion vector search unit 303732 refers to the motion vector selected by the first motion vector search unit 303731, and searches for a motion vector by a matching process for each of a plurality of sub-blocks included in the prediction block. To do.
  • the second motion vector search unit 303732 searches for a motion vector by performing a local search after performing an initial vector search for a sub-block.
  • the matching motion deriving unit 30373c prohibits a search in an oblique direction in at least one of the local search by the first motion vector search unit 303731 and the local search by the second motion vector search unit 303732. That is, the matching motion deriving unit 30373c sets the search direction of the candidate vector in the horizontal direction or the vertical direction centering on the initial vector in at least one of the first motion vector search unit 303731 and the second motion vector search unit 303732. Limit to crab.
  • the image decoding device 31a may read an image by reusing an already read image. Therefore, according to the above-described configuration, it is possible to generate a predicted image while reducing the memory band, compared to a case where search for a motion vector in an oblique direction is not prohibited.
  • FIG. 44 is a flowchart showing an outline of the motion prediction mode determination flow.
  • the motion prediction mode determination flow is executed by the inter prediction parameter decoding unit 303c.
  • the motion prediction mode is a mode for determining a method for deriving a motion vector used for motion compensation prediction.
  • the inter prediction parameter decoding control unit 3031 determines whether or not the mode is the merge mode (step S1501), and if not the merge mode (N in step S1501). AMVP mode. On the other hand, if it is determined that the mode is the merge mode (Y in step S1501), it is determined whether the mode is the matching mode (step S1502). If it is determined that the mode is the matching mode (Y in step S1502), the mode is the matching mode. If it is determined that the mode is not the matching mode (N in step S1502), the mode is the merge mode.
  • FIG. 45 is a sequence diagram showing the flow of the motion prediction mode determination flow.
  • the matching mode also called FRUC (Frame Rate Up Up Conversion) merge mode
  • the matching motion deriving unit 30373c derives a pattern match vector by bilateral matching when fruc_mode_idx is MODE_BM (eg, 1).
  • fruc_mode_idx is MODE_TM (for example, 2)
  • a pattern match vector is derived by template matching.
  • the parameter fruc_mode_idx indicating the matching mode serves as both a flag indicating whether the matching mode is used and a parameter indicating the matching method in the matching mode, but the present invention is not limited to this. That is, instead of the parameter fruc_mode_idx indicating the matching mode, a flag fruc_merge_flag indicating whether to use the matching mode and a parameter fruc_merge_param indicating the matching method may be used.
  • step S402 the AMVP mode is selected. More specifically, the AMVP prediction parameter derivation unit 3032 decodes the difference vector mvdLX in step S421, and the prediction vector index mvp_LX_idx is decoded in step S422. Further, in step S423, a prediction vector candidate pmvCand is derived. Subsequently, in step S424, a motion vector mvLX is derived from the following equation.
  • FIG. 46 is a flowchart showing the flow of the pattern match vector derivation process.
  • FIG. 46 shows the details of the process in step S405 in the sequence diagram shown in FIG.
  • the processing shown in FIG. 46 is executed by the matching motion derivation unit 30373c.
  • steps S4051 to S4054 are block searches executed at the block level. That is, a motion vector is derived for the entire block (CU or PU) using pattern matching.
  • Steps S4055 to S4060 are sub-block searches executed at the sub-block level constituting the block, and a motion vector is derived for each sub-block constituting the block using pattern matching.
  • a motion vector is derived every 8 ⁇ 8 or 4 ⁇ 4 unit.
  • the first motion vector search unit 303731 in the matching motion derivation unit 30373c derives a block-level initial vector in the target block (initial vector search).
  • the initial vector is a motion vector that serves as a search base.
  • a limited motion vector candidate such as a spatial merge candidate, a temporal merge candidate, a combined merge candidate, a zero vector, and an ATMVP vector of the target block
  • a vector that minimizes is derived as an initial vector.
  • the initial vector candidate is a motion vector derived based on the motion vector of the processed reference point.
  • the ATMVP vector is a vector derived from the average (or weighted average, median) of the motion vector around the target block and the motion vector of the reference image.
  • step S4054 the first motion vector search unit 303731 in the matching motion derivation unit 30373c performs a block level local search (local search) in the target block.
  • a local region centered on the derived initial vector is further searched, a vector having the lowest matching cost is searched, and a final block-level motion vector of the target block is obtained.
  • the local search may be a step search, raster search, or spiral search. Details of the local search will be described later.
  • step S4058 the second motion vector search unit 303732 in the matching motion derivation unit 30373c derives an initial vector of the sub-block in the target block (initial vector search). Specifically, vector candidates (motion vector derived in step S4054, zero vector, center collocated vector of the subblock, lower right collocated vector of the subblock, ATMVP vector of the subblock, upper adjacent vector of the subblock And the vector having the lowest matching cost among the left adjacent vectors of the sub-block and the like is set as the initial vector of the sub-block. Note that the vector candidates used for the initial vector search of the sub-block are not limited to the vectors described above.
  • step S4059 a local search centering on the initial vector of the sub-block selected in step S4058 is performed. Then, the matching cost of vector candidates near the initial vector of the sub-block is derived, and the minimum vector is derived as the sub-block motion vector.
  • fruc_mode_idx is MODE_BM
  • the matching cost is derived by bilateral matching.
  • fruc_mode_idx is MODE_TM
  • template matching is derived by template matching.
  • the matching motion derivation unit 30373c prohibits a search in an oblique direction in at least one of the local search in step S4054 and the local search in step S4059. Details will be described later.
  • FIG. 47 is a diagram for explaining a motion search pattern. Note that the number of steps (stepIter, maximum number of rounds) indicating how many times the method used for motion search (stepMethod) is repeated is set to a predetermined value. As will be described later, the maximum round number stepIterSubPU at the sub-block level may be less than the maximum round number stepIterPU at the block level.
  • the matching motion deriving unit 30373c considers the search candidate point that gives the smallest matching cost among the search candidate points evaluated for the matching cost in the motion search as the optimal search point, and selects the motion vector bestMV of the search candidate point. To do. Examples of functions used for deriving the matching cost include SAD (Sum of Absolute Difference, sum of absolute value errors), SATD (Hadamard transform absolute value error sum), SSD (Sum of Square difference), and the like. .
  • the local search for motion vectors performed by the matching motion deriving unit 30373c is not limited to this.
  • FIG. 47A is a diagram showing a motion search pattern when diamond search is applied.
  • FIG. 47B shows a motion search pattern when a cross search is applied.
  • step search a search candidate point is set around an initial vector (search start point), a matching cost for the set search candidate point is derived and evaluated, and a search candidate point that provides an optimal matching cost is selected. I do.
  • This process is called “step round process”.
  • this “step round process” is repeatedly executed.
  • the search round number numIter is incremented by one.
  • the initial vector startMV at each number of searches is indicated by a white diamond.
  • the optimal vector bestMV in each search round is indicated by a black diamond.
  • search candidate points at each number of searches are indicated by black circles.
  • points that have already been searched at each number of searches are indicated by white circles.
  • the matching motion deriving unit 30373c initializes the search round numIter to 0 before starting the search. Then, at the start time of each search round, the matching cost of the search start point is set to the minimum cost minCost, and the initial value ( ⁇ 1) is set to the optimal candidate index bestIdx.
  • mcost (X) is a function for deriving a matching cost in the search vector X.
  • the matching motion derivation unit 30373c selects and evaluates search candidate points centered on the search start point in each search round. Specifically, for each search candidate index Idx, the matching motion derivation unit 30373c adds the value (offsetCand [Idx]) of the offset candidate (offsetCand) to the coordinate (position) startMV of the search start point. Select point coordinates.
  • candMV startMV + offsetCand [Idx]
  • offsetCand [x] is an offset candidate that is added to the coordinates of the search start point in order to set the search candidate point.
  • the first motion vector search unit 303731 offsetCand [8] ((0, 2), (1, 1), (2, 0), (1, -1), (0, -2), (-1, -1), (-2, 0), (-1, 1) ⁇ is used.
  • the first motion vector search unit 303731 offsetCand [4] ⁇ (0, 2), (2, 0), (0, -2), (-2, 0) ⁇ may be used.
  • the first motion vector search unit 303731 excludes points 1, 3, 5, and 7 from the search candidate points among the points 0 to 7 in FIG. 47A, and sets the search candidate points to 0, 2 4 and 6.
  • the search candidate point indicated by the optimal vector bestMV is used as the search start point of the next round.
  • startMV bestMV (here P (1)) Whether or not the optimal vector bestMV has been updated is determined based on whether or not the optimal vector bestMV is different from the search start point, and whether or not bestIdx is updated to a value other than the initial value ( ⁇ 1), or , MinCost can also be determined based on whether or not the value is updated to a value other than the initial cost of the starting point.
  • search start index nDirectStart and the search end index nDirectEnd used in the next round may be determined by the following formulas depending on the position of the optimal vector bestMV (optimum candidate index bestIdx). This makes it possible to search efficiently without searching again for search points that have already been searched.
  • nStep 2-(bestIdx & 1)
  • nDirectStart bestIdx-nStep
  • nDirectEnd bestIdx + nStep
  • the search candidate point 2 is set as the initial vector startMV (search start point P1) in the current search, and is a plurality of points arranged in a diamond shape with the search start point P1 as the center, and is still selected as the search candidate point. Any of the points that are not used is set as a search candidate point.
  • the first motion vector search unit 303731 uses points 0 to 4 as search candidate points as shown in the second row of FIG.
  • Point 1 that was a search candidate is set as an initial vector startMV (search start point P2) in the current search.
  • the first motion vector search unit 303731 is a plurality of points arranged in a diamond shape with the search start point P2 as the center, and is not yet selected as a search candidate point but is present within the search range. One of them is set as a search candidate point.
  • the first motion vector search unit 303731 may also prohibit a search in an oblique direction when selecting a search candidate point in the second search and the third search.
  • the first motion vector search unit 303731 may prohibit a search in an oblique direction for one or both of the second search and the third search. In this case, the first motion vector search unit 303731 may determine whether or not to prohibit the search in the oblique direction for each search.
  • the first motion vector search unit 303731 limits the offset candidates by the same method as the method in which the search in the oblique direction is prohibited in the first search described above, and sets the search candidate points. limit.
  • the first motion vector search unit 303731 may perform another step search.
  • the following values are used as offset candidates (offsetCand).
  • FIG. 47B shows an example in which the cross search is performed after the diamond search.
  • the first motion vector search unit 303731 uses the search start point (the search start point P2 in the third row in FIG. 47A) as the center, and points at the top, bottom, left, and right (cross) positions as search candidate points. Select in turn.
  • the search candidate points including the search start point P2 that give the smallest matching cost among the search candidate points 0 to 3 on the upper, lower, left, and right sides of the search start point P2 are the points to the right of P2.
  • the first motion vector search unit 303731 selects a point on the right side of P2 as the end point of the optimal vector bestMV for the prediction block PU.
  • the second motion vector search unit 303732 may also prohibit the search in the oblique direction, like the first motion vector search unit 303731.
  • ⁇ Step search change example 1> In the above-described example, the case where the search in the oblique direction is prohibited in the local search (block search) by the first motion vector search unit 303731 and the second motion vector search unit 303732 is described. However, the present embodiment is not limited to this. In the present embodiment, the search in the oblique direction is not prohibited in the block search by the first motion vector search unit 303731, and the search in the oblique direction is performed in the local search (sub-block search) by the second motion vector search unit 303732. It may be prohibited.
  • the first motion vector search unit 303731 arranges in a diamond shape with the search start point as the center, and selects all points that have not yet been selected as search candidate points. It is set as a search candidate point.
  • the second motion vector search unit 303732 is arranged in a diamond shape with the search start point as the center, and searches for points that are oblique to the search start point among the points not yet selected as search candidate points. Exclude from candidate points.
  • the second motion vector search unit 303732 when searching for a motion vector centering on P0, the second motion vector search unit 303732 excludes points that are diagonally located with respect to P0 from the search candidate points among the points arranged in a diamond shape. Limit search candidate points.
  • the sub-block search is more detailed than the block search, so the amount of processing tends to increase.
  • the second motion vector search unit 303732 can reduce the processing amount of the sub-block search.
  • the first motion vector search unit 303731 may prohibit a search in an oblique direction in the step round process after a predetermined number of times among these local searches.
  • the first motion vector search unit 303731 does not prohibit the search in the diagonal direction in the step round processing up to the Mth time (M is a natural number), and prohibits the search in the diagonal direction in the step round processing after the (M + 1) th time. Also good.
  • the search in the oblique direction is allowed and the search in the oblique direction is prohibited in the step round process after approaching the optimum search point while searching for a suitable search point with a low matching cost. , Can reduce the processing amount.
  • the second motion vector search unit 303732 may also prohibit a search in an oblique direction in a step round process after a predetermined number of times.
  • the first motion vector search unit 303731 may prohibit a search in an oblique direction according to a search candidate point in a block search.
  • the first motion vector search unit 303731 may not prohibit the search in the oblique direction when the search candidate point is one pixel position (full-pel position).
  • the first motion vector search unit 303731 may prohibit the search in an oblique direction and limit the search direction to up, down, left, and right. A specific example is shown below.
  • the first motion vector search unit 303731 uses an offset candidate offsetCand [Idx] including all search candidate points as an offset candidate.
  • the first motion vector search unit 303731 uses the offset candidate offsetCand [Idx] to derive a motion vector candidate from the following equation.
  • candMV startMV + offsetCand [Idx]
  • the first motion vector search unit 303731 moves the motion compensated image (predicted image) of the motion vector candidate candMV. And calculate the matching cost.
  • the search candidate point is a decimal pixel position (for example, a position that is not an intersection of each pixel in the first row in FIG. 47A)
  • the first motion vector search unit 303731 Only when the search start point startMV is in the vertical and horizontal directions, the matching cost is calculated.
  • the first motion vector search unit 303731 When the search candidate point is oblique to the search start point startMV, the first motion vector search unit 303731 does not generate a motion compensated image (predicted image) of the motion vector candidate candMV. In this case, the first motion vector search unit 303731 may derive a predetermined large value (for example, 2 15 ⁇ 1) as the matching cost, or may not derive the matching cost. Thereby, the processing amount can be reduced.
  • a predetermined large value for example, 2 15 ⁇ 1
  • the matching cost may be calculated in the sub-block search as in the first motion vector search unit 303731.
  • the first motion vector search unit 303731 may prohibit the search for the motion vector when at least a part of the reference block indicated by the searched motion vector is outside the screen of the reference image. In other words, the first motion vector search unit 303731 may not generate a predicted image when at least a part of the reference block indicated by the motion vector is outside the screen of the reference image.
  • FIG. 48 is a flowchart showing an outline of a flow for determining whether or not to generate a predicted image.
  • the predicted image generation determination process by the first motion vector search unit 303731 includes the following three steps (Step S431) to (Step S433).
  • Step S431 the first motion vector search unit 303731 determines whether at least a part of the reference block is outside the screen. If first motion vector search section 303731 determines that at least a part of the reference block is outside the screen (Y in step S431), the first motion vector search unit 303731 proceeds to step S432. If the first motion vector search unit 303731 determines that at least a part of the reference block is not outside the screen (N in step S431), the process proceeds to step S433.
  • the first motion vector search unit 303731 may determine that the reference block is outside the screen of the reference image when at least one of the following formulas B-1 to B-4 is satisfied.
  • Equation B-1 is an equation used when determining whether or not the left end of the reference block indicated by the motion vector is outside the screen of the reference image.
  • Expression B-2 is an expression used when determining whether the right end of the reference block is outside the screen of the reference image.
  • Expression B-3 is an expression used when determining whether or not the upper end of the reference block is outside the screen of the reference image.
  • Expression B-4 is an expression used when determining whether or not the lower end of the reference block is outside the screen of the reference image.
  • xInt-NTAPS / 2 + 1 ⁇ 0 (Formula B-1) xInt + BLKW + NTAPS / 2-1> pic_width-1 (Formula B-2) yInt-NTAPS / 2 + 1 ⁇ 0 (Formula B-3) yInt + BLKH + NTAPS / 2-1> pic_height-1 (Formula B-4)
  • xInt and yInt in the equations B-1 to B-4 are shifted by the motion vector (mv_x, mv_y) from the upper left block coordinates (xPb, yPb) when the motion vector accuracy is 1 / M pel accuracy. Points to the integer position (xInt, yInt) of the reference image corresponding to the in-block coordinates (x, y).
  • NTAPS in Equations B-1 to B-4 represents the number of filter taps of the motion compensation filter.
  • BLKW and BLKH in the formulas B-1 to B-4 indicate the horizontal width and height of the corresponding block, respectively.
  • pic_width and pic_height in Expressions B-1 to B-4 indicate the horizontal width and height of the reference image, respectively.
  • Step S432 the first motion vector search unit 303731 determines not to generate a motion compensated image (predicted image) of the motion vector candMV.
  • the first motion vector search unit 303731 may derive a predetermined large value (for example, 2 15 ⁇ 1) as the matching cost, or may not derive the matching cost. This eliminates the need for padding processing that occurs when the reference block is outside the screen of the reference image, thereby reducing the amount of processing.
  • Step S433 the first motion vector search unit 303731 determines to generate a motion compensated image (predicted image) and calculate a matching cost.
  • the second motion vector search unit 303732 may also perform the above-described prediction image generation process in the sub-block search, similarly to the first motion vector search unit 303731.
  • the first motion vector search unit 303731 does not determine whether or not the reference block is outside the screen of the reference image in Step S431 of Modification 4 described above, but the reference block is within a certain range from the boundary of the reference image. You may determine whether it is outside.
  • FIG. 49 is a flowchart showing an outline of a flow for determining whether or not to generate a predicted image.
  • the predicted image generation determination process by the first motion vector search unit 303731 includes three steps (step S441) to (step S443). Steps S442 and S443 are the same as steps S432 and S433 described above, respectively, and thus description thereof is omitted.
  • Step S441 the first motion vector search unit 303731 determines whether at least a part of the reference block is outside a certain range from the boundary of the reference image.
  • step S441 determines that at least a part of the reference block is outside the screen (Y in step S441), the process proceeds to step S442. If the first motion vector search unit 303731 determines that at least part of the reference block is not outside the screen (N in step S441), the process proceeds to step S443.
  • the first motion vector search unit 303731 uses the following expressions C-1 to C-4: When at least one of the conditions is satisfied, it may be determined that at least a part of the reference block is outside a certain range from the boundary of the reference image.
  • the following expression C-1 is an expression used when determining whether or not the left end of the reference block indicated by the motion vector is outside a certain range from the boundary of the reference image.
  • Expression C-2 is an expression used when determining whether the right end of the reference block is outside a certain range from the boundary of the reference image.
  • Expression C-3 is an expression used when determining whether or not the upper end of the reference block is outside a certain range from the boundary of the reference image.
  • Expression C-4 is an expression used when determining whether or not the lower end of the reference block is outside a certain range from the boundary of the reference image.
  • padW and padH in the formulas C-1 to C-4 mean padW indicating a predetermined range in the horizontal direction and padH indicating a predetermined range in the vertical direction, respectively.
  • the first motion vector search unit 303731 when at least a part of the reference block protrudes from the boundary of the reference image but does not protrude outside the certain range, the first motion vector search unit 303731 generates a motion compensated image (predicted image). Can be generated.
  • a motion compensated image predicted image
  • the second motion vector search unit 303732 may also perform the above-described prediction image generation process in the sub-block search, similarly to the first motion vector search unit 303731.
  • raster search will be described.
  • the matching motion deriving unit 30373c performs a motion search to which the raster search is applied, the matching motion deriving unit 30373c comprehensively selects search points within the search range at regular intervals, and the matching cost is determined by a raster scan.
  • Evaluate in order the raster scan starts from the upper left of the search range, examines the pixels from the left side to the right until it reaches the right end, and if it reaches the right end, it goes down one row and goes from the left end to the right again. This is an exhaustive search method that sequentially examines the pixels.
  • the matching motion deriving unit 30373c selects a search vector that gives the smallest matching cost among the matching costs calculated for each search vector from the start point to the end point set in the raster scan order.
  • an extended raster scan may be used instead of the raster scan.
  • each point in the block is scanned in a predetermined scan order like the raster scan. For example, a spiral scan that scans spirally from the center toward the periphery is used.
  • At least one of the first motion vector search unit 303731 and the second motion vector search unit 303732 may prohibit the search in the oblique direction as in the above-described step search.
  • the image encoding device 11c includes a prediction parameter encoding unit 111c instead of the prediction parameter encoding unit 111b in the third embodiment.
  • the prediction parameter encoding unit 111c includes an inter prediction parameter encoding unit (motion vector deriving unit) 112c instead of the inter prediction parameter encoding unit 112b in the third embodiment.
  • FIG. 42 is a block diagram illustrating a configuration of the inter prediction parameter encoding unit 112c of the image encoding device 11c according to the fourth embodiment.
  • the inter prediction parameter encoding unit 112c includes a subblock prediction parameter deriving unit 1125c instead of the subblock prediction parameter deriving unit 1125b in the third embodiment.
  • the sub-block prediction parameter deriving unit 1125c includes a matching motion deriving unit 11253c instead of the matching motion deriving unit 11253 in the third embodiment.
  • the inter prediction parameter encoding unit 112c has the same configuration as the inter prediction parameter encoding unit 112b in the third embodiment.
  • the matching motion deriving unit 11253c has the same configuration as the above-described matching motion deriving unit 30373c, description thereof is omitted here.
  • the image encoding devices 11, 11a, 11b, and 11c and part of the image decoding devices 31, 31a, 31b, and 31c in the above-described embodiment for example, the entropy decoding unit 301, the prediction parameter decoding units 302, 302b, and 302c, Loop filter 305, predicted image generation units 308 and 308a, inverse quantization / inverse transformation unit 311, addition unit 312, predicted image generation units 101 and 101a, subtraction unit 102, transformation / quantization unit 103, entropy encoding unit 104,
  • the inverse quantization / inverse transform unit 105, the loop filter 107, the encoding parameter determination unit 110, and the prediction parameter encoding units 111, 111b, and 111c may be realized by a computer.
  • the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed.
  • the “computer system” is a computer system built in any of the image encoding devices 11, 11a, 11b, and 11c and the image decoding devices 31, 31a, 31b, and 31c. It shall include hardware such as equipment.
  • the “computer-readable recording medium” refers to a storage device such as a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a hard disk built in a computer system.
  • the “computer-readable recording medium” is a medium that holds a program dynamically for a short time, such as a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line,
  • a volatile memory inside a computer system serving as a server or a client may be included and a program that holds a program for a certain period of time.
  • the program may be a program for realizing a part of the above-described functions, or may be a program that can be realized by a combination with a program already recorded in the computer system.
  • part or all of the image encoding devices 11, 11a, 11b, and 11c and the image decoding devices 31, 31a, 31b, and 31c in the above-described embodiment are realized as an integrated circuit such as an LSI (Large Scale Integration). May be.
  • LSI Large Scale Integration
  • Each functional block of the image encoding devices 11, 11a, 11b, and 11c and the image decoding devices 31, 31a, 31b, and 31c may be individually converted into a processor, or a part or all of them may be integrated into a processor.
  • the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor.
  • an integrated circuit based on the technology may be used.
  • the above-described image encoding devices 11, 11a, 11b, and 11c and the image decoding devices 31, 31a, 31b, and 31c can be mounted and used in various devices that perform transmission, reception, recording, and reproduction of moving images.
  • the moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.
  • FIG. 50 (a) is a block diagram showing a configuration of a transmission apparatus PROD_A equipped with the image encoding apparatuses 11, 11a, 11b, and 11c.
  • the transmission apparatus PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image, and with the encoded data obtained by the encoding unit PROD_A1.
  • a modulation unit PROD_A2 that obtains a modulation signal
  • a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided.
  • the above-described image encoding devices 11, 11a, 11b, and 11c are used as the encoding unit PROD_A1.
  • Transmission device PROD_A as a source of moving images to be input to the encoding unit PROD_A1, a camera PROD_A4 that captures moving images, a recording medium PROD_A5 that records moving images, an input terminal PROD_A6 for inputting moving images from the outside, and An image processing unit A7 that generates or processes an image may be further provided.
  • FIG. 50A illustrates a configuration in which the transmission apparatus PROD_A includes all of these, but some of them may be omitted.
  • the recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 in accordance with the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.
  • FIG. 50 is a block diagram showing a configuration of a receiving device PROD_B equipped with the image decoding devices 31, 31a, 31b, and 31c.
  • the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a demodulator A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2.
  • the above-described image decoding devices 31, 31a, 31b, and 31c are used as the decoding unit PROD_B3.
  • the receiving device PROD_B is a display destination PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording a moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3 PROD_B6 may be further provided.
  • FIG. 50B a configuration in which all of these are provided in the receiving device PROD_B is illustrated, but a part may be omitted.
  • the recording medium PROD_B5 may be for recording a non-encoded moving image, or is encoded by a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.
  • the transmission medium for transmitting the modulation signal may be wireless or wired.
  • the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.
  • a terrestrial digital broadcast broadcasting station (broadcasting equipment, etc.) / Receiving station (television receiver, etc.) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals by wireless broadcasting.
  • a broadcasting station (broadcasting equipment, etc.) / Receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wired broadcasting.
  • a server workstation, etc.
  • Client television receiver, personal computer, smartphone, etc.
  • VOD Video On Demand
  • video sharing service using the Internet transmits and receives a modulated signal by communication.
  • PROD_A / receiving device PROD_B usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN.
  • the personal computer includes a desktop PC, a laptop PC, and a tablet PC.
  • the smartphone also includes a multi-function mobile phone terminal.
  • the moving image sharing service client has a function of encoding a moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.
  • FIG. 51A is a block diagram showing a configuration of a recording apparatus PROD_C equipped with the above-described image encoding apparatuses 11, 11a, 11b, and 11c.
  • the recording apparatus PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M.
  • a writing unit PROD_C2 for writing.
  • the above-described image encoding devices 11, 11a, 11b, and 11c are used as the encoding unit PROD_C1.
  • the recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) and SSD (Solid State Drive). Further, (2) an SD memory card, a USB (Universal Serial Bus) flash memory, or the like connected to the recording device PROD_C may be used. Further, (3) DVD (Digital Versatile Disc) and BD (Blu-ray Disc: registered trademark), etc. may be loaded into a drive device (not shown) built in the recording device PROD_C. .
  • the recording device PROD_C is a camera PROD_C3 that captures moving images as a source of moving images to be input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting moving images from the outside, and a reception for receiving moving images
  • a unit PROD_C5 and an image processing unit PROD_C6 for generating or processing an image may be further provided.
  • FIG. 51A a configuration in which the recording apparatus PROD_C includes all of these is illustrated, but a part may be omitted.
  • the receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.
  • Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main source of moving images). .
  • a camcorder in this case, the camera PROD_C3 is a main source of moving images
  • a personal computer in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images
  • a smartphone In this case, the camera PROD_C3 or the reception unit PROD_C5 is a main source of moving images
  • the like is also an example of such a recording apparatus PROD_C.
  • FIG. 51 (b) is a block diagram showing a configuration of a playback device PROD_D equipped with the above-described image decoding devices 31 and 31a.
  • the playback device PROD_D reads a moving image by decoding a read unit PROD_D1 that reads encoded data written to the recording medium PROD_M and a read unit PROD_D1 that reads the encoded data. And a decoding unit PROD_D2 to obtain.
  • the above-described image decoding devices 31, 31a, 31b, and 31c are used as the decoding unit PROD_D2.
  • the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD and SSD. Further, (2) a type connected to the playback device PROD_D, such as an SD memory card and a USB flash memory, may be used. Also, (3) a drive device (not shown) built in the playback device PROD_D, such as a DVD and a BD, may be loaded.
  • the playback device PROD_D has a display unit PROD_D3 that displays a moving image as a supply destination of the moving image output by the decoding unit PROD_D2, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image.
  • PROD_D5 may be further provided.
  • FIG. 51B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but some of them may be omitted.
  • the transmission unit PROD_D5 may transmit a non-encoded moving image, or transmits encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, an encoding unit (not shown) that encodes a moving image by a transmission encoding method may be interposed between the decoding unit PROD_D2 and the transmission unit PROD_D5.
  • Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, the output terminal PROD_D4 to which a television receiver or the like is connected is a main moving image supply destination).
  • a television receiver in this case, the display PROD_D3 is a main supply destination of moving images
  • a digital signage also referred to as an electronic signboard or an electronic bulletin board
  • the display PROD_D3 or the transmission unit PROD_D5 is the main supply of moving images.
  • Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a video)
  • An image is a main supply destination
  • a smartphone in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main supply destination of moving images
  • the like are examples of such a playback device PROD_D.
  • Each block of the above-described image decoding devices 31, 31a, 31b, and 31c and the image encoding devices 11, 11a, 11b, and 11c is realized by hardware by a logic circuit formed on an integrated circuit (IC chip). Alternatively, it may be realized by software using a CPU (Central Processing Unit).
  • IC chip integrated circuit
  • CPU Central Processing Unit
  • each of the above devices includes a CPU that executes instructions of a program that realizes each function, a ROM (Read Memory) that stores the program, a RAM (RandomAccess Memory) that expands the program, the program, and various data
  • a storage device such as a memory for storing the.
  • the object of the embodiment of the present invention is to record the program code (execution format program, intermediate code program, and source program) of the control program for each of the above devices, which is software that realizes the above-described functions, in a computer-readable manner. This can also be achieved by supplying a medium to each of the above devices, and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).
  • Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and CD-ROMs (Compact Disc Read-Only Memory) / MO discs (Magneto-Optical discs).
  • tapes such as magnetic tapes and cassette tapes
  • magnetic disks such as floppy (registered trademark) disks / hard disks
  • CD-ROMs Compact Disc Read-Only Memory
  • MO discs Magnetic-Optical discs
  • IC cards including memory cards
  • Cards such as optical cards
  • Semiconductor memories such as flash ROM, or PLD (Programmable logic device ) And FPGA (Field Programmable Gate Gate Array) and the like.
  • each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network.
  • the communication network is not particularly limited as long as it can transmit the program code.
  • Internet intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Area Antenna / television / Cable Television), Virtual Private Network (Virtual Private Network) Network), telephone line network, mobile communication network, satellite communication network, and the like.
  • the transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type.
  • IEEE Institute of Electrical and Electronic Engineers 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc. wired such as IrDA (Infrared Data Association) and remote control , BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It can also be used wirelessly.
  • the embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.
  • Embodiments of the present invention are preferably applied to an image decoding apparatus that decodes encoded data in which image data is encoded, and an image encoding apparatus that generates encoded data in which image data is encoded. it can. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.
  • inter prediction parameter encoding unit (motion vector deriving unit) DESCRIPTION OF SYMBOLS 1121 ... Merge prediction parameter derivation
  • AMVP prediction parameter derivation unit 3033 ... vector candidate derivation unit 3036 ... merge prediction parameter derivation unit 3038 ... addition unit 30361 ... merge candidate derivation unit 303611 ... merge candidate storage unit 30362 ... merge candidate selection unit 304 ... intra prediction parameter decoding unit 306 ... Reference picture memory 307 ... Prediction parameter memories 308 and 308a ... Prediction image generation units 309 and 309a ... Inter prediction image generation units 3091 and 3091a ... Motion compensation unit 30911 ... Motion compensation gradient unit 309122 ... Slope correction coefficient derivation unit 309112 ... Slope derivation Unit 3094 ... weight prediction unit 310 ... intra prediction image generation unit 311 ...

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon l'invention, un dispositif (31) de décodage vidéo comporte une unité (308) de génération d'image de prévision mettant en oeuvre au moins un mode parmi un mode de prévision unique, un mode de prévision double et un mode de prévision BIO pour générer une image de prévision. Cette unité (308) de génération d'image de prévision empêche la génération d'image de prévision mettant en oeuvre le mode BIO, dans le cas d'un bloc de référence à l'extérieur du plan d'une image de référence dans au moins une image parmi une première image de référence et une deuxième image de référence.
PCT/JP2018/022196 2017-06-14 2018-06-11 Dispositif de décodage vidéo, dispositif de codage vidéo, dispositif de génération d'image de prévision et dispositif de dérivation de vecteur de mouvement Ceased WO2018230493A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2017117273 2017-06-14
JP2017-117273 2017-06-14
JP2017-123731 2017-06-23
JP2017123731 2017-06-23

Publications (1)

Publication Number Publication Date
WO2018230493A1 true WO2018230493A1 (fr) 2018-12-20

Family

ID=64658656

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/022196 Ceased WO2018230493A1 (fr) 2017-06-14 2018-06-11 Dispositif de décodage vidéo, dispositif de codage vidéo, dispositif de génération d'image de prévision et dispositif de dérivation de vecteur de mouvement

Country Status (1)

Country Link
WO (1) WO2018230493A1 (fr)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019082985A1 (fr) * 2017-10-27 2019-05-02 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Dispositif de codage, dispositif de décodage, procédé de codage et procédé de décodage
WO2020130020A1 (fr) * 2018-12-21 2020-06-25 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Dispositif de codage, dispositif de décodage, procédé de codage et procédé de décodage
WO2020142762A1 (fr) * 2019-01-06 2020-07-09 Beijing Dajia Internet Information Technology Co., Ltd. Commande de largeur de bit destinée à un flux optique bidirectionnel
JP2020145644A (ja) * 2019-03-08 2020-09-10 株式会社Ｊｖｃケンウッド 画像符号化装置
WO2020181997A1 (fr) * 2019-03-08 2020-09-17 Huawei Technologies Co., Ltd. Encodeur, décodeur et procédés correspondants pour l'interprédiction
JP2021506178A (ja) * 2017-12-08 2021-02-18 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 画像符号化装置、画像復号装置、画像符号化方法、および画像復号方法
JPWO2020045050A1 (ja) * 2018-08-27 2021-08-10 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 符号化装置、復号装置、符号化方法及び復号方法
CN113498607A (zh) * 2019-03-13 2021-10-12 腾讯美国有限责任公司 用于小子块仿射帧间预测的方法和装置
CN113615197A (zh) * 2019-03-26 2021-11-05 北京达佳互联信息技术有限公司 用于双向光流的比特深度控制的方法和设备
JP2022506161A (ja) * 2018-11-05 2022-01-17 北京字節跳動網絡技術有限公司 精緻化を伴うインター予測のための補間
EP3874756A4 (fr) * 2019-03-19 2022-01-26 Tencent America LLC Procédé et appareil de codage vidéo
US20220038727A1 (en) * 2020-08-03 2022-02-03 Alibaba Group Holding Limited Systems and methods for bi-directional gradient correction
CN114175659A (zh) * 2019-06-25 2022-03-11 北京达佳互联信息技术有限公司 用于双向光流的比特宽度控制的装置和方法
EP4476911A4 (fr) * 2022-02-07 2025-12-31 Tencent America LLC Contrainte basée sur un sous-bloc sur une bi-prédiction de conditions hors limite

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016195451A (ja) * 2014-02-21 2016-11-17 パナソニック株式会社 画像符号化方法

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016195451A (ja) * 2014-02-21 2016-11-17 パナソニック株式会社 画像符号化方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEN, J. L. ET AL.: "Algorithm description of joint exploration test model 4", JVET-C1001_V3, ITU-T, 26 May 2016 (2016-05-26), pages 12 - 22, Retrieved from the Internet <URL:http://phenix.it-sudparis.eu/jvet/doc_end_user/documents/4_Chengdu/wg11/JVET-D0002-v1.zip> *

Cited By (80)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021180494A (ja) * 2017-10-27 2021-11-18 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America 復号装置、復号方法、及び、非一時的記憶媒体
JP7732053B2 (ja) 2017-10-27 2025-09-01 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ ビットストリーム送信装置、ビットストリーム送信方法、ビットストリーム受信装置及びビットストリーム受信方法
US12389028B2 (en) 2017-10-27 2025-08-12 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
JP2024156889A (ja) * 2017-10-27 2024-11-06 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ ビットストリーム送信装置、ビットストリーム送信方法、ビットストリーム受信装置及びビットストリーム受信方法
JP7535642B2 (ja) 2017-10-27 2024-08-16 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置、及び、符号化方法
JPWO2019082985A1 (ja) * 2017-10-27 2020-11-12 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 符号化装置、復号装置、符号化方法及び復号方法
US10856005B2 (en) 2017-10-27 2020-12-01 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
WO2019082985A1 (fr) * 2017-10-27 2019-05-02 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Dispositif de codage, dispositif de décodage, procédé de codage et procédé de décodage
US12058366B2 (en) 2017-10-27 2024-08-06 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
JP2024003221A (ja) * 2017-10-27 2024-01-11 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置、及び、符号化方法
JP7389282B2 (ja) 2017-10-27 2023-11-29 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 復号装置、及び、復号方法
JP2023040271A (ja) * 2017-10-27 2023-03-22 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 復号装置、及び、復号方法
US11240530B2 (en) 2017-10-27 2022-02-01 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US12063383B2 (en) 2017-12-08 2024-08-13 Panasonic Intellectual Property Corporation Of America Image encoding device, image decoding device, image encoding method, and image decoding method
JP2022082608A (ja) * 2017-12-08 2022-06-02 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置および復号装置
JP2023099740A (ja) * 2017-12-08 2023-07-13 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置および復号装置
US11218717B2 (en) 2017-12-08 2022-01-04 Panasonic Intellectual Property Corporation Of America Image encoding device, image decoding device, image encoding method, and image decoding method
US11601669B2 (en) 2017-12-08 2023-03-07 Panasonic Intellectual Property Corporation Of America Image encoding device, image decoding device, image encoding method, and image decoding method
JP7753488B2 (ja) 2017-12-08 2025-10-14 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置および復号装置
JP2021506178A (ja) * 2017-12-08 2021-02-18 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 画像符号化装置、画像復号装置、画像符号化方法、および画像復号方法
JP7286832B2 (ja) 2017-12-08 2023-06-05 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置および復号装置
JP7562760B2 (ja) 2017-12-08 2024-10-07 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置および復号装置
JP7046186B2 (ja) 2017-12-08 2022-04-01 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 画像符号化装置、画像復号装置、画像符号化方法、および画像復号方法
JP2024174030A (ja) * 2017-12-08 2024-12-13 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置および復号装置
JP2025015641A (ja) * 2018-08-27 2025-01-30 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ ビットストリームの送信装置
US11212549B2 (en) 2018-08-27 2021-12-28 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US12200243B2 (en) 2018-08-27 2025-01-14 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
JP7728950B2 (ja) 2018-08-27 2025-08-25 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ ビットストリームの送信装置
JP7590477B2 (ja) 2018-08-27 2024-11-26 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置及び符号化方法
JP7461885B2 (ja) 2018-08-27 2024-04-04 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 復号装置及び復号方法
JP2023060057A (ja) * 2018-08-27 2023-04-27 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置及び符号化方法
JPWO2020045050A1 (ja) * 2018-08-27 2021-08-10 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 符号化装置、復号装置、符号化方法及び復号方法
US11785243B2 (en) 2018-08-27 2023-10-10 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
JP7529827B2 (ja) 2018-11-05 2024-08-06 北京字節跳動網絡技術有限公司 精緻化を伴うインター予測のための補間
US11706421B2 (en) 2018-11-05 2023-07-18 Beijing Bytedance Network Technology Co., Ltd. Inter prediction with refinement in video processing
JP2022506161A (ja) * 2018-11-05 2022-01-17 北京字節跳動網絡技術有限公司 精緻化を伴うインター予測のための補間
JP7231727B2 (ja) 2018-11-05 2023-03-01 北京字節跳動網絡技術有限公司 精緻化を伴うインター予測のための補間
US11902535B2 (en) 2018-11-05 2024-02-13 Beijing Bytedance Network Technology Co., Ltd Prediction precision improvements in video coding
JP2023062073A (ja) * 2018-11-05 2023-05-02 北京字節跳動網絡技術有限公司 精緻化を伴うインター予測のための補間
US12108073B2 (en) 2018-12-21 2024-10-01 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
TWI840475B (zh) * 2018-12-21 2024-05-01 美商松下電器（美國）知識產權公司 編碼裝置、解碼裝置、編碼方法及解碼方法
WO2020130020A1 (fr) * 2018-12-21 2020-06-25 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Dispositif de codage, dispositif de décodage, procédé de codage et procédé de décodage
CN113170179B (zh) * 2018-12-21 2024-10-18 松下电器（美国）知识产权公司 编码装置、解码装置、编码方法和解码方法
US11743484B2 (en) 2018-12-21 2023-08-29 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US11425411B2 (en) 2018-12-21 2022-08-23 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
JP7189234B2 (ja) 2018-12-21 2022-12-13 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置及び復号装置
CN113170179A (zh) * 2018-12-21 2021-07-23 松下电器（美国）知识产权公司 编码装置、解码装置、编码方法和解码方法
JPWO2020130020A1 (ja) * 2018-12-21 2021-09-30 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 符号化装置、復号装置、符号化方法及び復号方法
TWI891333B (zh) * 2018-12-21 2025-07-21 美商松下電器（美國）知識產權公司 編碼裝置、解碼裝置及非暫時性電腦可讀取之媒體
JP2024099044A (ja) * 2018-12-21 2024-07-24 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 送信装置及び送信方法
JP7697101B2 (ja) 2018-12-21 2025-06-23 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 送信装置及び送信方法
JP7483836B2 (ja) 2018-12-21 2024-05-15 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化方法、復号方法、符号化装置及び復号装置
JP2023014296A (ja) * 2018-12-21 2023-01-26 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化方法及び復号方法
CN113613019B (zh) * 2019-01-06 2022-06-07 北京达佳互联信息技术有限公司 视频解码方法、计算设备和介质
US12137244B2 (en) 2019-01-06 2024-11-05 Beijing Dajia Internet Information Technology Co., Ltd. Bit-width control for bi-directional optical flow
US11388436B2 (en) 2019-01-06 2022-07-12 Beijing Dajia Internet Information Technology Co., Ltd. Bit-width control for bi-directional optical flow
US12238331B2 (en) 2019-01-06 2025-02-25 Beijing Dajia Internet Information Technology Co., Ltd. Bit-width control for bi-directional optical flow
CN116708772B (zh) * 2019-01-06 2024-01-23 北京达佳互联信息技术有限公司 视频解码方法、计算设备和介质
WO2020142762A1 (fr) * 2019-01-06 2020-07-09 Beijing Dajia Internet Information Technology Co., Ltd. Commande de largeur de bit destinée à un flux optique bidirectionnel
CN116708772A (zh) * 2019-01-06 2023-09-05 北京达佳互联信息技术有限公司 视频解码方法、计算设备和介质
US11743493B2 (en) 2019-01-06 2023-08-29 Beijing Dajia Internet Information Technology Co., Ltd. Bit-width control for bi-directional optical flow
CN113613019A (zh) * 2019-01-06 2021-11-05 北京达佳互联信息技术有限公司 双向光流的比特宽度控制
JP7215243B2 (ja) 2019-03-08 2023-01-31 株式会社Ｊｖｃケンウッド 画像符号化装置
WO2020181997A1 (fr) * 2019-03-08 2020-09-17 Huawei Technologies Co., Ltd. Encodeur, décodeur et procédés correspondants pour l'interprédiction
US11968387B2 (en) 2019-03-08 2024-04-23 Huawei Technologies Co., Ltd. Encoder, a decoder and corresponding methods for inter prediction using bidirectional optical flow
JP2020145644A (ja) * 2019-03-08 2020-09-10 株式会社Ｊｖｃケンウッド 画像符号化装置
US12382084B2 (en) 2019-03-08 2025-08-05 Huawei Technologies Co., Ltd. Encoder, a decoder and corresponding methods for inter prediction using gradient parameters
CN113498607B (zh) * 2019-03-13 2024-04-05 腾讯美国有限责任公司 视频编码方法、解码方法、装置和可读介质
CN113498607A (zh) * 2019-03-13 2021-10-12 腾讯美国有限责任公司 用于小子块仿射帧间预测的方法和装置
US11343525B2 (en) 2019-03-19 2022-05-24 Tencent America LLC Method and apparatus for video coding by constraining sub-block motion vectors and determining adjustment values based on constrained sub-block motion vectors
EP3874756A4 (fr) * 2019-03-19 2022-01-26 Tencent America LLC Procédé et appareil de codage vidéo
US11683518B2 (en) 2019-03-19 2023-06-20 Tencent America LLC Constraining sub-block motion vectors and determining adjustment values based on the constrained sub-block motion vectors
CN113615197B (zh) * 2019-03-26 2023-03-07 北京达佳互联信息技术有限公司 用于双向光流的比特深度控制的方法和设备
CN113615197A (zh) * 2019-03-26 2021-11-05 北京达佳互联信息技术有限公司 用于双向光流的比特深度控制的方法和设备
CN114175659A (zh) * 2019-06-25 2022-03-11 北京达佳互联信息技术有限公司 用于双向光流的比特宽度控制的装置和方法
CN115699744B (zh) * 2020-08-03 2024-03-26 阿里巴巴达摩院(杭州)科技有限公司 双向预测校正系统和方法
US11582474B2 (en) * 2020-08-03 2023-02-14 Alibaba Group Holding Limited Systems and methods for bi-directional gradient correction
CN115699744A (zh) * 2020-08-03 2023-02-03 阿里巴巴集团控股有限公司 双向预测校正系统和方法
US20220038727A1 (en) * 2020-08-03 2022-02-03 Alibaba Group Holding Limited Systems and methods for bi-directional gradient correction
EP4476911A4 (fr) * 2022-02-07 2025-12-31 Tencent America LLC Contrainte basée sur un sous-bloc sur une bi-prédiction de conditions hors limite

Similar Documents

Publication Publication Date Title
JP7421586B2 (ja) 復号装置および符号化装置
US11979602B2 (en) Motion vector generation device, a prediction image generation device, a video decoding device and a video coding device
WO2018230493A1 (fr) Dispositif de décodage vidéo, dispositif de codage vidéo, dispositif de génération d&#39;image de prévision et dispositif de dérivation de vecteur de mouvement
JP7026049B2 (ja) アフィン動きベクトル導出装置、予測画像生成装置、動画像復号装置、および動画像符号化装置
JP7706621B2 (ja) 動画像復号装置および動画像符号化装置
WO2018221368A1 (fr) Dispositif de décodage d&#39;image animée et dispositif de codage d&#39;image animée
US20200021837A1 (en) Video decoding apparatus and video coding apparatus
WO2019221072A1 (fr) Dispositif de codage d&#39;images, dispositif d&#39;extraction de flux codés et dispositif de décodage d&#39;images
WO2020116456A1 (fr) Dispositif de décodage d&#39;image mobile
WO2019139013A1 (fr) Dispositif d&#39;élaboration de vecteurs de mouvement, dispositif de décodage d&#39;images animées, et dispositif de codage d&#39;images animées
WO2017195608A1 (fr) Dispositif de codage d&#39;image animée
WO2018173895A1 (fr) Dispositif de génération d&#39;image prédite, dispositif de décodage vidéo et dispositif de codage vidéo
WO2018110180A1 (fr) Dispositif de génération de vecteur de mouvement, dispositif de génération d&#39;image prédite, dispositif de décodage d&#39;image animée, et dispositif de codage d&#39;image animée
WO2018173432A1 (fr) Dispositif de génération d&#39;image prédictive, dispositif de décodage d&#39;image en mouvement et dispositif d&#39;encodage d&#39;image en mouvement
WO2018037919A1 (fr) Dispositif de décodage d&#39;image, dispositif de codage d&#39;image, procédé de décodage d&#39;image et procédé de codage d&#39;image
JP2020145650A (ja) 画像復号装置および画像符号化装置
JP2020096279A (ja) 予測画像生成装置、動画像復号装置および動画像符号化装置
WO2018061550A1 (fr) Dispositif de décodage d&#39;image et dispositif de codage d&#39;image
JP2020108012A (ja) 画像復号装置および画像符号化装置
JP2020088660A (ja) 動画像符号化装置、動画像復号装置
JP2020005200A (ja) 画像復号装置、および画像符号化装置
WO2019065537A1 (fr) Dispositif de filtre de compensation de mouvement, dispositif de décodage d&#39;image et dispositif de codage d&#39;image animée
JP2020036101A (ja) 画像復号装置および画像符号化装置
JP2019213018A (ja) 画像復号装置および画像符号化装置
RU2812275C2 (ru) Устройство декодирования видеосигналов

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18817591

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18817591

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP