[go: up one dir, main page]

WO2014097937A1 - Dispositif de traitement d'image, et procédé de traitement d'image - Google Patents

Dispositif de traitement d'image, et procédé de traitement d'image Download PDF

Info

Publication number
WO2014097937A1
WO2014097937A1 PCT/JP2013/083154 JP2013083154W WO2014097937A1 WO 2014097937 A1 WO2014097937 A1 WO 2014097937A1 JP 2013083154 W JP2013083154 W JP 2013083154W WO 2014097937 A1 WO2014097937 A1 WO 2014097937A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
unit
motion vector
list
prediction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2013/083154
Other languages
English (en)
Japanese (ja)
Inventor
碩 陸
佐藤　数史
央二 中神
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Publication of WO2014097937A1 publication Critical patent/WO2014097937A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding

Definitions

  • the present technology relates to an image processing device and an image processing method, and in particular, it is possible to improve encoding efficiency when a motion vector of a base layer image is used as a motion vector prediction vector candidate of an enhancement layer image.
  • the present invention relates to an image processing apparatus and an image processing method.
  • MPEG compressed by orthogonal transform such as discrete cosine transform and motion compensation is used for the purpose of efficient transmission and storage of information.
  • a device compliant with a method such as Moving (Pictures Experts Group) phase) is becoming popular in both information distribution at broadcast stations and information reception in general households.
  • the MPEG2 (ISO / IEC 13818-2) system is defined as a general-purpose image encoding system, and is a standard that covers both interlaced and progressively scanned images, standard resolution images, and high-definition images. Widely used in a wide range of applications for consumer and consumer applications.
  • a standard resolution interlaced scanning image having 720 ⁇ 480 pixels is 4 to 8 Mbps
  • a high resolution interlaced scanning image having 1920 ⁇ 1088 pixels is 18 to 22 MBps.
  • MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but it did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the MPEG4 image coding system, the standard was approved as an international standard in December 1998 as ISO / IEC 449 14496-2.
  • H.264 Based on 26L, H. Standardization to achieve higher coding efficiency by incorporating functions that are not supported by 26L is performed as JointJModel of Enhanced-Compression Video Coding. This standardization was implemented in March 2003 by H.C. It was internationally standardized under the names of H.264 and MPEG-4® Part 10 (AVC (Advanced Video Coding)).
  • AVC Advanced Video Coding
  • ITUHEVC High Efficiency Video Coding
  • JCTVC Joint Collaboration Team Coding
  • AMVP Advanced Motion Vector Prediction
  • Merge Merge
  • motion vectors of neighboring PUs are regarded as motion vector prediction vector candidates for the processing target PU and registered in the list.
  • a prediction vector is determined from prediction vector candidates, and prediction vector identification information for identifying a prediction vector in the list is transmitted from the encoding device to the decoding device.
  • the prediction vector identification information is information indicating the order of registration in a list given to prediction vector candidates.
  • AMVP the difference between the prediction vector and the motion vector of the PU to be processed is transmitted from the encoding device to the decoding device.
  • the prediction vector is the motion vector of the PU to be processed.
  • peripheral PU a temporally neighboring PU, that is, a region having the same position in the screen as the processing target PU and having a different time, or a PU around that region (hereinafter referred to as a temporal peripheral PU), spatially
  • peripheral PUs that is, peripheral PUs in the same screen as the processing target PU (hereinafter referred to as spatial peripheral PUs).
  • the time-peripheral PU includes a block C0 that is a PU having the same position of the center of the PU1 to be processed and the center of the screen at different times, and PU1 and the screen in the screen.
  • the space peripheral PU is a block A1 and a block A2 that are two PUs on the lower left side of PU1, and blocks B0 to B2 that are three upper PUs.
  • FIG. 2 is a diagram for explaining a method of registering a prediction vector candidate in a list.
  • a PU having the same processing target PU, reference image specifying information (ref_idx) for specifying a reference image, and a prediction direction are called VEC1 PUs, and the reference image specifying information is the same, but the prediction directions are different.
  • PU is called VEC2 PU.
  • a PU to be processed and a PU having different reference image specifying information but having the same prediction direction are referred to as a VEC3 PU, and a PU having both reference image specifying information and a prediction direction being referred to as a VEC4 PU.
  • AMVP motion vectors of up to two spatial surrounding PUs are registered in the list as prediction vector candidates. Specifically, as shown in FIG. 2A, in AMVP, it is first determined whether or not there is a motion vector of block A0 and block A1 of VEC1, and then the determination is VEC2, VEC3, VEC4. Are performed in order. This determination is terminated when it is determined that it exists, and the motion vector of the existing block A0 or block A1 is registered in the list as a prediction vector candidate.
  • the motion vector of block H is registered in the list as a candidate for a prediction vector
  • the motion vector of block C0 is The prediction vector candidate is registered in the list.
  • the number of prediction vector candidates is less than a predetermined number, 0 as a motion vector is registered in the list as a prediction vector candidate.
  • motion vectors of up to five spatial surrounding PUs are registered in the list as prediction vector candidates.
  • FIG. 2B first, motion vectors existing among the motion vectors of the block A0 and the block A1, and the block B0 and the block B1 are set as prediction vector candidates.
  • the motion vector of the block B2 is registered in the list as a prediction vector candidate instead of the non-existing motion vector.
  • the motion vector of block H is registered in the list as a candidate for a prediction vector, and when there is no motion vector of block H, the motion vector of block C0 is The prediction vector candidate is registered in the list.
  • image encoding methods such as MPEG-2 and AVC have a scalability function for encoding an image in a hierarchical manner. According to the scalability function, it is possible to transmit encoded data according to the processing capability on the decoding side without performing a transcoding process.
  • a coded stream of an image of a base layer (base layer) that is a base layer can be transmitted to a terminal with low processing capability such as a mobile phone.
  • a terminal with low processing capability such as a mobile phone.
  • an encoded stream of an image of a base layer and an enhancement layer (enhancement layer) that is a layer other than the base layer may be transmitted to a terminal having high processing capability such as a television receiver or a personal computer. it can.
  • Such a scalability function is also provided in the HEVC system.
  • the above-described motion vector encoding is performed in each layer. Therefore, it is considered to improve the encoding efficiency by using the motion vector of the base layer image as a prediction vector candidate when encoding the enhancement layer motion vector (see, for example, Non-Patent Document 1).
  • the order of registration in the motion vector list of base layer images is fixed. Accordingly, although the motion vector of the base layer image is likely to be a prediction vector, the prediction vector identification information of the motion vector is large or low, but the prediction vector of the motion vector is low. The identification information is small. As a result, encoding efficiency is poor.
  • the present technology has been made in view of such a situation, and can improve coding efficiency when a motion vector of a base layer image is used as a motion vector prediction vector candidate of an enhancement layer image. It is something that can be done.
  • An image processing apparatus based on a feature of a processing unit that performs compensation processing on an image of a first layer of an image having a hierarchical structure using a motion vector, and an image having the hierarchical structure,
  • An image processing apparatus comprising: a list registration unit that registers a motion vector prediction vector candidate of a first layer image including a motion vector of a second layer image in a list.
  • the image processing method according to one aspect of the present technology corresponds to the image processing apparatus according to one aspect of the present technology.
  • compensation processing is performed on an image in a first layer of an image having a hierarchical structure using a motion vector, and the second layer is based on the characteristics of the image having the hierarchical structure.
  • the motion vector prediction vector candidates including the motion vector of the first layer are registered in the list.
  • the image processing apparatus can be realized by causing a computer to execute a program.
  • a program to be executed by a computer can be provided by being transmitted via a transmission medium or by being recorded on a recording medium.
  • encoding efficiency can be improved when a motion vector of a base layer image is used as a candidate for a motion vector prediction vector of an enhancement layer image.
  • FIG. 6 is a block diagram illustrating a configuration example of an encoding unit in FIG. 5.
  • FIG. 8 is a block diagram illustrating a configuration example of a motion prediction / compensation unit in FIG. 7. It is a figure which shows the example of PU corresponding to the prediction vector candidate.
  • FIG. 6 is a flowchart for explaining generation processing of the encoding device in FIG. 4.
  • 13 is a flowchart for explaining details of the encoding process of FIG. 12. It is a flowchart explaining the detail of the list production
  • enhancement layer there is one enhancement layer unless otherwise specified, but the number of enhancement layers may be plural.
  • FIG. 3 is a diagram for explaining the scalability function.
  • SNRabilityscalability is a scalability function that encodes an image layered with SNR (signal-noise ratio). Specifically, in SNR scalability, a low SNR image is encoded as a base layer image, and a difference image between a high SNR image and a low SNR image is encoded as an enhancement layer image.
  • the encoding device transmits only the encoded data of the base layer image to the decoding device with low processing capability, so that the decoding device can generate a low SNR image, that is, a low quality image. it can. Further, the encoding device transmits the encoded data of the base layer and enhancement layer images to the decoding device having high processing capability, so that the decoding device decodes and synthesizes the base layer and enhancement layer images.
  • a high SNR image that is, a high-quality image can be generated.
  • spatial scalability is a scalability function that encodes an image by layering it at a spatial resolution.
  • a low resolution image is encoded as a base layer image
  • a difference image between the high resolution image and the low resolution image is encoded as an enhancement layer image.
  • the encoding device transmits only the encoded data of the base layer image to the decoding device with low processing capability, so that the decoding device can generate a low-resolution image. Further, the encoding device transmits the encoded data of the base layer and enhancement layer images to the decoding device having high processing capability, so that the decoding device decodes and synthesizes the base layer and enhancement layer images. High-resolution images can be generated.
  • a scalability function there is also temporal-scalability for hierarchizing and encoding an image by a frame rate, and bit-depth-scalability for encoding an image by hierarchizing by a bit number.
  • a scalability function there is also a chroma-scalability for hierarchizing and encoding an image in a color difference signal format.
  • FIG. 4 is a block diagram illustrating a configuration example of an embodiment of an encoding device to which the present technology is applied.
  • the encoding apparatus 10 encodes an image using a scalability function in accordance with a scheme conforming to the HEVC scheme.
  • a base layer image (hereinafter referred to as a base image) is input to the base encoding unit 11 of the encoding device 10 from the outside.
  • the base encoding unit 11 is configured in the same manner as a conventional HEVC encoding device, and encodes a base image using the HEVC method. However, the base encoding unit 11 supplies a motion vector used at the time of inter encoding of the base image to the enhancement encoding unit 12.
  • the base encoding unit 11 supplies an encoded stream including encoded data, SPS, PPS, and the like obtained as a result of encoding to the synthesizing unit 13 as a base stream.
  • the enhancement coding unit 12 receives an enhancement layer image (hereinafter referred to as an enhancement image) from the outside.
  • the enhancement encoding unit 12 encodes the enhancement image by a method according to the HEVC method. Further, the enhancement encoding unit 12 uses the motion vector of the base image supplied from the base encoding unit 11 to encode the motion vector used at the time of inter-encoding of the enhancement image by the AMVP method, and obtains motion vector information. Generate.
  • the enhancement encoding unit 12 generates an encoded stream by adding motion vector information or the like to the encoded data of the enhancement image, and supplies the encoded stream to the synthesizing unit 13 as an enhancement stream.
  • the synthesizing unit 13 synthesizes the base stream supplied from the base encoding unit 11 and the enhancement stream supplied from the enhancement encoding unit 12 to generate an encoded stream of all layers.
  • the synthesizing unit 13 supplies the encoded stream of all layers to the transmission unit 14.
  • the transmission unit 14 transmits the encoded stream of all layers supplied from the synthesis unit 13 to a decoding device described later.
  • the encoding apparatus 10 shall transmit the encoding stream of all the layers here, it can also transmit only a base stream as needed.
  • FIG. 5 is a block diagram illustrating a configuration example of the enhancement encoding unit 12 of FIG.
  • the enhancement encoding unit 12 in FIG. 5 includes a setting unit 21 and an encoding unit 22.
  • the setting unit 21 of the enhancement encoding unit 12 sets a parameter set such as SPS and PPS and supplies the parameter set to the encoding unit 22.
  • the encoding unit 22 encodes an enhancement image input from the outside by a method according to the HEVC method, and generates encoded data. At this time, the encoding unit 22 uses the motion vector from the base encoding unit 11 to encode a motion vector used for inter prediction of the enhancement image by the AMVP method, and generates motion vector information.
  • the encoding unit 22 adds motion vector information or the like as header information to the encoded data of the enhancement image.
  • the encoding unit 22 adds the parameter set supplied from the setting unit 21 to the encoded data of the enhancement image to which the header information is added, and generates an enhancement stream.
  • the encoding unit 22 supplies the enhancement stream to the synthesis unit 13 in FIG.
  • FIG. 6 is a diagram illustrating a configuration example of SPS syntax set by the setting unit 21 of FIG.
  • the SPS includes a base vector flag (sps_enable_BLMV_flag) (identification information) for identifying that the motion vector of the base image is a prediction vector candidate.
  • the base vector flag is, for example, 1 when representing that the motion vector of the base image is a prediction vector candidate, and is 0 when representing that the motion vector of the base image is not a prediction vector candidate.
  • FIG. 7 is a block diagram illustrating a configuration example of the encoding unit 22 of FIG.
  • the encoding unit 22 includes an inverse quantization unit 108, an inverse orthogonal transform unit 109, a calculation unit 110, a filter 111, a decoded picture buffer 112, a selection unit 113, an intra prediction unit 114, a motion prediction / compensation unit 115, and a prediction.
  • An image selection unit 116 is included.
  • the A / D conversion unit 101 of the encoding unit 22 performs A / D conversion on the input enhancement image, and supplies the image that is the converted digital data to the screen rearrangement buffer 102 for storage.
  • the screen rearrangement buffer 102 rearranges the stored frames in the display order in the order of frames for encoding in accordance with GOP (Group Of Picture).
  • the screen rearrangement buffer 102 supplies the image with the rearranged frame order to the arithmetic unit 103.
  • the screen rearrangement buffer 102 supplies the image in which the frame order is rearranged to the intra prediction unit 114 and the motion prediction / compensation unit 115.
  • the calculation unit 103 subtracts the predicted image supplied from the intra prediction unit 114 or the motion prediction / compensation unit 115 via the predicted image selection unit 116 from the image read from the screen rearrangement buffer 102.
  • the calculation unit 103 outputs the difference information obtained as a result to the orthogonal transform unit 104.
  • the calculation unit 103 subtracts the prediction image supplied from the intra prediction unit 114 from the image read from the screen rearrangement buffer 102. Further, when inter coding is performed, the arithmetic unit 103 subtracts the predicted image supplied from the motion prediction / compensation unit 115 from the image read from the screen rearrangement buffer 102.
  • the orthogonal transform unit 104 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform on the difference information supplied from the computation unit 103. Note that this orthogonal transformation method is arbitrary.
  • the orthogonal transform unit 104 supplies the transform coefficient to the quantization unit 105.
  • the quantization unit 105 quantizes the transform coefficient supplied from the orthogonal transform unit 104.
  • the quantization unit 105 sets a quantization parameter based on information on the code amount target value, and performs the quantization. Note that this quantization method is arbitrary.
  • the quantization unit 105 supplies the quantized transform coefficient to the lossless encoding unit 106.
  • the lossless encoding unit 106 losslessly encodes the transform coefficient quantized by the quantization unit 105 using an arbitrary encoding method, and generates encoded data. Further, the lossless encoding unit 106 acquires intra prediction information including information indicating an intra prediction mode from the intra prediction unit 114, and moves inter prediction information including information indicating an inter prediction mode, motion vector information, and the like. Obtained from the prediction / compensation unit 115. Further, the lossless encoding unit 106 acquires filter coefficients used in the filter 111 and the like.
  • the lossless encoding unit 106 performs lossless encoding of these various types of information using an arbitrary encoding method, and uses (multiplexes) a part of the header information of the encoded data.
  • the lossless encoding unit 106 supplies the encoded data in which the header information is multiplexed to the accumulation buffer 107 for accumulation.
  • Examples of the encoding method of the lossless encoding unit 106 include variable length encoding or arithmetic encoding.
  • Examples of variable length coding include H.264.
  • CAVLC Context-Adaptive Variable Length Coding
  • Examples of arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding).
  • the accumulation buffer 107 temporarily holds the encoded data supplied from the lossless encoding unit 106.
  • the accumulation buffer 107 reads out the encoded data held at a predetermined timing, and supplies it as an enhancement stream to the synthesis unit 13 in FIG. 4 together with the parameter set supplied from the setting unit 21 in FIG.
  • the transform coefficient quantized by the quantization unit 105 is also supplied to the inverse quantization unit 108.
  • the inverse quantization unit 108 inversely quantizes the quantized transform coefficient by a method corresponding to the quantization by the quantization unit 105.
  • the inverse quantization method may be any method as long as it is a method corresponding to the quantization processing by the quantization unit 105.
  • the inverse quantization unit 108 supplies the obtained transform coefficient to the inverse orthogonal transform unit 109.
  • the inverse orthogonal transform unit 109 performs inverse orthogonal transform on the transform coefficient supplied from the inverse quantization unit 108 by a method corresponding to the orthogonal transform process by the orthogonal transform unit 104.
  • the inverse orthogonal transform method may be any method as long as it corresponds to the orthogonal transform processing by the orthogonal transform unit 104.
  • the inversely orthogonally transformed output (difference information restored locally) is supplied to the calculation unit 110.
  • the calculation unit 110 converts the inverse orthogonal transform result supplied from the inverse orthogonal transform unit 109, that is, locally restored difference information, into the intra prediction unit 114 or the motion prediction / compensation unit 115 via the predicted image selection unit 116. Are added to the predicted image to obtain a locally decoded image (hereinafter referred to as a reconstructed image).
  • the reconstructed image is supplied to the filter 111 or the decoded picture buffer 112.
  • the filter 111 includes a deblock filter, an adaptive loop filter, and the like, and appropriately performs a filtering process on the reconstructed image supplied from the calculation unit 110. For example, the filter 111 removes block distortion of the reconstructed image by performing deblocking filter processing on the reconstructed image. Further, for example, the filter 111 improves the image quality by performing loop filter processing using a Wiener filter on the deblock filter processing result (reconstructed image from which block distortion has been removed). Do.
  • the filter 111 may perform arbitrary filter processing on the reconstructed image.
  • the filter 111 can supply information such as filter coefficients used for the filter processing to the lossless encoding unit 106 and perform lossless encoding as necessary.
  • the filter 111 supplies a filter processing result (hereinafter referred to as a decoded image) to the decoded picture buffer 112.
  • the decoded picture buffer 112 stores the reconstructed image supplied from the calculation unit 110 and the decoded image supplied from the filter 111.
  • the decoded picture buffer 112 supplies the stored reconstructed image to the intra prediction unit 114 via the selection unit 113 at a predetermined timing or based on a request from the outside such as the intra prediction unit 114.
  • the decoded picture buffer 112 receives a stored decoded image at a predetermined timing or based on an external request from the motion prediction / compensation unit 115 or the like via the selection unit 113. 115.
  • the selection unit 113 indicates the supply destination of the reconstructed image or decoded image output from the decoded picture buffer 112. Specifically, when intra coding is performed, the selection unit 113 reads a reconstructed image that has not been filtered from the decoded picture buffer 112, and an image of a peripheral region (peripheral image) located around the prediction target region. Is supplied to the intra prediction unit 114.
  • the selection unit 113 reads the decoded image that has been filtered from the decoded picture buffer 112, and supplies the decoded image to the motion prediction / compensation unit 115 as a reference image.
  • the intra prediction unit 114 when acquiring a peripheral image from the decoded picture buffer 112, performs intra prediction using the pixel value of the peripheral image to generate a prediction image using the PU as a processing unit.
  • the intra prediction unit 114 performs this intra prediction in all candidate intra prediction modes.
  • the intra prediction unit 114 evaluates the cost function value of each prediction image using the prediction image generated by the intra prediction of all candidate intra prediction modes and the input image supplied from the screen rearrangement buffer 102. Select the optimal intra prediction mode. Then, the intra prediction unit 114 supplies the predicted image and the cost function value generated in the optimal intra prediction mode to the predicted image selection unit 116.
  • the intra prediction unit 114 when the selection is notified from the prediction image selection unit 116, the intra prediction unit 114 appropriately supplies intra prediction information including information related to intra prediction such as an optimal intra prediction mode to the lossless encoding unit 106, and Make it.
  • the motion prediction / compensation unit 115 reads out the decoded image stored in the decoded picture buffer 112 as a reference image.
  • the motion prediction / compensation unit 115 uses the input image supplied from the screen rearrangement buffer 102 and the reference image as a unit of processing, and performs motion prediction using inter-time correlation as inter prediction.
  • the motion prediction / compensation unit 115 performs a compensation process according to the motion vector detected as a result of the inter prediction, and generates a predicted image.
  • the motion prediction / compensation unit 115 performs such inter prediction and compensation processing in all candidate inter prediction modes.
  • the motion prediction / compensation unit 115 evaluates the cost function value of each prediction image using the prediction images in all candidate inter prediction modes and the input image supplied from the screen rearrangement buffer 102, and determines the optimum value. Select inter prediction mode. Then, the motion prediction / compensation unit 115 supplies the predicted image and the cost function value generated in the optimal inter prediction mode to the predicted image selection unit 116.
  • the motion prediction / compensation unit 115 when the selection is notified from the prediction image selection unit 116, the motion prediction / compensation unit 115, based on the motion vector from the base encoding unit 11 and the base vector flag included in the SPS from the setting unit 21, A motion vector corresponding to the predicted image generated in the optimal inter prediction mode is encoded by the AMVP method.
  • the motion prediction / compensation unit 115 supplies the motion vector information obtained as a result and inter prediction information including information related to inter prediction such as the optimal inter prediction mode to the lossless encoding unit 106 and performs lossless encoding.
  • the predicted image selection unit 116 selects the supply source of the predicted image supplied to the calculation unit 103 and the calculation unit 110 based on the cost function values supplied from the intra prediction unit 114 and the motion prediction / compensation unit 115.
  • the predicted image selection unit 116 uses the intra prediction unit as a source of the predicted image. 114 is selected. Thereby, the prediction image supplied from the intra estimation part 114 is supplied to the calculating part 103 and the calculating part 110, and intra encoding is performed.
  • the prediction image selection unit 116 serves as a prediction image supply source. Select. Thereby, the predicted image supplied from the motion prediction / compensation unit 115 is supplied to the calculation unit 103 and the calculation unit 110, and inter coding is performed. The predicted image selection unit 116 notifies the selected supply source of the selection.
  • FIG. 8 is a block diagram illustrating a configuration example of the motion prediction / compensation unit 115 of FIG.
  • a processing unit 131 includes a processing unit 131, a prediction information buffer 132, a list registration unit 133, a vector buffer 134, and a vector encoding unit 135.
  • the processing unit 131 of the motion prediction / compensation unit 115 reads out the decoded image stored in the decoded picture buffer 112 of FIG. 7 as a reference image.
  • the processing unit 131 uses the input image and the reference image supplied from the screen rearrangement buffer 102 to perform inter prediction using the PU as a processing unit.
  • the processing unit 131 performs compensation processing according to the motion vector detected as a result of inter prediction, and generates a predicted image.
  • the processing unit 131 performs such inter prediction and compensation processing in all candidate inter prediction modes.
  • the processing unit 131 evaluates the cost function value of each prediction image using the prediction images and input images of all candidate inter prediction modes, and selects an optimal inter prediction mode. Then, the processing unit 131 supplies the predicted image and the cost function value generated in the optimal inter prediction mode to the predicted image selection unit 116.
  • the processing unit 131 displays the PU size, prediction direction, motion vector, and reference image specifying information corresponding to the prediction image generated in the optimal inter prediction mode.
  • the prediction information is supplied to the prediction information buffer 132 as prediction information.
  • the processing unit 131 supplies a motion vector corresponding to the predicted image generated in the optimal inter prediction mode to the vector encoding unit 135.
  • the processing unit 131 supplies the optimal inter prediction mode, the reference image specifying information, and the prediction direction representing the PU size to the lossless encoding unit 106 in FIG. 7 as inter prediction information.
  • the prediction information buffer 132 stores the prediction information supplied from the motion prediction / compensation unit 115.
  • the list registration unit 133 determines whether or not to use the motion vector of the base image as a prediction vector candidate based on the base vector flag included in the SPS supplied from the setting unit 21 of FIG.
  • the list registration unit 133 reads the size of the PU from the prediction information buffer 132 when the motion vector of the base image is a prediction vector candidate. Then, based on the PU size, the list registration unit 133 hierarchically predicts motion vectors of peripheral peripheral PUs (hereinafter referred to as hierarchical peripheral PUs), spatial peripheral PU motion vectors, and temporal peripheral PU motion vectors. The order of registration as candidates is determined.
  • hierarchical peripheral PUs peripheral peripheral PUs
  • spatial peripheral PU motion vectors spatial peripheral PU motion vectors
  • temporal peripheral PU motion vectors temporal peripheral PU motion vectors
  • the list registration unit 133 determines the registration order as the motion vector candidate of the spatial peripheral PU and the prediction vector of the motion vector of the temporal peripheral PU.
  • the list registration unit 133 registers motion vector identification information, which is identification information of motion vectors that are prediction vector candidates, in the list in the determined registration order.
  • the registration method of the motion vector of the spatial peripheral PU and the motion vector of the temporal peripheral PU is the same as the method described in FIG. 2, and the motion vector of the spatial peripheral PU is stored in the prediction information buffer 132.
  • the prediction direction of the surrounding spatial PU and the PU to be processed and the reference image specifying information are used.
  • the list registration unit 133 registers motion vector identification information of a motion vector that is 0 in the list. Then, the list registration unit 133 supplies the generated list to the vector encoding unit 135. Note that the motion vector identification information in the list is given prediction vector identification information in the order of registration.
  • the vector buffer 134 stores the motion vector supplied from the base encoding unit 11.
  • the vector encoding unit 135 reads the motion vector identified by the motion vector identification information registered in the list supplied from the list registration unit 133 from the prediction information buffer 132 or the vector buffer 134 as a prediction vector candidate.
  • the vector encoding unit 135 determines the prediction vector by comparing the motion vector supplied from the processing unit 131 with the prediction vector candidate. Specifically, the vector encoding unit 135 selects a prediction vector candidate having a small difference from the motion vector supplied from the processing unit 131 as a prediction vector.
  • the vector encoding unit 135 encodes the motion vector by generating a difference between the motion vector supplied from the processing unit 131 and the prediction vector as motion vector information.
  • the vector encoding unit 135 supplies motion vector information and prediction vector identification information to the lossless encoding unit 106 as inter prediction information.
  • FIG. 9 is a diagram illustrating an example of PUs corresponding to prediction vector candidates.
  • the spatial peripheral PU corresponding to the prediction vector candidate is one of the block A1 or the block A2 and the blocks B0 to B2, as in the case of FIG.
  • the time periphery PU is the block C0 or the block H as in the case of FIG.
  • the hierarchical peripheral PU is an area where the position in the screen is the same as the PU to be processed or the PU of the base image around the area.
  • the positions of the center of the PU 151 and the center in the screen Are the blocks ColBL that are PUs of the base images having the same.
  • FIG. 10 is a diagram illustrating an example of a PU size.
  • the PU is set by dividing CU. Therefore, for example, as shown in FIG. 10, the CU 170 can be used as the PU 171 as it is. In this case, the size of the PU 171 is the same as the size of the CU 170.
  • the CU 170 can be divided into upper and lower halves to form PU 172-1 and PU 172-2, or divided into left and right halves to form PU 173-1 and PU 173-2.
  • the size of PU172-1, PU172-2, PU173-1, and PU173-2 is half of the size of CU170.
  • the CU 170 can be divided into four parts in the vertical and horizontal directions to form PUs 174-1 to 174-4. In this case, the sizes of the PUs 174-1 to 174-4 are 1/4 times the size of the CU 170.
  • 2/3 on the left side of CU170 can be PU175-1, and 1/3 on the right side can be PU175-2.
  • 1/3 on the left side of CU170 can be PU176-1, and 2/3 on the right side can be PU175-1. It can also be PU176-2.
  • the upper 2/3 of the CU 170 can be the PU 177-1 and the lower 1/3 can be the PU 177-2, or the upper 1/3 of the CU 170 can be the PU 178-1, and the lower 2 / 3 can also be PU178-2.
  • the size of PU175-1, PU176-2, PU177-1, and PU178-2 is 2/3 times that of CU170, and PU175-2, PU176-1, PU177-2 and PU178-1 The size is 1/3 times that of CU170.
  • the processing unit 131 performs inter prediction and compensation processing in the inter prediction mode representing the size of the PU divided as described above, and generates a prediction image. Then, the processing unit 131 evaluates the cost function value of each predicted image and selects an optimal inter prediction mode.
  • FIG. 11 is a diagram illustrating a method for determining the registration order of prediction vector candidates by the list registration unit 133 in FIG. 8.
  • the list registration unit 133 determines that the motion vector of the base image is a prediction vector candidate
  • the list registration unit 133 sets the motion vector registration order of the block ColBL based on the size of the PU to be processed. Or at the end.
  • the image of the entire CU is uniform, and the correlation between the motion vectors of the base image and the enhancement image is relatively strong. Therefore, in this case, the registration order of the motion vector of the block ColBL is determined at the head. That is, the prediction vector identification information of the motion vector of the block ColBL is the minimum value.
  • the registration order after the motion vector of the block ColBL is the same as that in the conventional case shown in FIG. That is, the registration order is determined so that the motion vector of the block A0 or the block A1, the motion vector of any of the blocks B0 to B2, and the motion vector of the block H or the block C0 are in order from the front.
  • the registration order of the motion vectors of the block ColBL is determined last. That is, the prediction vector identification information of the motion vector of the block ColBL is the maximum value.
  • the order of registration before the motion vector of the block ColBL is the same as in the conventional case.
  • the predetermined size is half the size of the CU
  • the registration order of the motion vector of the block ColBL in the list for the PU 178-2 is first.
  • the motion vector registration order of the block ColBL in the list for PUs 174-1 to 174-4, PU 175-2, PU 176-1, PU 177-2, and PU 178-1 in FIG. 10 is the last.
  • the registration order of the motion vectors of the block ColBL may not be the first or last. That is, if the registration order is earlier than the registration order of the space surrounding CU or the time surrounding CU, it may be the second or the like instead of the top, and if the registration order is later than the registration order of the space surrounding CU or the time surrounding CU, It may be the second from the end, not the last.
  • the list registration unit 133 sequentially starts from the front of the block A0 or the block A1 as in the conventional case illustrated in A of FIG.
  • the registration order is determined so as to be a motion vector, a motion vector of any of blocks B0 to B2, and a motion vector of block H or block C0.
  • FIG. 12 is a flowchart illustrating the generation process of the encoding device 10 of FIG.
  • the base encoding unit 11 of the encoding device 10 encodes a base image input from the outside using the HEVC method.
  • the base encoding unit 11 supplies a motion vector used when the base image is inter-encoded to the enhancement encoding unit 12.
  • the base encoding unit 11 generates an encoded stream including encoded data obtained as a result of encoding, SPS, PPS, and the like as a base stream, and supplies the encoded stream to the synthesizing unit 13.
  • step S11 the setting unit 21 (FIG. 5) of the enhancement encoding unit 12 sets SPS.
  • step S12 the setting unit 21 sets the PPS.
  • the setting unit 21 supplies a parameter set such as SPS or PPS to the encoding unit 22.
  • step S13 the encoding unit 22 performs an encoding process that encodes an enhancement image input from the outside using a motion vector supplied from the base encoding unit 11 in a method according to the HEVC method. Details of this encoding process will be described with reference to FIG.
  • step S14 the synthesizing unit 13 synthesizes the base stream supplied from the base encoding unit 11 and the enhancement stream supplied from the enhancement encoding unit 12, and generates an encoded stream of all layers.
  • the synthesizing unit 13 supplies the encoded stream of all layers to the transmission unit 14.
  • step S15 the transmission unit 14 transmits the encoded stream of all layers supplied from the synthesis unit 13 to a decoding device to be described later, and ends the process.
  • FIG. 13 is a flowchart illustrating details of the encoding process in step S13 of FIG.
  • step S100 of FIG. 13 the A / D conversion unit 101 (FIG. 7) of the encoding unit 22 performs A / D conversion on the input enhancement image, and the screen rearrangement buffer 102 converts the image that is the converted digital data. To supply and memorize.
  • step S101 the screen rearrangement buffer 102 rearranges the images of the frames in the stored display order into the frame order for encoding in accordance with the GOP.
  • the screen rearrangement buffer 102 also supplies the image with the rearranged frame order to the arithmetic unit 103, the intra prediction unit 114, and the motion prediction / compensation unit 115.
  • step S102 the intra prediction unit 114 performs intra prediction using the PU as a processing unit using the peripheral image supplied from the decoded picture buffer 112 via the selection unit 113.
  • the intra prediction unit 114 performs this intra prediction in all candidate intra prediction modes.
  • the intra prediction unit 114 calculates the cost function value of each prediction image using the prediction image generated by the intra prediction of all candidate intra prediction modes and the input image supplied from the screen rearrangement buffer 102. Evaluate and select the optimal intra prediction mode. Then, the intra prediction unit 114 supplies the predicted image and the cost function value generated in the optimal intra prediction mode to the predicted image selection unit 116.
  • step S103 the processing unit 131 (FIG. 8) of the motion prediction / compensation unit 115 uses the decoded image read as the reference image from the decoded picture buffer 112 and the input image from the screen rearrangement buffer 102, Inter prediction is performed using the PU as a processing unit. Then, the processing unit 131 performs compensation processing according to the motion vector detected as a result of inter prediction, and generates a predicted image. The processing unit 131 performs such inter prediction and compensation processing in all candidate inter prediction modes.
  • the processing unit 131 evaluates the cost function value of each prediction image using the prediction images and input images of all candidate inter prediction modes, and selects an optimal inter prediction mode. Then, the processing unit 131 supplies the predicted image and the cost function value generated in the optimal inter prediction mode to the predicted image selection unit 116.
  • step S104 the predicted image selection unit 116, based on the cost function values supplied from the intra prediction unit 114 and the processing unit 131, serves as the source of the predicted image supplied to the calculation unit 103 and the calculation unit 110. Determine whether to select.
  • step S104 When it is determined in step S104 that the intra prediction unit 114 is selected, the prediction image selection unit 116 supplies the prediction image supplied from the intra prediction unit 114 to the calculation unit 103 and the calculation unit 110, and selects the intra prediction unit 114. To be notified.
  • step S105 the intra prediction unit 114 supplies the intra prediction information to the lossless encoding unit 106, and the process proceeds to step S108.
  • the predicted image selection unit 116 supplies the predicted image supplied from the processing unit 131 to the calculation unit 103 and the calculation unit 110, and selects the processing unit 131. To be notified.
  • the processing unit 131 supplies the PU size, the prediction direction, the motion vector, and the reference image specifying information corresponding to the prediction image generated in the optimal inter prediction mode to the prediction information buffer 132 as the prediction information, and stores it.
  • the processing unit 131 supplies the motion vector to the vector encoding unit 135, and the optimal inter prediction mode, prediction direction, and reference image specifying information indicating the PU size are supplied to the lossless encoding unit 106 as inter prediction information. Supply.
  • step S106 the motion prediction / compensation unit 115 performs list generation processing for generating a list of motion vector prediction vectors corresponding to the predicted image generated in the optimal inter prediction mode. Details of this list generation processing will be described with reference to FIG.
  • step S107 the vector encoding unit 135 determines a prediction vector by comparing the prediction vector candidate with the motion vector from the processing unit 131, and generates a difference between the prediction vector and the motion vector as motion vector information.
  • the prediction vector candidate is a motion vector identified by the motion vector identification information registered in the list supplied from the list registration unit 133, and is read from the prediction information buffer 132 and the vector buffer 134.
  • step S108 the calculation unit 103 calculates a difference between the image read from the screen rearrangement buffer 102 and the predicted image supplied from the intra prediction unit 114 or the motion prediction / compensation unit 115 via the predicted image selection unit 116. Calculate.
  • the calculation unit 103 outputs the difference information obtained as a result to the orthogonal transform unit 104.
  • step S109 the orthogonal transform unit 104 performs orthogonal transform on the difference information supplied from the calculation unit 103, and supplies the transform coefficient obtained as a result to the quantization unit 105.
  • step S110 the quantization unit 105 quantizes the transform coefficient supplied from the orthogonal transform unit 104, and supplies the quantized transform coefficient to the lossless encoding unit 106 and the inverse quantization unit 108.
  • step S111 the inverse quantization unit 108 inversely quantizes the quantized transform coefficient supplied from the quantization unit 105 by a method corresponding to the quantization by the quantization unit 105, and inversely orthogonalizes the obtained transform coefficient. This is supplied to the conversion unit 109.
  • step S112 the inverse orthogonal transform unit 109 performs inverse orthogonal transform on the transform coefficient supplied from the inverse quantization unit 108 by a method corresponding to the orthogonal transform processing by the orthogonal transform unit 104, and locally restores the resultant result.
  • the obtained difference information is supplied to the calculation unit 110.
  • step S113 the arithmetic unit 110 is supplied with the locally restored difference information supplied from the inverse orthogonal transform unit 109 from the intra prediction unit 114 or the motion prediction / compensation unit 115 via the predicted image selection unit 116. To obtain a reconstructed image.
  • the reconstructed image is supplied to the filter 111 or the decoded picture buffer 112.
  • step S114 the filter 111 appropriately performs a filtering process on the reconstructed image supplied from the calculation unit 110, and supplies the decoded image obtained as a result to the decoded picture buffer 112.
  • step S115 the decoded picture buffer 112 stores the reconstructed image and the decoded image supplied from the calculation unit 110.
  • the reconstructed image stored in the decoded picture buffer 112 is read via the selection unit 113 and supplied to the intra prediction unit 114 as a peripheral image when intra coding is performed.
  • the decoded image stored in the decoded picture buffer 112 is read via the selection unit 113 and supplied to the motion prediction / compensation unit 115 as a reference image.
  • step S116 the lossless encoding unit 106 losslessly encodes the quantized transform coefficient from the quantization unit 105 to obtain encoded data.
  • the lossless encoding unit 106 losslessly encodes the intra prediction information from the intra prediction unit 114 or the inter prediction information from the motion prediction / compensation unit 115, the filter coefficient used in the filter 111, and the like. As part of the information.
  • the lossless encoding unit 106 supplies encoded data in which header information is multiplexed to the accumulation buffer 107.
  • step S117 the accumulation buffer 107 temporarily accumulates the encoded data in which the header information supplied from the lossless encoding unit 106 is multiplexed. And a process returns to step S13 of FIG. 12, and progresses to step S14.
  • FIG. 14 is a flowchart for explaining the details of the list generation processing in step S106 of FIG.
  • the vector buffer 134 stores the motion vector supplied from the base encoding unit 11.
  • the list registration unit 133 determines whether the base vector flag included in the SPS supplied from the setting unit 21 in FIG.
  • step S132 determines that the base vector flag is 1
  • the list registration unit 133 determines that the motion vector of the base image is a prediction vector candidate, and the process proceeds to step S133.
  • step S133 the list registration unit 133 determines the registration order so that the motion vector registration order of the block ColBL comes first.
  • step S134 the list registration unit 133 of the motion prediction / compensation unit 115 determines whether or not the block ColBL exists.
  • step S135 the list registration unit 133 determines whether the optimal prediction mode of the block ColBL is the intra prediction mode.
  • step S135 If it is determined in step S135 that the optimal prediction mode of the block ColBL is the intra prediction mode, that is, if there is a motion vector of the block ColBL, the process proceeds to step S136.
  • step S136 the list registration unit 133 determines whether the size of the processing target PU held in the prediction information buffer 132 is greater than or equal to N ⁇ N (N is an arbitrary positive integer) pixels. If it is determined in step S136 that the size of the PU to be processed is N ⁇ N pixels or more, the process proceeds to step S137.
  • step S137 based on the registration order determined in step S133, the list registration unit 133 sets the motion vector of the block ColBL as a prediction vector candidate and registers the motion vector identification information of the motion vector in the list. Then, the process proceeds to step S141.
  • step S138 the list registration unit 133 changes the registration order so that the registration order of the block ColBL is last. To do.
  • step S139 the list registration unit 133 holds the motion vector identification information of the motion vector of the block ColBL based on the registration order changed in step S138, and the process proceeds to step S141.
  • step S134 If it is determined in step S134 that the block ColBL does not exist or the optimal prediction mode of the block ColBL is not the intra prediction mode in step S135, the process proceeds to step S141.
  • step S132 determines that the base vector flag is not 1, that is, if the base vector flag is 0, the list registration unit 133 determines that the motion vector of the base image is not a candidate for a prediction vector, The process proceeds to step S140.
  • step S140 the list registration unit 133 determines the registration order of motion vectors of blocks other than the block ColBL, that is, the spatial peripheral PU and the temporal peripheral PU, and the process proceeds to step S141.
  • step S141 based on the registration order, the list registration unit 133 sets the motion vector of the spatial surrounding PU as a prediction vector candidate, and registers the motion vector identification information of the motion vector in the list.
  • the list registration unit 133 reads the block A0, the block A1, and the blocks B0 to B2 and the prediction direction of the processing target PU and the reference image specifying information from the prediction information buffer 132.
  • the list registration unit 133 sequentially determines whether or not the motion vectors of the block A0 and the block A1 of VEC1 exist based on the read prediction direction and the reference image specifying information, and then determines the determination as VEC2, VEC3. , VEC4 is sequentially performed until it is determined that it exists. Then, the list registration unit 133 uses the motion vector of the existing block A0 or block A1 as a prediction vector candidate, and registers the motion vector identification information of the motion vector in the list.
  • the list registration unit 133 sequentially determines whether or not motion vectors of the blocks B0 to B2 of the VEC1 exist based on the read prediction direction and the reference image specifying information, and then determines the determination as VEC2, The processing performed in order for VEC3 and VEC4 is performed until it is determined that it exists. Then, the list registration unit 133 sets any one motion vector of the existing blocks B0 to B2 as a prediction vector candidate, and registers motion vector identification information of the motion vector in the list.
  • step S142 based on the registration order, the list registration unit 133 sets the motion vector of the temporal peripheral PU as a prediction vector candidate, and registers the motion vector identification information of the motion vector in the list.
  • list registration unit 133 uses the motion vector of block H as a candidate for a prediction vector, and registers motion vector identification information of the motion vector in the list.
  • the list registration unit 133 sets the motion vector of block C0 as a candidate for a prediction vector, and registers motion vector identification information of the motion vector in the list.
  • step S143 the list registration unit 133 determines whether or not the motion vector identification information of the motion vector of the block ColBL is held, that is, whether or not the process of step S139 has been performed. If it is determined in step S143 that the motion vector identification information of the motion vector of the block ColBL is held, the process proceeds to step S144.
  • step S144 the list registration unit 133 registers the motion vector identification information of the motion vector of the block ColBL that is held in the list, and the process proceeds to step S145.
  • step S143 when it is determined in step S143 that the motion vector identification information of the motion vector of the block ColBL is not held, the process proceeds to step S145.
  • step S145 the list registration unit 133 determines whether the number of registered motion vector identification information is less than a predetermined number. If it is determined in step S145 that the number of motion vector identification information registered is less than the predetermined number, in step S146, the list registration unit 133 registers motion vector identification information of a motion vector that is 0 in the list. Then, the list registration unit 133 supplies the list to the vector encoding unit 135, returns the process to step S106 in FIG. 13, and proceeds to step S107.
  • step S145 if it is determined in step S145 that the number of motion vector identification information registered is not less than the predetermined number, the list registration unit 133 supplies the list to the vector encoding unit 135. And a process returns to step S106 of FIG. 13, and progresses to step S107.
  • the encoding apparatus 10 registers motion vector prediction vector candidates of the enhancement image including the motion vector of the base image in the list based on the PU size as the feature of the enhancement image.
  • the motion vector of the base image is set before the motion vector of the enhancement image. You can register to the list. As a result, the information amount of prediction vector identification information can be reduced and encoding efficiency can be improved.
  • the enhancement image motion vector is listed before the motion vector of the base image. Can be registered. As a result, the information amount of prediction vector identification information can be reduced and encoding efficiency can be improved.
  • FIG. 15 is a block diagram illustrating a configuration example of an embodiment of a decoding device to which the present technology is applied, which decodes an encoded stream of all layers transmitted from the encoding device 10 of FIG.
  • the 15 includes a receiving unit 181, a separating unit 182, a base decoding unit 183, and an enhancement decoding unit 184.
  • the receiving unit 181 receives the encoded stream of all layers transmitted from the encoding device 10 in FIG. 4 and supplies it to the separating unit 182.
  • the separating unit 182 separates the base stream from the encoded streams of all layers and supplies the base stream to the base decoding unit 183, and separates the enhancement stream and supplies it to the enhancement decoding unit 184.
  • the base decoding unit 183 is configured in the same manner as a conventional HEVC decoding device, decodes the base stream supplied from the separation unit 182 using the HEVC method, and generates a base image. However, the base decoding unit 183 supplies the motion vector used at the time of inter decoding of the base image to the enhancement decoding unit 184. The base decoding unit 183 outputs the generated base image.
  • the enhancement decoding unit 184 decodes the enhancement stream supplied from the separation unit 182 by a method according to the HEVC method, and generates an enhancement image. At this time, the enhancement decoding unit 184 refers to the motion vector supplied from the base decoding unit 183. The enhancement decoding unit 184 outputs the generated enhancement image.
  • FIG. 16 is a block diagram illustrating a configuration example of the enhancement decoding unit 184 of FIG.
  • the enhancement decoding unit 184 in FIG. 16 includes an extraction unit 201 and a decoding unit 202.
  • the extraction unit 201 of the enhancement decoding unit 184 extracts SPS, PPS, encoded data, and the like from the enhancement stream supplied from the separation unit 182 in FIG.
  • the decoding unit 202 refers to the motion vector of the base image supplied from the base decoding unit 183 in FIG. 15, and decodes the encoded data supplied from the extraction unit 201 by a method according to the HEVC method. At this time, the decoding unit 202 also refers to SPS, PPS, and the like supplied from the extraction unit 201 as necessary. The decoding unit 202 outputs an image obtained as a result of decoding as an enhancement image.
  • FIG. 17 is a block diagram illustrating a configuration example of the decoding unit 202 of FIG.
  • the decoding unit 202 includes a decoded picture buffer 309, a selection unit 310, an intra prediction unit 311, a motion compensation unit 312, and a selection unit 313.
  • the decoding unit 202 decodes the encoded data of the enhancement image supplied from the extraction unit 201 in FIG.
  • the accumulation buffer 301 accumulates the encoded data of the enhancement image supplied from the extraction unit 201 and supplies the encoded data to the lossless decoding unit 302 at a predetermined timing.
  • the lossless decoding unit 302 performs lossless decoding of the encoded data supplied from the accumulation buffer 301 by a method corresponding to the encoding method of the lossless encoding unit 106 in FIG.
  • the lossless decoding unit 302 supplies the quantized transform coefficient obtained by the lossless decoding to the inverse quantization unit 303.
  • the lossless decoding unit 302 supplies the intra prediction information to the intra prediction unit 311 when the intra prediction information is obtained by decoding the encoded data, and when the inter prediction information is obtained, the inter prediction information is obtained.
  • the prediction information is supplied to the motion compensation unit 312.
  • the inverse quantization unit 303 inversely quantizes the quantized transform coefficient supplied from the lossless decoding unit 302 by a method corresponding to the quantization method of the quantization unit 105 in FIG. 7, and reverses the obtained transform coefficient. This is supplied to the orthogonal transform unit 304.
  • the inverse orthogonal transform unit 304 performs inverse orthogonal transform on the transform coefficient supplied from the inverse quantization unit 303 by a method corresponding to the orthogonal transform method of the orthogonal transform unit 104 in FIG. 7 to obtain difference information.
  • the difference information obtained by the inverse orthogonal transform is supplied to the calculation unit 305.
  • a prediction image is supplied from the intra prediction unit 311 or the motion compensation unit 312 to the calculation unit 305 via the selection unit 313.
  • the calculation unit 305 adds the difference information and the predicted image to obtain a reconstructed image.
  • the arithmetic unit 305 supplies the reconstructed image to the loop filter 306 or the decoded picture buffer 309.
  • the loop filter 306 performs a filtering process on the reconstructed image supplied from the calculation unit 305 as appropriate, similarly to the filter 111 in FIG. 7, and generates a decoded image. Note that when a filter coefficient is obtained by the lossless decoding by the lossless decoding unit 302, the loop filter 306 performs a filter process using the filter coefficient.
  • the loop filter 306 supplies the decoded image to the screen rearrangement buffer 307 and the decoded picture buffer 309.
  • the screen rearrangement buffer 307 rearranges the decoded images supplied from the loop filter 306. That is, the order of frames rearranged for the encoding order by the screen rearrangement buffer 102 in FIG. 7 is rearranged in the original display order.
  • the D / A conversion unit 308 D / A converts the decoded image supplied from the screen rearrangement buffer 307.
  • the D / A conversion unit 308 outputs the enhancement image obtained as a result.
  • the decoded picture buffer 309 stores the reconstructed image supplied from the calculation unit 305.
  • the decoded picture buffer 309 stores the decoded image supplied from the loop filter 306.
  • the decoded picture buffer 309 supplies the stored reconstructed image to the intra prediction unit 311 via the selection unit 310 at a predetermined timing or based on an external request from the intra prediction unit 311 or the like. .
  • the decoded picture buffer 309 also stores the decoded image stored at a predetermined timing or based on a request from the outside such as the motion parallax prediction / compensation unit 115 via the selection unit 310. To supply.
  • the intra prediction unit 311 performs basically the same processing as the intra prediction unit 114 in FIG. However, the intra prediction unit 311 performs intra prediction in the optimal intra prediction mode of the intra prediction information supplied from the lossless decoding unit 302 only for a region where a prediction image is generated by intra prediction at the time of encoding. . The intra prediction unit 311 supplies a prediction image obtained as a result of the intra prediction to the selection unit 313.
  • the motion compensation unit 312 reads the decoded image specified by the reference image specifying information included in the inter prediction information supplied from the lossless decoding unit 302 from the decoded picture buffer 309 as a reference image.
  • the motion compensation unit 312 is based on the motion vector from the base decoding unit 183 and the base vector flag included in the SPS from the extraction unit 201, similarly to the motion parallax prediction / compensation unit 115 in FIG. A list in which motion vector identification information of motion vectors that are candidates for the above is registered is generated. The motion compensation unit 312 determines a motion vector identified by the motion vector identification information identified by the prediction vector identification information included in the inter prediction information from the generated list as a prediction vector. Then, the motion compensation unit 312 decodes the motion vector of the processing target PU by adding the prediction vector and the motion vector information included in the inter prediction information.
  • the motion compensation unit 312 performs a compensation process for the optimal inter prediction mode included in the inter prediction information, using the PU as a processing unit, based on the generated motion vector and the reference image, and generates a predicted image. Based on the inter prediction information supplied from the lossless decoding unit 302, the motion compensation unit 312 performs this compensation processing only on a region where inter prediction has been performed at the time of encoding. The motion compensation unit 312 supplies the generated predicted image to the calculation unit 305 via the selection unit 313.
  • the selection unit 313 supplies the prediction image supplied from the intra prediction unit 311 or the prediction image supplied from the motion compensation unit 312 to the calculation unit 305.
  • FIG. 18 is a block diagram illustrating a configuration example of the motion compensation unit 312 in FIG.
  • the processing unit 331 of the motion compensation unit 312 refers to the decoded image stored in the decoded picture buffer 309 based on the prediction direction and the reference image specifying information included in the inter prediction information from the lossless decoding unit 302 in FIG. Read as an image.
  • the processing unit 331 uses the motion vector supplied from the vector decoding unit 335 and the reference image to perform compensation processing in an optimal inter prediction mode, and generates a predicted image.
  • the processing unit 331 supplies the predicted image to the selection unit 313 in FIG.
  • the prediction information buffer 332 stores, as prediction information, the PU size, prediction direction, motion vector, and reference image specifying information represented by the optimal inter prediction mode, included in the inter prediction information supplied from the lossless decoding unit 302. .
  • the list registration unit 333 determines whether or not to use the motion vector of the base image as a prediction vector candidate based on the base vector flag included in the SPS supplied from the extraction unit 201 of FIG.
  • the list registration unit 333 sets the motion vector of the base image as a prediction vector candidate, the PU size, prediction direction, and reference image stored in the prediction information buffer 332 are the same as the list registration unit 133 in FIG. Based on the specific information, the motion vector identification information is registered in the list.
  • the list registration unit 333 is based on the prediction direction and the reference image specifying information stored in the prediction information buffer 332 as in the list registration unit 133. Register motion vector identification information in the list. Then, the list registration unit 333 supplies the generated list to the vector decoding unit 335. Note that the motion vector identification information in the list is given prediction vector identification information in the order of registration.
  • the vector buffer 334 stores the motion vector supplied from the base decoding unit 183.
  • the vector decoding unit 335 acquires motion vector identification information identified by the prediction vector identification information included in the inter prediction information supplied from the lossless decoding unit 302 from the list supplied from the list registration unit 333. Then, the vector decoding unit 335 reads the motion vector identified by the motion vector identification information from the prediction information buffer 332 or the vector buffer 334 as a prediction vector.
  • the vector decoding unit 335 performs motion vector decoding by adding the motion vector information and the prediction vector included in the inter prediction information supplied from the lossless decoding unit 302.
  • the vector decoding unit 335 supplies the motion vector to the processing unit 331.
  • FIG. 19 is a flowchart illustrating image generation processing of the decoding device 180 of FIG.
  • the reception unit 181 of the decoding device 180 receives the encoded stream of all layers transmitted from the encoding device 10 of FIG. 4 and supplies the encoded stream to the separation unit 182.
  • the separation unit 182 separates the base stream and the enhancement stream from the encoded streams of all layers supplied from the reception unit 181 and supplies the separated base stream and enhancement stream to the enhancement decoding unit 184.
  • step S172 the base decoding unit 183 decodes the base stream supplied from the separation unit 182 using the HEVC method, and generates a base image.
  • the base decoding unit 183 supplies the motion vector used at the time of inter decoding of the base image to the enhancement decoding unit 184.
  • the base decoding unit 183 outputs the generated base image.
  • step S173 the extraction unit 201 (FIG. 16) of the enhancement decoding unit 184 extracts the SPS from the enhancement stream supplied from the separation unit 182 and supplies the SPS to the decoding unit 202.
  • step S174 the extraction unit 201 extracts PPS from the enhancement stream and supplies the PPS to the decoding unit 202.
  • step S175 the extraction unit 201 extracts encoded data from the enhancement stream and supplies the encoded data to the decoding unit 202.
  • step S176 the decoding unit 202 uses the motion vector supplied from the base decoding unit 183 to perform decoding processing that decodes the encoded data supplied from the extraction unit 201 in a method according to the HEVC method, and the processing ends. To do. Details of the decoding process will be described with reference to FIG.
  • FIG. 20 is a flowchart for explaining the details of the decoding process in step S176 of FIG.
  • step S301 in FIG. 20 the accumulation buffer 301 accumulates the encoded data of the enhancement image supplied from the extraction unit 201 in FIG. 16, and supplies the encoded data to the lossless decoding unit 302 at a predetermined timing.
  • step S302 the lossless decoding unit 302 performs lossless decoding of the encoded data supplied from the accumulation buffer 301 by a method corresponding to the encoding method of the lossless encoding unit 106 in FIG.
  • the lossless decoding unit 302 supplies the quantized transform coefficient obtained by the lossless decoding to the inverse quantization unit 303.
  • the lossless decoding unit 302 supplies the intra prediction information to the intra prediction unit 311 when the intra prediction information is obtained by decoding the encoded data, and when the inter prediction information is obtained, the inter prediction information is obtained.
  • the prediction information is supplied to the motion compensation unit 312.
  • step S303 the inverse quantization unit 303 inversely quantizes the quantized transform coefficient supplied from the lossless decoding unit 302 by a method corresponding to the quantization method of the quantization unit 105 in FIG.
  • the transform coefficient is supplied to the inverse orthogonal transform unit 304.
  • step S304 the inverse orthogonal transform unit 304 performs inverse orthogonal transform on the transform coefficient supplied from the inverse quantization unit 303 by a method corresponding to the orthogonal transform method of the orthogonal transform unit 104 in FIG. 7 to obtain difference information.
  • the difference information obtained by the inverse orthogonal transform is supplied to the calculation unit 305.
  • step S305 the intra prediction unit 311 determines whether or not intra prediction information is supplied from the lossless decoding unit 302. When it is determined in step S305 that intra prediction information has been supplied, in step S306, the intra prediction unit 311 performs intra prediction basically similar to the intra prediction unit 114 in FIG. The intra prediction unit 311 supplies the prediction image obtained as a result of the intra prediction to the selection unit 313, and the process proceeds to step S310.
  • step S305 if it is determined in step S305 that intra prediction information is not supplied, that is, if the motion compensation unit 312 acquires inter prediction information, the process proceeds to step S307.
  • step S307 the motion compensation unit 312 performs a list generation process similar to the list generation process of FIG.
  • step S308 the vector decoding unit 335 (FIG. 18) of the motion compensation unit 312 is based on the list generated by the process of step S307 and the prediction vector identification information included in the inter prediction information from the lossless decoding unit 302. Then, the motion vector information included in the inter prediction information is decoded by the AMVP method. And the vector decoding part 335 supplies the motion vector obtained as a result to the process part 331, and advances a process to step S309.
  • step S309 the processing unit 331 uses the PU as a processing unit based on the motion vector from the vector decoding unit 335 and the reference image read from the decoded picture buffer 309, and performs the optimal inter prediction included in the inter prediction information.
  • a mode compensation process is performed to generate a predicted image.
  • the processing unit 331 supplies the generated predicted image to the calculation unit 305 via the selection unit 313, and advances the processing to step S310.
  • step S310 the calculation unit 305 adds the difference information supplied from the inverse orthogonal transform unit 304 and the predicted image supplied from the selection unit 313 to obtain a reconstructed image.
  • the arithmetic unit 305 supplies the reconstructed image to the loop filter 306 or the decoded picture buffer 309.
  • step S311 the loop filter 306 generates a decoded image by appropriately performing filter processing on the reconstructed image supplied from the calculation unit 305, similarly to the filter 111 in FIG. Note that when a filter coefficient is obtained by the lossless decoding by the lossless decoding unit 302, the loop filter 306 performs a filter process using the filter coefficient.
  • the loop filter 306 supplies the decoded image as the filter processing result to the screen rearrangement buffer 307 and the decoded picture buffer 309.
  • step S312 the screen rearrangement buffer 307 rearranges the decoded images supplied from the loop filter 306.
  • step S313 the D / A conversion unit 308 D / A converts the decoded image supplied from the screen rearrangement buffer 307, and outputs the enhancement image obtained as a result.
  • step S314 the decoded picture buffer 309 stores the reconstructed image supplied from the calculation unit 305 and the decoded image supplied from the loop filter 306.
  • the reconstructed image stored in the decoded picture buffer 309 is supplied to the intra prediction unit 311 via the selection unit 310.
  • the decoded image stored in the decoded picture buffer 309 is supplied to the motion compensation unit 312 via the selection unit 310. After the process of step S314, the process returns to step S176 of FIG. 19 and ends.
  • the decoding device 180 registers motion vector prediction vector candidates of the enhancement image including the motion vector of the base image in the list based on the PU size as the feature of the enhancement image. Therefore, it is possible to decode the enhancement stream transmitted from the encoding device 10 and improved in encoding efficiency.
  • the registration order of the motion vector of the block ColBL is determined at the head or the end based on the size of the PU to be processed. However, whether or not the motion vector of the block ColBL is registered is determined. May be. This case will be described with reference to FIGS. 21 and 22 below.
  • FIG. 21 is a diagram illustrating a method for determining the registration order of prediction vector candidates in this case.
  • the registration order is the same as in the case described with reference to FIG. Is determined. That is, the registration order of the motion vectors of the block ColBL is determined at the top.
  • the motion vector of the block ColBL is not regarded as a prediction vector candidate. Therefore, as in the conventional case shown in FIG. 2A, the motion vector of the block A0 or the block A1, the motion vector of any of the blocks B0 to B2, and the motion vector of the block H or the block C0 are sequentially from the front.
  • the registration order is determined as follows.
  • the predetermined size is half the size of the CU
  • the registration order of the motion vector of the block ColBL in the list for the PU 178-2 is first.
  • the motion vectors of the block ColBL for the PUs 174-1 to 174-4, PU175-2, PU176-1, PU177-2, and PU178-1 in FIG. 10 are not registered in the list.
  • the registration order is determined as in the case described with reference to FIG.
  • the motion vectors of the base image are listed. Not registered. Therefore, it is possible to reduce the prediction vector identification information of the motion vector of the enhancement image that is highly likely to be a prediction vector, and to improve the encoding efficiency.
  • FIG. 22 is a flowchart for explaining the list generation processing by the list registration unit 133 when the registration order is determined by the determination method of FIG.
  • step S332 determines that the base vector flag is not 1
  • the list registration unit 133 determines that the motion vector of the base image is not a prediction vector candidate, and the process proceeds to step S338. If it is determined in step S336 that the size of the PU to be processed is not greater than or equal to N ⁇ N pixels, the process proceeds to step S338.
  • step S338 the list registration unit 133 determines the registration order of motion vectors of blocks other than the block ColBL, and advances the process to step S339.
  • steps S339 to S342 are the same as the processes in steps S141, S142, S145, and S146 in FIG.
  • the processes in steps S138 and S139 and the processes in steps S143 and S144 in FIG. 14 are not performed, so that the list can be easily generated. 22 is performed by the list registration unit 133, the list generation unit 333 performs the same list generation process as the list generation process of FIG.
  • the registration order is determined based on the PU size of the enhancement image.
  • the registration order may be determined based on the PU size of the base image corresponding to the PU.
  • the registration order may be determined based on the sizes of both the enhancement image PU and the corresponding base image PU.
  • the motion vector of the block ColBL is registered at the end of the list or not registered.
  • the motion vector of the block ColBL is registered at the top of the list.
  • the registration order may be determined based on the CU size instead of the PU size.
  • the motion vector of the block ColBL is registered at the top of the list.
  • the motion vector of the block ColBL is registered at the end of the list or not registered.
  • the size of the CU used for determining the registration order is the size of at least one of the CU to be processed for the enhancement image and the CU of the base image corresponding to the CU, as in the case of the PU.
  • the registration order may be determined based on the sizes of both the CU and the PU. In this case, for example, when the size of the CU is 32 ⁇ 32 pixels or more, the motion vector of the block ColBL is registered at the top of the list. If the CU size is 16 ⁇ 16 pixels or 8 ⁇ 8 pixels smaller than 32 ⁇ 32 pixels, and the PU size is N ⁇ N pixels or more, the motion vector of the block ColBL is registered at the top of the list.
  • the registration order may be determined based on the TU size, not the PU or CU size.
  • the registration order may be determined based on the type of scalability function instead of the PU size.
  • the type of the scalability function is spatial scalability where the division value ⁇ obtained by dividing the spatial resolution of the enhancement image (EL) by the spatial resolution of the base image (BL) is greater than 1.
  • the motion vector of block ColBL is registered at the end of the list or not registered.
  • the division value ⁇ is larger than 1, it is necessary to scale the motion vector in order to make the motion vector of the base image a candidate for a prediction vector.
  • the spatial resolution of the enhancement image is twice the spatial resolution of the base image, it is necessary to scale the motion vector of the base image to twice. Therefore, in this case, the correlation between the motion vectors of the base image and the enhancement image is relatively weak. Therefore, the motion vector of the block ColBL of the base image is registered at the end of the list or not registered.
  • the motion vector need not be scaled, so the motion vector of the block ColBL is registered at the top of the list.
  • the registration order of the list may be determined based on the quantization parameter of the base image and the enhancement image instead of the PU size.
  • the quantization parameter QP E of the enhancement image is larger than the quantization parameter QP B of the base image
  • the motion vector of the block ColBL is registered at the top of the list.
  • the quantization parameter QP E is equal to or less than the quantization parameter QP B
  • the motion vector of the block ColBL is registered at the end of the list or not registered.
  • QP E As the quantization parameter QP E (QP B ), a quantization parameter in units of pictures (PicQP) and a quantization parameter in units of slices (Slice QP) can be used in addition to the quantization parameter in units of CUs.
  • PicQP quantization parameter in units of pictures
  • Slice QP quantization parameter in units of slices
  • the hierarchy peripheral PU is not limited to the block ColBL, and may be another PU as long as it is the position of the base image whose position in the screen is the same as the PU to be processed or the PU around the position.
  • the number of hierarchical peripheral PUs may be plural.
  • the hierarchical peripheral PU may be a block ColBL and a block HBL that is the lower right PU of the base image region whose position in the screen is the same as the processing target PU.
  • the motion vector of the block ColBL or the block HBL is selected as a prediction vector candidate and registered in the list. Then, the motion vector of the block C0 or the block H at the position corresponding to the unregistered motion vector is registered in the list.
  • the motion vector of the block ColBL when the motion vector of the block ColBL is registered in the list, the motion vector of the block H is registered in the list, and when the motion vector of the block HBL is registered in the list, the motion vector of the block C0 is registered in the list. be registered.
  • the motion vector of the block C0 or the block H is selected as a prediction vector candidate and registered in the list. Then, the motion vector of the block ColBL or the block HBL at the position corresponding to the motion vector not registered is registered in the list.
  • the motion vector of the block HBL is registered in the list.
  • the motion vector of the block ColBL is registered in the list. be registered.
  • the base vector flag may be set not in sequence units but in picture units or sequence units.
  • the base vector flag (pps_enable_BLMV_flag) in units of pictures is included in the PPS as shown in the 19th line of FIG.
  • the base vector flag (slice_enable_BLMV_flag) in units of slices is included in the slice header as shown in the sixth line of FIG.
  • the enhancement stream may include a change flag and a switch flag instead of the base vector flag.
  • the change flag is a flag for identifying a motion vector of a base image as a candidate for a prediction vector and determining that the order of registration of the base image in the motion vector list is determined based on the feature of the image.
  • the switching flag is a flag for identifying whether the motion vector of the base image is a candidate for a prediction vector and switching the presence or absence of registration of the motion vector of the base image in the list based on the feature of the image.
  • the prediction vector candidate is also based on the feature of the image. A list of is generated.
  • FIG. 29 shows an example of a multi-view image encoding method.
  • the multi-viewpoint image includes a plurality of viewpoint images, and a predetermined one viewpoint image among the plurality of viewpoints is designated as the base view image.
  • Each viewpoint image other than the base view image is treated as a non-base view image.
  • dQP (base view) Current_CU_QP (base view)-LCU_QP (base view) (1-2)
  • dQP (base view) Current_CU_QP (base view)-Previsous_CU_QP (base view) (1-3)
  • dQP (base view) Current_CU_QP (base view)-Slice_QP (base view)
  • base view Current_CU_QP (base view)-Slice_QP (base view)
  • non-base-view: (2-1) dQP (non-base view) Current_CU_QP (non-base view)-LCU_QP (non-base view) (2-2)
  • dQP (non-base view) Current QP (non-base view)-Previsous QP (non-base view) (2-3)
  • dQP (non-base view) Current_CU_QP (non-base view)-Slice
  • the above (1) to (4) can be used in combination.
  • a method of obtaining a quantization parameter difference at the slice level between the base view and the non-base view (combining 3-1 and 2-3), between the base view and the non-base view
  • the method of taking the difference of the quantization parameter at the LCU level (combining 3-2 and 2-1) can be considered.
  • the difference can be improved even when multi-viewpoint encoding is performed.
  • a flag for identifying whether or not there is a dQP whose value is not 0 can be set for each of the above dQPs.
  • FIG. 30 shows another example of encoding by the scalability function.
  • dQP (base layer) Current_CU_QP (base layer)-LCU_QP (base layer) (1-2)
  • dQP (base layer) Current_CU_QP (base layer)-Previsous_CU_QP (base layer) (1-3)
  • dQP (base layer) Current_CU_QP (base layer)-Slice_QP (base layer)
  • non-base-layer (2-1)
  • dQP (non-base layer) Current_CU_QP (non-base layer)-LCU_QP (non-base layer) (2-2)
  • dQP (non-base layer) Current QP (non-base layer)-Previsous QP (non-base layer) (2-3)
  • the above (1) to (4) can be used in combination.
  • the method of taking the difference of the quantization parameter at the LCU level (combining 3-2 and 2-1) can be considered. In this manner, by applying the difference repeatedly, the encoding efficiency can be improved even when hierarchical encoding is performed.
  • a flag for identifying whether or not there is a dQP whose value is not 0 can be set for each of the above dQPs.
  • the series of processes described above can be executed by hardware or can be executed by software.
  • a program constituting the software is installed in the computer.
  • the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.
  • FIG. 31 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processing by a program.
  • a CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • An input / output interface 605 is further connected to the bus 604.
  • An input unit 606, an output unit 607, a storage unit 608, a communication unit 609, and a drive 610 are connected to the input / output interface 605.
  • the input unit 606 includes a keyboard, a mouse, a microphone, and the like.
  • the output unit 607 includes a display, a speaker, and the like.
  • the storage unit 608 includes a hard disk, a nonvolatile memory, and the like.
  • the communication unit 609 includes a network interface or the like.
  • the drive 610 drives a removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • the CPU 601 loads the program stored in the storage unit 608 to the RAM 603 via the input / output interface 605 and the bus 604 and executes the program, for example. Is performed.
  • the program executed by the computer (CPU 601) can be provided by being recorded on a removable medium 611 as a package medium, for example.
  • the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
  • the program can be installed in the storage unit 608 via the input / output interface 605 by attaching the removable medium 611 to the drive 610. Further, the program can be received by the communication unit 609 via a wired or wireless transmission medium and installed in the storage unit 608. In addition, the program can be installed in the ROM 602 or the storage unit 608 in advance.
  • the program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.
  • FIG. 32 illustrates a schematic configuration of a television apparatus to which the present technology is applied.
  • the television apparatus 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, and an external interface unit 909. Furthermore, the television apparatus 900 includes a control unit 910, a user interface unit 911, and the like.
  • the tuner 902 selects a desired channel from the broadcast wave signal received by the antenna 901, demodulates it, and outputs the obtained encoded bit stream to the demultiplexer 903.
  • the demultiplexer 903 extracts video and audio packets of the program to be viewed from the encoded bit stream, and outputs the extracted packet data to the decoder 904. Further, the demultiplexer 903 supplies a packet of data such as EPG (Electronic Program Guide) to the control unit 910. If scrambling is being performed, descrambling is performed by a demultiplexer or the like.
  • EPG Electronic Program Guide
  • the decoder 904 performs packet decoding processing, and outputs video data generated by the decoding processing to the video signal processing unit 905 and audio data to the audio signal processing unit 907.
  • the video signal processing unit 905 performs noise removal, video processing according to user settings, and the like on the video data.
  • the video signal processing unit 905 generates video data of a program to be displayed on the display unit 906, image data by processing based on an application supplied via a network, and the like.
  • the video signal processing unit 905 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data of the program.
  • the video signal processing unit 905 generates a drive signal based on the video data generated in this way, and drives the display unit 906.
  • the display unit 906 drives a display device (for example, a liquid crystal display element or the like) based on a drive signal from the video signal processing unit 905 to display a program video or the like.
  • a display device for example, a liquid crystal display element or the like
  • the audio signal processing unit 907 performs predetermined processing such as noise removal on the audio data, performs D / A conversion processing and amplification processing on the processed audio data, and outputs the audio data to the speaker 908.
  • the external interface unit 909 is an interface for connecting to an external device or a network, and transmits and receives data such as video data and audio data.
  • a user interface unit 911 is connected to the control unit 910.
  • the user interface unit 911 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 910.
  • the control unit 910 is configured using a CPU (Central Processing Unit), a memory, and the like.
  • the memory stores a program executed by the CPU, various data necessary for the CPU to perform processing, EPG data, data acquired via a network, and the like.
  • the program stored in the memory is read and executed by the CPU at a predetermined timing such as when the television device 900 is activated.
  • the CPU executes each program to control each unit so that the television device 900 operates in accordance with the user operation.
  • the television device 900 includes a bus 912 for connecting the tuner 902, the demultiplexer 903, the video signal processing unit 905, the audio signal processing unit 907, the external interface unit 909, and the control unit 910.
  • the decoder 904 is provided with the function of the image processing apparatus (image processing method) of the present application. For this reason, when the motion vector of the base layer image is used as the motion vector prediction vector candidate of the enhancement layer image, it is possible to decode the encoded stream that improves the encoding efficiency.
  • FIG. 33 illustrates a schematic configuration of a mobile phone to which the present technology is applied.
  • the cellular phone 920 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, and a control unit 931. These are connected to each other via a bus 933.
  • an antenna 921 is connected to the communication unit 922, and a speaker 924 and a microphone 925 are connected to the audio codec 923. Further, an operation unit 932 is connected to the control unit 931.
  • the mobile phone 920 performs various operations such as transmission / reception of voice signals, transmission / reception of e-mail and image data, image shooting, and data recording in various modes such as a voice call mode and a data communication mode.
  • the voice signal generated by the microphone 925 is converted into voice data and compressed by the voice codec 923 and supplied to the communication unit 922.
  • the communication unit 922 performs audio data modulation processing, frequency conversion processing, and the like to generate a transmission signal.
  • the communication unit 922 supplies a transmission signal to the antenna 921 and transmits it to a base station (not shown).
  • the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and supplies the obtained audio data to the audio codec 923.
  • the audio codec 923 performs data expansion of the audio data and conversion to an analog audio signal and outputs the result to the speaker 924.
  • the control unit 931 receives character data input by operating the operation unit 932 and displays the input characters on the display unit 930.
  • the control unit 931 generates mail data based on a user instruction or the like in the operation unit 932 and supplies the mail data to the communication unit 922.
  • the communication unit 922 performs mail data modulation processing, frequency conversion processing, and the like, and transmits the obtained transmission signal from the antenna 921.
  • the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores mail data. This mail data is supplied to the display unit 930 to display the mail contents.
  • the mobile phone 920 can also store the received mail data in a storage medium by the recording / playback unit 929.
  • the storage medium is any rewritable storage medium.
  • the storage medium is a removable medium such as a semiconductor memory such as a RAM or a built-in flash memory, a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.
  • the image data generated by the camera unit 926 is supplied to the image processing unit 927.
  • the image processing unit 927 performs encoding processing of image data and generates encoded data.
  • the demultiplexing unit 928 multiplexes the encoded data generated by the image processing unit 927 and the audio data supplied from the audio codec 923 by a predetermined method, and supplies the multiplexed data to the communication unit 922.
  • the communication unit 922 performs modulation processing and frequency conversion processing of multiplexed data, and transmits the obtained transmission signal from the antenna 921.
  • the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores multiplexed data. This multiplexed data is supplied to the demultiplexing unit 928.
  • the demultiplexing unit 928 performs demultiplexing of the multiplexed data, and supplies the encoded data to the image processing unit 927 and the audio data to the audio codec 923.
  • the image processing unit 927 performs a decoding process on the encoded data to generate image data.
  • the image data is supplied to the display unit 930 and the received image is displayed.
  • the audio codec 923 converts the audio data into an analog audio signal, supplies the analog audio signal to the speaker 924, and outputs the received audio.
  • the image processing unit 927 is provided with the function of the image processing device (image processing method) of the present application. For this reason, when the motion vector of the base layer image is used as a candidate for the motion vector prediction vector of the enhancement layer image, the encoding efficiency can be improved. In addition, when the motion vector of the base layer image is used as a motion vector prediction vector candidate for the enhancement layer image, it is possible to decode an encoded stream that improves the encoding efficiency.
  • FIG. 34 illustrates a schematic configuration of a recording / reproducing apparatus to which the present technology is applied.
  • the recording / reproducing apparatus 940 records, for example, audio data and video data of a received broadcast program on a recording medium, and provides the recorded data to the user at a timing according to a user instruction.
  • the recording / reproducing device 940 can also acquire audio data and video data from another device, for example, and record them on a recording medium. Further, the recording / reproducing apparatus 940 decodes and outputs the audio data and video data recorded on the recording medium, thereby enabling image display and audio output on the monitor apparatus or the like.
  • the recording / reproducing apparatus 940 includes a tuner 941, an external interface unit 942, an encoder 943, an HDD (Hard Disk Drive) unit 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) unit 948, a control unit 949, A user interface unit 950 is included.
  • Tuner 941 selects a desired channel from a broadcast signal received by an antenna (not shown).
  • the tuner 941 outputs an encoded bit stream obtained by demodulating the received signal of a desired channel to the selector 946.
  • the external interface unit 942 includes at least one of an IEEE 1394 interface, a network interface unit, a USB interface, a flash memory interface, and the like.
  • the external interface unit 942 is an interface for connecting to an external device, a network, a memory card, and the like, and receives data such as video data and audio data to be recorded.
  • the encoder 943 performs encoding by a predetermined method when the video data and audio data supplied from the external interface unit 942 are not encoded, and outputs an encoded bit stream to the selector 946.
  • the HDD unit 944 records content data such as video and audio, various programs, and other data on a built-in hard disk, and reads them from the hard disk during playback.
  • the disk drive 945 records and reproduces signals with respect to the mounted optical disk.
  • An optical disk such as a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.), a Blu-ray (registered trademark) disk, or the like.
  • the selector 946 selects one of the encoded bit streams from the tuner 941 or the encoder 943 and supplies it to either the HDD unit 944 or the disk drive 945 when recording video or audio. Further, the selector 946 supplies the encoded bit stream output from the HDD unit 944 or the disk drive 945 to the decoder 947 at the time of reproduction of video and audio.
  • the decoder 947 performs a decoding process on the encoded bit stream.
  • the decoder 947 supplies the video data generated by performing the decoding process to the OSD unit 948.
  • the decoder 947 outputs audio data generated by performing the decoding process.
  • the OSD unit 948 generates video data for displaying a menu screen for selecting an item and the like, and superimposes it on the video data output from the decoder 947 and outputs the video data.
  • a user interface unit 950 is connected to the control unit 949.
  • the user interface unit 950 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 949.
  • the control unit 949 is configured using a CPU, a memory, and the like.
  • the memory stores programs executed by the CPU and various data necessary for the CPU to perform processing.
  • the program stored in the memory is read and executed by the CPU at a predetermined timing such as when the recording / reproducing apparatus 940 is activated.
  • the CPU executes the program to control each unit so that the recording / reproducing device 940 operates according to the user operation.
  • the decoder 947 is provided with the function of the image processing apparatus (image processing method) of the present application. For this reason, when the motion vector of the base layer image is used as the motion vector prediction vector candidate of the enhancement layer image, it is possible to decode the encoded stream that improves the encoding efficiency.
  • FIG. 35 illustrates a schematic configuration of an imaging apparatus to which the present technology is applied.
  • the imaging device 960 images a subject, displays an image of the subject on a display unit, and records it on a recording medium as image data.
  • the imaging device 960 includes an optical block 961, an imaging unit 962, a camera signal processing unit 963, an image data processing unit 964, a display unit 965, an external interface unit 966, a memory unit 967, a media drive 968, an OSD unit 969, and a control unit 970. Have. In addition, a user interface unit 971 is connected to the control unit 970. Furthermore, the image data processing unit 964, the external interface unit 966, the memory unit 967, the media drive 968, the OSD unit 969, the control unit 970, and the like are connected via a bus 972.
  • the optical block 961 is configured using a focus lens, a diaphragm mechanism, and the like.
  • the optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962.
  • the imaging unit 962 is configured using a CCD or CMOS image sensor, generates an electrical signal corresponding to the optical image by photoelectric conversion, and supplies the electrical signal to the camera signal processing unit 963.
  • the camera signal processing unit 963 performs various camera signal processing such as knee correction, gamma correction, and color correction on the electrical signal supplied from the imaging unit 962.
  • the camera signal processing unit 963 supplies the image data after the camera signal processing to the image data processing unit 964.
  • the image data processing unit 964 performs an encoding process on the image data supplied from the camera signal processing unit 963.
  • the image data processing unit 964 supplies the encoded data generated by performing the encoding process to the external interface unit 966 and the media drive 968. Further, the image data processing unit 964 performs a decoding process on the encoded data supplied from the external interface unit 966 and the media drive 968.
  • the image data processing unit 964 supplies the image data generated by performing the decoding process to the display unit 965. Further, the image data processing unit 964 superimposes the processing for supplying the image data supplied from the camera signal processing unit 963 to the display unit 965 and the display data acquired from the OSD unit 969 on the image data. To supply.
  • the OSD unit 969 generates display data such as a menu screen and icons made up of symbols, characters, or figures and outputs them to the image data processing unit 964.
  • the external interface unit 966 includes, for example, a USB input / output terminal, and is connected to a printer when printing an image.
  • a drive is connected to the external interface unit 966 as necessary, a removable medium such as a magnetic disk or an optical disk is appropriately mounted, and a computer program read from them is installed as necessary.
  • the external interface unit 966 has a network interface connected to a predetermined network such as a LAN or the Internet.
  • the control unit 970 reads the encoded data from the media drive 968 in accordance with an instruction from the user interface unit 971, and supplies the encoded data to the other device connected via the network from the external interface unit 966. it can.
  • the control unit 970 may acquire encoded data and image data supplied from another device via the network via the external interface unit 966 and supply the acquired data to the image data processing unit 964. it can.
  • any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory is used.
  • the recording medium may be any type of removable medium, and may be a tape device, a disk, or a memory card. Of course, a non-contact IC (Integrated Circuit) card may be used.
  • media drive 968 and the recording medium may be integrated and configured by a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive).
  • a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive).
  • the control unit 970 is configured using a CPU.
  • the memory unit 967 stores a program executed by the control unit 970, various data necessary for the control unit 970 to perform processing, and the like.
  • the program stored in the memory unit 967 is read out and executed by the control unit 970 at a predetermined timing such as when the imaging device 960 is activated.
  • the control unit 970 controls each unit so that the imaging device 960 performs an operation according to a user operation by executing a program.
  • the image data processing unit 964 is provided with the function of the image processing apparatus (image processing method) of the present application. For this reason, when the motion vector of the base layer image is used as a candidate for the motion vector prediction vector of the enhancement layer image, the encoding efficiency can be improved. In addition, when the motion vector of the base layer image is used as a motion vector prediction vector candidate for the enhancement layer image, it is possible to decode an encoded stream that improves the encoding efficiency.
  • Scalable encoding is used for selection of data to be transmitted, for example, as in the example shown in FIG.
  • the distribution server 1002 reads the scalable encoded data stored in the scalable encoded data storage unit 1001, and via the network 1003, the personal computer 1004, the AV device 1005, the tablet This is distributed to the terminal device such as the device 1006 and the mobile phone 1007.
  • the distribution server 1002 selects and transmits encoded data of appropriate quality according to the capability of the terminal device, the communication environment, and the like. Even if the distribution server 1002 transmits unnecessarily high-quality data, the terminal device does not always obtain a high-quality image, and may cause a delay or an overflow. Moreover, there is a possibility that the communication band is unnecessarily occupied or the load on the terminal device is unnecessarily increased. On the other hand, even if the distribution server 1002 transmits unnecessarily low quality data, there is a possibility that an image with sufficient image quality cannot be obtained in the terminal device. Therefore, the distribution server 1002 appropriately reads and transmits the scalable encoded data stored in the scalable encoded data storage unit 1001 as encoded data having an appropriate quality with respect to the capability and communication environment of the terminal device. .
  • the scalable encoded data storage unit 1001 stores scalable encoded data (BL + EL) 1011 encoded in a scalable manner.
  • the scalable encoded data (BL + EL) 1011 is encoded data including both a base layer and an enhancement layer, and is a data that can be decoded to obtain both a base layer image and an enhancement layer image. It is.
  • the distribution server 1002 selects an appropriate layer according to the capability of the terminal device that transmits data, the communication environment, and the like, and reads the data of the layer. For example, the distribution server 1002 reads high-quality scalable encoded data (BL + EL) 1011 from the scalable encoded data storage unit 1001 and transmits it to the personal computer 1004 and the tablet device 1006 with high processing capability as they are. . On the other hand, for example, the distribution server 1002 extracts base layer data from the scalable encoded data (BL + EL) 1011 for the AV device 1005 and the cellular phone 1007 having a low processing capability, and performs scalable encoding. Although it is data of the same content as the data (BL + EL) 1011, it is transmitted as scalable encoded data (BL) 1012 having a lower quality than the scalable encoded data (BL + EL) 1011.
  • BL scalable encoded data
  • scalable encoded data By using scalable encoded data in this way, the amount of data can be easily adjusted, so that the occurrence of delay and overflow can be suppressed, and the unnecessary increase in the load on the terminal device and communication medium can be suppressed. be able to.
  • scalable encoded data (BL + EL) 1011 since scalable encoded data (BL + EL) 1011 has reduced redundancy between layers, the amount of data can be reduced as compared with the case where encoded data of each layer is used as individual data. . Therefore, the storage area of the scalable encoded data storage unit 1001 can be used more efficiently.
  • the hardware performance of the terminal device varies depending on the device.
  • the application which a terminal device performs is also various, the capability of the software is also various.
  • the network 1003 serving as a communication medium can be applied to any communication network including wired, wireless, or both, such as the Internet and a LAN (Local Area Network), and has various data transmission capabilities. Furthermore, there is a risk of change due to other communications.
  • the distribution server 1002 communicates with the terminal device that is the data transmission destination before starting data transmission, and the hardware performance of the terminal device, the performance of the application (software) executed by the terminal device, etc. Information regarding the capability of the terminal device and information regarding the communication environment such as the available bandwidth of the network 1003 may be obtained. The distribution server 1002 may select an appropriate layer based on the information obtained here.
  • the layer extraction may be performed by the terminal device.
  • the personal computer 1004 may decode the transmitted scalable encoded data (BL + EL) 1011 and display a base layer image or an enhancement layer image. Further, for example, the personal computer 1004 extracts the base layer scalable encoded data (BL) 1012 from the transmitted scalable encoded data (BL + EL) 1011 and stores it or transfers it to another device. The base layer image may be displayed after decoding.
  • the numbers of the scalable encoded data storage unit 1001, the distribution server 1002, the network 1003, and the terminal devices are arbitrary.
  • the example in which the distribution server 1002 transmits data to the terminal device has been described, but the usage example is not limited to this.
  • the data transmission system 1000 may be any system as long as it transmits a scalable encoded data to a terminal device by selecting an appropriate layer according to the capability of the terminal device or a communication environment. Can be applied to the system.
  • scalable coding is used for transmission via a plurality of communication media, for example, as in the example shown in FIG.
  • a broadcasting station 1101 transmits base layer scalable encoded data (BL) 1121 by terrestrial broadcasting 1111. Also, the broadcast station 1101 transmits enhancement layer scalable encoded data (EL) 1122 via an arbitrary network 1112 including a wired or wireless communication network or both (for example, packetized transmission).
  • BL base layer scalable encoded data
  • EL enhancement layer scalable encoded data
  • the terminal apparatus 1102 has a reception function of the terrestrial broadcast 1111 broadcast by the broadcast station 1101 and receives base layer scalable encoded data (BL) 1121 transmitted via the terrestrial broadcast 1111.
  • the terminal apparatus 1102 further has a communication function for performing communication via the network 1112, and receives enhancement layer scalable encoded data (EL) 1122 transmitted via the network 1112.
  • BL base layer scalable encoded data
  • EL enhancement layer scalable encoded data
  • the terminal device 1102 decodes the base layer scalable encoded data (BL) 1121 acquired via the terrestrial broadcast 1111 according to, for example, a user instruction, and obtains or stores a base layer image. Or transmit to other devices.
  • BL base layer scalable encoded data
  • the terminal device 1102 for example, in response to a user instruction, the base layer scalable encoded data (BL) 1121 acquired via the terrestrial broadcast 1111 and the enhancement layer scalable encoded acquired via the network 1112 Data (EL) 1122 is combined to obtain scalable encoded data (BL + EL), or decoded to obtain an enhancement layer image, stored, or transmitted to another device.
  • BL base layer scalable encoded data
  • EL enhancement layer scalable encoded acquired via the network 1112 Data
  • the scalable encoded data can be transmitted via a communication medium that is different for each layer, for example. Therefore, the load can be distributed, and the occurrence of delay and overflow can be suppressed.
  • the communication medium used for transmission may be selected for each layer. For example, scalable encoded data (BL) 1121 of a base layer having a relatively large amount of data is transmitted via a communication medium having a wide bandwidth, and scalable encoded data (EL) 1122 having a relatively small amount of data is transmitted. You may make it transmit via a communication medium with a narrow bandwidth. Further, for example, the communication medium for transmitting the enhancement layer scalable encoded data (EL) 1122 is switched between the network 1112 and the terrestrial broadcast 1111 according to the available bandwidth of the network 1112. May be. Of course, the same applies to data of an arbitrary layer.
  • the number of layers is arbitrary, and the number of communication media used for transmission is also arbitrary.
  • the number of terminal devices 1102 serving as data distribution destinations is also arbitrary.
  • broadcasting from the broadcasting station 1101 has been described as an example, but the usage example is not limited to this.
  • the data transmission system 1100 can be applied to any system as long as it is a system that divides scalable encoded data into a plurality of layers and transmits them through a plurality of lines.
  • scalable encoding is used for storing encoded data as in the example shown in FIG. 38, for example.
  • the imaging device 1201 performs scalable coding on image data obtained by imaging the subject 1211, and as scalable coded data (BL + EL) 1221, a scalable coded data storage device 1202. To supply.
  • the scalable encoded data storage device 1202 stores the scalable encoded data (BL + EL) 1221 supplied from the imaging device 1201 with quality according to the situation. For example, in the normal case, the scalable encoded data storage device 1202 extracts base layer data from the scalable encoded data (BL + EL) 1221, and the base layer scalable encoded data ( BL) 1222. On the other hand, for example, in the case of attention, the scalable encoded data storage device 1202 stores scalable encoded data (BL + EL) 1221 with high quality and a large amount of data.
  • the scalable encoded data storage device 1202 can store an image with high image quality only when necessary, so that an increase in the amount of data can be achieved while suppressing a reduction in the value of the image due to image quality degradation. And the use efficiency of the storage area can be improved.
  • the imaging device 1201 is a surveillance camera.
  • the monitoring target for example, an intruder
  • the content of the captured image is likely to be unimportant, so reduction of the data amount is given priority, and the image data (scalable coding) Data) is stored in low quality.
  • the image quality is given priority and the image data (scalable) (Encoded data) is stored with high quality.
  • whether it is normal time or attention time may be determined by the scalable encoded data storage device 1202 analyzing an image, for example.
  • the imaging apparatus 1201 may make a determination, and the determination result may be transmitted to the scalable encoded data storage device 1202.
  • the criterion for determining whether the time is normal or noting is arbitrary, and the content of the image as the criterion is arbitrary. Of course, conditions other than the contents of the image can also be used as the criterion. For example, it may be switched according to the volume or waveform of the recorded sound, may be switched at every predetermined time, or may be switched by an external instruction such as a user instruction.
  • the number of states is arbitrary, for example, normal, slightly attention, attention, very attention, etc.
  • three or more states may be switched.
  • the upper limit number of states to be switched depends on the number of layers of scalable encoded data.
  • the imaging apparatus 1201 may determine the number of layers for scalable coding according to the state. For example, in a normal case, the imaging apparatus 1201 may generate base layer scalable encoded data (BL) 1222 with low quality and a small amount of data, and supply the scalable encoded data storage apparatus 1202 to the scalable encoded data storage apparatus 1202. For example, when attention is paid, the imaging device 1201 generates scalable encoded data (BL + EL) 1221 having a high quality and a large amount of data, and supplies the scalable encoded data storage device 1202 to the scalable encoded data storage device 1202. May be.
  • BL base layer scalable encoded data
  • BL + EL scalable encoded data
  • the monitoring camera has been described as an example.
  • the use of the imaging system 1200 is arbitrary and is not limited to the monitoring camera.
  • the method for transmitting such information is not limited to such an example.
  • these pieces of information may be transmitted or recorded as separate data associated with the encoded bitstream without being multiplexed into the encoded bitstream.
  • the term “associate” means that an image (which may be a part of an image such as a slice or a block) included in the bitstream and information corresponding to the image can be linked at the time of decoding. Means. That is, information may be transmitted on a transmission path different from that of the image (or bit stream).
  • Information may be recorded on a recording medium (or another recording area of the same recording medium) different from the image (or bit stream). Furthermore, the information and the image (or bit stream) may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part of the frame.
  • This technology receives bitstreams compressed by orthogonal transform such as discrete cosine transform and motion compensation, such as MPEG and H.26x, via network media such as satellite broadcasting, cable TV, the Internet, and mobile phones.
  • orthogonal transform such as discrete cosine transform and motion compensation
  • the present invention can be applied to an encoding device or a decoding device that is used when processing on a storage medium such as an optical, magnetic disk, or flash memory.
  • the present technology can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is jointly processed.
  • each step described in the above flowchart can be executed by one device or can be shared by a plurality of devices.
  • the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
  • a processing unit that performs compensation processing of an image of a first layer of an image having a hierarchical structure using a motion vector;
  • An image comprising: a list registration unit for registering motion vector prediction vector candidates of the first layer image including the motion vector of the second layer image in a list based on the characteristics of the image having the layer structure. Processing equipment.
  • the list registration unit registers the prediction vector candidates in the list based on a feature of the first layer image.
  • the list registration unit registers the prediction vector candidates in the list based on a feature of the second layer image.
  • the list registration unit registers the prediction vector candidates in the list based on characteristics of the images of the first and second hierarchies.
  • the list registration unit determines a motion vector of the second layer image as a motion vector of the first layer image.
  • the list registration unit determines a motion vector of the first layer image from a motion vector of the second layer image.
  • the image processing apparatus according to any one of (1) to (5), wherein the image vector is registered in the list as a candidate for the prediction vector first.
  • the list registration unit registers the prediction vector candidates including the motion vector of the second layer image in the list when the size of a PU (Prediction Unit) of the image having the hierarchical structure is equal to or larger than a predetermined size.
  • the image processing apparatus according to any one of (1) to (4).
  • the list registration unit determines the motion vector of the second layer image as the motion vector of the first layer image.
  • the image processing device according to any one of (1) to (6), wherein the image vector is registered in the list as a candidate for the prediction vector earlier.
  • the list registration unit uses the motion vector of the first layer image based on the motion vector of the second layer image.
  • the image processing apparatus according to any one of (1) to (6) or (8), wherein the image is registered in the list as candidates for the prediction vector first.
  • the list registration unit registers the prediction vector candidates including the motion vector of the second layer image in the list when the size of a CU (Coding Unit) of the image having the hierarchical structure is equal to or larger than a predetermined size.
  • the image processing apparatus according to any one of (1) to (4) or (7).
  • the list registration unit obtains a motion vector of the second layer image as the first layer image.
  • the list registration unit determines the motion vector of the first layer image as the motion vector of the second layer image.
  • the list registration unit selects the prediction vector candidate including the motion vector of the image of the second layer.
  • the list registration unit obtains a motion vector of the second layer image of the first layer.
  • the image processing apparatus wherein the image vector is registered in the list as a candidate for the prediction vector before an image motion vector.
  • the list registration unit obtains a motion vector of the first layer image from the second layer.
  • the image processing apparatus according to (1) or (14), wherein the image vector is registered in the list as a candidate for the prediction vector prior to a motion vector of the image.
  • the list registration unit when a quantization parameter of the first layer image is larger than a quantization parameter of the second layer image, the prediction vector candidate including a motion vector of the second layer image The image processing apparatus according to (1).
  • the list registration unit may include a first region or a second region that is temporally surrounding the block that is the target of the compensation processing in the first layer image based on the characteristics of the image having the hierarchical structure.
  • the motion vector of one of the images is registered in the list as a candidate for the prediction vector, and the motion vector of the region of the second layer image corresponding to the other is registered in the list.
  • the image processing apparatus according to any one of 16).
  • the image processing device according to any one of (1) to (17), further including: a setting unit configured to set identification information for identifying that the motion vector of the second layer image is a candidate for the prediction vector.
  • the image processing device A processing step of performing compensation processing of an image of a first layer of an image having a hierarchical structure using a motion vector; An image including a list registration step of registering, in a list, motion vector prediction vector candidates of the first layer image including a motion vector of the second layer image based on the characteristics of the image having the layer structure. Processing method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un dispositif de traitement d'image et un procédé de traitement d'image adaptés pour améliorer une efficacité de codage quand un vecteur de mouvement d'une image d'une couche de base est employé comme vecteur de prédiction candidat d'un vecteur de mouvement d'une image d'une couche d'amélioration. Un module de traitement utilise un vecteur de mouvement pour exécuter une opération de correction d'une image d'amélioration. Sur la base de caractéristiques d'une image formée à partir de l'image de base et de l'image d'amélioration, un module d'enregistrement sur liste enregistre, dans une liste, des vecteurs de prédiction candidats d'un vecteur de mouvement de l'image d'amélioration, qui comprend le vecteur de mouvement de l'image de base. La présente invention peut-être mise en œuvre, par exemple, dans un dispositif de codage adapté pour exécuter un codage modulable.
PCT/JP2013/083154 2012-12-20 2013-12-11 Dispositif de traitement d'image, et procédé de traitement d'image Ceased WO2014097937A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2012-277663 2012-12-20
JP2012277663 2012-12-20

Publications (1)

Publication Number Publication Date
WO2014097937A1 true WO2014097937A1 (fr) 2014-06-26

Family

ID=50978272

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/083154 Ceased WO2014097937A1 (fr) 2012-12-20 2013-12-11 Dispositif de traitement d'image, et procédé de traitement d'image

Country Status (1)

Country Link
WO (1) WO2014097937A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010507352A (ja) * 2006-10-18 2010-03-04 アップル インコーポレイテッド 下層のフィルタリングを備えたスケーラブルビデオ符号化
WO2011048903A1 (fr) * 2009-10-20 2011-04-28 シャープ株式会社 Dispositif d'encodage vidéo, dispositif de décodage vidéo et structure de données

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010507352A (ja) * 2006-10-18 2010-03-04 アップル インコーポレイテッド 下層のフィルタリングを備えたスケーラブルビデオ符号化
WO2011048903A1 (fr) * 2009-10-20 2011-04-28 シャープ株式会社 Dispositif d'encodage vidéo, dispositif de décodage vidéo et structure de données

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JIE DONG ET AL.: "Description of scalable video coding technology proposal by InterDigital Communications", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP3 AND ISO/IEC JTC1/SC29/WG11 JCTVC-K0034 R1, ITU-T, pages 1 - 30 *
KIRAN MISRA ET AL.: "Description of scalable video coding technology proposal by Sharp (proposal 1", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP3 AND ISO/IEC JTC1/SC29/WG11 JCTVC-K0031, ITU-T, pages 1 - 35 *

Similar Documents

Publication Publication Date Title
JP6670812B2 (ja) 符号化装置および符号化方法
JP6296180B2 (ja) 画像処理装置および方法、記録媒体、並びに、プログラム
CA2874723C (fr) Dispositif et procede de traitement d'image
US10499079B2 (en) Encoding device, encoding method, decoding device, and decoding method
US20150139303A1 (en) Encoding device, encoding method, decoding device, and decoding method
US20150043637A1 (en) Image processing device and method
JP6287035B2 (ja) 復号装置および復号方法
WO2014002898A1 (fr) Dispositif et procédé de décodage
WO2015005367A1 (fr) Dispositif et procédé de décodage d'image
JP6497562B2 (ja) 画像符号化装置および方法
JP2015005899A (ja) 復号装置および復号方法、並びに、符号化装置および符号化方法
JPWO2015098561A1 (ja) 復号装置および復号方法、並びに、符号化装置および符号化方法
US20140205007A1 (en) Image processing devices and methods
JP2015076861A (ja) 復号装置および復号方法、並びに、符号化装置および符号化方法
JPWO2013108688A1 (ja) 画像処理装置および方法
US20150304678A1 (en) Image processing device and method
US20150208097A1 (en) Coding apparatus, coding method, decoding apparatus, and decoding method
WO2014156707A1 (fr) Dispositif et procédé de codage d'image et dispositif et procédé de décodage d'image
WO2014103764A1 (fr) Dispositif et procédé de traitement d'image
JP6477930B2 (ja) 符号化装置および符号化方法
JPWO2014002900A1 (ja) 画像処理装置および画像処理方法
WO2014141899A1 (fr) Dispositif et procédé de traitement d'image
WO2014097937A1 (fr) Dispositif de traitement d'image, et procédé de traitement d'image
JP2015050738A (ja) 復号装置および復号方法、並びに、符号化装置および符号化方法
WO2014203762A1 (fr) Dispositif de décodage, procédé de décodage, dispositif de codage, et procédé de codage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13864017

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13864017

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP