JP2009278609A - Information processing device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To transmit image data with high quality and low delay. <P>SOLUTION: A wavelet transform section 101 applies wavelet transform to image data using a reversible filter that performs data transform with a reversible method that completely ensures forward direction and backward direction transform. An entropy coding section 103 encodes coefficient data using a predetermined entropy coding scheme, by a reversible method that completely ensures forward direction and backward direction transform. The present invention is applicable to, for example, an encoding apparatus or a decoding apparatus. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method capable of transmitting image data with high quality and low delay.

  Conventionally, it has been required to transmit video and audio data without delay in a communication system using video and audio such as an interactive video conference system, and in a mixed system with an uncompressed environment in a broadcasting station or the like. ing. In recent years, in particular, the amount of data has increased along with the improvement in quality of video and audio, and image compression at the time of transmission has become indispensable. There is a demand for data transmission with low delay.

  For example, MPEG (Moving Picture Experts Group) and H.264. In the 26x compression method, the compression rate is increased by motion prediction. However, if motion prediction is performed, the algorithm becomes complicated, and the processing time increases in proportion to the square of the frame size. Encoding delay occurs. When bi-directional real-time communication is performed, the delay time is a marginal delay time of 250 ms, which cannot be ignored.

  In addition, since an intra-frame codec such as JPEG (Joint Photographic Experts Group) 2000 does not use difference information between frames, the above-described delay does not occur. However, since compression is performed in units of frames, it is necessary to wait at least one frame until encoding starts. In the current general system, there are many 30 frames per second, so a waiting time of about 16 ms is required before starting encoding.

  In addition, with the improvement of the quality of video and audio, there is a demand for higher quality of image compression. JPEG2000 employs a compression coding scheme that combines bit modeling for each bit plane and high-efficiency entropy coding by arithmetic coding in wavelet transform. For example, Patent Document 1 describes a wavelet transform method that further improves encoding efficiency.

JP-A-9-130800

  However, it has been difficult for the conventional method to transmit image data with high quality and low delay.

  The present invention has been proposed in view of such conventional circumstances, and enables image data to be encoded by a reversible method capable of restoring the original data and transmitted with low delay. It is.

  According to one aspect of the present invention, image data is decomposed for each frequency band by a reversible method capable of restoring the original image data from the converted coefficient data by reverse conversion, and the coefficient data for each frequency band is used. Analysis filter means for performing analysis filter processing for generating subbands for each line block including the image data for the number of lines necessary to generate coefficient data for one line of at least the subband of the lowest frequency component; The coefficient data generated by the analysis filter processing by the analysis filter means can be restored to the original coefficient data from the encoded data by decoding processing for each coefficient data group generated from a plurality of the line blocks. It is an information processing apparatus provided with the encoding means to encode by a reversible method.

  The encoding including information indicating that at least the encoded data is generated by encoding in a reversible method for each predetermined data unit with respect to the encoded data obtained by encoding by the encoding means Generation means for generating information about the data can be further provided.

  The generation means can also generate information on the type of the analysis filter process executed by the analysis filter means as information on the encoded data.

  The generating means can also generate information relating to the number of line blocks necessary for the coding means to generate the coefficient data group that is the processing unit of the coding as information relating to the coded data. .

  The coefficient data generated by the analysis filter processing by the analysis filter means is combined with the coefficient data of each subband for each line block in order of executing synthesis filter processing for restoring the original image data. Rearrangement means for rearranging further, wherein the encoding means encodes the coefficient data in the order rearranged by the rearrangement means for each coefficient data group generated from a plurality of the line blocks. Can do.

  For the encoded data obtained by encoding by the encoding means, at least information indicating that the encoded data has been generated by encoding in a reversible method for each predetermined data unit, and the arrangement Generation means for generating information related to the encoded data including information related to the rearrangement of the coefficient data by the replacement means may be further provided.

  The analysis filter means further decomposes the image data for each frequency band by an irreversible method in which restoration of the original image data by reverse conversion is not guaranteed, and from the coefficient data for each frequency band The analysis filter processing for generating a subband is performed for each line block including the image data for the number of lines necessary to generate coefficient data for one line of at least the subband of the lowest frequency component, and the analysis filter Control means for controlling whether the analysis filter processing by the means is performed by a reversible method or an irreversible method can be further provided.

  According to another aspect of the present invention, the analysis filter unit decomposes the image data for each frequency band by a reversible method capable of restoring the original image data from the coefficient data after conversion by reverse conversion, and the frequency For each line block including the image data corresponding to the number of lines necessary to generate at least one line of coefficient data for the subband of the lowest band component in the analysis filter processing for generating subbands including coefficient data for each band And the encoding means converts the coefficient data generated by the analysis filter processing from the encoded data by decoding processing to the original coefficient data for each of the coefficient data groups generated from the plurality of line blocks. This is an information processing method for encoding by a reversible and reversible method.

  According to another aspect of the present invention, information related to encoded data obtained by encoding image data is analyzed, and the encoded data is encoded by a reversible method capable of restoring data before encoding by decoding processing. When it is determined that the encoded data is generated by the encoding process of the reversible method, as a result of the analysis by the analyzing unit that identifies whether or not the generated data is analyzed, Decoding means for performing decoding processing on the encoded data by a method corresponding to the encoding processing, and coefficient data generated from the encoded data by decoding processing by the decoding means An information processing apparatus including synthesis filter means for generating image data.

  Another aspect of the present invention is also a reversible method in which the analysis unit analyzes information related to encoded data obtained by encoding image data, and the encoded data can restore data before encoding by decoding processing. The decoding unit determines whether the encoded data is generated by the reversible encoding method as a result of analysis. A decoding process is performed on the encoded data by a method corresponding to the encoding process, and a synthesis filter unit synthesizes the coefficient data generated from the encoded data by the decoding process, and This is an information processing method for generating image data.

  In one aspect of the present invention, image data is decomposed for each frequency band by a reversible method capable of restoring the original image data from the converted coefficient data by reverse conversion, and consists of coefficient data for each frequency band. The analysis filter processing for generating the subband is performed for each line block including the image data for the number of lines necessary to generate the coefficient data for one line of at least the subband of the lowest band component. The coefficient data generated by the above is encoded by a reversible method capable of restoring the original coefficient data from the encoded data by decoding processing for each coefficient data group generated from a plurality of line blocks.

  In another aspect of the present invention, information on encoded data obtained by encoding image data is analyzed, and the encoded data is subjected to a reversible encoding process that can restore the data before encoding by the decoding process. If it is determined whether or not the encoded data is generated by a reversible encoding process, the encoded data is encoded. A decoding process is performed by a method corresponding to the process, and coefficient data generated from the encoded data by the decoding process is combined to generate image data.

  According to the present invention, image data can be transmitted. In particular, image data can be transmitted with high quality and low delay.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a block diagram showing a configuration example of an encoding apparatus to which the present invention is applied.

  In FIG. 1, an encoding apparatus 100 is an apparatus that encodes image data. The encoding apparatus 100 decodes the encoded data obtained by encoding the baseband image data so as to restore the baseband image data before encoding. That is, the encoding apparatus 100 encodes the image data by a reversible method in which conversion between the forward direction and the reverse direction is completely guaranteed. Thereby, the encoding apparatus 100 can encode image data so that high-quality image data can be obtained even after decoding. The encoding apparatus 100 encodes image data in units of precincts described later. Thereby, the encoding apparatus 100 can reduce the delay time until the image data is encoded and decoded.

  The encoding apparatus 100 includes a wavelet transform unit 101, a midway calculation buffer unit 102, and an entropy encoding unit 103.

  The image data input to the encoding device 100 is temporarily stored in the midway calculation buffer unit 102 via the wavelet transform unit 101. The wavelet transform unit 101 performs wavelet transform on the image data stored in the midway calculation buffer unit 102 using a reversible filter that performs data transformation in a reversible manner in which forward and reverse transformations are completely guaranteed. Apply. The wavelet transform unit 101 reads out the image data from the midway calculation buffer unit 102, performs filter processing using a reversible analysis filter, generates low-frequency component and high-frequency component coefficient data, and generates the generated coefficient data in the middle The data is stored in the calculation buffer unit 102.

  The wavelet transform unit 101 includes a horizontal analysis filter and a vertical analysis filter, and performs reversible analysis filter processing on the image data group in both the screen horizontal direction and the screen vertical direction. The wavelet transform unit 101 reads the low-frequency component coefficient data stored in the midway calculation buffer unit 102 again, performs a filtering process using an analysis filter on the read coefficient data, and outputs the high-frequency component and the low-frequency component. Further generate coefficient data. The generated coefficient data is stored in the midway calculation buffer unit 102. That is, the wavelet transform unit 101 recursively repeats the filtering process for separating the input data into the low frequency component and the high frequency component a predetermined number of times for the low frequency component.

  When the decomposition level reaches a predetermined level by repeating this process, the wavelet transform unit 101 reads coefficient data from the midway calculation buffer unit 102 and writes the read coefficient data to the entropy encoding unit 103.

  The wavelet transform unit 101 and the midway calculation buffer unit 102 are also collectively referred to as an analysis filter processing unit 111.

  The entropy encoding unit 103 uses the predetermined entropy encoding method such as Huffman encoding or arithmetic encoding for the coefficient data supplied from the wavelet transform unit 101 (analysis filter processing unit 111), for example, in the forward and reverse directions. Is encoded in a reversible manner with a completely guaranteed conversion. That is, the entropy encoding unit 103 performs encoding so that the baseband image data before encoding can be restored by decoding the encoded data obtained by encoding the baseband image data.

  At this time, the entropy encoding unit 103 improves encoding efficiency by performing encoding processing using a larger amount of information. In particular, when encoding processing is performed using a reversible method, higher quality and higher efficiency of image compression are often prioritized over delay time. For this purpose, the entropy encoding unit 103 waits until a predetermined number of precinct coefficient data equal to or greater than “2 precincts” is obtained, and entropy encodes the coefficient data for each precinct number. . By performing encoding using coefficient data for a plurality of precincts as described above, the entropy encoding unit 103 compresses an image with higher efficiency and higher quality than when encoding is performed for each precinct. be able to.

  The entropy encoding unit 103 outputs the generated encoded data to the outside of the encoding apparatus 100.

  Next, processing performed by the wavelet transform unit 101 in FIG. 1 will be described in more detail. First, data input to the wavelet transform unit 101 will be described.

  In the wavelet transform unit 101, in order to perform the analysis filter processing reversibly, for example, an integer type 5 × 3 filter, which will be described later, that performs calculation with integer precision is used. The wavelet transform unit 101 uses up to the least significant bit of the input data as an integer part.

  FIG. 2 is a schematic diagram illustrating an example of input data of the wavelet transform unit 101. In FIG. 2, each square represents each bit of input data, the left end in the figure indicates the most significant bit, and the right end in the figure indicates the least significant bit.

  A point P shown in FIG. 2 indicates the position of the decimal point. In FIG. 2, this point P exists at the right end (lowest side). In this integer precision calculation, one bit on the most significant side (left side in the figure) is used as a bit indicating a sign (+/−).

  Next, the wavelet transform will be schematically described. FIG. 3 is a diagram schematically illustrating a configuration example of a wavelet transform operation on image data. The example of FIG. 3 shows a configuration example in which octave division, which is the most general wavelet transform among several methods, is performed over a plurality of levels. In the case of FIG. 3, the number of levels is 3 (level 1 to level 3), the image signal is divided into a low frequency and a high frequency, and only the low frequency components are hierarchically divided. Yes. FIG. 3 illustrates wavelet transform for a one-dimensional signal (for example, a horizontal component of an image) for convenience, but this is expanded to two dimensions (for example, applied to a vertical component of an image). It can correspond to a two-dimensional image signal.

The input image signal 140 to the wavelet transform unit shown in FIG. 3 is obtained by first performing band division by the low-pass filter 131 (transfer function H ₀ (z)) and the high-pass filter 132 (transfer function H ₁ (z)). The low-frequency component and the high-frequency component are thinned out by a half (level 1) respectively by the corresponding down-sampler 133a and down-sampler 133b. There are two outputs at this time: an L component 141 and an H component 146. Here, L is low and indicates a low frequency, and H is high and indicates a high frequency. The low-pass filter 131, the high-pass filter 132, the down sampler 133a, and the down sampler 133b shown in FIG.

  Only the low-frequency components of the signals thinned out by the down-sampler 133a and the down-sampler 133b described above, that is, only the signal from the down-sampler 133a, are further bandpassed by the low-pass filter and the high-pass filter of the level 2 circuit unit 122. The resolution is divided and the resolution is thinned by a factor of two by the corresponding downsampler (level 2). The configuration of the circuit unit 122 including these level 2 low-pass filters, high-pass filters, and down-samplers is the same as that of the circuit unit 121 including the level 1 low-pass filter 131, high-pass filter 132, down-sampler 133a, and down-sampler 133b described above. It is.

  By performing such processing to a predetermined level, band components obtained by hierarchically dividing the low frequency component into bands are sequentially generated. The band components generated at level 2 are the LL component 142 and the LH component 145. FIG. 3 shows an example of band division up to level 3, and the output from the downsampler on the low-pass filter side of the level 2 circuit unit 122 is the level 3 circuit having the same configuration as the circuit unit 121 described above. Supplied to the unit 123. As a result of the band division to level 3, the LLL component 143, the LLH component 144, the LH component 145, and the H component 146 are generated.

  As schematically shown in FIG. 4, by the wavelet transform as described above, the process of dividing the image data into a high spatial frequency band and a low spatial frequency is performed on the low spatial frequency band data obtained as a result of the division. It is recursively repeated. The encoding apparatus 100 can perform efficient compression encoding in the entropy encoding unit 103 by driving data in a low spatial frequency band into a smaller region in the analysis filter processing unit 111 in this way. Like that.

  In FIG. 4, the division processing into the low-frequency component region L and the high-frequency component region H with respect to the lowest-frequency component region of the image data is repeated three times to obtain a division level = 3 indicating the total number of division layers. This is an example. In FIG. 4, “L” and “H” represent a low-frequency component and a high-frequency component, respectively, and the order of “L” and “H” is a band obtained by dividing the front side in the horizontal direction (horizontal direction). In the figure, the rear side shows a band obtained as a result of division in the vertical direction (vertical direction). The numbers before “L” and “H” indicate the hierarchy of the area, and the lower-frequency component hierarchy is represented by a smaller value. The maximum value of this hierarchy indicates the current division level (number of divisions) of the wavelet transform.

  That is, “0LL”, “1HL”, “1LH”, and “1HH” in FIG. 4 are subbands obtained by the level 3 analysis filter processing (2LL is divided) in FIG. “2HL”, “2LH”, and “2HH” are subbands obtained by the level 2 analysis filter processing of FIG. 3 (1LL is divided), and in FIG. 4, “3HL”, “3LH”, And “3HH” are subbands obtained by analyzing the level 1 analysis filter of FIG. 3 (by dividing the baseband image data).

  That is, in the example of FIG. 4, “3HH” is the subband with the lowest low frequency component (the highest frequency component is included most), and “0LL” is the subband with the highest low frequency component.

  The reason why the low-frequency component is repeatedly converted and divided is that the energy of the image is concentrated on the low-frequency component. This is because, as the division level is advanced from the division level = 1 example shown in FIG. 5A to the division level = 3 example shown in FIG. 5B, FIG. It is understood from the fact that subbands are formed as shown in FIG. In the example of FIG. 4, the division level of the wavelet transform is 3, and as a result, 10 subbands are formed.

  The wavelet transform unit 51 normally performs the above-described processing using a filter bank composed of a low-pass filter and a high-pass filter. Since a digital filter usually has an impulse response having a plurality of taps, that is, a filter coefficient, it is necessary to buffer in advance input image data or coefficient data that can be filtered. Similarly, when wavelet transform is performed in multiple stages, it is necessary to buffer the wavelet transform coefficients generated in the previous stage as many times as can be filtered.

A reversible integer type wavelet transform operation by an integer precision type 5 × 3 analysis filter defined by FPEG (Final Committee Draft) of JPEG-2000 Part-1 will be described. FIG. 6 is a diagram for explaining the operation of an integer precision type 5 × 3 analysis filter (hereinafter also referred to as an integer type 5 × 3 analysis filter), in which one-dimensional analysis filter processing is performed once. An operation of converting the leftmost input image data into a low-frequency component s and a high-frequency component d is shown. Here, it is assumed that d _m ⁿ is the m-th high-frequency component coefficient whose wavelet transform level is n. Similarly, ^let s _m ⁿ be the m-th low-frequency component coefficient whose wavelet transform level is n. When n = 0, as shown in FIG. 6, it is the input image itself.

  As shown in FIG. 6, the formula for calculating the low frequency component coefficient s can be generally expressed as the following formula (1).

s _{m + 1} ^{n + 1} = s _{m + 1} ⁿ + ((d _m ^{n + 1} + d _{m + 1} ^{n + 1} +2) / 4) (1)

  Here, the ¼ operation in the equation can be realized by a bit shift, and when the 2-bit right shift is expressed as “>> 2”, the above equation (1) is expressed as the following equation (1 ′): Can be expressed as

s _{m + 1} ^{n + 1} = s _{m + 1} ⁿ + ((d _m ^{n + 1} + d _{m + 1} ^{n + 1} +2) >> 2) (1 ′)

  Note that +2 in parentheses in the above formula (1) is for compensating for a rounding error that occurs during a right shift.

  Further, the calculation formula of the high frequency component coefficient d can be generally expressed as the following formula (2).

d _m ^{n + 1} = d _m ⁿ − ((s _m ⁿ + s _{m + 1} ⁿ ) / 2) (2)

  Alternatively, 1-bit right shift can be expressed by “>> 1” and expressed as the following expression (2 ′).

d _m ^{n + 1} = d _m ⁿ − ((s _m ⁿ + s _{m + 1} ⁿ ) >> 1) (2 ′)

Here, in the example of FIG. 6, the first-stage wavelet transform coefficient, that is, the low-frequency component coefficient and the high-frequency component coefficient of level n = 1 are calculated for the input image data. s and d do not represent a low-frequency component coefficient or a high-frequency component coefficient. Further, at a position corresponding to the edge of the screen, data corresponding to the outside of the screen is used for returning data outside the screen. For example, in order to obtain the data s ₀ ¹ in FIG. 6, a s ₀ ^0, and d ₀ ^1, further using the same d ₀ ¹ as wrapping d ₀ ^1. Similarly, two s ₇ ⁰ are used to obtain d ₇ ¹ .

A portion SP surrounded by a broken line in FIG. 6 is a portion for obtaining a low-frequency component coefficient s ₂ ¹ corresponding to n = 0 and m = 1 in the above equation (1), and in the following equation (3): Such a calculation is performed.

s ₂ ¹ = s ₂ ⁰ + ((d ₁ ¹ + d ₂ ¹ +2) / 4) (3)

Further, a part DP surrounded by a broken line in FIG. 6 is a part for obtaining a high frequency component coefficient d ₀ ¹ corresponding to n = 0 and m = 0 in the above formula (2). ) Is calculated.

d ₀ ¹ = d ₀ ⁰ − ((s ₀ ⁰ + s ₁ ⁰ ) / 2) (4)

  Here, as described above, ¼ multiplication can be realized by 2-bit right shift “>> 2” and ½ multiplication can be realized by 1-bit right shift “>> 1”.

  The amount of calculation required to calculate the above-described equations (1) and (2) is as follows: multiplication “0”, addition / subtraction “5”, and shift operation “2”, assuming that multiplication is realized by shift operation as described above. It becomes. By performing this operation in the same manner from top to bottom, all the coefficients can be calculated. As described above, for a two-dimensional signal such as an image, a coefficient group generated by a wavelet transform in a one-dimensional direction (for example, the vertical direction), in another direction (for example, a horizontal direction), The wavelet transform may be performed as described above.

  In practice, the above-described filter processing calculation can be reduced by using a lifting technique. This lifting will be described. FIG. 7 shows an example in which one-dimensional filter processing by lifting of an integer type 5 × 3 filter (analysis filter and synthesis filter) is performed up to decomposition level = 2. In FIG. 7, the part shown as the analysis filter on the left side of the figure is the filter of the wavelet transform unit 101 in FIG. In addition, a portion shown as a synthesis filter on the right side of the figure is a filter of a wavelet inverse transform unit described later.

  In FIG. 7, the leftmost column indicates continuous pixel data on one line (or one column) of original image data (that is, image data input to the encoding device 100) that is baseband image data. Yes. The first through third columns from the left end in FIG. 7 show the filter processing at the division level = 1, and the fourth through sixth columns show the filter processing at the division level = 2.

  In the analysis filter processing at the decomposition level = 1, high-frequency component coefficient data is calculated based on the original image data as the first-stage analysis filter processing, and the first-stage analysis filter processing is performed as the second-stage analysis filter processing. The coefficient data of the low frequency component is calculated based on the coefficient data of the high frequency component calculated in step 1 and the original image data. In FIG. 7, the second column from the left end indicates the high frequency component output, and the third column from the left end indicates the low frequency component output.

  The calculated coefficient data of the high frequency component is supplied to the entropy encoding unit 103 in FIG. The calculated low frequency component coefficient data is stored in the midway calculation buffer unit 102 in FIG. In FIG. 7, the entropy encoding unit 103 is shown as a part surrounded by a one-dot chain line, and the midway calculation buffer unit 102 is shown as a part surrounded by a dotted line.

  Based on the analysis filter processing result of decomposition level = 1 held in the midway calculation buffer unit 102, analysis filter processing of decomposition level = 2 is performed. In the filter processing with the decomposition level = 2, the coefficient data calculated as the low-frequency component coefficient in the filter processing with the decomposition level = 1 is regarded as the coefficient data including the low-frequency component and the high-frequency component, and the decomposition level is determined. Filter processing similar to that for = 1 is performed. The high frequency component coefficient data and the low frequency component coefficient data calculated by the filter processing at the decomposition level = 2 are supplied to the entropy encoding unit 103 in FIG.

  The wavelet transform unit 101 in FIG. 1 performs the filter processing as described above in the horizontal and vertical directions of the screen. For example, the wavelet transform unit 101 first performs a filtering process of decomposition level = 1 in the horizontal direction, and stores the generated high frequency component and low frequency component coefficient data in the midway calculation buffer unit 102. Next, the wavelet transform unit 101 performs a filter process with the decomposition level = 1 in the vertical direction on the coefficient data stored in the midway calculation buffer unit 102. By processing in the horizontal and vertical directions with this decomposition level = 1, the sub-band HH and sub-band HL by the coefficient data obtained by further decomposing the high-frequency component into the high-frequency component and the low-frequency component, and the low-frequency component further higher Four subbands of a subband LH and a subband LL are formed by the coefficient data decomposed into the component and the low-frequency component, respectively.

  At the decomposition level = 2, the analysis filter processing is further performed on the coefficient data of the low frequency component generated at the decomposition level = 1 for each of the horizontal direction and the vertical direction. That is, by the analysis filter processing at the decomposition level = 2, the subband LL divided and formed at the decomposition level = 1 is further divided into four to form the subband HH, subband HL, subband LH, and subband LL. The

  In the vertical direction of the screen, the wavelet transform unit 101 divides the reversible analysis filter processing by the wavelet transform as described above into processes for every several lines and performs them step by step in a plurality of times. . In the example of FIG. 7, the first process that is the process from the first line on the screen performs the filter process for 7 lines, and the second and subsequent processes that are the processes from the 8th line are performed every 4 lines. Filter processing is performed. The number of lines at each time is based on the number of lines necessary for generating the lowest band component for one line by the analysis filter processing described above.

  In the following, a collection of lines including other subbands necessary for generating one line of the lowest frequency component (coefficient data for one line of the subband of the lowest frequency component) is precinct ( Or a line block). Here, the line indicates a picture (frame or field) corresponding to image data before wavelet transform, or pixel data or coefficient data for one row formed in each subband. That is, the precinct (line block) is a pixel data group corresponding to the number of lines necessary for generating coefficient data for one subband of the lowest frequency component after wavelet transformation in the original image data before wavelet transformation. Or the coefficient data group of each subband obtained by wavelet transforming the pixel data group.

  According to FIG. 7, the coefficient C5 obtained as a result of the filter processing of decomposition level = 2 is calculated based on the coefficient C4 and the coefficient Ca stored in the midway calculation buffer unit 102, and the coefficient C4 is calculated based on the midway calculation buffer unit. The coefficient Ca, coefficient Cb, and coefficient Cc stored in 102 are calculated. Further, the coefficient Cc is calculated based on the coefficient C2 and the coefficient C3 supplied to the entropy encoding unit 103 and the pixel data of the fifth line. The coefficient C3 is calculated based on the pixel data of the fifth line to the seventh line. As described above, in order to obtain the low-frequency component coefficient C5 at the division level = 2, the pixel data of the first line to the seventh line are required.

  On the other hand, in the second and subsequent filter processing, the coefficient data already calculated in the previous filter processing and supplied to the entropy encoding unit 103 (actually the same as the supplied coefficient data is used as the wavelet transform unit 101). (Or held by the midway calculation buffer unit 102) can be used, and the number of necessary lines can be reduced.

  That is, according to FIG. 7, the coefficient C9, which is the coefficient next to the coefficient C5 among the coefficients of the low-frequency component obtained from the filter processing result of the decomposition level = 2, is the coefficient C4, the coefficient C8, and the intermediate calculation It is calculated based on the coefficient Cc stored in the buffer unit 102. The coefficient C4 is already calculated by the first filtering process described above, and is supplied to the entropy encoding unit 103 (actually, the same data as the supplied coefficient data is the wavelet transform unit 101 (or intermediate calculation buffer unit 102). Is holding). Similarly, the coefficient Cc has already been calculated by the first filtering process described above, and is stored in the midway calculation buffer unit 102. Therefore, in the second filtering process, only the filtering process for calculating the coefficient C8 is newly performed. This new filtering process is performed by further using the eighth to eleventh lines.

  In this way, the second and subsequent filtering processes are calculated by the previous filtering process and supplied to the midway calculation buffer unit 102 and the entropy encoding unit 103 (actually, the wavelet transform unit 101 (or the midway calculation Since the data held in the buffer unit 102) can be used, processing for each four lines is sufficient.

  If the number of lines on the screen does not match the number of lines to be encoded, the original image data lines are copied by a predetermined method, and the number of lines is matched with the number of lines to be encoded.

  In this way, the wavelet transform unit 101 divides the filtering process to obtain coefficient data for one line of the lowest frequency component into a plurality of times step by step for the entire screen (precinct (line block) unit). By doing so, it is possible to obtain a decoded image with low delay when transmitting encoded data.

  As already described, in the wavelet transform, coefficients are generated from the high frequency component side to the low frequency component side. In the example of FIG. 7, at the first time, the high-frequency component coefficient C1, coefficient C2, and coefficient C3 are sequentially generated from the pixel data of the original image by the filter processing of decomposition level = 1. Then, the filter processing of decomposition level = 2 is performed on the coefficient data of the low frequency component obtained by the filter processing of decomposition level = 1, and the low frequency component coefficient C4 and the coefficient C5 are sequentially generated. That is, in the first time, coefficient data is generated in the order of coefficient C1, coefficient C2, coefficient C3, coefficient C4, and coefficient C5. The generation order of the coefficient data is always in this order (order from high to low) on the principle of wavelet transform.

  On the other hand, the decoding process is performed first from the low frequency component. This will be described more specifically with reference to the example of FIG. The right side of FIG. 7 shows an example of synthesis filter processing for performing wavelet inverse transformation, which is the inverse transformation of the above-described wavelet transformation (analysis filter processing). The first synthesis filter process (wavelet inverse transform process) including the first line of the output image data on the decoding side is performed by the coefficient C4 and the coefficient C5 of the lowest frequency component generated by the first filter process on the encoding side. And the coefficient C1.

  That is, in the first synthesis process, the synthesis filter process is performed on the coefficient C5 and the coefficient C4 in the synthesis level = 2 process that is the synthesis process corresponding to the decomposition level = 2, and the coefficient Cf is generated. The coefficient Cf is stored in the buffer. A synthesis filter process is performed on the coefficient Cf and the coefficient C1 in a process of synthesis level = 1, which is a synthesis process corresponding to the decomposition level = 1, and a first line of baseband image data is generated. .

  As described above, the first synthesis filter processing is performed by converting coefficient data generated in the order of the coefficient C1, the coefficient C2, the coefficient C3, the coefficient C4, and the coefficient C5 by the analysis filter processing on the encoding side into the coefficient C5, the coefficient C4, Processing is performed in the order of the coefficients C1,.

  On the synthesis filter side shown on the right side of FIG. 7, for the coefficients supplied from the coding side, the line order on the analysis filter side is shown in parentheses, and the line order on the synthesis filter side is shown outside the parentheses. For example, the coefficient C1 (5) indicates that it is the fifth line on the left analysis filter side in FIG. 7 and the first line on the synthesis filter side.

  The synthesis filter process for the coefficient data generated in the second and subsequent analysis filter processes uses coefficient data and image data synthesized in the previous synthesis filter process in addition to the coefficient data supplied from the analysis filter side. Done. In the case of the example in FIG. 7, the second synthesis filter process is performed by using the coefficient data supplied from the analysis filter side as well as the low-frequency component coefficient C8 and coefficient C9 generated by the second analysis filter process. This is performed using the coefficient C2 and the coefficient C3 generated by the second analysis filter process. Through the second synthesis filter processing, the second to fifth lines of the baseband image data are decoded.

  That is, in the second combining process, the coefficient C9, the coefficient C8, the coefficient C2, and the coefficient C3 are processed in this order. That is, in the synthesis filter process of synthesis level = 2, the coefficient Cg is generated using the coefficients C8 and C9 and the coefficient C4 supplied from the encoding side in the first synthesis filter process, and stored in the buffer. To do. In addition, the synthesis filter process of synthesis level = 2 generates a coefficient Ch using the coefficient Cg, the above-described coefficient C4, and the coefficient Cf generated by the first synthesis filter process and stored in the buffer, To store.

  Then, the synthesis filter process of synthesis level = 1 includes the coefficient Cg and coefficient Ch generated by the synthesis filter process of synthesis level = 2 and stored in the buffer, and the coefficient C2 supplied from the analysis filter side (coefficient C2 on the synthesis filter side). C6 (2)) and coefficient C3 (denoted as coefficient C7 (3) on the synthesis filter side) are used to decode the second to fifth lines of baseband image data.

  In this way, in the second synthesis filter process, coefficient data generated in the order of coefficient C2, coefficient C3, (coefficient C4, coefficient C5), coefficient C6, coefficient C7, coefficient C8, and coefficient C9 on the analysis filter side. Are processed in the order of coefficient C9, coefficient C8, coefficient C2, coefficient C3,.

  Similarly, in the third and subsequent synthesis filter processing, the coefficient data generated by the analysis filter processing is rearranged in a predetermined order, and the lines of the baseband image data are decoded by four lines. .

  In the synthesis filter processing corresponding to the analysis filter processing including the line at the lower end of the screen (hereinafter referred to as the last round), all the coefficient data generated in the previous processing and stored in the buffer are output. As a result, the number of output lines increases. In the example of FIG. 7, 8 lines are output in the last round.

  The processing up to the above will be described more specifically with reference to FIG. FIG. 8 is an example in the case of performing filter processing by wavelet transform up to decomposition level = 2 using an integer type 5 × 3 analysis filter. In the wavelet transform unit 101, as shown in FIG. 8A, the first analysis filter processing is performed in the horizontal and vertical directions for the first to seventh lines of the input image data, respectively (FIG. 8). In-1 of A).

  The coefficient data that corresponds to the coefficient C1, the coefficient C2, and the coefficient C3 described above in at least one of the processes in the horizontal and vertical directions with the decomposition level = 1 of the first analysis filter process is shown in FIG. As shown in FIG. 8B, three lines are generated for each of subband HH, subband HL, and subband LH formed at decomposition level = 1 (WT-1 in FIG. 8B). .

  In addition, the subband LL formed by coefficient data that is a low-frequency component in both the horizontal and vertical processing at the decomposition level = 1 of the first analysis filter processing is in the horizontal and vertical directions according to the decomposition level = 2. Is further divided into four by the division filter processing. Coefficient data corresponding to the coefficient C5 in both processes is generated for one line of the subband LL in the subband LL by the horizontal and vertical processes at the decomposition level = 2 of the first analysis filter process. Is done. In addition, the coefficient data that is assumed to correspond to the coefficient C4 in at least one of the processes in the horizontal and vertical directions at the decomposition level = 2 of the first analysis filter process is subband HH in subband LL, One line is generated for each of the subband HL and the subband LH.

  In the second and subsequent filter processing by the wavelet transform unit 51, filter processing is performed every four lines (In-2 in FIG. 8A), and for each subband, two lines at a decomposition level = 1. Coefficient data is generated (WT-2 in FIG. 8B), and coefficient data for each line is generated at the decomposition level = 2.

  When decoding the wavelet-transformed coefficient data as shown in FIG. 8B, as shown in FIG. 8C, the first analysis filter processing by the first line to the seventh line is performed for the first time. The first line is output by the synthesis filter processing (Out-1 in FIG. 8C). Thereafter, image data is output line by line (Out-2... In FIG. 8C) by the synthesis filter processing for the analysis filter processing from the second time to the last previous time. Then, 8-line image data is output by the synthesis filter processing for the last analysis filter processing.

  The coefficient data generated by the wavelet transform unit 101 from the high frequency component side to the low frequency component side is sequentially supplied to the entropy encoding unit 103.

  The entropy encoding unit 103 sequentially encodes the supplied coefficient data using a reversible method, and outputs the generated encoded data to the outside of the encoding apparatus 100.

  FIG. 9 is a block diagram illustrating a configuration example of a decoding apparatus corresponding to the encoding apparatus 100 of FIG. A decoding apparatus 200 shown in FIG. 9 decodes encoded data obtained by encoding image data by the encoding apparatus 100 by a method corresponding to the encoding apparatus 100 of FIG. Restore band image data. That is, the decoding device 200 decodes the image data by a reversible method in which the forward and reverse conversions are completely guaranteed. Thereby, the decoding apparatus 200 can decode the encoded data obtained by the encoding apparatus 100 so that high-quality image data can be obtained. Also, the decoding device 200 decodes the encoded data obtained by encoding the image data in units of precincts by the encoding device 100 for each precinct. Thereby, the decoding apparatus 200 can reduce the delay time until the image data is encoded and decoded.

  The decoding apparatus 200 includes an entropy decoding unit 201, a coefficient rearranging unit 202, a coefficient buffer unit 203, and a wavelet inverse transform unit 204.

  The entropy decoding unit 201 decodes the supplied encoded data by a reversible decoding method corresponding to the encoding method by the entropy encoding unit 103 (FIG. 1) to obtain coefficient data. The coefficient data is supplied to the coefficient rearranging unit 202. The coefficient rearranging unit 202 rearranges the supplied coefficient data, that is, coefficient data supplied in the order from the high frequency component to the low frequency component in the order from the low frequency component to the high frequency component. The coefficient rearranging unit 202 stores the rearranged coefficient data in the coefficient buffer unit 203. The wavelet inverse transformation unit 204 performs synthesis filter processing (wavelet inverse transformation) using the coefficient data stored in the coefficient buffer unit 203, and stores the result of the synthesis filter processing in the coefficient buffer unit 203 again. The wavelet inverse transform unit 204 repeats this process according to the decomposition level to obtain baseband image data. This baseband image data is equivalent to the baseband image data input to the encoding device 100 of FIG.

  That is, the decoding apparatus 200 can restore the baseband image data before encoding by decoding the encoded data obtained by encoding the baseband image data by the encoding apparatus 100. When the baseband image data is obtained, the wavelet inverse transform unit 204 outputs it to the outside of the decoding device 200.

  The wavelet inverse transform unit 204 includes a horizontal synthesis filter and a vertical synthesis filter, and performs reversible filter processing on the coefficient data group in both the screen horizontal direction and the screen vertical direction. The wavelet inverse transform unit 204 reads the low-frequency component coefficient data and the high-frequency component coefficient data stored in the coefficient buffer unit 203, performs synthesis filter processing, and performs low-frequency component coefficient data of the next lower level. Is generated. The generated coefficient data is stored in the coefficient buffer unit 203. That is, the wavelet inverse transform unit 204 recursively performs a filter process for synthesizing a low-frequency component and a high-frequency component, which are input data, to generate a low-frequency component of the next lower level until the level 1 processing is completed. repeat.

  When the wavelet inverse transform unit 204 repeats this processing and finishes the synthesis filter processing at level 1 (when baseband image data is obtained), the baseband image data is read from the coefficient buffer unit 203 and read out. Output baseband image data.

  The coefficient buffer unit 203 and the wavelet inverse transform unit 204 are also collectively referred to as a synthesis filter processing unit 211.

  Next, processing performed by the wavelet transform unit 101 in FIG. 10 will be described in more detail. First, data input to the wavelet transform unit 101 will be described.

  Next, the wavelet inverse transformation will be schematically described. FIG. 10 is a diagram schematically illustrating a configuration example of wavelet inverse transform operation on coefficient data. The output of the wavelet transform process described with reference to FIG. 3 is the LLL component 243 corresponding to the LLL component 143, the LLH component 244 corresponding to the LLH component 144, the LH component 245 corresponding to the LH component 145, and the H component 146. Of the corresponding H component 246, the LLL component 243 and the LLH component 244 are first up-sampled to double the resolution by the up-sampler 231a and the up-sampler 231b, respectively. The low frequency component (LLL component 243) upsampled by the upsampler 231a is filtered by the low pass filter 232 and supplied to the adder 234. The high frequency component (LLH component 244) upsampled by the upsampler 231b is filtered by the low pass filter 233 and supplied to the adder 234.

  The adder 234 synthesizes the low frequency component and the high frequency component (LLL component 243 and LLH component 244). Through the processing of the circuit unit 221 so far, the inverse conversion corresponding to the forward conversion in the circuit unit 123 of level 3 in FIG. 3 is completed, and the LL component 247 that is the low-frequency component of level 2 is obtained.

  By repeating this process to level 1 at each level thereafter, a final decoded image 249 after inverse transformation is output. That is, the level 2 circuit unit 222 and the level 1 circuit unit 223 have the same configuration as the level 3 circuit unit 221 described above. That is, the output (LL component 247) of the level 3 circuit unit 221 is input to the low frequency side of the level 2 circuit unit 222. The circuit unit 222 synthesizes the LL component 247 input to the low frequency side and the LH component 245 input to the high frequency side in the same manner as in the circuit unit 221, and the L component 248 that is the low frequency component of level 1 is combined. Output. This L component 248 is input to the low frequency side of the level 1 circuit unit 223. The circuit unit 223 combines the L component 248 input to the low frequency side and the H component 246 input to the high frequency side in the same manner as the circuit unit 221, and outputs baseband image data (decoded image) 249. To do. The above is the basic configuration of the wavelet inverse transform process.

The operation of the integer precision type 5 × 3 synthesis filter defined by the FCD of Part-1 of JPEG-2000 will be described. FIG. 11 is a diagram for explaining an operation of an integer precision type 5 × 3 synthesis filter (hereinafter also referred to as an integer type 5 × 3 synthesis filter). One-dimensional synthesis filter processing is performed once. It shows how the output image data at the right end in the figure is calculated from the low end component s at the left end and the high frequency component d at the center. In FIG. 11, it is assumed that d _m ⁿ is the m-th high frequency component coefficient whose wavelet transform level is n. Similarly, ^let s _m ⁿ be the m-th low-frequency component coefficient whose wavelet transform level is n. Here, when n = 0, an output image is obtained as shown in FIG.

  As shown in FIG. 9, the calculation formula for the low-frequency component coefficient s of one lower level can be generally expressed as the following formula (5).

s _{m + 1} ⁿ = s _{m + 1} ^{n + 1} − ((d _m ^{n + 1} + d _{m + 1} ^{n + 1} +2) / 4) (5)

  Here, by multiplying 1/4 in the equation (5) by bit shift and expressing a 2-bit right shift by “>> 2”, equation (5) can be expressed by the following equation (5 ′) It can be expressed as

s _{m + 1} ⁿ = s _{m + 1} ^{n + 1} − ((d _m ^{n + 1} + d _{m + 1} ^{n + 1} +2) >> 2) (5 ′)

  “+2” in parentheses in the equation (5) or the equation (5 ′) relates to a rounding error that occurs in the right shift, and this is an integral part of the rounding process of the analysis filter described above. Of course, it is necessary for the rounding process to coincide on the analysis / synthesis side in order to maintain reversibility.

  Similarly, the calculation formula of the high frequency component coefficient d can be generally expressed as the following formula (6).

d _m ⁿ = d _m ^{n + 1} + ((s _m ⁿ + s _{m + 1} ⁿ ) / 2) (6)

  Here, when the 1-bit right shift is represented by “>> 1”, Expression (6) can be represented as the following Expression (6 ′).

d _m ⁿ = d _m ^{n + 1} + ((s _m ⁿ + s _{m + 1} ⁿ ) >> 1) (6 ′)

Here, a portion SP surrounded by a broken line in FIG. 11 is a portion for obtaining a low-frequency component coefficient s ₂ ⁰ corresponding to n = 0 and m = 1 in the above-described equation (5). The calculation shown in (7) is performed.

s ₂ ⁰ = s ₂ ¹ − ((d ₁ ¹ + d ₂ ¹ +2) / 4) (7)

A portion DP surrounded by a broken line in FIG. 11 is a portion for obtaining a high frequency component coefficient d ₀ ⁰ corresponding to n = 0 and m = 0 in the above-described equation (6). The calculation shown in 8) is performed.

d ₀ ⁰ = d ₀ ¹ + ((s ₀ ⁰ + s ₁ ⁰ ) / 2) (8)

  Here, 1/4 multiplication can be realized by 2-bit right shift “>> 2”, and 1/2 multiplication can be realized by 1-bit right shift “>> 1”.

  The above-described equations (5) and (6), in particular, the amount of calculation required to calculate the equations (5 ′) and (6 ′) using the shift operation, that is, 1 of even-numbered and odd-numbered coefficients. The amount of calculation required to generate a set is multiplication “0”, addition / subtraction “5”, and shift calculation “2”. By performing this operation in the same manner from top to bottom, all the coefficients can be calculated. As described above, with respect to a two-dimensional signal such as an image, another direction (with respect to a coefficient group generated by wavelet inverse transform (synthesis filter processing) in a one-dimensional direction (for example, vertical direction) ( For example, the wavelet inverse transform may be performed in the same manner as described above in the horizontal direction.

  Next, the flow of processing by the encoding device 100 and the decoding device 200 will be described. Initially, the example of the flow of the encoding process performed by the encoding apparatus 100 is demonstrated with reference to the flowchart of FIG.

  When the encoding process is started, the wavelet transform unit 101 of the encoding apparatus 100 initializes the number A of the processing target precinct in step S101. For example, the number A is set to “1”. When the setting is completed, in step S102, the wavelet transform unit 101 acquires image data of the number of lines (that is, one precinct) necessary to generate the Ath one line from the top of the lowest subband after the conversion. To do.

  When the image data for one precinct is acquired, the wavelet transform unit 101 performs reversible vertical analysis filtering processing that performs analysis filtering on the image data in a reversible manner with respect to the image data arranged in the screen vertical direction in step S103. In step S104, a reversible horizontal analysis filtering process is performed in which the analysis filtering process is performed on the image data arranged in the horizontal direction on the screen in a reversible manner.

  In step S105, the wavelet transform unit 101 determines whether the reversible vertical analysis filtering process and the reversible horizontal analysis filtering process have been performed to the final level. If it is determined that the decomposition level has not reached the predetermined final level, the process returns to step S103, and the analysis filtering processes in steps S103 and S104 are repeated for the current decomposition level.

  If it is determined in step S105 that the analysis filtering process has been performed to the final level, the process proceeds to step S106.

  In step S106, the entropy encoding unit 103 determines whether or not the wavelet transform unit 101 has processed a predetermined number of precincts. If the entropy encoding unit 103 determines that the processing has been performed, the entropy encoding unit 103 proceeds to step S107 and adds the accumulated coefficient data to the accumulated coefficient data. Entropy encoding is performed on the image. When the encoding is completed, the entropy encoding unit 103 outputs the obtained encoded data, and the process proceeds to step S108. If it is determined in step S106 that the predetermined number of precincts has not been processed, the process of step S107 is omitted, and the process proceeds to step S108.

  In step S108, the wavelet transform unit 101 increments the value of the precinct number A by “1”. In step S109, the wavelet transform unit 101 determines whether there is an unprocessed line. If it is determined that there is an unprocessed line, the wavelet transform unit 101 returns the process to step S102 and repeats the subsequent processes. If it is determined in step S109 that there is no unprocessed line, the wavelet transform unit 101 ends the encoding process.

  The encoding apparatus 100 repeats the encoding process as described above for each picture (frame or field).

  As described above, the wavelet transform unit 101 continuously performs the reversible vertical analysis filtering process and the reversible horizontal analysis filtering process up to the final level in units of precincts, so that it is held at one time (simultaneously) as compared with the conventional method. The amount of data that needs to be buffered is small, and the amount of buffer memory to be prepared can be greatly reduced. Therefore, the delay time can be greatly reduced as compared with the method of performing wavelet transform on the entire screen. Also, entropy encoding processing can be performed in units of a plurality of precincts. Therefore, the image data can be entropy-encoded with higher efficiency and higher quality than when encoding is performed for each precinct unit. Furthermore, by performing wavelet transform and entropy encoding in a reversible manner, the encoding apparatus 100 can transmit image data with high quality and low delay.

  Next, an example of the flow of decoding processing executed by the encoding device 200 will be described with reference to the flowchart of FIG.

  When the decoding process is started, the entropy decoding unit 201 performs entropy decoding on the input encoded data in step S201. In step S202, the coefficient rearranging unit 202 rearranges the coefficient data in the order of low frequency components to high frequency components. In step S203, when the coefficient data supplied from the coefficient rearranging unit 202 is held for one precinct or more in the coefficient buffer unit 203, the wavelet inverse transform unit 204 reads the coefficient data for one precinct and performs reversible. Vertical synthesis filter processing is performed, and reversible horizontal synthesis filter processing is performed in step S204.

  In step S205, the wavelet inverse transform unit 204 determines whether or not the synthesis filter processing has been performed up to level 1, and if it is determined that the level has not yet reached level 1, the processing returns to step S203, and the subsequent processing is performed. repeat. If it is determined in step S205 that the synthesis filter process has been performed up to level 1, the process proceeds to step S206.

  In step S206, the entropy decoding unit 201 determines whether or not to end the decoding process. If it determines not to end the process, the entropy decoding unit 201 returns the process to step S201 and repeats the subsequent processes. If it is determined in step S206 that the decoding process is to be terminated, the decoding process is terminated.

  The decoding apparatus 200 repeats the above decoding process for each picture (frame or field).

  In the case of the conventional wavelet inverse transformation method, the horizontal synthesis filtering process is first performed in the horizontal direction on the screen and then the vertical synthesis filtering process is performed in the vertical direction on the screen with respect to all coefficients of the decomposition level to be processed. In other words, for each synthesis filtering process, the result of the synthesis filtering process needs to be held in the buffer. At this time, the buffer stores the synthesis filtering result of the current decomposition level and all coefficients of the next decomposition level. Must be held, and a large memory capacity is required (a large amount of data is held).

  Further, in this case, since image data output is not performed until all wavelet inverse transformations are completed in the picture (frame or field), the delay time from input to output increases.

  On the other hand, in the case of the wavelet inverse transform unit 203 of the decoding device 200, the vertical synthesis filtering process and the horizontal synthesis filtering process are continuously performed up to level 1 in units of precincts as described above. The amount of data that needs to be buffered at the same time (at the same time) is small, and the amount of buffer memory to be prepared can be greatly reduced. Further, by performing synthesis filtering processing (wavelet inverse transformation processing) up to level 1, image data can be sequentially output (in precinct units) before all image data in a picture is obtained. In comparison, the delay time can be greatly reduced.

  Also, the decoding apparatus 200 can obtain a high-quality restored image by performing entropy decoding processing and wavelet inverse transformation processing by a reversible method.

  That is, the encoding apparatus 100 and the decoding apparatus 200 can transmit image data with high quality and low delay.

  Next, an irreversible method encoding process and a decoding process for which conversion between the forward direction and the reverse direction is not completely guaranteed with respect to the reversible method encoding process and decoding process as described above will be described.

  FIG. 14 is a block diagram illustrating a configuration example of an encoding device that encodes image data using an irreversible method. In FIG. 14, the encoding apparatus 300 performs an encoding process by an irreversible method. The encoding apparatus 300 includes a wavelet transform unit 301, a midway calculation buffer unit 302, a quantization unit 303, a coefficient rearrangement buffer unit 304, a coefficient rearrangement unit 305, an entropy encoding unit 306, and a rate control unit 307. .

  The wavelet transform unit 301 is basically the same as the wavelet transform unit 101, but performs irreversible wavelet transform using a fixed-point precision type 5 × 3 analysis filter described later. The intermediate calculation buffer unit 302 is basically the same as the intermediate calculation buffer unit 102.

  The quantization unit 303 quantizes the coefficient data converted by the wavelet transform unit 301 with a predetermined quantization coefficient, and supplies the quantized coefficient data to the coefficient rearranging buffer unit 304 for holding. Of course, the quantization unit 303 may be omitted. By performing the quantization, further improvement of the compression effect can be expected. Any quantization method may be used. For example, a method of dividing the coefficient data W by the quantization step size Δ may be used. The quantization step size Δ at this time is calculated by the rate control unit 307, for example. The coefficient rearranging unit 305 is basically the same as the coefficient rearranging unit 202. The coefficient rearranging unit 305 receives the coefficient data held in the coefficient rearranging buffer unit 304 in order from the high frequency component to the low frequency component. The components are rearranged in the order of the band components and supplied to the entropy encoding unit 306.

  The entropy encoding unit 306 is basically the same as the entropy encoding unit 103, and encodes the supplied coefficient data by a predetermined entropy encoding method such as Huffman encoding or arithmetic encoding.

  The entropy encoding unit 306 operates in conjunction with the rate control unit 307 and is controlled so that the bit rate of the output compression encoded data becomes a substantially constant value. That is, the rate control unit 307, based on the encoded data information from the entropy encoding unit 306, immediately when the bit rate of the data compressed and encoded by the entropy encoding unit 306 reaches the target value or immediately before reaching the target value. Then, a control signal for controlling to end the encoding process by the entropy encoding unit 306 is supplied to the entropy encoding unit 306. The entropy encoding unit 306 outputs the encoded data to the outside of the encoding apparatus 300 when the encoding process is completed according to the control signal supplied from the rate control unit 307.

  The wavelet transform unit 301 and the midway calculation buffer unit 302 are also collectively referred to as an analysis filter processing unit 311. FIG. 15 is a block diagram showing a more detailed configuration example of the analysis filter processing unit.

  As illustrated in FIG. 15, the fixed-point wavelet transform unit 301 includes a bit shifter 321 and a wavelet transformer 322.

The bit shifter 321 performs bit shift processing of the input image data only once, and generates input data with fixed point precision as shown in FIG. In FIG. 16, each square represents each bit of input data. The point P indicates the position of the fixed point, the bit on the upper side (left side in the figure) from this point P indicates the integer part, and the bit on the lower side (right side in the figure) indicates the decimal part. In the example of FIG. 16, since the decimal part has 6 bits, the fixed point precision can be ensured with an accuracy of 1/2 ⁶ = 1/64. In the example of FIG. 16, it is assumed that the integer part handles 8-bit data, and 16-bit calculation is performed as a whole (the calculation accuracy is 16 bits). Furthermore, in the example of FIG. 16, a guard bit (Guard-bit) of 2 bits is provided on the most significant side (the left end side in the figure), and 1 bit of this is a bit indicating a sign (+/−). Yes, the other bit is a bit for avoiding overflow.

  Returning to FIG. 15, the fixed-point precision data generated by the bit shifter 321 is stored in the midway calculation buffer unit 302 by the wavelet transformer 322. When a predetermined amount (one precinct) of fixed-point type data is accumulated in the midway calculation buffer unit 302, the wavelet transformer 322 applies the data for one precinct accumulated in the midway calculation buffer unit 302. Analysis filter processing is performed using a 5 × 3 analysis filter, and coefficient data of each subband is generated.

FIG. 17 illustrates a fixed-point wavelet transform operation on the analysis side using a fixed-point precision 5 × 3 analysis filter (hereinafter also referred to as a fixed-point type 5 × 3 analysis filter). FIG. 17 shows an operation of performing one-dimensional wavelet transform and converting the leftmost input image data into a low-frequency component s and a high-frequency component d. The detailed operation will be described below with reference to FIG. Here, it is assumed that d _m ⁿ is the m-th high-frequency component coefficient whose wavelet transform level is n. In the case of n = 0, it is clear from the figure that the input image. Similarly, ^let s _m ⁿ be the m-th low-frequency component coefficient whose wavelet transform level is n.

  As can be seen from FIG. 17, the formula for calculating the low-frequency component coefficient s can be generally expressed as the following formula (9a) or formula (9b).

s _{m + 1} ^{n + 1} = s _{m + 1} ⁿ + β × (d _m ^{n + 1} + d _{m + 1} ^{n + 1} +2) (9a)
s _{m + 1} ^{n + 1} = s _{m + 1} ⁿ + β × (d _m ^{n + 1} + d _{m + 1} ^{n + 1} ) (9b)
Given in.

  Similarly, the calculation formula of the high frequency component coefficient d can be generally expressed by the following formula (10).

d _m ^{n + 1} = d _m ⁿ −α × (s _m ⁿ + s _{m + 1} ⁿ ) (10)

  Here, α = 0.5 and β = 0.25, and d corresponding to the high frequency component is Nyquist gain = 2 on the analysis side, so that the gain is adjusted so that Nyquist gain = 1. This is the reason why the high frequency component coefficient d is multiplied by SH = 0.5. Further, in the case of the integer type 5 × 3 analysis filter described above, if gain adjustment is performed, reversible conversion is not guaranteed, and thus gain adjustment is not necessary. This peripheral technology is known as a general digital signal processing technology.

When calculating the data of the position of the screen edge, when data outside the screen is required, the data inside the adjacent screen is used by folding (for example, d ₀ ¹ for obtaining s ₀ ^{1 in} FIG. 17). This is the same as in the case of the integer analysis filter described above, and also in the case of the synthesis filter.

  Next, the difference between the above formulas (9a) and (9b) will be described. Since the fixed-point type 5 × 3 analysis filter has higher precision than the integer precision, the expression (9a) (compatible) for rounding “+2” described in the integer type 5 × 3 analysis filter and “ Both equations (9b) (incompatible) with no +2 "rounding are conceivable.

  When the configuration of the arithmetic means is made common between the fixed-point type 5 × 3 analysis filter and the integer type 5 × 3 analysis filter described above, it is necessary to use Expression (9a). That is, when the configuration for performing the calculation of Expression (9a) is used as the wavelet transformer used for the fixed-point type 5 × 3 analysis filter, it is also commonly used as the wavelet transformer used for the integer type 5 × 3 analysis filter. be able to.

  In the fixed-point type 5 × 3 analysis filter described with reference to FIG. 17, the amount of calculation required to generate one set of coefficients of the low-frequency component and the high-frequency component is multiplication “3”, addition / subtraction “5” ( Compatible) or addition / subtraction “4” (incompatible).

  Next, a decoding apparatus corresponding to the above irreversible encoding apparatus 300 will be described. FIG. 18 is a block diagram illustrating a configuration example of a decoding device that decodes encoded data in an irreversible manner.

  18 performs a decoding process by a method corresponding to the encoding process of the encoding apparatus 300 described above. In other words, the decoding device 400 performs the decoding process using an irreversible method in which forward and reverse conversions are not completely guaranteed. The decoding apparatus 400 includes an entropy decoding unit 401, an inverse quantization unit 402, a coefficient buffer unit 403, and a wavelet inverse transform unit 404.

  The entropy decoding unit 401 is basically the same as the entropy decoding unit 201, and decodes supplied encoded data by a method corresponding to the entropy encoding unit 306 to generate coefficient data. The entropy decoding unit 401 supplies the generated coefficient data to the inverse quantization unit 402. The inverse quantization unit 402 performs inverse quantization by a method corresponding to the quantization unit 303 and supplies the coefficient data subjected to inverse quantization to the coefficient buffer unit 403 to be held. When the quantization unit 303 is omitted in the encoding device 300, the inverse quantization unit 402 is also omitted.

  The coefficient buffer unit 403 is basically the same as the coefficient buffer unit 203, holds the coefficient data supplied from the inverse quantization unit 402, and appropriately supplies the coefficient data to the wavelet inverse transform unit 404, The intermediate data and image data generated by the wavelet inverse transform unit 404 are held.

  The wavelet inverse transformation unit 404 is basically the same as the wavelet inverse transformation unit 204, but using a fixed-point precision type 5 × 3 synthesis filter (hereinafter also referred to as a fixed-point type 5 × 3 synthesis filter) described later. Performs irreversible inverse wavelet transform.

  The coefficient buffer unit 403 and the wavelet inverse transform unit 404 are also collectively referred to as a synthesis filter processing unit 411. FIG. 19 is a block diagram showing a more detailed configuration example of the synthesis filter processing unit 411.

  As shown in FIG. 19, the fixed-point wavelet inverse transform unit 404 includes a wavelet inverse transformer 421 and a bit shifter 422.

  Regarding fixed-point precision, the difference from integer precision has been described above. However, in the case of fixed-point precision, the data is bit-shifted to the left on the analysis filter side. Is restored to its original level. This right bit shift is performed on the final decoded image (level 0). That is, the wavelet inverse transformer 421 repeatedly performs synthesis filter processing for each level using the coefficient buffer unit 403, and finally supplies baseband image data to the bit shifter 422. To do. The bit shifter 422 performs a right bit shift process on the supplied baseband image data, and outputs the shifted data.

Next, the fixed-point wavelet inverse transform operation using the fixed-point type 5 × 3 synthesis filter will be described with reference to FIG. FIG. 20 shows an operation of performing one-dimensional wavelet transform (inverse transform) and converting the low-frequency component s and the high-frequency component d at the left end in the figure into output pixels at the right end in the figure. Here, it is assumed that d _m ⁿ is the m-th high-frequency component coefficient whose wavelet transform level is n. Similarly, ^let s _m ⁿ be the m-th low-frequency component coefficient whose wavelet transform level is n. When n = 0, s and d are odd-numbered and even-numbered pixels, respectively.

  As is clear from FIG. 20, the calculation formula for the odd-numbered pixel s at the right end of the drawing or the low-frequency component coefficient s is generally expressed as the following formula (11a) or formula (11b). Can do.

s _{m + 1} ⁿ = s _{m + 1} ^{n + 1} −β × (d _m ^{n + 1} + d _{m + 1} ^{n + 1} +2) (11a)
s _{m + 1} ⁿ = s _{m + 1} ^{n + 1} −β × (d _m ^{n + 1} + d _{m + 1} ^{n + 1} ) (11b)

  Similarly, the even-numbered pixel d at the right end of the figure or the calculation formula for the high frequency component coefficient d can be generally expressed by the following formula (12).

d _m ⁿ = d _m ^{n + 1} + α × (s _m ⁿ + s _{m + 1} ⁿ ) (12)

  Here, α = 0.5 and β = 0.25. On the analysis filter side, since d corresponding to the high frequency component is Nyquist gain = 2, the gain is adjusted so that Nyquist gain = 1. That is, the high frequency component coefficient d is multiplied by SH = −0.5 on the analysis filter side. On the other hand, on the synthesis filter side, it is necessary to return Nyquist gain from 1 to 2. For this reason, the high frequency coefficient d is multiplied by SH = −2.0 on the synthesis filter side, and further subjected to the calculation of Expression (12).

  Next, the difference between the above formulas (11a) and (11b) will be described. Since the fixed-point type 5 × 3 synthesis filter can perform conversion processing with higher precision than in the case of integer precision, as in the case of the rounding process in the analysis filter described above, the case of rounding “+2” Both formula (11a) (compatible) and formula (11b) (not compatible) without “+2” are conceivable.

  When the configuration of the arithmetic means is shared between the fixed-point type 5 × 3 synthesis filter and the above-described integer type 5 × 3 synthesis filter, it is necessary to use the expression (11a).

  In the fixed-point type 5 × 3 synthesis filter that performs the operation as shown in FIG. 10, the amount of calculation required to generate one set of coefficients of the low-frequency component and the high-frequency component is multiplication “3”, addition / subtraction “5” ( Compatible) or addition / subtraction “4” (incompatible).

  Next, the flow of the irreversible encoding process and decoding process as described above will be described. First, an example of the flow of encoding processing by the encoding device 300 will be described with reference to the flowchart of FIG. This encoding process is a process of encoding image data by an irreversible method in which conversion between the forward direction and the reverse direction is not completely guaranteed.

  As described above, the analysis filter processing unit 311 basically performs the analysis filter in the same manner as in the case of reversibility (in the case of the analysis filter processing by the analysis filter processing unit 111) except that processing is performed with fixed-point precision instead of integer precision. Process. That is, the processes in steps S301 through S305 are executed in the same manner as the processes in steps S101 through S105 in FIG.

  When the analysis filter processing unit 311 (wavelet transform unit 301) performs analysis filter processing on the image data for one precinct to the final level with fixed-point precision, in step S306, the quantization unit 306 determines the coefficient for the one precinct. Quantize the data. Actually, the quantization unit 306 sequentially quantizes the coefficient data supplied from the wavelet transform unit 301.

  In step S307, the coefficient rearranging unit 305 rearranges the quantized coefficient data arranged in the order from the high frequency component to the low frequency component in the order from the low frequency component to the high frequency component. In step S308, the entropy encoding unit 306 performs entropy encoding on the rearranged coefficient data. When the encoding is completed, the wavelet transform unit 301 increments the value of the processing target precinct number A in step S309 and changes the processing target to the next precinct. In step S310, the wavelet transform unit 301 determines whether there is an unprocessed line. If it is determined that there is an unprocessed line, the wavelet transform unit 301 returns the process to step S302 and repeats the subsequent processes. If it is determined in step S310 that there is no unprocessed line, the wavelet transform unit 301 ends the encoding process.

  The encoding apparatus 300 repeats the above encoding process in units of pictures (frames or fields).

  Next, an example of the flow of decoding processing by the decoding device 400 will be described with reference to the flowchart of FIG. This decoding process is a process of decoding the encoded data by an irreversible method in which conversion between the forward direction and the reverse direction is not completely guaranteed.

  As described above, the decoding device 400 performs decoding processing basically in the same manner as in the case of lossless (in the case of decoding processing by the decoding device 200), except that processing is performed with fixed-point accuracy instead of integer accuracy. However, in the case of irreversible, reduction of delay time has the highest priority, so that the rearrangement of coefficients is performed in the encoding apparatus 100. In the case of irreversible, the encoding apparatus 300 also performs quantization. Therefore, the decoding apparatus 400 performs inverse quantization instead of rearranging the coefficients.

  When the decoding process is started, the entropy decoding unit 401 entropy decodes the input encoded data in step S401. In step S402, the inverse quantization unit 402 inversely quantizes the coefficient data by a method corresponding to step S306 (FIG. 21). In step S403, when the coefficient data supplied from the inverse quantization unit 402 is held in the coefficient buffer unit 403 for one precinct or more, the wavelet inverse transform unit 404 reads the fixed precinct coefficient data and fixes it. A irreversible vertical synthesis filter process with decimal point precision is performed, and an irreversible horizontal synthesis filter process with fixed point precision is performed in step S404.

  In step S405, the wavelet inverse transform unit 404 determines whether or not the synthesis filter processing has been performed up to level 1. If it is determined that the level has not yet reached level 1, the processing returns to step S403, and the low-frequency component is processed. Repeat the process after that. If it is determined in step S405 that the synthesis filter process has been performed up to level 1, the process proceeds to step S406.

  In step S406, the entropy decoding unit 401 determines whether or not to end the decoding process. If it determines not to end the process, the entropy decoding unit 401 returns the process to step S401 and repeats the subsequent processes. If it is determined in step S406 that the decoding process is to be terminated, the decoding process is terminated.

  The decoding apparatus 400 repeats the above decoding process for each picture (frame or field).

  The various processes as described above can be appropriately executed in parallel as shown in FIG. 23, for example.

  FIG. 23 is a diagram schematically illustrating an example of parallel operation of each element of processing in the case of irreversible executed by each unit of the encoding device 300 illustrated in FIG. 14 and the decoding device 400 illustrated in FIG. . FIG. 23 corresponds to FIG. 7 and FIG. 8 described above. The wavelet transform unit 301 (FIG. 14) performs the first wavelet transform WT-1 (B in FIG. 23) on the input In-1 of the image data (A in FIG. 23). The first wavelet transform WT-1 is started when the first three lines are input, and the coefficient C1 is generated. That is, a delay of 3 lines occurs from the input of the image data In-1 until the wavelet transform WT-1 is started.

  The generated coefficient data is stored in the coefficient rearranging buffer unit 304 (FIG. 14) after quantization. Thereafter, wavelet transform is performed on the input image data, and when the first process is completed, the process proceeds to the second wavelet transform WT-2.

  In parallel with the input of the image data In-2 for the second wavelet transform WT-2 and the processing of the second wavelet transform WT-2, the coefficient rearrangement unit 305 (FIG. 14) The rearrangement Ord-1 of the coefficient C1, the coefficient C4, and the coefficient C5 is executed (C in FIG. 23).

  Note that the delay from the end of the wavelet transform WT-1 to the start of the reordering Ord-1 is, for example, a delay associated with transmission of a control signal instructing the coefficient reordering unit 305 to perform a reordering process, This delay is based on the apparatus and system configuration, such as the delay required for starting the processing of the coefficient rearranging unit 305 and the delay required for the program processing, and is not an essential delay in the encoding process.

  The coefficient data is read from the coefficient rearranging buffer unit 304 in the order in which the rearrangement is completed, and is supplied to the entropy encoding unit 306 (FIG. 14) to perform entropy encoding EC-1 (D in FIG. 23). . This entropy encoding EC-1 can be started without waiting for the end of the rearrangement of all the three coefficients C1, C4, and C5. For example, when the rearrangement of one line by the coefficient C5 output first is completed, entropy coding for the coefficient C5 can be started. In this case, the delay from the start of the reordering Ord-1 to the start of the entropy coding EC-1 is one line.

  The encoded data that has undergone the entropy encoding EC-1 by the entropy encoding unit 306 is subjected to predetermined signal processing and then transmitted to the decoding device 400 (FIG. 18) (E in FIG. 23).

  As described above, the image data is sequentially input to the encoding apparatus 300 up to the lowermost line on the screen, following the input of image data for seven lines in the first process. In the encoding apparatus 300, in accordance with input In-n (n is 2 or more) of image data, as described above, the wavelet transform WT-n, the reordering Ord-n, and the entropy encoding EC-n are performed every four lines. I do. The rearrangement Ord and entropy encoding EC for the last processing in the encoding apparatus 300 are performed for six lines. These processes are performed in parallel in the encoding apparatus 300 as illustrated in A of FIG. 23 to D of FIG.

  The encoded data encoded by the entropy encoding EC-1 by the encoding device 300 is supplied to the decoding device 400. The entropy decoding unit 401 (FIG. 18) of the decoding device 400 sequentially performs entropy decoding iEC-1 on the supplied encoded data encoded by the entropy encoding EC-1, and provides coefficient data. Is restored (F in FIG. 23). The restored coefficient data is inversely quantized and then sequentially stored in the coefficient buffer unit 403. The wavelet inverse transform unit 404 reads the coefficient data from the coefficient buffer unit 403 and stores the wavelet inverse transform iWT− using the read coefficient data when the coefficient data is stored in the coefficient buffer unit 403 so that wavelet inverse transform can be performed. 1 is performed (G in FIG. 23).

  As described with reference to FIGS. 7 and 8, the wavelet inverse transformation iWT-1 by the wavelet inverse transformation unit 404 can be started when the coefficient C4 and the coefficient C5 are stored in the coefficient buffer unit 403. Therefore, the delay from the start of decoding iEC-1 by the entropy decoding unit 401 to the start of wavelet inverse transformation iWT-1 by the wavelet inverse transformation unit 404 is two lines.

  When the wavelet inverse transformation unit 404 finishes the wavelet inverse transformation iWT-1 for three lines by the first wavelet transformation, the output Out-1 of the image data generated by the wavelet inverse transformation iWT-1 is performed (FIG. 23). H). In the output Out-1, the image data of the first line is output as described with reference to FIGS.

  Coefficient data encoded by entropy encoding EC-n (n is 2 or more) is sequentially decoded after the input of the encoded coefficient data for three lines by the first processing in the encoding apparatus 300. 400 is input. The decoding device 400 performs entropy decoding iEC-n and wavelet inverse transformation iWT-n for every four lines on the input coefficient data as described above, and an image restored by the wavelet inverse transformation iWT-n. Data output Out-n is sequentially performed. Entropy decoding iEC and inverse wavelet transformation iWT corresponding to the last round of the encoding apparatus 300 are performed on 6 lines, and output Out is output on 8 lines. These processes are performed in parallel in the decoding apparatus 400 as illustrated in F of FIG. 23 to H of FIG.

  As described above, the image compression processing and the image decoding processing can be performed with lower delay by performing the processes in the encoding device 300 and the decoding device 400 in parallel in the order from the top to the bottom of the screen. It becomes possible.

  Referring to FIG. 23, from the image input to the image output when wavelet transform and wavelet inverse transform are performed up to decomposition level = 2 using a fixed-point type 5 × 3 analysis filter and a fixed-point type 5 × 3 synthesis filter Let's calculate the delay time. The delay time from when the image data of the first line is input to the encoding device 300 to when the image data of the first line is output from the decoding device 400 is the sum of the following elements. Here, delays that differ depending on the system configuration, such as delays in the transmission path and delays associated with actual processing timing of each unit of the apparatus, are excluded.

(1) Delay D_WT from completion of first line input to completion of wavelet transform WT-1 for 7 lines
(2) Time D_Ord associated with count rearrangement Ord-1 for three lines
(3) Time D_EC associated with entropy encoding EC-1 for 3 lines
(4) Time D_iEC associated with entropy decoding iEC-1 for 3 lines
(5) Time D_iWT associated with wavelet inverse transformation iWT-1 for 3 lines

  Referring to FIG. 23, an attempt is made to calculate a delay by each of the above-described elements. The delay D_WT in (1) is a time for 10 lines. The time D_Ord in (2), the time D_EC in (3), the time D_iEC in (4), and the time D_iWT in (5) are times for three lines, respectively. In the encoding device 300, the entropy encoding EC-1 can be started one line after the rearrangement Ord-1 is started. Similarly, in decoding apparatus 400, wavelet inverse transform iWT-1 can be started two lines after entropy decoding iEC-1 is started. In addition, the entropy decoding iEC-1 can start processing when encoding for one line is completed in the entropy encoding EC-1.

  Therefore, in the example of FIG. 23, the delay time from when the first line of image data is input to the encoding apparatus 300 to when the first line of image data is output from the decoding apparatus 400 is 10 + 1 + 1 + 2 + 3. = 17 lines.

  Consider the delay time with a more specific example. When the input image data is an HDTV (High Definition Television) interlaced video signal, for example, one frame is configured with a resolution of 1920 pixels × 1080 lines, and one field is 1920 pixels × 540 lines. Therefore, when the frame frequency is 30 Hz, one field of 540 lines is input to the encoding device 300 at a time of 16.67 msec (= 1 sec / 60 fields).

  Accordingly, the delay time associated with the input of the image data for 7 lines is 0.216 msec (= 16.67 msec × 7/540 lines), which is a very short time with respect to the update time of one field, for example. Also, the total number of lines to be processed is the sum of the delay D_WT of (1), the time D_Ord of (2), the time D_EC of (3), the time D_iEC of (4), and the time D_iWT of (5). Since the delay time is small, the delay time is greatly reduced.

  As described above, in the case of the encoding process and the decoding process using the irreversible method described with reference to FIGS. 14 to 23, priority is given to the reduction of the delay time. Therefore, the entropy encoding unit 306 of the encoding apparatus 300 is preferred. Encodes coefficient data for each precinct. In order to further reduce the delay time, the coefficient data is rearranged in the encoding device 300 (coefficient rearranging unit 305). Further, in the case of the irreversible method, the coefficient data can be quantized. By performing this quantization in the encoding device 300 (quantization unit 303), an improvement in compression effect can be expected.

  In the case of the irreversible method, the encoding apparatus 300 can use an irreversible filter such as a fixed-point 5 × 3 analysis filter in the wavelet transform unit 301. Thereby, the encoding apparatus 300 can perform analysis filter processing with higher accuracy than when a reversible analysis filter such as an integer type 5 × 3 analysis filter is used. For example, an irreversible analysis filter other than the fixed-point type 5 × 3 analysis filter such as a 9 × 7 analysis filter, a 13 × 3 analysis filter, or a 9 × 2 analysis filter is also applicable. Similarly, an irreversible synthesis filter other than the fixed-point type 5 × 3 synthesis filter, such as a 9 × 7 synthesis filter, a 13 × 3 synthesis filter, or a 9 × 2 synthesis filter, is applied to the decoding device 400. You can also

  On the other hand, in the case of encoding processing and decoding processing by the reversible method described with reference to FIGS. 1 to 13, when baseband image data is encoded by the encoding device 100 and decoded by the decoding device 200, The baseband image data before encoding is restored. That is, it is possible to obtain a high-quality decoded image in which the image quality is not deteriorated compared to before encoding. Therefore, in the case of this reversible method, it is natural to reduce the data transmission delay time as compared with the conventional MPEG, JPEG2000, and other methods, but to some extent consider high-quality data transmission. Also good. In general, there is a trade-off between reducing the delay time and improving the quality of data transmission. Therefore, in order to improve the quality of data transmission, it is necessary to suppress the reduction of the delay time to some extent.

  For example, the analysis filter processing unit 111 of the encoding apparatus 100 performs the analysis filter processing in units of one precinct, but the entropy encoding unit 103 improves the compression efficiency and improves the quality of the data by using a plurality of precincts. Until the coefficient data is obtained, and entropy coding is performed using the coefficient data for the plurality of precincts. Also, since it is a reversible method, the encoding apparatus 100 does not quantize coefficient data (the decoding apparatus 200 does not perform inverse quantization). Although the rearrangement of the coefficients has been described as being performed in the decoding apparatus 200, the encoding apparatus 100 may perform the rearrangement if importance is placed on reducing the delay time.

  Further, the wavelet transform unit 101 of the encoding device 100 has been described as performing wavelet transform using an integer type 5 × 3 analysis filter. For example, an integer type 5 × 3 analysis filter such as a 2 × 1 analysis filter is used. Other reversible analysis filters can be applied. Similarly, for the wavelet inverse transform unit 204 of the decoding apparatus 200, a reversible analysis other than an integer type 5 × 3 synthesis filter such as a 2 × 1 synthesis filter is used instead of the integer type 5 × 3 synthesis filter. A filter may be applied.

  As described above, by performing the encoding process and the decoding process by a reversible method, the encoding apparatus 100 and the decoding apparatus 200 can transmit image data with high quality and low delay.

  In the above description, when encoding / decoding processing is performed by a reversible method and data transmission is performed, the coefficient rearrangement is performed in the decoding device. However, in order to reduce the delay time, the coefficient rearrangement is performed. You may make it perform in an encoding apparatus.

  When transmitting encoded data, generally, encoded data is packetized and transmitted. At this time, information related to encoding, such as encoding using a reversible method, filters used for encoding, the number of processing units of entropy encoding, or whether or not coefficient data is rearranged, is added to the header information of the packet. Thus, the decoding device may be configured to perform an appropriate decoding process according to the content of the encoding process based on the header information.

  These cases will be described below. FIG. 24 is a block diagram showing another configuration example of the encoding apparatus to which the present invention is applied. In FIG. 24, the encoding apparatus 500 encodes baseband image data by a reversible method, like the encoding apparatus 100, to generate encoded data. However, encoding apparatus 500 further packetizes the encoded data and outputs the packetized data. At this time, the encoding apparatus 500 generates header information using information related to encoding, and supplies the header information together with the encoded data to the decoding apparatus. Thereby, the encoding apparatus 500 can provide the information regarding an encoding process to a decoding apparatus, and a decoding apparatus performs the suitable decoding process according to the content of the encoding process based on the header information Can be able to.

  As illustrated in FIG. 24, the encoding device 500 includes a control unit 501, an encoding unit 502, a header generation unit 503, a packetization unit 504, and a transmission unit 505.

  The control unit 501 controls the encoding process by the encoding unit 502 by supplying control information to the encoding unit 502, thereby specifying the processing unit (number of precincts) of the encoding process of the entropy encoding unit 515. . The control unit 501 is a control unit according to, for example, a user instruction input via an input unit (not shown), predetermined setting information input in advance, setting information supplied from an external device, or a predetermined condition Control information is generated or acquired in accordance with the setting information created by itself 501 and supplied to the encoding unit 502.

  The encoding unit 502 is basically a processing unit corresponding to the encoding device 100 of FIG. 1 and has the same configuration as the encoding device 100, that is, a wavelet transform unit 511 corresponding to the wavelet transform unit 101, and an intermediate calculation An intermediate calculation buffer unit 512 corresponding to the buffer unit 102 (an analysis filter processing unit 521 corresponding to the analysis filter processing unit 111) and an entropy encoding unit 515 corresponding to the entropy encoding unit 103 are included. However, the encoding unit 502 has a function of rearranging the coefficient data output from the analysis filter processing unit 521 in order from the high frequency component to the low frequency component, in order from the low frequency component to the high frequency component. The configuration includes a coefficient rearranging buffer unit 513 and a coefficient rearranging unit 514.

  The coefficient rearranging buffer unit 513 holds the coefficient data generated and supplied in the order in which the wavelet transform unit 511 encodes image data by a reversible method. The coefficient rearranging unit 514 rearranges the coefficient data held in the coefficient rearranging buffer unit 513 in the order from the high frequency component to the low frequency component, and supplies the coefficient data to the entropy encoding unit 515. To do.

  Similar to the analysis filter processing unit 511, the wavelet transform unit 511 of the analysis filter processing unit 521 uses the midway calculation buffer unit 512 to perform reversible analysis filter processing on the input image data, and converts the encoded data into Generated and supplied to the coefficient rearranging buffer unit 513. The wavelet transform unit 511 may select a reversible analysis filter to be used. In that case, for example, the wavelet transform unit 511 selects a filter based on the designation included in the control information supplied from the control unit 501.

  The entropy encoding unit 515 encodes the coefficient data whose order is rearranged in the coefficient rearranging unit 514 by a reversible method for each preset number of precincts, and the obtained encoded data is sent to the packetization unit 504. Supply.

  In addition, the encoding unit 502 performs setting and control of each unit in the encoding unit 502 based on the control information supplied from the control unit 501. Also, the encoding unit 502 generates related information regarding the encoding process including the setting content of the encoding, and supplies the related information to the header generating unit 503.

  Based on the supplied related information, the header generation unit 503 generates header information for each predetermined data unit, for example, for each picture.

  FIG. 25 is a diagram schematically illustrating an example of a picture header generated by the header generation unit 503. A picture header 530 illustrated in FIG. 25 is header information including information related to a picture added to each picture of the image data. That is, this picture header 530 is added to the header portion of the first packet in the packet group corresponding to one picture. For example, the picture header 530 is added to the header portion of the top packet of each picture. Of course, it may be added to other than the top packet.

  As shown in FIG. 25, in the picture header 530, a picture header flag which is a flag indicating that it is a picture header is arranged at the head. Further, the picture header 530 includes related information supplied from the encoding unit 502, such as lossless lossy identification information 532, filter type information 533, processing unit precinct number information 534, and coefficient rearrangement information 535. Information related to the encoding process is included.

  The reversible irreversible identification information 532 indicates whether the encoding process by the encoding unit 502 is a reversible method or a irreversible method. For example, it is composed of 1-bit flag information. The filter type information 533 indicates the filter type of the analysis filter used in the wavelet transform unit 511. For example, it is configured by a predetermined identification number, a filter type name, or the like. The processing unit precinct number information 534 indicates a processing unit of encoding by the entropy encoding unit 515. For example, it is indicated by the number of precincts. The coefficient rearrangement information 535 indicates whether or not the coefficient rearrangement has been performed in the encoding unit 502. For example, it is composed of 1-bit flag information.

  Of course, the picture header 530 may include information other than this, a part of the information related to the encoding process described above may be omitted, or information related to the encoding process other than the above described. Further, it may be included.

  The content of the encoding process performed in the encoding unit 502 can be easily grasped by the decoding device that decodes the encoded data generated in the encoding unit 502 acquiring such related information. Therefore, for example, when the encoded data generated in the encoding unit 502 is decoded by a decoding device corresponding to various encoding methods, the decoding device can easily perform an appropriate decoding process based on the header information. Can be set.

  The packetization unit 504 packetizes the encoded data supplied from the encoding unit 502 for each predetermined data unit, and appropriately adds the header information supplied from the header generation unit 503 to the packet and generates the packet. The packet is supplied to the transmission unit 505. The transmission unit 505 transmits the packet to the decoding device via the transmission path according to the format of the communication network serving as the transmission path.

  In the above description, the header generation unit 503 generates the picture header based on the related information. However, the header generation unit 503 may include information on the encoding in a header other than the picture header. That is, the timing at which the header generation unit 503 generates information related to encoding is arbitrary and may be other than a picture unit.

  An example of the flow of the encoding process by the encoding apparatus 500 as described above will be described with reference to the flowchart of FIG.

  In step S501, the encoding unit 502 sets each unit, that is, the encoding process, based on the control information supplied from the control unit 501. In step S502, the encoding unit 502 starts encoding image data. Details of this encoding process will be described later with reference to the flowchart of FIG. Thereafter, the encoding process by the encoding unit 502 is executed in parallel with the following process.

  When the encoding process for the image data is started, the packetizing unit 504 packetizes the encoded data supplied from the encoding unit 502 for each predetermined data unit in step S503. In step S504, the header generation unit 503 determines whether image data for one picture has been encoded. If it is determined that the image data for one picture has been encoded, the header generation unit 503 advances the processing to step S505, and includes a picture header including information on the encoded data based on the related information supplied from the encoding unit 502 530 is generated. This picture header generation process will be described later with reference to the flowchart of FIG.

  In step S506, the packetizer 504 adds the generated picture header to the header of the packet, and the process proceeds to step S507. If it is determined in step S504 that the image data for one picture has not been encoded, the packetizing unit 504 skips steps S505 and S506, and proceeds to step S507. In step S507, the transmission unit 505 transmits the generated packet to the transmission path.

  In step S508, the encoding unit 502 determines whether or not the control information supplied from the control unit 501 has been updated. If it is determined that the control information has been updated, the process proceeds to step S509 to update the encoding setting. . When the setting is updated, the encoding unit 502 advances the process to step S510. If it is determined in step S508 that the control information supplied from the control unit 501 has not been updated, the encoding unit 502 omits the process of step S509 and advances the process to step S510. In step S510, the encoding unit 502 determines whether or not to end the encoding process. For example, when it is determined that the image data is still supplied and the encoding process is to be continued, the process returns to step S503, The subsequent processing is repeated. In Step S510, when it is determined that the encoding process is to be ended, the encoding unit 502 ends the encoding process.

  Next, an example of the flow of encoding processing executed by the encoding unit 502 will be described with reference to the flowchart of FIG.

  This encoding process is basically executed in the same manner as the encoding process by the encoding apparatus 100 described with reference to the flowchart of FIG. That is, the processes in steps S531 to S535 are executed in the same manner as the processes in steps S101 to S105 in FIG. However, in the case of the encoding process of FIG. 27, in step S536, the coefficient rearranging unit 514 rearranges the coefficient data arranged in the order of the high frequency component to the low frequency component in the order of the low frequency component to the high frequency component. Further, the processes in steps S537 and S540 are executed in the same manner as the processes in steps S106 to S109 in FIG.

  The encoding process by the encoding unit 502 is executed as described above.

  Next, an example of the flow of the picture header generation process executed in step S505 in FIG. 26 will be described with reference to the flowchart in FIG.

  When the picture header generation process is started, the header generation unit 503 sets the picture header flag 531 in step S61. In step S562, the header generation unit 503 sets the reversible irreversible identification information 532 based on the related information supplied from the encoding unit 502 depending on whether the encoding method is the reversible method or the irreversible method. .

  In step S563, the header generation unit 503 sets the filter type information 533 according to the type of filter used for the wavelet transform, based on the related information supplied from the encoding unit 502. In step S564, the header generation unit 503 sets the processing unit precinct number information 534 according to the encoding processing unit based on the related information supplied from the encoding unit 502. In step S565, the header generation unit 503 sets the coefficient rearrangement information 535 to indicate that the coefficients have been rearranged based on the related information supplied from the encoding unit 502. Furthermore, in step S566, the header generation unit 503 sets other information based on the related information supplied from the encoding unit 502.

  When the process of step S566 ends, the process returns to step S505 in FIG. 26, and the processes after step S506 are executed.

  As described above, encoding apparatus 500 encodes image data, packetizes the encoded data and outputs the data, and also generates and outputs information related to the encoding processing, so that the encoded data is decoded. It is possible for the decoding device to easily grasp the content of the encoding process and perform the decoding process with appropriate settings. That is, the encoding apparatus 500 can suppress the occurrence of unnecessary data loss or the like that leads to image quality degradation during transmission, and can realize higher quality data transmission.

  In addition, the encoding apparatus 500 rearranges the coefficient data generated in the order from the high frequency component to the low frequency component, by rearranging the coefficient data from the low frequency component to the high frequency component, which is the order of the decoding process (inverse wavelet transform process). Further, the delay time (from the start of the encoding process to the end of the decoding process) due to data transmission can be reduced.

  A decoding process corresponding to this encoding process will be described later. Before that, description of the encoding device will be continued.

  In the above description, the encoding apparatus has been described as performing encoding using a reversible method or an irreversible method. However, in addition to this, the encoding apparatus may select an encoding method. Good. For example, the encoding apparatus can execute both the encoding process by a reversible method and the encoding process by an irreversible method, and can select which method to encode. Also good. Furthermore, the encoding apparatus may be able to select whether or not to perform coefficient rearrangement, or may be able to arbitrarily set the processing unit for entropy encoding. That is, the encoding device may be configured to arbitrarily set the encoding process.

  FIG. 29 is a block diagram showing still another configuration example of the encoding apparatus to which the present invention is applied.

  In FIG. 29, an encoding apparatus 600 is an encoding apparatus that encodes image data, and has basically the same configuration as the above-described encoding apparatus 500 and the like. However, the encoding apparatus 600 can select whether to perform the analysis filter process by a reversible method or an irreversible method, and whether to perform a rearrangement process. Also, the encoding apparatus 600 can arbitrarily set the processing unit of the entropy encoding process.

  As illustrated in FIG. 29, the encoding device 600 includes a control unit 601, an encoding unit 602, a header generation unit 603, a packetization unit 604, and a transmission unit 605. The control unit 601, the encoding unit 602, the header generation unit 603, the packetization unit 604, and the transmission unit 605 are the control unit 501, the encoding unit 502, the header generation unit 503, and the packetization unit 504 of the encoding device 500. , And the transmission unit 505, respectively, have basically the same configuration and perform the same processing.

  However, the encoding unit 602 can change settings for each of the analysis filter process, the rearrangement process, and the entropy encoding process, as described above. As illustrated in FIG. 29, the encoding unit 602 includes an analysis filter processing unit 621, a rearrangement processing unit 622, and an encoding unit 623.

  The analysis filter processing unit 621 corresponds to the analysis filter processing unit 521 and includes a wavelet transform unit 611 and a midway calculation buffer unit 612. The rearrangement processing unit 622 includes a switching unit 613 and a switching unit 616 in addition to a coefficient rearranging buffer unit 614 similar to the coefficient rearranging buffer unit 513 and a coefficient rearranging unit 615 similar to the coefficient rearranging unit 514. Have The switching unit 613 and the switching unit 616 switch whether to rearrange the coefficients based on the control of the control unit 601.

  For example, when the coefficient rearrangement unit 615 performs the rearrangement of coefficient data based on the control of the control unit 601, the rearrangement processing unit 622 converts the coefficient data supplied from the wavelet transform unit 611 into the coefficient to the switching unit 613. The connection is switched so as to be supplied to the rearrangement buffer unit 614, and the switching unit 616 is switched so that the output of the coefficient rearrangement unit 615 is supplied to the encoding buffer unit 617 of the encoding processing unit 623. .

  Further, for example, the reordering processing unit 622 performs the coefficient supplied from the wavelet transform unit 611 to the switching unit 613 when the coefficient reordering unit 615 does not reorder the coefficient data based on the control of the control unit 601. The connection is switched so that data is supplied to the switching unit 616, and the connection is switched so that the switching unit 616 supplies the output of the switching unit 613 to the encoding buffer unit 617 of the encoding processing unit 623.

  The encoding processing unit 623 includes an encoding buffer unit 617 and an entropy encoding unit 618. The encoding buffer unit 617 sequentially stores the coefficient data supplied from the switching unit 616. The entropy encoding unit 618 performs entropy encoding similar to the entropy encoding unit 515 of the encoding device 500 on the coefficient data stored in the encoding buffer unit 617, but at this time, designated by the control unit 601. Entropy encoding is performed using the number of precincts to be processed as a processing unit. By storing the coefficient data in the encoding buffer unit 617, the entropy encoding unit 618 uses, as a processing unit, coefficient data for an arbitrary number (plural) of precincts, such as 2 precincts and 3 precincts. An encoding process can also be performed.

  FIG. 30 is a block diagram illustrating a detailed configuration example of the analysis filter processing unit 621. As illustrated in FIG. 30, the wavelet transform unit 611 includes a buffer unit 631, a switching unit 632, an integer wavelet transform unit 633, a fixed-point wavelet transform unit 634, and a switch unit 635.

  The buffer unit 631 accumulates input image data. The switching unit 632 supplies the input image data output from the buffer unit 631 to either the integer type wavelet transform unit 633 or the fixed-point type wavelet transform unit 634 by switching the connection. The integer wavelet transform unit 633 basically has the same configuration as the wavelet transform unit 111 and the wavelet transform unit 511, and performs analysis filter processing by a reversible method. The fixed-point wavelet transform unit 634 has basically the same configuration as the wavelet transform unit 301, and performs analysis filter processing by an irreversible method. The switching unit 635 supplies either the output of the integer wavelet transform unit 633 or the fixed-point wavelet transform unit 634 to the switch unit 613 by switching the connection.

  The control unit 601 controls the operations of the switching unit 632, the integer type wavelet transform unit 633, the fixed-point type wavelet transform unit 634, and the switching unit 635 by supplying control information. The wavelet transform process (analysis filter process) is performed in either the type wavelet transform unit 633 or the fixed-point type wavelet transform unit 634. That is, the analysis filter processing unit 621 (wavelet transform unit 611) performs wavelet transform processing by a reversible method or an irreversible method under the control of the control unit 601.

  The flow of the encoding process performed by the encoding device 600 is basically the same as that described with reference to FIG. With reference to the flowchart of FIG. 31, an example of the flow of an encoding process that is started in the process corresponding to step S502 of FIG. 26 by the wavelet transform unit 611 of FIG. 29 will be described.

  Steps S601 to S605 are basically executed in the same manner as steps S531 to S535. However, in step S603 and step S604, the wavelet transform unit 611 is either an integer wavelet transform unit 633 or a fixed-point wavelet transform unit 634, that is, a reversible method based on the control of the control unit 601. Alternatively, analysis filtering (vertical analysis filtering and horizontal analysis filtering) is performed by an irreversible method.

  In step S606, the rearrangement processing unit 622 determines whether or not the coefficient data is rearranged based on the control of the control unit 601, and when it is determined that rearrangement is performed, the rearrangement processing unit 622 controls the switching unit 613 and the switching unit 616. The coefficient rearranging unit 615 can be used, and in step S607, the coefficient data output from the wavelet transform unit 611 is arranged in the order of the high frequency component to the low frequency component, and is arranged in the order of the low frequency component to the high frequency component. Change. When it is determined in step S606 that the coefficient data is not rearranged, the rearrangement processing unit 622 omits the process of step S607.

  The processes in steps S608 to S611 are executed basically in the same manner as the processes in steps S537 to S540. However, the entropy encoding unit 618 performs entropy encoding using the data unit (number of precincts) set by the control unit 601 as a processing unit.

  The encoding unit 602 repeatedly executes the above encoding process for each picture.

  Next, an example of the flow of the setting process executed in the process corresponding to step S501 in FIG. 26 will be described with reference to the flowchart in FIG.

  When the setting process is started, in step S631, the wavelet transform unit 611 determines whether to perform lossless encoding based on the control information supplied from the control unit 601. If it is determined that encoding by the lossless method is to be performed, the wavelet transform unit 611 advances the processing to step S632, controls the switch unit 632 and the switch unit 635 to select the integer wavelet transform unit 633, and controls the integer wavelet. The conversion unit 633 is activated.

  If it is determined in step S631 that encoding by the irreversible method is to be performed, the wavelet transform unit 611 advances the processing to step S633 and selects the fixed-point wavelet transform unit 634 so as to select the fixed-point wavelet transform unit 634. And the fixed-point wavelet transform unit 634 is activated.

  In step S634, the rearrangement processing unit 622 determines whether or not to perform rearrangement based on the control information supplied from the control unit 601. When it is determined that the coefficient data is to be rearranged, the rearrangement processing unit 622 advances the process to step S635, controls the switching unit 613 and the switching unit 616 to perform rearrangement, and the coefficient rearrangement unit 615 and the coefficient rearrangement. The buffer unit 614 is activated. On the other hand, when it is determined in step S634 that the rearrangement is not performed, the rearrangement processing unit 622 advances the process to step S636 and controls the switching unit 613 and the switching unit 616 so as not to perform the rearrangement.

  In step S637, the entropy encoding unit 618 sets an entropy encoding processing unit (for example, the number of precincts) based on the control information supplied from the control unit 601. That is, the entropy encoding unit 618 waits until the coefficient data of the set number of processing unit precincts is accumulated in the encoding buffer unit 617, and the coefficient data of the set number of processing unit precincts is stored in the encoding buffer unit. When stored in 617, entropy encoding processing is executed using those coefficient data.

  When the process of step S637 ends, the process returns to the process corresponding to step S501 of FIG. 26, and the subsequent processes are executed.

  As described above, the encoding apparatus 600 can perform settings related to the encoding method of the encoding unit 602 based on the control information. Note that the setting can be updated by basically the same process. Also, the header generation unit 603 of the encoding device 600 embeds the setting of the encoding process performed as described above in the picture header as information related to encoding. The method is the same as that described with reference to the flowchart of FIG. 28, and the header generation unit 603 embeds each piece of information in the picture header based on the related information supplied from the encoding unit 602.

  In the above, switching between the case of performing the wavelet transform by a reversible method and the case of performing the wavelet transform by an irreversible method is performed by either one of the integer wavelet transform unit 633 and the fixed-point wavelet transform unit 634. However, as described above, the reversible analysis filter processing and the irreversible analysis filter processing can be made common except for the bit accuracy.

  FIG. 33 is a block diagram showing another configuration example of the analysis filter processing 621. As shown in FIG.

  As shown in FIG. 33, in this case, the wavelet transform unit 611 includes a buffer unit 641, a switching unit 642, a bit shifter 643, a switching unit 644, and a wavelet transformer 645. The bit shifter 643 and the wavelet transformer 645 are the same as the bit shifter 321 and the wavelet transformer 322 (FIG. 15). The buffer unit 641 temporarily holds input image data and supplies it to the switching unit 642 at a predetermined timing. The switching unit 642 and the switching unit 644 switch connection so as to select whether or not to use the bit shifter 643 under the control of the control unit 601.

  For example, when the wavelet transform unit 611 performs integer precision analysis filter processing (analysis filter processing by a reversible method) based on the control information of the control unit 601, the switching unit 642 and the switching unit 644 are buffer units. The connection is switched so that the image data output from 641 is supplied to the wavelet transformer 645 without passing through the bit shifter 643.

  Further, for example, when the wavelet transform unit 611 performs analysis filter processing with fixed-point precision (analysis filter processing by an irreversible method) based on control information of the control unit 601, the switching unit 642 and the switching unit 644. Switches the connection so that the image data output from the buffer unit 641 is supplied to the wavelet transformer 645 via the bit shifter 643.

  That is, the wavelet transformer 645 performs wavelet transformation processing with integer precision (in a reversible manner) when wavelet transformation is performed without shifting the input image data. On the other hand, when the wavelet transform is performed after the input image data is bit-shifted by the bit shifter 643, the wavelet transformer 645 performs a wavelet transform process with a fixed-point precision (in an irreversible manner).

  By sharing a part of the configuration in this way, an increase in hardware (circuit scale) can be suppressed, and power consumption and cost of the encoding device 600 can be reduced. Note that the switching method and the picture header generation method are the same as those in the configuration of FIG.

  Next, a decoding apparatus corresponding to the encoding apparatus 500 and the encoding apparatus 600 described above and analyzing a picture header (information related to encoding) and performing an appropriate decoding process will be described. FIG. 34 is a block diagram illustrating another configuration example of the decoding device.

  In FIG. 34, a decoding apparatus 700 corresponds to the encoding apparatus 500 and the encoding apparatus 600, analyzes a picture header (information regarding encoding) supplied from them, and performs an appropriate decoding process. The decoding device 700 includes a receiving unit 701, a packet analyzing unit 702, a header analyzing unit 703, and a decoding unit 704.

  The reception unit 701 receives a packet supplied from the encoding device and supplies the packet to the packet analysis unit 702. The packet analysis unit 702 analyzes the supplied packet, extracts header information such as a picture header from the packet, supplies the header information to the header analysis unit 703, and extracts encoded data from the packet. The converted data is supplied to the decoding unit 704. The header analysis unit 703 analyzes the supplied header information, for example, extracts information related to the encoding included in the picture header described above, and optimizes the setting of the decoding process based on the information related to the encoding Then, control information for decoding processing is generated, and the control information is supplied to the control unit 711 of the decoding unit 704.

  The decoding unit 704 has basically the same configuration as the decoding device 200 and the decoding device 400, decodes the supplied encoded data by a method corresponding to the encoding process performed in the encoding device, and Generate band image data.

  As shown in FIG. 34, the decoding unit 704 includes a control unit 711, an entropy decoding unit 712, a rearrangement processing unit 731 and a synthesis filter processing unit 732.

  The control unit 711 controls the operations of the entropy decoding unit 712, the rearrangement processing unit 731, and the synthesis filter processing unit 732 based on the control information supplied from the header analysis unit 703. Under the control of the control unit 711, the entropy decoding unit 712 performs entropy decoding processing on the encoded data supplied from the packet analysis unit 702 by a method corresponding to the entropy encoding processing performed in the encoding device. The obtained coefficient data is supplied to the switching unit 713 of the rearrangement processing unit 731.

  The rearrangement processing unit 731 has basically the same configuration as that of the rearrangement processing unit 622, and can select whether or not to rearrange coefficient data based on the control of the control unit 711. As illustrated in FIG. 34, the rearrangement processing unit 731 includes a switching unit 713, a coefficient rearranging buffer unit 714, a coefficient rearranging unit 715, and a switching unit 716. That is, the rearrangement processing unit 731 uses the coefficient rearrangement buffer unit 714 and the coefficient rearrangement unit 715 by switching the connection between the switching unit 713 and the switching unit 716 based on the control of the control unit 711. Switch whether or not to rearrange data. The rearrangement processing unit 731 rearranges the coefficient data when the coefficient rearrangement is not performed on the encoding device side, and the rearrangement of the coefficient data when the coefficient rearrangement is performed on the encoding device side. Do not change.

  As described above, the rearrangement processing unit 731 can easily select whether or not to rearrange the coefficient data according to whether or not the coefficient data is rearranged or processed on the encoding device side. it can. The switching unit 716 supplies the supplied coefficient data to the gain adjustment unit 717 of the synthesis filter processing unit 732.

  The synthesis filter processing unit 732 includes a gain adjustment unit 717, a coefficient buffer unit 718, and a wavelet inverse transformation unit 719. The coefficient buffer unit 718 and the wavelet inverse transformation unit 719 basically have the same configuration as the coefficient buffer unit 203 and the wavelet inverse transformation unit 204 (FIG. 9), and the coefficient buffer unit 403 and the wavelet transformation unit 404 (FIG. 18). And perform the same processing. However, the wavelet inverse transform unit 719 can perform both reversible and irreversible synthesis filter processing. The gain adjustment unit 717 adjusts the gain of the high frequency component of the coefficient data as necessary.

  FIG. 35 is a block diagram illustrating a detailed configuration example of the synthesis filter processing unit 732.

  As shown in FIG. 35, the gain adjustment unit 717 includes a switching unit 751, a high-frequency component gain adjustment unit 752, and a switching unit 753, and adjusts the gain of the high-frequency component based on the control of the control unit 711. Select whether or not to do. When performing gain adjustment, the gain adjustment unit 717 receives the coefficient data input to the switching unit 751 and the switching unit 753 via the high-frequency component gain adjustment unit 752 based on the control of the control unit 711. The connection is switched so as to be supplied to the switching unit 753. The high frequency component gain adjustment unit 752 performs a predetermined gain adjustment on the high frequency component of the supplied coefficient data. For example, when gain adjustment is performed on the encoding side, the gain adjustment unit 717 performs gain adjustment in the opposite direction to that during encoding.

  In addition, when gain adjustment is not performed, the gain adjustment unit 717 receives the coefficient data input to the switching unit 751 and the switching unit 753 based on the control of the control unit 711 and the high frequency component gain adjustment unit. The connection is switched so as to be supplied to the switching unit 753 without going through 752.

  As described above, the gain adjustment unit 717 can easily select whether or not to perform gain adjustment according to the analysis filter processing method (with or without gain adjustment processing) on the encoding device side. The switching unit 753 supplies the supplied coefficient data to the coefficient buffer unit 718 and holds it.

  The wavelet inverse transform unit 719 performs synthesis filter processing on the coefficient data stored in the coefficient buffer unit 718 to generate baseband image data. Similarly to the case of the wavelet transform unit 611 (FIG. 30), It has both an integer type wavelet inverse transform unit 754 that performs synthesis filter processing by a reversible method and a fixed-point type inverse wavelet transform unit 755 that performs synthesis filter processing by an irreversible method. The wavelet inverse transform unit 719 further includes a switching unit 756 and a buffer unit 757, and the switching unit 756 switches connection based on the control of the control unit 711, whereby the integer wavelet inverse transform unit 754 or the fixed-point type Synthesis filter processing is performed using one of the inverse wavelet transform units 755. The buffer unit 757 temporarily holds the output of the integer wavelet inverse transform unit 754 or the fixed-point inverse wavelet transform unit 755 supplied via the switching unit 756, that is, baseband image data, at a predetermined timing. To output.

  For example, when encoding is performed in a reversible manner in the encoding device, the wavelet inverse transformation unit 719 selects the integer type wavelet inverse transformation unit 754 based on the control of the control unit 711 and performs the integer type wavelet inverse transformation. The unit 754 is used to perform synthesis filter processing. The wavelet inverse transformation unit 719 connects the integer type wavelet inverse transformation unit 754 and the buffer unit 757 to the switching unit 756 based on the control of the control unit 711, and the output of the integer type wavelet inverse transformation unit 754 receives the buffer unit. 757 is stored.

  Further, for example, when encoding is performed by an irreversible method in the encoding device, the wavelet inverse transform unit 719 selects the fixed-point inverse wavelet transform unit 755 based on the control of the control unit 711 and fixes it. The synthesis filter process is performed using the decimal type inverse wavelet transform unit 755. The wavelet inverse transform unit 719 connects the fixed-point inverse wavelet transform unit 755 and the buffer unit 757 to the switching unit 756 based on the control of the control unit 711, and the output of the fixed-point inverse wavelet transform unit 755 is output. The data is stored in the buffer unit 757.

  As described above, the wavelet inverse transform unit 719 can easily perform the synthesis filter processing by an appropriate method according to the analysis filter processing method on the encoding device side.

  Next, an example of the flow of decoding processing executed by the decoding device 700 will be described with reference to the flowchart of FIG.

  When the decoding process is started, the receiving unit 701 of the decoding device 700 receives a packet in step S701. In step S702, the packet analysis unit 702 analyzes the packet. In step S703, the packet analysis unit 702 determines whether the packet includes a picture header (information on encoding). In addition, when the information regarding encoding is contained in another header, the packet analysis part 702 determines whether the header information is included.

  If it is determined that the picture header is included, the packet analysis unit 702 advances the process to step S704. In step S704, the header analysis unit 703 analyzes the picture header, creates control information, and sets the decoding unit 704. Details of this processing will be described with reference to the flowchart of FIG. When the process of step S704 ends, the header analysis unit 703 advances the process to step S705.

  If it is determined in step S703 that the picture header is not included in the packet, the packet analysis unit 702 omits the process in step S704 and advances the process to step S705.

  In step S705, the decoding unit 704 determines whether or not a predetermined amount of encoded data has been accumulated in the entropy decoding unit 712 by the above-described reception and analysis of the packet. Returning to S701, the packet acquisition and analysis are continued. If it is determined in step S705 that a predetermined amount of encoded data has been obtained, the decoding unit 704 advances the processing to step S706 and executes the encoded data decoding process. Details of this decoding process will be described later with reference to the flowchart of FIG.

  In step S707, the receiving unit 701 determines whether or not to end the decoding process. If it is determined not to end the process, the receiving unit 701 returns the process to step S701 and repeats the subsequent processes. If it is determined in step S707 that the decoding process is to be ended, the reception unit 701 ends the reception of the packet and ends the decoding process.

  Next, an example of the flow of the picture header analysis process executed in step S704 in FIG. 36 will be described with reference to the flowchart in FIG.

  In step S721, the header analysis unit 703 refers to the reversible lossy identification information 532 of the picture header 530. In step S722, the header analysis unit 703 determines whether or not the encoding method of the encoded data is the reversible method based on the reversible lossy identification information 532. Proceed to step S723.

  In step S723, the header analysis unit 703 controls the switching unit 751 and the switching unit 753 so as not to perform gain adjustment. In practice, the header analysis unit 703 only creates control information. Actual control is performed by the control unit 711 based on the control information created by the header analysis unit 703. The same applies to the following.

  In step S724, the header analysis unit 703 controls the switching unit 756 to select the integer wavelet inverse transform unit 754, and activates the integer wavelet inverse transform unit 754.

  If it is determined in step S722 that the encoding method is the irreversible method based on the reversible irreversible identification information 532, the header analysis unit 703 advances the process to step S725.

  In step S725, the header analysis unit 703 controls the switching unit 751 and the switching unit 753 to perform gain adjustment. In step S726, the header analysis unit 703 controls the switching unit 756 to select the fixed-point wavelet inverse transform unit 755. The fixed-point wavelet inverse transform unit 755 is activated.

  When the process of step S724 or step S726 ends, the header analysis unit 703 selects a filter of the type indicated in the filter type information 533 in step S727. This is a case where the filter used by the integer type wavelet inverse transform unit 754 or the fixed-point type wavelet inverse transform unit 755 (the activated wavelet inverse transform unit) can be selected. The integer type wavelet inverse transform unit 754 can be selected. Alternatively, when the fixed-point wavelet inverse transform unit 755 (activated wavelet inverse transform unit) can use only one filter, this process is omitted.

  In step S728, the header analysis unit 703 refers to the coefficient rearrangement information 535 of the picture header 530. In step S729, the header analysis unit 703 determines whether to rearrange the coefficient data, and determines that the coefficient data is rearranged. If so, the process proceeds to step S730.

  In step S730, the header analysis unit 703 controls the switching unit 713 and the switching unit 716 to perform rearrangement, and activates the coefficient rearranging unit 715 and the coefficient rearranging buffer unit 714. If it is determined in step S729 that the coefficient data is not rearranged, the header analysis unit 703 advances the processing to step S731, and controls the switching unit 713 and the switching unit 716 so that the rearrangement is not performed.

  In step S732, the header analysis unit 703 reflects the processing unit indicated in the processing unit number precinct information 534 in the setting of entropy decoding. Further, in step S733, the header analysis unit 703 reflects other information included in the picture header 530 in various settings as necessary, ends the picture header analysis process, and returns the process to step S704 in FIG. The subsequent processing is executed.

  As described above, the header analysis unit 703 analyzes the picture header (information related to encoding) supplied from the encoding device, and creates control information so that decoding settings are appropriately set according to the encoding settings. . Accordingly, the decoding apparatus 700 can easily perform decoding processing by an appropriate method according to the encoding setting, suppress unnecessary data deterioration and occurrence of delay time, and transmit data with high quality and low delay. It can be performed.

  Next, an example of the flow of the decoding process executed by the decoding unit 704 in step S706 of FIG. 36 will be described with reference to the flowchart of FIG.

  In step S751, the entropy decoding unit 712 performs entropy decoding on the encoded data. In step S752, the coefficient rearranging unit 715 determines whether to rearrange the coefficients. If it is determined to rearrange, the coefficient rearranging unit 715 advances the process to step S753 and rearranges the coefficient data. If it is determined in step S752 that the coefficients are not rearranged, the coefficient rearranging unit 715 omits the process of step S753.

  In step S754, the gain adjustment unit 717 determines whether or not to perform gain adjustment. If it is determined that gain adjustment is to be performed, the process proceeds to step S755 to adjust the gain of the high frequency component. If it is determined in step S754 that the gain adjustment is not performed, the gain adjustment unit 717 omits the process in step S755.

  In step S756, the wavelet inverse transform unit 719 performs vertical synthesis filtering by the reversible method or the irreversible method using either the integer wavelet inverse transform unit 754 or the fixed-point wavelet inverse transform unit 755. In step S757, the wavelet inverse transform unit 719 performs horizontal synthesis filtering by the reversible method or the irreversible method using either the integer wavelet inverse transform unit 754 or the fixed-point wavelet inverse transform unit 755.

  In step S758, the wavelet inverse transform unit 719 determines whether or not the synthesis filter processing has been performed up to level 1. If it is determined that the synthesis filter processing of level 1 has not been completed, the processing returns to step S756, and the next The synthesis filter processing of the level is performed. If it is determined in step S758 that the level 1 synthesis filter processing has been completed, the wavelet inverse transform unit 719 advances the processing to step S759. In step S759, the decoding unit 704 determines whether to end the decoding process. If it is determined that the encoded data continues to be supplied and the decoding process is not terminated, the decoding unit 704 returns the process to step S751 and repeats the subsequent processes. If it is determined in step S759 that the decoding process is to be ended, the decoding unit 704 ends the decoding process.

  The decoding unit 704 executes such decoding processing for each picture.

  Note that in the case of the wavelet inverse transform unit 719, as in the case of the wavelet transform unit 611, a part of the configuration of the integer type wavelet inverse transform unit 754 and the fixed-point type wavelet inverse transform unit 755 can be shared.

  FIG. 39 is a block diagram illustrating another configuration example of the synthesis filter processing unit 732. In FIG. 39, the wavelet transform unit 719 includes a wavelet inverse transformer 764, a switching unit 765, a bit shifter 766, a switching unit 767, and a buffer unit 768 that are common to the reversible method and the irreversible method. The wavelet inverse transform unit 719 shares a part other than the bit accuracy of the synthesis filter process between the reversible method and the irreversible method. That is, in the case of the irreversible method, the wavelet inverse transformer 764 performs synthesis filter processing with fixed-point precision, so that the wavelet inverse transform unit 719 sends the bit to the switching unit 765 and the switching unit 766 according to the control of the control unit 711. The shifter 766 switches the connection so that the data is bit-shifted in the direction opposite to that at the time of encoding.

  On the contrary, in the case of the reversible method, the wavelet inverse transformer 764 performs synthesis filter processing with integer precision, so that the wavelet inverse transformer 719 transfers data to the switching unit 765 and the switching unit 766 according to the control of the control unit 711. The connection is switched so as not to go through the bit shifter 766.

  The buffer unit 768 temporarily holds the baseband image data supplied from the switching unit 767 and outputs it at a predetermined timing.

  40 and 41 show an example of a hardware configuration for calculating the coefficients of the high-frequency and low-frequency components on the synthesis side. First, a configuration for calculating the coefficient value or pixel value (in the case of level 0) of the low frequency component will be described with reference to FIG. Here, for convenience of explanation, three registers Rm, Rn, and Rd, adders 771, 772, and 775 respectively indicated by +, +2, and − in the figure, a multiplier 773 that performs an operation of × 0.25, and a floor And a rounding device 774 shown in FIG. Each of the Rm, Rn, and Rd registers is configured to store, for example, four values x, y, z, and w, and the predetermined filtering described above is performed on the coefficient values stored in the registers. By performing the above calculation, the coefficient value or pixel value of one lower level (in the case of level 0) is calculated.

That is, four values from the register Rm (for example, d ₀ ¹ , d ₀ ¹ , d ₁ ¹ , d ₂ ¹ ) and four values from the register Rn (for example, d ₀ ¹ , d ₁ ¹ , d ₂ ¹ , d ₃ ¹ ) are respectively added by the adder 771, 2 is added to each addition result by the adder 772, and 0.25 is respectively added to the four output values from the adder 772 by the multiplier 773. The four values obtained by multiplication (or 2-bit right shift) and rounding by the rounder 774 are sent to the adder 775, and these four values are sent to the four values (for example, s ₀₎ from the register Rd. ¹ , s ₁ ¹ , s ₂ ¹ , s ₃ ¹ ), respectively, by subtracting the coefficient value or pixel value (for example, s ₀ ⁰ , s ₁ ⁰ , s ₂ ⁰ , s) ₃ ⁰ ) is calculated.

  For example, if an integer register having a certain bit length is provided, the above-described filtering with integer precision can be executed, and if a fixed-point register having a certain bit length is used, the above-described filtering with fixed point precision can be performed. In addition to the register accuracy, the hardware configuration need not be changed at all. In this way, common hardware components can be realized.

  In the configuration of the specific example of FIG. 40, the low frequency component is stored in the register Rd, the high frequency component is stored in the registers Rm and Rn, and the operation result is newly stored in the register Rd. By doing so, the coefficient that has been used and overwritten is overwritten, so that it is not necessary to use an extra register, which is efficient. It also contributes to hardware reduction.

40 is a specific example of the rounding process of the rounding device 774 indicated by floor in FIG. 40. The truncation by floor is defined to perform the process of rounding to the smaller integer as a value, not truncation after the decimal point. . According to this definition, for example,
+2.3 → +2 (positive value)
-2.3 → -3 (negative value)
become.

  Next, a configuration for calculating the coefficient value or pixel value (in the case of level 0) of the high frequency component will be described with reference to FIG. Here, for convenience of explanation, there are provided three registers Rm, Rn, and Rd, adders 781 and 785, a multiplier 782 that performs an operation of × 0.5, and a rounder 783 indicated by a floor. It shall be configured. Each of the Rm, Rn, and Rd registers is configured to store, for example, four values x, y, z, and w, and the predetermined filtering described above is performed on the coefficient values stored in the registers. By performing the above calculation, the coefficient value or pixel value of one lower level (in the case of level 0) is calculated.

That is, four values from the register Rn (for example, s ₀ ⁰ , s ₁ ⁰ , s ₂ ⁰ , s ₃ ⁰ ) and four values from the register Rm (for example, s ₁ ⁰ , s ₂ ⁰ , s ₃ ⁰ , s ₄ ⁰ ) are respectively added by an adder 781, 0.5 is multiplied (or 1-bit right shifted) by a multiplier 782 for each addition result, and a rounding process is performed by a rounder 783. The four values are sent to the adder 784, and these four values are subtracted from the four values (eg, d ₀ ¹ , d ₀ ¹ , d ₁ ¹ , d ₂ ¹ ) from the register Rd. The coefficient value or pixel value (for example, d ₀ ⁰ , d ₁ ⁰ , d ₂ ⁰ , d ₃ ⁰ ) of the high-frequency component at the lower level is calculated. Here, of the four values (d ₀ ¹ , d ₀ ¹ , d ₁ ¹ , d ₂ ¹ ) of the register Rd, two d ₀ ¹ are used as described above. This is because the data located outside the screen is supplemented by the adjacent data inside the screen (see d ₀ ^{1 in} FIG. 11 and a broken-line circle that is turned back).

  In the specific example of FIG. 41, as in the specific example of FIG. 40, the hardware configuration is completely different only by using each register Rm, Rn, Rd as an integer register or a fixed-point register. The configuration of the integer type synthesis filter and the fixed-point type synthesis filter can be made common without changing. In addition, the rounding process of the rounder 783 indicated by floor in FIG. 41 is also defined so as to perform the process of rounding to the smaller integer as a value instead of rounding down after the decimal point, similarly to the rounder 774 of FIG. Keep it. In the specific example of FIG. 41, the low frequency components are stored in the registers Rm and Rn, the high frequency components are stored in the register Rd, and the calculation result is newly stored in the Rd register. In this way, since the coefficient already used and overwritten is overwritten, it is not necessary to use an extra register, which is efficient. It also contributes to hardware reduction. It is self-evident that there are other uses for these registers. If the register is large, the amount of data that can be stored increases. Therefore, the amount that can be simultaneously filtered increases, which can contribute to speeding up the processing. .

  Incidentally, the multipliers 773 and 782 shown in FIGS. 40 and 41 can be replaced by shift arithmetic units. Specifically, it is clear that the × 0.25 multiplier 773 in FIG. 40 corresponds to a right 2-bit shift, and the × 0.5 multiplier 782 in FIG. 41 corresponds to a right 1-bit shift. In general, it is known that the shift calculator is less hardware than the multiplier, and is more efficient.

  Even when a part of the structure of the wavelet inverse transform unit is shared between the reversible method and the irreversible method as shown in FIG. 39, the method selection (switching unit switching, etc.) and the decoding processing method are basically the same. Since it is similar to the case of FIG. 35, its description is omitted.

  In the case of the irreversible method, the encoding apparatus 600 may be able to select whether or not to perform quantization processing. The method may be the same as that of the rearrangement processing unit 622 described with reference to FIG. Information indicating whether or not quantization processing has been performed may be embedded in the picture header 530 as in the case of coefficient rearrangement. Similarly, in the case of the irreversible method, the decoding apparatus 700 may be able to select whether or not to perform inverse quantization processing. The method may be the same as that of the rearrangement processing unit 731 described with reference to FIG. The decoding device may refer to the picture header and perform inverse quantization when quantization processing is performed in the encoding device.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium into a general-purpose personal computer or an information processing apparatus of an information processing system including a plurality of apparatuses.

  FIG. 42 is a block diagram illustrating an example of a configuration of an information processing system that executes the above-described series of processing by a program.

  As shown in FIG. 42, an information processing system 800 includes an information processing device 801, a storage device 803 connected to the information processing device 801 by a PCI bus 802, and a VTR 804- that is a plurality of video tape recorders (VTRs). 1 to VTR 804 -S, a system comprising a mouse 805, a keyboard 806, and an operation controller 807 for a user to perform an operation input thereto, and an image encoding process and an image as described above are performed by an installed program. This is a system that performs a decoding process and the like.

  For example, the information processing apparatus 801 of the information processing system 800 stores encoded data obtained by encoding moving image content stored in a large-capacity storage device 803 configured by RAID (Redundant Arrays of Independent Disks) in the storage device 803. The decoded image data (moving image content) obtained by storing or decoding the encoded data stored in the storage device 803 is stored in the storage device 803, or the encoded data and decoded image data are stored in the VTRs 804-1 to 8044-1. It can be recorded on a video tape via the VTR804-S. The information processing apparatus 801 is also configured to be able to import the moving image content recorded on the video tape mounted on the VTRs 804-1 to VTR804-S into the storage device 803. At that time, the information processing apparatus 801 may encode the moving image content.

  The information processing apparatus 801 includes a microprocessor 901, a GPU (Graphics Processing Unit) 902, an XDR (Extreme Data Rate) -RAM 903, a south bridge 904, an HDD (Hard Disk Drive) 905, a USB interface (USB I / F) 906, and A sound input / output codec 907 is provided.

  The GPU 902 is connected to the microprocessor 901 via a dedicated bus 911. The XDR-RAM 903 is connected to the microprocessor 901 via a dedicated bus 912. The south bridge 904 is connected to the I / O controller 944 of the microprocessor 901 via a dedicated bus. An HDD 905, a USB interface 906, and a sound input / output codec 907 are also connected to the south bridge 904. A speaker 921 is connected to the sound input / output codec 907. A display 922 is connected to the GPU 902.

  The south bridge 904 is further connected to a mouse 805, a keyboard 806, VTR 804-1 to VTR 804 -S, a storage device 803, and an operation controller 807 via a PCI bus 802.

  The mouse 805 and the keyboard 806 receive a user operation input, and supply a signal indicating the content of the user operation input to the microprocessor 901 via the PCI bus 802 and the south bridge 904. The storage device 803 and the VTRs 804-1 to VTR804-S can record or reproduce predetermined data.

  Further, a drive 808 is connected to the PCI bus 802 as necessary, and a removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is necessary. Is installed in the HDD 905 accordingly.

  The microprocessor 901 includes a general-purpose main CPU core 941 that executes a basic program such as an OS (Operating System), and a plurality of (in this case, eight) RISC (Reduced) connected to the main CPU core 941 via an internal bus 945. An instruction set computer) type signal processor, a sub CPU core 942-1 to a sub CPU core 942-8, a memory controller 943 for performing memory control on an XDR-RAM 903 having a capacity of, for example, 256 [MByte], and a south A multi-core configuration in which an I / O (In / Out) controller 944 that manages input / output of data to / from the bridge 904 is integrated on one chip, and realizes an operating frequency of 4 [GHz], for example.

  At startup, the microprocessor 901 reads a necessary application program stored in the HDD 905 based on a control program stored in the HDD 905 and develops it in the XDR-RAM 903. Thereafter, the microprocessor 901 executes this application program and an operator operation. Perform the necessary control processing.

  Further, the microprocessor 901 realizes the above-described encoding process and decoding process, for example, by executing software, and supplies the encoded stream obtained as a result of encoding to the HDD 905 via the south bridge 904. The playback video of the moving image content obtained as a result of decoding can be transferred to the GPU 902 and displayed on the display 922.

  The usage method of each CPU core in the microprocessor 901 is arbitrary. For example, the main CPU core 941 performs processing related to control of image encoding processing and image decoding processing, and includes eight sub CPU cores 942-1 to 942-1. Each processing such as wavelet transform, coefficient rearrangement, entropy coding, entropy decoding, inverse wavelet transform, quantization, and inverse quantization in the sub CPU core 942-8 is described with reference to FIG. 39, for example. It may be executed simultaneously in parallel. At this time, if the main CPU core 941 allocates processing in units of precincts to each of the eight sub CPU cores 942-1 to 942-8, the encoding processing and decoding processing are performed as described above. As described above, it is executed in parallel in units of precincts. That is, it is possible to improve the efficiency of the encoding process and the decoding process, shorten the delay time of the entire process, and further reduce the load, the processing time, and the memory capacity required for the process. Of course, each process may be performed by other methods.

  For example, a part of the eight sub CPU cores 942-1 to 942-8 of the microprocessor 901 performs the encoding process and the other part executes the decoding process simultaneously in parallel. Is also possible.

  For example, when an independent encoder or decoder or a codec processing device is connected to the PCI bus 802, the eight sub CPU cores 942-1 to 942-8 of the microprocessor 901 are connected to the south. The processing executed by these devices may be controlled via the bridge 904 and the PCI bus 802. Further, when a plurality of these devices are connected, or when these devices include a plurality of decoders or encoders, the eight sub CPU cores 942-1 to 942-8 of the microprocessor 901 are displayed. May share and control processes executed by a plurality of decoders or encoders.

  At this time, the main CPU core 941 manages the operations of the eight sub CPU cores 942-1 to 942-8, assigns processing to each sub CPU core, and collects processing results. Further, the main CPU core 941 performs processes other than those performed by these sub CPU cores. For example, the main CPU core 941 accepts commands supplied from the mouse 805, the keyboard 806, or the operation controller 807 via the south bridge 904, and executes various processes according to the commands.

  In addition to the final rendering processing related to texture embedding when moving the playback video of the moving image content displayed on the display 922, the GPU 902 displays a plurality of playback images of the moving image content and still images of the still image content on the display 922 at a time. It controls coordinate transformation calculation processing for display, enlargement / reduction processing for a playback image of moving image content and a still image of a still image content, and the like, thereby reducing the processing load on the microprocessor 901.

  Under the control of the microprocessor 901, the GPU 902 performs predetermined signal processing on the supplied video data of moving image content and image data of still image content, and displays the obtained video data and image data as a result. The image signal is sent to 922 and the image signal is displayed on the display 922.

  For example, reproduced videos of a plurality of moving image contents decoded in parallel in parallel by the eight sub CPU cores 942-1 to 942-8 in the microprocessor 901 are transferred to the GPU 902 via the bus 911. However, the transfer rate at this time is, for example, a maximum of 30 [Gbyte / sec], and even a complex playback video with a special effect can be displayed at high speed and smoothly.

  Further, the microprocessor 901 performs audio mixing processing on the audio data among the video data and audio data of the moving image content, and the edited audio data obtained as a result is sent via the south bridge 904 and the sound input / output codec 907. By sending the sound to the speaker 921, sound based on the sound signal can be output from the speaker 921.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIG. 42, the recording medium is distributed to distribute the program to the user separately from the apparatus main body, and includes a magnetic disk (including a flexible disk) on which the program is recorded, an optical disk ( In addition to being constituted by a removable medium 811 made up of a CD-ROM, DVD), a magneto-optical disk (including MD), or a semiconductor memory, etc., it is delivered to the user in a state of being pre-installed in the apparatus body. It is composed of an HDD 905 in which a program is recorded, a storage device 803, and the like. Of course, the recording medium may be a semiconductor memory such as a ROM or a flash memory.

In the above description, eight sub CPU cores are configured in the microprocessor 901. However, the present invention is not limited to this, and the number of sub CPU cores is arbitrary. Further, the microprocessor 901 does not have to be configured by a plurality of cores such as a main CPU core and a sub CPU core, and a CPU configured by a single core (one core) may be used.
Further, a plurality of CPUs may be used in place of the microprocessor 901, or a plurality of information processing devices are used (that is, a program for executing the processing of the present invention is executed in a plurality of devices operating in cooperation with each other). Execute).

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus composed of a plurality of devices (apparatuses).

  In the above, the configuration described as one device may be divided and configured as a plurality of devices. Conversely, the configurations described above as a plurality of devices may be combined into a single device. Of course, configurations other than those described above may be added to the configuration of each device. Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device may be included in the configuration of another device.

  The present invention described above is a device or system that performs data transmission with high quality and low delay, transmits an image after compression encoding, and decodes and outputs the compression code at the transmission destination. It can be applied to various things. The present invention is particularly suitable for use in an apparatus or system that requires a short delay from image compression encoding to decoding and output.

  For example, the present invention is suitable for use in medical telemedicine diagnosis in which a magic hand is operated while performing a therapeutic action while watching an image taken with a video camera. Further, the present invention is suitable for use in a system in which an image is encoded and transmitted and decoded and displayed or recorded in a broadcasting station or the like.

  Furthermore, the present invention can be applied to a system that distributes live-streamed video, a system that enables interactive communication between students and teachers in an educational setting, and the like.

  Furthermore, for transmission of image data taken by a mobile terminal having an imaging function, such as a mobile phone terminal with a camera function, a system using a video conference system, a surveillance camera, and a recorder that records video taken by the surveillance camera. The present invention can be applied.

It is a block diagram which shows the structural example of an encoding apparatus. It is a schematic diagram which shows the example of input data of integer precision. It is a figure which shows typically the structural example of the calculation of the wavelet transformation with respect to image data. It is an approximate line figure for explaining wavelet transform roughly. It is an approximate line figure for explaining wavelet transform roughly. It is a figure for demonstrating the calculation of an integer type 5x3 analysis filter. It is a basic diagram which shows the example of the mode of the lifting of a 5x3 filter. It is a basic diagram which shows roughly the flow of a wavelet transformation and a wavelet inverse transformation. It is a block diagram which shows the structural example of a decoding apparatus. It is a figure which shows typically the structural example of the calculation of the wavelet inverse transformation with respect to encoding data. It is a figure for demonstrating the calculation of an integer type 5x3 synthetic | combination filter. It is a flowchart for demonstrating the example of the flow of an encoding process. It is a flowchart for demonstrating the example of the flow of a decoding process. It is a block diagram which shows the other structural example of an encoding apparatus. It is a block diagram which shows the detailed structural example of the analysis filter process part of FIG. It is a schematic diagram which shows the example of input data of fixed point precision. It is a figure for demonstrating the calculation of a fixed point type 5x3 analysis filter. It is a block diagram which shows the other structural example of a decoding apparatus. It is a block diagram which shows the detailed structural example of a synthetic | combination filter process part. It is a figure for demonstrating the calculation of a fixed point type 5x3 synthetic | combination filter. It is a flowchart for demonstrating the other example of the flow of an encoding process. It is a flowchart for demonstrating the other example of the flow of a decoding process. It is a basic diagram which shows roughly the example of the mode of the parallel operation | movement which each element of an encoding part and a decoding part performs. It is a block diagram which shows the further another structural example of an encoding apparatus. It is a schematic diagram which shows the structural example of a picture header. It is a flowchart for demonstrating the further another example of the flow of an encoding process. It is a flowchart for demonstrating the further another example of the flow of an encoding process. It is a flowchart for demonstrating the example of the flow of a picture header production | generation process. It is a block diagram which shows the further another structural example of an encoding apparatus. It is a block diagram which shows the detailed structural example of the analysis filter process part of FIG. It is a flowchart for demonstrating the further another example of the flow of an encoding process. It is a flowchart for demonstrating the example of the flow of a setting process. FIG. 30 is a block diagram illustrating another detailed configuration example of the analysis filter processing unit in FIG. 29. It is a block diagram which shows the further another structural example of a decoding apparatus. It is a block diagram which shows the detailed structural example of the synthetic | combination filter process part of FIG. It is a flowchart for demonstrating the further another example of the flow of a decoding process. It is a flowchart for demonstrating the example of the flow of a picture header analysis process. It is a flowchart for demonstrating the further another example of the flow of a decoding process. FIG. 35 is a block diagram illustrating another detailed configuration example of the synthesis filter processing unit in FIG. 34. It is a figure which shows an example of the hardware constitutions for calculating the coefficient or pixel value of the synthetic | combination low-frequency component. It is a figure which shows an example of the hardware constitutions for calculating the coefficient or pixel value (in the case of level 0) of the high frequency component by the side of composition. It is a figure which shows the structural example of the information processing system to which this invention is applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 encoding apparatus, 101 wavelet transformation part, 102 intermediate calculation buffer part, 103 entropy encoding part, 111 analysis filter processing part, 200 decoding apparatus, 201 entropy decoding part, 202 coefficient rearrangement part, 203 coefficient buffer part, 204 Wavelet inverse transform unit, 211 synthesis filter processing unit, 300 encoding device, 321 bit shifter, 322 wavelet transformer, 400 decoding device, 421 wavelet inverse transform unit, 422 bit shifter, 500 encoding device, 501 control unit, 502 encoding unit, 503 header generating unit, 504 packetizing unit, 505 transmitting unit, 513 coefficient rearranging buffer unit, 514 coefficient rearranging unit, 600 encoding device, 601 control unit, 60 Coding section, 603 header generation section, 604 packetization section, 605 transmission section, 611 wavelet transform section, 613 switching section, 616 switching section, 622 rearrangement processing section, 623 coding processing section, 632 switching section, 633 integer type Wavelet transform unit, 634 fixed-point wavelet transform unit, 635 switching unit, 542 switching unit, 644 switching unit, 700 decoding device, 701 receiving unit, 702 packet analysis unit, 703 header analysis unit, 704 decoding unit, 713 switching unit, 716 switching unit, 717 gain adjustment unit, 731 rearrangement processing unit, 732 synthesis filter processing unit, 751 switching unit, 752 high-frequency component gain adjustment unit, 753 switching unit, 754 integer wavelet inverse transformation unit, 755 fixed-point type c Over wavelet inverse transformation unit, 756 switching unit, 765 switching unit, 767 switching unit

Claims (10)

Analysis that decomposes image data for each frequency band in a reversible manner that can restore the original image data from the converted coefficient data by reverse conversion, and generates subbands composed of the coefficient data for each frequency band Analysis filter means for performing filtering for each line block including the image data for the number of lines necessary to generate coefficient data for one line of at least the subband of the lowest frequency component;
The coefficient data generated by the analysis filter processing by the analysis filter means can be restored to the original coefficient data from the encoded data by decoding processing for each coefficient data group generated from a plurality of the line blocks. An information processing apparatus comprising: encoding means for encoding by a reversible method. The encoding including information indicating that at least the encoded data is generated by encoding in a reversible method for each predetermined data unit with respect to the encoded data obtained by encoding by the encoding means The information processing apparatus according to claim 1, further comprising a generation unit configured to generate information regarding data. The information processing apparatus according to claim 2, wherein the generation unit also generates information about a type of the analysis filter process executed by the analysis filter unit as information about the encoded data. The generation unit also generates information on the number of the line blocks necessary for generating the coefficient data group that is used as the processing unit of the encoding by the encoding unit as information on the encoded data. The information processing apparatus described in 1. The coefficient data generated by the analysis filter processing by the analysis filter means is combined with the coefficient data of each subband for each line block in order of executing synthesis filter processing for restoring the original image data. It further comprises a sorting means for sorting,
The information processing apparatus according to claim 1, wherein the encoding unit encodes the coefficient data in the order in which the coefficient data group generated from the plurality of line blocks is rearranged by the rearrangement unit. For the encoded data obtained by encoding by the encoding means, at least information indicating that the encoded data has been generated by encoding in a reversible method for each predetermined data unit, and the arrangement The information processing apparatus according to claim 5, further comprising a generation unit that generates information regarding the encoded data including information regarding rearrangement of the coefficient data by a replacement unit. The analysis filter means further decomposes the image data for each frequency band by an irreversible method in which restoration of the original image data by reverse conversion is not guaranteed, and from the coefficient data for each frequency band An analysis filtering process for generating a subband is performed for each line block including the image data for the number of lines necessary to generate at least one line of coefficient data of the subband of the lowest frequency component,
The information processing apparatus according to claim 1, further comprising a control unit that controls whether the analysis filter processing by the analysis filter unit is performed in a reversible method or a non-reversible method. In the information processing method of the information processing apparatus,
The information processing apparatus includes:
Analysis filter means;
Encoding means, and
The analysis filter means decomposes the image data for each frequency band by a reversible method capable of restoring the original image data from the converted coefficient data by reverse conversion, and includes the coefficient data for each frequency band. Analysis filter processing for generating subbands is performed for each line block including the image data for the number of lines necessary to generate coefficient data for one line of at least the subband of the lowest frequency component,
The encoding means can restore the original coefficient data from the encoded data by decoding the coefficient data generated by the analysis filter process for each coefficient data group generated from a plurality of the line blocks. Information processing method that encodes in a reversible manner. Whether or not the encoded data obtained by encoding the image data is analyzed, and the encoded data is generated by a reversible encoding process capable of restoring the data before encoding by the decoding process. An analysis means for identifying
As a result of analysis by the analysis means, when it is determined that the encoded data is generated by the reversible encoding process, a method corresponding to the encoding process for the encoded data Decryption means for performing decryption processing in
An information processing apparatus comprising: synthesis filter means for synthesizing the coefficient data generated from the encoded data by the decoding processing by the decoding means to generate the image data. In the information processing method of the information processing apparatus,
The information processing apparatus includes:
Analysis means;
Decryption means;
A synthesis filter means, and
The analysis unit analyzes information related to encoded data obtained by encoding image data, and the encoded data is generated by a reversible method encoding process capable of restoring data before encoding by a decoding process. Whether it ’s a thing,
When the decoding means determines that the encoded data is generated by the reversible encoding process as a result of the analysis, the decoding unit corresponds to the encoding process for the encoded data. Decryption with the method
An information processing method in which the synthesis filter unit generates the image data by synthesizing the coefficient data generated from the encoded data by a decoding process.

Images