JP2002094991A - Concerned area encoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a concerned area encoding method that can enhance a compression rate of an entire image in the case of assigning a coding bit rate to a concerned area of an optional shape. SOLUTION: A one or plural concerned areas are designated as a mask area and a coding quantity is assigned to each mask area and each non mask area. In this case, the code quantity assigned to the mask area is set to a value not reaching a value required for reversible compression. Applying nonreversible compression to the concerned area can enhance the compression rate.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention compares the image quality of a region of interest with that of another region by preferentially assigning a coding bit rate to a specific region of interest such as a person or an object in the entire image. And a method of coding a region of interest that keeps it high.

[0002]

2. Description of the Related Art As a method for efficiently encoding an image,
By preferentially assigning a coding bit rate to a specific region of interest such as a person or an object in the entire image, the image quality of the region of interest is kept higher than that of other regions.
There is a method of dividing into OI (Region of Interest) and non-ROI of insignificant information such as background.

As a typical example of this system, Maxshi
ft method (first conventional example) and EB as bit plane method
COT (Embedded Block Coding with Optimized Trunc
ation: second conventional example).

[0004] The Maxshift method (first conventional example) uses R
The OI part is specified in an arbitrary form, and the part is losslessly compressed, while the non-ROI part is lossy compressed. Specifically, first, a known wavelet transform is performed on the original image to obtain a distribution of wavelet coefficients as shown in FIG. 13, and among these distributions, the largest of the coefficient distributions conceived in the non-ROI portion is obtained. The value Vm of the wavelet coefficient is obtained in advance. Then, the number of bits S that satisfies S> = max (Vm) is obtained, and only the wavelet coefficients of the ROI portion Ar1 are shifted by S bits in the increasing direction as shown in FIG. For example, the value of Vm is "255" in decimal.
(Ie, “11111111” in binary), S = 8 bits, and the value of Vm is “1” in decimal.
Similarly, S = 8 bits in the case of "28" (that is, "10000000" in binary). In this case, the wavelet coefficient of the ROI part Ar1 is increased in the direction of increasing S = 8 as shown in FIG. It will be shifted by bits. As a result, the non-RO
The compression ratio can be set lower than that of the I part,
It is possible to obtain reversibly compressed data for the I part Ar1. According to this method, on the decoding side,
This is convenient because it is not necessary to obtain definition information such as the shape of the ROI part in advance, and the ROI part can be reversibly decoded just by decoding it.

In the EBCOT (second conventional example) of the bit plane method, as shown in FIG. 15, an image is divided into a plurality of rectangular blocks B, and the rectangular blocks B are compressed with a priority of a bit rate for each rectangular block B. Things. Therefore, E
According to BCOT, image compression at a higher bit rate can be performed for some rectangular blocks B than for other rectangular blocks B.

[0006]

In the Maxshift method (first conventional example), since the lossless compression is performed in the ROI part, there is a certain limit in the compression ratio of that part.
The size of the compressed image file had to be relatively large as a whole. Conversely, Maxshi
In order to increase the compression ratio in the ft method and reduce the size of the compressed image file, the bit rate of the non-ROI part must be significantly reduced, and thus the image quality of the non-ROI part is lower than that of the ROI part. Would be extremely poor.

On the other hand, in EBCOT, since the bit rate is changed for each rectangular block B, definition information such as block coordinates, size, and boundary is required in advance on the decoding side.

In EBCOT, since the ROI portion is specified only as the rectangular block B, the ROI portion of an arbitrary shape cannot be defined. For example, the bit rate of only the face of a person in the background is increased. In such a case, the processing must be extremely difficult.

It is an object of the present invention to efficiently encode an image by irreversibly compressing an ROI portion having an arbitrary shape, and to preliminarily determine the shape and size of the ROI portion on the decoding side. It is an object of the present invention to provide a region-of-interest coding method that does not require the acquisition of definition information such as

[0010]

Means for Solving the Problems In order to solve the above problems,
The invention according to claim 1 is a first step of performing a wavelet transform on an original image, and a second step of developing a mask region of an arbitrary shape specified for the original image on a wavelet plane corresponding to the wavelet transform. And a third step of assigning a code amount when compressing the mask area and a non-mask area other than the mask area, and performing quantization and encoding according to the code amount assigned in the third step. And a fourth step of performing, wherein in the third step, the code amount when compressing the mask area is set to a value less than a value necessary for lossless compression.

According to a second aspect of the present invention, there is provided the method of encoding a region of interest according to the first aspect, wherein the fourth step includes separately performing the quantization and the encoding on the mask region and the non-mask region. And generates compressed data for each.

According to a third aspect of the present invention, in the method of encoding a region of interest according to the first aspect, a plurality of the mask areas are designated for the original image, and the mask areas are different in the third step. The code amount is individually assigned to the mask area.

According to a fourth aspect of the present invention, there is provided the region-of-interest coding method according to the third aspect, wherein the fourth step includes separately performing the quantization on each of a plurality of the mask regions and the non-mask regions. And the above encoding to generate respective compressed data.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a flowchart showing a method of coding a region of interest according to one embodiment of the present invention. The method of coding the region of interest will be described with reference to this flowchart.

<Compression Method> Image compression is performed according to the flow on the side of the compression device Co in FIG.

As shown in FIG. 1, the compression device Co first performs a wavelet transform on digital image data 1 as an original image. The wavelet transform is a transform method in which an image is divided into a low-pass side and a high-pass side by using a two-divided filter bank and coding and compression are performed. Here, as a representative method of the wavelet transform, the mallat type shown in FIG. 2 and FIG. 3 and the spat type shown in FIG. 4 and FIG.
Cle type and a plurality of types such as the packet type shown in FIGS. 6 and 7. Here, one of these is selected to perform the wavelet transform. FIGS. 2, 4 and 6 show one-dimensional wavelet transform modes, and FIGS. 3, 5 and 7 show two-dimensional wavelet transform modes. However, the development hierarchy of the wavelet transform is not limited to the number of hierarchies shown in each figure, and is arbitrary.

Here, malla shown in FIG. 2 and FIG.
In the t-type, a low-frequency component (low-pass component: see an image represented by a relatively small block in FIG. 2 and a path indicated by “L” in FIG. 3) has a high-frequency component (high-pass component: FIG. This is a method in which only the low-pass filter is repeated under the assumption that it contains more information than the image shown by the relatively large block and the path shown by "H" in FIG. On the other hand, since the wavelet packet base can correspond to an arbitrary binary tree structure, FIG.
In the space type shown in FIG. 5, not only low-frequency components (low-pass components: see the image shown by relatively small blocks in FIG. 4 and the path shown by “L” in FIG. 5) but also high-frequency components The component (high-pass component: see the image indicated by the largest block in FIG. 4 and the path indicated by “H” in the first branch in FIG. 5) also has a lower frequency component (low-pass component: “L”) and a higher frequency component. Only one stage is developed for the frequency component (high-pass component: “H”). FIG.
In the packet type shown in FIG. 7, a frequency component (low-pass component: “L”) and a high frequency component (high-pass component:
"H") to adopt a complete tree configuration. Here, which of these types is selected is limited by a rate determined based on cost and capacity, and a type that minimizes restoration distortion is selected.

Next, in the compression device Co, the important part of the wavelet-transformed image data 2 is represented by R
A mask signal 3 for designating an OI portion is given.
For example, in the image data (original image) 1 of a person as shown in FIG. 8, when only the face portion below the forehead is used as the ROI portion, as shown in a white portion in FIG.
Is set, and this is designated as the ROI part. The mask area 4 can be designated corresponding to the original image 1 using a pointing input device such as a mouse while viewing the original image 1 on the screen of the display device.

FIG. 9 shows a single RO for the original image 1.
In the example in which the I portion is specified, a plurality of regions may be specified as the ROI portion. These are defined by different mask signals 3 respectively. The remaining portion obtained by removing all the mask regions 4 for all the ROI portions becomes the non-ROI portion 5 (FIG. 9).

Next, when a plurality of mask signals 3 are given as shown in FIG. 1, the priorities are assigned to the plurality of mask signals 3. The higher the priority, the more information,
For example, the bit rate is increased, and the loss at the time of decompression is reduced. In this case, the priority is “1”
Designation is performed in ascending numerical order such as “2”.

The type (ma) selected as described above
The mask region 4 is expanded on a wavelet plane to generate a mask signal 3 in accordance with the wavelet transform of the llat type / space type / packet type.

Here, the method of transforming the mask signal into a portion corresponding to the wavelet plane depends on the number of taps of the filter of the wavelet transform.

For example, as shown in FIG. 11, reversible (Reversible) 5.times.
3 filters (the number of taps of the low-pass filter on the decomposition side is 5
If the tap is a filter in which the number of taps of the high-pass filter on the decomposition side is 3 taps), the original image 1
Is specified as the ROI portion, the n-th data of the low-pass filter (low-pass side) 7 and (n-1) of the high-pass filter (high-pass side) 8 ) -Th and n-th data are ROI
As a part, the mask signal is developed on a wavelet plane. If the odd-numbered (2n + 1) -th pixel data of the original image 1 is designated as the ROI portion, the n-th and (n + 1) -th data of the low-pass filter (low-pass side) 7 and the high-pass filter ( Assuming that the (n-1) -th, n-th, and (n + 1) -th data of the (high-frequency side) 8 are ROI portions, the mask signal is developed on the wavelet plane. FIG. 11 shows the original image 1
Only the correspondence between the first hierarchical level and the wavelet plane of the first hierarchical level is shown, but the same recursive expansion is performed for the development of a deeper hierarchical level.

Alternatively, for example, as shown in FIG. 12, in the arithmetic processing of the wavelet transform, Debussy (Daubec) is used.
hies) If a 9 × 7 filter (a filter in which the number of taps of the low-pass filter on the decomposition side is 9 taps and the number of taps of the high-pass filter on the decomposition side is 7 taps) is applied, the even-numbered (2n-th) ) Is R
When it is designated as the OI part, the (n−1) th, nth, and (n +) th of the low-pass filter (low-pass side) 7
1) -th data and (n−2) -th, (n−1) -th, n-th, and (n +) of the high-pass filter (high-pass side) 8
1) Assuming that the data is the ROI part, the mask signal is developed on the wavelet plane. The odd-numbered (2n + 1) th pixel data of the original image 1 is the ROI
If it is specified as a part, the (n−1) th, nth, (n + 1) th and (n + 2) th data of the low-pass filter (low-pass side) 7 and the high-pass filter (high-pass side) 8 (N−2), (n−1), n, (n + 1) and (n + 2) th data of R
The mask signal is developed on the wavelet plane assuming that it is the OI portion. Although FIG. 12 shows only the correspondence between the original image 1 and the wavelet plane of the first layer, the same recursive expansion is performed for expansion of a deeper layer.

The low-pass filter (low-pass side) 7 and the high-pass filter (high-pass side) 8 are shown in FIGS.
Regarding the correspondence relationship of 2, the non-ROI portion corresponding to a certain pixel data of the original image 1 and the ROI portion overlapping with the other pixel data of the original image 1 are:
The mask signal is developed on the wavelet plane assuming that it is the ROI part.

The blank portion 4a in FIG. 10 is a region (hereinafter referred to as a "developed mask region") 4 in which the mask region (ROI portion) 4 is developed on a mallat type wavelet plane as described above. A mask signal 3 corresponding to the developed mask area 4a is generated and given to the wavelet-transformed image data 2. Reference numeral 5a in FIG. 10 indicates a region where the non-ROI portion 5 is developed on the wavelet plane (hereinafter, referred to as a "developed non-mask region"). In a portion where the mask regions (ROI portions) 4 overlap with each other and in a portion where the mask regions (ROI portions) 4 and the non-ROI portions 5 overlap with each other, the higher priority is set as the mask region (ROI portion) 4. , Lower is non-RO
Process as I part 5.

In FIG. 1, the wavelet coefficients of the wavelet-transformed image data 2 are extracted in association with the respective mask signals 3.

Next, as shown in FIG. 10, a priority is assigned to each mask area 4a developed on the wavelet plane. Note that, as described above, a plurality of ROs
When the I portion 4 is set, the mask regions 4a corresponding to each other correspond to the number of developed mask regions 4a developed on the wavelet plane for each of the plurality of ROI portions 4.
Are given the same priority, and finally the priority is set to all the development mask areas 4a.

Here, when a plurality of ROI portions 4 are set for the original image 1, a portion which has passed through a low-pass filter (low-pass side) 7 on a wavelet plane has a plurality of development mask areas 4 a. Can overlap. In this case, the priority is determined for the overlapping portion as the higher priority development mask region 4a among the plurality of overlapping development mask regions 4a.

Then, in the image data 2 subjected to the wavelet transform, the wavelet coefficients of the expanded non-mask area 5a which are not affected by any mask signal are extracted.
In this case, the priority of the development non-mask area 5a is lower than that of all the development mask areas 4a (that is, a larger number is given in ascending numerical order). Adopted.

In this way, the plurality of ROI portions 6a,
The wavelet coefficient 6b is output. In addition, this RO
The output data of the wavelet coefficients of the I parts 6a, 6b,... And the unmasked unmasked area 5a is hereinafter referred to as “ROI signal”.

Next, according to the previously set priority, R
A bit amount is allocated to each of the OI portions 6a and 6b and the developed non-mask area 5a.

As a method of assigning the bit amount at this time, the information amount of the ROI portions 6a and 6b, for example, the bit amount is set to a value smaller than the bit amount necessary for the lossless compression.
Specifically, for example, first, the compression rate bit amount of the entire image is determined, and a predetermined ratio of bit amount is sequentially allocated to the ROI portions 6a, 6b,... A first method of allocating the bit amount to the expanded non-mask area 5a, and a second method of directly determining the bit amount at a predetermined ratio according to the priority of each of the ROI portions 6a, 6b,. However, any one of the methods is selected in advance, and each R is selected according to the selected method.
.. And the unmasked area 5a are determined. At this time, as described above, the ROI portion 6
The bit amounts of a and 6b are set to values that are less than the bit amounts required for lossless compression. Note that a mode in which the bit amount of the ROI portions 6a and 6b is allocated to a bit amount necessary for lossless compression or a mode in which the bit amount is set to a value less than the bit amount required for lossless compression may be selected.

Then, the ROI signals of the ROI portions 6a, 6b... And the unmasked unmasked area 5a are quantized, and the data 10a, 10b.
Generate 0z. In addition, as a method of this quantization processing,
EBCOT and SPIHT (Image Compression with Set
Binarization may be performed by a method similar to a bit plane coding method such as Partitioning in Hierarchical Trees.

Then, each binarized data 1
Entropy coding is performed on 0a, 10b... 10z using a predetermined method such as arithmetic coding such as MQ coder or QM coder or Huffman coding.

By doing so, each ROI portion 6
a, 6b... and compressed data 1 for each of the developed non-mask areas 5a
.. 11z are generated.

Then, each of the compressed data 11a, 11b.
1z are arranged in order according to their priorities and sent out, for example, on a predetermined path or recorded on a predetermined recording medium such as a hard disk drive.

<Decoding Method> Decoding of an image is executed according to the flow on the side of the decompression device Ex in FIG.

In the decompression device Ex, as shown in FIG. 1, each of the compressed data 11a, 11b... 11z is entropy-decoded according to a predetermined method, and is sequentially arranged in an order (that is, the order of the priority of the bit rate). .., 21z are generated.

Next, each of the binarized data 21
.. 21z are inversely quantized (multi-valued) to generate a plurality of ROI signals 22a, 22b. At this time, inverse quantization (multi-level quantization) is performed in the order in which the bits are arranged (that is, in the order of higher priority of the bit rate), and the plurality of ROI signals 22a, 22b,. Generate in the highest order.

Then, on the wavelet plane, the ROI signals 22a, 22b,. At this time, each ROI signal 22a, 22b.
Since the priority order of z is the order in which the data was sent (that is, the order in which the data is arranged), the wavelet coefficients are sequentially synthesized in accordance with the given order to form the wavelet-transformed image data 23. . Note that a plurality of R
If there is an overlap between the OI signals 22a, 22b,..., 22z, the higher priority is selected. In addition, the wavelet coefficient of the non-ROI part 5 is set to “0”.
, The ROI portions 6a, 6b,...
And ROI signals 22a and 22 corresponding to the non-ROI part 5
It is only necessary to add the values of b ... 22z.

Finally, the type of the wavelet transform (mal
In accordance with (lat type / space type / packet type), inverse wavelet transform is performed to restore the image data 24.

As described above, ROI portions 6a, 6 of arbitrary shapes converted into portions corresponding to the wavelet plane
Since the lossy compression is performed for b ..., the size of the compressed image file can be reduced as a whole as compared with the Maxshift method (first conventional example) that has performed the lossless compression. In addition, compared to EBCOT (second conventional example), ROI portions 6a, 6b,... Of an arbitrary shape can be specified. For example, when it is desired to increase the bit rate of only the face of a person in the background, the processing is performed. It will be easier.

[0044]

According to the first and second aspects of the present invention, the code amount when compressing a region of interest (ROI) into a designated mask region is satisfied with a value necessary for lossless compression. Since it is set to a value that does not exist, a mask region (ROI) of an arbitrary shape converted to a portion corresponding to the wavelet plane
Part). Therefore, the size of the compressed image file can be reduced as a whole as compared with the Maxshift method (first conventional example) in which lossless compression is performed, and EBCOT (second conventional example).
The ROI portion having an arbitrary shape can be designated as compared with the case of, for example, when it is desired to increase the bit rate of only the face of the person in the background, the processing is facilitated.

According to the third and fourth aspects of the present invention, even when a plurality of mask regions having an arbitrary shape are designated, irreversible compression can be easily performed on each mask region. . In this case, by individually assigning the code amount to each mask area, there is an effect that the assigned code amount can be freely set.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a region of interest encoding method according to one embodiment of the present invention.

FIG. 2 is a diagram illustrating a mallat type wavelet packet tree.

FIG. 3 is a diagram illustrating a mallat-type wavelet plane.

FIG. 4 is a diagram showing a space type wavelet packet tree.

FIG. 5 is a diagram showing a space type wavelet plane;

FIG. 6 is a diagram illustrating a packet type wavelet packet tree.

FIG. 7 is a diagram illustrating a packet-type wavelet plane.

FIG. 8 is a diagram illustrating an example of an original image.

FIG. 9 is a diagram illustrating a single mask area set for the original image of FIG. 8;

10 is a diagram showing a state in which the mask region of FIG. 9 is developed on a mallat type wavelet plane.

FIG. 11 is a diagram illustrating a correspondence relationship of a mask area between a low-frequency side and a high-frequency side and an input side in an inverse wavelet 5 × 3 filter.

FIG. 12 is a diagram illustrating a correspondence relationship of a mask area between a low-frequency side and a high-frequency side and an input side in an inverse wavelet 9 × 7 filter.

FIG. 13 is a diagram showing a distribution of wavelet coefficients after performing a known wavelet transform on an original image.

FIG. 14 is a diagram showing a distribution of wavelet coefficients in the first conventional example.

FIG. 15 is a diagram illustrating the concept of EBCOT according to a second conventional example.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image data (original image) 10a-10z Binarized data 11a-11z Compressed data 2 Wavelet-transformed image data 21a-21z Binarized data 22a-22z ROI signal 3 Mask signal 4, 4a Mask area 5, 5a Non-ROI part 6a, 6b ... ROI part Co Compressor Ex Expander

Claims (4)

[Claims] A first step of performing a wavelet transform on an original image; a second step of developing a mask region of an arbitrary shape designated for the original image on a wavelet plane corresponding to the wavelet transform; A third step of allocating a code amount when compressing a region and a non-mask region other than the mask region, and a fourth step of performing quantization and encoding according to the code amount allocated in the third step Wherein the code amount when compressing the mask region in the third step is set to a value less than a value required for lossless compression. 2. The region of interest encoding method according to claim 1, wherein the fourth step performs the quantization and the encoding individually on the mask region and the non-mask region, and compresses the respective regions. A method for coding a region of interest, comprising generating data. 3. The region of interest encoding method according to claim 1, wherein a plurality of the mask regions are designated for the original image, and in the third step, the mask regions are individually assigned to different ones of the mask regions. A region of interest encoding method, wherein a code amount is assigned. 4. The region of interest encoding method according to claim 3, wherein the fourth step performs the quantization and the encoding on each of the plurality of mask regions and the non-mask regions individually, A method of coding a region of interest, characterized by generating respective compressed data.