JPH10257484A - Image compression / decompression system - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To reproduce an impression of an original image when transmitting and compressing an image which has been subjected to preprocessing for compression ratio efficiency such as a low-pass filter and expanding the image on a reproducing side. To SOLUTION: Two filters, a pre-processing filter 12 and a post-processing filter 24, have properties opposite to each other.
In the transmitting unit 1, if high-frequency components are removed by filtering with the pre-processing filter 12 having a low-pass function, it is possible to increase the compression ratio without significantly reducing the image quality during image compression. On the other hand, the receiving side unit 2 expands the compressed image and passes it through the post-processing filter 24 having a high-pass filter function, thereby enhancing the outline of the image and reproducing the impression of the original image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for compressing and transmitting an image, and for expanding and reproducing the transmitted compressed image.

[0002]

2. Description of the Related Art When performing conventional image compression, if the image quality of an original image is very good and the amount of information is large, deterioration of the image quality due to compression is conspicuous. A method of suppressing image quality deterioration due to compression by reducing image quality has been used.

For example, the image compression by the MPEG method will be described. When the image quality of the original image is high, that is, when the information amount of the original image is large, the bit size required for output is reduced.
The rate (code amount) increases. However, the output bit rate must be a fixed value according to the request from the transmission system. Therefore, in the MPEG system, when the information amount of the input image is large, it is necessary to increase the compression ratio.
In this method, the information itself is dropped by making the quantization step rough, and as a result, the image quality of the output image is deteriorated.

Particularly, in the MPEG system, 16 × 1
Due to the processing performed on macroblocks in units of 6 pixels, when the compression ratio is increased, square noise in units of macroblocks may appear on the screen. The noise in units of macroblocks is extremely annoying to human senses, and the image quality is extremely poor. Therefore, in such a case, a predetermined preprocessing is performed to reduce the deterioration of the image quality at the time of compression.

[0005] In the MPEG system, discrete cosine transform (DCT transform) and quantization are used as image compression means. That is, the DCT transform of the input image specifies the image bias due to the spatial frequency, and extracts only the bias component by quantization to reduce the data amount. This means that in order to increase the compression ratio in the MPEG system, an image biased in spatial frequency is preferable.

In general, human vision is sharp for low spatial frequency components and dull for high frequency components.
In other words, if high frequency components are removed by a low-pass filter as pre-processing, the compression rate can be increased without degrading image quality so much. Therefore, for images with a large amount of information, and thus macroblocks are likely to appear, the amount of information is reduced by removing high-frequency components with a low-pass filter as pre-processing, and then the compressed image information is transmitted and reproduced. The side has been stretched to improve the visual image quality.

[0007]

However, by applying a low-pass filter as pre-processing in this way, although the image quality of the reproduced image itself is improved, an image having a different impression is reproduced when compared with the original image. There was something. Of course, the present invention is not limited to the low-pass filter, and the same applies as long as the pre-processing of the compression processing for improving the compression efficiency leads to the reproduction of an image having a different impression.

The present invention has been made in order to solve the above-mentioned problems. For example, an image which has been subjected to pre-processing for compression ratio efficiency such as a low-pass filter is compressed and transmitted, and the reproduced data is transmitted to the reproducing side. It is an object of the present invention to provide an image compression / decompression system that can reproduce the impression of an original image when decompressing.

[0009]

In order to achieve the above object, an image compression / decompression system according to claim 1, comprising: a compression means for compressing an input original image;
Preprocessing means for performing a first filtering process on an original image before compression by the compression means in order to improve compression efficiency in the compression means; and transmission means for transmitting the image compressed by the compression means. Expansion means for restoring the compressed image transmitted by the transmission means,
Post-processing means for subjecting the image decompressed by the decompression means to a second filter processing having an effect opposite to the first filter processing in the pre-processing means. And

According to the present image compression / decompression system, the input original image is subjected to the first filter processing by the preprocessing means and then compressed by the compression means. The first filter processing by the pre-processing means is processing for improving the compression efficiency in the compression means, and for example, a low-pass filter or the like can be considered. As described above, the amount of information can be reduced by removing high-frequency components by the low-pass filter, so that the compression efficiency can be increased. Also, since human vision is sharp for low spatial frequency components and dull for high frequency components,
If high frequency components are removed by the low-pass filter, the compression rate can be increased without significantly reducing the image quality.

On the reproducing side to which the image thus compressed is transmitted, after the compressed image is restored by the decompression means, the first processing in the pre-processing means is performed by the post-processing means.
A second filter process having an operation effect opposite to that of the filter process is performed. That is, when the first filter processing is, for example, a low-pass filter, a smoothing function for an image is exhibited, and a high-pass filter having the opposite effect is applied as the second filter processing. In this case, since the high-pass filter performs edge enhancement on the image, an image that has been given an impression different from that of the original image by the low-pass filter returns to an image close to the impression of the original image by the high-pass filter.

As described above, as the first and second filter processing, a low-pass filter having an image smoothing function is defined as a first filter processing, and a high-pass filter having an image contour enhancement function is defined as a second filter processing. However, the content of the filtering process is not limited to this. The first filter processing only needs to be “processing for improving the compression efficiency of the compression unit”,
The second filter processing may be of any type as long as the second filter processing can exert an effect opposite to that of the first filter processing.

Further, in the above-mentioned image compression / decompression system, there is provided a pre-processing control means for instructing the content or execution mode of the first filter processing by the pre-processing means, and the compression means is instructed by the pre-processing control means. Instruction data indicating the contents of the image and the data of the image are multiplexed and can be compressed.On the other hand, the decompression means converts the instruction data from the compressed data in which the instruction data and the image data are multiplexed. The post-processing unit may be configured to be separable, and may further include a post-processing control unit that controls the content or execution mode of the second filter processing by the post-processing unit based on the separated instruction data.

For example, when it is desired to select not only all frames of an image but only those that need to be subjected to the first filter processing by the pre-processing means, the pre-processing control means instructs the frame to be selected. It will be. And
In the post-processing control means, based on the instruction data,
The second filter processing is performed only on the frame on which the first filter processing has been performed. This eliminates unnecessary filtering and improves image quality.

As a first filter processing, high-frequency components can be removed by applying a low-pass filter to the image. However, when an image having a large number of high-frequency components such as a natural image is targeted, the entire image is blurred. As a result, the image is degraded. Therefore, image deterioration can be appropriately prevented by actively controlling whether or not to execute the filter processing according to the target image.

In this case, the pre-processing control means instructs, so to speak, "instructs the execution mode of the first filter processing", but may instruct "the contents of the first filter processing". For example, when a plurality of types of first filtering processes are provided, the instruction is to specify which of them is specified. Naturally, the post-processing means also includes a plurality of types of second filter processing corresponding to the second filter processing, and the second filter processing corresponding to the first filter processing is selected and executed based on the instruction data. . For example, it is conceivable to prepare a plurality of types of low-pass filters having different low-pass function strengths. In this case, the strength of the low-pass filter can be actively adjusted according to the frequency characteristics of the target image, so that image degradation can be appropriately prevented.

The first and second filter processes are a low-pass filter process and a high-pass filter process capable of executing a weighted averaging process by using a matrix-type coefficient assigned to a central pixel of interest and surrounding pixels. If so, by changing the size and coefficient value of the matrix, the strength and the like can be changed even for the same low-pass.

The image handled by the image compression / decompression system of the present invention may be a still image or a moving image. When handling moving images, it is necessary that the compression means and the decompression means conform to the standard for compressing and expanding moving images. Standards for compressing and expanding moving images include, for example, MPEG and H.264. 261 and the like. As described above, the effect is considered to be greater in the case of handling a moving image than in the case of a still image.
A method based on a standard method for compressing a still image such as PEG can be similarly applied.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. In the image compression / decompression system of the present embodiment, image data is compressed by the transmission unit 1 and the compressed image data is transmitted to the reception unit 2 via the transmission path 3, and transmitted by the reception unit 2. It is configured as a system for expanding compressed image data.

First, the transmitting unit 1 will be described.
The transmission unit 1 has an input interface (hereinafter, referred to as an input I / F) for inputting a digital video signal.
11, a pre-processing filter 12, and a frame memory 13
, A compression device 14, a multiplexing unit 15, a transmitting unit 16, and a filter control unit 17.

The pre-processing filter 12 includes a line memory 12a for two lines and a convolution operation unit 12b. The operation result of the convolution operation unit 12b is written in the frame memory 13. In this sense, the pre-processing filter 12 and the frame memory 13 are called “pre-processing means”.
Is equivalent to The function and the like of the pre-processing filter 12 will be described later in detail together with the post-processing filter 24 of the receiving unit 2 having the same configuration.

The image data pre-processed by the pre-processing filter 12 is compressed by a compression device 14 and subjected to a predetermined multiplexing process by a multiplexing unit 15. Sent. Also, the input I / F 11
A path for inputting data without passing through the pre-processing filter 12 described above is also set between the frame memory 13 and a frame memory 13. A switch is provided for switching between a path through the pre-processing filter 12 and a path not passing through. ing. This switch is switch-controlled (filter ON / OFF) by the filter control unit 17.

A time code indicating time information for each frame of the digital video signal is input to the filter control unit 17. Then, as shown in FIG. 6, control information listing time codes corresponding to frames to be filtered is prepared in advance, and at the time of compression, this control information is read by the filter control unit 17. Then, ON / OFF control of the filter is performed at a necessary timing. Furthermore, the filter control unit 17 adds this control information to the private data and sends the private data to the multiplexing unit 15. This private data is multiplexed with the compressed image data from the compression device 14 and sent to the transmission unit 16.

Here, the internal configuration of the compression device 14 will be described. The compression device 14 according to the present embodiment conforms to the MPEG system. As shown in FIG. 2, moving image data is input from the frame memory 13 and output to the multiplexing unit 15 as compressed digital data. A subtraction unit 102 that performs a subtraction process between the pixel value data input from the frame memory 13 and the reference data, and a DCT as an orthogonal transform unit that performs a DCT (discrete cosine transform) process, which is a type of orthogonal transform. Unit 103 and its DCT
A quantization unit 104 as quantization means for quantizing the DCT coefficient data input from the unit 103;
A variable length coding unit 105 as variable length coding means for performing variable length coding on the DCT coefficient data quantized in 04, and buffering the data coded by the variable length coding unit 105 A transmission buffer unit 106 for outputting to the subsequent multiplexing unit 15;
An inverse quantization unit 107 for inversely quantizing the DCT coefficient data quantized in 04, and an IDCT (inverse discrete cosine transform) process is performed on the DCT coefficient data inversely quantized by the inverse quantization unit 107. An IDCT unit 108, an addition unit 109 for adding motion compensation to the data returned to the image data before the IDCT processing by the IDCT unit 108, and an addition unit 109 for inter-motion frame prediction
A frame memory unit 110 for storing the output from
And a motion compensator 112.

In the compression apparatus 14, the input moving image data to be compressed is taken into the frame memory 13 and temporarily stored therein. In this embodiment, 16 × 16
Since the macroblock of pixels is processed as one unit for compression, the data is rearranged in macroblock units and output to the compression device 14. The pixel data in macroblock units, which is the output of the frame memory 13,
Sent to The prediction data from the motion compensation unit 112 is also input to the subtraction unit 102, and the difference data (prediction error) between the pixel data and the prediction data is output to the DCT unit 103.

The DCT unit 103 performs a DCT (Discrete Cosine Transform) process on the difference data output from the subtraction unit 102. At this time, the calculation is performed in predetermined block units (for example, every 8 × 8 pixels). In DCT operation,
An 8 × 8 coefficient is obtained as an output. In the present embodiment, the coefficients (hereinafter, also referred to as “DCT coefficients”) are arranged in the order of frequency components and output to the quantization unit 104.

The quantization unit 104 quantizes the DCT coefficient input from the DCT unit 103 with a predetermined step size, and outputs the quantized value to the variable length coding unit 105. The variable length coding unit 105 performs variable length coding on the quantized DCT coefficients and outputs the result to the transmission buffer unit 106. In this variable-length coding, for example, it is conceivable that after the output of the quantization unit 104 is run-length-transformed, the coefficient is converted into a variable-length code and output as compressed data. Although the run-length encoding results in two coefficients, which are a set of the number of zero coefficients and the value of the next non-zero coefficient, the output of the quantizing unit 104 has a tendency to continue with many zero coefficients. The data amount is reduced by performing the length encoding. Next, two coefficients that are the result of the run-length encoding are put together, and one variable-length code is assigned and output, so that compressed data is obtained.

On the other hand, the inverse quantization unit 107 includes the quantization unit 1
Since the quantized DCT coefficient output from the input unit 04 is input, inverse quantization is performed using the quantization step size selected in the quantization unit 104, the DCT coefficient is calculated and output to the IDCT unit 108. I do. Since the result at this time is a value including an error due to quantization, IDCT section 108 performs IDCT (inverse discrete cosine transform) processing on the DCT coefficient input from inverse quantization section 107 and outputs the result to addition section 109. I do.

The addition section 109 adds the output from the IDCT section 108 and the output from the motion compensation section 112 and outputs the result to the frame memory section 110. Frame memory unit 11
Reference numeral 0 denotes a frame memory for motion compensated inter-frame prediction, which is composed of at least two frame memories. Then, at the same time as accumulating the pixel value output from the addition unit 109, the pixel data of the previous frame is output to the motion compensation unit 112 for motion compensation frame prediction.

Transmission unit 1 via input I / F 11
Is assumed to be a general video signal. In a video signal, a set of pixel data in one row in the horizontal direction is called a scanning line, and the scanning lines are arranged in the vertical direction to form a two-dimensional screen. The scanning lines have pixel data sequentially arranged from the left end to the right, and are arranged in a downward direction from the scanning line at the upper end of the screen. In the present embodiment, the number of scanning lines is 480, and one scanning line includes 704 pixels. This will be sent sequentially in time. FIG. 2 shows the display positions of the pixels on the screen. When the scanning order is indicated by pixels, V (0,0) → V (0,1) →... → V (0, n)
→ V (1,0) → V (1,1) → ... → V (m, 0) →
... → V (m, n).

Next, the receiving unit 2 will be described. The receiving unit 2 includes a receiving unit 21 that receives the stream data transmitted via the transmission path 3, a system decoding unit 22, a decompression device 23, and a post-processing filter 2.
4, a frame memory 25, an output driver 26, and a frame counter 27.

The system decoding section 22 separates and extracts the private data from the stream data and sends it to the frame counter 27. Compressed data such as video data is sent to the decompression device 23. Stretching device 2
3 is omitted, but since the compression device 14 described above performs DCT, quantization, and variable-length coding as shown in FIG. To restore the data before compression.

The image data reproduced by the decompression device 23 is post-processed by the post-processing filter 24 and written into the frame memory 25. Post-processing filter 24
Is a line memory 24a for two lines and a convolution unit 2
4b, and the operation result of the convolution operation unit 24b is written to the frame memory 25. In this sense, the post-processing filter 24 and the frame memory 13 correspond to “post-processing means”. The function of the post-processing filter 24 will be described later in detail together with the pre-processing filter 12 described above.

The image written in the frame memory 25 is output as a video signal by the output driver 26 and is displayed on, for example, a monitor. Also,
A path for inputting data without passing through the post-processing filter 24 described above is also set between the decompression device 23 and the frame memory 25, and a switch for switching between a path through the post-processing filter 12 and a path not passing through the post-processing filter 12. Is provided. This switch is connected to the frame counter 27
Switching control (filter ON / OFF) is performed by this.

The frame counter 27 receives the private data separated from the stream data by the system decoding unit 22 as described above. As described above, in the transmitting unit 1, the control information in which the time code corresponding to the frame to be filtered is listed is added to the private data. Therefore, the frame counter 27 extracts the filter control information from the private data. Then, while counting the number of playback frames, the filter ON / OFF switching control is performed on the frame of the designated time code.

Next, the functions of the pre-processing filter 12 of the transmitting unit 1 and the post-processing filter 24 of the receiving unit 2 will be described. In the present embodiment, a digital filter based on a 3 × 3 convolution operation is used. The configurations of the pre-processing filter 12 and the post-processing filter 24 are completely the same,
By changing the coefficients used for convolution, a filter having the opposite effect is realized. This filter is shown in FIG.
As shown in (b), for example, weighting is performed on nine pixel data including eight pixels in the vicinity of the pixel at point V (i, j) and the pixel at point V (i, j) itself. , That is, spatial filtering is performed by a convolution operation. Assuming that the 3 × 3 coefficient matrix is A, B, C, D, E, F, G, H, I as shown in FIG. 4A, the output data Vo (i, j) is expressed by the following equation. Is calculated as follows.

Vo (i, j) = A * V (i-1, j)
-1) + B * V (i-1, j) + C * V (i-1, j +
1) + D * V (i, j-1) + E * V (i, j) + F *
V (i, j + 1) + G * V (i + 1, j-1) + H * V
(I + 1, j) + I * V (i + 1, j + 1) As a filter operation, the lines of V (i-1) and V (i) stored in the line memories 12a and 24a and the line of the input V (i + 1) are A convolution operation is performed by multiplying the data of each line by a preset coefficient A to I by a multiplier and adding the result by an adder. This calculation is performed once for each pixel of the video signal, and is sequentially calculated for the next pixel by a pipeline method.

The video data is stored in frame memories 13 and 25 capable of holding video data for one screen.
Are compressed by the compression device 14 during compression, and are output as video signals by the output driver 26 during decompression. Regarding the preprocessing filter 12, the coefficients (A to I) are set to values as shown in FIG. 5A, that is, the coefficients E corresponding to V (i, j) to be the target pixel are set to (8/2).
0), and the coefficients B, D,
Let F and H be (2/20) and coefficients A, B, G and I for the remaining pixels be (1/20). This results in a low-pass filter. Since the amount of information can be reduced by removing high-frequency components using a low-pass filter, compression efficiency can be increased. Also, since human vision is sharp for low spatial frequency components and dull for high frequency components, removing high frequency components with a low-pass filter increases the compression ratio without significantly reducing image quality. You can do it. That is, an image smoothing function is exhibited.

On the other hand, for the post-processing filter 24, the coefficients (A to I) are set to values as shown in FIG.
That is, the coefficient E corresponding to V (i, j) that is the pixel of interest
Is (8/20), the coefficients B, D, F, and H of the pixels corresponding to the upper, lower, left, and right are (−2/20), and the coefficients A, B, G, and I for the remaining pixels are (1/20). ). This is because, for the coefficient of FIG. 5A, V (i,
The signs B, D, F, and H of the pixels corresponding to the upper, lower, left, and right of j) are inverted. By setting such a value, a high-pass filter is provided, and a contour enhancement function of an image is exhibited.

That is, since the two filters, the pre-processing filter 12 and the post-processing filter 24, have properties opposite to each other, the image on the transmitting side is filtered by the low-pass filter using the coefficients shown in FIG. On the other hand, the receiving unit 2 can reproduce the impression of the original image, though the image quality is reduced, by decompressing the compressed image and passing it through a high-pass filter using the coefficients of FIG. it can.

However, preprocessing by image filtering is not always used for all screens (frames). This is because, for a frame having a relatively small amount of information, the image quality is better if the frame is compressed without filtering. Therefore, at the time of compression in the transmission side unit 1, the ON / OF of the filter is performed such that the filtering is not performed by the frame.
When F is switched, it is necessary to switch ON / OFF of the filter even at the time of extension in the receiving unit 2. For this purpose, the above-described filter information is used.

As described above, in this embodiment, control information listing time codes corresponding to frames to be filtered is prepared in advance.
At the time of compression, the filter control unit 17 reads this control information, and controls ON / OFF of the filter at necessary timing. Then, the filter control unit 17 adds this control information to the private data and sends it to the multiplexing unit 15. The private data is multiplexed with the compressed image data from the compression device 14 and transmitted from the transmission unit 16 to the receiving unit 2 via the transmission path 3.

In the receiving unit 2, private data is separated from the stream data by the system decoding section 22 and input to the frame counter 27. The frame counter 27 extracts the filter control information from the private data and counts the number of reproduction frames, and controls the switching of the filter ON / OFF with the frame of the designated time code.

Thus, the frame that has been filtered by the transmitting unit 1 can be filtered by the receiving unit 2 as well, and if the type of filter described above is used, a frame that has been subjected to smoothing processing as preprocessing at the time of compression is used. Is subjected to contour enhancement processing as post-processing at the time of decompression. This eliminates unnecessary filtering and improves image quality.

As described above, the present invention is not limited to such an embodiment, and can be implemented in various forms without departing from the gist of the present invention. For example, as the types of filters, filters of smoothing and edge enhancement are used here, but this does not limit the types of filters. Any filter may be used as long as the filters have the opposite effects.

Further, the specific example shown in FIG. 5 has been described as a coefficient for the above-described filtering, but the present invention is not limited to this. In particular, if it is a low-pass filter process and a high-pass filter process capable of executing weighted averaging with a matrix-type coefficient allocated to a target pixel at the center and surrounding pixels, changing the size and coefficient value of the matrix Thus, even with the same low pass, the strength and the like can be changed.

Further, in the above embodiment, DCT is employed as the orthogonal transform at the time of the compression processing in the compression device 14, but other orthogonal transforms such as Hadamard transform, Fourier transform, Haar transform, KL transform and the like are also available. There are things. However, in the MPEG presupposed in the present embodiment, it is general to use DCT which has a performance close to the most efficient KL transform among the orthogonal transforms and is easy to be hardened.

The image handled by the image compression / decompression system may be a still image or a moving image. In the above embodiment, the compression means 1 is assumed on the assumption that a moving image is handled.
H.4 conforms to the MPEG system, but H.4 is a standard for compressing and expanding the same moving image. H.261 may be more effective in handling moving images than in the case of still images. Of course, JPEG or other standard methods for compressing still images may be used. It is equally applicable.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an image compression / decompression system according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a schematic configuration of a compression device of a transmitting unit.

FIG. 3 is an explanatory diagram showing a configuration of video data.

FIG. 4 is an explanatory diagram showing a filter method.

FIG. 5 is an explanatory diagram showing a specific example of a filter coefficient.

FIG. 6 is an explanatory diagram showing filter control information.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Transmission side unit 2 ... Receiving side unit 3 ... Transmission path 12 ... Preprocessing filter 12a ... Line memory 12b ... Convolution calculation part 13 ... Frame memory 14 ... Compression device 15 ... Multiplexing part 16 ... Transmission part 17 ... Filter control Unit 21 receiving unit 22 system decoding unit 23 decompression device 24 post-processing filter 24a line memory 24b convolution unit 25 frame memory 26 output driver 27 frame counter

Claims (5)

[Claims] A compression unit for compressing an input original image; and a first filtering process on the original image before compression by the compression unit in order to improve a compression efficiency in the compression unit. Processing means, transmission means for transmitting the image compressed by the compression means, decompression means for restoring the compressed image transmitted by the transmission means, and for the image decompressed by the decompression means, An image compression / decompression system comprising: a post-processing unit that performs a second filtering process having an operation effect opposite to that of the first filtering process in the pre-processing unit. 2. The apparatus according to claim 1, further comprising: a pre-processing control unit configured to instruct a content or an execution mode of the first filtering process performed by the pre-processing unit, wherein the compression unit includes instruction data indicating the content instructed by the pre-processing control unit. The image data is configured to be multiplexable and compressible, while the decompression means is configured to be able to separate the instruction data from the compressed data in which the instruction data and image data are multiplexed. 2. The image compression / decompression apparatus according to claim 1, further comprising a post-processing control unit that controls the content or execution mode of the second filter processing by the post-processing unit based on the separated instruction data. system. 3. The image compression / decompression system according to claim 1, wherein said first filter processing is low-pass filter processing, and said second filter processing is high-pass filter processing. 4. The first and second filter processes are a low-pass filter process and a high-pass filter process capable of executing a weighted averaging process by using a matrix-type coefficient assigned to a central pixel of interest and surrounding pixels. 4. The image compression / decompression system according to claim 3, wherein: 5. The image compression / decompression system according to claim 1, wherein said compression means and decompression means conform to standards for compressing and decompressing moving images, respectively.