JP2001258040A - Color interpolation method for single-chip solid-state imaging device and recording medium storing color-interpolation processing program for single-chip solid-state imaging device - Google Patents

Abstract

(57) [Summary] A good image cannot be obtained if the same color interpolation processing is performed on an edge portion and a flat portion of an image. An edge is determined from an image to be processed, a color interpolation process is performed on the edge portion by a bicubic method, and nxm (n, m is set around a target pixel) for a portion other than the edge. Color interpolation processing is performed by uniform averaging over a pixel range of (positive integer). At that time, the color interpolation at the edge portion is a mixture of the image obtained by the color interpolation process by the bicubic method and the image obtained by the color interpolation process by the uniform average, and the mixing ratio Is changed according to the magnitude of the output of the edge determination filter (steps s11 to s20). Further, when performing the color interpolation based on the uniform average, a low-pass filter is performed after the color interpolation based on the uniform average is performed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color interpolation method for a single-chip solid-state imaging device, and more particularly to a color interpolation method for a single-chip solid-state imaging device capable of appropriately performing color interpolation on an edge portion and a flat portion of an image. The present invention relates to a recording medium storing an interpolation method and a color interpolation processing program for a single-chip solid-state imaging device.

[0002]

2. Description of the Related Art Image input devices such as a digital camera capable of extracting an image as digital data include:
A solid-state imaging device called a CCD (Charge Coupled Device) is used. In a general household appliance, a so-called single-panel type CCD is used for extracting three basic colors from a single CCD.

This single-plate CCD has four basic colors (described here using complementary colors): cyan (C), yellow (Y), magenta (M), and green (G). As a set of colors, for example, as shown in FIG. 6, a two-dimensional two-dimensional array is provided for each pixel.

In FIG. 6, when attention is paid to the pixel portion corresponding to cyan C22, the color of this pixel (referred to as a pixel of interest) can only obtain color information of only cyan. Therefore, in order to reproduce the color of this portion, it is necessary to perform color interpolation using nearby colors. For example, magenta uses the magenta M12 immediately above cyan C22 and the magenta M22 immediately below cyan C22, and adds the values of the two and divides by 2 to obtain a value obtained by a so-called simple average, which is the magenta value of the target pixel.

Similarly, yellow uses the yellow Y21 to the left of cyan C22 and the yellow Y22 to the right of cyan C22, adds the values of both, and divides the value obtained by simple averaging by 2 to the yellow of the pixel of interest C22. Value.

Similarly, green is green G11 at the upper left corner of cyan C22 and green G1 at the upper right corner of cyan C22.
2. There is a green G21 at the diagonally lower left and a green G22 at the diagonally lower right, and a value obtained by a simple average of adding these values and dividing by 4 is defined as the green value of the target pixel.

As described above, n × m around the pixel of interest
(N and m are positive integers, and here, n = m = 3). The color interpolation of the target pixel is performed using the color component corresponding to each pixel existing in the pixel range. This is referred to herein as color interpolation by 3 × 3 uniform averaging.

Alternatively, for example, color interpolation may be performed on pixels that do not exist originally. Conventionally, there is an interpolation method called a bicubic method.

[0009]

As described above,
There are various methods for performing color interpolation. But,
Each of these color interpolation methods may have different characteristics depending on which part of the image is used. For example, the bicubic method can reproduce a sharp image in an edge portion of an image, but tends to cause unevenness in an image in a flat portion of the image. This is considered to be mainly because the bicubic method in which the pixels used for the calculation used practically are limited performs substantially the same operation as the high-pass filter. In other words, the data input from the CCD originally has a sense of unevenness due to the influence of noise or the like, but the operation is performed to emphasize the unevenness. In addition, the bicubic method is disadvantageous in that the amount of calculation is large.

On the other hand, color interpolation based on 3 × 3 uniform averaging is a simple color interpolation method, and requires less calculation than the bicubic method. Although it has some problems, it has a feature that a straightforward image can be obtained in a flat portion.

In the present invention, the color interpolation based on the bicubic method and the color interpolation based on the uniform averaging are selectively used for the image portion, thereby suppressing an increase in the amount of calculation and enabling appropriate color interpolation as a whole. It is intended to be.

[0012]

In order to achieve the above-mentioned object, a color interpolation method for a single-chip solid-state imaging device according to the present invention is provided.
An edge is determined from the image to be processed, a color interpolation process is performed on the edge portion by the bicubic method, and a portion other than the edge is set to nx set around the pixel of interest.
The color interpolation processing is performed by a uniform average over a pixel range of m (n and m are positive integers).

In the invention of the color interpolation method for a single-chip solid-state image pickup device, the edge determination is performed based on a magnitude of a change in a pixel value of a pixel present in the vicinity, and the color at the edge portion determined thereby is determined. Interpolation is performed by mixing the image obtained by the color interpolation processing by the bicubic method and the image obtained by the color interpolation processing by the uniform average, and the mixing ratio is determined by the pixels existing in the vicinity. Is changed according to the degree of change of the pixel value of. It should be noted that the magnitude of the change from the pixel value of the neighboring pixels can be obtained by an edge determination filter.

Further, when performing the color interpolation based on the uniform average, a low-pass filter is passed after the color interpolation based on the uniform average is performed.

Further, the color interpolation processing program in the recording medium storing the color interpolation processing program of the single-chip solid-state imaging device according to the present invention includes a procedure for judging an edge from an image to be processed, and a procedure for judging an edge from an image to be processed. A procedure of performing color interpolation processing by the cubic method, and performing color interpolation processing by uniform averaging over a pixel range of n × m (n and m are positive integers) set around a target pixel for a portion other than an edge; Is included.

Further, in the invention of the recording medium recording the color interpolation processing program for the single-chip solid-state image pickup device, the edge determination is performed based on a magnitude of a change in a pixel value of a pixel present in the vicinity. The color interpolation at the edge portion obtained is a mixture of the image obtained by the color interpolation process by the bicubic method and the image obtained by the color interpolation process by the uniform average, and the mixing ratio is determined. , According to the magnitude of the degree of change in the pixel value of the neighboring pixels. It should be noted that the magnitude of the change in the pixel value of the neighboring pixels can be obtained by an edge determination filter.

Further, when performing color interpolation based on the uniform average, a low-pass filter is passed after the color interpolation based on the uniform average is performed.

As described above, according to the present invention, an edge is determined from an image to be processed, a color interpolation process is performed on the edge portion by the bicubic method, and an nx set around the pixel of interest for a portion other than the edge. The color interpolation processing is performed by a uniform average over a pixel range of m (n and m are positive integers). As a result, it is possible to perform color interpolation utilizing the characteristics of the bicubic method and the uniform average of n × m, to obtain a good image with less unevenness in the flat part, and to obtain a sharp image in the edge part. .

The color interpolation at the edge portion is
It is assumed that the image obtained by the color interpolation processing by the bicubic method and the image obtained by the color interpolation processing by the uniform average are mixed, and the mixing ratio is determined by the neighboring pixels in a portion determined to be an edge. Are changed in accordance with the change in the pixel value of. by this,
There is no unnatural edge switching in the edge part,
An image with a natural gradation is obtained. Note that an edge determination filter can be used for edge determination, and by using the edge determination filter,
Edge determination processing can be simplified.

Further, in the method of obtaining a uniform average of n × m, by applying a low-pass filter to the obtained value, it is possible to further improve the unevenness of the image of the flat portion, and to further improve the image quality. Image.

[0021]

Embodiments of the present invention will be described below. The contents described in this embodiment are as follows.
In addition to the description of the color interpolation method of the single-chip solid-state imaging device of the present invention, the specific processing content of the color interpolation processing program in a recording medium that stores the color-interpolation processing program of the single-chip solid-state imaging device of the present invention is also described. Including.

According to the present invention, simply, in a flat portion of an image, color interpolation is performed with a uniform average of n × m (here, n = m = 3), and a bicubic is performed in an edge portion of the image. This is to perform color interpolation by the method. When actually processing an image, in a flat portion of the image, it can be confirmed that the use of the 3 × 3 uniform average visually reduces the sense of unevenness. On the other hand, regarding the vicinity of the edge of the image, It was found that a sharper image could be obtained by using the bicubic method.

In order to realize this, first, it is necessary to separately perform the color interpolation processing on the edge portion and the flat portion of the image, and it is necessary to perform the edge judgment. The method of performing the edge determination is not particularly limited, but here, a luminance signal obtained by the bicubic method is used. Here, the edge determination is performed using an edge determination filter, and when the absolute value | F | of the output F of the edge determination filter exceeds a predetermined value th1, that is, | F | ≧ th
When it is 1, the part is determined to be an edge.

Accordingly, such edge judgment is performed on the image to be processed, and if | F | ≧ th1, color interpolation is performed by the bicubic method, and | F |
If <th1, color interpolation based on a 3 × 3 uniform average is performed. Thereby, a good image with less unevenness is obtained in the flat portion, and a sharp image is obtained in the edge portion.

The flowchart of FIG. 1 shows the basic concept of the present invention, in which edge determination processing is performed (step s1), and if | F | ≧ th1 (step s1).
2) Assuming that the edge is an edge, color interpolation by the bicubic method is performed (step s3). | F | <t
If it is h1, it is regarded as not an edge, and color interpolation based on a 3 × 3 uniform average is performed (step s4).

In this embodiment, an edge judgment filter as shown in FIG. 2 is used as an example. And
When the original image is an image having a full-scale gradation of 0 to 1023, th1 is set to 32, and if the edge determination output | F | is | F | ≧ 32, the edge is determined. Color interpolation by the bicubic method for |
If F | <32, 3 *
When the color interpolation based on the uniform average of 3 is performed, a good image with little unevenness is obtained in the flat portion, and a sharp image is obtained in the edge portion.

However, if the color interpolation method is switched by simply performing the edge judgment, some problems remain. For example, in a portion where the change in luminance is near th1, the influence of noise or the like causes the bicubic method and
Switching between two color interpolation methods having a uniform average of 3 frequently occurs. In this case, different values are alternately changed for each pixel, and a natural gradation may not be drawn. A method for solving this will be described below.

In order to solve this problem, it is desirable not to rapidly switch between the two color interpolation methods of the bicubic method and the 3 × 3 uniform average, but to switch gradually in accordance with the edge amount. Specifically, two color interpolated images of the bicubic method and the uniform average of 3 × 3 are mixed at an appropriate ratio depending on the magnitude of the output F of the edge determination filter shown in FIG.

Assuming that a pixel value after color interpolation obtained by the bicubic method for a certain pixel is B, and a pixel value after color interpolation obtained by a uniform average of 3 × 3 is T, FIG. The magnitude of the absolute value | F | of the output F of the edge determination filter is divided into, for example, five stages, and the pixel value after two color interpolations of the bicubic method and the uniform average of 3 × 3 is calculated for each stage. The mixing ratio has been changed.

Here, as shown in FIG. 3, the magnitude of the absolute value | F | of the output F of the edge determination filter is | F | ≧
32, 32> | F | ≧ 24, 24> | F | ≧ 16, 16
> | F | ≧ 8, 8> | F | Here, a larger value of | F | indicates a larger edge amount. The numerical values in FIG. 3 are examples in which the original image is set to 0 to 1023 full scale.

In FIG. 3, when | F | ≧ 32, the pixel value after color interpolation is set to (4 · B + 0 · T) / 4.
32> | F | ≧ 24, the pixel value after color interpolation is set to (3
· B + 1 · T) / 4, and when 24> | F | ≧ 16, the pixel value after color interpolation is set to (2 · B + 2 · T) / 4.
16> | F | ≧ 8, the pixel value after color interpolation is (1 ·
B + 3 · T) / 4, and when 8> | F |, the pixel value after color interpolation is (0 · B + 4 · T) / 4.

By performing such mixing, unnatural edge switching in the edge portion is eliminated, and an image with a natural gradation is obtained.

In the flowchart of FIG. 4, two color interpolated images of the bicubic method and the 3 × 3 uniform average are mixed at an appropriate ratio according to the magnitude of the absolute value | F | of the output F of the edge judgment filter. The above processing will be described with reference to a specific example of FIG. First, edge determination is performed (step s11). If the absolute value | F | of the output F of the edge determination filter is | F | ≧ 32 (step s1)
2) The pixel value after color interpolation is set to (4 · B + 0 · T) / 4 (step s13), and if the absolute value | F | of the output F of the edge determination filter is 32> | F | ≧ 24 (step s13). s
14), the pixel value after color interpolation is set to (3 · B + 1 · T) / 4 (step s15).

If the absolute value | F | of the output F of the edge determination filter is 24> | F | ≧ 16 (step s1)
6) The pixel value after color interpolation is set to (2 · B + 2 · T) / 4 (step s17), and if the absolute value | F | of the output F of the edge determination filter is 16> | F | ≧ 8 (step s17) s1
8) The pixel value after color interpolation is set to (1 · B + 3 · T) / 4 (step s19). If the absolute value | F | of the output F of the edge determination filter is not 16> | F | ≧ 8 (step s18), the absolute value | F | of the output F of the edge determination filter is regarded as 8> | F |. Then, the pixel value after color interpolation is set to (0 · B + 4 · T) / 4 (step s20).

As described above, the two color interpolated images of the bicubic method and the 3 × 3 uniform average are mixed at an appropriate ratio depending on the magnitude of the absolute value | F | of the output F of the edge judgment filter. A good image can be obtained by performing such processing as described above, but in the present invention, in order to further improve the image of the flat portion, the value obtained by the 3 × 3 uniform average is used. Apply a low-pass filter.

That is, first, a process of mixing two color interpolated images of a bicubic method and a 3 × 3 uniform average at an appropriate ratio is performed depending on the size of the output F of the edge judgment filter. Further, in a method of obtaining a uniform average of 3 × 3, a low-pass filter as shown in FIG. 5 is applied to the obtained value. By performing such processing, it is possible to obtain a good image in which the edge portion becomes sharp and the unevenness of the flat portion is further improved.

As described above, according to the present invention, the edge portion is subjected to the bicubic method, and the flat portion is color-interpolated by the uniform average of 3.times.3. An image is obtained.

Further, by performing a process of mixing two color interpolated images of the bicubic method and the uniform average of 3 × 3 at an appropriate ratio according to the size of the output F of the edge judgment filter, Although a good image can be obtained, in the present invention, in order to further improve the image of the flat portion, a method of obtaining a 3 × 3 uniform average is used.
By applying a low-pass filter to the obtained value, it is possible to obtain a good image without sharpness in the edge portion and no unevenness in the image in the flat portion.

The present invention is not limited to the embodiment described above, but can be variously modified without departing from the gist of the present invention. For example, although an edge determination filter as shown in FIG. 2 has been described as a specific example of the edge determination filter, the present invention is not limited to such an edge determination filter. Further, the edge can be determined by calculation based on a change in the pixel value of a neighboring pixel without using an edge determination filter. Further, although a low-pass filter is shown as a specific example in FIG. 5, the low-pass filter is not limited to the low-pass filter shown in FIG.

The above-described color interpolation processing program for performing the processing of the present invention can be recorded on a recording medium such as a floppy disk, an optical disk, or a hard disk. The present invention also includes the recording medium.
Further, the processing program may be obtained from a network.

[0041]

As described above, according to the present invention, an edge is determined from an image to be processed, a color interpolation process is performed on the edge portion by a bicubic method, and a portion other than the edge is around a target pixel. N set to
The color interpolation process is performed by uniform average over a pixel range of × m (n and m are positive integers). As a result, it is possible to perform color interpolation utilizing the characteristics of the bicubic method and the uniform average of n × m, to obtain a good image with less unevenness in the flat part, and to obtain a sharp image in the edge part. .

The color interpolation at the edge portion is
It is assumed that the image obtained by the color interpolation processing by the bicubic method and the image obtained by the color interpolation processing by the uniform average are mixed, and the mixing ratio is determined by the neighboring pixels in a portion determined to be an edge. Is changed in accordance with the change of the pixel value. As a result, unnatural edge switching in the edge portion is eliminated, and an image in which a natural gradation is drawn can be obtained.
Note that the edge determination process can be simplified by using an edge determination filter for edge determination.

Further, in the method of obtaining a uniform average of n × m, by applying a low-pass filter to the obtained value, it is possible to improve the unevenness of the image of the flat portion, and it is possible to further improve the image quality. It can be an image.

[Brief description of the drawings]

FIG. 1 is a flowchart illustrating a basic concept of the present invention.

FIG. 2 is a diagram showing a specific example of an edge determination filter used in the embodiment of the present invention.

FIG. 3 is a diagram illustrating a process of mixing color interpolation based on a bicubic method and color interpolation based on a 3 × 3 uniform average according to the size of an edge determination filter output according to the present invention.

FIG. 4 is a flowchart illustrating a processing procedure in the example illustrated in FIG. 3;

FIG. 5 is a diagram showing a specific example of a low-pass filter used in the embodiment of the present invention.

FIG. 6 is a diagram illustrating an example of a pixel array for explaining color interpolation processing based on a 3 × 3 uniform average.

[Explanation of symbols]

 F Size of output of edge judgment filter B Pixel value obtained by bicubic method T Pixel value obtained by uniform averaging of 3 × 3

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) H04N 1/46 H04N 1/46 Z F term (reference) 4M118 AA10 AB01 BA10 FA06 GC09 GC14 GC20 5B057 AA20 CA01 CA08 CA12 CB01 CB08 CB12 CC01 CD06 CE03 CE05 CE16 CH18 DB02 DB06 DB09 5C065 AA01 BB13 BB22 CC01 CC09 DD02 DD17 EE05 EE07 GG13 GG17 GG31 GG32 5C079 JA23 LA14 LA28 NA02 NA27

Claims (8)

[Claims] 1. An edge is determined from an image to be processed, an edge portion is subjected to a color interpolation process by a bicubic method, and an nxm (n, m is set around a pixel of interest for a portion other than the edge. A color interpolation method for a single-chip solid-state imaging device, wherein color interpolation processing is performed by uniform averaging over a pixel range of (positive integer). 2. The edge determination is made based on a magnitude of a change in pixel value of a pixel present in the vicinity, and color interpolation at an edge portion determined by the determination is obtained by a color interpolation process using a bicubic method. It is assumed that the image and the image obtained by the color interpolation processing based on the uniform average are mixed, and the mixing ratio is changed according to the degree of the change in the pixel value of the neighboring pixels. 2. The color interpolation method for a single-chip solid-state imaging device according to claim 1, wherein 3. The color interpolation method for a single-chip solid-state imaging device according to claim 2, wherein the magnitude of a change in the pixel value of a pixel present in the vicinity is obtained by an edge determination filter. 4. The unit according to claim 1, wherein, when performing color interpolation based on the uniform average, the color interpolation is performed through the low-pass filter after performing the color interpolation based on the uniform average. A color interpolation method for a plate-type solid-state imaging device. 5. A recording medium on which a color interpolation processing program for a single-chip solid-state imaging device is recorded, the color interpolation processing program comprising: a procedure for determining an edge from an image to be processed; Procedure of performing color interpolation processing by the bicubic method, and performing color interpolation processing by uniform averaging over a pixel range of n × m (n and m are positive integers) set around a target pixel for a portion other than an edge. A recording medium storing a color interpolation processing program for a single-chip solid-state imaging device, characterized by comprising: 6. The edge determination is performed based on a magnitude of a change in a pixel value of a pixel present in the vicinity, and color interpolation at the edge portion determined by the determination is obtained by a color interpolation process using a bicubic method. It is assumed that the image and the image obtained by the color interpolation processing based on the uniform average are mixed, and the mixing ratio is changed according to the degree of the change in the pixel value of the neighboring pixels. 6. A recording medium in which a color interpolation processing program for a single-chip solid-state imaging device according to claim 5 is recorded. 7. The recording according to claim 6, wherein the magnitude of the change in the pixel value of the pixel present in the vicinity is obtained by an edge determination filter. Medium. 8. The simple interpolation method according to claim 5, wherein when performing the color interpolation based on the uniform average, the color interpolation based on the uniform average is performed, and then the low-pass filter is passed. A recording medium that stores a color interpolation processing program for a plate-type solid-state imaging device.