JPH04165887A - Image signal processing device - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to adaptive processing that changes the method of signal processing depending on the screen situation.

(Related technology) FIGS. 7, 9, and 11 are diagrams showing examples of arrangement configurations of color filters of conventionally known color solid-state imaging devices. Transmission filter (more than “
It has a stripe shape directly at the IF and is called a red light transmitting filter (hereinafter referred to as "Rd filter").
) and r'? Color transmission filter (more than “Bu”)
They are arranged between Gr filters in the underwater direction in every second row and one column.

Figure 9 shows a magenta transmission filter (hereinafter referred to as "Mg filter"), a green light 4A filter, a cyan light aA filter (hereinafter referred to as "Cy filter"), a yellow light 4A filter (hereinafter referred to as "Ye filter"), and a water filter.゛A total of 8 color filters, 2 pixels in the P direction and 4 pixels in the vehicle direction
They are placed in the order shown in the figure.

Furthermore, FIG. 11 shows Rd, Gr, Bj! Each filter has 3 pixels horizontally and 1 pixel vertically.

They are arranged in an offset sampling structure with an offset amount of 1.5 pixels in the horizontal direction.

Figures 8, 10, and 12 refer to Figures 7 and 9, respectively.
The color light carriers in the color filter array configuration shown in FIG.

FIG. 3 is a characteristic diagram of the first quadrant when expressed as fv). In both cases, PH is the pixel pitch in the horizontal direction.

Let the pixel pitch in the vertical direction be Pv, 0≦f, ≦17P,
, 0≦fv≦1/2 PV. In both figures, arrows represent carriers of each color, the length of the arrow represents the size of the carrier, and the direction represents the phase relationship.

In FIG. 8, the colored light carriers are, in addition to (0,0),
(1/2P,,O), (1/P,,O), (0,1/4
Pv), (1/2P,,t/4Pv), (1/P+-
+, 1/4Pv). Of these, (o
, o) and (1/PH, O) are carriers generated for achromatic light and cause aliasing distortion.
Other carriers completely cancel out and disappear against achromatic light, but do not disappear against chromatic light, causing aliasing distortion.

Similarly, in FIG. 10, the colored light carriers are (0,0
), (1/2PH, o), (1/PH, 0), (1
/2PH11/4PV), (.

, 1/2Pv ), (1/2P++ , 1/2Pv )
.

(1/PH, 1/2 Pv), of which (o, o) and (1/Po, o) are carriers generated for achromatic light; the others are carriers for achromatic light. It is a career that disappears against.

Furthermore, in FIG. 12, if the horizontal offset amount of the solid-state image sensor is P u / 2, the colored light carriers are (273PH, O") and (1/3P) in addition to (o, o).
H, 1/2Pv), (1/P,.

1/2Pv), among which (o, o) and (1/Pv)
, , 1/2Pv ) are carriers that are generated in response to achromatic light, and other carriers are carriers that disappear in response to achromatic light.

This offset sampling structure has the following characteristics:
In a rectangular sampling structure as shown in FIGS. 7 and 9, (1/P ++ in the horizontal direction).

Since there is a carrier generated for achromatic light at the frequency of position 0), f s -1/2 P ++ becomes the Nyquist frequency, and no higher frequency components can be obtained, so f □ = 1/ Although it is not possible to obtain water resolution up to 2P□, the offset sampling structure shown in FIG. 11 has a horizontal resolution of (1/P□).

It is generally known that there are no carriers generated for achromatic light at the position 0), and that fH=1/Pu can be set as the Nyquist frequency. For this reason,
In the color filter arrangement shown in FIG. 11, although the sampling pitch is the same as that of the rectangular sampling structure, the horizontal resolution can be obtained up to twice fl+"1/P□.

However, general subjects are not necessarily achromatic, but generally have colors, so FIG.

Folding distortion occurs from the colored light carriers shown at all positions in FIGS. 10 and 12, and depending on the scene, it becomes very unsightly. For this reason, it is necessary to use an optical low-pass filter or the like to cut out harmful colored light carriers, which leads to a decrease in resolution.

For example, in the color solid-state imaging device with the offset sampling structure shown in FIGS. 11 and 12, (2/3P□,
Since colored light carriers are generated at the position of 0), an optical low-pass filter is required to cut frequency components higher than f ++ = 2 / 3 P ++ in the horizontal direction. / P u can obtain water resolution by 2/3 f,, = 2
It is possible to obtain water resolution only up to /3P.

In order to resolve such problems, the applicant has separately proposed the third
We have proposed a color filter array as shown in the figure.

A circumferential image sensor using the color filter array shown in FIG.
Horizontal pixel pitch P□, vertical pixel pitch Pv
, horizontal pixel offset amount P, has an offset sampling structure of /2, and has four types of color light transmission filters (color filters) Mg: magenta, green, cyan, and yellow.
, Gr, Cy, and Ye are arranged at positions corresponding to each pixel. A combination of these color filters is called a color filter array. As shown in Fig. 3, this color filter array has an offset sampling structure in which each type of color filter has a horizontal pitch of 2P, a vertical pitch of 2Pv, and a horizontal offset amount of PH. It has become.

The fourth characteristic diagram shows the color light carriers in such a color filter array in terms of two-dimensional frequency loss plane (flI, fv).
In the figure, 0≦f□≦1/pH in the first quadrant, 0≦fv≦1
/ 2 P v range is shown.

In the same figure, the colored light carriers are -(1) in addition to (0,0)
/P+-+, O), (1/2P++, 1/4Pv)
, (o, 1/2PV), (t/p++, 1/2Pν
), among which (0,0) and (1/P,,
, 1/2Pv) are carriers generated in response to achromatic light, and the others are carriers that disappear in response to achromatic light.

As is clear from the figure, in the horizontal direction (1/p, , o
), no colored light carriers are generated at all, and therefore frequency components up to fs=1/P can be obtained. That is, the color image capturing device in this example is as follows:
Figure 11.

A horizontal resolution up to fu=1/P+-+, which is 1.5 times the horizontal resolution of the color solid-state imaging device shown in the sampling structure of FIG. 12, can be obtained.

Furthermore, the color light carrier closest to the origin is (
1/2PH, 1/4PV'), but this is a colored light carrier that is sufficiently far from the origin and disappears in the presence of achromatic light, so for horizontal and vertical frequency components, No significant aliasing distortion occurs. Especially when using color filters or complementary colors, aliasing distortion is small.

As described above, the color solid-state imaging device using the color filter array shown in FIG. 3 has high resolution and less occurrence of moiré.

(Problems to be Solved by the Invention) An example of signal processing in the above-mentioned color imaging device is shown in FIG. 13.

However, when attempting to obtain a luminance signal by simply low-pass filtering the IH signal from the sensor I using the interpolation filter 25, the following phenomenon occurs.

In general, four color filters Mg for a certain-peripheral tautomer
, Cy, Ye, and Gr are slightly different, so for example, even if the brightness of the subject is uniform, the 13th
The structure of the array shown in FIG. 3 is visible in the luminance signal resulting from the signal processing using the configuration shown in the figure, and this appears as a checker pattern.

The characteristics of the colored light carriers when four color filters are arranged as shown in FIG. 3 are as shown in FIG. 4. The above-mentioned phenomenon can be understood as a phenomenon in which the DC component is returned to the position of the carrier indicated by P in FIG. The easiest way to prevent this is to perform signal processing to make the response of the carrier at point P in FIG. 4 zero. For example, if the brightness No. 43 Y at the position of the Gr exit in Fig. 3 is taken, the water/V in Fig. 4 and the directional frequency component 1/
An upward low-pass filter is implemented that traps 2P, so the P point is also trapped and the checker pattern is erased.

Furthermore, by adding up the pixel information of 2 prη elements in the vertical direction, a vertical low-pass filter that traps the vehicle weight direction frequency 1/4 Pv is realized, and the checker pattern can be removed.

However, if processing is performed in the horizontal direction or vertical direction as described above, the band in the horizontal direction or vertical direction is restricted, and the resolution deteriorates.

The present invention has been made to solve such problems, and it is an object of the present invention to provide an image signal processing device that can remove checker patterns without reducing resolution.

[Means to solve the problem]

In order to achieve the above object, one aspect of the present invention is to configure an image signal processing device as shown in (1) and (2) below.

(1) A discriminating means for discriminating whether or not there is a uniform area with little luminance change in a predetermined direction in an image, a processing means for low-pass filtering a luminance signal in the predetermined direction, and a discriminating means that and selecting means for selecting the processing means to perform low-pass filter song processing when it is determined that there is a similar region.

(2) An image signal processing device in which the discrimination means discriminates the luminance signal based on the absolute values of the outputs of horizontal and vertical band pass filters.

(Operation) According to the configurations (1) and (2) above, when there is a uniform ItI region with little luminance change in a predetermined direction in the image, low-pass filtering of the luminance signal is performed in that direction.

In the configuration (2) above, the −-like region with little change in brightness is determined based on the absolute values of the outputs of the horizontal and vertical bandpass filters for the brightness signal.

(Example) The present invention will be explained in detail below with reference to Examples.

FIG. 2 is a block diagram of a "video camera signal processing device" which is an embodiment of the present invention.

A color filter array shown in FIG. 3 is attached to the sensor 201, and as shown in FIG. 3, two rows are read out in a zigzag pattern in one horizontal scanning period (IH).

Therefore, every pixel will be read out once per field. Now set the frequency of this read clock to fs
Then, the output of the sensor 101 becomes Mg-+cy→Gr→Ye for each clock. This signal is processed by an analog processing unit 102 such as AGC (automatic gain control), and then converted to an A/D (analog-digital) converter 10.
A-D conversion is performed at step 3. It is desirable that the A/D converter used be one with a tolerance of 10 bits or more.

This signal is manually inputted to a luminance signal adaptive filter processing unit 205, which will be described later, to form a high-band luminance signal without the aforementioned checker pattern. This result is subjected to standard video signal processing such as γ conversion and blanking in the luminance processing section 206, and then digitally processed in the D/A converter 207.
converted to analog. Furthermore, the same Kll (3-Q addition part 2
08, the standard television signal synchronization number 8 is given to this.

The switch 204 switches the human input signal at half of the clock f8, and as a result, the Mg-Gr signal is output to S1, and the Cy-Ye signals A and - are output to $2.

The signals Mg, Gr, Cy,
It is separated into Ye. Since each color signal has an offset structure as shown in FIG. 3, missing information is interpolated by two-dimensional interpolation filters 211, 212, 213, and 214.

The matrix calculation unit 215 obtains primary color signals of R, G, and H from the four interpolated color signals Mg, Gr, Cy, and Ye by Madrisk calculation.

Here, matrix A is Mg, Gr of sensor 201,
Spectral characteristics Mg(λ) of Cy, Ye.

Ideal spectral characteristics R(λ), G of R, G, and B determined using Gr(λ), Cy(λ), and Ye(λ) according to the NTSC method
(λ), 3 rows optimized to approach B(λ) 4
It is a madrisk in a row.

A color processing unit 216 performs color processing such as white balance and γ processing on the 13 primary colors of R, G, and B obtained in this way, and inputs them to a color difference calculation unit 217. This is R-Y.

Two color difference signals of B-Y are formed, and the valley D/A change tl
! ! Digital-to-analog conversion is performed by the converters 218 and 219.

Thereafter, the chrominance signal was orthogonally modulated by the encoder 220, mixed with the luminance signal Y, sync, I, Jz, and S by the mixer 221, and outputted as a standard television signal.

FIG. 1 shows the configuration of the brightness signal adaptive filter processing section 205. Ii degree signal human power Y is Mg-Cy-Gr-Ye
It was read out in a zigzag pattern. As mentioned above, in the horizontal direction, the frequency of 1/4 of the clock f8 (1/4
Since the carrier exists at the position of 240 TV lines (1/4P) in the vertical direction, it is sufficient to determine the magnitude of the frequency component of this component.

The horizontal bandpass filter 101 is composed of a 5-tap digital filter (-1/2, 0, 1, 0.-172), as shown in the lower half of FIG. The vertical band pass filter 102 uses an IH (horizontal scanning period) memory as shown in the lower half of FIG.
It consists of a tap digital filter.

The determining unit 103 outputs a selection signal S to the switch 107 depending on the magnitude of the absolute values of the outputs of the horizontal hunter filter 101 and the vertical bandpass filter 102.

The switch 107 selects a signal output 104 that does nothing according to selection No. 15 S from the discriminator 103, an output of the horizontal low-pass filter 105, and a vertical low-pass filter 1.
06 output can be selected. As shown in the upper half of Figure 7J1.5, the horizontal low-pass filter 105 can have a configuration 1 by sharing a delay and a tap with the water hunt-pass filter 101. Here (1/2.
There is a 5-tap low-pass filter of 0, 1, 0° 1/2), and the checker pattern can be eliminated by setting the TH carry response to 0 in FIG. Vertical low pass filter 1
06 can also be constructed by sharing the one-tone direct band pass filter 101, IH memory, and tap, as shown in the upper part of FIG. Here, (1/2, 1.1/2)
There is a low-pass filter called TV' in Figure 4.
The checker pattern can be erased because the ki-answer on ksA is set to 0.

The operation of the discriminator 03 can be performed as shown in the following table.

Incidentally, a time when the absolute value of the output of the horizontal band-pass filter 101 is greater than a certain value is defined as an event A, and a time when the absolute value of the output of the vertical band-pass filter 102 is greater than a certain value is defined as an event B. The counter-events of A, Bt, iA, and B are shown.

For example, case 1 is a case where the structure of the subject changes significantly both in the horizontal direction and in the vertical direction, so there are A and B. In this case, even if there is a checker pattern, it is not noticeable. Conversely, if low-pass filtering is applied, the image will be blurred, so switch Sl is selected.

For example, in case 2, there is a component in the vertical direction of the subject, so even if you select switch S2 and apply a low-pass filter toward the water/P horn, the subject information will not be transmitted to #f1,
In addition, the checker pattern can also be removed.

In the above embodiment, the outputs of the vertical band-pass filters are used in combination to identify whether or not the part of the body changes rapidly. It is also possible to identify changes in the subject in the diagonal direction by using the filter and perform low-pass filtering in the diagonal direction when the change is small.

(Effects of the Invention) As explained above, according to the present invention, the checker pattern can be prevented without deteriorating the resolution.

[Brief explanation of drawings]

FIG. 1 is a block diagram of a luminance signal adaptive filter processing unit according to an embodiment of the present invention, FIG. 2 is a block diagram of the embodiment, and FIG. 3 is a diagram showing the arrangement of color filters used in the embodiment. , FIG. 4 is a characteristic diagram of colored light carriers in the color filter array shown in FIG. 8 shows a characteristic diagram of colored light carriers in the color filter array of FIG. 7, FIG. 10 shows a characteristic diagram of colored light carriers in the color filter array of FIG. 9, and FIG. FIG. 11 is a characteristic diagram of color light carriers in a color filter array, and FIG. 13 is a block diagram for explaining related technology. 101...Horizontal band pass filter 102...
... Vertical band pass filter 103 ... Discrimination section 105 ... Horizontal low pass filter 106 ...
...Vertical low-pass filter 107...Switch

Claims (2)

[Claims] (1) A discriminating means for discriminating whether or not there is a uniform area with little luminance change in a predetermined direction in an image, a processing means for low-pass filtering a luminance signal in the predetermined direction, and a discriminating means that and selecting means for selecting the processing means to perform low-pass filtering processing when it is determined that there is a similar region. (2) The image signal processing device according to claim 1, wherein the determining means determines the luminance signal based on the absolute value of the output of the horizontal and vertical band pass filters.