CN1300505A - Solid-state color imager - Google Patents

Abstract

A solid-state color imager comprising a solid-state color imaging element (2) having a color separating filter in which arrangement patterns are repeatedly provided, each arrangement pattern has four pixels adjacent to one another vertically and horizontally, the two upper pixels are a full color transmission filter and a cyan transmission filter arranged from the left, and the two lower pixels are a yellow transmission filter and a full color transmission filter (2) arranged from the left, storage means (5) for receiving color signals outputted from the pixels and storing them therein, correlation calculating means (6) for calculating the correlations of pixels around interpolation object pixels comprised of the cyan signal pixels and yellow signal pixels stored in the storage means, and interpolating means (7) for performing interpolation in the direction in which the correlation increases and calculating full color transmission signals in the positions of the interpolation object pixels.

Description

solid color camera

technical field

The present invention relates to an all-pixel readout solid-state imaging device, and more particularly to a method of arranging a dichroic filter for reducing resolution deterioration of luminance signals that occurs when information from the dichroic filter is matrix-calculated, and matrix calculation methods. In addition, in particular, it relates to a signal processing method for a high-resolution solid-state color imaging device by performing interpolation processing between pixels. The frequency characteristic of the signal is adjusted according to the color of the color separation filter, and frequency components including foldback distortion that cause false colors are reduced.

Background technique

Image signals are usually represented by the three primary colors of light of red (R), green (G), and blue (B), and also by luminance signal (Y) and two color difference signals (R-Y, B-Y). The three primary colors of RGB are generally in the form of an input signal to a monitor for a computer, and the luminance and color difference are in the form of a digital part in an element of a TV system. In addition, in recent years, image signals of solid-state color imaging devices have been used not only for image display but also for digital recording or image communication between components. The image signal has a large amount of information, and due to the constraints of recording capacity or communication capacity, image compression processing is usually performed. The format of the image signal used at that time was called 4:2:0 or 4:1:1 format, and compared with the 4:2:2 format used in the past, the color information was often half.

Next, a conventional solid-state color imaging device will be described with reference to FIG. 2 .

Fig. 2 (a) shows the solid-state color imaging device of prior art, and 1 is the optical system that makes the object to be imaged on the surface of the solid-state imaging element, and 2 is to convert the imaged object image (optical image) into an image signal (electrical image) signal) with a solid-state imaging element with a color separation filter, 3 is an AD converter that converts the image signal converted by the solid-state imaging element into a digital image signal, and 4 is converting the digital image signal into a luminance signal and a color difference signal image signal processing circuit. The color separation filter equipped on the surface of the aforementioned solid-state imaging element 2 is often composed of magenta (Mg), green (G), cyan (Cy), and yellow (Y), as shown in FIG. 2( b ). Dichroic filters in the complementary color and alternate mode shown.

Next, the operation of the solid-state color imaging device configured as above will be described.

In FIG. 2 , an object image (optical image) is formed on a solid-state imaging device 2 by an optical system 1 . The solid-state imaging device 2 outputs the captured subject image as an image signal color-separated by a color separation filter. The image signal is converted into a digital signal by the AD converter 3, supplied to the signal processing circuit 4, converted into a luminance signal Y and two kinds of color difference signals, and becomes a color image signal. In the image processing circuit, the luminance signal (Y) and a pair of color difference signals (R-Y, B-Y) of one pixel are generated according to four pixels of Mg, G, Cy, and Ye complementary color systems.

An example of the generation process of this luminance signal is shown below, Y(h, v) is

Y(0,1)=Mg(0,0)+G(1,0)+Cy(0,1)+Ye(1,1)

Y(1,0)=G(1,0)+Mg(2,0)+Ye(1,1)+Cy(2,1)

Y(0,1)=Cy(0,1)+Ye(1,1)+G(0,2)+Mg(1,2)

Y(1,1)=Ye(1,1)+Cy(2,1)+Mg(1,2)+G(2,2),

A luminance signal is generated from the outputs of the four pixels of the solid-state imaging element.

The color difference signal R-Y(h,v) is

R-Y(0,0)=Mg(0,0)-G(1,0)-Cy(0,1)+Ye(1,1)

R-Y(0,1)=-G(1,0)+Mg(2,0)+Ye(1,1)-Cy(2,1)

R-Y(1,0)=-Cy(0,1)+Ye(1,1)-G(0,2)+Mg(1,2)

R-Y(1,1)=Ye(1,1)-Cy(2,1)+Mg(1,2)-G(2,2),

B-Y(h,v) is

B-Y(0,0)=Mg(0,0)-G(1,0)+Cy(0,1)-Ye(1,1)

B-Y(0,1)=-G(1,0)+Mg(2,0)-Ye(1,1)+Cy(2,1)

B-Y(1,0)=Gy(0,1)-Ye(1,1)-G(0,2)+Mg(1,2)

B-Y(1,1)=-Ye(1,1)+Cy(2,1)+Mg(1,2)-G(2,2).

The output luminance and color-difference signals have the same number as the number of pixels of the solid-state imaging element, and have a 4:4:4 format. The conversion to the 4:2:2 format, 4:2:0, 4:1:1 format is carried out in comparison with the target output device.

Since false colors occur at the edge of luminance, edge determination is performed based on the edge signal of the combined luminance signal, and false color suppression is performed by reducing the gain of the color difference signal corresponding to the pixel determined to be an edge.

However, in the solid-state color imaging device of the prior art, no consideration has been made about image compression, and the output of the color separation filter in the form of 4:4:4 is taken as a premise, and in the form of 4:2:0, 4:1: In the case of device output based on the premise of image compression such as 1 format, 3/4 of the color information becomes unnecessary. In addition, the luminance signal is averaged by four pixels, for example, in the case of making Y(0,0) and Y(0,1) in Figure 2(b), because G(1,0) and Ye(1 ,1), so the luminance signal is no longer purely a sampling signal, but passes through a low-pass filter in the vertical and horizontal directions. deterioration. As for the color-difference signal, the conversion based on four adjacent pixels is not pure sampling information, but the vertical and horizontal directions are also passed through a low-pass filter, and the resolution is also deteriorated.

In order to solve the aforementioned problems, in the solid-state color imaging device of the present invention, the color separation filter on the surface of the solid-state imaging element is used as two panchromatic transmission filters, a cyan transmission filter, and a yellow transmission filter. A total of four pixels of the chip are used as an arrangement pattern, which is made to repeat the aforementioned arrangement pattern, and four luminance information are output from the aforementioned arrangement pattern, one for each of the two kinds of color information, and then when converting to the luminance signal and the color difference signal, The relationship between the pixel imaged by the subject and the surrounding pixels is obtained by correlation detection processing to calculate pixels in a direction with high correlation and convert them into luminance signals and color difference signals.

In addition, in such a solid-state imaging device of the present invention, a plurality of color signals are output via a plurality of color transmissive filters. This color signal is independent of each color signal, and its sampling rate is lower than that of the full signal if one focuses on a specific color signal. Therefore, aliasing occurs in each color signal, and frequency components having foldback distortion may be included.

FIG. 15 shows how the wrapping distortion occurs when a specific color signal is interpolated. In FIG. 15 , the horizontal axis represents the frequency, 2π represents the sampling frequency of the entire signal, and the vertical axis represents the amplitude of the signal. In addition, the solid line represents the color signal, and the dotted line represents the foldback distortion component. In the case of interpolating composite color signals containing high-frequency components, the foldback distortion components are contained in the passband lower than π/2 as shown in FIG. 15 . Therefore, new problems such as poor interpolation accuracy and false color signals occur.

In order to solve the above-mentioned problems, the solid-state color imaging device according to the present invention is characterized in that the output from the solid-state imaging element using each color separation filter such as panchromatic transmission, cyan transmission, and yellow transmission is adjusted by means of a frequency characteristic adjustment device. The frequency characteristic of each color signal is interpolated to synthesize the color difference signal by using the color signal adjusted for the characteristic.

In addition, it has an edge judgment function in the correlation detection process, which determines the gain in the color difference signal and adds it to the corresponding color difference signal, so as to suppress false colors.

Accordingly, in the present invention, it is possible to provide a solid-state color imaging device with less degradation in gradation resolution and less false color.

disclosure of invention

In order to solve the above-mentioned problems, the solid-state color imaging device described in claim 1 of the present invention is characterized in that it is provided with: a color separation filter having an arrangement pattern of four adjacent pixels vertically and horizontally, and the color separation filter of the arrangement pattern Filter, two pixels are panchromatic transmission filters, one pixel is cyan transmission filter, and one pixel is yellow transmission filter, which is composed of repeating the arrangement pattern of the aforementioned four pixels vertically and horizontally, and has individual A solid-state imaging element of a device for extracting information of each pixel of the color separation filter; and from the image information individually extracted from the aforementioned solid-state imaging element, for one of the aforementioned arrangement patterns, four luminance signals and two At this time, two of the aforementioned four luminance signals are made based on the information of only the aforementioned panchromatic transmission filter, and based on the information of the aforementioned panchromatic transmission filter and the surrounding pixels of the four vertically and horizontally adjacent pixels The information is used to make the other two, and the signal processing circuit for making the aforementioned two kinds of color-difference signals is based on the information of the aforementioned cyan or yellow transmission filter and the aforementioned surrounding pixel information. Thereby, two of the four luminance signals are created based on only the information of the panchromatic transmission filter, thereby improving the luminance resolution.

In addition, the solid-state color imaging device described in claim 2 of the present invention is the solid-state color imaging device described in claim 1, wherein the aforementioned color separation filter with four pixels adjacent vertically and horizontally as an arrangement pattern The slice is composed of two pixels in the vertical direction and two pixels in the horizontal direction. A total of six signals consisting of four luminance signals and two color-difference signals are generated based on the information extracted from the above-mentioned arrangement pattern, and output to the 4:2 :0 way elements. This has an effect of improving the luminance resolution in the device of the 4:2:0 system.

In addition, the solid-state color imaging device described in claim 3 of the present invention is the solid-state color imaging device described in claim 1, wherein the aforementioned color separation filter with four pixels adjacent vertically and horizontally as an arrangement pattern The slice is composed of one pixel in the vertical direction and four pixels in the horizontal direction, and a total of six signals consisting of four luminance signals and two color-difference signals are generated based on the information extracted from the above-mentioned arrangement pattern, and output to 4:1: 1 way element. This has an effect of improving the luminance resolution in the device of the 4:1:1 system.

In addition, the solid-state color imaging device described in claim 4 of the present invention is the solid-state color imaging device described in claim 1, characterized in that it includes: the aforementioned color separation method in which four pixels adjacent vertically and horizontally are used as an arrangement pattern. Filters, the upper two pixels are the panchromatic transmission filter and the cyan transmission filter from the left, the lower two pixels are the yellow transmission filter and the panchromatic transmission filter from the left, and The repeated pattern person of forming; respectively accept and store the color signal output from each pixel of the aforementioned solid-state imaging element and store it; with the cyan signal pixel and the yellow signal pixel stored in the storage device as interpolated pixels, the calculation For each of the pixels to be interpolated, a correlation calculation device for the correlation of a plurality of pixels located around each of the pixels to be interpolated; An interpolation processing device for the panchromatic transmission signal at the position of the pixel to be interpolated. Thereby, since the input image is converted into a luminance signal after being interpolated with pixels with high correlation, there is an effect of reducing deterioration of luminance resolution.

In addition, the solid-state color imaging device according to claim 5 of the present invention is the solid-state color imaging device according to claim 4, wherein the correlation calculation means calculates the interpolated pixel and its surrounding pixels In , the horizontal or vertical correlation including interpolated pixels. Thereby, there is an effect of reducing deterioration of luminance resolution in the vertical and horizontal directions.

Furthermore, the solid-state color imaging device according to claim 6 of the present invention is the solid-state color imaging device according to claim 4, wherein the correlation calculation means calculates the correlation between the interpolated pixel and its surrounding pixels Among them, the horizontal or vertical correlation including interpolated pixels, and further, the oblique correlation. Thereby, there is an effect of reducing deterioration of luminance resolution in the vertical and horizontal directions, and in the oblique direction.

In addition, the solid-state color imaging device according to claim 7 of the present invention is the solid-state color imaging device according to claim 4, wherein the correlation calculation means calculates the correlation between the interpolated pixel and its surrounding pixels In , the horizontal or vertical correlation including the interpolated pixel, and then the right and upward, or right and downward, or left and upward, or left and downward correlation. Thereby, there is an effect of reducing deterioration of luminance resolution in the vertical direction, the horizontal direction, and the upper right L-shaped direction, the lower right L-shaped direction, the upper left L-shaped direction, and the lower left L-shaped direction.

In addition, the solid-state color imaging device according to claim 8 of the present invention is the solid-state color imaging device according to claim 4, wherein the correlation calculation means calculates the correlation between the interpolated pixel and its surrounding pixels In , the horizontal or vertical correlation including interpolated pixels, and then the oblique correlation, and then right and up, or right and down, or left and up, or left and downward correlation. Thereby, there is an effect of reducing deterioration of luminance resolution in the vertical, horizontal, oblique, upper right L-shaped directions, right lower L-shaped directions, upper left L-shaped directions, and left lower L-shaped directions.

In addition, the solid-state color imaging device according to claim 9 of the present invention is the solid-state color imaging device according to claim 4, wherein the correlation degree calculating means calculates the difference between the pixel to be interpolated and the pixels located around it. Between, the operation of the same color signals to calculate the correlation. Thereby, by using the same color signal to calculate the correlation degree, there is an effect that the calculation accuracy of the correlation degree is improved.

In addition, the solid-state color imaging device according to claim 10 of the present invention is the solid-state color imaging device according to claim 4, wherein the correlation calculation means calculates , which becomes an operation between adjacent pixels of different color signals to calculate the correlation. Thereby, by using adjacent pixels close to the interpolated pixel for calculation, there is an effect of improving the calculation accuracy of the correlation degree even for signals of different colors.

Furthermore, the solid-state color imaging device described in claim 11 of the present invention is the solid-state color imaging device described in claim 4, wherein the interpolation processing means does not use the correlation calculated by the correlation calculation means. For the color signal of the pixel to be interpolated in the direction with a large degree, the interpolation process is performed using only signals of the same color as the color signal to be generated around the pixel to be interpolated. Thereby, the interpolation accuracy is improved and the luminance resolution is improved.

Furthermore, the solid-state color imaging device described in claim 12 of the present invention is the solid-state color imaging device described in claim 4, wherein the interpolation processing means uses the correlation For the color signal of the pixel to be interpolated in the direction with a large degree, the shortage of the color signal to be generated is calculated from the pixels around the pixel to be interpolated, and an interpolation process is performed. In this way, only the insufficient color components are interpolated, and the other components are interpolated using the color signals of the pixels to be interpolated. Therefore, the interpolation accuracy is improved and the luminance resolution is less likely to be deteriorated.

Furthermore, the solid-state color imaging device described in claim 13 of the present invention is the solid-state color imaging device described in any one of claims 5 to 10, wherein if calculated by the above-mentioned correlation calculation means If the correlation degree is smaller than a given threshold value, the above-mentioned interpolation processing device executes a process of reducing the gain of the color difference signal corresponding to the pixel. Thereby, there is an effect of reducing the false color that occurs at the edge of the luminance signal.

Furthermore, the solid-state color imaging device described in claim 14 of the present invention is the solid-state color imaging device described in any one of claims 5 to 10, wherein if calculated by the above-mentioned correlation calculating means If the degree of correlation is smaller than a given threshold, the interpolation processing means executes a process of gradually reducing the gain of the color difference signal corresponding to the pixel according to the degree of correlation. Thereby, there is an effect of adaptively reducing the false color that occurs at the edge of the luminance signal.

Furthermore, the solid-state color imaging device according to claim 15 of the present invention is the solid-state color imaging device according to claim 4, wherein the interpolation processing means is provided with a function for adjusting each output from the solid-state imaging element. A frequency characteristic adjustment device for adjusting the frequency characteristic of a color signal interpolates and synthesizes a color difference signal using the color signal subjected to the frequency characteristic adjustment. Thereby, there is an effect of reducing false color signals that occur when interpolating and combining color-difference signals by interpolation using color signals including high-frequency components.

In addition, the solid-state color imaging device according to claim 16 of the present invention is the solid-state color imaging device according to claim 15, wherein the interpolation processing means is provided with a function for adjusting each output from the solid-state imaging element. A frequency characteristic adjustment device for the frequency characteristic of a color signal, using the color signal adjusted for the frequency characteristic to interpolate and synthesize R-Y color difference signals at the position of the cyan transmission filter, and to interpolate and synthesize B-Y at the position of the yellow transmission filter color difference signal. Thereby, there is an effect of reducing false color signals that occur when interpolating and combining color-difference signals by interpolation using color signals including high-frequency components.

Furthermore, the solid-state color imaging device according to claim 17 of the present invention is the solid-state color imaging device according to claim 15, wherein the interpolation processing means is provided with When there is a direction with a large correlation degree, the frequency characteristic adjustment is performed, and when there is no direction with a large correlation degree, the frequency characteristic adjustment is not performed. Thereby, in the interpolation synthesis using the color difference of the color signal having no correlation direction, the frequency components are preserved, and there is an effect of maintaining the color reproducibility of the image.

Furthermore, the solid-state color imaging device described in claim 18 of the present invention is the solid-state color imaging device described in claim 16, wherein the interpolation processing means is provided with When there is a direction with a large correlation degree, the frequency characteristic adjustment is performed, and when there is no direction with a large correlation degree, the frequency characteristic adjustment is not performed. Thereby, in the interpolation synthesis using the color difference of the color signal having no correlation direction, the frequency components are preserved, and there is an effect of maintaining the color reproducibility of the image.

As above, the present invention has two panchromatic transmission filters, one cyan transmission filter, and one yellow transmission filter in the adjacent vertical and horizontal four pixels of the color separation filter on the surface of the solid-state imaging element, with By repeating the pattern of four pixels and installing a circuit for extracting four luminance information and two color information from the four pixels in the repeating pattern, high luminance resolution and no deterioration in color resolution can be realized. , an excellent solid-state color camera device. In addition, by adding a device respectively receiving and storing the signals output by each pixel to the above-mentioned device, the cyan signal pixel and the yellow signal pixel stored in the storage device are used as interpolated pixels, and the calculation corresponding to the above-mentioned interpolated pixel A correlation calculation device for the correlation of a plurality of surrounding pixels, and a device for interpolating in the direction of a large correlation to calculate the panchromatic transmission signal of the position of the interpolated pixel, providing a device that can reduce the brightness resolution It becomes possible to implement a solid-state color imaging device that suppresses false colors occurring at the edge of the luminance signal without adding a large processing circuit.

Brief description of the drawings

Fig. 1(a) is a block diagram of the solid-state color imaging device of the present invention.

FIG. 1( b ) is a schematic diagram of a dichroic filter disposed on the solid-state imaging device of FIG. 1( a ).

FIG. 1( c ) is a schematic diagram of a dichroic filter arranged on the solid-state imaging device of FIG. 1( a ).

FIG. 2(a) is a block diagram of a conventional solid-state color imaging device.

FIG. 2( b ) is a schematic diagram of a dichroic filter disposed on the solid-state imaging device of FIG. 2( a ).

Fig. 3(a) is a diagram showing positions of luminance and color difference signals in the 4:2:0 format of the solid-state color imaging device in the first embodiment of the present invention.

Fig. 3(b) is a diagram showing positions of luminance and color difference signals in the 4:1:1 format of the solid-state color imaging device in the second embodiment of the present invention.

Fig. 4 is an example diagram of an arrangement pattern of dichroic filters suitable for output in a 4:2:0 format in the first embodiment of the present invention.

Fig. 5 is an example diagram of an arrangement pattern of color separation filters suitable for output in a 4:1:1 format in the second embodiment of the present invention.

6 is an explanatory view of third to seventh solid-state color imaging devices according to the present invention, (a) is a configuration diagram of the solid-state color imaging device, and (b) is a configuration diagram of a color separation filter on a solid-state imaging device.

Fig. 7 is a diagram illustrating correlation degree calculation, correlation direction (vertical, horizontal) and interpolation processing in the sixth embodiment of the present invention.

Fig. 8 is a diagram illustrating correlation degree calculation, correlation direction (right obliquely downward, left obliquely downward), and interpolation processing in the eighth embodiment of the present invention.

Fig. 9 is a diagram illustrating correlation calculation, correlation directions (L-shaped four directions), and interpolation processing in the fifth embodiment of the present invention.

Fig. 10 is a diagram illustrating the relationship between the degree of correlation and the gain added to the color-difference signal in the ninth embodiment of the present invention.

Fig. 11 is a diagram illustrating the relationship between the degree of correlation and the gain added to the color-difference signal in the ninth embodiment of the present invention.

Fig. 12 is a configuration diagram of a solid-state color imaging device according to a tenth embodiment of the present invention.

Fig. 13 is a diagram for explaining the frequency characteristic adjustment operation in the tenth embodiment of the present invention.

Fig. 14 is a diagram for explaining the frequency characteristic adjustment operation in the tenth embodiment of the present invention.

Fig. 15 is a diagram illustrating the frequency characteristic adjustment operation in the tenth embodiment of the present invention.

Fig. 16 is a configuration diagram of a solid-state color imaging device according to an eleventh embodiment of the present invention.

The best form for carrying out the invention

The embodiments of the present invention are described below

first embodiment

Next, a first embodiment corresponding to claim 1 and claim 2 of the present invention will be described.

FIG. 1(a) shows a solid-state color imaging device according to a first embodiment of the present invention. In the figure, 1 is an optical system composed of lenses and the like for realizing imaging of an object to be photographed on the surface of the solid-state imaging device. 2 is to realize the conversion of the imaged subject image (optical image) into an image signal (electrical signal), and is composed of a solid-state imaging element with a color separation filter. Reference numeral 3 realizes the conversion of the image signal obtained from the solid-state imaging element into a digital image signal, and is constituted by an AD converter. 4 is to realize the conversion of the digital image signal obtained from the AD converter into a luminance signal and a color difference signal, and is composed of an image signal processing circuit.

Fig. 1(b) shows a dichroic filter of a solid-state imaging element attached to the surface of the solid-state imaging element 2 in Fig. 1(a), showing an example pattern in which two pixels in the vertical direction and two pixels in the horizontal direction are repeated , the filter configuration consists of the upper two pixels from left to panchromatic transmission filter, cyan transmission filter, the lower two pixels from left to yellow transmission filter, panchromatic transmission filter pieces to form.

FIG. 3(a) shows input and output signals of the image signal processing circuit 4 in FIG. 1(a) in the first embodiment.

In Fig. 1(a), the subject is imaged on the surface of the solid-state imaging device through the optical system 1, and the imaged subject image (optical image) is converted into an image by the solid-state imaging device 2 with a color separation filter. signal (electrical signal), the image signal obtained from the solid-state imaging element is converted into a digital image signal by the AD converter 3, and the digital image signal obtained from the AD converter is converted by the image signal processing circuit 4 into a luminance signal and color difference signal. Furthermore, the arrangement of the dichroic filters attached to the solid-state imaging element 2 is a structure in which two pixels in the vertical direction and two pixels in the horizontal direction are repeated as shown in FIG. The two pixels are panchromatic transmission filter and cyan transmission filter from left, and the bottom two pixels are yellow transmission filter and panchromatic transmission filter from left, making the four pixels Two have panchromatic transmission filters, and the other two pixels are composed of cyan transmission filters and yellow transmission filters. At this time, the image signal obtained from the solid-state imaging element becomes four in total of two panchromatic information, one cyan information, and one yellow information, and matrix calculation is performed on these four, and four luminances are output from the image signal processing circuit 4 Signal, one R-Y color difference signal, one B-Y color difference signal.

Next, the conversion operation to the luminance signal and the color difference signal in the image signal processing circuit 4 will be described with reference to FIG. 3( a ).

If each transmission filter is used to represent the primary color components of light (red, green, blue, respectively R, G, B), then usually W=R+G+B, Cy=G+B, Ye=R +G. The luminance information is composed of all components of R, G, and B, and the luminance signal Y about the position of the panchromatic transmission filter is made only by the signal from the panchromatic transmission filter which becomes pure sampling information, which is approximated as

Y(h,v)=a×W(h,v)

In the formula, a is a coefficient for adjusting the dynamic range, and h+v is always an even number in the example of Fig. 3(a).

In addition, the luminance signal Y of the position without the panchromatic transmission filter is also created using the information of the surrounding positions. As a simple creation method, it is approximated as

Y(h,v)=a×((W(h-1,v)+W(h+1,v)+W(h,v-1)+W(h,v+1))÷4)

In the formula, a is a coefficient for adjusting the dynamic range, and h+v is always an odd number in the example of Fig. 3(a).

Alternatively, to generate color information at an appropriate position as pure sampling information, if it is the luminance signal Y at the position of the cyan filter, since Cy does not have the R component in the luminance signal component, it is interpolated according to the surrounding pixels. complement, approximately

R(h,v)=a×(W(h-1,v)+W(h+1,v)+W(h,v-1)+W(h,v+1))÷4

-b×(Cy(h,v)×4+Cy(h-2,v)+Cy(h+2,v)+Cy(h,v-2)+Cy(h,v+2))÷ 8

Y(h,v)=b×Cy(h,v)+R(h,v)

If it is the luminance signal Y at the location of the yellow filter, since Ye has no B component in the luminance signal component, it is interpolated according to the surrounding pixels, which is approximately

B(h,v)=a×(W(h-1,v)+W(h+1,v)+W(h,v-1)+W(h,v+1))÷4

-c×(Ye(h,v)×4+Ye(h-2,v)+Ye(h+2,v)+Ye(h,v-2)+Ye(h,v+2))÷ 8

Y(h,v)=c×Ye(h,v)+B(h,v)

In the formula, b and c are the coefficients used to adjust the dynamic range, h+v is always an odd number in the example of Figure 3(a), h is an odd number at the location of the cyan filter, v is an even number, and the yellow filter h at the location of the slice is an even number, and v is an odd number.

The Y, R, and B components of the luminance signal obtained from the Cy pixel and the Ye pixel are generated by interpolation using information from surrounding pixels, and are not obtained from pure sampling information by the solid-state imaging device. However, for the Cy pixel, the G+B component, and for the Ye pixel, the R+G component is retained as pure sampling information, and the interpolated R and B components are at most one-third of the luminance signal Y, The influence is small, and the luminance signal Y remains in a high-resolution state.

In addition, as far as the color-difference signal (R-Y, B-Y) is concerned, one piece of information is taken out for each of the four luminance signals, and two pixels in the vertical direction and two pixels in the horizontal direction of the luminance signal are used as a model. With a simple calculation method, first Convert to R, G, B. The R and B components used for color difference signal conversion are

R(h,v)=a×W((h div 2)*2,(v div 2)*2)-b×Cy((h div 2)*2+1,(v div 2)*2)

B(h,v)=a×W((h div 2)*2+1,(v div 2)*2+1)-c×Ye((h div 2)*2,(v div 2)* 2+1)

Then G becomes

G(h,v)=a×(W((h div 2)*2,(v div 2)*2)+W((h div 2)*2+1,(v div 2)*2+1 ))

÷2-R(h,v)-B(h,v)

According to this RGB, the color difference signal is approximately approximated as

R-Y(h,v)=2×R(h,v)-G(h,v)

B-Y(h,v)=2×B(h,v)-G(h,v)

In the formula, a, b, c are the coefficients used to adjust the dynamic range, div represents the calculation of only taking out the quotient of the division operation of integers and discarding the remainder, and * represents the multiplication operation.

The pair of color-difference signals obtained in this way is converted without repeating the output of the solid-state imaging element with respect to another adjacent color-difference signal, so that the color resolution is improved.

By performing the above operations, as shown in Figure 3(a), four luminance signals Y for two pixels in the vertical direction and two pixels in the horizontal direction are obtained, and two color difference signals R-Y and B-Y are obtained respectively, as shown in the figure The format shown in 3(a) is suitable as the format of the input signal to the element of the 4:2:0 format.

Furthermore, by setting the transmittance of the panchromatic transmission filter to 0.3 with respect to R, 0.59 with respect to B, and 0.11 with respect to G in addition to the above, since the transmissivity of the panchromatic transmission filter The ratio of the transmittance is equal to the primary color mixing ratio of the luminance signal Y, so a pure luminance signal can be obtained, which can further improve the resolution. That is to say, although the primary color components (red, green, blue, respectively, R, G, B) are used to represent each transmission filter, it becomes W=R+G+B, Cy=G+B, Ye =R+G, but if the above settings are made, the RGB mixing ratio of the luminance signal Y is expressed as

Y=0.30R+0.59G+0.11B

Each transmission filter used in this first embodiment is adjusted to become

W=0.30R+0.59G+0.11B

Cy=(0.59G+0.11B)÷0.7

Ye=(0.30R+0.59G)÷0.89

At this time, the luminance signal Y at the position of the panchromatic transmission filter is created only from the signal from the panchromatic transmission filter, which becomes pure sampling information, and becomes

Y(h,v)=a×W(h,v)

In the formula, a is a coefficient for adjusting the dynamic range, and h+v is always an even number in the example of Fig. 3(a). In addition, the luminance signal Y at the position where there is no panchromatic transmission filter is also created using the information of the surrounding position. As a simple creation method, it is

Y(h,v)=a×((W(h-1,v)+W(h+1,v)+W(h,v-1)+W(h,v+1))÷4)

Alternatively, to generate color information at an appropriate position as pure sampling information, if it is the luminance signal Y at the position of the cyan filter, then it can also be

Y(h,v)=a×((W(h-1,v)+W(h+1,v)+W(h,v-1)+W(h,v+1))÷4) +0.7×b×(Cy(h-2,v)÷2-(Cy(h+2,v)+Cy(h,v-2)+Cy(h,v+2)+Cy(h,v ))÷8)

If it is the luminance signal Y at the location of the yellow filter, it can also become

Y(h,v)=a×((W(h-1,v)+W(h+1,v)+W(h,v-1)+W(h,v+1))÷4) +0.7×c×(Ye(h-2,v)÷2-(Ye(h+2,v)+Ye(h,v-2)+Ye(h,v+2)+Ye(h,v ))÷8)

In the formula, b and c are the coefficients used to adjust the dynamic range, h+v is always an odd number in the example of Figure 3(a), h is an odd number at the location of the cyan filter, v is an even number, and the yellow filter h at the location of the slice is an even number, and v is an odd number.

In addition, as far as the color-difference signal (R-Y, B-Y) is concerned, one piece of information is taken out for each of the four luminance signals, and two pixels in the vertical direction and two pixels in the horizontal direction of the luminance signal are used as a model. With a simple calculation method, it becomes

R-Y(h,v)=0.7÷0.3×(a×(W((h div 2)*2,(v div 2)*2)+W((h div 2)*2+1,(vdiv 2) *2+1))÷2-b×Cy((h div 2)*2+1,(v div 2)*2))

B-Y(h,v)=0.89÷0.11×(a×(W((h div 2)*2,(v div 2)*2)+W((h div 2)*2+1,(vdiv 2) *2+1))+2-b×Ye((h div 2)*2,(v div 2)*2+1))

Alternatively, considering the sampling position of the color-difference signal, it can also be

R-Y(h,v)=0.7+0.3×(a×(W((h div 2)*2,(v div 2)*2)+W((h div 2)*2+1,(vdiv 2) *2+1))÷2-b×(Cy((h div 2)*2+1,(v div 2)*2)×2+Cy((h div 2)*2-1,(vdiv 2 )*2)+Cy((h div 2)*2+1,(v div 2)*2+2))÷4)

B-Y(h,v)=0.89÷0.11×(a×(W((h div 2)*2,(v div 2)*2)+W((h div 2)*2+1,(vdiv 2) *2+1))÷2-b×(Ye((h div 2)*2,(v div 2)*2+1)×2+Ye((h div 2)*2,(vdiv 2)* 2-1)+Ye((h div 2)*2+2,(v div 2)*2+1))÷4)

In addition, although in the above-mentioned first embodiment, the arrangement of the panchromatic filters is set to be an alternate arrangement, it is also possible to place the upper side in the repeated pattern of two pixels in the vertical direction and two pixels in the horizontal direction as shown in Figure 4(a). Two pixels are taken as panchromatic transmission filters, and the lower two pixels are taken as blue-green transmission filters and yellow transmission filters, and the panchromatic transmission filters are horizontally and continuously arranged to improve the horizontal resolution as an advantage. Furthermore, the disposition of the panchromatic filter can also be like Fig. 4 (b) in the repeated pattern of two pixels in the vertical direction and two pixels in the horizontal direction. The repeating pattern of the filter, the bottom two pixels from the left are panchromatic transmission filter and yellow transmission filter, and the panchromatic transmission filter is arranged vertically and continuously, as an advantage, the effect of vertical resolution improvement is obtained. In addition, the same resolution can also be obtained by adopting a structure in which the pattern of the above four pixels is changed for each pattern.

Furthermore, although the above-mentioned first embodiment has been described for the occasion of performing 4:2:0 output, although the color resolution is lowered when the color difference signal is converted, it is possible to arrange R at any position by using surrounding pixels. , G, B components, so not only this 4:2:0 output, but also 4:4:4, 4:2:2, 4:1:1 output is also possible.

In addition, although in the above-mentioned first embodiment, the cyan transmission filter and the yellow transmission filter are used in addition to the panchromatic transmission filter, it is also possible to use the red transmission filter in addition to this panchromatic transmission filter. Light sheet, blue transmitting filter. The advantage is that it is not necessary to take out the R component and B component from the filter, which can simplify the calculation. The disadvantage is that because the green component G is not included in the red filter and the blue filter, the G component is also supplemented from surrounding pixels, so the resolution of the luminance information is reduced.

In addition, although in the description of the above-mentioned first embodiment, it has been described that the upper two pixels of the dichroic filter having two pixels in the vertical direction and two pixels in the horizontal direction are composed of a panchromatic transmission filter, a cyan transmission filter from the left, and a cyan transmission filter. For the light sheet, the lower two pixels are composed of a yellow transmission filter and a panchromatic transmission filter from the left, but the arrangement of the other filters is changed as the transmission color remains unchanged, or two Change the transmission color of the cyan and yellow transmission filters to cyan and magenta, or magenta and yellow, or red and blue, or red and green, or green with the panchromatic transmission filter intact The same can be done with the composition of blue.

2nd embodiment

Next, a second embodiment corresponding to claim 1 and claim 3 of the present invention will be described.

Fig. 1(c) shows the color separation filter of the solid-state color imaging element attached to the surface of the solid-state imaging element 2 in Fig. 1(a), showing an example pattern in which one pixel in the vertical direction and four pixels in the horizontal direction are repeated. , The configuration of the filters is composed of panchromatic transmission filter, cyan transmission filter, panchromatic transmission filter, and yellow transmission filter from the left.

FIG. 3(b) shows input and output signals of the image signal processing circuit 4 in FIG. 1(a) in the second embodiment.

In Fig. 1(a), the subject is imaged on the surface of the solid-state imaging device through the optical system 1, and the imaged subject image (optical image) is converted into an image by the solid-state imaging device 2 with a color separation filter. signal (electrical signal), the image signal obtained from the solid-state imaging element is converted into a digital image signal by the AD converter 3, and the digital image signal obtained from the AD converter is converted by the image signal processing circuit 4 into a luminance signal and color difference signal. Furthermore, the arrangement of the dichroic filters attached to the solid-state imaging device 2 is a pattern of repeating one pixel in the vertical direction and four pixels in the horizontal direction as shown in Figure 1 (c). Panchromatic transmissive filter, cyan transmissive filter, panchromatic transmissive filter, yellow transmissive filter from left, making two of the four pixels have panchromatic transmissive filters, and the other two pixels It is composed of cyan transmission filter and yellow transmission filter. At this time, the image signal obtained from the solid-state imaging element becomes four in total of two panchromatic information, one cyan information, and one yellow information, and matrix calculation is performed on these four, and four luminances are output from the image signal processing circuit 4 Signal, one R-Y color difference signal, one B-Y color difference signal.

Next, the conversion operation to the luminance signal and the color difference signal in the image signal processing circuit will be described with reference to FIG. 3(b).

If each transmission filter is used to represent the primary color components of light (red, green, blue, respectively R, G, B), then usually W=R+G+B, Cy=G+B, Ye=R +G. The luminance information is composed of all components of R, G, and B, and the luminance signal Y about the position of the panchromatic transmission filter is made only by the signal from the panchromatic transmission filter which becomes pure sampling information, which is approximated as

Y(h,v)=a×W(h,v)

In the formula, a is a coefficient for adjusting the dynamic range, and h is always an even number in the example of Fig. 3(b).

In addition, the luminance signal Y of the position where there is no panchromatic transmission filter is also created using the information of the surrounding position. As a simple creation method, it is approximated as Y(h,v)=a×((W(h-1,v )+W(h+1,v))÷2)

In the formula, a is a coefficient for adjusting the dynamic range, and h is always an odd number in the example of Fig. 3(b).

Alternatively, to generate color information at an appropriate position as pure sampling information, if it is the luminance signal Y at the position of the cyan filter, since Cy does not have the R component in the luminance signal component, it is interpolated according to the surrounding pixels. complement, approximately

R(h,v)=a×((W(h-1,v)+W(h+1,v))×2+W(h-1,v-1)+W(h+1,v -1)+W(h-1,v+1)+W(h+1,v+1))÷8-b×(Cy(h,v)×2+Cy(h,v-1)+ Cy(h,v+1))÷4Y(h,v)=b×Cy(h,v)+R(h,v)

If it is the luminance signal Y at the position of the yellow filter, since Ye has no B component in the luminance signal component, it can be interpolated according to the surrounding pixels, which can also be approximated as

B(h,v)=a×((W(h-1,v)+W(h+1,v))×2+W(h-1,v-1)+W(h+1,v -1)+W(h-1,v+1)+W(h+1,v+1))÷8-c×(Ye(h,v)×2+Ye(h,v-1)+ Ye(h,v+1))÷4Y(h,v)=c×Ye(h,v)+B(h,v)

In the formula, b and c are the coefficients used to adjust the dynamic range. If mod is the calculation of the remainder of the division operation that only takes out integers, then the h at the position of the cyan filter is (hmod 4)=1, and the yellow filter The h of the position of the light sheet is (h mod 4)=3.

The R and B components necessary for obtaining the above-mentioned luminance signal Y from Cy pixels and Ye pixels are created by interpolation using information of surrounding pixels, so the above-mentioned luminance signal Y is not purely generated by the solid-state imaging element. obtained from sampling information. However, for the Cy pixel, the G+B component, and for the Ye pixel, the R+G component is retained as pure sampling information, and the interpolated R and B components are at most one-third of the luminance signal Y, The influence is small, and the luminance signal Y remains in a high-resolution state.

In addition, as far as the color-difference signal (R-Y, B-Y) is concerned, one piece of information about each of the four luminance signals is taken out. Taking one pixel in the vertical direction and four pixels in the horizontal direction of the luminance signal as a model, use a simple calculation method. First, transform into R, G, B. The R and B components used for color difference signal conversion are

R(h,v)=a×W((h div 4)*4,v)-b×Cy((h div 4)*4+1,v)

B(h,v)=a×W((h div 4)*4+2,v)-c×Ye((h div 4)*4+3,v)

Then G becomes

G(h,v)=a×(W((h div 4)*4,v)+W((h div 4)*4+2,v))÷2-R(h,v)-B( h, v)

According to this RGB, the color difference signal is approximately approximated as

R-Y(h,v)=2×R(h,v)-G(h,v)

B-Y(h,v)=2×B(h,v)-G(h,v)

In the formula, a, b, c are the coefficients used to adjust the dynamic range.

The pair of color-difference signals obtained in this way is converted without repeating the output of the solid-state imaging device for other adjacent color-difference signals, so that the color resolution is improved.

By performing the above operations, as shown in Figure 3(b), four luminance signals Y for one pixel in the vertical direction and four pixels in the horizontal direction are obtained, and two color difference signals R-Y and B-Y are obtained respectively, as shown in Figure 3 The format shown in (b) is suitable as the format of the input signal to the element of the 4:1:1 format.

Furthermore, by setting the transmittance of the panchromatic transmission filter to 0.3 with respect to R, 0.59 with respect to B, and 0.11 with respect to G in addition to the above, since the transmissivity of the panchromatic transmission filter The ratio of the transmittance is equal to the primary color mixing ratio of the luminance signal Y, so a pure luminance signal can be obtained, which can further improve the resolution. This point can be calculated by the same method as the matrix change in the first embodiment.

In addition, similarly to the above-mentioned first embodiment, although the color resolution decreases when the color-difference signal is converted, since the R, G, and B components can be arranged at any position by using surrounding pixels, not only this 4:1 :1 output, and 4:4:4, 4:2:2, 4:2:0 output is also possible.

Furthermore, the configuration of the panchromatic filter is different from that in Fig. 1(c) and Fig. 3(a). When vertically repeating, the transmissive filters are staggered by one pixel in turn, and the panchromatic transmissive filters are made into alternate configurations, thereby improving the oblique resolution of the luminance signal. Furthermore, as shown in Figure 5(b), the configuration of the transmission filter other than panchromatic is replaced with the panchromatic transmission filter of Figure 1(c), and a transmission filter that replaces cyan and yellow is made. The repeated configuration of the pattern of eight pixels of the slice can realize the 4:1:1 method of one pixel in the vertical direction and four pixels in the horizontal direction, and the 4:2:0 method of two pixels in the vertical direction and two pixels in the horizontal direction. Two-sided, filter configuration with color resolution.

In addition, although in the description of the second embodiment, it is explained that the color separation filter with one pixel in the vertical direction and four pixels in the horizontal direction is composed of a panchromatic transmission filter, a cyan transmission filter, a panchromatic transmission filter from the left. filter, yellow transmission filter to form an example, but the transmission color of the other filters is unchanged and the arrangement is changed, or two panchromatic transmission filters are intact and the cyan and yellow The configuration in which the transmission color of the transmission filter is changed to cyan and magenta, or magenta and yellow, or red and blue, or red and green, or green and blue can be implemented similarly. The same effect can be obtained by arranging two panchromatic transmission filters and two kinds of color filters each of which transmits a non-panchromatic color as a repeating pattern of four pixels in the repeating pattern of transmission filters. In addition, as far as the color separation filter pattern of one pixel in the vertical direction and four pixels in the horizontal direction is prepared, four patterns are prepared in the vertical direction, and the arrangement of the color separation filters in the four patterns is different. able to implement.

3rd embodiment

Next, a third embodiment corresponding to claim 4, claim 5, claim 9 and claim 12 of the present invention will be described with reference to Fig. 6 and Fig. 7 .

In FIG. 6( a ), 1 is an optical system, which realizes imaging of an object to be photographed on a solid-state imaging device, and is composed of lenses and the like. 2 is a solid-state imaging element with a color separation filter, which converts the imaged object image (optical image) into an image signal (electrical signal). 3 is an AD converter, which converts the image signal obtained from the solid-state imaging device 2 into a digital image signal. 5 is a storage circuit that stores the digital image signal converted by the AD converter 3 for one screen. 6 is a correlation calculation circuit, which calculates the correlation between any pixel of the digital image signal stored in the storage circuit 6 and surrounding pixels. 7 is an interpolation processing circuit, which performs interpolation processing based on the correlation degree calculated from the correlation degree calculation circuit 6, and outputs a luminance signal and a color difference signal. By these optical system 1, solid-state imaging element 2 with dichroic filter, AD converter 3, storage circuit 5, correlation degree calculation circuit 6, and interpolation processing circuit 7, each circuit produces a luminance signal and a color difference signal .

FIG. 6( b ) shows the configuration of the dichroic filter on the solid-state imaging device 2 . Taking the four adjacent pixels vertically and horizontally as an arrangement unit, the configuration of the filter consists of panchromatic transmission filter and cyan transmission filter from the left for the upper two pixels, and yellow transmission filter for the lower two pixels from the left. film, panchromatic transmission filter to form. This array unit is continuously and repeatedly arranged vertically and horizontally.

If RGB components are used to represent the W pixel, Cy pixel, and Ye pixel of the income storage circuit (not shown), it can be expressed as W=(R+G+B)/3, Cy=(G+B)/3, Ye=(R+B)/3, if the W pixel W≈Y is set, the output signal of the W pixel can be expressed as a luminance signal intact. As for the Cy pixel and the Ye pixel, the R component and the B component are obtained by interpolation and added to the Cy pixel and Ye pixel, respectively, to express the luminance signal. Although the signals of surrounding pixels are used in interpolation, the correlation degree calculation circuit 6 calculates the degree of correlation with the pixel to be interpolated to determine the surrounding pixels used in interpolation. Firstly, the calculation method of the correlation degree will be described at the beginning.

FIG. 7 shows the arrangement of surrounding pixels when the cyan pixel Cyn is used as the pixel to be interpolated, and the symbols ● and ○ are Ye pixels and W pixels that are unnecessary in the interpolation process of this Cyn pixel. Let Vc be the vertical correlation degree in the ①-①' direction shown in FIG. 7, and Hc be the horizontal correlation degree in the ②-②' direction, respectively, to be calculated by the following equations.

Vc=｜Wu-Wd｜+｜Cyu-Cyn｜+｜Cyd-Cyn｜ (1)

Hc=｜Wl-Wr｜+｜Cyl-Cyn｜+｜Cyr-Cyn｜ (2)

Using this result, the correlation direction is determined according to the following conditional expression.

Vc+Th<Hc (3)

Hc+Th<Vc (4)

In addition, Th is a threshold value and is a specific constant. The relevant direction is judged to be the vertical direction when the expression (3) holds true, and it is judged to be the horizontal direction when the expression (4) holds true. When neither the expression (3) nor the expression (4) holds, it is judged that there is no relevant direction.

The interpolation processing will be described below.

When the relevant direction is judged to be vertical, the pixels used in the interpolation process use only the peripheral pixels in the vertical direction for the interpolated pixel Cyn, and the deficit component RCy is calculated by the following equation.

RCy=(Wu+Wd)/2-(2*Cyn+Cyu+Cyd)/4 (5)

When it is judged to be in the horizontal direction, the pixels used in the interpolation process use only the surrounding pixels in the horizontal direction for the pixel Cyn to be interpolated to calculate the deficit component RCy using the following equation.

RCy=(Wl+Wr)/2-(2*Cyn+Cyl+Cyr)/4 (6)

Also, when it is judged that there is no relevant direction, the deficit component RCy is calculated using the following equation using surrounding pixels in both the horizontal and vertical directions for the pixel Cyn to be interpolated.

RCy=(Wu+Wd+Wl+Wr)/4-(4*Cyn+Cyu+Cyd+Cyl+Cyr)/8 (7)

Using the insufficient component RCy obtained from equations (5) to (7), the W component of the interpolated pixel Cyn can be obtained as W'=Cyn+RCy by the following equation.

W' is calculated similarly for all interpolated pixels Cyn by the above operation.

When the Ye pixel is the pixel to be interpolated, the Cy in the formulas (1)-(2) can be replaced by Ye to calculate the correlation, and the right-hand side of the formulas (5)-(7) can be Replace Cy with Ye to find the insufficient weight BYe. The obtained insufficient component is not the R component, but becomes the B component. By setting W'=Yen+BYe, the W component of the Ye pixel can be obtained. The above operations are similarly performed for all interpolated pixels Yen.

By performing this interpolation process, the luminance W' at the Cy pixel and the Ye pixel is obtained, and all the luminance signals can be obtained. With this method, since the signal of the W pixel is used as it is, the Cy pixel and the Ye pixel are interpolated with surrounding pixels with high correlation respectively, so that the decrease in resolution can be reduced.

As described above, in the third embodiment, since interpolation is performed by detecting the vertical and horizontal correlations between the pixel to be interpolated and the surrounding pixels including the pixel to be interpolated, high-precision The luminance signal can prevent the reduction of resolution.

4th embodiment

Next, a fourth embodiment corresponding to claim 6 of the present invention will be described with reference to FIG. 8 .

The structure of the fourth embodiment is basically the same as that of the above-mentioned third embodiment. In the fourth embodiment, further, in the calculation of the correlation degree by the correlation degree calculation circuit 6, the processing of calculating the degree of correlation in the oblique direction is added. , In addition, the interpolation processing during the oblique correlation is added in the interpolation processing circuit 7.

Firstly, the calculation method of the correlation degree will be described below.

FIG. 8 shows the arrangement of surrounding pixels when the cyan pixel Cyn is used as the pixel to be interpolated, and the symbols ● and ○ are Ye pixels and W pixels that are unnecessary in the interpolation process of this pixel Cyn.

In the third embodiment described above, only the degree of correlation in the direction ①-①' and the direction ②-②' shown in FIG. 7 is obtained. Here further let the correlation degree shown in Figure 8 as the ③-③' direction in the right oblique downward direction be Nr, let the correlation degree in the left oblique downward direction as the ④-④' direction be Nl, and use the following formulas to calculate respectively .

Nr=｜(Wu+Wl)/2-(Wd+Wr)/2｜+｜(Cyul-Cyn)｜+｜(Cydr-Cyn)｜ (8)

Nl=｜(Wu+Wr)/2-(Wd+Wl)/2｜+｜(Cydl-Cyn)｜+｜(Cyur-Cyn)｜ (9)

Using this result, and Vc and Hc obtained from the formulas (1) to (2), determine the correlation direction according to the following conditional formula.

Vc+Th<min(Hc,Nr,Nl) (10)

Hc+Th<min(Vc,Nr,Nl) (11)

Nr+Th<min(Hc,Vc,Nl) (12)

Nl+Th<min(Hc,Vc,Nr) (13)

In addition, Th is a threshold value and is a specific constant, and min is a function which takes the minimum value among each element in parentheses. The relevant direction is judged to be vertical when formula (10) is established, judged to be horizontal when formula (11) is true, judged to be obliquely down to the right when formula (12) is true, and judged when formula (13) is true to the left obliquely downward direction. When none of Expressions (10) to (13) holds true, it is determined that there is no relevant direction.

The interpolation processing will be described below.

In the above-mentioned third embodiment, the interpolation processing when the relevant direction is determined to be vertical and horizontal and when it is determined that there is no relevant direction has been described, but here the processing for the case where the relevant direction is determined to be oblique is further added.

When the correlation direction is judged to be the right-down direction, only the surrounding pixels in the right-down direction are used for the interpolated pixel Cyn, and the deficit component RCy is calculated by the following equation.

RCy=(Wu+Wd+Wl+Wr)/4-(2*Cyn+Cyul+Cydr)/4 (14)

When it is judged to be in the downward diagonal direction, only the surrounding pixels in the downward diagonal direction are used for the pixel Cyn to be interpolated, and the deficit component RCy is calculated by the following equation.

RCy=(Wu+Wd+Wl+Wr)/4-(2*Cyn+Cyur+Cydl)/4 (15)

Thereafter, interpolation processing is performed in the same manner as in the third embodiment described above to obtain all luminance signals.

In the fourth embodiment, interpolation is performed by detecting not only the correlation in the vertical and horizontal directions but also in the oblique direction, so that not only the vertical and horizontal but also oblique resolution decreases can be reduced.

fifth embodiment

Next, a fifth embodiment corresponding to claim 7 of the present invention will be described with reference to FIG. 9 .

The configuration of the fifth embodiment is basically the same as that of the third embodiment above. In the fifth embodiment, further, in the calculation of the correlation by the correlation calculation circuit 6, an additional function of calculating the correlation in the L-shaped direction is added. For processing, interpolation processing at the time of correlation in the L-shaped direction is added to the interpolation processing circuit 7 .

Firstly, the calculation method of the correlation degree is described.

FIG. 9 shows the arrangement of surrounding pixels when the cyan pixel Cyn is used as the pixel to be interpolated, and the symbols ● and ○ are Ye pixels and W pixels that are unnecessary in the interpolation process of this pixel Cyn.

In the third embodiment described above, only the degree of correlation in the direction ①-①' and the direction ②-②' shown in FIG. 7 is obtained. Here further let the correlation degree of the upper left L-shaped direction as the ⑤-⑤' direction shown in FIG. The correlation degree of the lower left L word direction of direction is Ldl, makes the correlation degree of the lower right L word direction as 8-8' direction be Ldr, calculates with following formula respectively.

Lul=｜Wu-Wl｜+｜Cyu-Cyn｜+｜Cyl-Cyn｜ (16)

Lur=｜Wu-Wr｜+｜Cyu-Cyn｜+｜Cyr-Cyn｜ (17)

Ldl=｜Wd-Wl｜+｜Cyd-Cyn｜+｜Cyl-Cyn｜ (18)

Ldr=｜Wd-Wr｜+｜Cyd-Cyn｜+｜Cyr-Cyn｜ (19)

Using this result, and Vc and Hc obtained from the formulas (1) to (2), determine the correlation direction according to the following conditional formula.

Vc+Th<min(Hc,Lul Ldl,Lur,Ldr) (20)

Hc+Th<min(Vc,Lul,Ldl,Lur,Ldr) (21)

Lul+Th<min(Hc,Vc Ldl,Lur,Ldr) (22)

Lur+Th<min(Hc,Vc,Lul,Ldl,Ldr) (23)

Ldl+Th<min(Hc,Vc Lul,Lur,Ldr) (24)

Ldr+Th<min(Hc,Vc,Lul,Ldl,Lur) (25)

In addition, Th is a threshold value and is a specific constant, and min is a function which takes the minimum value among each element in parentheses. The relevant direction is judged as vertical when formula (20) is established, judged as horizontal when formula (21) is true, judged as upper left L-shaped direction when formula (22) is true, and judged when formula (23) is true It is an upper-right L-shape direction, and it is judged as a lower-left L-shape direction when Expression (24) holds, and it is judged as a lower-right L-shape direction when Expression (25) holds. When none of Expressions (20) to (25) holds true, it is determined that there is no relevant direction.

The interpolation processing will be described below.

In the above-mentioned third embodiment, the interpolation processing when the relevant direction is judged to be vertical and horizontal and when it is judged to have no relevant direction has been described, but here the processing for the case where the relevant direction is judged to be an L-shaped direction is further added. .

When the correlation direction is judged to be the upper-left L-shape direction, only the surrounding pixels in the upper-left L-shape direction are used for the interpolated pixel Cyn, and the deficit component RCy is calculated by the following equation.

RCy=(Wu+Wl)/2-(2*Cyn+Cyu+Cyl)/4 (26)

When it is judged to be in the upper right L-shape direction, only the surrounding pixels in the upper-right L-shape direction are used for the pixel Cyn to be interpolated, and the deficit component RCy is calculated by the following equation.

RCy=(Wu+Wr)/2-(2*Cyn+Cyu+Cyr)/4 (27)

When it is judged to be in the lower left L-shape direction, only the surrounding pixels in the lower-left L-shape direction are used for the pixel Cyn to be interpolated, and the deficit component RCy is calculated by the following equation.

RCy=(Wd+Wl)/2-(2*Cyn+Cyd+Cyl)/4 (28)

When it is judged to be in the lower right L-shape direction, only the surrounding pixels in the lower right L-shape direction are used for the interpolated pixel Cyn, and the deficit component RCy is calculated by the following equation.

RCy=(Wd+Wr)/2-(2*Cyn+Cyd+Cyr)/4 (29)

Thereafter, interpolation processing is performed in the same manner as in the third embodiment described above to obtain all luminance signals.

In this fifth embodiment, interpolation is performed by detecting the correlation not only in the vertical and horizontal directions but also in the L-shaped direction in this way, so that it is possible to reduce the decrease in resolution not only in the vertical and horizontal directions but also in the L-shaped direction.

sixth embodiment

Next, a sixth embodiment corresponding to claim 8 of the present invention will be described.

The configuration of the sixth embodiment is basically the same as the configuration of the third embodiment above. In the calculation of the correlation degree by the correlation degree calculation circuit 6, the processing of also calculating the correlation degree of the oblique direction and the L-shaped direction is added. In the processing of the processing circuit 7, interpolation processing for oblique correlation and L-shaped correlation is added.

Firstly, the calculation method of the correlation degree is described.

In the third embodiment described above, only the correlations Vc and Hc in the directions ①-①' and ②-②' shown in FIG. 7 are obtained. Let the correlation degree of the right oblique downward direction as the ③-③' direction shown in FIG. 8 be Nr, and the correlation degree of the left oblique downward direction as the ④-④' direction be Nl, and the above-mentioned fourth embodiment Find similarly, let the correlation degree of the upper left L-shaped direction as the ⑤-⑤' direction shown in FIG. The correlation degree in the lower left L-shaped direction in the -⑦' direction is Ldl, and the correlation degree in the lower right L-shaped direction in the ⑧-⑧' direction is Ldr, which is obtained in the same manner as in the fifth embodiment above.

Using this result, the correlation direction is determined according to the following conditional expression.

Vc+Th<min(Hc,Nr,Nl,Lul,Lur,Ldl,Ldr) (30)

Hc+Th<min(Vc,Nr,Nl,Lul,Lur,Ldl,Ldr) (31)

Nr+Th<min(Hc,Vc,Nl,Lul,Lur,Ldl,Ldr) (32)

Nl+Th<min(Hc,Vc,Nr,Lul,Lur,Ldl,Ldr) (33)

Lul+Th<min(Hc,Vc,Nr,Nl,Lur,Ldl,Ldr) (34)

Lur+Th<min(Hc,Vc,Nr,Nl,Lul,Ldl,Ldr) (35)

Ldl+Th<min(Hc,Vc,Nr,Nl,Lul,Lur,Ldr) (36)

Ldr+Th<min(Hc,Vc,Nr,Nl,Lul,Lur,Ldl) (37)

In addition, Th is a threshold value and is a specific constant, and min is a function which takes the minimum value among each element in parentheses. The relevant direction is judged to be vertical when formula (30) is established, judged to be horizontal when formula (31) is established, judged to be oblique to the right when formula (32) is established, and judged when formula (33) is established It is an obliquely downward direction to the left. When formula (34) is established, it is judged as an upper-left L-shape direction. When formula (35) is true, it is judged as an upper-right L-shape direction. When formula (36) is true, it is judged as a lower-left L-shape direction. , when formula (37) holds true, it is judged to be in the lower right L-shape direction. When none of Expressions (30) to (37) holds true, it is determined that there is no relevant direction.

The interpolation processing will be described below.

Although in the above-mentioned third embodiment, the interpolation processing when the relevant direction is judged to be vertical and horizontal and when it is judged to have no relevant direction is described, but here it is further added that the relevant direction is judged to be oblique and the L-shaped direction handling of the occasion.

When the correlation direction is determined to be obliquely downward to the right, only the surrounding pixels in the downward oblique direction to the interpolated pixel Cyn are used to calculate the deficit component RCy using equation (14). When it is judged to be in the downward diagonal direction, only the surrounding pixels in the downward diagonal direction are used for the pixel Cyn to be interpolated, and the deficiency component RCy is calculated using Equation (15). When the correlation direction is judged to be the upper-left L-shape direction, only the surrounding pixels in the upper-left L-shape direction are used for the interpolated pixel Cyn, and the deficit component RCy is calculated using Equation (26). When it is judged to be in the upper right L-shape direction, only the surrounding pixels in the upper-right L-shape direction are used for the pixel Cyn to be interpolated, and the deficit component RCy is calculated using Equation (27). When it is judged to be in the lower left L-shape direction, only the surrounding pixels in the lower-left L-shape direction are used for the interpolated pixel Cyn, and the deficit component RCy is calculated using Equation (28). When it is judged to be in the lower right L-shape direction, only the surrounding pixels in the lower right L-shape direction are used for the interpolated pixel Cyn, and the deficit component RCy is calculated using Equation (29).

Thereafter, interpolation processing is performed in the same manner as in the third embodiment described above to obtain all luminance signals.

In this sixth embodiment, interpolation is performed by detecting not only the correlation between the vertical and horizontal directions but also the oblique and L-shaped directions, so that not only the vertical and horizontal but also the oblique and L-shaped directions can reduce the reduction in resolution. .

Seventh embodiment

Next, a seventh embodiment corresponding to claim 10 of the present invention will be described.

The configuration of the seventh embodiment is the same as that of the above-mentioned third to sixth embodiments, except that the correlation calculation method in the correlation calculation circuit 6 is different from them. Although the vertical correlation degree Vc, the horizontal correlation degree Hc, the correlation degree Nr in the right downward direction, the correlation degree N1 in the left diagonal direction, the correlation degree Lul in the upper left L-shaped direction obtained in the third to sixth embodiments above, In the correlation degree Lur of the upper right L-shaped direction, the correlation degree Ldl of the lower left L-shaped direction, and the correlation degree Ldr of the lower right L-shaped direction, the same color pixels are used to calculate each other, but here, the calculation with the adjacent different color pixels is performed as follows The formula shown is found.

Vc=｜Wu-Cyn｜+｜Wd-Cyn｜ (38)

Hc=｜Wl-Cyn｜+｜Wr-Cyn｜ (39)

Nr=｜(Wu+Wl)/2-Cyn｜+｜(Wd+Wr)/2-Cyn｜ (40)

Nl=｜(Wu+Wr)/2-Cyn｜+｜(Wd+Wl)/2-Cyn｜ (41)

Lul=｜Wu-Cyn｜+｜Wl-Cyn｜ (42)

Ldl=｜Wd-Cyn｜+｜Wl-Cyn｜ (43)

Lur=｜Wu-Cyn｜+｜Wr-Cyn｜ (44)

Ldr=｜Wd-Cyn｜+｜Wr-Cyn｜ (45)

Hereafter, as far as the determination of the relevant direction and the interpolation processing are concerned, they are the same as those in the third to sixth embodiments described above.

In this way, in the seventh embodiment, since the correlation degree can be obtained by calculation with adjacent pixels of different colors, the calculation accuracy of the correlation degree can be improved.

Eighth embodiment

Next, an eighth embodiment corresponding to claim 11 of the present invention will be described.

The configuration of the eighth embodiment is the same as that of the above-mentioned third to sixth embodiments, but the interpolation processing method in the interpolation processing circuit 7 is different from them.

That is to say, in the eighth embodiment, regarding the pixel to be interpolated Cyn obtained in the third to sixth embodiments above, in the interpolation process, after obtaining the luminance W' of the pixel to be interpolated Cyn, Actually, instead of Cyn itself, only the surrounding W pixels are used for interpolation according to the formula.

If the relative direction is vertical, press

W'=(Wu+Wd)/2 (46)

If the relevant direction is horizontal, press

W'=(Wl+Wr)/2 (47)

If the relevant direction is the upper left L-shaped direction, press

W'=(Wu+Wl)/2 (48)

If the relevant direction is the lower left L-shaped direction, press

W'=(Wd+Wl)/2 (49)

If the relevant direction is the upper right L-shaped direction, press

W'=(Wu+Wr)/2 (50)

If the relevant direction is the lower right L-shaped direction, press

W'=(Wd+Wr)/2 (51)

If the relevant direction is other than the above occasions, press

W'=(Wu+Wd+Wl+Wr)/4 (52)

to interpolate.

In this eighth embodiment, since only W is used to calculate the luminance signal, the interpolation accuracy is improved, and a high-resolution image free from uneven luminance can be obtained.

9th embodiment

Next, a ninth embodiment corresponding to claim 13 and claim 14 of the present invention will be described with reference to Fig. 10 and Fig. 11 .

In the ninth embodiment, when it is judged by the correlation degree detection in the above embodiment that the interpolated pixel is related in a specific direction, it is not based on the correlation degree, but is processed so that the position of the interpolated pixel Add a gain of less than 1 to the color difference signal (R-Y, B-Y).

FIG. 10 shows the relationship between the degree of correlation in the direction determined to have the strongest correlation and the gain added to the color-difference signal. Although a false color signal tends to appear at the edge of the luminance, by adding the gain of less than 1 to the interpolation processing circuit 7, the false color generated at the edge of the luminance can be suppressed. Furthermore, since the correlation detection circuit 6 can also be configured to have an edge detection function, false color suppression processing can be performed without additionally adding a luminance edge detection circuit.

In addition, the gain applied to the color-difference signal may be changed according to the magnitude of the degree of correlation in the direction determined to have the strongest correlation. FIG. 11 shows an example of the relationship between the degree of correlation in the direction determined to be the strongest correlation in this case and the gain added to the color-difference signal. But here the stronger the correlation, the smaller the correlation.

In FIG. 11, a certain width Th1 is given for the degree of correlation, and the gain added to the color difference is gradually reduced within each such width. In general, the larger the luminance level difference is at the edge, the denser the false color becomes. That is, the more the correlation is eliminated, the higher the possibility of dark false color is generated, so the level of the color-difference signal can be lowered according to this possibility, and the false color can be effectively suppressed.

10th embodiment

Next, a tenth embodiment corresponding to claim 15 and claim 16 of the present invention will be described with reference to Fig. 12, Fig. 13, Fig. 14 and Fig. 15. Note that the same reference numerals are used for the same configurations as those of the above-mentioned embodiments, and description thereof will be omitted.

FIG. 12 shows the configuration of a solid-state color imaging device according to the tenth embodiment, which is basically the same as the configuration shown in FIG. 6 , except that a frequency characteristic adjustment circuit 10 is added to the configuration of FIG. 6 . The interpolation processing circuit 7 in FIG. 6 is decomposed into a luminance signal interpolation processing circuit 8 and a color-difference signal interpolation processing circuit 9 in FIG. 10 composition. The frequency characteristic of the image signal stored in the storage circuit 5 is adjusted in the frequency characteristic adjustment circuit 10 , and input to the color difference signal interpolation processing circuit 9 .

FIG. 13 is a schematic diagram for explaining the operation of adjusting the frequency characteristics of color signals by limiting the frequency band to low pass in the frequency characteristic adjustment circuit 10. FIG. The frequency characteristic adjustment of adding a low-pass filter can be carried out in the vertical, horizontal, and vertical and horizontal directions. When adding a low-pass filter in the vertical direction, use the vertical 2n+1 points (n=1,2...) of the same color signal centered on the pixel with the frequency characteristic to be adjusted, and calculate the coefficients that determine the characteristics of the filter at each point. Average. For example, the output signal Cy23' of the frequency characteristic adjustment circuit 10 in the case of adding a low-pass filter to the frequency characteristic adjustment pixel Cy23 in FIG. 13 can be expressed as formula (53).

Cy23′=(Cy03+Cy23+Cy43)/3 (53)

Similarly, when adding a low-pass filter in the horizontal direction, use the horizontal 2n+1 points (n=1,2...) of the same color signal centered on the adjusted frequency characteristic pixel to calculate and add the coefficients that determine the characteristics of the filter to each point average of those. For example, with Cy23 in Fig. 13 as the adjusted frequency characteristic pixel, and using horizontal three-point same-color signals, if the coefficients are all set to 1, then the output signal Cy23' of the frequency characteristic adjustment circuit 10 in the case of adding a low-pass filter can be Expressed like formula (54).

Cy23′=(Cy21+Cy23+Cy25)/3 (54)

Similarly, when a low-pass filter is added vertically and horizontally, use the vertical and horizontal (2n+1)×(2m+1) points (n,m=1,2...) of the same color signal centered on the adjusted frequency characteristic pixel, and calculate it in The average of coefficients that determine the characteristics of the filter are added to each point. For example, with Cy23 in Fig. 13 as the adjusted frequency characteristic pixel, using the vertical and horizontal nine points of the same color signal, if the coefficients are all taken as 1, then the signal with the addition of a low-pass filter can be expressed like formula (55).

Cy23'=(Cy01+Cy03+Cy05+Cy21+Cy23+Cy25

  +Cy41+Cy43+Cy45)/9 (55)

Similarly, when adding a low-pass filter in an oblique direction, use 2n+2m+1 points (n, m=1, 2...) of the same color signal in the vertical and horizontal cross directions centered on the pixel with the adjusted frequency characteristic, and calculate and add to each point. The average of the coefficients of the filter characteristics. For example, with Cy23 in Fig. 13 as the adjusted frequency characteristic pixel, using five same-color signals in the vertical and horizontal cross directions, if all the coefficients are set to 1, the signal with the addition of a low-pass filter can be expressed as in formula (56).

Cy23′=(Cy03+Cy21+Cy23+Cy25+Cy43)/5 (56)

This operation of adjusting frequency characteristics is performed for all color signals required for interpolation and synthesis of color difference signals.

For example, when Cy23 is used as the interpolated pixel and the surrounding pixels of W22 and W24 are used to synthesize the color-difference signal, when a low-pass filter is added in the vertical and horizontal directions, as in formulas (57), (58), and (59) , to calculate the color signal required for interpolation.

W22′=(W00+W02+W04+W20+W22+W24+W40+W42+W44)/9 (57)

Cy23'=(Cy01+Cy03+Cy05+Cy21+Cy23+Cy25

+Cy41+Cy43+Cy45)/9 (58)

W24′=(W02+W04+W06+W22+W24+W26+W42+W44+W46)/9 (59)

In the color-difference signal interpolation processing circuit 9 in Fig. 12, the R-Y color-difference signal is output by the following formula using Cy23', W22', and W24'.

R-Y=A×(W22′+W24′)-B×Cy23′ (60)

In the formula, A and B are constants determined by the white balance or the like. Also at the Ye position, the B-Y color-difference signal can be output by performing the above-mentioned frequency characteristic adjustment and color-difference signal interpolation processing in the same positional relationship.

FIG. 14 shows a combination of the amplitude characteristic 11 when three-point averaging is used for frequency characteristic adjustment as a color signal and the amplitude characteristic 12 when linear interpolation is used for interpolation processing as a color difference signal, as shown by a solid line. The horizontal axis is frequency, and the sampling frequency of each color signal is represented by π. If linear interpolation is performed using the color signal adjusted for the frequency characteristic as shown in FIG. 14, frequency components including the foldback distortion included in the vicinity of π/2 of the color signal shown by the dotted line in FIG. 15 can be interpolated to reduce .

In the tenth embodiment, with such a configuration, when the color signal stored in the memory circuit 5 contains high-frequency components, since frequency components including foldback distortion are reduced by the frequency characteristic adjustment circuit 10, the color-difference signal In the interpolation processing circuit 9, the color signal whose frequency characteristic has been adjusted is used to interpolate and synthesize the color difference signal, so that the false color signal can be reduced.

11th embodiment

Next, an embodiment of the invention described in claim 17 and claim 18 corresponding to the present invention will be described with reference to FIG. 16 . Note that the same reference numerals are assigned to the same configurations as those of the above-mentioned embodiments, and description thereof will be omitted.

16 shows the configuration of a solid-state color imaging device according to the eleventh embodiment, which is basically the same as that shown in FIG.

With such a configuration, when the correlation detection circuit 6 determines that there is a correlation direction, the frequency characteristic of the color signal of the pixel to be interpolated is adjusted in the frequency characteristic adjustment circuit 10 . Furthermore, the color difference signal is calculated in the color difference signal interpolation processing circuit 9 using the color signal of the interpolated pixel whose frequency characteristic has been adjusted. The processing in this case is exactly the same as that of the tenth embodiment described above.

Conversely, when it is determined by the correlation detection circuit 6 that there is no correlation direction, the color signal of the pixel to be interpolated is not processed in the frequency characteristic adjustment circuit 10, and is output to the color difference signal interpolation processing circuit 9 as it is. Together with the luminance signal from the luminance signal interpolation processing circuit 8, it is used to complement the color difference signal.

In this way, in the eleventh embodiment, the frequency characteristic is adjusted by the frequency characteristic adjustment circuit 10 for color signals including high-frequency components with a direction of correlation, and the frequency characteristic is reduced similarly to that shown in the tenth embodiment above. Occurrence of false color signals. On the other hand, for a color signal without a relevant direction, since it does not contain false color components, there is no need to adjust the frequency characteristics, because the frequency components of the color signal are not attenuated by the adjustment of the frequency characteristics, so the color reproducibility is maintained .

Industrial Applicability

As described above, according to the solid-state color imaging device of the present invention, four luminance information and two color information are taken out from adjacent vertical and horizontal four pixels of the color separation filter on the surface of the solid-state imaging element, whereby the luminance resolution is high. It is useful as a signal processing method for a solid-state color imaging device that can reduce deterioration in color resolution and perform interpolation processing between pixels to obtain a high-resolution solid-state color imaging device.

Claims (18)

1. A solid-state color imaging device, characterized in that it is equipped with: It has a color-separation filter with four adjacent pixels vertically and horizontally as an arrangement pattern. For the color-separation filter in this arrangement pattern, two pixels are panchromatic transmission filters, and one pixel is a turquoise transmission filter. One pixel is a yellow transmissive filter, which is a solid-state imaging element that repeats the arrangement pattern of the aforementioned four pixels vertically and horizontally, and has a device for individually extracting the information of each pixel of the color separation filter; and Among the image information individually extracted from the solid-state imaging element, four luminance signals and two kinds of color difference signals are extracted for one of the aforementioned array patterns, and at this time, the aforementioned four luminance signals are created based on the information of only the aforementioned panchromatic transmission filter. Two of the degree signals are prepared according to the information of the aforementioned panchromatic transmission filter and the surrounding pixel information of the four vertically and horizontally adjacent pixels. The information is made into the signal processing circuit of the aforementioned two kinds of color-difference signals. 2. The solid-state color imaging device according to claim 1, wherein The above-mentioned color separation filter with four adjacent pixels vertically and horizontally as an arrangement pattern is composed of two pixels in the vertical direction and two pixels in the horizontal direction. Based on the information extracted from the above-mentioned arrangement pattern, four luminance signals and two A total of six signals consisting of one color-difference signal each are output to the element of the 4:2:0 system. 3. The solid-state color imaging device according to claim 1, wherein The above-mentioned color separation filter with four adjacent pixels vertically and horizontally as an arrangement pattern is composed of one pixel in the vertical direction and four pixels in the horizontal direction. Based on the information extracted from the above-mentioned arrangement pattern, four luminance signals and two A total of six signals composed of one color-difference signal each are output to the element of the 4:1:1 system. 4. The solid-state color imaging device according to claim 1, comprising: The above-mentioned color separation filter with four pixels adjacent vertically and horizontally as an arrangement mode, the upper two pixels are panchromatic transmission filter and cyan transmission filter from left, and the lower two pixels are from left to right The yellow transmission filter, the panchromatic transmission filter, and the repeating pattern; A storage device for respectively receiving and storing color signals output from each pixel of the aforementioned solid-state imaging element; Taking the cyan signal pixel and the yellow signal pixel stored in the storage device as interpolated pixels, calculating the correlation of each interpolated pixel with respect to the correlations of a plurality of pixels located around each interpolated pixel degree computing device; and An interpolation processing device for performing pixel interpolation in a direction in which the calculated correlation is large and calculating a panchromatic transmission signal at the position of the interpolated pixel. 5. The solid-state color imaging device according to claim 4, wherein The correlation calculating means calculates the horizontal or vertical correlation between the pixel to be interpolated and pixels located around it, including the pixel to be interpolated. 6. The solid-state color imaging device according to claim 4, wherein The correlation calculating means calculates the horizontal or vertical correlation between the interpolated pixel and the surrounding pixels, including the interpolated pixel, and further, the oblique correlation. 7. The solid-state color imaging device according to claim 4, wherein The above-mentioned correlation calculation means calculates the horizontal or vertical correlation between the above-mentioned interpolated pixel and the pixels located around it, including the interpolated pixel, and then to the right and upward, or to the right and downward, or Affinity to the left and up, or left and down. 8. The solid-state color imaging device according to claim 4, wherein The above-mentioned correlation calculation means calculates the horizontal or vertical correlation including the interpolated pixel, and further the oblique correlation, and further rightward and upward, among the above-mentioned interpolated pixel and pixels located around it, Either right and down, or left and up, or left and down. 9. The solid-state color imaging device according to claim 4, wherein The correlation calculation means calculates the correlation by calculating the same-color signals between the pixel to be interpolated and its surrounding pixels. 10. The solid-state color imaging device according to claim 4, wherein The correlation calculation means calculates the correlation by calculation between adjacent pixels serving as different color signals between pixels located around the pixel to be interpolated. 11. The solid-state color imaging device according to claim 4, wherein The interpolation processing means does not use the color signal of the pixel to be interpolated in the direction in which the degree of correlation is large calculated by the correlation degree calculation means, but uses only signals of the same color as the color signal to be generated around the pixel to be interpolated. Perform interpolation processing. 12. The solid-state color imaging device according to claim 4, wherein The interpolation processing means uses the color signal of the pixel to be interpolated in the direction in which the degree of correlation is high calculated by the correlation degree calculation means, and calculates the insufficient part of the color signal to be generated from the pixels around the pixel to be interpolated. , perform interpolation processing. 13. The solid-state color imaging device according to any one of claims 5 to 10, wherein If the degree of correlation calculated by the degree-of-correlation calculation means is smaller than a predetermined threshold value, the interpolation processing means performs processing of reducing the gain of the color-difference signal corresponding to the pixel. 14. The solid-state color imaging device according to any one of claims 5 to 10, wherein If the degree of correlation calculated by the degree of correlation calculation means is smaller than a predetermined threshold, the interpolation processing means executes a process of stepwise reducing the gain of the color difference signal corresponding to the pixel according to the degree of correlation. 15. The solid-state color imaging device according to claim 4, wherein The interpolation processing device includes frequency characteristic adjustment means for adjusting frequency characteristics of various color signals output from the solid-state imaging device, and interpolates and synthesizes color difference signals using the frequency characteristic-adjusted color signals. 16. The solid-state color imaging device according to claim 15, wherein The above-mentioned interpolation processing device is equipped with a frequency characteristic adjustment device for adjusting the frequency characteristics of various color signals output from the above-mentioned solid-state imaging device, and interpolates at the position of the cyan transmission filter using the color signals subjected to the frequency characteristic adjustment. Synthesize the R-Y color difference signal, and interpolate and synthesize the B-Y color difference signal at the position of the yellow transmission filter. 17. The solid-state color imaging device according to claim 15, wherein The above-mentioned interpolation processing means is equipped with a function of determining a correlation direction based on the correlation degree calculated by the above-mentioned correlation degree calculation means, performing frequency characteristic adjustment when there is a direction with a high correlation degree, and not performing frequency characteristic adjustment when there is no direction with a high correlation degree. . 18. The solid-state color imaging device according to claim 16, wherein The above-mentioned interpolation processing means is equipped with a function of determining a correlation direction based on the correlation degree calculated by the above-mentioned correlation degree calculation means, performing frequency characteristic adjustment when there is a direction with a high correlation degree, and not performing frequency characteristic adjustment when there is no direction with a high correlation degree. .

Images