JP2018093284A - Visible and near infrared simultaneous imaging device - Google Patents

Abstract

Kind Code: A1 When a visible image and a near-infrared image are simultaneously captured using a DBPF that transmits light in a visible light band and a near-infrared light band and a color filter having a white W pixel region, a white W component is obtained. Provided is a simultaneous visible and near-infrared imaging apparatus capable of reducing moiré generated by obtaining signal outputs of infrared and IR components through calculation using signal outputs. A visible/near-infrared simultaneous imaging device includes a double bandpass filter (DBPF2) that transmits visible light and near-infrared light, and red, green, blue, and white pixel regions arranged in a predetermined pattern. In addition, each pixel region includes a color filter having a transmission characteristic for near-infrared light, and a solid-state imaging device 3 . The visible and near-infrared simultaneous imaging apparatus includes an IR signal generator 6 that calculates an output signal of the infrared light component, and a low-pass filter that reduces moiré generated in the output signal of the infrared light component calculated by the IR signal generator 6. 21. [Selection drawing] Fig. 1

Description

  The present invention relates to a visible and near infrared simultaneous imaging apparatus capable of simultaneously capturing and outputting a visible image and an infrared image.

  In general, there is a demand for a camera capable of both photographing with visible light and photographing with infrared light, such as a monitoring camera. As a camera that captures visible and infrared images with a single solid-state imaging device, a double bandpass filter that transmits light in the visible light band and light in the near infrared light band, red R, green G, and blue An imaging system using a color filter having an infrared IR pixel region in a mosaic shape in addition to the B pixel region has been proposed (for example, see Patent Document 1).

  The DBPF has a transmission characteristic in the visible light band, a cutoff characteristic in the first wavelength band adjacent to the long wavelength side of the visible light band, and a second wavelength band that is a part of the first wavelength band. Is an optical filter having transmission characteristics. A wavelength band (a part of the first wavelength band) between the visible light band and the second wavelength band has a blocking characteristic with respect to light. In the camera as the imaging system described above, the infrared cut filter used in a normal camera is not used. Therefore, the infrared light transmits the infrared light band (second wavelength band) of DBPF, and the color filter. Is transmitted through the infrared IR pixel region. At this time, the infrared light passes not only through the IR pixel region of the dichroic color filter but also through the R, G, and B pixel regions. That is, the color filter has a characteristic of transmitting infrared light, and an ordinary camera uses an infrared cut filter to eliminate the influence of infrared light.

  In the above-described imaging system, in a solid-state imaging device having a color filter, in each of the red R, green G, and blue B pixel regions, there is a risk that the signal intensity increases due to the influence of infrared light, resulting in an unnatural image. . Therefore, in the above-described imaging system, the IR signal output is subtracted from the RGB signal outputs as image processing. Here, since the transmitted light of each pixel region includes IR, the transmitted light of the red R pixel region is represented by R + IR, the transmitted light of the green G pixel region is represented by G + IR, and the transmitted light of the blue B pixel region is represented by B + IR. Therefore, when the infrared IR component is removed from the output signal corresponding to the pixel area of each color, (R + IR) −IR = R, (G + IR) −IR = G, and (B + IR) −IR = B. Red R, green G, and blue B signals from which external IR components have been removed can be obtained. Thereby, in the above-described imaging system, an IR image and a visible image with reduced influence of IR can be output simultaneously. The IR pixel region blocks visible light and transmits near-infrared light corresponding to the second wavelength band of DBPF.

Japanese Patent Laid-Open No. 2006-103786

  By the way, in a camera that outputs a visible image and an infrared image simultaneously as described above, even if a color filter in which a white (W) pixel region is added to each RGB pixel region instead of an IR pixel region is used as a color filter, R It is possible to remove the IR component signal output from the G, B signal outputs. Since the IR pixel area does not transmit visible light, when the IR pixel area is provided with one IR pixel for each of R, G, B, for example, the number of pixels used in the visible image is 1 /. Decrease by 4. Normally, the ratio of the IR pixel area is further reduced, but in any case, by providing the color filter with an IR pixel area that blocks light in the visible light band, the resolution of visible light (particularly luminance) is lowered.

Therefore, when the white W pixel region is used, more pixels can be used for the visible image than when the infrared IR pixel region is used. The white W pixel region is basically a pixel region that transmits both visible light and infrared light. When each of the red R, green G, and blue B pixel regions includes IR, the transmission of each pixel is performed. The light can be represented by R + IR, G + IR, and B + IR as described above, whereas the transmitted light of the white W pixel region can be represented by R + G + B + IR (W + IR). Note that the white W pixel region of the color filter is not actually white, but is a pixel region that transmits visible light and infrared light, and is basically a colorless and transparent region. Here, this transparent pixel region is referred to as white W.
FIG. 6 shows the color arrangement of a color filter having such red R, green G, blue B, and white W pixel regions, and each of the red R, green G, blue B, and white W pixel regions has 2 pixels. One by one is arranged in the × 2 pixel array, and this array is repeated. Further, the transmitted light of each pixel region includes IR as described above.

  The graph in FIG. 7 shows the spectral sensitivity of a sensor such as a photodiode used in a solid-state imaging device and the spectral transmittance of each filter. The horizontal axis is the frequency of light, and the vertical axis is the sensitivity or transmittance. It is shown as illuminance, and is normalized so that the maximum scale is 100 and the upper limit of the transmittance is below the sensitivity of the sensor on the graph. The spectral transmittance of DBPF is not normalized, and the total transmission is 100 on the vertical axis. In the graph of FIG. 7, the double line is the spectral sensitivity of the sensor, the thick line is the spectral transmittance of DBPF, the two-dot chain line is the spectral transmittance of the blue B color filter, and the one-dot chain line is green G. The spectral transmittance of the color filter, the broken line is the spectral transmittance of the red R color filter, and the solid line is the spectral transmittance of the NIR filter.

  As shown in FIG. 7, the transmittance of each color filter of red R, green G, and blue B on the infrared side is total transmission. FIG. 7 shows the spectral transmittance of the NIR filter as a color filter in the above-described IR pixel region, not the white W color filter, and the wavelength range of light transmitted by the NIR filter and red R, green Each pixel region of the G and blue B color filters overlaps with a wavelength range in which the total transmission is performed on the infrared side, and these wavelength ranges and an infrared light band (second wavelength band) that transmits DBPF light It is supposed to overlap. In the white W pixel region not shown in FIG. 7, visible light and near-infrared light are basically totally transmitted, and when normalized as described above, the spectral sensitivity of the photodiode (sensor) in FIG. It is almost the same as the heavy line.

In such a camera, in order to obtain an infrared IR output signal using a color filter having white W, the pixel output from the red R pixel region is obtained from the R + IR, green G pixel region as described above. If the pixel output is G + IR, the pixel output from the blue B pixel region is B + IR, and the pixel output from the white W pixel region is R + G + B + IR, that is, R + G + B−W = 0, then IR = ((R pixel output) + (G pixel output) + (B pixel output) − (W pixel output)) / 2 = ((R + IR) + (G + IR) + (B + IR) − (R + G + B + IR)) / 2. More specifically, since the sensitivity (spectral sensitivity) of a sensor such as a photodiode differs depending on the frequency, the coefficient of red R is Kr, the coefficient of green G is Kg, and the coefficient of blue B is a coefficient of each color corresponding to the sensitivity of the sensor. Is Kb and Kr * R + Kg * G + Kb * B−W = 0, IR is represented by the following equation.
IR = ((Kr * (R pixel output) + Kg * (G pixel output) + Kb * (B pixel output) − (W pixel output)) / (Kr + Kg + Kb−1) = (kr * (R + IR) + Kg * (G + IR) + Kb * (B + IR)-(W + IR)) / (Kr + Kg + Kb-1).

In fact, when the color filter including the DBPF and the white W pixel region is used, if the IR signal component is obtained by calculation using the signal output of the white W pixel region, it is found that moire occurs in the IR signal. It was. In this case, when moire occurs in the IR signal, moire occurs in the output of red R, green G, and blue B signals by subtracting the IR signal in which moire has occurred.
In particular, moire occurs at 1/2 and 1/4 of the sampling frequency fs in the spatial frequency graph shown in FIG. In this graph, the vertical axis indicates the spatial frequency in the vertical direction, the horizontal axis indicates the spatial frequency in the horizontal direction, and the sampling frequency indicates the number of pixels per unit length in the horizontal and vertical directions of the solid-state imaging device ( Pixel pitch).

  When such moire is generated by photographing, colors and patterns that are not present in an actual subject are generated, making it difficult to photograph the subject properly. In other words, if a white W pixel area is provided in the color filter in place of the infrared IR pixel area in order to increase the resolution of the visible image, moire occurs in the IR signal output, causing a problem in the infrared image and the visible image. Problems also arise in the image, and the merit of using the white W pixel region is lost.

  The present invention has been made in view of the above circumstances, and a visible image and a near infrared light using a DBPF that transmits light in a visible light band and a near infrared light band and a color filter having a white W pixel region. An object of the present invention is to provide a visible-near-infrared simultaneous imaging apparatus capable of reducing moire generated by obtaining a signal output of an infrared IR component by calculation using a signal output of a white W component when simultaneously capturing images. To do.

In order to solve the above-mentioned problems, the visible and near infrared simultaneous imaging device of the present invention has a transmission characteristic in the visible light band and a cutoff characteristic in the first wavelength band adjacent to the long wavelength side of the visible light band. A double band pass filter that is an optical filter having transmission characteristics in a second wavelength band that is a part of the first wavelength band;
A color filter in which each pixel region of red, green, blue, and white is arranged in a predetermined pattern, and each pixel region has transmission characteristics in the second wavelength band;
A solid-state imaging device including a sensor for each pixel region of the color filter;
A lens that forms an image of a subject on the solid-state image sensor;
Red corresponding to the second wavelength band from the output signal of the solid-state imaging device corresponding to each infrared calculation region set by combining the pixel regions of red, green, blue, and white adjacent to each other on the color filter Infrared calculation means for calculating an output signal of the external light component;
Infrared moiré reduction means for reducing moiré generated in the output signal of the infrared light component calculated by the infrared calculation means is provided.

  According to such a configuration, the output signal of the infrared IR light component is output as a solid-state image corresponding to each of the red R, green G, blue B, and white W pixel regions of the color filter having transmission characteristics in the infrared IR. It is possible to eliminate or weaken the moire generated by calculating from the output signal of each sensor (for example, photodiode) of the element. Thus, in order to eliminate the influence of infrared light that passes through the second wavelength band from the visible image, the infrared IR component from the output signal corresponding to each of the red R, green G, and blue B pixel regions of the color filter. By subtracting the output signal in which the moiré is generated, the moiré is also prevented from being generated in the red R, green G, and blue B output signals by using the infrared IR output signal in which the moiré is reduced. can do.

  The said structure of this invention WHEREIN: It is preferable that the said infrared moire reduction means has a low-pass filter between the said solid-state image sensor and the said lens.

According to such a configuration, moire can be reduced by the low-pass filter. The low-pass filter here uses, for example, a crystal having a birefringence, blocks light having a high spatial frequency as a blurred state, and transmits light having a low spatial frequency. Here, it is preferable to blur light having a spatial frequency that is ½ of the sampling frequency or a spatial frequency in the vicinity of ¼ of the sampling frequency. Note that the sampling frequency is based on the pixel pitch in the solid-state imaging device, and corresponds to the number of pixels per unit length. Therefore, in the solid-state imaging device of square pixels, the pixel pitch is the same both in the horizontal direction and in the vertical direction, so in the graph of FIG. 8, the scale interval is the same on the X axis and the Y axis.
Here, when the solid-state imaging device is a square pixel, it is preferable that the optical axis direction of the crystal serving as the low-pass filter is an angle shifted by 45 degrees with respect to the above-described horizontal and vertical directions of the solid-state imaging device. .

In the configuration of the present invention, the infrared moire reduction means includes
Edge detection means for detecting an edge from an output signal from the solid-state imaging device;
When the edge detected by the edge detection unit overlaps the infrared calculation region, the pixel region included in the infrared calculation region is changed so that the infrared calculation region overlaps the edge. It is preferable to change the position of the infrared calculation region.

  According to such a configuration, for example, when an edge overlaps an infrared calculation region composed of pixel regions of red R, green G, blue B, and white W of the color filter, a line that becomes an edge of the pixel region of each color The output of the pixel area is relatively large (brighter) on the side where the output of the pixel becomes higher, and the output of the pixel area becomes relatively smaller (darker) on the side where the output of the edge becomes smaller. The brightness changes and the output of the infrared IR component cannot be accurately calculated using these outputs that are not correlated with each other, and an erroneous signal is output, which causes moire.

  Here, each infrared calculation area is basically arranged in a matrix in the vertical and horizontal directions, and in the infrared calculation area where the edges detected by edge detection overlap, the side where the light intensity is high and the side where the light is low The infrared calculation region is set as an edge by re-setting the infrared calculation region by combining the pixel regions of red R, green G, blue B, and white W on the low light intensity side so as not to exceed the boundary of Avoid overlapping. Similarly, the infrared calculation area is overlapped with the edge by resetting the infrared calculation area by combining the red R, green G, blue B, and white W pixel areas on the side of the boundary where the light intensity is high. Do not become. Moire can be reduced by changing the arrangement of these infrared calculation regions.

  ADVANTAGE OF THE INVENTION According to this invention, generation | occurrence | production of a moire can be reduced in the visible near-infrared simultaneous imaging device which has DBPF and the color filter provided with the pixel area | region of red R, green G, blue B, and white W.

It is a block diagram which shows the visible near infrared simultaneous imaging device of embodiment of this invention. It is a figure for demonstrating the synchronization process of a perpendicular direction similarly. It is a figure for demonstrating the synchronization process of a horizontal direction similarly. It is a graph for demonstrating the optical axis direction of a low-pass filter similarly. FIG. 4 is a diagram illustrating a color filter, and (a), (b), and (c) are diagrams illustrating a relationship between an edge and an infrared calculation region. It is a figure for demonstrating the color scheme of the color filter which has a white W pixel area. It is a graph which shows the spectral sensitivity of the sensor in the visible near infrared simultaneous imaging device, and the spectral transmittance of each filter. It is a graph for demonstrating the spatial frequency which a moire tends to generate | occur | produce.

Embodiments of the present invention will be described below.
As shown in FIG. 1, the visible-near-infrared simultaneous imaging device of this Embodiment is provided with the lens 1, DBPF2, the solid-state image sensor 3 provided with a sensor and a color filter. Further, this visible-near-infrared simultaneous imaging apparatus outputs an output signal for each pixel that is output from the solid-state imaging device corresponding to each of the red R, green G, blue B, and white W pixel regions of the color filter shown in FIG. And a luminance signal generation unit to which outputs of red R, green G, blue B, and white W determined for each pixel by the synchronization processing unit 4 are respectively input. 5. An IR signal generation unit 6 and an RGB signal generation matrix unit 7 are provided.

  In the visible-near-infrared simultaneous imaging device, part of the luminance signal generated by the luminance signal generation unit 5 is output as an IR uncorrected luminance signal without performing a process of removing the infrared IR component. The remaining part of the luminance signal is subtracted from the IR signal generated by the IR signal generation unit 6 in the IR subtraction processing unit 8, and is processed by the gamma processing unit 9 as a luminance signal in which infrared IR is corrected, and luminance Y The signal is transmitted to the outside of the visible / near infrared simultaneous imaging apparatus. The IR signal generated by the IR signal generation unit 6 is output to the outside as a signal for an infrared image. The IR signal generated by the IR signal generation unit 6 is removed by the IR subtraction processing unit 8 from the red R, green G, and blue B signals output from the RGB signal generation matrix unit 7. The RGB signal from which the IR signal has been removed is subjected to white balance adjustment by the W / B processing unit 10, gamma correction by the gamma processing unit 11, and the RGB signal is converted into a color difference signal by the color difference signal generation unit 12. ing.

  The lens 1 forms an image of a subject, and usually includes a plurality of lenses. The DBPF 2 has a transmission characteristic in the visible light band, a cut-off characteristic in the first wavelength band adjacent to the long wavelength side of the visible light band, and a second wavelength band that is a part of the first wavelength band. The upper and lower limits of the wavelength of each band can be adjusted during manufacturing. As shown in FIG. 7, the spectral sensitivity of the sensor and the spectral transmittance of each pixel region of the color filter are adjusted. Correspondingly, the upper and lower wavelengths of each band are determined.

  The solid-state imaging device 3 has a sensor such as a photodiode for each pixel, and each pixel region of the color filter is arranged. Therefore, the output from the solid-state imaging device 3 is an output signal of a color corresponding to each pixel region, and an output signal of any one of red R, green G, blue B, and white W is output from one pixel. The The synchronization processing unit 4 performs synchronization so that signals of four colors of red R, green G, blue B, and white W are simultaneously output for each pixel. In the present embodiment, the synchronization processing is performed by a method using a line memory, but other methods may be used.

  However, in order to prevent moire, it is preferable not to perform an operation using the white W output signal. For example, if the red R and green G output signals are subtracted from the white W signal, the blue B signal is obtained, and if the green G and blue B output signals are subtracted from the white W signal, the red R signal is obtained. By subtracting the red R and blue B output signals from the white W signal, a green G signal is obtained. In this embodiment, the synchronization processing is performed using the line memory without performing such calculation using white W.

  As shown in FIG. 2, the synchronization processing unit 4 uses the two line memories 41 and 42, the addition processing unit 43, and the line changeover switch 44 to output signals output from the solid-state imaging device 3 in the vertical direction. Process. In the pixel array shown in FIG. 6, the output from the solid-state imaging device 3 is, for example, one line of W · R · W · R · W · R · W · R... Is output repeatedly for one line of B, G, B, G, B, G, B, G... For the next row of pixels. Signal.

  On the other hand, in the first line memory 41, since the above-mentioned one line is stored and output, the output is delayed by one line from the through output signal. Furthermore, in the second line memory 42, since the output of one line of the first line memory 41 is stored and then output, it is delayed by two lines with respect to the through. When this is considered based on the output of the first line memory 41, the through output is earlier by one line than the output of the first line memory 41, and the output of the second line memory 42 is delayed by one line.

  Further, the output for one line is only the output of white W and red R or only the output of green G and blue B. In this case, when the output of the first line memory 41 is used as a reference, when the output of the first line memory 41 is green G and blue B, the through and the output of the second line memory 42 are white W and red R. When the output of the first line memory 41 is white W and red R, the output of the through and second line memory 42 is green G and blue B. Therefore, by combining the output of the first line memory 41 and the output of the through and second line memories 42, the output of white W and red R and the output of green G and blue B are performed for the output of one line. I am doing so.

As shown in FIG. 2, 1/2 of the through output and 1/2 of the output of the second line memory 42 are added together by the addition processing unit 43 to output white W and red R or green G and blue B. Is generating the output. That is, in a matrix-like pixel group along both the horizontal direction and the vertical direction, an output signal of a horizontal column of pixels serving as a reference and an output of a horizontal column of pixels above and a horizontal pixel below the reference are output. The outputs that are averaged with the outputs of pixels in one column are combined.
In addition, for each output for one line, the first line memory 41, the second line memory 42, the outputs of the white W and red R and the outputs of the green G and blue B are interchanged, and the first line memory The output of the through and second line memory 42 is always opposite to the output of white W and red R and the output of green G and blue B with respect to the output of 41.
Therefore, the output of the first line memory 41 is switched between the outputs of white W and red R and the outputs of green G and blue B, and the output of the addition processing unit 43 is the output of green G and blue B and white W and red. Switch with R output. Therefore, the line changeover switch 44 switches between the output of the first line memory 41 and the output of the addition processing unit 43, and white W and red R signals are always output from one terminal during photographing, The output terminal always outputs green G and blue B signals during shooting. Thus, the vertical synchronization process has been performed.

  Next, the horizontal synchronization processing of the synchronization processing unit 4 will be described with reference to FIG. In the horizontal synchronization process, the horizontal synchronization process is performed using the first register 45, the second register 46, the pixel addition processing unit 47, and the pixel selector switch 48. The configuration shown in FIG. 3 requires two terminals, one for outputting white W and red R and the other for outputting green G and blue B shown in FIG. Here, the horizontal synchronization process of the outputs of white W and red R will be described. First, the signal to be through is a signal in which white W and red R are repeated, and the output of the first register 45 is designed to output after storing the output of one pixel of the through. Is delayed by one pixel. The output of the second register 46 is output after storing the output of the first register 45 for one pixel, and is delayed by one pixel with respect to the output of the first register 45. .

  Therefore, when the output of the first register 45 is used as a reference, the through output is accelerated by one pixel, and the output of the second register 46 is delayed by one pixel. Here, when the outputs of white W and red R are switched for each pixel, when the output of the first register 45 is white W, the output of the through and second registers 46 becomes red R, and the first register 45 And the output of the second register 46 is white W. Therefore, by combining the output of the first register 45 and the output of the through and second registers 46, it is possible to obtain both white W and red R outputs for one pixel. Here, ½ of the through signal output and ½ of the signal output of the second register 46 are added and output by the pixel addition processing unit 47. This output is the average of the output of the pixel immediately before and the pixel immediately after the pixel that outputs the signal of the first register 45.

  The output of the first register 45 is switched between white W and red R for each pixel, and the output of the pixel addition processing unit 47 is switched between red R and white W for each pixel. 48, the output of the first register 45 and the output of the pixel addition processing unit 47 are switched for each pixel, white W is always output from the white W terminal during photographing, and red is output from the red R output terminal. R is always output during shooting. Green G and blue B are similarly processed. As a result, four signals of white W, red R, green G, and blue B are output per pixel from the synchronization processing unit 4. Here, the calculation of the output signals of the plurality of colors including the white W is not performed, and only the process of taking the average of the signals of the same color at different timings as described above, and the white W and other colors as described above are performed. Since no pixel color interpolation processing is performed by calculation, the occurrence of moire can be suppressed.

  In the visible and near infrared simultaneous imaging apparatus of the present embodiment, as described above, an uncorrected luminance signal that is not subtracted from the output of the infrared IR component is output. For example, image recognition is performed. In this case, an uncorrected luminance signal without moire can be used.

As described above, the IR signal generation unit 6 calculates the output signal of the infrared IR component using the output signals of red R, green G, blue B, and white W as the infrared calculation means. There is a possibility that moire occurs in the external IR signal. In the present embodiment, in order to prevent moiré, a low-pass filter 21 as an infrared moiré reducing means is disposed between the lens 1 and the solid-state imaging device 3. The low-pass filter 21 utilizes, for example, the birefringence of quartz (SiO ₂ ), blocks high frequency components with high spatial frequency, and transmits low frequency components with low spatial frequency. In the present embodiment, the low-pass filter 21 set corresponding to the sampling frequency (fs) corresponding to the number of pixels per unit length (or pixel pitch) in the solid-state imaging device 3 is used. The frequency components including 1/2 and 1/4 of the frequency are blurred and cut off.

  The graph shown in FIG. 4 shows horizontal (H) and vertical (V) spatial frequencies, and positions where the sampling frequency is 1/2 · 1/4 are shown. Here, when the pixels in the solid-state imaging device 3 are square pixels, the graduations in the horizontal direction and the vertical direction are equally spaced, and the positions of the sampling frequencies of 1/2 and 1/4 are the horizontal axis and the vertical direction. The line segment connecting 1/2 and 1/4 of these sump frequencies is 45 degrees as shown in FIG. Here, the horizontal direction (H) of the horizontal axis is 0 degree, and the vertical direction (V) is 90 degrees. A line segment connecting 1/2 and 1/4 of the sump frequency is inclined upward at an angle of 45 degrees from right to left.

  On the other hand, the low-pass filter 21 is set so that the optical axis of the low-pass filter 21 is at an angle of 45 degrees with respect to the horizontal and vertical directions on the imaging surface of the solid-state imaging device 3. Such a low-pass filter 21 can suppress moire generated in the output of the infrared IR component as described above. Thereby, when the infrared IR component is subtracted from red R, green G, blue B and luminance Y, it is possible to suppress the occurrence of moire in red R, green G, blue B and luminance Y.

  Further, when the IR signal is generated in the IR signal generation unit 6 of the present embodiment, for example, red R, green G, blue B and white included in 2 × 2 = 4 pixels in the color arrangement in FIG. The infrared IR component is calculated from the output signal corresponding to W as described above. Therefore, the solid-state imaging device 3 is divided into four pixels of infrared calculation areas, and IR signals are calculated from the four output signals of each infrared calculation area. Here, when there is an edge detected by edge detection in the captured image, and this edge crosses the infrared calculation region, it causes moire in the calculated IR signal. The edge is a portion where the brightness changes discontinuously. For example, one side of the edge is bright and the other side is dark. As shown in FIG. 5A, when the line segment 51 connecting the portions where the brightness of the edge changes discontinuously exists in the horizontal direction, when the infrared calculation region overlaps the line segment 51, white A line segment 51 is arranged between W and red R and green G and blue B. In this case, for example, white W and red R are bright, green G and blue B are dark, there is no correlation between them, and infrared IR components are output from the outputs of red R, green G, blue B, and white W. Is calculated, it becomes an error signal different from the actual infrared IR component, and this causes moire.

  Therefore, the IR signal generation unit 6 serving as an infrared moire reduction unit is provided with an edge detection unit, and detects a portion (line segment 51) where the brightness between pixels serving as edges is discontinuously different. For edge detection, a known edge detection method can be used. In a situation where the edge line segment 51 is horizontally disposed between the upper and lower pixels, the edge line segment 51 is located at the upper and lower centers of the infrared calculation region composed of two pixels each of the upper, lower, left and right pixels. In this case, the red R, green G, blue B, and white W pixel areas constituting the infrared calculation area are changed so that the infrared calculation area does not overlap with the line segment 51 that is an edge, and the upper side of the line segment 51 Or, the changed infrared calculation area 52 is set so as to be arranged on the lower side. In this way, the infrared IR component is calculated by combining the output of the pixel region with the higher light intensity on one side of the edge and the output of the pixel region with the lower light intensity on the other side of the boundary. Can be prevented.

  Also, as shown in FIG. 5B, in the situation where the above-described line segment 51 of the edge is arranged vertically so as to separate the pixels left and right, the line segment 51 is perpendicular to the center of the left and right of the infrared calculation region. In the case of overlapping, the red R, green G, blue B, and white W in the infrared calculation area are changed, and the position is set so that the changed infrared calculation area 52 is arranged on the right or left side of the line segment 51. change. Further, as shown in FIG. 5C, when the above-described line segment 51 of the edge is arranged obliquely, when the line segment 51 overlaps obliquely with the infrared calculation area, the red color of the infrared calculation area R, green G, blue B, and white W are changed, and the position is changed so that the infrared calculation area 52 after the change does not overlap the line segment 51.

  Thereby, the brightness changes discontinuously and no IR signal is generated by a combination of white W, red R, green G, and blue B having no correlation, and moire can be suppressed. In this embodiment, the IR signal is generated from the pixel output corresponding to the pixel area of each color before the synchronization process, and the IR signal is generated from the pixel output of the four colors before the synchronization process. Is calculated.

  In the present embodiment, when the white W, red R, green G, and blue B color filters and the DBPF2 are used to simultaneously output the visible image layer and the infrared image, the above-described white W, red R, and green are output. By calculating the IR signal from the G and blue B outputs, the moiré is generated in the IR signal, and the pixels used when generating the IR signal by adapting to the edge as described above are changed. Can be prevented. This also prevents moiré from occurring in each color when the infrared component contained in the output signal of each color is subtracted from the infrared signal calculated from the output signal of each color. In this embodiment, both the low-pass filter installation and the change corresponding to the edge of the pixel used for infrared signal calculation are performed, but only one of them may be performed.

1 Lens 2 DBPF (Double Band Pass Filter)
3 Solid-state image sensor 6 IR signal generator (infrared calculation means, infrared moire reduction means)
21 Low-pass filter (infrared moire reduction means)

Claims (3)

It has a transmission characteristic in the visible light band, a cutoff characteristic in the first wavelength band adjacent to the long wavelength side of the visible light band, and a transmission characteristic in the second wavelength band that is a part of the first wavelength band A double band pass filter that is an optical filter having
A color filter in which each pixel region of red, green, blue, and white is arranged in a predetermined pattern, and each pixel region has transmission characteristics in the second wavelength band;
A solid-state imaging device including a sensor for each pixel region of the color filter;
A lens that forms an image of a subject on the solid-state image sensor;
Red corresponding to the second wavelength band from the output signal of the solid-state imaging device corresponding to each infrared calculation region set by combining the pixel regions of red, green, blue, and white adjacent to each other on the color filter Infrared calculation means for calculating an output signal of the external light component;
An infrared and near-infrared simultaneous imaging apparatus comprising: an infrared moire reducing unit that reduces moire generated in an output signal of an infrared light component calculated by the infrared calculating unit.   The visible-infrared simultaneous imaging apparatus according to claim 1, wherein the infrared moire reduction unit includes a low-pass filter between the solid-state imaging device and the lens. The infrared moire reducing means is
Edge detection means for detecting an edge from an output signal from the solid-state imaging device;
When the edge detected by the edge detection unit overlaps the infrared calculation region, the pixel region included in the infrared calculation region is changed so that the infrared calculation region overlaps the edge. The visible and near infrared simultaneous imaging apparatus according to claim 1, wherein the position of the infrared calculation region is changed.

Images