JP2001211388A - Image signal processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image signal processing apparatus which properly corrects defective pixels in exposure for a long time and reduces an effect of detective pixels to generate an image signal of less false signal occurrence. SOLUTION: Exposure time information in photographing is acquired by an exposure time determination part which measures the exposure time of an imaging device in a step 100, and it is decided in accordance with exposure time information whether or not defective pixel correction should be performed in a step 102. When defective pixel correction is performed, image signal processing is performed after photographic data are subjected to defective pixel correction by a defective pixel correction part in step 104 and 106. When defective pixel correction is not performed, the image signal processing is performed without defective pixel correction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing apparatus, and more particularly to an image signal processing apparatus for correcting a defective pixel of a solid-state image sensor such as a CCD sensor.

[0002]

2. Description of the Related Art In a solid-state imaging device such as a CCD sensor, a pixel defect may occur due to a physical defect of a semiconductor or the like. This pixel defect causes deterioration of image quality in image signal processing. Therefore, in the conventional image signal processing device, defective pixel correction is performed to eliminate the image quality deterioration.

For example, in the technique described in Japanese Patent Publication No. Hei 7-97838, data of a pixel that becomes a defective pixel is stored in advance in a storage means such as a ROM, and upon photographing, based on an adjacent pixel of the position data. A method of performing correction by performing an interpolation process has been proposed.

In the technique described in Japanese Patent Application Laid-Open No. 4-235472, a method of correcting a defective pixel by performing a median filter process or the like on image data after A / D conversion has been proposed.

[0005]

However, in long-time exposure (exposure) photographing for photographing night scenes and the like, a temperature difference of a solid-state image pickup device such as a CCD sensor and the like may cause a difference from the initial shipment. A defective pixel may newly occur at the position, or the level of the conventional defective pixel may increase.

Further, in the technique described in Japanese Patent Publication No. 7-97838, the defective pixel position of the solid-state image pickup device is recorded in the recording means at the time of shipment selection.
Defective pixel correction is performed based on the data. Therefore, there is a problem that a correction process cannot be performed on a defective pixel newly generated after shipment.

In the technique described in Japanese Patent Laid-Open No. 4-235472, a median filter process is always performed on image data representing a photographed image. Also, there is a problem that the filtering process is performed and the image quality of the captured image is impaired as a result.

The present invention has been made in order to solve the above-mentioned problem, and it is possible to appropriately perform defective pixel correction during long-time exposure, reduce the influence of defective pixels, and reduce the occurrence of false signals. It is an object of the present invention to provide an image signal processing device capable of generating a signal.

[0009]

According to one aspect of the present invention, there is provided an image pickup device that photoelectrically converts light that changes according to a subject and outputs an analog image signal. An analog-to-digital converter for converting the analog image signal output from the image sensor into a digital image signal; and interpolating defective pixels in the image sensor based on the digital image signal converted by the analog-to-digital converter. Interpolating means, measuring means for measuring at least the exposure time or temperature in the image sensor, and control means for controlling interpolation of the defective pixel by the interpolating means according to the measurement result by the measuring means. It is characterized by.

According to the first aspect of the present invention, the light that changes according to the subject is photoelectrically converted by an image pickup device (for example, a CCD sensor) to output an image signal. Since the image signal is an analog image signal, it is converted into a digital image signal by analog-digital conversion means. Here, some of the image sensors have defective pixels from the beginning of shipment. Therefore, interpolation of the defective pixels is performed by the interpolation means, and an image signal having no defective pixels is obtained. However, a defective pixel of the image sensor may not appear depending on the exposure time of the image sensor. In addition, the number of defective pixels tends to increase when the exposure time of the image sensor is increased or when the image sensor generates heat. Therefore, the invention according to claim 1 has a configuration in which the exposure time or the temperature in the image sensor is measured by the measurement means, and the interpolation by the interpolation means is controlled by the control means according to the measured exposure time. .

That is, it is possible to appropriately perform defective pixel correction during long-time exposure, reduce the influence of defective pixels, and generate an image signal with less occurrence of false signals. Is characterized in that, in the invention described in claim 1, the control means controls whether or not interpolation is performed by the interpolation means.

Depending on the exposure time of the image pickup device, it may not be necessary to interpolate defective pixels by the interpolation means. Alternatively, depending on the degree of defective pixels, processing without interpolation can reduce the processing time of the image signal.
Therefore, according to the second aspect of the present invention, in the first aspect of the present invention, the exposure time is increased by controlling whether or not to perform the interpolation of the defective pixel by the interpolation means performed by the control means. In such a case, the interpolation of the defective pixel by the interpolation means can be performed, and when the exposure time is short, the interpolation of the defective pixel by the interpolation means can be prevented. That is, when the exposure time is short, the processing time for performing the interpolation of the defective pixel performed by the interpolation unit can be reduced.

According to a third aspect of the present invention, in the first or second aspect of the present invention, the interpolation means performs interpolation using a median filter.

According to the invention described in claim 3, according to claim 1 of the present invention,
Alternatively, in the invention described in claim 2, the interpolation means can perform interpolation using a median (median) filter.

According to a fourth aspect of the present invention, in the third aspect of the present invention, the control means switches the shape of the median filter in accordance with the exposure time of the image sensor.

According to the invention set forth in claim 4, according to claim 3,
According to the invention described in (1), the control means switches the shape of the median filter according to the exposure time of the image sensor, thereby making it possible to perform appropriate correction according to the state of the image sensor (such as an increase in defective pixels). Become.

According to a fifth aspect of the present invention, in the third aspect of the present invention, the control means determines a correlation of each pixel of the image sensor, and according to a result of the determination, determines a shape of the median filter. Is switched.

According to the invention described in claim 5, according to claim 3,
In the invention described in (1), the control means may determine the correlation of each pixel of the image sensor, and switch the shape of the median filter according to the determination result.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, the present invention is applied to a digital camera.

FIG. 1 shows a configuration diagram of a digital camera 10 according to an embodiment of the present invention.

As shown in FIGS. 1A and 1B, the main body 12 of the digital camera 10 is substantially box-shaped, and a projection (gripping portion) for facilitating the grip of the main body 12 is provided on the left side when viewed from the front. )
Is formed. A lens 14 is attached to the center of the front side of the main body 12. Above the lens 14 of the main body 12, an optical viewfinder 16 for a user to visually check a photographing range and the like is provided. A strobe 18 for emitting auxiliary light is attached.

A power switch 20 is provided on the upper surface of the main body 12 on the right side when viewed from the front, and a shutter button 22 is provided on the left side (a position corresponding to the grip portion). Is provided with a slot 24 in which a memory card (not shown) can be inserted.

Further, below the strobe 18, there is provided an AC adapter connection port 28 for connecting an AC adapter for obtaining power from an AC power supply, so that the AC power can be supplied to the digital camera 10. ing.

Further, as shown in FIG.
A color display 26 made of a transmissive liquid crystal (may be a semi-transmissive liquid crystal) is provided on the back surface of the device, and the display 26 is configured to include a backlight including a fluorescent tube, an LED, and the like. . The display 26 has the function of the finder 16.

On the rear side of the main body 12, on the left side as viewed, a menu (MENU) switch 30, an execution / screen switching (E)
X / VIEW CHG) switch 32 and selection (S
An ELECT) switch 34 is provided, and a cancel (CANCEL) is provided above the display 26.
A switch 36 is provided. Note that the selection switch 3
Reference numeral 4 denotes a switch for executing a mode such as a menu screen displayed by pressing the menu switch 30, and a cancel switch 36 is a switch for canceling various modes and the like.

FIG. 2 shows the configuration of the electric system of the digital camera 10. More specifically, the lens 14 is a zoom lens (variable focal length lens) having an autofocus focus (AF) mechanism. The AF mechanism and the zoom mechanism of the lens 14 are driven by a drive circuit 38. In addition,
A variable focal length lens having only an AF mechanism may be used as the lens 14 instead of the zoom lens.

At a position corresponding to the focal position of the lens 14 inside the main body 12, an image pickup device 40 composed of an area CCD sensor or the like is arranged. An image is formed on the light receiving surface of the imaging device 40. The imaging device 40 outputs, as an image signal, an analog signal representing the amount of received light in each of the plurality of photoelectric conversion cells arranged in a matrix on the light receiving surface. The imaging device 40 is driven at a timing synchronized with a timing signal generated by a timing generation circuit (not shown) incorporated in the driving circuit 38 and outputs an image signal.

A shutter / aperture 42 is arranged between the lens 14 and the imaging device 40. The shutter and the aperture 42 are driven by the drive circuit 38. The shutter is for preventing the occurrence of smear due to light entering the light receiving surface of the imaging device 40 when an image signal is output from the imaging device 40, and is omitted depending on the configuration of the imaging device 40. It is possible. Further, the aperture may be configured by a single aperture capable of continuously changing the aperture amount, or may be configured to switch a plurality of apertures having different aperture amounts. The strobe 18 is further connected to the drive circuit 38, and the strobe 18 is controlled to emit light when it is detected that the illuminance is low or when the user instructs to emit light.

An analog signal processing unit 44, an A / D converter 46, a defective pixel correction unit 48, a digital signal processing unit 50, and a memory 52 are sequentially connected to a signal output terminal of the imaging device 40. The analog signal processing unit 44 amplifies the image signal output from the imaging device 40 and performs correction such as white balance on the amplified image signal. The image signal output from the analog signal processing unit 44 is converted into digital image data by the A / D converter 46 and input to the defective pixel correction unit 48. The defective pixel correction unit 48 corrects a defect occurring in a pixel of the imaging device 40 and outputs the corrected image data to the digital signal processing unit 5.
Input to 0. The correction in the defective pixel correction unit 48 can be performed by, for example, performing interpolation using a median (median) filter.
The digital signal processing unit 50 performs various processes such as color correction, γ correction, and Y / C conversion on the input image data. The image data output from the digital signal processing unit 50 is temporarily stored in a memory 52 including a RAM or the like.

The driving circuit 38, the analog signal processing section 44,
A / D converter 46, defective pixel correction unit 48, digital signal processing unit 50, memory 552, and compression / decompression unit 54 (described later)
Is connected to a bus 56, which has a CP
U58 is connected, and switches 60 such as a power switch 20, a menu switch 30, and an execution / screen changeover switch 32, and a shutter switch 62 that is turned on and off by operating a shutter button 22 are connected. I have. Although not shown, the CPU 58 includes peripheral circuits such as a ROM, a RAM, and an input / output port.

The display 26 and the compression / expansion unit 54 are connected to the memory 52, respectively. When displaying an image on the display 26, the CPU 58
After a predetermined process is performed on the image data selected by operating the selection switch 34 from the plurality of image data temporarily stored in the display device 26, the image data is transferred to the display 26. Thereby, an image represented by the image data temporarily stored in the memory 52 is displayed on the display 26.

When the shutter button 22 is operated and the shutter switch 62 is turned on, the slot 2
When the instruction to store the image data in the memory card inserted in the memory card 4 is given, the CPU 58 reads out the image data temporarily stored in the memory 52 and transfers it to the compression / decompression unit 54. Thus, the image data is stored in the memory card after being compressed by the compression / expansion unit 54. The image data may be stored in the memory card without being compressed depending on the shooting mode or the like.

When the reproduction (display) of the image represented by the image data stored in the memory card inserted into the slot 24 is instructed, the image data is read from the memory card and read. After the image data is expanded (decompressed) by the compression / expansion unit 54, it is temporarily stored in the memory 52. Then, display (reproduction) of the image on the display 26 is performed using the image data temporarily stored in the memory 52.

By the way, some imaging devices 40 such as CCD sensors have a defect of a pixel (photoelectric conversion cell) due to a physical defect of a semiconductor or the like from the time of shipment. Also,
When the exposure time of the imaging device 40 becomes longer, heat is generated by the photoelectric conversion of the imaging device 40, and the generation of the heat tends to increase the number of defective pixels of the imaging device 40 (become noticeable).

Therefore, the digital camera 1 of the present embodiment
At 0, an exposure time determination unit 64 is connected to the bus 56 to switch whether or not to perform correction of a defective pixel according to the exposure time of the imaging device 40. The exposure time of the defective pixel correction unit 4 is measured in accordance with the measured exposure time.
8. Whether or not to perform the correction in 8 is switched. Note that the temperature near each photoelectric conversion cell of the imaging device 40 may be measured instead of the exposure time, and whether to perform the correction may be switched according to the measured temperature.

Next, detection of a defective pixel of the imaging device 40 will be described. As described above, the digital camera 10 according to the present embodiment is provided with the shutter / aperture 42 between the lens 14 and the imaging device 40. Detection of a defective pixel is performed by performing dark correction with the shutter closed. When the shutter is closed, an image signal is extracted from each photoelectric conversion cell of the imaging device 40, and a defective pixel is extracted from the image signal (in the case of a defective pixel, the output of the photoelectric conversion cell is different from other pixels. The pixel is extracted), and the position data of the defective pixel is stored in a storage medium such as the RAM of the CPU 58. Then, the position data is read out by the defective pixel correction section 48 to correct the defective pixel.

Next, the operation of the digital camera 10 configured as described above will be described with reference to the flowchart of FIG.

First, in step 100, exposure time information at the time of photographing is obtained by the exposure time determining unit 64, and the process proceeds to step 102.

In step 102, it is determined whether or not to perform defective pixel correction. The determination is made in step 10
It is performed by determining whether the exposure time acquired at 0 is longer than a predetermined time. If the determination is affirmative, the process proceeds to step 104.

In step 104, the defective pixel correcting section 48
Accordingly, defective pixels are corrected for the photographing data (image data acquired by the imaging device 40).

Subsequently, in step 106, the digital signal processing section 50 performs various processes such as the above-described color correction, γ correction, and Y / C conversion, and a series of image signal processes ends.

If the determination in step 102 is negative, step 10 is performed without performing defective pixel correction.
Then, the process proceeds to step 6, where the image signal processing is performed and a series of image signal processing ends.

As described above, by correcting the defective pixel according to the exposure time, the defective pixel can be properly corrected during the long-time exposure, and the influence of the defective pixel can be reduced to generate the false signal. Can generate an image signal with a small number of pixels. Further, at the time of short-time exposure, since the influence of defective pixels is extremely small, image signal processing is performed without performing defective pixel correction, so that signal processing time can be reduced.

Next, a modification of the above embodiment will be described. In the modified example of the above embodiment, whether to perform the defective pixel correction is switched according to the exposure time,
A median filter process using a median filter is performed in defective pixel correction. The median filter used in the median filter processing has a configuration in which the shape of the median filter is switched in accordance with the exposure time and the correlation between the neighboring pixels. The median filter processing using the median filter is a filter processing in which the median of the peripheral pixels of the target pixel is interpolated. For example, as shown in FIG. 4, A (YIN (m-1, n)), B
Interpolate the median value of three pixels (YIN (m, n)) and C (YIN (m + 1, n)).

In this embodiment, the median filter is
A plurality of filters having different sizes and shapes are prepared. For example, filters as shown in FIGS. 6 to 13 are prepared, and the processing is performed by switching the median filter according to the exposure time of the imaging device 40. . FIG. 5 shows an example of a color filter used in the present embodiment, and the median filters of FIGS. 6 to 13 show filters of different shapes set based on the color filter. ing.

Next, the operation of the modification will be described with reference to the flowchart of FIG.

First, in step 200, exposure time information at the time of photographing is obtained by the exposure time determining unit 64, and the process proceeds to step 202.

In step 202, it is determined whether or not to perform defective pixel correction. The determination is made in step 20
This is performed by determining whether the exposure time acquired at 0 is longer than a predetermined time. If the determination is affirmative, the process proceeds to step 204.

In step 204, a median filter to be processed is determined based on the exposure time information obtained in step 202.

Then, in step 206, the correlation between the peripheral pixels is calculated for each pixel, and the process proceeds to step 208.
As shown in FIG. 15, for example, as shown in FIG. 15, the correlation between the peripheral pixels is obtained by subtracting the image data in the vertical direction using the pixel signals of the same color to obtain the correlation in the vertical direction (detecting an edge (boundary)). . Or, as shown in FIG.
It is possible to obtain the correlation by subtracting the image data in the horizontal direction and the image data in the vertical direction together with not performing the boundary detection in the horizontal direction using the pixel signals of the same color. Further, as shown in FIG. 17, it is possible to obtain the vertical correlation using the median value (1/2) of each pixel using the upper and lower pixel signals of the same color and the preceding and succeeding pixel signals. is there.

In step 208, it is determined whether or not there is a horizontal correlation based on the correlation calculated in step 206. If the determination is negative, it is determined that there is no horizontal correlation, and the process proceeds to step 210.

In step 210, it is determined whether or not there is a vertical correlation based on the correlation calculated in step 206. If the determination is negative, step 21
Move to 2.

In step 212, the digital signal processing section 50 performs various processes such as the above-described color correction, gamma correction, and Y / C conversion, and a series of image signal processing ends.

If the determination in step 208 is affirmative, the flow shifts to step 214 to perform a median filtering process on the photographed data in the vertical direction, and then to step 210, where the determination in step 210 is affirmative. If so, the flow shifts to step 216 to perform median filter processing in the horizontal direction on the photographed data, and shifts to step 212 described above.

On the other hand, if the determination in step 202 is negative, the process proceeds to step 212 without performing the defective pixel correction by the defective pixel correction unit 48, and the above-described image signal processing is performed to perform a series of processing. The process ends.

As described above, in the digital camera 10 of the present embodiment, whether or not to perform defective pixels is switched according to the exposure time of the photographing device 40, and
The correlation is determined from the peripheral pixels of the target pixel, and control is performed so as to switch whether to perform the median filter processing in the horizontal or vertical direction. That is, it is possible to appropriately perform defective pixel correction during long-time exposure, reduce the influence of defective pixels, and generate an image signal with less occurrence of false signals. In addition, by performing the median filter processing on a place that does not need to be corrected, it is possible to prevent the image quality from deteriorating in the image.

In the above embodiment, the digital camera 10 has been described as an example.
The present invention is not limited to this, but can be applied to any image signal processing device as long as an image is processed using an imaging device (element) such as a CCD sensor and a defective pixel of the imaging device is corrected. It is.

[0058]

As described above, according to the present invention, it is possible to appropriately perform defective pixel correction during long-time exposure by controlling the interpolation of defective pixels according to the exposure time or temperature of the image sensor. Thus, there is an effect that it is possible to provide an image signal processing device capable of reducing the influence of defective pixels and generating an image signal with less occurrence of false signals.

[Brief description of the drawings]

FIG. 1 is an external view of a digital camera according to an embodiment of the present invention, in which (A) is a front view and (B) is a back view.

FIG. 2 is a block diagram showing a configuration of an electric system of the digital camera according to the embodiment of the present invention.

FIG. 3 is a flowchart illustrating an operation of the digital camera according to the embodiment of the present invention.

FIG. 4 is a diagram for explaining a median filter.

FIG. 5 is a diagram illustrating an example of a color filter.

FIG. 6 is a diagram illustrating a first example of a median filter.

FIG. 7 is a diagram illustrating a second example of a median filter.

FIG. 8 is a diagram illustrating a third example of the median filter.

FIG. 9 is a diagram illustrating a fourth example of the median filter.

FIG. 10 is a diagram showing a fifth example of the median filter.

FIG. 11 is a diagram illustrating a sixth example of the median filter.

FIG. 12 is a diagram illustrating a seventh example of the median filter.

FIG. 13 is a diagram illustrating an eighth example of the median filter.

FIG. 14 is a flowchart illustrating an operation of a modification.

FIG. 15 is a diagram illustrating a case of performing vertical detection using pixel signals of the same color in pixel correlation determination.

FIG. 16 is a diagram showing an example in which pixel detection of a pixel is not performed in the horizontal direction using pixel signals of the same color in pixel correlation determination.
It is a figure at the time of detecting a vertical direction.

FIG. 17 is a diagram in the case of detecting in the vertical direction using upper and lower pixel signals of the same color and preceding and succeeding pixel signals.

[Explanation of symbols]

 Reference Signs List 10 digital camera 40 imaging device 46 A / D converter 48 defective pixel correction unit 64 exposure time determination unit 58 CPU

Claims (5)

[Claims] An image sensor that photoelectrically converts light that changes according to a subject and outputs an analog image signal, and an analog-digital converter that converts the analog image signal output from the image sensor into a digital image signal Conversion means, interpolation means for interpolating defective pixels in the image sensor based on the digital image signal converted by the analog-digital conversion means, and measurement means for measuring at least the exposure time or temperature in the image sensor An image signal processing apparatus comprising: a control unit that controls interpolation of the defective pixel by the interpolation unit according to a measurement result by the measurement unit. 2. The image signal processing apparatus according to claim 1, wherein said control means controls whether or not interpolation is performed by said interpolation means. 3. The image signal processing apparatus according to claim 1, wherein said interpolation means performs interpolation using a median filter. 4. The image signal processing apparatus according to claim 3, wherein the control unit switches the shape of the median filter according to an exposure time of the image sensor. 5. The apparatus according to claim 3, wherein the control unit determines a correlation between the pixels of the image sensor, and switches the shape of the median filter according to a result of the determination.
5. The image signal processing device according to claim 1.