JP2017055168A - Solid-state imaging device - Google Patents

Abstract

A solid-state imaging device capable of performing highly accurate interpolation processing on both imaging pixels and focus detection pixels is provided. A solid-state imaging device includes a row-direction interpolation circuit, a correction processing circuit, and an interpolation processing circuit. A pre-interpolator 33, which is a row-wise interpolator, uses the signal value of the first pixel arranged in the same row as the second pixel to generate a first interpolated value for the second pixel. A flaw correction circuit 36, which is a correction processing circuit, calculates a correction value for the center pixel of the first pixel block in which the second pixel is included in the first peripheral pixels, using the first interpolation value. The interpolation processing circuit 37 uses the signal values of the second peripheral pixels included in the second pixel block to perform interpolation processing on the second pixel. The interpolation processing circuit 37 replaces the first interpolation value for the second pixel with a second interpolation value calculated using the signal values of the second peripheral pixels. [Selection drawing] Fig. 5

Description

  The present embodiment relates to a solid-state imaging device.

  As one of autofocus methods for camera systems, a method using focus detection pixels arranged in a pixel region of a solid-state imaging device is known. The camera system performs an image signal interpolation process on the focus detection pixels.

  A solid-state imaging device may include a defect correction circuit that corrects a defective portion of an image signal by signal processing. The defect correction circuit calculates a correction value for the correction target by using a signal of the pixel to be corrected and a signal of a pixel around the correction target. The solid-state imaging device is desired to be able to suppress a decrease in the accuracy of flaw correction on the correction target when focus detection pixels are included around the correction target.

JP 2012-186789 A

  One embodiment provides a solid-state imaging device that enables high-precision correction processing to pixels arranged in a pixel region including focus detection pixels and high-precision interpolation processing to focus detection pixels. Objective.

  According to one embodiment, a solid-state imaging device includes a pixel region, a row direction interpolation circuit, a correction processing circuit, and an interpolation processing circuit. The pixel region includes pixels arranged in the row direction and the column direction. The column direction is a direction perpendicular to the row direction. The pixel region includes a first pixel and a second pixel. The second pixel detects a subject and a defocus. The row direction interpolation circuit generates a first interpolation value to the second pixel by using the signal value of the first pixel arranged in the same row as the second pixel. The correction processing circuit performs correction processing for the center pixel using the signal values of the first peripheral pixels. The first peripheral pixel is included in the first pixel block. The first pixel block is a matrix of pixels. The center pixel is located at the center of the first pixel block. The interpolation processing circuit performs an interpolation process on the second pixel located at the center of the second pixel block using the signal value of the second peripheral pixel. The second peripheral pixel is included in the second pixel block. The correction processing circuit calculates a correction value for the center pixel of the first pixel block in which the first peripheral pixel includes the second pixel by using the first interpolation value. The interpolation processing circuit replaces the second pixel with the second interpolation value calculated using the signal value of the second peripheral pixel from the first interpolation value.

FIG. 1 is a block diagram illustrating a configuration of a camera system including the solid-state imaging device according to the embodiment. FIG. 2 is a block diagram showing a schematic configuration of the solid-state imaging device shown in FIG. FIG. 3 is a diagram illustrating a first arrangement example of the phase difference pixels in the pixel region illustrated in FIG. 2. FIG. 4 is a diagram illustrating a second arrangement example of the phase difference pixels in the pixel region illustrated in FIG. 2. FIG. 5 is a block diagram showing a configuration of the imaging processing circuit shown in FIG. FIG. 6 is a diagram for explaining processing in the map scratch processing circuit shown in FIG. FIG. 7 is a diagram for explaining processing in the pre-interpolation circuit shown in FIG. FIG. 8 is a diagram for explaining a first example of the defect correction and interpolation processing in the embodiment. FIG. 9 is a flowchart illustrating a procedure of flaw correction and interpolation processing in the embodiment. FIG. 10 is a diagram illustrating a second example of the defect correction and interpolation processing in the embodiment. FIG. 11 is a diagram illustrating a third example of the defect correction and interpolation processing in the embodiment. FIG. 12 is a block diagram illustrating a configuration example of the interpolation processing circuit illustrated in FIG. FIG. 13 is a diagram for explaining interpolation processing by the interpolation processing circuit shown in FIG. FIG. 14 is a diagram for explaining interpolation processing by the interpolation processing circuit shown in FIG. FIG. 15 is a flowchart for explaining the procedure of interpolation processing by the interpolation processing circuit shown in FIG. FIG. 16 is a diagram illustrating a third arrangement example of the phase difference pixels in the pixel region illustrated in FIG. 2.

  Hereinafter, a solid-state imaging device according to an embodiment will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

(Embodiment)
FIG. 1 is a block diagram illustrating a configuration of a camera system including the solid-state imaging device according to the embodiment. The camera system 1 is an electronic device including the camera module 2 and is, for example, a mobile terminal with a camera. The camera system 1 may be an electronic device such as a digital camera.

  The camera system 1 includes a camera module 2 and a post-processing unit 3. The camera module 2 includes a lens module 4 and a solid-state imaging device 5. The lens module 4 includes an imaging optical system 6 and a lens driving unit 7.

  The imaging optical system 6 takes in light from the subject. The imaging optical system 6 includes an imaging lens (not shown) that forms a subject image. The lens driving unit 7 is a driving mechanism that moves the imaging lens. The lens driving unit 7 operates the imaging lens in a direction parallel to the optical axis of the imaging optical system 6 in accordance with a control signal from the focus driver 11. The lens driving unit 7 adjusts the focus of the imaging optical system 6 by adjusting the amount of extension of the imaging lens.

  The solid-state imaging device 5 includes an image sensor 8, an imaging processing circuit 9, a phase difference detection circuit 10, and a focus driver 11. The image sensor 8 captures a subject image. The imaging processing circuit 9 performs various signal processes on the image signal from the image sensor 8. The imaging processing circuit 9 outputs a signal from an imaging pixel described later to the ISP 12. The imaging processing circuit 9 outputs a signal from a later-described phase difference pixel to the phase difference detection circuit 10.

  The camera module 2 performs imaging by a so-called pupil division method using phase difference pixels. The phase difference detection circuit 10 obtains a defocus amount based on the phase difference between two images obtained by imaging. The focus driver 11 that is a focus adjustment unit generates a control signal corresponding to the defocus amount. The focus driver 11 outputs a control signal to the lens driving unit 7.

  The image sensor 8 is a backside illumination type CMOS (Complementary Metal Oxide Semiconductor) image sensor. In the back-illuminated CMOS image sensor, a wiring layer is provided on the opposite side of the semiconductor layer including the photoelectric conversion element from the incident light incident side. Note that the image sensor 8 is not limited to a back-illuminated CMOS image sensor, and may be a front-illuminated CMOS image sensor, a CCD (Charge Coupled Device), or the like.

  The post-processing unit 3 includes an image signal processor (ISP) 12, a recording unit 13, and a display unit 14. The ISP 12 performs signal processing of the image signal from the camera module 2. The ISP 12 performs various signal processing such as demosaic processing, white balance adjustment, color matrix processing, and gamma correction. The recording unit 13 records an image that has undergone signal processing in the ISP 12 on a storage medium or the like.

  The display unit 14 displays an image according to the image signal from the ISP 12 or the image signal read from the recording unit 13. The display unit 14 is, for example, a liquid crystal display. The camera system 1 performs feedback control of the solid-state imaging device 5 based on data that has undergone signal processing in the ISP 12.

  FIG. 2 is a block diagram showing a schematic configuration of the solid-state imaging device shown in FIG. The image sensor 8 includes a pixel region 20, a control circuit 21, a row scanning circuit 22, a column scanning circuit 23, and a column processing circuit 24.

  The pixel region 20 is a region including pixels arranged in the row direction and the column direction. The row direction and the column direction are directions perpendicular to each other. Each pixel includes a photodiode that is a photoelectric conversion element. The photoelectric conversion element generates a signal charge corresponding to the amount of incident light. The pixel accumulates signal charges generated according to the amount of incident light.

  The pixel region 20 includes an imaging pixel as a first pixel and a phase difference pixel as a second pixel. The imaging pixel is a pixel for detecting a subject image. The phase difference pixel that is a focus detection pixel is a pixel for detecting a defocus amount that is a shift amount between the focus of the imaging optical system and the subject.

  The control circuit 21, the row scanning circuit 22, the column scanning circuit 23, the column processing circuit 24, the imaging processing circuit 9, the phase difference detection circuit 10 and the focus driver 11 are integrated on the chip on which the pixel region 20 is mounted. The circuit unit is configured.

  Various data and clock signals for driving the image sensor 8 are supplied from the ISP 12 outside the chip to the control circuit 21 via the imaging processing circuit 9. The control circuit 21 generates various pulse signals for controlling the driving of the peripheral circuit unit according to the clock signal. The control circuit 21 supplies a pulse signal instructing drive timing to each of the row scanning circuit 22, the column scanning circuit 23, the column processing circuit 24, and the imaging processing circuit 9.

  The row scanning circuit 22 includes a shift register and an address decoder. The row scanning circuit 22 which is a pixel driving circuit supplies a driving signal to the pixels in the pixel region 20. The control circuit 21 supplies a pulse signal corresponding to the vertical synchronization signal to the row scanning circuit 22. The row scanning circuit 22 sequentially selects pixel rows from which pixel signals are read according to the pulse signal from the control circuit 21. The row scanning circuit 22 performs readout scanning by sequentially supplying a readout signal for each pixel in the selected pixel row. The read signal is a drive signal for reading a pixel signal generated according to the amount of incident light from the pixel.

  The row scanning circuit 22 performs sweep-out scanning by supplying a reset signal to each pixel prior to supplying a readout signal to each pixel. The reset signal is a drive signal for discharging the charge remaining in the photoelectric conversion element. Each pixel accumulates signal charges generated according to the amount of incident light from when the reset signal is supplied to when the readout signal is supplied.

  The drive signal is transmitted from the row scanning circuit 22 to each pixel through the pixel drive line 25. The pixel drive line 25 is provided for each pixel row in the pixel region 20. A pixel row consists of pixels arranged in the row direction (horizontal direction).

  The pixel signal is transmitted from each pixel to the column processing circuit 24 through the vertical signal line 26. The vertical signal line 26 is provided for each pixel column in the pixel region 20. The pixel column is composed of pixels arranged in the column direction (vertical direction).

  The column processing circuit 24 processes the pixel signal transmitted through the vertical signal line 26 by a unit circuit (not shown) provided for each pixel column. The column processing circuit 24 performs correlated double sampling processing (CDS) for reducing fixed pattern noise on the pixel signal. The column processing circuit 24 performs AD conversion, which is conversion to a digital signal, on the pixel signal, which is an analog signal. The column processing circuit 24 may perform processing other than CDS and AD conversion. The column processing circuit 24 holds the pixel signal that has undergone CDS and AD conversion for each unit circuit.

  The column scanning circuit 23 includes a shift register and an address decoder. The control circuit 21 supplies a pulse signal corresponding to the horizontal synchronization signal to the column scanning circuit 23. The column scanning circuit 23 sequentially selects pixel columns from which pixel signals are read according to the pulse signal from the control circuit 21. The column processing circuit 24 sequentially outputs pixel signals held in each unit circuit in accordance with the selective scanning by the column scanning circuit 23. The image sensor 8 outputs a signal having the pixel signal from the column processing circuit 24 as a component.

  3 and 4 are diagrams illustrating an example of arrangement of the phase difference pixels in the pixel region illustrated in FIG. In FIG. 3 and FIG. 4, locations indicated as “R”, “G”, and “B” represent red (R) pixels, green (G) pixels, and blue (B) pixels, which are imaging pixels, respectively.

  The Y direction shown in FIGS. 3 and 4 is the column direction of the pixel region 20, and the X direction is the row direction of the pixel region 20. A direction indicated by an arrow in the Y direction is a plus Y direction, and a direction opposite to the arrow is a minus Y direction. Of the X directions, the direction indicated by the arrow is the plus X direction, and the direction opposite to the arrow is the minus X direction.

  The R pixel, G pixel, and B pixel each include an R color filter, a G color filter, and a B color filter (all not shown). The R color filter is a color filter that selectively transmits R light. The G color filter is a color filter that selectively transmits G light. The B color filter is a color filter that selectively transmits B light. 3 and 4, the image pickup pixels of each color form a Bayer array.

  The phase difference pixel 41 includes a light shielding layer. 3 and 4, the hatched portion of the phase difference pixel 41 indicates a portion covered with a light shielding layer. The light shielding layer shields light. The light shielding layer is a layer containing a metal material that reflects light. The light shielding layer may be a layer containing a light absorbing material.

  In the portion of the phase difference pixel 41 other than the portion covered with the light shielding layer, an opening through which the light traveling to the photoelectric conversion element passes is provided. The phase difference pixel 41L is a phase difference pixel 41 in which an opening is provided in a half area on the minus X side and a light shielding layer is provided in a half area on the plus X side. The phase difference pixel 41R is a phase difference pixel 41 in which an opening is provided in a half area on the plus X side and a light shielding layer is provided in a half area on the minus X side.

  In the first arrangement example shown in FIG. 3, for example, in a Bayer array of 8 rows and 8 columns (8 × 8), one B pixel is replaced with the phase difference pixel 41L, and one R pixel is replaced with the phase difference pixel 41R. Yes. In the first arrangement example, the phase difference pixel 41L and the phase difference pixel 41R are adjacent to each other in a direction oblique to the X direction and the Y direction. The phase difference pixel 41R is in a position in an oblique direction between the plus X direction and the plus Y direction with respect to the phase difference pixel 41L.

  The phase difference pixel 41L and the phase difference pixel 41R may be arranged so that the positional relationship in the X direction or the Y direction is opposite to that shown in FIG. The phase difference pixel 41L and the phase difference pixel 41R may be arranged with one or more pixels apart from each other in addition to the arrangement adjacent to each other. In the pixel area 20, a plurality of combinations of two phase difference pixels 41L and 41R having such a positional relationship are arranged. Note that one R pixel and one B pixel in the Bayer array may be replaced with a phase difference pixel 41L and a phase difference pixel 41R, respectively.

  The combination of the two phase difference pixels 41L and 41R may be arranged in any ratio in the pixel region 20, and is arranged in one ratio in the 8 × 16 Bayer array or one ratio in the 16 × 16 Bayer array. May be.

  In the second arrangement example shown in FIG. 4, for example, one B pixel in an 8 × 8 Bayer array is replaced with a phase difference pixel 41L. Further, the B pixel at a position separating three pixels in the plus Y direction from the phase difference pixel 41L is replaced with the phase difference pixel 41R.

  The phase difference pixel 41L and the phase difference pixel 41R may be arranged so that the positional relationship in the Y direction is opposite to that shown in FIG. The phase difference pixel 41L and the phase difference pixel 41R may be arranged with one or three or more pixels apart from the three pixels. In the pixel area 20, a plurality of combinations of two phase difference pixels 41L and 41R having such a positional relationship are arranged. Note that two R pixels in the Bayer array may be replaced with two phase difference pixels 41L and 41R, respectively. The combination of the two phase difference pixels 41L and 41R may be arranged at any ratio in the pixel region 20 as in the first arrangement example.

  The signal component detected by the G pixel contains more luminance information of the subject than the signal component detected by the R pixel and the B pixel. The G pixel is a pixel having a larger influence on the luminance and resolution of the image than the R pixel and the B pixel. In the first arrangement example and the second arrangement example, the G pixels in the Bayer array are not replaced with the phase difference pixels 41L and 41R, and both are used for detection of image information. For this reason, the solid-state imaging device 5 can acquire an image with high brightness and high resolution.

  The phase difference pixels 41 </ b> L and 41 </ b> R are arranged dispersed in the entire pixel region 20 or a partial region. The arrangement mode of the phase difference pixels 41L and 41R in the pixel region 20 is not limited to the first and second arrangement examples, and is arbitrary. The phase difference pixels 41 </ b> L and 41 </ b> R only need to be arranged in a predetermined pattern in at least a part of the pixel region 20.

  Transparent filters are provided in the openings of the phase difference pixels 41L and 41R instead of the color filters in the imaging pixels. The transparent filter transmits light in a wider wavelength range than the color filter provided in the imaging pixel. By providing a transparent filter for the phase difference pixels 41L and 41R, the phase difference pixels 41L and 41R can obtain luminance information with high sensitivity.

  The phase difference pixels 41L and 41R may be provided with a G color filter instead of the transparent filter. Since the phase difference pixels 41L and 41R are provided with the G color filter, it is possible to obtain luminance information with higher sensitivity than when the R color filter or the B color filter is provided.

  The camera module 2 simultaneously acquires an image obtained by the plurality of phase difference pixels 41L and an image obtained by the plurality of phase difference pixels 41R. The camera module 2 performs imaging by the pupil division method by using the phase difference pixels 41L and 41R whose openings are symmetric in the X direction.

  In the in-focus state, the image obtained by the plurality of phase difference pixels 41L matches the image obtained by the plurality of phase difference pixels 41R. The phase difference between the image obtained by the plurality of phase difference pixels 41L and the image obtained by the plurality of phase difference pixels 41R is zero.

  In a state where the focus of the imaging optical system 6 is deviated from the subject, a phase difference occurs between an image obtained by the plurality of phase difference pixels 41L and an image obtained by the plurality of phase difference pixels 41R. The phase difference detection circuit 10 calculates the phase difference between the two images, and calculates the defocus amount of the imaging optical system 6 from the phase difference. The focus driver 11 generates a control signal for a focus operation for focusing on the subject according to the defocus amount obtained by the phase difference detection circuit 10.

  The phase difference pixels 41L and 41R are not limited to those arranged in place of the B pixel or the R pixel in the Bayer array. The phase difference pixels 41L and 41R may be arranged instead of the G pixels in the Bayer array.

  FIG. 5 is a block diagram showing a configuration of the imaging processing circuit shown in FIG. The imaging processing circuit 9 includes a black level correction circuit 31, a map scratch processing circuit 32, a pre-interpolation circuit 33, a digital gain circuit 34, a line memory 35, a scratch correction circuit 36, an interpolation processing circuit 37, a selector 38, and an OTP (One Time Programmable). memory) 39.

  The black level correction circuit 31 corrects the black level of the signal from the image sensor 8. The black level is a signal level used as a reference when the luminance level is expressed as a gradation, and is a signal level indicating the lowest gradation. The black level correction circuit 31 outputs an image signal from the imaging pixel to the map scratch processing circuit 32.

  The solid-state imaging device 5 holds in advance position information indicating the position of the phase difference pixel 41 in the pixel region 20. The control circuit 21 supplies a pulse signal corresponding to the position information of the phase difference pixel 41 to the imaging processing circuit 9. The pulse signal indicates the timing at which the signal from the phase difference pixel 41 is input to the imaging processing circuit 9. The black level correction circuit 31 outputs a signal from the phase difference pixel 41 to the phase difference detection circuit 10 in accordance with the pulse signal from the control circuit 21.

  The map scratch correction circuit 32, the pre-interpolation circuit 33, the scratch correction circuit 36, and the interpolation processing circuit 37 are configured by appropriately combining various logic circuits and storage elements for holding calculation results and various data. The storage element may be either a register or a memory.

  The map scratch processing circuit 32 performs map scratch correction on the image signal. The map scratch correction is a scratch correction for a map scratch which is a scratch mapped in advance. The map scratch processing circuit 32 is a row direction correction circuit that calculates a first correction value for a correction target using signal values of the imaging pixels arranged in the same row as the correction target pixel that is set in advance. is there.

  The OTP 39 is a memory that holds position information on scratches in the pixel region 20. The map scratch processing circuit 32 performs map scratch correction based on the position information read from the OTP 39.

  The map defect correction circuit 32 holds signals of a predetermined number of pixels in the row direction, and refers to the signal values of the imaging pixels in the same row as the correction target. The map flaw correction circuit 32 can perform flaw correction with a small memory configuration compared to the case of processing using pixel information in a two-dimensional direction.

  The pre-interpolation circuit 33 calculates an interpolation value temporarily set in the phase difference pixel 41. The pre-interpolation circuit 33 is a row direction interpolation circuit that interpolates the signal of the phase difference pixel 41 using the signal values of the imaging pixels arranged in the same row as the phase difference pixel 41. The pre-interpolation circuit 33 calculates a pre-interpolation value that is a first interpolation value for the phase difference pixel 41 in accordance with the pulse signal from the control circuit 21.

  The pre-interpolation circuit 33 holds signals of a predetermined number of pixels in the row direction, and refers to the signal values of the imaging pixels in the same row as the correction target. The pre-interpolation circuit 33 can perform the interpolation process with a small memory configuration compared to the case of the process using the pixel information in the two-dimensional direction.

  Note that the interpolation value finally set for the phase difference pixel 41 in the imaging processing circuit 9 is calculated by the interpolation processing circuit 37. In the following description, the interpolation performed by the interpolation processing circuit 37 may be referred to as “main interpolation”, and the interpolation performed by the pre-interpolation circuit 33 may be referred to as “pre-interpolation”. The imaging processing circuit 9 performs pre-interpolation in the pre-interpolation circuit 33 before the main interpolation in the interpolation processing circuit 37.

  The digital gain circuit 34 performs digital gain adjustment on the image signal. The line memory 35 holds pixel signals in units of rows. The line memory 35 delays the image signal for each row of the pixel region 20 by temporarily holding the image signal. The line memory 35 holds signals (4H) of four pixel rows. The line memory 35 is an SRAM, for example.

  The scratch correction circuit 36 and the interpolation processing circuit 37 are supplied with signals of four pixel rows held in the line memory 35. Further, a signal from one pixel row immediately before being stored in the line memory 35 is input to the defect correction circuit 36 and the interpolation processing circuit 37. Signals from five pixel rows are input to the defect correction circuit 36 and the interpolation processing circuit 37.

  The defect correction circuit 36 delays signals from five pixel rows in the row direction. The defect correction circuit 36 matches the timing of the signal of the pixel block in which the pixels are arranged in a 5 × 5 matrix. The defect correction circuit 36 performs direct defect correction on the image signal using the pixel block signal. The direct flaw correction is a flaw correction performed based on the result of flaw determination using an image signal obtained by imaging.

  The defect correction circuit 36 is a correction processing circuit that performs correction processing on the center pixel located at the center of the first pixel block using the signal value of the first peripheral pixel included in the first pixel block. . The first pixel block is a matrix of pixels. The defect correction circuit 36 performs defect determination for the center pixel and direct defect correction for the pixel determined to be a defect.

  The interpolation processing circuit 37 delays signals from the five pixel rows in the row direction. The defect correction circuit 36 matches the timing of the signal of the pixel block in which the pixels are arranged in a 5 × 5 matrix. The interpolation processing circuit 37 grasps the timing at which the center of the pixel block becomes the phase difference pixel 41 from the pulse signal corresponding to the position information of the phase difference pixel 41, and performs the interpolation process.

  The interpolation processing circuit 37 uses the signal value of the imaging pixel that is the second peripheral pixel included in the second pixel block, and is the second interpolation value to the phase difference pixel 41 located at the center of the second pixel block. This interpolation value is calculated. The interpolation processing circuit 37 calculates the main interpolation value based on the output two-dimensional image information, and replaces the pre-interpolation value with the main interpolation value.

  The selector 38 selects either the signal from the defect correction circuit 36 or the signal from the interpolation processing circuit 37. The selector 38 grasps the timing at which the interpolation value for the phase difference pixel 41 is output from the interpolation processing circuit 37 from the pulse signal corresponding to the position information of the phase difference pixel 41. The selector 38 selects the output from the interpolation processing circuit 37 at the timing when the interpolation value for the phase difference pixel 41 is output. The selector 38 selects the output from the defect correction circuit 36 when the signal value of the imaging pixel is output from the defect correction circuit 36. The imaging processing circuit 9 outputs the signal selected by the selector 38.

  The defect correction circuit 36 and the interpolation processing circuit 37 are connected in parallel to each other and share the line memory 35. The defect correction circuit 36 and the interpolation processing circuit 37 use a common line memory 35 to obtain a signal of the first pixel block and a signal of the second pixel block, respectively. The solid-state imaging device 5 can greatly reduce the circuit scale as compared with the case where the line memory for the defect correction circuit 36 and the line memory for the interpolation processing circuit 37 are provided separately.

  FIG. 6 is a diagram for explaining processing in the map scratch processing circuit shown in FIG. The adjustment process for storing the position information in the OTP 39 is performed, for example, in an inspection process at the time of manufacturing the solid-state imaging device 5. In the adjustment process, a test for detecting scratches from the pixel region 20 is performed. The OTP 39 stores position information set based on the result of the test.

  The OTP 39 holds the address of the scratch set as the master (M) among the map scratches to be corrected by the map scratch processing circuit 32 in association with the type type representing the scratch arrangement mode. In FIG. 6, type 0 to type 4 are shown as examples of five types.

  Type 0 to type 3 represent arrangement modes of two pixel scratches, which are scratches of two pixels close to each other. Of the two pixel scratches, a pixel scratch in the pixel region 20 in which the pixel signal readout order is first is a master, and a pixel scratch in which the readout order is later is slave (S). The master and slave have the same color pixel. The same color pixels are imaging pixels that detect the same color light. The slave address is determined by the master address and the type.

  The map scratch processing circuit 32 reads the master address and type type. The map flaw processing circuit 32 performs flaw correction on the master whose position is specified by the read address and the slave whose address is obtained from the read address and type type.

  Type 0 represents a state in which the slave is positioned across one different color pixel in an oblique direction between the plus Y direction and the minus X direction from the master. The different color pixel is an imaging pixel that detects color light different from the color light detected by the same color pixel. As information about the two-pixel scratches in the positional relationship, the master address and the data “0” indicating the type type are registered in the OTP 39.

  Type 1 represents a state in which the slave is located across one different color pixel in the plus Y direction from the master. Type 2 represents a state in which there is a slave across one different color pixel from the master in an oblique direction between the plus Y direction and the plus X direction. Type 3 represents a state in which the slave is positioned across one different color pixel in the plus X direction from the master.

  The map scratch processing circuit 32 calculates a correction value from the type 0 to the type 2 master using the signal values of the two adjacent same-color pixels P1 and P2. The adjacent same color pixels P1 and P2 are the same color pixels adjacent to the master via one different color pixel, and are located in the same pixel row as the master. The map scratch processing circuit 32 calculates the correction value from the type 0 to the type 2 slave using the signal values of the two adjacent same color pixels P3 and P4. The adjacent same color pixels P3 and P4 are the same color pixels adjacent to the slave via one different color pixel, and are located in the same pixel row as the slave. The map scratch processing circuit 32 calculates, for example, an average of signal values of two adjacent same-color pixels as correction values for the master and the slave.

  The map scratch processing circuit 32 calculates a correction value for the type 3 master using the signal values of the two adjacent pixels of the same color P1 and P2. The adjacent same color pixel P1 is one same color pixel adjacent to the master via one different color pixel. The adjacent same color pixel P2 is one same color pixel adjacent to the slave through one different color pixel. The adjacent same color pixels P1 and P2 are located in the same pixel row as the master and slave.

  As a correction value for the master, the map scratch processing circuit 32 calculates, for example, a weighted average obtained by weighting the signal values of two adjacent same-color pixels P1 and P2 according to the distance from the master. The map flaw processing circuit 32 calculates the correction value for the slave in the same manner as the correction value for the master. The solid-state imaging device 5 can correct two-pixel scratches in the map scratch processing circuit 32 by registering scratch information about types 0 to 3 in the OTP 39.

  Type 4 represents one pixel scratch of an arrangement mode that does not correspond to any of types 0 to 3. In the case of type 4, the address of the master that is one pixel scratch and the data “4” indicating the type type are registered in the OTP 39. The map scratch processing circuit 32 calculates a correction value for the master using the signal values of the two adjacent same-color pixels P1 and P2 of the master. The solid-state imaging device 5 can correct one pixel scratch in the map scratch processing circuit 32 by registering the scratch information about the type 4 in the OTP 39.

  The solid-state imaging device 5 can use the correction value calculated by the map flaw correction for the direct flaw correction in the flaw correction circuit 36 and the main interpolation process in the interpolation processing circuit 37. The solid-state imaging device 5 can reduce the influence of scratches in the direct scratch correction and the main interpolation process by performing the map scratch correction.

  FIG. 7 is a diagram for explaining processing in the pre-interpolation circuit shown in FIG. The pre-interpolation circuit 33 performs pre-interpolation on the phase difference pixel 41 according to the pulse signal corresponding to the position information of the phase difference pixel 41.

  The pre-interpolation circuit 33 compares the signal values of two adjacent same color pixels P5 and P6. The adjacent same color pixels P5 and P6 are the same color pixels adjacent to the phase difference pixel 41 through one different color pixel, and are located in the same pixel row as the phase difference pixel 41. The pre-interpolation circuit 33 employs one of the smaller signal values of the two adjacent same-color pixels P5 and P6 as a pre-interpolation value that is an interpolation value to the phase difference pixel 41.

  The pre-interpolation circuit 33 regards one of the two adjacent same-color pixels P5 and P6 having a larger signal value as a pixel having a possibility of white flaws. White scratches are scratches that show a higher signal level than when the pixel is functioning normally. The pre-interpolation circuit 33 uses the smaller one of the signal values of the two adjacent pixels of the same color P5 and P6 as the pre-interpolation value, so that the correction value calculated by the defect correction circuit 36 may be affected by white defects. Reduce.

  Note that scratches that may occur in the pixel region 20 include black scratches in addition to white scratches. Black scratches are scratches that indicate a lower signal level than when the pixel is functioning normally. In the present embodiment, each black scratch in the pixel region 20 may be a target for map scratch correction in the map scratch processing circuit 32. It is known that black scratches are generated less frequently than white scratches. For this reason, the position information of each black scratch can be stored in the OTP 39 without significantly increasing the storage capacity of the OTP 39. By inputting the signal subjected to the map defect correction to the black defect to the pre-interpolation circuit 33, the pre-interpolation circuit 33 can reduce the influence of the black defect on the generation of the pre-interpolation value.

  FIG. 8 is a diagram for explaining a first example of the defect correction and interpolation processing in the embodiment. FIG. 9 is a flowchart illustrating a procedure of flaw correction and interpolation processing in the embodiment. As shown in the upper part of FIG. 8, in the first example, it is assumed that white scratches are generated at the positions of the three pixels P11, P12, and P13 around the phase difference pixel 41. Three white scratches are detected in a test in the inspection process.

  In the first example, the phase difference pixel 41 is arranged in place of the B pixel in the Bayer array, and the pixels P11, P12, and P13 are all B pixels that are the same color pixels. The pixel P111 is located at a position through one different color pixel from the phase difference pixel 41 in an oblique direction between the minus Y direction and the minus X direction. The pixel P12 is located from the phase difference pixel 41 through one different color pixel in an oblique direction between the minus Y direction and the plus X direction. The pixel P13 is located from the phase difference pixel 41 via one different color pixel in the plus Y direction. Pixels P <b> 11, P <b> 12, and P <b> 13 are pixels included in a 5 × 5 pixel block centered on the phase difference pixel 41.

  Two white scratches in the pixels P11 and P12 are subject to map scratch correction. Two white scratches are registered in the OTP 39 as type 4 scratches, respectively. One white defect in the pixel P13 is not registered in the OTP 39, and is a target for direct defect correction.

  The map flaw processing circuit 32 performs map flaw correction on the two white flaws in the pixels P11 and P12 based on the position information registered in the OTP 39 (step S1). The map scratch processing circuit 32 calculates correction values for two white scratches using the signal values of B pixels arranged in the same row as the pixels P11 and P12. When the map defect correction is completed, one white defect in the pixel P13 is left around the phase difference pixel 41 as shown in the lower part of FIG.

  The pre-interpolation circuit 33 generates a pre-interpolation value for the phase difference pixel 41 (step S2). The pre-interpolation circuit 33 compares the signal values of two B pixels located in the minus X direction and the plus X direction from the phase difference pixel 41 via the different color pixels. The pre-interpolation circuit 33 outputs the smaller one of the two signal values as a pre-interpolation value.

  The defect correction circuit 36 performs defect determination for the pixel P13 using the signal of the first pixel block, which is a 5 × 5 pixel block having the pixel P13 as a central pixel. The scratch correction circuit 36 may perform the scratch determination by any method. When the determination result that the pixel P13 is white defect is obtained, the defect correction circuit 36 performs direct defect correction on the pixel P13 (step S3).

  The defect correction circuit 36 refers to signal values of eight peripheral pixels that are the first peripheral pixels included in the 5 × 5 pixel block. The peripheral pixels include seven B pixels and one phase difference pixel 41 shown in FIG. The defect correction circuit 36 calculates a correction value for the pixel P13 using the signal values of the seven B pixels and the pre-interpolation value that is the signal value of the phase difference pixel 41.

  The interpolation processing circuit 37 uses the signal of the second pixel block, which is a 5 × 5 pixel block with the phase difference pixel 41 as the central pixel, to calculate the main interpolation value for the phase difference pixel 41 (step S4). . The interpolation processing circuit 37 refers to the signal value of the imaging pixel that is the second peripheral pixel included in the pixel block, and interpolates the B component signal at the position of the phase difference pixel 41. The interpolation processing circuit 37 may calculate the interpolation value by any method. The interpolation processing circuit 37 may calculate the interpolation value by using a median filter, for example.

  The imaging processing circuit 9 outputs a signal from the defect correction circuit 36 selected by the selector 38 and a signal from the interpolation processing circuit 37. The solid-state imaging device 5 performs scratch correction and interpolation processing for each frame.

  In the first example, the solid-state imaging device 5 performs map flaw correction on two of the three flaws included in the pixel block centered on the phase difference pixel 41 and direct flaw correction on one. The solid-state imaging device 5 uses any one of a plurality of flaws included in the pixel block as a target for direct flaw correction, so that the information held in the OTP 39 is compared with a case where all of a plurality of flaws are subject to map flaw correction. Reduce the amount. The solid-state imaging device 5 can reduce the capacity of the OTP 39.

  In the imaging processing circuit 9, the image information at the position of the phase difference pixel 41 is lost in a step before the image signal interpolation processing for the phase difference pixel 41 is performed. Since the phase difference pixel 41 is included in the peripheral pixels whose signal value is referred to in the direct defect correction, the missing portion of the image information is referred to in the calculation of the correction value. Even though the image information is missing, the phase difference pixel 41 is treated in the same manner as the imaging pixel, so that the accuracy of the direct flaw correction is greatly reduced. When a measure is taken to exclude the phase difference pixel 41 from the peripheral pixel target, the image information of the position of the phase difference pixel 41 is not reflected in the calculated correction value. In this case also, the accuracy of direct scratch correction is reduced.

  In this embodiment, the pre-interpolation circuit 33 simply fills in image information at the position of the phase difference pixel 41 by generating a pre-interpolation value from the signal value of the imaging pixel in the same pixel row as the phase difference pixel 41. To do. The solid-state imaging device 5 can suppress a decrease in accuracy of direct scratch correction by performing direct scratch correction after a pre-interpolated value is temporarily placed as image information of the phase difference pixel 41. By providing the pre-interpolation circuit 33, the solid-state imaging device 5 can realize highly accurate direct defect correction with a relatively simple circuit configuration.

  Further, the interpolation processing circuit 37 calculates this interpolation value from the signal values of the peripheral pixels included in the pixel block, instead of the pre-interpolation value that is a simple hole filling of the image information. The interpolation processing circuit 37 can perform interpolation processing with higher accuracy than the interpolation processing in the pre-interpolation circuit 33 by calculating the interpolation value from the signal value in the two-dimensional direction with the phase difference pixel 41 as the center. To do. The solid-state imaging device 5 can reduce luminance deviation (artifact) in the phase difference pixel 41 and obtain an image with reduced unnaturalness by high-precision interpolation processing in the interpolation processing circuit 37.

  The solid-state imaging device 5 obtains a high-quality image by performing the flaw correction and the interpolation processing of the present embodiment even if a plurality of flaws are detected around the phase difference pixel 41 in the test in the inspection process. be able to. The solid-state imaging device 5 can improve the yield by reducing the occurrence frequency of defective products, and can reduce the manufacturing cost.

  FIG. 10 is a diagram illustrating a second example of the defect correction and interpolation processing in the embodiment. In the second example, it is assumed that white scratches are generated in the two pixels P14 and P15 around the phase difference pixel 41. Two white scratches are detected in a test in the inspection process.

  In the second example, the phase difference pixels 41 are arranged in place of the B pixels in the Bayer array, and the pixels P14 and P15 are both B pixels that are the same color pixels. The pixel P14 is located at a position via one different color pixel from the phase difference pixel 41 in an oblique direction between the minus Y direction and the plus X direction. The pixel P15 is located at a position from the phase difference pixel 41 via one different color pixel in the minus X direction. The pixels P14 and P15 are pixels included in a 5 × 5 pixel block with the phase difference pixel 41 as the center.

  One white scratch on the pixel P14 is registered in the OTP 39 as a type 4 scratch. One white scratch in the pixel P15 is not registered in the OTP 39. The map flaw processing circuit 32 performs map flaw correction for one white flaw in the pixel P14 based on the position information registered in the OTP 39. When the map defect correction is completed, one white defect in the pixel P15 is left around the phase difference pixel 41 as shown in the lower part of FIG.

  The pre-interpolation circuit 33 generates a pre-interpolation value for the phase difference pixel 41. Since the pre-interpolation circuit 33 selects the smaller one of the two signal values, a signal value other than the signal value of the pixel P15 that is a white defect is employed as the pre-interpolation value.

  The defect correction circuit 36 performs direct defect correction on the pixel P15. The peripheral pixels in the 5 × 5 pixel block include seven B pixels and one phase difference pixel 41. The defect correction circuit 36 calculates a correction value for the pixel P15 using the signal values of the seven B pixels and the pre-interpolation value that is the signal value of the phase difference pixel 41. The interpolation processing circuit 37 calculates the actual interpolation value for the phase difference pixel 41.

  Also in the case of the second example, the solid-state imaging device 5 can reduce the capacity of the OTP 39 by setting one white defect as a target for direct defect correction. Moreover, the solid-state imaging device 5 can suppress a decrease in accuracy of direct scratch correction by generating a pre-interpolated value. The solid-state imaging device 5 can reduce the artifact in the phase difference pixel 41 by calculating this interpolation value, and obtain an image with reduced unnaturalness. The solid-state imaging device 5 can improve the yield and reduce the manufacturing cost.

  FIG. 11 is a diagram illustrating a third example of the defect correction and interpolation processing in the embodiment. In the third example, it is assumed that white flaws occur in the pixel P16 that is intermediate between the two phase difference pixels 41L and 41R. In the third example, the phase difference pixels 41L and 41R are arranged in place of the B pixels in the Bayer array, and the pixel P16 is a B pixel. The phase difference pixel 41L is located from the pixel P16 via one different color pixel in the minus Y direction. The phase difference pixel 41R is located from the pixel P16 via one different color pixel in the plus X direction.

  In the third example, one white defect in the pixel P16 is not registered in the OTP 39. The pre-interpolation circuit 33 generates pre-interpolation values for the phase difference pixels 41L and 41R, respectively. The scratch correction circuit 36 calculates a correction value for the pixel P16 using the signal values of the six B pixels and the two pre-interpolated values that are the signal values of the phase difference pixels 41L and 41R.

  Also in the case of the third example, the solid-state imaging device 5 can reduce the capacity of the OTP 39 by making one white defect a target for direct defect correction. Moreover, the solid-state imaging device 5 can suppress a decrease in accuracy of direct scratch correction by generating a pre-interpolated value. The solid-state imaging device 5 can reduce the artifact in the phase difference pixel 41 by calculating this interpolation value, and obtain an image with reduced unnaturalness.

  Even if there is a scratch between the two phase difference pixels 41, the solid-state imaging device 5 can obtain a high-quality image by performing the scratch correction and interpolation processing of the present embodiment. The solid-state imaging device 5 can improve the yield by reducing the occurrence frequency of defective products, and can reduce the manufacturing cost.

  In the first and second examples, one white defect is left in the pixel block that is referred to when the interpolation value is calculated. It is desirable that the interpolation processing circuit 37 can calculate the interpolation value by a technique that can reduce the influence of the presence of one flaw in the pixel block. Next, a calculation example of the interpolation value in the interpolation processing circuit 37 will be described.

  FIG. 12 is a block diagram illustrating a configuration example of the interpolation processing circuit illustrated in FIG. The interpolation processing circuit 37 includes a change amount comparison circuit 51, a color difference interpolation circuit 52, an average circuit 53, and a clip circuit 54.

  The change amount comparison circuit 51 compares luminance change amounts in a plurality of directions centered on the phase difference pixel 41. The color difference interpolation circuit 52 calculates an interpolation value for the phase difference pixel 41. The averaging circuit 53 averages the interpolation values generated by the color difference interpolation circuit 52. The clip circuit 54 performs clip processing on the average value calculated by the average circuit 53.

  13 and 14 are diagrams for explaining the interpolation processing by the interpolation processing circuit shown in FIG. 13 and 14 show a pixel block 55 centered on the phase difference pixel 41 arranged in place of the B pixel in the Bayer array. In this example, the interpolation processing circuit 37 uses the signal value of the G pixel and the signal value of the B pixel in the pixel block 55 in the interpolation process. The G pixel is a first pixel that detects G light that is first color light. The B pixel is a second pixel that detects B light that is second color light.

  As shown in FIGS. 13 and 14, the first direction d1, the second direction d2, the third direction d3, and the fourth direction d4 are a negative Y direction, a positive Y direction, a negative X direction, and a positive X direction, respectively. The fifth direction d5 is an oblique direction between the minus Y direction and the minus X direction. The sixth direction d6 is an oblique direction between the plus Y direction and the plus X direction. The seventh direction d7 is an oblique direction between the minus Y direction and the plus X direction. The eighth direction d8 is an oblique direction between the plus Y direction and the minus X direction.

  FIG. 15 is a flowchart for explaining the procedure of interpolation processing by the interpolation processing circuit shown in FIG. The change amount comparison circuit 51 calculates a score for each direction in the eight directions d1 to d8 centering on the phase difference pixel 41 (step S11). The score is an evaluation value for comparing the amount of change in luminance. The change amount comparison circuit 51 calculates a score using the signal value of the G pixel that is the first pixel included in the pixel block.

The change amount comparison circuit 51 calculates SC1, which is a score in the first direction d1, using the signal value of the G pixel included in the pixel group PG1 shown in FIG. The pixel group PG1 includes pixels located on the first direction d1 side from the phase difference pixel 41. The change amount comparison circuit 51 calculates SC1 by the following equation (1), for example.
SC1 = | G2-G5 | + | G4-G12 | (1)

  | G2-G5 | in Expression (1) is the absolute value of the difference between the signal values of the two G pixels “G2” and “G5”, and is on the minus X side of the phase difference pixel 41 in the range of the pixel group PG1. This represents the amount of change in luminance in the portion. | G4-G12 | is the absolute value of the difference between the signal values of the two G pixels “G4” and “G12”, and changes in luminance in the plus X side portion of the phase difference pixel 41 in the range of the pixel group PG1. Represents an amount. The change amount comparison circuit 51 considers that the amount of change in luminance in the first direction d1 is smaller as SC1 is lower.

  Similar to the case of the first direction d1, the change amount comparison circuit 51 calculates SC2 to SC8 which are scores in the second direction d2 to the eighth direction d8. The change amount comparison circuit 51 obtains SC2, SC3, and SC4 from the signal values of the pixel groups PG2, PG3, and PG4 shown in FIG. The pixel group PG2 includes pixels located on the second direction d2 side from the phase difference pixel 41. The pixel group PG3 includes pixels located on the third direction d3 side from the phase difference pixel 41. The pixel group PG4 includes pixels located on the fourth direction d4 side from the phase difference pixel 41.

The change amount comparison circuit 51 calculates SC2, SC3, and SC4 by the following equations (2) to (4), for example.
SC2 = | G2-G8 | + | G4-G9 | (2)
SC3 = | G1-G6 | + | G3-G7 | (3)
SC4 = | G1-G11 | + | G3-G10 | (4)

  The change amount comparison circuit 51 obtains SC5, SC6, SC7, and SC8 from the signal values of the pixel groups PG5, PG6, PG7, and PG8 shown in FIG. The pixel group PG5 includes pixels located on the fifth direction d5 side from the phase difference pixel 41. The pixel group PG6 includes pixels located on the sixth direction d6 side from the phase difference pixel 41. The pixel group PG7 includes pixels located on the seventh direction d7 side from the phase difference pixel 41. The pixel group PG8 includes pixels located on the eighth direction d8 side from the phase difference pixel 41.

The change amount comparison circuit 51 calculates SC5, SC6, SC7, and SC8 by the following equations (5) to (8), for example.
SC5 = | G4-G5 | + | G3-G6 | (5)
SC6 = | G1-G10 | + | G2-G9 | (6)
SC7 = | G2-G12 | + | G3-G11 | (7)
SC8 = | G1-G7 | + | G4-G8 | (8)

  The change amount comparison circuit 51 compares the calculated scores SC1 to SC8. The change amount comparison circuit 51 detects the smallest two of SC1 to SC8. The change amount comparison circuit 51 selects the two directions dm1 and dm2 having the smallest score among the eight directions d1 to d8 (step S12). The change amount comparison circuit 51 determines that the two directions dm1 and dm2 are the two directions having the smallest luminance change amount among the eight directions d1 to d8. The two directions dm1 and dm2 are directions that are highly likely to be directions of a linear pattern included in the image among the eight directions d1 to d8.

  Note that the change amount comparison circuit 51 may compare the amount of change in luminance based on evaluation values other than the scores shown in the equations (1) to (8). The change amount comparison circuit 51 may compare the amount of change in luminance in each direction around the phase difference pixel 41 by any method.

  The color difference interpolation circuit 52 calculates an interpolation value to the phase difference pixel 41 using the signal value of the reference pixel selected based on the comparison result by the change amount comparison circuit 51. The reference pixels are a B pixel at a position in the direction dm1 from the phase difference pixel 41 and a B pixel at a position in the direction dm2 from the phase difference pixel 41. In the present embodiment, the color difference interpolation circuit 52 obtains eight interpolation values for the eight directions d1 to d8 (step S13). The color difference interpolation circuit 52 selects an interpolation value for the direction dm1 and an interpolation value for the direction dm2 from the obtained eight interpolation values. The eight interpolation values obtained by the color difference interpolation circuit 52 are used for setting reference values in the clip circuit 54 described later.

The color difference interpolation circuit 52 calculates the interpolation value Vd1 in the first direction d1 using the signal values of the G pixel and the B pixel included in the pixel group PG1 shown in FIG. The color difference interpolation circuit 52 calculates Vd1 by the following equation (9), for example. Expression (9) is an expression modified from Expression (9 ′).
Vd1 = B1- (G5 + G12) / 2 + (G2 + G4) / 2 (9)
Vd1− (G2 + G4) / 2 = B1− (G5 + G12) / 2 (9 ′)

  The right side of Expression (9 ′) represents the color difference between the G component and the B component in the G pixels “G5” and “G12” and the B pixel “B1”. The left side of the equation (9 ′) represents the color difference between the G component and the B component in the phase difference pixel 41 and the G pixels “G2” and “G4”. The G pixels “G2” and “G4” are G pixels adjacent to the phase difference pixel 41 in a direction perpendicular to the first direction d1.

  Expression (9 ′) is obtained by calculating the color difference between the phase difference pixel 41 and the adjacent G pixels “G2” and “G4” and the color difference between the B pixel “B1” and the adjacent G pixels “G5” and “G12”. Represents a correlation that is equal. The color difference interpolation circuit 52 calculates an interpolation value Vd1 that is a signal value for the B component, assuming that the relationship of Expression (9 ') is satisfied.

  When one of the two directions dm1 and dm2 selected by the change amount comparison circuit 51 is the first direction d1, the color difference interpolation circuit 52 selects the interpolation value Vd1. The B pixel “B1” at a position in the first direction d1 from the phase difference pixel 41 is a reference pixel selected based on the comparison result in the change amount comparison circuit 51. The G pixels “G2” and “G4” are the first pixels around the phase difference pixel 41. The G pixels “G5” and “G12” are first pixels around the reference pixel. In the present embodiment, the first pixel around the phase difference pixel 41 is assumed to include the first pixel adjacent to the phase difference pixel 41. The first pixel around the reference pixel includes a first pixel adjacent to the reference pixel.

  In equation (9), (G5 + G12) / 2, which is the second term on the right side, is an average value of signal values of G pixels adjacent to the reference pixel. The third term (G2 + G4) / 2, which is the third term on the right side, is an average value of signal values of G pixels adjacent to the phase difference pixel 41. According to equation (9), the color difference interpolation circuit 52 adds these differences {(G2 + G4) / 2− (G5 + G12) / 2} to the signal value “B1” of the reference pixel. The color difference interpolation circuit 52 employs the signal value of the reference pixel that has been adjusted by the addition of the difference as the interpolation value Vd1.

Similar to the case of the first direction d1, the color difference interpolation circuit 52 calculates the interpolation values Vd2 to Vd8 in the second direction d2 to the eighth direction d8. The color difference interpolation circuit 52 calculates Vd2 to Vd8 by the following equations (10) to (16), for example.
Vd2 = B3- (G8 + G9) / 2 + (G2 + G4) / 2 (10)
Vd3 = B2- (G6 + G7) / 2 + (G1 + G3) / 2 (11)
Vd4 = B4- (G10 + G11) / 2 + (G1 + G3) / 2 (12)
Vd5 = B5- (G5 + G6) / 2 + (G4 + G3) / 2 (13)
Vd6 = B7− (G9 + G10) / 2 + (G1 + G2) / 2 (14)
Vd7 = B8− (G12 + G11) / 2 + (G2 + G3) / 2 (15)
Vd8 = B6- (G7 + G8) / 2 + (G1 + G4) / 2 (16)

  The color difference interpolation circuit 52 selects two interpolation values for the two directions dm1 and dm2 from the eight interpolation values Vd1 to Vd8. The color difference interpolation circuit 52 outputs the two selected interpolation values to the averaging circuit 53. The color difference interpolation circuit 52 outputs the eight interpolation values Vd1 to Vd8 to the clip circuit 54.

  In a direction that is determined to be highly likely to be a pattern direction, it is determined that the amount of change in luminance is small compared to other directions, but a slight luminance change may be included. For this reason, when the signal value of the B pixel as the reference pixel is directly used as the interpolated value for the phase difference pixel 41, a luminance shift (artifact) due to insufficient interpolation to the phase difference pixel 41 may remain in the image. There is.

  In the present embodiment, the interpolation processing circuit 37 considers that there is almost no color difference change in the pattern direction, adjusts the signal value of the reference pixel, and calculates an interpolation value. The interpolation processing circuit 37 performs color difference compensation on the signal value of the reference pixel based on the correlation between the color difference between the phase difference pixel 41 and the G pixel and the color difference between the reference pixel and the G pixel. The color difference interpolation circuit 52 reduces artifacts by such adjustment. Note that the color difference interpolation circuit 52 may obtain the interpolation values Vd1 to Vd8 by a calculation other than the calculations according to the expressions (9) to (16).

  The averaging circuit 53 calculates the average value Vav of the two interpolation values for the two directions dm1 and dm2 (step S14). The average circuit 53 is an average value Vav that is an interpolation value in which luminance information and color difference information in two directions determined to be highly likely to be the direction of the pattern included in the image among the eight directions d1 to d8. Is calculated.

  The interpolation processing circuit 37 averages the interpolation values in the two directions dm1 and dm2 that are highly likely to be the pattern directions in the averaging circuit 53. When a pattern in two different directions from the phase difference pixel 41 is included in the image, the interpolation processing circuit 37 can obtain an interpolation value that reflects the image information in the two directions. The solid-state imaging device 5 can reduce the artifact in the phase difference pixel 41 and obtain an image with reduced unnaturalness.

  The clip circuit 54 sets Rmax and Rmin as reference values in the clip process (step S15). The clip circuit 54 performs clip processing for limiting the level of the interpolation value to the phase difference pixel 41 to a level having Rmax as an upper limit and Rmin as a lower limit.

  The clip circuit 54 rearranges the eight interpolation values Vd1 to Vd8 obtained in step S13 in order of level. The clip circuit 54 sets the interpolation value at the second level from the top of the interpolation values Vd1 to Vd8 to Rmax, which is one of the reference values. The clip circuit 54 sets the interpolation value of the second lowest level among the interpolation values Vd1 to Vd8 to Rmin which is one of the reference values. The clip circuit 54 performs clip processing according to the procedure of the next step S16 and step S17.

  In step S16, the clipping circuit 54 compares the average value Vav obtained in step S14 with the reference value Rmin. In step S17, the clipping circuit 54 compares the average value Vav obtained in step S14 with the reference value Rmax.

  When Vav is smaller than Rmin (step S16, Yes), the clip circuit 54 selects Rmin. Thereby, the clipping circuit 54 performs the clipping process by Rmin as the lower limit of the interpolation value on the Vav from the averaging circuit 53. The interpolation processing circuit 37 sets Rmin to the interpolation value Vin for the phase difference pixel 41 (step S18).

  When Vav is equal to or greater than Rmin (No in step S16) and Vav is greater than Rmax (Yes in step S17), the clip circuit 54 selects Rmax. As a result, the clipping circuit 54 performs the clipping process with Rmax, which is the upper limit of the interpolation value, on the Vav from the averaging circuit 53. The interpolation processing circuit 37 sets Rmax to the interpolation value Vin for the phase difference pixel 41 (step S19).

  When Vav is equal to or greater than Rmin (step S16, No) and Vav is equal to or less than Rmax (step S17, No), the clip circuit 54 selects Vav. The clip circuit 54 allows the Vav from the averaging circuit 53 to pass without being clipped. The interpolation processing circuit 37 sets Vav as the interpolation value Vin for the phase difference pixel 41 (step S20).

  In this way, the interpolation processing circuit 37 calculates the interpolation value that is the interpolation value Vin set in any of steps S18 to S20. The interpolation processing circuit 37 replaces the phase difference pixel 41 from the pre-interpolation value to the main interpolation value. Thereby, the interpolation processing circuit 37 ends the interpolation processing to the phase difference pixel 41.

  Similarly to the case where the phase difference pixel 41 arranged instead of the B pixel is targeted, the interpolation processing circuit 37 performs an interpolation process targeting the phase difference pixel 41 arranged instead of the R pixel. In this case, the interpolation processing circuit 37 performs an interpolation process using the G pixel as the first pixel and the R pixel as the second pixel.

  For the calculation of the interpolated value in one of the two directions dm1 and dm2, the signal value of the second pixel that is a white defect may be used. By using an abnormal signal due to white flaws, a high interpolation value is calculated regardless of the color information of the subject.

  The interpolation processing circuit 37 regards an interpolation value at a level higher than Rmax obtained from the interpolation values Vd1 to Vd8 calculated by the color difference interpolation circuit 52 as a result of being affected by white scratches. The interpolation processing circuit 37 can reduce the influence of the second pixel that is a white defect by performing the clipping process by Rmax on the interpolation value determined to include the influence of the white defect.

  In addition, the signal value of the second pixel that is a black defect may be used for calculation of the interpolation value for one of the two directions dm1 and dm2. By using an abnormal signal due to black flaws, a low interpolation value is calculated regardless of the color information of the subject.

  The interpolation processing circuit 37 regards the interpolation value at a level lower than Rmin obtained from the interpolation values Vd1 to Vd8 calculated by the color difference interpolation circuit 52 as a result of being affected by black scratches. The interpolation processing circuit 37 can reduce the influence of the second pixel that is a black flaw by performing a clipping process with Rmin on the interpolation value determined to include the influence of the black flaw.

  If the direction of the pattern included in the image does not coincide with any of the eight directions d1 to d8, a deviation occurs between the two selected directions dm1 and dm2 and the direction of the actual pattern included in the image. . In this case, in the color difference interpolation circuit 52, there is a possibility that the signal value of the first pixel other than the first pixel in the actual pattern direction is used to calculate the interpolation values in the two directions dm1 and dm2.

  The interpolation processing circuit 37 may calculate an interpolation value of an abnormally high level or a lower level than a normal level due to an error in calculation of the interpolation value in the color difference interpolation circuit 52. There is. The interpolation processing circuit 37 corrects the interpolation value to a value close to a normal level by performing clipping processing in the clipping circuit 54 on the interpolation value having such an abnormal level.

  The clip circuit 54 corrects the interpolation value that has become an abnormal level due to factors that cannot be dealt with by the processing up to the averaging circuit 53 to a normal level. The solid-state imaging device 5 can reduce the artifact in the phase difference pixel 41 and obtain an image with reduced unnaturalness. Note that the reference value in the clipping process is not limited to one of the interpolation values Vd1 to Vd8. The reference value may be set by any method. For example, the clip processing circuit 54 may perform clip processing using a preset reference value.

  The pixel block may include a first pixel that is white. When the signal of the first pixel, which is a white defect, is used for calculation of the score in the change amount comparison circuit 51, a high value is calculated for the score regardless of the luminance information of the subject. The direction having the score having such a high value is excluded in the selection of the direction in the change amount comparison circuit 51. The interpolation processing circuit 37 can exclude an interpolation value including the influence of the first pixel that is a white defect.

  The phase difference pixels 41 are not limited to those arranged in the Bayer array. The phase difference pixels 41 may be arranged in a pixel array other than the Bayer array. For example, the pixel array may be provided with white (W) pixels instead of the Bayer array G pixels. The W pixel is an imaging pixel that detects white light. White light includes light having a wavelength in the entire visible region. The solid-state imaging device 5 can capture a subject image with high sensitivity by including W pixels in the imaging pixels.

  FIG. 16 is a diagram illustrating a third arrangement example of the phase difference pixels in the pixel region illustrated in FIG. 2. In the pixel array shown in FIG. 16, all G pixels in the Bayer array are replaced with W pixels. The W pixel includes a transparent filter (not shown) that transmits white light. The pixel array includes W pixels, R pixels, and B pixels. The imaging processing circuit 9 converts an image signal, which is a detection result of light of each color component of W, R, and B, into an image signal corresponding to the Bayer array.

  In the third arrangement example, similarly to the first arrangement example and the second arrangement example, combinations of the two phase difference pixels 41L and 41R are arranged in the pixel region 20 at a predetermined ratio. The W pixel detects light in a wider wavelength region than the R pixel and the B pixel. The signal component detected by the W pixel contains more luminance information of the subject than the signal component detected by the R pixel and the B pixel. In the third arrangement example, the W pixels in the pixel array are not replaced with the phase difference pixels 41L and 41R, and both are used for detection of image information. The solid-state imaging device 5 can acquire an image having high brightness and high resolution.

  FIG. 16 shows a 5 × 5 pixel block 60 centered on the phase difference pixel 41 arranged in place of the B pixel. The interpolation processing circuit 37 uses the signal value of the W pixel and the signal value of the B pixel in the pixel block 60 in the interpolation process. In this example, the W pixel is a first pixel that detects W light that is first color light. The B pixel is a second pixel that detects B light that is second color light. In the present embodiment, white light is assumed to be included in “color light”. Also in the case of the third arrangement example, the solid-state imaging device 5 can suppress a decrease in accuracy of direct scratch correction.

  According to the embodiment, the solid-state imaging device 5 can suppress a decrease in accuracy of direct scratch correction by performing direct scratch correction after generating a pre-interpolated value for the phase difference pixel 41. The solid-state imaging device 5 can perform highly accurate interpolation processing by replacing the pre-interpolation value with the main interpolation value for the phase difference pixel 41. Thereby, the solid-state imaging device 5 can obtain an effect that high-accuracy correction processing to pixels arranged in the pixel region 20 including focus detection pixels and high-accuracy interpolation processing of image signals can be realized. .

  Although the embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  5 solid-state imaging device, 20 pixel area, 32 map scratch processing circuit, 33 pre-interpolation circuit, 35 line memory, 36 scratch correction circuit, 37 interpolation processing circuit, 39 OTP, 41, 41L, 41R phase difference pixel.

Claims (6)

A pixel region including pixels arranged in a row direction and a column direction perpendicular to the row direction, the pixel region including a first pixel and a second pixel that detects a subject and a defocus;
Using a signal value of the first pixel arranged in the same row as the second pixel, and generating a first interpolation value to the second pixel;
A correction processing circuit that performs correction processing on a central pixel located at the center of the first pixel block using a signal value of a first peripheral pixel included in the first pixel block that is a matrix of pixels;
An interpolation processing circuit for performing an interpolation process on a second pixel located at the center of the second pixel block using a signal value of a second peripheral pixel included in the second pixel block which is a matrix of pixels; With
The correction processing circuit calculates a correction value for a central pixel of a first pixel block including a second pixel in the first peripheral pixel using the first interpolation value,
The interpolation processing circuit replaces the second pixel with the second interpolation value calculated using the signal values of the second peripheral pixels from the first interpolation value to the second pixel. apparatus. A row direction correction circuit for calculating a first correction value for the correction target using a signal value of the first pixel arranged in the same row as the first pixel registered in advance as the correction target;
The correction processing circuit uses the first correction value to calculate a second correction value that is the correction value for the central pixel of the first pixel block in which the correction target is included in the first peripheral pixel. The solid-state imaging device according to claim 1, wherein   The row direction interpolation circuit employs, as the first interpolation value, the smaller one of the signal values of two first pixels arranged in the same row as the second pixel. Or the solid-state imaging device of 2.   The solid-state imaging device according to claim 2, further comprising a memory that holds position information of the correction target in the row direction correction circuit. A line memory that holds pixel signals in units of rows,
5. The correction processing circuit and the interpolation processing circuit obtain a signal of a first pixel block and a signal of a second pixel block, respectively, using the common line memory. The solid-state imaging device according to one item. A pixel region including pixels arranged in a row direction and a column direction perpendicular to the row direction, the pixel region including a first pixel and a second pixel that detects a subject and a defocus;
A row direction interpolation circuit for generating an interpolation value for the second pixel using a signal value of the first pixel arranged in the same row as the second pixel;
A correction processing circuit that performs correction processing on a central pixel located at the center of the pixel block using signal values of peripheral pixels that are pixels included in the pixel block that is a pixel matrix; and
The solid-state imaging device, wherein the correction processing circuit calculates a correction value for a central pixel of a pixel block in which a second pixel is included in the peripheral pixels by using the interpolation value.

Images