JP2018101940A - Photoelectric conversion device, photoelectric conversion method, and image forming apparatus - Google Patents

Abstract

Even when a plurality of defective pixels exist continuously, it is possible to perform an appropriate interpolation process. A photoelectric conversion unit converts light received by each pixel into an electrical signal. The detection unit detects a defective pixel that outputs an electrical signal of an abnormal level among the pixels of the photoelectric conversion unit. The specifying unit specifies a defective pixel having a continuous physical position on the photoelectric conversion unit among the detected defective pixels. The plurality of interpolation units perform an interpolation process using the electric signal of the defective pixel as an electric signal of a normal pixel. The separation unit separates the electrical signal from the photoelectric conversion unit and supplies the signal to the interpolation unit so that the defective pixels at the specified consecutive positions are interpolated by different interpolation units. The synthesizer synthesizes the electrical signals that have been subjected to the interpolation processing by the interpolation units, and reproduces and outputs a series of electrical signals. As a result, even when there are a plurality of defective pixels in succession, an electric signal subjected to appropriate interpolation processing can be output. [Selection] Figure 11

Description

  The present invention relates to a photoelectric conversion device, a photoelectric conversion method, and an image forming apparatus.

  Nowadays, an interpolation process for interpolating abnormal image data generated at a defective pixel of an image sensor using image data from pixels around the defective pixel is known. For example, Patent Document 1 (Japanese Unexamined Patent Application Publication No. 2009-130553) discloses a defective pixel correction method for the purpose of appropriately interpolating consecutively generated defective pixel groups. This defective pixel correction method compares the degree of defects in the divided areas, leaves defective pixels with a small degree of defect, corrects defective pixels with a large degree of defect, and then detects the defective pixels again after remaining defective pixels, When it is determined as a defective pixel, it is generated by interpolation from surrounding pixel values.

  However, in addition to the defective pixel correction method disclosed in Patent Document 1, when a plurality of defective pixels exist continuously, the conventional interpolation processing performs interpolation processing using another defective pixel adjacent to the defective pixel. Therefore, there is a problem that the interpolation processing becomes insufficient or is replaced with an incorrect value.

  The present invention has been made in view of the above-described problems, and provides a photoelectric conversion device, a photoelectric conversion method, and an image forming apparatus that enable appropriate interpolation processing even when a plurality of defective pixels exist continuously. Objective.

  In order to solve the above-described problems and achieve the object, the present invention provides a photoelectric conversion unit that converts light received by each pixel into an electrical signal, and a defective pixel that outputs an electrical signal of an abnormal level among the pixels. A detection unit that detects a defect pixel, a specifying unit that identifies a defective pixel having a continuous physical position on the photoelectric conversion unit, and an electrical signal of the defective pixel as an electrical signal of a normal pixel A plurality of interpolation units that perform interpolation processing and an interpolation unit that separates electrical signals from the photoelectric conversion unit so that each defective pixel at a specified continuous position is interpolated by different interpolation units. And a synthesizing unit that synthesizes the electrical signals that have been subjected to the interpolation processing by each interpolation unit and reproduces and outputs a series of electrical signals.

  According to the present invention, even when a plurality of defective pixels exist continuously, there is an effect that an appropriate interpolation process can be performed.

FIG. 1 is a cross-sectional view of the MFP according to the first embodiment. FIG. 2 is a cross-sectional view of a reading device provided in the MFP according to the first embodiment. FIG. 3 is a diagram for explaining RTS noise appearing on an image due to defective pixels. FIG. 4 is a schematic diagram illustrating an example of an interpolation method for a plurality of consecutive defective pixels. FIG. 5 is a diagram for explaining a problem that occurs when interpolation processing of defective pixels is performed using surrounding pixels. FIG. 6 is a diagram for explaining a problem in performing defective pixel interpolation processing based on similar patterns. FIG. 7 is a diagram for explaining how abnormal pixels appear due to RTS noise generated in a photoelectric conversion unit that performs driving in column units. FIG. 8 is a diagram for explaining how abnormal pixels appear due to consecutive defective pixels in the area sensor. FIG. 9 is a diagram for explaining how abnormal pixels appear due to consecutive defective pixels in a linear sensor. FIG. 10 is a diagram illustrating a configuration of a main part of the MFP according to the first embodiment. FIG. 11 is a schematic diagram illustrating a flow of interpolation processing operations for consecutive defective pixels in the MFP according to the first embodiment. FIG. 12 shows a state in which the influence of abnormal pixels appearing on an image is improved by performing interpolation processing of consecutive defective pixels on the area sensor by the interpolation processing applied to the MFP of the first embodiment. FIG. FIG. 13 is a diagram illustrating a state in which the influence of abnormal pixels appearing on an image is improved by performing interpolation processing of consecutive defective pixels on the linear sensor in the MFP according to the first embodiment. FIG. 14 is a diagram for explaining the column configuration of the photoelectric conversion unit provided in the MFP according to the second embodiment. FIG. 15 is a partially enlarged view of the photoelectric conversion unit provided in the MFP according to the third embodiment. FIG. 16 is a diagram illustrating an example in which both defective pixels that cause RTS noise and defective pixels that are white defects exist on the photoelectric conversion unit of the MFP according to the fourth embodiment. . FIG. 17 is a diagram illustrating a state in which an interpolation process is performed by separately classifying defective pixels that cause RTS noise and defective pixels that are white defects in the MFP according to the fourth embodiment. FIG. 18 is a diagram for explaining how defective pixels appear in an image when there is contrast in peripheral pixels. FIG. 19 is a diagram for explaining a problem when an interpolation method for performing an interpolation process when the noise amount of the target pixel is higher than the shot noise amount of the peripheral pixel average value is applied to the linear sensor. FIG. 20 is a block diagram of a main part of the MFP according to the fifth embodiment. FIG. 21 is a diagram for explaining a calculation operation performed by the interpolation determination unit in the interpolation determination process. FIG. 22 is a diagram for explaining an image pattern in which it is erroneously determined that interpolation processing is unnecessary even though continuous defective pixel interpolation processing is necessary. FIG. 23 is a diagram for explaining an image pattern that is erroneously determined that the interpolation processing is unnecessary even though the interpolation processing of the continuous defective pixels is unnecessary. FIG. 24 is a diagram for explaining an interpolation processing operation of defective pixels that are continuous for two or more pixels in the MFP according to the fifth embodiment. FIG. 25 is a block diagram of an interpolation processing unit provided in the MFP of each embodiment. FIG. 26 is a diagram for explaining the operation of calculating the SAD value while moving the template on the target pattern. FIG. 27 is a flowchart showing the flow of the interpolation processing operation in the interpolation processing unit. FIG. 28 is a block diagram of a main part of the MFP according to the sixth embodiment. FIG. 29 is a schematic diagram for explaining the flow of the interpolation processing operation of the MFP according to the sixth embodiment in the case where there are a plurality of consecutive defective pixels. FIG. 30 is a schematic diagram for explaining the flow of the interpolation processing operation of the MFP according to the sixth embodiment when the defective pixel is a single pixel. FIG. 31 is a schematic diagram for explaining the flow of the interpolation processing operation in the MFP according to the seventh embodiment. FIG. 32 shows that the entire image information is separated into a plurality of systems so that each defective pixel is included in a different system based on a plurality of continuous defective pixels, and defective pixel interpolation processing is performed for each system. It is a figure for demonstrating the problem which arises. FIG. 33 is a schematic diagram for explaining the flow of the interpolation processing operation in the MFP according to the eighth embodiment. FIG. 34 is a schematic diagram for explaining the flow of the interpolation processing operation in the MFP according to the ninth embodiment.

  First, the field of application will be described. The photoelectric conversion device and the photoelectric conversion method can be applied to devices that perform predetermined information processing by sensing the presence or absence of light, in addition to devices that read images. Specifically, a photoelectric conversion device and a photoelectric conversion method are provided on a multifunction peripheral (MFP) linear sensor, an autofocus line sensor of a camera device or a video camera device, and an interactive whiteboard device (electronic blackboard). The present invention can be applied to a line sensor or the like that reads characters, symbols, or figures written on the. Hereinafter, as an example, an MFP to which a photoelectric conversion device and a photoelectric conversion method are applied will be described.

(First embodiment)
(MFP configuration)
First, FIG. 1 shows a diagram of the MFP according to the first embodiment viewed from the side. FIG. 1 shows a state in which the main body of the MFP is seen through. As shown in FIG. 1, the MFP includes a reading device 1 and a main body 2. The reading device 1 includes an automatic document feeder (ADF) 3 and a scanner mechanism 4.

  In the main body 2, a tandem type image forming unit 5, a registration roller 7 that supplies recording paper from the paper supply unit 13 to the image forming unit 5 through a conveyance path 6, an optical writing device 8, a fixing conveyance unit 9, and The double-sided tray 10 is provided. The image forming unit 5 includes four photosensitive drums 11 corresponding to four colors of yellow (Y), magenta (M), cyan (C), and black (K). Around each photosensitive drum 11, image forming elements including a charger, a developing device 12, a transfer device, a cleaner, and a static eliminator are arranged. Further, an intermediate transfer belt 14 is provided between the transfer unit and the photosensitive drum 11 so as to be stretched between the driving roller and the driven roller while being sandwiched between the nips thereof.

  In the tandem-type image forming apparatus configured as described above, optical writing is performed on the photosensitive drum 11 corresponding to each color of YMCK, and development is performed for each color toner by the developing unit 12, for example, Y, M, C, K. In this order, primary transfer is performed on the intermediate transfer belt 14. Then, a full-color image in which four colors are superimposed by primary transfer is secondarily transferred onto a recording sheet, and then fixed and discharged. Thereby, a full-color image is formed on the recording paper.

(Configuration of ADF and scanner mechanism)
FIG. 2 is a cross-sectional view of the ADF 3 and the scanner mechanism 4. The scanner mechanism 4 includes a contact glass 15 on which an original is placed. The scanner mechanism 4 includes a first carriage 18 having a light source 16 for document exposure and a first reflection mirror 17, and a second carriage 24 having a second reflection mirror 19 and a third reflection mirror 20. Yes. Further, the scanner mechanism 4 includes a lens unit 22 for forming an image of the light reflected by the third reflecting mirror 20 on the light receiving region of the photoelectric conversion unit 21. The scanner mechanism 4 includes a reference white plate 23 for correcting various distortions by a reading optical system and the like, and a sheet-through reading slit 24. The scanner mechanism 4 receives reflected light from a document illuminated with light emitted from the light source 16 by the photoelectric conversion unit 21, converts the light into an electrical signal (image data), and outputs the electrical signal (image data).

  The ADF 3 is connected to the main body 2 via a hinge member (not shown) so that the ADF 3 can be opened and closed with respect to the contact glass 15. The ADF 3 includes a document tray 28 on which a document bundle 27 composed of a plurality of documents can be placed. The ADF 3 also includes a separation feeding unit including a feeding roller 29 that separates documents one by one from the document bundle 27 placed on the document tray 28 and automatically feeds them toward the sheet-through reading slit 25. It also has.

(Original scanning operation)
Such a reading apparatus 1 has a scan mode for reading a document placed on the contact glass 15 and a sheet-through mode for reading a document automatically fed by the ADF 3. Before reading the image in the scan mode or the sheet through mode, the reference white plate 23 is illuminated by the light source 16 that is turned on, and the image by the reflected light is read by the photoelectric conversion unit 21. Then, shading correction data is generated and stored so that each pixel of the image data for one line has a uniform level. The stored shading correction data is used for shading correction of image data read in a scan mode or a sheet through mode described below.

  In the scan mode, the first carriage 18 and the second carriage 24 are moved in the arrow A direction (sub-scanning direction) by a stepping motor (not shown) to scan the document. At this time, the second carriage 24 moves at a speed half that of the first carriage 18 in order to keep the optical path length from the contact glass 15 to the light receiving region of the photoelectric conversion unit 21 constant.

  At the same time, the image surface, which is the lower surface of the document placed on the contact glass 15, is illuminated (exposed) by the light source 16 of the first carriage 18. Then, the reflected light from the image plane is sequentially reflected by the first reflecting mirror 17 of the first carriage 18, the second reflecting mirror 19 of the second carriage 24, and the third reflecting mirror 20. Then, the light flux reflected by the third reflection mirror 20 is focused by the lens unit 22 and imaged on the light receiving region of the photoelectric conversion unit 21. The photoelectric conversion unit 21 photoelectrically converts light received for each line to generate image data. The image data is digitized, gain-adjusted, and output. The document whose image has been read is discharged to a discharge port (not shown).

  In the sheet through mode, the first carriage 18 and the second carriage 24 move to the lower side of the sheet through reading slit 25 and stop. Thereafter, the lowermost originals of the original bundle 27 placed on the original tray 28 of the ADF 3 are sequentially automatically fed in the arrow B direction (sub-scanning direction) by the feeding roller 29, and the sheet through reading slit 25 When the document passes through the position, the document is scanned.

  At this time, the lower surface (image surface) of the automatically fed document is illuminated by the light source 16 of the first carriage 18. Then, the reflected light from the image plane is sequentially reflected by the first reflecting mirror 17 of the first carriage 18, the second reflecting mirror 19 of the second carriage 24, and the third reflecting mirror 20. Then, the light flux reflected by the third reflection mirror 20 is focused by the lens unit 22 and imaged on the light receiving region of the photoelectric conversion unit 21. The photoelectric conversion unit 21 photoelectrically converts light received for each line to generate image data. The image data is digitized, gain-adjusted, and output. The document whose image has been read is discharged to a discharge port (not shown).

(RTS noise due to defective pixels)
Here, RTS noise generated when a defective pixel is present in the photoelectric conversion unit 21 will be described. As an example, the photoelectric conversion unit 21 is formed of a semiconductor imaging device such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. In such a photoelectric conversion unit 21, when one of the carriers moving in the channel of a MOS (Metal Oxide Semiconductor) transistor is trapped by a trap level existing in a gate insulating film or the like, the output level greatly fluctuates, and the image It is known that random telegraph signal noise (hereinafter referred to as RTS noise) occurs.

  In FIG. 3A, a distribution peak appears small at a level that is a fixed amount away from the average value of the pixel values in the positive and negative directions due to fluctuations in output level. And the above-mentioned fixed quantity becomes large, so that RTS noise amount ((sigma) rts) is large.

  In the case of an area sensor, the generated RTS noise appears in a dot shape with respect to each frame image. The RTS noise that appears has a small level and changes randomly for each frame. For this reason, the RTS noise generated by the area sensor is not so noticeable on the image.

  On the other hand, in the case of a linear sensor having photoelectric conversion elements for one row (or a plurality of rows), as shown in FIG. 3B, when pixels that generate RTS noise exist, Level changes appear on each frame image. In addition, even if the RTS noise (σrts) is within the normal noise distribution range (σsht) shown in FIG. 3A, changes due to the RTS noise are concentrated on the same pixel. Appears as vertical lines on a still image.

  Such RTS noise is generated by a single color (individually generated by each color MOS transistor) due to the configuration of the MOS transistor for each pixel of the photoelectric conversion unit 21. Alternatively, RTS noise is generated at the same position in all color channels (generated by MOS transistors common to all colors due to a column configuration or the like).

(Reference example of interpolation method for multiple consecutive defective pixels)
FIG. 4 is a schematic diagram illustrating an example of an interpolation method for a plurality of consecutive defective pixels. In the case of the interpolation method shown in FIG. 4, in the area sensor, when there are a plurality of continuous defective pixels as shown in FIG. 4 (a) in the divided constant region, as shown in FIG. 4 (b). The image information of defective pixels having a small defect degree is not interpolated, and the image information of defective pixels having a large defect degree is interpolated. Then, as shown in FIG. 4C, when defective pixels are detected again and it is determined that the remaining defective pixels are defective pixels again, defective pixel interpolation is performed based on the pixel values of surrounding pixels. Process. Thereby, as shown in FIG.4 (d), the interpolation process of a several continuous defective pixel can be performed.

(Problems when interpolating multiple consecutive defective pixels)
However, in the case of such an interpolation method, the other defective pixel is interpolated in a state where one defective pixel is left among a plurality of consecutive defective pixels. For this reason, as shown in FIG. 5, for example, the number of surrounding pixels that can be used in the linear interpolation process or the replacement process is reduced, and there is a problem that the interpolation accuracy is lowered. Further, as shown in FIG. 6, even when interpolation processing by pattern matching described later is performed, a pattern including a defective pixel in the target pattern may be erroneously detected as a similar pattern.

  FIG. 7 is a diagram illustrating how defective pixels appear due to RTS noise in a CMOS (Complementary Metal Oxide Semiconductor) sensor that performs drive control for each predetermined plurality of pixels (for each column). When the CMOS sensor has a column configuration, RTS noise may be generated in an amplifier circuit or a MOS (Metal Oxide Semiconductor) transistor or the like that is shared by each pixel of one column. In this case, since each pixel of one column appears as a continuous defective pixel, as shown in FIG. 7, the abnormality appears more conspicuous than when one pixel is a defective pixel.

  The example of FIG. 7 is an example in which two consecutive pixels are driven as one column, for example. In this case, when RTS noise is generated in an amplifier circuit or a MOS transistor in the subsequent stage, an abnormality corresponding to a defective pixel for two pixels appears. A rectangular square shown in FIG. 7 indicates a defect for two pixels. . Thus, the CMOS sensor having the column configuration has a great influence on the image by the generated RTS noise. The example in FIG. 7 is an example in which two consecutive pixels are driven as one column. For example, when driving three consecutive pixels as one column, there is an abnormality corresponding to defective pixels for three pixels. appear. That is, an abnormality corresponding to consecutive defective pixels appears by the number of pixels constituting the column.

  In the case of an area sensor, even if a plurality of defective pixels are continuous, abnormalities due to the defective pixels appear in a scattered manner in one image as shown in FIG. On the other hand, in the case of a linear sensor, abnormalities due to defective pixels appear continuously along the sub-scanning direction, and thus appear as a vertical line abnormality as shown in FIG. For this reason, when the interpolation process of the defective pixel in a linear sensor is not performed appropriately, there exists a problem which appears as a bigger abnormality than an area sensor.

(Configuration of main part of the first embodiment)
FIG. 10 shows a configuration of a main part of the MFP 1 according to the first embodiment. As shown in FIG. 10, the MFP 1 includes a photoelectric conversion unit 51, an abnormal pixel detection unit 52 (an example of a detection unit), a storage unit 53, a separation unit 54, an interpolation unit 55, a synthesis unit 56, and a determination unit 57 (specification unit). Example). The abnormal pixel detection unit 52 detects defective pixels present in the image. The determination unit 57 determines whether two or more defective pixels are continuous. The storage unit 53 stores an image signal of each pixel and address information of the pixel determined as a defective pixel. The separation unit 54 separates the image signal into a plurality of systems. The interpolation unit 55 performs interpolation processing on image signals corresponding to defective pixels in each system. The synthesizer 56 synthesizes (combines) the image signals separated into the respective systems and outputs them.

  The photoelectric conversion unit 51 shown in FIG. 10 corresponds to the photoelectric conversion unit 21 shown in FIG. In addition, hereinafter, the photoelectric conversion unit 51 to the determination unit 57 are described as being configured by hardware, but some of them may be configured by software.

  For example, when the abnormal pixel detection unit 52 and the separation unit 54 to the determination unit 57 are configured by software, an image processing program for realizing them is stored in a storage unit such as a hard disk drive or a flash memory of the MFP 1. Then, the central processing unit (CPU) of the MFP 1 executes the image processing program to realize the abnormal pixel detection unit 52 and the separation unit 54 to the determination unit 57.

  The image processing program may be provided by being recorded on a recording medium readable by a computer device such as a CD-ROM or a flexible disk (FD) as a file in an installable or executable format. Further, the program may be provided by being recorded on a recording medium readable by a computer device such as a CD-R, a DVD (Digital Versatile Disk), a Blu-ray Disc (registered trademark), or a semiconductor memory. The image processing program may be provided by being installed via a network such as the Internet. The image processing program may be provided by being incorporated in advance in a ROM or the like in the device.

(Interpolation processing of consecutive defective pixels)
When there are two or more consecutive defective pixels, the MFP 1 uses the photoelectric conversion unit 51 to the determination unit 57 to perform interpolation processing of the consecutive defective pixels. FIG. 11 is a schematic diagram showing the flow of this interpolation processing. The abnormal pixel detection unit 52 detects defective pixels from the image signal supplied from the photoelectric conversion unit 51 as shown in FIG. 11A and supplies address information of the detected defective pixels to the storage unit 53. The example of FIG. 11A shows three rows of image signals generated by the photoelectric conversion elements 51 configured by arranging the first to nth pixels in parallel (one-dimensionally). Of these, the two hatched pixels in the middle row indicate defective pixels.

  The abnormal pixel detection unit 52 stores such defective pixel address information in the storage unit 53. The storage unit 53 is, for example, a hard disk drive or a RAM (Random Access Memory) in the MFP 1. The storage unit 53 stores all image signals including defective pixels.

  The determination unit 57 determines whether or not the defective pixels are continuous based on the address information of the defective pixels stored in the storage unit. When the address information of each defective pixel indicates an adjacent address, the determination unit 57 supplies determination information indicating that each defective pixel is continuous to the separation unit 54 together with the address information of each defective pixel. Further, when the address information of each defective pixel indicates an address separated by one or more pixels, the determination unit 57 indicates that each defective pixel is a defective pixel that occurs only once. The determination information is supplied to the separation unit 54 together with the address information of each defective pixel.

  Next, in the example of FIG. 11A, two defective pixels are continuous. For this reason, the separation unit 54 displays all image signals stored in the storage unit 53 based on the defective pixel address information notified from the determination unit 57 so that the two defective pixels are separated. 11 (b) and FIG. 11 (c), the image signal system of odd pixels (1, 3, 5...) And the image signal of even pixels (2, 4, 6...). Separate into lines.

  When N defective pixels are consecutive, the separation unit 54 converts all image signals stored in the storage unit 53 into N systems so that the N defective pixels are in different systems. To separate.

  Next, each interpolation unit 55 to which the image signals of each system are supplied performs defective pixel interpolation processing for each system. In the case of the example in FIG. 11, since each defective pixel is separated into two systems, the two interpolation units 55 each perform defective pixel interpolation processing as shown in FIGS. 11 (d) and 11 (e). . The synthesizing unit 56 synthesizes the image signals in which the defective pixels are interpolated by the interpolating units 55 as shown in FIG. 11F, and reproduces and outputs the original series of image signals.

  Thereby, since there is no continuous defective pixel in each system, it is possible to prevent the inconvenience of performing the interpolation process using the defective pixel during the interpolation process. For this reason, even when there are two or more consecutive defective pixels, the defective pixels can be appropriately interpolated, and deterioration in image quality can be prevented.

  Specifically, FIG. 12 is a diagram illustrating a result of interpolation processing of consecutive defective pixels in the area sensor. Among these, FIG. 12A shows an image before the interpolation processing, and FIG. 12B shows an image after the interpolation processing. As can be seen by comparing FIG. 12A and FIG. 12B, the area sensor is formed on the image due to the influence of successive defective pixels by the interpolation process performed by separating the defective pixels described above. It can be seen that the rectangular defect portion is appropriately interpolated.

  Similarly, FIG. 13 is a diagram illustrating a result of interpolation processing of consecutive defective pixels in the linear sensor. Among these, FIG. 13A shows an image before the interpolation processing, and FIG. 13B shows an image after the interpolation processing. As can be seen by comparing FIG. 13 (a) and FIG. 13 (b), in the linear sensor, the interpolation processing performed by separating the above-described defective pixels continuously in a line on the image due to the influence of the continuous defective pixels. It can be seen that the rectangular defect portion that appeared as a result is appropriately interpolated.

(Second Embodiment)
Next, the MFP 1 according to the second embodiment will be described. FIG. 14 is a partially enlarged view of the photoelectric conversion unit 51 provided in the MFP 1 according to the second embodiment. The photoelectric conversion unit 51 is formed by a CMOS linear sensor, and a high-speed drive is realized by adopting a column configuration in which a post-stage amplifier circuit and a MOS transistor MOSTR are shared for each column, which is a group of a plurality of pixels. Can do.

  However, if there are two or more consecutive defective pixels in at least one of the RGB pixels in the column shown in FIG. 14, it is highly likely that all the pixels in the column are defective pixels. . For this reason, the interpolation unit 55 shown in FIG. 10 interpolates all the pixels in the column where the defective pixel exists when interpolating each defective pixel after separation. Thereby, the influence of the defective pixel can be further suppressed, and the same effect as in the first embodiment described above can be obtained.

  The second embodiment is different from the first embodiment described above only in this point. For the explanation and effects of other parts, refer to the explanation of the first embodiment described above. Further, in FIG. 14, as an example, a configuration in which a total of 9 pixels of 3 rows × 3 columns is formed as one column is shown, but even when the number of pixels constituting the column and the combination are different, as described above, The same effect can be obtained by performing an interpolation process using pixels.

(Third embodiment)
Next, the MFP 1 according to the third embodiment will be described. FIG. 15 is a partially enlarged view of the photoelectric conversion unit 51 provided in the MFP 1 according to the third embodiment. FIG. 15 shows an example in which each column of RGB is composed of a total of 6 pixels of 3 rows × 2 columns, and one column. In this way, when pixels of each color of RGB are included in one column, if correction is performed for each color, the degree of correction may change for each color. In this case, the color is different from that of the surrounding pixels, and inconvenient results in conspicuous abnormalities of defective pixels.

  Therefore, in the MFP 1 according to the third embodiment, when there are two or more defective pixels continuously in at least one of the colors including the RGB colors, the interpolation unit 55 illustrated in FIG. When interpolating each defective pixel, the same interpolation processing is performed for the pixels of other colors having addresses corresponding to the addresses of the defective pixels (physical positions on the photoelectric conversion unit 51) in the column. Do. That is, when the pixel at the third position (address) of G is a defective pixel, the interpolation unit 55 performs the interpolation process on each pixel at the third position of R and the third position of B. Apply. As a result, it is possible to prevent inconvenience that the pixel after the interpolation processing is different in color from the surrounding pixels and conspicuously abnormal defective pixels.

  The third embodiment is different from the above-described embodiments only in this point. For the description and effects of other parts, refer to the description of each embodiment described above. Further, in FIG. 15, as an example, a configuration in which a total of 6 pixels of 3 rows × 2 columns is formed as one column is shown. However, even when the number of pixels constituting the column and the combination are different, The same effect can be obtained by performing the same interpolation process on the pixels of other colors having addresses corresponding to the addresses of defective pixels of that color.

(Fourth embodiment)
Next, the MFP 1 according to the fourth embodiment will be described. For example, in the photoelectric conversion unit 51 which is a CMOS image sensor, there is a cause of pixel abnormality called “white scratch” in addition to “RTS noise”. A white defect is a defect that can cause an image to be affected by an offset larger than that of a normal pixel due to an abnormal dark current generated by a defect on a photodiode in a CMOS image sensor. When this white scratch is generated in the linear sensor, the offset is increased in all lines as in the case of the RTS noise, so that the influence of defective pixels in the form of vertical lines appears on the image.

  Since “RTS noise” and “white scratches” have different appearances of abnormality, they are detected by different methods, and the presence / absence of interpolation processing is determined by each determination threshold value. For this reason, even when a defective pixel due to white flaws and a defective pixel due to RTS noise exist adjacent to each other, each is determined as a defective pixel separately.

  However, if the “RTS noise” defective pixel and the “white scratch” defective pixel are separately interpolated, the “white scratch” defective pixel is used for the interpolation processing of the “RTS noise” defective pixel, and the interpolation processing is performed. On the contrary, the interpolation processing is performed by using the defective pixel of “RTS noise” for the interpolation processing of the defective pixel of “white scratch”, and there is a possibility that the interpolation accuracy may be lowered. In addition, since the interpolation process is selectively used for each type of defective pixel, there is a problem that the interpolation process becomes complicated.

  For this reason, the MFP 1 according to the fourth embodiment stores the address information on the photoelectric conversion unit 51 of the defective pixel of “RTS noise” and the photoelectric of the defective pixel of “white scratch” in the storage unit 53 illustrated in FIG. Address information on the conversion unit 51 is stored. Specifically, a pixel indicated by a right oblique line block illustrated in FIG. 16 indicates a defective pixel of “RTS noise”, and a pixel indicated by a left oblique line block indicates a “white scratch” defective pixel. The abnormal pixel detection unit 52 controls the storage unit 53 to store such defective pixel address information.

  The determination unit 57 determines whether the address information of the defective pixels is continuous based on the address information of each defective pixel stored in the storage unit 53. Specifically, as shown in FIG. 17, the address information “4”, “6”... Of “RTS noise” defective pixels and the address information “defective white” are stored in the storage unit 53. “2”, “7”, “10”... Are stored. The determination unit 53 compares the address information and determines whether the address information is continuous. In the case of the example of FIG. 17, the address information “6” of the defective pixel “RTS noise” and the address information “7” of the defective pixel “white scratch” are continuous. When the determination unit 57 detects continuous address information, it adds “1” flag information indicating continuous address information to the continuous address information as shown in FIG.

  Based on the flag information, the separation unit 54 illustrated in FIG. 10 separates defective pixels having consecutive addresses so as to have different systems as described above. The interpolation unit 55 performs interpolation processing on each system of the separated defective pixels. As a result, even when two or more defective pixels having different generation factors are consecutive, each defective pixel can be interpolated in different systems, so that appropriate interpolation processing can be performed. In addition, since the interpolation processing can be performed using the same one type of interpolation processing instead of using the interpolation processing for each occurrence factor, the inconvenience of complicating the interpolation processing can be prevented, and each of the above-described implementations can be performed. The same effect as the form can be obtained.

  The fourth embodiment is different from the above-described embodiments only in this point. For the description and effects of other parts, refer to the description of each embodiment described above.

(Fifth embodiment)
(Influence of defective pixels when surrounding pixels have contrast)
Here, FIG. 18A shows an example of an image in which the influence of the defective pixel (RTS noise) appears when one or a plurality of pixels (surrounding pixels) adjacent to the left and right of the defective pixel have contrast. . FIG. 18A shows an example in which a defective pixel exists in a pixel forming a central vertical line among vertical lines having a thickness of three pixels. Further, FIG. 18A shows an example in which the density of the vertical lines formed by the left and right pixels of the defective pixel is not uniform, and there is a contrast (contrast) between the vertical lines. Further, as a comparative example, FIG. 18B shows an image of a vertical line having a thickness of three pixels drawn by normal pixels.

  When a dark vertical line is drawn in a pixel adjacent to the defective pixel, the influence of the defective pixel is less likely to appear as shown in FIG. 18A, and the vertical line drawn by the defective pixel is recognized as a thin line. It becomes possible. Thus, even when a defective pixel exists, the influence of the defective pixel may not be noticeable depending on the pattern of the surrounding pixels of the defective pixel. The above-described example is an example in which contrast exists between three vertical lines drawn by the defective pixel and the left and right pixels. For this reason, when three vertical lines drawn by the defective pixel and the left and right pixels are substantially uniform, there is a concern that the influence of the defective pixel appears.

  However, when the color of the image to be drawn is white or a color close to white, the influence of the defective pixel is difficult to appear. This is because the RTS noise is a noise generated due to an electrical defect inside the defective pixel. Therefore, the influence of the defective pixel is that the output level of the image data output from the surrounding pixels of the defective pixel is not less than a predetermined level. This is because, if the level is reached, it is reduced by shot noise as shown in FIG.

(Problems when interpolation processing is applied to linear sensors)
Next, an interpolation method for performing interpolation processing when the noise amount of the target pixel is higher than the shot noise amount of the peripheral pixel average value in a linear sensor that performs one-dimensional photoelectric conversion processing for one column (or a plurality of columns) The problem when applying is explained.

  In the case of this interpolation method, when the shot noise amount σsht converted from the average level Dave of the surrounding pixels and the noise amount σrts of the defective pixels are large (σrts> σsht to be interpolated) and the level difference from the surrounding pixels is large. Perform interpolation judgment. For this reason, in the case of the defective pixel described with reference to FIG. 3A, “σrts <σsht”, which is not a correction target. That is, when the above-described interpolation method is applied to a linear sensor, interpolation processing is not performed on the defective pixel described with reference to FIG. 3A, and vertical stripes to be corrected are present as shown in FIG. This causes an uncorrected problem.

  Further, if σrts <σsht, the contrast described with reference to FIG. 18A exists between the vertical lines even when the vertical stripe interpolation processing shown in FIG. 19A is performed. Among the three vertical lines shown in FIG. 18A, the pixel corresponding to the central vertical line is set as an interpolation target. The level of the interpolation pixel is replaced with the average value or the median value of the surrounding pixels. For this reason, the level of the pixel corresponding to the central vertical line is replaced with the average value or the median value of the surrounding pixels, and an image having a white line at the center of the three vertical lines is obtained as shown in FIG. Cause inconvenience. This means that the defective pixel interpolation process has made the defective pixel portion stand out.

  In the MFP 1 according to the fifth embodiment, the interpolation process is performed only in the portion where the influence of the defective pixel is visible. FIG. 20 is a block diagram of a main part of the MFP 1 according to the fifth embodiment. As shown in FIG. 20, in the case of the MFP 1 of the fifth embodiment, an interpolation determination unit 61 that determines whether or not to perform interpolation processing is provided between the abnormal pixel detection unit 52 and the storage unit 53. doing. The interpolation determination unit 61 determines the presence / absence of the interpolation processing for each line of the image based on whether or not the periphery of each defective pixel is a high-density area.

(Calculation operation performed by judgment of interpolation processing)
FIG. 21 is a diagram for explaining a calculation operation performed by the interpolation determination unit 61 in the interpolation determination process. The interpolation determination unit 61 determines whether or not the surrounding pixels of each defective pixel correspond to a solid area with high density for each line of the image. The example of FIG. 21 shows an example in which it is determined whether or not the five pixels on the left and right of the defective pixel correspond to solid areas with high density as surrounding pixels. In this example, the left and right 5 pixels of the defective pixel are described as the calculation target pixels, but the left and right 2 pixels, the left and right 3 pixels, and the left and right 8 pixels of the defective pixel may be used.

  The interpolation determining unit 61 calculates a difference value ΔL between the maximum value and the minimum value of image data formed by five pixels adjacent to the left side (L: Left) of the defective pixel for each RGB color (ΔRL, ΔGL, ΔBL). Further, the interpolation determining unit 61 calculates, for each color, a difference value ΔL between the maximum value and the minimum value of image data formed by five pixels adjacent to the right side (R: Right) of the defective pixel (ΔRR, ΔGR, ΔBR). ).

  The interpolation determining unit 61 calculates the level change degree of the surrounding pixels of the defective pixel by calculating the difference value. The larger the calculated difference value, the larger the level change of the image data. When the calculated difference value is large, the interpolation determination unit 61 determines that the level change of the image data is large, the contrast is present in the surrounding pixels of the defective pixel, and an image with a nonuniform pattern is formed.

  In addition, the interpolation determination unit 61 calculates an average value Level of image data formed by five pixels adjacent to the left side (L) of the defective pixel for each color of RGB (RLave, GLave, and BLave). In addition, the interpolation determination unit 61 calculates an average value Rave of image data formed by five pixels adjacent to the right side (R) of the defective pixel for each RGB color (RRave, GRave, BRave). This average value indicates the density of surrounding pixels. The interpolation determination unit 61 determines that the density is lower (white side) as the average value is larger.

(Interpolation judgment operation)
The interpolation determination unit 61 uses the difference value and the average value of the surrounding pixels to accurately grasp the degree of influence on the defective pixel and determine the presence / absence of interpolation processing, as will be described below. That is, the interpolation determination unit 61 calculates the difference value between the maximum value and the minimum value of the surrounding pixels of the defective pixel and the average value of the surrounding pixels, and whether or not the difference value is equal to or less than a predetermined first threshold value. Is determined.

  When the difference value is larger than the first threshold value, the interpolation determining unit 61 determines that the image area around the defective pixel is an image area having a contrast such as a pattern or pattern area. As described above with reference to FIG. 18A, the influence of defective pixels due to pixel abnormality is less likely to appear in an image area with contrast. Further, the difference value being larger than the first threshold means that the image formed by the surrounding pixels has a low density (white side = white or a color close to white), and the influence of the defective pixel is difficult to appear. means. For this reason, when the difference value is larger than the first threshold value, the interpolation determination unit 61 determines that the interpolation process is not performed, and stores information indicating the determination result in the storage unit 53.

  On the other hand, when the difference value is equal to or smaller than the first threshold value, the interpolation determining unit 61 determines whether or not the average value is equal to or smaller than the predetermined second threshold value. As described with reference to FIG. 18C, when the image data of the surrounding pixels becomes larger than a specific level (second threshold value), the influence of noise caused by the electrical inside of the pixel is caused by shot noise. It is buried and becomes difficult to appear on the image. On the other hand, when the image data of surrounding pixels is below a specific level, the influence of noise appears on the image without being buried in shot noise.

  For this reason, when the average value is equal to or smaller than the second threshold value, the interpolation determination unit 61 determines that the interpolation process is performed because the influence of noise is not buried in the shot noise, and stores information indicating the determination result. Store in the unit 53. On the other hand, when the average value is larger than the second threshold value, the interpolation determination unit 61 determines that the interpolation process is not performed because the influence of noise is buried in the shot noise, and stores information indicating the determination result. Store in the unit 53.

  The separation unit 54 separates consecutive defective pixels into different systems from the defective pixels determined to be “perform interpolation processing” by the interpolation determination unit 61. Each interpolation unit 55 interpolates the defective pixels separated from each system. As a result, the degree of influence on the image when the defective pixel is interpolated can be accurately grasped, and the disadvantage that the image is adversely affected by the interpolation processing can be prevented, and the defective pixel can be accurately interpolated. In addition, the same effects as those of the above-described embodiments can be obtained.

(Determination of necessity of interpolation processing before separation)
Here, when there are two or more defective pixels in succession, separation is performed so that each defective pixel is interpolated in different systems. However, when the necessity of interpolation processing is determined after the separation, the resolution decreases. Whether or not interpolation processing is necessary is determined in the state. For example, when continuous defective pixels are separated into two systems, as shown in FIG. 22 (a), the defective pixels are present in a high-density solid region, so that the defective pixels are conspicuous. Because of the contrast, as shown in FIGS. 22 (b) and 22 (c), there arises a problem in that it is erroneously determined that no interpolation is necessary in the interpolation determination after separation.

  In addition, as shown in FIG. 23A, in the interpolation determination after separation, although the defective pixel exists but is in a region with contrast, the defective pixel is inconspicuous. As shown in FIGS. 23 (b) and 23 (c), the high density and no contrast cause the inconvenience that the interpolation process is erroneously determined to be necessary. When such a misjudgment occurs, as a result of performing the interpolation process, the defective pixel becomes conspicuous on the image, or the interpolation process is not executed even though the defective pixel is conspicuous on the image. Arise.

  For this reason, in the MFP 1 according to the fifth embodiment, the necessity of interpolation processing is determined before separating defective pixels that are continuous for two or more pixels. That is, as shown in FIG. 20, the interpolation determination unit 61 determines whether or not the interpolation processing is necessary before the separation unit 54. Thus, after determining that the interpolation processing is to be executed because the image is a high-density solid image as shown in FIG. 24A, each defective pixel is replaced with each other as shown in FIGS. 24B and 24C. Can be separated. Also, as shown in FIG. 24D, when it is determined that the interpolation process is unnecessary due to the presence of contrast, the interpolation process can be prevented from being executed.

(Interpolation processing operation)
Next, an interpolation processing operation performed by the interpolation unit 55 of the MFP 1 according to the fifth embodiment will be described. FIG. 25 is a detailed block diagram of the interpolation unit 55 of the MFP 1 according to the fifth embodiment. As shown in FIG. 25, the interpolation unit 55 has a correlation degree calculation unit 67 and a replacement processing unit 68 along with the memory 66.

  As an example, as shown in FIG. 26, the interpolating unit 55 uses the 16 pixels before and after the defective pixel determined to be interpolated by the interpolation determining unit 61 and the 3 lines before and after the pixel to be an interpolation candidate as a search region (target pattern). Explore. In addition, the interpolation unit 55 sets a region corresponding to three lines formed by a defective pixel, two pixels before and after the defective pixel, and two pixels above and below the two pixels before and after the defective pixel as templates. Then, as shown in FIG. 26, the interpolation unit 55 compares the above-mentioned target pattern (image data of 16 pixels before and after the defective pixel and 3 lines before and after) while moving the template pixel by pixel, Calculate the correlation of the template.

  For example, a SAD (Sum of Absolute Difference) value is used as the degree of correlation. For example, in the case of the example shown in FIG. 26, the interpolation unit 55 calculates 24 SAD values based on the following formula (for example, when obtaining SAD values S1 to S24). Then, the interpolation unit 55 replaces the defective pixel with the center pixel of the portion of the target pattern for which the smallest SAD value (= maximum correlation) is calculated.

  In the following formula and FIG. 26, “S” means an SAD value. “S1” indicates the first SAD value, and S24 indicates the 24th SAD value. “P” means a pixel of the target pattern. “P11” means the first pixel of the first line of the target pattern, and “P321” means the 32nd pixel of the first line of the target pattern. Similarly, “P13” means the first pixel on the third line of the target pattern, and “P323” means the 32nd pixel on the third line of the target pattern. “T” means a pixel of the template. “T11” means the first pixel on the first line of the template, and “T41” means the fourth pixel on the first line of the template. Similarly, “P13” means the first pixel on the third line of the target pattern, and “P43” means the fourth pixel on the third line of the target pattern.

  S1 = (P11−T11) + (P12−T12) + (P13−T13) + (P21−T21) + (P22−T22) + (P23−T23) + (P31−T31) + (P32−T32) + (P33−T33)... S24 = (P281−T11) + (P282−T12) + (P283−T13) + (P291−T21) + (P292−T22) + (P293−T23) + (P301−T31) ) + (P302-T32) + (P303-T33)

  The interpolation unit 55 calculates SAD values S <b> 1 to S <b> 24 by performing arithmetic processing based on such a mathematical expression. For example, when the value of the SAD value S22 is the minimum value, the pixel of P282 that is the central pixel of the portion of the target pattern for which the minimum value is calculated is used as a replacement pixel and replaced with a defective pixel.

  FIG. 27 is a flowchart showing the flow of such an interpolation processing operation. In step S21, the correlation degree calculation unit 67 shown in FIG. 25 sets “1” as the SAD value calculation count in the counter (n = 1: n is the calculation count). That is, in the example shown in FIG. 26, the correlation degree calculation unit 67 calculates a total of 24 SAD values. Therefore, in step S21, the correlation degree calculation unit 67 first sets “1”, which indicates the first SAD value calculation operation, to the counter. Each time the calculation of the correlation degree 67 completes the calculation of one SAD value, the value of the counter is incremented one by one, such as “1” → “2” → “3”. Increment. The increment timing is a timing at which the template is moved by one pixel with respect to the target pattern.

  Next, in step S22, the correlation degree calculation unit 67 calculates an SAD value (Sn: S is an SAD value, and n is a number from 1 to 24 of the calculated SAD value) based on the above-described formula. To do. In step S23, the correlation degree calculation unit 67 determines whether or not the number of calculations is 24 (n = 24: n is the number of calculations). As described above, the correlation degree calculation unit 67 performs the above-described calculation while moving the template one by one with respect to the target pattern. For this reason, when the number of calculations is less than 24 (step S23: No), the process proceeds to step S25, the number of calculations is incremented by one (n = n + 1), and the process returns to step S22. In step S22, the SAD value is calculated again.

  When 24 SAD values are calculated in this way (step S23: Yes), the process proceeds to step S24, and the replacement processing unit 68 shown in FIG. 25 calculates the minimum value among the 24 SAD values. The replacement processing (interpolation processing) with the defective pixel is performed using the central pixel of the pattern portion as the replacement pixel, and the processing of the flowchart in FIG.

  The MFP 1 according to the fifth embodiment determines whether or not interpolation processing is necessary before performing separation processing on consecutive defective pixels. When determining whether or not interpolation processing is necessary after separating consecutive defective pixels, determining whether or not interpolation processing is necessary based on image information whose resolution has been reduced due to fewer pixels used for the determination. Therefore, the determination accuracy decreases. However, it is possible to determine the necessity of the interpolation process using many pixels by performing the determination of the necessity of the interpolation process before separating the consecutive defective pixels, preventing erroneous determination, The determination accuracy can be improved.

  The fifth embodiment is different from the above-described embodiments only in this point. For the description and effects of other parts, refer to the description of each embodiment described above.

(Sixth embodiment)
Next, the MFP 1 according to the sixth embodiment will be described. The MFP 1 according to the sixth embodiment includes a duplication unit 72 and a selector 71 together with the photoelectric conversion unit 51 to the determination unit 57, as shown in FIG. When the defective pixel is a single pixel, the duplicating unit 72 creates a copy of the image information that is not subjected to the separation process by the number of the interpolating unit 55. Based on the determination result of the determination unit 57, the selector 71 supplies the image information to the separation unit 54 when the defective pixels are continuous, and copies the image information when the defective pixel is a single pixel. To the unit 72.

  FIG. 29A to FIG. 29E are schematic diagrams illustrating the flow of processing of image information when there are two consecutive defective pixels. When there are two consecutive defective pixels, the separation unit 54 separates the image information of each defective pixel into two systems as shown in FIG. 29A, and FIG. 29C and FIG. As shown in FIG. 4, the data are supplied to the first interpolation unit 55a and the second interpolation unit 55b. The replacement processing unit 68 (an example of a control unit) illustrated in FIG. 25 controls the first interpolation unit 55a and the second interpolation unit 55b so that each defective pixel is subjected to interpolation processing. The image information generated by the interpolation processing is combined by the combining unit 56 and reproduced as a series of image information as shown in FIG. The interpolation process is performed one by one by N interpolation units 55 when the defective pixels are N consecutive pixels.

  On the other hand, FIG. 30A to FIG. 30E are schematic diagrams showing the flow of processing of image information when the defective pixel is a single pixel. As shown in FIG. 30A, when the defective pixel is a single pixel, the image information is supplied to the duplicating unit 72 without being separated. As shown in FIG. 30B, the duplicating unit 72 duplicates and generates image information in a state where it is not separated by the number of pieces of the interpolating unit 55. Then, the duplicating unit 72 supplies the duplicated image information to all the interpolating units 55 as shown in FIGS. 30 (c) and 30 (d). 30C and 30D are examples in which two interpolation units 55 are provided in the MFP 1. In this case, the duplicating unit 72 generates one piece of image information to be duplicated in addition to the original image information, supplies the original image information to the first interpolating unit 55a, and supplies the duplicated image information to the first image information. 2 is supplied to the second interpolation unit 55b.

  The replacement processing unit 68 controls each interpolation unit 55 so as to perform interpolation processing on the defective pixels of single pixels. Then, the replacement processing unit 68 controls the predetermined one interpolation unit 55 to supply the interpolated image information to the synthesis unit 56, and discards the other interpolation unit 55 from the interpolated image information. To control. That is, in the example of FIG. 30, the first interpolation unit 55 a discards the interpolated image information, and the second interpolation unit 55 b supplies the interpolated image information to the synthesis unit 56. The combining unit 56 outputs the image information supplied from one interpolation unit 55 as it is without combining. Thereby, it is possible to prevent inconvenience that an unnecessary image signal combining process is performed in the combining unit 56.

  The MFP 1 according to the sixth embodiment does not need to include two types of circuits, that is, a single defective pixel circuit and two or more consecutive defective pixel circuits. In addition to preventing an increase in cost, the same effects as those of the above-described embodiments can be obtained.

  The sixth embodiment is different from the above-described embodiments only in this point. For the description and effects of other parts, refer to the description of each embodiment described above.

(Seventh embodiment)
Next, the MFP 1 according to the seventh embodiment will be described. In the sixth embodiment described above, when the defective pixel is a single pixel, all image information is supplied to the first and second interpolation units 55a and 55b without separation. Then, after performing interpolation processing in each of the interpolation units 55a and 55b, the image information of the first interpolation unit 55a is discarded, and only the image information of the second interpolation unit 55b is output.

  On the other hand, in the MFP 1 according to the seventh embodiment, even when the defective pixel is a single pixel as shown in FIG. 31A, the separation unit 54 is provided as shown in FIG. Based on the defective pixel, the image information is separated into a plurality of systems. FIG. 31C and FIG. 31D are examples in which image information is separated into two systems based on defective pixels, and image information that does not include defective pixels is supplied to the first interpolation unit 55a. This is an example in which image information including defective pixels is supplied to the second interpolation unit 55b.

  In this case, the replacement processing unit 68 (an example of the control unit) illustrated in FIG. 25 controls the first interpolation unit 55a to which image information not including a defective pixel is supplied so as not to perform the interpolation process. Further, the replacement processing unit 68 controls the second interpolation unit 55b to which the image information including the defective pixel is supplied so as to execute the interpolation processing. Then, as shown in FIG. 31E, the synthesis unit 56 performs image information that has not been subjected to interpolation processing by the first interpolation unit 55a, and image information that has undergone interpolation processing by the second interpolation unit 55b. Are combined and output.

  As a result, the selector 71 and the duplicating unit 72 described in the sixth embodiment can be eliminated, the circuit scale can be reduced and the cost can be reduced, and the same as in each of the above-described embodiments. The effect of can be obtained.

  The seventh embodiment is different from the above-described embodiments only in this point. For the description and effects of other parts, refer to the description of each embodiment described above.

(Eighth embodiment)
Next, the MFP 1 according to the eighth embodiment will be described. As shown in FIG. 32 (a), when interpolation processing is performed on defective pixels that exist continuously for two or more pixels, the image information is separated for each defective pixel as shown in FIGS. 32 (b) and 32 (c). Then, interpolation processing using pattern matching is performed for each system (see FIG. 26). At this time, there is a possibility that a similar pattern (= template shown in FIG. 26) is selected from a different position for each system. Then, when the interpolated image information is synthesized based on the similar patterns selected at different positions for each system, the subtle differences in the similar patterns become abnormal pixels as shown in FIG. There is concern about appearing on the image.

  For this reason, the MFP 1 of the embodiment of FIG. 8 compares similar patterns selected in each system (pattern matching), and the similar patterns used for the interpolation processing in each system are similar patterns each having a high similarity. Adjust so that Specifically, when there are a plurality of consecutive defective pixels as shown in FIG. 33 (a), the separation unit 54, as described above, distributes the entire image so that each defective pixel is distributed to a different system. Isolate information. In the example of FIG. 33A, there are two consecutive defective pixels, and each defective pixel has two systems as shown in FIGS. 33B and 33C. 2 shows a state of being separated into two interpolation units 55b).

  As shown in FIGS. 33B and 33C, the first interpolation unit 55a and the second interpolation unit 55b respectively detect similar patterns used for interpolation processing (see FIG. 26). The replacement processing unit 68 (an example of a control unit) illustrated in FIG. 25 uses the first similar pattern detected by the first interpolation unit 55a and the second similar pattern detected by the second interpolation unit 55b. Comparison processing (pattern matching) is performed. As shown in FIGS. 33 (b) and 33 (c), when the similarity of each similar pattern is low (when the matching rate is low), the replacement processing unit 68 performs each interpolation so as to select a different similar pattern. The units 55a and 55b are controlled, and each similar pattern is compared again. The replacement processing unit 68 repeatedly controls such similar pattern detection processing and comparison processing.

  As a result, as shown in FIG. 33 (d), a similar pattern (third similar pattern) having a high degree of similarity (a high matching rate) is detected in each of the interpolation units 55a and 55b. The The synthesizing unit 56 synthesizes and outputs each piece of image information interpolated with the third similar pattern having a high similarity in the interpolating units 55a and 55b. As a result, as shown in FIG. 33 (e), the occurrence of abnormal images due to subtle differences between similar patterns used in each system can be prevented, and the same effects as those of the above-described embodiments can be obtained. be able to.

  Note that this is the only difference between the eighth embodiment and the above-described embodiments. For the description and effects of other parts, refer to the description of each embodiment described above.

(Ninth embodiment)
Next, the MFP 1 according to the ninth embodiment will be described. In the case of the MFP 1 of the ninth embodiment, as in the eighth embodiment, an example in which an inconvenience that an abnormal image occurs due to a subtle difference between similar patterns used for interpolation processing in each system is prevented. It is.

  In the case of the MFP 1 according to the ninth embodiment, as shown in FIG. 34A, the replacement processing unit 68 (an example of a control unit) uses a plurality of continuous images based on image signals before separation into each system. A reference pattern for performing interpolation processing of defective pixels is generated. When the reference pattern is generated, the separation unit 54 separates the entire image information so as to separate each defective pixel into each system, and the interpolation unit 55 of each system performs the processes shown in FIGS. 34 (b) and 34 (c). ), A similar pattern for interpolating the defective pixels of each system is detected. The example of FIGS. 34B and 34C shows an example in which two consecutive defective pixels are separated into two systems of a first interpolation unit 55a and a second interpolation unit 55b, respectively. .

  Each of the interpolation units 55a and 55b detects a similar pattern for interpolating each defective pixel. FIG. 34B shows an example in which the first interpolation unit 55a detects the first similar pattern for defective pixel interpolation processing. FIG. 34C shows an example in which the second interpolation unit 55b detects the second similar pattern for defective pixel interpolation processing. The replacement processing unit 68 compares the reference pattern generated based on the image signal before separation into each system and the similar pattern detected by the interpolation units 55a and 55b. Then, when the degree of similarity between the reference pattern and the similar pattern is low, the replacement processing unit 68 controls the interpolation unit 55 so as to detect a different similar pattern again.

  The example of FIG. 34B is an example in which the first interpolation unit 55a detects a first similar pattern for defective pixel interpolation processing. The replacement processing unit 68 compares the first similar pattern detected by the first interpolation unit 55a with the reference pattern. In the case of the example in FIG. 34B, the similarity (matching rate) between the two is high. Therefore, the replacement processing unit 68 recognizes that the first interpolation unit 55a uses the first similar pattern for the interpolation processing, and does not perform redetection control of the similar pattern. The first interpolation unit 55 a performs defective pixel interpolation processing using the first similar pattern having a high degree of similarity to the reference pattern, and supplies the interpolated image information to the combining unit 56.

  On the other hand, the example of FIG. 34C is an example in which the second interpolation unit 55b detects the second similar pattern for interpolation processing of defective pixels. The replacement processing unit 68 compares the first similar pattern detected by the second interpolation unit 55b with the reference pattern. In the case of the example in FIG. 34C, the similarity (matching rate) between the two is low. For this reason, the replacement processing unit 68 controls the second interpolation unit 55b so as to re-detect similar patterns other than the second similar pattern.

  As a result, as shown in FIG. 34D, the second interpolation unit 55b detects a third similar pattern different from the previously detected similar pattern. The replacement processing unit 68 compares the third similar pattern detected by the second interpolation unit 55b with the reference pattern. In the example of FIG. 34 (d), the similarity (matching rate) between the two is high. Accordingly, the replacement processing unit 68 recognizes that the second interpolation unit 55b uses the third similar pattern for the interpolation processing, and ends the similar pattern re-detection control. The second interpolation unit 55 b performs defective pixel interpolation processing using the second similar pattern having a high similarity to the reference pattern, and supplies the interpolated image information to the synthesis unit 56.

  As shown in FIG. 34 (e), the synthesizer 56 performs image information on which defective pixel interpolation processing has been performed with the first similar pattern and image on which defective pixel interpolation processing has been performed with the third similar pattern. A series of image data is reproduced and output by combining the information.

  In the MFP 1 according to the ninth embodiment, interpolation processing of defective pixels in image information after separation is performed using a similar pattern having a high degree of similarity with respect to a reference pattern generated based on all image information before separation. be able to. For this reason, it is possible to perform interpolation processing using appropriate similar patterns rather than performing interpolation processing using similar patterns detected based on image information whose resolution has been reduced by separating the image information into each system. Besides, it is possible to prevent the inconvenience that an abnormal image is generated due to a subtle difference between similar patterns used for interpolation processing in each system, and it is possible to obtain the same effects as those of the above-described embodiments.

  Note that the ninth embodiment is different from the above-described embodiments only in this point. For the description and effects of other parts, refer to the description of each embodiment described above.

  Finally, each of the above-described embodiments has been presented as an example, and is not intended to limit the scope of the present invention. Each of the novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. Further, the embodiments and modifications of the embodiments are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 Reading apparatus of MFP (MFP) 2 MFP main body 21 Photoelectric conversion unit 51 Photoelectric conversion unit 52 Abnormal pixel detection unit 53 Storage unit 54 Separation unit 55 Interpolation unit 55a First interpolation unit 55b Second interpolation unit 56 Combining unit 57 Determination Unit 61 Interpolation Determination Unit 67 Correlation Level Calculation Unit 68 Replacement Processing Unit 71 Selector 72 Replication Unit

JP 2009-130553 A

Claims (14)

A photoelectric conversion unit that converts light received by each pixel into an electrical signal;
A detection unit that detects a defective pixel that outputs an electrical signal of an abnormal level among the pixels,
Of each detected defective pixel, a specifying unit that specifies a defective pixel having a continuous physical position on the photoelectric conversion unit;
A plurality of interpolation units for performing an interpolation process using the defective pixel electrical signal as a normal pixel electrical signal;
A separation unit that separates the electrical signal from the photoelectric conversion unit and supplies the signal to the interpolation unit so that each defective pixel at a specified continuous position is interpolated by the different interpolation unit;
And a synthesizing unit that synthesizes the electric signals that have been subjected to the interpolation processing by the interpolating units, and reproduces and outputs a series of the electric signals. The photoelectric conversion device according to claim 1, wherein the photoelectric conversion unit is formed by arranging a plurality of pixels in a one-dimensional manner for each of the three primary colors of light received. The photoelectric conversion unit is driven for each column in which a predetermined number of pixels are grouped,
The photoelectric conversion apparatus according to claim 1, wherein the interpolation unit performs the interpolation processing on all pixels in the column when defective pixels exist in the column. In the interpolation unit, when a pixel of any one of the three primary colors is the defective pixel, the other color pixel at a position corresponding to the position of the defective pixel on the photoelectric conversion unit The photoelectric conversion device according to claim 2, wherein the interpolation processing is performed on the photoelectric conversion device. The detection unit detects the defective pixel for each occurrence factor,
The specifying unit specifies each defective pixel at successive positions for each occurrence factor,
The separation unit separates the electrical signal from the photoelectric conversion unit so that each defective pixel at successive positions specified for each occurrence factor is interpolated by different interpolation units, respectively, It supplies to an interpolation part. The photoelectric conversion apparatus of Claim 1 or Claim 2 characterized by the above-mentioned. Whether or not a normal interpolation processing result is obtained when the defective pixel is interpolated is determined based on pixel values of pixels around the defective pixel, and an electric signal of the defective pixel is set to a normal level electric signal. An interpolation determination unit that performs an interpolation determination process as to whether or not to perform an interpolation process that approximates a signal,
The photoelectric conversion apparatus according to claim 1, wherein the interpolation unit performs pixel interpolation processing on the defective pixel from which a determination result indicating that the interpolation processing is executed is obtained. The photoelectric conversion apparatus according to claim 6, wherein the interpolation determination unit executes the interpolation determination process before separating the electrical signal from the photoelectric conversion unit. The interpolating unit arranges a template having a partial size of the target pattern from an end of the target pattern on a target pattern including a pixel column including the defective pixel and one or a plurality of pixel columns before and after. The degree of correlation between the target pattern and the template is calculated while moving at least one pixel along the direction, and the defective pixel is obtained by using the central pixel of the part of the target pattern for which the maximum degree of correlation is calculated as a replacement pixel. The photoelectric conversion device according to claim 6, wherein the photoelectric conversion device is subjected to an interpolation process. When the defective pixel on the photoelectric conversion unit is a single defective pixel, a duplication unit that generates the number of duplicates of the electric signal that is not separated by the number of the interpolation units, and supplies each of the interpolation units,
When there are a plurality of defective pixels whose positions are continuous on the photoelectric conversion unit, an electrical signal from the photoelectric conversion unit is supplied to the separation unit, and the defective pixel existing on the photoelectric conversion unit In the case of a defective pixel, a selector that supplies an electrical signal from the photoelectric conversion unit to the replication unit;
When the electrical signal separated according to each defective pixel is supplied from the separation unit to the interpolation unit, the interpolation unit is controlled so that the interpolation process is performed for each interpolation unit, When a copy of the electrical signal that is not separated from a replication unit is supplied to each interpolation unit, the electrical signal from one interpolation unit is supplied to the synthesis unit and the synthesis unit supplies the electrical signal from the other interpolation unit. A control unit that controls each of the interpolation units so as to discard the electrical signal,
When there are a plurality of defective pixels whose positions are continuous on the photoelectric conversion unit, the combining unit combines and outputs the electrical signals from the interpolation units, and the defective pixels on the photoelectric conversion unit 3. The photoelectric conversion device according to claim 1, wherein, when the pixel is a defective pixel, the electric signal supplied from the one interpolation unit is output. When the defective pixel on the photoelectric conversion unit is a single defective pixel, an interpolation unit that performs interpolation processing of the electrical signal of the defective pixel is controlled to perform the interpolation processing, and the electrical signal of the defective pixel The other interpolation unit to which the electrical signal not including the signal is supplied further includes a control unit that controls not to perform the interpolation process,
The synthesizing unit does not include the electrical signal that has been subjected to the interpolation processing of the defective pixel by the interpolation processing and the electrical signal of the defective pixel, and thus is output from the interpolation unit without performing the interpolation processing. The photoelectric conversion device according to claim 1, wherein the electrical signals are combined and output. A controller that controls each of the interpolation units so as to detect a template having a predetermined height when the templates generated by the interpolation units are compared with each other and the similarity of the templates is low. And more,
The photoelectric conversion apparatus according to claim 8, wherein each of the interpolation units performs the interpolation process using a template having a predetermined height of similarity. Based on the electrical signal before separation, a reference pattern for performing interpolation processing of the defective pixel is generated, the template generated by each interpolation unit is compared with the reference pattern, and the reference pattern A controller that controls the interpolation unit that has generated a template having a low similarity to detect a template having a similarity with a predetermined height that is similar to the reference pattern;
The photoelectric conversion apparatus according to claim 8, wherein each of the interpolation units performs the interpolation process using a template whose similarity with the reference pattern is a predetermined height. A photoelectric conversion step in which the photoelectric conversion unit converts the light received by each pixel into an electrical signal;
A detection step of detecting a defective pixel that outputs an electrical signal of an abnormal level among the pixels;
A specifying step of specifying a defective pixel having a continuous physical position on the photoelectric conversion unit among the detected defective pixels; and
An interpolation step in which a plurality of interpolation units perform an interpolation process in which an electrical signal of the defective pixel is an electrical signal of a normal pixel;
The separation unit separates the electrical signal from the photoelectric conversion unit and supplies the signal to the interpolation unit so that the defective pixels at the specified consecutive positions are interpolated by different interpolation units. A separation step;
And a synthesis step of synthesizing the electrical signals that have been subjected to the interpolation processing by the interpolation units, and reproducing and outputting a series of the electrical signals. An image forming apparatus that irradiates a document placed on a placement table with light, receives reflected light with a photoelectric conversion unit, and reads the document,
Among each pixel of the photoelectric conversion unit, a detection unit that detects a defective pixel that outputs an electrical signal of an abnormal level;
Of each detected defective pixel, a specifying unit that specifies a defective pixel having a continuous physical position on the photoelectric conversion unit;
A plurality of interpolation units for performing an interpolation process using the defective pixel electrical signal as a normal pixel electrical signal;
A separation unit that separates the electrical signal from the photoelectric conversion unit and supplies the signal to the interpolation unit so that each defective pixel at a specified continuous position is interpolated by the different interpolation unit;
An image forming apparatus comprising: a synthesizing unit that synthesizes the electric signals that have been subjected to the interpolation processing by the interpolating units, and reproduces and outputs a series of the electric signals.