JP2009188894A - Imaging device - Google Patents

Abstract

An imaging apparatus capable of recovering luminance information at the position of a defective pixel and acquiring single image data that prevents the luminance information from being completely lost even if defective pixels occur.
An optical system that collects incident light from the outside, a reflecting plate that divides a light beam L that has passed through the optical system 1 in a plurality of directions, and split light beams L1 and L2 that pass through the reflecting plate 4 are connected. The photoelectric conversion element 5 that outputs the electric signal 11 corresponding to the luminance information of the plurality of pixels P1 and P2, the element driver 10 that converts the electric signal 11 into the composite image data 12, and the composite image data 12 are simply set. And an image recovery unit 13 for converting into one image data 14.
[Selection] Figure 1

Description

  The present invention relates to an image pickup apparatus having a photoelectric conversion element, and more particularly to an image pickup apparatus that can obtain a signal of an image pickup region corresponding to a defective pixel even when a defective pixel occurs in the photoelectric conversion element.

  In recent years, photoelectric conversion elements are often used in imaging devices such as digital cameras and digital videos, and the number of pixels has been increased from tens of thousands to several millions. However, as the number of pixels in the imaging device has increased, it has become difficult to completely eliminate pixels (defective pixels) that cannot output a correct signal. In particular, some infrared photoelectric conversion elements have a relatively large number of defective pixels for structural reasons.

  Such a defective pixel becomes a white or black point in the photographed image, which is obstructive and may adversely affect the automatic contrast adjustment function of the image. Therefore, in order to avoid this problem from the past, it is examined in advance which defective pixel is, and the luminance information output by the defective pixel is replaced with the luminance information obtained by surrounding healthy pixels. A correction method has been proposed (see, for example, Patent Document 1).

  Normally, when an image is acquired by a photoelectric conversion element, luminance information at the position of a defective pixel is lost. However, in the case of a search / tracking device, for example, detection failure occurs when a target exists at the position of a defective pixel. It will be. Therefore, it is necessary to compensate the luminance information of the position of the defective pixel by some method.

  Normally, the luminance at the defective pixel is similar to the luminance information obtained by neighboring pixels, so when replaced with the luminance information obtained by neighboring normal pixels, the defective pixels are not noticed and the contrast is adjusted normally. Will be able to.

  However, the information formed on the defective pixel is not lost, and this may be a fatal problem in terms of the system. For example, with regard to an imaging device applied to a part of a search / tracking device, the target to be detected may be extremely small and may be 1 pixel or less, and if the pixel at the position where the target is imaged is defective The target will be missed. Further, when the target being tracked forms an image on a defective pixel, tracking may fail.

Japanese Patent Laid-Open No. 5-48974

In a conventional imaging device, a known defective pixel is replaced with luminance information obtained from a neighboring normal pixel. For example, when the target to be detected is 1 pixel or less when applied to a search / tracking device. If the target imaging position coincides with the defective pixel, the target cannot be detected, and tracking fails when the target being tracked forms an image on the defective pixel.
In addition, if the luminance information of the defective pixel is simply replaced with the luminance information of the surrounding normal pixels, the replacement pixel portion is replaced with an inappropriate luminance at the edge portion where the color or luminance changes abruptly. In some cases, the presence of a defective pixel is recognized.

  The present invention has been made to solve the above-described problems, and avoids the complete loss of information of an image formed at the position of the defective pixel even if a defective pixel occurs in the photoelectric conversion element. An object of the present invention is to obtain an image pickup apparatus.

  An image pickup apparatus according to the present invention includes an optical system that collects incident light from the outside, a splitting unit that splits a light beam that has passed through the optical system in a plurality of directions, and a plurality of split light beams formed through the splitting unit are imaged. A photoelectric conversion element that outputs an electrical signal corresponding to luminance information in a pixel, an element driver that converts the electrical signal into composite image data, and an image recovery means that converts the composite image data into single image data It is.

  According to the present invention, the light beam that should be focused on one pixel is divided (or dispersed) and focused on a plurality of pixels on the photoelectric conversion element, and the received signal is temporarily imaged into composite image data. Then, the luminance information is aggregated again to restore the image. Thereby, even if a defective pixel occurs, it is possible to recover the luminance information at the position of the defective pixel and obtain single image data that prevents the luminance information from being completely lost.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a schematic configuration of an image pickup apparatus according to Embodiment 1 of the present invention together with an optical system.
In FIG. 1, the imaging apparatus includes an optical system 1, a reflection plate 4 that changes the direction of a light beam L that has passed through the optical system 1, and a photoelectric conversion element 5 that receives the divided light beams L 1 and L 2 reflected by the reflection plate 4. , An element driver 10 that controls the photoelectric conversion element 5, and an image recovery unit 13 that converts the composite image data 12 from the element driver 10 into final single image data 14.

  The optical system 1 includes a pair of lenses 2 disposed to face each other so as to form an intermediate focal point, and a light shielding plate 3 disposed on an intermediate focal plane of each lens 2. Further, the light shielding plate 3 of the optical system 1 has a rectangular opening 3a, whereby an image finally formed on the photoelectric conversion element 5 is cut out in a rectangular shape. .

The light beam L incident from the outside and passed through the optical system 1 is individually reflected on the front and back surfaces of the reflecting plate 4 and divided into divided light beams L1 and L2, and then imaged on the photoelectric conversion element 5. .
At this time, the reflecting plate 4 is configured such that 50% of the total amount of light is reflected by the front surface and the back surface, and the light beam L (one-dot chain line) incident on the reflecting plate 4 is the first divided light beam. It is divided into two, L1 (two-dot chain line) and a second split light beam L2 (broken line).

The divided light beams L1 and L2 are respectively imaged on the two pixels P1 and P2 arranged on the upper surface of the photoelectric conversion element 5, and each luminance information is converted into an electric signal 11 and input to the element driver 10.
Thereafter, the element driver 10 converts the electrical signal 11 into composite image data 12 and inputs it to the image recovery unit 13, and the image recovery unit 13 outputs the final single image data 14 to the outside.

  In practice, the divided light beam L1 (two-dot chain line) and the divided light beam L2 (broken line) have different focal lengths until they reach the photoelectric conversion element 5, but the thickness of the reflector 4 is equal to the focal length. On the other hand, the size is negligible (for the sake of convenience in FIG. 1), the influence of the difference in focal length is not a problem. Accordingly, each of the light beams L1 and L2 is imaged almost accurately on the upper surface of the photoelectric conversion element 5.

As apparent from FIG. 1, the image formed on the surface of the photoelectric conversion element 5 is originally reflected on the front and back surfaces of the reflection plate 4 by the single light beam L and is connected to the pixels P1 and P2 at individual positions. Therefore, the image has a double outline (hereinafter referred to as “double image”).
The photoelectric conversion element 5 is driven and controlled by the element driver 10, converts luminance information of an image formed on the surface of the photoelectric conversion element 5 into an electric signal 11, and transmits the electric signal 11 to the element driver 10.

The element driver 10 includes a drive control diagram, an amplifier, an AD converter, and a signal processing unit (not shown). The element driver 10 amplifies and AD converts the electric signal 11 and converts the signal format to generate composite image data 12 of a double image. Is generated.
The image recovery unit 13 recovers the composite image data 12 (double image) to an original image that is not double (hereinafter referred to as “original image”), and outputs it as final single image data 14.

FIG. 2 is an explanatory diagram showing a specific example of the composite image data 12.
In FIG. 2, an image G <b> 1 is an imaging frame by surface reflection (divided light beam L <b> 1) of the reflecting plate 4, and an image G <b> 2 is an imaging frame by back surface reflection (divided light beam L <b> 2) of the reflecting plate 4.
Here, each image G1, G2 is shown as an image in which a round figure is shown. For ease of recognition, the other image G2 is shown slightly lowered with respect to one image G1, but in reality, there is no vertical displacement between the images G1 and G2. .

The composite image data 12 is composed of composite images of the images G1 and G2. In the composite image data 12, the round figure in each of the images G1 and G2 appears with double outlines as shown in FIG. Become.
Further, in the composite image data 12, the surface reflection image G 1 is normally shown in the shift area Ra, but the back reflection image G 2 is blocked by the light shielding plate 3 and becomes completely dark (uniform). This is the same as the state in which the image G2 is not shown.

On the other hand, the image in the adjacent region Rb on the right side of the shift region Ra (having the same width as the shift amount) is surface-reflected with the same image as the image G1 in the shift region Ra of the image G2 reflected from the back surface. Of the obtained image G1, an image in the right adjacent region of the image G1 in the shift region Ra is combined.
In this case, since the image G1 in the shift region Ra is known, the image due to surface reflection is also obtained for the adjacent region Rb by subtracting the image G1 (data) in the shift region Ra from the composite image in the adjacent region Rb. Only the component of G1 can be calculated.

Therefore, the image recovery unit 13 repeatedly executes the above signal processing for each region Ra, Rb,..., Rn in the right direction in FIG. One image data 14 can be recovered.
Hereinafter, this image restoration procedure is referred to as “image restoration procedure 1”.

Next, as shown in FIG. 2, a defective pixel Pa (see black square) occurs at one point in the adjacent region Rb among the horizontal pixels Pa, Pb,..., Pn of the photoelectric conversion element 5. The case will be described.
In this case, in the image recovery procedure 1 described above, the luminance information at the position of the defective pixel Pa in the image G1 cannot be recovered. Further, the luminance information of the position of the pixel Pb in the right adjacent image G2 is not obtained, and the same applies to the subsequent right adjacent pixel. Hereinafter, the luminance of the pixels having the same positional relationship until the right end pixel Pn is reached. Information cannot be obtained.

Therefore, in this case, the image recovery unit 13 switches the image recovery procedure 1 and sequentially recovers the single image data 14 from the right end shift region Rn in which the images G1 and G2 are in the inverse relationship to the adjacent region on the left side. Go.
In this way, the luminance information is obtained up to the position of the pixel Pb in the image G2 by reading the composite image data 12 of the image G2 from R right to. The single image data 14 can be recovered as described above. Hereinafter, this image restoration procedure is referred to as “image restoration procedure 2”.

Here, the positional relationship of the “pixel Pb in the image G2” is the same as the positional relationship of the “defective pixel Pa in the image G1,” and considering that the image G1 and the image G2 are the same image, the defect Luminance information at the position of the pixel Pa is obtained.
That is, regardless of the position of the defective pixel, in principle, if the image G1 and the image G2 are shifted by at least one pixel, one defective pixel position in the lateral direction is obtained by the correction method according to the image restoration procedures 1 and 2. It is possible to recover the luminance information.

Further, if the shift amounts of the images G1 and G2 are two pixels, if there are no two or more defective pixels in either the odd-numbered pixel series or the even-numbered pixel series in the horizontal direction, for example, the luminance Information recovery is possible.
Further, even when the image shift amount is 3 pixels or more, the luminance information can be recovered from the same concept.

  In this way, by applying the correction methods of the image restoration procedures 1 and 2, even if a defective pixel Pa occurs, a part of the image information is not lost completely. For example, when applied to a search / tracking device In this case, it is possible to prevent a detection failure of a minute tracking target.

  In other words, in the search / tracking device, since the occurrence of defective pixels becomes a fatal problem, a photoelectric conversion element 5 having no defect is required. Therefore, the yield of the photoelectric conversion element 5 is deteriorated and the procurement price is increased. However, by allowing the generation of defective pixels to be acceptable, these problems can be avoided.

  As described above, the imaging apparatus according to Embodiment 1 of the present invention includes the optical system 1 that collects incident light from the outside, and the reflecting plate 4 that divides the light beam L that has passed through the optical system 1 into a plurality of directions ( Splitting means), the photoelectric conversion element 5 that outputs the electric signal 11 corresponding to the luminance information in the plurality of pixels P1 and P2 by forming the divided light beams L1 and L2 via the reflector 4 and the electric signal 11 An element driver 10 for converting into the composite image data 12 and an image recovery unit 13 for converting the composite image data 12 into the single image data 14 are provided.

The reflector 4 (dividing means) divides the light beam L that has passed through the optical system 1 into a first divided light beam L1 and a second divided light beam L2, and divides the first and second divided light beams L1, L2 into two. Images are individually formed on different pixels P1 and P2 of the photoelectric conversion element 5.
Further, the element driver 10 outputs the composite image data 12 corresponding to the first and second split light beams L1 and L2, and the image recovery unit 13 receives the first split light beam L1 or the second split light from the composite image data 12. The single image data 14 corresponding to only one of the divided light beams L2 is output.

The optical system 1 has a light shielding plate 3 having a rectangular opening 3a for cutting the light beam L into a rectangular shape, and the dividing means is constituted by a reflecting plate 4 that reflects incident light by 50% on the front and back surfaces, respectively. Yes.
Further, the image recovery unit 13 performs the composite image data 12 for each region (Ra, Rb,..., Rn) corresponding to the shift amount of the composite image data 12 corresponding to the first and second divided light beams L1 and L2. The image corresponding to one of the first divided light beam L1 and the second divided light beam L2 is subtracted from the composite image data 12 sequentially from one end of the image data to form single image data.

In this way, the light beam L that should be focused on one pixel is split and received through the reflector 4 (dividing means) so that it is intentionally focused on the two pixels P1 and P2. By correcting the composite image data 12 based on the electrical signal 11, even if a defective pixel Pa occurs, single image data 14 in which luminance information at the defective pixel Pa is recovered can be obtained.
Therefore, particularly when applied to a search / tracking device, a target recognized with a size of one pixel or less can be accurately detected even if a defective pixel Pa occurs.

  Furthermore, not only in the search / tracking device, even when applied to a normal imaging device, if the luminance information of the defective pixel Pa is simply replaced with the luminance information of the surrounding normal pixels as in the conventional device, the edge portion Although the presence of the defective pixel Pa is recognized, according to the first embodiment of the present invention, such erroneous recognition can be avoided.

Embodiment 2. FIG.
In the first embodiment (FIG. 2), in the image recovery unit 13, the image recovery procedures 1 and 2 (each region Ra) start from the “region where the images do not overlap (Ra or Rn)” at the left and right ends. ,..., A single image data 14 (original image) is acquired using a subtraction process for each Rn), but instead of the image restoration procedures 1 and 2, it is called deconvolution (deconvolution). The single image data 14 may be acquired using a general calculation method. The inverse convolution calculation can be referred to a known document (for example, Japanese Patent Application Laid-Open No. 2003-255231, paragraphs 0033-).

In this case, the image recovery unit 13 forms single image data 14 from the composite image data 12 by using an inverse convolution operation.
At this time, the composite image data 12 made up of the double image is considered to be the same as the image generated by applying a certain transfer function to the original image in principle. By applying an inverse transfer function (inverse convolution calculation) to the data 12 by signal processing, the single image data 14 (original image) can be recovered.

  In Embodiment 2 of the present invention, an inverse convolution operation is used instead of the image restoration procedures 1 and 2 in the image restoration device 13 as the original image restoration method, and the luminance information of the defective pixel Pa is the luminance information of the surrounding pixels. The original image is restored by performing an inverse convolution operation on the complemented image.

As a result, the same effects as those of the first embodiment can be obtained, and as the image restoration procedure, the number of defective pixels is not limited to only a single pixel in a series of steps of a certain number of pixels in the horizontal direction. There is an effect that even if a pixel occurs, it can be dealt with.
In the above-described first embodiment (FIG. 2), it is necessary to set the shift areas Ra and Rn in which only one of the images G1 and G2 of the divided light beams L1 and L2 by the front surface reflection or the back surface reflection is shown. According to the second embodiment of the present invention, it is not necessary to set the shift areas Ra and Rn, and the light shielding plate 3 for limiting each area is also unnecessary.

Embodiment 3 FIG.
In the first embodiment (FIG. 1), the reflecting plate 4 that divides the light beam L into two is used. However, as shown in FIG. 3, a dispersion optical element 23 may be used as the dividing means.
FIG. 3 is a block diagram showing a schematic configuration of an image pickup apparatus according to Embodiment 3 of the present invention together with an optical system. Components similar to those described above (see FIG. 1) are denoted by the same reference numerals as described above, or A detailed description will be omitted by adding “A” after the reference numeral.

In FIG. 3, a dispersion optical element 23 is inserted in the optical path from the optical system 22 to the photoelectric conversion element 5.
The optical system 22 is composed of a single lens (convex lens) that does not include the light-shielding plate 3, condenses incident light from the outside into a light beam L (one-dot chain line), and forms an image on the surface of the photoelectric conversion element 5. (Refer to light beam La) It arrange | positions.

  The dispersive optical element 23 is composed of a single lens (concave lens) and disperses the original light beam La (two-dot chain line) to be focused on the surface of the photoelectric conversion element 5 by the optical system 22 as a dispersive light beam Lb (broken line). Let me blur. As a result, the light beam La that should originally be concentrated at one point becomes the dispersed light beam Lb diffused in a certain region, and thus becomes a blurred image as a whole (hereinafter referred to as “blurred image”), and the photoelectric conversion element 5. Reach a plurality of pixels on the surface.

The dispersive optical element 23 is designed to generate a blurred image stably and accurately over the entire screen on the photoelectric conversion element 5, and the transfer function for generating the blurred image from the original image is known. It is.
Therefore, the image recovery unit 13A can recover the original image by performing a reverse convolution operation on the composite image data 12A, as in the second embodiment.

As described above, the dividing unit according to the third embodiment of the present invention includes the dispersive optical element 23 that disperses the light beam L that has passed through the optical system 22, and converts a blurred image based on the dispersive light beam Lb into a plurality of photoelectric conversion elements 5. An image is formed on the pixel.
Further, the image recovery unit 13A forms single image data 14 from the composite image data 12A based on the blurred image by using the inverse convolution operation.

As a result, the same effects as those of the second embodiment are obtained.
In the first and second embodiments, the light beam L (image information) that should be concentrated on one pixel is only divided into two, but in the third embodiment of the present invention, the original concentrated pixel is used. (The number of dispersion can be freely set by design), the influence of information loss due to defective pixels can be relatively reduced, and the image resilience can be improved.

Note that the entire system of the imaging apparatus may be configured so that the dispersive optical element 23 inserted in the optical path of the light beam L is unnecessary, and as a result, a divided light beam is obtained.
For example, the image recovery unit 13A forms the single image data 14 from the composite image data 12A based on the blurred image by using the inverse convolution operation, and the dividing unit is incorporated in advance as the blur characteristic of the optical system 22. Also good. In other words, the imaging state may be degraded in advance (focal length is intentionally shifted) when the optical system 22 is designed.

Alternatively, one pixel on the surface of the photoelectric conversion element 5 may be set smaller than the imaging performance of the optical system 22 even in a configuration that optically forms an image sufficiently.
In this case, for example, the dividing unit includes a distance setting unit (not shown) that variably sets the optical distance between the optical system 22 and the photoelectric conversion element 5, and passes through the position of the photoelectric conversion element 5 and the optical system 22. By setting the optical distance so that the position of the condensing point of the light beam L is different, a blurred image based on the dispersed light beam is formed on a plurality of pixels of the photoelectric conversion element 5.

Thereby, the image recovery unit 13A can form single image data 14A from the composite image data 12A based on the blurred image by using the inverse convolution operation.
Therefore, by intentionally dispersing the light flux L to be collected into a plurality of pixels, the influence of defective pixels can be reduced and the image resilience can be improved as described above.

Embodiment 4 FIG.
In the third embodiment (FIG. 3), the dispersion optical element 23 (dividing means) and the image recovery unit 13A are always inserted. However, as shown in FIG. 4, the dispersion process and the correction process are unnecessary. Further, a selector 24 for generating a switching signal, a switch means 25 and a driving device 26 responsive to the switching signal may be provided so that the dispersive optical element 23 and the image recovery device 13A can be removed (invalidated).

FIG. 4 is a block diagram showing a schematic configuration of an image pickup apparatus according to Embodiment 4 of the present invention together with an optical system. The same components as those described above (see FIG. 3) are denoted by the same reference numerals as those described above and described in detail. Is omitted.
In FIG. 4, the selector 24 outputs a switching signal in response to an external operation in an operation mode that does not require dispersion processing and correction processing.

The switch means 25 is inserted between the element driver 10A and the image recovery unit 13A, and shorts between both ends of the image recovery unit 13A in response to a switching signal from the selector 24 (switches to a broken line position).
In addition, the driving device 26 drives the dispersion optical element 23 to the invalid position (see the broken line) in response to the switching signal from the selector 24.

  When the switching signal from the selector 24 is no longer input (switched again to an operation mode that requires dispersion processing and correction processing), the switch means 25 and the drive device 26 return the dispersion optical element 23 and the image recovery device 13A to the original state. Return to the effective position (solid line position).

  According to the fourth embodiment of the present invention, when the dispersion process of the luminous flux L and the correction process of the composite image data 12 are not required, the switching signal is generated from the selector 24 by the external operation of the operator, and the drive device 26 The dispersion optical element 23 is removed from the optical path of the light beam L, and the image recovery device 13A is short-circuited by the switch means 25, whereby the output data from the element driver 10A is directly converted into single image data 14 as usual. Output.

For example, in the above-described first to third embodiments, a procedure in which a double image (dispersed image) is once formed and intentional image deterioration is generated and then the original image is restored by correction processing is always applied. Therefore, compared with the case where the original image is directly acquired by the normal method, the image after the image restoration becomes a rougher image than the true original image, or characteristic noise is superimposed. There is.
In addition, a search / tracking device to which such an imaging device is applied may have not only a search / tracking function but also an image output function. Even if some defective pixels are generated during image supply, the search / tracking device is clear. It is preferable to supply a correct image.

  On the other hand, according to the fourth embodiment of the present invention, with the above configuration, it is possible to return to the normal image acquisition method as necessary, so that a clear image is acquired in the image output mode, and search / tracking is performed. In the mode, it is possible to selectively use an image that is not affected by defective pixels.

Therefore, when loss of luminance information due to defective pixels is not a problem in use, dispersion processing and correction processing can be prohibited in order to wire the image quality, so that optimum performance for the needs can be exhibited. it can.
Here, although the case where the dividing means is the dispersion optical element 23 is shown, as described above, the distance setting means (not shown) for variably setting the optical distance between the optical system 22 and the photoelectric conversion element 5 is driven. You may comprise.

Embodiment 5 FIG.
In the fourth embodiment (FIG. 4), the selector 24 that generates a switching signal in response to an external operation is used. However, as shown in FIG. A selector 24B that generates a signal may be used.
FIG. 5 is a block diagram showing a schematic configuration of an image pickup apparatus according to Embodiment 5 of the present invention together with an optical system. Components similar to those described above (see FIG. 4) are denoted by the same reference numerals as described above, or A “B” is appended to the reference numeral and the detailed description is omitted.

  In FIG. 5, the selector 24B determines whether the dispersion optical element 23 and the image recovery unit 13 are necessary based on at least one of the electrical signal 11A from the photoelectric conversion element 5 and the single image data 14 from the image recovery unit 13A. When it is determined that the dispersion optical element 23 and the image recovery unit 13A are unnecessary, a switching signal is output.

  For example, if the selector 24B determines from the state of the electrical signal 11A or the single image data 14 that there is no defective pixel in the photoelectric conversion element 5 (or a level that can be ignored even if there is a defective pixel), A switching signal for invalidating the optical element 23 and the image recovery unit 13A is output.

As described above, according to the fifth embodiment of the present invention, when the selector 24B determines that the correction process is unnecessary, the driving device 26 controls the dispersion optics from the optical path of the light beam L under the control of the selector 24B. While the element 23 is removed, the image recovery unit 13A is short-circuited by the switch means 25, and the output data from the element driver 10A is directly output as the single image data 14.
Thereby, when there is no defective pixel, or when loss of luminance information due to the defective pixel is not a problem, the single image data 14 based on the normal path can be automatically acquired.

1 is a block diagram showing a schematic configuration of an imaging apparatus according to Embodiment 1 of the present invention together with an optical system. It is explanatory drawing which shows the image image | photographed with the imaging device which concerns on Embodiment 1 of this invention. It is a block diagram which shows schematic structure of the imaging device which concerns on Embodiment 3 of this invention with an optical system. It is a block diagram which shows schematic structure of the imaging device which concerns on Embodiment 4 of this invention with an optical system. It is a block diagram which shows schematic structure of the imaging device which concerns on Embodiment 5 of this invention with an optical system.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1,22 Optical system, 3 Light-shielding plate, 3a Rectangular aperture, 4 Reflecting plate (dividing means), 5 Photoelectric conversion element, 10, 10A Element driver, 11, 11A Electrical signal, 12, 12A Composite image data, 13, 13A image Recovery unit, 14 Single image data (original image), 23 Dispersion optical element (dividing means), 24, 24B selector, 25 switch means, 26 drive unit, G1, G2 Split beam, L beam, L1, L2 split Luminous flux, Lb dispersed luminous flux, P1, P2 pixels, Pa defective pixels, Pb to Pn pixels (normal pixels), Ra, Rn deviation areas (corresponding to deviation amounts), Rb adjacent areas.

Claims (10)

An optical system that collects incident light from the outside;
Splitting means for splitting the light beam that has passed through the optical system in a plurality of directions;
A photoelectric conversion element that outputs an electric signal corresponding to luminance information in a plurality of pixels by forming an image of the divided luminous flux through the dividing unit;
An element driver for converting the electrical signal into composite image data;
An image pickup device comprising: image recovery means for converting the composite image data into single image data. The splitting unit splits the light beam that has passed through the optical system into a first split light beam and a second split light beam, and the first and second split light beams are placed on different pixels of the photoelectric conversion element. Image individually,
The element driver outputs the composite image data corresponding to the first and second split light beams,
The said image restoration means outputs the said single image data corresponding to only one of the said 1st division | segmentation light beam or the said 2nd division | segmentation light beam from the said composite image data. Imaging device. The optical system has a light shielding plate having a rectangular opening for cutting out the light beam into a rectangular shape,
The imaging apparatus according to claim 2, wherein the dividing unit includes a reflecting plate that reflects incident light by 50% on the front surface and the back surface.   The image restoration means sequentially starts from one end of the composite image data from the composite image data for each region corresponding to the shift amount of the composite image data corresponding to the first and second divided light beams. The imaging apparatus according to claim 3, wherein the single image data is formed by performing subtraction processing on data corresponding to one of the divided light beam and the second divided light beam.   The imaging apparatus according to claim 3, wherein the image restoration unit forms the single image data from the synthesized image data by using an inverse convolution operation. The dividing unit includes a dispersion optical element that disperses a light beam that has passed through the optical system, and forms a blurred image based on the dispersed light beam on a plurality of pixels of the photoelectric conversion element,
The imaging apparatus according to claim 1, wherein the image restoration unit forms the single image data from the composite image data based on the blurred image by using an inverse convolution operation. The dividing unit includes a distance setting unit that variably sets an optical distance between the optical system and the photoelectric conversion element, and a position of the photoelectric conversion element is different from a position of a condensing point of a light beam that has passed through the optical system. By setting the optical distance as follows, a blurred image based on the dispersed light beam is formed on a plurality of pixels of the photoelectric conversion element,
The imaging apparatus according to claim 1, wherein the image restoration unit forms the single image data from the composite image data based on the blurred image by using an inverse convolution operation.   The image restoration means forms the single image data from the composite image data based on the blurred image using an inverse convolution operation, and the dividing means is incorporated in advance as a blur characteristic of the optical system. The imaging apparatus according to claim 1. A selector that outputs a switching signal in response to an external operation;
Switch means for short-circuiting both ends of the image recovery means in response to the switching signal;
The imaging apparatus according to claim 6, further comprising: a driving device that drives the dividing unit to an invalid position in response to the switching signal. Based on at least one of the electrical signal and the single image data, it is determined whether or not the dividing unit and the image restoration unit are necessary, and a switching signal is output when the division unit and the image restoration unit are unnecessary. A selector,
Switch means for short-circuiting both ends of the image recovery means in response to the switching signal;
The imaging apparatus according to claim 6, further comprising: a driving device that drives the dividing unit to an invalid position in response to the switching signal.

Images