JP2012049791A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which allows reduction in production cost by dispensing with a period for initial adjustment for the correction of variation in pixel output.SOLUTION: While ordinary operation is performed, storage operation for measurement is performed during a shorter storage period than a normal storage period. On the basis of a pair of sheets of image data obtained by the storage operation for measurement and the ordinary operation just after the storage operation for measurement, photoelectric transfer characteristic is estimated, and the pixel output is corrected using the estimated photoelectric transfer characteristic.

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus having an imaging element that converts incident light into an electrical signal by photoelectric conversion.

  Conventionally, an imaging device such as a digital camera has been provided with an imaging device that converts incident light into an electrical signal. In recent years, image sensors having a plurality of types of photoelectric conversion characteristics have been proposed. For example, a linear log conversion type sensor is proposed in which a linear conversion operation for linearly converting incident light into an electrical signal based on an incident light amount and a logarithmic conversion operation for logarithmic conversion are switched depending on the incident light amount.

  In the linear log conversion type sensor, the point (inflection point) at which the conversion characteristic is switched from linear to log in units of pixels varies due to variations in characteristics of transistors that constitute the log characteristics. As a result, noise is included in the obtained image. Further, the variation of the inflection point changes under the influence of the ambient temperature.

  As a method for correcting the variation of such an inflection point, for example, Patent Document 1 discloses a method of correcting based on a characteristic measurement result measured in advance.

JP 2008-28623 A

  As described above, conventionally, in an imaging apparatus having an imaging element composed of a linear log conversion type sensor, conventionally, photoelectric conversion characteristics and temperature characteristics are measured at the manufacturing stage of the imaging apparatus, and based on the results. Since the configuration for correcting the variation of the inflection point is adopted, there is a problem that the initial adjustment takes time and the manufacturing cost increases.

  The present invention has been made to solve the above-described problems, and it is an object of the present invention to provide an image pickup apparatus that does not require time for initial adjustment for correcting pixel output variation and reduces manufacturing costs. And

  A first aspect of an imaging apparatus according to the present invention includes a plurality of pixels configured by photoelectric conversion elements, an imaging element that converts incident light into an electrical signal by photoelectric conversion, and an output between the plurality of pixels. A variation correction unit that corrects variations, wherein the variation correction unit outputs a first output based on the charge accumulated in the first accumulation time in the imaging element, and the first in the imaging element. A second output that is output based on the charge accumulated in the second accumulation time shorter than the first accumulation time, and from each of the first and second outputs for each pixel, Data on conversion characteristics is acquired, and a photoelectric conversion characteristic at each pixel is obtained from a ratio of the first and second accumulation times for the obtained plurality of pixels and an output value at the first and second accumulation times. Estimate and estimate Correcting the variation among the plurality of pixels based on the photoelectric conversion characteristics were.

  In a second aspect of the imaging apparatus according to the present invention, the photoelectric conversion characteristic has a characteristic that a plurality of characteristics continuously change at an inflection point, and based on data on the photoelectric conversion characteristic, The calculated unknown coefficient of the photoelectric conversion characteristic, the offset value, and the position of the inflection point are used for estimation.

  In the third aspect of the imaging apparatus according to the present invention, the first accumulation time corresponds to an accumulation time for one frame of image data, and only the first output is used for generating image data.

  In a fourth aspect of the imaging apparatus according to the present invention, a plurality of types of ratios of the second accumulation time with respect to the first accumulation time are set and changed in a predetermined cycle.

  According to a fifth aspect of the imaging apparatus of the present invention, the imaging apparatus has a temperature sensor, and when the temperature change is detected, the estimated photoelectric conversion characteristic is discarded, and the acquired photoelectric conversion characteristic is obtained again. Based on the data, the photoelectric conversion characteristics at each pixel are estimated.

  According to the first aspect of the imaging apparatus of the present invention, data on photoelectric conversion characteristics representing an output with respect to the incident light amount is acquired from the first and second outputs for each pixel, and the acquired plurality of pixels A photoelectric conversion characteristic at each pixel is estimated from a ratio of the first and second accumulation times and an output value at the first and second accumulation times, and a plurality of values are calculated based on the estimated photoelectric conversion characteristics. Since variations between pixels are corrected, initial adjustment such as measurement of photoelectric conversion characteristics and temperature characteristics becomes unnecessary at the manufacturing stage of the imaging apparatus, and manufacturing costs can be reduced.

  According to the second aspect of the imaging apparatus of the present invention, the photoelectric conversion characteristic can be estimated by acquiring the unknown coefficient, offset value, and inflection point of the photoelectric conversion characteristic.

  According to the third aspect of the imaging apparatus according to the present invention, it is possible to estimate the photoelectric conversion characteristics at the spot of imaging.

  According to the 4th aspect of the imaging device which concerns on this invention, it can respond to the case where the brightness of an imaging scene changes frequently.

  According to the fifth aspect of the imaging apparatus according to the present invention, it is possible to always calculate an appropriate characteristic coefficient with the data on the spot without acquiring data in advance under any temperature environment.

1 is a block diagram illustrating an overall configuration of an imaging apparatus according to an embodiment of the present invention. It is a block diagram which shows the structure of the image pick-up element of the imaging device of embodiment which concerns on this invention. It is a figure which shows the structure of the photoelectric conversion element which comprises an image sensor. It is a figure which shows the photoelectric conversion characteristic in an image sensor. It is a figure which shows the dispersion | variation in a photoelectric conversion characteristic. It is a figure which shows typically the image output in normal operation | movement. It is a figure which shows typically the timing of the accumulation | storage operation | movement for a dispersion | variation characteristic measurement. It is a figure which shows typically the timing of the accumulation | storage operation | movement for a dispersion | variation characteristic measurement. It is a figure which shows the output obtained by the accumulation | storage operation | movement at the time of measurement, and the normal operation | movement immediately after that on a photoelectric conversion characteristic. It is a flowchart which shows the accumulation | storage method of the data set for calculation of the characteristic coefficient of a photoelectric conversion characteristic.

<Embodiment>
<Device configuration>
FIG. 1 shows an overall configuration of an imaging apparatus 1 according to the present embodiment. As shown in FIG. 1, the imaging apparatus 1 is configured to receive incident light on the imaging surface of the imaging element 4 via a lens system 2 and a diaphragm 3. As the lens system 2 and the diaphragm 3, conventionally known ones are used.

  Here, the imaging device of the present invention is an imaging device having a plurality of pixels that convert incident light into an electric signal by a plurality of types of conversion characteristics, and an output signal of the plurality of conversion characteristics continuously changes via a switching point. It is supposed to be.

  The imaging device 4 is configured by a linear log conversion type sensor capable of switching between a linear conversion operation for linearly converting incident light into an electrical signal and a logarithmic conversion operation for logarithmic conversion based on the amount of incident light. The output signal of the image sensor 4 is configured to continuously change from the linear region to the log region via an inflection point. Here, the “inflection point” means a boundary between the linear conversion operation and the logarithmic conversion operation, and is a term that is a subordinate concept of the “switching point” of the output signal of the image sensor having a plurality of types of conversion characteristics.

  The imaging element 4 having a plurality of types of conversion characteristics is not limited to a linear log conversion type sensor in which an output signal continuously changes from a linear region to a log region, and an output signal is continuously from the log region to the linear region. However, the present invention can be applied even to an image sensor having three or more types of output characteristics.

  As shown in FIG. 2, the image pickup device 4 has a plurality of pixels G11 to Gmn (where n and m are integers of 1 or more) arranged in a matrix (matrix arrangement).

  Each of the pixels G11 to Gmn is composed of a photoelectric conversion element that photoelectrically converts incident light and outputs an electrical signal. These pixels G11 to Gmn are configured to switch the conversion operation to an electric signal based on the incident light quantity. As will be described later, the linear conversion for linearly converting the incident light with respect to the incident light quantity less than the predetermined incident light quantity. As for the operation, a logarithmic conversion operation for logarithmically converting incident light is performed for an incident light amount equal to or greater than a predetermined incident light amount.

  A Bayer array RGB primary color filter is mounted on the lens system 2 side of the pixels G11 to Gmn. The type of color filter is not limited to this, and other color filters such as cyan, magenta, and yellow may be provided in each of the pixels G11 to Gmn. Moreover, it is good also as a structure which does not mount a color filter.

  Further, as shown in FIG. 2, a power supply line 5, signal application lines LA1 to LAn, LB1 to LBn, LC1 to LCn, and signal readout lines LD1 to LDm are connected to the pixels G11 to Gmn. Note that lines such as a clock line and a bias supply line are also connected to the pixels G11 to Gmn, but these are not shown in FIG.

  The signal application lines LA1 to LAn, LB1 to LBn, and LC1 to LCn supply signals φv and φVPS (see FIG. 3) to the pixels G11 to Gmn. A vertical scanning circuit 6 is connected to these signal application lines LA1 to LAn, LB1 to LBn, and LC1 to LCn. The vertical scanning circuit 6 applies signals to the signal application lines LA1 to LAn, LB1 to LBn, and LC1 to LCn based on signals from a timing generation circuit 22 (see FIG. 1) described later. The target signal application lines LA1 to LAn, LB1 to LBn, and LC1 to LCn are sequentially switched in the X direction.

  The electric signals generated by the pixels G11 to Gmn are derived from the signal readout lines LD1 to LDm. Constant current sources D1 to Dm and selection circuits S1 to Sm are connected to these signal read lines LD1 to LDm.

  A DC voltage VPS is applied to one end of the constant current sources D1 to Dm (lower end in the figure).

  The selection circuits S1 to Sm sample-hold the noise signals given from the pixels G11 to Gmn and the electric signals at the time of imaging through the signal readout lines LD1 to LDm. A horizontal scanning circuit 7 and a correction circuit 8 are connected to these selection circuits S1 to Sm. The horizontal scanning circuit 7 sequentially switches selection circuits S1 to Sm that sample and hold an electric signal and transmit the signal to the correction circuit 8 in the Y direction. Further, the correction circuit 8 removes the noise signal from the electric signal based on the noise signal transmitted from the selection circuits S1 to Sm and the electric signal at the time of imaging.

  As the selection circuits S1 to Sm and the correction circuit 8, those disclosed in Japanese Patent Application Laid-Open No. 2001-223948 can be used. In this embodiment, the description will be made assuming that only one correction circuit 8 is provided for the entire selection circuits S1 to Sm. However, one correction circuit 8 is provided for each of the selection circuits S1 to Sm. It is good as well.

  Subsequently, the pixels G11 to Gmn will be described. Each of the pixels G11 to Gmn includes a photodiode P and transistors T1 to T3 as shown in FIG. The transistors T1 to T3 are N-channel MOS transistors whose back gates are grounded.

  The light passing through the lens system 2 and the aperture stop 3 strikes the photodiode P. A DC voltage VPD is applied to the cathode Pk of the photodiode P, and the drain T1D and gate T1G of the transistor T1 and the gate T2G of the transistor T2 are connected to the anode PA.

  A signal application line LC (corresponding to LC1 to LCn in FIG. 2) is connected to the source T1S of the transistor T1, and a signal φVPS is input from the signal application line LC. Here, the signal φVPS is a binary voltage signal, and more specifically, the voltage value VH for operating the transistor T1 in the subthreshold region when the incident light quantity exceeds a predetermined value, and the transistor T1 in the conductive state. The voltage value VL is set to two values.

  When the amount of incident light exceeds a predetermined threshold, the image sensor 4 can read out incident light as a logarithmically converted logarithm of natural light by operating the transistors T1 of the pixels G11 to Gmn in the subthreshold region. It has become. Thereby, as shown in FIG. 4, the output signal of the image sensor 4 has a linear region and a logarithmic region that change continuously according to the amount of incident light. That is, the pixels G11 to Gmn are configured to switch the conversion operation to the electric signal based on the incident light amount. As shown by the solid line in FIG. 4, the incident light is less than the predetermined incident light amount th. A linear conversion operation for linearly converting the incident light is performed as a logarithmic conversion operation for logarithmically converting the incident light with respect to an incident light amount equal to or greater than a predetermined incident light amount th.

  As shown in FIG. 3, a DC voltage VPD is applied to the drain T2D of the transistor T2, and the source T2S of the transistor T2 is connected to the drain T3D of the row selection transistor T3.

  Further, a signal application line LA (corresponding to LA1 to LAn in FIG. 2) is connected to the gate T3G of the transistor T3, and a signal φV is input from the signal application line LA. The source T3S of the transistor T3 is connected to a signal read line LD (corresponding to LD1 to LDm in FIG. 2).

  As the pixels G11 to Gmn as described above, those disclosed in Japanese Patent Application Laid-Open No. 2002-77733, Japanese Patent Application Laid-Open No. 2006-303768, or Japanese Patent Application No. 2009-24261 by the present applicant can be used.

  Here, it returns to description of FIG. 1 again. An amplifier 9 and an A / D converter 10 are electrically connected to the image sensor 4, and a determination circuit 11, a proximity pixel memory 13, and a variation correction unit 14 are electrically connected to the A / D converter 10. ing. The determination circuit 11, the proximity pixel memory 13, and the variation correcting unit 14 are electrically connected to each other. A variation correction data memory 12 is electrically connected to each of the determination circuit 11 and the variation correction unit 14.

  As the amplifier 9, a conventionally known amplifier is used, and a signal photoelectrically converted by the image sensor 4 is amplified.

  The A / D converter 10 converts the electric signal amplified by the amplifier 9 from an analog signal to a digital signal.

  The determination circuit 11 includes a correction unit determination unit 11a that determines a correction unit for output signals of the pixels G11 to Gmn, an output characteristic determination unit 11b that determines the characteristics of output signal values of the pixels G11 to Gmn, and the pixels G11 to Gmn. The color determination unit 11c for determining the color of the output signal value is provided.

  The adjacent pixel memory 13 temporarily stores pixel data for performing interpolation processing between adjacent pixels.

  The variation correction unit 14 corrects output variations among the pixels G11 to Gmn, and includes a memory correction unit 14a and an interpolation correction unit 14b.

  Further, the black reference correction unit 15, the linearization unit 16, and the image processing unit 17 are electrically connected to the variation correction unit 14 in this order.

  The black reference correction unit 15 sets a minimum level of the digital signal.

  The linearization unit 16 functions as a characteristic conversion unit, and converts the output signal of the image sensor 4 having a plurality of types of conversion characteristics into a state that is uniformly converted by one conversion characteristic. ing. The linearization unit 16 of the present embodiment is a state in which the logarithmic region of the output signal of the image sensor 4 is linearly converted based on inflection point information transmitted from the output characteristic control circuit 21 described later via the control device 19. It is designed to convert to an electrical signal.

  The image processing unit 17 performs image processing such as white balance processing, color interpolation processing, color correction processing, gradation conversion, and color space conversion on the image data formed by the entire electrical signals from the pixels G11 to Gmn. It is like that.

  In addition, a control device 19 is electrically connected to the linearization unit 16 via an evaluation value calculation unit 18.

  The evaluation value calculation unit 18 calculates an AWB evaluation value used for white balance processing (AWB processing) in the image processing unit 17 and an AE evaluation value used for (AE processing) in the exposure control processing unit 20 described later. It is like that.

  The control device 19 includes a CPU, a RAM, and the like, and controls each component of the imaging device 1. The control device 19 is electrically connected to the determination circuit 11, the variation correction data memory 12, the proximity pixel memory 13, the variation correction unit 14, and the linearization unit 16.

  Further, the diaphragm 3 is electrically connected to the control device 19 via the exposure control processing unit 20. Further, the imaging device 4 and the determination circuit 11 are electrically connected to the control device 19 via the output characteristic control circuit 21, and the imaging device 4 and the determination circuit 11 are connected via the timing generation circuit 22. Each is electrically connected.

  The exposure control processing unit 20 is configured by an aperture control mechanism or the like, and controls the aperture 3 by a control signal from the control device 19 while receiving feedback such as an output signal of the evaluation value calculation unit 18.

  The output characteristic control circuit 21 determines the ratio of the linear region and logarithmic region (the position of the inflection point) of the output signal of the image sensor 4. This inflection point information is output to the output characteristic determination unit 11 b and also output to the linearization unit 16 via the control device 19. As described above, the ratio between the linear region and the logarithmic region can be changed by adjusting the voltage (output characteristic control voltage) applied to each pixel G11 to Gmn of the image sensor 4 and setting the register for output characteristic control. ing.

  The timing generation circuit 22 controls the shooting operation of the image sensor 4. That is, a predetermined timing pulse (a pixel drive signal, a horizontal synchronization signal, a vertical synchronization signal, a horizontal scanning circuit drive signal, a vertical scanning circuit drive signal, etc.) is generated on the basis of the imaging control signal from the control device 19 and the image sensor 4. To output. The timing generation circuit 22 is also configured to generate a timing signal for AD conversion.

  In the element configuration shown in FIG. 3, the transistors T2 and T3 have a switching function for reading. It is the characteristics of the transistor T1 that determine the photoelectric conversion characteristics. In the analog operation range of the transistor, an inflection point th, a change in the position of H moving from the linear region to the logarithmic region, and the inclination and coefficient of the logarithmic region vary from one imaging device to another.

  FIG. 5 shows the output characteristics with respect to the amount of incident light, and shows that the linear region and the logarithmic region vary for each photoelectric conversion element. The logarithmic slopes c1, c2, c3, the inflection point position d1, This shows that d2 and d3 vary. Further, since the characteristics of the transistor fluctuate with temperature, these values fluctuate depending on the environment and cause noise.

<Outline of the present invention>
In the imaging apparatus according to the present invention, these variation characteristics are acquired in an adjustment / calibration process, stored in a memory or the like, and not used for correction. Image data with varying brightness of each pixel is collected and stored, and variation characteristics are measured using the stored image data. A good image is obtained by performing correction using the measured variation characteristic. It should be noted that the measurement and correction of the variation characteristic according to the present invention is mainly executed in the variation correction unit 14 in FIG.

<Device operation>
<Measurement of variation characteristics>
The image sensor 4 described with reference to FIG. 2 outputs an image by an operation as shown in FIG. 6 in a normal operation.

  That is, first, photoelectric conversion is performed in each pixel to accumulate charges, and after a predetermined accumulation time T, the accumulated charges are read as a voltage. One block shown as the normal accumulation operation in FIG. 6 represents one frame, and the accumulation time T (first accumulation time) corresponds to 1/30 second. Further, the read time in one block (frame) shown as the read operation corresponds to 1/30 second.

  Thus, the charges of all the pixels accumulated in 1/30 seconds are read out in the next 1/30 seconds to generate an image. As a result, an image of 30 images / second is generated and used for recording and displaying as a moving image.

  The accumulation time T at this time is 1/30 seconds as the maximum (fastest) and is usually determined by exposure control. However, since the image sensor 4 has a wider dynamic range than a general image sensor, 1/30 The image is captured by controlling the fixed seconds.

  During such normal operation, an accumulation operation for measuring variation characteristics is performed using a time of one frame.

  Specifically, an accumulation operation for measuring variation characteristics is performed at a timing as shown in FIG. That is, during the normal operation, for example, at a rate of 1 frame per 15 frames or 1 frame per 30 frames, the accumulation time (T / N) (second accumulation time) is shorter than the normal accumulation time T. Accumulate. This is referred to as a measurement accumulation operation.

  Here, since the image data obtained by the accumulation operation at the time of measurement is used as a set of two pieces of image data obtained by the normal operation immediately after the measurement, the accumulation operation at the time of measurement is performed immediately before, for example, as shown in FIG. It is conceivable to perform in a time zone close to the end of the reading operation of the image data stored in the normal operation. Further, the accumulation operation at the time of measurement may be performed in a time zone close to the start of the reading operation of the image data accumulated in the immediately preceding normal operation as shown in FIG.

  This set of two sets of image data can be regarded as image data obtained at very short intervals of input time, and the incident light quantity (light quantity × accumulation time) has become 1 / N in almost the same scene. It can be understood as image data. That is, the output of each pixel obtained by the accumulation operation during measurement indicates 1 / N output of the output of each pixel obtained by the normal accumulation operation.

  FIG. 9 is a diagram showing the output obtained in this way on the photoelectric conversion characteristics. In the following, based on these characteristics, the variation characteristic detection method will be described with the simplest model.

  In FIG. 9, three different photoelectric conversion characteristics are shown, and each characteristic is estimated to correct variations. In addition, the three photoelectric conversion characteristics shown in FIG. 9 are characteristics in which the positions of the inflection points that change from the linear area to the log area are different and the inclination of the log area is different. In the three photoelectric conversion characteristics, a characteristic having a log area represented by y = c1Lnx + d1 is a characteristic P, a characteristic having a log area represented by y = c2Lnx + d2, a characteristic Q, and a log area represented by y = c3Lnx + d3 The characteristic possessed is called characteristic R. Further, the linear regions of the three photoelectric conversion characteristics are expressed by a common formula y = ax.

  In the imaging of a scene that gives a certain cell surface illuminance α, the log region outputs A1, A2, and A3 can be obtained with three photoelectric conversion characteristics during normal operation.

  On the other hand, the outputs of sensors having three different photoelectric conversion characteristics as outputs when the integration time is set to 1 / N in the accumulation operation for measuring the variation characteristics are the same A in the linear region. The ratio of the amount of light received by the sensor during integration with different two integration times is a ratio of N: 1.

  Next, when time has passed and transition is made to imaging of a scene that gives uniform cell surface illuminance β, during normal operation in this scene, the output of the log areas B1, B2, and B3 is obtained with three photoelectric conversion characteristics, respectively. It is done. If the integration time in a scene that can be assumed to be in the same state as the scene from which this output is obtained is 1 / N, the outputs of sensors having three different photoelectric conversion characteristics are the same B in the linear region.

  In this way, the characteristics of each pixel can be analyzed when the output data of two points in the linear region and two points in the log region corresponding to the linear region are aligned.

  First, the illuminance ratio of α and β can be calculated from the ratio of A: B. Further, since the incident light quantity ratio during the variation measurement operation and the normal operation is 1: N, all points can be plotted with the ratio of each measurement point based on α × T / N.

  Here, the two points in the log area, A1 and B1, are represented by the following formulas (1) and (2), respectively.

  From the above formulas (1) and (2), c1 and d1 of the characteristic P are expressed by the following formulas (3) and (4), respectively.

  Similarly, c2 and d2 of the characteristic Q are expressed by the following formulas (5) and (6), respectively.

  Also, c3 and d3 of the characteristic R are expressed by the following formulas (7) and (8), respectively.

  Also, the output value at point A is originally a value multiplied by N, and is equal to the value obtained by multiplying the illuminance α by the accumulation time T, so the following formula (9) is derived.

  As described above, the unknown coefficient in the photoelectric conversion characteristic of each pixel can be calculated. As a result, the inflection point position that is the intersection of the straight line y = ax representing the linear region and each log characteristic curve can also be calculated.

  Here, even if the linear region has an offset component b and the linear region is expressed by the equation y = ax + b, the third data includes two scenes in which the variation measurement operation and the normal operation are both linear regions. If the output values p and q are input, b can be calculated, and based on this, the characteristics of each pixel can be calculated from other similar scene data.

  That is, b can be calculated by the following mathematical formulas (10) to (14).

  The calculation of the unknown coefficient of each log characteristic curve and the inflection point position that is the intersection of each log characteristic curve is as described above.

  Further, in the above description, for simplification, the description has been made on the assumption that each of the two scenes is given a uniform illuminance on the sensor surface. However, as can be understood from the above-described characteristic coefficient calculation process, If the characteristic is an illuminance state where the characteristic shifts from the log area to the linear area with an integration time of 1 / N, two points of the linear area obtained by the continuous operation of the dispersion measuring operation and the normal operation, and It is possible to grasp the output and the illuminance relationship between the two log areas related to each other, and to calculate the unknown coefficient of each log characteristic curve and the position of the inflection point that is the intersection of the linear characteristic and the log characteristic curve.

  Therefore, the operation is continued with a measurement operation for different integration times, i.e., 1 / N integration time variations, during normal operation. During this time, for each pixel, a set of output values that are assumed to be a log area for normal operation and a set of output values that are assumed to be linear characteristics during the variation measurement operation are stored.

  When the scene changes, the output value of each pixel fluctuates, and data of different output value sets, that is, output values under different illuminances are obtained. This is repeated to determine whether or not the characteristic coefficient can be calculated for all pixels. If the output value data set is not sufficient, the combination of the normal operation and the variation measuring operation is repeated.

  In this case, two data sets are required at a minimum, but the minimum number of sets is two, and the characteristic coefficients such as the slope of the linear area, the offset, the slope of the log area, and the offset are calculated based on more data sets. It is more accurate to calculate.

  Therefore, data sets up to the limit allowed by the memory may be left and statistically obtained from these data sets.

  In the above description, the characteristics of the sensor are switched at the inflection point between the characteristics of the linear region and the log region, and the inflection point varies from pixel to pixel. However, the application of the present invention is limited to this. Similarly, if the change is to a characteristic that is similarly modeled by two coefficients, the number of unknowns is also two, so that a solution can be obtained mathematically.

  For example, the characteristic coefficient can be calculated in the same manner with photoelectric conversion characteristics having two types of linear regions with different inclinations.

  A data set accumulation method for calculating the characteristic coefficient of the photoelectric conversion characteristic described above will be described with reference to the flowchart shown in FIG.

  As shown in FIG. 10, when the characteristic measurement routine is started, the count of the number of data (output value) sets is initialized in step S1. Here, the data of each pixel is represented by T (i, j), and i and j represent the X and Y coordinates of each pixel.

  Next, normal operation for 14 frames is performed (step S2).

  Next, a measurement accumulation operation for variation measurement is performed for a time of 1 / N of the normal accumulation time (step S3).

  Data of all pixels obtained up to this stage is stored in the output memory OUT2 (step S4). The maximum value of the X coordinate is represented as Xmax, the maximum value of the Y coordinate is represented as Ymax, and pixel data of X = 1 to Xmax and Y = 1 to Ymax are stored. The output memory OUT2 is included in the variation correction data memory 12 described with reference to FIG.

  As for the output, in an ordinary image system, an output of one frame is supplied at equal intervals in 1/30 seconds. In order to adapt to such a system, a normal operation of 15 frames is performed with an integration time of 1/32 seconds, and 1 / N integration for variation measurement is included once. For this reason, a normal operation of 30 frames is performed in one second, and it is possible to perform the same function as a normal image system by outputting this at equal intervals. This operation can be realized by providing an extra memory having a storage capacity for one screen, and can be realized very easily in a digital image data system.

  Next, in step S5, a normal operation (for one frame) immediately after the measurement accumulation operation is performed. As shown in steps S6 to S9, pixel data (output values) defined by the X and Y coordinates are changed to X, The initial value of the Y coordinate is stored in the output memory OUT1 (step S10). The output memory OUT1 is included in the variation correction data memory 12 described with reference to FIG.

  Next, the pixel data obtained in the normal operation immediately after the measurement accumulation operation stored in the output memory OUT1 is compared with a predetermined threshold th2. If the pixel data is larger, It is determined that the data is measured in the log area, and if it is equal to or less than the threshold th2, it is determined that the data is measured in the linear area (step S11). In this case, the process proceeds to step S15.

  On the other hand, if it is determined that the measurement is performed in the log area, the pixel data obtained by the accumulation operation during measurement stored in the output memory OUT2, the same X and Y coordinates as the pixel in which the data was stored in step S10 are stored. The pixel data is read (step S12) and compared with a predetermined threshold value th1. If the pixel data is smaller, it is determined that the data is measured in the linear region, and the threshold value th1 or higher. If there is, it is determined that it is measured in the log area (step S13). In this case, the process proceeds to step S15.

  Note that the threshold value th2 in step S11 and the threshold value th1 in step S13 may be the same value, or may be different values.

  On the other hand, when it is determined that the measurement is performed in the linear region, the pixel data determined to be measured in the log region in step S11 is set as a set and compared with the data set that has already been stored ( Step S14). If the comparison reveals that the data set is different from the previously stored data set, the data set is stored as a new data set, and the integration time ratio N and the number of stored sets T (i, j) are incremented by one (T ( i, j) = T (i, j) +1) and store.

  Next, in step S15, it is determined whether or not the coordinate has been changed until the X coordinate (i) reaches the maximum value (Xmax). If the coordinate has reached the maximum value, the X coordinate (i) is set to the initial value. (Step S16). On the other hand, if the maximum value has not been reached, the process proceeds to step S9, the X coordinate (i) is incremented by 1, and the operations in and after step S10 are repeated.

  Next, in step S17, it is determined whether or not the coordinate has been changed until the Y coordinate (j) reaches the maximum value (Ymax). If the maximum value has not been reached, the process proceeds to step S8, where the Y coordinate (j ) Is incremented by 1, and the operations in and after step S9 are repeated. On the other hand, if the maximum value has been reached, it is determined whether or not the number of stored data sets (T (i, j)) is two or more (step S18). If there are two sets or more, the process proceeds to the characteristic estimation routine (step S19), and the unknown coefficient of the log characteristic curve and the unknown of the straight line representing the linear region are processed through the processes described using the equations (1) to (14). An inflection point position that is an intersection of a coefficient, an offset value, a log characteristic curve, and a straight line representing a linear region is calculated, and photoelectric conversion characteristics are estimated.

  Based on the photoelectric conversion characteristics obtained in this manner, the variation correction unit 14 shown in FIG. 1 corrects output variations among pixels. Note that the method disclosed in Japanese Patent Laid-Open No. 2008-28623 may be used as a method for correcting variation.

<Effect>
As described above, during the normal operation, the accumulation operation at the time of measurement with the accumulation time (T / N) shorter than the normal accumulation time T is performed, and the accumulation operation at the time of measurement and the normal operation immediately after that are performed. The photoelectric conversion characteristics are estimated based on one set of two sets of image data obtained by the above, and the pixel output is corrected using the estimated photoelectric conversion characteristics. Initial adjustment such as measurement of characteristics is unnecessary, and the manufacturing cost can be reduced.

  In the above description, the variation measuring operation at the time of imaging a general scene has been described. However, it is needless to say that the same idea can be used in a calibration environment or a prior data collection environment.

  In the case of a calibration environment, it is necessary to calculate the photoelectric conversion characteristics by setting a lot of subject brightness and inputting data. However, the brightness conditions of the environment that must be set by using the present invention are greatly increased. The data can be collected at high speed.

<Modification 1>
In the above description, the integration time ratio N is a fixed value, but it is not necessary to be a fixed value. A plurality of known values may be changed and used in a predetermined cycle.

  In the variation measurement operation, for example, three values may be sequentially repeated so that the value of N becomes 16, 64, and 256, and data for variation measurement may be acquired within the integration time according to each N. .

  Even when such a method is adopted, the characteristic coefficient can be calculated by recording the integration time ratio N as described in step S14 of FIG.

  When the integration time ratio N is increased, the time required for the accumulation operation during measurement is shortened. Therefore, it is suitable for imaging a bright scene, but is not suitable for imaging a dark scene, and the integration time ratio N is fixed to a large value. If this is the case, it will not be possible to deal with the case where the brightness of the imaging scene changes frequently. However, such a case can be dealt with by changing the integration time ratio N as needed.

<Modification 2>
One possible cause of variations in photoelectric conversion characteristics in a linear log conversion sensor is a change in element characteristics due to a temperature change in a semiconductor element constituting the sensor. When a temperature change occurs, the value of the measured photoelectric conversion characteristic coefficient also changes, so that the correction effect is lost.

  Therefore, if a temperature sensor is provided in the system, preferably on the sensor, and a temperature change of a predetermined temperature or more has occurred from the temperature at the previous calculation of the characteristic coefficient, a data acquisition and characteristic estimation routine for calculating a new characteristic coefficient is provided. What is necessary is just to set it as the structure to start.

  As a result, an appropriate characteristic coefficient can always be calculated from the in-situ data without any prior data acquisition under any temperature environment, and a system with appropriate characteristic correction can be constructed.

DESCRIPTION OF SYMBOLS 1 Imaging device 4 Image sensor 14 Variation correction part

Claims (5)

An image sensor that has a plurality of pixels composed of photoelectric conversion elements and converts incident light into an electrical signal by photoelectric conversion;
A variation correction unit that corrects variations in output among the plurality of pixels, and an imaging device comprising:
The variation correction unit
A first output that is output based on the charge accumulated in the first accumulation time in the imaging device;
Receiving a second output that is output based on the charge accumulated in the second accumulation time shorter than the first accumulation time in the imaging element;
From the first and second outputs for each pixel, data on photoelectric conversion characteristics representing an output with respect to an incident light amount is acquired,
A photoelectric conversion characteristic in each pixel is estimated from a ratio of the first and second accumulation times for the obtained plurality of pixels and an output value in the first and second accumulation times, and the estimated photoelectric An imaging apparatus that corrects variations among the plurality of pixels based on conversion characteristics.   The photoelectric conversion characteristic has a characteristic that a plurality of characteristics continuously change at an inflection point, and is calculated based on data on the photoelectric conversion characteristic, an unknown coefficient of the photoelectric conversion characteristic, an offset value The imaging apparatus according to claim 1, wherein the imaging apparatus is estimated using the position of the inflection point. The first accumulation time corresponds to an accumulation time for one frame of image data,
The imaging apparatus according to claim 1, wherein only the first output is used for generating image data.   The imaging apparatus according to claim 1, wherein a plurality of types of ratios of the second accumulation time with respect to the first accumulation time are set and changed in a predetermined cycle. The imaging device has a temperature sensor;
The imaging according to claim 1, wherein when the temperature change is detected, the estimated photoelectric conversion characteristic is discarded, and the photoelectric conversion characteristic at each pixel is estimated based on the data on the photoelectric conversion characteristic acquired again. apparatus.

Images