JP2012231324A - Imaging apparatus and control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately perform subject tracking when subject tracking is performed based on data used for exposure control.
An image sensor that outputs a signal corresponding to a captured subject image, an exposure control unit that performs exposure control using image data based on a signal output from the image sensor, and a plurality of sensors used for the exposure control Tracking means that performs tracking of a target that satisfies a predetermined condition using image data, and correction means that performs correction so that values relating to the luminance of each of the plurality of image data used for tracking are approximated. When the correction is performed, the means performs the tracking based on the corrected result.
[Selection] Figure 4

Description

  The present invention relates to an imaging apparatus that performs subject tracking based on data used for exposure control.

  2. Description of the Related Art Conventionally, there is known an imaging apparatus having a so-called subject tracking function that recognizes where a main subject is in an imaging screen and automatically tracks its position. By using this subject tracking function, it is possible to continuously perform automatic focus adjustment control (AF control) and exposure control (AE control) on a moving main subject.

  As an example of an imaging apparatus having such a subject tracking function, in an imaging apparatus such as a single-lens reflex camera having a photometry unit used for photometry separately from an image sensor, subject tracking is performed based on photometric data output from the photometry unit. Japanese Patent Application Laid-Open No. H10-228707 describes that.

JP-A-7-110429

  However, when subject tracking is performed based on photometric data output from the photometric unit as in the imaging device described in Patent Document 1, the following problems occur.

  Since the photometric data output from the photometric unit is also used for exposure control, the accumulation time of the photometric unit is controlled with priority given to the output of photometric data suitable for exposure control. For this reason, when two photometric data are subjected to correlation calculation in the subject tracking process, if the luminance values indicated by the two photometric data are greatly different, the accuracy of the correlation calculation is reduced, and subject tracking cannot be performed. The accuracy of the will decrease.

  Therefore, an object of the present invention is to enable accurate subject tracking when subject tracking is performed based on data used for exposure control.

  In order to achieve the above object, an imaging apparatus according to the present invention performs exposure control using an imaging device that outputs a signal corresponding to a captured subject image and image data based on the signal output from the imaging device. Exposure control means, tracking means for tracking a target that satisfies a predetermined condition using a plurality of image data used for the exposure control, and correction so that values related to the respective luminances of the plurality of image data used for tracking are approximated Correction means for performing, when the correction is performed, the tracking means performs the tracking based on the corrected result.

  In order to achieve the above object, a method for controlling an imaging apparatus according to the present invention is a method for controlling an imaging apparatus having an imaging element that outputs a signal corresponding to a captured subject image, and is output from the imaging element. An exposure control step for performing exposure control using image data based on the generated signal, a tracking step for tracking a target that satisfies a predetermined condition using a plurality of image data used for exposure control in the exposure control step, And a correction step for performing correction so that the values related to the brightness of each of the plurality of image data used for tracking in the step approach, and the tracking step is corrected when the correction in the correction step is performed. The tracking is performed based on the result obtained.

  According to the present invention, when subject tracking is performed based on data used for exposure control, subject tracking can be performed with high accuracy.

1 is a block diagram showing a configuration of an imaging apparatus according to an embodiment of the present invention. 6 is a timing chart showing a continuous shooting sequence. The timing chart shown in detail of AE processing and tracking processing. The figure which shows the detail of AE process and a tracking process. The figure which shows the rearrangement process of data. The figure which shows the conversion characteristic at the time of converting the luminance value based on the output of the image pick-up element before performing a development process into the luminance value as an image. The figure which shows the correction table for determining the correction amount used for a correction process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram of an imaging apparatus according to an embodiment of the present invention. The imaging apparatus includes a camera body 100 and a lens unit 200 that is an interchangeable lens that can be attached to and detached from the camera body 100.

  The configuration of the camera body 100 will be described. The control unit 101 comprehensively controls various arithmetic processes and the entire imaging apparatus.

  The mirror 102 guides the subject image that has passed through the photographing lens 202 of the lens unit 200 to the finder and the image sensor B116 (first image sensor) for exposure control when using the finder, and jumps up the subject image to the image sensor A (first image sensor) when photographing. 2 imaging device) 106. Hereinafter, regarding the state of the mirror 102, the state in which the subject image that has passed through the photographing lens 202 is guided to the viewfinder and the image sensor B116 is the down state, and the state in which the subject image that has passed through the photographing lens 202 is guided to the image sensor A106. Set to the up state. In addition, an operation for setting the mirror 102 in the down state is a down operation, and an operation for setting the mirror 102 in the up state is an up operation. The mirror driving unit 103 drives the mirror 102 in accordance with a control signal from the control unit 101.

  The shutter mechanism 104 has a shutter curtain corresponding to a focal plane type front curtain / rear curtain, and controls the time during which the subject image that has passed through the photographing lens 202 is guided to the image sensor A106. The shutter drive unit 105 drives the shutter mechanism 104 in accordance with a control signal from the control unit 101.

  The image sensor A106 is a (solid) image sensor that outputs an image signal corresponding to the image formed by the photographing lens 202, and is, for example, a CCD sensor or a CMOS sensor.

  The imaging signal processing unit A107 performs various processing such as amplification processing of the imaging signal output from the imaging device A106, A / D conversion processing for converting from analog to digital, and defect correction for image data after A / D conversion. Correction processing or compression processing for compressing image data is performed. The timing generator A108 outputs various timing signals to the image sensor A106 and the image signal processor A107.

  The memory unit 109 temporarily stores image data and the like processed by the imaging signal processing unit A 107, and stores various adjustment values and programs for executing various controls by the control unit 101.

  The recording medium control interface (I / F) unit 110 is for performing recording processing of image data or the like on the recording medium 111 or reading processing of image data or the like from the recording medium 111. The recording medium 111 is a detachable recording medium composed of a semiconductor memory or the like for recording various data such as image data.

  The display driving unit 112 drives the display unit 113 that displays a captured still image, moving image, or the like in accordance with a control signal from the control unit 101.

  An external interface (I / F) unit 114 exchanges information such as image data and control signals with an external device 115 such as a computer or a printer.

  The image sensor B116 is an image sensor for acquiring data used for exposure control and light source detection, and outputs an image signal corresponding to the captured subject image. In the present embodiment, it is assumed that a CCD sensor having RGB stripe arrangement is used. Note that a CMOS sensor may be used as long as the sensor has a short readout time.

  The imaging signal processing unit B117 performs various processing such as amplification processing of the imaging signal output from the imaging element B116, A / D conversion processing for converting from analog to digital, and defect correction for image data after A / D conversion. Correction processing or compression processing for compressing image data is performed. The timing generator B118 outputs various timing signals to the image sensor B116 and the image signal processor B117.

  The pentaprism 119 guides the light beam bent by the mirror 102 to a finder (not shown) and the image sensor B116.

  The focus detection unit 121 performs focus detection by a phase difference method, and acquires a pair of images by light rays that are transmitted through and reflected by the mirror 102. The control unit 101 calculates the defocus amount of the subject from the shift amount of the pair of images, obtains the driving amount of the photographing lens 202, and transmits it to the lens control unit 201. The focus detection drive unit 120 drives the focus detection unit 121 according to a control signal from the control unit 101.

  Next, the configuration of the lens unit 200 will be described. The photographing lens 202 forms a subject image on the image sensor A106. A lens driving unit 203 drives the photographing lens 202 in accordance with a control signal from the lens control unit 201 in order to perform focus adjustment and zooming. A diaphragm mechanism 204 controls the amount of light flux that has passed through the photographing lens 202. The aperture driving unit 205 drives the aperture mechanism in accordance with a control signal from the lens control unit 201.

  The control unit 101 and the lens control unit 201 communicate with each other between the camera body 100 and the lens unit 200 to manage transmission / reception.

  FIG. 2 is a timing chart showing a continuous shooting sequence of the imaging apparatus according to the present embodiment. The main image exposure of the (N + 2) th frame ends from the start of the main image exposure of the Nth frame (N ≧ 1) in the continuous shooting. Shown until later.

  In continuous shooting, the main image exposure of the Nth frame is performed after the mirror 102 which has been in the down state is brought into the up state by the up operation until the down operation is started. That is, the image sensor A106 is accumulated to obtain the main image.

  Subsequently, during the period from the completion of the down operation of the mirror 102 to the down state until the next up operation is started, accumulation / calculation of exposure control processing (hereinafter referred to as AE processing) and subject tracking processing ( (Hereinafter referred to as tracking processing) accumulation / calculation. The tracking process is a process of calculating where the tracking target has moved from the luminance pattern and color pattern distribution of the tracking target (main subject etc.) that satisfies a predetermined condition in the imaging screen in a plurality of continuous images. is there. The accumulation here is accumulation by the image sensor B116.

  Then, the main image exposure for the (N + 1) th frame is performed after the up operation of the mirror 102 is completed and the up state is reached until the next down operation is started.

  At this time, exposure control is performed based on the result of accumulation / calculation of the AE process in the immediately previous down state to perform the main image exposure. Further, based on the results of accumulation and calculation of the tracking process in the down state immediately before the (N + 1) th frame and the down state immediately before the Nth frame, the main subject area targeted for focus adjustment in the Nth frame is tracked. The focus adjustment process for the (N + 1) th frame is performed.

  Subsequently, the accumulation / calculation of the AE process and the accumulation / calculation of the tracking process are performed again after the down operation of the mirror 102 is completed and becomes the down state until the next up operation is started. Thereafter, the main image exposure of the (N + 2) th frame is performed after the up operation of the mirror 102 is completed and the down state is started after the up state is reached. Thereafter, the above sequence is repeated.

  Next, processing related to the main image shooting will be described. When the mirror 102 is in the up state, the main image exposure, that is, the image sensor A106 is accumulated. When the accumulation is completed, the imaging signal processing unit A107 reads the accumulated imaging signal. As shown in the timing chart, the readout of the accumulated imaging signal must be completed before the accumulation of the next main image starts.

  When the readout of the imaging signal is completed, the imaging signal processing unit A107 performs development processing and compression processing on the readout imaging signal. When the development process and the compression process are completed, the recording medium control interface (I / F) unit 110 starts the recording process. In this way, the image data that has been subjected to various processes is recorded on the recording medium 111.

  In the above description, the process from accumulating the main image to the recording process has been described. As shown in FIG. 2, in parallel with the development process for the image signal of the Nth frame, the image signal processing unit A107 performs the process of the image of the N + 1th frame. The imaging signal accumulated by exposure is read out. Further, the imaging signal processing unit A107 performs development processing for the imaging signal of the (N + 1) th frame and reading of the imaging signal accumulated by the main image exposure of the (N + 2) th frame in parallel with the recording processing for the image data of the Nth frame. Multiple processes are performed in parallel.

  FIG. 3 is a timing chart showing details of the AE process and the tracking process shown in FIG.

  When the mirror 102 is in the down state, the image sensor B116 performs accumulation 1 that is accumulation for tracking. The tracking process must be completed by the tracking limit set based on the timing of the focus adjustment process in order to reflect the result of the tracking calculation in the focus adjustment. Therefore, this accumulation 1 is a minimum accumulation time during which a signal that can be used for the tracking calculation can be obtained.

  When the accumulation 1 is completed, the imaging signal processing unit B117 reads the imaging signal accumulated in the accumulation 1. When the reading is completed, the imaging signal processing unit B117 performs a development process in order to perform a tracking calculation. When the development process is completed, the imaging signal processing unit B117 performs a tracking calculation. Since the tracking operation uses a data format called YUV, development processing including conversion to the YUV format is performed. The result of the tracking calculation obtained in this way is used for the focus adjustment processing for the next main photographing.

  Further, when the accumulation 1 is completed, the image sensor B116 performs accumulation 2 which is accumulation for AE while reading the imaging signal accumulated in the accumulation 1. When the accumulation 2 is completed, the imaging signal processing unit B117 reads the imaging signal accumulated in the accumulation 2, and the image data based on the imaging signal accumulated in the accumulation 1 (first image data) and the accumulation 2 are accumulated. The arithmetic operation is performed with the image data (second image data) based on the image pickup signal.

  By performing the arithmetic mean operation of the first image data and the second image data, the flicker of the same extent as when accumulation is performed with the accumulation time of the accumulation 1 and the accumulation time of the accumulation 2 being performed. Image data with reduced influence can be obtained.

  The image data obtained by performing the averaging operation in this way corresponds to image data based on the imaging signal accumulated in the accumulation time longer than the image data based on the imaging signal accumulated in the accumulation 1 or the accumulation 2. Therefore, the influence of the flicker light source under a fluorescent lamp or the like is reduced compared to the first image data or the second image data, and exposure control is performed using image data based on the imaging signal accumulated in the accumulation 1 or accumulation 2. Exposure control can be performed more accurately than when it is performed.

  When the reading and the addition average calculation are completed, the imaging signal processing unit B117 divides the image data obtained by the addition average calculation into a plurality of areas corresponding to the imaging screen, performs an integration operation in each area, and performs integration data in each area. Is calculated. Then, the imaging signal processing unit B117 performs AE calculation using this integration result. The imaging signal processing unit B117 determines exposure control values such as a shutter speed (exposure time), an aperture value, and an ISO value (imaging sensitivity) at the time of actual image shooting by the AE calculation, and the control unit 101 controls these exposures. Based on the value, each part is controlled to perform actual image shooting.

  FIG. 4 shows details of the AE process and the tracking process shown in FIG.

  FIG. 4A shows the flow of imaging signals accumulated in the accumulation 1. The imaging signal processing unit B117 reads the imaging signal accumulated in the imaging element B116 in the accumulation 1, and then stores the image data based on the readout imaging signal in the memory unit 109 as the stripe data 1.

  FIG. 4B shows details from after reading in the tracking process to the tracking calculation. The imaging signal processing unit B117 reads the stripe data 1 stored in the memory unit 109 and performs data rearrangement processing before performing development processing. Details of the rearrangement process will be described later.

  Next, the imaging signal processing unit B117 develops the rearranged image data to generate a YUV image. Further, in parallel with the development process, the imaging signal processing unit B117 calculates integration data for the rearranged image data in the same manner as the integration operation of the AE process described above.

  The imaging signal processing unit B117 performs correction processing on YUV image data (hereinafter referred to as the current YUV image) generated based on the imaging signal accumulated in the current accumulation 1 by using the integration data. . Details of this correction processing will be described later.

  Next, the imaging signal processing unit B117 compares the current YUV image that has been corrected and the previous YUV image that has not been corrected. Then, the imaging signal processing unit B117 calculates how much the main subject area of the current YUV image is shifted in the X direction and the Y direction of the imaging screen with respect to the main subject area of the previous YUV image. The imaging signal processing unit B117 obtains where the main subject area has moved in the imaging screen using this shift amount, and performs a tracking calculation. The control unit 101 uses this tracking result to perform a focus adjustment process for the current main image shooting.

  FIG. 4C shows a flow of a weighted average calculation between the image data based on the imaging signal accumulated in the accumulation 1 and the image data based on the imaging signal accumulated in the accumulation 2. The imaging signal processing unit B117 reads the imaging signal accumulated in the imaging element B116 in the accumulation 2, and then stores the image data based on the readout imaging signal in the memory unit 109 as the stripe data 2.

  Next, the imaging signal processing unit B 117 reads out the stripe data 1 and the stripe data 2 stored in the memory unit 109, performs an addition average operation, and stores the obtained image data as the stripe data 3 in the memory unit 109. If the stripe data 1 and the stripe data 2 are simply added, the integrated data value of the stripe data 3 may be too large. Therefore, the value of the integral data of the stripe data 3 is prevented from becoming too large by performing an averaging operation. If the integrated data value of the stripe data 3 does not exceed a predetermined value, the stripe data 1 and the stripe data 2 may be simply added.

  FIG. 4D shows details from the arithmetic mean calculation to the AE calculation. First, the imaging signal processing unit B117 reads the stripe data 3 stored in the memory unit 109 and performs data rearrangement processing. The imaging signal processing unit B117 calculates integration data for the rearranged image data in the same manner as the integration operation of the AE process described above.

  The imaging signal processing unit B117 performs an AE calculation using this integral data, and determines exposure control values such as a shutter speed, an aperture value, and an ISO value used at the time of actual image capturing.

  FIG. 5 shows an example of rearrangement processing for rearranging the image data in the stripe arrangement described above into image data in the Bayer arrangement.

  In the present embodiment, since the image sensor B116 is mainly used for acquiring data used for exposure control and light source detection, a stripe-array CCD sensor is used for the image sensor B116. For this reason, the image pickup signal read from the image pickup device B116 has a stripe arrangement as shown in FIG.

  The imaging signal processing unit B117 performs image data rearrangement processing so that this imaging signal becomes a Bayer array as shown in FIG. 5B, which is a general imaging signal array. When a Bayer array image sensor is used as the image sensor B 116, it is not necessary to perform such image data rearrangement processing.

  FIG. 6 shows a conversion characteristic (generally a gamma characteristic) when converting a luminance value (pre-development luminance value) based on the output of the image sensor before development processing into a YUV image luminance value (development luminance value). Is called). As shown in FIG. 6, the luminance value before development and the luminance value after development are not in a proportional relationship.

  In the present embodiment, the tracking calculation is performed using the current YUV image and the previous YUV image, but subject tracking is performed when the luminance value of the current YUV image and the luminance value of the previous YUV image are significantly different. Cannot be performed or the accuracy of subject tracking is reduced. Specifically, when the luminance values of two YUV images are greatly different, the tracking calculation is performed because the luminance pattern and the color pattern distribution are greatly different between the previous YUV image and the current YUV image even in the same main subject region. , The same main subject region is not determined. Alternatively, in the tracking calculation, an area different from the main subject area in the current YUV image is determined as an area corresponding to the main subject area in the previous YUV image.

  In this embodiment, in order to reduce the decrease in tracking accuracy as described above, the imaging signal processing unit B117 is configured to make the luminance value of the current YUV image substantially the same as the luminance value of the previous YUV image. After the correction, the two images are compared. Since the current YUV image and the previous YUV image used for the tracking calculation are both developed images, the luminance value of the current YUV image is corrected to be equal to the luminance value of the previous YUV image. However, it does not mean that it was corrected accurately. Therefore, the imaging signal processing unit B117 determines the correction amount of the current YUV image based on the luminance value of the YUV image, that is, not the luminance value after development but the luminance value before development.

  FIG. 7 shows a correction table for determining an appropriate correction amount based on the previous pre-development luminance value and the current pre-development luminance value. Note that each luminance value in the correction table of FIG. 7 is a luminance value obtained by averaging the integration data of each region of the imaging screen. In this embodiment, in order to simplify the explanation, a luminance value obtained by simply averaging the integration data of each region is used for determining the correction amount. However, weighting according to the shooting conditions is weighted and averaged over each region. A luminance value may be used.

  The values shown in FIG. 7 are set based on the previous pre-development luminance value, the current pre-development luminance value, and the conversion characteristics described above. Therefore, the imaging signal processing unit B117 performs the correction process of the current YUV image using the correction amount determined based on FIG. 7, thereby calculating the difference between the previous pre-development luminance value and the current pre-development luminance value. An accurately corrected YUV image can be obtained.

  By performing the tracking calculation using the current YUV image corrected in this way and the previous YUV image, even if the tracking calculation is performed based on image data obtained at different accumulation times, the accuracy is improved. The subject can be tracked well.

  Even if the accumulation times are not different, if the accumulation time is shorter as the influence of flicker becomes larger, a large difference in luminance value occurs between a plurality of image data. Even in such a case, subject tracking can be performed with high accuracy by performing the tracking process using the image subjected to the correction process as described above.

  In addition, as in the above embodiment, the tracking calculation is not performed using the image data after the development processing, but is used for the tracking calculation if the tracking calculation is performed using the image data before the development processing. The correction amount of the correction process may be determined based on the difference between the luminance values of the two image data. Further, if the tracking calculation is performed using the image data before the development processing, the correction amount of the correction processing may be determined based on the difference between the exposure amounts of the two image data used for the tracking calculation.

  In the above-described embodiment, the AE calculation is performed based on the result of the arithmetic mean calculation of the image data based on the imaging signal accumulated in the accumulation 1 and the image data based on the imaging signal accumulated in the accumulation 2. However, the present invention can also be applied when the accumulation is not performed twice.

  In the above embodiment, the subject tracking at the time of continuous shooting has been described. However, the present invention can also be applied to the case where the subject tracking is performed before the main image shooting.

  Moreover, the structure which performs at least one part of the process which imaging signal process part B117 performed in said embodiment performed at least one of imaging signal process part A107 and the control part 101 may be sufficient. Alternatively, the imaging signal processing unit B117 may not be provided, and all the processing performed by the imaging signal processing unit B117 may be performed by at least one of the imaging signal processing unit A107 and the control unit 101.

  Moreover, the structure by which the lens unit was incorporated in the camera main body may be sufficient.

  Further, in the correction process, it is not necessary to correct the previous pre-development luminance value and the current pre-development luminance value so that a plurality of pieces of image data used for tracking can be obtained to such an extent that the subject tracking accuracy is sufficiently obtained. What is necessary is just to correct | amend so that the value regarding each brightness | luminance may approach. Note that the value relating to the luminance here includes a luminance value, an exposure amount, and the like.

  The correction amount of the correction process does not have to be determined using a correction table as shown in FIG. 7. The previous pre-development luminance value, the current pre-development luminance value, and conversion when converting to the YUV format You may make it calculate based on a characteristic.

DESCRIPTION OF SYMBOLS 100 Camera main body 101 Control part 106 Image sensor A
109 Memory part 116 Image sensor B
117 Imaging signal processor B
121 Focus detection unit

Claims (9)

An image sensor that outputs a signal corresponding to the captured subject image;
Exposure control means for performing exposure control using image data based on a signal output from the image sensor;
Tracking means for tracking a target that satisfies a predetermined condition using a plurality of image data used for the exposure control;
Correction means for performing correction so that the values related to the luminance of each of the plurality of image data used for tracking approach,
The tracking device, when the correction is performed, performs the tracking based on a corrected result. A second image sensor that outputs a signal corresponding to the captured subject image;
The said exposure control means performs said exposure control using the image data based on the signal output from the said image pick-up element, when imaging with the said 2nd image pick-up element. Imaging device.   The imaging apparatus according to claim 2, further comprising a focus adjustment unit that performs focus adjustment based on a tracking result of the tracking unit before imaging with the second imaging element.   4. The imaging apparatus according to claim 1, wherein the tracking unit performs the tracking using a plurality of continuous image data based on a signal output from the imaging device. 5.   5. The image pickup apparatus according to claim 1, wherein the tracking unit performs the tracking using a plurality of image data after performing a development process. 6.   The imaging apparatus according to claim 5, wherein the exposure control unit performs the exposure control using image data before the development processing.   7. The correction unit according to claim 5, wherein the correction unit corrects the image data after the development processing based on a correction amount determined using the image data before the development processing. Imaging device.   The imaging device according to claim 5, wherein the development processing includes processing for converting image data based on a signal output from the imaging device into a YUV format. A method for controlling an imaging apparatus having an imaging element that outputs a signal corresponding to a captured subject image,
An exposure control step for performing exposure control using image data based on a signal output from the image sensor;
A tracking step of tracking a target that satisfies a predetermined condition using a plurality of image data used for exposure control in the exposure control step;
A correction step for performing correction so that the values related to the brightness of each of the plurality of image data used for tracking in the step approach,
In the tracking step, when the correction in the correction step is performed, the tracking is performed based on a corrected result.

Images