JP2008289032A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire a video image capable of responding also to sharp variation of an amount of relative motion of a subject and an imaging apparatus in addition to sharp brightness variation of the subject and with less fluctuation and brightness variation. <P>SOLUTION: The imaging apparatus has: an image pickup device 103 which performs photoelectric conversion of a subject image; a video image generation means 105 for generating a video image based on an imaging signal output from the image pickup device; a gain adjustment means 104 for performing gain adjustment to the imaging signal; motion information generation means 107, 108, 601, 602 for generating motion information using at least either one of a plurality of unit images in the video image and a sensor which detects a motion of the imaging apparatus; and exposure control means 113, 114 for controlling a diaphragm value of a diaphragm 102 which adjusts a quantity of light received by the image pickup device, shutter speed of the image pickup device and a gain in the gain adjustment means based on brightness information and the motion information. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus or an imaging apparatus having an image stabilization function that suppresses shake of a captured image due to movement of a subject.

  Many imaging devices such as video cameras are equipped with a so-called AE (Auto Exposure) function. The AE function is a function for measuring brightness (luminance) information of a subject using a photometric sensor or a video signal and automatically determining an exposure value based on the brightness information.

  In an imaging apparatus equipped with an AE function, an iris (aperture) mechanism and a gain circuit are controlled according to the determined exposure value. This makes it possible to react quickly even if time fluctuations occur in the luminance of the subject, and to reduce the brightness fluctuations of the acquired video.

  However, when imaging a subject with extremely low brightness, the iris is opened, and the shutter speed is greatly reduced. In this way, when the normal AE range is exceeded, not only the adjustment of the above elements but also the gain amplification is performed in the image sensor or the signal processing unit at the subsequent stage. Conversely, when the subject brightness is too high, an ND (Neutral Density) filter is inserted (see Patent Document 1).

  In addition to the fully automated AE function, there is also a program AE that includes an aperture priority operation and a shutter priority operation. Furthermore, there is a so-called mode select function (also referred to as a scene select or image select function) in which the exposure mode is switched by selecting a use scene and a preferable exposure preset value is set (see Patent Document 2).

  As a function of capturing a subject moving at a high speed with respect to the imaging apparatus, there are a shutter priority AE mode and a sports mode that can be selected by a mode selection function.

  In the shutter priority AE mode, it is necessary for the user to estimate a shutter speed at which no image shake occurs from the movement of the subject. When the shutter speed is input, the aperture amount is adjusted in accordance with the input shutter speed and the photometric value, and the brightness variation of the image can be suppressed even if the luminance of the subject varies.

Further, in the sport mode, a shutter speed that does not cause a shake of the image assuming a fast movement of the subject is automatically set. Then, in accordance with the photometric value, the aperture amount is adjusted according to a program diagram defined so as to obtain an image with less brightness fluctuation while maintaining the set shutter speed. The program diagram is a parameter table showing combinations of aperture amounts and shutter speeds to be selected for photometric values.
JP 2001-145017 A Japanese Patent No. 3658361 JP 2001-330882 A

  On the other hand, when not only the subject but also the imaging apparatus moves, a steep change in the amount of motion occurs in addition to a sharp change in brightness information. The conventional exposure control function based only on the photometric value copes with changes in brightness information, but cannot sufficiently cope with a steep change in the amount of motion and tends to cause a shake in the image.

  In an exposure mode that sets a constant shutter speed, such as a shutter priority mode or a sports mode, the shutter speed must be set very high in order to cope with a steep change in the amount of movement. In this case, the relative movement between the imaging device and the subject is actually slow, and even if the shutter speed is set in the normal sports mode, the light quantity may be insufficient even in a situation where a sufficiently bright image can be obtained. is there. Or there is a risk that a noisy video may be obtained due to gain amplification.

  Low quality images, such as images with a lot of vibration and noise, and images with insufficient light intensity, are particularly low when viewed on a large display device with high definition. Also, when performing advanced image processing such as image stabilization processing, dynamic range processing, super-resolution processing, frame rate up-conversion processing, face recognition processing (see JP 2001-330882 A) based on the acquired video, If the video quality is low, it is difficult to perform the processing satisfactorily.

  The present invention can cope with a sudden change in the relative amount of movement of the subject and the imaging apparatus in addition to a sudden change in the brightness of the subject, and performs exposure control for obtaining an image with little shake and brightness fluctuation. An imaging device is provided.

  An imaging apparatus according to one aspect of the present invention includes an imaging device that performs photoelectric conversion on a subject image, video generation means that generates a video based on an imaging signal output from the imaging device, and a gain that performs gain adjustment on the imaging signal. Based on the brightness information and the motion information, an adjustment unit, a motion information generation unit that generates motion information using at least one of a plurality of unit images in the video and a sensor that detects the motion of the imaging device, And an exposure control means for controlling the aperture value of the diaphragm for adjusting the amount of light received by the shutter, the shutter speed of the image sensor, and the gain in the gain adjusting means.

  According to the present invention, even when the brightness of a subject or the relative amount of movement of a subject and an imaging device changes abruptly, it is possible to acquire a high-quality video with little shake and brightness fluctuation.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows the configuration of a video camera as an image pickup apparatus that is Embodiment 1 of the present invention.

  Reference numeral 101 denotes a lens optical system. The lens optical system 101 includes a plurality of lenses and forms an optical image of a subject. Reference numeral 102 denotes an iris (aperture). The iris 102 is a light amount adjustment unit that mechanically adjusts the amount of light that the image sensor 103 receives on its light receiving surface. The lens optical system 101 and the iris 102 constitute an imaging optical system.

  Reference numeral 103 denotes an image sensor. The image sensor 103 is composed of a CCD sensor or a CMOS sensor, is formed by the lens optical system 101, and photoelectrically converts an object image whose light amount is adjusted by the iris 102 to generate an image signal.

  Reference numeral 114 denotes a system control unit that constitutes a part of the exposure control means. The system control unit 114 is configured by a microcomputer, and comprehensively controls the entire imaging apparatus.

  Reference numeral 104 denotes an AGC (auto gain control) unit as a gain adjusting means. The AGC unit 104 is generated by the imaging device 103 by changing the gain of an amplifier included in at least one of the imaging device 103 and a camera signal processing unit described later in accordance with a gain control signal from the system control unit 114. Gain adjustment is performed on the imaging signal.

  Reference numeral 105 denotes a camera signal processing unit as video generation means. The camera signal processing unit 105 performs various video processes on the gain-adjusted imaging signal from the AGC unit 104 to generate a video signal. Video processing includes, for example, color difference / brightness separation, sharpening, white balance adjustment, black level adjustment, and code / combination processing. In addition, as described above, the camera signal processing unit 105 may include an amplifier. In this case, a portion of the camera signal processing unit 105 subsequent to the amplifier corresponds to a video generation unit.

  Reference numeral 106 denotes a work memory. The work memory 106 stores an arbitrary frame image of the video generated by the camera signal processing unit 105 in order to calculate a motion vector described later.

  In the present embodiment, the unit image constituting the video is described as a frame image, but the unit image may be a field image constituting the frame image. When a field image is used as a unit image, basically, a frame in the following description may be read as a field image.

  Reference numeral 107 denotes an image motion detection unit. The image motion detection unit 107 calculates a motion vector between a frame image (hereinafter simply referred to as a frame) stored in the work memory 106 and another frame, for example, the latest frame (current frame).

  Reference numeral 108 denotes a motion amount calculation unit that constitutes a motion information generation unit together with the image motion detection unit 107. The motion amount calculation unit 108 calculates a motion amount (motion information) based on the motion vector calculated by the image motion detection unit 107.

  Reference numeral 109 denotes a zoom / focus adjustment unit. The zoom / focus adjustment unit 109 moves the zoom lens and the focus lens included in the lens optical system 101 according to the zoom control signal and the focus control signal supplied from the system control unit 114, respectively. Further, the zoom / focus adjustment unit 109 obtains the zoom amount (zoom lens position) and the focus lens position from the output value of the zoom and focus encoder (not shown) built in the lens optical system 101, and sends it to the system control unit 114. send.

  Reference numeral 110 denotes an iris adjustment unit. The iris adjustment unit 110 drives the iris 102 according to the iris control signal supplied from the system control unit 114 and adjusts the amount of light passing through the aperture opening.

  Reference numeral 111 denotes an image sensor control unit. The image sensor control unit 111 outputs charge accumulation, readout, and reset timing pulses to the image sensor 103 to control basic operations of the image sensor 103. Further, according to the shutter speed control signal from the system control unit 114, the timing pulse output interval is controlled to control the charge accumulation time of the image sensor 103, that is, the electronic shutter speed.

  Reference numeral 112 denotes a photometry unit as photometry means. The photometry unit 112 integrates the signal output from the AGC unit 104 to obtain the average light amount, thereby outputting a photometric value as brightness information for exposure calculation. That is, brightness information is generated. The photometry unit 112 performs mask processing and weighting processing on the image area at the time of integration, specifies a recording area in the entire image, or divides the entire image into a plurality of areas and performs weighted integration on each area. .

  In this embodiment, TTL (Through The Lens) method is used to perform photometry using light that has passed through the imaging optical system, but an external photometry method that performs photometry using light that does not pass through the imaging optical system is employed. May be.

  Reference numeral 113 denotes an AE calculation unit that constitutes an exposure control unit together with the system control unit 114. The AE calculation unit 113 receives the motion amount calculated by the motion amount calculation unit 108 and the photometric value output from the photometry unit 112. The AE calculation unit 113 uses the iris value of the iris 102, the electronic shutter speed (hereinafter simply referred to as the shutter speed) of the image sensor 103, and the AGC unit 104 (or the camera signal processing unit 105) based on the amount of movement and the photometric value. To control the gain. Then, these exposure control values are output to the system control unit 114.

  Reference numeral 115 denotes an operation signal input unit. The operation signal input unit 115 includes buttons that are operated by the user to select functions of the imaging apparatus and perform various settings. It also includes an imaging start / end button. Note that the operation signal input unit 115 may be integrated with a display unit 116 described later using a touch panel display element.

  In accordance with the exposure mode selected by the operation signal input unit 115, the system control unit 114 controls each processing unit and each control unit in accordance with a parameter table read from a nonvolatile memory 118 described later. And send information. Specifically, a control signal for controlling the characteristics of the camera signal processing unit 105, information on the motion amount detection area for the image motion detection unit 107, and information on the photometry area for the AE calculation unit 113 are transmitted. Further, the system control unit 114 outputs control signals to the zoom / focus adjustment unit 109, the iris adjustment unit 110, and the image sensor control unit 111.

  Reference numeral 116 denotes a display unit. The display unit 116 includes an LCD, LED, EL, and the like, and displays various operation mode setting displays, various warning displays, video data acquired by imaging, recorded video data that has been decompressed, and the like.

  Reference numeral 117 denotes a recording unit. The recording unit 117 records video data (compressed data) on a recording medium such as a semiconductor memory, a magnetic tape, and an optical disk, and reads video data from the recording medium.

  Reference numeral 118 denotes a nonvolatile memory. The nonvolatile memory 118 stores various data such as an imaging mode item and exposure control photometric area information corresponding thereto. In addition, a data table representing a basic control guideline of an exposure control element consisting of a code diagram is also stored.

  When a certain imaging mode is selected in the operation signal input unit 115, the corresponding various data and control pointer table are read from the nonvolatile memory 118 by the system control unit 114. Generally, a parameter table corresponding to a program exposure mode called a shutter priority mode, an aperture priority mode, a sport mode, or a landscape mode is read.

  Reference numeral 119 denotes an external I / F. When at least one of the aperture value, shutter speed, and gain is outside the adjustment range (out of the controllable range) in the exposure control based on the photometric value, the system control unit 114 notifies that via the external I / F 119. Outputs information to notify the outside.

  Although FIG. 1 shows the configuration of a lens-integrated video camera, an interchangeable-lens video camera is also included in the embodiment of the present invention. In the case of an interchangeable lens video camera, the lens optical system 101, the iris 102, the zoom / focus adjustment unit 109, and the iris adjustment unit 110 in FIG. 1 are mounted on the interchangeable lens.

  FIG. 2 shows a processing procedure of exposure control in the present embodiment. This processing is executed according to a computer program (imaging processing program) stored in a memory (not shown). In the following description, “S” indicates a step.

  In step S201, the system control unit 114 confirms a processing start instruction (ON). If the process start instruction has not been issued (OFF), this process ends.

  In S202, the system control unit 114 confirms the exposure mode. Then, various data related to calculation of a photometric value or motion information corresponding to the exposure mode, for example, photometric area information and motion amount calculation area information are read from the nonvolatile memory 118. Further, the above-described control guide table information is read.

  FIG. 11 shows a setting example of the motion amount calculation area in the imaging area (full screen). In this figure, a motion amount calculation area MA is set in the central portion of the imaging area IA, and motion vectors in this area MA are preferentially detected and used for motion amount calculation. Such a setting is referred to as “central part emphasis area setting”. The movement amount calculation area MA and the photometry area may be set to be the same.

  In step S <b> 203, the system control unit 114 causes the image motion detection unit 107 and the motion amount calculation unit 108 to calculate a motion vector and a motion amount. Specifically, the motion vector is calculated between the frame stored in the work memory 106 (stored frame) and the current frame output most recently from the camera signal processing unit 105 (adjacent frame immediately after the stored frame). Let it be done. For the calculation of the motion vector, a matching process based on block matching, cross-correlation calculation, gradient method or the like is used. The plurality of frames used for calculating the motion vector may not be two adjacent frames as described above.

  FIG. 10 shows an example of block matching. Here, the left image 1001 is a reference image, and the right image 1002 is a search image. First, a template 1003 as a partial area of a predetermined size is set around one of the attention points 1004 arranged in a grid in the reference image 1001.

  Next, an arbitrary search area 1007 is set in the search image 1002, and a position that most closely matches the template 1003 is searched while sequentially moving the search area 1007. Specifically, the similarity with the template 1003 in the region 1006 with the target pixel 1005 in the search image 1002 as a reference is calculated. As an index of similarity, SSD (Sum of Square Difference), SAD (Sum of Absolute Difference), normal cross-correlation, or the like is used. As a result of calculating the similarity of each region 1006 in the search region 1007, the region 1006 with the highest similarity is regarded as the corresponding region. Then, a motion vector is calculated between the template 1003 and the corresponding region. If there is no occlusion, motion vectors are calculated for the number of attention points 1004 set on the reference image 1001.

  By the above matching process, motion vectors in a plurality of regions in the motion amount calculation area MA are calculated, and the motion amount is calculated from the motion vectors.

  The amount of motion is an index that represents the relationship between shake and shutter speed caused by relative motion between the video camera and the subject. Therefore, any value can be used as the amount of motion as long as it corresponds to the relationship. In this embodiment, as an example, the maximum amount between frames in the motion amount calculation area set by the exposure mode is used. The movement amount is [pixel / frame]. The greater the movement of the image between frames, the greater the value of motion.

  In S204, the system control unit 114 causes the photometry unit 112 to calculate a photometry value as brightness information. The photometry unit 112 takes in the output signal from the AGC unit 104 and integrates the output signal (luminance level). Then, an exposure value (photometric value) Ev corresponding to the integration result is output.

  The exposure value Ev is used to determine the relationship between the aperture value, shutter speed, and gain based on the photometric value. The exposure value Ev is also called an apex system. Ev = 0 is defined as an appropriate exposure value when the aperture value is F1, the shutter speed is 1 second, and the gain is standard (for example, 0 dB).

  In order to simplify the calculation, Av and Tv are associated with the aperture value and the shutter speed, and Sv is associated with the gain. When F is an aperture value (F value) and T is a shutter speed (seconds), the following relationship is established.

  ISO represents ISO sensitivity, and Sv represents an index of sensitivity based on ISO sensitivity 100. Sv is associated with a log having an amplification factor of 2 on the basis of sensitivity when Ev = 0 is defined. As a result, the sum of the Av value and the Tv value becomes the Ev value.

  Tables 1 and 2 show the relationship between Av value and F and the relationship between Tv value and shutter speed T, respectively.

  A code diagram as shown in FIG. 6 is used as a method for indicating the relationship between the exposure value and the aperture and shutter speed.

  In the drawing, the vertical axis on the left represents the aperture value F, and the horizontal axis on the lower side represents the shutter speed T. The parameter “−” on each axis is a value that cannot be set. The axis from the upper side to the right side represents the Ev value.

  The thick line graph in the code diagram corresponds to the control policy based on the exposure mode set by the parameter table, and is read when the exposure mode is confirmed (S202) as described above. The aperture value and the shutter speed are set so as to have values on the graph as much as possible.

  FIG. 6 shows a graph in the basic exposure mode in moving image capturing. As indicated by a circled point 1, the shutter speed T = 1/60 is maintained as much as possible until the aperture value reaches a maximum value (F = 22) that can be set. After the aperture value reaches the maximum value that can be set, the shutter speed is increased. The gain does not appear on the graph.

  Setting Sv to +1 is equivalent to setting Av and Tv to +1, respectively. For example, when the photometric value calculated in S204 is Ev = 10, according to this code diagram, F = 4 and T = 1/60. The gain is 0 dB.

  Returning to FIG. 2, in S205, the system control unit 114 causes the AE calculation unit 113 to set Tv, Av, and Sv. The AE calculator 113 calculates Tv, Av, and Sv values corresponding to the aperture value, shutter speed, and gain based on the amount of motion and the photometric value.

  FIG. 3 shows a procedure for calculating values of Tv, Av, and Sv based on the motion amount and the photometric value according to the exposure mode.

In S301, the AE calculation unit 113 calculates Tv _min based on the motion amount. The maximum shutter speed T at which the amount of motion within the exposure time (1 / shutter speed) of one frame of the video signal does not exceed the maximum motion amount threshold MV _max is assigned to Tv _min . A value such as 1/10 pixel is set in MV _max . The MV _max can be changed by an operation in the operation signal input unit 115.

  As a result, for example, as shown in FIG. 7A, a parameter area that cannot be used for the shutter speed is set in the parameter space of the exposure control element. In FIG. 7A, the left half rectangular hatched area is an unusable parameter area.

  In S302, the AE calculation unit 113 checks the exposure mode corresponding to the calculation of the exposure control parameter in the next S303. Then, it is confirmed that the parameter table corresponding to the exposure mode is read. If the parameter table does not correspond to the exposure mode, the parameter table is read from the nonvolatile memory 118.

  In S303, values of Tv, Av, and Sv are calculated from the exposure value Ev. The control parameter is determined according to the control policy read from the nonvolatile memory 118.

In S304, the AE calculation unit 113 confirms that the shutter speed (Tv) does not fall below the minimum shutter speed Tv _min that does not cause shake.

The control parameters Tv, Av, Sv are set independently of the minimum shutter speed Tv _min . For this reason, Tv may exceed Tv _min as shown in FIG. 7B, but Tv may fall below Tv _min as shown in FIG. 7C.

When Tv exceeds Tv _min , the calculated control parameter set is used as it is. When Tv is lower, the minimum shutter speed Tv _min that does not cause shake is adopted, and other control parameters are adjusted.

In S305, when Tv is lower than Tv _min , Tv = Tv _min is set, and the shutter speed is set to a shutter speed that does not cause shake.

In S306, Av and Sv are corrected along with the replacement of Tv. For example, when Tv _min = 9 (1/500 seconds), Av is adjusted as indicated by an arrow in FIG. 7C. If the aperture value cannot be changed with aperture priority, Tv is changed as shown by the arrow in FIG. 7D. In either case, this corresponds to an increase of two stages of gain (Sv).

  For the Av and Sv correction accompanying the change in the Tv value, the three-dimensional graph as shown in FIG. 8 is easier to understand than the code diagram. The program diagrams shown in FIGS. 6, 7C and 7D are shown in FIG. FIG. 8 shows a control policy in which when the photometric value indicates low luminance, the gain (Sv) is first controlled, the aperture value (Av) is then controlled, and finally the shutter speed (Tv) is controlled. Show.

FIG. 9 shows a recalculation method of the Av value and the Sv value due to the change in the minimum shutter speed Tv _min .

(0) in the figure indicates a state in which a parameter set in an initial state where Tv _min is lower is set. In (1), Tv _min = 8 is obtained by calculation of the minimum shutter speed in S301, and it is confirmed that the shutter speed is lower than the minimum shutter speed in S304, and Tv = Tv _min is set in S305. It is. Since the exposure formula is not satisfied as it is, Av and Sv are corrected in S306.

  In the Av and Sv correction, a method of correcting the aperture value (Av) as shown in (2), a method of correcting the gain (Sv) as shown in (3), and an aperture as shown in (4). There are three options for correcting both (Av) and gain (Sv). Basically, a correction method according to the control policy corresponding to the exposure mode is selected from the above three options.

  Returning to FIG. 3, in S <b> 307, the AE calculation unit 113 determines the values of Tv, Av, and Sv set by the above processing.

  Returning to FIG. 2, in S206, the system control unit 114 checks whether the values of Tv, Av, and Sv calculated (set) in S205 are within the adjustment range (controllable range).

  In step S2061, information indicating that the exposure control element exceeds the adjustment range (out of the controllable range) with respect to the motion amount and the photometric value is displayed on the display unit 116, or through the external I / F 119. Output to the outside.

  For example, when the video camera according to the present embodiment is mounted on a vehicle, the notification information is output to the outside, thereby making it possible to promote a decrease in the traveling speed of the vehicle or to promote light emission. As a result, the conditions of the amount of motion and the photometric value are relaxed, and the control parameters in exposure control are within the adjustment range, and it may be possible to perform good exposure control.

  In S2062, Tv, Av, and Sv are reset. That is, the set value that exceeds the adjustment range is reset to a value within the adjustment range, so that the exposure can be controlled.

  In S207, the shutter speed, aperture value, and gain are controlled. That is, according to the set Tv, Av, Sv, a shutter speed control signal, an aperture control signal, and a gain control signal are output from the system control unit 114 to perform exposure control.

  In S208, it is confirmed whether the process is continued. When continuing, it returns to S201.

  As described above, in this embodiment, a high-quality image with less fluctuations in shake and brightness is obtained by sequentially controlling the shutter speed, aperture value, and gain, which are exposure control elements, based on the amount of motion and the photometric value. Can be obtained.

  A video camera that is Embodiment 2 of the present invention will be described below with reference to FIG.

  In this embodiment, instead of calculating the motion amount from the image by the image motion detection unit in the first embodiment, the motion amount is obtained using the motion sensor (vibration sensor) 601 and the motion sensor control unit 602. That is, in the first embodiment (FIG. 1), a motion vector between arbitrary frames is extracted by the image motion detection unit 107, and within the exposure time (1 / shutter speed) of one frame of the video signal by the motion amount calculation unit 108. The amount of movement was calculated as an index of movement. On the other hand, in this embodiment, the amount of motion is obtained by the motion sensor 601 and the motion sensor control unit 602.

  In FIG. 4, the same components as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment and are not described. The processing procedure in the present embodiment is the same as the processing described in the first embodiment with reference to FIGS.

  The motion sensor 601 is configured by an acceleration sensor or an angular acceleration sensor. The motion sensor 601 outputs changes in the position and orientation of the video camera as shown in FIGS. 5A and 5B. By integrating the acceleration signal or angular velocity signal output from the motion sensor 601 by the motion sensor control unit 602, the movement amount or the angle change amount per unit time is calculated.

  For example, as shown in FIG. 5A, the amount of movement of the video camera in the X, Y, and Z axis directions is

(Mm / frame)
As shown in FIG. 5B, let α, β, γ (rad / frame) be the amount of change in angle about the X, Y, and Z axes. In this case, the movement amount x ″, y ″ (mm) on the image sensor 103 is obtained by using a distance d [mm] from the focus lens position or the like to the subject.

It is represented by However, n is an outward normal when the subject is approximated to the surface, and in this embodiment,

And

  The rotation matrix R is

It is. The power T represents transposition. The amount of motion in pixel coordinate units can be calculated by dividing the amount of motion x ″, y ″ by the pixel size (mm).

  When calculating the amount of motion from the unit image (frame) described in the first embodiment, the relative motion between the subject and the video camera is detected as the amount of motion. In addition to shake, image shake due to subject movement can also be suppressed.

  On the other hand, when the motion sensor 601 is used as in this embodiment, only the motion of the video camera is detected. Accordingly, image blur due to camera movement can be suppressed, but image blur due to subject movement cannot be suppressed.

  However, when the motion sensor 601 is used, it is possible to detect a motion at a frequency much higher than the sampling rate of the video, and therefore, there is an advantage that more accurate changes in the position and orientation of the camera can be monitored sequentially.

  Even when it is difficult to extract a motion vector from an image of a low-luminance subject, the amount of motion can be obtained in this embodiment.

  Next, a video camera that is Embodiment 3 of the present invention will be described with reference to FIG.

  In the present embodiment, a rough motion amount is obtained by the motion sensor 601 and the motion sensor control unit 1302 as in the second embodiment, and the motion amount is also calculated from the image as in the first embodiment. Thereby, a more accurate and robust motion amount can be obtained.

  In FIG. 12, components common to the first embodiment are denoted by the same reference numerals as those of the first embodiment and are not described. The contents of the processing in the present embodiment are the same as the processing described in the first embodiment with reference to FIGS.

  In this embodiment, in addition to the motion sensor 601 and the motion sensor control unit 1302, the work memory 106 and the image motion detection unit 1301 are mounted for motion vector calculation. By setting the search region using the motion amount output by the motion sensor 601 and the motion sensor control unit 1302 as the initial position of the point of interest in the search image, the motion vector and the motion amount can be calculated at high speed and with high reliability. It can be carried out.

  Further, by combining the motion detection by the motion sensor 601 and the motion detection by the image motion detection unit 1301, not only the image shake due to the camera motion but also the image shake due to the subject motion can be suppressed. it can.

  Further, when the subject has low luminance, the amount of motion can be obtained using the detection result by the motion sensor 601 without using the detection result by the image motion detection unit 1301. Thereby, even when the subject has low luminance, a good image can be obtained.

  Next, a video camera that is Embodiment 4 of the present invention will be described with reference to FIGS. 13A and 13B. FIG. 13A shows an example of the values of Tv, Av, Sv set in S205 of FIG. 2 over time. From the left, the frame number, Tv value, Av value, Sv value, and Ev value are shown. The shutter speed (Tv) changes abruptly as the frame numbers 112 to 113 change (that is, the time elapses), and the aperture value (Av) changes abruptly to cope with this.

  When such control is actually performed, the iris 102 is mechanically driven, so that the followability is low. For this reason, a phenomenon occurs in which the brightness fluctuates in a short time such that an image becomes dark or bright for a moment. On the other hand, the gain, which is another exposure control element, is electrically controlled, so that the followability is high.

  Therefore, in this embodiment, when it is necessary to change the aperture value rapidly with a sudden change in the shutter speed, the gain is also temporarily changed. In other words, when the shutter speed is changed by a specific value or more between a specific number of consecutive frames (between unit images), for example, between two adjacent frames or between three consecutive frames (for example, when the Tv value is changed by two or more steps). The gain is changed with the aperture value.

  FIG. 13B shows a change in Av value when the brightness variation due to the adjustment of the gain (Sv) is smoothed. As a whole of FIG. 13B, the frame number, Tv value, Av value, Sv value, and Ev value are shown from the left. From this figure, it can be seen that the change in the aperture value (Av) is moderated by changing the gain (Sv).

  Thus, by replacing a part of the rapid change of Av with the change of Sv, it is possible to suppress the brightness variation of the image due to the low followability of the iris 102. Moreover, by replacing the change in Av due to the mechanical drive of the iris 102 with the electrical change in Sv, it is possible to suppress the generation of sound and battery consumption in the video camera.

  Next, a fifth embodiment of the present invention will be described. Also in this embodiment, the shutter speed, aperture value, and gain are adjusted in S205 based on the motion information obtained in S203 of FIG. 2 and the brightness information (photometric value) obtained in S204.

  However, the aperture value is related not only to the amount of light but also to the depth of field. Decreasing the aperture value (opening the aperture) decreases the depth of field, and increasing the aperture value (closing the aperture) increases the depth of field. In other words, when the aperture value is changed, the range in the depth direction that is in focus also changes, and an image different from the user's intention to focus on a specific range in the depth direction of the imaging field of view can be obtained. There is a case.

  Even in conventional exposure control, only one of an aperture value and a shutter speed, which is called an aperture priority mode or a shutter priority mode, can be specified. In other words, it was only possible to specify a depth of field or a constant shutter speed for suppressing movement.

  In this embodiment, a function called a so-called aperture priority / shake suppression mode is realized. In the first, second, and third embodiments, the iris (iris 102) is used in the exposure adjustment, whereas in this embodiment, the iris is fixed and the exposure is adjusted.

  Specifically, a table with a fixed aperture value can be designated as the control policy table through the operation signal input unit 115. When a fixed aperture value is input, the aperture value in the setting of Tv, Av, Sv in S205 is fixed. Except for fixing the aperture value, exposure control in Examples 1 to 3 is performed.

Further, with reference to FIG. 3, first, in S303 (exposure mode confirmation), a table with a fixed aperture value is read as a parameter table. In S306 (Av, Sv correction), Av (aperture value) is fixed and Tv (shutter speed) is corrected to Tv _min (minimum shutter speed causing no shake) by controlling Sv (gain). To compensate. In other words, the selection of Av and Sv recalculation accompanying the change in the minimum shutter speed shown in FIG. 9 is limited to the correction of the gain (Sv) shown in (3).

  In this way, by fixing the aperture value, it is possible to obtain an image in which shake and rapid brightness fluctuation are suppressed while maintaining the setting of the depth of field intended by the photographer.

  14A and 14B show changes in exposure control elements in the aperture priority / shake suppression mode.

  FIG. 14A shows the Ev value calculated in S204 of FIG. 2 and the Tv, Av, and Sv values calculated from the program diagram for each frame. Here, a case is shown in which Av calculated in S303 is 3, and that Av = 3 is fixed because of aperture priority.

On the other hand, FIG. 14B shows the result of changing the shutter speed to Tv _min and correcting it. Since the shutter speed is equal to or higher than Tv _min and the aperture is given priority, correction is performed using only the gain (Sv).

  As described above, according to each of the above embodiments, motion information representing the amount of motion is generated using at least one of a plurality of frames in a video signal and a motion sensor that detects motion of a video camera. Then, the aperture value, shutter speed, and gain are controlled based on the brightness information of the subject and the motion information. For this reason, it is possible to realize an imaging device capable of acquiring a high-quality image with less shake and abrupt brightness fluctuations, corresponding to a sudden change in the amount of motion in addition to a sudden change in brightness information. .

  In addition, each said Example is only an example, The Example of this invention is not limited to these. Various modifications and variations of each embodiment are possible.

1 is a block diagram illustrating a configuration of an imaging apparatus (video camera) that is Embodiment 1 of the present invention. 5 is a flowchart showing a procedure for exposure control in Embodiment 1. 3 is a flowchart showing a procedure for calculating Tv, Av, Sv in the first embodiment. FIG. 3 is a block diagram illustrating a configuration of an imaging apparatus according to a second embodiment. FIG. 10 is a diagram illustrating motion detection by a motion sensor of the imaging apparatus that is Embodiment 2 of the present invention. The figure which shows the angular displacement detection by the said motion sensor. FIG. 3 is a diagram illustrating an example of a program diagram in the first embodiment. FIG. 3 is a diagram illustrating an example of parameter control in the first embodiment. FIG. 3 is a diagram illustrating an example of parameter control in the first embodiment. FIG. 3 is a diagram illustrating an example of parameter control in the first embodiment. FIG. 3 is a diagram illustrating an example of parameter control in the first embodiment. FIG. 5 is a diagram illustrating a parameter space of an exposure control element in the first embodiment. FIG. 6 is a diagram illustrating a method for determining an exposure control element based on a movement amount in the first embodiment. FIG. 6 is a diagram for explaining block matching processing in the first embodiment. FIG. 6 is a diagram illustrating a motion amount calculation area in the first embodiment. FIG. 6 is a block diagram illustrating a configuration of an imaging apparatus that is Embodiment 3 of the present invention. The figure which shows the change of an exposure control element. FIG. 10 is a diagram illustrating changes in exposure control elements in an imaging apparatus that is Embodiment 4 of the present invention. FIG. 10 is a diagram illustrating changes in exposure control elements in an aperture priority / shake suppression mode in an imaging apparatus that is Embodiment 5 of the present invention. FIG. 10 is a diagram illustrating changes in exposure control elements in an aperture priority / shake suppression mode in an imaging apparatus that is Embodiment 5 of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Lens optical system 102 Iris 103 Image pick-up element 104 AGC part 105 Camera signal processing part 106 Work memory 107 Image motion detection part 108 Motion amount calculation part 109 Zoom / focus adjustment part 110 Iris adjustment part 111 Image sensor control part 112 Photometry part 113 AE Calculation unit 114 System control unit 115 Operation signal input unit 116 Display unit 118 Non-volatile memory 119 External I / F

Claims (4)

An image sensor that photoelectrically converts a subject image;
Video generation means for generating video based on an imaging signal output from the imaging device;
Gain adjusting means for performing gain adjustment on the imaging signal;
Motion information generating means for generating motion information using at least one of a plurality of unit images constituting the video and a sensor for detecting motion of the imaging device;
Exposure control means for controlling the aperture value of the diaphragm for adjusting the amount of light received by the image sensor based on the brightness information and the motion information, the shutter speed of the image sensor, and the gain in the gain adjustment means. An imaging device that is characterized.   The imaging apparatus according to claim 1, wherein the exposure control unit changes the gain together with the aperture value when the shutter speed is changed by a specific value or more between a specific number of consecutive unit images.   The imaging apparatus according to claim 1, wherein the exposure control unit has a mode in which the aperture value is fixed and the shutter speed and the gain are changed. 4. The exposure control unit according to claim 1, wherein when at least one of the shutter speed, the aperture, and the gain is out of a controllable range, information for notifying the output is output. The imaging device described in 1.