CN1732406B - Shake correction camera system - Google Patents

Abstract

A shake correction camera system includes: a shake correction lens, driven by a vibration detection signal obtained from an angular velocity sensor, to correct image shake; a point image function calculation unit, which calculates a point image distribution function; and an image restoration calculation unit, which uses the point image distribution function to perform image restoration on the captured image through image processing, thereby correcting image shake. Furthermore, image restoration corrects image shake that was not fully corrected in the shake correction lens, resulting in an image of higher quality.

Description

Shake correction camera system

The application is based on Japanese Patent Application No. 2002-374644 (applied on December 25, 2002), No. Japanese application), Japanese patent application No. 2002-374704 (applied on December 25, 2002), Japanese patent application No. 2002-374724 (applied on December 25, 2002), Japanese patent application No. 25), Japanese Patent Application No. 2002-374739 (filed on December 25, 2002), and Japanese Patent Application No. 2003-026098 (filed on February 3, 2003) as the basis, the contents of which are incorporated as references in in.

technical field

The present invention relates to a technology applicable to a camera or the like, detecting vibration due to hand shake or the like and correcting the shake of an image.

Background technique

Conventionally, cameras having a shake correction function are known in order to prevent deterioration of captured images due to hand shake that occurs during shooting. As a method of correcting camera shake, there are roughly two methods shown below.

The first shake correction method is an optical shake correction method as follows: the vibration of the camera is detected by a vibration detection sensor such as an angular velocity sensor and an acceleration sensor, and an optical system such as a photographic lens or a variable vertex prism is driven according to the detected amount to perform Shake correction (for example, JP-A-61-240780).

The second shake correction method is an electronic correction method as follows: the amount of shake is obtained from the difference between the captured image and the previous image temporarily stored in the memory, and the shake correction is performed when the image is read out (for example, JP-A 63-187883 bulletin). Either of these two methods is a method of performing camera shake correction in real time during shooting.

On the other hand, as a hitherto known technique in a shake correction method different from the above method, it is known to restore a degraded image to an image free from hand shake or blur. For example, Japanese Patent Application Laid-Open No. 62-127976 discloses a method of expressing image degradation caused by camera shake during shooting with a point image distribution function, and restoring a shake-free image based on the point image distribution function. In addition, there is known a technique of recording hand-shake information by providing only a shake detection device on a camera, and performing image restoration processing using this information during playback to correct the shake (for example, JP-A-6-276512).

Here, a specific method of image restoration processing will be described. The so-called image restoration refers to: using the shaking information to process the shaken image, and restore it to an image with little shake.

Now, let (x, y) be the position coordinates on the screen, the image without shaking (hereinafter referred to as the original image) is o(x, y), and the image degraded by shaking (hereinafter referred to as the shaken image) is z (x, y), the information of the point image expanded due to dithering (hereinafter referred to as point image function) is p(x, y), and these three satisfy the following relationship.

z(x, y) = o(x, y) ^* p(x, y) (1)

Here, ^* represents a convolution integral operation, so it is specifically represented by the following formula.

z(x,y)=∫∫σ(x,y)p(x-x′,y-y′)dx′dy′ (2)

By Fourier transforming it into the space frequency (u, v) region, formulas (1) and (2) become the following formula (3).

Z(u, v)=O(u, v) P(u, v) (3)

Among them, Z(u, v), O(u, v), P(u, v) are the spectrums of z(x, y), o(x, y), p(x, y) respectively. Furthermore, in formula (3), P(u, v) is specifically called a spatial frequency transfer function.

Here, when the point image function p(x, y) can be known by any method other than the dithered image z(x, y), each frequency spectrum is calculated, and the original Spectrum O(u,v) of an image.

$\mathrm{<span\; class="notranslate">\; o</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Z</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

In formula (4), 1/P(u, v) is specifically called an inverse filter. The original image o(x, y) can be obtained by performing inverse Fourier transform on the frequency spectrum calculated by formula 4. FIGS. 6( a ) to ( c ) and FIGS. 7 ( a ) to ( d ) are diagrams illustrating image restoration in the prior art.

Here, for the sake of simplicity, as shown in FIG. 6( b ), vibration is generated uniformly in the direction of one axis (X axis).

Select the profile of the point image distribution function, as shown in Figure 7(a). Figure 7(b) Fourier transforms it, which is the spatial frequency transfer function of the dither shown in Figure 6(a). The transfer function should note that there are several places where the value is 0. When using an inverse filter, as shown in FIG. 7(c), there is a point where it becomes infinite. Applying it to formula (4), regarding a specific spatial frequency, it becomes as shown in formula (5) below, and the spectrum value of the original image is uncertain.

The so-called transfer function being 0 means that there are frequency components that are not transmitted due to jitter (=information loss), and this formula indicates that the lost frequency components cannot be recovered. This means that the original image cannot be fully restored.

In addition, in practice, a Wiener filter represented by the following equation is used for image restoration so that the inverse filter does not become infinite.

c：常数    (6)

c: constant (6)

Fig. 7(d) is a graph in which the Wiener filter is used.

By making it a Wiener filter, O(u, v) does not become uncertain like in formula (5).

However, the above-mentioned conventional optical blur correction and image restoration have the following problems.

<Problems with Optical Shake Correction>

In optical shake correction, an angular velocity sensor is generally used as a sensor for detecting vibration. In order to convert the angular velocity detected by the angular velocity sensor into an angle, an output value (reference value) of the sensor at rest during operation is required, and it is known that the reference value is easily affected by drift due to temperature. This problem will be described in detail with reference to FIGS. 8( a ) and ( b ).

8( a ) and ( b ) are diagrams showing an output of an angular velocity sensor including a drift component, an output of a reference value, and a shake amount on an image plane.

FIG. 8( a ) shows changes in the angular velocity sensor with respect to time. For convenience of explanation, it is assumed that a hand shake occurs in a sinusoidal wave. In FIG. 8( a ), a waveform e0 represents the output of the shake sensor when hand shake is generated with a sinusoidal wave. In addition, either one of the waveforms e1 and e2 is a reference value calculated by a low-pass filter, and the interruption frequency of the waveform e1 is set to be lower than that of the waveform e2. In FIG. 8( a ), the output value deviates from the vibration center with the passage of time due to the influence of the environmental conditions, thereby drifting.

FIG. 8( b ) shows the amount of image blur when performing blur correction using the output of the angular velocity sensor and the reference value in FIG. 8( a ). Waveforms f0 , f1 , and f2 in FIG. 8( b ) correspond to waveforms e0 , e1 , and e2 in FIG. 8( a ), respectively. Waveform f0 represents the shake amount of the image plane when shake correction is not fully performed. The waveform f1 is compared with the waveform f2. Since the reference value e1 with a low cutoff frequency is used, the high-frequency components are cut off and the amount of jitter increases with time. Conversely, the reference value of the waveform f2 has a higher interruption frequency than the waveform f1, so the drift is smaller than that of the f1, but the high-frequency components due to hand shake cannot be removed.

In this way, removing image shake caused by hand shake and reducing the influence of drift are opposite problems, and it is very difficult to sufficiently correct image shake and set the cutoff frequency of the low-pass filter to reduce the influence of drift. For this reason, there is a problem that detection errors inevitably occur in the amount of detected blur, and even if optical blur correction is performed, blur remains in the obtained image.

Furthermore, in the optical shake correction device, a switch for switching ON/OFF of the shake correction operation is often provided. In this case, there is also the following problem: the switch is forgotten to be ON, and the shake correction operation is not performed during shooting, so the camera is shot. A jittery image was obtained.

<Problems with image recovery>

Next, the problem of image restoration will be described.

It is conventionally known that an image obtained by restoring a dithered image using a Wiener filter has higher resolution than the original image. But when At the spatial frequency (u', v') of , since the value of the filter becomes larger, when the clutter included in the image includes the spatial frequency component, the clutter component is amplified. As a result, there is a problem that unnecessary stripes are generated in the image, which degrades the image quality. This stripe does not become a big problem when the original jitter is small, but it becomes conspicuous when the jitter is large, so it becomes a problem more often.

In addition, cameras equipped with existing image restoration processing functions do not allow optical correction, but simply record output data from a shake sensor such as an angular velocity sensor, and perform restoration processing based on the shake information during playback. Therefore, if the shake of the image is too large, there will be a problem that the image quality cannot be improved due to the influence of the above-mentioned streaks and the like even if the image restoration process is performed.

Furthermore, the point image function required for image restoration is calculated based on information such as the output of the angular velocity sensor, and image restoration calculations are performed based on the calculation results. However, the amount of data output from the angular velocity sensor is very large, and a very large number of calculations are required, resulting in computational efficiency. Poor, a problem that requires a high-speed arithmetic processing unit.

Furthermore, even when the data required for image restoration is recorded on a recording medium or sent to the outside without performing point image function calculations and image restoration calculations, there is the following problem: as mentioned above, due to the large amount of data, A large-capacity recording medium is required, or a high-speed recording device and a high-speed communication device are required, and the implementation is relatively difficult, which causes an increase in cost.

Contents of the invention

The present invention is constituted as follows.

(1) The shake correction camera system of the present invention includes: a vibration detection part, which detects vibration and outputs a vibration detection signal; a shake correction optical system, which is driven according to the vibration detection signal, and corrects image shake; The imaging optical system of the optical system captures an image; and the image restoration calculation unit performs image restoration on the image captured by the imaging unit through image processing, and corrects image shake. Preferably, it further includes a point image distribution function calculation unit for calculating the point image distribution function, and the image restoration calculation unit performs image restoration by processing the image with the point image distribution function. Preferably, it further includes a reference value calculation unit for calculating a reference value of the vibration detection signal, and a point image distribution function calculation unit which calculates a point image distribution function based on a calculation result of the reference value calculation unit. Preferably, the shake correction camera system includes: a camera having a vibration detection unit, a shake correction optical system, an imaging unit, a point image distribution function calculation unit, a reference value calculation unit, and an image recording unit for recording an image; and an external device having The image restoration operation unit is a device separate from the camera, and performs image restoration by inputting the image recorded by the image recording unit and the point image distribution function.

The camera shake correction camera of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; a shake correction optical system that is driven according to the vibration detection signal to correct image shake; The image formed by the optical system is taken; the image recording unit records the image taken by the imaging unit; and the point image distribution function calculation unit calculates the point image distribution function required for the image restoration operation. Preferably, a point image distribution function output device is further provided, and the point image distribution function calculated by the point image distribution function calculation part is output to the outside by using an image recording part or a communication device. Preferably, it further includes a reference value calculation unit for calculating a reference value of the vibration detection signal, and the point image distribution function calculation unit calculates the point image distribution function based on the calculation result of the reference value calculation unit.

The image restoration device of the present invention includes: a data input unit, which receives image data and a point image distribution function obtained when the image data is captured via an external communication and/or medium; The function performs image processing on the image data to perform image restoration and correct image shake.

The computer program product of the present invention that can be read by a computer has a control program for shake correction, and the control program has: a data input command to receive image data and a point image distribution function obtained when the image data is captured; and an image restoration operation command, By performing image processing on image data with a point image distribution function, image restoration is performed and image shake is corrected. Preferably, the computer program product is a recording medium on which a control program is recorded. The control program of the computer program product may also be a carrier wave contained as a data signal.

(2) The shake correction camera system of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; a reference value calculation unit that calculates a reference value of the vibration detection signal; a shake correction optical system that detects vibration based on the vibration detection signal and the reference value driven to correct image shake; the imaging unit shoots an image formed by a photographic optical system including a shake correction optical system; the point image distribution function calculation unit uses a reference value or a vibration detection signal to calculate a point image distribution function; and The image restoration calculation unit performs image restoration and corrects image blur by performing image processing on the image captured by the imaging unit using the point image distribution function. Preferably, a point image distribution function calculation switching unit is further provided for switching between the following two cases: whether to use the reference value or the vibration detection signal, and the point image distribution function calculation unit performs the calculation of the point image distribution function. The point image distribution function calculation switching unit may also serve as a shake correction operation setting unit for switching whether to perform the shake correction operation by the shake correction optical system. Preferably, the point image distribution function calculation unit calculates the point image distribution function using the reference value when the shake correction operation is performed by the shake correction optical system. When the shake correction operation is not performed by the shake correction optical system, the point image distribution function calculation unit may calculate the point image distribution function using the vibration detection signal. Preferably, the shake correction camera system includes: a camera having a vibration detection unit, a shake correction optical system, an imaging unit, a point image distribution function calculation unit, a reference value calculation unit, and an image recording unit for recording images; and an external device, It has an image restoration calculation unit, which is a device separate from the camera, and performs image restoration by inputting the image recorded by the image recording unit and the point image distribution function.

The shake correction camera of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; a reference value calculation unit that calculates a reference value of the vibration detection signal; a shake correction optical system that is driven according to the vibration detection signal and the reference value, correcting image shake; an imaging section that captures an image formed by a photographing optical system including a shake correction optical system; an image recording section that records an image captured by the imaging section; and a point image distribution function calculation section that utilizes a reference value or vibration detection Signal, which operates on point-like distribution functions. Preferably, a point image distribution function output device is further provided, and the point image distribution function calculated by the point image distribution function calculation part is output to the outside by using an image recording part or a communication device.

(3) The shake correction camera of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; a reference value calculation unit that calculates a reference value of the vibration detection signal; and a shake correction optical system based on the vibration detection signal and the reference value. Driven to correct image shake; the imaging unit shoots the image formed by the photographic optical system including the shake correction optical system; the point image distribution function calculation unit calculates the point image distribution function required for image restoration calculation according to the reference value; and The information amount reducing unit reduces the reference value used for calculating the point image distribution function and/or the information amount of the calculated point image distribution function. Preferably, the information amount reducing unit reduces the amount of information by extracting the data of the reference value and/or the computed point image distribution function. Preferably, the information amount reducing unit reduces the amount of information so as to ensure the amount of information required for the image restoration calculation.

(4) The shake correction camera system of the present invention includes: a vibration detection section that detects vibration and outputs a vibration detection signal; an imaging section that takes an image formed by a photographing optical system including a shake correction optical system as an original image; and preserves the original image The image restoration part is used to save the original image; the image restoration operation part can change the parameters related to the image processing, and use the parameters to restore the original image through image processing to make a restored image corrected for image shake; and the restoration result preservation part will be used for The parameters of the image processing in the image restoration calculation unit and/or the restored image are associated with the original image and saved. Preferably, there is also a point image distribution function calculation unit for calculating the point image distribution function, and the image restoration calculation unit performs image restoration by processing the image with the point image distribution function, and the parameters include the point image distribution function. Preferably, the restoration result storage unit can store a plurality of parameters and/or a plurality of restoration images corresponding to a plurality of sets of restoration images. Preferably, the shake correction camera system includes: a camera having a shake detection unit, a shake correction optical system driven to correct image shake according to the shake detection signal, an imaging unit, a point image distribution function calculation unit, and a reference for calculating the shake detection signal. The reference value calculation part of the value, the original image storage part; and the external device, which has the image restoration calculation part and the restoration result storage part, is a device separate from the camera, and the original image and the point image distribution function recorded by the original image storage part are input , for image recovery.

The image recovery device of the present invention includes: a data input unit, which receives the original image data and the point image distribution function obtained when the original image data is photographed via external communication and/or media; Processing related parameters, using the parameters including the point image distribution function, performing image restoration on the original image data through image processing, and making a restored image with corrected image shake; The processed parameters and/or restored image are associated with the original image and saved.

The computer program product of the present invention that can be read by a computer has a control program for shake correction, and the control program includes: a data input command, receiving the original image data and the point image distribution function obtained when the original image data is taken; an image restoration operation command , you can change the parameters related to image processing, use the parameters including the point image distribution function to restore the original image data through image processing, and make a restored image with corrected image shake; and the restoration result save command will be used for image restoration The parameters of the image processing in the computing unit and/or the restored image are associated with the original image and saved. Preferably, the computer program product is a recording medium on which a control program is recorded. The control program of the computer program product may also be a carrier wave contained as a data signal.

(5) The shake correction camera of the present invention includes: a vibration detection part, which detects vibration and outputs a vibration detection signal; an optical shake correction device, which drives a shake correction optical system according to the vibration detection signal, and corrects image shake; a point image distribution function calculation part, calculating a point image distribution function required for restoring image shake by image processing; Prepare for shake correction. Preferably, the image restoration judging unit judges whether the image restoration mode needs to be performed based on the vibration detection signal. The image restoration judging unit may also judge whether the image restoration mode needs to be performed according to the shutter speed. The image restoration judging unit may also judge whether to perform the image restoration mode according to the focal length of the photographing optical system. The image restoration judging unit may also judge whether the image restoration mode needs to be performed according to the point image distribution function. A notification device may be further provided for notifying the information when the image restoration judging unit judges that the image restoration mode is unnecessary. When the image restoration judging unit judges that the image restoration mode is unnecessary, the image restoration mode may not be performed. When the image restoration judging unit judges that the image restoration mode is unnecessary, the point-image separation function may not be stored.

(6) The shake correction camera of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; a reference value calculation unit that calculates a reference value of the vibration detection signal; and a shake correction optical system based on the reference value and the vibration detection signal. driven to correct image shake; the drive control unit controls the operation of the shake correction optical system according to the vibration detection signal and the reference value; The required point image distribution function is calculated; and the shake correction mode selection part selects whether to perform an image restoration mode in addition to the optical shake correction operation of performing shake correction using the shake correction optical system, and the image restoration mode is performed by image restoration. Shake correction, or preparation for shake correction by image restoration, the drive control unit changes the control content of the shake correction optical system according to the selection state of the shake correction mode selection unit. Preferably, the drive control unit changes the control content of the correction optical system by changing the calculation method of the reference value according to the selection state of the shake correction mode selection unit. Preferably, the reference value calculation unit calculates the reference value using a low-pass filter, and the drive control unit changes the control content of the shake correction optical system by changing the cutoff frequency of the low-pass filter. Preferably, the drive control unit sets a cutoff frequency in a selected state in which the shake correction mode selection unit performs image restoration to a frequency higher than a cutoff frequency in a selection state in which the shake correction mode selection unit does not perform image restoration.

(7) Shake correction camera, including: a vibration detection part, which detects vibration and outputs a vibration detection signal; an optical shake correction device, which drives the shake correction optical system according to the vibration detection signal, and corrects image shake; a point image distribution function calculation part, which calculates the image Restoring the required point image distribution function, the image restoration restores the shake that has not been completely corrected by the optical shake correction device through image processing; Shake Correction Mode and Image Restoration Mode are selected. Image Restoration Mode performs Shake Correction by Image Restoration or prepares for Shake Correction by Image Restoration. Shake Correction Mode Selector Selects Image Restoration Mode. Also select the optical shake correction mode.

The shake correction camera of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; an optical shake correction device that drives a shake correction optical system according to the vibration detection signal to correct image shake; a point image distribution function calculation unit that calculates the image Restoring the required point image distribution function, the image restoration restores the shake that has not been completely corrected by the optical shake correction device through image processing; Shake correction mode and image restoration mode are selected, image restoration mode performs shake correction by image restoration, or prepares for shake correction by image restoration, and shake correction mode selection section does not select optical shake correction mode, the image recovery mode cannot be selected.

The shake correction camera of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; an optical shake correction device that drives a shake correction optical system according to the vibration detection signal to correct image shake; a point image distribution function calculation unit that calculates the image Restoring the required point image distribution function, the image restoration restores the shake that has not been completely corrected by the optical shake correction device through image processing; Shake correction mode and image restoration mode are selected, image restoration mode performs shake correction by image restoration, or prepares for shake correction by image restoration, and shake correction mode selection section does not select optical shake correction In the state of the mode, a warning is given when selecting the image recovery mode.

The shake correction camera of the present invention includes: a vibration detection unit that detects vibration and outputs a vibration detection signal; an optical shake correction device that drives a shake correction optical system according to the vibration detection signal to correct image shake; and a point image distribution function calculation unit that calculates The point image distribution function required for image restoration. The image restoration restores the shake that has not been completely corrected by the optical shake correction device through image processing. The calculation of the point image distribution function performed by the point image distribution function calculation unit can be performed by Operate optical shake correction equipment.

(8) The shake correction camera system of the present invention includes: a shake correction optical system for correcting image shake; a vibration detection unit for detecting vibration and outputting a vibration signal;

The reference value calculation unit calculates the reference value of the vibration signal; the drive unit drives the shake correction optical system; the position detection unit detects the position of the shake correction optical system, and outputs a position signal; the control unit, based on the reference value, the vibration signal and the position signal, Controlling the drive of the shake correction optical system to correct the shake of the subject image due to vibration; the imaging unit photographs the image formed by the photographing optical system including the shake correction optical system; the position error output unit is controlled by the control unit The difference between the obtained drive target position of the shake correction optical system and the actual drive position of the shake correction optical system output from the position detection part is output as a control position error; Image restoration is performed by image processing to control position error, and image shake is corrected. Preferably, it also includes: a point image distribution function calculation unit, which performs calculations on the point image distribution function required for image restoration calculation; and a function correction unit, which uses the control position error to correct the point image distribution function, and the image recovery calculation unit, by using The point image distribution function corrected by the function correction unit is subjected to image processing to perform image restoration. Preferably, it also includes a point image distribution function calculation unit, which calculates the point image distribution function required for the image restoration operation, and the point image distribution function is based on (a) the reference value and the control position error, (b) the vibration signal and the control position error. The position error, (c) the reference value, the vibration signal, and the control position error, or (d) the control position error are calculated, and the image restoration calculation unit performs image restoration by performing image processing with a point image distribution function. Preferably, the shake correction camera system includes: a camera having at least a vibration detection unit, a shake correction optical system, an imaging unit, a point image distribution function calculation unit, a reference value calculation unit, and an image recording unit for recording an image; and an external device having at least It has an image restoration calculation unit, which is a device separate from the camera, and performs image restoration by inputting the image recorded by the image recording unit and the point image distribution function. The shake correction camera system may also include: a camera having at least a vibration detection unit, a shake correction optical system, an imaging unit, a reference value calculation unit, and an image recording unit for recording an image; and an external device having at least a point image distribution function calculation unit and an image recording unit. The restoration calculation unit is a device separate from the camera, and performs image restoration by inputting the image recorded by the image recording unit and the point image distribution function.

The shake correction camera of the present invention includes: a shake correction optical system for correcting image shake; a vibration detection unit for detecting vibration and outputting a vibration signal; a reference value calculation unit for calculating a reference value of the vibration signal; a drive unit for driving the shake correction optical system; The position detection part detects the position of the shake correction optical system and outputs a position signal; the control part controls the driving of the shake correction optical system according to the reference value, the vibration signal and the position signal to correct the shake of the subject image caused by the vibration; The part shoots the image formed by the photographing optical system including the shake correction optical system; the image recording part records the image; the control position error output part converts the drive target position of the shake correction optical system obtained by the control part from the position detection part The difference of the actual driving position of the output shake correction optical system is output as a control position error; the point image distribution function calculation part is used to calculate the point image distribution function required for the image restoration operation; the function correction part is used to control the position error correction point an image distribution function; and an external output device that outputs the point image distribution function corrected by the correction unit to the outside using an image recording unit or a communication device.

The shake correction camera of the present invention includes: a shake correction optical system for correcting image shake; a vibration detection unit for detecting vibration and outputting a vibration signal; a reference value calculation unit for calculating a reference value of the vibration signal; a drive unit for driving the shake correction optical system; The position detection part detects the position of the shake correction optical system and outputs a position signal; the control part controls the driving of the shake correction optical system according to the reference value, the vibration signal and the position signal to correct the shake of the subject image caused by the vibration; The part shoots the image formed by the photographing optical system including the shake correction optical system; the image recording part records the image; the control position error output part converts the drive target position of the shake correction optical system obtained by the control part from the position detection part The difference in the actual driving position of the output shake correction optical system is output as a control position error; the point image distribution function operation part is used to calculate the point image distribution function required for the image restoration operation; and the external output device is used by the image recording part or A communication device that outputs a point image distribution function to the outside. The point image distribution function is based on (a) a reference value and a control position error, (b) a vibration signal and a control position error, and (c) a reference value and a vibration signal and a control position error. , or (d) control the position error and perform calculations.

The shake correction camera of the present invention includes: a shake correction optical system for correcting image shake; a vibration detection unit for detecting vibration and outputting a vibration signal; a reference value calculation unit for calculating a reference value of the vibration signal; a drive unit for driving the shake correction optical system; The position detection part detects the position of the shake correction optical system and outputs a position signal; the control part controls the driving of the shake correction optical system according to the reference value, the vibration signal and the position signal to correct the shake of the subject image caused by the vibration; The part shoots the image formed by the photographing optical system including the shake correction optical system; the image recording part records the image; the control position error output part converts the drive target position of the shake correction optical system obtained by the control part from the position detection part The output difference of the actual driving position of the shake-corrected optical system is output as a control position error; and the external output device outputs the point image distribution function to the outside using an image recording unit or a communication device.

The image restoration device of the present invention includes: a data input unit that receives the difference between the drive target position of the shake correction optical system and the actual drive position of the shake correction optical system output from the position detection unit via external communication and/or media The obtained control position error, the image data, and the point image distribution function obtained when the image data is taken; the function correction part uses the control position error to correct the point image distribution function; The point image distribution function performs image processing on the image data to restore the image and correct the image shake.

The image restoration device of the present invention includes: a data input unit that receives the difference between the drive target position of the shake correction optical system and the actual drive position of the shake correction optical system output from the position detection unit via external communication and/or media The control position error obtained, the image data, and the vibration signal obtained when the image data is taken; the point image distribution function calculation part calculates the point image distribution function required for the image restoration operation; the function correction part uses the control position error The point image distribution function is corrected; and the image restoration operation part performs image processing on the image data with the point image distribution function corrected by the function correction part to restore the image and correct the image shake.

The image restoration device of the present invention includes: a data input unit that receives the difference between the drive target position of the shake correction optical system and the actual drive position of the shake correction optical system output from the position detection unit via external communication and/or media The control position error obtained, the image data, and the vibration signal obtained when the image data is taken; the point image distribution function calculation part is used to calculate the point image distribution function required for the image restoration operation; and the image restoration operation part is used to The point image distribution function performs image processing on the image data to restore the image and correct the image shake. The point image distribution function, according to (a) reference value and control position error, (b) vibration signal and control position error, (c) reference Value and vibration signal and control position error, or (d) control position error, perform calculation.

The computer program product that can be read by a computer according to the present invention has a control program for shake correction, and the control program includes: a data input command, through communication with the outside and/or a medium, receiving the drive target position of the shake correction optical system and the slave The control position error, image data, and point image distribution function obtained when the image data is captured are obtained from the difference in the actual driving position of the shake correction optical system output by the position detection unit; the function correction command is used to correct the point image distribution using the control position error function; and an image restoration operation command for performing image restoration and correcting image shake by performing image processing on the image data with the point image distribution function corrected by the function correction unit.

The computer program product that can be read by a computer according to the present invention has a control program for shake correction, and the control program includes: a data input command, through communication with the outside and/or a medium, receiving the drive target position of the shake correction optical system and the slave The control position error obtained from the difference in the actual driving position of the shake correction optical system output by the position detection unit, the image data, and the vibration signal obtained when the image data is captured; the point image distribution function operation command is required for the image restoration operation The point image distribution function is operated; the function correction command is used to correct the point image distribution function using the control position error; and the image restoration operation command is used to perform image processing on the image data by using the point image distribution function corrected by the function correction part to perform image restoration. , corrections like dithering.

The computer program product that can be read by a computer according to the present invention has a control program for shake correction, and the control program includes: a data input command, through communication with the outside and/or a medium, receiving the drive target position of the shake correction optical system and the slave The control position error obtained from the difference between the actual driving position of the shake correction optical system output by the position detection unit, the image data, and the vibration signal obtained when the image data is taken; the point image distribution function operation command, which is required for the image restoration operation The point image distribution function is calculated according to (a) reference value and control position error, (b) vibration signal and control position error, (c) reference value and vibration signal and control position error, or (d) control position error; With the image restoration operation command, the image restoration is performed by performing image processing on the image data with the point image distribution function, and the image shake is corrected. A computer program product is a recording medium on which a control program is recorded. The control program is a carrier wave included as a data signal.

Description of drawings

FIG. 1 is a block diagram showing the system configuration of the first embodiment of the shake correction camera of the present invention.

FIG. 2 is an example of a frame arrangement diagram of a shake correction camera in the first embodiment.

3 is a control block diagram illustrating control operations of a shake correction control unit of the optical correction system.

4( a ) to ( c ) are diagrams illustrating image restoration in this embodiment.

5( a ) to ( d ) are diagrams illustrating image restoration in this embodiment.

6( a ) to ( c ) are first diagrams illustrating image restoration in the prior art.

7( a ) to ( d ) are second diagrams illustrating image restoration in the prior art.

8( a ) and ( b ) are diagrams showing an output of an angular velocity sensor including a drift component, an output of a reference value, and a shake amount on an image plane.

FIG. 9 is a block diagram showing a system configuration of a second embodiment of the shake correction camera of the present invention.

FIG. 10 is an example of a frame arrangement diagram of a shake correction camera in the second embodiment.

FIG. 11 is a block diagram illustrating the operation of the shake correction control unit in the second embodiment.

FIG. 12 is a flowchart showing the operation of the camera in the second embodiment.

FIG. 13 is a flowchart showing exposure and image restoration operations in the second embodiment.

FIG. 14 is a block diagram showing a system configuration of a third embodiment of the shake correction camera of the present invention.

FIG. 15 is a flowchart showing the basic operation of the camera when a shake correction operation is performed.

FIG. 16 is a flowchart showing the basic operation of the camera in the optical shake correction operation mode in the third embodiment.

FIG. 17 is a flowchart showing the basic operation of the camera in the image restoration operation mode in the third embodiment.

18 is a flowchart showing detailed operations of an image restoration judging unit that judges whether or not to perform point image function calculation based on shake detection data in the third embodiment.

Fig. 19 is a flowchart showing in detail the operation of obtaining data for point image function calculation.

Fig. 20 is a flowchart showing the basic operation of the image playback apparatus.

FIG. 21 is a diagram showing a specific example of image display and operation of various parameters.

22( a ) to ( c ) are diagrams showing specific image display and operation examples of various parameters.

FIG. 23 is a schematic diagram showing the relationship between the restored image, parameters and the original image saved in S2110.

Fig. 24 is a flowchart showing an image restoration operation in the fourth embodiment.

25 is a flowchart showing detailed operations of an image restoration judging unit that judges whether or not to perform image restoration based on a point image function in the fourth embodiment.

FIG. 26 is a block diagram showing a system configuration of a fifth embodiment of the shake correction camera of the present invention.

FIG. 27 is a flowchart showing the operation procedure of the camera body and the interchangeable lens at the time of shooting in the fifth embodiment.

FIG. 28 is a flowchart showing the operation procedure of the camera body and the interchangeable lens at the time of shooting in the fifth embodiment following FIG. 27 .

Detailed ways

Hereinafter, embodiments of the present invention will be described in further detail with reference to the drawings and the like.

<First Embodiment>

FIG. 1 is a block diagram showing the system configuration of the first embodiment of the shake correction camera of the present invention.

The shake correction camera 1 in this embodiment forms a camera system that has an optical shake correction function and can perform image restoration. FIG. 2 is a frame configuration diagram when the shake correction camera 1 is applied to a single-lens reflex camera with an interchangeable lens barrel. As shown in FIG. 2 , the camera system includes a camera body 101 and a lens barrel 102 .

The shake correction camera 1 is a so-called digital camera that captures images electronically, and has an optical correction system (or a shake optical correction system) 500 .

The angular velocity sensor 10 of the optical correction system 500 is a vibration detection unit that detects a vibration applied to the shake correction camera 1 by an angular velocity value. The angular velocity sensor 10 detects an angular velocity using Coriolis force, and outputs the detection result as a voltage signal.

The angular velocity sensor 10 and the processing of its output signal will be described with reference to FIG. 2 .

The angular velocity sensors 10 are provided one on each of the X-axis perpendicular to the optical axis I of the photographing lens and one on the Y-axis perpendicular to the X-axis, and two-dimensionally detect vibration of the shake correction camera 1 . The angular velocity sensor 10 can detect the angular velocity only while power is supplied from the power supply unit 90 . In addition, only one angular velocity sensor 10 is shown in FIG. 1 for easy understanding, and angular velocity sensors 10 a and 10 b in the X-axis direction and the Y-axis direction are shown in FIG. 2 .

The amplifier 420 is an amplifier that amplifies the output of the angular velocity sensor 10 .

The A/D conversion unit 20 is a converter for converting an analog signal into a digital signal, and converts the vibration signal from the angular velocity sensor 10 from an analog signal to a digital signal, and sends it to the shake correction control unit 30 .

The A/D converter 440 is a converter that converts the position information (analog signal) of the shake correction lens 70 transmitted from the position detection unit 60 into a digital signal. The converted camera shake correction lens position information is sent to the camera shake correction control unit 30 .

The D/A converter 430 is a D/A converter for converting the drive signal (digital signal) calculated by the shake correction control unit 30 into an analog signal. The converted analog signal is sent to the optical system driver 50 .

Generally speaking, since the output from the angular velocity sensor 10 is small, even if it is directly digitized by the A/D converter 20 and processed in the microcomputer 460, the resolution capability of the angular velocity value is too low (the angular velocity value corresponding to 1 bit is too high). Large), still cannot perform correct vibration detection. Therefore, the accuracy of shake correction cannot be improved. Therefore, the angular velocity signal is amplified by the amplifier 420 before being input to the A/D converter 20 . In this way, the resolution capability of the angular velocity value in the microcomputer 460 can be improved (the angular velocity value corresponding to 1 bit can be reduced), and the accuracy of shake correction can be improved.

In the amplifier unit 420, two amplifier units 420a, 420b are provided corresponding to the angular velocity sensors 10a, 10b, respectively. In addition, here, a low-pass filter may be added for the purpose of not only amplifying the signal but also reducing high-frequency noise included in the sensor output.

In addition, as shown in FIG. 2, the A/D conversion unit 20, the shake correction control unit 30, the optical system drive unit 50, the position detection unit 60, the D/A converter 430, and the A/D converter 440 include X 20a, 20b; 30a, 30b; 50a, 50b; 60a, 60b; However, since the operations in the X-axis direction and the Y-axis direction of each part are the same, the X-axis direction and the Y-axis direction will not be described separately below.

Hereinafter, it demonstrates based on FIG. 1. FIG.

The shake correction control unit 30 is a part that calculates the position information for driving the shake correction lens 70 based on the vibration signal detected by the angular velocity sensor 10 and the position information of the shake correction lens 70 detected by the position detection unit 60 described later. drive signal, and output the drive signal to the optical system drive unit 50. In addition, the shake correction control unit 30 also functions as a control position error output unit that outputs an error (control position error) described later.

The shake correction control unit 30 includes a reference value calculation unit 31 (see FIG. 3 ). The reference value calculation unit 31 is a part that calculates a reference value of the vibration signal obtained from the angular velocity sensor 10 , and in this embodiment, a digital low-pass filter (LPF) is used, and the output of the LPF is used as a reference value. The control operation performed by the shake correction control unit 30 will be described later.

The optical drive unit 50 is an actuator that drives the shake correction lens 70 based on the drive signal output from the shake correction control unit 30 .

The position detection unit 60 detects the positions of the shake correction lens 70 in the X-axis direction and the Y-axis direction in order to perform shake correction. The output (position signal) of the position detection unit 60 is sent to the shake correction control unit 30 via the A/D converter 440 .

The shake correction lens 70 is a part of the photographic optical system of the camera, and is a shake correction optical lens composed of a single lens that can move in a plane approximately perpendicular to the optical axis I of the photographic optical system, or a lens group formed by a plurality of lenses. system. The shake correction lens 70 is driven in a direction substantially perpendicular to the optical axis I by the optical system driving unit 50, and deflects the optical axis I of the photographing optical system.

Shake in images such as photographs is caused by movement of the image on the imaging surface during exposure due to vibrations applied to the camera such as hand shake. In the shake correction camera 1 of the present embodiment, vibrations applied to the shake correction camera 1 can be detected by the angular velocity sensor 10 . And, if the vibration applied to the shake correction camera 1 is detected, the movement of the image on the imaging surface caused by the vibration can be known, so the shake correction lens 70 can be driven to suppress the movement of the image on the imaging surface, and the imaging surface can be corrected. The movement of the image on the image is the image jitter.

The shake correction camera 1 includes, in addition to the above-mentioned optical correction system 500, a control unit 80, a power supply unit 90, a point image function calculation unit 100, an imaging unit 110, an image recording unit 120, an exposure control unit 150, and a focus lens position The detection unit 160, the focus detection unit 170, the flash control unit 180, the operation unit 190, and the like.

The control unit 80 is a control unit that controls the overall operation of the shake correction camera 1, and controls the shake correction control unit 30, the point image function calculation unit 100, the exposure control unit 150, the focus lens position detection unit 160, the focal length detection unit 170, and flash control. Various control operations of the unit 180 and the like.

The power supply unit 90 is a part that continuously supplies power to places requiring power in the camera starting with the angular velocity sensor 10 while the half-press timer 195 shown in FIG. 2 is ON. Also, when the half-press timer is OFF, the supply of power is stopped. Therefore, only while the half-press timer 195 of the camera is ON, the vibration detection of the camera can be performed by the angular velocity sensor 10 .

The point image function calculation unit 100 is a point image distribution function calculation unit that calculates the amount of exposure during exposure based on various information obtained from the shake correction control unit 30, the exposure control unit 150, the focus lens position detection unit 160, the focal length detection unit 170, and the like. The point image function (point image distribution function).

When the optical shake correction by the shake correction lens 70 is complete, the point image function is 1 point, but actually the optical shake correction is not complete, so the point image function is not 1 point. That is, image blur not completely corrected by the blur correction lens 70 remains (shake correction residue). The point image function calculated here is used later when image processing is used to further correct the shake correction residue left on the image plane without being completely corrected by the shake correction lens 70 .

The imaging unit 110 includes an imaging element 111 , an A/D conversion unit 112 , a signal processing unit 113 , etc., and is a unit that captures an image formed on an imaging surface by the imaging optical system and outputs image data to the image recording unit 120 . The imaging element 111 is an element that receives light from a subject image formed on an imaging surface by a photographing optical system, and converts it into image data of an analog signal. The A/D conversion unit 112 is a converter that converts an analog image into a digital image. The signal processing unit 113 is a unit that processes the image data converted into a digital signal by the A/D conversion unit 112 .

The image recording unit 120 associates the image captured by the imaging unit 110, the point image function calculated by the point image function calculation unit 100, and various information (parameters) required for various image restoration processes with the image, and records and saves them. part. These point-like functions and various information can be embedded and recorded in the image file as headers, or directly embedded in the image as in electronic watermarking technology. In addition, another file corresponding to the image file may be created and information may be written therein.

A specific form of the image recording unit 120 may be, for example, a removable recording medium such as Compact Disc (trademark) or smart media (trademark), or a buffer memory for image transmission.

The exposure control unit 150 is a part that controls the exposure time of the imaging element from the set value of the exposure time set on an unillustrated command dial or the like. Exposure time information and exposure start/end timing information are sent to the point image function calculation unit 100 .

The focus lens position detection unit 160 is a part that detects the position of a focus lens not shown. By detecting the position of the focusing lens, it is possible to calculate the distance from the imaging surface to the subject required for the calculation of the point image function.

The focus detection unit 170 is a part that detects the focal length f of the lens during shooting by the shooting optical system. The focal length f of the lens is also information required for the calculation of the point image function.

The flash control unit 180 is a part that controls light emission of the flash light emitting unit 181 .

The operation unit 190 has a half-press switch ( SW1 ) 191 , a full-press switch ( SW2 ) 192 , and the like.

The half-press switch 191 is a switch that is turned ON in conjunction with the half-press operation of a release button (not shown). By turning ON the half-press switch 191 , photometry calculations, autofocus driving, and the like by a photometry unit (not shown) are started. Also, when the half-press timer 195 is OFF, the half-press timer 195 is turned ON in synchronization with the ON of the half-press switch 191 .

The full-press switch 192 is a switch that is turned ON in conjunction with the full-press operation of the release button (not shown). By turning ON the full-press switch 192 , a series of photographing operations such as opening and closing of the shutter by a shutter mechanism (not shown) and writing of an image by the image sensor are performed.

The image restoration computing section 210 is a section that performs correction based on the image data transmitted from the image recording section 120 of the shake correction camera 1, point image function information corresponding to the image data, and various parameters used for image restoration processing. Image restoration processing of jitter contained in an image. In the image restoration processing in the image restoration calculation unit 210, the Wiener filter (Wina-filter) described in the formula (6) is used, but the present invention is not limited to this, and other methods may be used.

The image display unit 220 is a part for displaying an image taken by the photographer or an image after restoration of the image, and in this embodiment, the monitor unit of the camera corresponds to this part.

Next, the parts related to the shake correction control unit 30 , including the control of the optical shake correction operation, will be described.

FIG. 3 is a control block diagram illustrating the control operation of the shake correction control unit 30 of the optical correction system 500 .

First, vibration applied to the camera is detected by the angular velocity sensor 10 . The angular velocity sensor 10 generally uses a piezoelectric vibration angular velocity sensor that detects Coriolis force. The output of the angular velocity sensor 10 is input to a reference value calculation unit (low frequency component extraction) 31 via the A/D converter 20 .

The reference value computing unit 31 is a part that computes a reference value of vibration from the output of the angular velocity sensor 10 . Usually, the reference value of hand shake should be the output (hereinafter referred to as zero output) value when the angular velocity sensor 10 is completely still. However, since this zero output value fluctuates according to environmental conditions such as drift and temperature, the reference value cannot be fixed. Therefore, it is necessary to obtain zero output from the state of actual use, that is, the calculation reference value of the hand shake signal of the photographer. In the calculation of the reference value, a digital low-pass filter (LFP) is used.

The cut-off frequency fc of the digital low-pass filter is preferably set as low as possible, but as described in the description of the prior art, if the cut-off frequency fc is set too low, it is easily affected by sensor drift. In addition, when the value is set high conversely, frequency components below fc are not optically corrected, and therefore remain as image shake. The details will be described later, but the point image function can be obtained based on the reference value output without this optical correction, and the image blur that was not completely obtained by the optical correction can be restored by post-processing by performing image restoration processing.

Then, after the calculation of the reference value, the shake detection signal obtained by subtracting the reference value from the shake detection signal from the angular velocity sensor 10 is sent to the integrating unit 32 .

In the integration unit 32, the shake detection signal expressed in units of angular velocity is time-integrated and converted into a camera shake angle. For example, calculation is performed according to the following formula (7).

θ(t)=θ(t-1)+C·(ω(t)-ω ₀ (t)) (7)

In each symbol of the formula (7), θ(t) is the target driving position, ω(t) is the shaking detection signal, ω ₀ (t) is the reference value, t is the time (integer value), and C is determined by the A constant determined by conditions such as focal length.

The target driving position information calculated by the integrating unit 32 is sent to the target driving position calculating unit 33 .

In the target driving position calculation part 33, information such as the lens focal length f from the focus detection part 170, the object distance D from the focus lens position detection part 160, etc. are added to the shaking angle information transmitted from the integration part 32, Target drive position information for driving the shake correction lens 70 is calculated.

In the shake correction control unit 30, by using known PID control or the like, in order to move the shake correction lens 70 based on the target drive position information, and obtain the difference between the target drive position information and the position information from the position detection unit 60 of the shake correction lens 70, A drive signal for driving the optical system drive unit 50 is sent out. The drive signal is transmitted to the optical system drive section 50 via the D/A converter 430 . The shake correction lens 70 can be driven in a direction perpendicular to the optical axis by passing a current through the coil of the optical system driving unit 50 according to the sent driving signal.

The position detection unit 60 monitors the position of the shake correction lens 70 , and uses the detected lens position information to feedback-control the shake correction lens 70 through the shake correction control unit 30 .

Next, the operation of the point image function calculation unit 100 will be described.

In the description of the prior art, the following problem was described: even if the shake correction is performed by the optical correction system 500, the shake is not completely corrected, and some shake remains on the image (generation of shake correction residue) (refer to FIG. 8( a)(b)). The main reason for this jitter correction residual is that the place based on the reference value is very large. Therefore, in the point image function calculation unit 100 of the present embodiment, the point image function of the blur correction residual is calculated on the basis of the reference value.

Here, the calculation method of the point image function will be briefly described.

The point image function is calculated using the following formula. That is, the calculated average value ω ₀ ave of the reference value is subtracted from the obtained reference value ω ₀ , and this is integrated to obtain the error angle θ(t). Furthermore, the point image distribution function X(t) on the image plane is obtained from the focal length information f.

ω ₀ ave = Σω ₀ (t)/N (8)

θ′(t)=θ′(t-1)+C·(ω ₀ (t)-ω ₀ ave) (9)

X(t)＝f·θ(t)

In addition, when installing a teleconverter, it is necessary to change the focal length according to the magnification of the teleconverter. In addition, the accuracy of the point image distribution function increases when correction is performed using subject distance information. In this case, the following formula (11) may be used.

X(t)＝β·Rf·θ(t)          (11)

β: Lateral magnification

R: subject distance

These calculations are performed on the X direction and the Y direction respectively, and the point image distribution function is obtained when it is expanded on the X-Y plane.

In addition, the above-mentioned example is an example of point image function calculation, and other methods may be used for point image function calculation.

The point image function calculated here is sent to the image restoration computing unit 210 . The image restoration computing unit 210 performs image restoration computation based on the transmitted point image function. By correcting image shake that has not been completely corrected by the shake correction operation of the shake correction lens 70, a high-quality image with a good shake correction effect can be obtained.

Data used in conventional image restoration processing is almost always an example in which a point image function is directly obtained from shake detection data detected by an angular velocity sensor or the like to perform image restoration. However, as described above, in this method, there is a problem that image quality cannot be improved even if the image is restored when the image shakes greatly. However, according to the present embodiment, by correcting a certain degree of blurring by the optical blurring correction mechanism and performing image restoration processing using the blurring information at that time, the image quality can be greatly improved.

4( a ) to ( c ) and FIGS. 5( a ) to ( d ) are diagrams illustrating image restoration in this embodiment.

In the present embodiment, since the image data and the blur information after the blur correction by the optical blur correction mechanism are used, there is no problem that the amount of blur is too large. The effect of this point can be understood by comparing FIGS. 7( a ) to ( d ). The greater the jitter, the more the frequency components that are not transmitted increase, and the more difficult it is to restore the image. It can be seen that there are fewer points at which the spatial frequency transfer function shown in FIG. 5(b) becomes 0 than in FIG. 7(b). This means that efficient image restoration is possible due to the reduction of non-transmitted frequency components.

As described in detail above, according to the first embodiment, the following effects can be achieved.

Since there is a shake correction optical system that corrects image shake, and an image restoration calculation unit that corrects image shake by performing image restoration on an image captured by the imaging unit through image processing, it is possible to supplement the optical shake correction using the optical correction optical system. Problem points, and problem points of shake correction in image restoration, improve shake correction effect.

There is a point image distribution function computing unit that calculates point image distribution functions, and an image restoration computing unit performs image restoration by processing images according to point image distribution functions. Therefore, if the point image distribution function is calculated and stored during shooting, it can be used after shooting Image recovery at any time.

The point image distribution function calculation unit calculates the point image distribution function based on the calculation result of the reference value calculation unit, so that the shake that is not completely corrected in the shake correction by the shake correction optical system can be used as the point image distribution function, and image restoration can be performed. Shakes that are not completely corrected in the shake correction by the shake correction optical system are corrected.

<Second Embodiment>

This embodiment is a configuration in which the shake correction operation selection switch 194 and the signal flow control unit 452 are provided in the system configuration of the first embodiment.

FIG. 9 is a block diagram of the system configuration of this embodiment, and FIG. 10 is a block diagram of these configurations. Parts that perform the same functions as those in the first embodiment are given the same reference numerals, and redundant descriptions are appropriately omitted.

FIG. 11 is a block diagram illustrating the operation of the shake correction unit. As shown in FIG. 11 , when the output from the reference value calculation unit 31 is sent to the point image function calculation unit 100 , a signal flow control unit 452 is provided for switching whether or not to perform a shake correction operation.

A shake correction operation selection switch (VRSW) 194 is a switch for switching ON/OFF of an optical shake correction operation. In addition, the signal flow control unit 452 is a unit that switches the data sent to the point image function calculation unit 100 according to the state of the VRSW 194. In addition, in the interchangeable lens barrel camera, as shown in FIG. 10 , the correction operation selection switch 194 is provided on the outer peripheral portion of the lens barrel 102 of the interchangeable lens. The operation by the selection switch 194 in this embodiment is performed as follows.

When the VRSW 194 is ON, the shake correction lens 70 is driven to perform operations necessary for optical correction operations and image restoration. Image restoration data necessary for image restoration processing is sent from the reference value computation unit 31 to the point image function computation unit 100 . On the other hand, when the VRSW 194 is OFF, the drive of the shake correction lens 70 is stopped, and an optical shake correction operation is performed. The target driving position signal output from the target driving position computing unit 33 is sent to the point image function computing unit 100 as data for image restoration. In addition, regardless of the setting state of the VRSW 194, it is sent to the point image function calculation unit 100.

Next, basic operations of the shake correction camera 1 of this embodiment will be described.

Fig. 12 is a flow chart explaining the basic operation of the camera of the present invention.

Hereinafter, the flow of camera operations in this embodiment will be described based on the flowchart shown in FIG. 12 .

In addition, since the content described later is the content common to the X direction and Y direction, it demonstrates without specifying a direction in particular.

In step S1010, it is determined whether the half-press switch SW1 is ON. When the half-press switch SW1 is ON, enter S1020, and when the half-press switch SW1 is OFF, enter S1140.

In S1020, the counter T _SW1 is reset to make the count value 0. The counter T _SW1 is a counter for measuring the elapsed time after half-pressing the switch SW1 to turn OFF, and the count value is an integer. This counter is 0 while the half-press switch SW1 is ON, and operates only while the half-press switch SW1 is OFF and the half-press timer is ON.

In S1030, it is determined whether the half-press timer 195 is OFF. When the half-press timer 195 is OFF, the process proceeds to S1040, and when the half-press timer 195 is ON, the process proceeds to S1170.

In S1040, the counter t is reset to make the count value 0. The counter t is a counter for measuring the time during which the half-press timer 195 is ON. This counter is an integer-valued counter, and starts counting operation at the same time as the half-press timer 195 is turned ON, and continues the count operation while the half-press timer 195 is ON.

In S1050, the half-press timer 195 is turned on. In S1060, the angular velocity sensor 10 is turned ON, and detection of vibration is started. In addition, the conversion operation by the A/D converter 20 also starts here. In S1070 , the calculation of the reference value is started based on the output from the angular velocity sensor 10 . In S1080, calculation of a drive signal for driving the shake correction lens 70 is started. In S1090, it is judged whether VRSW 194 is set to ON. When the VRSW 194 is set to ON, the process proceeds to S1100 in order to drive the shake correction lens 70 . On the other hand, when VRSW 194 is set to OFF, the process proceeds to S1130.

In S1100, driving of the shake correction lens 70 is started. In S1100, an exposure operation and an image recovery operation are performed. This operation will be described in detail later using FIG. 13 . In S1120, the counter t of the half-press timer 195 is incremented by one. In S1130, the driving of the shake correction lens 70 is stopped. In addition, when the drive of the shake correction lens 70 has stopped at the time of entering this step, the stopped state continues as it is.

In S1140, it is determined whether the half-press timer 195 is ON. When the half-press timer 195 is ON, enter S1150, and when the half-press timer 195 is OFF, return to S1010, and continue to detect the half-press switch SW1. In S1150, when entering this step, the camera is in a state where the half-press switch SW1 is OFF and the half-press timer 195 is ON. To measure how long this state lasts, the counter T _SW1 is incremented by 1.

In S1160, it is judged whether the value of the counter T _SW1 is smaller than the threshold T_SW1. Here, the threshold T_SW1 is a constant for determining the upper limit of the counter T _SW1 , and determines the time from when the half-press switch SW1 is turned off to when the half-press timer 195 is turned off.

When the counter T _SW1 is less than the threshold T_SW1 , that is, when the determination is affirmative, the process proceeds to S1170 without turning off the half-press timer 195 . On the other hand, when the counter T _SW1 is equal to the threshold T_SW1, that is, when the judgment of this step is negative, the process proceeds to S1220, and the process of turning off the half-press timer 195 and the processing accompanying the half-press timer 195 being turned OFF are performed. .

In S1170, the ON state of the angular velocity sensor 10 is continued, and vibration is detected. In addition, the conversion operation by the A/D converter 20 is also continued. In S1180, the calculation of the reference value is continued. In S1190, the calculation of the drive signal for driving the shake correction lens 70 is continued from the output of the angular velocity sensor 10 and the reference value calculated in S1180.

In S1200, it is judged whether VRSW 194 is set to ON. When the VRSW 194 is set to ON, the process proceeds to S1210 in order to continue driving the shake correction lens 70 . When VRSW194 is set to OFF, it progresses to S1130.

In S1210, the driving of the shake correction lens 70 is continued. In S1220, the driving of the shake correction lens 70 is stopped. In S1230, the calculation of the reference value is stopped. In S1240, the power supply to the angular velocity sensor 10 is stopped, and the angular velocity sensor 10 is turned OFF. In S1250, the half-press counter 195 is turned off, and the process returns to S1010 to detect the state of the half-press switch (SW1) 191 .

Next, the exposure operation and image restoration operation of the camera of this embodiment will be described.

FIG. 13 is a flowchart showing details of the exposure operation and the image restoration operation of S1110 in FIG. 12 .

In S1500, it is determined whether or not the full push switch (SW2) 192 is ON. S1510 is entered when the switch SW2 is fully pressed to be ON, and S1520 is entered when the switch SW2 is fully pressed to be OFF.

In S1510, it is judged whether or not the process of starting exposure has been completed. When the exposure start process is completed, it proceeds to S1520, and when the exposure start process is not completed, it proceeds to S1530. The full press of the switch SW2 is a switch that triggers the exposure operation. When this switch is ON, if the exposure has not been started, the exposure is started here, and if the exposure has been started, the exposure control is performed.

In S1520, it is determined whether exposure is in progress. Enter S1540 when it is in exposure, and enter S1570 when it is not in exposure. In S1530 , processes for starting exposure, such as raising the mirror (not shown) and opening the shutter, are performed. In S1540, it is judged whether VRSW 194 is ON. Enter S1550 when VRSW 194 is ON, and enter S1560 when VRSW 194 is OFF.

In S1550, the reference value is integrated. This is equivalent to calculating the shake of an image that has not been completely corrected by the optical shake correction operation. The integrated value is stored in a memory or the like, and used for calculation of the point image function after exposure.

In S1560, the target drive position signal is read and stored in a memory or the like. Use this to perform point image function calculations after exposure. In S1570, it is judged whether the processing for ending the exposure is completed. When the processing is completed, go to S1590, and if the processing is not completed, go to S1580. In S1580, processing for ending the exposure, such as lowering of the mirror and closing of the shutter, is performed.

In S1590, it is judged whether the operation of the point image function is finished. When the operation of the point image function is completed, it proceeds to S1610, and when the operation of the point image function is not completed, it proceeds to S1600.

In S1600, the operation of the point-like function is started or continued. When entering this step, if the operation of the point-like function has not been started, the operation is started, and if the operation has been started, the operation is continued.

In S1610, it is judged whether the operation of image restoration is finished. When the calculation of image restoration is completed, it proceeds to S1120, and when the calculation of image restoration is not completed, it proceeds to S1620. In S1620, start or continue the operation of image restoration. When entering this step, if the operation of image restoration has not been started, the operation is started, and if the operation of image restoration has been started, the operation is continued.

In addition, when the optical shake correction operation is not performed, the point image function can be calculated from the output of the angular velocity sensor 10, and the shake can be reduced by image restoration processing after shooting. Therefore, even when the shake correction lens 70 is not operated due to any malfunction such as insufficient battery, a switch being left on, etc., the shake can be reduced by the image restoration process.

As described in detail above, according to the present invention, the following effects can be achieved.

Image restoration is performed by having a correcting optical system for correcting image shake, a point image distribution function calculation unit for calculating a point image distribution function using a reference value or a vibration detection signal, and performing image processing on an image captured by an imaging unit according to a point image distribution function. Since the image restoration calculation unit corrects image shake, image restoration can be performed regardless of whether optical shake correction using the shake correction optical system is performed.

Since there is a point image distribution function calculation switching section for switching whether the calculation of the point image distribution function is performed by the point image distribution function calculation section using the reference value or the vibration detection signal, the calculation method of the point image distribution function can be selected as required , which can efficiently perform image restoration when needed.

The point image distribution function calculation switching unit is also used as a shake correction operation setting unit for switching whether to perform shake correction operation by the shake correction optical system, so that the shake correction operation by the shake correction optical system can be linked and changed. The calculation method of the point image distribution function can calculate the optimal point image distribution function corresponding to the presence or absence of the shake correction operation by the shake correction optical system.

When the shake correction operation by the shake correction optical system is not performed, the point image distribution function calculation unit uses the vibration detection signal to calculate the point image distribution function, so the calculated point image distribution function is related to the camera shake itself, and can be used by image restoration. Image shake recovery due to camera shake.

Since there is a shake correction optical system that corrects image shake, and a point image distribution function calculation unit that calculates a point image distribution function using a reference value or a vibration detection signal, it is possible to correct optical shake regardless of whether optical shake is corrected using the shake correction optical system. By calculating the point image distribution function necessary for image restoration with an optimal calculation method, image recovery can be performed efficiently on the captured image afterwards by an external device or the like.

<Third Embodiment>

The third embodiment is a method of separating the image restoration device from the camera body. Parts that perform the same functions as those in the first embodiment are assigned the same reference numerals, and redundant descriptions are appropriately omitted.

Hereinafter, embodiments of the present invention will be described in further detail with reference to the drawings. FIG. 14 is a block diagram showing a system configuration of a third embodiment of the shake correction camera of the present invention.

The shake correction camera body 1 of this embodiment is provided with a function correction unit 105, an interface unit 130, and an image restoration determination unit 140 in addition to the structure of the first embodiment, and the image restoration calculation unit 210 becomes a part of the image playback device 2. A structure separate from the camera body 1.

The image playback device 2 is an image restoration device that connects the image recording unit 120 or the shake correction camera 1 and a playback device with a transmission cable or the like, plays back an image captured by the shake correction camera 1, and can perform image restoration.

The function correction unit 105 corrects the point image function using the position signal of the shake correction lens 70 obtained from the position detection unit 60 . More specifically, correction of the point image function is performed using the position signal based on the control position error which is the difference between the drive target position and the position signal (actual drive position).

The image recording unit 120 connects the interface unit 130 and the image playback device 2 with the transmission cable 300, and transmits the images stored in the image recording unit 120 and information required for image restoration processing to the image playback device 2 as needed.

The interface unit 130 is a communication device having a terminal for connecting the transmission cable 300 when connecting the shake correction camera 1 and the image playback device 2 or the like.

The connection cable 300 is a cable for connecting the connector of the interface unit 130 to a communication port (for example, RS-232C, USB, parallel port, IEEE1394, etc.) of the image playback device 2 . Data is transmitted and received between the shake correction camera 1 and the image playback device 2 via the connection cable 300 .

The image restoration judging unit 140 is a part that judges whether or not to perform point image function calculation. That is, a part that determines whether or not to perform point image function calculation on the output data from the reference value calculation unit of the shake correction control unit 30 based on information such as the shutter speed and the amount of blur.

Since the necessity of point image function calculation is judged by the image restoration judging unit 140, the information stored in the image recording unit 120 can be kept as much as necessary data, and unnecessary calculation operations and memory capacity can be reduced.

The operation unit 190 has a half-press switch (SW) 191 , a full-press switch (SW) 192 , a shake correction mode selection switch (SW) 194 , and the like.

The shake correction mode selection switch 194 is an operation member for selecting a combination of an optical correction operation mode and an image restoration mode. In this embodiment, as a switch for the shake correction operation mode from which three modes can be selected, it operates as follows.

When the "shake correction OFF mode" is selected, optical correction and image restoration are not performed. That is, the drive of the shake correction lens 70 is stopped, no shake correction operation is performed, and data for image restoration is not recorded and saved.

When the "optical correction operation mode" is selected, only the optical correction operation is performed, and the shake correction lens 70 is driven to perform the image shake correction operation, but the calculation of the point image function for image restoration processing and the data conversion for image restoration are not performed. record keeping, etc.

When "Image Restoration Operation Mode" is selected, operations required for optical correction operations and image restoration are performed. Data for image restoration necessary for the image restoration process is sent from the optical correction system 500 to the point image function calculation unit 100 via the image restoration determination unit 140 .

Next, the image playback device 2 will be described.

The image playback device 2 includes an image restoration calculation unit 210 for performing image restoration processing, an image display unit 220 for displaying an image, and a restoration result storage unit 230 .

The image playback device 2 in this embodiment functions as the image playback device 2 by using a personal computer and installing application software including a dedicated shake correction program necessary for image restoration to the personal computer.

In this shake correction program, in addition to the image restoration calculation part that performs image restoration processing

In addition to the (image restoration operation program), it also includes a data input unit (data input program) that receives image data, point image distribution functions, and various parameters transmitted from the camera side, and performs parameter setting on the camera side from the image playback device side. Certain settings, etc.

In addition, the image playback device 2 is not limited to the use of a personal computer, and may be a dedicated playback device or incorporated into a camera, for example.

The image restoration calculation section 210 is a section that, based on the image data transmitted from the image recording section 120 of the shake correction camera 1, point image function information corresponding to the image data, and various parameters used for image restoration processing, Image restoration processing for correcting shakes contained in an image is performed.

For the image restoration processing in the image restoration calculation unit 210, the Wiener filter described in the formula (6) is used, but the present invention is not limited thereto, and other methods may be used.

The image display unit 220 is a part for displaying an image taken by a photographer or an image after image restoration, and in this embodiment, a monitor unit of a personal computer corresponds to this part.

The restoration result storage unit 230 is a part that associates and stores the restored image after the image restoration process performed by the image restoration calculation unit 210 and the parameters used in the image restoration process with the original image.

Next, basic operations of the shake correction camera 1 of this embodiment will be described.

FIG. 15 is a flowchart showing the basic operation of the camera when a shake correction operation is performed.

In step S210, when the half-press switch 191 is ON, it progresses to S220. In S220, the state of the shake correction mode selection switch 194 is determined. In the "optical correction operation mode", enter the optical correction operation flow of S230, and in the "image restoration operation mode", enter the image restoration processing flow of S240 for optical shake correction operation and image restoration processing operation. When the "shake correction OFF mode" is selected, the process proceeds to S250.

In S250, since the "shake correction OFF mode" is selected, as described above, optical correction and image restoration are not performed, the drive of the shake correction lens 70 is stopped, no shake correction operation is performed, and no image restoration is performed. Data record keeping.

Hereinafter, operations of the camera shake correction camera 1 in the case of the "optical correction operation mode" and the "image restoration operation mode" will be described respectively.

First, the operation of the camera shake correction in the "optical correction operation mode" will be described.

FIG. 16 is a flowchart showing the basic operation of the camera in the optical shake correction operation mode.

In S400, the cut-off frequency fc of the LPF unit used in the reference value calculation unit 31 is set to fc=0.1 Hz. In S410, the angular velocity sensor 10 serving as the vibration detection unit is turned ON. In S420, the locked shake correction lens 70 is unlocked. In S430, a shake correction operation is started. The shake correction started here refers to an optical shake correction operation as follows: Based on the output of the angular velocity sensor 10, in order to eliminate the image shake, the shake correction lens 70 is moved in a direction approximately perpendicular to the direction of the optical axis to correct shake.

In S440, the state of the half-press timer is detected, and if the half-press timer is OFF, the process proceeds to S450, and if the half-press timer is ON, the process proceeds to S470. In S450, the shake correction operation is stopped, in S460 the correction lens 70 is locked, and the optical correction mode ends. In S470, the state of the full-press switch 192 is detected, and if the full-press switch 192 is ON, the process proceeds to S480, and if the full-press switch 192 is OFF, the process returns to S440.

In S480, after the centering operation of the shake correction lens 70 is performed, the shake correction is started again. In a state where the optical system driving unit 50 is not driven, the optical axis I of the photographing optical system and the optical axis of the shake correction lens 70 do not necessarily coincide. Generally, the shake correction lens 70 is mostly in the state of moving to the end of its movable range, and when the shake correction operation is performed directly, the direction cannot be driven, so the shake correction lens 70 is driven to make the shake correction lens 70 The optical axis of the camera and the optical axis I of the photographic optical system are approximately the same.

In S490, a shutter opening operation is performed to start exposure to the imaging unit 110 . In S500, it is judged whether or not the flash (SB) is emitted, and if the flash is emitted, the process proceeds to S510, and when the flash is not emitted, the process proceeds to S520. Lighting of the flash is performed in S510. In S520, the shutter is closed to end the exposure. Thereafter, it returns to the half-press timer judgment routine of S440.

Next, the operation of the camera shake correction in the "image restoration operation mode" will be described.

Fig. 17 is a flowchart showing the basic operation of the camera in the image restoration operation mode.

In S600, the cutoff frequency fc of the LPF unit used in the reference value calculation unit 31 is set to fc=1 Hz. Compared with fc=0.1 Hz in the above-mentioned "optical correction operation mode", in this "image restoration operation mode", the cutoff frequency fc is increased.

By setting in this way, in the "image restoration operation mode", compared with the "optical correction operation mode", the point image function calculated by the point image function calculation part 100 can appear within the vibration of the shake correction camera 1. The component of driving the shake correction lens 70 to perform shake correction is reduced by increasing the component. Accordingly, the amount of driving the shake correction lens 70 can be reduced, and the shake correction lens 70 can be driven with a margin within the drivable range. At this time, the amount of shake corrected by the optical shake correction operation decreases, and the amount of shake in the captured image increases, but this increased shake is corrected by image restoration later, so finally a high shake correction effect and no image shake can be obtained. Or like an image with very little jitter.

In this way, in the "image restoration operation mode", the cut-off frequency for calculation of the reference value is increased compared with the "optical correction operation mode", and the components for performing shake correction are allocated to the optical shake correction and image restoration, so as to be compatible with Compared with the "optical correction operation mode", it is possible to perform appropriate shake correction even for larger hand shakes.

The flow from S610 to S670 in FIG. 17 is the same as the operation in the flow from S410 to S470 in FIG. 16 , so detailed description is omitted here.

In S680, a recovery processing judgment is performed. The detailed description of the recovery process judgment in S680 will be described later using FIG. 18 . According to the restoration processing judgment in this step, if it is judged that the image restoration processing is unnecessary, go to S690, and if it is judged that the image restoration processing is necessary, go to S720.

In S690, similarly to S480 in FIG. 16, after the centering operation of the shake correction lens 70 is performed, the shake correction is started again. In S700 , a shutter opening operation is performed to start exposure to the imaging unit 110 . In S710, close the shutter and end the exposure. Thereafter, the process returns to the half-press timer judgment routine of S640. In S720, similarly to S480 in FIG. 16, after the centering operation of the shake correction lens 70 is performed, the shake correction is started again.

In S730, a shutter opening operation is performed to start exposure to the imaging unit 110 . In S740, data acquisition for point image function calculation is performed during the exposure period. Data for calculation of the point image function includes a reference value calculated from an output from the angular velocity sensor 10 and error information calculated from position information of the shake correction lens 70 obtained by the position detection unit 60 . The data acquisition for the point image function calculation in S740 will be described in detail later using FIG. 19 .

In S745, close the shutter and end the exposure. In S750, after the point image function calculation data is acquired, the point image function calculation is performed using the acquired data. The calculation example of the point image function is omitted since it has already been described in the first embodiment. In S755, the point image function calculated in S740 is corrected using the error data (operation by the function correcting section 105).

Here, a method of correcting the point image function using error data will be described.

As a function of time t, the driving target position of the shake correction lens 70 is set as lc(t), and the actual driving position is set as lr(t), then the control position errors e(t) in the X-axis direction and the Y-axis direction respectively , obtained by the following formula (as the operation of the control position error output section).

ex(t)＝lcx(t)-lrx(t) (12)

ey(t)=lcy(t)-lry(t) (13)

Set the function after two-dimensionally expanding these formulas (12) and (13) as e(x, y), then the corrected point image function p'(x, y) can use the point image shown in Mathematical Formula 1 The function p(x, y) is obtained according to the following formula.

p'(x,y)=p(x,y)+e(x,y) (14)

After the point image function is corrected, in S760, a dithering mark is added to the image to be processed for image restoration.

In S770, the corrected point image function is recorded as shake information, and the process returns to S640.

Next, processing of shake information output from the optical correction system 500 and acquisition of point image function calculation data will be described.

FIG. 18 is a flowchart showing detailed operations of the image restoration judging section 140 (S680 in FIG. 17 ) that judges whether to perform point image function calculation based on shake detection data.

Based on the judgment of the image restoration judging unit 140 , it is judged whether or not to record the shake detection data necessary for image restoration.

In S310, the validity of the image restoration process is judged according to the magnitude of the shake detection amount. In this step, the range of conditions within which image recovery is possible is set in advance based on the shake information and camera shooting information, and it is judged based on the conditions whether the shake can be effectively corrected by performing image recovery processing from the calculation result of the target drive position.

For example, if the shake amount is too large (the maximum limit shake amount), streaks will be noticeable in the image even if the image restoration process is performed, and image quality degradation due to the streaks cannot be avoided. Also, when the amount of shake is too small (minimum limit shake amount), the improvement effect will not appear even if image restoration is performed.

Therefore, these limit jitter amounts are set in advance based on experimental or empirical results.

In S320, the necessity of image restoration processing is judged based on the shutter speed (exposure time). In this step, the magnitude of a certain degree of shake is predicted based on the shutter speed, and whether or not image restoration processing is necessary is determined based on the predicted shake amount. When the shutter speed is fast, for example, even if there is a very small amount of vibration, it is judged to be an image that can be enjoyed. The amount of blur at this time is obtained from both the focal length and the shutter speed. When optical shake correction is not performed, it generally means that the occurrence of hand shake is slower than the shutter speed of (1/focal length). However, in the present embodiment, since optical blur correction is also performed, image restoration processing is performed only when the following formula (15) is satisfied, for example.

(A/focal length)＜shutter seconds (exposure time) (15)

Here, "A" in (Formula 15) may be a predetermined value or a variable that changes according to other conditions.

When both the shutter speed determination and the shake detection amount determination in S310 and S320 determine that recovery processing is necessary, the process changes to the exposure step of S330 with recovery processing, and proceeds to S720 in FIG. 17 .

On the other hand, if either of the shutter speed judgment or the shake detection amount judgment in S310 and S320 judges that restoration processing is unnecessary, the process proceeds to S340 to warn and display (inform) that the image restoration operation is not to be performed. The notification may be, for example, a warning sound, or a predetermined display may be performed.

After performing S340, it changes to the exposure step of S350 without restoration processing, and proceeds to S690 in FIG. 17 .

As shown in FIG. 18, by judging whether image restoration is necessary, the amount of dither information used for image restoration processing can be reduced, and memory capacity can be reduced.

FIG. 19 is a flowchart showing in detail the operation of acquiring data for point image function calculation (S740 in FIG. 17).

In this embodiment, the extraction processing (processing as an information amount reducing unit) shown in FIG. 19 is performed with the main purpose of saving memory capacity and the like.

After exposure starts, the counter is reset in S910. Specifically, let N=1, K=0. Here, N is a counter that is a number added to distinguish a plurality of reference values, and K is a counter that is a timer for counting time.

In S920, ω ₀ (1) output as the initial reference value is stored.

In S925, the error e(1) at this time is calculated and stored. Here, the so-called error refers to the difference between the drive target position of the shake correction lens 70 calculated by the target drive position calculation unit 33 and the actual drive position of the shake correction lens 70 output from the position detection unit 60 (control position error). The shake correction control unit 30 performs calculations. The shake correction control unit 30 outputs a drive signal to compensate for the difference between the drive target position and the actual drive position, but sometimes the shake correction lens 70 does not follow the drive target completely, and an error occurs in this case.

In S930, the average value ω ₀ ave of the reference value output is calculated according to the following formula.

ω ₀ ave = {ω ₀ (N)+ω ₀ ave×(N-1)}/N (16)

In S940, counter confirmation is performed. When K=100, it progresses to S950, and in other cases, it progresses to S970. In S950, the reference value output ω ₀ (N) is saved.

In S955, the error e(N) at this time is calculated and stored. In S960, set K=0 to reset the timer counter. In the present embodiment, since the sampling frequency of the angular velocity sensor 10 is 1 KHz, the reference value output is stored every 0.1 sec, so the reference value output is extracted. In S970, check whether the shutter is closed. If the shutter is open, go to S990. If the shutter is closed, go to S980.

In S980, the last reference value output ω ₀ (N) is stored. This is because, when the shutter second time is fast due to the decimated saving of the reference value output, it is avoided to save only the initial point of the reference value output. For example, in this embodiment, since the reference value output is saved every 0.1 sec when the sampling frequency is 1KHz, at a shutter speed faster than 1/10 sec, only the initial reference value output is saved, and it is impossible to configure Point-like functions. In addition, in this S980, the average value ω ₀ ave of the reference value output calculated in S930 is simultaneously stored.

In S985, the error e(N) at this time is calculated and stored. In S990 and S1000, count up, return to S930, and perform average value calculation of the reference value output.

Here, the above-mentioned extraction processing will be described.

In this embodiment, the point image function used in the image restoration process is calculated based on the reference value output. The reference value output is an LPF output having a cutoff frequency of 1 Hz (in the flow of FIG. 17 for image restoration) as described above, and therefore is lower than the frequency component of hand shake. Therefore, it is also possible to reduce the number of data used for point image function operations.

When the point image function calculation is performed, when the point image function is calculated for all the shake detection data sent from the optical correction system 500, a large amount of calculation and memory capacity are required.

The number of shake detection data obtained from the calculation result of the target position, for example, when the sampling frequency of the reference value calculation is 1KHz, the number of reference value data per second is N=1000, which is a very large amount of data. The frequency of hand shake is about 0.1 to 10 Hz, and the cutoff frequency of the low-pass filter provided in the reference value calculation unit 31 for calculating the reference value of hand shake is about 1 Hz. That is, in the point image function calculation unit 100, frequencies of 1 Hz or less are main components. About 10 times the frequency representing 1 Hz, that is, data with a period of 0.1 sec is sufficient. Therefore, the data sampled at 1KHz can be decimated up to 1/100.

Also, when changing the cutoff frequency of the LPF used for the reference value output calculation, it is necessary to change the amount of decimation from the cutoff frequency.

Through such processing, the time for calculation processing can be shortened, and the capacity of a memory or the like can be saved.

After the extraction process, dithering information is recorded on a recording medium for image recovery processing by the image playback device, and the data is transferred to the image playback device 2 . In this embodiment, the minimum number of data required for image restoration processing is recorded and transmitted through extraction processing, thereby greatly reducing transmission time and calculation processing time, and particularly saving memory capacity.

Here, the operation of the point image function calculation unit 100 performed in S750 of FIG. 17 will be described.

The description of the prior art describes the problem that even if the shake correction is performed by the optical correction system 500 , the shake is not completely corrected and some shake remains on the image (generation of shake correction residue). The cause of such shake correction residue is mainly that the position of the reference value and the position of the error generated between the actual drive position of the shake correction lens and the drive target are relatively large. Therefore, in the point image function computing unit 100 of the present embodiment, the point image function is calculated based on the reference value, and in the function correcting unit 105 , the point image function is corrected by an error. The point image function corrected here is sent to the image restoration computing unit 210 . The image restoration calculation unit 210 performs image restoration calculations based on the sent corrected point image function, and can obtain high image quality with good shake correction effect by correcting the image shake that has not been completely corrected in the shake correction operation of the shake correction lens 70. Image.

Next, the operation of the image playback device 2 will be described.

FIG. 20 is a flowchart showing the basic operation of the image playback apparatus 2. As shown in FIG.

In the image playback device 2, a shake correction program for performing image restoration has been installed.

As described above, in this embodiment, image data on the camera side is transmitted to the image playback device 2 via the transmission cable 300 .

In FIG. 20 , the transfer of the image has already been performed, the camera shake correction (image restoration process) process is started, and the menu screen is displayed.

In S2010, by clicking a restoration processing button or the like with an input device such as a mouse, the process of image restoration is entered. In S2020, an image judged to be an image to be restored on the camera side in advance is recorded with a jitter mark attached to it, so only the image to which the jitter mark has been added when the display image read-in operation has started is read out at the time of playback. image. In S2030, an image for which the user performs image restoration processing is selected and displayed while looking at various parameters related to the image or image shake.

In S2040, the shake track data and point image shake related to the selected image and used as parameters required for image restoration are displayed in more detail. More specifically, on the image display part (display) 220, the correction information recorded by the shake correction camera 1, such as shake locus data, point image shake, etc., and shooting information are displayed, and the operator can directly operate the appropriate image on the image display part 220. jitter trajectory data.

21 and 22( a ) to ( c ) are diagrams showing specific image display and operation examples of various parameters. FIG. 22( a ) shows shake locus data, FIG. 22( b ) shows shake locus data in the rough adjustment operation mode, and FIG. 22( c ) shows shake locus data in the fine adjustment operation mode.

In S2050, the above-mentioned parameters for performing image restoration are arbitrarily changed and set. In S2060, recovery processing is performed according to the parameters set in S2050. In S2070, the shaken image before restoration processing and the restoration image after restoration processing are displayed in comparison on the image display unit 220 of the image reproduction device 2 . In S2080, compare the shaken image before image restoration with the restored image after image restoration by eyes, and judge whether the restored image is acceptable (whether to perform image restoration again). If the image can be restored, go to S2085, and return to S2040 if the image is restored again.

In S2085, the user judges and decides whether to save the restored image and parameters. When saving the restored image and parameters, enter S2090, and end if not saving. In S2090, the user judges and selects whether to overwrite and save the restored image and parameters. Go to S2110 when saving without overwriting, and go to S2100 when saving with overwriting. In addition, when overwriting and saving, the data to be overwritten and erased (data that has already been saved) is also selected at the same time.

In S2100, the past restored image and parameters stored in association with the original image (data selected for overwriting in S2090) are deleted. In S2110, the restored image and the new parameters used in this image restoration process are associated with the original image and saved.

In this step, the file that saved the original image before restoration (dithered image), the file that saved the parameters, and the three files of the restored image are used as different files. In the file where the parameters are saved, write the same as Information about the file in which the original image was saved and the file in which the recovered image was saved. Thereafter, when the file in which the parameters are stored is opened in the image playback device 2, the associated file in which the original image is stored and the file in which the restored image is stored are also opened and displayed.

FIG. 23 is a schematic diagram showing the relationship among the restored image, parameters, and original image stored in S2110.

As shown in FIG. 23 , in this embodiment, there may be a plurality of combinations of restored images and parameters associated with one original image.

In this way, even if the image restoration process is performed several times, the parameters are associated with the image and stored, so that the work can be carried out smoothly without confusion.

In addition, the storage of restored images or parameters and original images is not limited to the manner shown in this embodiment, and various embodiments may be adopted. These combinations are shown in Table 1 below.

Table 1

Nos. 1 to 6 in Table 1 are methods that do not save the original image.

For example, if you are satisfied with the restoration result and do not perform image restoration again, there may be cases where the original image is completely unnecessary. At this time, methods No. 1, 2, 5, and 6 are effective.

On the other hand, when restoring the original image and the obtained restored image is not satisfactory, and the original image and the restored image are unnecessary, it is desirable to save it as a reference when performing restoration processing on another original image later. For parameters, No.3 and No.4 methods are valid.

No.7-12 in Table 1 are ways to save the original image. Among them, Nos. 7 and 8 are the above-mentioned embodiments.

For example, when the processing speed of the image restoration calculation unit 210 of the image playback device 2 is high, even if the restored image is not saved, only the parameters are saved by associating with the original image, and the saved parameters and the original image can be used immediately when necessary. Image performs image restoration and displays the restored image. Therefore, maintaining the same usage as the above-described embodiment, the capacity of the recording medium and the like can be saved (No. 9, 10).

In addition, No. 11 and No. 12 are effective when the restoration result is satisfactory and there is no need to restore the image under the same conditions again, but the original image is saved for confirmation so that the image restoration can be performed again with other parameters.

In addition, for all the above methods, you can choose re-save and overwrite save as the save method.

In the example shown in FIG. 21, on the image display unit 220, the dithered image before the restoration process, the restored image after the restoration process, the information related to the point image function, and the dithering locus data are associated with each other and displayed in a window. . By comparing and displaying on the same screen in this way, it is possible for the operator to intuitively judge at a glance which position to correct.

In addition, in the portion displayed on the lower right of FIG. 21 , a display is made that can be manipulated to shake the locus data. In the present embodiment, in this way, the shake locus data displayed on the image display unit 220 can be locally operated with a mouse or the like. By performing re-restoration processing on the basis of the jitter trace data thus operated, more detailed comparison and judgment can be made.

In addition, in the present embodiment, the rough adjustment of the image data operation shown in FIG. The fine-tuning shown in Figure 22(c). Fig. 22(c) is an example of fine-tuning mode, in which the number of data can be finely manipulated, so it is easy to evaluate the parameters of the restored image obtained, and it is possible to increase the degree of freedom in image manipulation and perform effective processing.

Conventionally, in the point image function calculation, the output obtained from the sensor output such as the angular velocity sensor 10 is directly used in the calculation, so the point image function contains a lot of error factors, and it is difficult to obtain high image quality even if the image display operation is performed. Image. On the other hand, in the present embodiment, image restoration processing is performed using output data with less clutter error that has been shake-corrected by an optical shake correction operation, so that a very high-quality restored image can be obtained. In addition, image manipulation can be performed directly with a mouse, for example, on shake locus data or point image data, and it is also easy to evaluate the effect of image restoration on parameters used in image restoration processing, enabling efficient processing operations.

Thus, in this embodiment, since the shake information is recorded in association with the image, the user can obtain the shake information simply by viewing the image using the image playback device 2 (image viewing software). Therefore, it is not necessary for the user to associate the image with shake information before performing image restoration, thereby improving work efficiency. In addition, since information on whether or not image restoration is required is also displayed through the shake mark, work efficiency is further improved.

According to the present embodiment, the actual driving position of the shake correction lens 70 is detected, the difference from the driving target position is obtained as an error, and the point image function reflecting the error is calculated, so image restoration is performed by using the point image function. In other words, the shake correction residue caused by the driving error of the shake correction lens 70 can also be corrected by image restoration, which can improve the shake correction effect.

In addition, it is not limited to embodiment demonstrated above, Various deformation|transformation and changes are possible, and those are also within the scope of the present invention.

For example, in this embodiment, an example is shown in which a digital low-pass filter is used for the reference value calculation, but the present invention is not limited to this, and the reference value calculation may be performed by other methods such as moving average.

In this embodiment, an example is shown in which the cutoff frequency of the LPF is changed depending on whether or not to perform image restoration, but the present invention is not limited to this, and the cutoff frequency of the LPF may not be changed depending on whether or not to perform image restoration.

In this embodiment, an example in which the camera shake correction camera 1 and the image playback device 2 are connected via the transmission cable 300 to transmit and receive data is shown, but it is not limited thereto. The point image function of the image, other parameters necessary for image restoration processing, photographic information, and other widely used recording media can also be transmitted by wireless communication.

In this embodiment, an example in which the restoration result calculation unit 230 is provided on the image playback device 2 is shown, but it is not limited thereto. For example, for a camera having an image restoration calculation unit, the restoration result calculation unit may be installed on camera.

In this embodiment, an example is shown in which the data extraction process is performed before the point image function calculation, but the present invention is not limited thereto, and the data extraction process may be performed after the point image function calculation.

In this embodiment, an example in which the shake correction operation mode selection switch 194 is a 3-state switch is shown. However, the present invention is not limited to this, and for example, a switch capable of separately turning ON/OFF the "optical correction operation mode" and the "image restoration operation mode" may be used, or a software switch may be used. In this case, when the "image restoration operation mode" is selected without selecting the "optical correction operation mode", the user will be warned as follows: a warning sound will be issued, a warning display will be displayed, or there will be an additional indication related to the image. A warning mark for optical shake correction, etc.

In this embodiment, the following example is shown: the result of judging whether image restoration is necessary, and when it is judged that image restoration is not to be performed, the dither information such as the point image function is not calculated or recorded and saved, but it is not limited to this, for example, it may be additionally performed. The recording of the jitter information and the notification (warning) of information on images that do not require image restoration may be marked, and a flag indicating information not to record and save the jitter information may be added.

In this embodiment, an example is shown in which the original image file, the parameter file, and the restored image are used as different files, and the parameters are written and saved in the parameter file. Information about the file that saved the original image and the file that saved the restored image. However, it is not limited to this. For example, these three types of data may be stored as one file, or information related to the relationship between the three types of data may be stored in another file, and the three types of data may be processed by referring to the other file on the application software. The associated display, etc.

As described above in detail, according to the present invention, the following effects can be exhibited.

Since there is a camera including a point image distribution function calculation unit and an external device including an image restoration calculation unit, it is not necessary to perform image restoration with a large amount of calculation by the camera, the camera can be made inexpensive, and the power consumption of the camera can be reduced.

Since there is a point image distribution function output device that outputs the point image distribution function calculated by the point image distribution function calculation part to the outside by using an image recording part or a communication device, image restoration can be easily performed by an external device without complicated work. .

The point image distribution function calculation unit calculates the point image distribution function based on the calculation result of the reference value calculation unit, so that the shake that is not completely corrected in the shake correction by the shake correction optical system can be used as the point image distribution function, and can be corrected by image restoration. Shake Correction Shake that is not fully corrected in the shake correction performed by the optical system.

Since there is a data input unit that receives image data and a point image distribution function, and an image restoration calculation unit that performs image restoration and corrects image shake, it is possible to perform image restoration on image data including shake using the point image distribution function, and to correct the image after shooting. shake.

Since it is a shake correction program including a data input program for receiving image data and point image distribution functions and an image restoration operation program for image restoration and correction of image shake, image restoration can be performed using widely used computers. Therefore, image restoration can be performed without using a dedicated external device, making it possible to make a low-cost system as a whole.

Since there is an information reduction unit for reducing the reference value used in the calculation of the point image distribution function and/or the information volume of the calculated point image distribution function, it is possible to shorten the calculation processing time, save memory capacity, etc., and eliminate the need for a high-speed calculation processing unit. , large-capacity recording media, high-speed recording equipment and high-speed communication equipment, image recovery can be performed.

The amount of information reduction unit reduces the amount of information by extracting the data of the reference value and/or the calculated point image distribution function, so that the amount of information can be reduced easily and reliably.

The information amount reduction unit reduces the amount of information to ensure the amount of information required for the image restoration calculation, so that the image quality after image restoration will not be degraded, and a high-quality restored image can be obtained.

Since there is a restoration result storage unit or a restoration result storage program that associates and saves the parameters used in the image processing in the image restoration calculation unit and/or the restored image with the original image, when the user changes the parameters necessary for image restoration , image data and parameter settings can be easily organized and managed.

The restoration result storage unit can save multiple parameters and/or restoration images corresponding to multiple sets of restoration images, so that even when the parameters are changed and the image restoration is tried many times at the same time, the history of the trials can be retained and can be easily sorted out .

The shake correction control unit changes the control content of the shake correction optical system according to the selection state of the shake correction mode selection unit, so that an optical shake correction operation suitable for a situation of performing image restoration can be performed.

The shake correction control unit changes the control content of the shake correction optical system by changing the calculation method of the reference value according to the selection state of the shake correction mode selection unit. Thus, the shake correction amount of the optical shake correction operation can be changed during image restoration, and by reducing the shake correction amount of the optical shake correction operation, an appropriate shake correction operation can also be performed for larger hand shakes. In addition, residual image shake can be corrected by image restoration, so a sufficiently shake-corrected high-quality image can be obtained.

Since the shake correction control unit changes the control content of the shake correction optical system by changing the cutoff frequency of the low-pass filter, it is possible to easily change the shake correction amount of the optical shake correction operation, and this embodiment can be easily performed.

The shake correction control unit changes the cutoff frequency of the selected state where the shake correction mode selection unit performs image restoration to a frequency higher than the cutoff frequency of the selected state where the shake correction mode selection unit does not perform image restoration. Therefore, when performing image restoration, the optical formula The shake correction amount of the shake correction operation is reduced, and appropriate shake correction operations can also be performed for larger hand shakes.

Since there is an image restoration judging unit for judging whether the image restoration mode is necessary, image restoration can be performed when image restoration is desired to improve image quality, and unnecessary image restoration can be prevented.

The image restoration judging unit judges whether the image restoration mode needs to be performed based on the vibration detection signal, so that it can judge whether the image shake is too large or the image shake hardly occurs.

The image restoration judging unit judges whether or not to perform the image restoration mode based on the shutter speed, so that the image restoration can be performed at a shutter speed that may cause image blur.

The image restoration judging unit judges whether the image restoration mode needs to be performed based on the focal length of the photographing optical system, so that image restoration can be performed at a focal length at which image shake may occur.

The image restoration judging unit judges whether the image restoration mode is necessary according to the point image distribution function, so it can judge whether the image restoration is necessary according to the change of the shake amount during exposure, and can make a more accurate judgment.

When the image restoration judging unit judges that the image restoration mode is not necessary, since there is a notification device for notifying the information, the photographer can easily grasp the shooting state, and it can be used more conveniently.

When the image restoration judging unit judges that the image restoration mode is not necessary, the image restoration mode is not executed, so unnecessary arithmetic processing is not performed, which is effective in terms of processing speed, power consumption, and the like.

When the image restoration judging unit judges that the image restoration mode is unnecessary, since the point image distribution function is not stored, the capacity of the memory and the recording medium can be saved.

When the image restoration mode is selected by the shake correction mode selection unit, the optical correction mode is also selected at the same time, so the image shake for image restoration can be reduced by optical shake correction, and high-quality images can be reliably obtained during image restoration. .

Since the image stabilization mode selection unit cannot select the image restoration mode when the optical image stabilization mode is not selected, it is possible to prevent erroneous restoration of an image captured without optical image stabilization. Therefore, high-quality images can always be obtained when performing image restoration.

The camera shake correction mode selection unit issues a warning when the image restoration mode is selected without selecting the optical shake correction mode, so it is possible to prevent erroneous image restoration of a captured image without performing optical shake correction. Therefore, high-quality images can always be obtained when performing image restoration.

The calculation of the point image distribution function by the point image distribution function calculation unit can be performed by operating the optical shake correction device, so the image shake at the time of image restoration can be reduced by the optical shake correction, and the calculated point image distribution function , fully contains the information needed for image restoration, so high-quality images can always be obtained when performing image restoration.

<Fourth Embodiment>

Hereinafter, a fourth embodiment of the present invention will be described using FIG. 24 .

The fourth embodiment is different from the third embodiment in the following point, and other parts are the same as the third embodiment, that is: after performing the point image function operation (S750), perform S680 (image resume the operation of the judging section). Therefore, the parts that perform the same functions as those in the above-mentioned third embodiment are given the same reference numerals, and redundant descriptions are appropriately omitted.

After the point image function operation is performed in S750, it is determined in S755 whether image restoration is required (whether an image restoration operation mode is required). This step is an operation having the same purpose as S680 in FIG. 17 in the third embodiment (as an operation of the image restoration judging unit), but the judging method is different (see FIG. 25 ). As a result of the determination in S755, if it is determined that image restoration is to be performed, go to S760, and if it is determined that image restoration is not to be performed, return to S640.

FIG. 25 is a flowchart showing detailed operations of an image restoration judging unit that judges whether or not to perform image restoration based on a point image function, and corresponds to FIG. 18 in the third embodiment.

In S5310, it is judged according to the point image function whether an image restoration operation mode is required. If the result of judgment is to perform image restoration, go to S5320, and enter the exposure program with image restoration (S760 in FIG. 24). On the other hand, when image restoration is not performed, the process proceeds to S5330.

As a specific method of judging whether the image restoration operation mode is necessary based on the point image function, for example, the following method is used: calculating the width of the point image function, which is smaller than a predetermined amount (for example, 30 μm) for image restoration. Here, as the width of the point image function, for example, a rectangle circumscribing the point image function can be obtained, and the length of the diagonal line adjoining the rectangle can be taken as the width of the point image function.

In S5330, warning, display (notification) of information that the image recovery operation is not performed. The notification may be, for example, a warning sound or a predetermined display.

In S5340, the exposure program without recovery processing is entered (S640 in FIG. 24).

In the third embodiment, before the calculation of the point image function, information such as shake detection amount, shutter speed, and focal length is used to determine whether image restoration is required. However, since this information is obtained from half-pressing the release button to full-pressing, the vibration during the exposure is predicted and judged, and there is a possibility that it cannot be correctly judged if the amount of vibration changes from the time the release button is fully pressed to the end of the exposure. Condition. Therefore, in the present embodiment, whether or not to perform the image restoration operation mode is judged based on the point image function calculated by using the information obtained during exposure, and the judgment reflecting the actual vibration information during photography can be made.

In addition, since the amount of image shake on the image plane is obtained as a specific numerical value from the point image function, it is possible to more accurately determine whether image restoration is necessary or not.

According to the present embodiment, since the determination of whether the image recovery operation mode is necessary is performed based on the point image function, it is possible to determine whether the image recovery is necessary based on the change in the shake amount during exposure, and a more accurate determination can be made.

As described above in detail, according to the present invention, the following effects can be exhibited.

Since there is an image restoration judging unit for judging whether the image restoration mode is necessary, image restoration can be performed when image restoration is desired to improve image quality, and unnecessary image restoration can be prevented.

The image restoration judging unit judges whether the image restoration mode needs to be performed based on the vibration detection signal, so that it can judge whether the image shake is too large or the image shake hardly occurs.

The image restoration judging unit judges whether or not to perform the image restoration mode based on the shutter speed, so that the image restoration can be performed at a shutter speed that may cause image blur.

The image restoration judging unit judges whether the image restoration mode needs to be performed based on the focal length of the photographing optical system, so that image restoration can be performed at a focal length at which image shake may occur.

The image restoration judging unit judges whether the image restoration mode is necessary according to the point image distribution function, so it can judge whether the image restoration is necessary according to the change of the shake amount during exposure, and can make a more accurate judgment.

When the image restoration judging unit judges that the image restoration mode is not necessary, since there is a notification device for notifying the information, the photographer can easily grasp the shooting state, and it can be used more conveniently.

When the image restoration judging unit judges that the image restoration mode is not necessary, the image restoration mode is not executed, so unnecessary arithmetic processing is not performed, which is effective in terms of processing speed, power consumption, and the like.

When the image restoration judging unit judges that the image restoration mode is unnecessary, since the point image distribution function is not stored, the capacity of the memory and the recording medium can be saved.

<Fifth Embodiment>

The fifth embodiment is an interchangeable camera in which the taking lens part can be replaced, and the parts that perform the same functions as those in the third embodiment are denoted by the same reference numerals, and redundant descriptions are appropriately omitted.

FIG. 26 is a block diagram showing a system configuration of a fifth embodiment of the shake correction camera of the present invention.

The camera body 101 is provided with a main body side control unit 80A, a power supply unit 90, a point image function calculation unit 100, an imaging unit 110, an image recording unit 120, an interface unit 130, a shake correction mode determination unit 145, an exposure control unit 150, A flash control unit 180, an operation unit 190, and the like.

The interchangeable lens 102 is provided with a lens side control unit 80B, a RAM 121, a focus lens position detection unit 160, a focal length detection unit 170, a shake correction mode selection switch 194, an optical correction system 500, and the like.

In addition, a signal transmission unit 310 is provided at a portion connecting the camera body 101 and the interchangeable lens 102 so that signals can be exchanged between the camera body 101 and the interchangeable lens 102 .

A function correction unit 240 is provided on the image playback device 2 .

Next, operations of the camera body 101 and the interchangeable lens 102 at the time of shooting in this embodiment will be described.

In addition, as described in the third embodiment, the camera shake correction mode selection switch 194 is a switch that can select three modes: "shake correction OFF mode", "optical correction operation mode", and "image restoration operation mode", but here , only the operations of the "optical correction operation mode" and "image restoration operation mode" related to the description of the present invention will be described.

27 and 28 are flowcharts showing the flow of operations of the camera body 101 and the interchangeable lens 102 during shooting according to this embodiment, and show only the parts related to shake correction during general single-shot release. In addition, on paper, the flowchart is divided and shown in FIG. 27 and FIG. 28 . In addition, among these flows, the flow on the camera body 101 side (S3000 part) is shown on the left side of the figure, and the flow (S4000 part) on the interchangeable lens 102 side is shown on the right side, and the dotted line connects them. Operations, expressed as operations at approximately the same time.

In S3010 , the use of the power supply unit 90 is permitted for the interchangeable lens 102 .

The interchangeable lens 102 that has obtained the permission to use the power source (S4010) supplies power to the angular velocity sensor 10 and other circuits in S4020.

In S3020, it is judged whether the half-press switch 191 is ON, and when the half-press switch 191 is ON, it proceeds to S3030, and when the half-press switch 191 is OFF, the judgment in S3020 is repeated. In S3030, a command instructing to start a shake correction operation performed while the half-press switch 191 is ON (hereinafter referred to as half-press shake correction) is sent to the lens side.

In the interchangeable lens 102 that has received the half-press shake correction start command (S4030), in S4040, the lock of the shake correction lens 70 is released, and the half-press shake correction operation is started.

In S3040, it is judged whether the full-press switch 192 is ON. When the full-press switch 192 is ON, the process proceeds to S3050. When the full-press switch 192 is OFF, the process returns to S3030, and the judgment in S3040 is repeated. In S3050, the ON of the full-press switch 192 is received, and a command instructing to start blur correction during exposure is sent to the lens side. Exposure preparations are performed in S3060, such as raising the mirror, etc., exposure is started in S3070, and exposure is ended in S3080.

In the interchangeable lens 102 that has received the in-exposure shake correction start instruction (S4050), the in-exposure shake correction operation is started in S4060. In addition, in this S4060, in the interchangeable lens 102, the shake correction operation during exposure is performed, and the actual drive position of the shake correction lens 70 is detected by the position detection unit 60, at least during the period from the exposure start time to the end time, The difference from the target driving position is used as error information (error data), and information from the angular velocity sensor 10 is continuously stored in RAM 121 as vibration data.

In S3090 , an instruction instructing to stop the shake correction (shake correction stop instruction) is sent to the interchangeable lens 102 . In the interchangeable lens 102 that has received the shake correction stop instruction (S4070), the shake correction control is stopped at S4080.

In S3100, post-exposure processing, such as mirror lowering and charging, is performed. In S3110 , a shake correction lens lock instruction indicating the lock of the shake correction lens 70 is sent to the interchangeable lens 102 . In the interchangeable lens 102 that has received the shake correction lens lock command (S4090), the shake correction lens is locked at S4100.

Entering FIG. 28, in S3120, it is judged in the shake correction mode judging section 145 provided on the camera body 101 that the "optical correction operation mode" (only Perform optical shake correction), or select the "image restoration operation mode" (perform optical shake correction and image restoration). If it is the "image restoration operation mode", go to S3130, and if it is the "optical correction operation mode", go to S3170.

In addition, even on the interchangeable lens side, corresponding to the operation in S3120, if it is "image recovery operation mode", it will go to S4120, otherwise it will go to S4150 (S4110).

In S3130, an instruction to request the error data and vibration data previously stored in the RAM 121 is sent to the interchangeable lens 102. The interchangeable lens 102 having received the requested error data and vibration data (S4120) transmits the error data to the camera body 101 in S4130.

In S4140, it is judged whether the transmission of the two data is completed. If the transmission is completed, go to S4150. If the transmission is not completed, return to S4130 to continue sending the error data. In the camera body 101 that has received the error data and the vibration data (S3140), it is judged in S3150 whether the transmission of the two data is completed, and when the transmission is completed, it proceeds to S3160, and when the transmission is not completed, it returns to S3140 to continue the transmission of the two data.

In S3155, a point image function is calculated using the received vibration data. In S3160 , the error data and the point image function are associated with the captured image and stored in the image recording unit 120 . In S3170 , the captured image is stored in the image recording unit 120 . In addition, in the case of this step, unlike S3160, error data is not saved. In S3180, a command is sent to the interchangeable lens 102 to urge turning off the power supply, and thereafter, in S3190, the camera body 101 is turned off, and the operation ends.

When the interchangeable lens 102 receives the command to urge to turn off the power supply (S4150), the angular velocity sensor 10 and other circuits are turned off at S4160, and the operation ends.

The stored image data, the point image function calculated by the point image function calculation unit 100 and the error data are sent to the image reproduction device 2, and the point image function is corrected by the function correction unit 240 based on the error data.

Thereafter, image restoration is performed in the image restoration computing unit 210 . At this time, image restoration is performed using the point image function corrected based on the error data. The restored image obtained in this way is subject to shake correction in consideration of the tracking error of the shake correction lens 70, and an image with a high shake correction effect can be obtained.

According to the present embodiment, in the interchangeable-lens camera system, the drive error of the shake correction lens 70 is detected for image restoration, so when the drive characteristics of the shake correction lens are different for each interchangeable lens, any interchangeable lens can be used The best shake correction effect can be obtained.

Not limited to the embodiment described above, various deformation|transformation and changes are possible, and these are all within the scope of the present invention.

In the first and third embodiments, examples were shown in which data extraction processing was performed to reduce the amount of data, but the present invention is not limited thereto, and calculation and storage of reference values and calculation and storage of errors may be performed without performing data extraction.

In this embodiment, an example is shown in which a point image function is calculated using a reference value or vibration data output from the angular velocity sensor 10 and an example is corrected using error information. Correction is carried out with the operation function of the two sides of vibration data.

In addition, the point image function may be obtained from either or both of the reference value and the vibration data and the error information, and image restoration may be performed without correcting the point image function.

Furthermore, the output of the angular velocity sensor 10 may not be used at all, and the point image function may be calculated only from the error information. When the point image function is calculated only from this error information, the reference value or vibration data does not need to be stored or transmitted through communication, so that the work efficiency can be improved and the work time can be shortened.

In this embodiment, an example is shown in which the point image function calculation unit 100 is mounted on the camera body 101 of the camera 1 , but the present invention is not limited thereto. For example, the point image function calculation unit 100 may be mounted on the image playback device 2 . Similarly, in the fifth embodiment, the function correcting unit 240 is installed in the image playback device 2 , but the present invention is not limited thereto, and the function correcting unit may be provided in the camera body 101 of the camera 1 , for example. In this way, various combinations are possible as a method of utilizing information related to the control position error. This combination example is shown in Table 2 below.

Table 2

No. camera, camera body Image playback device Remark 1 Point Image Function Calculation Section Function Correction Section Image Restoration Calculation Section 2 Point image function operation part (reflecting error) image restoration operation part 3 Point image function operation part (calculation based on error only) image restoration operation part

No. camera, camera body Image playback device Remark 4 Point Image Function Operation Department Function Correction Department Image Restoration Computing Department Corresponding to the third embodiment 5 Point image function operation department Function Correction Department Image Restoration Operation Department Corresponding to the fifth embodiment 6 Point image function operation unit (reflecting error) Image Restoration Computing Department 7 Point image function calculation department (calculation based on error only) Image Restoration Computing Department 8 Point Image Function Calculation Section Function Correction Section Image Restoration Calculation Section 9 Point image function operation part (reflecting error) image restoration operation part 10 Point image function operation part (calculation based on error only) image restoration operation part

No. 4 in Table 2 corresponds to the third embodiment, and No. 5 corresponds to the fifth embodiment. Any of the aspects shown in Table 2 can similarly exhibit the effects of the present invention.

In addition, according to the methods shown in No. 3, 7, and 10 in Table 2, as described above, the reference value or vibration data does not need to be stored or transmitted through communication, and the work efficiency can be improved and the work time can be shortened.

As described above in detail, according to the embodiment of the present invention, the following effects can be achieved.

Owing to having: control position error output part, the difference of the drive target position of shake correction optical system obtained by control part and the actual drive position of shake correction optical system output from position detection part is output as control position error; and image restoration operation The part, by adding the image processing of the image captured by the camera part to the image processing of the control position error, corrects the image shake; can correct the remaining image shake according to the drive control error of the shake correction optical system, and also includes not as the target. When optical blur correction is performed, image blur can always be reliably corrected with a high blur correction effect.

It includes a function correction unit that corrects the point image distribution function by controlling the position error, and an image restoration calculation unit that processes the point image distribution function corrected by the function correction unit to perform image restoration, so it is possible to correct the drive control of the optical system by shaking Image blur remaining after error correction includes cases where image blur is not corrected by optical blur correction as intended, and image blur can always be reliably corrected with a high blur correction effect.

Since there is a point image distribution function calculation unit for calculating the point image distribution function using the control position error, it is possible to calculate the point image distribution function reflecting the drive control error of the shake correction optical system. Therefore, residual image shake can be corrected by the drive control error of the shake correction optical system, and image shake can always be reliably corrected with a high shake correction effect even when the shake correction is not performed as intended by optical shake correction.

Possibility of industrial use

The above has described a digital camera that captures still images, but the present invention can be similarly applied to a digital camera that captures moving images.

Claims (4)

1. blur correction camera system comprises:

Vibration detecting part detects vibration and output vibration detection signal;

The reference value operational part, the reference value of the described vibration detection signal of computing;

The jitter correction optical system is driven according to the signal that deducts described reference value from described vibration detection signal, carries out the first picture jitter correction;

Image pickup part is taken the picture that is formed by the photographic optical system that comprises described jitter correction optical system;

Point is as the distribution function operational part, and is needed as distribution function according to described reference value arithmograph picture recovery computing; With

Image recovers operational part, carries out image recovery computing according to described as distribution function, and the image of being taken by described image pickup part is carried out the second picture jitter correction,

Even described second shake of proofreading and correct as jitter correction is to have carried out described first as the also residual shake composition that has of jitter correction.

2. blur correction camera system according to claim 1,

With described image pickup part take rest image time correlation temporal information and take the relevant time sequence information of the sequential of described rest image with described image pickup part and be provided for describedly as the distribution function operational part, describedly utilize described temporal information and described time sequence information computing described as distribution function as the distribution function operational part.

3. blur correction camera system according to claim 1,

Also have image and recover judging part, even utilize the focal length of described photographic optical system, described image pickup part to take the time of rest image and carried out described first, judge whether that recovering operational part by described image carries out the described second picture jitter correction as in the also residual shake composition that has of jitter correction any.

4. according to each described blur correction camera system in the claim 1 to 3,

Described second shake of proofreading and correct as jitter correction is the rest image shake of the rest image of described image pickup part shooting.

Images