HK1184864A - Method and system for an adaptive auto-focus algorithm - Google Patents

Abstract

A method and system for auto-focusing is presented that receives an image at an image capture device, determines a focus measure of the image, and adjusts a focus of the image by moving a lens in the image capture device in a first direction until a maximum focus measure position is reached. The method and system then evaluates a noise level of the focus measure and continues to adjust the focus of the image by further moving the lens in the first direction until the focus measure is decreased by an adaptive threshold amount, wherein the adaptive threshold amount is based on the noise level of the focus measure. The method and system then adjusts the focus of the image by moving the lens in a second direction to the maximum focus measure position.

Description

Method and system for adaptive auto-focus algorithm

Technical Field

The present invention relates to image processing, and more particularly, to image autofocus based on noise levels of focus measurements.

Background

Many current digital cameras provide an auto-focus feature consisting of an optical system with one or more sensors and a control system that positions the lens along the optical axis to automatically detect the best focus. The sensor evaluates the focus measurement (usually the sum of the gradients in the image) and stops the lens at the position of the maximum focus measurement (which is the position of the sharpest focus).

To determine the actual sharpest focus position that has been found, many digital cameras employ an auto-focus algorithm that includes a process of overshooting the sharpest focus by continually moving the lens past the suspected sharpest focus, thereby blurring the image. If this blur occurs, the camera assumes that the sharpest focus position has actually been found and returns the lens to the position where the focus measurement is maximized.

The time taken for the camera to reach the sharpest focus is critical to being able to acquire the desired image. In an attempt to minimize the time to reach the sharpest focus position, the autofocus algorithm may limit the amount of overshoot while attempting to verify the sharpest focus. However, doing so results in unreliable verification and may result in missing the sharpest focus position. More deliberate auto-focus algorithms may significantly increase overshoot to ensure verification of the location of the sharpest focus, but may result in an unpleasant blur being displayed to the camera operator.

The ideal choice of overshoot in the autofocus algorithm actually depends on the particular scene (scene) being focused, including the movement of objects, the illumination of the scene, and the contrast within the image. In highly illuminated images with high contrast, only a small amount of overshoot is required to verify the location of the sharpest focus. In low light conditions, the image and focus measurements are blurred with noise in the image, thereby causing a longer time overshoot.

Typical autofocus algorithms introduce a predefined blur overshoot threshold that is designed for a set of "normal" conditions, but is not most effective in high brightness/low noise environments and is not available in low brightness/high noise environments. These deficiencies result in non-optimal autofocus in high and low focus measurement situations and may not autofocus in low brightness conditions.

Therefore, there is a need for an adaptive auto-focus algorithm that is capable of determining an overshoot threshold based on the noise level of the focus measurement. Further, the adaptive threshold is designed to be a number of current noise levels of the focus measurement.

Disclosure of Invention

In order to solve the above problems, the present invention provides the following auto-focusing system and auto-focusing method:

(1) a system for auto-focusing, comprising:

an image acquisition device configured to receive an image;

a controller configured to determine a focus measurement of the image and a noise level of the focus measurement; and

a focusing system configured to:

determining an adaptive threshold based on the noise level; and

adjusting the focus of the image by moving a lens in the image acquisition device using an adaptive threshold level until a maximum focus measurement position is reached.

(2) The system of (1), wherein the controller continuously determines the focus measurements and the noise levels of the focus measurements.

(3) The system of (1), wherein the focusing system determines the adaptive threshold for the focus measurement as a plurality of the noise levels for the focus measurement.

(4) The system of (1), wherein the focus system verifies the maximum focus measurement location by overshooting the maximum focus measurement location by an adaptive threshold amount and then returning to the maximum focus measurement location.

(5) The system of (1), wherein the controller determines the focus measurement based on intensity.

(6) The system of (1), wherein the controller determines the focus measurement based on a gradient sum of the images.

(7) A system for auto-focusing, comprising:

an image acquisition device configured to receive an image;

a controller configured to determine a focus measurement of the image and a noise level of the focus measurement; and

a focusing system configured to:

determining an adaptive threshold for the focus measurement based on the noise level of the focus measurement;

adjusting the focus of the image by moving a lens in the image acquisition device in a first direction until the focus measurement decreases by a value equal to the adaptive threshold; and

moving the lens in the image capture device in a second direction.

(8) The system of (7), wherein the focusing system moves the lens in the second direction until the focus measurement reaches a maximum value and then decreases from the maximum value by a value equal to the adaptive threshold.

(9) The system of (7), wherein the focusing system moves the lens in the second direction until a maximum focus measurement position is reached.

(10) A method for auto-focusing, comprising:

receiving an image in an image acquisition device;

determining a focus measurement for the image;

adjusting the image focus by moving a lens in the image capture device in a first direction;

estimating a noise level of the focus measurement;

continuing to adjust the focus of the image by further moving the lens in the first direction until the focus measurement decreases by an adaptive threshold amount; and

adjusting the focus of the image by moving the lens in a second direction to a maximum focus measurement position,

wherein the adaptive threshold amount is based on a noise level of the focus measurement.

(11) The method of (10), wherein said estimating a noise level of said focus measurements is performed continuously.

(12) The method of (10), wherein said estimating the noise level of the focus measurement is performed periodically.

(13) The method of (10), wherein the adaptive threshold amount is continuously estimated.

(14) The method of (10), wherein the adaptive threshold amount is a plurality of noise levels of the focus measurements.

(15) The method of (10), wherein the adaptive threshold is increased due to a decrease in illumination of the image.

(16) The method of (10), wherein the determination of the focus measurement is based on light intensity.

(17) The method of (10), wherein the determination of the focus measurement comprises summing gradients of the images.

(18) The method of (10), further comprising, before adjusting the focus of the image by moving the lens in the image acquisition device in the first direction until a maximum focus measurement position is reached, moving the lens in the second direction, wherein the focus measurement is decreased.

(19) The method of (18), further comprising:

adjusting the focus of the image by moving the lens in the first direction once the focus measurement is determined to be decreasing.

(20) The method of (19), wherein determining that the focus measurement is decreasing comprises moving the lens in the second direction until the focus measurement decreases by an adaptive threshold amount.

Drawings

Exemplary embodiments are described with reference to the accompanying drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Additionally, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears.

FIG. 1 illustrates an example of an autofocus system for an image acquisition device;

FIG. 2 is an example of a diagram showing lens positions for a given focus measurement in an auto-focus system, excluding noise level components of the focus measurement;

FIG. 3 is an example of a diagram showing lens position given a focus measurement in an auto-focus system, showing noise sampling of the focus measurement estimate;

FIG. 4 is a detailed diagram illustrating lens positions given a focus measurement in an auto-focus system showing noise sampling of the focus measurement estimate, according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating lens positions given focus measurements in an auto-focus system with low noise, according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating lens position for a given focus measurement in an auto-focus system with high noise, according to an embodiment of the present invention;

FIGS. 7 and 8 show block diagrams of exemplary methods for adaptive auto-focus algorithms based on noise level of focus measurements, according to embodiments of the present invention; and

FIG. 9 is a diagram of an example implementation of an auto-focus system according to an embodiment of the invention.

Exemplary embodiments will now be described with reference to the accompanying drawings.

Detailed Description

The following detailed description refers to the accompanying drawings to illustrate exemplary embodiments. References in the detailed description to "one exemplary embodiment," "an exemplary embodiment," etc., indicate that the exemplary embodiment described may include a particular feature, structure, or characteristic, but every exemplary embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same exemplary embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an exemplary embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other exemplary embodiments whether or not explicitly described.

The exemplary embodiments described herein are provided for illustrative purposes and are not intended to be limiting. Other embodiments are possible, and modifications to the exemplary embodiments are possible within the spirit and scope of the disclosure. Therefore, the detailed description is not intended to be limiting. Rather, the scope of the invention is to be defined only by the claims and their equivalents.

Implementations may be performed in hardware (e.g., digital cameras and/or circuitry), firmware, software, or any combination thereof. Embodiments may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by one or more processors. A machine-readable medium may include any mechanism for storing information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include Read Only Memory (ROM); random Access Memory (RAM); a magnetic disk storage medium; an optical storage medium; a flash memory device. Further, firmware, software, routines, instructions may be described herein as performing certain actions. However, it should be understood that such descriptions are merely for convenience and that such actions in fact result from computing devices, processors, controllers, or other devices executing the firmware, software, routines, instructions, etc.

For purposes of discussion, the term "module" should be understood to include at least one of software, firmware, and hardware (such as one or more of a circuit, microchip, or device, or any combination thereof), and any combination thereof. Further, it should be clear that each module may comprise one or more than one component within the actual device, and that each component forming part of the module may cooperate or function independently with any other component forming part of the module. Rather, the various modules described herein may represent a single component within an actual device. Further, the components within a module may be in a single device, or distributed among multiple devices in a wired or wireless manner.

The following detailed description of exemplary embodiments will so fully reveal the general nature of the disclosure that others can, by applying knowledge known to those skilled in the art, readily modify and/or adapt for various applications such specific embodiments without undue experimentation, without departing from the spirit and scope of the present disclosure. Accordingly, such changes and modifications are within the meaning and equivalents of the exemplary embodiments in accordance with the teachings and guidance presented herein. It is to be understood that the phraseology or terminology herein is for the purpose of description and not of limitation, such that the terminology or phraseology of the present specification is to be interpreted by the skilled artisan in light of the teachings herein.

Although the description of the embodiments is in terms of an image acquisition device, in particular, a digital camera, those skilled in the art will recognize that the embodiments may be applied to other imaging devices without departing from the spirit and scope of the present disclosure.

Digital imaging

Fig. 1 shows a block diagram of an auto-focus (AF) system 100 according to an embodiment of the invention. The AF system 100 includes an image acquisition apparatus 110 capable of receiving and acquiring an image of a subject 120. Image capture device 110 includes a movable lens 112, an image sensor 114, an analog-to-digital converter (ADC) 116, a digital Image Signal Processor (ISP) 118, a camera controller 120, and a focus drive system 122.

The lens 112 is moved in any direction by the focus drive system 122, but is shown moving in a plane indicated by the direction of movement 113. Camera controller 120 analyzes the output of image sensor 114 to determine whether lens 112 should be moved by focus drive system 122 in order to focus an image of object 120 (referred to herein as image 120) on image sensor 114. The ADC116 converts analog image data of the image 120 into digital image data.

The camera controller 120 determines whether the image 120 is in focus by measuring a focus measurement that is evaluated by processing the image 120 in the ISP 118. The focus measurement is a mathematical value obtained from the pixel value processing of the image 120 on the image sensor 114. There are many algorithms for processing the image 120, including simple focus measurements of the sum of gradients across the image, such as the sum of absolute differences between adjacent pixels. With this algorithm, if the image is blurred, neighboring pixels will have very similar values and thus the focus measurement will be relatively low. If the image is sharp, the difference between adjacent pixels, and therefore the focus measurement, will be relatively high. The sharpest image corresponds to the largest sum of adjacent pixel difference values and represents the point at which the image 120 is in focus.

In more complex AF algorithms, additional processing may be used to perform additional image analysis. For example, only pixels of a particular region of the image are analyzed, e.g., pixels in the center of the image, or filters for pixel brightness, color, and stereo frequency, for example, may be applied.

Camera controller 120 is also confronted with other information, such as noise, contained within image 120 as image sensor 114 analyzes image 120. Image noise is a common problem in digital photography. Image noise can be viewed as simulating film grain in film photography. Image noise is typically a random variation of illumination or color in the acquired image, which adds false and extraneous information.

Image noise becomes more noticeable under low light conditions. At low light, proper exposure of the image requires the use of longer shutter speeds, higher ISO settings (e.g., gain or exposure index), or a combination of both. In a digital camera, longer shutter speeds and/or higher ISO settings result in a relative increase in noise within the image 120. To achieve focus quickly and efficiently, as discussed later, the ISP118 introduces analysis of the noise level in the image 120 and subsequent control of the focus drive system 122 and lens 112.

Overshoot during autofocus

Fig. 2 is an example of a diagram showing a basic cycle of an AF algorithm according to an embodiment of the present invention, showing lens positions given focus measurements. The diagram in fig. 2 does not show any image noise or associated effects on the focus point measurement as discussed later.

As mentioned previously, to determine the actual sharpest focus position that has been found, digital cameras typically employ an autofocus algorithm that includes overshooting the sharpest focus by continuously moving the lens past the suspected sharpest focus, thereby blurring the image. When this blur occurs, the camera assumes that the sharpest focus position has been found and passed, and returns the lens to the position where the focus measurement is maximized.

Fig. 2 shows a graph of focus measurements as a function of lens position. The location where the focus measurement is maximized (at point D) indicates the sharpest focus location. Point a indicates a possible starting position of the lens 112. The focus drive system 122 arbitrarily selects the direction 113 in which the lens 112 starts to move. In the case where the "correct" direction is selected, the lens 112 begins to move from point a to point D. However, to determine whether point D is the maximum focus measurement, camera controller 120 instructs focus drive system 122 to continue moving lens 112 in the same direction until camera controller 120 determines that the focus measurement of image 120 on image sensor 114 is decreasing. Once the camera controller 120 makes such a decision, it instructs the focus drive system 122 to move the lens 112 in the opposite direction, returning to point D.

When the focus drive system 122 incorrectly selects the starting direction 113 to move the lens 112, the lens 112 begins to move from point a to point B. The camera controller 120 senses that the focus is decreasing and then reverses the direction of movement 113 of the lens 112 so that the lens 112 moves towards point D. As in the previous example, to determine whether point D is the maximum focus measurement, camera controller 120 instructs focus drive system 122 to continue moving in the same direction until camera controller 120 determines that the focus measurement of image 120 on image sensor 114 is decreasing. Once the camera controller 120 makes such a decision, it instructs the focus drive system 122 to move the lens 112 in the opposite direction, returning to point D.

The movement of the lens 112 from points a to B and from points D to C may be considered to be counterproductive to the process of reaching the sharpest focus point. Likewise, if there is no noise in the image 120, the movement of the lens in the incorrect initial direction from A to B and in the case of over-adjustment verification from point D to C can be minimized to a point where substantially no inefficiencies arise due to lens movement. However, since the digital image itself includes noise, overshoot verification is necessary, especially in low light and high ISO settings.

Noise impact during auto-focus

Fig. 3 is a diagram example of a basic loop of an AF algorithm according to an embodiment of the present invention, which includes the influence of image noise. Although FIG. 2 shows a smooth transition between points, FIG. 3 includes random image noise, as shown by the smaller points with reference point A, B, C for comparison, and D. In fact, as shown in fig. 2, the AF algorithm does not have a smooth clear focus measurement curve, but rather is faced with noisy samples of focus measurement estimates at certain points in time when the lens position is modified.

The measurements in fig. 3 are noisy and fluctuate randomly due to inherent noise in the image, but also due to other factors such as random vibration of the camera, motion, varying illumination, ISO value settings, etc. Thus, movement from the initial lens position A to either B or D in any direction will result in some point focus value being reduced. In this case, rather than indicating that the sharpest focus point has been reached, it is merely indicating that a "local" peak point has been found.

Fig. 4 shows the case where the lens 112 in the AF system has an initial position shown by point a, a. The focus drive system 122 moves the lens 116 toward point D in an attempt to locate the sharpest focus point, e.g., the point at which the focus measurement is maximized. As the lens 116 begins to move toward point D, the focus measurement increases and then decreases. In a noise-free environment, camera controller 120 may conclude that the maximum focus measurement was reached because the focus measurement began to decrease. However, as shown in fig. 4, the "maximum" detection point is not the sharpest focus point but only a local maximum due to image noise.

As the lens 116 continues to move past the local maximum point, the local minimum point is reached, after which the focus measurement again increases and overshoots the local maximum. This situation indicates that the true maximum focus measurement has not been reached and that the suspected maximum focus measurement is only a local maximum.

To avoid such false detection, a large number of focus measurements must be detected before it is confirmed that a true focus measurement point has been detected and the direction of movement of the inverse lens 112 is returned to the actual sharp focus point. The value of overshoot is a key tradeoff within any AF algorithm. If the amount of overshoot is too small, then a local maximum may be erroneously identified at the maximum focus measurement point. If the value of overshoot is too large, then overshoot occurs after the sharp focus position is exceeded, wasting time and producing a blurred image that is observed. Typical AF algorithms attempt to mitigate the occurrence of false identifications and excessive overshoot by defining intermediate range overshoot values that are not optimal over the range of noise conditions, which in most cases play a role. Fig. 5 and 6 will discuss adaptive auto-focusing with a full range noise scenario.

Adaptive auto-focusing in low noise

Fig. 5 is an example of a diagram showing a loop of an adaptive AF algorithm according to an embodiment of the present invention, showing lens positions given focus measurements in an autofocus system with low noise information content. As mentioned previously, image noise increases in low light, high ISO situations, while noise information decreases in high brightness, low ISO environments. The adaptive autofocus algorithm identifies relatively low noise information contained in the image and thus reduces the threshold of the corresponding overshoot value. Fig. 5 again shows the initial starting position of the lens 112 at point a. The relationship between the focus measurement and the lens position in the noise-free case is also shown, labeled as a noise-free trajectory. Camera controller 120 periodically or continuously determines the amount of noise information in the received image. The amount of noise information is illustrated as an offset between a particular point and a noise-free trajectory. An example of a point W and a marked low noise delta (Δ) is shown.

With low noise Δ, camera controller 120 assigns an overshoot threshold based on the amount of noise Δ (e.g., low noise Δ). In an embodiment, the threshold is set to a number of noise measurements, e.g. 3x noise Δ, at a particular point in time. Thus, the amount of overshoot required to verify that the true sharpest focus position is point D is shown as the overshoot Δ between point D and point X.

Since the image noise level is not necessarily constant, in an embodiment, camera controller 120 continuously monitors the amount of noise in the received image and can continuously modify the overshoot threshold. In another embodiment, camera controller 120 periodically determines the amount of noise in the received focus measurements and modifies the overshoot threshold. In any event, the overshoot threshold adapts to the noise level in the received focus measurements.

Although the lens 112 is moved in the embodiment, the current focus measurement is continuously monitored within the AF system 100. The initial current focus measurement value is stored within the AF system 100 and is referred to as the maximum focus measurement. When the current focus measurement exceeds the stored maximum focus measurement, the stored maximum focus measurement is replaced with a new maximum focus measurement. Lens movement continues until the focus measurement begins to decrease. Since the reduction in focus measurements can be caused by random noise fluctuations, lens movement continues in the same direction until a statistically significant reduction in focus measurements is observed. Two different situations are possible at this time.

The first situation occurs where the focus measurement is decreasing all the time during the lens movement, and a statistically significant maximum is not reached and passed during the lens movement. In this case, the lens movement direction is reversed, and the process is repeated. Given this reverse direction, maximum focus is reached, and then the focus measurement drops.

The second situation occurs where the focus measurement initially increases during lens movement, reaches a statistically significant maximum, and then the focus measurement begins to decrease. In this case, the position of maximum focus is determined and the lens is moved back to the position of maximum focus.

In the conventional prior art, the AF algorithm determines the overshoot threshold from a predetermined or heuristically defined constant. In the present disclosure, the desired overshoot is determined as a statistically significant maximum of the focus measurement noise. This approach gives the advantage of minimizing overshoot. Users generally believe that the AF system pre-forms unnecessary, or excessive, autofocus searches when the perceived image is significantly blurred. In the present embodiment, the lens is moved just to a point where blur due to defocus is comparable to blur due to noise, and therefore no overshadowing blur is observed. However, in the case of significant image noise, the threshold value will be increased accordingly and robust and reliable AF operation will be maintained.

The same approach is used for the case where, from the initial lens start position a, the focus drive system 122 selects the advancing direction of the lens 112 in the opposite direction of the focal point D. In this case, the camera controller 120 will determine the noise content of the image 120 based on the plurality of noise measurements and select an overshoot threshold. When the threshold is exceeded, the focusing system 113 reverses the direction of travel of the lens 112.

Adaptive auto-focusing in high noise

FIG. 6 is an example of a diagram showing an adaptive AF algorithm loop showing lens positions given a focus measurement in an auto-focus system with high noise information content, according to an embodiment of the present invention. The adaptive autofocus algorithm identifies relatively high noise information contained in the image, thus increasing the threshold of the corresponding overshoot value to ensure accurate maximum focus measurement determination. While adaptive auto-focusing in low noise allows for a faster and more efficient determination of the sharpest focus position, adaptive auto-focusing in high noise environments allows for an accurate maximum focus measurement determination where typical AF algorithms cannot determine any type of maximum focus measurement position.

Fig. 6 again shows the initial starting position of the lens 112 at point a. The relationship between the focus measurement and the lens position in the noise-free case is also shown, labeled as a noise-free trajectory. Camera controller 120 periodically or continuously determines the amount of noise information in the received image. The amount of noise information is illustrated as an offset between a particular point and a noise-free trajectory. An example of a point Y and a marked high noise delta is shown. Note that the high noise Δ is larger than the low noise Δ shown in fig. 5.

In the case of high noise Δ, camera controller 120 assigns an overshoot threshold, such as high noise Δ, based on the amount of noise Δ. In an embodiment, the threshold is set at a number of noise measurements at any particular point in time, such as 3x noise Δ. Thus, the amount of overshoot required to verify that the true sharpest focus position is point D is shown as the overshoot Δ between points D and Z. Again, note that high noise overshoot Δ is greater than the low noise overshoot shown in FIG. 5.

As previously mentioned, the image noise level need not be constant, and the camera controller 120 can continuously or periodically monitor the amount of noise in the received image and modify the associated overshoot threshold.

Once the threshold level has been obtained, camera controller 120 instructs focusing system 113 to return lens 112 to the maximum focus measurement position, which is indicated by point D in fig. 6. Since the amount of noise level at a particular point is known, and has been determined to be relatively large in this example, a reasonable amount of overshoot verification point D is required to be actually the maximum focus measurement location. Thus, in practice, the sharpest focus position D may be determined, however typical AF algorithms use a "standard" or nominally overshoot threshold that is not based on the current noise information content of the image, and cannot determine any type of non-focus position.

The same approach is used in the case where, from the initial lens start position a, the focus drive system 122 selects the advancing direction of the lens 112 in the opposite direction of the focal point D. In this case, the camera controller 120 will determine the noise content of the image 120 based on the plurality of noise measurements and select an overshoot threshold. When the threshold is exceeded, the focusing system 113 will reverse the direction of advance of the lens 112.

Exemplary method of adaptive auto-focusing Algorithm

The method according to the embodiment is described in connection with the adaptive auto-focus algorithm shown in fig. 2 to 6 and the system described in fig. 1, but is not limited thereto.

Fig. 7 is a flow chart of an exemplary method 700 of an adaptive auto-focus algorithm based on a noise level of a focus measurement in accordance with an embodiment of the present invention. For ease of explanation, method 700 is described with reference to the adaptive auto-focus system of fig. 1 using the methods of fig. 2-6, but method embodiments are not limited thereto.

The method 700 begins at step 702 where an image is received at an image acquisition device. In an embodiment, the autofocus system 100 includes an image acquisition device 110 that is capable of receiving and acquiring an image of an object 120.

The method 700 continues by determining a focus measurement for the image in step 704. In an embodiment, image acquisition device 110 receives an image of object 120 at image sensor 114. The camera controller 120 determines a focus measurement of the image sensor 114.

The method 700 continues by adjusting the focus of the image by moving a lens in the image acquisition device in a first direction until a maximum focus measurement is obtained in step 706. In an embodiment, the focus drive system 122 arbitrarily selects the direction 113 in which to start moving the lens 112. As shown in FIG. 2, in the case where the "correct" direction is selected, the lens 112 begins to move from point A to point D, which is the maximum focus measurement.

The method 700 continues by estimating the noise level of the focus measurement at step 708. In an embodiment, camera controller 120 periodically or continuously determines the amount of noise information in the received image. The amount of noise information is shown as the offset between a particular point and the noise-free trajectory. Fig. 5 shows an example of point W and labeled low noise delta.

The method 700 continues by moving the lens further in the first direction in step 710 to continue adjusting the focus of the image until the focus measurement decreases by the adaptive threshold amount. In an embodiment, to determine whether point D is the maximum focus measurement, camera controller 120 instructs focus drive system 122 to continue moving lens 120 in the same direction until camera controller 120 determines that the focus measurement of image 120 on image sensor 114 decreases. Once camera controller 120 makes such a determination, it instructs focus drive system 122 to move lens 112 in the opposite direction back to point D.

The method 700 continues by verifying that the maximum focus measurement represents a sharp focus location based on the adaptive threshold amount in step 712. In an embodiment, in the case of a relatively low noise Δ as shown in fig. 5, camera controller 120 assigns an overshoot threshold based on the amount of noise Δ (e.g., low noise Δ). In an embodiment, the threshold is set at a plurality of noise measurements, e.g., 3x noise Δ, at any particular point in time. Thus, the amount of overshoot required to verify that the true sharpest focus position is point D is shown as the overshoot Δ between points D and X.

The method 700 continues by adjusting the focus of the image by moving the lens in a second direction to a sharp focus position in step 714, where the adaptive threshold amount is based on a noise level of the focus measurement. In an embodiment, once the threshold level has been obtained, camera controller 120 instructs focusing system 113 to return lens 112 to the maximum focus measurement position, which is indicated by point D in fig. 5 and 6. The method 700 then ends.

Fig. 8 is a flow chart of an exemplary method 800 for an adaptive auto-focus algorithm based on a noise level of a focus measurement according to an embodiment of the present disclosure. For ease of explanation, method 800 is described using the methods of fig. 2-6 with reference to the adaptive auto-focus system of fig. 1, but embodiments of the method are not limited thereto.

The method 800 begins at step 802 by selecting a direction of movement of a lens and moving the lens in the selected direction. In an embodiment, the autofocus system 100 includes an image acquisition device 110 that is capable of receiving and acquiring an image of an object 120. In an embodiment, the focus drive system 122 arbitrarily selects the direction 113 in which to start moving the lens 112, and starts moving the lens 112 in that direction.

The method 800 continues by determining a focus measurement for the image in step 804. In an embodiment, image acquisition device 110 receives an image of object 120 at image sensor 114. Camera controller 120 determines a focus measurement of image 120 at image sensor 114.

The method 800 continues by moving the lens in step 806 until the focus measurement is statistically significantly reduced compared to the noise level of the focus measurement. In an embodiment, the focus drive system 122 moves the position of the lens 112 in the direction 113 while the focus measurements are monitored by the ISP 118. If the current focus measurement value exceeds the previous focus measurement, then ISP118 updates the stored maximum focus measurement with the new value. This process is continued until a statistically significant decrease in focus measurements is observed.

The method 800 continues by evaluating whether a statistically significant focus maximum has been obtained in step 808. In an embodiment, the camera controller 120 periodically or continuously determines the amount of noise information in the received image. The amount of noise information is shown as the offset between a particular point and the noise-free trajectory. An example of a point W and a label with low noise delta is shown in fig. 5. In an embodiment, to determine whether point D is the maximum focus measurement, camera controller 120 instructs focus drive system 122 to continue moving lens 112 in the same direction until camera controller 120 determines that the focus measurement of image 120 on image sensor 114 is decreasing.

The method 800 continues by determining whether a maximum focus measurement has been obtained at step 810. If it is determined that the maximum focus measurement has actually been obtained, the method 800 proceeds directly to step 812 so that the lens is moved to the sharpest focus position. If the maximum focus measurement is not obtained, then step 810 changes the direction of movement of the lens and moves the lens until a statistically significant decrease in focus measurement is observed. In an embodiment, once camera controller 120 makes a determination that a focus maximum has not been obtained, focus drive system 122 is instructed to move lens 112 in the opposite direction, returning to point D.

The method 800 continues by moving the lens to the sharpest focus position at step 812. In an embodiment, once the threshold level has been obtained, camera controller 120 instructs focusing system 113 to return lens 112 to the maximum focus measurement position, which is indicated by point D in fig. 5 and 6. The method 800 then ends.

One skilled in the relevant art will recognize that the method may additionally or alternatively include any of the functionality of the autofocus system 100 discussed above, and the above description of the exemplary method should not be taken as limiting the method, nor the description of the adaptive autofocus system.

Example processor System implementation

Fig. 9 shows a block diagram of an auto-focus (AF) system 900 according to an embodiment of the invention. The AF system 900 includes an image acquisition device 910 capable of receiving and acquiring an image of a subject 920. The image acquisition apparatus 910 includes a lens 912 movable along an optical axis 913, an image sensor 914, an analog-to-digital converter (ADC) 916, a digital Image Signal Processor (ISP) 918, a camera controller 920, a focus drive system 922, a storage unit 930, a RAM940, and a video output port 950.

In an embodiment, a signal is obtained from the image sensor 914 and converted to a digital format by the ADC 916. The digital format image is then processed by the ISP918, and the ISP918 processes the digital image and generates an output video of the image capture device 910 at the video output port 950. The video output signal may be in digital or analog form for display on a camera screen, and/or storage, and/or transmission via cable or wireless, and/or further signal processing and/or playback, compression and/or encoding, and so forth.

ISP918 also evaluates parameters such as image brightness and contrast for camera control and image tuning. ISP918 estimates these parameters by processing the image or video stream. In particular, ISP918 estimates focus measurements, which are transmitted to camera controller 920, which in turn controls lens position via focus drive system 922.

Conclusion

It is to be understood that the detailed description, and not the abstract, is intended to be used to interpret the claims. The abstract may set forth one or more, but not all exemplary embodiments, and is therefore not intended to limit the claims in any way.

The embodiments have been described above with the aid of functional building blocks illustrating the implementation of specific functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed.

It will be apparent to persons skilled in the relevant art that various changes in form and detail can be made therein without departing from the spirit and scope of the invention. Accordingly, the present invention is not limited by any of the above-described exemplary embodiments, but is only limited by the following claims and their equivalents.

Claims (10)

1. A system for auto-focusing, comprising:

an image acquisition device configured to receive an image;

a controller configured to determine a focus measurement of the image and a noise level of the focus measurement; and

a focusing system configured to:

determining an adaptive threshold based on the noise level; and

adjusting the focus of the image by moving a lens in the image acquisition device using an adaptive threshold level until a maximum focus measurement position is reached.

2. The system of claim 1, wherein the controller continuously determines the focus measurement and the noise level of the focus measurement, or the focusing system determines the adaptive threshold for the focus measurement as a plurality of the noise levels of the focus measurement.

3. The system of claim 1, wherein the focus system validates the maximum focus measurement location by overshooting an adaptive threshold amount and then returning to the maximum focus measurement location.

4. The system of claim 1, wherein the controller determines the focus measurement based on intensity or based on a gradient sum of the images.

5. A system for auto-focusing, comprising:

an image acquisition device configured to receive an image;

a controller configured to determine a focus measurement of the image and a noise level of the focus measurement; and

a focusing system configured to:

determining an adaptive threshold for the focus measurement based on the noise level of the focus measurement;

adjusting the focus of the image by moving a lens in the image acquisition device in a first direction until the focus measurement decreases by a value equal to the adaptive threshold; and

moving the lens in the image capture device in a second direction.

6. The system of claim 5, wherein the focusing system moves the lens in the second direction until the focus measurement reaches a maximum value and then decreases from the maximum value by a value equal to the adaptive threshold, or the focusing system moves the lens in the second direction until a maximum focus measurement position is reached.

7. A method for auto-focusing, comprising:

receiving an image in an image acquisition device;

determining a focus measurement for the image;

adjusting the image focus by moving a lens in the image capture device in a first direction;

estimating a noise level of the focus measurement;

continuing to adjust the focus of the image by further moving the lens in the first direction until the focus measurement decreases by an adaptive threshold amount; and

adjusting the focus of the image by moving the lens in a second direction to a maximum focus measurement position,

wherein the adaptive threshold amount is based on a noise level of the focus measurement.

8. The method of claim 7, further comprising, before adjusting the focus of the image by moving the lens in the image acquisition device in the first direction until a maximum focus measurement position is reached, moving the lens in the second direction, wherein the focus measurement is decreased.

9. The method of claim 7, further comprising:

adjusting the focus of the image by moving the lens in the first direction once the focus measurement is determined to be decreasing.

10. The method of claim 7, wherein determining that the focus measurement is decreasing comprises moving the lens in the second direction until the focus measurement decreases by an adaptive threshold amount.