CN102811309A - Method and device for generating images with shallow depth of field - Google Patents

Abstract

The invention provides a method and a device for generating a shallow depth-of-field image. The subject is photographed according to the first aperture value, so as to generate a first aperture value image. And shooting the subject according to the second aperture value to generate a second aperture value image. Wherein the second aperture value is larger than the first aperture value. In addition, the first aperture value image and the second aperture value image are analyzed to obtain an image difference value. In addition, if the image difference value is judged to be larger than the critical value, the first aperture value image is subjected to image processing to obtain a shallow depth-of-field image.

Description

Method and device for generating shallow depth-of-field images

technical field

The invention relates to an image processing method and device, in particular to a method and device for generating shallow depth of field images.

Background technique

FIG. 1 is a schematic diagram of a conventional camera lens focusing on a subject plane. Please refer to FIG. 1, when the camera lens 10 focuses on the subject plane 20, after the subject plane 20 forms the image at the focal plane 30 to achieve the clearest picture, the distance between the camera lens 10 and the subject plane 20 is distance Y, and the distance between the camera lens 10 and the focal plane 30 is the lens focal length (focal length) y. When using a camera to shoot images, in order to highlight the subject in the captured image, the shooting technique of the so-called shallow depth of field is generally used, that is, objects within the shooting distance Y can be clearly imaged, but outside the shooting distance Y objects are gradually blurred.

However, the shallow depth of field effect that general camera lenses can produce is quite limited. To obtain a better shallow depth of field effect, it is necessary to take a series of continuous shots of the same scene with different focal lengths, and find out the position of each pixel at each point. The clearest position in the image, and then use the relationship between the focal length and the depth of field to deduce the relative depth of field of each pixel in the image. However, this method requires a long processing time and consumes a large storage space, which is not conducive to commercialization.

If you take 2 to 3 images of the same scene with different focal lengths, that is to say, use a few images to deduce the relationship between the relative depth of each pixel, although the processing time can be slightly reduced, but the result is easily affected by image noise. As a result, the processed image has a discontinuous or unnatural depth of field.

Contents of the invention

In view of this, the present invention provides a method for generating an image with a shallow depth of field, which can determine the relative depth of field of images captured by two different aperture values, thereby preserving the clarity of the main body of the image and enhancing the blurring of the non-main part of the image.

The invention provides a device for generating images with shallow depth of field, which can use different aperture values to shoot the same scene, and after image processing, the clarity of the main body of the image can be preserved and the blurring degree of the non-main part of the image can be enhanced.

From one point of view, the present invention proposes a method for generating an image with a shallow depth of field, and the method includes the following steps. The subject is photographed according to the first aperture value, so as to generate the first aperture value image. The subject is photographed according to the second aperture value, so as to generate the second aperture value image, wherein the second aperture value is greater than the first aperture value. In addition, the first aperture value image and the second aperture value image are analyzed to obtain an image difference value. In addition, if it is determined that the image difference is greater than the critical value, image processing is performed on the image with the first aperture value to obtain an image with a shallow depth of field.

In an embodiment of the present invention, the step of photographing the subject according to the first aperture value so as to generate the image with the first aperture value includes focusing on the subject according to the first aperture value and then photographing the subject, and Obtain this first f-stop image. And a clear area including the subject is selected in the first aperture value image.

In an embodiment of the present invention, after the step of photographing the subject according to the second aperture value to generate the second aperture value image, it further includes using the clear area to calculate the second aperture value image Geometry transformation parameters. And perform geometric transformation on the second aperture value image according to the geometric transformation parameter to obtain the converted second aperture value image.

In an embodiment of the present invention, if it is determined that the image difference is greater than a threshold value, the step of performing image processing on the first aperture value image to generate a shallow depth-of-field image includes the following steps. If it is determined that the image difference is greater than the critical value, smoothing is performed on the first aperture value image and the converted second aperture value image, so as to obtain a relative depth of field map. And a blurring process is performed on the relative depth-of-field map to generate a blurred image. Then average the blurred image and the first aperture value image to obtain the shallow depth of field image.

In an embodiment of the present invention, the smoothing process uses an image interpolation method.

In an embodiment of the present invention, the method for generating a shallow depth-of-field image further includes directly outputting the first aperture value image if it is determined that the image difference is not greater than a critical value.

From another point of view, the present invention provides a device for generating images with a shallow depth of field, which includes an image acquisition module and a processing module. The image acquisition module shoots the subject according to the first aperture value and the second aperture value, so as to generate the first aperture value image and the second aperture value image respectively. Wherein, the second aperture value is greater than the first aperture value. The processing module is coupled to the image acquisition module, and analyzes the first aperture value image and the second aperture value image to obtain image difference. If the processing module determines that the image difference is greater than the critical value, it performs image processing on the first aperture value image to generate a shallow depth-of-field image.

In an embodiment of the present invention, the image acquisition module focuses on the subject according to the first aperture value and shoots, and generates an image with the first aperture value.

In an embodiment of the present invention, the processing module selects a clear area including the subject from the first aperture value image generated by the image acquisition module.

In an embodiment of the present invention, the device for generating a shallow depth-of-field image further includes a geometric conversion unit. The geometric conversion unit is coupled to the processing module. The geometric conversion unit uses the clear area to calculate the geometric conversion parameters of the second aperture value image, and uses the geometric conversion parameters to perform geometric conversion on the second aperture value image to generate the converted first aperture value image. Two f-stop images.

In an embodiment of the present invention, the processing module includes a smoothing processing unit and a blurring processing unit. If the processing module determines that the image difference is greater than the critical value, the processing module controls the smoothing processing unit to perform smoothing processing on the first aperture value image and the converted second aperture value image to obtain a relative depth of field map. The blurring processing unit performs blurring processing on the relative depth of field image to generate a blurred image, wherein the processing module averages the blurred image and the first aperture value image to generate a shallow depth of field image.

In an embodiment of the present invention, if the processing module determines that the image difference is not greater than a threshold value, it directly outputs the first aperture value image.

Based on the above, the method and device for generating shallow depth-of-field images provided by the present invention use different aperture sizes to cause different characteristics of image depth-of-field, shoot the same scene with different aperture values, and compare the differences between images, thereby Determine the relative depth of field of the image content. When the image difference is large enough, image synthesis can be performed to preserve the sharpness of the subject of the image and enhance the blur of non-subjects; captured image.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail with reference to the accompanying drawings.

Description of drawings

FIG. 1 is a schematic diagram of a conventional camera lens focusing on a subject plane.

FIG. 2 is a block diagram of an apparatus for generating images with a shallow depth of field according to an embodiment of the present invention.

FIG. 3 is a flowchart of a method for generating a shallow depth-of-field image according to an embodiment of the present invention.

Fig. 4 is a block diagram of an apparatus for generating images with a shallow depth of field according to another embodiment of the present invention.

FIG. 5 is a flowchart of a method for generating a shallow depth-of-field image according to another embodiment of the present invention.

Reference signs:

10: Camera Lens

20: Subject plane

30: Focus plane

200, 400: Devices that produce shallow depth-of-field images

210, 410: image acquisition module

220, 420: processing module

422: smoothing processing unit

424: Fuzzy processing unit

430: Geometric Transformation Unit

Y: Photography distance

y: lens focal length

S310-S340: each step of the method for generating a shallow depth-of-field image according to an embodiment of the present invention

S510-S580: each step of the method for generating a shallow depth-of-field image according to another embodiment of the present invention

Detailed ways

The invention proposes a method for synthesizing shallow depth of field images by using the relationship between the depth of field and the size of the aperture. Focus on close objects with a large aperture first and then shoot, then roughly frame the clear area in the image, then change to a smaller aperture to shoot the second image, analyze the two images, and use the difference between the two images The magnitude of the difference represents the relative depth of field between the subject area and other background areas, and it is determined whether to perform image synthesis with a shallow depth of field according to the image difference. In order to make the content of the present invention clearer, the following examples are listed as examples in which the present invention can actually be implemented.

FIG. 2 is a block diagram of an apparatus for generating images with a shallow depth of field according to an embodiment of the present invention. Please refer to FIG. 2 , the device 200 for generating images with a shallow depth of field in this embodiment is, for example, a digital camera, a video camera, or a smart phone with a camera function, etc. The device 200 for generating images with a shallow depth of field includes an image acquisition module 210 and a processing module 220. Its functions are described as follows:

The image acquisition module 210 includes a lens, a photosensitive element and an aperture. The lens is, for example, a standard lens, a wide-angle lens, a zoom lens, or the like. The photosensitive element is, for example, a Charge Coupled Device (CCD), a Complementary Metal-Oxide Semiconductor (CMOS) element or other elements, and the lens, photosensitive element or combination thereof are not limited herein.

The aperture refers to a group of movable blades made in the lens. By controlling the opening and closing of the blades, the amount of light entering the image acquisition module 210 within a certain period of time can be controlled. The aperture will be enlarged or reduced as the aperture value (also referred to as f-number by those skilled in the art) on the lens is adjusted. Common f values are: f1.4, f2, f2.8, f4, f5.6, f8, f11, f16, f22, f32. It should be noted here that the smaller the f value, the larger the opening of the aperture, and the more light entering; the larger the f value, the smaller the opening of the aperture, and the less light entering. Accordingly, the "large aperture" mentioned in this embodiment refers to an aperture value with a smaller f value. The image capture module 210 of this embodiment mainly uses two different aperture values to shoot the same scene, so as to generate images of the first and second aperture values.

The processing module 220 is, for example, a central processing unit (Central Processing Unit, CPU), or other programmable microprocessor (Microprocessor), digital signal processor (Digital Signal Processor, DSP), programmable controller, special application integration circuit (Application Specific Integrated Circuits, ASIC), programmable logic device (Programmable Logic Device, PLD) or other similar devices, the processing module 220 is coupled to the image acquisition module 210, in order to the image acquisition module 210 received The first and second aperture value images are analyzed and processed to generate a shallow depth of field image.

FIG. 3 is a flowchart of a method for generating a shallow depth-of-field image according to an embodiment of the present invention. Please refer to FIG. 3 , the method of this embodiment is applicable to the device 200 for generating a shallow depth of field image in FIG. 2 , and the detailed steps of the method for generating a shallow depth of field image in this embodiment are described below with the components in FIG. 2 :

Firstly, in step S310 , the image acquisition module 210 focuses on the subject according to the first aperture value and shoots, so as to generate an image with the first aperture value. Next, in step S320, the image acquisition module 210 shoots the subject according to the second aperture value without changing other conditions (such as focal length, shutter or shooting distance, etc.), so as to generate the second aperture value image. Wherein, the second aperture value is greater than the first aperture value, that is, the amount of light entering the image at the first aperture value is greater than the second aperture value. Therefore, under the same conditions, the results of pictures taken with different apertures are different. The larger the aperture (the smaller the f value), the smaller the range of clear objects, the more blurred the background, the more three-dimensional the subject, and the clearer the subject.

In more detail, when we focus on the subject with a large aperture and shoot, the objects near the plane where the subject is located (subject plane 20 as shown in Figure 1, please refer to Figure 1) will be clear Yes, if you do not change other conditions, just change to a smaller aperture to shoot the same scene, in addition to the objects near the plane where the subject is located are clear, the objects farther away from the plane where the subject is located are also clear. for clarity. In this way, by comparing two images captured with different aperture values, it is possible to determine where in the image is a close object and where is a distant object.

Next, in step S330 , the processing module 220 analyzes the first and second aperture value images according to the above point of view to obtain image difference values. The method is, for example, to calculate the difference between the grayscale values of each pixel in the first and second aperture value images, and an image edge detection algorithm can also be used to distinguish the difference between the subject area and other background areas. If the image difference is low, it means that the distance between the objects in the entire image and the focal plane where the subject is located is not far away, so the images captured with two different aperture values are clear. If the image difference is large, it means that the distance between each object in the entire image and the focal plane where the subject is located is relatively large.

Therefore, in step S340 , if the processing module 220 determines that the image difference is greater than the critical value, image processing is performed on the first aperture value image to obtain a shallow depth-of-field image. Wherein, the threshold value can be automatically selected by the processing module 220 according to the current shooting mode or freely set by the user according to the shooting environment, and there is no limitation here. Image processing is to blur and enhance the background area (objects that are far away from the focal plane where the subject is located) in order to highlight the subject and make the subject more three-dimensional.

It should be noted here that the ratio of the first aperture value to the second aperture value is related to the characteristics of the lens used and the shooting distance (the shooting distance Y shown in Figure 1), so the first aperture value and the second aperture value The ratio must be adjusted in advance according to the lens characteristics of the device 200 that produces shallow depth-of-field images and different shooting distances, and then the appropriate first aperture value and the distance between the lens and the subject can be adjusted according to the distance between the lens and the subject in actual shooting. Shooting is performed at the second aperture value, therefore, the first aperture value and the second aperture value are not fixed, and can be adjusted by the user according to the actual shooting situation.

Another embodiment is given below to illustrate the present invention. Fig. 4 is a block diagram of an apparatus for generating images with a shallow depth of field according to another embodiment of the present invention. Referring to FIG. 4 , the apparatus 400 for generating a shallow depth-of-field image in this embodiment includes an image acquisition module 410 , a processing module 420 and a geometric conversion unit 430 . The device 400 for generating a shallow depth-of-field image shown in FIG. 4 is substantially similar to the device 200 for generating a shallow depth-of-field image shown in FIG. 2 , so only the differences between the two will be described below.

The processing module 420 includes a smoothing processing unit 422 and a blurring processing unit 424 . The smoothing processing unit 422, for example, adopts an image interpolation method to perform a smoothing process on the two images. The blurring processing unit 424, for example, adopts a spatial filter, a linear filter, a non-linear filter, or a blur filter to blur the image. . The geometric transformation unit 430 is coupled to the processing module 420, and uses an affine transformation matrix to perform displacement correction, so as to correct the starting pixels of two different images to the same position.

FIG. 5 is a flowchart of a method for generating a shallow depth-of-field image according to another embodiment of the present invention. Please refer to Figure 4 and Figure 5 together.

The image acquisition module 410 shoots the subject according to the first aperture value, so as to generate the first aperture value image. The processing module 420 then selects a clear area including the subject in the first aperture value image, that is, the focus area of the plane where the subject is located, and the image frame of the focus area is a clear image (step S510) . The image capturing module 410 then shoots the subject according to the second aperture value to generate a second aperture value image, wherein the second aperture value is greater than the first aperture value (step S520 ).

The geometric transformation unit 430 uses an affine transformation matrix to calculate a geometric transformation parameter for the clear area (step S530), and uses the geometric transformation parameter to perform geometric transformation on the second aperture value image, so that the converted second aperture The starting pixel position of the clear area of the value image is the same as the starting pixel position of the clear area of the first aperture value image (step S540). The processing module 420 analyzes the first aperture image and the converted second aperture image to obtain an image difference (step S550 ), and then the processing module 420 determines whether the image difference is greater than a threshold (step S560 ).

If the image difference is greater than the critical value, image processing is performed on the image with the first aperture value to obtain a shallow depth-of-field image with enhanced background blur (step S570 ). For example, the processing module 420 can control the smoothing processing unit 422 to interpolate the first f-stop image and the converted second f-stop image to obtain a relative depth of field map. In detail, since the image difference is large enough, it means that the depth of field of the image with the first aperture value is relatively deep, and the depth of field of the image with the second aperture value is relatively shallow, so the relative depth of field map with continuous depth of field can be produced after the smoothing process by interpolation . Then, the blurring processing unit 424 can be used to perform blurring processing on the relative depth map to generate a blurred image, wherein the degree of blurring can be predetermined by the user. Finally, the processing module 420 averages the blurred image and each pixel of the first aperture value image (for example, weighted average, etc.). Accordingly, it is possible to generate a shallow depth-of-field image that maintains the sharpness of the subject area and enhances the blurring of other background areas.

However, returning to step 560, if it is judged that the image difference is not greater than the critical value, it means that the positions of objects other than the subject and other background regions are all close to the plane where the subject is located, that is to say, with the first and The images captured by the second two different aperture values have little difference, and both are clear images, that is, images with a shallow depth of field that enhance background blur cannot be synthesized, so the image with the first aperture value can be directly output (step S580 ). In one embodiment, when the processing module 420 determines that the image difference is not greater than the critical value, a prompt screen can be displayed on the screen (not shown) of the device 400 for generating shallow depth-of-field images, so as to remind the user of the current shooting scene It is not suitable for compositing images with a shallow depth of field. Therefore, the user can find another scene with a long depth of field for shooting. Accordingly, the device 400 for generating an image with a shallow depth of field can save unnecessary calculations and shorten its processing time. In another embodiment, if the image taken by the user with the maximum aperture and the minimum aperture provided by the existing image acquisition module 410, the image difference cannot be greater than the critical value, that is, it is not enough to judge the relative depth of field , it can be supplemented by changing the focal length (focal length) of the lens before shooting to produce two images with a large image difference. However, the focal length of the lens is also directly related to the shooting distance, so the adjustment of the focal length of the lens must also be based Only by adjusting the actual shooting situation can two images with an image difference greater than the critical value be taken.

To sum up, the method and device for generating shallow depth-of-field images of the present invention can synthesize images with better shallow depth-of-field effects only by having two images taken with two different apertures. The calculation is simple and generally consumer-friendly. The camera can do it. It doesn't take an expensive zoom lens like a high-end camera to take a series of continuous shots to calculate and produce a shallow depth-of-field image. In addition, the method and device of the present invention can also prompt the user to find another more suitable scene for shooting when the image difference between the two images is too low to be suitable for shallow depth-of-field image synthesis, thereby saving calculation and processing time .

Although the present invention has been disclosed above with the embodiments, it is not intended to limit the present invention, and any person of ordinary skill in the art may make some changes and modifications without departing from the spirit and scope of the present invention.

Claims (12)

1. method that produces shallow depth image, this method comprises the following steps:

According to one first f-number one shot subject is taken, used producing one first f-number image;

According to one second f-number this shot subject is taken, used producing one second f-number image, wherein this second f-number is greater than this first f-number;

Analyze this first f-number image and this second f-number image, to obtain an image difference; And

If judge that this image difference greater than a critical value, then carries out an image processing to obtain a shallow depth image to this first f-number image.

2. the method for the shallow depth image of generation according to claim 1 is wherein being taken this shot subject according to this first f-number, uses the step that produces this first f-number image, comprising:

According to this first f-number this shot subject is carried out defocused shooting, and obtain this first f-number image; And

In this first f-number image, choose a clear area that comprises this shot subject.

3. the method for the shallow depth image of generation according to claim 2 is wherein being taken this shot subject according to this second f-number, uses after the step that produces this second f-number image, also comprises:

Utilize this clear area to calculate a geometric transformation parameter of this second f-number image; And

According to this geometric transformation parameter this second f-number image is carried out geometric transformation, with this second f-number image after the acquisition conversion.

4. the method for the shallow depth image of generation according to claim 3, wherein if judge that this image difference greater than this critical value, then carries out this image processing to this first f-number image and comprises with the step that produces this shallow depth image:

If judge this image difference, this second f-number image after this first f-number image and the conversion is carried out a smoothing handle, to obtain a relative depth map greater than this critical value;

This relative depth map is carried out a Fuzzy processing to produce an obfuscation image; And

This obfuscation image and this first f-number image are carried out equalization processing, use this shallow depth image of acquisition.

5. the method for the shallow depth image of generation according to claim 4, wherein this smoothing is handled and is adopted an image interpolation method.

6. the method for the shallow depth image of generation according to claim 1 wherein also comprises:

If judge that this image difference is not more than this critical value, then directly export this first f-number image.

7. device that produces shallow depth image comprises:

One image acquisition module is taken a shot subject according to one first f-number and one second f-number respectively, uses and produces one first f-number image and one second f-number image respectively, and wherein, this second f-number is greater than this first f-number;

One handles module; Be coupled to this image acquisition module, analyze this first f-number image and this second f-number image, to obtain an image difference; If judge that this image difference greater than a critical value, then carries out an image processing to produce a shallow depth image to this first f-number image.

8. the device of the shallow depth image of generation according to claim 7, wherein this image acquisition module carries out defocused shooting this shot subject according to this first f-number, and produces this first f-number image.

9. the device of the shallow depth image of generation according to claim 8, wherein:

Choose a clear area that comprises this shot subject in this first f-number image that this processing module is produced this image acquisition module.

10. the device of the shallow depth image of generation according to claim 9 wherein also comprises:

One geometric transformation unit; Be coupled to this processing module; This this clear area of geometric transformation unit by using is to calculate a geometric transformation parameter of this second f-number image; And utilize this geometric transformation parameter that this second f-number image is carried out geometric transformation, to produce this second f-number image after changing.

11. the device of the shallow depth image of generation according to claim 10, wherein this processing module comprises:

One smoothing processing unit; This handles module if judge that this image difference is greater than this critical value; Then this smoothing processing unit of this processing module control carries out a smoothing to this second f-number image after this first f-number image and the conversion and handles, to obtain a relative depth map; And

One Fuzzy processing unit carries out a Fuzzy processing producing an obfuscation image to this relative depth map,

Wherein, this processing module carries out equalization processing with this obfuscation image and this first f-number image, uses this shallow depth image of generation.

12. the device of the shallow depth image of generation according to claim 7, wherein:

This processing module is then directly exported this first f-number image if judge that this image difference is not more than this critical value.

Images