JP2005332381A - Image processing method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform an appropriate blur correction to an image, only a part of which is blurred. <P>SOLUTION: An edge detecting means detects an edge for each of the eight different directions to a reduced image. A block dividing means divides a reduced image into 16. An analyzing means executes an analysis from the amount of characteristic of the edge of the image in each block, and determines whether or not the image in each block is a blurred image. A blurring width L, a degree of blurring, and a direction of blurring are determined as blur information Q when the image is the blurred image. A parameter setting means sets a correction parameter E from the blur information Q to the blurred image in the block, and sets a correction intensity α for the image in the block corresponding to the blurring width L included in the blur information Q in such a manner that the larger the blurring width is, the greater the correction intensity is. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method and apparatus for correcting blur on a digital photographic image, and a program therefor.

  Digital photographic images obtained by photoelectrically reading photographic images recorded on photographic films such as negative films and reversal films with a reading device such as a scanner, digital photographic images obtained by taking images with a digital still camera (DSC), etc. On the other hand, printing is performed by performing various image processing. As one of these image processes, there is a blurred image restoration process that removes a blur from a blurred image (blurred image).

  The reason why the photographic image obtained by imaging the subject is blurred is that the focal length is out of focus, the subject is moving, and the blurring caused by the camera shake of the photographer (hereinafter referred to as “photographer”) This is called blurring). In the case of out-of-focus, the point image spreads two-dimensionally, that is, the spread on the photographic image exhibits non-directionality, whereas in the case of blur, a locus with a point image is drawn on the image one-dimensionally. Spreads, that is, has a direction with a spread on a photographic image.

  In the field of digital photographic images, various methods have been conventionally proposed for restoring blurred images. If you know information such as blur direction and blur width when capturing a photographic image, you can restore it by applying a restoration filter such as a Wiener filter or inverse filter to the photographic image. There is widely known a method of providing an apparatus (for example, an acceleration sensor) that can be acquired in an imaging apparatus, acquiring information such as a blur direction and a blur width together with imaging, and performing repair based on the acquired information (for example, , See Patent Document 1).

  Also, a degradation function is set for the blurred image (image with blur), the blurred image is repaired with a restoration filter corresponding to the set degradation function, the restored image is evaluated, and the evaluation result is evaluated. A method is also known in which the deterioration function is reset, and repair is performed by repeating the repair, evaluation, and resetting of the deterioration function until a desired image quality is obtained. This method has a problem that it takes processing time because it is necessary to repeat the process of setting, repairing, evaluating the deterioration function, resetting the deterioration function, and so on. In Patent Document 2, the user specifies a small area including an edge in a blurred image, and instead of the entire blurred image, the above-described degradation function is set, repaired, evaluated, and the specified small area. Repeat the process of resetting the degradation function to find the optimal degradation function, apply the restoration filter corresponding to this degradation function to the entire blurred image, and use the above-mentioned small area for the image used to obtain the degradation function A method has been proposed in which the amount of calculation is reduced and the efficiency is improved by using this image.

  On the other hand, with the rapid spread of mobile phones, the functions of mobile phones have improved, and among them, the improvement of the functions of digital cameras attached to mobile phones (hereinafter referred to as mobile phone cameras) has attracted attention. In recent years, the number of pixels of mobile phone cameras has increased to the order of 1 million, and mobile phone cameras are used in the same way as ordinary digital cameras. In addition to commemorative photos when traveling with friends, it is a common scene to photograph favorite talents and athletes with a mobile phone camera. In such a background, a photographic image obtained by capturing with a mobile phone camera is not limited to being viewed on a monitor of a mobile phone, and for example, can be printed in the same manner as a photographic image acquired with a normal digital camera. It is increasing.

On the other hand, the mobile phone camera has a problem in that the holdability at the time of imaging is poor because the main body (cell phone) is not manufactured exclusively for imaging. In addition, since the mobile phone camera does not have a flash, the shutter speed is slower than that of a normal digital camera. For this reason, camera shake is more likely to occur when shooting a subject with a mobile phone camera than with a normal camera. Extreme camera shake can be confirmed on the monitor of the mobile phone camera, but small camera shake cannot be confirmed on the monitor, and since it is often noticed only after printing, it is captured by the mobile phone camera. Therefore, it is highly necessary to perform blur correction on the photographic image obtained.
Japanese Patent Laid-Open No. 2002-112099 JP-A-7-121703

  However, downsizing of mobile phones is one of the focus of competition among mobile phone manufacturers, along with their performance and cost, and a camera attached to a mobile phone is provided with a device that acquires the direction and width of blur. Since it is not realistic, the method proposed in Patent Document 1 cannot be applied to a mobile phone camera.

  Further, the method proposed in Patent Document 2 is troublesome because it requires a user to specify a small area. Furthermore, photographic images include images with a shallow depth of field that are captured with the main subject such as the face up, images captured when the subject or part of the subject moves, and a dynamic feeling There are images shot in the panning, and in such an image, only a part of the image is blurred. If the small area designated by the user is located in a blurred portion, there is a problem that an appropriate deterioration function cannot be obtained, and the image quality of the corrected image is deteriorated.

  In view of the above circumstances, the present invention makes it possible to efficiently perform blur correction in a digital photographic image without requiring a special device to be provided in the imaging device, and as an image with a shallow depth of field. An object of the present invention is to provide an image processing method and apparatus capable of performing appropriate correction even on an image in which only a part is blurred, and a program therefor.

An image processing method of the present invention is an image processing method for correcting blur in a digital photographic image.
Obtain blur information including the degree of blur of the image of each area using the image of each area constituting the digital photographic image,
Based on the blur information, blur correction is performed on the image of each region so that the region with a higher degree of blur increases the intensity of the correction on the image of the region.

  Here, the “degree of blur” indicates the degree of blur when the target image (hereinafter referred to as the target image. Here, the image of each area constituting the digital photographic image) is a blur image. For example, it may be a blur width. In the case of an image that is not blurred, the degree of blur can be set to zero. In addition, as described above, blur includes non-directional defocus and directional blur. In the case of blur, “the degree of blur” corresponds to the blur degree, and can be, for example, the blur width.

  In addition, increasing the intensity of correction for an image with a higher degree of blur means that the image with an area with a lower degree of blur has the same effect as decreasing the intensity of correction for the image. In other words, the correction is not performed on an image in a region where is less than or equal to a predetermined threshold, that is, the correction intensity is set to zero.

  Further, “to perform blur correction on the image of each area” is not limited to correcting each area over the entire image according to the blur information of each area, but blur information of one or a plurality of areas Accordingly, the correction of the entire image is included, and correction of two or more areas is also included according to the blur information of one or a small number of areas.

  In the image processing method of the present invention, when obtaining the degree of blur in the target image, an edge is detected from the image of each region, the feature amount of the detected edge is acquired, and the acquired feature amount The blur information of the area can be obtained based on the above.

  As described above, the blur causes the spread of the point image in the image. Therefore, in the blurred image (hereinafter referred to as the blur image), the edge spreads according to the spread of the point image. That is, the manner of edge spreading in the image is directly related to the blur in the image. Focusing on this point, the present invention can determine the blur information of this area based on the feature amount of the edge in the image of each area.

  Here, the “edge feature amount” means a feature amount related to an aspect of the edge spread in the target image, and may include, for example, the edge sharpness and the edge sharpness distribution.

  As the “edge sharpness”, any parameter that can express the sharpness of the edge may be used. For example, in the case of the edge shown by the edge profile in FIG. The edge profile is not only used as the edge sharpness so that the sharpness is low, but also the sharpness of the edge is higher as the sharpness of the brightness change of the edge (the slope of the profile curve in FIG. 3) is higher. The slope of the curve may be used as the edge sharpness.

  In addition, as described above, there are blurs that cause non-directional blur and blurs that cause directional blur, and in order to appropriately correct the blur in the image, the direction of the blur (out-of-focus) In this case, the non-directional blur direction is corrected by applying an isotropic filter that works in the same direction for non-directional blur. It is desirable to correct directional blur by applying an anisotropic filter that acts in the direction of blur (that is, blur direction). In the image processing method of the present invention, an edge is detected in each of a plurality of different directions with respect to the image of each region, the feature amount of the edge in each of the directions is acquired, and the region is based on the edge feature amount. It is preferable to acquire not only the degree of blur of the image but also the direction of blur as blur information.

  Here, the “plurality of different directions” means directions for specifying the direction of blur in the target image, and it is necessary to include directions close to the direction of blur. Although the specific accuracy is higher, it is preferable to use an appropriate number according to the balance with the processing speed, for example, eight directions as shown in FIG. In the present invention, in the case of non-directional blur, that is, out-of-focus, “non-direction” is defined as the blur direction.

  The blur width that can represent the degree of blur means the blur width in the blur direction, and can be, for example, the average value of the edge widths of the edges in the blur direction. In addition, in the case where the blur is non-directional out-of-focus, the average value of the edge width in one arbitrary direction may be used as the blur width, but the edge width of the edge in all the different directions included in the plurality of different directions. It is preferable to make it an average value.

An image processing apparatus of the present invention is an image processing apparatus for correcting blur in a digital photographic image,
Blur information acquisition means for acquiring blur information including the degree of blur of the image of the area using the image of each area constituting the digital photographic image;
Based on the blur information, the image processing apparatus includes correction execution means for performing blur correction on the image in each area so that the area with the greater degree of blur increases the intensity of the correction on the image in the area. It is characterized by being.

  The blur information acquisition unit detects an edge in the image of each region, acquires a feature amount of the detected edge, and acquires the blur information of the region based on the feature amount. be able to.

  The blur information preferably includes a blur direction, and the blur information acquisition unit detects an edge for each of a plurality of different directions from the image of each region, and the edge information in each direction is detected. It is preferable that the feature amount is acquired, and the blur information of the image of the region is acquired based on the feature amount in each direction.

  The image processing method of the present invention may be provided as a program for causing a computer to execute the image processing method.

  The area may be the entire image divided into predetermined areas, or may be divided into areas for each size of an object in the image, such as a face, an eye, and the like. The image may be divided at a division rate according to the image size.

  According to the image processing method, apparatus, and program of the present invention, for each image in each area constituting a digital photographic image, blur information including the degree of the blur is obtained using the image in the area, and the image of the area is determined. Since correction is performed, it is not necessary for the user to designate a region, and it is not necessary to provide a device for acquiring information about blurring at the time of imaging, thereby avoiding an increase in the size of the imaging device. This is particularly advantageous in the case of a digital camera attached to a mobile phone, which requires miniaturization.

  Also, according to the degree of blur of the image of each area constituting the digital photographic image, the greater the degree of blur, the higher the correction strength of the image of that area, so that the correction is performed. Even in the case of a digital photograph image with a shallow depth of field or an image in which only a part is blurred, such as a digital photograph image obtained by panning, appropriate correction can be performed.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. Note that the image processing apparatus according to the present embodiment performs a blur correction process for correcting blur in an input digital image (hereinafter referred to as an image), and the blur read in the auxiliary storage device is performed. This is realized by executing the correction processing program on a computer (for example, a personal computer). The blur correction processing program is stored in an information storage medium such as a CD-ROM or distributed via a network such as the Internet and installed in a computer.

  Further, since the image data represents an image, the following description will be given without particularly distinguishing the image from the image data.

  As shown in FIG. 1, the image processing apparatus according to the embodiment of the present invention uses a reduction unit 5 that performs reduction processing on an image D to obtain a reduced image D0 of the image D, and a reduced image D0. The edge detection means 10 for detecting edges in each of the eight different directions shown in FIG. 6 and the reduced image D0 are divided into 16 parts, and the edges detected by the edge detection means 10 are output to the edge profile creation means 13 described later for each block. A block dividing unit 11; an edge profile generating unit 13 for generating an edge profile output from the block dividing unit 11 for each block; an edge narrowing unit 14 for removing invalid edges for each block; Edge feature quantity acquisition means 16 for acquiring the edge feature quantity S obtained by the narrowing means 14 for each block, and edge characteristics for each block Using the amount S, the blur direction and the blur degree N in the image of each block are calculated to determine whether the image of each block is a blur image or a normal image. While information P indicating that the image is an image is transmitted to the parameter setting means 30 described later, in the case of a blurred image, the blur degree K and the blur width L of the image of this block are further calculated, and the blur information along with the blur direction is calculated. Analyzing means 20 transmitted to the parameter setting means 30 as Q, parameters E for correcting the image of each block of the image D based on information P and blur information Q from the analyzing means 20, and correction intensity α Correction execution for obtaining the corrected image D ′ of the image D by correcting the image of each region of the image D using the parameter setting means 30 to be set, the parameter E, and the correction intensity α. A stage 40, comprising a storage means 50 which stores various databases for analyzing unit 20 and the parameter setting means 30.

  The edge detection means 10 uses the reduced image D0 to detect edges having a predetermined intensity or more in every eight directions as shown in FIG. 2, obtains the coordinate positions of these edges, and outputs them to the block division means 11. .

  The block dividing unit 11 divides each edge detected by the edge detecting unit 10 into 16 blocks obtained by dividing the image D into 16 and outputs the result to the edge profile creating unit 13. Note that the edge profile creation unit 13, the edge narrowing unit 14, the edge feature quantity acquisition unit 16, and the analysis unit 20 perform respective processes for each block. Hereinafter, an image of one block is converted into a block image Db. The above means will be described by taking the block image Db as an example.

  Based on the coordinate position of each edge in each direction in the block image Db output from the block dividing unit 11, the edge profile creation unit 13 uses the block image Db to perform these edges in FIG. An edge profile as shown is created and output to the edge narrowing means 14.

  The edge narrowing unit 14 is based on the edge profile of the block image Db output from the edge detection unit 10, an edge having a complex profile shape, an edge including a light source (for example, an edge having a certain brightness or more), and the like. , And the remaining edge profile is output to the edge feature quantity acquisition means 16.

  Based on the edge profile of the block image Db output from the edge narrowing means 14, the edge feature quantity acquisition means 16 obtains an edge width as shown in FIG. 3 for each edge, as shown in FIG. 2 is generated for each of the eight directions shown in FIG. 2, and is output to the analysis unit 20 as the edge feature amount S of the block image Db together with the edge width.

    The analysis means 20 mainly performs the following two processes.

  1. The blur direction in the block image Db and the blur degree N of the image D are obtained to determine whether the block image Db is a blur image or a normal image.

  2. When the block image Db is determined to be a blurred image, a blur width L and a blurring degree K are calculated.

  Here, the first process will be described.

  In order to obtain the blur direction in the block image Db, the analysis unit 20 first sets two directions orthogonal to each other to a histogram of edge widths in eight directions shown in FIG. As a result, the correlation value of the histogram of each direction set (1-5, 2-6, 3-7, 4-8) is obtained. Note that there are various types of correlation values, depending on how they are obtained.If the correlation value is large, the correlation type is small, and if the correlation value is the same as the correlation level, that is, if the correlation value is small, the correlation type is small. It can be roughly divided into two types. In this embodiment, as an example, a correlation value of a type in which the magnitude of the correlation value matches the magnitude of the correlation is used. As shown in FIG. 5, when there is blur in the image, the correlation between the blur direction histogram and the histogram in the direction orthogonal to the blur direction is small (see FIG. 5A). The correlation of the histogram is large in the orthogonal direction set not related to or in the orthogonal direction set when there is no blur in the image (no blur or out of focus) (see FIG. 5B). The analysis unit 20 in the image processing apparatus according to the present embodiment pays attention to such a tendency, obtains a correlation value of the histogram of each group with respect to the four direction groups, and determines two directions of the direction group having the smallest correlation. Find out. If there is a blur in the block image Db, one of the two directions can be considered as a direction closest to the blur direction among the eight directions shown in FIG.

  FIG. 5C is a histogram of edge widths in the blur direction obtained for each image obtained by imaging the same subject under imaging conditions without blur, blur, and blur (out of focus and blur). Show. As can be seen from FIG. 5 (c), the normal image without blur has the smallest average edge width, that is, of the two directions found above, the one with the largest average edge width is the most blurred. The direction should be close to.

  In this way, the analysis unit 20 finds the direction set having the smallest correlation, and sets the direction having the larger average edge width as the blur direction among the two directions of the direction set.

  Next, the analysis means 20 calculates | requires the blurring degree N of the block image Db. The degree of blur of the image indicates the magnitude of the degree of blur in the image. For example, the average edge width in the direction most blurred in the image (here, the blur direction obtained above) may be used. However, here, the edge width of each edge in the blur direction is used to obtain more accurately using the database based on FIG. FIG. 6 is based on a normal image database for learning and a blurred (blurred and blurred) image database, and the direction in which the image is most blurred (in the case of a normal image, a direction corresponding to this direction is desirable, but arbitrary The edge width distribution histogram (which may be the direction of the image) is created, and the ratio between the frequency in the blurred image and the frequency in the normal image (the vertical axis in the drawing) is obtained for each edge width as an evaluation value (the score in the drawing). Is. Based on FIG. 6, a database (hereinafter referred to as a score database) in which edge widths and scores are associated with each other is created and stored in the storage unit 50.

  The analysis unit 20 refers to the score database created based on FIG. 6 and stored in the storage unit 50, obtains a score from the edge width of each edge in the blur direction of the block image Db, and blur direction Is obtained as the degree of blur N of the block image Db. If the obtained blur degree N of the block image Db is smaller than a predetermined threshold T, the analysis unit 20 determines that the block image Db is a normal image, that is, determines that the block image Db is not blurred, and also determines the block image Db. Is output to the parameter setting means 30.

  On the other hand, if the degree of blur N of the block image Db is equal to or greater than the threshold value T, the analysis unit 20 determines that the block image Db is a blurred image, that is, the block image Db is blurred, and performs the second process. enter.

  As the second process, the analysis unit 20 first obtains the degree of blur K of the block image Db.

  The degree of blur K indicating the degree of blur in the blur image can be obtained based on the following factors.

1. Correlation value of the direction group with the smallest correlation (hereinafter referred to as the minimum correlation group): The smaller the correlation value, the greater the degree of blurring. The analysis means 20 pays attention to this point and based on the curve shown in FIG. First blur degree K1 is obtained. Note that the LUT (look-up table) created according to the curve shown in FIG. 7A is stored in the storage unit 50, and the analysis unit 20 uses the first correlation value corresponding to the correlation value of the minimum correlation set. The first blur degree K1 is obtained by reading the blur degree K1 from the storage means 50.

2. Of the two directions of the minimum correlation set, the average edge width in the direction where the average edge width is large: The larger the average edge width, the greater the degree of blurring. The analysis means 20 pays attention to this point, and FIG. A second blurring degree K2 is obtained based on the curve shown in FIG. Note that the LUT (look-up table) created according to the curve shown in FIG. 7B is stored in the storage unit 50, and the analysis unit 20 calculates the average of the smallest correlation pair in the direction where the average edge width is large. The second blur degree K2 corresponding to the edge width is read from the storage means 50 to obtain the second blur degree K2.

3. Difference between the average edge widths in the two directions of the minimum correlation set: The greater the difference, the greater the degree of blurring. The analysis means 20 pays attention to this point, based on the curve shown in FIG. 3 is calculated. Note that an LUT (lookup table) created according to the curve shown in FIG. 7C is stored in the storage unit 50, and the analysis unit 20 calculates the average edge in each of the two directions of the minimum correlation set. The third blur degree K3 corresponding to the width difference is read from the storage means 50 to obtain the third blur degree K3.

  In this way, the analysis means 20 obtains the first blur degree K1, the second blur degree K2, and the third blur degree K3, and blurs using K1, K2, and K3 according to the following equation (1). The degree of blur K of the block image Db to be an image is obtained.

K = K1 × K2 × K3 (1)
Where K: degree of blurring K1: first degree of blurring K2: second degree of blurring K3: third degree of blurring

Next, the analysis unit 20 obtains a blur width L of the block image Db that becomes a blurred image. Here, regardless of the blurring degree K, the average edge width in the blur direction may be obtained as the blur width L. In this embodiment, the average edge width of the edges in all eight directions shown in FIG. Is determined as the blur width L.

  The analysis unit 20 obtains the blur degree K and the blur width L for the block image Db which is a blurred image, and outputs it to the parameter setting unit 30 as the blur information Q together with the blur direction.

  The parameter setting means 30 first sets a one-dimensional correction parameter W1 for directionality correction and a two-dimensional correction parameter W2 for isotropic correction according to the following equation (2).

W1 = K × M1
W2 = (1-K) × M2 (2)
However, W1: One-dimensional correction parameter
W2: Two-dimensional correction parameter
K: Degree of blur
M1: One-dimensional correction mask
M2: Two-dimensional correction mask

That is, the correction parameters W1 and W2 (collectively, parameter E) are set so that the weight of directionality correction increases as the degree of blur K increases with respect to the block image Db that becomes a blurred image.

  Then, the parameter setting unit 30 sets the correction strength α of the block image Db according to the blur width L of the block image Db so as to increase as the blur width L increases.

  In this way, the parameter setting unit 30 sets the correction parameter E and the correction strength α to the image of each block (in the present embodiment, 16 blocks as an example) and outputs the image to the correction execution unit 40. It should be noted that the correction parameter is not set so as not to be corrected for an image of a block that is determined not to be a blurred image, that is, a block image for which information P has been output from the analysis unit 20.

  The correction execution unit 40 corrects the blur of the image D by emphasizing the high frequency component of each block image. Specifically, the correction execution unit 40 sets the high frequency component (Dh) of the block image Db. ) And the blur of the block image Db is corrected by emphasizing the high frequency component Dh using the correction parameter E and the correction intensity α obtained by the parameter setting unit 30 according to the following equation (3).

Db ′ = Db + α × E × Dh (3)
Where Db ′: corrected block image
Db: Block image before correction
Dh: high frequency component of the block image Db
E: Correction parameter
α: Correction strength

Since the correction intensity α is set so as to increase as the blur width L increases in accordance with the blur width L in the block image, a block image (blurred image) having a large blur width L can be seen from Equation (3). (Block image) is more strongly corrected. The correction execution unit 40 operates so as not to correct the image of the block for which the correction parameter E and the correction strength α are not set.

  Next, the correction execution unit 40 combines each corrected block image obtained by correcting the block image that is a blurred image and a block image that is not the blurred image to obtain a corrected image D ′ of the image D.

  FIG. 8 is a flowchart showing processing performed in the image processing apparatus of the embodiment shown in FIG. As shown in the figure, the reduction means 5 first performs a reduction process on the input image D to obtain a reduced image D0 (S10). Next, the edge detection means 10 uses the reduced image D0 to detect edges in each of the eight different directions shown in FIG. 2, and obtains the coordinate position of each detected edge (S15). The block dividing means 11 divides the reduced image D0 into 16 and outputs the edges detected by the edge detecting means 10 to the edge profile creating means 13 described later for each divided block (S20). The edge profile creation means 13, the edge narrowing means 14, the edge feature quantity acquisition means 16 and the analysis means 20 respectively create edge profiles, edge narrowing to eliminate invalid edges, and edge features for the images of 16 blocks. By acquiring the amount and performing analysis based on the feature amount, it is determined whether or not each block image is a blurred image, and the blur width L, blur degree, and blur direction of the block image that is the blurred image are determined. Is obtained as blur information Q (S25). The parameter setting unit 30 sets the correction parameter E based on the blur information Q for the block image that is a blurred image, and the larger the blur width L is, the larger the blur width L is according to the blur width L included in the blur information Q. The correction strength α for the image of the block is set so as to increase. The parameter setting unit 30 does not set correction parameters for block images that are not blurred images (S30). The correction execution unit 40 corrects the corresponding block image with the correction parameter E and the correction strength α obtained by the parameter setting unit 30 to obtain a corrected block image, and the corrected block image and the parameter setting unit 30 The corrected image D ′ of the image D is obtained by synthesizing with the block image for which the correction parameter is not set (S35).

  As described above, according to the image processing apparatus of the present embodiment, blur information is acquired from a digital photographic image and the blur is corrected. Therefore, the blur of the image is corrected without requiring a special device at the time of shooting. can do.

  Further, since correction parameters are set and corrected based on the acquired blur information, it is not necessary to repeat trial and error of parameter setting, correction, evaluation, resetting,.

  Further, according to the degree of blur of the image of each block constituting the image D (in this case, the blur width L), the correction is performed by increasing the correction strength of the image of the block as the degree of blur is larger. Therefore, even in the case of a photographic image with a shallow depth of field, a photographic image in which a part of the subject has moved during imaging, or a photographic image obtained by taking a picture, only a part of it is blurred. Appropriate correction can be performed.

  The preferred embodiments of the present invention have been described above. However, the image processing method and apparatus of the present invention and the program therefor are not limited to the above-described embodiments, and each of the above-described embodiments is not departed from the gist of the present invention. The configuration of the embodiment can be combined, increased, decreased, or changed.

  For example, although the image processing apparatus according to the above-described embodiment divides the image D into 16 regions, the image D may be divided at other division ratios. In addition, even if the entire image is not divided into predetermined areas as described above, the area is divided into areas for each size of an object in the image, such as a face and an eye, for example. It is also possible to divide the region with different division ratios according to the size of the image D, such as dividing the image of 1 million pixels into 16 and dividing the image of 2 million pixels into 32. Good.

  In the image processing apparatus according to the above-described embodiment, the direction with the largest average edge width among the two directions of the minimum correlation set is the blur direction. For example, the minimum correlation set (the direction set with the smallest correlation value) is used. ) And the direction set with the second smallest correlation value, the degree of blurring is calculated for each direction set, and the direction with the larger average edge width of the two directions in the direction set is set as the blurred direction. Candidate directions are acquired, and the obtained blur candidate directions are weighted according to the two calculated blur degrees, the greater the blur group, the greater the weight of the blur candidate directions included in the direction group. The blur direction may be obtained by weighting as described above. In this case, the blur width is also weighted so that the average edge width in each of the two blur candidate directions has a greater weight for the blur candidate direction included in the direction set in a direction set with a higher degree of blurring. The bokeh width can be obtained.

  Further, in the above-described embodiment, the analysis unit 20 does not distinguish whether the blur is out of focus with respect to the block image that becomes the blur image, and if the block image is determined as the blur image, the analysis unit 20 calculates the blur degree. A correction parameter is obtained by weighting and summing the isotropic correction parameter and the directionality correction parameter with a weighting coefficient corresponding to the degree of blurring (in the image processing apparatus of the present embodiment, the blurring degree itself is used as the weighting coefficient). Although correction is made, for example, a blur image whose blurring degree is smaller than a predetermined threshold value may be set as a blur image, and only the isotropic correction parameter may be set for the blur image.

  The image processing apparatus according to the embodiment shown in FIG. 1 performs block division after performing edge detection on an image. However, edge detection may be performed for each block after the image is divided. .

  Furthermore, as described in the embodiment, the mode for performing blur correction on the image of each area is not limited to correction for each area over the entire image according to the blur information of each area. It also includes correcting the entire image according to blur information of a plurality of areas. For example, it may be assumed that the entire area is similarly blurred by the blur information of one area, and the same correction may be applied to the entire area, or depending on the blur information of two adjacent or separated areas. If they are the same, the whole may be assumed to be the same, and the same correction may be applied to the whole. You may make it correct | amend it as what was prorated. Further, if one area is not blurred, it can be estimated that the entire image is out of focus and correction is not performed over the entire area.

  It is also possible to correct a plurality of areas according to blur information of one or a few areas. For example, by applying a Gaussian filter or the like to neighboring pixels adjacent to an area where the degree of blurring is high, the change in the correction amount between the adjacent areas is smoothed, and the degree of blurring correction of the image at the boundary of the area Can be made smooth.

  The correction processing according to the present invention described above can be applied not only to a mobile phone camera but also to a normal digital camera, as well as to a printer that prints digital image data.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. Diagram showing the direction used when detecting edges Diagram showing edge profile Diagram showing edge width histogram The figure for demonstrating operation | movement of the analysis means 20 Diagram for explaining the calculation of the degree of blur Diagram for explaining the calculation of blurring degree The flowchart which shows the process of the image processing apparatus of embodiment shown in FIG.

Explanation of symbols

5 Reduction means 10 Edge detection means 11 Block division means 13 Edge profile creation means 14 Edge narrowing means 16 Edge feature amount acquisition means 20 Analysis means 30 Parameter setting means 40 Correction execution means 50 Storage means D Digital photograph image D 'Corrected image E correction parameter K blurring degree L blur width M1 one-dimensional correction mask M2 two-dimensional correction mask N blur degree P information indicating that it is not a blur image Q blur information S edge feature amount α correction strength

Claims (16)

In an image processing method for correcting blur in a digital photographic image,
Obtain blur information including the degree of blur of the image of each area using the image of each area constituting the digital photographic image,
An image processing method, wherein blur correction is performed on an image in each area such that an area having a higher degree of blur increases the intensity of the correction on the image in the area based on the blur information. Detecting an edge in the image of each region,
Obtain the feature amount of the detected edge,
The image processing method according to claim 1, wherein the blur information of the region is acquired based on the feature amount. The blur information includes a direction of blur;
Detecting an edge in each of a plurality of different directions for the image of each region;
Obtaining feature values of the edge in each of the directions;
The image processing method according to claim 2, wherein the blur information of the image in the region is acquired based on the feature amount in each direction.   4. The image processing method according to claim 1, wherein the blur correction for the image of each area is performed for each area over the entire image in accordance with blur information of each area.   4. The image processing method according to claim 1, wherein the blur correction for the image of each area is for correcting the entire image in accordance with blur information of one or a plurality of areas.   4. The image processing method according to claim 1, wherein the blur correction for the image of each area is performed for a plurality of areas according to blur information of one or a small number of areas.   2. The image processing method according to claim 1, wherein the blur correction for the image in each area is not performed over the entire area unless one area is blurred. An image processing apparatus for correcting blur in a digital photographic image,
Blur information acquisition means for acquiring blur information including the degree of blur of the image of the area using the image of each area constituting the digital photographic image;
Based on the blur information, the image processing apparatus includes correction execution means for performing blur correction on the image of each area so that the area with the greater degree of blur increases the intensity of the correction on the image of the area. An image processing apparatus comprising: The blur information acquisition unit detects an edge in the image of each region,
Obtain the feature amount of the detected edge,
The image processing apparatus according to claim 8, wherein the blur information of the area is acquired based on the feature amount. The blur information includes a direction of blur;
The blur information acquisition unit detects an edge for each of a plurality of different directions with respect to the image of each region,
Obtaining feature values of the edge in each of the directions;
The image processing apparatus according to claim 9, wherein the blur information of an image in the region is acquired based on the feature amount in each direction.   The image processing apparatus according to claim 8, 9 or 10, wherein the area is obtained by dividing the entire image into predetermined areas.   The image processing apparatus according to claim 8, 9 or 10, wherein the area is divided into areas for each size of an object in the image.   The image processing apparatus according to claim 8, wherein the area is divided at a division ratio according to the size of the image. A program for causing a computer to execute image processing for correcting blur in a digital photographic image,
The image processing is a blur information acquisition process for acquiring blur information including the degree of blur of the image of the area using the image of each area constituting the digital photographic image, respectively.
Based on the blur information, the blur correction processing is performed on the image of each region so that the region with the higher degree of blur increases the strength of the correction on the image of the region. Program to do. The blur information acquisition process detects an edge for the image of each region,
Obtain the feature amount of the detected edge,
15. The program according to claim 14, wherein the program is processing for obtaining the blur information of the area based on the feature amount. The blur information includes a direction of blur;
The blur information acquisition process detects an edge for each of a plurality of different directions with respect to the image of each region,
Obtaining the feature amount of the edge in each direction;
The program according to claim 15, wherein the blur information of the image of the region is acquired based on the feature amount in each direction.

Images