JP2018037857A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: When a plurality of captured images are synthesized while changing an in-focus position, the front-rear positional relationship of the subject may not be correctly reflected from the distribution of the contrast value of the subject. An image processing apparatus sets a composition ratio for each region of a plurality of images captured while changing an in-focus position of an optical system, performs composition of the plurality of images using the composition ratio, A synthesis unit that generates a composite image; and a contrast value calculation unit that calculates a contrast value for each region of the plurality of images. The composition ratio is set based on the result of comparing the contrast values of a plurality of images for each region where the images overlap and the in-focus position when the plurality of images are captured. [Selection] Figure 3

Description

  The present invention relates to an image processing apparatus used for image composition.

  In recent years, there has been a high frequency of taking pictures of things that you like and uploading them to social media. For example, when imaging a small object that is close to the user, depending on the imaging conditions, the depth of the subject may become shallow, and it may be difficult to focus on the entire subject. For this reason, only a blurred image of a part of the subject can be obtained, and there are cases where the image is not satisfactory for the photographer. Alternatively, when taking a picture with a family member or a friend, if all the members do not fall within the depth of field, an image in a state where all the members are in focus cannot be obtained. For this reason, when trying to capture an image focused on everyone, the degree of freedom of arrangement of the person inevitably has been limited.

  Therefore, in order to solve these problems, Patent Document 1 discloses a technique for generating a composite image that combines a plurality of images captured while changing the focus position and focuses the entire subject. In Patent Document 1, a plurality of captured images are aligned, and in each of the plurality of images, pixel values of pixels existing at overlapping positions when combining are combined according to the contrast value of each pixel. Synthesis is performed using the ratio. By giving a large composition ratio to an image with a large contrast value, the images are combined so that the ratio of the image in focus is increased.

JP 2014-17539 A

  However, when image synthesis is performed using the technique described in Patent Document 1, the following problems may occur. FIG. 8 is a diagram for explaining processing for combining a plurality of images with different in-focus positions. In FIG. 8, it is assumed that there are a plurality of subjects having different distances from the camera at the same position on the image. FIG. 8A shows an imaging composition, where the subject 801 is a front subject and the subject 802 is a rear subject. 8B is a captured image when the front subject 801 is focused, FIG. 8C is a captured image when the rear subject 802 is focused, and FIG. 8D is a view where the subject 801 and the subject 802 overlap. It is the figure which paid its attention to the area | region. When the front subject 801 is focused as shown in FIG. 8B, the rear subject 802 is shielded by the front subject 801. However, when the rear subject 802 is focused as shown in FIG. When the subject 801 is blurred, the rear subject 802 may be seen through. When the contrast value of the rear subject 802 in the in-focus state is higher than the contrast value of the front subject 801 in the in-focus state, combining is performed according to the method described in Patent Document 1, as shown in FIG. The rear subject is output at the same time. Therefore, a composite image in which a part of the subject in front is missing is generated.

  The present invention has been made in view of such a situation, and when combining a plurality of images taken while changing the focus position, a part of the subject in the foreground is missing and the subject in the back is seen through. An object of the present invention is to provide an image processing apparatus that can suppress the formation of a correct image.

  The present invention sets a composition ratio for each region of a plurality of images captured while changing the focus position of the optical system, and combines the plurality of images using the composition ratio to generate a composite image. Synthesizing means, and contrast value calculating means for calculating a contrast value for each region of the plurality of images, and the synthesizing means performs the synthesis of the plurality of images. An image processing apparatus, wherein the composition ratio is set based on a result of comparing contrast values of the plurality of images for each region where positions overlap and a focus position when the plurality of images are captured. provide.

  According to the present invention, when a plurality of images are captured and combined while changing the in-focus position, it is possible to suppress an image in which a part of the subject in front is lost and the subject in the back is transparent. can do.

It is a block diagram of a digital camera in an embodiment of the present invention. It is a flowchart for demonstrating embodiment of this invention. It is a figure for demonstrating preparation of the synthetic | combination map in embodiment of this invention. It is a figure which shows the contrast value of the corresponding pixel of the some image in the 1st Embodiment of this invention. It is a flowchart for demonstrating the judgment process of the object pixel of a composition ratio in the 1st Embodiment of this invention. It is a flowchart for demonstrating the judgment process of the object pixel of a composition ratio in the 2nd Embodiment of this invention. It is a figure for demonstrating correction | amendment of the contrast value in the 2nd Embodiment of this invention. It is a figure for demonstrating the process which synthesize | combines several image from which a focus position differs.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, a digital camera will be described as an example. However, the present invention is not limited to a digital camera, and is not limited to a digital camera, and is an image processing apparatus capable of taking in focus position information, calculating a contrast value, and synthesizing an image. If is used, the present invention can be applied.

(First embodiment)
FIG. 1 is a block diagram showing the structure of a digital camera according to this embodiment. The digital camera 100 can capture a still image, records information on the in-focus position, and can calculate a contrast value and synthesize an image. Furthermore, the digital camera 100 can perform an enlargement process or a reduction process on an image captured and stored or an image input from the outside.

  The control unit 101 is a signal processor such as a CPU or an MPU, for example, and controls each part of the digital camera 100 while reading a program built in the ROM 105 described later. For example, as will be described later, the control unit 101 instructs the imaging unit 104 described later to start and end imaging. Alternatively, an image processing command is issued to an image processing unit 107 described later based on a program built in the ROM 105. A user command is input to the digital camera 100 by an operation unit 110 described later, and reaches each part of the digital camera 100 through the control unit 101.

  The drive unit 102 is configured by a motor or the like, and mechanically operates an optical system 103 to be described later under a command from the control unit 101. For example, based on a command from the control unit 101, the driving unit 102 moves the position of the focus lens included in the optical system 103 and adjusts the focal length of the optical system 103.

  The optical system 103 includes a zoom lens, a focus lens, a diaphragm, and the like. The diaphragm is a mechanism that adjusts the amount of transmitted light. The focus position can be changed by changing the position of the lens.

  The imaging unit 104 is a photoelectric conversion element, and performs photoelectric conversion that converts an incident optical signal into an electrical signal. For example, a CCD sensor or a CMOS sensor can be applied to the imaging unit 104. The imaging unit 104 is provided with a moving image capturing mode, and can capture a plurality of temporally continuous images as each frame of the moving image.

  The ROM 105 is a read-only nonvolatile memory as a recording medium, and stores parameters and the like necessary for the operation of each block in addition to the operation program for each block provided in the digital camera 100. The RAM 106 is a rewritable volatile memory, and is used as a temporary storage area for data output in the operation of each block included in the digital camera 100.

  The image processing unit 107 performs various image processing, such as white balance adjustment, color interpolation, and filtering, on the image output from the imaging unit 104 or image signal data recorded in the internal memory 109 described later. . Further, compression processing is performed on the data of the image signal captured by the imaging unit 104 in accordance with a standard such as JPEG.

  The image processing unit 107 includes an integrated circuit (ASIC) that collects circuits for performing specific processing. Alternatively, the control unit 101 may perform some or all of the functions of the image processing unit 107 by performing processing according to a program read from the ROM 105. When the control unit 101 has all the functions of the image processing unit 107, it is not necessary to have the image processing unit 107 as hardware.

  The display unit 108 is an image temporarily stored in the RAM 106, an image stored in an internal memory 109, which will be described later, or a liquid crystal display or an organic EL display for displaying a setting screen of the digital camera 100. is there.

  The built-in memory 109 is a place for recording an image captured by the image capturing unit 104, an image obtained by the processing of the image processing unit 107, information on a focus position at the time of image capturing, and the like. A memory card or the like may be used instead of the built-in memory.

  The operation unit 110 is, for example, a button, switch, key, or mode dial attached to the digital camera 100, or a touch panel that is also used as the display unit 108. A command from the user reaches the control unit 101 via the operation unit 110.

  FIG. 2 is a flowchart for explaining the present embodiment.

  In step S201, the digital camera 100 captures a plurality of images while automatically changing the in-focus position by a predetermined amount in accordance with an imaging instruction from the user. The imaging unit 104 captures an image of the subject via the optical system 103, saves the image captured by the control unit 101 in the built-in memory 109, and acquires information on the in-focus position of the optical system 103 when each image is captured. And stored in the built-in memory 109. Note that the user may manually rotate the zoom ring to capture images a plurality of times while changing the focus position. In this embodiment, the driving unit 102 changes the in-focus position by driving a lens provided in the optical system 103. However, the present invention is not limited to this, and for example, a plurality of digital cameras are used, and each digital camera is different. You may make it image-capture in the state focused on the position. However, in this case, it is necessary to devise so that the optical axes of the respective digital cameras are close to each other so that the parallax between images obtained by a plurality of digital cameras does not increase.

  In step S202, the image processing unit 107 performs alignment processing on the plurality of images captured in step S201. An example of the alignment method will be described below. While shifting the relative positions of the two images to be aligned, the sum of absolute values (SAD: Sum Of Absolute Difference) of the output differences of the pixels of the plurality of images is obtained. The relative moving amount and moving direction of the two images when the absolute value sum is the smallest are obtained. Then, after calculating the conversion coefficient of the affine transformation or projective transformation according to the obtained movement amount and the movement direction, minimize the error between the movement amount by the conversion coefficient and the movement amount calculated from the absolute value sum. Optimize transform coefficients using the square method. Based on the optimized conversion coefficient, a deformation process is performed on the image to be aligned.

  In step S203, the control unit 101 reads out the in-focus position at the time of capturing each image from the built-in memory 109, arranges the captured images based on the in-focus position when each image is captured, and assigns numbers. To do.

  In step S204, the image processing unit 107 calculates a contrast value for a plurality of captured images. As an example of the contrast value calculation method, an edge detection filter is applied to the pixel output, and the result is used as the contrast value. As the edge detection filter, for example, a band pass filter, a Sobel filter, a Laplacian filter, or the like is used.

  In step S205, the image processing unit 107 creates a composite map based on the contrast value comparison result calculated in step S204. Specifically, the image processing unit 107 first sets the composition ratio of the pixels of the image with the highest contrast value to 100% among the corresponding pixels of each of the plurality of images captured while changing the focus position. The other image pixels are set to 0%. The corresponding pixel here means a pixel that is located at the same position or overlapping position among a plurality of images as a result of the alignment in step S202. In the following processing, the composition ratio given to each pixel will be described based on the contrast value of each pixel.

  FIG. 3 is a diagram for explaining creation of a composite map. FIG. 4 is a diagram illustrating contrast values of corresponding pixels of a plurality of captured images. FIG. 3A shows the subject composition of a certain imaging scene. A portion 304 in FIG. 3A is a portion of the subject 301 and does not overlap the subject 302.

  FIG. 4A shows the arrangement order of the images and the in-focus position of the optical system 103 when each image is captured. FIG. 4B is a diagram showing the contrast value of any one pixel included in the portion 304. Corresponding pixels included in the images from the first to the eighth array are shown as pixels 411 to 418, respectively. As shown. In FIG. 4B, the contrast value of the pixel 412 included in the portion 304 in the image captured by the digital camera 100 in focus at the in-focus position 312 in FIG. Indicates that it will be the highest. Therefore, the image processing unit 107 sets the composition ratio of the pixels 412 included in the second image in the arrangement order to 100% at the position of the portion 304. At the same time, the image processing unit 107 sets the combination ratio of the first pixel 411 in the arrangement order and the corresponding pixels 413 to 418 of the third to eighth images to 0%. The image processing unit 107 performs such a composition ratio setting for each corresponding pixel of each image. Note that if the composition ratio suddenly changes from 0% to 100% (or changes from 100% to 0%) between adjacent pixels, unnaturalness at the composition boundary becomes conspicuous. Therefore, the image processing unit 107 applies a filter having a predetermined number of pixels (number of taps) to the composite map so that the composite ratio changes stepwise between adjacent pixels.

  Note that the above-described method for creating a composite map is merely an example, and can be modified as appropriate. For example, instead of setting the composition ratio of the image with the highest contrast value to 100%, the image processing unit 107 performs composition ratios other than 0 for a plurality of images so that the composition ratio becomes larger as the pixel with the higher contrast value. May be set.

  In step S <b> 206, the control unit 101 determines a pixel for adjusting the composition ratio. FIG. 5 is a flowchart for explaining the synthesis ratio adjustment processing. First, in step S501, as illustrated in FIG. 4B, the control unit 101 detects a local maximum value from the relationship between the arrangement order of the images and the contrast value of the corresponding pixel of each image. In the case of FIG. 4A, the contrast value in the corresponding pixel of the image in the second order is the maximum value. In the case of FIG. 4B, the contrast value in the corresponding pixel of the images in the second and seventh order is the maximum value.

  Next, in step S502, the control unit 101 determines whether there are two or more local maximum values in the corresponding pixel. If there are two or more local maximum values, the process proceeds to step S503, and it is determined that the pixel being processed is a pixel that needs to be adjusted in the composition ratio. If two or more local maximum values do not exist, step S504 is performed. Then, it is determined that the pixel being processed is a pixel that does not require adjustment of the composition ratio. For example, in the case of FIG. 4B, in the corresponding pixels, the contrast values of the pixels corresponding to the second and seventh images in the arrangement order are the maximum values. The control unit 101 determines that these pixels are pixels that require adjustment of the composition ratio.

  The reason for the above processing will be briefly described with reference to FIG. For example, as shown in FIG. 3A, a subject 301 and a subject 302 exist, FIG. 3C shows an image captured when focusing on the subject 301, and FIG. An image taken when focusing is shown. However, it is assumed that the subject 321 in FIG. 3C and the subject 331 in FIG. 3D are a part (head) of the subject 301, and when the subject 301 is focused, the subject 421 is also focused. Similarly, it is assumed that the subject 322 in FIG. 3C and the subject 332 in FIG. 3D are part of the subject 302, and that when the subject 302 is focused, the subject 332 is also focused. When focusing on the subject 301, the contrast value of the region corresponding to the subject 321 increases, and when focusing on the subject 302, the contrast value of the region corresponding to the subject 332 increases. The contrast value of the pixel in the region 323 where the subject 301 and the subject 302 overlap is the two focus positions 312 and 317 when the focus position is changed from the focus position 311 to 318 in FIG. Maximum value. When the relationship between the arrangement order based on the focus position and the contrast value is graphed, it has two local maximum values. On the other hand, in the pixel of the region 324 where only one subject exists, two or more local maximum values do not exist. Therefore, in the above-described processing, when there are two or more maximum values, the control unit 201 determines that the pixel needs to be adjusted in the synthesis ratio.

  The above is the pixel determination process for adjusting the composition ratio in step S206.

  In step S207, the image processing unit 107 corrects the composite map. Here, in step S206, the composition ratio of the pixel determined to be necessary to adjust the composition ratio determined by the control unit 101 is changed. Specifically, the image processing unit 107 increases the combination ratio of the pixels of the closest image having the maximum value with respect to the pixels determined to need adjustment of the combination ratio in step S206. For example, in FIG. 4B, the composition ratio of the second pixel in the arrangement order is set to 100%. Thus, as shown in FIG. 3E, the composition ratio of the subject 321 can be set to 100%, and the subject 332 behind can be completely hidden. Alternatively, although the composition ratio of the subject 321 is increased, the composition ratio of the pixels at the edge portion is not limited to 100% and is slightly suppressed, so that the image after composition can be made a blurred image.

  As shown in FIG. 4C, there may be a local maximum 432 in which the difference in contrast between the pixels on both sides is extremely small compared to other local maximums. Since these values are considered to be highly likely to be affected by noise or the like, the composition ratio is not set to 100% even if it is closest. In such a case, among the maximum values other than the maximum value 432, the composition ratio of the corresponding pixels of the closest image is set to 100%. In the case of FIG. 4C, the composition ratio of the fourth pixel in the arrangement order is set to 100%.

  Specifically, the control unit 101 determines a threshold value in advance, and compares and adjusts the contrast value described above when the average of the difference between the maximum value and the contrast value between both sides is smaller than the threshold value.

  When the processing in step S207 is completed, in step S208, the control unit 101 determines whether or not the composite map correction processing for all the pixels has been completed. If not completed, the image processing unit 107 receives the next pixel from the built-in memory 109 and performs the processing from step S206. If completed, the process enters step S210, and the images are combined based on the combination map. Also in this case, similarly to step S205 described above, the image processing unit 107 applies a filter having a predetermined number of pixels to the composite map corrected in step S207 as necessary, and combines the adjacent pixels with a composite ratio. To gradually change.

  According to the first embodiment, when synthesizing using a plurality of images picked up by changing the in-focus position, by adjusting the composition ratio between the plurality of images of pixels whose positions overlap at the time of composition, A composite image that correctly reflects the positional relationship before and after can be generated. Here, the configuration for obtaining the contrast value and the composition ratio for each pixel has been described as an example. However, in order to reduce the processing load, the contrast value and the composition ratio may be obtained for each block composed of a plurality of pixels. . In other words, if the configuration is such that a contrast value and a composition ratio are obtained for each of a plurality of regions included in an image, the size of each region can be set to an arbitrary size in consideration of processing load and image quality. is there.

(Second Embodiment)
In the second embodiment, unlike the first embodiment, the contrast value calculated by the image processing unit 107 is used for determination of a pixel that needs to be adjusted in the composition ratio through correction using a correction coefficient. The overall processing in the second embodiment is the same as the flowchart shown in FIG. 2 of the first embodiment, but the composition ratio adjustment processing in step S206 is different from that in the first embodiment. In the following, the second embodiment will be described focusing on the adjustment process of the synthesis ratio. Note that description of processing similar to that of the first embodiment is omitted.

  FIG. 6 is a diagram for explaining a synthesis ratio adjustment process according to the second embodiment. In step S601, the control unit 101 detects the maximum value of the contrast value as in step S501 of the first embodiment. In step S602, the control unit 101 detects the maximum value of the contrast value as in step S502 of the first embodiment.

  In step S603, the control unit 101 corrects the contrast value of the corresponding pixel of each image. FIG. 7 is a diagram for explaining the correction of the contrast value. In FIG. 7A, the front subject and the rear subject are focused on the second and seventh images in the order of arrangement, and in FIG. 7B, the fifth and seventh images in the order of arrangement are shown. Focus on the front subject and the rear subject, respectively. 7A and 7B, the contrast values when focusing on the front subject are C_near1 and C_near2, respectively, and the contrast values when focusing on the rear subject are both C_far.

  When focusing on one of the front and rear subjects in a pixel that requires adjustment of the composition ratio, the other degree of blur increases as the distance between the front and rear subjects increases. Therefore, the contrast value calculated in step S601 is corrected in consideration of the in-focus position. As an example, a correction coefficient is applied to the contrast value of the pixel of the closest image, and the result is used as a corrected contrast value. In FIG. 7C, the correction value W_near1 is applied to the contrast value C_near1 of the pixel having the second number in the arrangement order to obtain a corrected contrast value C_near1 ′. Similarly, the correction coefficient W_near2 is applied to the contrast value C_near2 of the pixel having the number 5 in the arrangement order in FIG. 7D to obtain a corrected contrast value C_near2 ′. In FIGS. 7 (c) and (d), the seventh image in the rearrangement order is focused in the rear, but FIG. 7 (c) is a front subject and a rear subject as compared with FIG. 7 (d). The distance to is large. In FIG. 7C, the degree of blurring of the second image in the arrangement order is larger, and the front-rear position of the subject in the composite image is likely to be disturbed. Therefore, it is necessary to increase the correction coefficient. That is, the control unit 101 sets the correction coefficient W_near1 to be larger than the correction coefficient W_near2. Specifically, in FIG. 7C and FIG. 7D, a correction coefficient larger than 1 is given according to the distance from the most infinite pixel among the corresponding pixels having the maximum contrast value. . On the other hand, among the corresponding pixels having the maximum contrast value, 1 is given as a correction coefficient to the most infinite pixel, or no correction is performed.

  In step S604, the control unit 101 determines whether or not the local maximum contrast value is the maximum value. As shown in FIG. 7C, when the local maximum on the closest side is the maximum value, the process proceeds to step S605, and the control unit 101 determines that the pixel being processed is a pixel that needs to be adjusted in the composition ratio. To do. As shown in FIG. 7D, when the local maximum on the nearest side is not the maximum value, the process proceeds to step S606, and the control unit 101 determines that the pixel being processed is not a pixel that needs to be adjusted in the composition ratio.

  The reason for the above determination will be described. The in-focus position of the pixel having the local maximum value 702 on the closest side is set to have a larger distance from the rear subject and a larger correction coefficient as described above than the pixels having other local maximum values. As long as a sufficiently large correction coefficient is applied to the contrast value 702 of the corresponding pixel of the front subject image, the contrast value 703 of the corresponding pixel of the image when focusing on the rear subject should be larger. Nevertheless, if the corrected value 712 of the contrast value 702 of the corresponding pixel of the front subject image may be smaller than the contrast value 703 of the corresponding pixel of the rear image, the value due to the influence of noise or the like. it is conceivable that. That is, the contrast value 702 is not focused on the front subject in the first place, and it is considered that the contrast value 802 is the maximum value due to the influence of noise or the like. Therefore, in such a case, the adjustment process of the composition ratio of the pixel being processed is not performed.

  According to the second embodiment, when synthesizing using a plurality of images picked up by changing the in-focus position, the influence of the blur condition due to the distance between the subjects and the influence of noise is considered, and the front-rear positional relationship of the subjects is determined. It can be reflected correctly.

(Other embodiments)
The above embodiment has been described based on the implementation with a digital camera, but is not limited to a digital camera. For example, it may be implemented by a portable device with a built-in image sensor or a network camera capable of capturing an image.

  Note that the present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus execute the program. It can also be realized by a process of reading and operating. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Digital camera 101 Control part 102 Drive part 103 Optical system 104 Imaging part 105 ROM
106 RAM
107 Image processing unit 108 Display unit 109 Built-in memory 110 Operation unit

Claims (12)

Synthesizing means for setting a synthesis ratio for each region of a plurality of images captured while changing the focus position of the optical system, and synthesizing the plurality of images using the synthesis ratio to generate a synthesized image; ,
Contrast value calculating means for calculating a contrast value for each region of the plurality of images,
The synthesizing unit, based on a result of comparing the contrast values of the plurality of images for each region where the positions of the plurality of images overlap, and a focus position at the time of capturing the plurality of images when performing the synthesis. An image processing apparatus, wherein the composition ratio is set.   The synthesizing unit compares the contrast values of the plurality of images for each of the regions, and makes the composition ratio of the image having the largest contrast value among the plurality of images highest for each of the regions. The image processing apparatus according to claim 1.   When the contrast value of the plurality of images is arranged in the order of the in-focus position of the optical system for each region, the synthesizing unit has two or more maximum values of the contrast value. The image processing apparatus according to claim 1, wherein the composition ratio of any one of the plurality of images corresponding to the maximum value is set to be the highest.   The combining means, for each of the regions, when arranging the contrast values of the plurality of images in the order of the in-focus position of the optical system, among the plurality of images corresponding to the maximum value of the contrast value, The image processing apparatus according to claim 3, wherein the composition ratio of an image in which the in-focus position is closest to the closest side is set to be the highest.   The synthesizing means, when there are two or more maximum values of the contrast value, except for a maximum value that is smaller than a predetermined threshold, the image of the image in which the in-focus position is closest to the closest side. The image processing apparatus according to claim 4, wherein the composition ratio is set to be the highest.   6. The composition means, wherein the composition ratio of one of the plurality of images corresponding to the maximum value is set to 1, and the composition ratio of another image is set to 0. The image processing apparatus according to any one of the above.   The combining means corrects one or more maximum values using a correction coefficient when two or more maximum values of the contrast value exist, and based on the corrected maximum values, the plurality of the maximum values The image processing apparatus according to claim 3, wherein the composition ratio of any one of the images corresponding to the maximum value is set to be the highest.   The synthesizing unit is configured to combine the image having the in-focus position closest to the closest position when the corrected maximum value of the image having the in-focus position closest to the closest position is larger than the other maximum values. The image processing apparatus according to claim 7, wherein the ratio is set to be the highest.   The composition unit, when there are two or more maximum values of the contrast value, sets the correction coefficient based on the in-focus position of an image corresponding to each maximum value. The image processing apparatus according to 7 or 8.   The composition means, when there are two or more local maximum values of the contrast value, makes the correction coefficient for the local maximum value of the image in which the in-focus position is located closest to the largest value. The image processing apparatus according to 9. A composition step of setting a composition ratio for each region of the plurality of images captured while changing the focus position of the optical system, and performing composition of the plurality of images using the composition ratio to generate a composite image; ,
A contrast value calculating step for calculating a contrast value for each region of the plurality of images,
In the synthesis step, when performing the synthesis, based on the result of comparing the contrast values of the plurality of images for each region where the positions of the plurality of images overlap, and the focus position when the plurality of images are captured. An image processing method characterized in that the composition ratio is set. A program for causing a computer of an image processing apparatus to operate,
A composition step of setting a composition ratio for each region of the plurality of images captured while changing the focus position of the optical system, and performing composition of the plurality of images using the composition ratio to generate a composite image; ,
A contrast value calculating step for calculating a contrast value for each region of the plurality of images,
In the synthesis step, when performing the synthesis, based on the result of comparing the contrast values of the plurality of images for each region where the positions of the plurality of images overlap, and the focus position when the plurality of images are captured. And a computer program for setting the composition ratio.

Images