JP2018098764A - Imaging apparatus and image composition method - Google Patents

Abstract

Kind Code: A1 An imaging apparatus is provided that synthesizes a plurality of images to automatically generate an image with a deeper depth of field. An imaging device (100) includes an imaging unit (140) for capturing a subject image while changing a focus position to generate a plurality of image data, and synthesizing the plurality of image data generated by the imaging unit to obtain a plurality of images. and an image processing unit 160 that generates still image data having a deeper depth of field than the depth of field of the image data of the image processing unit 160, and a control unit 180 that controls the image processing unit. The control unit 180 detects a main subject from an image represented by one image data out of a plurality of image data, determines a range for synthesizing the plurality of image data based on the position of the detected main subject, and determines the determined range. The image processing unit 160 is controlled so as to synthesize the image data focused inside and generate the still image data. [Selection drawing] Fig. 1

Description

  The present disclosure relates to an imaging device having a function of combining a plurality of images and generating an image having a deep depth of field, and an image combining method.

  Patent Document 1 discloses an imaging device that captures a plurality of images with different focus and generates an image with an expanded depth of field based on the captured images. In the imaging apparatus of Patent Document 1, the operator selects a plurality of subjects to be focused, and images are captured based on the focus positions of the subjects selected by the operator. By synthesizing these images, a synthesized image in accordance with the operator's intention can be obtained.

JP 2014-207502 A

  The present disclosure provides an imaging apparatus that automatically generates an image with a deeper depth of field by combining a plurality of images. The present disclosure also provides an image composition method for automatically generating an image having a deeper depth of field by combining a plurality of images.

  An imaging apparatus according to an aspect of the present disclosure combines an imaging unit that captures a subject image while changing a focus position and generates a plurality of image data, and the plurality of image data generated by the imaging unit. An image processing unit that generates still image data having a depth of field deeper than the depth of field of each of the plurality of image data, and a control unit that controls the image processing unit. The control unit detects a main subject from an image indicated by one image data of the plurality of image data, and based on the detected position of the main subject, a range in which the plurality of image data are combined to include And the image processing unit is controlled to generate the still image data by synthesizing image data to be focused within the determined range among the plurality of image data.

  According to another aspect of the present disclosure, an image composition method combines a plurality of pieces of image data so that the subject image of the main subject in the still image data is larger than the depth of field of the main subject in each of the plurality of image data. In the image composition method, the still image data is generated so that the depth of field becomes deeper. In the image composition method, when each of the plurality of image data includes a first subject and a second subject, areas of the first subject and the second subject on the image are included. And the first subject and the second subject have different in-focus positions, and the first subject and the second subject have the same center position on the image, the first subject And the second subject having the larger area on the image is set as the main subject. In the image composition method, the areas of the first subject and the second subject on the image are equal, the in-focus positions of the first subject and the second subject are different, and the When the center position of the first subject and the second subject on the image is different, the one closer to the center of the image is selected as the main subject between the first subject and the second subject.

  According to the imaging apparatus and the image composition method of the present disclosure, a main subject is detected from an image, a composition range is determined based on the position of the main subject, and a plurality of pieces of image data to be focused within the determined composition range are obtained. By combining, an image with a deep depth of field can be obtained.

FIG. 1 is a diagram illustrating a configuration of a digital camera according to the present disclosure. FIG. 2 is a rear view of the digital camera. FIG. 3 is a flowchart showing a processing flow when the depth synthesis processing is performed. FIG. 4 is a diagram for explaining the movement of the focus lens at the time of recording a multi-focus moving image. FIG. 5 is a diagram for explaining the concept of depth synthesis using a frame image of a multifocus moving image. FIG. 6 is a diagram for explaining (A) a plurality of AF areas set in an image area, and (B) a change in contrast value accompanying movement of a focus lens in one AF area. FIG. 7A is a diagram illustrating an example of the focusing information table. FIG. 7B is a diagram illustrating an example of the focusing information table. FIG. 8 is a flowchart showing focus search and multifocus moving image recording processing. FIG. 9 is a diagram for explaining a moving range of the focus lens at the time of recording a multi-focus moving image. FIG. 10 is a flowchart showing the automatic depth synthesis process. FIG. 11 is a flowchart showing the setting process of the depth synthesis range. FIG. 12A is a diagram illustrating an example of a frame image including a plurality of subjects. FIG. 12B is a diagram illustrating an example of a plurality of subjects extracted for each AF area. FIG. 13 is a diagram illustrating an example of the subject determination information table. FIG. 14 is a diagram illustrating an example of the determination threshold value table. FIG. 15A is a diagram illustrating an example of setting weights for each AF area. FIG. 15B is a diagram illustrating an example of a weight table. FIG. 16 is a flowchart showing a process for detecting an AF area including the same subject. FIG. 17 is a flowchart showing processing for calculating the importance (cumulative importance) of the subject included in each AF area. FIG. 18 is a diagram illustrating an example of the subject importance degree table. FIG. 19 is a flowchart illustrating the depth synthesis range calculation process. FIG. 20 is a diagram for explaining a specific example of the calculation process of the depth synthesis range. FIG. 21 is a diagram for explaining the automatically set depth synthesis range. FIG. 22A is a diagram illustrating an example of a frame image including a plurality of subjects having different areas. FIG. 22B is a diagram in which a frame indicating an AF area is superimposed on the frame image of FIG. 22A. FIG. 23A is a diagram illustrating an example of a frame image including a plurality of subjects having different center positions. FIG. 23B is a diagram in which a frame indicating an AF area is superimposed on the frame image of FIG. 23A.

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, a detailed description of already well-known matters and a redundant description of substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the claimed subject matter. .

  Hereinafter, an embodiment of an imaging device according to the present disclosure will be described with reference to the drawings.

(Embodiment 1)
[1. Constitution]
An electrical configuration of the digital camera (an example of an imaging device) according to Embodiment 1 will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of the digital camera 100. The digital camera 100 is an imaging device that captures a subject image formed by an optical system 110 including one or a plurality of lenses with a CCD (Charge Coupled Device) 140 as an imaging unit. The image data generated by the CCD 140 is subjected to various processes by the image processing unit 160 and stored in the memory card 200. Hereinafter, the configuration of the digital camera 100 will be described in detail.

  The optical system 110 includes a focus lens 111, a zoom lens 112, and a camera shake correction lens 113. The subject image can be enlarged or reduced by moving the zoom lens 112 along the optical axis. Further, the focus (in-focus state) of the subject image can be adjusted by moving the focus lens 111 along the optical axis. Further, the camera shake correction lens 113 corrects image blur due to blur of the digital camera 100.

  The lens driving unit 120 drives various lenses (for example, the zoom lens 112 and the focus lens 111) included in the optical system 110. The lens driving unit 120 includes, for example, a zoom motor that drives the zoom lens 112 and a focus motor that drives the focus lens 111.

  The diaphragm unit 300 adjusts the amount of light transmitted through the opening by adjusting the size of the opening in accordance with a user setting or automatically.

  The shutter 130 is a means for shielding the light transmitted through the CCD 140. The shutter 130 controls the optical information indicating the subject image together with the optical system 110 and the diaphragm unit 300. The optical system 110 and the diaphragm unit 300 are housed in a lens barrel.

  The CCD 140 captures the subject image formed by the optical system 110 and generates image data. The CCD 140 includes a color filter, a light receiving element, and an AGC (Auto Gain Controller). The light receiving element converts the optical signal collected by the optical system 110 into an electrical signal, and generates image information. The AGC amplifies the electric signal output from the light receiving element.

  An ADC (A / D converter: analog-digital converter) 150 converts analog image data generated by the CCD 140 into digital image data.

  The image processing unit 160 performs various processes on the digital image data generated and converted by the CCD 140 under the control of the controller 180. The image processing unit 160 generates image data to be displayed on the display monitor 220 or generates image data to be stored in the memory card 200. For example, the image processing unit 160 performs various processes such as gamma correction, white balance correction, and flaw correction on the image data generated by the CCD 140. Further, the image processing unit 160 converts the image data generated by the CCD 140 into H.264. It compresses by the compression format etc. based on H.264 standard and MPEG2 standard. Further, the image processing unit 160 can generate, for example, moving image data (4K moving image data) having about 4000 × 2000 pixels based on the image data generated by the CCD 140. The image processing unit 160 can perform various processes described later on the generated 4K moving image data. For example, the image processing unit 160 performs a depth synthesis (focus synthesis) process using frame images constituting the generated 4K moving image data (details will be described later).

  The controller 180 is a control unit that controls the entire digital camera 100. The controller 180 can be realized by a semiconductor element or the like.

  The image processing unit 160 and the controller 180 may be configured only by hardware, or may be realized by combining hardware and software. The controller 180 may be a microcontroller, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Cycle F), or an FP (Aggregated Circuit). .

  The buffer 170 functions as a work memory for the image processing unit 160 and the controller 180. The buffer 170 is realized by, for example, a DRAM (Dynamic Random Access Memory), a ferroelectric memory, or the like.

  The card slot 190 is a means for attaching the memory card 200 to the digital camera 100. The card slot 190 can connect the memory card 200 and the digital camera 100 mechanically and electrically.

  The memory card 200 includes a flash memory or a ferroelectric memory, and can store data such as an image file generated by the image processing unit 160.

  The built-in memory 240 is configured by a flash memory or a ferroelectric memory. The built-in memory 240 stores a control program and data for controlling the entire digital camera 100.

  The operation member 210 is a generic name for a user interface that receives an operation from a user. The operation member 210 includes at least one of a button, a lever, a dial, a touch panel, and a switch that receives an operation from the user. The operation member 210 includes a focus ring provided on the outer periphery of the lens barrel. The focus ring is a member that is rotated by the user to move the focus lens 111.

  The display monitor 220 can display an image (through image) indicated by image data generated by the CCD 140 and an image indicated by image data read from the memory card 200. The display monitor 220 can also display various menu screens for performing various settings of the digital camera 100. The display monitor 220 includes a liquid crystal display device or an organic EL (Electro Luminescence) display device.

  The gyro sensor 250 is a sensor that detects a shake (movement) of the digital camera 100. Based on the output signal from the gyro sensor 250, a camera shake correction operation and stillness determination are performed.

  FIG. 2 is a view showing the back surface of the digital camera 100. In FIG. 2, a release button 211, a selection button 213, a determination button 214, and a touch panel 222 are shown as examples of the operation member 210. When the operation member 210 receives an operation by the user, the operation member 210 transmits various instruction signals to the controller 180.

  The release button 211 is a two-stage push button. When the release button 211 is pressed halfway by the user, the controller 180 executes autofocus control (AF control), auto exposure control (AE control), and the like. When the release button 211 is fully pressed by the user, the controller 180 records image data captured at the timing of the pressing operation on the memory card 200 or the like as a recorded image.

  The selection buttons 213 are a plurality of push buttons provided respectively in the up / down / left / right directions of the determination button 214. The user can move a cursor and a frame to be described later, or select various condition items displayed on the display monitor 220 by pressing one of the selection buttons 213 in the up / down / left / right directions.

  The decision button 214 is a push button. When the determination button 214 is pressed by the user while the digital camera 100 is in the shooting mode or the playback mode, the controller 180 displays a menu screen on the display monitor 220. The menu screen is a screen for setting various conditions for photographing (recording) and reproduction. When the determination button 214 is pressed when a setting item for various conditions is selected, the controller 180 finalizes the setting of the selected item.

  The touch panel 222 is arranged so as to overlap with the display screen of the display monitor 220, and detects a touch operation on the display screen by the user's finger. As a result, the user can perform an operation such as designating an area for the image displayed on the display monitor 220.

[2. Operation]
The operation of the digital camera 100 having the above configuration will be described. The digital camera 100 has a depth synthesis (focus synthesis) function. The depth synthesis function is a function for synthesizing a plurality of images taken at different in-focus positions (hereinafter also referred to as focus lens positions) to generate one still image having a pseudo deep depth of field. is there. Hereinafter, the depth composition operation using the depth composition function of the digital camera 100 will be described.

[2-1. Depth composition]
FIG. 3 is a diagram for explaining the flow of processing when the depth synthesis operation of the digital camera 100 is executed. The digital camera 100 performs a process of recording a moving image while changing a focus position (focus position) in order to obtain a plurality of still images for use in depth synthesis (S1). That is, the controller 180 of the digital camera 100 causes the CCD 140 to capture an image while moving the focus lens 111 along the optical axis, and generates moving image data. Thereafter, the digital camera 100 performs a depth synthesis process (S2) described later using a frame image (still image) included in the recorded moving image, and generates a still image having a deep depth of field. It should be noted that the depth synthesis process (S2) may be executed continuously following the moving image recording process (S1). Alternatively, the depth synthesis process (S2) is executed at an arbitrary timing after the completion of the moving image recording process (S1), for example, when a predetermined operation (operation on the menu screen or operation by an operation button) is performed by the user. May be.

[2-1-1. Recording video for depth composition]
With reference to FIG. 4 to FIG. 7A and FIG. 7B, the moving image recording process (S1) for use in depth synthesis will be described. In this process, in order to obtain a plurality of still images (frame images) having different in-focus positions, a moving image is recorded while continuously changing the in-focus positions. Hereinafter, the moving image recorded in this way is referred to as “multi-focus moving image”. The digital camera 100 has a specific shooting mode for recording this multifocus moving image. For example, the user can set the digital camera 100 to the specific shooting mode by operating a menu screen or an operation dial.

  A multi-focus moving image is a moving image that is recorded while continuously changing the focus position (that is, the focus lens position). As a multi-focus moving image, for example, a high-resolution 4K moving image having about 4000 × 2000 pixels is recorded. The depth synthesis process is executed using a plurality of frame images included in the frame images constituting the 4K moving image. Since the images synthesized in this way are based on 4K moving images, they have high image quality.

  In recording a multi-focus moving image, moving image shooting is performed while moving the focus lens 111 from the closest end to the infinite end (or vice versa) as shown in FIG. 4, that is, while changing the in-focus position. . The multi-focus moving image recorded in this way includes a plurality of frame images Pn recorded at different in-focus positions as shown in FIG. In the depth synthesis process, the range of depth synthesis is automatically set by the digital camera 100. Then, a frame image (hatched image in the figure) focused in the automatically set composition range is selected from the photographed plural frame images Pn and synthesized. Hereinafter, the synthesis range is also referred to as a depth synthesis range. Details of the automatic setting of the synthesis range will be described later.

  In the digital camera 100 of the first embodiment, a plurality of AF (Auto Focus) areas are set in the image area 400 as shown in FIG. In the first embodiment, 49 (7 rows × 7 columns) AF areas are set. For each AF area, a number (1-49) for specifying each AF area is added. For each AF area, a frame in focus on the subject in the AF area (hereinafter referred to as “focus frame”) is specified, and information regarding the specified focus frame is recorded in a focus information table (described later). The Note that the digital camera 100 performs a focus search operation before recording (capturing) a multifocus moving image, and detects the in-focus position of each AF area to generate an in-focus information table.

  7A and 7B are diagrams showing the data structures of the focusing information table 60 and the focusing information table 61, respectively. The in-focus information table 60 has information (recently in-focus lens position) indicating the focus lens position (Pnear) when the focus is closest to the closest end in the focus search operation and the focus is closest to the infinite end. The information (the farthest focusing lens position) indicating the current focus lens position (Pfar) is stored (see FIG. 7A). Further, the focusing information table 61 associates each AF area with the position of the focus lens 111 when focusing on each AF area and the frame number of the frame having the focusing state in each AF area. Manage (see FIG. 7B).

  For example, in the focus search operation, the contrast value is calculated for each AF area while moving the focus lens 111. For example, in the 19th AF area 41 shown in FIG. 6A, it is assumed that the peak of the contrast value is detected at the focus lens position P as shown in FIG. In that case, the focus lens position P is recorded in the column of the 19th AF area of the focusing information table 61 shown in FIG. 7B. Thereafter, in the multifocus moving image recording operation, the frame number of the frame shot at the focus lens position P (in this example, “50”) corresponds to the position of the focus lens 111 (in this example, “P”). And recorded in the focusing information table 61. The in-focus information table 60 and the in-focus information table 61 are stored, for example, in a header portion of moving image data obtained by moving image shooting.

  Details of the multifocus moving image recording process (S1) for use in depth synthesis will be described with reference to the flowchart of FIG.

  In the digital camera 100, a desired angle of view is set by a user's operation of the zoom lens 112 or the like in a state where a specific shooting mode for recording a multi-focus moving image is set. After the angle of view is set, when the user presses the release button 211 halfway (YES in S11), the controller 180 detects the in-focus position for each of the AF areas of the image to detect the in-focus information table 60 and the in-focus information table 60. A focus search for creating the focus information table 61 is performed (S12).

  In the focus search, the controller 180 detects the contrast value for each AF area while moving the focus lens 111 from the closest end to the infinite end (or vice versa) (see FIGS. 6A and 6B). ).

  Then, the controller 180 creates the focusing information table 61 based on the detected contrast value. Specifically, the controller 180 moves the focus lens 111 from the closest end to the infinite end, and the position of the focus lens 111 when the contrast value is maximum in a plurality of images for each AF area. Is calculated (see FIG. 6B), and the position of the focus lens 111 is recorded in the focusing information table 61 (see line c in FIG. 7B). When the contrast value of any image is lower than a predetermined threshold value for one AF area, the in-focus position is not determined for the AF area, and the in-focus position is unknown in the in-focus information table 61. A predetermined value indicating is recorded.

  After executing the focus search in the entire AF area, the controller 180 further moves the focus lens 111 from the closest end to the infinite end, and the in-focus position closest to the closest end ( Pnea) and the focus position (Pfar) closest to the infinite end are recorded in the focus information table 60 (see line a in FIG. 7A). Thus, the focus search ends. In this state, the focusing information table 61 does not yet include frame number information.

  After the focus search is completed, the controller 180 determines whether or not the release button 211 is half-pressed by the user (S13).

  If the user has not pressed the release button 211 halfway after the focus search ends (NO in S13), the controller 180 returns the process to step S11. As a result, the user can redo the focus search.

  If the user continues to press the release button 211 halfway after the focus search is completed (YES in S13), the controller 180 continues to determine whether or not the release button 211 has been fully pressed (S14).

  Thereafter, when release button 211 is fully pressed by the user (YES in S14), a moving image recording operation for recording a multifocus moving image is started (S15).

  That is, the controller 180 returns the focus lens 111 to the closest end, and moves the focus lens 111 from the latest focus lens position (Pnear) to the farthest focus lens position (Pfar) as shown in FIG. (Multi-focus video) is recorded (S15). By limiting the moving range of the focus lens 111 during moving image recording to a range (Pnear to Pfar) obtained by the focus search, moving image recording is not performed in a range where focus cannot be obtained. As a result, the time required for moving image recording can be shortened. In the moving image recording, a moving image is recorded according to a predetermined format for moving image data. For example, a moving image is recorded according to the MP4 standard (H.264 / MPEG-4 AVC format). During execution of moving image recording, for example, an icon or a message indicating that moving image recording is in progress may be displayed on the display monitor 220.

  Further, the controller 180 associates the position of the focus lens 111 with the frame number of each frame constituting the moving image for each AF area during moving image recording. That is, for each AF area, the number of the frame shot at the focus lens position indicated by the focus lens position (see line c in FIG. 7B) of the focus information table 61 is specified, and the focus information is associated with the focus lens position. Record in table 61. Thus, the frame number is associated with each AF area in the focusing information table 61 (see the b line in FIG. 7B).

  During the recording of the moving image, the image being recorded is displayed on the display monitor 220. At that time, highlighting may be performed for the user to recognize the in-focus area with respect to the in-focus area in the image. . By performing this highlighting, the user can easily grasp the focused area in the image.

  Returning to FIG. 8, when the moving image recording (S15) is completed, the moving image data in which the focusing information table 60 and the focusing information table 61 are recorded in the header portion is recorded in the memory card 200 (S16). Thereby, the recording process (S1) of the moving image (multifocus moving image) ends.

  After the moving image recording (S1) is completed, a depth synthesis process (S2) is executed.

[2-1-2. Depth synthesis process]
Hereinafter, the depth synthesis process (S2) will be described. FIG. 10 is a flowchart showing the depth synthesis process. In the depth synthesis process, the controller 180 first automatically sets the depth synthesis range (synthesis range) in the depth synthesis process using the frame images constituting the moving image data (S21), and performs the synthesis process based on the set synthesis range. Perform (S22). Hereinafter, each process (S21, S22) will be described.

[2-1-2-1. Depth composition range setting]
The automatic setting of the depth synthesis range will be described with reference to FIGS.

  FIG. 11 is a flowchart showing the setting process of the depth synthesis range. In the depth composition range setting process, the controller 180 first refers to the focus information table 61 and identifies the subject included in each AF area for each AF area (S31). That is, it is specified which of the plurality of subjects included in the image each AF region includes. For example, when the frame image constituting the multifocus moving image includes a plurality of subjects as shown in FIG. 12A, the subject included in the AF region is specified for each AF region. Next, the controller 180 specifies the importance of the subject for each AF area (S32). Here, the importance of the subject is set according to the position of the subject in the image. Next, the controller 180 calculates the cumulative importance for each subject (S33). A method for calculating the cumulative importance will be described later. Next, the controller 180 sets a synthesis range for depth synthesis (S34). Specifically, the controller 180 determines a main subject (hereinafter referred to as “main subject”) based on the cumulative importance of the subject, and sets the composite range so that the determined main subject is the center of the composite range. Hereinafter, each process (S31 to S34) will be described in detail.

  First, a subject determination information table, a determination threshold table, and a weight table used in each process (S31 to S34) will be described.

  FIG. 13 is a diagram illustrating a configuration example of the subject determination information table 70. The subject determination information table 70 is information for managing which subject (all or a part of the subject) of a plurality of subjects included in the image is included in each AF area. As shown in FIG. 13, the subject determination information table 70 includes a focus lens position in the AF area, an AF area number indicating the AF area, a subject number indicating the subject included in the AF area, and a subject number included in the AF area. The importance level indicating the importance level is managed in association with each other. In the subject determination information table 70, each data is sorted in the order of the focus lens position.

  FIG. 13 shows an example of the subject determination information table 70 when the frame image includes a plurality of subjects 1 to 8 as shown in FIG. 12A. The subject 1 shows the lower right flower on the frame image shown in FIG. 12A. The subject 2 shows a flower arranged above the flower of the subject 1 on the image. The subject 3 shows a central person on the image. The subject 4 shows the flower on the lower left on the image. A subject 5 shows a person on the left side of the image. The subject 6 shows the trees on the upper right on the image. The subject 7 shows the upper sky on the image. The subject 8 shows the upper left cloud on the image. A subject number 1 shown in FIG. Similarly, subject number 2 to subject number 8 correspond to subject 2 to subject 8, respectively.

  FIG. 14 is a diagram illustrating a configuration example of the determination threshold value table 72. The determination threshold table 72 manages the difference (threshold) of the focus lens position when determining whether or not the subject is the same subject for each predetermined range of the focus lens position. The determination threshold is set to be different for each predetermined range of the focus lens position. For example, when the focus lens position is 0 or more and less than 100, the determination threshold is set to 10. That is, for two adjacent AF areas, when the focus lens positions of the respective areas are in the range of 0 to less than 100, the difference between the focus lens positions of those AF areas should be within 10 (determination threshold). For example, it is determined that the subjects in the two AF areas are the same subject, and the same subject number is assigned. The determination threshold is set to a larger value as the value of the focus lens position increases (closer to the infinite end). In addition, the predetermined range is set so that the width of the focus lens position increases as the focus lens position approaches the infinite end. By setting in this way, when the focus lens is close to the close end, if there is any difference in the focus lens position in the two areas, the subject in the two areas is determined to be another subject. Is done. On the other hand, when the focus lens is at a position close to the infinity end, it is determined that the subjects in the two areas are the same subject even if the difference between the focus lens positions in the two areas is large.

  The weight table will be described with reference to FIGS. 15A and 15B. The weight table is a table for managing the importance (weight) of the subject included in each AF area for each AF area. In the first embodiment, as shown in FIG. 15A, the weight indicating the importance of the subject is set for each AF area. In particular, in the example of FIG. 15A, the weight (importance) is set so that the value becomes larger as it is closer to the center of the image. This is because the subject considered to be important by the photographer is usually considered to be placed in the central area of the image. FIG. 15B is a diagram showing the configuration of the weight table 74 when the weight is set for each AF area as shown in FIG. 15A. The way of setting the weight is not limited to the mode shown in FIG. 15A.

  With reference to the flowchart of FIG. 16, the subject calculation processing (step S31) for each AF area in the flowchart of FIG. 11 will be specifically described. First, the controller 180 refers to the focus information table 61 and stores values in the “focus lens position (focus position)” and “AF area number” in the subject determination information table 70 (S41). At that time, the controller 180 rearranges the data in the focus information table 61 in ascending order of the focus lens position (focus position) (ascending order), and “focus lens position (focus position)” in the subject determination information table 70. And “AF area number” are stored.

  For this reason, the controller 180 reads data from the focus information table 61 (see FIG. 7B) in the order of focus lens position (ascending order), and stores the AF area number in association with the focus lens position in the subject determination information table 70. To go. At the time of completion of the processing in step S41, the subject determination information table 70 has not yet been stored with the subject number and importance, and is incomplete.

  Next, the controller 180 performs processing for storing the subject number in the subject determination information table 70 (S42 to S50). First, the controller 180 initializes each of the counter i and the variable “subject No” to “1” (S42).

  Next, the controller 180 sets the value of the i-th focus lens position (focus position) in the subject determination information table 70 to the variable “reference position” (S43).

  Next, the controller 180 refers to the determination threshold value table 72 and acquires a threshold value that serves as a determination criterion when determining whether or not the subject is the same subject (S44). In the first embodiment, the controller 180 determines whether or not the subject is the same subject based on the focus lens position. Specifically, if the difference between the in-focus position of one subject and the in-focus position of another subject adjacent to the subject is equal to or less than a threshold value, the subjects are determined to be the same subject. The threshold value at this time is provided by the determination threshold value table 72. As described above, in the determination threshold value table 72, the threshold value is set for each predetermined range of the focus lens position.

  The controller 180 sets the value of the i-th focus lens position (focus position) in the subject determination information table 70 to the variable “current position” (S45). Next, the controller 180 determines whether or not the difference between the current position and the reference position is equal to or less than a threshold value (S46).

  If the difference between the current position and the reference position is equal to or smaller than the threshold (YES in S46), the controller 180 sets the value of the subject No as the value of the i-th subject number in the subject determination information table 70 (S47). This is because if the difference between the current position and the reference position is less than the threshold, the subject at the current position and the subject at the reference position can be determined to be the same subject because the focus lens positions are close to each other.

  Then, the value of the counter i is incremented by 1 (S48). Thereafter, in step S49, it is determined whether or not the value of the counter i is 49 or less. If the value of counter i is 49 or less (YES in step S49), the process returns to step S45. If the value of counter i exceeds 49 (NO in step S49), the process ends.

  On the other hand, if the difference between the current position and the reference position is larger than the threshold value in step S46 (NO in S46), the controller 180 increments the subject number by 1 (S50) and returns to step S43.

  With the above processing, the focus lens position (focus position), the AF area number, and the subject number are set in the subject determination information table 70. As a result, as shown in FIG. 12B, a plurality of subjects 1 to 8 are specifically extracted from the frame image based on the focus lens position for each AF area. However, at this time, the importance level is not yet set in the subject determination information table 70.

  Next, referring to the flowchart of FIG. 17, in the flowchart of FIG. 11, the process of specifying the importance of the subject included in each AF area (step S32) and the process of calculating the cumulative importance for each subject (step S33) Will be described in detail.

  First, the controller 180 sets “importance” in the subject determination information table 70 (S51). Specifically, the controller 180 acquires the “weight” of each AF area from the weight table 74 and stores it in “importance” in the subject determination information table 70. Thereby, as shown in FIG. 13, the subject determination information table 70 in which all values are stored is completed. Step S51 is a detailed step of step S32 in FIG.

  Next, the controller 180 performs processing for calculating the cumulative importance for each subject (S52 to S60). Therefore, the controller 180 first initializes the counter i to “1” and the variable “cumulative importance” to “0” (S52).

  Thereafter, the controller 180 sets the value of “subject number” corresponding to the i-th focus lens position (focus position) in the subject determination information table 70 to the variable “reference subject No” (S53). Further, the controller 180 sets the value of “subject number” corresponding to the i-th focus lens position (focus position) to the variable “current subject No” (S54).

  Next, the controller 180 compares the reference subject No with the current subject No (S55). When the reference subject No and the current subject No are equal (YES in S55), the controller 180 sets the importance value corresponding to the i-th focus lens position of the subject determination information table 70 to the value of the variable “cumulative importance”. Are added (S56). That is, in step S56, the importance is added to the AF areas having the same subject number in the subject determination information table 70. By summing the importance of AF areas having the same subject number in this way, a subject having a larger area on the image has a higher cumulative importance. For the importance of each AF area, a value weighted based on the position of the subject in the image is used. As described above, in the first embodiment, the cumulative importance of the subject is set based on the area and position of the subject on the image.

  Thereafter, the controller 11 increments the counter i by 1 (S57), and determines whether the counter i is 49 or less (S58). If the value of counter i is 49 or less (YES in step S58), the process returns to step S54. If the value of counter i exceeds 49 (NO in step S58), the process ends.

  On the other hand, when the reference subject No and the current subject No are not equal (NO in S55), the controller 180 determines the value (subject number) indicated by the variable “reference subject No” and the current value of the variable “cumulative importance”. Are stored in a subject importance table for managing the cumulative importance for each subject (S59). FIG. 18 is a diagram illustrating a configuration example of the subject importance degree table. The subject importance level table 76 manages subject numbers and cumulative importance in association with each other.

  Thereafter, the value of the variable “cumulative importance” is reset to 0 (S60), and the process returns to step S53. As a result, the target whose cumulative importance is calculated shifts to the subject indicated by the next subject number in the subject determination information table 70.

  By repeating the above processing, the cumulative importance is calculated for each of all subjects indicated by the subject numbers recorded in the subject determination information table 70, and the calculation result is stored in the subject importance table 76. These steps S52 to S60 are detailed steps of step S33 in FIG.

  With reference to the subject importance degree table 76 recorded as described above, the range of the image used in the depth synthesis process is set. FIG. 19 is a flowchart showing a detailed process of the depth synthesis range calculation process (step S34) in the flowchart of FIG. In this process, the range of the image used in the depth synthesis process is set.

  The controller 180 refers to the subject importance degree table 76 and acquires the subject number corresponding to the highest cumulative importance (S61). For example, in the example illustrated in FIG. 18, the controller 180 acquires the subject number “3” corresponding to the highest cumulative importance “101”. Thus, the main subject that the user considers important is detected by specifying the subject number corresponding to the highest cumulative importance.

  Next, the controller 180 refers to the subject determination information table 70, and is located closest to the focus lens position (focus position) associated with the acquired subject number (that is, the main subject). The focus lens position (hereinafter referred to as “most recent focus position”) and the focus lens position closest to the infinite end (hereinafter referred to as “farthest focus position”) are acquired (S62). For example, when “3” is acquired as the subject number corresponding to the highest cumulative importance (that is, when “subject 3” is detected as the main subject), the subject is determined from the subject determination information table 70 (FIG. 13). Among the focus lens positions associated with the number “3”, “400” is acquired as the most recent focus position and “470” is acquired as the farthest focus position.

  Next, the controller 180 calculates a focus lens position (hereinafter referred to as “the center position of the synthesis range”) that is the center of the synthesis range from the latest focus position and the farthest focus position (S63). Specifically, the controller 180 acquires from the subject determination information table 70 the focus lens position that is closest to the intermediate value between the latest focus position and the farthest focus position. In the subject determination information table 70, when the focus lens position candidate closest to the intermediate value is on both the near end side and the infinite end side with respect to the intermediate value, the one with the larger number of AF areas is set to the intermediate value. The focus lens position closest to. That is, the number of AF areas in which the focus lens position closest to the intermediate value is recorded is compared with the number of AF areas in which the focus lens position closest to the intermediate value is recorded. The focus lens position corresponding to the larger number is set as the focus lens position closest to the intermediate value. Note that the focus lens position closest to the intermediate value may be simply determined as the closest focus lens position on either the closest end side or the infinity side, regardless of the number of AF areas. Then, the controller 180 sets the focus lens position closest to the intermediate value as the center position of the synthesis range.

  For example, from the subject determination information table 70 shown in FIG. 13, “400” is the closest focus lens position associated with the subject number “3”, and “470” is the most infinite focus lens position. Is acquired, the intermediate value is calculated as “435”. In this case, the focus lens position “430” on the closest end side and the focus lens position “440” on the infinity side are candidates for the focus lens position closest to the intermediate value “435”. The AF area where the focus lens position is “430” is only one area having the AF area number “11”. On the other hand, there are three areas having AF area numbers “17”, “19”, and “32” as AF areas where the focus lens position is “440”. Therefore, since there are more AF areas where the focus lens position is “440” than there are AF areas where the focus lens position is “430”, the focus lens position closest to the intermediate value is “440”. In this way, “440” is calculated as the focus lens position closest to the intermediate value “435” in the subject determination information table 70. This focus lens position “440” is set as the center position of the synthesis range.

  Returning to FIG. 19, the controller 180 then sets the depth synthesis range based on the center position of the synthesis range (S <b> 64). Specifically, the controller 180 sets, as a lower limit of a range that can be set as a synthesis range (hereinafter referred to as a settable range), a position that is a predetermined value away from the focus lens position toward the closest end at the center position of the synthesis range. calculate. Further, the controller 180 calculates a position where the focus lens position is separated by a predetermined value from the focus lens position at the center position of the synthesis range toward the infinite end as an upper limit of the settable range.

  Here, the above-mentioned predetermined value is set to such a distance that the combined image does not fail when the images are combined. In the case where two adjacent areas in one image are set as the depth synthesis range, if the difference in focus lens position (focus position) between the two areas is large, an unnatural synthesized image may be obtained. This is because when the difference in focus lens position (focus position) between the two areas is large, the focus state (blurring method) in each area is greatly different. That is, when these two regions are combined, the connection is not smooth in the vicinity of the boundary. In the first embodiment, therefore, the range of the focus lens position corresponding to the synthesis range is limited so that the synthesis is not performed between the areas where the difference in focus lens position (focus position) is larger than a predetermined value when performing the depth synthesis. To do.

  Then, the controller 180 refers to the subject determination information table 70 and identifies a focus lens position closest to the closest end (hereinafter referred to as “composition range start point position”) corresponding to a range that can be set as the composition range. . Further, the controller 180 refers to the subject determination information table 70 and specifies the focus lens position closest to the infinite end (hereinafter referred to as “composition range end point position”) corresponding to the range that can be set as the composition range. As a result, a predetermined range centered on the main subject is set as the synthesis range.

  With reference to FIG. 20, the above-described synthesis range setting process will be described using a specific example. An example will be described in which the focus lens position (focus position) at the center of the synthesis range is “440” and the predetermined value for preventing failure is “300”. In this case, “140” (= 440−300) is calculated as the lower limit of the range that can be set as the synthesis range, and “740” (= 440 + 300) is calculated as the upper limit of the range that can be set as the synthesis range. Referring to the subject determination information table 70, “150” is specified as the closest focus lens position and “640” is specified as the farthest focus lens position in the settable range (range between 140 and 740). Accordingly, the range of the focus lens position (focus position) from 150 to 640 is set as the synthesis range of the depth synthesis process.

  FIGS. 21A to 21C are diagrams for explaining the synthesis range of the depth synthesis process set for the main subject. For example, in the first embodiment, the subject 3 is specified as the main subject. In this case, as shown in FIG. 21A, a composition range is set with the subject 3 as the center. When the subject 2 shown in FIG. 12B is a main subject, a synthesis range is set around the subject 2 as shown in FIG. When the subject 4 shown in FIG. 12B is the main subject, a synthesis range is set around the subject 4 as shown in FIG. In (A) to (C) of FIG. 21, the non-compositing area is an area where there is a high possibility that an image will fail when depth synthesis is performed. That is, the difference between the focus lens positions (in-focus positions) becomes very large at the boundary between the synthesis range area and the non-synthesis area, and it is highly possible that an unnatural image is generated when the depth synthesis is performed. For this reason, in the first embodiment, on each of the closest end side and the infinite end side, a region where the focus lens position is a predetermined value or more away from the focus lens position at the center position of the synthesis range is not used for the synthesis process. It is set as an area.

[2-1-2-2. Compositing process]
When the synthesis range (that is, the start point position and the end point position) of the depth synthesis process is set as described above, the frame images that are focused within the synthesis range are depth-synthesized, and a synthesized image is generated. For this reason, the controller 180 refers to the focus information table 61 shown in FIG. 7B, and the frame number corresponding to each of the start point position (focus lens position) of the composite range and the end point position (focus lens position) of the composite range. Is identified. The controller 180 performs the depth synthesis process using all or part of the frame images indicated by the two specified frame numbers and the frame image between them. A known technique can be used for the depth synthesis process.

[3. Effect]
As described above, the digital camera 100 (an example of an imaging device) according to the first embodiment includes the CCD 140 (an example of an imaging unit) that captures a subject image while changing a focus position and generates a plurality of image data. An image processing unit 160 that combines a plurality of frame image data (an example of a plurality of image data) generated by the CCD 140 to generate still image data having a depth of field deeper than the depth of field of the plurality of image data. An example of an image processing unit), and a controller 180 (an example of a control unit) that controls the image processing unit 160. The controller 180 detects a main subject from an image indicated by one frame image data among the plurality of frame image data, determines a range in which the plurality of image data are combined based on the detected position of the main subject, The image processing unit 160 is controlled so as to generate still image data by combining image data to be focused within a determined range among a plurality of image data.

  The digital camera 100 having the above configuration automatically detects a main subject from an image, sets a depth composition range around the main subject, and performs a depth composition process using image data that is focused within the set range. To do. For this reason, the user can omit operations such as designating the depth synthesis range, and can easily obtain an image with the deepest depth of field within a range in which image failure does not occur during synthesis.

  In the first embodiment, in the digital camera 100, the controller 180 calculates the importance (for example, the cumulative importance shown in FIG. 18) for each of a plurality of subjects included in the image indicated by one frame image data. Based on the importance, the main subject is set from among a plurality of subjects. As a result, a composite image centered on the main subject can be generated, so that an image focused on the subject considered more important by the user can be obtained.

  The importance (cumulative importance) may be calculated based on any one parameter among the area, position, type, and the like of the subject on the image, or may be calculated based on a plurality of parameters. In the first embodiment, the importance is set to a higher value as the area of the subject on the image is larger. Further, in the first embodiment, the importance is set to a higher value as the position of the subject is closer to the center of the image. By setting the importance based on both the area and the position of the subject on the image in this way, an image focused on the subject that the user considers more important can be obtained more efficiently.

  In the first embodiment, the controller 180 identifies the image by dividing it into a plurality of AF areas (an example of areas). Then, the controller 180 has a difference in focus position between two adjacent areas (for example, the first area and the second area) of the plurality of areas equal to or less than the determination threshold value (an example of the predetermined value) illustrated in FIG. In this case, it is determined that two adjacent areas include at least one same subject. In this way, the controller 180 can determine the difference between subjects.

  In the first embodiment, the determination threshold is set to be different depending on the in-focus position (see determination threshold table 72). For example, the determination threshold may be set smaller as the in-focus position is closer to the close end, and the determination threshold may be set to be larger as the in-focus position is farther from the close end (that is, closer to the infinite end). (See FIG. 14). Since images taken at different in-focus positions on the close side have a large amount of blurring of the image, there is a high possibility that the image will fail during image synthesis. For this reason, the determination threshold is set smaller on the close side. On the other hand, an image taken at the far-in-focus position is in focus on the entire image, and is therefore less likely to fail during image composition. For this reason, the determination threshold is set large on the far side. This determination threshold value may be changed according to the lens used for photographing.

  In the first embodiment, the controller 180 detects the frame number (an example of information for specifying a plurality of image data) and the focused AF area in each of the plurality of image data when detecting the main subject. The main subject is detected using the focusing information table 61 that associates the information shown with the focus lens position (focusing position) when generating each of the plurality of image data. By referring to the focusing information table 61, it is possible to associate the focus lens position (focusing position) with the frame image.

  In the first embodiment, the CCD 140 (an example of an imaging unit) captures a subject image and generates moving image data. The plurality of image data is data of a plurality of frame images constituting the moving image data. Thus, in the first embodiment, a plurality of image data can be obtained in a short time, and the time required for the synthesis process can be shortened.

[4. Specific example of automatic main subject setting]
In the first embodiment, the main subject is set when setting the depth synthesis range. In the first embodiment, when setting the main subject, the importance is calculated based on the area and position of the subject, and the main subject is set based on the importance. Hereinafter, the setting of the main subject will be described using another specific example.

  FIG. 22A is a diagram illustrating an example of a frame image including a plurality of subjects. The areas of these subjects on the image are different from each other. FIG. 22B is a diagram in which a frame indicating an AF area is superimposed on the frame image of FIG. 22A.

  In FIG. 22A, a tunnel 81 and an automobile 82 are arranged as subjects on the image. On the image, the area of the tunnel 81 is larger than the area of the automobile 82. For example, as shown in FIG. 22B, the area of the tunnel 81 corresponds to the size of the region 91 and is equivalent to 18 AF regions. The area of the automobile 82 corresponds to the size of the area 92 and is equivalent to one AF area. On the image, the in-focus positions of the tunnel 81 and the automobile 82 are sufficiently different. That the in-focus positions are sufficiently different means that they are separated from a predetermined range or more as a range in which image breakdown occurs when combined. The in-focus position of the tunnel 81 is closer to the closest end side, and the in-focus position of the automobile 82 is closer to the infinite end side. Further, the center positions of the tunnel 81 and the car 82 on the image are the center of the image and are equal.

  When the image of FIG. 22A as described above is captured, the digital camera 100 of Embodiment 1 uses the tunnel 81 having a larger area on the image as a main subject. A depth synthesis range is set around the tunnel 81.

  FIG. 23A is a diagram illustrating an example of a frame image including a plurality of subjects having different center positions. FIG. 23B is a diagram in which a frame indicating an AF area is superimposed on the frame image of FIG. 22A.

  In FIG. 23A, a large ship 83 and a small ship 84 are arranged as subjects on the image. The area of the large ship 83 on the image is equal to the area of the small ship 84 on the image. For example, as shown in FIG. 23B, the area of the large ship 83 corresponds to the size of the region 93 and is equivalent to eight AF regions. The area of the small boat 84 corresponds to the size of the region 94 and is equivalent to eight AF regions. On the image, the focal positions of the large ship 83 and the small ship 84 are sufficiently different. As described above, “sufficiently different” means that the image is broken beyond a predetermined range as a range in which image breakdown occurs when the images are combined. The center positions of the large ship 83 and the small ship 84 on the image are different. The large ship 83 is located at the left end on the image, and the small ship 84 is located closer to the center of the image than the large ship 83.

  When the image of FIG. 23A is captured, the digital camera 100 according to Embodiment 1 uses the small ship 84 whose center position on the image is closer to the center of the image as a main subject. A depth synthesis range is set around the small ship 84.

  As described above, if the digital camera of the first embodiment (an example of an imaging device) is expressed in another aspect, the digital camera combines a plurality of image data, and the main subject in each of the plurality of image data. Still image data is generated so that the depth of field of the main subject in the still image data is deeper than the depth of field. In the digital camera, when the first subject and the second subject are arranged on each image of the plurality of image data, the areas of the first subject and the second subject on the image are different, and When the in-focus positions of the first subject and the second subject are different and the center positions on the images of the first subject and the second subject are equal, the first subject and the second subject The larger subject area is the main subject. In addition, in the digital camera, when the first subject and the second subject are arranged on each of the plurality of image data, the areas of the first subject and the second subject on the image are equal. And when the in-focus positions of the first subject and the second subject are different and the center positions on the images of the first subject and the second subject are different, of the first subject and the second subject, The one closer to the center of the image is the main subject.

  The digital camera 100 according to the first embodiment can set the synthesis range centering on the subject that the user considers more important when automatically setting the depth synthesis range. Therefore, it is possible to easily obtain an image with a deep depth of field desired by the user.

(Other embodiments)
As described above, the first embodiment has been described as an example of the technique disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can also be applied to an embodiment in which changes, replacements, additions, omissions, and the like are appropriately performed. Moreover, it is also possible to combine each component demonstrated in the said Embodiment 1, and it can also be set as a new embodiment. Therefore, other embodiments will be exemplified below.

  (1) In Embodiment 1 described above, depth synthesis is performed using frame images that constitute a multi-focus moving image. However, a plurality of still images generated by continuous shooting are used instead of frame images that constitute a moving image. Depth synthesis may be performed.

  (2) In Embodiment 1 described above, the importance level (cumulative importance level) of the subject is set based on both the area of the subject on the image and the position in the image. However, the importance level is set based on one of the conditions. May be set. Alternatively, the importance may be set based on other conditions in addition to or instead of the area of the subject on the image and the position in the image. For example, the importance may be set according to the type of subject. Specifically, when the subject is a person's face, the importance may be set to be high, and when it is not, the importance may be set to be low. Furthermore, the importance may be set according to the focus position of the subject. Specifically, it may be set so that the degree of importance of the subject focused on the closest end side becomes high. In the first embodiment, the center of the main subject is the center of the depth synthesis range, but the center of the main subject may be the start point or the end point of the depth synthesis range. For example, the depth composition range may be determined using the in-focus position closest to the end as a starting point.

  (3) The idea disclosed in the first embodiment can be applied to both types of digital cameras, that is, an interchangeable lens camera and a lens-integrated camera.

  (4) In Embodiment 1 described above, the digital camera is used as an example of the imaging apparatus, but the imaging apparatus is not limited to this. The idea of the present disclosure can be applied to various imaging devices capable of shooting moving images, such as a digital video camera, a smartphone, and a wearable camera.

  (5) In the above embodiment, the image pickup device as the image pickup unit is configured by the CCD, but the image pickup device is not limited to this. The imaging device may be configured by an NMOS (n-Channel Metal-Oxide Semiconductor) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

  As described above, the embodiments have been described as examples of the technology in the present disclosure. For this purpose, the accompanying drawings and detailed description are provided. Accordingly, among the components described in the accompanying drawings and the detailed description, not only the components essential for solving the problem, but also the components not essential for solving the problem in order to illustrate the above technique. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential as those non-essential components are described in the accompanying drawings or the detailed description. Moreover, since the above-mentioned embodiment is for demonstrating the technique in this indication, a various change, replacement, addition, abbreviation, etc. can be performed in a claim or its equivalent range.

  The present disclosure is applicable to an imaging apparatus that can capture a moving image. Specifically, the present invention can be applied to various imaging devices that can shoot moving images, such as digital cameras, digital video cameras, smartphones, and wearable cameras.

60 Focus Information Table 61 Focus Information Table 70 Subject Determination Information Table 72 Determination Threshold Table 74 Weight Table 81 Tunnel 82 Automobile 83 Large Ship 84 Small Ship 91 Area 92 Area 93 Area 94 Area 100 Digital Camera (Imaging Device)
DESCRIPTION OF SYMBOLS 110 Optical system 111 Focus lens 112 Zoom lens 113 Camera shake correction lens 120 Lens drive part 130 Shutter 140 CCD (imaging part)
150 A / D converter 160 Image processing unit 170 Buffer 180 Controller (control unit)
190 card slot 200 memory card 210 operation member 220 display monitor 240 built-in memory 300 aperture unit 400 image area

Claims (9)

An imaging unit that captures a subject image while changing a focus position and generates a plurality of image data;
An image processing unit that synthesizes the plurality of image data generated by the imaging unit and generates still image data having a depth of field deeper than a depth of field of each of the plurality of image data;
A control unit for controlling the image processing unit,
The controller is
A main subject is detected from an image indicated by one image data of the plurality of image data, a range in which the plurality of image data is combined is determined based on the detected position of the main subject, and the plurality of image data An image pickup apparatus that controls the image processing unit so as to generate the still image data by combining image data that is focused within the determined range.   The imaging apparatus according to claim 1, wherein the control unit calculates importance for each of a plurality of subjects included in the image, and sets the main subject from the plurality of subjects based on the calculated importance. .   The imaging apparatus according to claim 2, wherein the importance is set to a higher value as an area of each of the plurality of subjects on the image is larger.   The imaging apparatus according to claim 2, wherein the importance is set to a higher value as the position of each of the plurality of subjects is closer to the center of the image.   The control unit divides and identifies the image into a plurality of regions, and the difference between the in-focus positions of the first region and the second region adjacent to each other among the plurality of regions is a predetermined value or less. The imaging device according to claim 2, wherein the first region and the second region are determined to include at least one same subject among the plurality of subjects.   The imaging apparatus according to claim 5, wherein the predetermined value is set to be different depending on the in-focus position.   The control unit, when detecting the main subject, information identifying each of the plurality of image data, information indicating a focused area in each of the plurality of image data, and the plurality of image data The imaging apparatus according to claim 1, wherein detection is performed using information that associates the in-focus position when generating each. The imaging unit captures the subject image to generate moving image data,
The imaging apparatus according to claim 1, wherein the plurality of image data is data of a plurality of frame images constituting the moving image data. Combining a plurality of image data, the still image data is converted so that the depth of field of the main subject in the still image data is deeper than the depth of field of the main subject in each of the plurality of image data. An image composition method for generating,
When a first subject and a second subject are arranged on each of the plurality of image data,
The areas of the first subject and the second subject on the image are different, the in-focus positions of the first subject and the second subject are different, and the first subject and the second subject are different. When the center position of the subject on the image is equal, the larger of the area on the image of the first subject and the second subject is the main subject,
And,
The areas of the first subject and the second subject on the image are equal, the in-focus positions of the first subject and the second subject are different, and the first subject and the first subject are different. When the center positions of the two subjects on the image are different, the one closer to the center of the image of the first subject and the second subject is the main subject.
Image composition method.