JP2009111716A - Imaging apparatus, program, and template generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a means for generating a template more suitable for subject tracking processing.
An imaging apparatus includes an imaging unit, a template generation unit, and a subject tracking unit. The imaging unit captures a subject and generates image data. The template generation unit identifies a feature region indicating the characteristics of the subject to be tracked from the inside of the candidate region set in the image, and generates a template using the image data of the feature region. The subject tracking unit continuously detects the subject to be tracked from the input image using the template.
[Selection] Figure 1

Description

  The present invention relates to an imaging apparatus having a subject tracking function and its peripheral technology.

Conventionally, various imaging apparatuses having a subject tracking function have been proposed in order to perform focus adjustment, calculation of exposure conditions, and the like following a moving main subject. As an example, Patent Document 1 discloses an imaging apparatus that performs subject tracking by performing template matching processing on an input image.
JP 2004-112550 A

  By the way, when subjecting the imaging device to subject tracking processing, it is important how a template reflecting the characteristics of the subject to be tracked can be generated in the initial state. However, the initial template generated at the position specified by the user is often inappropriate for the imaging device to perform subject tracking processing, and there is still room for improvement in that the performance of the subject tracking function cannot be fully demonstrated. It was.

  The present invention is to solve the above-mentioned problems of the prior art. An object of the present invention is to provide a means for generating a template more suitable for subject tracking processing.

  An imaging apparatus according to a first aspect includes an imaging unit, a template generation unit, and a subject tracking unit. The imaging unit captures a subject and generates image data. The template generation unit identifies a feature region indicating the characteristics of the subject to be tracked from the inside of the candidate region set in the image, and generates a template using the image data of the feature region. The subject tracking unit continuously detects the subject to be tracked from the input image using the template.

  In a second aspect based on the first aspect, the template generation unit specifies the position of the feature region within the candidate region.

  In a third aspect based on the first aspect or the second aspect, the template generation unit specifies the size of the feature region.

  In a fourth aspect based on the second aspect, the template generation unit performs an extraction process for extracting a main subject within the candidate area. Then, the template generation unit specifies the position of the feature region based on the position of the center of gravity of the main subject.

  In a fifth aspect based on the third aspect, the template generation unit performs an extraction process for extracting a main subject in the candidate area. Then, the template generation unit specifies the size of the feature region based on the size of the main subject.

  In a sixth aspect based on the second or third aspect, the imaging apparatus further includes a memory for storing tracking history data. This tracking history data indicates the correspondence between the result of subject tracking by the subject tracking unit and the template generation conditions applied in the subject tracking. Then, the template generation unit specifies the feature region with reference to the generation condition extracted from the tracking history data according to the subject tracking result.

  In a seventh aspect based on the second or third aspect, the imaging apparatus further includes a distance measuring unit that detects subject distances at a plurality of locations on the photographing screen corresponding to the image. Then, the template generation unit identifies the feature region with reference to the subject distance in the candidate region.

  In an eighth aspect based on the first aspect, the imaging apparatus further includes a determination processing unit and a candidate area changing unit. The determination processing unit determines whether or not a portion indicating the characteristics of the subject can be detected from the inside of the candidate area. The candidate area changing unit enlarges the size of the candidate area when the determination processing unit determines that the portion indicating the above feature cannot be detected.

  In a ninth aspect based on the eighth aspect, the template generation unit specifies a feature region from the inside of the candidate region after being enlarged.

  In a tenth aspect based on the eighth aspect, the template generation unit specifies the candidate area after being enlarged as the characteristic area.

  Note that the configuration of the imaging apparatus described above includes a program for causing a computer to generate a template, a storage medium storing the above-described program, a template generation method for causing the computer to generate a template, and image processing for generating a template. What is expressed by being converted into a format such as a device is also effective as a specific aspect of the present invention.

  In the present invention, a template that is more suitable for subject tracking processing can be generated by specifying a feature region indicating the feature of the subject to be tracked from the inside of the candidate region.

<Description of the configuration of the electronic camera>
FIG. 1 is a block diagram illustrating the configuration of the electronic camera of this embodiment. The electronic camera of the present embodiment has a subject tracking function based on subject recognition.

  The electronic camera includes an imaging optical system 11, a lens driving unit 12, an imaging element 13, an AFE 14, an image processing unit 15, a first memory 16, a recording I / F 17, a monitor 18, an operation member 19, A release button 20, a CPU 21, a second memory 22 and a bus 23 are provided. Here, the image processing unit 15, the first memory 16, the recording I / F 17, the monitor 18, the CPU 21, and the second memory 22 are connected via a bus 23. The lens driving unit 12, the operation member 19, and the release button 20 are each connected to the CPU 21.

  The imaging optical system 11 includes a plurality of lens groups including a zoom lens and a focusing lens. The lens position of the focusing lens of the imaging optical system 11 is adjusted in the optical axis direction by the lens driving unit 12. For simplicity, the imaging optical system 11 is illustrated as a single lens in FIG.

  The image pickup device 13 is disposed on the image space side of the image pickup optical system 11, and the light receiving devices are two-dimensionally arranged on the light receiving surface thereof. Then, the image sensor 13 photoelectrically converts the subject image on the shooting screen by the light beam that has passed through the imaging optical system 11 to generate an analog image signal. Note that the output of the image sensor 13 is connected to the AFE 14.

  In addition to the light receiving element for generating image data, the imaging element 13 of the present embodiment is provided with a plurality of distance measuring pixels for performing pupil division type phase difference AF. The distance measuring pixels are arranged at regular intervals on the entire imaging surface of the imaging element 13. Therefore, the CPU 21 of the present embodiment can obtain the subject distance of an arbitrary part on the shooting screen based on the output of the ranging pixels of the image sensor 13. In addition, about an image pick-up element which has a ranging pixel, the thing of well-known structures, such as Unexamined-Japanese-Patent No. 2000-156823, for example is applicable.

  Here, in a shooting mode which is one of the operation modes of the electronic camera, the image sensor 13 captures a recorded image (main image) in response to a full press operation of the release button 20. Further, the imaging element 13 in the shooting mode captures a through image at predetermined intervals even during standby for shooting. The through image data is thinned and read out from the image sensor 13 and used for image display on the monitor 18 and various arithmetic processes by the CPU 21.

  The AFE 14 is an analog front end circuit that performs analog signal processing on the output of the image sensor 13. The AFE 14 performs correlated double sampling, image signal gain adjustment, and A / D conversion of the image signal. The output of the AFE 14 is connected to the image processing unit 15.

  The image processing unit 15 performs various types of image processing (color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment, etc.) on the digital image signal for one frame. The image processing unit 15 also executes image resolution conversion processing and main image data compression / decompression processing. Note that the image processing unit 15 generates and outputs a monitor display view image from the through image during shooting standby in the shooting mode.

  The first memory 16 is a buffer memory that temporarily stores image data in the pre-process and post-process of image processing by the image processing unit 15. The first memory is constituted by an SDRAM which is a volatile storage medium.

  The recording I / F 17 is formed with a connector for connecting a storage medium 24 that is a nonvolatile storage medium. The recording I / F 17 writes / reads data to / from the storage medium 24 connected to the connector. The storage medium 24 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 24.

  The monitor 18 displays various images according to instructions from the CPU 21. Note that the monitor 18 in the present embodiment may be configured with any of an electronic viewfinder having an eyepiece and a liquid crystal display panel provided on the back surface of the camera housing. On the monitor 18, the view image is displayed as a moving image under the control of the CPU 21 during shooting standby in the shooting mode. At this time, the CPU 21 can also display various information necessary for photographing on the view image of the monitor 18 in an overlay manner. The CPU 21 can also display a menu screen on which various setting items can be input on the monitor 18.

  The operation member 19 includes, for example, a command dial, a cross-shaped cursor key, a determination button, and the like. The operation member 19 receives various inputs to the electronic camera from the user. For example, the operation member 19 is used for an input operation on the above menu screen, an operation for switching the operation mode of the electronic camera, and the like.

  The release button 20 receives from the user an instruction input for starting an autofocus (AF) operation before shooting by a half-press operation and an instruction input for starting an imaging operation by a full-press operation. Note that the release button 20 in the present embodiment is also used when the user gives an instruction to start template generation processing described later.

  The CPU 21 is a processor that comprehensively controls the operation of the electronic camera. As an example, the CPU 21 executes AF using a contrast detection method using through image data in the shooting mode. Further, the CPU 21 obtains the subject brightness in the shooting screen using the through image data, and executes the AE calculation based on the subject brightness.

  The CPU 21 functions as a template generation unit 25 and a subject tracking unit 26 by executing a program stored in a second memory 22 described later.

  The template generation unit 25 generates template data for identifying the tracking target object from the input image. In the electronic camera of this embodiment, it is possible to generate template data for any object including a human, an animal, a building, a vehicle, and the like as a tracking target.

  As an example, the template generation unit 25 according to the present embodiment specifies a feature region indicating the characteristics of the subject to be tracked from the inside of the candidate region set in the shooting screen. Then, the template generation unit 25 generates template data from the through image of the specified feature region. Note that the template data includes all of the YCbCr channel data.

  In addition, the template generation unit 25 performs a determination process for determining whether or not a portion indicating the characteristics of the subject can be detected from the inside of the candidate area. And the template production | generation part 25 performs the process which expands the size of a candidate area | region, when a determination process part determines that the part which shows the characteristic cannot be detected.

  The subject tracking unit 26 continuously detects the position of the subject to be tracked in the shooting screen using the template. Thereby, the CPU 21 can perform AF control and AE calculation of spot metering following the movement of the subject corresponding to the template. The CPU 21 can also superimpose and display a mark indicating the position of the subject corresponding to the template on the view image of the monitor 18.

  In addition, the subject tracking unit 26 generates tracking history data indicating a correspondence relationship between a result of subject tracking and a template generation condition applied in subject tracking. The tracking history data is referred to by the template generation unit 25 when generating a template.

  The second memory 22 is a non-volatile storage medium such as a flash memory. The second memory 22 stores the template data and the tracking history data generated by the subject tracking unit 26. Further, the second memory 22 stores a counter indicating the number of times of expansion of a candidate area, which will be described later. The second memory 22 stores a program executed by the CPU 21. Examples of programs are shown in FIGS. 3 and 5, which will be described in detail later.

  Here, the contents of the tracking history data will be described in detail. The tracking history data in the present embodiment has a first tracking history table and a second tracking history table. Then, depending on the success or failure of the subject tracking process, tracking histories composed of combinations of the subject tracking results and the template generation conditions are registered in separate tables. The template generation conditions include (1) the size of the template, (2) the type of template generation channel used to generate the template (any of YCbCr), (3) AE information when the template is generated, (4) Object distance information (AF information) when the template is generated is included.

  FIG. 2A is a diagram schematically showing a first tracking history table showing a case where tracking of a subject is successful. In the first tracking history table, (a) “tracking history when the main image is captured after subject tracking”, (b) “subject tracking is completed by user's cancel operation or subject loss, The tracking history when the tracking duration time is long is recorded by the CPU 21. In these cases, it can be considered that the subject intended by the user has been tracked.

  The tracking history registered in the first tracking history table is sorted by the CPU 21 according to the priority order. As an example, in the first tracking history table, the tracking history of (A) is set higher in priority than the tracking history of (B). In addition, the tracking history of (a) has a higher priority as the photographing time of the main image is newer. On the other hand, the tracking history of (b) has a higher priority as the duration of subject tracking is longer. In the first tracking history table, only a predetermined number of tracking histories are stored in descending order of priority among the tracking histories satisfying the requirements (a) and (b). FIG. 2A shows an example of a first tracking history table in which five tracking histories are stored in descending order of priority (H1 to H5) from the top.

  FIG. 2B is a diagram schematically showing a second tracking history table showing a case where subject tracking has failed. In the second tracking history table, the CPU 21 records (c) “tracking history when subject tracking is completed by user cancel operation or subject lost and subject tracking duration is short”. In this case, it can be considered that the tracking of the subject intended by the user is not performed.

  The tracking history registered in the second tracking history table is also sorted by the CPU 21 according to the priority order. As an example, in the second tracking history table, the priority is set lower as the subject tracking duration is shorter. In the second tracking history table, only a predetermined number of tracking histories are stored in order of increasing priority among the tracking histories satisfying the requirement (c). FIG. 2B shows an example of a second tracking history table in which five tracking histories are stored in descending order of priority (L1 to L5) from the upper side.

<Description of electronic camera operation>
Next, an operation example in the shooting mode of the electronic camera in the present embodiment will be described with reference to the flowchart of FIG. In the following operation example, description will be made on the assumption that the subject tracking function is set to ON.

  Step S101: The CPU 21 drives the imaging device 13 to start capturing a through image. Thereafter, the through images are sequentially generated at predetermined intervals. Further, the CPU 21 displays the view image generated from the through image on the monitor 18 as a moving image. Therefore, the user can perform framing for determining the shooting composition with reference to the view image on the monitor 18.

  When the subject tracking function is on, the CPU 21 superimposes a guide mark (rectangular green frame display) indicating the registered position of the subject to be tracked on the view image (see FIG. 4). The frame display is displayed in a blinking state at the center of the monitor 18 under the control of the CPU 21. It should be noted that the user operates the release button 20 half-pressed in a state where the subject to be tracked is contained in the frame display described above, thereby causing the electronic camera to generate a template for the subject to be tracked and a desired subject to be selected. Can be tracked.

  Step S102: The CPU 21 determines whether or not the release button 20 is half-pressed. If the release button 20 is pressed halfway (YES side), the process proceeds to S103. On the other hand, when the release button 20 is not half-pressed (NO side), the CPU 21 waits for half-pressing of the release button 20.

  Step S103: The CPU 21 executes a template generation process to be described later in response to the half-press operation. As a result, the tracking target template data is generated from the through image.

  When the tracking target template is generated, the CPU 21 outputs a display (subject tracking mark) indicating the subject tracking execution state to the monitor 18. Specifically, the CPU 21 changes the guide mark frame display (S101) from the blinking state to the lighting state. As a result, the user can confirm the completion of template generation and the start of subject tracking on the monitor 18 (note that the subject tracking mark display screen is substantially the same as in FIG. 4 and is not shown).

  Step S104: The CPU 21 determines whether or not the half-press of the release button 20 has been released. When the half-press of the release button 20 is released (YES side), the process proceeds to S105. On the other hand, when the half-press of the release button 20 is continued (NO side), the process proceeds to S107.

  Step S105: In this case, the CPU 21 cancels the subject tracking process by a user operation. Then, the CPU 21 switches the frame display of the subject tracking mark from the lighting state to the blinking state (guide mark state).

  Step S106: The subject tracking unit 26 of the CPU 21 performs a process of updating the tracking history data in the second memory 22. As a result, the result of the current subject tracking process can be reflected in the first tracking history table or the second tracking history table. Thereafter, the CPU 21 returns to S102 and repeats the above operation.

  Step S107: The CPU 21 determines whether or not a determination frame for determining subject tracking is captured in the through image. When the determination frame is captured (YES side), the process proceeds to S108. On the other hand, when the determination frame is not captured, the CPU 21 waits for the determination frame to be captured. Note that the CPU 21 may be configured to adjust the interval for performing the subject tracking process from the through image (the frame rate of the determination frame) according to the user's setting operation.

  Step S108: The CPU 21 searches for the subject to be tracked from the determination frame of the through image. Specifically, the subject tracking unit 26 of the CPU 21 executes a matching process for obtaining the similarity between the template and the subject of the determination frame. Then, the subject tracking unit 26 determines the portion of the determination frame where the similarity is equal to or higher than the threshold and the highest similarity value is the position of the subject to be tracked. On the other hand, the subject tracking unit 26 determines that the subject to be tracked is lost when both the similarity between the template and the subject in the determination frame is less than the threshold.

  Step S109: The CPU 21 determines whether or not the tracking target has been detected from the shooting screen in S107. When the tracking target is detected (YES side), the process proceeds to S111. On the other hand, when the tracking target cannot be detected and lost (NO side), the process proceeds to S110.

  Step S110: The subject tracking unit 26 of the CPU 21 stops the display of the subject tracking mark and superimposes and outputs a lost display (display indicating that the subject cannot be tracked and is in a lost state) on the view image. For example, the subject tracking unit 26 performs the lost display by changing the frame display superimposed on the view image from the lighting state to the blinking state again. Thereby, the user can confirm on the monitor 18 that the subject to be tracked has been lost. Thereafter, the CPU 21 proceeds to the process of S106.

  Step S111: The subject tracking unit 26 of the CPU 21 updates the position of the subject tracking mark to be superimposed on the view image based on the position of the subject to be tracked (S108). As a result, the position of the frame display on the monitor 18 is shifted in accordance with the movement of the subject to be tracked. As a result, the user can check the subject tracking state on the monitor 18.

  Step S112: The subject tracking unit 26 of the CPU 21 executes AF using the position of the subject to be tracked (S108) as a focus detection area, and executes a known AE calculation. Note that the CPU 21 may execute AE calculation by spot metering based on the position of the subject to be tracked (S108).

  Step S113: The CPU 21 determines whether or not the release button 20 has been fully pressed. When the release button 20 is fully pressed (YES side), the process proceeds to S114. On the other hand, when the release button 20 is not fully pressed (NO side), the CPU 21 returns to S104 and repeats the above operation. Thereby, the subject tracking unit 26 can continuously detect a change in the position of the subject to be tracked in the shooting screen.

  Step S114: The CPU 21 drives the image pickup device 13 to execute a main image pickup process. The data of the main image is recorded in the storage medium 24 via the recording I / F 17 after predetermined processing is performed by the AFE 14 and the image processing unit 15.

  Step S115: The subject tracking unit 26 of the CPU 21 performs a process of updating the tracking history data in the second memory 22. As a result, the result of the current subject tracking process can be reflected in the first tracking history table. Above, description of the flowchart of FIG. 3 is complete | finished.

  Next, the subroutine of the template generation process (S103) of FIG. 3 will be described with reference to the flowchart of FIG.

  Step S201: The template generation unit 25 of the CPU 21 initializes the size of the candidate area. Here, a candidate area | region shows the range which the template production | generation part 25 searches the characteristic of the to-be-tracked subject. The template data can be generated from some or all of the images of the candidate area.

  FIG. 6 shows an example of the positional relationship between the through image and the candidate area. The candidate area in S201 is set to a rectangular area that is larger than the guide mark frame size, centered on the position of the guide mark frame. In addition, on the view image of the monitor 18, the display which shows the range of a candidate area | region is abbreviate | omitted.

  Step S202: The template generation unit 25 acquires through image data in the candidate area (S201) for each channel of YCbCr. Thereby, the template generation unit 25 generates data of each channel of YCbCr of the through image in the candidate area.

  Step S203: The template generation unit 25 obtains the average value of the Y channel gradation of the through image in the candidate area. Then, the template generation unit 25 determines whether or not the average value of the Y channel gradation exceeds the threshold value. If the above requirement is satisfied (YES side), the process proceeds to S204. On the other hand, if the above requirement is not satisfied (NO side), the image in the current candidate region has a low luminance, and it is likely that the feature of the subject to be tracked cannot be detected. Therefore, the CPU 21 shifts the process to S206.

  Step S204: The template generation unit 25 calculates the dispersion value of the gradation of the through image in the candidate area for each channel of YCbCr.

  Step S205: The template generation unit 25 determines whether or not the variance values of the respective channels in the candidate area (each obtained in S204) exceed the threshold value. If the above requirement is satisfied (YES side), the process proceeds to S208. On the other hand, when the above requirement is not satisfied (NO side), the image in the current candidate region has a substantially uniform pattern, and it is highly likely that the feature of the subject to be tracked cannot be detected. Therefore, the CPU 21 shifts the process to S206.

  Step S206: The template generation unit 25 refers to the second memory 22 and determines whether or not the number of times the range of the candidate area has been expanded is equal to or less than a threshold (for example, twice or less). If the number of times of expansion is less than or equal to the threshold (YES side), the process proceeds to S207. On the other hand, if the number of enlargements exceeds the threshold (NO side), the CPU 21 terminates the process of enlarging the candidate area and proceeds to S208.

  Step S207: The template generation unit 25 expands the range of the currently set candidate area. Then, the template generation unit 25 increments the number of expansions of the candidate area range stored in the second memory 22. Thereafter, the CPU 21 returns to S202 and repeats the above operation. Therefore, when there is a high possibility that the characteristics of the subject to be tracked cannot be detected in the candidate area, the template is generated based on the candidate area wider than the previous one by the above loop.

  Step S208: The template generation unit 25 binarizes the gradation of the through image in the candidate area with a median value for each channel of YCbCr. Thereby, the template generation unit 25 generates a binarized image of the candidate area for each channel of YCbCr.

  Step S209: The template generation unit 25 obtains the degree of concentration of the high frequency component at the center of the image for each binary image of each channel of YCbCr.

  Hereinafter, the contents of the arithmetic processing in S209 will be described in detail. Here, the contents of the arithmetic processing for each binary image of YCbCr are common. Therefore, for the sake of simplicity, in the following example, only an example of a binarized image of the Y channel will be described, and description of an example of the binarized image of the CbCr channel will be omitted.

  (1) First, the template generation unit 25 obtains a score for each pixel by multiplying each pixel of the binary image of the Y channel by the pixel value in the binarized image and the distance from the image center. Here, when the Y gradation value is equal to or higher than the median value in the image before binarization, the pixel value in the binarized image is “1”. Further, when the gradation value of Y is less than the median value in the image before binarization, the pixel value in the binarized image is “0”. Further, the value of the distance from the center of the image is assumed to be “1” as the center pixel of the candidate area, and the value increases by 1 as the distance from the center pixel increases by one pixel.

  Therefore, the pixel score “0” in the binarized image has a pixel score of 0. Further, the pixel score “1” in the binarized image has a higher pixel score as the distance from the center pixel of the candidate area increases. For example, when the pixel value of the central pixel is “1”, the pixel score is “1”. On the other hand, when the pixel value of the pixel 5 pixels away from the center pixel is “1”, the score of the pixel is “5”.

(2) Next, the template generation unit 25 obtains a total value (S _Y ) by accumulating the pixel scores obtained in (1) above. Here, if many pixels having a pixel value “1” are distributed around the periphery of the candidate area in the binary image of the Y channel, the total value S _Y is a large value. On the other hand, if the pixels with the pixel value “1” are distributed in the center of the candidate area in the binary image of the Y channel, the total value is obtained even if the number of pixels with the pixel value “1” is the same as in the above case. S _Y is a small value. Therefore, the template generation unit 25 can evaluate the degree of concentration of the high frequency component at the image center of the binary image of the Y channel based on the small value of the total value S _Y.

Step S210: The template generation unit 25 determines a template generation channel (any one of YCbCr) based on the total value (S _Y , S _Cr , S _Cb ) of the YCbCr channels obtained in S209. Specifically, the template generation unit 25 obtains the minimum value from the total values S _Y , S _Cr , and S _Cb . Then, the template generation unit 25 determines a channel corresponding to the binarized image whose total value is the minimum value as a template generation channel.

Here, when there are two or more minimum values among the total values S _Y , S _Cr , and S _Cb , the template generation unit 25 refers to the tracking history data in the second memory 22. Then, the template generation unit 25 determines a template generation channel according to at least one of the following (1) and (2).

  (1) The template generation unit 25 refers to the first tracking history table and prioritizes the template generation channel when the tracking of the subject is successful. Specifically, the template generation unit 25 refers to the combination of AE information and subject distance information in the tracking history of the first tracking history table in descending order of priority (in order of H1 to H5). Then, the template generation unit 25 sequentially determines whether or not the tracking history in the first tracking history table corresponds to the current shooting condition based on the AE calculation result and the subject distance obtained from the through image. When there is a tracking history corresponding to the current shooting condition, the template generation unit 25 sets the channel type indicated by the tracking history as a template generation channel. As a result, a template generation channel more suitable for subject tracking processing is selected based on the past subject tracking results.

  (2) The template generation unit 25 refers to the second tracking history table and excludes a template generation channel when tracking of the subject has failed. Specifically, the template generation unit 25 refers to the combination of the AE information and the subject distance information in the tracking history of the second tracking history table in ascending order of priority (in the order of L1 to L5). Then, the template generation unit 25 sequentially determines whether or not the tracking history in the second tracking history table corresponds to the current shooting condition based on the AE calculation result and the subject distance obtained from the through image. When there is a tracking history corresponding to the current shooting condition, the template generation unit 25 determines a template generation channel by excluding the channel type indicated by the tracking history. As a result, a template generation channel more suitable for subject tracking processing is selected based on the past subject tracking results.

  Step S211: The template generation unit 25 obtains the barycentric position of the high-frequency component cluster (pixel value “1” portion) by using a known barycentric calculation algorithm for the binarized image of the template generation channel (S210). . When the binarized image of the template generation channel includes a plurality of high-frequency component clusters, the template generation unit 25 determines the center of gravity position of the largest cluster.

  Step S212: The template generation unit 25 determines the provisional size of the feature region indicating the characteristics of the subject to be tracked with reference to the position of the center of gravity obtained in S211.

  As an example, in the binarized image of the template generation channel, the template generation unit 25 confirms the pixel value for each line in the vertical direction or the horizontal direction starting from the position of the center of gravity. At this time, the template generation unit 25 sets the line where the number of pixels with the pixel value “1” is equal to or less than the threshold as the end of the feature region. And the template production | generation part 25 determines the temporary size of a characteristic area | region according to the magnitude | size of the cluster of a high region part by performing the said process in four directions of up and down, right and left, respectively. In this case, the provisional size of the feature area is a range surrounded by the above-described line (see FIG. 7).

  Step S213: The template generation unit 25 determines whether or not the temporary size of the feature area (determined in S212) is smaller than a predetermined default size. If the above requirement is satisfied (YES side), the process proceeds to S214. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S215.

  Step S214: The template generation unit 25 replaces the temporary size of the feature area with a preset default size and enlarges it. When the template generation channel is determined with reference to the tracking history in the first tracking history table in S210, the template generation unit 25 sets the template size indicated by the tracking history as the default size and the feature area determined in S212. It may be replaced with a temporary size.

  Step S215: The template generation unit 25 corrects the range of the feature region based on the subject distance on the shooting screen.

  Specifically, the template generation unit 25 generates a distance image indicating the distribution of the subject distance in the candidate area based on the output of the ranging pixels of the image sensor 13. Next, the template generation unit 25 superimposes the above-described feature area and the distance image, and determines whether or not there is a portion in which a difference equal to or greater than a threshold value occurs in the subject distance within the feature area.

  When a difference equal to or greater than the threshold value is generated in the subject distance within the feature region, the template generation unit 25 determines a final feature region range by excluding a portion where the subject distance difference is equal to or greater than the threshold value. As a result, a portion that is likely to be different from that of the subject to be tracked is excluded from the feature region, so that the accuracy of subject tracking can be further improved.

  Step S216: The template generation unit 25 generates template data from the range of the feature region determined in S215 in the through image. In the present embodiment, it is assumed that the template generation unit 25 generates template data based on the through image of the entire range of the feature region. But the template production | generation part 25 may produce | generate the data of a template based on a part of feature area. For example, the template generation unit 25 may set a rectangle inscribed in the feature region when the outer periphery of the feature region is nonlinear, and may generate template data from a through image in the range of the rectangle (in this case) Is omitted).

  Note that the template generation unit 25 in S216 generates template data using not only the template generation channel but all YCbCr channels in order to improve subject tracking accuracy. Here, consider a case where a template is generated using only the Y channel and a case where a template is generated using all channels of YCbCr. If subject tracking is performed in a scene in which a subject having the same luminance as the tracking target is included in the shooting screen, the possibility that the subject tracking unit 26 erroneously recognizes the tracking target in the former increases. On the other hand, in the latter case, using the Cb channel or Cr channel data increases the possibility that the subject tracking unit 26 can correctly track the subject to be tracked. This is the end of the description of the subroutine of FIG.

  Hereinafter, the operational effects of this embodiment will be described. The electronic camera according to the present embodiment specifies the position and size of the feature region indicating the characteristics of the subject to be tracked from the inside of the through image candidate region when generating the initial template in the subject tracking process. And an electronic camera produces | generates a template using the data of the through image of said characteristic area. As a result, a template suitable for subject tracking processing is automatically generated by the electronic camera, so that the accuracy of subject tracking can be further improved.

<Supplementary items of the embodiment>
(1) In the electronic camera of the above-described embodiment, a phase difference AF module may be provided separately from the imaging device 13 and a phase difference AF operation by a known pupil division may be performed in addition to a contrast type AF operation. In the above-described embodiment, an example in which AE calculation is performed based on the output of the image sensor 13 has been described. However, a photometric element for AE calculation may be provided separately from the image sensor 13.

  (2) In the above embodiment, template data and tracking history data may be held in the first memory 16. In the above-described embodiment, the CPU 21 may hold tracking history data in the electronic camera only while the power is on, and initialize the tracking history data in accordance with a shooting mode switch or a power-off operation. .

  (3) In the above embodiment, an example has been described in which functions such as the template generation unit 25 and the subject tracking unit 26 are realized by software using a program. However, these configurations are realized by hardware using an ASIC. Of course it doesn't matter.

  (4) In the above embodiment, the example in which the CPU 21 of the electronic camera generates a template using the through image has been described. However, the present invention is not limited to the above-described embodiment, and, for example, an electronic camera may generate a template using a recorded image (main image or the like).

  Further, template data to be applied when subject tracking with an electronic camera may be generated from a recorded image by an external device (such as a personal computer) connected to the electronic camera. At this time, the template generation process shown in the above embodiment may be executed by a program on an external device. Furthermore, the algorithm of the above embodiment may be realized by causing the external device to share a part of the processing executed by the CPU 21 of the electronic camera in the above embodiment and causing the electronic camera and the external device to cooperate.

  (5) In the above embodiment, the example in which the template generation channel is selected from the YCbCr channels has been described. However, when applying RGB image data in the present invention, a template generation channel may be selected from among the RGB channels.

  (6) The template generation unit 25 in the above embodiment determines whether or not to enlarge the candidate area based on the luminance value and the variance value (S203, S205). However, the configuration of the present invention is not limited to the example of the above embodiment. For example, the template generation unit 25 generates a contour image (binarized image indicating the presence / absence of an edge component) of the candidate region, and adds the pixel values in the vertical and horizontal directions. When the sum value obtained from the contour image is less than the threshold value, the template generation unit 25 may enlarge the candidate region on the assumption that the image in the current candidate region has a substantially uniform pattern. Of course, the template generation unit 25 may execute a combination of determination of candidate area expansion based on the luminance value or variance value and candidate area expansion determination based on the contour image as appropriate.

  (7) In S203 of the above embodiment, the template generation unit 25 may further determine whether or not the average value of the gradation of the Cb and Cr channels exceeds the threshold value. In S205 of the above embodiment, the template generation unit 25 determines that the feature of the subject can be detected if any of the YCbCr variance values of the through image in the candidate region exceeds the threshold value, and determines on the YES side. You may do.

  (8) The template generation unit 25 in the above embodiment may specify the candidate area after being enlarged in the process of S206 as a feature area as it is, and generate template data. Further, the template generation unit 25 in the above embodiment, when the average value of the Y channel gradation is less than the threshold value (NO side of S203), or when one of the variance values of each channel is less than the threshold value (S205). On the NO side), the template generation process may be terminated and an error display may be displayed on the monitor 18.

  (9) The template generation unit 25 in the above embodiment, when the subject distance in the candidate area obtained by the ranging pixel is out of the predetermined range (when the subject distance is extremely short or extremely long), The template generation process may be terminated and an error display may be displayed on the monitor 18.

  (10) The shape of the feature region and the shape of the template in the above embodiment are not limited to a square or a rectangle, and may be, for example, a shape surrounded by a free curve.

  (11) In S212 of the above embodiment, the template generation unit 25 obtains the distances from the center of gravity to the ends of the top, bottom, left, and right feature regions, and applies the maximum value of the obtained distances to the feature region. The provisional size may be determined. In this case, the temporary size shape of the feature region is a square. In addition, the processing order of S215 in the above embodiment may be appropriately rearranged as long as it is after the processing of S202.

  (12) The configurations of the above-described embodiments and modifications are merely examples. Therefore, when implementing the imaging device, the program, and the template generation method of the present invention, all combinations of the configurations shown in the above-described embodiments and modifications and the order of the processes can be taken.

  It should be noted that the present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

A block diagram for explaining a configuration of an electronic camera of the present embodiment (A) The figure which shows the 1st tracking history table, (b) The figure which shows the 2nd tracking history table Flow chart explaining an operation example in the shooting mode of the electronic camera The figure which shows the example of a display of the view image on which the guide mark was superimposed FIG. 3 is a flowchart for explaining a subroutine of the template generation process (S103). The figure which shows an example of the positional relationship of a through image and a candidate area | region Explanatory drawing which shows the determination method of the temporary size of a template

Explanation of symbols

DESCRIPTION OF SYMBOLS 13 ... Image sensor, 15 ... Image processing part, 21 ... CPU, 22 ... 2nd memory, 25 ... Template production | generation part, 26 ... Subject tracking part

Claims (12)

An imaging unit that images a subject and generates image data;
A template generation unit that identifies a feature region indicating a feature of a subject to be tracked from inside a candidate region set in the image, and generates a template using image data of the feature region;
A subject tracking unit that continuously detects the subject to be tracked from an input image using the template;
An imaging apparatus comprising: The imaging device according to claim 1,
The said template production | generation part specifies the position of the said characteristic area within the said candidate area | region, The imaging device characterized by the above-mentioned. In the imaging device according to claim 1 or 2,
The template generation unit specifies the size of the feature region. The imaging device according to claim 2,
The template generating unit performs an extraction process of extracting a main subject in the candidate region, and specifies the position of the feature region based on the position of the center of gravity of the main subject. The imaging device according to claim 3.
The template generation unit performs an extraction process of extracting a main subject in the candidate region, and specifies the size of the feature region based on the size of the main subject. In the imaging device according to claim 2 or 3,
A memory for storing tracking history data indicating a correspondence relationship between a result of subject tracking by the subject tracking unit and a generation condition of a template applied in the subject tracking;
The template generation unit identifies the feature region with reference to the generation condition extracted from the tracking history data in accordance with the subject tracking result. In the imaging device according to claim 2 or 3,
A distance measuring unit for detecting subject distances at a plurality of locations on the shooting screen corresponding to the image;
The imaging apparatus, wherein the template generation unit specifies the feature region with reference to a subject distance in the candidate region. The imaging device according to claim 1,
A determination processing unit that determines whether or not a portion indicating the characteristics of the subject can be detected from the inside of the candidate region;
A candidate area changing unit for enlarging the size of the candidate area when the determination processing unit determines that the portion indicating the feature cannot be detected;
An image pickup apparatus further comprising: The imaging device according to claim 8,
The said template production | generation part specifies the said feature area from the inside of the said candidate area | region after being expanded, The imaging device characterized by the above-mentioned. The imaging device according to claim 8,
The said template production | generation part specifies the said candidate area | region after being expanded as said characteristic area, The imaging device characterized by the above-mentioned. For a computer having a data reading unit for reading image data and a calculation unit and generating a template for continuously detecting a subject to be tracked from an input image,
An image input step of reading the image data into the calculation unit by the data reading unit;
A template generation step of identifying a feature region indicating the characteristics of a subject to be tracked from inside a candidate region set in the image, and generating the template using image data of the feature region;
Is executed by the calculation unit. A template generation method for generating a template for continuously detecting an object to be tracked from an input image using at least one computer,
An image input step of reading image data into the computer;
A feature region specifying step in which the computer specifies a feature region indicating a feature of a subject to be tracked from inside a candidate region set in the image;
A template generating step in which the computer generates the template using image data of the feature region;
A template generation method comprising:

Images