JP2014202875A - Subject tracking device - Google Patents

Abstract

An imaging apparatus is provided that tracks a main subject and an object in front of the main subject and selects a distance measuring point where only the main subject exists at the time of focusing. A reference image generating unit that generates a reference image including at least a part of a tracking target, and a tracking target position in the captured image by evaluating a degree of correlation between the captured image from the previous tracking sensor and the reference image. Tracking means for detecting the tracking object, and the tracking means individually tracking the subject to be focused at the time of focus adjustment as a main subject and the subject positioned in front of the main subject as viewed from the imaging device as a shielding subject, The focus adjusting means is configured to perform focus adjustment at a distance measuring point that exists on the main subject and does not exist on the shielding subject. [Selection] Figure 1 (a)

Description

  The present invention relates to an imaging device, and more particularly to a subject tracking device mounted on the imaging device.

  2. Description of the Related Art Conventionally, in a camera equipped with a solid-state image sensor, a function of selecting an arbitrary subject from continuously captured images and keeping track of the subject between successive frames has been put into practical use. In such a camera, by detecting the position of the main subject in the image, it is possible to continue appropriate focus adjustment and automatic exposure for a subject that changes with time.

  In addition, in a camera having the above-described subject tracking function, a function for simultaneously tracking a plurality of subjects instead of a single subject in order to recognize a plurality of subjects individually or distinguish a main subject from a subject that is not. Some of them have

  Japanese Patent Application Laid-Open No. 2004-151561 discloses a technique for recognizing a plurality of subjects existing in an image and tracking them simultaneously to show the photographer the position of the subject on the image.

  The digital camera described in Patent Document 1 has a function of displaying a tracking frame surrounding a subject detected from an input image on a screen, and one or more specified by the photographer via a touch screen LCD or the like. The subject can be set as a tracking target. And, it has a function of performing processing such as white balance adjustment and gamma correction that are optimal for the tracking target for an image captured after the tracking target is specified.

JP 2010-183253 A

  However, in the conventional technique disclosed in Patent Document 1 described above, since all the tracking targets designated by the photographer are tracked, if there are a large number of subjects in the scene and all subjects are to be tracked, the calculation cost is low. There is a problem that becomes larger. Also, depending on how to use subject tracking, it may not be necessary to track all subjects.

  For example, consider a case where the position information of a subject detected by tracking processing is used for distance measurement point selection of a single-lens reflex camera having a phase difference AF method. In such a case, after identifying the position of the main subject, select a distance measurement point that excludes the distance measurement points affected by other subjects, such as distance conflict, from the distance measurement points that exist on the main subject. By doing so, it is possible to focus on the main subject. Perspective conflicts often occur when another subject cuts in between the main subject and the imaging device, so only the main subject and only the subject that can be interrupted between the imaging device rather than the main subject are tracked. By doing so, it is possible to exclude ranging points affected by the perspective conflict from being focused.

  Therefore, the object of the present invention is to track the main subject and an object in front of the main subject instead of all the subjects present in the image as the tracking target, and select a distance measuring point where only the main subject exists at the time of focusing. An imaging device is provided.

  Hereinafter, in this proposal, a subject that may interrupt between the main subject and the imaging device is referred to as a shielding subject.

To achieve the above object, the present invention provides a tracking sensor that captures an image from a finder optical system, a reference image generating unit that generates a reference image including at least a part of a tracking target,
A tracking means for detecting a tracking target position in the captured image by evaluating a degree of correlation between the captured image from the previous tracking sensor and the reference image;
It is possible to detect the focus state of a plurality of ranging points in the imaging screen, select a ranging point corresponding to the tracking target position from the plurality of ranging points based on the result of the tracking means, and automatically focus In an imaging apparatus having a focus adjusting means for adjusting,
The tracking unit individually tracks each of the subject to be focused on by the focus adjusting unit as a main subject and a subject positioned in front of the main subject as viewed from the imaging apparatus as a shielding subject,
The focus adjusting means adjusts the focus at a distance measuring point that exists on the main subject and does not exist on the shielding subject.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which excludes the focus detection area | region which has a high possibility of having a bad influence in AF from the object of focusing can be provided, reducing the load of tracking processing.

It is a figure which shows the operation | movement procedure of the imaging device in connection with 1st Embodiment. It is a figure which shows the operation | movement procedure of the imaging device in connection with 1st Embodiment. It is a mechanical drawing of the imaging device concerning 1st Embodiment. It is a block diagram which shows the basic composition of the imaging device concerning 1st Embodiment. It is a figure which shows the imaging range of the sensor mounted in the imaging device concerning 1st Embodiment. It is a figure which shows the example of the shielding subject detection of the imaging device concerning 1st Embodiment. It is a figure which shows the example of the ranging point selection of the imaging device concerning 1st Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is an operation procedure of the imaging apparatus according to the embodiment of the present invention.

[Example 1]
Hereinafter, with reference to FIG. 2, an imaging apparatus according to a first embodiment of the present invention will be described.

  In FIG. 2, reference numeral 201 denotes a digital single-lens reflex camera body, and a photographing lens 202 is attached to the front surface thereof. The photographing lens 202 can be exchanged, and the camera body 201 and the photographing lens 202 are also electrically connected via the mount contact group 215. The taking lens 202 includes a focusing lens 213 and an aperture shutter 214, and the amount of light taken into the camera and the focus can be adjusted by control via the mount contact group.

  Reference numeral 203 denotes a main mirror, which is a half mirror. The main mirror 203 is obliquely arranged on the photographing optical path in the finder observation state, and reflects the photographing light beam from the photographing lens 202 to the finder optical system. On the other hand, the transmitted light enters the distance measuring sensor 205 via the sub mirror 204.

  The distance measuring sensor 205 forms an image signal that can detect the focus adjustment state of the photographing lens 202 by a phase difference detection method by forming the secondary image formation surface of the photographing lens 202 on the focus detection line sensor. .

  Reference numeral 206 denotes a focusing plate disposed on the planned imaging plane of the photographing lens 202 constituting the finder optical system.

  Reference numeral 207 denotes a pentaprism for changing the finder optical path. Reference numeral 209 denotes an eyepiece, and the photographer can confirm the photographing screen and the distance measuring point mark by observing the focus plate 206 from here.

  Reference numeral 208 denotes a photometric sensor. The photometric sensor 208 generates image data having a luminance signal and a color difference signal from the irradiated light signal. Photometric image data generated by the photometric sensor 208 is used for processing of subject brightness detection, exposure calculation, and subject tracking.

  Reference numeral 210 denotes a focal plane shutter, and 211 denotes an image sensor. When performing exposure, the main mirror 203 and the sub mirror 204 are retracted from the photographing light beam, and the focal plane shutter 210 is opened, so that the image sensor 211 is exposed. Reference numeral 212 denotes a display unit that displays photographing information and a photographed image so that the user can check it.

  Next, functional blocks of the imaging apparatus according to the present embodiment will be described with reference to the schematic diagram of FIG. 2 and FIGS. 3 and 4. In FIG. 3, the same members as those in FIG. 2 are denoted by the same reference numerals as those in FIG.

  The operation unit 301 detects an operation performed by a user via a release button, a switch, a dial, a connected device, and the like (not shown) attached to the camera body unit 201, and sends a signal corresponding to the operation content to the system control unit 304. . The operation unit 301 outputs the SW1 signal to the system control unit 304 at the moment when the release button is pressed halfway, and the SW2 signal at the moment when the release button is pressed. Hereinafter, when the user presses the release button halfway is expressed as “pressing the SW1 button”, and maintaining the release button halfway pressed is expressed as SW1 state. In addition, pressing the release button is expressed as “pressing the SW2 button”, and a state where the release button is pressed is expressed as SW2 state. In addition, the operation unit 301 outputs the SW1 release signal to the system control unit 304 at the moment when the user releases the release button in the SW1 state, and the SW2 release signal at the moment when the user releases the release button in the SW2 state.

  The photometric sensor 302 generates image data for performing automatic exposure calculation and subject tracking from a light signal incident on the sensor, and outputs the image data to the system control unit 304. The system control unit 304 performs automatic exposure calculation based on the image data input from the photometric sensor 302, controls the aperture of the aperture shutter 214 based on the calculation result, and adjusts the amount of light incident on the camera body. . Furthermore, the shutter 210 is controlled at the time of release, and the exposure time of the image sensor 211 is adjusted.

  In the SW1 state and during continuous shooting, the system control unit 304 performs subject tracking using the image data input from the photometric sensor 302 to detect the position of the subject.

  The distance measuring sensor 303 includes a line sensor corresponding to the position of a distance measuring point mark 403 in FIG. 4 to be described later, generates an image signal from an optical signal incident through the sub-mirror 204, and outputs the image signal to the system control unit 304. To do. The system control unit 304 calculates a defocus value at each distance measurement point from the image signal input from the distance sensor 303 based on a phase difference detection method that is a known technique, and performs focus detection.

  Then, the system control unit 304 performs focus adjustment based on the focus detection result of either the distance measurement point near the subject position detected in the subject tracking process or the distance measurement point selected by the user's manual operation, The focusing lens 213 is controlled through the mount contact point group 215.

  The system control unit 304 controls the mirrors 203 and 204 and the shutter 210 based on a signal input from the operation unit 301. If the signal output from the operation unit 301 is the SW2 signal, the mirrors 203 and 204 are raised and the shutter 210 is controlled to irradiate the image sensor 211 with light. When the shutter control is finished, the mirrors 203 and 204 are lowered.

  The image sensor 211 generates image data from an optical signal incident through the photographing lens 202 and outputs the image data to the system control unit 304. The system control unit 304 displays the image data input from the imaging sensor 211 on the display 212 and stores the image data in the image storage device 305.

  FIG. 4 shows the shooting ranges of the image sensor 211 and the photometric sensor 302 and the arrangement of the distance measuring point marks of the distance measuring sensor 303 in this embodiment. The imaging range of the imaging sensor 211 is indicated by the imaging sensor range in FIG. 4, the imaging range of the photometric sensor 302 is indicated by the photometric sensor range 402, and the distance measuring point position of the distance measuring sensor 303 is indicated by the distance measuring point mark 403.

  When the main mirror 203 is in the down state, the subject image is projected onto the focus plate 206, so that the user can view the subject image being tracked by looking into the eyepiece 209 of FIG. Also, among the distance measuring point marks 403 in FIG. 4, the distance measuring point for which focus adjustment is being performed at that time is displayed superimposed on the focus plate 206, so that the user is at the currently tracked subject position. A distance measuring point can be confirmed.

  Subsequently, an operation procedure of the imaging apparatus and an object tracking algorithm in the present embodiment will be described with reference to FIGS. 1, 5, and 6.

  FIGS. 1A and 1B show the operation sequence of the camera after SW1 is pressed.

  In S101, the system control unit 304 sets the reference image generation flag to FALSE. This flag is set to TRUE if the reference image of the main subject has been generated even once after the start of tracking.

  In step S <b> 102, the photometric sensor 302 and the distance measuring sensor 303 accumulate incident optical signals and output image signals to the system control unit 304. Hereinafter, an image output from the photometric sensor 302 to the system control unit 304 is referred to as an input image.

  In S103, the system control unit 304 refers to the value of the reference image generation flag. If it is TRUE, it means that the reference image for tracking the main subject has been generated, and the process proceeds to the subject tracking process in S109. . If it is FALSE, since the reference image of the main subject has not been generated, the process shifts to the defocus calculation and distance measuring point selection process of S104, not the subject tracking process.

  S104 is processing executed when it is determined in S103 that the reference image of the main subject does not exist. The system control unit 304 performs all processing based on the image signal read from the distance measuring sensor 303 in S102. The defocus value of the distance measuring point is calculated. Then, a distance measuring point previously selected by the user by manual operation is set as the selected distance measuring point. The defocus value calculated here is used in the processing of S107 and S117 described later.

  In S105, the system control unit 304 sets a subject existing at the same position as the distance measuring point selected in S104 in the input image as a main subject. Then, a region having a certain size centered on the position of the distance measuring point selected in S104 is cut out from the input image, and this is used as the reference image of the main subject. In subsequent frames, the main subject is detected from the input image using the reference image of the main subject as a tracking target.

  In S <b> 106, the system control unit 304 detects the occluded subject from the input image by executing the processing of S <b> 116 to S <b> 121 based on the defocus value of each ranging point. Hereinafter, a specific method for detecting the shielding subject will be described with reference to FIG. FIG. 5 shows a scene in which three subjects 501, 502, and 503 exist within the distance measuring point range. FIG. 5A shows a state where the distance measuring point selected in S104 is the selected distance measuring point 504, and the main subject is the object 501 in S105.

  In S <b> 116, the system control unit 304 clears the information of the shielding subject currently set as the tracking target and excludes it from the tracking target.

  In S117, the system control unit 304 extracts and stores a distance measuring point having a defocus value in front of the main subject set in S105 as the defocus value of each distance measuring point.

  In S118, if at least one distance measuring point having a defocus value before the main subject is extracted in S117, the system control unit 304 proceeds to the process in S119. On the other hand, when none is extracted, the shielding subject detection process is terminated.

  In S119, the system control unit 304 groups the distance measurement points extracted in S117, the distance measurement points adjacent to each other vertically and horizontally and having a defocus value difference equal to or smaller than a predetermined value. Create

  FIG. 5B is a diagram in which distance measuring points having a defocus value in front of the main subject are grouped with respect to FIG. In FIG. 5B, distance measuring points located at the same position as the subject 502 located in front of the subject 501 are extracted and grouped, and a distance measuring point group 505 is obtained. On the other hand, a distance measuring point on the subject 503 existing behind the subject 501 as viewed from the imaging apparatus is not a grouping target.

  In S120, the system control unit 304 detects a subject existing at the same position as each distance measurement point group created in S119 in the input image as a shielding subject, and sets it as a tracking target in the next frame.

  In S121, the system control unit 304 generates a reference image for tracking processing for all the occluded subjects set in S120. As a specific process, an area of a certain size centered on the position of the center of gravity on the input image of the ranging point belonging to the ranging point group corresponding to each shielding subject is cut out, and this is used as a reference for the shielding subject. An image.

  This is shown in FIG. In FIG. 5C, in S105, an image having a certain size centered on the selected distance measuring point is used as the reference image 601 of the subject 501 as the main subject. In S 121, the reference image 602 of the subject 502 that is a shielding subject is an image having a certain size centered on the position of the center of gravity of the distance measuring point group 505 corresponding to the subject 502.

  In subsequent frames, the shielded subject is detected from the input image using the reference image of the shielded subject as a tracking target.

  The above is the description of the shielding subject detection process. By the processing from S116 to S121, it is possible to detect a subject existing in front of the main subject as viewed from the imaging device as a shielding subject and set it as a tracking target. In addition, since the shielded subject is initialized in S116, the subject that has moved deeper than the main subject according to the change in time can be excluded from the tracking target.

  In S107, the system control unit 304 generates the reference image of the main subject in S105, and therefore sets the reference image generation flag to TRUE.

  In S <b> 108, the system control unit 304 performs template matching processing on the input image using the reference image of the main subject and the shielded subject, and calculates the degree of correlation with each area of the input image. Then, the region having the highest correlation is set as the detection position of the main subject and the shielding subject.

  Here, for template matching, the following matching processing is performed. First, an arbitrary area of the input image is cut out so as to have the same size as the reference image to be tracked, and is set as the tracking input image.

  Then, the sum of the difference values of the luminance signal and the color difference signal is calculated for all corresponding pixels of the tracking input image and the reference image to be tracked, and the result is used as a matching evaluation value. The smaller the matching evaluation value, the higher the degree of correlation.

  Then, by calculating the matching evaluation value for the entire image while shifting the position of the area to be cut out from the input image by a predetermined value, the area of the tracking input image having the highest degree of correlation is set as the detection position of the tracking target. This is performed on the reference image of the main subject and all the shielding subjects, thereby detecting the positions of the main subject and each shielding subject in the input image.

  In S109, the system control unit 304 calculates the defocus value of each distance measurement point based on the image signal read from the distance measurement sensor 303 in S102, and selects the distance measurement point to be focused. Here, a method of selecting a distance measuring point will be described with reference to FIG. FIG. 6A shows the result of tracking the subjects 501 and 502 in the tracking process of S108 using the reference images 601 and 602 of FIG. The tracking result 701 is the result of tracking the subject 501 that is the main subject, and the tracking result 702 is the result of tracking the subject 502 that is the shielding subject.

  For such a result, first, a distance measuring point closest to the tracking result of each shielding object is selected one by one. In FIG. 6A, since the shielding subject is only the subject 502, the distance measuring point 703 closest to the tracking result 702 is selected. Subsequently, as shown in FIG. 6B, all the distance measuring points that are adjacent in the vertical and horizontal directions and have a defocus value difference equal to or smaller than a predetermined value are grouped starting from the selected distance measuring point. Thus, the shielding distance measuring point group 704 is created.

 Finally, a distance measuring point closest to the tracking result 701 of the subject 501 is selected as a selected distance measuring point 705 from the distance measuring points excluding the shielded distance measuring point group 704. As a result, the distance measuring point affected by the shielding subject can be excluded from the selected distance measuring point, so that the main subject is kept in focus even when the shielding subject hides a part of the main subject. be able to.

  In S110, the system control unit 304 performs the same occluded subject detection process as in S106, detects an object existing in front of the main subject, and sets it as a tracking target in the next frame.

  By detecting the shield subject at this timing, the shield subject to be tracked in the next frame can be detected based on the defocus calculation result of S109. Further, if there is a shielded subject that has moved to the back of the main subject as seen from the imaging device, it can be excluded from the tracking target.

  In S111, if the degree of correlation between the reference image of the main subject calculated in S108 and the input image is equal to or greater than a predetermined value, the system control unit 304 is an image of the area in which the main subject is detected in the input image. Update the reference image.

  In S112, the system control unit 304 performs automatic exposure calculation based on the input image. In S113, the system control unit 304 performs focus adjustment based on the focus detection result of the distance measuring point selected in S104 or S109. Further, exposure control is performed based on the automatic exposure calculation result of S116.

  In S114, the system control unit 304 determines whether the SW2 signal is received from the operation unit 301. If the SW2 signal is received, the system control unit 304 proceeds to the imaging process in S115. If it has not been received, imaging is not performed and the process proceeds to S102.

  In step S <b> 115, the system control unit 304 performs shooting by raising the mirrors 203 and 204 and exposing the image sensor 211. After shooting, the mirrors 203 and 204 are lowered, and the process proceeds to S102.

  The above processing is repeated while the SW1 state or SW2 state continues. The above is the operation procedure of the imaging apparatus in the present embodiment. By such an operation, it is possible to keep focusing on the main subject while reducing the computation load by tracking only the main subject and a shielding subject that is likely to have a bad influence on the focus on the main subject. it can.

  The present invention relates to an imaging apparatus, and in particular, may be applicable to a subject tracking apparatus mounted on the imaging apparatus.

201 Digital SLR Camera Main Body 202 Lens 203 Main Mirror 204 Sub Mirror 205 Distance Sensor 206 Focusing Plate 207 Penta Prism 208 Photometric Sensor 209 Eyepiece 210 Focal Plane Shutter 211 Image Sensor 212 Display Unit 213 Focusing Lens 214 Aperture Shutter 215 Mount Contact Group 301 Operation Unit 302 Photometric sensor 303 Distance sensor 304 System control unit 305 Image storage device

Claims (2)

A tracking sensor (208) for capturing an image from the viewfinder optical system;
Reference image generation means (S105) for generating a reference image including at least a part of the tracking target;
Tracking means (S108) for detecting the tracking target position in the captured image by evaluating the degree of correlation between the captured image from the previous tracking sensor and the reference image;
It is possible to detect the focus state of a plurality of ranging points in the imaging screen, select a ranging point corresponding to the tracking target position from the plurality of ranging points based on the result of the tracking means, and automatically focus In the imaging apparatus having the focus adjusting means (S117) for adjusting,
The tracking unit individually tracks the subject to be focused by the focus adjusting unit as a main subject and the subject positioned in front of the main subject as viewed from the imaging apparatus as a shielding subject (S106 to S108, S110). To S114),
The imaging apparatus according to claim 1, wherein the focus adjustment unit performs focus adjustment at a distance measuring point that exists on the main subject and does not exist on the shielding subject. 2. The tracking unit according to claim 1, wherein the tracking unit excludes the shielded subject from being tracked when the shielded subject moves far behind the main subject as viewed from the imaging device (S <b> 110, S <b> 113). Imaging device.