JP2009182624A - Target tracking device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a target tracking device performing a tracking process with high accuracy even if there are a plurality of objects in a color similar to that of a tracking object. <P>SOLUTION: The target tracking device 100 for tracking a target object in image data which is consecutively taken in includes: a tracking target specification accepting part 150 for accepting the specification of the tracking target in the image data; a target color setting part 160 for setting the color in the image data related to the specification as a target color; and a particle filter processing part 130 which measures the likelihood found by comparing the color in the vicinity of a particle to the target color using the particle moving in the image data according to a predetermined rule, compensates the measured likelihood for each particle by the likelihood found by comparing the color in a region determined based on the specified auxiliary region to an auxiliary target color, and estimates the region of the target object in the image data, based on the compensated likelihood. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a target tracking device that tracks a target object in continuously captured image data.

  As represented by video surveillance systems and video conference systems, camera systems that use images taken by cameras in various situations have been put into practical use. Some of such camera systems have a tracking function for automatically tracking a set target object and shooting while changing the shooting area. For example, in a video surveillance system having a tracking function, when a suspicious person is set as a target object, it is possible to continue shooting while tracking the person and storing it in the video. In addition, in a video conference system having a tracking function, it is possible to capture a conference image that tracks a set person.

  When shooting an image while tracking the target object, it is necessary to control the pan, tilt, zoom magnification, etc. of the camera according to the movement of the target object so that the target object does not deviate from the shooting angle of view. In order to realize this, it is necessary to recognize the target object in the image and detect its moving direction.

  Conventionally, as a method for recognizing a target object in an image and detecting its moving direction, a background difference method or a frame difference method using a luminance difference has been used. As described in Non-Patent Document 2, etc., a target tracking technique using a particle filter has been studied.

The particle filter is an approximate calculation method of a Bayes filter that uses a posteriori probability, and represents a probability distribution function by a finite number of particles and uses it to perform time-series prediction. In other words, the particle filter is a sequential Monte Carlo method based on sampling. Since the time series distribution is approximated by a collective expression based on the position and weight of the particle, it is possible to track even a distribution that cannot be approximated by a Gaussian distribution. In addition, since various time-series feature quantities can be treated as likelihood, the application range is wide. In particular, as described in Patent Document 1 and the like, when applied to tracking of a target object, the likelihood is measured using the color of the target object. In this case, the likelihood can be measured based on how many pixels of a color close to the color of the target object exist in the vicinity of the particle, and the position of the target object can be estimated based on the measurement result.
JP 2004-282535 A M.Isard, A.Blake: CONDENSATION? Conditional Density Propagation for Visual Tracking, Int 'IJ.Computer Vision, vol. 28, No.1, pp.5-28 (1998) Hiroyoshi Fujiyoshi: Video image comprehension technology and its application, Chubu University Faculty of Engineering Department of Information Engineering, Fujiyoshi Laboratory Text pp.76-80 (2007) http://www.vision.cs.chubu.ac.jp/VU/pdf/VU .pdf

  As described above, in the target object tracking process using the particle filter, the color of the target object is set, and the target object is tracked by increasing the likelihood of the particles arranged in the portion close to the color.

  However, if there are multiple objects with colors close to the color of the target object, both objects will be measured with a high likelihood of particles and the tracking accuracy of the target object will be reduced. Failure to track an object can occur.

  The present invention has been made in view of such a situation, and provides a target tracking device that can perform tracking processing with high accuracy even when there are a plurality of objects having colors close to the color of the tracking target. For the purpose.

  In order to solve the above-described problem, a target tracking device according to a first aspect of the present invention is a target tracking device that tracks a target object in image data that is captured in time series, and the target tracking device in the image data A tracking target designation receiving unit that receives a tracking target designation, an auxiliary region designation receiving unit that receives a designation of an auxiliary region associated with the tracking target in the image data, and a color in the image data relating to the specified tracking target Target color setting means for setting the target color as the target color, auxiliary target color setting means for setting the color of the designated auxiliary region as the auxiliary target color, and particles that move in the image data according to a predetermined rule , Measure the likelihood obtained by comparing the color around the particle and the set target color, the measured likelihood for each particle Correction is performed by the likelihood obtained by comparing the color in the region determined based on the determined auxiliary region and the auxiliary target color, and the tracking target region in the image data is estimated based on the corrected likelihood. Particle filter processing means.

  In this aspect, the color of the auxiliary area associated with the tracking target is set as the auxiliary target color, and the likelihood is measured using the auxiliary target color also in the auxiliary area. Then, by correcting the likelihood measured using the target color based on the likelihood, it is possible to perform a highly accurate tracking process even when there are a plurality of objects having a color close to the color of the tracking target object. become.

  In order to solve the above-described problem, a target tracking device according to a second aspect of the present invention is a target tracking device that tracks a target object in image data that is captured in time series, and the target tracking device includes: A tracking target designation receiving unit that receives a tracking target designation, an auxiliary region designation receiving unit that receives a designation of an auxiliary region associated with the tracking target in the image data, and a color in the image data relating to the specified tracking target Target color setting means for setting the target color as the target color, auxiliary target color setting means for setting the color of the designated auxiliary region as the auxiliary target color, and particles that move in the image data according to a predetermined rule The likelihood obtained by comparing the color around the particle and the set target color is measured, and the measured likelihood is distributed in the measured likelihood distribution. And a color in an area determined based on the designated auxiliary area and the degree of approximation of the auxiliary target color, and the tracking target area in the image data is estimated based on the corrected likelihood Particle filter processing means.

  In this aspect, the color of the auxiliary area associated with the tracking target is set as the auxiliary target color, and the degree of approximation with the auxiliary target color is calculated in the auxiliary area. Then, by correcting the likelihood measured using the target color based on this approximation, a highly accurate tracking process can be performed even when there are a plurality of objects having a color close to the color of the tracking target object. become.

  In order to solve the above-described problem, a target tracking device according to a third aspect of the present invention is a target tracking device that tracks a target object in image data that is captured in time series, and the target tracking device includes: A tracking target designation receiving unit that receives a tracking target designation, a target color setting unit that sets a color in the image data related to the specified tracking target as a target color, and a tracking target in the previously captured image data The image obtained later is measured based on the measured likelihood by measuring the likelihood obtained by comparing the color around the particle with the set target color using particles dispersed from the vicinity of the region assumed Particle filter processing means for estimating the tracking target area in the data, and the particle filter processing means A region that is the tail object, obtains a distance between the color of a region similar to the target color, characterized in that changing the movement amount in the dispersion of the particles in accordance with the distance obtained.

  In this aspect, in order to change the movement amount at the time of particle dispersion according to the distance between the target object region and the color region that approximates the color of the target object, the color of the color that approximates the color of the target object A decrease in tracking accuracy due to the influence of the area can be prevented.

  Embodiments of the present invention will be described in detail with reference to the drawings. First, the first embodiment will be described. FIG. 1 is a block diagram showing a configuration of a target tracking system 10 including a target tracking device 100 according to the first embodiment of the present invention. As shown in the figure, the target tracking system 10 includes a target tracking device 100, a camera 200 that performs shooting and outputs an image signal, and pan / tilt control and zoom magnification of the camera 200 according to control signals from the target tracking device 100. A control device 210 that performs control is provided. However, the target tracking system 10 may be configured by integrating these devices.

  The target tracking device 100 performs target tracking processing using an image input unit 110 that inputs an image signal output from the camera 200, a display unit 120 that displays an image based on the input image signal, and a particle filter. The particle filter processing unit 130, the camera control signal output unit 140 that generates a signal for controlling the camera 200 based on the tracking result and outputs the signal to the control device 210, and the tracking target in the image displayed on the display unit 120 A tracking target designation receiving unit 150 that receives designation of an object to be configured, a target color setting unit 160 that sets a target color according to the designated tracking target, and an auxiliary target color setting unit 170 that sets an auxiliary target color I have.

  The particle filter processing unit 130 includes an image storage unit 131 that stores image data to be processed in order to perform tracking processing of a target using a particle filter, a target color storage unit 132 that stores a set target color, And an auxiliary target color storage unit 133 that stores the set auxiliary target color.

  The target tracking device 100 can be configured using a general-purpose information processing device including a CPU, a memory, an input / output device, a communication processing device, and the like. A general-purpose information processing device such as a personal computer can function as the target tracking device 100 by executing a program developed for a specific application. Of course, the target tracking device 100 may be configured as a dedicated device.

Here, an image tracking algorithm using a particle filter introduced in Non-Patent Document 1, Non-Patent Document 2, and the like will be briefly described. The particle filter that is an image tracking algorithm is a posteriori probability density that is an estimated amount of the position of the target object in the image frame at time t.

Is a particle group consisting of N sets of weights π of state x

This is a method of tracking a target object by approximating by. Here, the state quantity of the i-th particle in the image frame at time t

Indicates the weight which is the likelihood.

When the likelihood observation value z _t is obtained from the image frame at time t, the probability density using the tracking target state x _t as a random variable is expressed as the posterior probability density [Equation 1]. This [Equation 1] can be expressed as follows using Bayes' law.

Here, α is a constant for normalization.

  The Condensation algorithm shown in Non-Patent Document 1 repeatedly executes the process of tracking the target object for each frame in three steps: Select, Predict, and Measure. ing.

  Specifically, in the selection step, particles are distributed and arranged around the target object, the likelihood is measured for each particle, and N particles are probable in the order of the likelihood in proportion to the likelihood height. Select the particle. In the prediction step, particles with high likelihood selected in the selection step are randomly dispersed and rearranged. In the measurement step, for each particle, the number of pixels in the color approximate to the target object is calculated in the vicinity, and this is used as the likelihood of the particle. Then, the position of the target object is estimated by a weighted average of the likelihoods of all particles.

  The target tracking device 100 according to the present embodiment improves the target tracking performance of the particle filter processing of such an algorithm, and performs the processing described below. However, the present invention is not limited to the above algorithm as long as it is an image tracking process using a particle filter, and can be applied to various deformation algorithms.

  FIG. 2 is a flowchart for explaining the flow of processing of the target tracking device 100. The target tracking device 100 inputs an image signal output from the camera 200 in units of frames (S101). The target tracking device 100 sequentially displays the input image signal as an image on the display unit 120, and waits for the tracking target to be designated (S102). In this state, the supervisor can specify a target object that is desired to be tracked in the image.

  The designation of the tracking object can be received, for example, by receiving an instruction from the monitor corresponding to the tracking object displayed on the display unit 120 via an input device such as a mouse or a pointer. Further, the display unit 120 may be configured by a touch panel display device, and designation may be received by recognizing a touch from a supervisor. Or you may make it receive designation | designated of a tracking target object through another external device etc.

  When the specification of the tracking target is received (S102: Yes), the target color setting unit 160 acquires the color of the pixel corresponding to the area in which the specification in the image frame is received and sets it as the target color (S103). The target color may be set based on the RGB value of a single pixel in the designated area, or may be set based on the RGB average value of a plurality of pixels. Alternatively, a histogram may be created with pixels included in a designated area and set as a target color. When the target color is set, it is recorded in the target color storage unit 132. The image frame is recorded in the image storage unit 131.

  In this embodiment, the particle filter processing unit 130 performs processing in the HSV color space expressed by hue (H), saturation (S), and lightness (V). Convert color space to HSV color space. When creating a histogram, it is created in the HSV color space. However, the processing may be performed in the RGB color space or may be performed in the YUV color space.

  In this embodiment, when the target color is set, the auxiliary target color is set in order to increase the tracking accuracy of the target object (S104). Here, the setting of the auxiliary target color, which is a characteristic process of the present embodiment, will be described in detail. FIG. 3 is a flowchart for explaining auxiliary target color setting processing. FIG. 4 is a diagram illustrating an example of image frames and particles in the auxiliary target color setting process. In the example illustrated in FIG. 4, it is assumed that a person's face is specified as a tracking target, and the face color is set as a target color.

  The particle filter processing unit 130 generates particles near the area designated as the tracking target for the image frame in the initial state in which the designation of the tracking target is received in the process (S102) in FIG. 2 (S201). The tracking accuracy and processing speed depending on the number of particles to be generated are in a trade-off relationship. FIG. 4A shows a large number of particles generated near the face of a person to be tracked.

  Next, each generated particle is moved according to a predetermined rule in order to estimate a target color region (S202). Here, the distance is determined by a random number with a standard normal distribution so that each particle is dispersed from the specified tracking target. As a result, as shown in FIG. 4B, particles are dispersed around the human face. However, if a region corresponding to the target color has already been specified as the tracking target, processing for estimating the target color region is not necessary.

  For each dispersed particle, the ratio of pixels near the target color is calculated for the pixels in the vicinity of each position in the image frame, for example, pixels in the surrounding rectangular area, and the result is the likelihood of each particle. Is measured (S203). When the target color is expressed by a histogram, an HSV histogram in a rectangular area can be created, the similarity to the target color can be calculated, and the likelihood can be measured. The black particles in FIG. 4C indicate particles with a high measured likelihood and are present in the face area. When the likelihood is measured, the weighted average of the likelihood of each particle is calculated, and the area of the tracking target in the image frame to be processed is estimated (S204). For example, the region of the tracking object can be estimated by binarizing the likelihood using a certain threshold value, estimating the mixed normal distribution using the EM algorithm, and further estimating the variance.

  In the present embodiment, the area of the tracking target object is expressed by a rectangle. FIG. 4D shows the frame estimated as the area of the tracking target object. In this figure, Frame has a size of vertical H and horizontal W.

  Then, the designation of the auxiliary tracking target area is accepted on the basis of the estimated tracking target area (S205). The auxiliary tracking target area is an area used for identifying the tracking target object when there are a plurality of color areas close to the tracking target object in the image frame. That is, in the present embodiment, when estimating the tracking target area based on the target color, the accuracy of the target object tracking is improved by further measuring the color likelihood in the auxiliary target area associated with the tracking target area. Like that.

  For example, as shown in FIG. 4E, the auxiliary tracking target area includes four rectangular areas adjacent to the tracking target area Frame, that is, an upper area subFrT, a lower area subFrB, a right area subFrR, and a left area It can be any of the regions subFrL. The size of each region is such that the side in contact with the tracking target region Frame is made to correspond to the tracking target region Frame, and the side that is not in contact is determined appropriately. For example, it may be determined in advance or may be determined according to the size of the tracking target area Frame.

  Which area is set as the auxiliary tracking target area can be determined based on an instruction from the supervisor, for example. In general, since a person can be identified by the color of clothes, if the tracking target is a face, the auxiliary tracking target area is preferably an area including clothes. For this reason, the lower region subFrB may be set in advance. For example, when the tracking target is identified by the face and the hat, the upper area subFrT can be set as the auxiliary tracking target area. When tracking a truck, a car, or the like whose color feature is in the horizontal direction, the right area subFrR or the left area subFrL can be set as the auxiliary tracking target area.

  When the auxiliary target color setting unit 170 receives the designation of the auxiliary tracking target region, the auxiliary target color setting unit 170 sets the color of the region that has become the auxiliary tracking target region as the auxiliary target color and stores it in the auxiliary target color storage unit 133 (S206). Here, the auxiliary target color can be an average color of pixels included in the auxiliary tracking target area Frame, a histogram, or the like. Which format is set corresponds to the target color setting format.

  Then, based on the likelihood of the particles measured in the process (S203), the particles with low likelihood are moved to positions where the likelihood becomes high so that the particles with high likelihood are probabilistically increased. As a result, as shown in FIG. 4 (f), the particles gather in the region of the tracking target object.

  When the above auxiliary target color setting process (S104) is completed, the target tracking process for recognizing the tracking object in the sequentially input image frame using the target color and the auxiliary target color and controlling the camera 200 is performed. This is performed (S105).

  Here, the target tracking process in the present embodiment will be described in detail. FIG. 5 is a flowchart for explaining the target tracking process. As described above, the present invention is not limited to the particle filter object tracking algorithm described here. FIG. 6 is a diagram illustrating an example of image frames and particles in the target tracking process. The example in FIG. 6 is a continuation of the example shown in FIG. 4, and it is assumed that the face of a person is designated as the tracking target and the face color is set as the target color.

  First, the particles generated in the process (S201) in FIG. 3 and aggregated in the tracking target area in the process (S207) are moved according to a predetermined rule (S401). Here, when the movement direction of the tracking target can be predicted, the movement direction of the particles can be determined according to the predicted movement direction. In the present embodiment, the tracking target is mainly assumed to be a suspicious person in the monitoring image, and the motion dynamics is assumed to be random. For this reason, each particle moves with a distance determined by random numbers with a standard normal distribution. As a result, as shown in FIG. 6A, particles are dispersed around the human face.

  Then, the next image frame is input (S402). This image frame becomes a processing target image frame for detecting the movement of the tracking target. FIG. 6B shows a state when the next image frame is input to the dispersed particles. In the input image frame, the face of the person who is the tracking target is moving in the right direction.

  Each particle calculates the ratio of pixels close to the target color for pixels in the vicinity of each position in the image frame to be processed, for example, pixels in the surrounding rectangular area, and the result is the likelihood of each particle. Measurement is performed (S403). At this time, for example, the likelihood can be measured by creating an HSV histogram in a rectangular area and calculating the similarity to the target color. In this case, the likelihood obtained by replacing the color value of the pixel corresponding to each particle with the frequency in the target color histogram can be used. The black particles in FIG. 6C indicate particles with high likelihood as a result of the measurement.

  In this embodiment, after the likelihood is measured using the target color, the likelihood is further measured using the auxiliary target color (S404). Here, the measurement of likelihood using the auxiliary target color will be described with reference to FIG. FIG. 7 shows a case where two persons are included in the image frame. When a person's face is designated as a tracking target, as shown in FIG. 7A, particles (p1, p2, p3, p4, dispersed on the face portions of one person Ob1 and the other person Ob2). High likelihood is measured at p5, p6).

  At this time, it is assumed that the area subFrB below the tracking target area Frame shown in FIG. 4D is set as the auxiliary tracking target area. For this reason, the color of the clothes of the person Ob1 is set as the auxiliary target color.

  In the measurement of the likelihood using the auxiliary target color, auxiliary particles are set at positions separated by a distance d in the direction in which the auxiliary tracking target area is set for each particle, and the likelihood is calculated using the auxiliary target color. Measure. Here, the distance d can be, for example, ½ of the length in the direction in which the auxiliary tracking target area of the tracking target area Frame is set. That is, in this example, it becomes 1/2 of the length H in the vertical direction of the tracking target area Frame. Of course, the distance d may be determined by other methods.

  As a result, as shown in FIG. 7A, the likelihood of the person Ob1 is increased by the auxiliary particles of the particles p2 and p3 whose likelihood is measured at the clothes position corresponding to the auxiliary target color. On the other hand, since the person Ob2 is dressed in a color different from the auxiliary target color, any auxiliary particle has a low likelihood.

  When the likelihood measurement with the auxiliary target color is performed, the likelihood of the particles measured in the process (S403) is corrected (S405). The likelihood correction can be performed, for example, by multiplying the likelihood of the high likelihood particle measured in the process (S403) and the likelihood of the corresponding auxiliary particle in the auxiliary target color. Of course, other calculation methods may be used.

  By correcting the likelihood, it is possible to determine which object is the tracking target when there are a plurality of objects having a color close to the target color. For example, in the example shown in FIG. 7B, the likelihood of the particles p2 and p3 near the person Ob1 remains high after correction, and the likelihood of the particles p4, p5 and p6 near the person Ob2 becomes low after correction. . As a result, it is possible to determine that the particles near the person Ob1 are particles near the tracking target, and the target object can be tracked correctly. The likelihood measurement using the auxiliary target color may be performed only when a plurality of objects having a color close to the target color exists in the image frame.

  Then, the weighted average of the likelihood of each particle is calculated, and the region of the tracking target in the image frame to be processed is estimated (S406). In this example, the area including the black particles in FIG. 6C is estimated as the area of the tracking target object.

  Next, a control signal for causing the region estimated as the region of the tracking target to fall within the angle of view of the camera 200 is generated, and the camera control signal output unit 140 outputs the control signal to the control device 210 (S407). When the camera 200 pans, tilts, and zooms based on the control signal, the tracking target can be tracked. When the angle of view of the camera 200 is changed, the subsequent image frame processing is performed in consideration of the relative movement amount of the tracking target in the image accompanying the change of the angle of view. Specifically, the position of the particle is adjusted so that the relative positional relationship between the particle and the tracking target is maintained. This process can use conventional techniques.

  Then, based on the measured likelihood of particles, particles with low likelihood are moved to positions where the likelihood becomes high so that particles with high likelihood increase stochastically. As a result, as shown in FIG. 6D, the particles gather on the tracking object.

  Thereafter, the particles collected on the tracking target are dispersed again in the same manner as described above (S401), and the same processing for the next image frame is repeated until the target tracking processing is completed (S409). The end determination criterion for the target tracking process can be, for example, a case where the tracking object is out of the tracking range of the camera 200, or a case where an end instruction is given from the supervisor.

  Next, a modification of the first embodiment of the present invention will be described. In the first embodiment described above, the likelihood of the auxiliary target color is measured with the auxiliary particles corresponding to the particles, and the likelihood of the particles is corrected. In the modified example, tracking of the target object using the auxiliary target color is performed by a simplified process. Below, it demonstrates centering on difference with 1st Embodiment. In addition, the modification of 1st Embodiment can be set as the structure similar to 1st Embodiment. For this reason, illustration of block diagrams and the like is omitted, and the same functional units and processes will be described with the same reference numerals. Further, functional units having different processing contents are distinguished by adding “a” to the end of the code of the corresponding functional unit.

  Similar to the first embodiment (see FIG. 2), in the modification of the first embodiment, an image frame is input (S101), and designation of the tracking target is received from the supervisor (S102). Based on this designation, the target color setting unit 160 sets a target color (S103) and stores it in the target color storage unit 132. Next, the tracking object designation receiving unit 150 receives designation of the auxiliary tracking target area. Further, the auxiliary target color setting unit 170 sets the color of the area that has become the auxiliary tracking target area as the auxiliary target color (S104), and stores it in the auxiliary target color storage unit 133. Then, the particle filter processing unit 130a performs target tracking processing (S105).

  The target tracking process in the modification of 1st Embodiment is demonstrated with reference to the flowchart of FIG. In the target tracking process in the modified example, the particles are dispersed (S501) as in the first embodiment, and the next image frame is input (S502). Then, the likelihood measurement using the target color is performed for each particle (S503).

  When the likelihood measurement is performed, the tracking target candidate is extracted in this modified example (S504). In the extraction of the tracking target object candidate, the weighted average of the likelihood of each particle is calculated and the tracking target object area in the processing target image frame is estimated, as in the tracking target object area estimation in the process (S204). Thus, the estimated region is set as a tracking target candidate.

  Then, it is determined whether there are a plurality of candidate areas (S505). If there is not more than one (S505: No), that is, if there is one candidate area, the candidate area is estimated as a tracking target (S508), and camera control is performed based on the estimated position (S509). Next, the low likelihood particle is moved to a position where the likelihood becomes high (S511), and these processes are repeated until the target tracking process is completed (S512).

  On the other hand, when there are a plurality of candidate areas (S505: Yes), an evaluation using the auxiliary target color is performed for the auxiliary tracking target area corresponding to the candidate tracking target area (S506). Here, the evaluation using the auxiliary target color in this modification will be described with reference to FIG. FIG. 9A shows a case where a person's face is designated as a tracking target and two persons are included in the image frame. As shown in this figure, the likelihood of particles dispersed in the face portions of one person Ob1 and the other person Ob2 is measured with high likelihood. Frame1 corresponding to the face portion of the person Ob1 and the face of the person Ob2 Frame 2 corresponding to the part is a candidate for the tracking target. It is assumed that the lower area subFrB is set as the auxiliary tracking target area.

In this modification, it is evaluated whether the color of the auxiliary tracking target area for the tracking target area candidate is close to the auxiliary target color. Here, the evaluation of whether or not it is close to the auxiliary target color can be performed, for example, by calculating a ratio of pixels having a color close to the auxiliary target color in the auxiliary tracking target region. Alternatively, this can be done by calculating the difference between the average color and the auxiliary target color in the auxiliary tracking target area. Further, when the auxiliary target color is set as a histogram, it can be performed by obtaining a histogram of the color of the pixel included in the auxiliary tracking target area and obtaining an intersection with the histogram of the auxiliary target color. The histogram intersection can be obtained by the following equation.

Here, H1i is a histogram of the auxiliary target color, and H2i is a color histogram of the auxiliary tracking target area. The larger the value of S, the more similar the histogram.

  When the evaluation is made as to whether or not it is close to the auxiliary target color in this way, the evaluation value is normalized so that, for example, the same color becomes 1, and the normalized evaluation value is measured in the processing (S503). The likelihood is corrected by multiplying with the likelihood (S507). Then, the region of the tracking target is estimated using the corrected likelihood (S508).

  In the example shown in FIG. 9A, the evaluation value of the auxiliary tracking target area subFrB1 of the person Ob1 is higher than the evaluation value of the auxiliary tracking target area subFrB2 of the person Ob2. For this reason, when the likelihood is corrected, as shown in FIG. 9B, the likelihood of the particles of the face portion of the person Ob1 remains high, and the likelihood of the particles of the face portion of the person Ob2 becomes low. As a result, the region Frame1 of the face portion of the person Ob1 is estimated as the tracking target.

  When the tracking target is estimated, camera control is performed based on the estimated position (S509). Next, the low likelihood particle is moved to a position where the likelihood becomes high (S511), and these processes are repeated until the target tracking process is completed (S512).

  Next, a second embodiment of the present invention will be described. Since the second embodiment can be configured substantially the same as the first embodiment, the differences will be mainly described, and the same functional units as those of the first embodiment will be simply denoted by the same reference numerals.

  FIG. 10 is a block diagram showing a configuration of a target tracking system 10b including the target tracking device 100b according to the second embodiment of the present invention. As shown in the figure, the target tracking system 10b includes a target tracking device 100b, a camera 200 that performs imaging and outputs an image signal, and pan / tilt control and zoom magnification of the camera 200 according to control signals from the target tracking device 100b. A control device 210 that performs control is provided. However, the target tracking system 10b may be configured by integrating these devices.

  The target tracking device 100 includes an image input unit 110, a display unit 120, a particle filter processing unit 130b that performs a target tracking process using a particle filter, a camera control signal output unit 140, and a tracking target designation receiving unit. 150 and a similar color distribution distance calculation unit 180. The particle filter processing unit 130 b includes an image storage unit 131 and a target color storage unit 132. The similar color distribution distance calculation unit 180 determines a color region similar to the target color in the image frame, and performs a process of calculating the shortest distance from the tracking target object. Specific contents of this processing will be described later.

  FIG. 11 is a flowchart for explaining the processing flow of the target tracking device 100b in the second embodiment. The target tracking device 100b inputs the image signal output from the camera 200 in units of frames (S601). The target tracking device 100b sequentially displays the input image signal as a video on the display unit 120, and waits for the tracking target to be designated (S602). In this state, the supervisor can specify a target object that is desired to be tracked in the image.

  When the specification of the tracking target is received (S602: Yes), the target color setting unit 160 acquires the color of the pixel corresponding to the area in the image frame that has received the specification, and sets it as the target color (S603). The target color can be set in the same manner as in the first embodiment. Then, target tracking processing for recognizing a tracking target in an image frame sequentially input using the target color and controlling the camera 200 is performed (S604). That is, unlike the first embodiment, the second embodiment performs the target tracking process without using the auxiliary target color.

  Here, the target tracking process in the present embodiment will be described in detail. FIG. 12 is a flowchart for explaining target tracking processing in the second embodiment. First, the particle filter processing unit 130b generates particles near the area designated as the tracking target (S701). The tracking accuracy and processing speed depending on the number of particles to be generated are in a trade-off relationship.

  Then, the distribution of the color area close to the target color in the image frame is determined, and the shortest distance between the area and the tracking target is calculated (S702). Here, the similar color distribution distance calculation processing performed by the similar color distribution distance calculation unit 180 will be described with reference to the flowchart of FIG. 13 and the image frame example of FIG.

In the similar color distribution distance calculation process, the degree of approximation A between the color (Ht, St, Vt) and the target color (H, S, V) of each pixel in the image frame is calculated, and the calculated degree of approximation A is calculated. Binarization is performed with a predetermined threshold T (S801). The calculation of the approximation A and the binarization of the color approximation A can be performed, for example, according to the following equation.

As a result, the area formed by the pixels with p = 1 is a color area close to the target color. For example, as shown in FIG. 14A, when two persons are included in the image frame and the face of one person is designated as the tracking target, as shown in FIG. A binary distribution image is obtained. Here, the white portion is p = 1, that is, a color region that approximates the face color that is the target color, and the black portion is p = 0, that is, a region of other colors.

  Since the binarized distribution image generally contains noise, noise removal processing is performed (S802). The noise removal process can be performed by, for example, a degeneration / expansion process that is a general technique. FIG. 14C shows a distribution image after noise removal. However, this process may be omitted.

  Next, a labeling process is performed for each block of regions, and the area of each region is calculated. Then, the small area whose area is equal to or smaller than the predetermined threshold is deleted as a tracking target object (S803). FIG. 14D shows a distribution image after performing a labeling process and deleting a small region.

  For the remaining approximate color area, the distance from the position to be tracked is calculated, and the shortest distance is extracted (S804). At this time, the area including the tracking target is excluded from the calculation of the distance. In the example shown in FIG. 14E, the face X of the left person is designated as a tracking target, and distances a and b between the area Y and the area Z are calculated. As a result, the distance a is extracted as the shortest distance from the similar color region.

  When the similar color distribution distance calculation process (S702) ends, the particles generated in the process (S701) are moved according to a predetermined rule (S703). Here, when the movement direction of the tracking target can be predicted, the movement direction of the particles can be determined according to the predicted movement direction. In the present embodiment, the tracking target is mainly assumed to be a suspicious person in the monitoring image, and the motion dynamics is assumed to be random. For this reason, each particle moves with its distance determined by a normally distributed random number.

In this embodiment, at this time, the dispersion of the normal distribution is changed according to the shortest distance from the similar color area obtained in the similar color distribution distance calculation process (S702). Specifically, regular random numbers

The variance sigma ² in, to correspond to the magnitude of the shortest distance between the similar color region. Here, rand (x) is a uniform random number from 0 to less than 1.

  That is, in the second embodiment, by controlling the distribution of the normal distribution according to the shortest distance between the target color and the similar color region, many particles are dispersed within the shortest distance from the tracking target. As a result, there are not many particles in the similar color region. Therefore, the likelihood of particles in the region including the original tracking target can be relatively increased in the subsequent processing.

  For example, as shown in FIG. 15A, when the shortest distance d1 between the tracking target object X and the approximate color region Y is relatively large, the tracking target object X is centered as shown in FIG. Particles are widely dispersed. Thereby, even when the tracking target object X moves greatly, it becomes possible to track with high accuracy. On the other hand, as shown in FIG. 15C, when the shortest distance d2 between the tracking object X and the approximate color region Y is relatively small, the tracking object X is centered as shown in FIG. Particles are dispersed narrowly. Thereby, it is possible to prevent a decrease in tracking accuracy due to the influence of the approximate color region Y.

  When the particles are moved by controlling the dispersion by the above processing (S703), the next image frame is input (S704). This image frame becomes a processing target image frame for detecting the movement of the tracking target. Each particle calculates the ratio of pixels close to the target color for pixels in the vicinity of each position in the image frame to be processed, for example, pixels in the surrounding rectangular area, and the result is the likelihood of each particle. Measurement is performed (S705). At this time, for example, the likelihood can be measured by creating an HSV histogram in a rectangular area and calculating the similarity to the target color.

  When the likelihood is measured, the weighted average of the likelihood of each particle is calculated, and the area of the tracking target in the image frame to be processed is estimated (S706). Then, a control signal is generated so that the region estimated as the tracking target region falls within the angle of view of the camera 200, and the camera control signal output unit 140 outputs the control signal to the control device 210 (S707). When the camera 200 pans, tilts, and zooms based on the control signal, the tracking target can be tracked.

  Then, based on the measured likelihood of particles, particles with low likelihood are moved to positions where the likelihood becomes high so that particles with high likelihood increase stochastically. Thereafter, similarly to the above, the shortest distance between the estimated tracking object and the similar color region in the image frame is calculated (S702), and the particles gathered on the tracking object are dispersed again according to the shortest distance ( In step S703, the same process for the next image frame is repeated until the target tracking process is completed (S709).

  In the first and second embodiments described above, the target tracking devices 100 and 100b are configured to receive the tracking target in the image displayed on the display unit 120 by the tracking target designation receiving unit 150. The position of the tracking object may be automatically specified based on the results of motion detection processing, color detection processing, and the like by a control unit (not shown). In this case, it is not necessary to provide the display unit 120 and the tracking target designation receiving unit 150.

It is a block diagram which shows the structure of the target tracking system containing the target tracking apparatus in 1st Embodiment. It is a flowchart explaining the flow of a process of a target tracking apparatus. It is a flowchart for demonstrating the setting process of an auxiliary target color. It is a figure which shows the example of the image frame and particle in the setting process of an auxiliary | assistant target color. It is a flowchart for demonstrating a target tracking process. It is a figure which shows the example of the image frame and particle in a target tracking process. It is a figure for demonstrating the measurement of the likelihood using an auxiliary | assistant target color. It is a flowchart for demonstrating the target tracking process in the modification of 1st Embodiment. It is a figure for demonstrating the evaluation using an auxiliary target color. It is a block diagram which shows the structure of the target tracking system containing the target tracking apparatus which is 2nd Embodiment. It is a flowchart explaining the flow of a process of a target tracking apparatus. It is a flowchart for demonstrating a target tracking process. It is a flowchart for demonstrating a similar color distribution distance calculation process. It is an example of an image frame for demonstrating a similar color distribution distance calculation process. It is an image frame example for demonstrating dispersion | distribution control of normal distribution according to the shortest distance of a target color and a similar color area | region.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 * 10b ... Target tracking system 100 * 100b ... Target tracking apparatus 110 ... Image input part 120 ... Display part 130 * 130a * 130b ... Particle filter processing part 131 ... Image storage part 132 ... Target color storage part Reference numeral 133: Auxiliary target color storage unit 140: Camera control signal output unit 150: Tracking target designation receiving unit 160 ... Target color setting unit 170: Auxiliary target color setting unit 180 ... Similar color distribution distance calculation unit 200 ... Camera, 210 ... Control device

Claims (3)

A target tracking device that tracks a target object in image data captured in time series,
Tracking target designation receiving means for receiving designation of a tracking target to be the target object in image data;
Auxiliary area designation accepting means for accepting designation of an auxiliary area associated with the tracking target in the image data;
Target color setting means for setting a color in the image data relating to the specified tracking target as a target color;
Auxiliary target color setting means for setting a color related to the specified auxiliary area as an auxiliary target color;
Using particles that move in the image data according to a predetermined rule, measure the likelihood obtained by comparing the color around the particle and the set target color, the measured likelihood for each particle Correction is performed by the likelihood obtained by comparing the color in the area determined based on the designated auxiliary area and the auxiliary target color, and the tracking target area in the image data is estimated based on the corrected likelihood. Particle filter processing means;
A target tracking device comprising: A target tracking device that tracks a target object in image data captured in time series,
Tracking target designation receiving means for receiving designation of a tracking target to be the target object in image data;
Auxiliary area designation accepting means for accepting designation of an auxiliary area associated with the tracking target in the image data;
Target color setting means for setting a color in the image data relating to the specified tracking target as a target color;
Auxiliary target color setting means for setting a color related to the specified auxiliary area as an auxiliary target color;
Using particles that move in the image data according to a predetermined rule, the likelihood obtained by comparing the color around the particle and the set target color is measured, and the measured likelihood is measured. The tracking target in the image data is corrected based on the degree of approximation between the color in the area determined based on the distribution of degrees and the designated auxiliary area and the auxiliary target color, and based on the corrected likelihood Particle filter processing means for estimating the area of
A target tracking device comprising: A target tracking device that tracks a target object in image data captured in time series,
Tracking target designation receiving means for receiving designation of a tracking target to be the target object in image data;
Target color setting means for setting a color in the image data relating to the specified tracking target as a target color;
Using particles dispersed from the vicinity of the region to be tracked in the previously captured image data, the likelihood obtained by comparing the color around the particle with the set target color is measured and measured. Particle filter processing means for estimating the tracking target area in the image data captured later based on the likelihood,
The particle filter processing unit obtains a distance between a region to be tracked in the previously captured image data and a region of a color similar to the target color, and in the dispersion of the particles according to the obtained distance A target tracking device characterized by changing a movement amount.