JP2005100388A - Object tracking method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To effectively control an imaging means by an object tracking method for tracking an object in an image based on an image signal obtained by an imaging means.
An object-in-image detecting step for detecting a feature amount of the object in an image, an imaging lens control step for controlling an imaging lens of the imaging means based on the detection result, and the detection based on the detection result A partial image setting step for setting a partial image of the image and an image enlargement step for enlarging the set partial image to a predetermined size are provided.
[Selection] Figure 1

Description

  The present invention relates to an object tracking method for tracking an object in an image to be captured, and more particularly to a technique for effectively controlling an imaging lens that performs imaging.

  For example, a remote monitoring type monitoring system using an imaging device such as a TV camera (television camera) has been widely used in the past, and many of them are monitored while watching an image displayed on the monitor. It is a so-called manned monitoring system. In a manned monitoring system, it is necessary for a monitor to constantly monitor the images displayed on the monitor and identify intruders such as humans and cars entering the monitored area in real time. It takes a big burden.

  In other words, since the human concentration is limited, in the monitoring system of the manned monitoring system, the occurrence of oversight of the intruding object cannot be ignored, and there is a problem in terms of reliability. In addition, with the explosive spread of surveillance cameras, there are also many scenes where one surveillance person monitors a number of TV camera images on multiple monitors. May be missed.

  Therefore, instead of monitoring by such a person, a camera equipped with a TV camera so that an intruding object is automatically detected by image processing from an image captured by a TV camera and the image of the intruding object is captured at the center of the screen. In recent years, there has been a strong demand for a so-called automatic tracking type monitoring system in which a pan head (a swivel base) is controlled to automatically adjust the viewing direction and the angle of view so as to obtain predetermined notifications and alarms. It has become like this.

  By the way, in order to realize such a system, a function of detecting a monitoring target object to be regarded as an intruding object from an image signal by using a predetermined monitoring method and detecting the movement of the intruding object is required. One example of a monitoring method for detecting such an intruding object is a method called a difference method, which has been widely used. The difference method is a method in which an input image obtained from a TV camera is compared with a reference background image created in advance, that is, an image in which an object to be detected is not captured, a difference in luminance value is obtained for each pixel, and the difference value is obtained. Is detected as an object. In addition, an application example of the difference method has been studied (for example, see Patent Document 1).

Furthermore, there is a method called a template matching method as an example of a monitoring method for detecting the amount of movement of an intruding object, which has been widely used conventionally as in the difference method. In the template matching method, an image of an intruding object detected by a difference method or the like is registered as a template, and a position most similar to the template image is detected among images sequentially input (see, for example, Non-Patent Document 1). .) Normally, when tracking a target object using template matching, in order to follow a change in the posture of the target object, an image of the position of the target object detected by the matching process is sequentially updated as a template.
JP-A-9-73541 Supervised by Hideyuki Tamura, “Introduction to Computer Image Processing”, Soken Publishing, 1985, p.149-153

  By the way, in an object tracking type monitoring system, there is a system called mechanical / optical tracking that involves mechanical and / or optical control of a camera head and an imaging lens. However, when a processing device such as an MPU (Micro Processing Unit) of the monitoring system determines that control of the camera pan head or control of the imaging lens is necessary, there is a delay time until the control is actually started. In addition, the control time of the camera platform and the imaging lens (the time required to complete the control to the target camera platform position and the focal length of the imaging lens) may be several seconds, during which multiple frames are processed by the processing device. The input image is processed.

  This situation will be specifically described with reference to FIG. FIG. 7 is an example in which a humanoid intruding object 801a detected at a certain time t1 is tracked while zooming in. In the figure, the time when the input image 801 is acquired is denoted by t1, and the acquisition time of the input images that are sequentially input at a predetermined time interval (for example, 100 ms) is (t1 + 1), (t1 + 2), in the order of input.・ It expresses.

  First, an intruding object 801a in the input image 801 obtained at time t1 is detected by a difference method. The image of the intruding object is registered as a template 801b. Since the intruding object 801a exists on the left side from the center of the input image 801, a command for rotating (panning) the camera head to the left is transmitted via the head control interface means, and the size of the image of the intruding object is further transmitted. A command to increase the focal length of the imaging lens is transmitted via the lens control interface means in order to make the predetermined size (for example, 80% of the screen vertical).

  Next, the intruding object 802a is detected by applying the template matching method to the input image 802 obtained at time (t1 + 1), and the image is updated as the template 802b. At this time, the operation of the command transmitted from the camera platform and the imaging lens at time t1 is not completed. In this case, a control command is transmitted again to the camera head and the imaging lens.

  Next, a template matching method is applied to the input image 803 obtained at time (t1 + 2) to detect the intruding object 803a, and the image is updated as a template 803b. Here, the intruding object 803a is located at the center of the screen, and the control of the camera pan head is completed. However, the size of the intruding object on the input image does not reach the target predetermined size. Therefore, the control command is transmitted to the imaging lens again.

  Next, the template matching method is applied to the input image 804 obtained at time (t1 + 3) to detect the intruding object 804a, and the image is updated as the template 804b. At this time, since the size of the intruding object on the input image has reached the target predetermined size, the control of the imaging lens is completed.

  In this way, there is a delay in the detection result of the intruding object of the processing device and the control of the camera pan head and the imaging lens, and due to this low response, the camera pan head in the movement of the intruding object in the monitoring area, Alternatively, the imaging lens may not be able to follow, and in that case, the intruding object cannot be caught in the field of view of the imaging apparatus, and it is difficult to improve the object tracking performance. This problem is noticeable when the focal length of the imaging lens is long (when zoomed in). Therefore, when performing mechanical / optical tracking, monitoring with a small focal length of the imaging lens is required. Required.

  In order to solve the above problems, there is an object tracking method called electronic tracking in which a part of an input image is electronically enlarged and tracked without controlling the camera platform and the imaging lens. Since this method cuts out and enlarges a part of the input image, it is possible to realize pseudo camera pan head control by adjusting the cutout position. Furthermore, since no mechanical control is involved, it is possible to solve the low responsiveness of the controlled device as in the mechanical / optical tracking described above, and to perform stable tracking. However, since this method cuts out and enlarges a part of the input image, there is a problem that when the resolution of the input image is low, the enlarged image becomes a block (mosaic) image.

  This problem will be specifically described with reference to FIG. FIG. 8 shows the result of processing the input image 901 (same as the input image 801) at the same time t1 as in the example described in FIG. In FIG. 8, when electronic tracking is performed on the input image 901, a block-like image 902 obtained by electronically enlarging the partial image 901c of the input image 901 is displayed on the image monitor. In the same figure, as shown in an image 902, when the electronic enlargement ratio increases, information such as a fine expression of the intruding object cannot be obtained. Further, in the electronic tracking, since the entire monitoring area must be imaged by the imaging device without controlling the camera platform and the imaging lens, it is necessary to use an imaging device with a wide angle of view. That is, in order to perform electronic tracking, an expensive imaging device with a high resolution and a wide angle of view is required.

  Here, in the automatic tracking type monitoring system, it is important to zoom in and monitor the target object to be monitored without reducing the reliability of the object tracking function. In mechanical / optical tracking, for example, there is an advantage that a wide range of monitoring and an image of an intruding object can be acquired with high resolution. On the other hand, as an example, the responsiveness of the camera head and imaging lens is low. There is a problem that it takes time to capture the intruding object with an appropriate size in the center of the screen, and a problem that the object tracking process cannot be performed if the intruding object comes off the image.

  Electronic tracking, for example, has the advantage that an intruding object can be captured at an appropriate size in the center of the screen at a high speed. On the other hand, as an example, if the enlargement ratio is increased with a low resolution input image, There is a problem that information such as a fine expression of an intruding object cannot be obtained, and a wide-angle imaging device is required.

  The present invention has been made in view of such a conventional situation, and an object of the present invention is to provide an object tracking method capable of effectively controlling an imaging unit when tracking an object in an image to be captured. And

In order to achieve the above object, an object tracking method according to the present invention is an object tracking method for tracking an object using an imaging unit capable of controlling an imaging direction and a zoom amount, and an input image obtained from the imaging unit. Detecting at least one feature amount of an image of the object in the image, controlling the imaging means based on the detected feature amount for tracking the object, and inputting the input based on the detected feature amount Set the range of the partial area including the image of the object in the image,
The image of the set partial area is enlarged.

  In this specification, the term “tracking” is used for explanation, but for example, the term “tracking” is also a similar term and is included in the present invention. In this specification, the term “image” is used for explanation. For example, the term “video” is a similar term and is included in the present invention. In addition, for example, an image referred to in the present specification refers to a moving image when it is temporally continuous. For example, as to a still image, an image of one frame in the moving image, Some of the images and still images that are unrelated to moving images.

  Here, various imaging means may be used, and for example, a camera or the like can be used. Various image signals such as an NTSC system and a PAL system may be used. Various objects such as people, vehicles, animals, etc. may be used as objects. Further, the object in the image corresponds to, for example, the image portion of the target object existing in the image.

In the object tracking method according to the present invention, the at least one feature amount includes at least one of a position, a size, and a movement amount of the image of the object.
In the object tracking method according to the present invention, the position of the range of the partial area is set based on the position of the image of the object, and the size of the range of the partial area is set based on the size of the image of the object. Set.

In the object tracking method according to the present invention, the image of the range of the partial area is enlarged at an enlargement ratio set based on the size of the range of the partial area and a predetermined image display size.
Further, in the object tracking method according to the present invention, an upper limit of the zoom amount of the imaging lens of the imaging unit is set based on the size of the image of the object, and the size of the range of the partial region is the size of the input image. The size is set to be less than or equal to a predetermined ratio smaller than 1.

In the object tracking method according to the present invention, the zoom amount of the imaging unit is changed depending on the movement amount of the image of the object.
In the object tracking method according to the present invention, the zoom amount of the imaging unit is changed depending on the size of the image of the object.

  In order to achieve the above object, the object tracking device according to the present invention includes an imaging device capable of controlling an imaging direction and a zoom amount, a display device, and a feature amount of an object image in an input image obtained by the imaging device. A detection unit for detecting; a control unit for controlling the imaging device based on the detected feature quantity for tracking the object; and a partial region including the object in the input image based on the feature quantity A setting unit for setting the range of the image, and an enlargement unit for enlarging the image of the range of the set partial area for display on the display device.

In the object tracking device according to the present invention, the feature amount is configured to include at least one of a position, a size, and a movement amount of the image of the object.
In the object tracking device according to the present invention, the setting unit sets a position of the range of the partial area based on the position of the image of the object, and sets the position of the partial area based on the size of the image of the object. Configured to set the size of the range.

  Further, in the object tracking device according to the present invention, the enlargement unit enlarges the image of the partial area range at an enlargement ratio set based on the size of the partial area range and a predetermined image display size. Configured.

  In the object tracking device according to the present invention, the control unit sets an upper limit of the zoom amount of the imaging lens of the imaging device based on the size of the image of the object, and the size of the range of the partial region is , The input image size is set to be smaller than a preset ratio smaller than 1.

In the object tracking device according to the present invention, the control unit is configured to include changing a zoom amount of the imaging lens of the imaging device depending on a movement amount of the object in the image.
In the object tracking device according to the present invention, the control unit is configured to change the zoom amount of the imaging device depending on the size of the image of the object.

  In order to achieve the above object, the computer program according to the present invention performs the following steps in order to perform object tracking by operating an object tracking device provided with an imaging device capable of controlling the imaging direction and the zoom amount. A computer program to be executed, the step of detecting at least one feature quantity of the image of the object in the image obtained from the imaging device, wherein the feature quantity is a position, a size, and a movement amount of the image of the object Controlling the imaging device based on the detected feature amount for tracking the object, and including an image of the object in the image based on the detected feature amount A step of setting a range of the partial area, and a step of enlarging an image of the set range of the partial area.

  In the computer program according to the present invention, the step of setting the range of the partial area may include setting the position of the range of the partial area based on the position of the image of the object; and Setting the size of the range of the partial area based on the size.

  Further, in the computer program according to the present invention, the step of controlling the imaging device includes setting an upper limit of a zoom amount of the imaging lens of the imaging device based on a size of an image of the object. The size of the area range is set to be less than or equal to a preset ratio of 1 to the size of the input image.

  In order to achieve the above object, a computer program realized on a computer-readable medium according to the present invention operates an object tracking device provided with an imaging device capable of controlling an imaging direction and a zoom amount. A computer program implemented on a computer-readable medium that performs the following steps to perform tracking: at least one feature amount of an image of the object in an input image obtained from the imaging device Detecting, the feature amount includes at least one of a position, a size, and a movement amount of an image of the object, and controlling the imaging device based on the detected feature amount for tracking the object Setting a partial area range including the object image in the image based on the detected feature amount; enlarging the set partial area range image; Provided with a step.

  In the computer program realized on the computer-readable medium according to the present invention, the step of setting the range of the partial area sets the position of the range of the partial area based on the position of the image of the object. And setting the size of the range of the partial area based on the size of the image of the object.

  Further, in the computer program realized on the computer-readable medium according to the present invention, the step of controlling the imaging device includes an upper limit of the zoom amount of the imaging lens of the imaging device based on the size of the image of the object. The size of the range of the partial area is set to be equal to or less than a predetermined ratio of the size of the input image.

  According to the object tracking method of the present invention, when an object is tracked using an imaging unit whose imaging direction and zoom amount can be controlled, the imaging unit is controlled based on the feature amount of the object image, and the object image is included. Since the image in the range of the partial area is enlarged, for example, the imaging means can be controlled effectively.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Like members are given like reference numerals.
FIG. 2 shows a hardware configuration example of an image monitoring apparatus to which the object tracking apparatus according to the present invention is applied. The image monitoring apparatus of this example includes an imaging device 201, a processing device 202, an operation device 203, an external storage device 204, an image monitor 205, and a warning lamp 206.

The imaging apparatus includes a TV camera 201a, an imaging lens 201b configured by, for example, a zoom lens, and a camera platform 201c configured by, for example, a swivel base.
The processing device 202 includes an image input unit 202a, a pan head control unit 202b, a lens control unit 202c, an operation input unit 202d, an image memory 202e, an MPU 202f, a work memory 202g, an external input / output unit 202h, The image output unit 202i, an alarm output unit 202j, and a data bus 202k are included.

The operating device 203 includes a joystick 203a, a first button 203b, and a second button 203c.
Specifically, the output of the TV camera 201a is connected to the data bus 202k via the image input unit 202a, and the control unit of the imaging lens 201b is connected to the data bus 202k via the lens control unit 202c. A camera head 201c on which the TV camera 201a is mounted is connected to a data bus 202k through a head control unit 202b, and an output of the operation device 203 is connected to the data bus 202k through an operation input unit 202d.

  The external storage device 204 is connected to the data bus 202k via the external input / output unit 202h, and the monitoring image monitor 205 is connected to the data bus 202k via the image output unit 202i. Is connected to the data bus 202k via an alarm output unit 202j. The MPU 202f, the work memory 202g, and the image memory 202e are directly connected to the data bus 202k.

  Here, the TV camera 201a captures a region to be monitored within a predetermined field of view, captures the region to be monitored, and outputs an image signal. For this reason, the TV camera 201 a includes an imaging lens 201 b and is mounted on the camera platform 201. The image signal captured by the TV camera 201a is stored in the image memory 202e from the image input unit 202a via the data bus 202k.

  The external storage device 204 functions to store programs and data, and the programs and data are read into the work memory 202g via the external input / output unit 202h as necessary. Is stored in the external storage device 204 from the work memory 202g.

  The MPU 202f executes processing according to a program stored in the external storage device 204 and read into the work memory 202g when the processing device 202 is operated, and analyzes the image stored in the image memory 202e in the work memory 202g. Then, the MPU 202f controls the imaging lens 201b via the lens control unit 202c or the camera pan 201c via the pan control unit 202b according to the processing result, and the TV camera 201a. In addition to changing the imaging field of view, an image as a result of detecting an intruding object is displayed on the image monitor 205 as necessary, and the warning lamp 206 is lit.

  In the above-described image monitoring apparatus, the configuration of the imaging apparatus 201, the processing apparatus 202, and the like and the connection format thereof are not limited to the above-described embodiments. For example, a configuration in which the imaging apparatus 201 and the processing apparatus 202 are connected via a network such as the Internet, or image data obtained by digitally compressing an image signal captured by the TV camera 201a is input to the processing apparatus 202. It is possible to use various configurations.

  FIG. 1 shows an example of an object tracking processing procedure using the difference method and the template matching method performed by the image monitoring apparatus of this example. The object tracking process will be described with reference to FIG.

First, in the initialization processing step 101, the image memory 202e and work memory 202g for executing the object tracking processing are initialized.
Next, an intruding object detection process 102 (step 102a to step 102e) by the difference method is performed.

That is, in the first image input processing step 102a, an input image having 320 pixels in width and 240 pixels in height is obtained from the TV camera 201a.
In the difference processing step 102b, a luminance value difference for each pixel is calculated between the input image obtained in the first image input step 102a and a reference background image that is created in advance and does not show an intruding object.

  In the binarization processing step 102c, the pixel value of the pixel in which the pixel value (difference value) of the difference image obtained in the difference processing step 102b is less than a predetermined threshold Th is set to “0”, and the threshold Th A binary image is obtained by setting the pixel value of the above pixel to “255”. Here, for example, Th = 20 is used as the predetermined threshold Th. Further, the pixel value of one pixel is calculated by 8 bits (“0” to “255”).

  In the labeling processing step 102d, a block of pixels having the pixel value “255” in the binarized image obtained in the binarizing processing step 102c is detected so that each can be identified and numbered.

  In the intruding object presence determination processing step 102e, it is determined that there is an intruding object in the monitoring target area when a group of pixels having the pixel value “255” numbered in the labeling processing step 102d satisfies a predetermined condition. . Here, as the predetermined condition, for example, a condition that the width is 20 pixels or more and the height is 50 pixels or more is used.

  If it is determined in step 102e that the intruding object is present, the process proceeds to the alarm / detection information display processing step 103. If it is determined that there is no intruding object, the above-described process is performed again. The process proceeds to the first image input processing step 102a, and the difference method is executed again.

  Here, with reference to FIG. 3, the above-described intruding object detection processing will be specifically described. FIG. 3 schematically shows an outline of an example of a process for detecting an intruding object using the difference method described above and an example of an example of a process for registering an image of the intruding object described later in a template.

  In FIG. 3, 401 indicates an input image obtained in the first image input processing step 102a, and 402 indicates a reference background image that is created in advance and recorded in the image memory 202e. Reference numeral 406 denotes a subtractor representing the processing of the difference processing step 102b, 403 denotes the difference image obtained in the difference processing step 102b, and 407 denotes a binary representing the processing of the binarization processing step 102c. The generator is shown. Reference numeral 404 denotes a binarized image obtained in the binarization processing step 102c.

  Then, the subtractor 406 calculates the difference in the luminance value of the pixel between the input image 401 and the reference background image 402 for each pixel, and outputs a difference image 403. Next, the binarizer 407 performs threshold processing on the difference image 403 for each pixel with the threshold value Th, sets the pixel value less than the threshold value Th to “0”, and the pixels of the pixels that are equal to or greater than the threshold value Th. A binarized image 404 is obtained with the value “255”. As a result, the humanoid object 409 shown in the input image 401 is calculated as a region (image signal changing region) 410 in which a difference is generated by the subtractor 406, and is detected as an image 411 by the binarizer 407.

  Next, the continuation of the processing procedure shown in FIG. 1 will be described. In the alarm / detection information display processing step 103, for example, in order to notify the monitor of an alarm indicating that an intruding object has been detected, the intruding object information is displayed on the image monitor 205 via the image output unit 202j. For example, the warning lamp 206 is turned on via the alarm output unit 202j. Here, as the information on the intruding object, for example, information such as the position and the number of people can be used.

  For example, an intruding object may be superimposed on the input image and displayed on the image monitor 205. Note that the display form of the intruding object superimposed on the input image is the binarized image itself of the intruding object obtained in the binarization processing step 102c, its circumscribed rectangle, or if the intruding object is a human, Various forms are possible, such as attaching a triangle mark or color display.

Next, detection processing 104 (104a to 104f) of the amount of movement of the intruding object by the template matching method is performed.
That is, in the template registration processing step 104a, the image of the intruding object in the input image is cut out based on the circumscribed rectangle 412 of the cluster of pixels having the pixel value “255” numbered in the labeling processing step 102d described above. Register as

  In the second image input processing step 104b, as in the first image input processing step 102a, an input image of, for example, 320 pixels wide and 240 pixels high is obtained from the TV camera 201a. At that time, the focal length of the imaging lens 201b of the TV camera 201a is set to f and recorded in the work memory 202g.

  In the template enlargement / reduction processing step 104c, the focal length f ′ recorded in the work memory 202g, that is, the focal length f ′ of the imaging lens 201b of the TV camera 201a at the time of execution of a template update processing step 104f described later, for example, In accordance with the ratio of the focal length f recorded in the memory 202g, the difference in the size of the tracking target object shown in the template and the input image generated by changing the focal length of the imaging lens 201b is corrected. In this example, the imaging lens 201b is controlled by a pan / lens control processing step 105 described later, and the focal length changes.

  In the template matching processing step 104d, an image having the highest matching degree with the template is detected from the input images obtained in the second image input step 104b. Usually, since it takes a long time to compare the template and the entire input image, an image having the highest degree of matching with the template is detected in the search region with a predetermined range as the search region.

  In the coincidence degree determination processing step 104e, a coincidence degree r (Δx, Δy) described later is determined, and when the normalized correlation value represented by the expression 1 described later is used, for example, the coincidence degree is 0.7 or more If the degree of coincidence is less than 0.7, the process proceeds to the first image input process step 102a.

  Here, the high degree of coincidence means that there is an image similar to the template in the input image, that is, there is an intruding object in the monitoring target area, and its position is viewed from the template position (x0, y0) described later. This means that the position is relatively (Δx, Δy). In this case, the amount of movement of the intruding object is subsequently detected. Further, the low degree of coincidence means that there is no image similar to the template in the input image, that is, there is no intruding object in the monitoring target region. In this case, the first image input processing step The process moves to 102a, and an intruding object is detected again by the difference method.

  In the template update processing step 104f, the input image obtained in the second image input processing step 104b is cut out based on the newly obtained position of the intruding object, and used as a new template image. In this way, by updating the template sequentially, the latest image of the intruding object is recorded in the template, and the amount of movement of the intruding object can be detected stably even when the intruding object changes its posture. .

  Here, the template enlargement / reduction processing step 104c will be described in detail with reference to FIGS. 6A and 6B. 6A and 6B show a case where the template is enlarged, but the same applies to the case where the template is reduced.

  FIG. 6A shows an example of an image 701 before enlargement of the template, and FIG. 6B shows an example of an image 703 after enlargement of the template. The zoom magnification r of the template is expressed as Equation 1.

For example, if the focal length f ′ of the imaging lens 201b of the TV camera 201a at the time of executing the template update processing step 104f is 20 mm and the current focal length f is 24 mm, then r = 24/20 = 1.2. The size of the target object on the image is increased by 1.2 times due to the change in the focal length of the imaging lens 201b. That is, if the template 701 is multiplied by 1.2 so that the center positions 702 and 704 of the template match before and after enlargement, and this result is used as a new template 703, the intruding object in the input image is displayed. The size and the size of the intruding object of the template can be matched.

  In the example of FIGS. 6A and 6B, the length (r × Tx), which is the result of multiplying the length Tx in the X-axis direction of the template 701 before enlargement by the zoom magnification r in the XY orthogonal coordinate system, The length of the template 703 after enlargement is the length in the X-axis direction. Similarly, the length (r × Ty), which is the result of multiplying the length Ty of the template 81 before enlargement in the Y-axis direction by a zoom factor r, is enlarged. The length of the subsequent template 703 in the Y-axis direction is used. At this time, the center positions 702 and 704 of the templates 701 and 703 are not changed.

  Further, immediately after detecting an intruding object in the intruding object detection processing 102 described above, the template update processing step 104f is not executed, so that the focal length f ′ of the imaging lens 201b of the TV camera 201a at the time of template updating is In this case, the template enlargement / reduction processing step 104c is not executed because it has not been acquired.

  Further, when the template enlargement / reduction processing step 104c is performed as in this example, the template update processing step 104f uses the focal length f ′ recorded in the work memory 202g at the time of execution of the process of the TV camera 201a. Updating is performed using the focal length f of the imaging lens 201b.

  Here, with reference to FIG. 3 and FIG. 4, the above-described detection processing of the amount of movement of the intruding object will be described in detail. FIG. 3 shows the cutout unit 408 and the template image 405.

  Then, the intruding object reflected in the input image 401 is cut out based on the circumscribed rectangle 412 of the intruding object 411 obtained as a cluster of the pixel value “255” in the binarized image by the labeling processing step 102d. A template image 405 is obtained by cutting out with the device 408. The template image 405 includes a template 413 of the intruding object 409, which becomes an initial template in the detection processing of the intruding object movement amount by the template matching method. Next, template matching is executed based on the initial template.

FIG. 4 schematically shows an example of a flow of processing for tracking the intruding object by sequentially executing the detection processing of the moving amount of the intruding object by the template matching method.
In FIG. 4, the time when the template image 501 is acquired is represented as t0, and the acquisition times of input images that are sequentially input at a predetermined time interval (for example, 100 ms) are (t0 + 1), (t0 + 2),. Represent.

  In FIG. 4, reference numeral 501 denotes a template image at time t0. The template image 501 includes a template 501a at time t0. These are the same as the template image 405 and the template 413 shown in FIG.

  Reference numeral 502 denotes an input image at time (t0 + 1). In the input image 502, the rectangular area 502b represents the position of the intruding object (the position of the template 501a) at time t0, and the rectangular area 502c represents the area (search area) to be subjected to template matching.

  Then, when the template matching process (step 104d) 509 is executed, the degree of coincidence becomes the largest in the image 502a that most closely matches the template 501a in the template matching search area 502c, and the intruding object has an image of the image 502a at time (t0 + 1). It can be seen that it exists at the position. This position is represented by a relative position (Δx, Δy) when viewed from the position (x0, y0) of the template 501a at time t0. That is, it can be seen that the intruding object has moved by the amount indicated by the arrow 502d.

  Thus, the template update process (step 104f) 510 updates the image 502a that most closely matches the template 501a as a new template at time (t0 + 1). That is, as shown in FIG. 4, the position 502a of the intruding object is cut out from the input image 502, and this is used as the template image 503, and the intruding object image 502a is updated as a new template 503a at time (t0 + 1).

  This process is applied to input images sequentially input from the TV camera 201a. Specifically, as shown in FIG. 4, a search region 504c is set based on the position 504b of the template 503a in the input image 504 at time (t0 + 2), and the template 503a in the template image 503 at time (t0 + 1). The position 504a of the intruding object is detected by template matching processing (step 104d) 509 using Then, it can be seen that the intruding object has moved as indicated by an arrow 504d.

  Further, as shown in FIG. 4, the template image 505 and the template 505a of the intruding object at the time (t0 + 2) are updated by the template update process (step 104f) 510.

  As shown in FIG. 4, a search area 506c is set in the input image 506 at the time (t0 + 3) based on the position 506b of the template 505a, and the template 505a in the template image 505 at the time (t0 + 2) is used. A template matching process (step 104d) 509 detects the position 506a of the intruding object. Then, it can be seen that the intruding object has moved as indicated by the arrow 506d.

  Furthermore, as shown in FIG. 4, the template update process (step 104f) 510 updates the template image 507 and the intruding object template 507a at time (t0 + 3).

  Then, as shown in FIG. 4, a search area 507c is set based on the position 508b of the template 507a in the input image 508 at time (t0 + 4), and the template 507a in the template image 507 at time (t0 + 3) is used. A template matching process (step 104d) 509 detects the position 508a of the intruding object. Then, it can be seen that the intruding object has moved as indicated by an arrow 508d.

In this way, an intruding object can be tracked by sequentially executing template matching.
Here, the search area and the degree of coincidence in the template matching process (step 104d) will be specifically described. The range of the search area described above is determined by, for example, the movement of the target object registered in the template on the input image.

  As a specific example, a condition that a 1/3 inch CCD (imaging device size: 4.8 mm × 3.6 mm) is used as the imaging device 201, the focal length of the imaging lens 201b is 32 mm, and the distance to the target object is 30 m. When the image is taken, the horizontal field of view of the TV camera 201a is 30 × 4.8 ÷ 32 = 4.5 m. When a target object with a moving speed of 5 km / h (about 1.39 m / s) is imaged with this TV camera 201a at an image size of 320 × 240 pixels and an input interval of 0.1 s (100 ms), the input image of the target object The moving amount on each image is 320 × 1.39 × 0.1 / 4.5≈9.88 pixels in the horizontal direction.

  Further, when the target object moves in the direction of the TV camera 201a, the amount of movement on the image also increases. Therefore, the actual search area range is set with a margin of about five times the value calculated above. That is, the horizontal size Mx of the search area is 50 pixels. The vertical size My of the search area depends on the elevation angle of the TV camera 201a, and changes depending on the mounting position of the TV camera 201a. Therefore, the search region has a value about 40% of the horizontal size. Therefore, in this example, the search range may be an area that is widened by Mx = 50 pixels on the left and right and My + 20 on the top and bottom with respect to the template.

  As the degree of coincidence, for example, a normalized correlation value r (Δx, Δy) can be applied, and is expressed as Expression 2.

Here, f (x, y) represents an input image. Also, referring to FIG. 5 described later, g (x, y) represents a template image 601, (x0, y0) represents the upper left coordinates of the template 602, and D represents the size of the template. Represents. In this example, the upper left corner of the image is the origin (0, 0) as the coordinate axis of the image. Referring to FIG. 3, D corresponds to the size of the circumscribed rectangle 412 of the intruding object detected in the binarized image 404. In this example, D corresponds to 50 pixels horizontally and 20 pixels vertically.

The normalized correlation value r (Δx, Δy) takes a value of −1 ≦ r (Δx, Δy) ≦ 1, and is “1” when the input image and the template completely match.
In template matching, when Δx and Δy are scanned within the search range, that is, in the above example, when −Mx ≦ Δx ≦ Mx and −My ≦ Δy ≦ My, the normalized correlation value r ( A process of detecting a position (Δx, Δy) where Δx, Δy) is largest is performed.

  Next, the continuation of the processing procedure shown in FIG. 1 will be described. In the focal length information acquisition processing step 105, the focal length f of the imaging lens 102b at the time when the current input image recorded in the work memory 202g is acquired is acquired.

  Next, in the camera platform / lens control step 106, the camera is selected according to the displacement between the position of the intruding object detected by the template matching processing step 104d in the detection processing 104 for the amount of movement of the intruding object and the center of the input image. The pan head 201c is controlled. Also, a new focal length (zoom magnification) is calculated according to the size of the detected intruding object on the image and the focal length of the TV camera 201a (obtained in step 105) at that time, and the focal point of the imaging lens 201b. Control the distance (zoom). The calculation of the zoom magnification will be described later.

  Here, with reference to FIG. 5, the control process of the camera pan head 201c described above will be specifically described. As an example, it is assumed that an intruding object is detected at a position 602 as shown in FIG. 5 in the template image 601. In this case, assuming that the center position of the intruding object is the center 603 of the template, the displacement dx in the X-axis direction and the displacement dy in the Y-axis direction from the center 604 of the template image 601 are calculated.

  If the center position 603 of the template is a predetermined amount s or more on the left side (dx <−s) compared to the center 604 of the input image, the camera head 201c is rotated (panned) to the left, and the right side (dx) > S), the camera head 201c is rotated (panned) to the right. If the center position 603 of the template is higher than the center 604 of the input image by a predetermined amount s or more (dy <−s), the camera platform 201c is tilted upward (tilt), and is lower than the predetermined amount s. If (dy> s), tilt down.

  When such a predetermined amount s is used, it is not necessary to control the camera platform 201c when an intruding object is present near the center of the image. Therefore, the control of the camera platform 201c is started by the predetermined amount s. The position of the intruding object to be specified can be specified. Note that various values may be used as the predetermined amount s for each of left, right, upper, and lower. For example, the same value may be used in the left, right, up, and down directions, or in the left, right, up, and down directions, respectively. Any value may be used.

  As an example, it is possible to use a value of a predetermined amount s = 50 in the left and right and up and down directions. Further, for example, as the predetermined amount s is smaller, there is a possibility that the camera pan head 201c is controlled if the intruding object deviates from the center even a little, and the image becomes difficult to see, but the predetermined amount s = 0 described above. It is also possible to use a small value as the value or the predetermined amount s.

  It is also possible to perform control such that the control speed of the panning or tilting motor is changed by the absolute values of the displacement dx in the X-axis direction and the displacement dy in the Y-axis direction with respect to the center 604 of the template image 601. . In this case, for example, the control speed is increased as the absolute value of the displacement dx in the X-axis direction or the displacement dy in the Y-axis direction is larger.

  Next, the control process of the imaging lens 201b described above will be specifically described. In the control of the imaging lens 201b, for example, based on the size of the detected intruding object on the image, that is, the size of the template, for example, the height of the template is less than a predetermined value (or below a predetermined value). If there is any), the image pickup lens 201b is zoomed in. If the height of the template is equal to or greater than a predetermined value (or exceeds a predetermined value), the image pickup lens 201b is zoomed out. As an example, a value of 400 pixels (when the size of the input image is 640 pixels in the horizontal direction and 480 pixels in the height direction) can be used as the predetermined value. In this case, for example, if the current template height is 300 pixels, and the focal length of the current zoom lens 201b recorded in the work memory 202g is f = 30 mm, the template height is 400 pixels. The focal length of the zoom lens may be f = 30 × (400/300) = 40 mm. That is, the zoom ratio may be increased by 1.33. Accordingly, the MPU 202f controls the focal length of the zoom lens 201b to 40 mm via the lens control unit 202c. In this way, an intruding object can be captured with an appropriate size within the field of view of the TV camera 201a.

  As an alternative example, the zoom lens 201b can be controlled by a simple process in which the focal length is increased by 1.0 mm as zoom-in and the focal length is reduced by 1.0 mm as zoom-out. Since the process of tracking the intruding object is repeatedly performed sequentially, even in this simple process, if the control of the focal length is insufficient, the control is similarly performed in the next processing frame. Therefore, by repeating the process of tracking the intruding object, the focal length of the zoom lens 201b is controlled to an appropriate value, and the height of the template can be adjusted to a predetermined size. Here, the change in focal length of 1.0 mm is calculated empirically. If this value is large, the height of the template can be quickly adjusted to a predetermined size. Near the value, there is a possibility that the focal length vibration decreases (over damping). On the other hand, if the amount of change in focal length is small, it may take time until the height of the template matches a predetermined size (under damping).

  In the above example, the height of the template is used to determine the size of the target object on the image. The reason is that the intruding object appears in a portrait in most cases and is input by the imaging device 201 for that. This is because the image is horizontally long. That is, when an intruding object is captured within the field of view of the imaging device 201, the distance from the upper and lower ends of the intruding object to the upper and lower ends of the input image is compared with the upper and lower ends of the input image. The interval is narrower. Therefore, when the size of the target object on the image is determined using the width of the template and the imaging lens 102b is controlled based on the result to zoom in, the top and bottom of the intruding object jumps out of the field of view on the image. This is because there is a possibility that such a problem may occur.

  Accordingly, when evaluating the size of the intruding object on the image, in the example where the focal length of the imaging lens 102b is adjusted based on the height of the template, the control of the stable camera pan 201c is maintained. Zoom in and out. In addition to the height of the template, for example, the focal length of the imaging lens 102b can be adjusted based on the width of the template. For example, when the intruding object is horizontally long like a car, the width of the template can be used.

As a result, it is possible to automatically control the camera platform 201c and track the intruding object while capturing the intruding object in the center of the visual field of the TV camera 201a.
Note that the imaging lens 201b can be controlled based on an element other than the above-described size of the intruding object on the image, for example, an amount of movement of the intruding object on the image. Specifically, when the amount of movement of the intruding object is less than a predetermined value (or less than the predetermined value), the imaging lens 201b is zoomed in, and the amount of movement of the intruding object on the image is If it is greater than or equal to a predetermined value (or exceeds a predetermined value), the imaging lens 201b is zoomed out. The control of the imaging lens 201b based on the amount of movement of the intruding object on the image will be described later in connection with another embodiment.

  In the image cut-out processing step 107 and the image enlargement processing step 108 described below, a partial area range including the video of the intruding object is set in the input image, and image processing (enlargement processing) is performed on the partial area image within the set range. ).

First, in the image cutout processing step 107, a partial image of the input image is cut out according to the position of the intruding object and the size of the template (the position, size, etc. of the partial image are set).
Here, with reference to FIG. 10, the partial image clipping process will be described in detail. FIG. 10 shows the relationship between the template 1101 of the intruding object detected in the input image at a certain time and the partial image 1103 to be cut out. In FIG. 10, the size of the partial image, that is, the horizontal Sx pixel and the vertical Sy pixel are expressed as Equation 3 as an example.

Here, Tx is the horizontal size (width) of the template, and Ty is the vertical size (height) of the template. Accordingly, the height Sy of the partial image is set to be 120% of the vertical size Ty of the template in the example shown in Expression 3. Here, the value of 120% is an example, and a value smaller than the vertical size Ty of the template, such as 80%, can be set as the height Sy of the partial image.

  Further, as an example, the partial image width Sx is set according to the aspect ratio of the image monitor 205 that outputs the enlargement result. For example, when the aspect ratio of the image monitor 205 is 4: 3, the width Sx of the partial image is set to 4/3 times the set height Sy of the partial image as shown in Expression 3. .

  In FIG. 10, the partial image clipping range 1103 is set with reference to the center (coordinates (Cx, Cy)) 1102 of the template. That is, in FIG. 10, the upper left coordinates (x0, y0) of the partial image cutout range 1103 are (Cx−Sx / 2, Cy−Sy / 2), and the lower right coordinates (x1, y0) of the partial image cutout range 1103. y1) is set as (Cx + Sx / 2, Cy + Sy / 2). In this way, the center of the template can coincide with the center of the partial image.

  In the above-described Expression 3, the size (height and width) of the partial image is set based on the vertical size Ty of the template. However, the size of the partial image is set based on the horizontal size Tx of the template. May be set.

  Next, the continuation of the process shown in FIG. 1 will be described. In the image enlargement processing step 108, the partial image obtained in the image cutout processing step 107 is enlarged to a size that can be output by the image monitor 205 in order to output it to the image monitor 205. That is, the size of the output image signal can be set to a desired size according to the screen size of the image monitor 205 or the like. For example, when the image monitor 205 can output an NTSC image signal, as an example, the image monitor 205 is enlarged to 640 pixels wide and 480 pixels high. In this case, the horizontal direction may be enlarged by 640 / Sx times, and the vertical direction may be enlarged by 480 / Sy times, but various values can be used as magnifications. It is also possible to set an upper limit value and a lower limit value for the magnification of enlargement. In the above example, the case where the image is enlarged to a certain size of 640 pixels in the horizontal direction and 480 pixels in the vertical direction has been described. However, the image size after the enlargement may not be a constant size. The enlarged image is displayed on the image monitor 205 in an alarm / tracking information display processing step 109 described later.

  Next, in the alarm / tracking information display processing step 109, the image enlarged in the image enlargement processing step 108 is displayed on the image monitor 205 via the image output unit 202i. In addition, for example, in order to transmit an alarm indicating that the intruding object is being tracked to the monitor, information on the intruding object is displayed on the image monitor 205 via the image output unit 202i, or via the alarm output unit 202j. For example, the warning lamp 206 is turned on. Here, as the information on the intruding object, for example, information such as a movement amount and a movement route can be used.

  As described above, according to the present embodiment, the low response of the camera head / imaging lens in mechanical / optical tracking is based on electronic image segmentation processing (step 107) and image enlargement processing (step 108), a wide-angle imaging device required for electronic tracking is supplemented by a mechanical camera pan head control process (step 106), and the low resolution is optical imaging lens control process (step 106), the tracking process can be performed while the intruding object is captured at the center of the image at a high speed and output to the image monitor with the highest possible resolution.

  Next, with reference to FIG. 9, the effect of a present Example is demonstrated concretely. FIG. 9 shows an example when the present embodiment is applied to the input image at the same time as the tracking processing result of the intruding object shown in FIG. In FIG. 9, the input images at times t1, (t1 + 1), (t1 + 2), and (t1 + 3) are 1001, 1003, 1005, and 1007 (same as the input images 801, 802, 803, and 804), and the image monitor at that time The enlargement results of the partial images output to 205 are 1002, 1004, 1006, and 1008, respectively.

  9, the positions 1001a, 1003a, 1005a, and 1007a of the intruding objects are the positions 801a, 802a, 803a, and 804a of the intruding objects in FIG. 8 and the templates 801b, 802b, and 803b, respectively. Same as 804b.

  In FIG. 9, an input image 1001 is obtained at time t1, and an intruding object 1001a is detected. At this time, in the image cutout process (step 107), the partial image 1001c is cut out as described above according to the template 1001b of the intruding object. In the image enlargement process (step 108), the partial image 1001c is enlarged for display on the image monitor 205. Then, a display result 1002 is displayed on the image monitor 205. Here, the intruding object is displayed at the center of the screen like 1002a (step 109).

  Next, at time (t1 + 1), the partial image 1003c is cut out (step 107), enlarged (step 108), and displayed on the image monitor 205 as a display result 1004 (step 109). Further, at time (t1 + 2), the partial image 1005c is cut out (step 107), enlarged (step 108), and displayed as a display result 1006 (step 109). At time (t1 + 3), since the partial image to be cut out has the same size as the input image, the input image 1007 becomes the display result 1008 as it is.

  Note that, for example, the size of the partial image to be cut out can be suppressed to 60% of the size of the input image so that the size of the partial image to be cut out is always smaller than the size of the input image. At that time, in the control of the imaging lens 201b, as an upper limit for zooming in, a value at which the template height is 400 pixels is used as an upper limit as an example, but a smaller value, for example, the template height is input. It is also possible to use a value that is 1/2 the height of the image, that is, 240 pixels as the upper limit of zoom-in. In this case, in the image enlargement processing step 108 described above, the partial image must be enlarged. However, since the input image is larger than the partial image to be cut out (the size of the template), The interval between the upper and lower ends and the left and right ends of the input image can be increased. Therefore, the possibility that the top, bottom, left and right of the intruding object will jump out of the field of view on the input image is reduced, and the intruding object tracking performance can be improved. Further, since the intruding object can be tracked by changing the cutout position of the partial image according to the movement of the intruding object, a high-speed response to the object movement can be realized.

  In addition, by correcting the position of the partial image to be cut out by the movement amount (Δx, Δy) of the intruding object, the intruding object can be tracked while following the movement of the intruding object, and the oversight of the intruding object on the display can be reduced. Can do. In this case, the upper left coordinates (x0, y0) of the partial image cutout range 1103 are (Cx−Sx / 2 + Δx, Cy−Sy / 2 + Δy), and the lower right coordinates (x1, y1) of the partial image cutout range 1103 are (Cx + Sx / 2 + Δx, Cy + Sy / 2 + Δy).

  Therefore, according to the present invention, the detected intruding object is captured in the center of the screen while suppressing the influence of the low response of the camera head and the imaging lens, and the resolution of the intruding object also depends on the focal length change of the imaging lens. High-resolution video can be displayed gradually.

  In the process shown in FIG. 10 above, the image cut-out position is determined based on the center of the template. For example, the position of the partial image is determined based on a location other than the center of the template, for example, above the center of the template. It is also possible to perform cutout processing. At this time, there is a possibility that the template protrudes from the display result, but since the target image of the processing apparatus is the input image before being cut out, the object tracking process is not disabled. For example, when the reference is set above the center of the template, a partial image centered on the face of the intruding object is obtained. Further, if only the vicinity of the face of the intruding object is cut out as a partial image, an image in which only the face of the intruding object is enlarged can be output to the image monitor 205.

  In this embodiment, the image enlargement processing step 108 is executed so as to keep the size of the intruding object on the display image of the image monitor 205, but the size of the intruding object on the input image is set. If the height is small, the resolution of the displayed image may be significantly reduced by the image enlargement process. In such a case, it is possible to set a lower limit for the width Sx and height Sy of the partial image in the partial image cutout processing step 107. For example, if the lower limits of the width Sx and the height Sy of the partial image are set to 160 and 120 pixels, respectively, the enlargement magnification in the image enlargement processing step 108 is the maximum, 640 / Sx = 640/160 = 4 times, 480 / Sy = 480/120 = 4 times, and enlargement of 4 times or more is not performed. Therefore, although the size of the intruding object on the display image is not constant, it is possible to suppress a decrease in the resolution of the display image.

  In addition to the partial image enlarged by the image enlargement process (step 108), the image monitor 205 may display an image that has not been electronically enlarged, that is, an input image in the image input processes 102a and 104b. Is possible. At this time, the input image and the enlarged partial image can be displayed side by side on the image monitor 205. It is also possible to display an image obtained by reducing the input image so as to overlap the enlarged partial image.

Another embodiment of the present invention is described below.
In the present invention, it is also possible to control the imaging lens 201b based on an element other than the above-described size of the intruding object on the image, for example, an amount of movement of the intruding object on the image. In this embodiment, the imaging lens 201b is controlled based on the amount of movement of the intruding object on the image. Specifically, when the amount of movement of the intruding object is less than a predetermined value (or less than the predetermined value), the imaging lens 201b is zoomed in, and the amount of movement of the intruding object on the image is If it is greater than or equal to a predetermined value (or exceeds a predetermined value), the imaging lens 201b is zoomed out. This process will be described below with reference to FIGS. 11 and 12.

  FIG. 11 shows another example of the procedure of the object tracking process using the difference method and the template matching method performed by the image monitoring apparatus of FIG. Note that steps 101 to 104 (and 105) in the procedure shown in FIG. 11 are the same as those in FIG. FIG. 12 is a diagram for explaining an example of setting the zoom magnification in the present embodiment.

As in FIG. 1, in step 105, the focal length f of the imaging lens 201b recorded in the work memory 202g when the current input image is acquired is acquired.
Next, based on the movement amount (Δx, Δy) of the intruding object obtained in the template matching process (step 104), the zoom magnification rf is calculated by equation 4 (step 110).

In Expression 4, Mx and My represent search ranges in the template matching method as already described. Kx and Ky represent the maximum movement amount on the image of the intruding object that can be stably tracked. For example, about half of the search range, that is, in this example, Mx = 50 and My = 20. , Kx = 25 and Ky = 10. It should be noted that the values of Kx and Ky may be values that satisfy the requirement of, for example, providing a margin at about half of the search range so that the object does not deviate from the search range. To set.

In Expression 4, when the moving amount of the intruding object is (Δx, Δy) = (0, 0), that is, when the moving amount of the intruding object is zero, the zoom magnification rf = 1.5. Yes.
Note that, for example, when the zoom magnification rf is equal to or greater than a predetermined value, the zoom magnification rf may be set to the predetermined value so as not to zoom in rapidly. As the predetermined value, for example, 1.5 can be used. In this case, it is possible to zoom in up to 50% with one zoom-in.

  In this way, when the maximum zoom-in magnification rf (upper limit value) in one zoom-in is set, for example, an object detected near the end of the input image jumps out of the field of view on the image due to zoom-in. Can be suppressed.

  Further, for example, a configuration in which the upper limit value of the zoom magnification rf is variable can be used. In such a configuration, for example, when the template is too small compared to the screen size of the input image, the upper limit value of the zoom magnification rf can be set to a value larger than 1.5.

  Further, for example, zoom-in may be limited based on the height of the template. For example, zoom-in may be performed only in a range where the image screen height is 120% or more of the template height, or the image screen width is 120% or more of the template width. You may make it zoom in only within the range. As a result, the amount of movement of the intruding object (Δx, Δy) is small, and it is possible to prevent the template from exceeding the size of the screen due to multiple zoom-ins, making it easy to see the image and stable operation. Can be secured.

  In addition, an upper limit may be set for zooming in based on the distance between the template and the upper end, lower end, left end, and right end of the screen. For example, as shown in FIG. 12, the distances between the template 172 and the top, bottom, left, and right edges of the screen 171 are du, db, dl, and dr, respectively, and the top, bottom, left, and right sides of the template 172 are zoomed in on the screen. Zoom out to the smallest magnification except for negative values of 120 / (120-du), 120 / (120-db), 160 / (160-dl), 160 / (160-dr) The upper limit of. Here, the screen size is assumed to be 320 pixels wide and 240 pixels high.

  Further, when the movement amount of the target object is predicted, when the movement amount of the target object in the previous frame is (Δx ′, Δy ′), 120 / [120− (du + Δy ′)], 120 / [ 120− (db−Δy ′)], 160 / [160− (dl + Δx ′)], 160 / [160− (dr−Δx ′)], the upper side, the lower side, the left side, and the right side of the template in the same manner as described above. Magnification that exceeds the outside of the screen by zooming in (upper limit of zooming-in magnification) is calculated.

  Note that, for example, the upper limit of the zoom-in magnification may be calculated based only on the shorter one of the distances du and db between the template 172 and the upper and lower ends of the screen 171, and the left and right edges of the template 172 and the screen 171 The upper limit of the zoom-in magnification may be calculated based only on the shorter one of the distances dl and dr.

  As described above, the zoom magnification of the imaging lens 201b is calculated based on the amount of movement of the intruding object on the image, and the focal length of the imaging lens 201b is set to f via the lens control unit 202c in the pan / lens control step 106. It will be adjusted to xrf.

  This is the end of the description of controlling the imaging lens 201b based on the amount of movement of the intruding object on the image. Steps 107-109 are the same as in the embodiment of FIG.

  In the above embodiment, the difference method is used as an example of a method for detecting an object from an image, and the template matching method is used as an example of a method for detecting the amount of movement of an object. For example, as in this example, Various methods may be used as long as they can be tracked while detecting the amount of movement of the intruding object.

Moreover, although the said description was made | formed about the Example, it is clear to those skilled in the art that this invention can make a various change and correction in the range of the mind and the claim.
For example, the configurations of the object tracking device and the image monitoring device according to the present invention are not necessarily limited to those described above, and various configurations may be used.

Furthermore, for example, in the embodiments, the zoom lens has been described as the imaging lens of the imaging apparatus, but the present invention is not limited to this, and various types may be used.
In the imaging lens control step, the imaging means performs imaging so that the size of the object in the input image satisfies a predetermined range, or the movement amount of the object in the image satisfies a predetermined value range. The zoom magnification of the lens may be calculated.

Various setting methods for partial image setting may be used. In addition to the means for electronically enlarging the image, various types of image enlargement means may be used.
Various modes may be used as a mode for controlling the imaging lens of the imaging unit based on the detection result by the object detection unit in the input image. Various methods may be used for calculating the zoom magnification of the imaging lens of the imaging unit based on the detection result of the object detection unit in the input image.

  Various methods may be used as a method of controlling the imaging lens of the imaging unit based on the calculation result of the zoom magnification. For example, the imaging lens of the imaging unit is used so as to realize the calculated zoom magnification. A way of moving can be used.

  The size and amount of movement of an object in the image correspond to, for example, the size of the object and the amount of movement of the object in the frame of the image. For example, the detection is performed based on the number of pixels constituting the frame. Is possible. In addition, various values may be used as the predetermined range related to the size of the object in the image. For example, it may be set in consideration that the size of the object is not too large in the frame. it can. Various values may be used as the predetermined range related to the amount of movement of the object in the image. For example, the predetermined range may be set in consideration of preventing the movement speed of the object from being too fast in the frame. it can.

  Therefore, according to the object tracking method and the object tracking device of the above embodiment, when tracking an object in an image based on an image signal obtained by imaging, control of an imaging lens that performs imaging and electronic enlargement processing of the image Thus, for example, the image of the object can be acquired with sufficient resolution by following the movement of the object at a sufficiently high speed.

  Here, the present invention can be provided as, for example, a method or method for executing the processing according to the present invention, a program for realizing such a method or method, and the like, for example, an object tracking device, etc. It can also be provided as various devices and systems.

The application field of the present invention is not necessarily limited to the above-described fields, and the present invention can be applied to various fields.
In addition, as various processes performed in the object tracking device and the image monitoring device according to the present invention, for example, a control program stored in a ROM (Read Only Memory) in a hardware resource including a processor, a memory, etc. A configuration controlled by execution may be used, and for example, each functional unit for executing the processing may be configured as an independent hardware circuit.

  Further, the present invention can also be grasped as a computer-readable recording medium such as a flexible disk or a CD (Compact Disc) -ROM storing the above control program, or the program (itself). Then, the processing according to the present invention can be performed by causing the processor to input to the computer.

It is a figure which shows an example of the procedure of the object tracking process performed by the image monitoring apparatus which concerns on one Example of this invention. It is a figure which shows the structural example of the image monitoring apparatus which concerns on one Example of this invention. It is a figure which shows the outline of an example of the process which detects an intruder using a difference method, and the outline of an example of the process which registers the image of the said intruding object into a template. It is a figure which shows an example of the flow of a process which performs the detection process of the movement amount of the intruding object by a template matching method sequentially, and tracks an intruding object. It is a figure which shows an example of the operation | movement which controls a camera platform based on the position of a detection object. It is a figure which shows an example of the mode of expansion / reduction of a template. It is a figure which shows an example of the mode of expansion / reduction of a template. It is a figure which shows the outline of an example of the operation | movement which controls a camera pan head and an imaging lens. It is a figure which shows an example of the mode of the process which expands a part of input image electronically. It is a figure which shows an example of the mode of the object tracking process performed by the image monitoring apparatus which concerns on one Example of this invention. It is a figure which shows an example of the mode of the extraction process of a partial image. It is a flowchart which shows the procedure of the object tracking process by another Example of this invention. It is a figure for demonstrating an example of the zooming magnification setting of the imaging lens in the Example of FIG.

Explanation of symbols

201: Imaging device, 201a: TV camera, 201b: Zoom lens, 201c: Camera head, 202: Processing device, 202a: Image input unit, 202b: Head control unit, 202c: Lens control unit, 202d: Operation input unit 202e: image memory, 202f: MPU, 203g: work memory, 202h: external input / output unit, 202i: image output unit, 202j: alarm output unit, 202k: data bus, 203: operation means, 203a: joystick, 203b: Button 203c: Button 204: External storage device 205: Image monitor 206: Warning light

Claims (7)

An object tracking method for tracking an object using an imaging means capable of controlling an imaging direction and a zoom amount,
Detecting at least one feature quantity of the image of the object in the input image obtained from the imaging means;
For tracking the object, controlling the imaging means based on the detected feature amount,
Setting a range of a partial region including the image of the object in the input image based on the detected feature amount;
Magnifying an image of the set partial area range;
An object tracking method characterized by the above. The object tracking method according to claim 1,
The at least one feature amount includes at least one of a position, a size, and a movement amount of the image of the object.
An object tracking method characterized by the above. The object tracking method according to claim 2,
Setting the position of the range of the partial region based on the position of the image of the object;
Setting the size of the range of the partial area based on the size of the image of the object;
An object tracking method characterized by the above. The object tracking method according to claim 3,
Magnifying the image of the partial area range at an enlargement ratio set based on the size of the partial area range and a predetermined image display size;
An object tracking method characterized by the above. The object tracking method according to claim 3,
An upper limit of the zoom amount of the imaging lens of the imaging unit is set based on the size of the image of the object, and the size of the range of the partial area is less than a preset ratio of 1 of the size of the input image Set to be
An object tracking method characterized by the above. The object tracking method according to claim 2,
Changing the zoom amount of the imaging means depending on the amount of movement of the image of the object;
An object tracking method characterized by the above. The object tracking method according to claim 2,
Changing the zoom amount of the imaging means depending on the size of the image of the object;
An object tracking method characterized by the above.