JP2007036782A - Moving object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving object detection device capable of detecting a moving object in a detection region without using camera pan / tilt / zoom control parameter values.
A moving object detection device according to the present invention includes a stationary point information storage unit 102b that stores information on a plurality of stationary points that are stationary in an image space captured by an imaging device, and a mutual relationship between a plurality of stationary points. A detection region tracking unit 104 that performs coordinate transformation of the image space based on the positional relationship and tracks the position of a predetermined detection region that moves on the image as the imaging device moves, and detection based on the luminance change of the detection region A moving object determination unit 105 that determines whether or not an object to be imaged in the region is a moving object, and calculates a coordinate conversion parameter by tracking a stationary point, whereby a camera pan / tilt / zoom control parameter value Even without using, it is possible to detect a moving object in the detection area, and to detect a moving object in a scene where the camera is moving.
[Selection] Figure 1

Description

  The present invention relates to a moving object detection device, and more particularly to a moving object detection device that tracks a predetermined detection area.

  Due to the development of image processing technology and significant performance improvement of processors, in recent years, even inexpensive recorders and surveillance cameras are equipped with image recognition functions such as moving object detection, and only moving images are acquired. In addition, it has been applied to the realization of searching for a moving image from the already acquired moving images.

  Detection of a moving object by image processing mainly includes a method using an image background difference and an inter-frame difference and a method using a motion vector.

  A general moving object detection method based on background difference will be described. In the moving object detection method based on background difference, an image in a state where no moving object exists is acquired and used as a background image. The difference between the detection target image and the luminance value of the background image is calculated, and the moving object is detected. This is performed for each pixel in a region to be performed (hereinafter referred to as “detection region”). When the sum of the differences of the calculated pixels is greater than or equal to a certain threshold value, it is determined that there is a moving object.

  Next, a general moving object detection method based on the interframe difference will be described. In the moving object detection method based on the inter-frame difference, the difference between the image of a certain frame and the image of the previous frame is calculated for each pixel in the detection region, and the sum of the differences exceeds a certain threshold. If it is determined that there is a moving object.

  According to the detection method of the moving object based on the background difference described above, a very stable detection result can be obtained when the same background is always guaranteed. There are few environments where the background is always constant due to the addition. On the other hand, the detection method of the moving object based on the difference between the frames described above determines the detection of the moving object using the difference between the two images captured in a short time, so the effects of environmental changes and the like are compared. Even if a new object is added as a background, it will be treated as part of the background and will not continue to be detected unless it has just been added. ing.

  However, in both of the above methods, when the camera performs pan / tilt / zoom operations, etc., the image changes greatly, so that it is detected as a moving object. In addition, as the camera moves, the coordinates of the detection area for detecting the moving object on the image also change. In order to obtain these coordinates, the movement information of the camera is necessary.

Therefore, an invention relating to a masking method for privacy protection in a camera capable of pan / tilt / zoom operations of the camera has been proposed (see Patent Document 1). The invention described in Patent Document 1 is a technique for calculating the coordinate value on the image of the masking area from the pan / tilt / zoom control parameters of the camera and always appropriately masking the area to be masked.
JP 2004-15362 A

  However, in the invention described in Patent Document 1, a pan / tilt / zoom control parameter for controlling pan / tilt / zoom of the imaging apparatus is always necessary. Therefore, the invention described in Patent Document 1 cannot be applied when control parameters cannot be acquired, for example, when an already acquired moving image is reproduced. In addition, in order to detect moving objects while moving the camera using image processing, it is necessary to accurately associate detection areas between multiple frames of images, but detection calculated from pan / tilt / zoom control parameters The accuracy is low in the region.

  The present invention has been made in view of such problems, and proposes a moving object detection method that keeps the detection area constant even when the camera performs pan / tilt / zoom operations and has a small amount of calculation. . Further, even when there is no camera movement information in the already acquired moving image, the moving object detection is realized.

  The moving object detection device of the present invention is based on a stationary point information storage unit that stores information on a plurality of stationary points that are stationary in an image space captured by the imaging device, and a mutual positional relationship between the plurality of stationary points. A detection area tracking unit that performs coordinate conversion of the image space and tracks a position of a predetermined detection area that moves on the image as the imaging device moves, and the detection area based on a luminance change of the detection area A moving object determination unit that determines whether or not the object to be imaged is a moving object.

  According to this configuration, it is possible to detect a moving object in the detection region without using the pan / tilt / zoom control parameter values of the camera by performing coordinate conversion based on the positional relationship between the stationary points. The moving object detection of the scene in which the camera is moving can be realized.

  In addition, the moving object detection device of the present invention includes a stationary point setting unit that sets, as the stationary point, a characteristic point that can be distinguished from other points based on pixel information of an image captured by the imaging device.

  According to this configuration, since the stationary point is distinguished from other points, the stationary point can be easily identified.

  In the moving object detection device of the present invention, the stationary point setting unit determines whether or not the amount of change in the pixel information exceeds a predetermined threshold in a state where the imaging device is fixed.

  According to this configuration, the stationary point can be accurately set by detecting the stationary point.

  Further, the moving object detection device of the present invention includes the plurality of stationary points that move on the image as the imaging device moves based on the information on the plurality of stationary points stored in the stationary point information storage unit. A stationary point tracking unit for tracking the position of

  According to this configuration, by tracking the position of the stationary point, it is possible to grasp the mutual positional relationship between the stationary points and perform coordinate conversion.

  In the moving object detection device of the present invention, the moving object determination unit calculates a change in luminance of the detection area, and when the luminance change exceeds a predetermined threshold, the object imaged in the detection area is a moving object. It is judged that.

  According to this configuration, it is possible to appropriately detect the moving object based on the luminance change.

  The moving object detection device of the present invention further includes a stationary point warning unit that outputs a warning signal when at least one stationary point out of the plurality of stationary points is out of a predetermined area.

  According to this configuration, it is possible to prevent a situation in which coordinate conversion cannot be performed by preventing loss of position information of a stationary point.

  The moving object detection device of the present invention further includes a moving object warning unit that outputs a warning signal to the user when the moving object determination unit detects a moving object.

  According to this configuration, when a moving object is detected, the presence of the moving object can be notified to the user.

  The moving object detection apparatus of the present invention can detect a moving object in the detection region with a small amount of calculation without using the pan / tilt / zoom control parameter values of the camera, and the camera is moving. The moving object detection of the scene can be realized.

  Hereinafter, embodiments of the moving object detection device of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of a moving object detection device according to an embodiment of the present invention. The moving object detection apparatus includes an image acquisition unit 101, a detection region setting unit 102a, a stationary information storage unit 102b, a stationary point setting unit 102c, a stationary point tracking unit 103, a detection region tracking unit 104, a moving object determination unit 105, a stationary point warning. Unit 106 and moving object warning unit 107.

  The image acquisition unit 101 is a means for acquiring an image, and may be realized as an imaging apparatus such as a camera, or has a function of acquiring an encoded image from an imaging apparatus connected to the outside via a communication network. The structure which has may be sufficient.

  The detection area setting unit 102a includes a display unit (not shown) and an input unit (not shown). The display unit displays the image acquired by the image acquisition unit 101. The input unit inputs, as information, an area set as a detection area for detecting a moving object on the image. In this case, the user designates the vertex of the detection area and inputs from the input unit. The detection area setting unit 102a may set an area that satisfies a predetermined condition as the detection area. For example, the detection area may be set with a stationary point described below as a vertex.

  The stationary point information storage unit 102b stores information on a plurality of stationary points that are stationary in the image space captured by the imaging device. That is, in order to identify a point that is stationary in the space displayed in the image acquired by the image acquisition unit 101, and to grasp the coordinates of the image space with reference to this stationary point, the stationary point information storage unit 102b Store the information of the stationary point. Here, the “point” may be one pixel or a region including surrounding pixels. The same applies to the following “points”. In addition, the information on the still point includes information characterizing each pixel in the image (hereinafter referred to as “pixel information”) such as a frequency component, luminance, and color at the still point. Furthermore, the information on the stationary point includes information (hereinafter referred to as “position information”) indicating the mutual positional relationship (coordinates, distance, angle, etc.) of the plurality of stationary points.

  The stationary point setting unit 102c sets, as the stationary point, a characteristic point that can be distinguished from other points based on pixel information of an image captured by the imaging device. That is, a plurality of still points are set based on the pixel information from the image acquired by the image acquisition unit 101, and information on the still points is stored in the still point information storage unit 102b. Note that the user may select a stationary point.

  The stationary point setting unit 102c sets a point where the pixel information does not substantially change as a stationary point that is stationary in the image space. That is, the still point setting unit 102c detects pixel information in a predetermined period, and sets a point where luminance or the like does not substantially change as a still point. Therefore, when the luminance or the like substantially changes at the point selected by the user, the stationary point setting unit 102c determines that the point selected by the user is not appropriate as the stationary point.

  The stationary point setting unit 102c determines whether or not the amount of change in pixel information exceeds a predetermined threshold in a state where the imaging device is fixed. That is, the still point setting unit 102c determines whether or not the amount of change in luminance exceeds a predetermined threshold in a predetermined period while the imaging device is fixed. The stationary point setting unit 102c sets the point or region as a stationary point when the amount of change in luminance does not exceed a predetermined threshold at the predetermined point or region.

  The stationary point tracking unit 103 tracks the positions of a plurality of stationary points that move on the image as the imaging apparatus moves based on information on the plurality of stationary points stored in the stationary point information storage unit 102b. That is, when the imaging apparatus moves and the image changes, the stationary point tracking unit 103 determines the position of the stationary point in the changed image based on the information of the plurality of stationary points stored in the stationary point information storage unit 102b. Is identified. By performing such identification continuously, the stationary point tracking unit 103 tracks the stationary point. The stationary point tracking unit 103 tracks the stationary point in the image acquired by the image acquiring unit 101, but may track the stationary point in the already acquired moving image such as recording. That is, the stationary point tracking unit 103 tracks a stationary point in association with a continuous frame of a moving image. The association is realized using a general pattern matching process.

  The detection area tracking unit 104 performs coordinate conversion of the image space based on the mutual positional relationship between a plurality of still points, and tracks the position of the detection area that moves on the image as the imaging apparatus moves. For example, the detection area 104 calculates a coordinate conversion parameter for an image when the imaging apparatus moves based on the positional relationship between a plurality of still points, and is set by the detection area setting unit 102a based on the coordinate conversion parameter. The coordinate of the position of the detected area is converted. That is, when the image pickup apparatus moves and the image changes, the detection area tracking unit 104 coordinates the space represented in the changed image based on the position information of the stationary point specified by the stationary point tracking unit 103. A conversion parameter is calculated. Then, the detection area tracking unit 104 specifies the position of the detection area in the space represented by the changed image based on the coordinate conversion parameter. That is, the detection area tracking unit 104 tracks the detection area by coordinate conversion for successive frames of the moving image. The coordinate conversion parameter can also be expressed as a projective conversion parameter.

  The moving object determination unit 105 determines whether the object imaged in the detection area is a moving object based on the luminance change of the detection area tracked by the detection area tracking unit 104. The moving object determination unit 105 calculates the luminance change of the detection area, and determines that the object imaged in the detection area is a moving object when the luminance change exceeds a predetermined threshold.

  The stationary point warning unit 106 outputs a warning signal when at least one stationary point out of a plurality of stationary points is out of a predetermined area.

  The moving object warning unit 107 outputs a warning signal when the moving object determination unit 105 determines that the object imaged in the detection area is a moving object. That is, the moving object warning unit 107 has a function of generating an alarm when the moving object determination unit 105 detects a moving object. Due to the occurrence of this alarm, the moving object detection device may be applied to an e-mail to an administrator or an image when an alarm is generated to an FTP server. However, the present invention does not limit this application method.

  With the configuration as described above, the moving object detection device of the present invention can always detect a moving object in a certain region even if the pan / tilt / zoom control parameter values of the camera are unknown. .

  Next, the operation of the moving object detection device according to the embodiment of the present invention will be described. First, an operation for setting a stationary point will be described. FIG. 2 is a flowchart showing the flow of processing for setting a stationary point.

  The image acquisition unit 101 acquires an image (step S201).

  The stationary point setting unit 102c selects a plurality of predetermined points in the image captured by the imaging device (step S202). The stationary point setting unit 102c selects a point that can be tracked by the stationary point tracking unit 103 based on the pixel information. The points that can be tracked by the stationary point tracking unit 103 are characteristic points that can be distinguished from other points. For example, the characteristic point is a point such as a marker installed at a predetermined position, a point where a plurality of lines intersect, a point where a plurality of surfaces intersect, a boundary point where the color changes. Note that the user may select a still point on the image. FIG. 3 is a diagram illustrating an example of points selected by the stationary point setting unit 102c. FIG. 3 shows a point P500 that is a marker, a point P501 that is a point where a plurality of lines intersect, a point P502 that is a point where a plurality of lines intersect and the color changes, and a part of one line. Point P503 is shown. Among these points, points P500 to P502 are characteristic points and are selected by the stationary point setting unit 102c. On the other hand, since there is only one edge at the point P503 and it is difficult to accurately track the point P503 when the orientation of the imaging apparatus changes, the point P503 is stopped by the stationary point setting unit 102c. It is judged that the point is not appropriate.

  The still point setting unit 102c determines whether the pixel information among the selected points is substantially changed (step S203). For example, in a state where the imaging device is fixed, the still point setting unit 102c divides the image into a plurality of regions, calculates the sum of squares of inter-frame differences in each region, and calculates this sum of squares over a certain period. When the integration is performed, it is determined whether or not the integration value exceeds a predetermined threshold value. For example, as shown in FIG. 3, the amount of change in pixel information at points P500 to P502 is calculated, and it is determined whether the calculated amount of change in pixel information exceeds a threshold value.

  If the change amount of the pixel information at the selected point does not exceed the predetermined threshold, the still point setting unit 102c sets the point as a still point, and stores the information on the still point in the still point information storage unit 102b. Store (step S204).

  When the amount of change in pixel information at the selected point exceeds a predetermined threshold, the still point setting unit 102c selects another point and re-inputs the selected other point (step S205). That is, even in a state where the imaging device is fixed, an area where the luminance change is large, that is, a point on a frequently moving object is determined not to be suitable as a stationary point, and re-input of another point is instructed. For example, as shown in FIG. 3, since the point P502 is a part of the door, the brightness value frequently changes when the door is opened and closed, and it is determined that the point P502 is not appropriate as a stationary point.

  By the process of setting the stationary point as described above, the moving object detection device of the present invention can set an optimal stationary point.

  Next, an operation for detecting a moving object will be described. FIG. 4 is a flowchart showing the flow of processing for detecting a moving object.

  The image acquisition unit 101 acquires an image (step S301).

  The stationary point tracking unit 103 tracks the position of the stationary point in the image when the imaging apparatus moves based on the information on the stationary point stored in the stationary point information storage unit 102b (step S302). 5A and 5B are diagrams illustrating an example of an image when the imaging apparatus moves. FIG. 5A is a diagram illustrating an example of an image before the imaging apparatus moves. FIG. 5B is a diagram illustrating an example of an image after the imaging apparatus has moved. As shown in FIG. 5A, the stationary points 402a to 402d are set by the stationary point setting unit 102c. Then, as illustrated in FIG. 5B, the stationary point tracking unit 103 identifies points 403a to 403d that coincide with the stationary points 402a to 402d, and tracks the stationary point.

  In step S302, the stationary point warning unit 106 determines whether or not the coordinate value of the tracked stationary point is within a predetermined area (for example, an image capturing range) (step S303). As shown in FIG. 5A, since the stationary points 402 a to 402 d exist in the predetermined area 400, it is determined that the coordinate value of the stationary point is in the area 400. On the other hand, as shown in FIG. 5B, since the stationary point 403 d is out of the predetermined area 405, it is determined that the coordinate value of the stationary point is not within the area 405.

  The stationary point warning unit 106 outputs a warning signal when at least one stationary point deviates from a predetermined area (step S304). For example, as shown in FIG. 5B, since the stationary point 403d is out of the region 405, the stationary point warning unit 106 outputs a warning signal. At this time, it is possible not only to issue a warning but also to limit the moving range of the imaging device so that the stationary point is within the imaging range.

  The detection area tracking unit 104 calculates a coordinate conversion parameter for an image when the imaging apparatus moves based on the mutual positional relationship between a plurality of still points, and determines the position of a predetermined detection area based on the coordinate conversion parameter. Coordinate conversion is performed (step S305). That is, the coordinate conversion parameter of the space represented in the changed image is calculated from the positional relationship between the coordinate values of the still points in the image acquired in each frame acquired in step S302. For example, the coordinate conversion parameter is calculated from the positional relationship between the stationary points 402a to 402d shown in FIG. 5A and the stationary points 403a to 403d shown in FIG. The coordinate conversion parameter is 3 × 3 having a degree of freedom of a constant multiple when the movement of the imaging device is limited to rotation and zoom operation around the optical center, or when the region set as the detection region is a plane. Represented by a matrix. For an image affected by the optical distortion of the lens, it is possible to correct the optical distortion in advance.

  The detection area tracking unit 104 specifies the position of the detection area in the space represented by the changed image based on the coordinate conversion parameter (step S306). That is, the detection area tracking unit 104 tracks the detection area based on the coordinate conversion parameter.

  The moving object determination unit 105 determines whether the object imaged in the detection region is a moving object based on the luminance difference of the detection region tracked by the detection region tracking unit 104 (step S307).

  When the luminance difference exceeds a preset threshold value, the moving body determination unit 105 determines that the object imaged in the detection area is a moving object, and the moving object warning unit 107 outputs a warning signal. (Step S308). And it returns to step S301 and repeats the same process.

  When the luminance difference does not exceed the preset threshold value, the moving body determination unit 105 determines that the object imaged in the detection area is not a moving object, and returns to step S301 to repeat the same processing.

  Through the process of detecting a moving object as described above, the moving object detection device of the present invention detects a moving object in a certain region with a small amount of calculation even during execution of pan / tilt / zoom control of the imaging device. can do. By setting the area surrounded by the stationary point as the detection area, it is possible to constantly monitor moving objects in the detection area by preventing the detection area from going outside the imaging range of the camera, and to pan, tilt, and zoom the camera. Control can be performed.

  The moving object detection apparatus of the present invention has an effect of detecting a moving object in a detection region without using the pan / tilt / zoom control parameter values of the camera, and the moving object detection that tracks a predetermined detection region. It is useful as a device.

The block diagram explaining the structure of the moving object detection apparatus of embodiment of this invention Flow diagram showing the flow of processing to set a still point The figure which shows an example of the point selected by the still point setting part Flow diagram showing the flow of processing to detect moving objects (A) The figure which showed an example of the image before an imaging device moves (b) The figure which showed an example of the image after an imaging device moved

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image acquisition part 102 Detection area setting part 102c Static point setting part 103 Static point tracking part 104 Detection area tracking part 105 Moving object detection part 106 Static point warning part 107 Moving object warning part

Claims (7)

A stationary point information storage unit that stores information of a plurality of stationary points that are stationary in an image space captured by the imaging device;
A detection area tracking unit that performs coordinate transformation of the image space based on the mutual positional relationship of the plurality of stationary points, and tracks the position of a predetermined detection area that moves on the image as the imaging device moves;
A moving object determination unit that determines whether the object imaged in the detection area is a moving object based on a change in luminance of the detection area;
A moving object detection apparatus comprising:   The moving object detection device according to claim 1, further comprising a stationary point setting unit configured to set, as the stationary point, a characteristic point that can be distinguished from other points based on pixel information of an image captured by the imaging device. .   The moving object detection device according to claim 2, wherein the stationary point setting unit determines whether or not a change amount of the pixel information exceeds a predetermined threshold value in a state where the imaging device is fixed.   A stationary point tracking unit that tracks the positions of the plurality of stationary points that move on the image as the imaging apparatus moves based on the information of the plurality of stationary points stored in the stationary point information storage unit; The moving object detection device according to claim 1, further comprising a moving object detection device.   5. The moving object determination unit according to claim 1, wherein the moving object determination unit calculates a luminance change of the detection area, and determines that the object imaged in the detection area is a moving object when the luminance change exceeds a predetermined threshold. The moving object detection apparatus in any one.   The movement according to claim 1, further comprising a stationary point warning unit that outputs a warning signal when at least one stationary point of the plurality of stationary points is out of an imaging region of the imaging device. Object detection device. 7. The moving object warning unit according to claim 1, further comprising a moving object warning unit that outputs a warning signal when the moving object determination unit determines that the object to be imaged in the detection area is a moving object. Moving object detection device.

Images