JP2007164690A - Image processing apparatus and image processing method - Google Patents

Abstract

An image processing apparatus and an image processing method for appropriately detecting a moving object by detecting a virtual image are provided.
An image monitoring apparatus 100 performs object extraction processing for extracting an extracted object from a current image and a background image by a predetermined procedure, and compares the extracted object with a previously recognized object and a foreground object by a predetermined procedure. And performing an object classification process for generating a recognition object and a background object, and a background image generation process for generating a background image from the background object.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method suitable for image monitoring for monitoring by comparing an input image with a background image that is automatically updated.

  Conventionally, in the object monitoring and tracking apparatus described in Patent Document 1, an object is detected using a background subtraction method, a dynamic binarization method, and template matching, and a feature amount of the detected object is extracted in advance. The moving object is recognized by comparing with the extracted feature quantity of the object to be specified.

  Moreover, the moving object detection apparatus described in Patent Document 2 will be described with reference to FIG. In FIG. 18, the moving object detection apparatus includes a change region extraction unit 1801, a continuation determination unit 1802, a background image update unit 1803, a continuation information storage unit 1804, and a background image storage unit 1805. Yes.

  The change area extraction unit 1801 calculates a difference absolute value for each pixel between the input current image and the background image, binarizes it with a predetermined threshold, and uses the binarized image as change area information to continue the determination unit 1802. Output to. Here, the change area information is a moving area and indicates a detected moving object.

  The continuation determination unit 1802 counts up the number of continuation frames of the change of each pixel in accordance with the change region information input from the change region extraction unit 1801, and takes a pixel position where the number of continuation frames is equal to or greater than a predetermined threshold as a capture region. While outputting to the background image update part 1803, the recognition information from which the continuation frame number does not exceed a threshold value from the counted continuation information is output to the continuation information storage part 1804 as update continuation information.

  The background image update unit 1803 replaces the background image of the capture area input from the continuation determination unit 1802 with the pixel value of the current image, and outputs the result to the background image storage unit 1805 as an updated background image.

  The continuation information storage unit 1804 stores the update continuation information input from the continuation determination unit 1802, and outputs the stored update continuation information to the continuation determination unit 1802 as the next continuation information.

  The background image storage unit 1805 stores the updated background image input from the background image update unit 1803 and outputs the stored updated background image to the change area extraction unit 1801 as the next background image.

  The operation of this moving object detection apparatus will be described with reference to FIGS. 19 and 20.

  First, the change area extraction unit 1801 generates a background difference image in FIG. 19C from the current image in FIG. 19A and the background image in FIG. 19B. Next, in the change area extraction unit 1801, (c) the background difference image is binarized to generate change area information (d) in the figure, and (d) the same figure (e) corresponding to the change area information (d). (F) Count-up continuation information is generated by counting up the continuation frame number of continuation information (continuation frame number = 4).

  In FIG. 19, the area of the vehicle is the same in the change area information and the continuation information, the number of continuation frames is counted up to “5”, the human area is newly added, and the number of continuation frames is “1”. Become.

  Next, the pixel position where the number of continuation frames exceeds the threshold value is output to the background image update unit 1803 as (g) capture area in the figure, and the pixel position where the threshold value does not exceed the threshold value is stored as continuation information as update continuation information in the figure (h). Output to the unit 1804. Here, the threshold value is set to “5”, and the area of the car having the number of continuing frames of “5” is output as the capture area, deleted from the update continuation information, and the number of continuing frames is “1”. Only the person's area is output as the update continuation information.

  Then, as shown in FIG. 20, the background image update unit 1803 generates the updated background image (i) by replacing the background update region (g) of the background image (g) of FIG. The image is output to the image storage unit 1805.

According to the configuration of the moving object detection apparatus as described above, even when the moving object is stationary in the screen, the moving object can be continuously detected without being immediately captured as a background image. it can.
JP 2002-157599 A JP 2003-123074 A

  However, when the background is updated as in the moving object detection device of Patent Document 2 described above, once the moving object once captured in the background image starts moving again, a virtual image is generated in the background difference, and the background of the virtual image portion is updated again. Until then, the virtual image continues to be generated, and there is a problem that the virtual image portion is erroneously detected as a moving object.

  For example, when (b) the moving object captured in the background image starts moving again as shown in FIG. 21, (d) the change area information includes not only the moving object area but also the moving object. A stationary area is also included, and a virtual image is detected as well as a moving object as shown in FIG.

  Further, there is an object monitoring and tracking apparatus disclosed in Patent Document 1 that does not detect erroneously even if a virtual image is generated in the background difference. That is, by detecting the detection target assumed in advance from the region where the background difference has occurred, no false detection occurs even if a virtual image occurs. However, it is necessary to prepare information for searching for a detection target in advance, and an unspecified moving object cannot be detected.

  This invention is made | formed in view of this point, and it aims at providing the image processing apparatus and image processing method which detect a moving body appropriately by detecting a virtual image.

  An image processing apparatus according to the present invention includes an object extraction unit that extracts an object from a current image and a background image, a comparison between an extracted object extracted by the object extraction unit, a previously recognized object, and a foreground object, An object classification unit that generates a recognition object and a background object, a background image generation unit that generates a background image from the background object, and a recognition object that stores the recognition object and outputs the recognition object to the object classification unit as the next previous recognition object A configuration is provided that includes a storage unit, and a background object storage unit that stores the background object and outputs the background object to the object classification unit as the next previous background object.

  According to this configuration, it is possible to detect that the object captured in the background starts to move again by comparing the changed part in the image with the object captured in the background. Can be updated.

  The image processing apparatus of the present invention also detects an object extracting unit that extracts an object from a current image and a background image, and compares the extracted object extracted by the object extracting unit with a previously recognized object and a foreground / background object. An object classifying unit that generates an object, a recognition object, and a background object; a background object updating unit that generates an updated background object from the current image and the background object; and a background image generating unit that generates a background image from the updated background object; A recognition object storage unit that stores the recognition object and outputs it as the next previous recognition object to the object classification unit; and a background object that stores the updated background object and outputs it as the next previous background object to the object classification unit. It adopts a configuration comprising a-object storage unit.

  According to this configuration, it is possible to detect that the object captured in the background starts to move again by comparing the change part in the image with the object captured in the background, and also follow the change in the feature amount of the object constituting the background. Therefore, it is possible to eliminate the virtual image from the change portion and generate a more appropriate background.

  The image processing apparatus according to the present invention is the image processing apparatus according to claim 1 or 2, wherein the object extraction unit generates a background difference image from the current image and the background image, and the background difference image is binarized. Processing to generate a background difference mask, divide the background difference mask into regions, extract feature amounts from the current image for each divided region, and output an extracted object. The object classification unit includes the extracted object and The matching foreground / background object is selected, and the background difference mask and the divided area of the selected background object are matched with the background difference mask and the divided area of the extracted object as a virtual image object, and the virtual image object The extracted object that does not match the detected object is set as a detected object, and the extracted object matches the previously recognized object The detected object is held as a temporary recognition object, and when the number of holding times reaches a predetermined number, the temporary recognition object is a new background object, the detection object that does not match the new background object is a recognition object, A new background object and the continuous background object are integrated and output as a background object.

  According to this configuration, the feature amount of the region to be captured in the background is stored, the change in the background region is detected by comparing with the feature amount of the region in which the change has occurred, and the change that has occurred in the region is masked. By doing so, the virtual image can be eliminated.

  The image processing apparatus according to the present invention is the image processing apparatus according to claim 2, wherein the background object update unit generates a background object map by combining the background objects, and combines an object mask of the detection object. Then, a composition detection object mask is generated, and a background object is updated from the current image, the background object, the background object map, and the composition detection object mask and output as an updated background object.

  According to this configuration, the updated background object can be made to follow minute changes in the current image, and the background image generated using the updated background object can be made more appropriate. In addition, the similarity of the feature quantity of the same object tends to be stronger when close in time, and since the background object is updated, it can be compared with the feature quantity of the object just before moving, so a virtual image Detection accuracy can be improved.

  The image processing method of the present invention includes an object extraction step of extracting an object from a current image and a background image, and compares the extracted object with a previous recognition object and a previous background object, thereby detecting a detection object and a recognition object. And an object classification step for generating a background object, a background image generation step for generating a background image from the background object, and a recognition object storage step for storing the recognition object in a recognition object storage unit and outputting it as the next previous recognition object And a background object storage step of storing the background object in a background object storage unit and outputting it as the next previous background object, and adopting a method of detecting an object from an image.

  According to this method, it is possible to detect that the object captured in the background starts to move again by comparing the changed part in the image with the object captured in the background. Can be updated.

  The image processing method of the present invention includes an object extraction step of extracting an object from a current image and a background image, and compares the extracted object with a previous recognition object and a previous background object, thereby detecting a detection object and a recognition object. And an object classification step for generating a background object, a background object updating step for generating an updated background object from the current image and the background object, a background image generating step for generating a background image from the updated background object, and the recognition object Is stored in the recognition object storage unit and output as the next previous recognition object, and the updated background object is stored in the background object storage unit and output as the next previous background object. Adopt a method of detecting an object from an image by having the object storage step.

  According to this method, it is possible to detect that the object captured in the background starts to move again by comparing the changed part in the image with the object captured in the background, and also follow the change in the feature amount of the object composing the background. Therefore, it is possible to eliminate the virtual image from the change portion and generate a more appropriate background.

  According to the present invention, it is possible to detect that the object captured in the background starts to move again by comparing the changed part in the image with the object captured in the background. Can be updated and the moving object can be detected appropriately.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a configuration of the image monitoring apparatus 100 according to the first embodiment. In FIG. 1, the image monitoring apparatus 100 includes an object extraction unit 101, an object classification unit 102, a background image generation unit 103, a recognition object storage unit 104, and a background object storage unit 105.

  As shown in the flowchart of FIG. 4, the image monitoring apparatus 100 according to the first embodiment performs an object extraction process for extracting an extracted object from a current image and a background image according to a predetermined procedure (step S101), and a predetermined procedure. An object classification process for comparing the extracted object with the previous recognition object and the previous background object to generate a recognition object and a background object (step S102), and a background image generation process for generating a background image from the background object (step S103) ).

  First, the configuration and function of the object extraction unit 101 will be described with reference to the block configuration diagram shown in FIG. 2 and the flowchart shown in FIG.

  As illustrated in FIG. 2, the object extraction unit 101 includes a background difference processing unit 201, a binarization unit 202, a region division unit 203, and a feature amount extraction unit 204.

  First, the background difference processing unit 201 calculates an absolute difference value of the input current image and background image in pixel units, and outputs a background difference image having the absolute value of the difference as a pixel value to the binarization unit 202 ( Step S201).

  Next, the binarization unit 202 binarizes the background difference image input from the background difference processing unit 201 with a predetermined threshold to generate a background difference mask. Here, the background difference mask is divided into regions using a binary image in which the pixel value of a pixel whose difference absolute value is equal to or greater than a predetermined threshold is “1” and the pixel value of other pixels is “0” as a background difference mask. The data is output to the unit 203 (step S202).

  Next, the region dividing unit 203 divides a pixel portion having a pixel value “1” from the difference background mask input from the binarizing unit 202, and zero or more pixel values “1” and “0” are obtained. Generate split regions. Here, a portion where the same pixel value is continuous is divided as the same region, and a region whose area is smaller than a predetermined threshold is deleted. In this case, the area is the number of pixels having a pixel value “1” in the region (step S203).

  Next, the feature amount extraction unit 204 extracts a feature amount from the current image for each divided region input from the region division unit 203, and outputs the feature amount to the object classification unit 102 as an extracted object (step S204).

  As the feature amount, an area, a color histogram, and the like are extracted. The area is an area in the divided region. The color histogram is obtained by dividing the divided area by a color space and totaling the number of pixels belonging to the same color space. In the first embodiment, the divided area, the background difference mask in the divided area, and the image of the divided area are set as a rectangular area, an object mask, and a rectangular area image, respectively, and those combined with the feature amount are output as extracted objects. To do.

  With the above configuration, the object extraction unit 101 has a function of generating an extraction object from the current image and the background image.

  Next, the configuration and function of the object classification unit 102 in FIG. 1 will be described with reference to the block configuration diagram shown in FIG. 3 and the flowchart shown in FIG.

  As shown in FIG. 3, the object classification unit 102 includes a virtual image determination unit 301, a recognition object update unit 302, and a background object integration unit 303.

  As illustrated in FIG. 3, the virtual image determination unit 301 compares and associates the extracted object input from the object extraction unit 101 with the foreground object input from the background object storage unit 105 in FIG. Select the foreground and background objects that match Then, the object having the same rectangular area and object mask of the matched foreground / background object is searched from the extracted object to be a virtual image object. Further, among the extracted objects, an extracted object other than the virtual image object is output to the recognition object update unit 302 and the outside as a detection object (step S301).

  Comparison between the extracted object and the foreground / background object is performed using an object mask, an area, and a color histogram.

  As for the coincidence determination with the object mask, the absolute value of the difference is calculated between each pixel of the extracted object and the object mask of the foreground / background object, and the difference absolute value is summed, and the total value is equal to or less than a predetermined threshold value. It is determined that the object mask matches.

  Regarding the area coincidence determination, when the areas of the extracted object and the foreground / background object are equal to or smaller than a predetermined threshold, it is determined that the areas coincide.

  Regarding the color histogram match determination, the Euclidean distance is calculated between the color histograms of the extracted object and the foreground / background object, and when the Euclidean distance is equal to or less than a predetermined threshold, it is determined that the color histograms match.

  Regarding the comparison of the rectangular areas, the distance between the upper left coordinates of the rectangular area of the extracted object and the foreground / background object is not more than a predetermined threshold, the width is not more than the predetermined threshold, and the height is not more than the predetermined threshold. In this case, it is determined that the rectangular areas match.

  Further, if the extracted object and the foreground / background object match all of the object mask, area, and color histogram, the virtual image determination unit 301 considers the extracted object and the foreground / background object to match, and Among these, the objects other than the matched foreground / background objects are output to the background object integration unit 303 as continuous background objects.

  Next, the recognition object update unit 302 compares and associates the detection object input from the virtual image determination unit 301 with the pre-recognition object input from the recognition object storage unit 104 in FIG. Let the object be a temporary recognition object. Then, the detected object recognized as the provisional recognition object for the predetermined number of frames or more is output to the background object integration unit 303 as a new background object. In addition, a recognition object is generated by updating a pre-recognition object other than a pre-recognition object that has not been recognized continuously for a predetermined number of frames or more and a pre-recognition object that has not become a new background object using a detection object to generate a recognition object The data is output to the storage unit 104 (step S302).

  Comparison between the detected object and the previously recognized object is performed using a rectangular area, an object mask, an area, and a color histogram.

  For comparison of rectangular areas, when the distance between the upper left coordinates of the rectangular area of the detected object and the previously recognized object is less than a predetermined threshold, the width is less than the predetermined threshold, and the height is less than the predetermined threshold It is determined that the rectangular areas match.

  For comparison with the object mask, the difference absolute value is taken between each pixel of the extracted object and the object mask of the foreground / background object, and the difference absolute value is summed, and when the total value becomes a predetermined threshold value or less, It is determined that the object mask matches.

  Regarding the area comparison, when the areas of the extracted object and the foreground / background object are equal to or less than a predetermined threshold, it is determined that the areas match.

  Regarding the comparison of color histograms, the Euclidean distance is calculated between the color histograms of the extracted object and the foreground / background object, and it is determined that the color histograms match when the Euclidean distance falls below a predetermined threshold.

  The recognized object update unit 302 determines that the extracted object and the foreground / background object match when all of the rectangular area, the object mask, the area, and the color histogram match.

  The recognized object updating unit 302 updates the number of recognized frames, the number of non-recognized frames, the number of consecutively recognized frames, and the number of consecutive unrecognized frames of the corresponding previous recognized object for the matching detected objects (the number of recognized frames). The number of consecutive recognition frames is incremented and the number of consecutive non-recognition frames is reset). If the number of consecutive recognition frames exceeds a predetermined threshold value, a new background object is added, and the background object time is added to the background object integration unit 303. Output to.

  The recognized object updating unit 302 adds the initialized number of recognized frames, number of unrecognized frames, and number of consecutive non-recognized frames to the detected objects that do not match, and recognizes the recognized object storage unit 104 as a recognized object. Output to.

  Further, the recognition object update unit 302 updates the number of non-recognized frames, the number of consecutive non-recognized frames, and the number of consecutive recognition frames for the previously recognized objects that do not match (the number of non-recognized frames and the number of consecutive non-recognized frames). And the number of consecutive non-recognized frames is reset to the recognition object storage unit 104 as a recognition object.

  Next, the background object integration unit 303 integrates the continuous background object input from the virtual image determination unit 301 and the new background object input from the recognition object update unit 302, generates a background object, and generates a background image generation unit. 103 and the background object storage unit 105 (step S303).

  With the above configuration, the object classification unit 102 has a function of generating a recognition object and a background object from the extracted object, the previous recognition object, and the front and background object.

  Next, the background image generation unit 103 in FIG. 1 generates a background image from the background object input from the object classification unit 102 (step S103). The background image generation unit 103 arranges the input background objects from the back to the front in order from the earliest backgrounding time, and considers overlapping of objects by overwriting the back objects with the front objects. In addition, the background image generation unit 103 writes an image of a portion where the pixel value of the object mask is “1” to the rectangular area image arranged in the rectangular area.

  Next, the recognition object storage unit 104 of FIG. 1 stores the recognition object input from the object classification unit 102, and outputs the stored recognition object to the object classification unit 102 as the next pre-recognition object.

  Next, the background object storage unit 105 in FIG. 1 stores the background object input from the object classification unit 102 and outputs the stored background object to the object classification unit 102 as the next foreground background object.

  With the above configuration, the image monitoring apparatus 100 according to the first embodiment can detect a virtual image by comparing an object captured in the background with the changed portion, appropriately detect the moving object, and also detect the background image. Update appropriately.

  The operation of the image monitoring apparatus 100 configured as described above will be described with reference to FIGS.

  FIG. 7 to FIG. 9 show an example of the operation of each part in the image monitoring apparatus 100 when a person enters the image monitoring screen and stops, and the car that is still in the screen starts to move. In this operation example, a case where a person and a car are detected as moving objects, a person is taken as a background, and a stopped car is deleted from the background will be described.

  First, the operation of the object extraction unit 101 will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of input / output images in each block in the object extraction unit 101.

  The background image of FIG. 7B is a background image generated by the initial background and the background object of the car and the bicycle, and it is assumed that the car and the bicycle that entered / stopped in the screen have already become the background. . On the other hand, the current image shown in FIG. 6A is an image that assumes that a person who has entered and stopped in the screen, a restarted car, and a dog that has entered the screen are shown. is there.

  The background difference processing unit 201 calculates a difference absolute value in pixel units between the input (a) current image and (b) background image, and performs background difference processing using the difference absolute value as a pixel value. The background difference image of (c) is generated and output to the binarization unit 202.

  Next, the binarization unit 202 binarizes the (c) background difference image input from the background difference processing unit 201 with a predetermined threshold, generates the background difference mask of FIG. The data is output to the unit 203.

  Next, the region dividing unit 203 divides the region where the same pixel values are continuous from the (d) background difference mask input from the binarizing unit 202 as the same region, and the region 1 in FIG. ˜4 are generated as divided regions and output to the feature amount extraction unit 204.

  Next, the feature amount extraction unit 204 extracts feature amounts for each of the divided regions 1 to 4 by the feature amount extraction processing from the input (a) current image and the divided regions input from the region dividing unit 203, The extracted objects 1 to 4 corresponding to the divided areas 1 to 4 in FIG. 4F are generated and output to the object classification unit 102.

  Here, a person, a car, a virtual image of a car, and a dog are extracted as extraction objects 1 to 4, respectively.

  Next, the operation of the object classification unit 102 will be described with reference to FIG. FIG. 8 is a diagram illustrating an example of input / output images in each block in the object classification unit 102.

  First, in the virtual image determination unit 301, the extracted objects 1 to 4 of (a) input from the object extraction unit 101 and the foreground / background objects 1 and 2 of FIG. As a result of the virtual image determination, detection objects 1 to 3 in FIG. 4D are generated and output to the recognition object update unit 302, and a continuous background object 1 in FIG. Is output.

  That is, as a result of comparison / association between the extracted objects 1 to 4 and the foreground / background objects 1 and 2, the extracted object 2 and the foreground / background object 1 match each other. The matching extracted object 3 is determined as a virtual image object. Accordingly, the extracted objects 1, 2, and 4 other than the virtual image object are output to the recognition object update unit 302 and the outside as the detection objects 1 to 3 in FIG. 4D, respectively, and the foreground object 2 in FIG. The continuous background object 1 is output to the background object integration unit 303.

  Next, the recognition object update unit 302 updates the recognition object from the pre-recognition object of FIG. 10C input from the recognition object storage unit 104 and (d) detection objects 1 to 3 input from the virtual image determination unit 301. As a result of the processing, recognition objects 1 and 2 in FIG. 8F are generated and output to the recognition object storage unit 104, and a new background object 1 in FIG. The data is output to the integration unit 303.

  That is, the pre-recognized object 1 and the detected object 1 match, and the number of recognized frames, the number of consecutively recognized frames, and the number of consecutive non-recognized frames are updated. Here, assuming that the backgrounding threshold is “10”, the number of continuously recognized frames is equal to or greater than the threshold, and therefore (g) a new backgrounding object 1 is generated from the previous recognition object 1 and the detection object 1.

  Next, the background object integration unit 303 integrates (e) the continuous background object 1 input from the virtual image determination unit 301 and (g) the new background object 1 input from the recognition object update unit 302. The background objects 1 and 2 in FIG. (H) are generated and output to the background image generation unit 103 and the background object storage unit 105.

  That is, (e) the continuous background object 1 and (g) the new background object 1 become the background objects 1 and 2 of (h), respectively.

  As described above, the object classification unit 102 eliminates the virtual image and generates the detection object, the background object, and the recognition object by performing the above-described operation.

  Next, the operation of the background image generation unit 103 will be described with reference to FIG. FIG. 9 is a diagram illustrating an example of an input / output image of the background image generation unit 103.

  In the background image generation unit 103, the background image shown in FIG. 10C is generated from the (b) background objects 1 and 2 input from the object classification unit 102 by the background image generation process and is output to the object extraction unit 101. Here, (a) the initial background image is overwritten with (b) the background objects 1 and 2 in the order of the backgrounding time, in the order of the background objects 1 and 2, and (c) the background image is obtained.

  As described above, the operation described with reference to FIGS. 7 to 9 enables extraction of a detection object that does not include a virtual image and generation of a background image in which the virtual image is appropriately updated.

  FIG. 10 shows data examples of the extracted object, the foreground / background object, the detection object, the continuous background object, the new background object, the recognition object, and the background object processed by the object classification unit 102. FIG. 11 shows a configuration example of the object data in FIG.

  As described above, according to the image monitoring apparatus of the first embodiment, it is reliably detected that the moving object once captured in the background image starts to move again, and the background image is updated to an image that does not generate a virtual image. It is possible to appropriately detect a moving object entering the monitoring screen.

  In the region dividing unit according to the first embodiment, the region is divided at a portion where the pixel values are not continuous. However, when the rectangular positions are close to each other, these may be integrated or similar features may be used. You may make it integrate what has quantity. In the image monitoring apparatus of the first embodiment, the feature amounts used for extraction and comparison are the area and the color histogram, but are not limited thereto.

(Embodiment 2)
A second embodiment of the present invention will be described with reference to FIGS.

  FIG. 12 is a block diagram showing the configuration of the image monitoring apparatus 1200 according to the second embodiment. The same components as those in the image monitoring apparatus 100 shown in FIG. Yes. In FIG. 12, an image monitoring apparatus 1200 includes an object extraction unit 101, an object classification unit 102, a background image generation unit 103, a recognized object storage unit 104, a background object storage unit 105, a background object update unit 1201, Consists of

  The object extraction unit 101, the object classification unit 102, and the recognized object storage unit 104 have the same functions as those described in the first embodiment. The background image generation unit 103 and the background object storage unit 105 handle the updated background object instead of the background object, but there is no difference in the data structure, and the operation is the same as that described in the first embodiment.

  Further, the image monitoring apparatus 1200 according to the second embodiment is configured to perform object extraction processing (step S101), object classification processing, and the like, as with the image monitoring apparatus 100 according to the first embodiment, as shown in the flowchart of FIG. (Step S102), a background image generation process (Step S103), and a background object update process in which the background object update unit 1201 generates an updated background object according to a predetermined procedure (Step S1403). Furthermore, it has a function to execute.

  The background object update unit 1201 generates an updated background object from the current image and the background object and detection object input from the object classification unit 102 by a predetermined procedure.

  Next, the configuration and function of the background object update unit 1201 will be described with reference to the block diagram shown in FIG. 13 and the flowchart shown in FIG.

  As illustrated in FIG. 13, the background object update unit 1201 includes a background object composition unit 1301, a detected object mask composition unit 1302, and an object data update unit 1303.

  First, the background object synthesis unit 1301 synthesizes the background objects input from the object classification unit 102, generates a background object map in which each pixel is associated with the background object, and outputs the background object map to the object data update unit 1303 (step S1501). ).

  Here, the background objects are arranged in ascending order of backgrounding time from the back. That is, when the background object overlaps, the pixel is associated with the background object with the later backgrounding time.

  Next, the detected object mask combining unit 1302 combines the object masks of the detected objects input from the object classifying unit 102, generates a combined detected object mask, and outputs the combined detected object mask to the object data update unit 1303 (step S1502).

  Next, the object data update unit 1303 receives the input current image, the background object input from the object classification unit 102, the background object map input from the background object composition unit 1301, and the detection object mask composition unit 1302. The background object is updated from the synthesized detection object mask to be output, and is output to the background image generation unit 103 and the background object storage unit 105 as an updated background object (step S1503).

  For a background object in which pixels not masked by the composite detection object mask or other background object occupy a ratio equal to or greater than the first predetermined threshold with respect to the area of the background object, the rectangular area image is re-extracted from the current image Using this, the feature amount is extracted again.

  For a background object that occupies a ratio equal to or greater than the second predetermined threshold with respect to the area of the background object, the average luminance value of the associated pixel and the average luminance value of the background object are calculated, and only the difference between them is calculated. All pixels of the rectangular image of the background object are shifted, and feature values are extracted again using this.

  Background objects are not updated for other background objects.

  With the configuration of the background object update unit 1201 as described above, the background object update unit 1201 generates an updated background object from the current image, the detected object, and the background object.

  The image monitoring apparatus 1200 configured as described above can detect a virtual image by comparing the object captured in the background with the changed portion, appropriately detect the moving object, and more appropriately the background image. Update.

  The operation of the image monitoring apparatus 1200 configured as described above will be described with reference to FIGS. Here, in particular, the operation of the background object update unit 1201 will be described. FIG. 16 is a diagram illustrating an example of input / output images in each block in the background object update unit 1201.

  First, assume that (a) the current image, (b) the background object, and (c) the detected object are input to the background object update unit 1201 as shown in FIG. The background object composition unit 1301 performs mapping by arranging the background object masks of the background objects 1 and 2 at the rectangular positions from the background object by the background object composition processing, and the background shown in FIG. An object map is generated and output to the object data update unit 1303.

  Next, in the detected object mask composition unit 1302, (c) the object masks of the detected objects 1 to 3 are arranged at the rectangular positions from the detected object by the detected object composition process, and the composite detected object mask of FIG. And output to the object data update unit 1303.

  Next, in the object data update unit 1303, (a) the current image, (b) the background object input from the object classification unit 102, (d) the background object map input from the background object composition unit 1301, and the detected object The background objects 1 and 2 in (b) are updated to the updated background objects 1 and 2 respectively by object data update processing from the (e) composition detection object mask input from the mask composition unit 1302 in FIG. An updated background object is generated and output to the background image generation unit 103.

  In this case, since the background object 1 is not masked at all by the (e) composite detection object mask and does not overlap with other background objects, all pixels are associated with each other. Are extracted again, and the feature amount is updated.

  Here, an updated background object 1 in which a minute change of the bicycle in the current image is captured is generated.

  Since the background object 2 is completely masked by the (e) composite detection object mask, the background object 2 becomes the updated background object 2 without being updated.

  FIG. 17 is a diagram illustrating an example in which the background object is partially mapped in the object data update unit 1303.

  Here, it is assumed that (a) the color of the car in the current image is slightly changed with respect to the background image before the update (f).

  (C) As shown in the background object map, the background object 1 and the background object 2 overlap each other, and the background object 1 is associated with all pixels, but the background object 2 is partially associated with the background object 2. For, the background object is updated using the average luminance value.

  The rectangular image of the background object 2 is shifted by the difference between the average luminance value of the background object 2 and the average luminance value of the pixel of the current image to which the background object 2 is mapped, and the feature amount is extracted again using this. The update background object 2 is assumed.

  In this way, the feature quantity that can be extracted from the screen is reflected in the area that cannot be extracted from the screen, thereby making the background object close to the current object.

  As described above, the operation of the object data update unit 1303 shown in FIGS. 16 and 17 allows the updated background object to follow a minute change in the current image. Thereby, the background image generated by the background image generation unit 103 using the updated background object can be made more appropriate.

  In addition, the similarity of the feature quantity of the same object tends to be stronger when close in time, and since the background object is updated, it can be compared with the feature quantity of the object just before moving, so a virtual image Detection accuracy can be improved.

  As described above, according to the image monitoring apparatus of the second embodiment, even when the object once captured in the background image has slightly changed, it is detected that the object has started moving again, and the image is not generated. While updating the background image, it is possible to appropriately detect the intruding object.

  In the object data update unit of the second embodiment, the background object is masked with the combined detection object mask obtained by combining the detection objects, but the background difference mask generated by the binarization unit of the object extraction unit is used instead. You may use for.

  In the object data updating unit of the second embodiment, the average luminance value is used. However, the average luminance value may be used as a feature amount extracted at the time of object extraction, and this may be used.

  Further, in the object data update unit of the second embodiment, when the feature amounts of adjacent background objects are close to each other, they may be integrated into one background object. Also, the object data update of the second embodiment In the section, a background object that has been a background for a sufficiently long time may be integrated into the initial background and the background object may be deleted.

  Further, in the object data update unit of the second embodiment, when the background object is partially mapped, the feature amount is extracted again from the rectangular image whose luminance value is shifted. May not be updated.

  In addition, in the object data update unit according to the second embodiment, when a background object is partially mapped, a rectangular image with a shifted luminance value is created, but the mapped pixels are extracted from the current image. Anyway.

  The image processing apparatus and the image processing method according to the present invention can be updated to an appropriate background image so as not to generate a virtual image in the background difference, and can be monitored with high accuracy. This is useful as an image monitoring apparatus for monitoring.

1 is a block diagram showing a configuration of an image monitoring apparatus according to Embodiment 1 of the present invention. The block diagram which shows the structure of the object extraction part which concerns on this Embodiment 1. FIG. The block diagram which shows the structure of the object classification | category part concerning this Embodiment 1. FIG. Flowchart showing operations and functions of the image monitoring apparatus according to the first embodiment. The flowchart which shows the operation | movement and function of the object extraction part which concerns on this Embodiment 1. The flowchart which shows the operation | movement and function of the object classification part which concerns on this Embodiment 1. The figure which showed the example of the input-output image in each block in the object extraction part which concerns on this Embodiment 1. The figure which showed the example of the input-output image in each block in the object classification part which concerns on this Embodiment 1. The figure which showed the example of the input-output image in the background image generation part which concerns on this Embodiment 1. The figure which shows the data example of the object data processed with the object classification | category part which concerns on this Embodiment 1. The figure which shows the structural example of the object data which concerns on this Embodiment 1. The block diagram which shows the structure of the image monitoring apparatus which concerns on Embodiment 2 of this invention. The block diagram which shows the structure of the background object update part which concerns on this Embodiment 2. FIG. Flowchart showing the operation and function of the image monitoring apparatus according to the second embodiment. The flowchart which shows operation | movement and a function of the background object update part which concerns on this Embodiment 2. The figure which showed the example of the input-output image in each block in the background object update part which concerns on this Embodiment 2. The figure which showed the example of the input-output image in the object data update part which concerns on this Embodiment 2. The block diagram which shows the structure of the conventional moving body detection apparatus The figure which shows operation | movement of the conventional moving body detection apparatus. The figure which shows operation | movement of the conventional moving body detection apparatus. The figure which shows the subject of operation | movement of the conventional moving body detection apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100, 1200 Image monitoring apparatus 101 Object extraction part 102 Object classification part 103 Background image generation part 104 Recognition object memory | storage part 105 Background object memory | storage part 201 Background difference process part 202 Binarization part 203 Area division part 204 Feature-value extraction part 301 Virtual image Determination unit 302 Recognition object update unit 303 Background object integration unit 1201 Background object update unit 1301 Background object composition unit 1302 Detected object mask composition unit 1303 Object data update unit

Claims (6)

An object extraction unit for extracting an object from the current image and the background image;
An object classification unit that compares the extracted object extracted by the object extraction unit with the previously recognized object and the background object, and generates a detected object, a recognized object, and a background object;
A background image generation unit for generating a background image from the background object;
A recognition object storage unit that stores the recognition object and outputs the recognition object as a next previous recognition object to the object classification unit;
A background object storage unit that stores the background object and outputs the background object as a next previous background object to the object classification unit;
An image processing apparatus comprising: An object extraction unit for extracting an object from the current image and the background image;
An object classification unit that compares the extracted object extracted by the object extraction unit with the previously recognized object and the background object, and generates a detected object, a recognized object, and a background object;
A background object update unit for generating an updated background object from the current image and the background object;
A background image generation unit that generates a background image from the updated background object;
A recognition object storage unit that stores the recognition object and outputs the recognition object as a next previous recognition object to the object classification unit;
A background object storage unit that stores the updated background object and outputs the updated background object to the object classification unit as a next previous background object;
An image processing apparatus comprising: The object extraction unit generates a background difference image from the current image and the background image, binarizes the background difference image to generate a background difference mask, divides the background difference mask into regions, and divides the divided region Extract the feature amount from each current image and output the extracted object,
The object classification unit selects the foreground / background object that matches the extracted object, and the background difference mask and the divided region of the selected foreground background object match the background difference mask and the divided region of the extracted object. When the object to be a virtual image object, the extracted object that does not match the virtual image object is a detection object, the detection object that matches the pre-recognition object is held as a temporary recognition object, The temporary recognition object is a new background object, the detected object that does not match the new background object is a recognition object, and the new background object and the continuous background object are integrated to form a background object. The image processing apparatus according to claim 1 or 2, wherein the to and output.   The background object update unit generates a background object map by combining the background objects, generates a combined detection object mask by combining an object mask of the detection object, the current image, the background object, and the 3. The image processing apparatus according to claim 2, wherein a background object is updated from a background object map and the composite detection object mask and output as an updated background object. An object extraction step for extracting an object from the current image and the background image;
An object classification step of comparing the extracted object with a previously recognized object and a previous background object to generate a detected object, a recognized object, and a background object;
A background image generating step for generating a background image from the background object;
A recognition object storage step of storing the recognition object in a recognition object storage unit and outputting as a next previous recognition object;
A background object storage step of storing the background object in a background object storage unit and outputting as a next previous background object;
An image processing method characterized by detecting an object from an image by having An object extraction step for extracting an object from the current image and the background image;
An object classification step of comparing the extracted object with a previously recognized object and a previous background object to generate a detected object, a recognized object, and a background object;
A background object update step of generating an updated background object from the current image and the background object;
A background image generation step of generating a background image from the updated background object;
A recognition object storage step of storing the recognition object in a recognition object storage unit and outputting as a next previous recognition object;
A background object storage step of storing the updated background object in a background object storage unit and outputting as a next previous background object;
An image processing method characterized by detecting an object from an image by having

Images