JP2010204704A - Image processing apparatus and program for threshold value setting processing - Google Patents

Abstract

In an image processing apparatus having an area extraction processing function for extracting a rectangular area including a change pixel from an image photographed by a monocular camera, adapting to wave fluctuations, reflection of the sun on the waves, and tree fluctuations, An automatic threshold value setting function capable of detecting a stable object with a stable pixel without fluctuation is realized.
An environment-adaptive automatic threshold calculation processing unit 20A calculates a standard deviation σ of a current pixel of interest and a moving average (weighting) of a pixel of interest weighted the lightest oldest image within a fixed period and the latest image most heavily. Based on (moving average), a value of (3σ weighted moving average) − (3σ standard deviation) is calculated, and this value is set as a threshold value of the corresponding pixel (target pixel) for extracting the change pixel. To do.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus having a region extraction processing function for extracting a region including a change pixel from a plurality of time-series images taken by a camera, and a threshold setting processing program for realizing the region extraction processing function.

  In an image processing apparatus having an area extraction processing function for extracting a rectangular area including a change pixel from a plurality of time-series images taken by a monocular camera (single camera), stable and highly accurate change pixel area extraction is performed. A threshold setting means is required to enable this. In order to meet this requirement, conventionally, the setting of the threshold value for extracting the change pixel has been performed depending on the experience and intuition of an image processing expert. Alternatively, the evaluation is performed by changing the threshold using a huge video source, and the best threshold is selected.

  As a technique for automatically setting a threshold in image processing, conventionally, there is a means for determining a mask extraction threshold according to the difference in image information between a reference frame and the current frame every time the current frame is updated. There is a monitoring system (see, for example, Patent Document 1). In the system of Patent Document 1, a histogram of frame differences is created, and the cumulative frequency is filtered with a fixed threshold value that is an index for extracting as a moving object region, thereby obtaining a mask extraction threshold value. Further, an absolute difference value of luminance is obtained between the input image and the set background image, threshold processing is performed on the absolute difference value of luminance, and pixels exceeding this threshold are set to 255, and pixels not exceeding are set to 0. There exists a binarization processing technique to perform (see, for example, Patent Document 2). In addition, there is a technique for calculating a threshold value for improving image quality by taking a difference between two images and removing a difference between two images with reference to a standard deviation value obtained from the difference image (for example, Patent Documents). 3). In contrast to these techniques, the inventor of the present application uses a continuous image for a certain period to obtain luminance fluctuations per pixel (average value and standard deviation), and uses the standard deviation as a pixel for extracting a change pixel. An automatic threshold value setting technique for setting a per-threshold value has been established (see Patent Document 4). However, in the threshold setting using this standard deviation, these corresponding pixels are also changed pixels for wave fluctuations, the influence of the western sun reflected in the waves, intense tree fluctuations, tree shadow fluctuations, etc. As a result, there is a problem in the stable operation performance following the fluctuation of the luminance described above.

JP 2001-325672 A JP 2001-160146 A JP-A-11-187288 JP 2008-250892 A

  As described above, in an image processing apparatus having an area extraction processing function for extracting a rectangular area including a change pixel from an image photographed by a monocular camera, conventionally, it is possible to stably extract a change pixel area with high accuracy. There was a performance problem in the automatic threshold setting means.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus and a threshold setting processing program that realizes an automatic threshold setting function capable of extracting a stable and highly accurate changed pixel region.

The present invention is directed to an image processing apparatus including a region extraction processing unit that sequentially inputs images captured by a camera and extracts a region including a change pixel from the input current image and a past image.
For a plurality of images within the input fixed period, standard deviation calculating means for calculating a luminance average value and a standard deviation per pixel, and for the pixel for which the standard deviation is calculated, the calculated standard deviation and the plurality of Threshold calculation means for calculating a threshold based on a weighted moving average of threshold values of corresponding pixels of the past image, and the change pixel of the pixel for which the standard deviation is calculated from the threshold calculated by the threshold calculation means. And a threshold value setting means for setting as a threshold value.

  The present invention is also directed to an image processing apparatus comprising region extraction processing means for sequentially inputting images captured by a camera and extracting a region including a change pixel from the input current image and past image. For a plurality of images within the input fixed period, standard deviation calculating means for calculating a luminance average value and a standard deviation per pixel, and for the pixel for which the standard deviation is calculated, the calculated standard deviation and the plurality of A threshold calculation unit that calculates a threshold based on a weighted moving average of thresholds of corresponding pixels of the past image, and a standard deviation calculated by the standard deviation calculation unit are set as a threshold for extracting the change pixel. Or an external setting means for setting whether to set the threshold value calculated by the threshold value calculation means as a threshold value for extracting the change pixel, and according to the setting of the external setting means, Threshold value setting means for setting the standard deviation calculated by the standard deviation calculation means or the threshold value calculated by the threshold value calculation means as a threshold value for extracting the change pixel of the pixel for which the standard deviation has been calculated. It is characterized by.

  In addition, the present invention is a method for causing a computer to function as an image processing apparatus having an area extraction processing function for sequentially inputting images captured by a camera and extracting a rectangular area including a change pixel from the input current image and past image. A threshold value setting processing program for setting a threshold value used for extraction of the change pixel for each pixel, and a standard for calculating a luminance average value and a standard deviation per pixel for the plurality of inputted images within a predetermined period. A threshold calculation function that calculates a deviation based on a weighted moving average of the calculated standard deviation and a threshold value of each corresponding pixel of the plurality of past images, with respect to a pixel for which the standard deviation has been calculated; A threshold value setting function for setting the threshold value calculated by the threshold value calculation function as a threshold value for extracting the change pixel of the pixel for which the standard deviation has been calculated; Characterized in that to.

  In addition, the present invention is a method for causing a computer to function as an image processing apparatus having an area extraction processing function for sequentially inputting images captured by a camera and extracting a rectangular area including a change pixel from the input current image and past image. A threshold value setting processing program for setting a threshold value used for extraction of the change pixel for each pixel, and a standard for calculating a luminance average value and a standard deviation per pixel for the plurality of inputted images within a predetermined period. A threshold calculation function that calculates a deviation based on a weighted moving average of the calculated standard deviation and a threshold value of each corresponding pixel of the plurality of past images, with respect to a pixel for which the standard deviation has been calculated; The standard deviation calculated by the standard deviation calculation function is set as a threshold for extracting the change pixel, or the threshold calculated by the threshold calculation means is set as the change pixel. In response to an external instruction on whether to set as a threshold value for output, according to this instruction, the standard deviation calculated by the standard deviation calculating means, or the threshold value calculated by the threshold calculating means, the pixel of which the standard deviation is calculated A threshold value setting function for setting a threshold value for extracting a change pixel is realized in the computer.

  According to the present invention, in an image processing apparatus having an area extraction processing function for extracting a rectangular area including a change pixel from an image photographed by a monocular camera, automatic threshold value extraction that enables stable and highly accurate change pixel area extraction is possible. A setting function can be realized. In particular, it is possible to realize an automatic threshold setting function that can adapt to wave fluctuations, reflections of the western sun on the waves, and tree fluctuations, and can detect a stable object with stable pixels without fluctuations. Accordingly, it is possible to provide an image processing apparatus and a threshold setting processing program that enable extraction of a stable and highly accurate changed pixel region with respect to various conditions such as camera installation conditions, use conditions, and imaging conditions.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. The figure which shows the component which implement | achieves the automatic threshold value setting process function stored in the image processing memory | storage part in the said embodiment. The figure which shows the setting concept of the threshold value using the standard deviation in the said embodiment. The flowchart which shows the procedure of the process in the said embodiment. The flowchart which shows the procedure of the process in the said embodiment. The flowchart which shows the procedure of the process in the said embodiment. The flowchart which shows the procedure of the process in the said embodiment. The flowchart which shows the procedure of the process in the said embodiment. The flowchart which shows the procedure of the process in the said embodiment. The flowchart which shows the procedure of the process in the said embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
In describing an embodiment of the present invention, an operation concept of an automatic threshold value calculation process in an image processing apparatus according to an embodiment of the present invention will be described with reference to FIG.

  The video input to the image processing apparatus according to the embodiment of the present invention is a camera video shot by a monocular camera (single camera), and the ambient brightness, the structure being imaged, video conditions (noise superposition) Depending on the situation, the operation of the automatic iris, etc., the luminance variation of the entire screen occurs. If only a fixed threshold is set, it is difficult to accurately extract only the target in the daytime and nighttime fluctuations, and the threshold is determined by the human system. The place due to uncertain factors such as Therefore, as shown in FIG. 3, the inventor of the present application, for all pixels of QVGA (320 × 240 pixels), the average (μ) of luminance (edge intensity) set of n frames (for example, 30 frames) per pixel. : Brightness average) and standard deviation (σ) were determined, and an automatic threshold setting technique was established to set this standard deviation (σ) as a threshold. As described above, this automatic threshold value setting means detects over-detection that is determined as a change pixel with respect to wave fluctuation, the influence of the western sun reflected in the wave, intense tree fluctuation, tree shadow fluctuation, and the like. It was seen.

  As a result of investigating the characteristics of the pixels with and without fluctuation, it was found that the pixels with fluctuation had a large variance in luminance used for setting the automatic threshold, and the pixels with no fluctuation had an extremely small variance.

  Therefore, in the case where the threshold value is set using the luminance standard deviation (σ), an integer multiple of 2 or more of the luminance standard deviation is set as the threshold value from the above characteristics. For example, it is considered that three times the standard deviation of luminance (3σ) is set as the threshold value. As a result, the threshold value of the pixel having a large variation is set to be larger (the threshold value of the pixel having a small variation is stably set to a smaller value).

  When this 3σ is used as a threshold value, a pixel that is constant on the time axis and has a continuously large threshold value (3σ) can be determined as a fluctuation pixel. Therefore, the final threshold value is determined by moving average. did. In addition, when using a moving average, a time lag is accompanied by the reflection to a threshold value (in order to estimate the present from the past data). Therefore, in order to reduce the influence as much as possible, a weighted moving average (weighted moving average) is used to obtain a result in which the influence of the latest (latest) calculation result is large. Further, not only the pixels that are constantly fluctuating, but also the pixels where the moving object is present temporarily, but the standard deviation of the luminance increases. Therefore, as shown in the following equation, the threshold (3σ) is used as a temporary threshold, and the weighted moving standard deviation of the threshold (3σ) is subtracted from the weighted moving average to reduce the influence as much as possible. Here, the minimum luminance value is 0 and the maximum luminance value is 255.

Luminance variance V = 1 / N × Σ (luminance) ² − (1 / N × Σ luminance) ²
Temporary threshold value (3σ) = 3 * sqrt (V), but when 255 exceeds 255, sqrt: square root threshold value = ΣWn × (3σ) / ΣWi−Sqrt (1 / N × Σ (3σ) ² −1 / N × Σ (3σ)) ² )
: (3σ weighted moving average)-(3σ standard deviation)
As a result, it has become possible to detect a stable object with stable pixels that are free from fluctuations, adapting to wave fluctuations, reflections of the western sun on the waves, and tree fluctuations. It is also possible to follow the dynamic brightness of the surroundings. This automatic setting of the threshold value can be applied to any of the original image and the edge image (spatial differential image, contour image) as the target image for which the difference is taken.

Hereinafter, embodiments of the present invention will be described with reference to FIGS.
FIG. 1 shows the overall configuration of an image processing apparatus according to an embodiment of the present invention. In this embodiment, image data for one frame (one screen) captured from a camera (monocular camera) is simply referred to as an image, an image on the screen, or an image for one screen.

  As shown in FIG. 1, the image processing apparatus according to the embodiment of the present invention includes a camera 11, a capture unit 12, an image processing storage unit 13, an image processing unit 14, and a display unit 15. Is done.

  The camera 11 includes a lens unit and an image pickup device (for example, a CCD solid-state image pickup device or a CMOS image sensor) provided at an image forming position of the lens unit, so that a subject (animal body) that moves outdoors or indoors can be used. A captured image for one screen is output to a target in a predetermined pixel unit (for example, one frame 320 × 240 pixels = QVGA).

  The capture unit 12 has a processing function of capturing an image in frame units captured by the camera 11 as a processing target image (input image) of the image processing unit 14 and holding it in an image buffer in the image processing storage unit 13. The input image on the screen captured in the image buffer is an original image here, but may be an edge image.

The image processing storage unit 13 stores various image data including the input image captured by the capture unit 12 and each image that is being processed / resulted by the image processing unit 14. In the image processing storage unit 13, under the control of the image processing unit 14, among the frame-unit images captured by the capture unit 12, an image for one screen (current image) captured this time and a plurality of images captured last time are stored. An area constituting an image buffer (ring buffer) that holds images for the screen (past images) as processing target images is secured, and an image area used for processing by the image processing unit 14 is secured. Here, one frame (one image) is 240 × 320 pixels (number of pixels in the height direction (IMG HEIGHT) x number of pixels in the width direction (IMG WIDTH)) is reserved as an image buffer (ring buffer) area for 30 frames. Further, the image processing storage unit 13 generates or obtains a storage area for storing various parameters and control data used for the processing of the image processing unit 14, registration of confirmed and unconfirmed objects, and tracking processing. A storage area for various information is also secured. Data elements for realizing the automatic threshold setting function for extracting the changed pixels in the image processing storage unit 13 will be described later with reference to FIG.

  The image processing unit 14 generates a difference binarized image obtained by binarizing the past image and the current image held in the image buffer of the image processing storage unit 13, and is included in the difference binarized image. A series of image processing is performed to remove noise, extract a rectangular area including a change pixel from the difference binarized image from which the noise has been removed, and trace the extracted rectangular area as a processing target. . The image processing unit 14 includes a preprocessing unit 141, a difference processing unit 142, a binarization processing unit 143, a noise removal filter 144, an area processing unit 145, an area tracking processing unit 146, and the like.

  Among the components of the image processing unit 14 described above, the preprocessing unit 141 compares the past image stored in the image buffer of the image processing storage unit 13 with the current image (difference processing for extracting a changed pixel region). At this time, predetermined preprocessing (smoothing processing, edge processing, etc.) is executed on the input image for one screen captured by the capture unit 12.

  The difference processing unit 142 compares the past image and the current image that have passed through the preprocessing unit 141 in pixel units, and extracts a difference region, and generates a difference image. This difference image is stored in an area (difference image area) secured in advance in the image processing storage unit 13. The threshold automatic calculation processing function according to the embodiment of the present invention is applied to this change pixel extraction.

  As processing means for realizing this threshold automatic calculation processing function, in this embodiment, an environment adaptive automatic threshold calculation processing unit 20A and a standard deviation setting type automatic threshold calculation processing unit 20B are prepared, and either one of them The threshold value calculation processing unit is selectively used by an instruction (setting) from the outside. An overview of the processing of the environment adaptive automatic threshold calculation processing unit 20A and the standard deviation setting type automatic threshold calculation processing unit 20B is shown in FIG.

  In the standard deviation setting type automatic threshold value calculation processing by the standard deviation setting type automatic threshold value calculation processing unit 20B, as shown in FIG. 3, the average luminance value and standard deviation per pixel for a plurality of input images within a certain period are input. σ is calculated, and this standard deviation σ is set as a threshold value of the corresponding pixel (target pixel) for extracting the change pixel. Details of the threshold calculation processing are described in Japanese Patent Application Laid-Open No. 2008-250892 filed by the same applicant. In the environment adaptive automatic threshold calculation processing by the environment adaptive automatic threshold calculation processing unit 20A, as shown in FIG. 3, the calculated standard deviation σ of the current pixel of interest and the oldest image within the predetermined period are the lightest and latest. A value of (3σ weighted moving average) − (3σ standard deviation) is calculated based on the moving average (weighted moving average) of the pixel of interest weighted most heavily on the image, and this value is used to extract a change pixel. Therefore, it is set as a threshold value for the corresponding pixel (target pixel). The processing procedure of the environment adaptive automatic threshold value calculation processing unit 20A will be described later with reference to the flowcharts of FIGS.

  The binarization processing unit 143 binarizes the image subjected to the difference processing by the difference processing unit 142 to generate a difference binarized image. The difference binarized image is stored in an area (difference binarized image area) secured in advance in the image processing storage unit 13. The threshold value automatic calculation processing function described above is applied to the creation of the difference binary image.

  The noise removal filter 144 is configured to include a diffusion filter and a contraction filter, and removes noise (sesame salt-like change pixels) unnecessary for extraction of the moving body region from the difference binary image. Among these, the diffusion filter sequentially sets a target pixel for each pixel except the outermost periphery of the difference binarized image, and a change pixel is present in a diagonal pixel (in four pixels) adjacent to the set target pixel. When it exists, the diffusion filtering process using the target pixel as the change pixel is performed. The contraction filter performs contraction filtering processing on the image processed by the diffusion filter based on the surrounding 8 pixels or 4 pixels adjacent to the target pixel. The difference binarized image for each frame (screen) from which noise has been removed by the noise removal filter 144 is stored in the image area of the image processing storage unit 13.

  The area processing unit 145 performs a process for converting the difference binarized image from which noise has been removed by the noise removal filter 144 into a rectangular area on the screen. In this area conversion processing, the change pixel area extracted as the difference binarized image is surrounded by a rectangle, and the rectangle area is extracted from the screen, so that the difference binarized image is displayed in the rectangle area on the screen. Turn into. The region processing unit 145 sends the upper left (x0, y0) and lower right (x1, y1) coordinate data representing the position and size of the rectangular screen to the region tracking processing unit 146 as rectangular region information. To do.

  The area tracking processing unit 146 performs processing for tracking a rectangular area based on the rectangular area information received from the area processing unit 145. Each time the area processing unit 145 extracts a rectangular area, the area tracking processing unit 146 performs a tracking process on the extracted rectangular area as a tracking process target object.

  The display unit 15 displays and outputs the moving object region image-processed by the image processing unit 14.

  The image processing storage unit 13 stores components and data elements that realize an automatic threshold setting function for extracting changed pixels as shown in FIG. As shown in FIG. 2, the image processing storage unit 13 includes an environment adaptation as a component that realizes an automatic threshold setting function for extracting a change pixel in an image processing program that controls the image processing unit 14. An automatic threshold setting processing program for realizing the type automatic threshold calculation processing unit 20A and the standard deviation setting type automatic threshold calculation processing unit 20B is stored. The processing procedure of this automatic threshold setting processing program will be described later with reference to FIGS. Further, the image processing storage unit 13 stores two sets of image buffers 131 and 132, a threshold buffer 133, a threshold statistic buffer 134, an external parameter setting file 135, and the like as information for realizing the automatic threshold setting process. .

  Each of the two sets of image buffers 131 and 132 is configured by a ring buffer, and has an area for storing, for example, a maximum of 30 frames of time-series images including the current image taken by the camera 11. Of these, 30 frames of original images (spatial differential images) captured by the camera 11 are sequentially stored in the image buffer 131 in time series. Similarly, 30 frames of edge images (contour images) are sequentially stored in the image buffer 132 in time series.

  Further, the image buffers 131 and 132 have storage areas such as an image size, a current element number of the buffer, and buffer identification information in addition to the above-described images. The image buffer 131 is designated at the head address by an input image buffer pointer (not shown) at the time of access. The image buffer 132 is designated at the head address by an edge image buffer pointer (not shown) at the time of access.

  The threshold buffer 133 is configured by a ring buffer, and a threshold calculation is performed on a set number of time-series frames with a predetermined number within the number of frames that can be stored in the image buffer 131 (or the image buffer 132) as a maximum value. And an area for storing the standard deviation (σ) per pixel of the entire captured image as a threshold value. Furthermore, it has a storage area for the brightness value of the captured image, the square of the brightness value, the sum of the squares of the brightness value, a fixed threshold, the upper and lower threshold values, the current element number of the buffer, the image buffer identification information, and the like . The threshold buffer 133 captures an image having a set number of frames from the image stored in the image buffer 131 (or the image buffer 132), and after calculating and setting the threshold per pixel of all the captured images, Image update by first-in first-out method for each image (image replacement processing for discarding the oldest image and capturing the latest image), and weight update processing for calculating a weighted moving average associated with this image update Etc. are implemented.

  FIG. 3 shows a concept of processing for calculating a threshold value per pixel for n images developed on the threshold buffer 133. Here, a buffer configuration (n = 30) for storing images for 30 frames is shown as an example.

  The threshold statistic buffer 134 is a buffer for calculating the weighted moving average used for the environment-adaptive automatic threshold calculation processing, and is applied to the weighted moving average calculation processing shown in FIG. 3, and is an image buffer (ring buffer). The threshold value information per pixel of the image stored in is stored.

  The external parameter setting file 135 includes a difference processing image type (for example, 1; original image (ORIGINAL), 2; edge image (EDGE)), the number of reference frames for calculating an automatic threshold (n; shown in FIG. 3). In the example, n = 30), various setting information such as the minimum threshold upper and lower limit values for the original image and the minimum threshold upper and lower limit values for the edge image are stored. In this embodiment, for an image (one image) for one screen, it is possible to specify an area where the tracking of the moving object is not performed as an arbitrary rectangular area, and the specified mask area is also set as a parameter as an external parameter. It is stored in the setting file 135.

  The processing procedure of the automatic threshold setting processing program for realizing the environment adaptive automatic threshold calculation processing unit 20A stored in the image processing storage unit 13 is shown in FIGS. Note that details of the processing procedure are omitted for the processing of the standard deviation setting type automatic threshold value calculation processing unit 20B.

  Here, the processing procedure of the environment adaptive automatic threshold value calculation processing unit 20A will be described with reference to the flowcharts shown in FIGS. In this flowchart, the multiplication symbol is represented by *.

  This embodiment is applied by an area extraction processing unit in an image processing apparatus including an area extraction processing unit that sequentially inputs images captured by a camera and extracts an area including a change pixel from the input current image and past image. With respect to the threshold setting process, any one of the threshold calculation processing means can be selected by selecting a standard deviation setting type automatic threshold calculation process and an environment adaptive type automatic threshold calculation process (step S1 in FIG. 4).

  Here, when the standard deviation setting type automatic threshold value calculation process is selected (set), the standard deviation setting type automatic threshold value calculation processing unit 20B calculates a threshold value for extracting the change pixel region for the target pixel. Is set. In the standard deviation setting type automatic threshold value calculation processing by the standard deviation setting type automatic threshold value calculation processing unit 20B, as shown in FIG. 3, the average luminance value per pixel and the standard value for a plurality of inputted images within a certain period of time are used. The deviation σ is calculated, and this standard deviation σ is set as a threshold value of the corresponding pixel (target pixel) for extracting the changed pixel (step S2 in FIG. 4).

  When the environment adaptive automatic threshold value calculation process is selected (set), the environment adaptive type automatic threshold value calculation processing unit 20A calculates and sets a threshold value for extracting a change pixel region for the target pixel. The In the environment adaptive automatic threshold calculation processing by the environment adaptive automatic threshold calculation processing unit 20A, as shown in FIG. 3, the calculated standard deviation σ of the current pixel of interest and the oldest image within the predetermined period are the lightest. Calculate the value of (3σ weighted moving average) – (3σ standard deviation) based on the moving average (weighted moving average) of the pixel of interest weighted most heavily on the latest image, and extract the changed pixels from this value Therefore, the threshold value is set as the threshold value of the corresponding pixel (target pixel).

  The automatic threshold value calculation processing procedure by the environment adaptive automatic threshold value calculation processing unit 20A is shown in step S11 to step S77 in FIG. This processing operation is based on the assumption that images for n frames are stored in the image buffer.

  In this environment adaptive automatic threshold value calculation process, first, member variables such as an image element number and a threshold buffer element number are initialized (step S11 in FIG. 4), and the current image to be processed in the image buffer 131 (or 132) is initialized. After obtaining the element number (step S12 in FIG. 4) and obtaining the current threshold buffer element number to be processed in the threshold buffer 133 (step S13 in FIG. 4), the image to be subjected to difference processing is the original image (ORIGINAL ) Or edge image (EDGE) (step S14 in FIG. 4).

  Here, when it is determined that the processing target image is the “original image” (step S14 in FIG. 4), the image buffer 131 is accessed according to the original image buffer pointer, and the minimum threshold for the original image given by the external parameter is exceeded. The lower limit value is set in the threshold buffer 133 (steps S15 and S16 in FIG. 4). When it is determined that the processing target image is an “edge image” (step S14 in FIG. 4), the image buffer 132 is accessed according to the edge image buffer pointer, and the minimum threshold value for the edge image given by the external parameter is set. Is set in the threshold buffer 133 (steps S17 and S18 in FIG. 4).

Next, for each pixel (pixel) excluding the mask area excluded from the processing target in the input image to be processed, a threshold value calculation process for extracting a change pixel, and a threshold value for the next screen are acquired. Various calculation processes are performed (steps S19 and S20 in FIG. 5 to steps S50 and S51 in FIG. 7). Here, each pixel is sequentially scanned in a certain direction with respect to the input image for one screen (j loop in the height direction (0 to IMG HEIGHT) and k-loops in the width direction (0 to IMG) WIDTH) is used to extract a change pixel for each pixel (pixel) except for a mask area excluded from the processing target by scanning each pixel by a loop process of image size (240 × 320 pixels). Threshold calculation processing, update of image for threshold calculation of next screen (replacement of image that discards oldest image and imports latest image), update of weighted threshold sum, and various values (standard deviation) associated with this update , Square standard deviation, luminance value, luminance value square, etc.) are calculated.

  In this process, first, the current target pixel number is calculated, and it is checked whether or not the target pixel is a mask pixel (whether or not it is a pixel in a preset mask area) (steps S21 and S22 in FIG. 5). .

  Here, when the target pixel is a mask pixel, the processing from step S23 to step S49 in FIG. 5 is skipped. When the target pixel is not a mask pixel (automatic threshold calculation target pixel), the image element number of the image buffer is incremented (+1) (step S23 in FIG. 5), and the luminance value of the input image and the luminance value The results of calculating the squares are stored in predetermined buffers (luminance value buffer and luminance value square buffer in threshold buffer 133), respectively (step S24 in FIG. 5).

  Next, the sum of the squares of the luminance values of the target pixel is obtained, and the average value of the squares of the luminance values is calculated from the sum (step S25 in FIG. 5). Similarly, the sum of the luminance values of the target pixel is obtained, and the average value of the luminance values is calculated from the sum (step S26 in FIG. 5). Based on these two average values, the standard deviation of the target pixel is obtained (step S27 in FIG. 5).

  The threshold buffer number (image element number of the image buffer) of the pixel of interest is acquired (step S28 in FIG. 6), and the threshold to be set is set using the already obtained weighted sum of thresholds, sum of thresholds, and sum of threshold squares. Calculate (Steps S29 to S32 in FIG. 6). Here, the threshold square average value (thresholdSqMean) is calculated using the already calculated threshold sum of squares (step S29 in FIG. 6), and the threshold average value is calculated using the already calculated threshold sum. (ThresholdMean) is calculated, the weighted threshold average value (weightThresholdMean) is calculated using the already calculated weighted threshold sum, the threshold squared average value (thresholdSqMean), the threshold average value (thresholdMean) and the weighted threshold A threshold (tempThreshold) to be set is calculated using the average value (weightThresholdMean).

  Next, the minimum and maximum of the calculated setting threshold are checked and set to appropriate values (steps S33 to S37 in FIG. 6). Here, when the calculated threshold value is smaller than the minimum luminance value (luminance value 0) (Tempthreshold <BRIGHTNESS_MIN), the calculated threshold value is replaced with the minimum luminance value (m_ThreBufSet.threshold [i] = BRIGHTNESS_MIN), and the calculated threshold value is the maximum luminance value When the value (brightness value 255) is greater (Tempthreshold> BRIGHTNESS_MAX), the calculated threshold value is replaced with the maximum luminance value (m_ThreBufSet.threshold [i] = BRIGHTNESS_MAX), and the calculated threshold value is within the range of the brightness value 0-255. Otherwise, set the calculated threshold value as it is (m_ThreBufSet.threshold [i] = tempThreshold).

  Next, 3σ is obtained from the standard deviation (σ) of luminance, and processing for calculating the threshold value of the next image is performed.

  In this process, a value (3σ) obtained by multiplying the standard deviation (σ) by 3 is obtained as the current threshold (calcThreshold). If the obtained current threshold is equal to or greater than the maximum luminance, the value is changed to the maximum luminance (step S41 in FIG. 7). ), The square of the obtained current threshold is obtained (step S42 in FIG. 7).

  Next, as a process associated with image update (replacement of the image that discards the oldest image and captures the latest image), the past threshold total is subtracted from the total of the past weighted threshold data, and the value obtained by multiplying the current threshold by the weight is calculated. Addition is performed to obtain the sum (step S43 in FIG. 7), the total threshold value of the target pixel is determined, and the square sum of the threshold value of the target pixel is determined (step S44 in FIG. 7). The calculated current threshold is stored in the buffer, the calculated current threshold square is stored in the buffer (step S45 in FIG. 7), and the calculated threshold storage buffer is updated (step S46 in FIG. 7).

  Next, the sum of the standard deviation and the square standard deviation for the image size for which the automatic threshold is calculated is obtained (step S47 in FIG. 7), and the oldest existing in the buffer (the current element number is set to the existing sum). The luminance value and luminance value square are subtracted, and the luminance value and luminance value square value calculated in step S2 in FIG. 5 are added (summation calculation) (step S48 in FIG. 7). Is stored in the buffer (step S49 in FIG. 7).

  The above-described processing is performed on the image for one screen (steps S50 and S51 in FIG. 7).

  After the above process, it is checked whether or not the automatic threshold value calculation pixel has reached 0. If the automatic threshold value calculation pixel is greater than 0, the following processes in FIG. 8 step S53 to FIG. When the calculated pixel reaches 0, the processing from step S53 to step S71 in FIG. 8 is not executed (step S52 in FIG. 8).

  When the automatic threshold value calculation pixel is larger than 0, the standard deviation for the automatic threshold value calculation pixel is calculated. Therefore, the average value of the standard deviation and the average value of the square standard deviation are obtained, and the standard deviation of the automatic threshold value calculation pixel is calculated. A standard deviation is obtained (step S53 in FIG. 8), and using the obtained data, an average value + 3σ is obtained and stored as a temporary minimum threshold A. (TempLowerLimitByStdev) (step S54 in FIG. 8). Further, it is checked whether or not the minimum threshold obtained here is within the luminance gradation. If not, the value is replaced with a value within the luminance gradation. That is, when the minimum threshold value is smaller than the minimum luminance value, it is replaced with the minimum luminance value, and when the minimum threshold value is higher than the maximum luminance value, it is replaced with the maximum luminance value (step S55 in FIG. 8). A value obtained by multiplying the average value of the standard deviation by the sensitivity is stored as a temporary minimum threshold value B. (TempLowerLimitByMean) (step S56 in FIG. 8). It is checked whether or not the minimum threshold value obtained here falls within the luminance gradation. If not, the values within the luminance gradation (the luminance minimum value and the luminance maximum value) are set as in step S55 in FIG. Replace (step S57 in FIG. 8).

  The lowest threshold values A and B are compared, and the larger one is adopted as the lowest threshold value (LowerLimit) (steps S58, S59, and S60 in FIG. 8). If the set minimum threshold value is smaller than the lower limit threshold value, the lower limit threshold value is set, and if it is larger than the upper limit threshold value, the upper limit threshold value is set (steps S61 to S64 in FIG. 9).

  Next, a loop is performed for the image size (steps S65 to S71 in FIG. 9), and if the threshold value of the pixel is lower than the minimum threshold value, the threshold value is replaced (steps S67 to S69 in FIG. 9).

  Thereafter, 1 is added to the current reference number in the threshold block (step S72 in FIG. 10), and it is checked whether or not the maximum reference number in the threshold block has been reached (step S73 in FIG. 10). Initialization (step S74 in FIG. 10), 1 is added to the current element number (step S75 in FIG. 10), and it is checked whether the element number exceeds the maximum value (step S76 in FIG. 10). (Step S77 in FIG. 10).

  In the image processing apparatus having an area extraction processing function for extracting a rectangular area including a change pixel from an image photographed by a monocular camera by the series of processes described above, the wave fluctuation, the reflection of the western sun on the wave, the tree shake It is possible to realize an automatic threshold value setting function that can detect a stable object with a stable pixel that does not fluctuate. Accordingly, it is possible to provide an image processing apparatus that can extract a variable pixel region that is stable and accurate with respect to various conditions such as camera installation conditions, use conditions, and imaging conditions.

  The automatic threshold setting processing program having the processing procedure shown in FIGS. 4 to 10 described above may be configured to be fixedly incorporated in a form resident in the image processing apparatus, or may be handled as needed by the program alone. It may be a configuration.

  Further, in the above-described embodiment, based on the standard deviation σ of the target pixel and the moving average (weighted moving average) of the target pixel weighted with the lightest oldest image within the fixed period and the latest image most heavily weighted (weight of 3σ) The value of (moving average) − (standard deviation of 3σ) is calculated, and this value is set as the threshold value of the corresponding pixel (target pixel) for extracting the change pixel. However, the threshold value here is limited to 3σ. For example, any of 2σ, 4σ, 5σ, and the like may be used. In short, a standard deviation value most suitable for an extraction (detection) target may be set.

  DESCRIPTION OF SYMBOLS 11 ... Camera, 12 ... Capture part, 13 ... Image processing memory | storage part, 14 ... Image processing part, 15 ... Display part, 20A ... Environment adaptive type automatic threshold value calculation processing part, 20B ... Standard deviation setting type automatic threshold value calculation processing part, 131, 132 ... Image buffer, 133 ... Threshold buffer, 134 ... Threshold statistics buffer, 135 ... External parameter setting file, 141 ... Pre-processing unit, 142 ... Difference processing unit, 143 ... Binarization processing unit, 144 ... Noise removal Filter, 145... Region processing unit, 146.

Claims (6)

In an image processing apparatus comprising region extraction processing means for sequentially inputting images taken by a camera and extracting a region including a change pixel from the input current image and past image,
In the region extraction processing means,
Standard deviation calculation means for calculating a luminance average value and standard deviation per pixel for a plurality of images within the input fixed period;
Threshold calculation means for calculating a threshold based on the calculated standard deviation and a weighted moving average of the threshold values of the corresponding pixels of the plurality of past images for the pixel for which the standard deviation has been calculated;
Threshold setting means for setting the threshold calculated by the threshold calculation means as a threshold for extracting the change pixel of the pixel for which the standard deviation has been calculated;
An image processing apparatus comprising: In an image processing apparatus comprising region extraction processing means for sequentially inputting images taken by a camera and extracting a region including a change pixel from the input current image and past image,
In the region extraction processing means,
Standard deviation calculation means for calculating a luminance average value and standard deviation per pixel for a plurality of images within the input fixed period;
Threshold calculation means for calculating a threshold based on the calculated standard deviation and a weighted moving average of the threshold values of the corresponding pixels of the plurality of past images for the pixel for which the standard deviation has been calculated;
Sets whether the standard deviation calculated by the standard deviation calculation means is set as a threshold for extracting the change pixel, or whether the threshold calculated by the threshold calculation means is set as a threshold for extracting the change pixel External setting means to
According to the setting of the external setting means, the standard deviation calculated by the standard deviation calculating means or the threshold calculated by the threshold calculating means is set as a threshold for extracting the change pixel of the pixel for which the standard deviation has been calculated. Threshold setting means;
An image processing apparatus comprising:   The threshold calculation means calculates a threshold for extracting the change pixel by subtracting the weighted moving standard deviation of the temporary threshold (3σ) from the weighted moving average of the temporary threshold (3σ) that is three times the standard deviation (σ). The image processing apparatus according to claim 1, wherein: The change pixels for causing the computer to function as an image processing apparatus having a region extraction processing function for sequentially inputting images captured by the camera and extracting a rectangular region including the change pixels from the input current image and past image. A threshold value setting processing program for setting a threshold value used for extraction for each pixel,
A standard deviation calculation function for calculating a luminance average value and a standard deviation per pixel for a plurality of images within the input fixed period;
A threshold calculation function for calculating a threshold for the pixel for which the standard deviation has been calculated, based on the calculated standard deviation and a weighted moving average of the thresholds of the corresponding pixels of the plurality of past images;
A threshold setting function for setting the threshold calculated by the threshold calculation function as a threshold for extracting the change pixel of the pixel for which the standard deviation is calculated;
A threshold setting processing program for causing the computer to realize the above. The change pixels for causing the computer to function as an image processing apparatus having a region extraction processing function for sequentially inputting images captured by the camera and extracting a rectangular region including the change pixels from the input current image and past image. A threshold value setting processing program for setting a threshold value used for extraction for each pixel,
A standard deviation calculation function for calculating a luminance average value and a standard deviation per pixel for a plurality of images within the input fixed period;
A threshold calculation function for calculating a threshold for the pixel for which the standard deviation has been calculated, based on the calculated standard deviation and a weighted moving average of the thresholds of the corresponding pixels of the plurality of past images;
Whether the standard deviation calculated by the standard deviation calculation function is set as a threshold for extracting the change pixel, or the threshold calculated by the threshold calculation means is set as a threshold for extracting the change pixel In response to the instruction, in accordance with the instruction, the standard deviation calculated by the standard deviation calculating unit or the threshold calculated by the threshold calculating unit is set as a threshold for extracting the change pixel of the pixel for which the standard deviation is calculated. A threshold setting function;
A threshold setting processing program for causing the computer to realize the above.   The threshold calculation function calculates a threshold for extracting the change pixel by subtracting the weighted moving standard deviation of the temporary threshold (3σ) from the weighted moving average of the temporary threshold (3σ) obtained by multiplying the standard deviation (σ) by three. The threshold value setting processing program according to claim 4 or 5, characterized in that:

Images