TWI389060B - Intelligent monitoring system and method thereof - Google Patents

Description

Intelligent monitoring system and method thereof

The invention relates to a monitoring system and a method thereof, in particular to a smart monitoring system and a method thereof.

The recent increase in crime rates has led people to pay attention to the importance of the preservation mechanism. The most important part of the security mechanism is the surveillance system. Its biggest function is to solve the problem of insufficient personnel in the security, so that users can quickly detect changes in environmental matters. A good monitoring system can achieve the goal of ensuring safety, reducing the degree of injury and reducing manpower waste.

Image motion detection technology has the ability to judge when an image changes. If the image motion detection technology is applied to a digital monitoring system or a closed circuit television (CTVTV) monitoring system, the monitoring system can start the recording function when the determination screen changes, so the image can be saved. Storage space. At the same time, a warning message can be generated to remind the security personnel to pay attention to the event, and to achieve the purpose of monitoring automation.

The resolution size refers to the number of imaging pixels of the picture. Please refer to the second figure (a), which shows an object. Please refer to the second figure (b), which shows the result of magnifying eight times after the object of the second figure (a) is photographed by a low resolution camera. Please refer to the second figure (c), which shows the result of the object of the second figure (a) after being photographed by a high-resolution camera. It can be clearly seen that the imaging size will be different when shooting the same object with different resolution cameras. If the image size of a low-resolution image is changed by digital zooming, there is a problem of blurring distortion, so it cannot be applied to security monitoring.

To solve the above problem, the user has traditionally used two cameras of the same resolution. One of the cameras (panorama camera) is stationary and is responsible for monitoring the panorama to establish the background of the surveillance area. After the current image is subtracted from the background to obtain the position of the moving object, the panoramic camera then informs another PTZ (Pan-Tile-Zoom) camera with wide viewing angle selection and adjustable zoom parameters for moving object tracking, accurately positioning and image zooming. . This allows for high-resolution images of moving objects.

The camera that controls the lens is called a PTZ camera, also known as a Speed Dome camera. PTZ cameras use motor-driven machine rotation and lens expansion to achieve the desired movement and magnification. However, the process of image taking is often limited by the drive and mechanical structure, and the machine itself is easily lost due to long-term monitoring. Moreover, the automatic zoom function of the PTZ camera also affects the camera taking time, and when the magnification is too high, it is easy to track the moving object and easily exceed the lens range and cause misjudgment.

In order to improve the above disadvantages of the prior art, it is an object of the present invention to provide an intelligent monitoring system.

In order to improve the above disadvantages of the prior art, it is still another object of the present invention to provide an intelligent monitoring method.

To achieve the above objective of the present invention, the intelligent monitoring system of the present invention comprises: an image sensor, a data temporary storage memory, a digital image processor and a program memory. The digital image processor is electrically connected to the image sensor, the data temporary storage memory and the program memory. The image sensor captures an original image and transmits the image to the digital image processor. The digital image processor adjusts the original image to obtain an adjusted image. The digital image processor detects one of the adjusted images to adjust the motion image. The digital image processor captures one of the original motion images in the original image according to the positioning information of the adjusted motion image.

In order to achieve the above object of the present invention, the intelligent monitoring method of the present invention utilizes an image sensor to capture an original image. The original image is adjusted to obtain an adjusted image. One of the adjusted images is detected to adjust the motion image. According to the positioning information of the motion image, one of the original motion images in the original image is captured.

The intelligent monitoring system and method thereof of the invention are combined with digital image processing technology, which can replace the PTZ camera and achieve the same benefit or even exceed the technology. It can identify and track people in specific scenarios, and send out alerts immediately so that relevant personnel can handle them immediately. All video files such as monitoring scenes, personnel movement characteristics, and fixed area monitoring are stored in the database as a research application for subsequent event retrieval.

The intelligent monitoring system and method thereof of the invention utilizes low-resolution panoramic images for object tracking, and then obtain high-definition tracked object images from high-resolution images, thereby replacing traditional PTZ cameras for intelligent monitoring systems. First, the high-resolution panoramic image is preserved and a low-resolution panoramic image is generated for smart moving object detection and tracking positioning, and then repositioned on the original high-resolution panoramic image, and the digital object is automatically digitally scaled. The intelligent monitoring system and the method thereof can effectively solve the problem of tracking dead angle and delay of the traditional PTZ camera, and can also track multiple objects at the same time, and eliminate the mechanical wear caused by the rotating mechanism and the influence of excessive reaction time to build Set up a smart instant unmanned surveillance system.

Please refer to the first figure, which is a block diagram of the intelligent monitoring system of the present invention. The intelligent monitoring system of the present invention comprises: an image sensor 10, a data temporary storage memory 20, a digital image processor 30, a program memory 40 and an image output device 50. The digital image processor 30 is electrically connected to the image sensor 10, the data temporary memory 20, the program memory 40, and the image output device 50.

The image sensor 10 captures an original image (not shown) and transmits it to the digital image processor 30. The digital image processor 30 adjusts the original image to obtain an adjusted image (not shown). The digital image processor 30 reconstructs the background as needed. The digital image processor 30 detects one of the adjusted images (not shown). The digital image processor 30 repairs and tags the adjusted motion image as needed to obtain a tagged image (not shown). The digital image processor 30 captures an original moving image (not shown) in the original image according to the positioning information of the tagged image; if the adjusted moving image does not need to be patched and labeled by the digital image processor 30 The digital image processor 30 captures one of the original motion images in the original image according to the positioning information of the adjusted motion image. Finally, the original moving image is displayed or stored by the image output device 50. The image sensor 10 is a high-resolution fixed camera; the image output device 50 is a display or a video device. The detailed operation of the intelligent monitoring system of the present invention will be described later in conjunction with the flowchart.

The digital image processor 30 adjusts the original image to: adjust the size of the original image to a predetermined size, perform color conversion on the original image, and filter out dot noise. First, the panoramic high-resolution image (that is, the original image) is retained, and the resolution is further reduced (for example, to 320×240 pixels) to reduce the amount of data calculation. The high-resolution image data is huge. Taking the 5 million pixel camera as an example, the original resolution of each image is 2560×1920 pixels, and it is a full-color 24-bit RGB image. Therefore, the amount of data calculation on image processing is too large, and it will cause too long reaction time for real-time monitoring. Therefore, according to an example of the present invention, the resolution of the panoramic high-resolution picture (ie, the original image) can be reduced to 320×240 pixels. The full-color 24-bit RGB picture is reduced to a grayscale 8-bit Gray Level picture to reduce the amount of data calculation. Finally, the noise is filtered out and can be achieved by the Image Low Pass Filter.

Among them, the so-called background is a very small difference in continuous image. Since the software program may have moving objects, or there are sunlight and light in the environment, the background image obtained is not complete, and the background must be reconstructed. When the incomplete part of the background is still for a while, the background can be re-established. When the difference between the background and the continuous image is found to exceed a certain level, the background reconstruction mechanism must be restarted to avoid affecting the subsequent moving object detection.

The digital image processor 30 detects the adjusted motion image in the adjusted image by using a fixed background image and three consecutive frames of adjusted images, which are detailed below. It uses a continuous three images with the previously established fixed background to calculate the real moving object. Please refer to the third figure, which is a schematic diagram of detecting a motion image adjustment method in the adjusted image. The grayscale image of the current frame is marked as M2; the grayscale image of the previous frame is indicated as M1; the grayscale image of the first two frames is denoted as M0; and the fixed background image is denoted by M3. And setting a binarized image threshold value as a basis for converting the grayscale images into a binary image. First, the current frame grayscale image M2 and the previous frame grayscale image M1 are mutually exclusive or (XOR) operated to obtain a first grayscale image (not shown). Then, comparing the pixels of the first grayscale image with the binarized image threshold: if the pixel of the first grayscale image is greater than or equal to the binarized image threshold, it is set as a bright spot; If the pixel of a grayscale image is smaller than the binarized image threshold, it is set as a dark point, thereby obtaining a first binarized image M5.

Similarly, the previous frame grayscale image M1 and the first two frame grayscale image M0 are mutually exclusive or (XOR) operated to obtain a second grayscale image (not shown), and the binary value is further The image threshold value is compared to obtain a second binarized image M4. By performing an AND operation on the first binarized image M5 and the second binarized image M4, a third binarized image M7 of the edge of the moving object is obtained. Furthermore, the previous frame grayscale image M1 and the fixed background image M3 are subjected to image subtraction operations to obtain a fourth grayscale image (not shown), and then compared with the binarized image threshold value. A fourth binarized image M6 of the foreground object separated from the background is obtained. Finally, the third binarized image M7 and the fourth binarized image M6 are subjected to an OR operation to obtain a fifth binarized image M8 of the most complete moving object (ie, the adjusted moving image) ).

The digital image processor 30 is described below to repair and label the adjusted motion image to obtain a tagged image flow. Please refer to the fourth figure, which is a flow chart for repairing and labeling the motion image. First, in steps S10 and S20, since the fifth binarized image M8 may still be a bit broken, it is necessary to perform image repair using Dilation and Erosion of image processing morphology.

Dilation is the most basic computational operation. Assume that in the plane, there is an image A1 and a mask B1, as shown in the thirteenth figure. The expansion calculation of the image A1 and the mask B1 is defined as: the mask B1 is scanned from left to right along the image A1, and is scanned from left to right (as shown in FIG. 14), if the mask B1 is in the mask B1 If there is any pixel in the range (slash) whose value of binarization is 1 (the diagonal part of the image A1), the center point of the mask of the image A1 is set to 1, otherwise 0. The image A1 is expanded by the mask B1 and the result is as shown in the fifteenth figure. The area after expansion is the part marked as X.

Erosion is another operation in morphology. Assume that in the plane, there is an image A1 and a mask B1, as shown in the thirteenth figure. The erosion calculation of the image A1 and the mask B1 is defined as: when the mask B1 is scanned from left to right along the image A1 (as shown in FIG. 14), if the mask is in the mask When there is any pixel in the range of the cover B1 (slash), if the value of the binarization is 0 (the non-slashed portion in the image A1), the center point of the mask of the image A1 is set to 0, otherwise it is 1. The image A1 is eroded by the mask B1 and the result is as shown in the sixteenth figure. The area after erosion is marked as part of X.

Considering that the monitoring system is mainly for monitoring human behavior, unlike the general pattern of Dilation and erosion (Erosion) with 3×3, 5×5 masking the basic structure, the method of the present invention adopts the fifth figure 3x3 The mask structure (Mask) performs erosion calculation; the sixth figure 3x1 mask structure is used for the expansion calculation. First, a 3×3 erosion (Erosion) is used to eliminate the residual noise from the complex background. For example, the seventh picture (a) shows the image originally filled with noise; the seventh picture (b) shows the image after the 3×3 erosion (Erosion) is eliminated. Then, in order to combine the possible separation of the head, the trunk, the foot and the like, three times of 3×1 expansion is obtained, and a complete image of the last moving object is obtained, as shown in the eighth figure.

Following steps S30 and S40, since there may be more than two moving objects, the number of moving objects must be calculated in a labeling manner. The main purpose of Connected Components Labeling is to aggregate adjacent groups of points into the same number to calculate the number of groups, which is the number of moving objects in the present invention. The ninth figure shows a schematic diagram of eight-adjacent. According to the eight-connected rule, from left to right, the binarized image is scanned from top to bottom, and all adjacent binarized pixels with a value of 1 are given the same and unique label symbol, by the number of label symbols. Calculate the number of moving objects. As shown in the tenth figure (where the right figure of the tenth figure is slashed, it can be clearly seen that there are three moving objects, that is, moving objects respectively given the label symbols 1, 2, and 3). Here, the aforementioned tagged images can be obtained.

Then, the digital image processor 30 captures an original motion image in the original image according to the positioning information of the tagged image. Through the above labeling process, the starting point position and the last point position of the object can be known, and the object size and the relative position of the object on the screen can be calculated. The positioning information is transmitted to the original high-resolution image (ie, the original image) to display the original image size (ie, the original moving image) on the image output device 50, and the digital zoom effect is achieved without amplification.

The intelligent monitoring system of the present invention finally displays or stores the original moving image. Please refer to the eleventh figure, which is an example of the image output of the present invention. The upper left corner of the eleventh figure shows a panoramic picture 320×240 pixels (that is, for example, the adjusted image whose original resolution is 2560×1920 pixels is reduced to 320×240 pixels), and includes a first moving object Ma, A second mobile object Mb and a user setting monitoring area A1 (details of the user-customizable monitoring area are detailed later). Corresponding to the image sensor, the first moving object Ma, the second moving object Mb, and the user setting monitoring area A1 captured by the panoramic picture 320×240 pixels shown in the upper left corner of the eleventh figure The position and size of the original resolution picture of 10 (ie, the original image, for example, 2560×1920 pixels), the first moving object Ma on the original resolution picture is presented in the 320×240 pixel size range, and the second movement The object Mb and the user set the monitoring area A1. For example, the size of the second moving object Mb in the upper left corner of the eleventh picture is 10×20 pixels, and is fixed by a fixed camera set with 5 million pixels (ie, 2560×1920 pixels), and is presented in the lower left sub-picture. It will be 80×160 pixels. In the case where no scaling is performed, the magnifications of eight times the width and height are achieved (as shown in the lower left corner of the eleventh figure). It is equal to the security personnel to operate a PTZ camera, and manually track the effect of moving objects at eight times the width and height. In addition, if the size of the tracked moving object relative to the original resolution of the camera exceeds 320×240 pixels or less than 320×240 pixels, the image processing will be automatically scaled to an appropriate size. For example, the first moving object Ma has a circle of 50×50 pixels in the upper left corner of the eleventh figure, and is 400×400 pixels with respect to the original resolution picture, and the object is presented in a 320×240 pixel picture. Therefore, the automatic digital zoom is adjusted. Finally, the 640×480 pixels are recorded and recorded to achieve the intelligent unmanned real-time monitoring system.

In addition to intelligent tracking and monitoring of moving objects, the intelligent monitoring system of the present invention can also select (and customize the scaling) scenes to be monitored, such as entrances and exits or no-smoking areas. First, the user sets the monitoring area in the adjusted image. Then, the digital image processor 30 captures the original monitoring area image in the original image according to the positioning information of the monitoring area. Finally, the original monitoring area image is displayed or stored by the image output device 50.

Please refer to the twelfth figure, which is a flow chart of the smart monitoring method of the present invention. First, in step S100, an image sensor is used to capture an original image. Then, the original image is adjusted in step S200 to obtain an adjusted image. Step S300 then reconstructs the background as needed. Following step S400, one of the adjusted images is detected to adjust the motion image. Then, in step S500, the adjusted motion image is repaired and tagged as needed to obtain a tagged image. Step S600 then captures an original motion image in the original image according to the positioning information of the tagged image. (If the motion image is not to be repaired and tagged, the step S600 is based on the positioning of the adjusted motion image. Information, grabbing one of the original motion images in the original image). The final step S700 displays or stores the original moving image. The image sensor is a high resolution fixed camera; the image output device is a display or a video device.

Step S200 (adjusting the original image) includes: adjusting a size of the original image to a predetermined size, performing color conversion on the original image, and filtering out dot noise. First, the panoramic high-resolution image (that is, the original image) is retained, and the resolution is further reduced (for example, to 320×240 pixels) to reduce the amount of data calculation. According to an embodiment of the present invention, the resolution of the panoramic high resolution picture (i.e., the original image) can be reduced to 320 x 240 pixels. The full-color 24-bit RGB picture is reduced to a grayscale 8-bit Gray Level picture to reduce the amount of data calculation. Finally, the noise is filtered out and can be achieved by the Image Low Pass Filter.

Step S400 (detecting one of the adjusted images in the adjusted image) uses a fixed background image and three consecutive frames of adjusted images to detect the adjusted moving image in the adjusted image. The flow is the same as that of the aforementioned intelligent monitoring system of the present invention, and thus will not be described herein.

Step S500 (repairing and labeling the adjusted motion image to obtain a label image process) is performed by digital image processing erosion (one erosion by 3x3 mask structure) and expansion (three expansions by 3x1 mask structure) The calculus technique fixes the adjusted motion image. And because there may be more than two moving objects, the number of moving objects must be calculated by labeling.

In addition to intelligent tracking and monitoring for mobile objects, the intelligent monitoring method of the present invention can also select (and customize the scaling) scenes to be monitored, such as entrances and exits or no-smoking areas. First, the user sets the monitoring area in the adjusted image. Then, according to the positioning information of the monitoring area, the original monitoring area image in the original image is captured. Finally, the original surveillance area image is displayed or stored.

The intelligent monitoring system and method of the invention have the following characteristics:

1. Take the 5 million pixel camera as an example. The panoramic video is full color 2560×1920 pixels, which consumes a lot of system resources and occupies a large amount of storage. The video frame of the present invention is 640×480 pixels, only the original resolution is required. With one-fold storage, you can achieve high-definition presentation of moving objects and user-defined areas to achieve the highest efficiency.

2, can replace the PTZ camera, exempt the mechanism, no monitoring dead angle, no movement delay and the ability to detect more than two moving objects at the same time.

3, a fixed high-resolution camera can replace a low-resolution camera plus a low-resolution PTZ camera, reducing the cost of human and material resources, greatly improving the quality of life, home security, making the monitoring system more efficient and Universal application in a variety of environments.

4. The invention has the advantages of easy expandability, easy integration, universal product, low cost, high quality and easy operation, and can be operated by non-professionals.

In summary, it is known that the present invention has industrial applicability, novelty and advancement, and the structure of the present invention has not been seen in similar products and public use, and fully complies with the requirements of the invention patent application, and is filed according to the patent law.

A1. . . image

B1. . . Mask

10. . . Image sensor

20. . . Data temporary storage memory

30. . . Digital image processor

40. . . Program memory

50. . . Image output device

M2. . . The current frame grayscale image is marked as

M1. . . The grayscale image of the previous frame is marked as

M0. . . The first two frames of grayscale images are marked as

M3. . . The fixed background image is marked as

M5. . . First binarized image

M4. . . Second binarized image

M7. . . Third binarized image

M6. . . Fourth binarized image

M8. . . Fifth binarized image

S10～S40. . . step

Ma. . . First moving object

Mb. . . Second moving object

A1. . . User setting monitoring area

S100～S700. . . step

The first figure is a block diagram of the intelligent monitoring system of the present invention.

The second figure (a) is a schematic view of an object.

The second figure (b) is an eight-fold image image obtained after shooting with a low-resolution camera.

The second picture (c) is a post-image image taken by a high-resolution camera.

The third figure is a schematic diagram of detecting a method for adjusting motion images in the adjusted image.

The fourth picture is a flow chart for repairing and labeling the adjusted motion image.

The fifth picture shows the 3x3 mask structure.

The sixth picture shows the 3x1 mask structure.

Figure 7 (a) is an image of the original noise.

Figure 7 (b) is an image of the noise removal after erosion.

The eighth picture shows the image sequence after three expansions.

The ninth figure is a schematic diagram of eight connections.

The tenth figure is a schematic diagram of the labeling material.

Figure 11 is a diagram showing an example of image output of the present invention.

The twelfth figure is a flow chart of the smart monitoring method of the present invention.

The thirteenth picture is an example of the Dilation and Erosion calculations.

Figure 14 is a schematic diagram of Dilation and Erosion scanning.

The fifteenth picture is a schematic diagram after the Dilation calculus.

The sixteenth figure is a schematic diagram of the Erosion calculus.

10. . . Image sensor

20. . . Data temporary storage memory

30. . . Digital image processor

40. . . Program memory

50. . . Image output device

Claims (14)

An intelligent monitoring system comprising: an image sensor; a digital image processor electrically connected to the image sensor; a data temporary storage memory electrically connected to the digital image processor; a program memory The image is electrically connected to the digital image processor; and an image output device is electrically connected to the digital image processor, wherein the image sensor captures an original image and transmits the image to the digital image processor; the digital image The processor adjusts the original image to obtain an adjusted image; the digital image processor detects one of the adjusted images to adjust the motion image; and the digital image processor captures the positioning information according to the adjusted motion image. An original moving image in the original image; the image output device displays or stores the original moving image; wherein the digital image processor adjusts the original image to adjust the size of the original image to a predetermined size, the original image Perform color conversion to filter out point noise. For example, the intelligent monitoring system of claim 1 is wherein the image sensor is a high resolution fixed camera. The intelligent monitoring system of claim 1, wherein the image output device is a display or a video device. A smart monitoring method that includes: A. using an image sensor to capture an original image; B. adjusting the original image to obtain an adjusted image; C. detecting one of the adjusted images to adjust the motion image; D. Positioning information of the image, capturing an original moving image in the original image; and E. displaying or storing the original moving image, wherein the step B comprises: b1. adjusting the size of the original image to a predetermined size; b2. Color conversion of the original image; and b3. filtering out point noise. For example, the smart monitoring method of claim 4, wherein the predetermined size described in the step b1 is 320x240 pixels. For example, the smart monitoring method of claim 4, wherein the step b2 is reduced from full color (24 bit RGB) to gray level (8 bit Gray Level). For example, the smart monitoring method of claim 4, wherein the step b3 is to filter out the dot noise through an image low pass filter. For example, the smart monitoring method of claim 4, wherein the step C uses a fixed background image and three consecutive frames of adjusted images to detect the adjusted moving image in the adjusted image. For example, the smart monitoring method of claim 4, wherein the step C further comprises: C1. patching and labeling the adjusted motion image to obtain a labeling Image; C2. According to the positioning information of the tagged image, capture one original motion image in the original image; and C3. display the original motion image. For example, the smart monitoring method of claim 9 includes: C4. storing the original moving image. For example, in the smart monitoring method of claim 9, wherein the step C1 repairs the adjusted motion image by using an Erosion and Dilation algorithm of digital image processing. For example, the intelligent monitoring method of claim 11 is to perform an Erosion calculus with a 3x3 mask structure and a Dilation calculus with a 3x1 mask structure (Mask). For example, the smart monitoring method of claim 4, wherein the step B further comprises the following steps: B1. setting the monitoring area in the adjusted image; B2. capturing the original image according to the positioning information of the monitoring area. The original monitoring area image; and B3. Display the original monitoring area image. For example, the smart monitoring method of claim 13 includes: B4. storing the original monitoring area image.