CN118612406A - An abnormal state detection method for Internet of Things image sensors - Google Patents

Abstract

The invention discloses a method for monitoring the state of a monitoring camera based on a data mining technology and an image processing algorithm, and relates to the field of image processing. The system comprises the following steps: initializing a camera to collect image and sound data of surrounding environment, converting the collected image into a gray image and calculating average brightness, separating RGB color channels of the image, performing bilateral filtering noise reduction treatment, performing Fourier transformation on the noise reduced image, centralizing a frequency spectrum, identifying interference and blurring phenomena of the image through amplitude spectrum analysis and Laplacian edge detection, and finally generating a report containing abnormal details. Compared with the traditional monitoring system, the system provided by the invention has the characteristics of instantaneity, accuracy and easiness in implementation, does not need to carry out hardware transformation on a camera, has higher user friendliness and wide applicability, and discloses the potential of detecting sensor abnormality through a software algorithm in the field of Internet of things.

Description

An abnormal state detection method for Internet of Things image sensors

Technical Field

The present invention belongs to the technical field of abnormal monitoring of Internet of Things (IoT) devices, and focuses on real-time monitoring of the status of cameras through image processing technology. By using methods such as image noise reduction, Fourier transform, brightness detection and edge detection to conduct in-depth analysis and comparison of the collected data, the present invention can effectively determine whether there is an abnormality in the working status of the device. This method not only improves the accuracy of monitoring, but also has important significance for timely diagnosis and response to potential equipment problems, thereby playing a key role in ensuring the stability and reliability of the IoT system.

Background Art

With the rapid development and widespread application of Internet of Things (IoT) technology, a large number of smart devices are deployed in various environments to collect and transmit key data. These devices, including cameras, play a vital role in monitoring systems, smart homes, industrial automation and other fields. However, due to various factors such as environmental changes, equipment aging, external interference, etc., sensors may experience performance degradation or failure, resulting in data anomalies, which not only affects the accuracy and reliability of the system, but may also cause safety risks and economic losses.

Traditional sensor anomaly detection methods usually rely on fixed thresholds or simple statistical analysis, which may not be accurate or robust enough when dealing with complex or changing environmental data. In addition, these methods often lack real-time and adaptability, and have difficulty responding to rapid changes in device status in a timely manner. Therefore, studying a technology that can detect and diagnose sensor anomalies in real time, accurately, and adaptively is of great significance to improving the performance and reliability of IoT devices.

The present invention aims to solve the above problems by combining advanced data mining technology and image processing algorithms, especially algorithms including image noise reduction, Fourier transform, brightness detection and edge detection, to conduct in-depth analysis of the hardware characteristics and real-time data stream of the sensor, thereby achieving real-time monitoring and abnormality detection of the device status. Through this method, the abnormal status of the sensor can be effectively identified, and corresponding processing measures can be taken to improve the stability and security of the Internet of Things system.

Summary of the invention

The present invention aims at the problem of abnormal state detection of image sensors (i.e., cameras) in Internet of Things (IoT) devices and proposes an abnormal state detection method for IoT image sensors. The method conducts in-depth analysis of the sensor data stream to detect and determine in real time whether the device is in an abnormal state.

During the implementation process, the hardware device is initialized first. Then, the real-time data stream of the sensor is collected through a specific hardware device, the initial frame is read, and a stable average brightness value is obtained by continuously reading 50 frames of images and calculating the average brightness of each frame. A frame of image is obtained from the camera, the three color channels of RGB of the image are separated, and each color channel is subjected to noise reduction processing using a bilateral filter, and the three channels after noise reduction are merged back into one image. Then, the noise-reduced image is converted into a grayscale image, the grayscale image is subjected to Fourier transform, and the result is moved to the center of the spectrum (the zero-frequency component is moved to the center). Then, the amplitude of the Fourier transform result is calculated, and logarithmic scaling is used to enhance the display effect. The average brightness of the current frame is calculated and compared with the average brightness calculated previously to detect whether there is external interference. The Laplace operator is used to detect the edge of the image, and the variance of the Laplace operator result is calculated to detect whether the image is blurred. Finally, the system will display the obtained data in real time and whether the device operating environment is normal.

After confirming that there is a sensor with suspected abnormality, the system will promptly feedback the abnormality to the user until the user handles the abnormality, thereby confirming its normal working status.

The invention not only improves the accuracy and real-time performance of sensor anomaly detection, but also enhances the ability to respond to abnormal conditions, which is of great significance for ensuring the stable operation of IoT devices and data security.

The anomaly detection method of the present invention is as follows:

As an important component of IoT devices, the normal working state of image sensors is crucial to the performance of the entire system. The hardware characteristics of the sensor, such as frequency response, signal-to-noise ratio, pixel quality, etc., are key indicators for judging its working state. During the operation of the sensor, these characteristics will change due to factors such as device aging, environmental changes, or external interference, which may cause data anomalies.

The present invention collects sensor data streams in real time and analyzes these data using algorithms such as image noise reduction, Fourier transform, brightness detection and edge detection to identify abnormal modes that are inconsistent with normal working modes.

Once an abnormal point is detected, the present invention will further determine the cause of the abnormality and promptly feedback relevant information to the user.

The core advantage of the present invention is that it can monitor the sensor status in real time and adaptively, and improve the accuracy and efficiency of anomaly detection through strategies such as frequency domain analysis and interference detection.

The technical solution adopted by the present invention to achieve the above-mentioned object is:

Step 1. After the system is started, the data stream of the image sensor is acquired in real time through the data acquisition module connected to the sensor;

Step 2. Preprocess the collected data stream, including filtering, denoising and normalization, to facilitate subsequent analysis;

Step 3. Using the preprocessed data, read the initial frame and convert it into a grayscale image;

Step 4. Calculate the average brightness of the grayscale image by continuously reading 50 frames of images and calculating the average brightness of each frame to obtain a stable average brightness value;

Step 5. Get a frame of image from the camera, separate the three color channels of RGB of the image, use a bilateral filter to perform noise reduction on each color channel, and merge the three channels after noise reduction back into one image.

Convert the denoised image into a grayscale image and perform Fourier transform on the grayscale image:

in is the rotation factor, and moves the result to the center of the spectrum (the zero-frequency component is moved to the center), calculates the magnitude of the Fourier transform result, and uses logarithmic scaling to enhance the display effect;

Step 6. Calculate the average brightness of the current frame and compare it with the previously calculated average brightness to detect whether there is external interference. Use the Laplacian operator to perform edge detection on the image:

Δf＝▽ ² f＝▽·▽f

Where f is a second-order differentiable real function, ▽f is the gradient, ▽·f is the divergence, and the variance of the Laplace operator result is calculated:

Among them, x is the average value of the group of data, n is the number of data, and the variance of the Laplace operator result is calculated to detect whether the image is blurred.

Further, data integration and sensor status assessment stage:

Integrate the processed data and conduct a comprehensive analysis based on the sensor's historical performance data, environmental monitoring data, etc. to assess whether the sensor's status is abnormal;

Abnormal reporting and decision support stage:

Generate anomaly reports to indicate the time, location and possible causes of the anomaly, and provide decision support, such as adjusting monitoring strategies, repairing or replacing sensors, etc.

In the above technical solution, we further adopt advanced data mining technology and image processing algorithms to improve the accuracy and adaptability of anomaly detection.

Furthermore, the system can be configured to promptly provide abnormal information to the user when an abnormality is detected, and guide the user to perform precise positioning and inspection.

The beneficial effects of the present invention are:

Real-time monitoring: The present invention can timely and accurately identify abnormal conditions of image sensors by collecting sensor data in real time. In addition, the system can adaptively adjust the threshold to adapt to data changes and the characteristics of different sensors, thereby improving the accuracy and efficiency of anomaly detection.

Versatility and scalability: The present invention does not rely on the detailed electromagnetic radiation characteristics of a specific sensor, so it has high versatility and can be applied to various types of image sensors. With the continuous development of IoT technology, new sensor types and application scenarios continue to emerge. The present invention can adapt to these changes by updating algorithms and parameters, maintaining its leading position in the field of sensor anomaly detection.

User-friendly operation experience: The present invention provides an intuitive user interface, allowing users to observe the sensor status through simple operations. When the system detects an abnormality, it will issue a clear alarm prompt to guide the user to take necessary treatment measures, greatly simplifying the user's operation process.

System integrability: The anomaly detection system of the present invention can be easily integrated into existing IoT devices and monitoring platforms, providing an additional layer of security for the devices. By seamlessly integrating with existing systems, the present invention enhances the reliability and stability of the entire IoT ecosystem.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a block diagram of an abnormality detection and diagnosis system for a device according to the present invention;

FIG2 is a diagram showing the working principle of an image sensor;

FIG3 is a flowchart of a process for processing the collected image data accordingly;

FIG4 is the amplitude spectrum of the environment image after Fourier transformation;

FIG5 is a diagram showing the user interface when the camera is disturbed;

DETAILED DESCRIPTION

In order to describe the present invention in more detail, it is described below in conjunction with the accompanying drawings.

1 , the system is mainly composed of a data acquisition and preprocessing module, an image processing and analysis module, an anomaly detection module, a state assessment and decision support module, and a user interface and report generation module.

Data collection and preprocessing stage:

Step 1: The data acquisition module is started and runs. The user turns on the data acquisition device, and the module obtains the data stream of the image sensor in real time. The working principle of the image sensor is shown in Figure 2.

Image processing and analysis stage:

Step 2: Process the received raw data stream. First, separate the acquired color image into independent red (R), green (G), and blue (B) channels, and then apply a bilateral filter to each channel for denoising to remove image noise without losing important edge information. Next, merge the denoised images and convert them into grayscale images to simplify data processing. Thereafter, the module performs Fourier transform to convert the image from the spatial domain to the frequency domain, and moves the zero-frequency component (i.e., the brightness information of the image) to the center of the spectrum through spectrum centering. Finally, calculate the amplitude of the transformation result, and use logarithmic scaling technology to enhance the display effect, providing visual frequency domain data for subsequent image analysis and anomaly detection. This module not only improves the image quality, but also lays the foundation for the anomaly detection function of the system. The workflow of this stage is shown in Figure 3.

Anomaly detection phase:

Step 3: The abnormality analysis module analyzes the preprocessed data by reading and processing the camera video stream in real time, aiming to identify potential problems in the camera image. First, this stage calculates the grayscale average brightness of the current frame and compares it with the predetermined average brightness benchmark to detect whether there is a brightness anomaly caused by interference from an external light source. Secondly, the Laplace operator is used to detect the edge of the image, and the image clarity is evaluated by calculating the variance of the Laplace result to determine whether the image has become blurred due to camera shaking or inaccurate focus. If an abnormality is detected, the system will generate a report containing the details of the abnormality, including the type of abnormality, the time of occurrence, and the possible cause, to provide decision support for taking appropriate maintenance or adjustment measures. This not only enhances the reliability of the monitoring system, but also improves the real-time monitoring capability of the equipment status.

Status assessment and decision support phase:

Step 5: The condition assessment module integrates the processed data and focuses on providing comprehensive evaluation and recommendations based on the anomaly detection results. In this stage, the system first classifies the detected anomalies, such as external interference or image blur, and analyzes their possible causes. Then, based on this information, the system generates a detailed anomaly report that clearly indicates the nature of the anomaly, when it occurred, where it is, and its possible impact. In addition, this stage also provides decision support, such as recommended maintenance measures, adjustments to monitoring strategies, or replacement of sensors to ensure the stability and safety of the system. Through this proactive evaluation and recommendation mechanism, the condition assessment and decision support stage not only improves the efficiency of problem response, but also helps prevent future failures and optimize long-term system performance.

User interface and report generation phase:

Step 6: Display the processed image and amplitude spectrum, as shown in Figure 4, provide a user interaction interface, generate and display anomaly reports, and provide decision support based on abnormal situations. Figure 5 is the feedback interface when the camera is disturbed.

The image sensor anomaly detection method of the present invention has the advantages of real-time monitoring, adaptive diagnosis and user-friendly operation. By collecting and analyzing sensor data in real time, the system can identify abnormal conditions in a timely and accurate manner. In addition, the constructed anomaly detection algorithm can adapt to different types of sensors and has high versatility. Users can perform anomaly detection through simple operations, and the system will also provide clear alarm prompts to help users deal with abnormal situations. This efficient and universal method greatly improves the monitoring capability and abnormal response efficiency of IoT devices, and provides a new solution for ensuring data security and stable operation of equipment.

The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as above in the preferred embodiment, it is not intended to limit the present invention. Any technician who is familiar with or not skilled in the art can make many possible changes and modifications to the technical solution of the present invention by using the technical content disclosed above, or modify it into an equivalent embodiment of equivalent changes without departing from the scope of the technical solution of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention should fall within the scope of protection of the technical solution of the present invention.

Claims (10)

1. Patent name: an abnormal state detection method for an image sensor of the Internet of things comprises the following steps:

Step 1: the equipment is started, and sound and image data in the surrounding environment are collected through the camera and the microphone.

Step 2: the initial frame is read, converted into a gray image, and the average brightness of the gray image is calculated.

Step 3: and reading a frame of image from the camera, separating three color channels of the image RGB, carrying out noise reduction treatment on each color channel by using a double filter, and merging the noise reduced channels into one image.

Step 4: the noise-reduced image is converted into a gray-scale image, fourier transform is performed on the gray-scale image, and the result is shifted to the center of the spectrum (zero-frequency component is shifted to the center).

Step 5: the magnitude of the fourier transform result is calculated and logarithmic scaling is used to enhance the display effect.

Step 6: the average brightness of the current frame is calculated and compared with the previously calculated average brightness to detect whether there is external interference.

Step 7: and carrying out edge detection on the image by using the Laplace operator, and calculating the variance of the Laplace operator result to detect whether the image is blurred.

Step 8: an exception report is generated indicating the time, location, and possible cause of the exception occurrence and providing decision support, such as adjusting monitoring strategies, servicing or replacing sensors, etc.

2. The abnormal state detection method for the image sensor of the internet of things according to the step 1 of the present invention is characterized in that the data output by the sensor is collected in real time through a data acquisition system connected with a sensor network interface. The original data is preprocessed, including noise removal, missing value filling, data normalization, and the like, so that subsequent analysis is facilitated.

3. The abnormal state detection method for the image sensor of the internet of things according to step 2 of claim 1, wherein the initial frame of the camera is read and converted from a color image to a gray image, which helps to simplify the subsequent processing process; the average brightness of the gray scale image is calculated, which can be used as a reference for the brightness comparison of the subsequent images. In addition, because the external environment changes at any time, the system can collect data again to update the normal brightness standard every one minute according to the change of the external environment.

4. The method for detecting abnormal states of an image sensor facing the internet of things according to step 3 of claim 1, wherein a new frame of image is read from a camera, a color image is separated into three independent channels of red (R), green (G) and blue (B), each separated color channel is subjected to noise reduction processing by using a bilateral filter to remove image noise, and the noise-reduced color channels are recombined to form a new image.

5. The abnormal state detection method for an image sensor of the internet of things according to step 4 of claim 1, wherein the noise-reduced image is converted into a gray image so as to perform frequency domain analysis, fourier transform is performed on the gray image, the image is converted from a spatial domain to a frequency domain, and a result of the fourier transform is moved to a center of a spectrum so that a low frequency component (typically brightness information of the image) is located at a center position of the spectrum.

6. The method for detecting abnormal states of an image sensor facing the internet of things according to step 5 of claim 1, wherein the amplitude of the fourier transform result is calculated, and logarithmic scaling is used to enhance the display effect, so that the dynamic range of the amplitude spectrum is more suitable for visualization.

7. The method for detecting abnormal states of an image sensor facing the internet of things according to step 6 of claim 1, wherein the average brightness of the gray image of the current frame is calculated again and compared with the average brightness calculated in step 2, and if the current brightness is significantly lower than the average brightness, it may indicate that the camera is subject to external interference.

8. The method for detecting an abnormal state of an image sensor facing the internet of things according to step 7 of claim 1, wherein the image is edge-detected by using a laplace operator, and a variance of a laplace operator result is calculated, and if the variance is lower than a certain threshold, the image may be indicated as blurred.

9. The method for detecting abnormal states of an image sensor facing the internet of things according to step 8 of claim 1, wherein a detailed abnormality report is generated based on the evaluation results of step 6 and step 7, including the time, location, possible cause and influence of occurrence of the abnormality. The report should provide decision support information such as recommended maintenance measures, adjustment of monitoring strategies, or advice to replace sensors.

10. An abnormal state detection method for an image sensor of the Internet of things comprises the following steps:

The data acquisition and preprocessing module is used for acquiring a camera image, converting the camera image into a gray level image and calculating initial brightness;

The image processing and analyzing module is used for carrying out image noise reduction, fourier transformation and amplitude spectrum calculation on the acquired data stream;

The system can dynamically adjust the threshold values of the brightness and the Laplace operator variance according to actual conditions so as to improve the detection accuracy;

The state evaluation and decision support module is used for performing state evaluation on the abnormal detection result, generating an abnormal report and providing corresponding decision support and suggestion;

the user interface and report generation module is used for displaying the processed image and amplitude spectrum, providing a user interaction interface, generating and displaying an abnormal report and providing decision support according to the abnormal condition.

Through the cooperative work of the modules, the real-time monitoring and the effective diagnosis of the abnormal state of the audio and image sensors in the Internet of things equipment are realized, the accuracy and the real-time performance of the abnormal detection are improved, and powerful guarantee is provided for the stable operation and the data safety of the Internet of things equipment.