CN107240094A - A kind of visible ray and infrared image reconstructing method for electrical equipment on-line checking - Google Patents

Abstract

The invention relates to a visible light and infrared image reconstruction method used for online detection of electrical equipment. The method comprises the following steps: performing effective feature analysis on infrared images and visible light images respectively; performing image registration based on contour information; utilizing image registration According to the results of the effective feature analysis, effective feature extraction is performed on the infrared image and visible light image after image registration, and the images are superimposed to complete the reconstruction of the visible light image and the infrared image. Compared with the prior art, the present invention has the advantages of intuitively reflecting the defect characteristics of electrical equipment, being conducive to comprehensive and accurate judgment of electrical equipment defects, stable algorithm, high degree of information retention and strong applicability.

Description

A Visible Light and Infrared Image Reconstruction Method for Online Inspection of Electrical Equipment

technical field

The invention relates to electrical equipment fault detection and diagnosis, in particular to a visible light and infrared image reconstruction method for electrical equipment on-line detection.

Background technique

With the continuous expansion of the scale of my country's power grid, the requirements for the safety and reliability of power grid operation are getting higher and higher, and the long-term operation of electrical equipment in the power grid will cause wear, oxidation, corrosion, aging, pollution and defects caused by external forces. It is extremely important to maintain the safe and stable operation of the power grid by carrying out condition-based maintenance of equipment and timely troubleshooting hidden troubles.

In the field of fault detection and diagnosis of electrical equipment, non-contact detection methods such as visible light image detection and infrared image detection have been more and more widely used due to their low cost, fast speed, and non-stop maintenance. Using the outline, texture, color and other features of visible light images can identify mechanical failures of electrical equipment, such as foreign objects in power lines, broken wires, insulators falling off, displacement of anti-vibration hammers, broken spacers, etc.; use infrared images to detect abnormal temperature of electrical equipment Faults, such as poor contact of fittings, overheating of terminals, discharge and heating of outer insulation surface, etc. Although there are many types of power grid equipment, most electrical equipment has defect characteristics such as incomplete outline texture, obvious color change, and high temperature rise. Making inferences based on one-sided parameter information often leads to misdiagnosis and missed diagnosis of electrical equipment failures. For this reason, the visible light and infrared images of electrical equipment are fused and reconstructed, and the effective features of the two types of images are integrated into one image, which can not only reduce information redundancy, but also facilitate comprehensive and accurate judgment of electrical equipment failures.

Contents of the invention

The object of the present invention is to provide a visible light and infrared image reconstruction method for on-line detection of electrical equipment in order to overcome the above-mentioned defects in the prior art. Through the image registration based on the contour information of the visible light image and infrared image of the same electrical equipment, the effective features of the two images are combined to reconstruct a two-dimensional image containing both visible light and infrared image information.

The purpose of the present invention can be achieved through the following technical solutions:

A visible light and infrared image reconstruction method for online detection of electrical equipment, the method includes the following steps:

S1. Perform effective feature analysis on infrared images and visible light images respectively;

S2. Perform image registration based on contour information;

S3. Using the image registration result, effective feature extraction is performed on the infrared image and the visible light image after the image registration according to the effective feature analysis result, and the images are superimposed to complete the reconstruction of the visible light image and the infrared image.

Step S2 image registration based on contour information includes the following steps:

S201. Perform image preprocessing on the infrared image and the visible light image respectively to obtain an infrared image contour map and a visible light image contour map;

S202. Perform an optimal affine transformation search on the infrared image contour map and the visible light image contour map, and perform the optimal affine transformation;

S203. Perform optimal affine transformation on the original infrared image to complete image registration.

The effective feature analysis is specifically to select features reflecting electrical equipment defects or faults through image processing, feature processing and feature selection steps.

The optimal affine transformation search uses the average closest distance between the infrared image profile and the visible light image profile to measure the image registration degree, and the average shortest distance D (A, B) calculation formula function is:

D(A,B)=min(d(A,B),d(B,A))

Among them, A and B are the visible light contour image and infrared contour image of the electrical equipment respectively, a _i and b _i are the i-th contour point in image A and the j-th contour point in B respectively, and a _j and b _i are the image The j-th contour point in A and the i-th contour point in B, n _A , n _B are the number of contour points in images A and B respectively, d(A,B), d(B,A) are the images The average closest distance from the point on A to image B, and the average closest distance from the point on image B to image A.

The reconstruction process of visible light image and infrared image in S3 is as follows:

Among them, I _inf and I _vis are the RGB pixel values of the infrared image and the visible light image respectively, I ₁ (x, y) is the reconstructed RGB pixel value, and T(x, y) is the coordinate (x, y) on the infrared image y) pixel temperature value, T _thresh is the temperature threshold of the infrared image.

The search for the best affine transformation is specifically to find the optimal combination of parameters for affine transformation.

Compared with the prior art, the present invention has the following advantages:

1. The present invention fuses and reconstructs the visible light and infrared images of electrical equipment, integrates the effective features of the two types of images into one image, greatly reduces information redundancy, and is also conducive to comprehensive and accurate judgment of electrical equipment defects;

2. The present invention reflects the abnormal temperature rise area of the infrared image of the electrical equipment on the visible light image, which reflects the defect characteristics of the electrical equipment very intuitively, and at the same time accurately locates the defect position of the electrical equipment;

3. The algorithm of the present invention is stable, the information retention is high, and the reliability is strong, and both visible light and infrared images with similar shooting angles and image sizes can be accurately reconstructed;

4. The present invention has strong applicability, and is not only suitable for fault detection of electrical equipment such as insulators, pole towers, and fittings, but also can be applied to fields such as remote sensing, security inspection, and mechanical wear detection that require combining infrared and visible light image information.

Description of drawings

Fig. 1 is the flow chart of visible light and infrared image reconstruction of the method of the present invention;

Fig. 2 is the particle swarm algorithm flowchart of the inventive method;

Fig. 3 is the visible light image of the power pole tower of example of the present invention;

Fig. 4 is the infrared image of the electric power tower of example of the present invention;

Fig. 5 is the visible light contour diagram of the example electric power tower of the present invention;

Fig. 6 is the infrared image profile diagram of the example electric power tower of the present invention;

Fig. 7 is the effect after the registration of the visible light and infrared image contour map of the example electric power tower of the present invention;

Fig. 8 is an effect diagram of visible light and infrared image reconstruction of an example electric power tower in the present invention.

In the figure: 1. Visible light image, 2. Infrared image, 3. Effective feature analysis, 4. Image registration based on contour information, 5. Grayscale, threshold segmentation, edge extraction, 6. Visible light and infrared image contour map, 7. Optimal affine transformation search, 8. Optimal affine transformation of the original infrared image, 9. Effective feature extraction and image superposition, 10. Visible light and infrared image reconstruction.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

Example

As shown in Figure 1, the visible light and infrared image reconstruction method suitable for online detection of electrical equipment, by performing image registration 4 based on contour information on the visible light image 1 and infrared image 2 of the same electrical equipment, the effective The features are combined and reconstructed into a two-dimensional image containing both visible light and infrared image information.

The effective features can be selected after effective feature analysis. Effective feature analysis refers to the selection of features that can best represent electrical equipment defects or faults after image processing, feature extraction and feature selection. By comparing various normal and existing defects The visible light and infrared images of the electrical equipment, the effective feature of the selected infrared image is the high temperature rise area, and the effective feature of the visible light image is the outline, color and texture of the electrical equipment. In addition, the background of the visible light image is also very important for the fault location of the electrical equipment. important.

The image registration based on contour information obtains contour maps of visible light and infrared images after image preprocessing such as grayscale conversion, threshold segmentation, and edge extraction are performed on infrared images and visible light images. The threshold in the threshold segmentation refers to For adaptive gray threshold, the gray threshold is generally determined by the method of maximum variance between classes, and the edge extraction is performed by a canny edge detection operator. The visible light and infrared image contours of the target object are overlapped through the best affine transformation search, and finally the best affine transformation is performed on the original infrared image to realize the image registration process. The best affine transformation search can be achieved through particle swarm Search algorithm implementation.

Described affine transformation comprises translation transformation, scaling transformation and rotation transformation, and translation transformation matrix is:

t _x , t _y are the horizontal translation and vertical translation of the image respectively, and the matrix of the scaling transformation is:

C _x , C _y are the image horizontal expansion and vertical expansion respectively, and the matrix of rotation transformation is:

θ is the image rotation angle. The optimal affine transformation search process is to find the optimal affine transformation parameter combination (t _x0 , _ty0 , C _x0 , C _y0 , θ ₀ ), so that after the infrared profile undergoes this affine change, the infrared The overlapping effect of image contour and visible light image contour is the best.

In the particle swarm optimization algorithm, the position of the i-th particle is set as Xi = (x _i1 , x _i2 ,..., x _i5 ), and the best position it has experienced _is recorded as P _i = (p _i1 , p _i2 ,...,p _i5 ). The velocity of particle i is represented by V _i =(v _i1 ,v _i2 ,...,v _i5 ). For each generation of particles, the particles use the following formula to update their speed and position:

v ^k+1 _id ＝v ^k _id +c ₁ *rand ₁ *(p ^k _id -x ^k _id )+c ₂ *rand ₂ *(p ^k _gd -x ^k _id ) (1)

x ^k+1 _id ＝x ^k _id +v ^k+1 _id (2)

k is the number of iterations, x ^k _id is the position of the current particle, c ₁ and c ₂ are learning factors, rand ₁ and rand ₂ are random numbers in the interval [0,1]. P _g is the best position among all particles, recorded as P _g =(p _g1 ,p _g2 ,...,p _g5 ).

The image registration degree (fitness) in the best affine transformation search process with the particle swarm algorithm is measured by the average shortest distance of visible light and infrared contour images, and the average shortest distance D (A, B) calculation formula is the adaptation The degree function is:

D(A,B)=min(d(A,B),d(B,A)) (3)

Among them, A and B are the visible light contour image and infrared contour image of the electrical equipment respectively, a _i and b _i are the i-th contour point in image A and the j-th contour point in B respectively, and a _j and b _i are the image The j-th contour point in A and the i-th contour point in B, n _A , n _B are the number of contour points in images A and B respectively, d(A,B), d(B,A) are the images The average closest distance from the point on A to image B, and the average closest distance from the point on image B to image A.

The visible light and infrared image reconstruction is achieved by performing effective feature extraction and image superposition on the visible light image and infrared image according to the results of effective feature analysis after the original infrared image is subjected to the optimal affine transformation. The image superposition will The elevated regions of the infrared image are overlaid on the visible image. The reconstruction process is as follows:

I _inf and I _vis are the RGB pixel values of the infrared image and the visible light image respectively, and I is the reconstructed RGB pixel value. T _thresh is the temperature threshold of the infrared image, which is used to distinguish high-temperature rise areas, and can also be calculated by the maximum between-class variance method.

As shown in Figure 3, the visible light image of electrical equipment is taken by a visible light camera, and as shown in Figure 4, the infrared image is taken by an infrared thermal imager. When taking visible light and infrared images of electrical equipment, the shooting angles of visible light and infrared should be kept as consistent as possible. The shooting distance can be different, but the size difference of the target in the two images should not be too large.

Perform effective feature analysis 3 on the captured visible light image 1 and infrared image 2, compare the visible light and infrared images of various normal and faulty electrical equipment, and perform image preprocessing such as grayscale and threshold segmentation, and calculate Its feature quantity is selected by using the Fisher criterion to select its effective features. The effective features that need to be preserved are the electrical equipment outline, texture, color features in the visible light image, and the area and maximum value of the high temperature rise area in the infrared image.

Image registration based on contour information 4 is performed on the visible light images and infrared images of power towers captured on site as shown in Figure 3 and Figure 4, and the visible light and infrared image contours 6 are obtained through grayscale, threshold segmentation, and edge extraction 5, The gray threshold of the visible light images in Figure 3 and Figure 4 determined by the method of maximum inter-class variance is 100, the gray threshold of the infrared image is 25, and the sensitivity threshold of the visible light and infrared images is both 0.2, the visible and infrared contours of the power tower after image processing are shown in Figure 5 and Figure 6.

Use the particle swarm optimization algorithm to search for the best affine transformation of the visible light and infrared contour maps of the power tower shown in Figures 5 and 6. First, it is necessary to determine the center of gravity and area of the binary image of the visible light and infrared images after threshold segmentation. Refine the search scope. Assuming that the barycentric coordinate difference of the binary image between the visible light and the infrared image is (x ₀ , y ₀ ), and the area ratio is r ₀ , the search intervals of the infrared image’s lateral translation and vertical translation t _x and _ty are respectively [x ₀ -100, x ₀ +100] and [y ₀ -100, y ₀ +100], the search intervals of the infrared image horizontal expansion and vertical expansion C _x and C _y are both [r ₀ -0.5, r ₀ +0.5 ], the search interval of the infrared image θ rotation angle defaults to [-0.5,0.5]. The difference in barycentric coordinates of the binary images of the visible light and infrared images of the power tower is (73, 22), and the area ratio is 0.7439. Then the particle swarm algorithm is used to search for the best affine transformation, and the search process is shown in Figure 2. Normalize the search interval of affine transformation parameters to [0,1], limit the maximum speed of particle search to 0.1, initialize the particle swarm speed and position, calculate the image matching degree, that is, fitness, and record the one with the smallest fitness Particle position. Constantly iteratively update and calculate the fitness of the particle swarm until the maximum number of iterations is reached or the fitness meets the requirements. At this time, the parameter represented by the recorded particle position with the smallest fitness is the best affine transformation parameter. The maximum number of iterations is generally set to 50, and the specified fitness is generally set to 0.1.

As shown in Figure 7, after the optimal affine transformation is performed on the infrared image contour map, the infrared image contour and the visible light image contour basically overlap, and the fitness at this time is 1.3. The original image of the infrared image is subjected to optimal affine transformation 8, and the images of the three RGB components of the infrared image are transformed according to the optimal affine transformation parameters, and the process of image registration 4 based on contour information is completed.

Visible light and infrared images after image registration are completed, and effective feature extraction and image superposition are carried out9. The area where the temperature rise of the infrared image is higher than the temperature threshold is extracted from the original infrared image, and superimposed on the visible light image in an overlay manner, that is, the pixel value of the corresponding position of the RGB component of the visible light image is replaced by the RGB component of the infrared image, and the completion Visible and Infrared Image Reconstruction10. The temperature threshold of the infrared image of the power tower in Figure 4 determined by the maximum inter-class variance algorithm is 28°C, and the effect of image superposition and reconstruction is shown in Figure 8.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of various equivalents within the technical scope disclosed in the present invention. Modifications or replacements shall all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (6)

1. A visible light and infrared image reconstruction method for online detection of electric equipment, is characterized in that, the method comprises the following steps: S1. Perform effective feature analysis on infrared images and visible light images respectively; S2. Perform image registration based on contour information; S3. Using the image registration result, effective feature extraction is performed on the infrared image and the visible light image after the image registration according to the effective feature analysis result, and the images are superimposed to complete the reconstruction of the visible light image and the infrared image. 2. A visible light and infrared image reconstruction method for online detection of electrical equipment according to claim 1, wherein the image registration based on contour information in step S2 comprises the following steps: S201. Perform image preprocessing on the infrared image and the visible light image respectively to obtain an infrared image contour map and a visible light image contour map; S202. Perform an optimal affine transformation search on the infrared image contour map and the visible light image contour map, and perform the optimal affine transformation; S203. Perform optimal affine transformation on the original infrared image to complete image registration. 3. A visible light and infrared image reconstruction method for on-line detection of electrical equipment according to claim 1, wherein said effective feature analysis is embodied through image processing, feature processing and feature selection steps A characteristic of a defect or failure in electrical equipment. 4. A kind of visible light and infrared image reconstruction method that is used for electrical equipment on-line detection according to claim 2, is characterized in that, described optimal affine transformation search adopts the infrared image contour map and the visible light image contour map The average shortest distance is used to measure the image registration degree, and the calculation formula function of the average shortest distance D(A,B) is: D(A,B)=min(d(A,B),d(B,A)) <mrow> <mi>d</mi> <mrow> <mo>(</mo> <mi>A</mi> <mo>,</mo> <mi>B</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mn>1</mn> <msub> <mi>n</mi> <mi>A</mi> </msub> </mfrac> <munder> <mo>&amp;Sigma;</mo> <mrow> <msub> <mi>a</mi> <mi>i</mi> </msub> <mo>&amp;Element;</mo> <mi>A</mi> </mrow> </munder> <munder> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> <mrow> <msub> <mi>b</mi> <mi>j</mi> </msub> <mo>&amp;Element;</mo> <mi>B</mi> </mrow> </munder> <mo>|</mo> <mo>|</mo> <msub> <mi>a</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>b</mi> <mi>j</mi> </msub> <mo>|</mo> <mo>|</mo> </mrow> <mrow> <mi>d</mi> <mrow> <mo>(</mo> <mi>B</mi> <mo>,</mo> <mi>A</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mn>1</mn> <msub> <mi>n</mi> <mi>B</mi> </msub> </mfrac> <munder> <mo>&amp;Sigma;</mo> <mrow> <msub> <mi>b</mi> <mi>i</mi> </msub> <mo>&amp;Element;</mo> <mi>B</mi> </mrow> </munder> <munder> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> <mrow> <msub> <mi>a</mi> <mi>j</mi> </msub> <mo>&amp;Element;</mo> <mi>A</mi> </mrow> </munder> <mo>|</mo> <mo>|</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>a</mi> <mi>j</mi> </msub> <mo>|</mo> <mo>|</mo> </mrow> Among them, A and B are the visible light contour image and infrared contour image of the electrical equipment respectively, a _i and b _i are the i-th contour point in image A and the j-th contour point in B respectively, and a _j and b _i are the image The j-th contour point in A and the i-th contour point in B, n _A , n _B are the number of contour points in images A and B respectively, d(A,B), d(B,A) are the images The average closest distance from the point on A to image B, and the average closest distance from the point on image B to image A. 5. A kind of visible light and infrared image reconstruction method for on-line detection of electrical equipment according to claim 1, is characterized in that, the visible light image and infrared image reconstruction process in S3 is as follows: <mrow> <msub> <mi>I</mi> <mn>1</mn> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <msub> <mi>I</mi> <mrow> <mi>i</mi> <mi>n</mi> <mi>f</mi> </mrow> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mrow> </mtd> <mtd> <mrow> <mi>T</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>&amp;GreaterEqual;</mo> <msub> <mi>T</mi> <mrow> <mi>t</mi> <mi>h</mi> <mi>r</mi> <mi>e</mi> <mi>s</mi> <mi>h</mi> </mrow> </msub> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mi>I</mi> <mrow> <mi>v</mi> <mi>i</mi> <mi>s</mi> </mrow> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mrow> </mtd> <mtd> <mrow> <mi>T</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>&lt;</mo> <msub> <mi>T</mi> <mrow> <mi>t</mi> <mi>h</mi> <mi>r</mi> <mi>e</mi> <mi>s</mi> <mi>h</mi> </mrow> </msub> </mrow> </mtd> </mtr> </mtable> </mfenced> </mrow> Among them, I _inf and I _vis are the RGB pixel values of the infrared image and the visible light image respectively, I ₁ (x, y) is the reconstructed RGB pixel value, and T(x, y) is the coordinate (x, y) on the infrared image y) pixel temperature value, T _thresh is the temperature threshold of the infrared image. 6. A visible light and infrared image reconstruction method for online detection of electrical equipment according to claim 2, characterized in that the search for the best affine transformation is specifically to find the optimal parameters for affine transformation combination.