CN119624761A

CN119624761A - Power distribution panel display method and device based on computer vision

Info

Publication number: CN119624761A
Application number: CN202411334613.9A
Authority: CN
Inventors: 陈一童; 杨志敏; 吴桂龙; 刘林; 冯晓芳; 黄昱; 张思拓
Original assignee: China Southern Power Grid Co Ltd
Current assignee: China Southern Power Grid Co Ltd
Priority date: 2024-09-24
Filing date: 2024-09-24
Publication date: 2025-03-14
Anticipated expiration: 2044-09-24
Also published as: CN119624761B

Abstract

The present invention provides a method and device for displaying a power distribution panel based on computer vision, the method comprising: using a camera to obtain a multi-angle entity image of the power distribution panel; extracting matching feature points in the multi-angle entity image, and mapping the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the camera; densifying the three-dimensional sparse point cloud to obtain a dense point cloud; constructing a distribution panel static model of the power distribution panel according to the dense point cloud; and displaying the real-time acquired operation information of the power distribution panel in the distribution panel static model. Through the above method, the operation data of the power distribution panel is mapped to the distribution panel static model in real time, the remote intelligent monitoring and management level is improved, the power grid operation decision is optimized, and the asset digitization of the power distribution panel is realized.

Description

Computer vision-based electric power distribution panel display method and device

Technical Field

The invention relates to the technical field of digital twinning, in particular to a method and a device for displaying an electric power distribution panel based on computer vision.

Background

In the process of new power system construction and digital transformation, power communication equipment such as a power distribution panel plays an irreplaceable role as an infrastructure of the power industry. Under the operation and maintenance business scene of communication equipment, the requirement of the digitization of the assets of the communication equipment is higher and higher, and a large number of businesses with strong mechanical property, high repeatability, high operation risk prevention and control difficulty and long operation time exist in the communication production line equipment. At present, development of digital application research of electric power communication assets is needed, and overall management efficiency is improved.

Disclosure of Invention

The invention provides a display method and device of an electric power distribution panel based on computer vision, which are used for mapping operation data of the electric power distribution panel into a static model of the distribution panel in real time, improving remote intelligent monitoring and management level, optimizing operation decisions of a power grid and realizing asset digitization of the electric power distribution panel.

The invention provides a computer vision-based electric power distribution panel display method, which comprises the following steps:

acquiring a multi-angle physical image of the power distribution panel by using the camera equipment;

extracting matching feature points in the multi-angle entity image, and mapping the matching feature points into three-dimensional sparse point clouds according to the space coordinates of the camera equipment;

performing point cloud densification on the three-dimensional sparse point cloud to obtain dense point cloud;

Constructing a static distribution panel model of the power distribution panel according to the dense point cloud;

And displaying the operation information of the power distribution panel acquired in real time into the static distribution panel model.

According to the computer vision-based power distribution panel display method provided by the invention, the matching characteristic points in the multi-angle entity image are extracted, and the matching characteristic points are mapped into three-dimensional sparse point clouds according to the space coordinates of the camera equipment, and the method comprises the following steps:

identifying a distribution panel area in the multi-angle physical image;

Dividing the distribution panel area according to GrabCut algorithm to obtain a foreground image of the power distribution panel;

extracting dense feature points in the foreground image, and carrying out feature point pair matching on the dense feature points to obtain matched feature points;

Rejecting mismatching feature points in the matching feature point pairs;

and mapping the matching characteristic points with the mismatching characteristic points removed into a three-dimensional sparse point cloud according to the space coordinates of the image pickup equipment.

According to the computer vision-based power distribution panel display method provided by the invention, the three-dimensional sparse point cloud is subjected to point cloud densification to obtain dense point cloud, and the method comprises the following steps:

Acquiring depth maps of the multi-angle entity image under different angles;

and according to the image field information acquired in the depth map, performing patch expansion and patch filtering on the three-dimensional sparse point cloud to obtain the dense point cloud.

According to the invention, a computer vision-based power distribution panel display method is provided, the power distribution panel static model is constructed according to the dense point cloud, and the method comprises the following steps:

And constraining the re-projection error of the dense point cloud according to a light speed adjustment method, and reconstructing a curved surface of the dense point cloud according to a poisson surface reconstruction algorithm to obtain the static model of the distribution panel.

According to the computer vision-based power distribution panel display method provided by the invention, the dense characteristic points in the foreground image are extracted, and the dense characteristic points are subjected to characteristic point pair matching to obtain a plurality of matching characteristic points, and the method comprises the following steps:

Extracting dense feature points in the foreground image by adopting a harris angular point detection algorithm;

and tracking and matching the dense feature points by using a KLT algorithm to obtain a plurality of matched feature points.

According to the method for displaying the electric power distribution panel based on computer vision provided by the invention, before the dense feature points in the foreground images are extracted, the method further comprises the following steps:

an emphasize operator is used to enhance the image contrast of the foreground image.

According to the computer vision-based power distribution panel display method provided by the invention, the step of eliminating the mismatching characteristic points in the matching characteristic points comprises the following steps:

Selecting a preset number of sample feature points from the matching feature points, wherein the preset number of sample feature points are not collinear;

constructing a matrix model according to the preset number of sample feature points;

calculating the projection error of each matching characteristic point and the matrix model;

and if the projection error is greater than or equal to a preset error threshold, the matching characteristic points are taken as mismatching characteristic points to be removed.

According to the invention, the electric power distribution panel display method based on computer vision further comprises the following steps:

Sparse feature points in the foreground image of the power distribution panel are extracted by adopting a SURF algorithm to obtain the space coordinates of the image pickup equipment.

The invention also provides a computer vision-based power distribution panel display device, comprising:

An image acquisition unit for acquiring a multi-angle physical image of the power distribution panel;

The feature point matching unit is used for extracting matching feature points in the multi-angle entity image and mapping the matching feature points into a three-dimensional sparse point cloud according to the space coordinates of the camera equipment;

the characteristic point processing unit is used for carrying out point cloud densification on the three-dimensional sparse point cloud to obtain dense point cloud;

A model building unit for building a static model of the power distribution panel from the dense point cloud;

And the information display unit is used for displaying the operation information of the electric power distribution panel acquired in real time into the static model of the distribution panel.

The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a computer vision based power distribution panel presentation method as described in any of the above when executing the program.

The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a computer vision based power distribution panel display method as described in any of the above.

The invention also provides a computer program product comprising a computer program which when executed by a processor implements a computer vision based power distribution panel display method as described in any one of the above.

According to the method and the device for displaying the power distribution panel based on the computer vision, provided by the invention, the static model of the distribution panel is constructed through the computer vision technology, and the real-time operation information of the power distribution panel is integrated into the static model of the distribution panel, so that the intuitiveness and the efficiency of operation and maintenance management are improved. The acquisition, the processing and the display of the multi-angle entity image of the power distribution panel are realized, and the digitizing level of the power distribution panel is improved.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a computer vision-based display method for an electric power distribution panel according to the present invention.

Fig. 2 is a schematic diagram of an operation information identification of an electric power distribution panel according to the present invention.

Fig. 3 is a second flow chart of a computer vision-based display method for an electric power distribution panel according to the present invention.

Fig. 4 is a schematic diagram of a principle of acquiring a three-dimensional sparse point cloud based on a triangulation principle.

Fig. 5 is a third flow chart of a computer vision-based electric power distribution panel display method according to the present invention

Fig. 6 is a flow chart of a computer vision-based display method for electric power distribution panels according to the present invention.

Fig. 7 is a schematic diagram of a static model of a distribution board according to the present invention.

Fig. 8 is a schematic diagram of the result of a computer vision based display device for the power distribution panel.

Fig. 9 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the description so used is interchangeable under appropriate circumstances such that the embodiments are capable of operation in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules that are expressly listed or inherent to such process, method, article, or apparatus. The naming or numbering of the steps in the present application does not mean that the steps in the method flow must be executed according to the time/logic sequence indicated by the naming or numbering, and the execution sequence of the steps in the flow that are named or numbered may be changed according to the technical purpose to be achieved, so long as the same or similar technical effects can be achieved. The division of the modules in the present application is a logical division, and may be implemented in another manner in practical applications, for example, a plurality of modules may be combined or integrated in another system, or some features may be omitted or not implemented, and further, coupling or direct coupling or communication connection between the illustrated or discussed modules may be through some interfaces, and indirect coupling or communication connection between the units may be electrical or other similar manners, which are not limited in the present application. The modules or sub-modules described as separate components may or may not be physically separate, may or may not be physical units, or may be distributed in a plurality of circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

The details of the present invention are described below in conjunction with fig. 1-9.

Fig. 1 is a schematic flow chart of a computer vision-based electric power distribution panel display method according to an embodiment of the present invention, which includes steps S101-S105.

Step S101, acquiring a multi-angle physical image of the power distribution panel by using the image capturing apparatus.

In an embodiment of the invention, the image capturing apparatus includes, but is not limited to, a camera, with which a multi-angle physical entity image of the distribution panel to be reconstructed is acquired, forming an initial image dataset.

Step S102, extracting matching feature points in the multi-angle entity image, and mapping the matching feature points into three-dimensional sparse point clouds according to the space coordinates of the image pickup equipment.

And step S103, carrying out point cloud densification on the three-dimensional sparse point cloud to obtain dense point cloud.

Step S104, a static model of the distribution panel of the power distribution panel is constructed from the dense point cloud.

Step S105, displaying the operation information of the electric power distribution panel acquired in real time into the static model of the distribution panel.

In the following, the process of acquiring operation information of the power distribution panel will be described in an embodiment of the present invention.

In the embodiment of the invention, a camera is utilized to timely shoot and acquire a front color image of the power distribution panel in real time, a fast R-CNN deep learning network is used for identifying and extracting objects such as a switch, a terminal, characters and a status lamp from the real-time image, and the object anchor frames are analyzed to acquire the operation information of the distribution panel.

The method comprises the steps of obtaining a certain number of images of a distribution panel, carrying out anchor frame marking on objects representing operation states such as switches, terminals, characters, status lamps and the like by utilizing a labelimg tool box to customize an image training set, inputting the image training set into a Faster R-CNN network to carry out training, inputting an image set to be identified, and identifying the operation states of the objects such as the switches, the terminals, the characters, the status lamps and the like by utilizing the Faster R-CNN network.

As shown in figure 2, for the images of the text areas selected by the interest anchor frame, the characters in the images are extracted and identified by utilizing an OCR technology, and the working state of a distribution panel is accurately obtained.

Finally, the text results are presented as running information of the power panel to the panel static model.

Fig. 3 is a second flowchart of a display method of an electric power distribution panel based on computer vision according to an embodiment of the present invention.

In a possible implementation manner, in step S102, the matching feature points in the multi-angle entity image are extracted, and the matching feature points are mapped into a three-dimensional sparse point cloud according to the spatial coordinates of the image capturing apparatus, as shown in fig. 3, including steps S301 to S305.

Step S301 identifies a distribution panel area in the multi-angle physical image.

In an embodiment of the present invention, the panel extraction anchor frame area image, i.e., the panel area, in the initial image is identified using the fast R-CNN network. The method comprises the steps of performing anchor frame calibration on the area occupied by a multi-angle entity image by utilizing a labelimg tool box, manufacturing an image training set of an electric power distribution panel, inputting the image training set into a Faster R-CNN network for training, inputting an image set to be identified, and framing the distribution panel area in the multi-angle entity image by utilizing the Faster R-CNN network.

Step S302, dividing the panel area according to the GrabCut algorithm, to obtain a foreground image of the power panel.

Specifically, the GrabCut algorithm is utilized to carry out semantic segmentation and background deletion on the image in the anchor frame, and a foreground image of the distribution panel is obtained.

The method includes the steps of taking an image of a distribution panel area obtained in the step S301 as input, defining a foreground area and a background area, carrying out Gaussian Mixture Modeling (GMM) on pixel points in the image of the distribution panel area, gathering foreground pixels and background pixels into K types by using a K-means algorithm to serve as K initialization models in the GMM, constructing a minimum energy formula of the following formula, and carrying out continuous iterative optimization based on a minimum segmentation or maximum flow algorithm to enable energy to tend to be minimum, so that image segmentation is completed.

Wherein  is a transparency coefficient, setting a background of 1;k epsilon {1,2, ··k }, a model number of GMM, θ= (μ, δ2) is a mean and covariance of GMM, z represents a pixel, D represents a matching degree of the transparency coefficient  and the pixel z, and is minimum when correctly classified, V is a boundary term reflecting similarity of adjacent pixels, and minimum when image boundary.

Step S303, extracting dense feature points in the foreground image, and carrying out feature point pair matching on the dense feature points to obtain matching feature point pairs.

And step S304, eliminating mismatching feature points in the matching feature point pairs.

Step S305, mapping the matching feature points, from which the mismatching feature points are removed, into a three-dimensional sparse point cloud according to the spatial coordinates of the image capturing apparatus.

In the embodiment of the application, dense characteristic points are extracted by utilizing a harris angular point detection algorithm, the matching of characteristic point pairs is completed by combining a KLT tracking algorithm, and mismatching characteristic points are removed by utilizing a Ransac random consistent sampling algorithm. Finally, based on the principle of triangulation, as shown in fig. 4, the three-dimensional space coordinates of the feature points can be calculated by using the space coordinates of the camera and the pixel coordinates of the matched feature points, and the three-dimensional sparse point cloud is mapped. See in particular the examples below.

In one possible implementation manner, the dense feature points in the foreground image are extracted, and feature point pairs are matched on the dense feature points, so as to obtain matched feature point pairs, which includes step S303A and step S303B.

And step S303A, extracting dense feature points in the foreground image by adopting a harris corner detection algorithm.

The method specifically comprises the steps of taking a window function omega (x, y) with a center as a coordinate (x, y) as a window to simultaneously perform tiny sliding in the x and y directions of a foreground image, calculating a pixel gray value variation E (u, v) in the window, and expanding the pixel gray value variation E by a two-dimensional Taylor formula, and calculating a corner response function in the window: Wherein H is a corner response function, and lambda 1 and lambda 2 are eigenvalues of the matrix. According to the threshold value threshold set in advance, it is determined that when H > threshold, the pixel point is regarded as a dense feature point.

And step S303B, tracking and matching the dense feature points by using a KLT algorithm to obtain matched feature point pairs.

The method specifically comprises the steps of tracking and matching dense characteristic points between images by utilizing a KLT algorithm, and judging whether a matched characteristic point pair exists in a window by comparing squares of pixel gray level differences in a preset window of two foreground images, wherein the implementation flow is as follows:

1. Feature points of the first picture are detected.

2. And carrying out translation or mapping on each characteristic point in the second picture to carry out motion track estimation.

3. And (5) tracking dense characteristic points according to the motion trail.

Before feature tracking, a square feature window W needs to be defined around the dense feature points, and the side length is h. Corresponding points defining the same point in 2 foreground images are (ux, uy), (ux+dx, uy+dy) respectively, wherein dx and dy are the image motion directions, namely the change directions of objects. If the two points match, then in window W, foreground image A takes e-h/2 as the window, foreground image B takes e+h/2 as the window, and the square difference of gray scale is minimum. The offset of the dense feature points is d= [ dx, dy ] T, the projection points are e= [ ux, uy ] T, and the square difference epsilon of the gray scale is expressed as an integral expression:

Wherein I (e) and J (e) are gray values on the image I, J, namely the gray values on the image 1 and the image 2, respectively, and W (e) is a function of the window W. And performing deviation derivation according to the direction d, and obtaining the minimum value of epsilon when the result is 0.

In one possible embodiment, before extracting the dense feature points in the foreground image, further comprising enhancing the image contrast of the foreground image with emphasize operators.

Fig. 5 is a third flow chart of a display method of an electric power distribution panel based on computer vision according to an embodiment of the invention.

In a possible implementation manner, in step S304, the mismatching feature points in the matching feature points are eliminated, as shown in fig. 5, including steps S501-S504.

In step S501, a preset number of sample feature points are selected from the matching feature points, and the preset number of sample feature points are not collinear.

Step S502, constructing a matrix model according to a preset number of sample feature points.

Illustratively, 4 sample points are randomly extracted from the set of matching feature point pairs, which sample points may not be collinear, and then a transformation matrix is calculated and recorded as a matrix model N.

In step S503, a projection error between each matching feature point and the matrix model is calculated.

Illustratively, the distance between the matching point and the matrix model N is calculated according to the matching point, and the average value of the calculated distances is calculated and set as a threshold Tr.

In step S504, if the projection error is greater than or equal to the preset error threshold, the matching feature points are removed as mismatching feature points.

For example, projection errors of all points in the matched feature point pair set and the model N are calculated, and if the projection errors are smaller than a threshold value Tr, the projection errors are added into the feature point set Bn. And if the projection error is greater than or equal to a preset error threshold, the matching characteristic points are taken as mismatching characteristic points to be removed.

In embodiments of the invention. If the number of elements of Bn is greater than the optimal number bn_best, then let bn_best=bn, and update the iteration number at the same timeThe method can be calculated by the following formula:

Where p is the confidence level, typically 0.995, n is the proportion of the feature point set, and z is the sample point number, 4. And ending when the iteration number is greater than kB, otherwise, adding one to the iteration number, and continuing to repeat the steps S501-S504.

Fig. 6 is a flowchart of a method for displaying an electric power distribution panel based on computer vision according to an embodiment of the present invention.

In a possible implementation manner, in step S103, the three-dimensional sparse point cloud is subjected to point cloud densification, so as to obtain a dense point cloud, as shown in fig. 6, including steps S601-S602.

Step S601, obtaining depth maps of multi-angle entity images under different angles.

In embodiments of the invention. A depth map is an image or image channel that contains information about the distance from each point in the scene to the point of observation, typically in gray scale, where lighter pixels represent objects closer to the point of observation and darker pixels represent objects farther away.

Step S602, according to the image field information obtained in the depth map, performing surface patch expansion and surface patch filtering on the three-dimensional sparse point cloud to obtain a dense point cloud.

Initializing matching features, namely extracting feature points in the image, searching for the feature points in the entity image of the I angle for the matching feature points in the entity images of other angles to form matching point pairs, and forming three-dimensional space points by utilizing triangulation.

The surface patch expansion, namely sorting the formed three-dimensional space points from small to large according to the distance from the optical center O (I) of the entity image at the I angle one by one until the surface patch is generated, and initializing the center cI (p), the normal vector nI (p) and the corresponding reference image RI (p) of the surface patch;

dough filtering-discontinuous dough that does not meet acceptable consistency criteria is filtered using three filters.

And repeating the steps to perform iterative patch expansion and patch screening on the point cloud to finally obtain a dense point cloud.

In a possible embodiment, in step S103, constructing a static model of the distribution panel of the electric power distribution panel from the dense point cloud comprises constraining the re-projection error of the dense point cloud according to the speed of light adjustment method and reconstructing the curved surface of the dense point cloud according to the poisson surface reconstruction algorithm to obtain the static model of the distribution panel.

Specifically, the error is minimized based on the beam adjustment theory, and the Levebberg-Marquardt algorithm in the least square method is used to reduce the re-projection error, as shown in the following formula:

Where K represents the number of spatial points, L represents the number of camera angles, Represent the firstThe three-dimensional point is calculated to be at the firstThe camera plane projection at a single angle,Representing three-dimensional points in spaceTo the cameraThe predicted projection point of the plane, d (x, y), represents the euclidean distance between the image points x and y. And (3) carrying out curved surface reconstruction on the dense point cloud by using a poisson surface reconstruction algorithm to form a static model of the distribution panel as shown in fig. 7.

In one possible implementation manner, the invention provides a computer vision-based power distribution panel display method, which further comprises the step of extracting sparse feature points in a foreground image of a power distribution panel by adopting a SURF algorithm to obtain space coordinates of an image pickup device.

The method specifically comprises the steps of extracting sparse feature points in a foreground image by using a SURF algorithm, forming 64-dimensional descriptors based on the characteristics of the harr wavelet for each sparse feature point, and specifically realizing the following steps:

1. building a Gaussian scale space and calculating a Hessian matrix . Given a point (x, y) in the foreground image, a Hessian matrixOn the scale ofThe specific expression is:

Wherein the method comprises the steps of Representing the foreground image at (x, y) and the gaussian second order derivativeIs used for the convolution of (a),Similarly.

2. And comparing each pixel point processed by the Hessian matrix with 26 points in the two-dimensional image space and the scale space neighborhood, preliminarily positioning key points, filtering out the key points with weaker energy and the key points positioned in error, and screening out final stable sparse feature points.

3. And determining the main direction distribution of the sparse feature points and generating 64-bit descriptors of the feature points.

The method comprises the steps of calculating the closest distance d1 and the next closest distance d2 between any two features in different pictures, and judging that the two features are matched when the ratio of the closest distance d1 to the next closest distance d2 is smaller than a certain threshold value. The distance L is expressed as:

Wherein, ,The ith bit of the 64-bit descriptor is respectively different from the first bit.

Finally, based on the sparse feature points (namely the matching feature points) which are successfully matched, calculating an intrinsic matrix by using a five-point method, and obtaining camera external parameters, namely camera pose, so as to obtain the space coordinates of the camera equipment, wherein the method comprises the following specific implementation steps:

1. five pairs of matching feature points are randomly selected to calculate an eigen matrix E, and the eigen matrix E is shown in the following formula:

Wherein, 、And、The coordinates of a matching feature point pair in the two foreground images, respectively.

2. And (3) carrying out singular value decomposition on the eigenvalue E, and calculating a rotation matrix R and a translation matrix T of the camera.

3. The world coordinates, namely the space coordinates of the camera shooting equipment, when the camera shoots the images can be obtained by solving the matching characteristic points among the foreground images and combining the corresponding relations among the coordinate systems. The following formula is shown:

wherein M1 represents an internal reference matrix of the camera and is obtained by a Zhang Zhengyou calibration method, and M2 represents an external reference matrix of the camera and is obtained by mainly obtaining the intrinsic matrix and performing singular value decomposition.

In the embodiment of the invention, a multi-angle entity image can be input into a Camera Calibrator toolbox in Matlab software to calculate and acquire camera parameters, namely distortion parameters, camera focal length and the like, so as to obtain the camera reference matrix.

By adopting the method, the invention can achieve at least one of the following beneficial effects:

1. Through computer vision technology, a static model of the power distribution panel is constructed, and real-time operation information of the power distribution panel is integrated into the static model of the power distribution panel, so that intuitiveness and efficiency of operation and maintenance management are improved. The acquisition, the processing and the display of the multi-angle entity image of the power distribution panel are realized, and the digitizing level of the power distribution panel is improved.

2. By identifying, dividing and extracting the feature points of the distribution panel area of the multi-angle entity image, and eliminating the mismatching feature points, the precision and quality of the three-dimensional sparse point cloud are improved, and the accuracy and reliability of the follow-up three-dimensional modeling are ensured.

3. By utilizing the depth map information, the three-dimensional sparse point cloud is densified, and the detail richness of the distribution panel model is further improved.

4. The light speed adjustment method and the poisson surface reconstruction algorithm are adopted, so that the re-projection error of the dense point cloud is effectively restrained, the high-precision curved surface reconstruction is realized, and the accuracy and the practicability of the static model of the distribution panel are improved.

5. Through the combination of the harris angular point detection algorithm and the KLT algorithm, the effective extraction and matching of dense feature points in the foreground image are realized, and the accuracy and stability of feature point pairs are improved.

6. Before dense feature points are extracted, contrast of a foreground image is enhanced through emphasize operators, image quality is improved, and accuracy and reliability of feature point extraction are further improved.

7. By constructing a matrix model and calculating projection errors, mismatching characteristic points are effectively removed, the precision and quality of the three-dimensional sparse point cloud are further improved, and a more reliable data base is provided for follow-up three-dimensional modeling and operation and maintenance management.

The computer vision-based power distribution panel display device provided by the invention is described below, and the computer vision-based power distribution panel display device described below and the computer vision-based power distribution panel display method described above can be referred to correspondingly.

Fig. 8 is a schematic structural diagram of a computer vision-based display device for electric power distribution panels according to the present invention, including:

An image acquisition unit 810 for acquiring a multi-angle physical image of the power distribution panel.

The feature point matching unit 820 is configured to extract matching feature points in the multi-angle entity image, and map the matching feature points into a three-dimensional sparse point cloud according to spatial coordinates of the image capturing apparatus.

The feature point processing unit 830 is configured to perform point cloud densification on the three-dimensional sparse point cloud to obtain a dense point cloud.

A model construction unit 840 for constructing a static model of the distribution panel of the electric power distribution panel from the dense point cloud.

And an information display unit 850 for displaying the operation information of the electric power distribution panel acquired in real time into the static model of the distribution panel.

In one possible implementation, the feature point matching unit 820 includes:

and the region identification subunit is used for identifying the distribution panel region in the multi-angle entity image.

And the image segmentation subunit is used for segmenting the distribution panel area according to the GrabCut algorithm to obtain a foreground image of the power distribution panel.

And the feature point matching subunit is used for extracting dense feature points in the foreground image, and matching feature point pairs of the dense feature points to obtain matched feature point pairs.

And the characteristic point eliminating subunit is used for eliminating mismatching characteristic points in the matching characteristic point pairs.

And the characteristic point mapping subunit is used for mapping the matching characteristic points with the mismatching characteristic points removed into a three-dimensional sparse point cloud according to the space coordinates of the image pickup equipment.

In one possible implementation manner, the feature point processing unit 830 includes:

And the depth map acquisition subunit is used for acquiring depth maps of the multi-angle entity image under different angles.

And the dense point cloud acquisition subunit is used for carrying out patch expansion and patch filtering on the three-dimensional sparse point cloud according to the image field information acquired in the depth map so as to obtain the dense point cloud.

In one possible implementation manner, the model building unit 840 is specifically configured to constrain the re-projection error of the dense point cloud according to the light speed adjustment method, and perform curved surface reconstruction on the dense point cloud according to the poisson surface reconstruction algorithm, so as to obtain a static model of the distribution panel.

In one possible implementation, the feature point matching subunit is provided with a feature point matching unit for extracting dense feature points in the foreground image by using a harris corner detection algorithm, and tracking and matching the dense feature points by using a KLT algorithm to obtain matching feature point pairs.

In a possible embodiment, the feature point matching unit 820 further comprises an image enhancement subunit for enhancing the image contrast of the foreground image using the emphasize operator.

In one possible implementation, the feature point eliminating subunit is specifically configured to select a preset number of sample feature points from the matched feature points, where the preset number of sample feature points are not collinear, construct a matrix model according to the preset number of sample feature points, calculate a projection error between each matched feature point and the matrix model, and eliminate the matched feature points as mismatching feature points if the projection error is greater than or equal to a preset error threshold.

In one possible implementation, the computer vision-based power distribution panel display device further comprises an image capturing device coordinate acquisition unit for extracting sparse feature points in a foreground image of the power distribution panel by using a SURF algorithm to obtain spatial coordinates of the image capturing device.

Fig. 9 illustrates a physical schematic diagram of an electronic device, which may include a processor (processor) 910, a communication interface (Communications Interface) 920, a memory 930, and a communication bus 940, where the processor 910, the communication interface 920, and the memory 930 perform communication with each other through the communication bus 940, as shown in fig. 9. The processor 910 may invoke logic instructions in the memory 930 to perform a computer vision based power distribution panel presentation method comprising:

A multi-angle physical image of the power distribution panel is acquired.

And extracting matching characteristic points in the multi-angle entity image, and mapping the matching characteristic points into a three-dimensional sparse point cloud according to the space coordinates of the image pickup equipment.

And carrying out point cloud densification on the three-dimensional sparse point cloud to obtain dense point cloud.

A static model of the power distribution panel is constructed from the dense point cloud.

And displaying the operation information of the power distribution panel acquired in real time into a static model of the distribution panel.

Further, the logic instructions in the memory 930 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product comprising a computer program storable on a non-transitory computer readable storage medium, the computer program when executed by a processor being capable of performing the computer vision-based power distribution panel display method provided by the methods described above, the method comprising:

A multi-angle physical image of the power distribution panel is acquired.

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which when executed by a processor is implemented to perform the computer vision based power distribution panel display method provided by the above methods, the method comprising:

A multi-angle physical image of the power distribution panel is acquired.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.

Claims

1. A computer vision-based power distribution panel display method, comprising:

2. The computer vision-based power distribution panel display method according to claim 1, wherein the extracting the matching feature points in the multi-angle solid image and mapping the matching feature points into a three-dimensional sparse point cloud according to the spatial coordinates of the image capturing apparatus comprises:

identifying a distribution panel area in the multi-angle physical image;

Extracting dense characteristic points in the foreground image, and carrying out characteristic point pair matching on the dense characteristic points to obtain matching characteristic point pairs;

Rejecting mismatching feature points in the matching feature point pairs;

3. The computer vision-based power distribution panel display method of claim 1, wherein the performing point cloud densification on the three-dimensional sparse point cloud to obtain a dense point cloud comprises:

Acquiring depth maps of the multi-angle entity image under different angles;

4. The computer vision based power distribution panel display method of claim 1, wherein said constructing a static model of the power distribution panel from the dense point cloud comprises:

5. The computer vision based power distribution panel display method of claim 2, wherein the extracting dense feature points in the foreground image and matching feature point pairs of the dense feature points to obtain matched feature point pairs comprises:

And tracking and matching the dense feature points by using a KLT algorithm to obtain matched feature point pairs.

6. The computer vision-based power distribution panel display method of claim 2, further comprising, prior to extracting dense feature points in the foreground image:

7. The computer vision based power distribution panel display method of claim 2 wherein said culling out mismatching ones of said matching feature points comprises:

8. The computer vision-based power distribution panel display method of claim 1, further comprising:

9. A computer vision-based power distribution panel display device, comprising:

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the computer vision-based power distribution panel presentation method of any one of claims 1 to 8 when the program is executed.