CN117911954B - Weak supervision target detection method and system for operation and maintenance of new energy power station - Google Patents

Abstract

The invention belongs to the technical field of operation and maintenance of new energy power stations, and particularly relates to a weak supervision target detection method and system for operation and maintenance of a new energy power station, comprising the following steps: acquiring an operation and maintenance scene picture of a new energy power station; performing weak supervision training on the acquired operation and maintenance scene pictures to generate a plurality of bounding boxes of small target operation and maintenance scenes; constructing a target detection model based on a transducer based on the acquired operation and maintenance scene picture; and finishing the weakly-supervised target detection for the operation and maintenance of the new energy power station according to the constructed target detection model and the boundary box of the generated small target operation and maintenance scene.

Description

A weakly supervised target detection method and system for operation and maintenance of new energy power stations

Technical Field

The present invention belongs to the technical field of operation and maintenance of new energy power stations, and specifically relates to a weakly supervised target detection method and system for operation and maintenance of new energy power stations.

Background Art

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

In the operation and maintenance of photovoltaic, wind power and other new energy power stations, due to the complex physical environment, there are problems such as inaccurate equipment inspection positioning and irregular manual operation; at the same time, new energy power stations also have problems with the large number and small size of complex targets. Therefore, in order to efficiently and reasonably monitor and troubleshoot power station equipment, target detection and recognition in such complex scenes is very important.

According to the inventors, the existing target detection methods used for the operation and maintenance of new energy power stations mostly adopt supervision-based monitoring methods, but this method is often more dependent on a large amount of data annotations (such as object bounding boxes) during the training process. In a complex power station environment, obtaining a large amount of labeled data is time-consuming and laborious; weakly supervised target detection can solve this problem. However, the existing weakly supervised target detection method uses a multi-instance learning method to perform weakly supervised target detection, which has obvious disadvantages such as redundant object bounding box generation, slow reasoning speed, and easy to focus detection on highlighting the most discriminative areas (for example: the head of an animal) rather than the entire target. The problem of focusing detection on highlighting the most discriminative areas can be alleviated to a certain extent by using Transformer's weakly supervised target detection, but it is very easy to ignore local details and small targets that exist in the target detection process.

Summary of the invention

In order to solve the above problems, the present invention proposes a weakly supervised target detection method and system for operation and maintenance of new energy power stations. In order to reduce the manpower and material costs brought by obtaining large-scale data annotation during the operation and maintenance of new energy power stations and minimize the possibility of errors in manual annotation, the operation and maintenance targets of new energy power stations are discovered through weakly supervised training based on Transformer to detect local detail features, which effectively solves the problem of small target detection that must use large-scale and accurate data annotation to obtain the operation and maintenance of new energy power stations, thereby effectively reducing the manpower and material resources consumed and improving the efficiency and accuracy of small target detection in the operation and maintenance of new energy power stations.

According to some embodiments, a first solution of the present invention provides a weakly supervised target detection method for operation and maintenance of a new energy power station, which adopts the following technical solution:

A weakly supervised target detection method for operation and maintenance of new energy power stations, comprising:

Get pictures of the operation and maintenance scenarios of new energy power stations;

Perform weak supervision training on the acquired operation and maintenance scene images to generate bounding boxes of several small target operation and maintenance scenes;

Build a Transformer-based object detection model based on the acquired operation and maintenance scene images;

Based on the constructed target detection model and the generated bounding box of the small target operation and maintenance scenario, weakly supervised target detection for the operation and maintenance of new energy power stations is completed.

As a further technical limitation, the constructed Transformer-based target detection model adopts a Transformer-based end-to-end target detection network (DETR); according to the obtained bounding box matching of several small target operation and maintenance scenarios, the loss function of the Transformer-based end-to-end target detection network is matched to complete the weakly supervised target detection for the operation and maintenance of new energy power stations.

As a further technical limitation, the constructed Transformer-based object detection model includes a full-category mapping module and a local awakening module for inducing the Transformer model to highlight weak local features in the local view; based on the full-category mapping module, an image block detail clue feature map is obtained. , an image block detail enhancement feature map M _LAM is obtained based on the local awakening module.

As a further technical limitation, in the process of weakly supervised training, the acquired operation and maintenance scene pictures are divided into several image blocks, the position information of the divided several image blocks in the acquired operation and maintenance scene pictures is perceived, the weights of the divided several image blocks are calculated, and the attention heat map is obtained.

Furthermore, attention calculation is performed on the divided image blocks to obtain the attention matrix A; based on the obtained attention matrix and the multi-layer perceptron MLP, the image block feature map F _L is obtained, that is, F _L = w _i × A, where W _i is the weight of each neuron in the multi-layer perceptron; i represents the number of elements in the matrix A, that is, the number of neurons in the MLP.

Further, the divided image blocks are subjected to attention calculation to obtain an attention matrix of the image blocks. The obtained image block attention matrix includes an image block matrix and an image block semantic perception block matrix. The image block matrix and the image block semantic perception matrix are fused to calculate the long-range dependency relationship between the attention matrices to obtain an image block detail clue feature map. ; According to the obtained image block semantic perception block matrix and image block feature map, the image block detail enhancement feature map M _LAM is obtained.

Further, according to the obtained image block detail clue feature map and the image block detail enhancement feature map M _LAM , calculate the image block detail feature map M ^f ; accumulate the obtained image block detail feature map and the attention heat map to obtain the image block containing the initial bounding box, that is, obtain the bounding boxes of several small target operation and maintenance scenarios.

According to some embodiments, a second solution of the present invention provides a weakly supervised target detection system for operation and maintenance of a new energy power station, which adopts the following technical solution:

A weakly supervised target detection system for operation and maintenance of new energy power stations, comprising:

An acquisition module configured to acquire operation and maintenance scene images of a new energy power station;

A generation module is configured to perform weak supervision training on the acquired operation and maintenance scene images to generate bounding boxes of several small target operation and maintenance scenes;

A construction module, which is configured to construct a Transformer-based object detection model based on the acquired operation and maintenance scene pictures;

The detection module is configured to complete weakly supervised target detection for operation and maintenance of new energy power stations based on the constructed target detection model and the generated bounding box of the small target operation and maintenance scenario.

According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution:

A computer-readable storage medium stores a program thereon, which, when executed by a processor, implements the steps of a weakly supervised target detection method for operation and maintenance of a new energy power station as described in the first solution of the present invention.

According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution:

An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the weakly supervised target detection method for operation and maintenance of new energy power stations as described in the first solution of the present invention are implemented.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention can better cope with the problem of weakly supervised target detection, so that the training data no longer depends on the accurately labeled data set, and any image data containing only category information can be used to train the model; global information and local detail information are combined to obtain a better representation; considering the redundancy of generating object bounding boxes and the problem of reasoning speed, the DETR detector is used to predict the location information of the object in a one-to-one matching manner, which solves the problem of redundant generating object bounding boxes and slow reasoning speed; considering how to use the interaction and detail information between image blocks in the Transformer model, under the guidance of the semantic perception block, the detail information inside the image block and the global information of the image block-semantic perception block are fused.

The present invention discovers the operation and maintenance targets of new energy power stations through weakly supervised training based on Transformer to detect local detail features, effectively solving the problem of small target detection that can only be obtained by using large-scale and accurate data annotation for the operation and maintenance of new energy power stations, thereby effectively reducing the manpower and material resources consumed and improving the efficiency and accuracy of small target detection in the operation and maintenance process of new energy power stations.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

FIG1 is a flow chart of a weakly supervised target detection method for operation and maintenance of a new energy power station in Embodiment 1 of the present invention;

FIG2 is a basic framework diagram of a weakly supervised target detection method for operation and maintenance of a new energy power station in Embodiment 1 of the present invention;

FIG3 is a basic architecture diagram of a full-category mapping module in Embodiment 1 of the present invention;

FIG4 is a basic architecture diagram of a local wake-up module in Embodiment 1 of the present invention;

FIG5 is a schematic diagram of a process of weakly supervised training in Embodiment 1 of the present invention;

FIG6 is a schematic diagram of a traditional supervised training process;

FIG7 is a structural block diagram of a weakly supervised target detection system for operation and maintenance of a new energy power station in Embodiment 2 of the present invention.

DETAILED DESCRIPTION

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and/or combinations thereof.

In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

In the present invention, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. For relevant scientific research or technical personnel in this field, the specific meanings of the above terms in the present invention can be determined according to specific circumstances, and they cannot be understood as limitations on the present invention.

In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

Embodiment 1

Embodiment 1 of the present invention introduces a weakly supervised target detection method for operation and maintenance of a new energy power station.

As shown in FIG1 , a weakly supervised target detection method for operation and maintenance of a new energy power station includes:

Get pictures of the operation and maintenance scenarios of new energy power stations;

Perform weak supervision training on the acquired operation and maintenance scene images to generate bounding boxes of several small target operation and maintenance scenes;

Build a Transformer-based object detection model based on the acquired operation and maintenance scene images;

Based on the constructed target detection model and the generated bounding box of the small target operation and maintenance scenario, weakly supervised target detection for the operation and maintenance of new energy power stations is completed.

In view of the fact that weakly supervised training methods tend to focus detection on highlighting the most discriminative areas, this embodiment uses a Transformer model with global information modeling capabilities to discover the entire target of interest through end-to-end weakly supervised training; the Transformer's attention mechanism is used to generate high-quality sparse bounding boxes, and the DETR detector is trained to predict the location information of the object in a one-to-one matching manner, solving the problems of redundant object bounding box generation and slow reasoning speed. This embodiment uses a full category mapping module and a local awakening module to induce the Transformer model to highlight weak local features in the local view, thereby retaining local details, solving the problem of ignoring local details caused by the very few previous methods of using Transformers for weakly supervised target detection.

This embodiment adopts the basic framework diagram of the weakly supervised target detection method based on the operation and maintenance of new energy power stations as shown in Figure 2, and implements the weakly supervised target detection of the operation and maintenance of new energy power stations based on the local detail perception Transformer; adopts the Transformer-based end-to-end target detection network (DETR); matches the loss function of the Transformer-based end-to-end target detection network according to the obtained bounding boxes of several small target operation and maintenance scenes, and completes the weakly supervised target detection for the operation and maintenance of new energy power stations.

In the matching process of this embodiment, the Hungarian matching algorithm in DETR is followed, and the loss functions adopted are L1 loss function and GIoU loss function. Specifically, a bounding box with a high similarity to the generated bounding box (i.e., the overlapping area of the two boxes is as large as possible) is found from multiple candidate boxes. The bounding box with the largest overlapping area between each candidate box and the generated bounding box is calculated by using GIoU loss function and L1 loss function, and the bounding box with the largest overlapping area is retained. The retained bounding box is the bounding box that is successfully matched. Among them, the GIoU loss function is Among them, I represents the area of the intersection of two boxes, ^A1 represents the area of a candidate box, ^A2 represents the area of a generated box, and ^Ac represents the area of the minimum circumscribed rectangle of the two boxes; the L1 loss function is L1loss=|Box1-Box2|, where BOX1 and BOX2 contain the coordinates of the four vertices of a box; || represents the absolute value; L1loss is also called absolute value error.

The Transformer-based object detection model constructed in this embodiment includes a full-category mapping module and a local awakening module for inducing the Transformer model to highlight weak local features in the local view; based on the full-category mapping module, an image block detail clue feature map M*r is obtained, and based on the local awakening module, an image block detail enhancement feature map M _LAM is obtained.

As one or more implementation methods, as shown in FIG2, this embodiment splices the image block output from the Transformer backbone network and the self-added semantic perception block, and then inputs it into the attention block, so that the semantic perception block can well integrate the category and position information contained in the image block to help identify different categories appearing in the same image; extract the weights generated by the attention calculation to generate the attention heat map; extract the attention matrix A _l after the attention calculation. Finally, the attention matrix A _l is calculated through two FFNs for classification loss, and a MLP generates a feature map _FL , which is input into the local awakening module for the next step of calculation; this embodiment perceives the position information of different targets in the image and mines the common semantic information in the image block and the semantic perception block.

As one or more implementations, the attention mechanism calculates the global interaction between the semantic perception block and the image block, but ignores the correlation within the image block, and loses some detail information to a certain extent. As shown in Figure 3, the full category mapping module in this embodiment focuses on the detail information within the image block and uses more local clues to model the global representation.

Specifically, in the classification module, the attention matrix A _l is extracted after attention calculation, and A _l contains the matrix containing the image block and the semantically aware block matrix .for , which reveals the global interaction between the semantic-aware patch and all image patches, ignoring the correlation between each image patch. It reflects the local relationship between image blocks. In order to make full use of the relationship between the two matrices, the corresponding elements are multiplied to fuse the two matrices to obtain the image block detail clue feature map , that is, the long-range dependency between the attention matrix A _l is obtained.

It should be noted that the method used in the process of matrix fusion is element multiplication. matrix The matrix C obtained by fusing matrix A and matrix B is

As one or more implementations, the local awakening module uses local features to guide the learning trend of the model, highlighting local weak responses, and alleviating the problem that the Transformer model easily ignores local details. The local awakening module uses a pooling-based method to mask the most significant areas, allowing the model to explore more in-depth details.

As shown in Figure 4, the semantic perception block matrix After transposing and will The feature map F _L output by the classification module is obtained by multiplying and convolving the corresponding elements Then, in order to eliminate the salient areas, Two separate operators are applied, namely global average pooling (GAP) and global maximum pooling (GMP). After pooling, they are summed and then a weighted convolution kernel is obtained by applying the sigmoid function. The convolution kernel is applied to the F _L and convolution operation to obtain the final detail feature map M _LAM ; the detail feature map M _LAM contains all the weak response areas in an image (such as the boundaries of objects).

As one or more implementations, the outputs of the full category mapping module and the local wake-up module are obtained. After M _LAM , First, the corresponding elements of M f are multiplied with M _LAM to obtain M ^f , and then the corresponding elements of M f are added with the attention heat map to obtain the image block containing the initial bounding box, that is, the bounding boxes of several small target operation and maintenance scenes are obtained.

It should be noted that the weakly supervised training method used in this embodiment only has category annotation, but no bounding box annotation; however, the training target detector must be supervised by the object bounding box, as shown in Figure 5. For supervised target detection as shown in Figure 6, I must manually annotate the object bounding box in the data of the new energy operation and maintenance scene, but the input data only contains category information, not object bounding box information).

Since traditional Transformers tend to ignore local details, some small targets are easily ignored when generating object bounding boxes (equivalent to low quality of generated target boxes), which does not meet the detection requirements of many small targets in new energy operation and maintenance scenarios. Therefore, this embodiment adds local detail information to the model through the full category mapping module and the local wake-up module, and can generate object bounding boxes for small targets when generating object bounding boxes.

This embodiment fully takes into account the complex scenario problems under the new energy operation and maintenance scenario, and greatly reduces the time cost of manually labeling the new energy operation and maintenance scenario data through simple technology, while avoiding the problem of target detector detection errors caused by manual labeling errors to a certain extent; through Transformer-based weak supervision training, the new energy power station operation and maintenance targets are discovered to detect local detail features, which effectively solves the problem of small target detection that can only be obtained by large-scale and accurate data labeling for new energy power station operation and maintenance, thereby effectively reducing the manpower and material resources consumed, and improving the efficiency and accuracy of small target detection in the operation and maintenance process of new energy power stations.

This embodiment can better cope with the problem of weakly supervised target detection, so that the training data no longer depends on the accurately labeled data set, and any image data containing only category information can be used to train the model; global information and local detail information are combined to obtain a better representation; considering the redundancy of generating object bounding boxes and the problem of reasoning speed, the DETR detector is used to predict the location information of the object in a one-to-one matching manner, which solves the problem of redundant generation of object bounding boxes and slow reasoning speed; consider how to use the interaction and detail information between image blocks in the Transformer model, and under the guidance of the semantic perception block, fuse the detail information inside the image block and the global information of the image block-semantic perception block.

Embodiment 2

Embodiment 2 of the present invention introduces a weakly supervised target detection system for operation and maintenance of new energy power stations.

As shown in FIG7 , a weakly supervised target detection system for operation and maintenance of a new energy power station includes:

An acquisition module configured to acquire operation and maintenance scene images of a new energy power station;

A generation module is configured to perform weak supervision training on the acquired operation and maintenance scene images to generate bounding boxes of several small target operation and maintenance scenes;

A construction module, which is configured to construct a Transformer-based object detection model based on the acquired operation and maintenance scene pictures;

The detection module is configured to complete weakly supervised target detection for operation and maintenance of new energy power stations based on the constructed target detection model and the generated bounding box of the small target operation and maintenance scenario.

The detailed steps are the same as the weakly supervised target detection method for operation and maintenance of new energy power stations provided in Example 1, and will not be repeated here.

Embodiment 3

Embodiment 3 of the present invention provides a computer-readable storage medium.

A computer-readable storage medium stores a program thereon, which, when executed by a processor, implements the steps of the weakly supervised target detection method for operation and maintenance of a new energy power station as described in Embodiment 1 of the present invention.

The detailed steps are the same as the weakly supervised target detection method for operation and maintenance of new energy power stations provided in Example 1, and will not be repeated here.

Embodiment 4

A fourth embodiment of the present invention provides an electronic device.

An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the weakly supervised target detection method for operation and maintenance of a new energy power station as described in the first embodiment of the present invention are implemented.

The detailed steps are the same as the weakly supervised target detection method for operation and maintenance of new energy power stations provided in Example 1, and will not be repeated here.

The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims (7)

1. A weakly supervised target detection method for operation and maintenance of a new energy power station, characterized by comprising: Get pictures of the operation and maintenance scenarios of new energy power stations; Perform weak supervision training on the acquired operation and maintenance scene images to generate bounding boxes of several small target operation and maintenance scenes; Build a Transformer-based object detection model based on the acquired operation and maintenance scene images; Based on the constructed target detection model and the generated bounding box of the small target operation and maintenance scenario, the weakly supervised target detection for the operation and maintenance of the new energy power station is completed; In the process of weak supervision training, the acquired operation and maintenance scene picture is divided into a number of image blocks, the position information of the divided image blocks in the acquired operation and maintenance scene picture is perceived, the weights of the divided image blocks are calculated, and the attention heat map is obtained; Perform attention calculation on the divided image blocks to obtain the attention matrix A; according to the obtained attention matrix and the multi-layer perceptron MLP, obtain the image block feature map F _L , that is, F _L = _i × A, where _Wi is the weight of each neuron in the multi-layer perceptron; i represents the number of elements in the matrix A, that is, the number of neurons in the MLP; Attention calculation is performed on the divided image blocks to obtain an attention matrix of the image blocks. The obtained image block attention matrix includes an image block matrix and an image block semantic perception block matrix. The image block matrix and the image block semantic perception matrix are fused, and the long-range dependency relationship between the attention matrices is calculated to obtain an image block detail clue feature map; based on the obtained image block semantic perception block matrix and the image block feature map, an image block detail enhancement feature map is obtained. 2. A weakly supervised target detection method for operation and maintenance of new energy power stations as described in claim 1, characterized in that the constructed Transformer-based target detection model adopts a Transformer-based end-to-end target detection network; the weakly supervised target detection for operation and maintenance of new energy power stations is completed according to the loss function of the Transformer-based end-to-end target detection network matched with the bounding boxes of several small target operation and maintenance scenarios obtained. 3. A weakly supervised target detection method for operation and maintenance of new energy power stations as described in claim 1, characterized in that the constructed Transformer-based target detection model includes a full-category mapping module and a local awakening module for inducing the Transformer model to highlight weak local features in the local view; an image block detail clue feature map is obtained based on the full-category mapping module, and an image block detail enhancement feature map is obtained based on the local awakening module. 4. A weakly supervised target detection method for operation and maintenance of a new energy power station as described in claim 1, characterized in that an image block detail feature map is calculated based on the obtained image block detail clue feature map and the image block detail enhancement feature map; the obtained image block detail feature map and the attention heat map are accumulated to obtain an image block containing an initial bounding box, that is, to obtain bounding boxes of several small target operation and maintenance scenes. 5. A weakly supervised target detection system for operation and maintenance of a new energy power station, based on a weakly supervised target detection method for operation and maintenance of a new energy power station as claimed in any one of claims 1 to 4, characterized in that it comprises: An acquisition module configured to acquire operation and maintenance scene images of a new energy power station; A generation module is configured to perform weak supervision training on the acquired operation and maintenance scene images to generate bounding boxes of several small target operation and maintenance scenes; A construction module, which is configured to construct a Transformer-based object detection model based on the acquired operation and maintenance scene pictures; The detection module is configured to complete weakly supervised target detection for operation and maintenance of new energy power stations based on the constructed target detection model and the generated bounding box of the small target operation and maintenance scenario. 6. A computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the weakly supervised target detection method for operation and maintenance of new energy power stations as described in any one of claims 1 to 4 are implemented. 7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the weakly supervised target detection method for operation and maintenance of new energy power stations as described in any one of claims 1 to 4 are implemented.