TWI877045B - Optimization method for defect detection model and electronic device - Google Patents

Abstract

The invention provides an optimization method for a defect detection model and an electronic device. The electronic device includes a processing device and a storage device electrically connected to the processing device. The processing device performs the optimization method, including: reading at least one negative sample training data, which has the same original image size; calculating an optimized segmentation range based on a target defect information; estimating an effective range of an overlapping area based on the optimized segmentation range; calculating a model segmentation quantity based on the original image size, the optimized segmentation range and the effective range to obtain an optimized modeling parameter; and applying the optimized modeling parameter to a defect detection model to perform model inference on at least one target image.

Description

Defect detection model optimization method and electronic device

This case is about a defect detection model optimization method and an electronic device for optimizing the defect detection model.

In the field of artificial intelligence manufacturing, high-resolution and ultra-high-resolution images are widely used to detect all defects as much as possible. However, using high-resolution images as input sources is not conducive to the training and inference of deep learning models. It will greatly increase the memory used and may cause memory overload (Out-of-Memory, OOM) problems, extend prediction time, reduce accuracy, etc., resulting in the model failing to meet user needs.

There are two common approaches to using high-resolution images as input for deep learning models. One is to reduce the resolution of the original image, which can easily cause tiny or fine defects to be undetectable. The other is to split the high-resolution image into several low-resolution images, but this approach will reduce the model's ability to analyze global or broader features, and easily lead to many false positives.

The present invention provides a defect detection model optimization method, comprising: reading at least one negative sample training data, the negative sample training data having the same original image size. According to a target defect information, an optimized segmentation range is calculated. According to the optimized segmentation range, an effective range of a repeated coverage area is estimated. According to the original image size, the optimized segmentation range and the effective range, a model segmentation quantity is calculated to obtain an optimized modeling parameter. The optimized modeling parameter is applied to a defect detection model to perform model inference on at least one target image.

The present invention further provides an electronic device, including a storage device and a processing device. A storage device stores at least one negative sample training data and at least one target image, and the negative sample training data has the same original image size. The processing device is electrically connected to the storage device and has a built-in defect detection model. The processing device performs the following steps: based on target defect information, an optimized segmentation range is calculated; based on the optimized segmentation range, an effective range of a repeated coverage area is estimated; based on the original image size, the optimized segmentation range and the effective range, a model segmentation quantity is calculated to obtain an optimized modeling parameter; the optimized modeling parameter is applied to the defect detection model, and the model is inferred using the target image.

In summary, this case proposes a defect detection model optimization method and electronic device, which uses target defect information to automatically generate optimized modeling parameters to apply to the defect detection model. The user does not need to decide how much the resolution needs to be reduced and how many low-resolution images need to be cut, so as to reduce the prediction time and improve the prediction accuracy, thereby making the defect prediction model effectively meet the user's needs.

The following is a detailed description of the preferred embodiments, however, the embodiments are only used as examples and do not limit the scope of protection of this case. In addition, some components are omitted in the drawings in the embodiments to clearly show the technical features of this case. The same reference numerals will be used to represent the same or similar components in all drawings.

Please refer to FIG. 1 , an electronic device 10 includes a processing device 12 and a storage device 14. The processing device 12 is electrically connected to the storage device 14. The processing device 12 has a built-in defect detection model 16, and the storage device 14 stores one or more negative sample training data (normal images) and one or more target images. In one embodiment, the negative sample training data is provided by the user, and the negative sample training data may include data enhancements or errors such as small rotations or translations, but each negative sample training data must have the same original image size. The processing device 12 reads the negative sample training data from the storage device 14, and directly optimizes the defect detection model 16 based on the negative sample training data, and performs model inference and training based on the target image. Furthermore, the electronic device 10 further includes a graphics processor 18, which is electrically connected to the processing device 12. When the processing device 12 performs calculations, inferences or training, the graphics processor 18 assists the processing device 12 in performing related calculations, so as to assist the calculations through the graphics processor 18 and accelerate the overall calculation speed. In one embodiment, the electronic device 10 is a personal computer, a laptop computer, or a tablet computer that can independently perform artificial intelligence inference and training operations, but the present invention is not limited thereto.

In one embodiment, the processing device 12 is a central processing unit (CPU), or other general-purpose or special-purpose microprocessor, microcontroller, micro control unit (MCU), digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), or other similar components or a combination of the above components to perform the operation of the entire algorithm, but the present invention is not limited to this.

In one embodiment, the storage device 14 can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD) or other similar components or a combination of the above components to store any model or parameter data required by the processing device 12, but the present case is not limited to this.

In the electronic device 10, the processing device 12 uses software to optimize the defect detection model 16. Please refer to FIG. 1, FIG. 2 and FIG. 3 at the same time. As shown in step S10, the processing device 12 reads at least one negative sample training data 20. The negative sample training data 20 must be kept the same size or adjusted to the same size to have the same original image size. As shown in step S12, the processing device 12 calculates an optimized segmentation range 22 based on a target defect information, wherein the target defect information provides a target defect size range or a target defect image and its defect location information for the user. The user can determine the defect size range or the target defect image and its defect location information expected to be detected in the negative sample training data 20.

In one embodiment, the step of calculating the optimal segmentation range 22 in step S12 further includes steps S121 and S122. Please refer to FIG. 1, FIG. 3 and FIG. 4 at the same time. As shown in step S121, the processing device 12 calculates the size of a perception range 24 according to an artificial intelligence model, such as a Backbone model. Specifically, the artificial intelligence model uses a first equation: , calculate the perception range 24, where R is the size of the perception range 24, n is the number of model convolution layers, s is an output layer step length and k is an output layer convolution kernel size, so that the processing device 12 obtains the perception range 24 by executing the operation of the first equation through the Backbone model. Then, as shown in step S122, the processing device 12 calculates the optimized segmentation range 22 according to the perception range 24 and the target defect information. In this step, the processing device 12 uses a second equation: The optimized segmentation range 22 is calculated, where S is the optimized segmentation range 22, a is the target defect information, here taking the smallest defect range 26 as an example, R is the size of the perception range 24, and N is a positive integer. The optimized segmentation range 22 needs to be able to completely cover the characteristics of the defect with the perception range 24. In addition, the processing device 12 can further estimate the length H _patch of the optimized segmentation range 22 and the width W _patch of the optimized segmentation range 22 according to the target inference speed to optimize the optimized segmentation range 22. When the target inference speed is high, a larger optimized segmentation range 22 (H _patch *W _patch ) will be used as much as possible; on the contrary, when the expected inference performance is high, a smaller optimized segmentation range 22 will be selected to improve the detection accuracy.

Please refer to FIG. 1, FIG. 2 and FIG. 3 at the same time. As shown in step S14, based on the optimized segmentation range 22, the processing device 12 estimates an effective range of an overlapping area 28. After obtaining the optimized segmentation range 22 and the effective range of the overlapping area 28, as shown in step S16, the processing device 12 calculates a model segmentation quantity according to the original image size, the optimized segmentation range 22 and the effective range to obtain an optimized modeling parameter. The processing device 12 uses a third-party program: The number of model segmentation is calculated, where H _ori is the length of the original image size, W _ori is the width of the original image size, H _patch is the length of the optimized segmentation range 22, W _patch is the width of the optimized segmentation range 22, and O is the effective range of the repeated coverage area 28. Therefore, the number of model segmentation as the optimized modeling parameter can be calculated by a third-party program. Finally, as shown in step S18, the optimized modeling parameter is applied to a defect detection model 16, and the processing device 12 reads at least one target image from the storage device 14, and inputs the target image into the defect detection model 16 for model inference. When the target image is input into the defect detection model 16, the algorithm in the defect detection model 16 will automatically segment, train, infer, and reorganize the target image to finally obtain a defect hotspot map 30 of the same size as the input target image, as shown in FIG5, or an output mask. This defect hotspot map 30 or output mask can indicate the predicted defect location and range size. The aforementioned steps of inputting the target image into the defect detection model 16 for model inference can be a general normal model training and inference process, and the present case is not limited to the aforementioned process.

Therefore, in this case, as long as the user provides the size of the pre-detected defect, the best defect detection model can be automatically generated, and the user does not need to decide by himself "how much resolution needs to be reduced" and "how much size needs to be cut". In addition, the user also provides the target inference speed and joins the optimization process to obtain the optimized modeling parameters.

In summary, this case proposes a defect detection model optimization method and electronic device, which uses target defect information to automatically generate optimized modeling parameters to apply to the defect detection model. The user does not need to decide how much the resolution needs to be reduced and how many low-resolution images need to be cut, so as to reduce the prediction time and improve the prediction accuracy, thereby making the defect prediction model effectively meet the user's needs.

The implementation examples described above are only for illustrating the technical ideas and features of this case. Their purpose is to enable those familiar with this technology to understand the content of this case and implement it accordingly. They cannot be used to limit the patent scope of this case. In other words, any equivalent changes or modifications made according to the spirit disclosed in this case should still be covered by the scope of the patent application of this case.

10: electronic device 12: processing device 14: storage device 16: defect detection model 18: graphics processor 20: negative sample training data 22: optimized segmentation range 24: perception range 26: defect range 28: repeated coverage area 30: defect hotspot map H _patch : length of optimized segmentation range W _patch : width of optimized segmentation range S10~S18: steps S121~S122: steps

FIG. 1 is a block diagram of an electronic device according to an embodiment of the present invention. FIG. 2 is a flow diagram of optimizing a defect detection model by an electronic device according to an embodiment of the present invention. FIG. 3 is a diagram of negative sample training data showing an optimized segmentation range, defect range, and perception range according to an embodiment of the present invention. FIG. 4 is a flow diagram of calculating an optimized segmentation range by an electronic device according to an embodiment of the present invention. FIG. 5 is a diagram of a defect hotspot map generated according to an embodiment of the present invention.

S10~S18: Steps

Claims (17)

A defect detection model optimization method includes: Reading at least one negative sample training data, the negative sample training data has the same original image size; Calculating an optimized segmentation range based on target defect information; Estimating an effective range of a repeated coverage area based on the optimized segmentation range; Calculating a model segmentation quantity based on the original image size, the optimized segmentation range and the effective range to obtain an optimized modeling parameter; and Applying the optimized modeling parameter to a defect detection model to perform model inference on at least one target image. The method for optimizing a defect detection model as described in claim 1, wherein the negative sample training data includes data enhancement or error such as small rotation or translation. The method for optimizing a defect detection model as described in claim 1, wherein the target defect information is a defect size range or a target defect image and its defect location information. The method for optimizing the defect detection model as described in claim 1, wherein the step of calculating the optimized segmentation range according to the target defect information further includes: calculating a perception range according to an artificial intelligence model; and calculating the optimized segmentation range according to the perception range and the target defect information. The method for optimizing a defect detection model as claimed in claim 4, wherein the artificial intelligence model is based on a first equation: , the perception range is calculated, where R is the size of the perception range, n is the number of model convolution layers, s is the step size of an output layer, and k is the size of an output layer convolution kernel. The method for optimizing the defect detection model as claimed in claim 4, wherein the optimization segmentation range is based on a second equation: Calculated, where S is the optimized segmentation range, a is the target defect information, R is the size of the perception range, and N is a positive integer. In the defect detection model optimization method as described in claim 1, in the step of calculating the optimized segmentation range, the optimized segmentation range can be estimated based on a target inference speed. The method for optimizing a defect detection model as claimed in claim 1, wherein the number of model segmentations is calculated using a third-party program: Calculated, where H _ori is the length of the original image size, W _ori is the width of the original image size, H _patch is the length of the optimized segmentation range, W _patch is the width of the optimized segmentation range, and O is the effective range. An electronic device includes: A storage device storing at least one negative sample training data and at least one target image, and the negative sample training data has the same original image size; and A processing device electrically connected to the storage device and having a built-in defect detection model, the processing device performing the following steps: Calculating an optimized segmentation range based on target defect information; Estimate an effective range of a repeated coverage area based on the optimized segmentation range; Calculating a model segmentation quantity based on the original image size, the optimized segmentation range and the effective range to obtain an optimized modeling parameter; and Applying the optimized modeling parameter to the defect detection model to perform model inference with the target image. An electronic device as described in claim 9, wherein the negative sample training data includes data enhancement or error such as small rotation or translation. An electronic device as described in claim 9, wherein the target defect information is a defect size range or a target defect image and its defect location information. The electronic device as described in claim 9, wherein the step of the processor calculating the optimized segmentation range further includes: calculating a perception range based on an artificial intelligence model; and calculating the optimized segmentation range based on the perception range and the target defect information. The electronic device of claim 12, wherein the artificial intelligence model is based on a first equation: , the perception range is calculated, where R is the size of the perception range, n is the number of model convolution layers, s is the step size of an output layer, and k is the size of an output layer convolution kernel. The electronic device of claim 12, wherein the processing device is configured to process the electronic device by a second equation: The optimal segmentation range is calculated, where S is the optimal segmentation range, a is the target defect information, R is the size of the perception range, and N is a positive integer. An electronic device as described in claim 12, wherein the processing device can further estimate the optimized segmentation range based on a target inference speed. The electronic device of claim 9, wherein the processing device is through a third party program: The number of segmentation of the model is calculated, where H _ori is the length of the original image size, W _ori is the width of the original image size, H _patch is the length of the optimized segmentation range, W _patch is the width of the optimized segmentation range, and O is the effective range. The electronic device as described in claim 9 further includes a graphics processor, which is electrically connected to the processing device and assists the processing device in performing calculations.