TWI843591B - Method for creating flaw image detection model, method for detecting flaw image and electronic device - Google Patents
Method for creating flaw image detection model, method for detecting flaw image and electronic device Download PDFInfo
- Publication number
- TWI843591B TWI843591B TW112120504A TW112120504A TWI843591B TW I843591 B TWI843591 B TW I843591B TW 112120504 A TW112120504 A TW 112120504A TW 112120504 A TW112120504 A TW 112120504A TW I843591 B TWI843591 B TW I843591B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- defect
- reconstructed
- reconstruction error
- error
- Prior art date
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 66
- 238000003709 image segmentation Methods 0.000 claims abstract description 38
- 230000007547 defect Effects 0.000 claims description 130
- 238000012360 testing method Methods 0.000 claims description 30
- 230000002950 deficient Effects 0.000 claims description 15
- 238000003860 storage Methods 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 5
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 claims 1
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 claims 1
- 230000001131 transforming effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 238000013528 artificial neural network Methods 0.000 description 6
- 230000009466 transformation Effects 0.000 description 6
- 238000007689 inspection Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000015654 memory Effects 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 1
- 101000827703 Homo sapiens Polyphosphoinositide phosphatase Proteins 0.000 description 1
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 1
- 102100023591 Polyphosphoinositide phosphatase Human genes 0.000 description 1
- 101100012902 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) FIG2 gene Proteins 0.000 description 1
- 101100233916 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) KAR5 gene Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
Description
本案有關於影像處理模型的建立及影像處理的方法,特別是有關於一種影像瑕疵檢測模型的建立方法、瑕疵影像的檢測方法,及執行這些方法的電子裝置。 This case is about the establishment of an image processing model and an image processing method, and in particular, about a method for establishing an image defect detection model, a method for detecting defective images, and an electronic device for executing these methods.
影像辨識技術廣泛運用於瑕疵檢測,相較於傳統以人工檢測提升檢測速度及精準度。瑕疵檢測技術仰賴機器視覺對待判讀影像及已知瑕疵影像來進行比對,故需事先定義瑕疵影像中的瑕疵特徵。基於不同的應用場域,瑕疵影像的內容也會隨著應用領域的影像而有所不同。使用者需依據應用場域定義對應的瑕疵特徵,然而,這樣的作法耗費大量的時間來準備前置作業。 Image recognition technology is widely used in defect detection, which improves the detection speed and accuracy compared to traditional manual detection. Defect detection technology relies on machine vision to compare the image to be read and the known defect image, so the defect features in the defect image must be defined in advance. Based on different application scenarios, the content of the defect image will also vary with the image of the application field. Users need to define the corresponding defect features according to the application scenario, however, this approach takes a lot of time to prepare for the pre-operation.
為有效率地找出影像中的瑕疵,神經網路演算法被運用來學習影像中的瑕疵特徵來實現自動化影像瑕疵檢測。神經網路演算法可以從大量輸入的未知影像自行推論出影像特徵,然而,在訓練階段中尚未有從未知影像中推論出瑕疵特徵的有效率的方法。 In order to efficiently find defects in images, neural network algorithms are used to learn the defect features in images to achieve automatic image defect detection. Neural network algorithms can infer image features from a large number of unknown images input by themselves. However, there is no efficient method to infer defect features from unknown images during the training stage.
此外,現有技術的自動編碼器可運用於影像重構,在重構過程中濾除瑕疵,來產生無瑕疵的重構影像。然而,若訓練階段的樣本數量不夠或者訓練影像不存在瑕疵,則自動編碼器模型無從學習瑕疵特徵。換言之,輸入自動編碼器的測試影像中的瑕疵仍會在重構影像中出現,導致自動編碼器的輸出結果不正確,即測試階段的判斷結果之精準度低。 In addition, the existing auto-encoder can be used for image reconstruction, filtering out defects during the reconstruction process to produce a defect-free reconstructed image. However, if the number of samples in the training phase is insufficient or the training image does not have defects, the auto-encoder model cannot learn the defect characteristics. In other words, the defects in the test image input to the auto-encoder will still appear in the reconstructed image, resulting in incorrect output results of the auto-encoder, that is, the accuracy of the judgment results in the test phase is low.
據此,在使用神經網路演算法來檢測影像中的瑕疵的方法中,判讀影像中的瑕疵之準確度不佳的問題仍有待解決。 Accordingly, in the method of using a neural network algorithm to detect defects in an image, the problem of poor accuracy in judging defects in the image remains to be solved.
根據本案的一實施例揭示一種影像瑕疵檢測模型的建立方法,影像瑕疵檢測模型用以檢測影像是否有異常。影像瑕疵檢測模型的建立方法包括:a)使用一無瑕疵樣本影像及一瑕疵樣本影像來生成一第一訓練影像;b)輸入該第一訓練影像至一自動編碼器以輸出一重構影像並計算一第一重構誤差;c)計算該第一訓練影像及該重構影像之間的差異以產生一重構誤差影像;d)使用該第一訓練影像、該重構影像及該重構誤差影像生成一第二訓練影像;e)輸入該第二訓練影像至一影像分割網路以產生一遮罩影像並計算一第二重構誤差;f)根據該第一重構誤差及該第二重構誤差計算一總重構誤差;g)根據該總重構誤差執行該自動編碼器及該影像分割網路的參數的優化;以及h)重複執行步驟a)至步驟g)以完成影像瑕疵檢測模型的建立及參數優化,其中影像瑕疵檢測模型包括自動編碼器及影像分割網路。 According to an embodiment of the present invention, a method for establishing an image defect detection model is disclosed. The image defect detection model is used to detect whether an image is abnormal. The method for establishing the image defect detection model includes: a) using a flawless sample image and a flawed sample image to generate a first training image; b) inputting the first training image to an automatic encoder to output a reconstructed image and calculate a first reconstruction error; c) calculating the difference between the first training image and the reconstructed image to generate a reconstructed error image; d) using the first training image, the reconstructed image, and the reconstructed error image to generate a second training image; e) inputting the first training image to an automatic encoder to output a reconstructed image and calculate a first reconstruction error; The second training image is fed to an image segmentation network to generate a mask image and calculate a second reconstruction error; f) a total reconstruction error is calculated based on the first reconstruction error and the second reconstruction error; g) the parameters of the automatic encoder and the image segmentation network are optimized based on the total reconstruction error; and h) steps a) to g) are repeatedly performed to complete the establishment and parameter optimization of the image defect detection model, wherein the image defect detection model includes the automatic encoder and the image segmentation network.
根據本案的一實施例揭示一種檢測瑕疵影像的方法,包括:a)獲取一待檢測影像;b)輸入待檢測影像至一自動編碼器並輸出一重構影像;c)計 算待檢測影像及該重構影像的差異以產生一重構誤差影像;d)根據待檢測影像、重構影像及重構誤差影像產生一測試影像;e)輸入測試影像至一影像分割網路以產生一瑕疵遮罩測試影像;以及f)根據瑕疵遮罩測試影像的一指示面積大小來判斷待檢測影像是否為一瑕疵影像。 According to an embodiment of the present invention, a method for detecting defective images is disclosed, including: a) obtaining an image to be detected; b) inputting the image to be detected into an automatic encoder and outputting a reconstructed image; c) calculating the difference between the image to be detected and the reconstructed image to generate a reconstructed error image; d) generating a test image according to the image to be detected, the reconstructed image and the reconstructed error image; e) inputting the test image into an image segmentation network to generate a defect mask test image; and f) judging whether the image to be detected is a defective image according to an indicated area size of the defect mask test image.
本案提出的影像處理方法可以自動地擴增用來訓練的影像,於後續測試影像中是否有瑕疵的測試階段,大幅地提升判讀影像中是否有瑕疵以及瑕疵在影像中位置的精準度。 The image processing method proposed in this case can automatically expand the images used for training, and in the subsequent testing phase of whether there are defects in the image, it can greatly improve the accuracy of judging whether there are defects in the image and the location of the defects in the image.
10:電子裝置 10: Electronic devices
110:儲存媒體 110: Storage media
120:處理器 120: Processor
150:影像瑕疵檢測模型 150: Image defect detection model
155:自動編碼器 155: Automatic encoder
157:影像分割網路 157: Image segmentation network
159:運算模組 159: Computation module
302:無瑕疵樣本影像 302: flawless sample image
304:瑕疵樣本影像 304: Defective sample image
310:第一訓練影像 310: First training video
312:重構影像 312: Reconstructing the image
314:瑕疵 314: Defects
316:重構誤差影像 316: Reconstructing erroneous images
318:瑕疵 318: Defects
320:第二訓練影像 320: Second training video
328:瑕疵 328: Defects
334:瑕疵遮罩影像 334: Defect mask image
336:過濾區域 336: Filter area
338:保留區域 338: Reserved area
E1:第一誤差函數 E1: First error function
E2:第二誤差函數 E2: Second error function
ES:加權公式 ES: Weighted formula
K1:第一重構誤差 K1: First reconstruction error
K2:第二重構誤差 K2: Second reconstruction error
KS:總重構誤差 KS: Total reconstruction error
S210~S280、S1110~S1180:步驟 S210~S280, S1110~S1180: Steps
圖1為本案根據一實施例所繪示的電子裝置的方塊圖。 FIG1 is a block diagram of an electronic device according to an embodiment of the present invention.
圖2為本案根據一實施例所繪示的影像瑕疵檢測模型的建立方法的流程圖。 FIG2 is a flow chart of a method for establishing an image defect detection model according to an embodiment of the present invention.
圖3為本案根據一實施例所繪示的生成第一訓練圖像的示意圖。 Figure 3 is a schematic diagram of generating a first training image according to an embodiment of the present invention.
圖4為本案根據一實施例所繪示的輸出重構影像的示意圖。 FIG4 is a schematic diagram of an output reconstructed image drawn according to an embodiment of the present invention.
圖5為本案根據一實施例所繪示的計算第一重構誤差的示意圖。 FIG5 is a schematic diagram of calculating the first reconstruction error according to an embodiment of the present invention.
圖6為本案根據一實施例所繪示的產生重構誤差影像的示意圖。 FIG6 is a schematic diagram of a reconstructed erroneous image generated according to an embodiment of the present invention.
圖7為本案根據一實施例所繪示的生成第二訓練影像的示意圖。 FIG7 is a schematic diagram of generating a second training image according to an embodiment of the present invention.
圖8為本案根據一實施例所繪示的產生瑕疵遮罩影像的示意圖。 FIG8 is a schematic diagram of generating a defect mask image according to an embodiment of the present invention.
圖9為本案根據一實施例所繪示的計算第二重構誤差的示意圖。 FIG9 is a schematic diagram of calculating the second reconstruction error according to an embodiment of the present invention.
圖10為本案根據一實施例所繪示的計算總重構誤差的示意圖。 FIG10 is a schematic diagram showing the calculation of the total reconstruction error according to an embodiment of the present invention.
圖11為本案根據一實施例所繪示的瑕疵影像的檢測方法的流程圖。 FIG11 is a flow chart of a defect image detection method according to an embodiment of the present invention.
以下結合圖式和實施例對本案作進一步說明,以使本發明所屬技術領域的相關人員可以更好的理解本發明並能據以實施,但所舉實施例不作為對本發明的限定。 The following further describes the present invention in combination with the drawings and embodiments, so that relevant personnel in the technical field to which the present invention belongs can better understand the present invention and implement it accordingly, but the embodiments are not intended to limit the present invention.
如本文中所使用的,諸如「第一」及「第二」等用語描述了各種元件、組件、區域、層及/或部分,這些元件、組件、區域、層及/或部分不應受這些術語的限制。這些術語僅可用於將一個元素、組件、區域、層或部分與另一個做區分。除非上下文明確指出,否則本文中使用的諸如「第一」及「第二」的用語並不暗示順序或次序。 As used herein, terms such as "first" and "second" describe various elements, components, regions, layers and/or parts, which should not be limited by these terms. These terms may only be used to distinguish one element, component, region, layer or part from another. Unless the context clearly indicates otherwise, the terms such as "first" and "second" used herein do not imply a sequence or order.
請參照圖1,其為本案根據一實施例所繪示的電子裝置的方塊圖。電子裝置10包括儲存媒體110及處理器120。儲存媒體110耦接於處理器120。電子裝置10用以建立影像瑕疵檢測模型150。
Please refer to FIG. 1 , which is a block diagram of an electronic device according to an embodiment of the present invention. The
於一實施例中,影像瑕疵檢測模型150用以檢測影像是否有異常。
In one embodiment, the image
儲存媒體110經配置以儲存影像瑕疵檢測模型150。於一實施例中,影像瑕疵檢測模型150係使用神經網路的模型架構來實現。影像瑕疵檢測模型150包括自動編碼器(AutoEncoder)155、影像分割網路157及運算模組159。
The
於一實施例中,自動編碼器155可以為卷積自動編碼器(CNN AutoEncoder)、稀疏自動編碼器(Sparse AutoEncoder)、降噪自動編碼器(Denoising AutoEncoder)或其他在多層神經網路架構以非監督式學習演算法來實現的自動編碼器。
In one embodiment, the
於一實施例中,影像分割網路(Image Segmentation Network)157可以為物件偵測演算法(例如U-Net或Mask RCNN演算法),用以偵測影像中的物件並於影像中標記物件輪廓並產生遮罩影像。
In one embodiment, the
運算模組159用以執行自動編碼器155及影像分割網路157以外的其他神經網路運算。
The
於一實施例中,影像瑕疵檢測模型150包括自動編碼器155、影像分割網路157及運算模組159。於一實施例中,自動編碼器155、影像分割網路157及運算模組159為軟體模型,是由多個程式碼實現,使得處理器120載入多個程式碼後執行自動編碼器155、影像分割網路157及運算模組159的操作。
In one embodiment, the image
於一實施例中,影像瑕疵檢測模型150可以為深度學習演算法,例如卷積神經網路(CNN,Convolutional neural network)、遞迴神經網路(RNN,Recurrent Neural Network)、生成對抗網路(GAN,Generative Adversarial Network)、多層感知器(MLP,Multilayer Perceptron)、深度波茲曼機(DBM,Deep Boltzmann Machine)或長短期記憶網路(LSTM,Long Short-Term Memory)。
In one embodiment, the image
於一實施例中,處理器可以為但不限於數位訊號處理器(Digital Signal Processor,DSP)、特定用途積體電路(Application Specific Integrated Circuit,ASIC)、中央處理器(Central Processing Unit,CPU)、系統單晶片(System on Chip,SoC)、現場可程式設計閘陣列(Field Programmable Gate Array,FPGA)、網路處理器(Network Processor)晶片或上述元件的組合。 In one embodiment, the processor may be, but is not limited to, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a central processing unit (CPU), a system on chip (SoC), a field programmable gate array (FPGA), a network processor chip, or a combination of the above components.
於一實施例中,儲存媒體可以為但不限於隨機存取記憶體(Random Access Memory,RAM)、快閃記憶體(Flash memory)、唯讀記憶體(Read Only Memory,ROM)、硬碟機(Hard Disk Drive,HDD)、固態硬碟(Solid State Drive,SSD)、光儲存器或上述元件的組合。 In one embodiment, the storage medium may be, but is not limited to, a random access memory (RAM), a flash memory (Flash memory), a read-only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), an optical storage device, or a combination of the above elements.
於一實施例中,電子裝置10可以為但不限於平板電腦、筆記型電腦、個人電腦、桌上型電腦、主機電腦系統、電腦工作站、影像電腦,或其他具備處理器及/或儲存裝置,而可執行安裝其上及/或雲端的應用程式或可執行儲存在本地端或雲端的指令之電子裝置。
In one embodiment, the
於一實施例中,影像瑕疵檢測模型150可應用於,例如工廠產線產品檢測、晶圓檢測、印刷電路板檢測、公共場域異常檢測,或任何使用機器視覺來實現目的性檢測的場域。
In one embodiment, the image
請參照圖2,其為本案根據一實施例所繪示的影像瑕疵檢測模型的建立方法的流程圖。影像瑕疵檢測模型的建立方法可以由圖1的電子裝置10所執行。電子裝置10的處理器120載入儲存媒體110中的多個程式碼以執行多個操作,來實現影像瑕疵檢測模型150的建立與訓練。
Please refer to FIG. 2, which is a flow chart of a method for establishing an image defect detection model according to an embodiment of the present invention. The method for establishing an image defect detection model can be executed by the
於一實施例中,使用者可基於檢測目的事先取得多個無瑕疵樣本影像。無瑕疵樣本影像係指基於所處場域或應用,其內容為正常或正確的影像。舉例而言,於印刷電路板的應用中,無瑕疵樣本影像係指其上的電路元件正確、電路元件的位置正確及配線正確等的印刷電路板影像。另一實施例中,無瑕疵樣本影像可以為被用於訓練學習的影像。 In one embodiment, the user can obtain multiple flawless sample images in advance for the purpose of detection. The flawless sample image refers to an image whose content is normal or correct based on the field or application. For example, in the application of printed circuit boards, the flawless sample image refers to an image of a printed circuit board with correct circuit components, correct positions of circuit components, and correct wiring. In another embodiment, the flawless sample image can be an image used for training and learning.
無瑕疵樣本影像可事先儲存於儲存媒體110的影像集合(圖未繪示)。 The flawless sample image can be stored in advance in an image collection of the storage medium 110 (not shown).
於步驟S210,運算模組159使用無瑕疵樣本影像及瑕疵樣本影像來生成第一訓練影像。
In step S210, the
於一實施例中,第一訓練影像是透過合成無瑕疵樣本影像及瑕疵樣本影像來生成。 In one embodiment, the first training image is generated by synthesizing a flawless sample image and a flawed sample image.
請參照圖3,其為本案根據一實施例所繪示的生成第一訓練圖像的示意圖。 Please refer to Figure 3, which is a schematic diagram of generating the first training image according to an embodiment of the present invention.
如圖3所示,無瑕疵樣本影像302與瑕疵樣本影像304被合成而生成第一訓練影像310。第一訓練影像310包括相同或相似於無瑕疵樣本影像302及瑕疵樣本影像304的影像特徵。
As shown in FIG3 , the
於一實施例中,瑕疵(flaw)樣本影像304包括雜訊(noise)樣本影像及不良(defect)樣本影像。
In one embodiment, the
於一實施例中,運算模組159使用隨機函式產生影像雜訊,並執行仿射變換、透視變換、色彩變換或上述的組合來處理影像雜訊後與無瑕疵樣本影像合成,以產生雜訊樣本影像。
In one embodiment, the
於一實施例中,運算模組159執行仿射變換、透視變換、色彩變換或上述的組合來處理任何可被作為瑕疵的影像後與無瑕疵樣本影像合成,以產生不良(defect)樣本影像。
In one embodiment, the
於一實施例中,儲存媒體110儲存的影像集合包括多個無瑕疵樣本影像及多個瑕疵樣本影像。瑕疵樣本影像的數量及無瑕疵樣本影像的數量之間具有一比例。舉例而言,此比例為1比100,即運算模組159取同一張瑕疵樣本影像及影像集合中所有無瑕疵樣本影像中的100張來分別產生100張的第一訓練影像。此比例用以調整第一訓練影像相對於無瑕疵樣本影像的差異性。
In one embodiment, the image set stored in the
於步驟S220,運算模組159輸入第一訓練影像至自動編碼器155,並由自動編碼器155輸出重構影像並計算第一重構誤差。
In step S220, the
請參照圖4,其為本案根據一實施例所繪示的輸出重構影像的示意圖。 Please refer to Figure 4, which is a schematic diagram of the output reconstructed image drawn according to an embodiment of the present case.
如圖4所示,自動編碼器155的輸入影像為第一訓練影像310。自動編碼器155的編碼器(圖未繪示)及解碼器(圖未繪示)對第一訓練影像310執行影像處理之後輸出無瑕疵樣本影像302的重構影像312。
As shown in FIG. 4 , the input image of the
於一實施例中,自動編碼器155具有處理第一訓練影像310中瑕疵的能力(例如濾除瑕疵314),而將第一訓練影像310轉換為相同或近似於無瑕疵樣本影像302的影像,即重構影像312。重構影像312為自動編碼器155的輸出影像。
In one embodiment, the
承上述實施例,於步驟S220,電子裝置10進一步計算第一重構誤差。於一實施例中,電子裝置10輸入無瑕疵樣本影像302的像素值及重構影像312的像素值至第一誤差函數來計算出第一重構誤差。
According to the above embodiment, in step S220, the
於一實施例中,第一誤差函數為:,其中X i 為無瑕疵樣本影像302的像素值,Y i 為重構影像312的像素值。
In one embodiment, the first error function is: , where Xi is the pixel value of the
請參照圖5,其為本案根據一實施例所繪示的計算第一重構誤差的示意圖。 Please refer to Figure 5, which is a schematic diagram of calculating the first reconstruction error according to an embodiment of the present invention.
電子裝置10以第一誤差函數E1逐每個像素計算無瑕疵樣本影像302的像素值及重構影像312的像素值之差平方和後計算平方和的平均值,而得到第一重構誤差K1。第一重構誤差K1呈現無瑕疵樣本影像302與重構影像312兩者整體上的誤差值或差異,反映自動編碼器155當下處理(消除)雜訊的程度。
The
第一重構誤差K1可被回饋至自動編碼器155作為調整自動編碼器155的參數的要素之一,以優化自動編碼器155處理雜訊的能力。
The first reconstruction error K1 can be fed back to the
於步驟S230,運算模組159計算第一訓練影像及重構影像的差異以產生重構誤差影像。
In step S230, the
請參照圖6,其為本案根據一實施例所繪示的產生重構誤差影像的示意圖。 Please refer to Figure 6, which is a schematic diagram of the reconstructed erroneous image generated according to an embodiment of the present invention.
如圖6所示,重構影像312與第一訓練影像310之間的差異為重構誤差影像316。
As shown in FIG6 , the difference between the
於一實施例中,第一訓練影像310包括瑕疵314,並且重構誤差影像316包括瑕疵318。其中,瑕疵314為被影像處理前的瑕疵特徵,瑕疵318為被影像處理後的瑕疵特徵。瑕疵318會相似於瑕疵314,故重構誤差影像316可被作為另一個類似於瑕疵樣本影像304(如圖3)的瑕疵影像。
In one embodiment, the
於步驟S240,運算模組159使用第一訓練影像310、重構影像312及重構誤差影像316生成第二訓練影像。
In step S240, the
請參照圖7,其為本案根據一實施例所繪示的生成第二訓練影像的示意圖。 Please refer to Figure 7, which is a schematic diagram of generating a second training image according to an embodiment of the present invention.
如圖7所示,運算模組159合併或疊合第一訓練影像310、重構影像312及重構誤差影像316來生成多維度影像(即第二訓練影像320)。
As shown in FIG. 7 , the
第二訓練影像320為合併或疊合第一訓練影像310、重構影像312及重構誤差影像316所產生,故第二訓練影像320的影像特徵包括瑕疵328,瑕疵328關聯於第一訓練影像310中的瑕疵314及重構誤差影像316中的瑕疵318。
The
於步驟S250,運算模組159輸入第二訓練影像至影像分割網路157,由影像分割網路157產生遮罩影像並由運算模組159計算第二重構誤差。
In step S250, the
請參照圖8,其為本案根據一實施例所繪示的產生瑕疵遮罩影像的示意圖。 Please refer to Figure 8, which is a schematic diagram of generating a defect mask image according to an embodiment of the present invention.
如圖8所示,第二訓練影像320被輸入至影像分割網路157。影像分割網路157根據第二訓練影像320的像素及影像特徵來對第二訓練影像320的像素分類為瑕疵或背景。
As shown in FIG8 , the
於一實施例中,影像分割網路157分類第二訓練影像320的所有像素後產生瑕疵遮罩影像334。瑕疵遮罩影像334包括保留區域338及過濾區域336。保留區域338對應於第二訓練影像320的瑕疵328。過濾區域336對應於第二訓練影像320的背景。
In one embodiment, the
於一實施例中,運算模組159使用瑕疵遮罩影像334進行影像處理,濾除被處理影像對應於過濾區域336的影像區塊,並留下對應於保留區域338的影像區塊。於此實施例中,被留下的影像區塊用於指示被處理影像的瑕疵區塊。
In one embodiment, the
於步驟S250,運算模組159進一步計算第二重構誤差。於一實施例中,運算模組159會輸入瑕疵樣本影像304及瑕疵遮罩影像334至第二誤差函數E2,以計算出第二重構誤差K2。
In step S250, the
於一實施例中,第二誤差函數為:E2=-α t (1-p t )γ log(p t ),其中p t 為使用此瑕疵遮罩影像334可以正確判斷被處理影像為瑕疵影像或非瑕疵影像的機率,α t 及γ為第二誤差函數E2的係數,屬於用於後續回饋調整影像分割網路157的權重值的超參數。
In one embodiment, the second error function is: E2 = -αt (1 - pt ) γlog ( pt ), where pt is the probability of correctly judging the processed image as a defect image or a non -defect image using the defect mask image 334 , αt and γ are coefficients of the second error function E2, which are hyperparameters used for subsequent feedback adjustment of the weight values of the
請參照圖9,其為本案根據一實施例所繪示的計算第二重構誤差的示意圖。 Please refer to Figure 9, which is a schematic diagram of calculating the second reconstruction error according to an embodiment of the present case.
運算模組159以第二誤差函數E2計算瑕疵樣本影像304及瑕疵遮罩影像334之間的差異,以獲得第二重構誤差K2。第二重構誤差K2呈現瑕疵樣本影像304及瑕疵遮罩影像334兩者整體上的誤差值或差異。
The
於步驟S260,根據第一重構誤差K1及第二重構誤差K2計算總重構誤差。 In step S260, the total reconstruction error is calculated based on the first reconstruction error K1 and the second reconstruction error K2.
於一實施例中,運算模組159計算加權公式來得到總重構誤差。
In one embodiment, the
於一實施例中,加權公式為:ES=w 1 E1+w 2 E2,其中w 1及w 2為加權係數(例如總和為100%的比例)。 In one embodiment, the weighting formula is: ES = w 1 E 1 + w 2 E 2, where w 1 and w 2 are weighting coefficients (eg, a ratio whose total is 100%).
請參照圖10,其為本案根據一實施例所繪示的計算總重構誤差的示意圖。 Please refer to Figure 10, which is a schematic diagram of calculating the total reconstruction error according to an embodiment of the present invention.
如上述說明,於步驟S220計算得到的第一重構誤差K1及於步驟S250計算得到的第二重構誤差K2分別被輸入至加權公式ES。運算模組159使用加權公式ES以加權係數w_1及w_2分別對第一重構誤差K1及第二重構誤差K2進行比例分配,加總後計算得到總重構誤差KS。
As described above, the first reconstruction error K1 calculated in step S220 and the second reconstruction error K2 calculated in step S250 are respectively input into the weighting formula ES. The
於步驟S270,運算模組159根據總重構誤差優化自動編碼器155及影像分割網路157的參數。
In step S270, the
於一實施例中,總重構誤差KS可以透過反向傳播法被回授至自動編碼器155及影像分割網路157進行參數優化。
In one embodiment, the total reconstruction error KS can be fed back to the auto-
自動編碼器155及影像分割網路157的參數經過調整後,電子裝置10再一次執行前述步驟S210至步驟S270,持續地訓練自動編碼器155及影像分割網路157並調整自動編碼器155及影像分割網路157的參數,以提高自動編碼器155及影像分割網路157的準確度。
After the parameters of the
於步驟S280,電子裝置10完成自動編碼器155及影像分割網路157的影像訓練及參數優化。
In step S280, the
於一實施例中,電子裝置10會判斷每一次的總重構誤差KS是否小於一閥值。若總重構誤差KS小於閥值,則判定已完成影像瑕疵檢測模型150的建立及參數優化。
In one embodiment, the
於建立及優化影像瑕疵檢測模型150後,影像瑕疵檢測模型150可應用於待檢測影像是否為瑕疵影像的檢測。
After establishing and optimizing the image
請參照圖11,其為本案根據一實施例所繪示的瑕疵影像的檢測方法的流程圖。瑕疵影像的檢測方法可以由圖1的電子裝置10所執行。電子裝置10的處理器120載入儲存媒體110中的多個程式碼以執行多個操作,來實現瑕疵影像的檢測。
Please refer to FIG. 11 , which is a flow chart of a defect image detection method according to an embodiment of the present invention. The defect image detection method can be executed by the
於步驟S1110,電子裝置10獲取待檢測影像。
In step S1110, the
於一實施例中,待檢測影像可以為工廠產線產品影像、晶圓影像、印刷電路板影像、公共場域影像,或拍攝自任何場域的影像。 In one embodiment, the image to be inspected can be a factory production line product image, a wafer image, a printed circuit board image, a public field image, or an image taken from any field.
於一實施例中,電子裝置10包括影像擷取模組(例如攝影機),用於拍攝待測物而獲取待檢測影像。
In one embodiment, the
於步驟S1120,運算模組159輸入待檢測影像至自動編碼器155,並由自動編碼器155輸出檢測重構影像。
In step S1120, the
於一實施例中,自動編碼器155對待檢測影像執行編碼器運算及解碼器運算後會輸出待檢測影像的檢測重構影像。
In one embodiment, the
於此實施例中,自動編碼器155已經過優化,故可以於此步驟中過濾掉待檢測影像的瑕疵。換言之,檢測重構影像為較待檢測影像更正確的影像。
In this embodiment, the
於步驟S1130,運算模組159計算待檢測影像及檢測重構影像的差異以產生檢測重構誤差影像。
In step S1130, the
檢測重構誤差影像為待檢測影像及檢測重構影像兩者的差異影像,換言之,此步驟的目的在於擷取出被判定是待檢測影像的瑕疵的影像區塊,並以檢測重構誤差影像來表示。 The detection and reconstruction error image is the difference image between the image to be detected and the detection and reconstruction image. In other words, the purpose of this step is to extract the image block that is determined to be a defect of the image to be detected and represent it with the detection and reconstruction error image.
於步驟S1140,運算模組159根據待檢測影像、檢測重構影像及檢測重構誤差影像來生成測試影像。
In step S1140, the
於一實施例中,運算模組159合併或疊合待檢測影像、檢測重構影像及檢測重構誤差影像來生成多維度影像(即測試影像)。
In one embodiment, the
由於待檢測影像客觀上存在瑕疵區塊、檢測重構影像為已濾除瑕疵的影像(自動編碼器155認為的無瑕疵影像),及檢測重構誤差影像為屬於待檢測影像的瑕疵區塊的影像(自動編碼器155認為的瑕疵影像),將此三個影像合併或疊合後產生的測試影像,可以凸顯客觀上存在的瑕疵及自動編碼器155判定的瑕疵之交集。
Since there are objective defect blocks in the image to be detected, the detected reconstructed image is an image with defects filtered out (an image without defects considered by the automatic encoder 155), and the detected reconstructed error image is an image belonging to the defect block of the image to be detected (an image considered by the automatic encoder 155), the test image generated by merging or superimposing these three images can highlight the intersection of the objective defects and the defects determined by the
於步驟S1150,運算模組159輸入測試影像至影像分割網路157,由影像分割網路157產生瑕疵遮罩測試影像。
In step S1150, the
於一實施例中,測試影像被輸入至影像分割網路157。影像分割網路157根據測試影像的像素及影像特徵來將測試影像的多個像素區塊分類為瑕疵區域或背景區域。
In one embodiment, the test image is input to the
於一實施例中,影像分割網路157分類測試影像的所有像素區塊後產生瑕疵遮罩測試影像。瑕疵遮罩測試影像包括瑕疵區域及背景區域。瑕疵區
域對應於測試影像的瑕疵的影像區塊。背景區域對應於測試影像的非瑕疵(背景)的影像區塊。
In one embodiment, the
於步驟S1160,運算模組159判斷瑕疵遮罩測試影像的指示面積是否大於閥值。若指示面積等於或小於閥值,則執行步驟S1170。若指示面積大於閥值,則執行步驟S1180。
In step S1160, the
於一實施例中,指示面積為瑕疵遮罩測試影像的瑕疵區域的面積。 In one embodiment, the indicated area is the area of the defect region of the defect mask test image.
於步驟S1170,由於指示面積小於或等於閥值,代表待測試影像的瑕疵量未超過容忍值,故運算模組159判定待檢測影像為無瑕疵影像。
In step S1170, since the indicated area is less than or equal to the valve value, it means that the defect amount of the image to be tested does not exceed the tolerance value, so the
於步驟S1180,由於指示面積大於閥值,代表待測試影像的瑕疵量超過容忍值,故運算模組159判定待檢測影像為瑕疵影像。
In step S1180, since the indicated area is larger than the valve value, it means that the defect amount of the image to be tested exceeds the tolerance value, so the
於一實施例中,電子裝置10會發出通知給使用者(例如物品的外觀出現異常的通知),以供使用者執行對應的處置。
In one embodiment, the
綜上所述,本案提出建立影像瑕疵檢測模型及使用影像瑕疵檢測模型來進行瑕疵影像的檢測,透過加入雜訊樣本及不良樣本至檢測模型的方式來提升訓練模型的多樣性。於檢測階段,檢測結果包括瑕疵在影像中的位置及大小,並根據檢測結果發出待檢測影像是否有瑕疵的通知(即被拍攝的待測物是否有異常)。藉由多個步驟組合而成的影像處理操作來提升模型的參數優化。依據本案的方法所建立的影像瑕疵檢測模型可以提升檢測瑕疵影像的準確性。 In summary, this case proposes to establish an image defect detection model and use the image defect detection model to detect defective images, and improve the diversity of the training model by adding noise samples and bad samples to the detection model. In the detection stage, the detection results include the position and size of the defect in the image, and a notification is issued based on the detection results to determine whether the image to be detected has defects (i.e., whether the object to be detected is abnormal). The model's parameter optimization is improved through image processing operations composed of multiple steps. The image defect detection model established according to the method of this case can improve the accuracy of detecting defective images.
以上所述僅為本案的具體實例,非因此即侷限本案的申請專利範圍,故舉凡運用本案內容所為的等效變化,均同理皆包含於本案的範圍內,合予陳明。 The above is only a specific example of this case, and does not limit the scope of the patent application of this case. Therefore, all equivalent changes made by applying the content of this case are also included in the scope of this case and should be stated.
S210~S280:步驟 S210~S280: Steps
Claims (15)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW112120504A TWI843591B (en) | 2023-06-01 | 2023-06-01 | Method for creating flaw image detection model, method for detecting flaw image and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW112120504A TWI843591B (en) | 2023-06-01 | 2023-06-01 | Method for creating flaw image detection model, method for detecting flaw image and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TWI843591B true TWI843591B (en) | 2024-05-21 |
| TW202449726A TW202449726A (en) | 2024-12-16 |
Family
ID=92077270
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW112120504A TWI843591B (en) | 2023-06-01 | 2023-06-01 | Method for creating flaw image detection model, method for detecting flaw image and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI843591B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI877045B (en) * | 2024-07-11 | 2025-03-11 | 華碩電腦股份有限公司 | Optimization method for defect detection model and electronic device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021142622A1 (en) * | 2020-01-14 | 2021-07-22 | 京东方科技集团股份有限公司 | Method for determining cause of defect, and electronic device, storage medium, and system |
| US20210374928A1 (en) * | 2020-05-26 | 2021-12-02 | Fujitsu Limited | Defect detection method and apparatus |
| TWI762193B (en) * | 2021-02-09 | 2022-04-21 | 鴻海精密工業股份有限公司 | Image defect detection method, image defect detection device, electronic device and storage media |
| TW202225674A (en) * | 2020-12-29 | 2022-07-01 | 鴻海精密工業股份有限公司 | Method for detecting defects of product and computer device |
| CN114764774A (en) * | 2021-01-12 | 2022-07-19 | 富泰华工业(深圳)有限公司 | Defect detection method, device, electronic equipment and computer readable storage medium |
| TW202232380A (en) * | 2021-02-09 | 2022-08-16 | 鴻海精密工業股份有限公司 | Image defect detection method, image defect detection device, electronic device and storage media |
-
2023
- 2023-06-01 TW TW112120504A patent/TWI843591B/en active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021142622A1 (en) * | 2020-01-14 | 2021-07-22 | 京东方科技集团股份有限公司 | Method for determining cause of defect, and electronic device, storage medium, and system |
| US20210374928A1 (en) * | 2020-05-26 | 2021-12-02 | Fujitsu Limited | Defect detection method and apparatus |
| TW202225674A (en) * | 2020-12-29 | 2022-07-01 | 鴻海精密工業股份有限公司 | Method for detecting defects of product and computer device |
| CN114764774A (en) * | 2021-01-12 | 2022-07-19 | 富泰华工业(深圳)有限公司 | Defect detection method, device, electronic equipment and computer readable storage medium |
| TWI762193B (en) * | 2021-02-09 | 2022-04-21 | 鴻海精密工業股份有限公司 | Image defect detection method, image defect detection device, electronic device and storage media |
| TW202232380A (en) * | 2021-02-09 | 2022-08-16 | 鴻海精密工業股份有限公司 | Image defect detection method, image defect detection device, electronic device and storage media |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI877045B (en) * | 2024-07-11 | 2025-03-11 | 華碩電腦股份有限公司 | Optimization method for defect detection model and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| TW202449726A (en) | 2024-12-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110619618B (en) | Surface defect detection method and device and electronic equipment | |
| US12217411B2 (en) | Inspection apparatus, unit selection apparatus, inspection method, and computer-readable storage medium storing an inspection program | |
| JP2024509411A (en) | Defect detection method, device and system | |
| WO2020031984A1 (en) | Component inspection method and inspection system | |
| WO2024208102A1 (en) | Deep-learning-based target detection method for defects of image of inner side of commutator | |
| CN112150460B (en) | Detection method, detection system, device and medium | |
| CN113807378A (en) | Training data increment method, electronic device and computer-readable recording medium | |
| CN106023154A (en) | Multi-temporal SAR image change detection method based on dual-channel convolutional neural network (CNN) | |
| CN113657539A (en) | Display panel micro-defect detection method based on two-stage detection network | |
| TWI843591B (en) | Method for creating flaw image detection model, method for detecting flaw image and electronic device | |
| CN120125585A (en) | A PCB surface defect detection system and method based on improved YOLOv10 | |
| CN117274258B (en) | Method, system, equipment and storage medium for detecting defects of main board image | |
| JP7070308B2 (en) | Estimator generator, inspection device, estimator generator method, and estimator generator | |
| JP2021143884A (en) | Inspection device, inspection method, program, learning device, learning method, and trained dataset | |
| CN114596244A (en) | Infrared image recognition method and system based on vision processing and multi-feature fusion | |
| JP7459697B2 (en) | Anomaly detection system, learning device, anomaly detection program, learning program, anomaly detection method, and learning method | |
| KR20230036650A (en) | Defect detection method and system based on image patch | |
| Supong et al. | PCB Surface Defect Detection Using Defect-Centered Image Generation and Optimized YOLOv8 Architecture | |
| CN119130886A (en) | Image defect detection model establishment method, image detection method and electronic device | |
| JP2022029262A (en) | Image processing apparatus, image processing method, image processing program, and learning device | |
| CN118261997A (en) | Template image generation model establishment method, image generation method, device and equipment | |
| JP7446697B2 (en) | Teacher data creation method and creation device | |
| CN116993698B (en) | Fabric flaw detection method, device, terminal equipment and medium | |
| CN107123105A (en) | Images match defect inspection method based on FAST algorithms | |
| CN118397006B (en) | Circuit board welding failure detection method and system based on image recognition |