[go: up one dir, main page]

TWI843591B - Method for creating flaw image detection model, method for detecting flaw image and electronic device - Google Patents

Method for creating flaw image detection model, method for detecting flaw image and electronic device Download PDF

Info

Publication number
TWI843591B
TWI843591B TW112120504A TW112120504A TWI843591B TW I843591 B TWI843591 B TW I843591B TW 112120504 A TW112120504 A TW 112120504A TW 112120504 A TW112120504 A TW 112120504A TW I843591 B TWI843591 B TW I843591B
Authority
TW
Taiwan
Prior art keywords
image
defect
reconstructed
reconstruction error
error
Prior art date
Application number
TW112120504A
Other languages
Chinese (zh)
Other versions
TW202449726A (en
Inventor
郭千豪
李彥志
Original Assignee
聯策科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 聯策科技股份有限公司 filed Critical 聯策科技股份有限公司
Priority to TW112120504A priority Critical patent/TWI843591B/en
Application granted granted Critical
Publication of TWI843591B publication Critical patent/TWI843591B/en
Publication of TW202449726A publication Critical patent/TW202449726A/en

Links

Images

Landscapes

  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

A method for creating a flaw image detection model includes: generating a first training image by using a flawless sample image and a flawed sample image; inputting the first training image to an autoencoder to output a reconstructed image and calculating a first reconstruction error; computing a difference between the first training image and the reconstructed image to generate a reconstruction error image; generating a second training image by using the first training image, the reconstructed image, and the reconstruction error image; inputting the second training image to an image segmentation network to generate a mask image and calculate a second reconstruction error; calculating a total reconstruction error based on the first reconstruction error and the second reconstruction error; optimizing parameters of the autoencoder and the image segmentation network by the total reconstruction error; and repeating above steps until the flaw image detection model is established and the parameters are optimized.

Description

影像瑕疵檢測模型的建立方法、瑕疵影像的檢測方法及電子裝置Method for establishing image defect detection model, method for detecting defective image and electronic device

本案有關於影像處理模型的建立及影像處理的方法,特別是有關於一種影像瑕疵檢測模型的建立方法、瑕疵影像的檢測方法,及執行這些方法的電子裝置。 This case is about the establishment of an image processing model and an image processing method, and in particular, about a method for establishing an image defect detection model, a method for detecting defective images, and an electronic device for executing these methods.

影像辨識技術廣泛運用於瑕疵檢測,相較於傳統以人工檢測提升檢測速度及精準度。瑕疵檢測技術仰賴機器視覺對待判讀影像及已知瑕疵影像來進行比對,故需事先定義瑕疵影像中的瑕疵特徵。基於不同的應用場域,瑕疵影像的內容也會隨著應用領域的影像而有所不同。使用者需依據應用場域定義對應的瑕疵特徵,然而,這樣的作法耗費大量的時間來準備前置作業。 Image recognition technology is widely used in defect detection, which improves the detection speed and accuracy compared to traditional manual detection. Defect detection technology relies on machine vision to compare the image to be read and the known defect image, so the defect features in the defect image must be defined in advance. Based on different application scenarios, the content of the defect image will also vary with the image of the application field. Users need to define the corresponding defect features according to the application scenario, however, this approach takes a lot of time to prepare for the pre-operation.

為有效率地找出影像中的瑕疵,神經網路演算法被運用來學習影像中的瑕疵特徵來實現自動化影像瑕疵檢測。神經網路演算法可以從大量輸入的未知影像自行推論出影像特徵,然而,在訓練階段中尚未有從未知影像中推論出瑕疵特徵的有效率的方法。 In order to efficiently find defects in images, neural network algorithms are used to learn the defect features in images to achieve automatic image defect detection. Neural network algorithms can infer image features from a large number of unknown images input by themselves. However, there is no efficient method to infer defect features from unknown images during the training stage.

此外,現有技術的自動編碼器可運用於影像重構,在重構過程中濾除瑕疵,來產生無瑕疵的重構影像。然而,若訓練階段的樣本數量不夠或者訓練影像不存在瑕疵,則自動編碼器模型無從學習瑕疵特徵。換言之,輸入自動編碼器的測試影像中的瑕疵仍會在重構影像中出現,導致自動編碼器的輸出結果不正確,即測試階段的判斷結果之精準度低。 In addition, the existing auto-encoder can be used for image reconstruction, filtering out defects during the reconstruction process to produce a defect-free reconstructed image. However, if the number of samples in the training phase is insufficient or the training image does not have defects, the auto-encoder model cannot learn the defect characteristics. In other words, the defects in the test image input to the auto-encoder will still appear in the reconstructed image, resulting in incorrect output results of the auto-encoder, that is, the accuracy of the judgment results in the test phase is low.

據此,在使用神經網路演算法來檢測影像中的瑕疵的方法中,判讀影像中的瑕疵之準確度不佳的問題仍有待解決。 Accordingly, in the method of using a neural network algorithm to detect defects in an image, the problem of poor accuracy in judging defects in the image remains to be solved.

根據本案的一實施例揭示一種影像瑕疵檢測模型的建立方法,影像瑕疵檢測模型用以檢測影像是否有異常。影像瑕疵檢測模型的建立方法包括:a)使用一無瑕疵樣本影像及一瑕疵樣本影像來生成一第一訓練影像;b)輸入該第一訓練影像至一自動編碼器以輸出一重構影像並計算一第一重構誤差;c)計算該第一訓練影像及該重構影像之間的差異以產生一重構誤差影像;d)使用該第一訓練影像、該重構影像及該重構誤差影像生成一第二訓練影像;e)輸入該第二訓練影像至一影像分割網路以產生一遮罩影像並計算一第二重構誤差;f)根據該第一重構誤差及該第二重構誤差計算一總重構誤差;g)根據該總重構誤差執行該自動編碼器及該影像分割網路的參數的優化;以及h)重複執行步驟a)至步驟g)以完成影像瑕疵檢測模型的建立及參數優化,其中影像瑕疵檢測模型包括自動編碼器及影像分割網路。 According to an embodiment of the present invention, a method for establishing an image defect detection model is disclosed. The image defect detection model is used to detect whether an image is abnormal. The method for establishing the image defect detection model includes: a) using a flawless sample image and a flawed sample image to generate a first training image; b) inputting the first training image to an automatic encoder to output a reconstructed image and calculate a first reconstruction error; c) calculating the difference between the first training image and the reconstructed image to generate a reconstructed error image; d) using the first training image, the reconstructed image, and the reconstructed error image to generate a second training image; e) inputting the first training image to an automatic encoder to output a reconstructed image and calculate a first reconstruction error; The second training image is fed to an image segmentation network to generate a mask image and calculate a second reconstruction error; f) a total reconstruction error is calculated based on the first reconstruction error and the second reconstruction error; g) the parameters of the automatic encoder and the image segmentation network are optimized based on the total reconstruction error; and h) steps a) to g) are repeatedly performed to complete the establishment and parameter optimization of the image defect detection model, wherein the image defect detection model includes the automatic encoder and the image segmentation network.

根據本案的一實施例揭示一種檢測瑕疵影像的方法,包括:a)獲取一待檢測影像;b)輸入待檢測影像至一自動編碼器並輸出一重構影像;c)計 算待檢測影像及該重構影像的差異以產生一重構誤差影像;d)根據待檢測影像、重構影像及重構誤差影像產生一測試影像;e)輸入測試影像至一影像分割網路以產生一瑕疵遮罩測試影像;以及f)根據瑕疵遮罩測試影像的一指示面積大小來判斷待檢測影像是否為一瑕疵影像。 According to an embodiment of the present invention, a method for detecting defective images is disclosed, including: a) obtaining an image to be detected; b) inputting the image to be detected into an automatic encoder and outputting a reconstructed image; c) calculating the difference between the image to be detected and the reconstructed image to generate a reconstructed error image; d) generating a test image according to the image to be detected, the reconstructed image and the reconstructed error image; e) inputting the test image into an image segmentation network to generate a defect mask test image; and f) judging whether the image to be detected is a defective image according to an indicated area size of the defect mask test image.

本案提出的影像處理方法可以自動地擴增用來訓練的影像,於後續測試影像中是否有瑕疵的測試階段,大幅地提升判讀影像中是否有瑕疵以及瑕疵在影像中位置的精準度。 The image processing method proposed in this case can automatically expand the images used for training, and in the subsequent testing phase of whether there are defects in the image, it can greatly improve the accuracy of judging whether there are defects in the image and the location of the defects in the image.

10:電子裝置 10: Electronic devices

110:儲存媒體 110: Storage media

120:處理器 120: Processor

150:影像瑕疵檢測模型 150: Image defect detection model

155:自動編碼器 155: Automatic encoder

157:影像分割網路 157: Image segmentation network

159:運算模組 159: Computation module

302:無瑕疵樣本影像 302: flawless sample image

304:瑕疵樣本影像 304: Defective sample image

310:第一訓練影像 310: First training video

312:重構影像 312: Reconstructing the image

314:瑕疵 314: Defects

316:重構誤差影像 316: Reconstructing erroneous images

318:瑕疵 318: Defects

320:第二訓練影像 320: Second training video

328:瑕疵 328: Defects

334:瑕疵遮罩影像 334: Defect mask image

336:過濾區域 336: Filter area

338:保留區域 338: Reserved area

E1:第一誤差函數 E1: First error function

E2:第二誤差函數 E2: Second error function

ES:加權公式 ES: Weighted formula

K1:第一重構誤差 K1: First reconstruction error

K2:第二重構誤差 K2: Second reconstruction error

KS:總重構誤差 KS: Total reconstruction error

S210~S280、S1110~S1180:步驟 S210~S280, S1110~S1180: Steps

圖1為本案根據一實施例所繪示的電子裝置的方塊圖。 FIG1 is a block diagram of an electronic device according to an embodiment of the present invention.

圖2為本案根據一實施例所繪示的影像瑕疵檢測模型的建立方法的流程圖。 FIG2 is a flow chart of a method for establishing an image defect detection model according to an embodiment of the present invention.

圖3為本案根據一實施例所繪示的生成第一訓練圖像的示意圖。 Figure 3 is a schematic diagram of generating a first training image according to an embodiment of the present invention.

圖4為本案根據一實施例所繪示的輸出重構影像的示意圖。 FIG4 is a schematic diagram of an output reconstructed image drawn according to an embodiment of the present invention.

圖5為本案根據一實施例所繪示的計算第一重構誤差的示意圖。 FIG5 is a schematic diagram of calculating the first reconstruction error according to an embodiment of the present invention.

圖6為本案根據一實施例所繪示的產生重構誤差影像的示意圖。 FIG6 is a schematic diagram of a reconstructed erroneous image generated according to an embodiment of the present invention.

圖7為本案根據一實施例所繪示的生成第二訓練影像的示意圖。 FIG7 is a schematic diagram of generating a second training image according to an embodiment of the present invention.

圖8為本案根據一實施例所繪示的產生瑕疵遮罩影像的示意圖。 FIG8 is a schematic diagram of generating a defect mask image according to an embodiment of the present invention.

圖9為本案根據一實施例所繪示的計算第二重構誤差的示意圖。 FIG9 is a schematic diagram of calculating the second reconstruction error according to an embodiment of the present invention.

圖10為本案根據一實施例所繪示的計算總重構誤差的示意圖。 FIG10 is a schematic diagram showing the calculation of the total reconstruction error according to an embodiment of the present invention.

圖11為本案根據一實施例所繪示的瑕疵影像的檢測方法的流程圖。 FIG11 is a flow chart of a defect image detection method according to an embodiment of the present invention.

以下結合圖式和實施例對本案作進一步說明,以使本發明所屬技術領域的相關人員可以更好的理解本發明並能據以實施,但所舉實施例不作為對本發明的限定。 The following further describes the present invention in combination with the drawings and embodiments, so that relevant personnel in the technical field to which the present invention belongs can better understand the present invention and implement it accordingly, but the embodiments are not intended to limit the present invention.

如本文中所使用的,諸如「第一」及「第二」等用語描述了各種元件、組件、區域、層及/或部分,這些元件、組件、區域、層及/或部分不應受這些術語的限制。這些術語僅可用於將一個元素、組件、區域、層或部分與另一個做區分。除非上下文明確指出,否則本文中使用的諸如「第一」及「第二」的用語並不暗示順序或次序。 As used herein, terms such as "first" and "second" describe various elements, components, regions, layers and/or parts, which should not be limited by these terms. These terms may only be used to distinguish one element, component, region, layer or part from another. Unless the context clearly indicates otherwise, the terms such as "first" and "second" used herein do not imply a sequence or order.

請參照圖1,其為本案根據一實施例所繪示的電子裝置的方塊圖。電子裝置10包括儲存媒體110及處理器120。儲存媒體110耦接於處理器120。電子裝置10用以建立影像瑕疵檢測模型150。 Please refer to FIG. 1 , which is a block diagram of an electronic device according to an embodiment of the present invention. The electronic device 10 includes a storage medium 110 and a processor 120. The storage medium 110 is coupled to the processor 120. The electronic device 10 is used to establish an image defect detection model 150.

於一實施例中,影像瑕疵檢測模型150用以檢測影像是否有異常。 In one embodiment, the image defect detection model 150 is used to detect whether the image is abnormal.

儲存媒體110經配置以儲存影像瑕疵檢測模型150。於一實施例中,影像瑕疵檢測模型150係使用神經網路的模型架構來實現。影像瑕疵檢測模型150包括自動編碼器(AutoEncoder)155、影像分割網路157及運算模組159。 The storage medium 110 is configured to store the image defect detection model 150. In one embodiment, the image defect detection model 150 is implemented using a neural network model architecture. The image defect detection model 150 includes an auto encoder 155, an image segmentation network 157, and a computing module 159.

於一實施例中,自動編碼器155可以為卷積自動編碼器(CNN AutoEncoder)、稀疏自動編碼器(Sparse AutoEncoder)、降噪自動編碼器(Denoising AutoEncoder)或其他在多層神經網路架構以非監督式學習演算法來實現的自動編碼器。 In one embodiment, the autoencoder 155 can be a convolutional autoencoder (CNN AutoEncoder), a sparse autoencoder (Sparse AutoEncoder), a denoising autoencoder (Denoising AutoEncoder) or other autoencoders implemented in a multi-layer neural network architecture using an unsupervised learning algorithm.

於一實施例中,影像分割網路(Image Segmentation Network)157可以為物件偵測演算法(例如U-Net或Mask RCNN演算法),用以偵測影像中的物件並於影像中標記物件輪廓並產生遮罩影像。 In one embodiment, the image segmentation network 157 can be an object detection algorithm (such as U-Net or Mask RCNN algorithm) for detecting objects in an image and marking the object outline in the image and generating a mask image.

運算模組159用以執行自動編碼器155及影像分割網路157以外的其他神經網路運算。 The computing module 159 is used to execute other neural network operations other than the automatic encoder 155 and the image segmentation network 157.

於一實施例中,影像瑕疵檢測模型150包括自動編碼器155、影像分割網路157及運算模組159。於一實施例中,自動編碼器155、影像分割網路157及運算模組159為軟體模型,是由多個程式碼實現,使得處理器120載入多個程式碼後執行自動編碼器155、影像分割網路157及運算模組159的操作。 In one embodiment, the image defect detection model 150 includes an automatic encoder 155, an image segmentation network 157, and an operation module 159. In one embodiment, the automatic encoder 155, the image segmentation network 157, and the operation module 159 are software models, which are implemented by multiple program codes, so that the processor 120 loads the multiple program codes and executes the operations of the automatic encoder 155, the image segmentation network 157, and the operation module 159.

於一實施例中,影像瑕疵檢測模型150可以為深度學習演算法,例如卷積神經網路(CNN,Convolutional neural network)、遞迴神經網路(RNN,Recurrent Neural Network)、生成對抗網路(GAN,Generative Adversarial Network)、多層感知器(MLP,Multilayer Perceptron)、深度波茲曼機(DBM,Deep Boltzmann Machine)或長短期記憶網路(LSTM,Long Short-Term Memory)。 In one embodiment, the image defect detection model 150 can be a deep learning algorithm, such as a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), a multilayer perceptron (MLP), a deep Boltzmann machine (DBM), or a long short-term memory network (LSTM).

於一實施例中,處理器可以為但不限於數位訊號處理器(Digital Signal Processor,DSP)、特定用途積體電路(Application Specific Integrated Circuit,ASIC)、中央處理器(Central Processing Unit,CPU)、系統單晶片(System on Chip,SoC)、現場可程式設計閘陣列(Field Programmable Gate Array,FPGA)、網路處理器(Network Processor)晶片或上述元件的組合。 In one embodiment, the processor may be, but is not limited to, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a central processing unit (CPU), a system on chip (SoC), a field programmable gate array (FPGA), a network processor chip, or a combination of the above components.

於一實施例中,儲存媒體可以為但不限於隨機存取記憶體(Random Access Memory,RAM)、快閃記憶體(Flash memory)、唯讀記憶體(Read Only Memory,ROM)、硬碟機(Hard Disk Drive,HDD)、固態硬碟(Solid State Drive,SSD)、光儲存器或上述元件的組合。 In one embodiment, the storage medium may be, but is not limited to, a random access memory (RAM), a flash memory (Flash memory), a read-only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), an optical storage device, or a combination of the above elements.

於一實施例中,電子裝置10可以為但不限於平板電腦、筆記型電腦、個人電腦、桌上型電腦、主機電腦系統、電腦工作站、影像電腦,或其他具備處理器及/或儲存裝置,而可執行安裝其上及/或雲端的應用程式或可執行儲存在本地端或雲端的指令之電子裝置。 In one embodiment, the electronic device 10 may be, but is not limited to, a tablet computer, a laptop computer, a personal computer, a desktop computer, a mainframe computer system, a computer workstation, a video computer, or other electronic devices equipped with a processor and/or a storage device that can execute applications installed thereon and/or in the cloud or can execute instructions stored locally or in the cloud.

於一實施例中,影像瑕疵檢測模型150可應用於,例如工廠產線產品檢測、晶圓檢測、印刷電路板檢測、公共場域異常檢測,或任何使用機器視覺來實現目的性檢測的場域。 In one embodiment, the image defect detection model 150 can be applied to, for example, factory production line product inspection, wafer inspection, printed circuit board inspection, public field anomaly inspection, or any field that uses machine vision to achieve purposeful inspection.

請參照圖2,其為本案根據一實施例所繪示的影像瑕疵檢測模型的建立方法的流程圖。影像瑕疵檢測模型的建立方法可以由圖1的電子裝置10所執行。電子裝置10的處理器120載入儲存媒體110中的多個程式碼以執行多個操作,來實現影像瑕疵檢測模型150的建立與訓練。 Please refer to FIG. 2, which is a flow chart of a method for establishing an image defect detection model according to an embodiment of the present invention. The method for establishing an image defect detection model can be executed by the electronic device 10 of FIG. 1. The processor 120 of the electronic device 10 loads multiple program codes in the storage medium 110 to execute multiple operations to achieve the establishment and training of the image defect detection model 150.

於一實施例中,使用者可基於檢測目的事先取得多個無瑕疵樣本影像。無瑕疵樣本影像係指基於所處場域或應用,其內容為正常或正確的影像。舉例而言,於印刷電路板的應用中,無瑕疵樣本影像係指其上的電路元件正確、電路元件的位置正確及配線正確等的印刷電路板影像。另一實施例中,無瑕疵樣本影像可以為被用於訓練學習的影像。 In one embodiment, the user can obtain multiple flawless sample images in advance for the purpose of detection. The flawless sample image refers to an image whose content is normal or correct based on the field or application. For example, in the application of printed circuit boards, the flawless sample image refers to an image of a printed circuit board with correct circuit components, correct positions of circuit components, and correct wiring. In another embodiment, the flawless sample image can be an image used for training and learning.

無瑕疵樣本影像可事先儲存於儲存媒體110的影像集合(圖未繪示)。 The flawless sample image can be stored in advance in an image collection of the storage medium 110 (not shown).

於步驟S210,運算模組159使用無瑕疵樣本影像及瑕疵樣本影像來生成第一訓練影像。 In step S210, the computing module 159 uses the flawless sample image and the flawed sample image to generate the first training image.

於一實施例中,第一訓練影像是透過合成無瑕疵樣本影像及瑕疵樣本影像來生成。 In one embodiment, the first training image is generated by synthesizing a flawless sample image and a flawed sample image.

請參照圖3,其為本案根據一實施例所繪示的生成第一訓練圖像的示意圖。 Please refer to Figure 3, which is a schematic diagram of generating the first training image according to an embodiment of the present invention.

如圖3所示,無瑕疵樣本影像302與瑕疵樣本影像304被合成而生成第一訓練影像310。第一訓練影像310包括相同或相似於無瑕疵樣本影像302及瑕疵樣本影像304的影像特徵。 As shown in FIG3 , the flawless sample image 302 and the flawed sample image 304 are synthesized to generate a first training image 310. The first training image 310 includes image features that are the same or similar to those of the flawless sample image 302 and the flawed sample image 304.

於一實施例中,瑕疵(flaw)樣本影像304包括雜訊(noise)樣本影像及不良(defect)樣本影像。 In one embodiment, the flaw sample image 304 includes a noise sample image and a defect sample image.

於一實施例中,運算模組159使用隨機函式產生影像雜訊,並執行仿射變換、透視變換、色彩變換或上述的組合來處理影像雜訊後與無瑕疵樣本影像合成,以產生雜訊樣本影像。 In one embodiment, the computing module 159 generates image noise using a random function, and performs affine transformation, perspective transformation, color transformation or a combination thereof to process the image noise and then synthesizes it with a flawless sample image to generate a noise sample image.

於一實施例中,運算模組159執行仿射變換、透視變換、色彩變換或上述的組合來處理任何可被作為瑕疵的影像後與無瑕疵樣本影像合成,以產生不良(defect)樣本影像。 In one embodiment, the computing module 159 performs affine transformation, perspective transformation, color transformation or a combination thereof to process any image that can be regarded as a defect and then synthesizes it with a flawless sample image to generate a defect sample image.

於一實施例中,儲存媒體110儲存的影像集合包括多個無瑕疵樣本影像及多個瑕疵樣本影像。瑕疵樣本影像的數量及無瑕疵樣本影像的數量之間具有一比例。舉例而言,此比例為1比100,即運算模組159取同一張瑕疵樣本影像及影像集合中所有無瑕疵樣本影像中的100張來分別產生100張的第一訓練影像。此比例用以調整第一訓練影像相對於無瑕疵樣本影像的差異性。 In one embodiment, the image set stored in the storage medium 110 includes a plurality of flawless sample images and a plurality of defective sample images. There is a ratio between the number of defective sample images and the number of flawless sample images. For example, the ratio is 1 to 100, that is, the computing module 159 takes the same flawed sample image and 100 flawless sample images in the image set to generate 100 first training images respectively. This ratio is used to adjust the difference between the first training image and the flawless sample image.

於步驟S220,運算模組159輸入第一訓練影像至自動編碼器155,並由自動編碼器155輸出重構影像並計算第一重構誤差。 In step S220, the computing module 159 inputs the first training image to the automatic encoder 155, and the automatic encoder 155 outputs the reconstructed image and calculates the first reconstruction error.

請參照圖4,其為本案根據一實施例所繪示的輸出重構影像的示意圖。 Please refer to Figure 4, which is a schematic diagram of the output reconstructed image drawn according to an embodiment of the present case.

如圖4所示,自動編碼器155的輸入影像為第一訓練影像310。自動編碼器155的編碼器(圖未繪示)及解碼器(圖未繪示)對第一訓練影像310執行影像處理之後輸出無瑕疵樣本影像302的重構影像312。 As shown in FIG. 4 , the input image of the automatic encoder 155 is the first training image 310. The encoder (not shown) and decoder (not shown) of the automatic encoder 155 perform image processing on the first training image 310 and output a reconstructed image 312 of the flawless sample image 302.

於一實施例中,自動編碼器155具有處理第一訓練影像310中瑕疵的能力(例如濾除瑕疵314),而將第一訓練影像310轉換為相同或近似於無瑕疵樣本影像302的影像,即重構影像312。重構影像312為自動編碼器155的輸出影像。 In one embodiment, the automatic encoder 155 has the ability to process defects in the first training image 310 (e.g., filter defects 314), and convert the first training image 310 into an image that is the same as or similar to the defect-free sample image 302, that is, the reconstructed image 312. The reconstructed image 312 is the output image of the automatic encoder 155.

承上述實施例,於步驟S220,電子裝置10進一步計算第一重構誤差。於一實施例中,電子裝置10輸入無瑕疵樣本影像302的像素值及重構影像312的像素值至第一誤差函數來計算出第一重構誤差。 According to the above embodiment, in step S220, the electronic device 10 further calculates the first reconstruction error. In one embodiment, the electronic device 10 inputs the pixel value of the flawless sample image 302 and the pixel value of the reconstructed image 312 into the first error function to calculate the first reconstruction error.

於一實施例中,第一誤差函數為:

Figure 112120504-A0305-02-0010-1
,其中X i 為無瑕疵樣本影像302的像素值,Y i 為重構影像312的像素值。 In one embodiment, the first error function is:
Figure 112120504-A0305-02-0010-1
, where Xi is the pixel value of the flawless sample image 302, and Yi is the pixel value of the reconstructed image 312.

請參照圖5,其為本案根據一實施例所繪示的計算第一重構誤差的示意圖。 Please refer to Figure 5, which is a schematic diagram of calculating the first reconstruction error according to an embodiment of the present invention.

電子裝置10以第一誤差函數E1逐每個像素計算無瑕疵樣本影像302的像素值及重構影像312的像素值之差平方和後計算平方和的平均值,而得到第一重構誤差K1。第一重構誤差K1呈現無瑕疵樣本影像302與重構影像312兩者整體上的誤差值或差異,反映自動編碼器155當下處理(消除)雜訊的程度。 The electronic device 10 calculates the square sum of the difference between the pixel value of the flawless sample image 302 and the pixel value of the reconstructed image 312 for each pixel using the first error function E1, and then calculates the average of the square sum to obtain the first reconstruction error K1. The first reconstruction error K1 presents the overall error value or difference between the flawless sample image 302 and the reconstructed image 312, reflecting the degree of noise processing (elimination) by the automatic encoder 155 at present.

第一重構誤差K1可被回饋至自動編碼器155作為調整自動編碼器155的參數的要素之一,以優化自動編碼器155處理雜訊的能力。 The first reconstruction error K1 can be fed back to the automatic encoder 155 as one of the factors for adjusting the parameters of the automatic encoder 155 to optimize the ability of the automatic encoder 155 to process noise.

於步驟S230,運算模組159計算第一訓練影像及重構影像的差異以產生重構誤差影像。 In step S230, the computing module 159 calculates the difference between the first training image and the reconstructed image to generate a reconstructed error image.

請參照圖6,其為本案根據一實施例所繪示的產生重構誤差影像的示意圖。 Please refer to Figure 6, which is a schematic diagram of the reconstructed erroneous image generated according to an embodiment of the present invention.

如圖6所示,重構影像312與第一訓練影像310之間的差異為重構誤差影像316。 As shown in FIG6 , the difference between the reconstructed image 312 and the first training image 310 is the reconstructed error image 316.

於一實施例中,第一訓練影像310包括瑕疵314,並且重構誤差影像316包括瑕疵318。其中,瑕疵314為被影像處理前的瑕疵特徵,瑕疵318為被影像處理後的瑕疵特徵。瑕疵318會相似於瑕疵314,故重構誤差影像316可被作為另一個類似於瑕疵樣本影像304(如圖3)的瑕疵影像。 In one embodiment, the first training image 310 includes a defect 314, and the reconstructed error image 316 includes a defect 318. The defect 314 is a defect feature before image processing, and the defect 318 is a defect feature after image processing. The defect 318 is similar to the defect 314, so the reconstructed error image 316 can be regarded as another defect image similar to the defect sample image 304 (as shown in FIG. 3).

於步驟S240,運算模組159使用第一訓練影像310、重構影像312及重構誤差影像316生成第二訓練影像。 In step S240, the computing module 159 generates a second training image using the first training image 310, the reconstructed image 312, and the reconstructed error image 316.

請參照圖7,其為本案根據一實施例所繪示的生成第二訓練影像的示意圖。 Please refer to Figure 7, which is a schematic diagram of generating a second training image according to an embodiment of the present invention.

如圖7所示,運算模組159合併或疊合第一訓練影像310、重構影像312及重構誤差影像316來生成多維度影像(即第二訓練影像320)。 As shown in FIG. 7 , the computing module 159 combines or overlaps the first training image 310 , the reconstructed image 312 , and the reconstructed error image 316 to generate a multi-dimensional image (i.e., the second training image 320 ).

第二訓練影像320為合併或疊合第一訓練影像310、重構影像312及重構誤差影像316所產生,故第二訓練影像320的影像特徵包括瑕疵328,瑕疵328關聯於第一訓練影像310中的瑕疵314及重構誤差影像316中的瑕疵318。 The second training image 320 is generated by merging or superimposing the first training image 310, the reconstructed image 312, and the reconstructed error image 316. Therefore, the image features of the second training image 320 include a defect 328, which is related to the defect 314 in the first training image 310 and the defect 318 in the reconstructed error image 316.

於步驟S250,運算模組159輸入第二訓練影像至影像分割網路157,由影像分割網路157產生遮罩影像並由運算模組159計算第二重構誤差。 In step S250, the computing module 159 inputs the second training image to the image segmentation network 157, the image segmentation network 157 generates a mask image and the computing module 159 calculates the second reconstruction error.

請參照圖8,其為本案根據一實施例所繪示的產生瑕疵遮罩影像的示意圖。 Please refer to Figure 8, which is a schematic diagram of generating a defect mask image according to an embodiment of the present invention.

如圖8所示,第二訓練影像320被輸入至影像分割網路157。影像分割網路157根據第二訓練影像320的像素及影像特徵來對第二訓練影像320的像素分類為瑕疵或背景。 As shown in FIG8 , the second training image 320 is input to the image segmentation network 157. The image segmentation network 157 classifies the pixels of the second training image 320 as defects or background according to the pixels and image features of the second training image 320.

於一實施例中,影像分割網路157分類第二訓練影像320的所有像素後產生瑕疵遮罩影像334。瑕疵遮罩影像334包括保留區域338及過濾區域336。保留區域338對應於第二訓練影像320的瑕疵328。過濾區域336對應於第二訓練影像320的背景。 In one embodiment, the image segmentation network 157 generates a defect mask image 334 after classifying all pixels of the second training image 320. The defect mask image 334 includes a reserved area 338 and a filter area 336. The reserved area 338 corresponds to the defect 328 of the second training image 320. The filter area 336 corresponds to the background of the second training image 320.

於一實施例中,運算模組159使用瑕疵遮罩影像334進行影像處理,濾除被處理影像對應於過濾區域336的影像區塊,並留下對應於保留區域338的影像區塊。於此實施例中,被留下的影像區塊用於指示被處理影像的瑕疵區塊。 In one embodiment, the computing module 159 uses the defect mask image 334 to perform image processing, filters out the image block corresponding to the filter area 336 of the processed image, and leaves the image block corresponding to the reserved area 338. In this embodiment, the left image block is used to indicate the defect block of the processed image.

於步驟S250,運算模組159進一步計算第二重構誤差。於一實施例中,運算模組159會輸入瑕疵樣本影像304及瑕疵遮罩影像334至第二誤差函數E2,以計算出第二重構誤差K2。 In step S250, the calculation module 159 further calculates the second reconstruction error. In one embodiment, the calculation module 159 inputs the defect sample image 304 and the defect mask image 334 to the second error function E2 to calculate the second reconstruction error K2.

於一實施例中,第二誤差函數為:E2=-α t (1-p t )γ log(p t ),其中p t 為使用此瑕疵遮罩影像334可以正確判斷被處理影像為瑕疵影像或非瑕疵影像的機率,α t 及γ為第二誤差函數E2的係數,屬於用於後續回饋調整影像分割網路157的權重值的超參數。 In one embodiment, the second error function is: E2 = -αt (1 - pt ) γlog ( pt ), where pt is the probability of correctly judging the processed image as a defect image or a non -defect image using the defect mask image 334 , αt and γ are coefficients of the second error function E2, which are hyperparameters used for subsequent feedback adjustment of the weight values of the image segmentation network 157.

請參照圖9,其為本案根據一實施例所繪示的計算第二重構誤差的示意圖。 Please refer to Figure 9, which is a schematic diagram of calculating the second reconstruction error according to an embodiment of the present case.

運算模組159以第二誤差函數E2計算瑕疵樣本影像304及瑕疵遮罩影像334之間的差異,以獲得第二重構誤差K2。第二重構誤差K2呈現瑕疵樣本影像304及瑕疵遮罩影像334兩者整體上的誤差值或差異。 The operation module 159 calculates the difference between the defect sample image 304 and the defect mask image 334 using the second error function E2 to obtain the second reconstruction error K2. The second reconstruction error K2 presents the overall error value or difference between the defect sample image 304 and the defect mask image 334.

於步驟S260,根據第一重構誤差K1及第二重構誤差K2計算總重構誤差。 In step S260, the total reconstruction error is calculated based on the first reconstruction error K1 and the second reconstruction error K2.

於一實施例中,運算模組159計算加權公式來得到總重構誤差。 In one embodiment, the computing module 159 calculates a weighted formula to obtain a total reconstruction error.

於一實施例中,加權公式為:ES=w 1 E1+w 2 E2,其中w 1w 2為加權係數(例如總和為100%的比例)。 In one embodiment, the weighting formula is: ES = w 1 E 1 + w 2 E 2, where w 1 and w 2 are weighting coefficients (eg, a ratio whose total is 100%).

請參照圖10,其為本案根據一實施例所繪示的計算總重構誤差的示意圖。 Please refer to Figure 10, which is a schematic diagram of calculating the total reconstruction error according to an embodiment of the present invention.

如上述說明,於步驟S220計算得到的第一重構誤差K1及於步驟S250計算得到的第二重構誤差K2分別被輸入至加權公式ES。運算模組159使用加權公式ES以加權係數w_1及w_2分別對第一重構誤差K1及第二重構誤差K2進行比例分配,加總後計算得到總重構誤差KS。 As described above, the first reconstruction error K1 calculated in step S220 and the second reconstruction error K2 calculated in step S250 are respectively input into the weighting formula ES. The operation module 159 uses the weighting formula ES to proportionally distribute the first reconstruction error K1 and the second reconstruction error K2 with weighting coefficients w_1 and w_2, respectively, and calculates the total reconstruction error KS after adding them up.

於步驟S270,運算模組159根據總重構誤差優化自動編碼器155及影像分割網路157的參數。 In step S270, the computing module 159 optimizes the parameters of the automatic encoder 155 and the image segmentation network 157 according to the total reconstruction error.

於一實施例中,總重構誤差KS可以透過反向傳播法被回授至自動編碼器155及影像分割網路157進行參數優化。 In one embodiment, the total reconstruction error KS can be fed back to the auto-encoder 155 and the image segmentation network 157 for parameter optimization via back propagation.

自動編碼器155及影像分割網路157的參數經過調整後,電子裝置10再一次執行前述步驟S210至步驟S270,持續地訓練自動編碼器155及影像分割網路157並調整自動編碼器155及影像分割網路157的參數,以提高自動編碼器155及影像分割網路157的準確度。 After the parameters of the automatic encoder 155 and the image segmentation network 157 are adjusted, the electronic device 10 executes the aforementioned steps S210 to S270 again, continuously training the automatic encoder 155 and the image segmentation network 157 and adjusting the parameters of the automatic encoder 155 and the image segmentation network 157 to improve the accuracy of the automatic encoder 155 and the image segmentation network 157.

於步驟S280,電子裝置10完成自動編碼器155及影像分割網路157的影像訓練及參數優化。 In step S280, the electronic device 10 completes the image training and parameter optimization of the automatic encoder 155 and the image segmentation network 157.

於一實施例中,電子裝置10會判斷每一次的總重構誤差KS是否小於一閥值。若總重構誤差KS小於閥值,則判定已完成影像瑕疵檢測模型150的建立及參數優化。 In one embodiment, the electronic device 10 determines whether the total reconstruction error KS is less than a threshold value each time. If the total reconstruction error KS is less than the threshold value, it is determined that the establishment and parameter optimization of the image defect detection model 150 have been completed.

於建立及優化影像瑕疵檢測模型150後,影像瑕疵檢測模型150可應用於待檢測影像是否為瑕疵影像的檢測。 After establishing and optimizing the image defect detection model 150, the image defect detection model 150 can be applied to detect whether the image to be detected is a defective image.

請參照圖11,其為本案根據一實施例所繪示的瑕疵影像的檢測方法的流程圖。瑕疵影像的檢測方法可以由圖1的電子裝置10所執行。電子裝置10的處理器120載入儲存媒體110中的多個程式碼以執行多個操作,來實現瑕疵影像的檢測。 Please refer to FIG. 11 , which is a flow chart of a defect image detection method according to an embodiment of the present invention. The defect image detection method can be executed by the electronic device 10 of FIG. 1 . The processor 120 of the electronic device 10 loads multiple program codes in the storage medium 110 to execute multiple operations to realize the detection of defect images.

於步驟S1110,電子裝置10獲取待檢測影像。 In step S1110, the electronic device 10 obtains the image to be detected.

於一實施例中,待檢測影像可以為工廠產線產品影像、晶圓影像、印刷電路板影像、公共場域影像,或拍攝自任何場域的影像。 In one embodiment, the image to be inspected can be a factory production line product image, a wafer image, a printed circuit board image, a public field image, or an image taken from any field.

於一實施例中,電子裝置10包括影像擷取模組(例如攝影機),用於拍攝待測物而獲取待檢測影像。 In one embodiment, the electronic device 10 includes an image capture module (such as a camera) for photographing the object to be tested to obtain an image to be tested.

於步驟S1120,運算模組159輸入待檢測影像至自動編碼器155,並由自動編碼器155輸出檢測重構影像。 In step S1120, the computing module 159 inputs the image to be detected to the automatic encoder 155, and the automatic encoder 155 outputs the detected reconstructed image.

於一實施例中,自動編碼器155對待檢測影像執行編碼器運算及解碼器運算後會輸出待檢測影像的檢測重構影像。 In one embodiment, the automatic encoder 155 performs encoder operations and decoder operations on the image to be detected and outputs a detected reconstructed image of the image to be detected.

於此實施例中,自動編碼器155已經過優化,故可以於此步驟中過濾掉待檢測影像的瑕疵。換言之,檢測重構影像為較待檢測影像更正確的影像。 In this embodiment, the automatic encoder 155 has been optimized, so it can filter out the defects of the image to be detected in this step. In other words, the detected reconstructed image is a more accurate image than the image to be detected.

於步驟S1130,運算模組159計算待檢測影像及檢測重構影像的差異以產生檢測重構誤差影像。 In step S1130, the computing module 159 calculates the difference between the image to be detected and the detected reconstructed image to generate a detected reconstructed error image.

檢測重構誤差影像為待檢測影像及檢測重構影像兩者的差異影像,換言之,此步驟的目的在於擷取出被判定是待檢測影像的瑕疵的影像區塊,並以檢測重構誤差影像來表示。 The detection and reconstruction error image is the difference image between the image to be detected and the detection and reconstruction image. In other words, the purpose of this step is to extract the image block that is determined to be a defect of the image to be detected and represent it with the detection and reconstruction error image.

於步驟S1140,運算模組159根據待檢測影像、檢測重構影像及檢測重構誤差影像來生成測試影像。 In step S1140, the computing module 159 generates a test image based on the image to be detected, the detected reconstructed image, and the detected reconstructed error image.

於一實施例中,運算模組159合併或疊合待檢測影像、檢測重構影像及檢測重構誤差影像來生成多維度影像(即測試影像)。 In one embodiment, the computing module 159 combines or overlaps the image to be detected, the detected reconstructed image, and the detected reconstructed error image to generate a multi-dimensional image (i.e., a test image).

由於待檢測影像客觀上存在瑕疵區塊、檢測重構影像為已濾除瑕疵的影像(自動編碼器155認為的無瑕疵影像),及檢測重構誤差影像為屬於待檢測影像的瑕疵區塊的影像(自動編碼器155認為的瑕疵影像),將此三個影像合併或疊合後產生的測試影像,可以凸顯客觀上存在的瑕疵及自動編碼器155判定的瑕疵之交集。 Since there are objective defect blocks in the image to be detected, the detected reconstructed image is an image with defects filtered out (an image without defects considered by the automatic encoder 155), and the detected reconstructed error image is an image belonging to the defect block of the image to be detected (an image considered by the automatic encoder 155), the test image generated by merging or superimposing these three images can highlight the intersection of the objective defects and the defects determined by the automatic encoder 155.

於步驟S1150,運算模組159輸入測試影像至影像分割網路157,由影像分割網路157產生瑕疵遮罩測試影像。 In step S1150, the computing module 159 inputs the test image to the image segmentation network 157, and the image segmentation network 157 generates a defect mask test image.

於一實施例中,測試影像被輸入至影像分割網路157。影像分割網路157根據測試影像的像素及影像特徵來將測試影像的多個像素區塊分類為瑕疵區域或背景區域。 In one embodiment, the test image is input to the image segmentation network 157. The image segmentation network 157 classifies multiple pixel blocks of the test image into defect areas or background areas based on the pixels and image features of the test image.

於一實施例中,影像分割網路157分類測試影像的所有像素區塊後產生瑕疵遮罩測試影像。瑕疵遮罩測試影像包括瑕疵區域及背景區域。瑕疵區 域對應於測試影像的瑕疵的影像區塊。背景區域對應於測試影像的非瑕疵(背景)的影像區塊。 In one embodiment, the image segmentation network 157 generates a defect mask test image after classifying all pixel blocks of the test image. The defect mask test image includes a defect area and a background area. The defect area corresponds to the image block of the defect of the test image. The background area corresponds to the image block of the non-defect (background) of the test image.

於步驟S1160,運算模組159判斷瑕疵遮罩測試影像的指示面積是否大於閥值。若指示面積等於或小於閥值,則執行步驟S1170。若指示面積大於閥值,則執行步驟S1180。 In step S1160, the computing module 159 determines whether the indicated area of the defect mask test image is greater than the valve value. If the indicated area is equal to or less than the valve value, step S1170 is executed. If the indicated area is greater than the valve value, step S1180 is executed.

於一實施例中,指示面積為瑕疵遮罩測試影像的瑕疵區域的面積。 In one embodiment, the indicated area is the area of the defect region of the defect mask test image.

於步驟S1170,由於指示面積小於或等於閥值,代表待測試影像的瑕疵量未超過容忍值,故運算模組159判定待檢測影像為無瑕疵影像。 In step S1170, since the indicated area is less than or equal to the valve value, it means that the defect amount of the image to be tested does not exceed the tolerance value, so the calculation module 159 determines that the image to be tested is a flawless image.

於步驟S1180,由於指示面積大於閥值,代表待測試影像的瑕疵量超過容忍值,故運算模組159判定待檢測影像為瑕疵影像。 In step S1180, since the indicated area is larger than the valve value, it means that the defect amount of the image to be tested exceeds the tolerance value, so the calculation module 159 determines that the image to be tested is a defective image.

於一實施例中,電子裝置10會發出通知給使用者(例如物品的外觀出現異常的通知),以供使用者執行對應的處置。 In one embodiment, the electronic device 10 will send a notification to the user (e.g., a notification that the appearance of an item is abnormal) so that the user can perform corresponding processing.

綜上所述,本案提出建立影像瑕疵檢測模型及使用影像瑕疵檢測模型來進行瑕疵影像的檢測,透過加入雜訊樣本及不良樣本至檢測模型的方式來提升訓練模型的多樣性。於檢測階段,檢測結果包括瑕疵在影像中的位置及大小,並根據檢測結果發出待檢測影像是否有瑕疵的通知(即被拍攝的待測物是否有異常)。藉由多個步驟組合而成的影像處理操作來提升模型的參數優化。依據本案的方法所建立的影像瑕疵檢測模型可以提升檢測瑕疵影像的準確性。 In summary, this case proposes to establish an image defect detection model and use the image defect detection model to detect defective images, and improve the diversity of the training model by adding noise samples and bad samples to the detection model. In the detection stage, the detection results include the position and size of the defect in the image, and a notification is issued based on the detection results to determine whether the image to be detected has defects (i.e., whether the object to be detected is abnormal). The model's parameter optimization is improved through image processing operations composed of multiple steps. The image defect detection model established according to the method of this case can improve the accuracy of detecting defective images.

以上所述僅為本案的具體實例,非因此即侷限本案的申請專利範圍,故舉凡運用本案內容所為的等效變化,均同理皆包含於本案的範圍內,合予陳明。 The above is only a specific example of this case, and does not limit the scope of the patent application of this case. Therefore, all equivalent changes made by applying the content of this case are also included in the scope of this case and should be stated.

S210~S280:步驟 S210~S280: Steps

Claims (15)

一種影像瑕疵檢測模型的建立方法,該影像瑕疵檢測模型用以檢測影像是否有異常,該方法包括:a)使用一無瑕疵樣本影像及一瑕疵樣本影像來生成一第一訓練影像;b)輸入該第一訓練影像至一自動編碼器以輸出一重構影像並計算一第一重構誤差;c)計算該第一訓練影像及該重構影像之間的差異以產生一重構誤差影像;d)使用該第一訓練影像、該重構影像及該重構誤差影像生成一第二訓練影像;e)輸入該第二訓練影像至一影像分割網路以產生一遮罩影像並計算一第二重構誤差;f)根據該第一重構誤差及該第二重構誤差計算一總重構誤差;g)根據該總重構誤差執行該自動編碼器及該影像分割網路的參數的優化;以及h)重複執行步驟a)至步驟g)以完成該影像瑕疵檢測模型的建立及參數優化,其中該影像瑕疵檢測模型包括該自動編碼器及該影像分割網路。 A method for establishing an image defect detection model, wherein the image defect detection model is used to detect whether an image is abnormal, the method comprising: a) using a flawless sample image and a flawed sample image to generate a first training image; b) inputting the first training image to an automatic encoder to output a reconstructed image and calculating a first reconstruction error; c) calculating the difference between the first training image and the reconstructed image to generate a reconstruction error image; d) using the first training image, the reconstructed image and the reconstruction error image to generate a second training image; e) inputting the second training image into an image segmentation network to generate a mask image and calculate a second reconstruction error; f) calculating a total reconstruction error according to the first reconstruction error and the second reconstruction error; g) optimizing the parameters of the automatic encoder and the image segmentation network according to the total reconstruction error; and h) repeatedly executing steps a) to g) to complete the establishment and parameter optimization of the image defect detection model, wherein the image defect detection model includes the automatic encoder and the image segmentation network. 如請求項1所述的方法,其中步驟a)之前包括:使用一隨機函式生成一雜訊樣本影像並合成該雜訊樣本影像及該無瑕疵樣本影像以獲得該瑕疵樣本影像;或者變換一不良樣本影像並合成變換後的該不良樣本影像及該無瑕疵樣本影像以獲得該瑕疵樣本影像。 As described in claim 1, before step a), the method includes: using a random function to generate a noise sample image and synthesizing the noise sample image and the flawless sample image to obtain the flawed sample image; or transforming a bad sample image and synthesizing the transformed bad sample image and the flawless sample image to obtain the flawed sample image. 如請求項1所述的方法,其中步驟b)包括: 輸入該無瑕疵樣本影像的像素值及該重構影像的像素值至一第一誤差函數進行計算以得到該第一重構誤差。 As described in claim 1, step b) includes: Inputting the pixel value of the flawless sample image and the pixel value of the reconstructed image into a first error function for calculation to obtain the first reconstruction error. 如請求項1所述的方法,其中步驟d)包括:合併該第一訓練影像、該重構影像及該重構誤差影像來生成該第二訓練影像。 As described in claim 1, step d) includes: merging the first training image, the reconstructed image and the reconstructed error image to generate the second training image. 如請求項1所述的方法,其中步驟e)包括:透過該影像分割網路分類該第二訓練影像的像素以獲得對應於瑕疵的一保留區域及對應於背景的一過濾區域;以及輸出包括該保留區域及該過濾區域的該遮罩影像。 The method as claimed in claim 1, wherein step e) comprises: classifying the pixels of the second training image through the image segmentation network to obtain a reserved area corresponding to the defect and a filtered area corresponding to the background; and outputting the mask image including the reserved area and the filtered area. 如請求項5所述的方法,其中步驟e)包括:輸入該瑕疵樣本影像的像素值及該遮罩影像的像素值至一第二誤差函數進行計算以得到該第二重構誤差。 As described in claim 5, step e) includes: inputting the pixel value of the defect sample image and the pixel value of the mask image into a second error function for calculation to obtain the second reconstruction error. 如請求項6所述的方法,其中步驟f)包括:使用一第一權重及一第二權重分別對該第一重構誤差及該第二重構誤差進行加權計算,以獲得該總重構誤差。 As described in claim 6, step f) includes: using a first weight and a second weight to perform weighted calculation on the first reconstruction error and the second reconstruction error respectively to obtain the total reconstruction error. 如請求項7所述的方法,其中步驟g)包括:根據該第二誤差函數的參數值、該第一權重及該第二權重調整該自動編碼器及該影像分割網路的參數。 The method as claimed in claim 7, wherein step g) comprises: adjusting the parameters of the automatic encoder and the image segmentation network according to the parameter value of the second error function, the first weight and the second weight. 如請求項8所述的方法,其中步驟h)包括:重複執行步驟a)至步驟g)至該總重構誤差小於一閥值時完成該影像瑕疵檢測模型的建立及參數優化。 As described in claim 8, step h) includes: repeatedly executing steps a) to g) until the total reconstruction error is less than a threshold value to complete the establishment of the image defect detection model and parameter optimization. 一種用於建立影像瑕疵檢測模型的電子裝置,包括:一儲存媒體,經配置以儲存該影像瑕疵檢測模型;以及一處理器,耦接該儲存媒體,經配置以執行如請求項1至9中任一項的方法。 An electronic device for establishing an image defect detection model comprises: a storage medium configured to store the image defect detection model; and a processor coupled to the storage medium and configured to execute a method as recited in any one of claims 1 to 9. 一種使用如請求項1至9中任一項的方法所建立的該影像瑕疵檢測模型來檢測瑕疵影像的方法,包括:a1)獲取一待檢測影像;b1)輸入該待檢測影像至該自動編碼器並輸出一檢測重構影像;c1)計算該待檢測影像及該檢測重構影像的差異以產生一檢測重構誤差影像;d1)根據該待檢測影像、該檢測重構影像及該檢測重構誤差影像產生一測試影像;e1)輸入該測試影像至該影像分割網路以產生一瑕疵遮罩測試影像;以及f1)根據該瑕疵遮罩測試影像的一指示面積大小來判斷該待檢測影像是否為一瑕疵影像。 A method for detecting defective images using the image defect detection model established by the method of any one of claims 1 to 9, comprising: a1) obtaining an image to be detected; b1) inputting the image to be detected into the automatic encoder and outputting a detection reconstructed image; c1) calculating the difference between the image to be detected and the detection reconstructed image to generate a detection reconstructed error image; d1) generating a test image based on the image to be detected, the detection reconstructed image and the detection reconstructed error image; e1) inputting the test image into the image segmentation network to generate a defect mask test image; and f1) judging whether the image to be detected is a defective image based on an indicated area size of the defect mask test image. 如請求項11所述的方法,其中步驟d1)包括:合併該待檢測影像、該檢測重構影像及該檢測重構誤差影像來生成該測試影像。 As described in claim 11, step d1) includes: merging the image to be detected, the detected reconstructed image and the detected reconstructed error image to generate the test image. 如請求項11所述的方法,其中步驟e1)包括:透過該影像分割網路分類該測試影像的像素以獲得一瑕疵區域及一背景區域;以及輸出包括該瑕疵區域及該背景區域的該瑕疵遮罩測試影像。 The method as claimed in claim 11, wherein step e1) comprises: classifying the pixels of the test image through the image segmentation network to obtain a defect area and a background area; and outputting the defect mask test image including the defect area and the background area. 如請求項12所述的方法,其中步驟f1)包括:計算該瑕疵遮罩測試影像的該瑕疵區域的面積以獲得該指示面積;若該指示面積大於一閥值,判斷該待檢測影像為該瑕疵影像;及若該指示面積小於或等於該閥值,判斷該待檢測影像為一無瑕疵影像。 The method of claim 12, wherein step f1) includes: calculating the area of the defect region of the defect mask test image to obtain the indication area; if the indication area is greater than a valve value, judging that the image to be tested is the defect image; and if the indication area is less than or equal to the valve value, judging that the image to be tested is a non-defective image. 一種用於檢測瑕疵影像的電子裝置,包括: 一儲存媒體,經配置以儲存該瑕疵影像檢測模型;以及一處理器,耦接該儲存媒體,經配置以執行如請求項11的方法。 An electronic device for detecting defective images, comprising: a storage medium configured to store the defective image detection model; and a processor coupled to the storage medium and configured to execute the method of claim 11.
TW112120504A 2023-06-01 2023-06-01 Method for creating flaw image detection model, method for detecting flaw image and electronic device TWI843591B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW112120504A TWI843591B (en) 2023-06-01 2023-06-01 Method for creating flaw image detection model, method for detecting flaw image and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW112120504A TWI843591B (en) 2023-06-01 2023-06-01 Method for creating flaw image detection model, method for detecting flaw image and electronic device

Publications (2)

Publication Number Publication Date
TWI843591B true TWI843591B (en) 2024-05-21
TW202449726A TW202449726A (en) 2024-12-16

Family

ID=92077270

Family Applications (1)

Application Number Title Priority Date Filing Date
TW112120504A TWI843591B (en) 2023-06-01 2023-06-01 Method for creating flaw image detection model, method for detecting flaw image and electronic device

Country Status (1)

Country Link
TW (1) TWI843591B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI877045B (en) * 2024-07-11 2025-03-11 華碩電腦股份有限公司 Optimization method for defect detection model and electronic device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021142622A1 (en) * 2020-01-14 2021-07-22 京东方科技集团股份有限公司 Method for determining cause of defect, and electronic device, storage medium, and system
US20210374928A1 (en) * 2020-05-26 2021-12-02 Fujitsu Limited Defect detection method and apparatus
TWI762193B (en) * 2021-02-09 2022-04-21 鴻海精密工業股份有限公司 Image defect detection method, image defect detection device, electronic device and storage media
TW202225674A (en) * 2020-12-29 2022-07-01 鴻海精密工業股份有限公司 Method for detecting defects of product and computer device
CN114764774A (en) * 2021-01-12 2022-07-19 富泰华工业(深圳)有限公司 Defect detection method, device, electronic equipment and computer readable storage medium
TW202232380A (en) * 2021-02-09 2022-08-16 鴻海精密工業股份有限公司 Image defect detection method, image defect detection device, electronic device and storage media

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021142622A1 (en) * 2020-01-14 2021-07-22 京东方科技集团股份有限公司 Method for determining cause of defect, and electronic device, storage medium, and system
US20210374928A1 (en) * 2020-05-26 2021-12-02 Fujitsu Limited Defect detection method and apparatus
TW202225674A (en) * 2020-12-29 2022-07-01 鴻海精密工業股份有限公司 Method for detecting defects of product and computer device
CN114764774A (en) * 2021-01-12 2022-07-19 富泰华工业(深圳)有限公司 Defect detection method, device, electronic equipment and computer readable storage medium
TWI762193B (en) * 2021-02-09 2022-04-21 鴻海精密工業股份有限公司 Image defect detection method, image defect detection device, electronic device and storage media
TW202232380A (en) * 2021-02-09 2022-08-16 鴻海精密工業股份有限公司 Image defect detection method, image defect detection device, electronic device and storage media

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI877045B (en) * 2024-07-11 2025-03-11 華碩電腦股份有限公司 Optimization method for defect detection model and electronic device

Also Published As

Publication number Publication date
TW202449726A (en) 2024-12-16

Similar Documents

Publication Publication Date Title
CN110619618B (en) Surface defect detection method and device and electronic equipment
US12217411B2 (en) Inspection apparatus, unit selection apparatus, inspection method, and computer-readable storage medium storing an inspection program
JP2024509411A (en) Defect detection method, device and system
WO2020031984A1 (en) Component inspection method and inspection system
WO2024208102A1 (en) Deep-learning-based target detection method for defects of image of inner side of commutator
CN112150460B (en) Detection method, detection system, device and medium
CN113807378A (en) Training data increment method, electronic device and computer-readable recording medium
CN106023154A (en) Multi-temporal SAR image change detection method based on dual-channel convolutional neural network (CNN)
CN113657539A (en) Display panel micro-defect detection method based on two-stage detection network
TWI843591B (en) Method for creating flaw image detection model, method for detecting flaw image and electronic device
CN120125585A (en) A PCB surface defect detection system and method based on improved YOLOv10
CN117274258B (en) Method, system, equipment and storage medium for detecting defects of main board image
JP7070308B2 (en) Estimator generator, inspection device, estimator generator method, and estimator generator
JP2021143884A (en) Inspection device, inspection method, program, learning device, learning method, and trained dataset
CN114596244A (en) Infrared image recognition method and system based on vision processing and multi-feature fusion
JP7459697B2 (en) Anomaly detection system, learning device, anomaly detection program, learning program, anomaly detection method, and learning method
KR20230036650A (en) Defect detection method and system based on image patch
Supong et al. PCB Surface Defect Detection Using Defect-Centered Image Generation and Optimized YOLOv8 Architecture
CN119130886A (en) Image defect detection model establishment method, image detection method and electronic device
JP2022029262A (en) Image processing apparatus, image processing method, image processing program, and learning device
CN118261997A (en) Template image generation model establishment method, image generation method, device and equipment
JP7446697B2 (en) Teacher data creation method and creation device
CN116993698B (en) Fabric flaw detection method, device, terminal equipment and medium
CN107123105A (en) Images match defect inspection method based on FAST algorithms
CN118397006B (en) Circuit board welding failure detection method and system based on image recognition