TWI908645B - Defect detection device and defect detection method - Google Patents

Abstract

A defect detection apparatus and method are provided, which can simultaneously improve defect detection sensitivity and suppress false alarms, while reducing user workload. The apparatus and method have the following components: An image acquisition unit for acquiring an image of a predetermined area of a sample to be inspected; a segmentation processing unit for segmenting the acquired image into multiple segments; and a defect candidate point extraction unit for extracting defect candidates from each segment segmented by the segmentation processing unit. The segmentation processing unit performs segmentation in a manner that minimizes at least two of the following losses: 1) mutual information loss, 2) shape loss, and 3) reconstruction loss.

Description

Defect detection device and defect detection method

This invention relates to a defect detection apparatus and a defect detection method for detecting defects in semiconductors and the like.

In semiconductor wafer manufacturing, it is crucial to quickly start the manufacturing process and rapidly transition to a high-yield mass production system to ensure profitability. To this end, various inspection, observation, and measurement devices are introduced into the production line. One of the primary inspection devices is the defect detection device, used to detect defects generated during the semiconductor manufacturing process.

One of the key performance indicators of a defect detection device is its ability to detect defects without ignoring them, while simultaneously suppressing false alarms (the result of misdetecting normal parts as defects). In semiconductor manufacturing, as the dimensions of circuit patterns formed on wafers become increasingly smaller, the size of defects that can fatally affect component operation also tends to shrink. Therefore, techniques for suppressing false alarms in defect detection are becoming increasingly important.

Patent document 1 discloses a defect detection method and a defect detection device, which has an optical system for acquiring an image of the entire wafer formed on the sample as the object of detection, and a function for displaying the image of the entire wafer and dividing it into multiple regions according to its culpability. The defect judgment threshold of each divided region can be changed, thereby making it difficult to detect defects erroneously.

Furthermore, Patent 2 discloses a defect detection method that divides the detection object into multiple regions. This method uses the following features as criteria for region division: 1) the grayscale values of the pixel and its surrounding pixels, and 2) the brightness gradient calculated using variance, entropy, and a Sobel filter. Additionally, as a method for grouping the divided regions, classification based on decision trees (using the aforementioned features), support vector machines (SVMs), nearest neighbor rules (NLMs), and commonly used pattern recognition and design data can be used to group patterns of the same shape into the same category. [Previous Artwork] [Patents]

[Patent Document 1] Japanese Patent Application Publication No. 2002-100660 [Patent Document 2] Japanese Patent Application Publication No. 2013-160629

[Problem Solved by the Invention] In Patent 1, the division of regions is performed by the device operator. Furthermore, Patent 2 lists previously known feature values and methods for region division, but does not teach which division method should be applied in each stage of semiconductor manufacturing. Since the appearance of the circuit pattern differs for each stage of semiconductor manufacturing, optimizing the segmentation process in each stage is crucial.

The purpose of this invention is to provide a defect detection device and method that, without requiring user instruction, can perform appropriate segmentation processing for each project, thereby easily adjusting the defect detection sensitivity in each segment. This balances defect detection sensitivity and false alarm suppression, while reducing the user's workload. [Solution]

To achieve the above objectives, the present invention is configured as follows: A defect detection apparatus and method, comprising: an image acquisition unit for acquiring an image of a predetermined area of a sample to be inspected; a segmentation processing unit for segmenting the captured image acquired by the image acquisition unit into multiple segments; and a defect candidate point extraction unit for extracting defect candidates from each segment segmented by the segmentation processing unit; the segmentation processing unit performs segmentation in a manner that minimizes at least two of the following 1) to 3): 1) calculating mutual information based on the segment identification results of two matched images. Information), setting the loss to maximize mutual information loss; 2) Shape loss is based on the correction error of superpixel segments, which are tiny regions obtained by grouping pixels with similar colors or textures in the input image; 3) Reconstruction loss is based on the reconstruction error of the grayscale image. [Invention Effects]

According to the present invention, a defect detection device and a defect detection method can be provided, which are configured to perform appropriate segmentation processing for each project without user instruction, thereby easily adjusting the defect detection sensitivity in each segment, balancing defect detection sensitivity and suppressing false alarms, and reducing the user's workload.

The issues, components, and effects other than those mentioned above will become clear through the following explanation of the implementation methods.

On a wafer prototype, regions can be divided according to each type of circuit pattern formed. For example, in a memory component, there are regions that form elements used for storing memory information (cell regions) and peripheral circuit regions that control these memory elements. The size of the formed circuit pattern and the magnitude of manufacturing variations often differ between cell regions and peripheral circuit regions, and the sensitivity required for defect detection also differs. Therefore, for example, adjusting the sensitivity to cell regions with fine pattern sizes increases the likelihood of erroneously detecting manufacturing variations in peripheral circuit regions. Conversely, adjusting the sensitivity in peripheral circuit regions carries the risk of overlooking critical defects in the cell regions.

To address this problem, the following embodiment of the invention is described with reference to the accompanying drawings. This embodiment can perform appropriate segmentation processing for each project without requiring user instruction. Furthermore, the following description is only an embodiment, and the invention is not limited to the following content. [Embodiment]

The defect detection device of the present invention will be described below. In this embodiment, the imaging device is described as an observation device equipped with a scanning electron microscope (SEM) that uses an electron beam. However, the imaging device of the present invention can also be a device other than SEM. Even imaging devices that utilize charged particle beams such as ions or imaging devices equipped with optical microscopes can achieve the effect of the present invention.

Figure 1 illustrates the configuration of the imaging device of the present invention. The device comprises: a SEM 101 for capturing images; a control unit 102 for overall control; a memory unit 103 for storing information on a disk or semiconductor memory; a calculation unit 104 for performing calculations according to a program; an external memory media input/output unit 105 for inputting/outputting information with an external memory media connected to the device; a user interface 106 for controlling information input/output with the user; and a network interface 107 for communicating with a defective image classification device via a network. Additionally, an input/output terminal 113, such as a keyboard, mouse, or display, is connected to the user interface 106.

SEM 101 includes: a movable stage 109 on which a sample wafer (also known as a semiconductor wafer) 108 is mounted; an electron source 110 for irradiating the sample wafer 108 with an electron beam 114; and a detector 111 for detecting secondary electrons and reflected electrons generated from the sample wafer 108. Additionally, SEM 101 also comprises: an electron microscope (not shown) for focusing the electron beam 114 onto the sample, and a deflector (112) for scanning the electron beam 114 onto the sample wafer 108.

The main process of defect detection in this embodiment will be explained with reference to Figure 2. Assume that the semiconductor wafer 108, which is the object of observation, is mounted on the movable stage 109, and the imaging conditions of the object wafer are preset.

First, the control unit 102 controls the SEM 101 to capture an image of the inspection area specified by the user (S201). Next, areas deemed defective are detected from the captured image as defect candidate points (S202). As a method for extracting defect candidate points, the images can be compared with images of areas with the same circuit pattern in the pre-captured design, and the areas showing differences can be considered defect candidates. Alternatively, an image equivalent to a good product image can be estimated from the captured images, and the estimated good product image can be compared for inspection.

The process of extracting defect candidate points is the so-called initial screening. The resulting defect candidate points may include "false alarms" that incorrectly detect normal areas as defects. Therefore, in the subsequent processing, which is a secondary screening, it is important to extract points that are determined to be truly defective from the defect candidate points. In this embodiment, the captured image is divided into segments (S203) independently (in parallel) with the detection of defect candidate points using pre-set segmentation parameters (M201). The result of this processing is the segment to which each pixel of the captured image belongs. Hereinafter, the image with the segment index as its grayscale value will be referred to as the segment image.

Next, for each defect candidate point obtained, the segment to which each defect candidate point belongs is determined by referring to the segment image (defect candidate point classification, S204). Since each defect candidate point has coordinate information, the segment index can be obtained by referring to the coordinates of the segment image.

Next, based on the pre-set sensitivity adjustment coefficient for each segment, the anomaly level of each defect candidate point belonging to each segment is corrected (S205). The anomaly level here is an index value representing the probability that each defect candidate point is defective, calculated by comparing the grayscale difference, area, and other features during detection. Finally, the corrected anomaly level is applied to the pre-set detection threshold to obtain the final output defect point (S206). The above describes the detection processing performed on the image captured in image acquisition processing S201. This processing can be repeated until the user-specified detection area is covered.

It can be configured to perform image acquisition processing using camera hardware (S201) and defect detection processing using software (S202~S206) in parallel, and memorize the images captured in sequence, read the captured images and apply them to defect detection processing.

Images are used to illustrate the aforementioned defect detection process (S202~S206). Figure 3 is a schematic representation of an example of an image, showing an example of an image of a unit area with circular patterns and an area adjacent to the surrounding circuit area. Examples of vertically patterned areas, horizontally patterned areas, and rectangular patterned areas existing as surrounding circuit areas are shown.

Figure 4 shows the processing result of the segmentation process (S203). It shows the result of segmenting into 8 segments, divided into substrate areas (floor areas) for forming various circuit patterns and circuit pattern forming areas within each floor area. Specifically, the floor area is segmented into unit areas (segment #1), vertical pattern areas (segment #3), rectangular pattern areas (segment #5), and horizontal pattern areas (segment #7). Additionally, it shows the result of segmenting the circuit pattern forming areas into circular patterns (segment #2), vertical patterns (segment #4), rectangular patterns (segment #6), and horizontal patterns (segment #8). However, the above is only an example of the segmentation result; the segmentation result does not necessarily have to be like this. For example, segmentation by each floor area might be sufficient.

Figure 5 shows an example of the result of detecting defect candidate points in the captured image through defect candidate point detection processing (S202), and the detected defect candidate points are indicated by cross marks.

Figure 6 shows the result of arranging the obtained defect candidate points according to their degree of abnormality for each segment. In the cell region (segment #1) in Figure 5, there are two defect candidate points, 601 and 604, indicated by a crosshair. For example, suppose 601 is a real defect. Without segmentation, it is impossible to use a detection threshold 602 to separate 604 from the real defect 601, and the false alarm 604 will also be judged as a defect. On the other hand, by using a detection threshold 603 set for each segment, the false alarm 604 can be separated from the real defect 601. Although Figure 6 illustrates the method of setting the detection threshold for each segment, the same effect can be achieved by correcting the coefficient by multiplying it by the degree of abnormality of the defect candidate points belonging to each segment. For example, multiplying the degree of abnormality of the defect candidate points belonging to segment 1 by 1/2 is equivalent to setting the detection threshold of segment 1 to 2.

As described above, by dividing the captured image into segments based on the appearance of circuit patterns, components, etc., and setting different detection sensitivities for each segment, the defect detection sensitivity of each segment can be easily adjusted. This allows for simultaneous improvement in defect detection sensitivity, suppression of false alarms, and reduction of user workload.

The image segmentation method of this embodiment is explained below. There are multiple semiconductor manufacturing processes that are the objects of inspection, and the appearance of the circuit patterns formed in each manufacturing process is different. Furthermore, depending on the SEM imaging conditions, even patterns with the same structure may show significant variations in the appearance of the captured images. Therefore, in order to perform segmentation processing suitable for each process, the processing parameters related to segmentation processing are automatically adjusted before inspection.

The automatically adjusted parameters are stored in memory unit M201 and used in the segmentation process (S203) during the detection process. The automatic parameter adjustment process is performed in the stage of creating the detection formula for the detection object process, and this process is performed by the operator.

The automatic adjustment of parameters related to segmentation processing is explained with reference to Figure 7. First, an image set for automatic adjustment is obtained (S701). This can be achieved by capturing images of the detection area in the object project. Alternatively, images appropriately sampled from captured images can be used.

Next, the parameters of the object to be adjusted are initialized (S702). Pre-set standard values can be used, or parameters automatically adjusted for similar projects can be used as initial values. Alternatively, a random number can be used to set the initial values. Then, the image set is segmented using the set parameters (S703), and the loss is calculated based on the segmentation results using the method described later (S704), and the parameters are updated (S705).

The above process is repeated, and a termination determination is made during the repetition (S706). If the desired conditions are met, the repetition ends, and the obtained parameters are stored in the memory unit M201 (S707). The termination condition for the repetition can be that a predetermined number of repetitions is reached, or that the repetition ends when the loss has been sufficiently reduced.

In the segmentation process (S703), various methods can be used to identify the segment of each pixel. Here, we will illustrate this using a method employing multiple convolutional processing and activation functions, i.e., a deep learning method. As an example of constructing a convolutional neural network, a three-layer neural network as shown in Figure 8 can be used. Here, Y is the captured image as input, F1(Y) and F2(Y) represent intermediate data, and F(Y) is the network output. The intermediate data and the network output are calculated using Equations 1 to 3. Here, "*" indicates a convolution operation. The final segment image L(Y) is calculated using Equation 4. Here, W1 is n1 filters of size c0×f1×f1, where c0 represents the number of input image channels and f1 represents the size of the spatial filter.

By convolving a filter of size c0×f1×f1 with the input image n1 times, an n1-dimensional feature map can be obtained. B1 is an n1-dimensional vector, which is the bias component corresponding to the n1 filters. Similarly, W2 is a filter of size n1×f2×f2, B2 is an n2-dimensional vector, W3 is a filter of size n2×f3×f3, and B3 is a c3-dimensional vector. Where c0 is the number of channels in the captured image, and c3 is a value determined by the number of segments to be segmented. Furthermore, f1, f2, n1, and n2 are hyperparameters determined by the user before learning the sequence; for example, they can be set to f1=9, f2=5, n1=128, and n2=64. The parameters adjusted through parameter update processing (S705) are W1, W2, W3, B1, B2, and B3.

In this adjustment process, backpropagation, a common error propagation technique in neural network learning, can be used. Alternatively, when calculating the estimated error, all acquired training images can be used, or a mini-batch method can be employed. In other words, several images can be randomly extracted from the training images and the parameters can be updated repeatedly. Furthermore, a patch image can be randomly cropped from a single image and used as the input image Y of the neural network. As a result, learning can proceed effectively.

Alternatively, other configurations can be used for the convolutional neural network shown above. For example, the number of layers can be varied, and networks with four or more layers can be used, or configurations with skip connections can be used (such as U-Net or ResNet (Residual Neural Network)). Furthermore, the Transformer model can also be used.

Referring to Figure 9, the loss calculation process (S704) in the automatic parameter adjustment related to segmentation processing in this embodiment will be explained. This process calculates three types of losses: "mutual information loss," "shape loss," and "reconstruction loss." Each loss is multiplied by a preset weight coefficient and summed to obtain the final loss.

"Mutual information loss" is calculated based on the segment identification results of the paired images, and a loss is set to maximize the mutual information loss. First, let's illustrate "mutual information loss" using image clustering as an example. Image segmentation can be viewed as clustering local image regions; therefore, if clustering is possible, it can be easily expanded. Given a probability distribution G, the information entropy H(G) is defined as Equation 5. As the distribution G approaches a uniform distribution, the value of the information entropy H increases; when the distribution G converges to a point, the information entropy H becomes 0.

Mutual information I is a measure of the interdependence of two probability distributions K and L, and is a value used to assess the consistency of the set of information entropy. If the two probability distributions K and L are independent, then the mutual information is zero. Mutual information I can be calculated using information entropy and Equation 6. Figure 10 illustrates the calculation method for "mutual information loss". In the figure, CNN represents a convolutional layer, FC represents a fully connected layer, and Softmax represents the activation function, corresponding to the aforementioned segmentation processing (S703). Assuming the input image is x, changes are added to the input image x without changing the classification category, thereby generating image x'. As image changes, such as "parallel movement", "zooming in/out", "rotation", and "adding noise" are applicable (in the example in the figure, the text image of the letter 'A' with changed font and position is represented as an image pair)).

Suppose that the outputs obtained after inputting images x and x' into the identifier are Φ(x) and Φ(x') respectively (Φ(x)∈[0,1]^C, where C is the number of segments to be segmented). Φ(x) is the discrete probability variable z∈1,... in image x, which can be interpreted as representing the probability distribution of C, and P(z=c|x)=Φc(x). Φ(x) and Φ(x') are calculated for the images in the batch, and the joint probability of the C×C matrix P is calculated as Φ(x) and Φ(x') (Formula (7). Where n represents the batch size). Next, by symmetricizing P, the matrix S=(P+P^T)/2 is obtained.

Learning is achieved by maximizing the mutual information between the marginal probabilities Sx and Sy of matrix S (Equation 8). By adding matrix S in the row and column directions, we can obtain the marginal probabilities Sx = S(z = x) and Sy = S(z = y).

Maximizing mutual information means maximizing the information entropy of Sx and Sy, while making the distributions of Sx and Sy closer. This means that when multiple images are given, they are equally classified into each class, and learning is performed to classify images x and x' into the same class.

Returning to Figure 9, the calculation of "mutual information loss" in this invention will be further explained. As mentioned above, when calculating mutual information, image pairs that can produce the same segment identification results (segment images) are required. Without user guidance, it is difficult to sample image pairs from the input image set. Therefore, a paired image x' is generated from the input image x (S801) and used to calculate the loss.

After generating the image pair, segment identification is performed on the input image x and its paired image x' (S802). The identification result L(x') of the paired image is inversely transformed (S803), and the loss is calculated based on the mutual information (S804). Specifically, the identification results L(x) and L(x') of the input image x and the image x' with added distortion to the input image x are treated as discrete probability variables, and the loss is calculated to maximize the mutual information.

The method for generating paired images can be a combination of processing such as magnification, rotation, inversion, distortion, and contrast modification on image x to generate image x'. When generating image pairs by magnification, rotation, or inversion, in the inverse transformation process (S803), in order to make the identification result L(x) of the input image correspond to the identification result L(x') of the paired image, L(x') is reduced, inversely rotated, or inversely inverted. Thus, scale invariance, rotational invariance, and inversion invariance can be obtained as identification properties.

"Shape loss" is a loss that focuses on the shape of segments. Superpixels are calculated for the input image (S805), and the calculated superpixels are used to correct the segment identification results F(x) or L(x) of the input image (S806). The shape error before and after correction is calculated as the loss (S807). Figure 11 shows an example of the segment correction result using superpixels. 1402 shows a visualization of the segment identification result of the input image 1401. When only the aforementioned "mutual information loss" is used for automatic adjustment of parameters related to segmentation processing, the outline of the circuit pattern may not match the outline of the segment. 1403 shows the result of calculating 6×6 superpixels based on the input image 1401. A superpixel is a tiny region formed by grouping pixels with similar colors or textures. Therefore, the boundaries of superpixels strongly correlate with the boundaries of circuit patterns. 1404 represents the result of calculating and visualizing the representative segment corresponding to the pixels belonging to that superpixel for each superpixel. In the shape loss calculation (S807), the cross-entropy and squared error of 1402 and 1404 are calculated as the shape error before and after correction.

By importing this loss, the direction of parameter adjustment can be determined so that the boundaries of the segments in the segment image are consistent with the boundaries of the circuit pattern.

The result is that segments #2, #4, #6, and #8 in Figure 4 correspond to the circuit pattern shape. Furthermore, superpixel calculation can be performed using image processing techniques or convolutional neural networks.

"Reconstruction loss" refers to the loss due to errors in recovering the input image from the segment image. If the segment identification result accurately reflects the structure of the input image, it is considered that the input image can be reconstructed from the segment image. Therefore, the segment identification result F(x) or L(x) of the input image is used as input to reconstruct the grayscale of the input image (S808), and the error between the reconstruction result and the input image is calculated as the loss (S809).

The absolute error or squared error of the grayscale can be used as the error. Furthermore, convolutional neural networks can be used to reconstruct the grayscale image from the segment identification results. In this case, the parameter adjustments related to the grayscale image reconstruction process can be performed simultaneously with the parameter adjustments for the segment identification process. The absolute error or squared error of the grayscale can be used as the loss associated with the grayscale image reconstruction process. By incorporating this loss, the direction of parameter adjustments can be determined, making the output of the segment identification process usable for reconstructing the grayscale image identification results.

The parameters of the segment identification process (S802) after loss adjustment as described above are used as parameters for the segment segmentation process and stored in the memory unit M201 (S707 in Figure 7).

While the above describes the method of using convolutional neural networks for segment recognition, rule-based processing can also be used. In this case, parameters such as the decision threshold become the objects of adjustment. The simplest method for optimizing rule-based processing is to use grid search. Alternatively, experimental design or Bayesian optimization methods can be used. In either case, adjustments can be made using "mutual information loss," "shape loss," and "reconstruction loss."

Furthermore, segment shaping processing can be applied to the segment identification results, which utilizes rule-based segment shaping. For example, as shown in Figure 12, after obtaining segments 1001 and 1002 through segment identification processing, a new segment 1003 can be generated according to a pre-set rule, such as shrinking segment 1002 by a size of (Δx, Δy).

As a GUI of this embodiment, Figure 13 shows a GUI for confirming and adjusting segmentation results. It includes: an interface 1101 that represents a list of captured image sets and allows the user to make selections; an interface 1102 that displays the image selected from the list; an interface 1103 that displays the segmentation results corresponding to the image; and an interface 1104 that adjusts the shaping rules of the segments.

Figure 14 illustrates the GUI for adjusting sensitivity. It includes: interface 1201 for displaying the distribution of anomaly severity in each segment; interface 1202 for displaying patch images (local images cropped out from the detection point) of detected defect candidates; and interface 1203 for adjusting the detection sensitivity of each segment. Here, the defect detection points selected in the anomaly severity distribution can be displayed in association with the patch images. For example, the patch images corresponding to the selected defect candidate points in the anomaly severity distribution are highlighted. This makes it easy to determine whether a defect should be detected or is a false alarm, and reduces the workload of sensitivity adjustment.

Furthermore, the patch image display interface also has the function of zooming in to display the patch image containing defect candidate points within a specified segment. The sensitivity adjustment interface 1203 has an interface that allows specifying a coefficient related to sensitivity adjustment for each segment. At this time, it has the function that the distribution display of the abnormality degree distribution display interface 1202 is updated in real time according to changes in the sensitivity adjustment coefficient.

By employing the methods described above, it is possible to perform appropriate segmentation processing for each project even without user instruction. This allows for easy adjustment of the defect detection sensitivity within each segment. Consequently, it enables simultaneous improvement in defect detection sensitivity, suppression of false alarms, and reduction of user workload.

101: Image Capture Device S201: Image Acquisition Processing S202: Defect Candidate Point Detection Processing S203: Segmentation Processing S204: Defect Candidate Point Classification Processing S205: Anomaly Level Correction Processing S206: Defect Point Extraction Processing M201: Parameter Memory Unit S703: Segmentation Processing S704: Loss Calculation Processing S705: Parameter Update Processing S706: Parameter Adjustment Completion Judgment Processing S707: Parameter Storage Processing S801: Image Pair Generation Processing S802: Segment Identification Processing S803: Inverse Transformation Processing S804: Mutual Information Loss Calculation Processing S805: Superpixel Calculation Processing S806: Segment Correction Processing S807: Shape Loss Calculation and Processing S808: Gray-Scale Image Reconstruction Processing S809: Reconstruction Loss Calculation and Processing 1101: Image List Display Interface 1102: Input Image Display Interface 1103: Segmentation Result Display Interface 1104: Segment Shaping Rule Input Interface 1201: Anomaly Level Display Interface 1202: Patch Image Interface 1203: Sensitivity Adjustment Interface

[Figure 1] is a structural diagram of the image capturing device. [Figure 2] is a diagram illustrating the processing flow of the defect detection process of the present invention. [Figure 3] is a diagram schematically showing an example of an captured image. [Figure 4] is a diagram showing an example of the processing result of segmenting the captured image. [Figure 5] is a diagram showing an example of the processing result of detecting candidate defects in the captured image. [Figure 6] is a diagram showing an example of displaying the degree of abnormality of candidate defects for each segment. [Figure 7] is a diagram illustrating the processing flow of automatic parameter adjustment processing related to the segmentation process of the present invention. [Figure 8] is a diagram showing an example of the structure of the convolutional neural network used to identify segments of each pixel in the segmentation process of the present invention. [Figure 9] is a diagram illustrating the calculation process for losses during automatic parameter adjustment in the segmentation processing of the present invention. [Figure 10] is a diagram explaining the calculation method for "mutual information loss". [Figure 11] is a diagram showing an example of the correction result for a segment using superpixels. [Figure 12] is an example of the result after applying rule-based formatting to the segmentation result obtained from the segmentation processing. [Figure 13] is a diagram showing the GUI used in the present invention for confirming and adjusting the segmentation result. [Figure 14] is a diagram showing the GUI for sensitivity adjustment in the present invention.

S701: Obtain image set

S702: Initialization parameters

S703: Perform segmentation

S704: Calculate Losses

S705: Update parameters

S706: Determining if a process has ended

S707: Save Parameters

M201: Memory Unit

Claims (14)

A defect detection apparatus, characterized by: comprising: an image acquisition unit for acquiring an image of a predetermined area of a sample to be inspected; a segmentation processing unit for segmenting the image acquired by the image acquisition unit into multiple segments; and a defect candidate point extraction unit for extracting defect candidates from each segment segmented by the segmentation processing unit; The aforementioned segmentation processing unit performs segmentation by minimizing at least two of the following losses (1) to (3): 1) calculating mutual information based on the segmentation identification results of the paired two images, and setting a loss such that the mutual information loss is maximized; 2) shape loss is based on the correction error of superpixel segments, which are small regions obtained by grouping pixels with similar colors or textures in the input image; 3) reconstruction loss is based on the grayscale image reconstruction error. As in the defect detection device of claim 1, the following processing is performed to address the mutual information loss based on the aforementioned mutual information: The recognition results of the input image x and the image x' to which the input image x has been distorted are treated as discrete probability variables, and a process is calculated to maximize the mutual information loss. As in the defect detection apparatus of claim 1, the loss due to correction error based on the aforementioned section is handled as follows: superpixels are calculated from the input image; the recognition result is shaped based on the calculated superpixels; and the error before and after shaping is treated as the loss. As in the defect detection apparatus of claim 1, the loss due to the aforementioned grayscale image reconstruction error is handled as follows: The input image is reconstructed from the identification results, and the error between the input image and the reconstructed image is considered as a loss. For any of the defect detection devices described in claims 1 to 4, the segmentation processing in the aforementioned segmentation processing unit and the defect candidate extraction processing in the aforementioned defect candidate point extraction unit are performed in parallel in two or more computers. The defect detection apparatus according to any one of claims 1 to 4, further comprises: a segment shaping processing unit, which applies rule-based shaping processing to the segmentation result obtained by the aforementioned segmentation processing unit to define new segments. The defect detection apparatus according to any one of claims 1 to 4, further comprises: a display unit that displays the distribution of the degree of abnormality in each segment for defect candidates extracted from the aforementioned defect candidate extraction unit. A defect detection method, characterized by: components including: an image acquisition step of obtaining an image of a predetermined area of a sample to be inspected; a segmentation processing step of dividing the captured image obtained in the aforementioned image acquisition step into multiple segments; and a defect candidate point extraction step of extracting defect candidates from each segment segmented in the aforementioned segmentation processing step; In the aforementioned segmentation processing steps, segmentation is performed by minimizing at least two of the following losses (1) to (3): 1) Calculating mutual information based on the segment identification results of the paired images, setting a loss to maximize the mutual information loss; 2) Shape loss is based on the correction error of superpixel segments, which are small regions obtained by grouping pixels with similar colors or textures in the input image; 3) Reconstruction loss is based on the grayscale image reconstruction error. As in the defect detection method of claim 8, the following processing is performed to address the mutual information loss based on the aforementioned mutual information: The identification results of the input image x and the image x' to which the input image x has been distorted are treated as discrete probability variables, and a process that maximizes the mutual information loss is calculated. As in the defect detection method of claim 8, the loss due to correction error based on the aforementioned section is handled as follows: superpixels are calculated from the input image; the recognition result is shaped based on the calculated superpixels; and the error before and after shaping is treated as the loss. As in the defect detection method of claim 8, the loss based on the aforementioned grayscale image reconstruction error is handled as follows: the input image is reconstructed from the identification results, and the error between the input image and the reconstructed image is treated as a loss. For any of the defect detection methods in claims 8 to 11, the segmentation process in the aforementioned segmentation step and the defect candidate extraction process in the aforementioned defect candidate point extraction step are performed in parallel. For example, the defect detection method described in any of claims 8 to 11, also includes segment shaping processing, which applies rule-based shaping to the segmentation results obtained in the aforementioned segmentation processing steps to define new segments. The defect detection method according to any one of claims 8 to 11, also includes: a display step that displays the distribution of the degree of abnormality in each segment for the defect candidates extracted from the aforementioned defect candidate point extraction step.