WO2025069808A1 - Computer program, information processing method, and information processing device - Google Patents

Abstract

The present invention provides a computer program, an information processing method, and an information processing device that can be expected to detect various patterns from a target image without preliminarily performing machine learning of a pattern to be detected.　A computer program according to the present embodiment causes a computer to execute processing for detecting a predetermined pattern formed on a substrate processed by a substrate processing apparatus from a target image capturing the substrate. The computer program causes the computer to execute processing involving: acquiring a template image showing a pattern to be detected; extracting a feature amount from each of the template image and the target image; and detecting the pattern to be detected from the target image on the basis of a result of comparison between the feature amount of the template image and the feature amount of the target image.

Description

COMPUTER PROGRAM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING APPARATUS

This disclosure relates to a computer program, an information processing method, and an information processing device.

Patent Document 1 proposes a pattern inspection system that inspects an image of a pattern to be inspected using a classifier configured by machine learning based on an image of a pattern to be inspected of an electronic device and data used to manufacture the pattern to be inspected. This pattern inspection system includes a memory unit that stores multiple pattern images of the electronic device and pattern data used to manufacture the pattern of the electronic device, and an image selection unit that selects a learning pattern image to be used for machine learning from the multiple pattern images based on the pattern data and pattern images stored in the memory unit.

JP 2020-35282 A

The present disclosure provides a computer program, an information processing method, and an information processing device that are expected to be able to detect various patterns from a target image without prior machine learning of the target patterns.

A computer program according to one embodiment is a computer program that causes a computer to execute a process of detecting a predetermined pattern formed on a substrate from a target image of the substrate that has been processed by a substrate processing apparatus, and causes the computer to execute a process of acquiring a template image onto which the pattern to be detected is captured, extracting features from the template image and the target image, and detecting the pattern to be detected from the target image based on a comparison result between the features of the template image and the features of the target image.

This disclosure is expected to enable detection of various patterns from target images without prior machine learning of the target patterns.

1 is a schematic diagram for explaining an overview of an information processing system according to an embodiment of the present invention; 1 is a block diagram showing an example of a configuration of an information processing device according to an embodiment of the present invention; FIG. 11 is a schematic diagram for explaining an example of a method for acquiring a template image. FIG. 11 is a schematic diagram for explaining an example of a pattern detection process. FIG. 11 is a schematic diagram for explaining an example of a pattern detection process. FIG. 11 is a schematic diagram for explaining an example of a pattern detection process. 10 is a flowchart showing an example of a procedure of a pattern detection process performed by the information processing device according to the first embodiment. 13 is a flowchart showing an example of a procedure of a pattern detection process performed by an information processing device according to a second embodiment. 13 is a flowchart showing an example of a procedure of a pattern detection process performed by an information processing device according to a third embodiment. 13 is a flowchart showing an example of a procedure of a pattern detection process performed by an information processing device according to a fourth embodiment.

Specific examples of information processing systems according to embodiments of the present disclosure are described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is set forth in the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[First embodiment]
<System Overview>
FIG. 1 is a schematic diagram for explaining an overview of an information processing system according to the present embodiment. The information processing system according to the present embodiment is configured to include an information processing device 1, a substrate processing device 101, a scanning electron microscope 102, and the like. The substrate processing device 10 is, for example, a device such as a process chamber that performs processing such as etching on a semiconductor wafer (substrate). The processed substrate that has been subjected to processing such as etching in the substrate processing device 101 is photographed by the scanning electron microscope 102 for surface shape and the like. The scanning electron microscope 102 is a device that irradiates an object with an electron beam and detects secondary electrons or transmitted electrons emitted from the object to observe the object, and outputs a so-called SEM (Scanning Electron Microscope) image of the object. The SEM image output by the scanning electron microscope 102 is provided to the information processing device 1. In the present embodiment, the SEM image photographed by the scanning electron microscope 102 is processed as a target, but is not limited thereto, and may be processed as a target image photographed by, for example, a transmission electron microscope or an optical microscope.

Structures of various shapes are formed on the surface of a substrate that has been processed by the substrate processing apparatus 101. For example, as shown in the SEM image in the center of FIG. 1, multiple structures of the same shape may be formed on the substrate. In the following, in this embodiment, each of these structures of the same shape will be referred to as a pattern. However, in reality, even if it is intended to form patterns of the same shape, the size or shape, etc. of each pattern will differ due to each process such as film formation, etching, or exposure. A pattern whose size or shape deviates beyond a predetermined threshold is determined to be abnormal. The information processing system of this embodiment is a system that detects multiple patterns formed on a target substrate based on an SEM image of the target substrate taken by a scanning electron microscope 102, and determines the state of each detected pattern.

In the present embodiment, the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 are described as separate devices, but this is not limited to the above. For example, the substrate processing apparatus 101 and the scanning electron microscope 102 may be one device, the scanning electron microscope 102 and the information processing apparatus 1 may be one device, or the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be one device. Furthermore, each of the substrate processing apparatus 101, the scanning electron microscope 102, and the information processing apparatus 1 may be configured by combining multiple devices.

The information processing device 1 acquires an SEM image of the target substrate captured by the scanning electron microscope 102. In this embodiment, the information processing device 1 also acquires an image in which the pattern to be detected is captured as a template image. The information processing device 1 detects the pattern captured in the template image from the SEM image of the target substrate. The lower part of Figure 1 shows a state in which the pattern detected from the SEM image is surrounded by a bounding box. The information processing device 1 outputs information such as the position and size of the bounding box relative to the SEM image for each detected pattern as the pattern detection result.

In this embodiment, the information processing device 1 extracts feature points from the template image and also from the target SEM image, and performs a matching process of the extracted feature points of both images to detect the pattern of the template image from the SEM image. By adopting a feature point matching process for pattern detection, the information processing system according to this embodiment does not require pre-processing, such as a learning model learning the shape of the pattern to be detected by machine learning in advance, and the user can arbitrarily specify the pattern to be detected. The user can specify the pattern to be detected by providing the information processing device 1 with a template image, such as an image with a pattern or an image with a pattern drawn on it. Details of the pattern detection process based on feature points will be described later.

The information processing device 1 also judges the state of each pattern detected from the SEM image. Even if multiple patterns formed on the substrate are intended to be formed in the same shape, differences in size or shape may occur due to each process of substrate processing. A pattern whose size or shape deviates beyond a predetermined threshold is considered to be an abnormal pattern, and a pattern whose size or shape is within the threshold range is considered to be a normal pattern. Therefore, for example, the information processing device 1 measures the length or width of each pattern detected from the SEM image based on a bounding box. The information processing device 1 can judge whether each pattern is normal or abnormal based on whether the length measured for each pattern is within an appropriate range. For example, the information processing device 1 also calculates the similarity between the pattern of the template image and each pattern detected from the SEM image. The information processing device 1 considers the pattern of the template image to be normal, judges the detected pattern to be normal if the similarity exceeds a threshold (i.e., the pattern of the template image and the detected pattern are similar), and judges the detected pattern to be abnormal if the similarity does not exceed the threshold. The method for determining the state of the pattern detected from the SEM image is not limited to the above method, and various methods may be adopted. Furthermore, the state of the pattern detected from the SEM image may be determined to be three or more types of states, rather than just two types, normal and abnormal.

<Device Configuration>
2 is a block diagram showing an example of the configuration of the information processing device 1 according to the present embodiment. The information processing device 1 according to the present embodiment can be realized by installing a predetermined application program or the like in a general-purpose information processing device such as a personal computer or a server computer. However, the information processing device 1 may be a dedicated information processing device that controls the substrate processing device 101 or the scanning electron microscope 102. The information processing device 1 according to the present embodiment is configured to include a processing unit 11, a storage unit 12, a communication unit 13, a display unit 14, an operation unit 15, and the like. Note that, in the present embodiment, the description will be given assuming that processing is performed by one information processing device 1, but the processing of the information processing device 1 may be distributed among a plurality of devices.

The processing unit 11 is configured using an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit) or a quantum processor, a ROM (Read Only Memory), and a RAM (Random Access Memory). The processing unit 11 reads out and executes a program 12a stored in the memory unit 12 to perform various processes such as obtaining a template image and an SEM image, detecting a pattern from the SEM image, and determining the state of the detected pattern.

The storage unit 12 is configured using a large-capacity storage device such as a hard disk or SSD (Solid State Drive). The storage unit 12 stores various programs executed by the processing unit 11 and various data required for the processing of the processing unit 11. In this embodiment, the storage unit 12 stores the program 12a executed by the processing unit 11. The storage unit 12 also includes a template image storage unit 12b that stores and accumulates a template image onto which a pattern of the target to be detected from the SEM image is copied.

In this embodiment, the program (computer program, program product) 12a is provided in a form recorded on a recording medium 99 such as a memory card or an optical disk, and the information processing device 1 reads the program 12a from the recording medium 99 and stores it in the memory unit 12. However, the program 12a may be written to the memory unit 12, for example, during the manufacturing stage of the information processing device 1. Also, for example, the program 12a may be distributed by a remote server device or the like and acquired by the information processing device 1 through communication. For example, the program 12a may be read from the recording medium 99 by a writing device and written to the memory unit 12 of the information processing device 1. The program 12a may be provided in a form distributed via a network, or may be provided in a form recorded on the recording medium 99.

The template image storage unit 12b stores and accumulates template images used for pattern detection. The information processing device 1 acquires a template image generated by, for example, a user by cutting out a portion of an SEM image, and stores it in the template image storage unit 12b. The template image storage unit 12b stores text information, such as title information of the template image set by the user or explanatory information regarding the pattern depicted in the template image, in association with information such as the date and time when the template image was acquired, together with the template image. The information processing device 1 can acquire a template image that is the pattern to be detected by, for example, displaying a list of multiple template images stored in the template image storage unit 12b as candidates and accepting the selection of one from the multiple candidates.

The communication unit 13 is connected to the scanning electron microscope 102 via a cable such as a communication line or a signal line, and transmits and receives data to and from the scanning electron microscope 102 via this cable. In this embodiment, the communication unit 13 receives data of an SEM image of a substrate transmitted from the scanning electron microscope 102, and provides the received data to the processing unit 11. Note that in this embodiment, the SEM image is transmitted from the scanning electron microscope 102 to the information processing device 1 via communication, but this is not limited to this, and the SEM image may be transmitted and received via a recording medium such as a memory card.

The display unit 14 is configured using a liquid crystal display or the like, and displays various images, characters, etc. based on the processing of the processing unit 11. In this embodiment, the display unit 14 displays, for example, the SEM image acquired from the scanning electron microscope 102, and the pattern detection results from the SEM image.

The operation unit 15 accepts user operations and notifies the processing unit 11 of the accepted operations. For example, the operation unit 15 accepts user operations through an input device such as a mechanical button or a touch panel provided on the surface of the display unit 14. For example, the operation unit 15 may also be an input device such as a mouse and a keyboard, and these input devices may be configured to be removable from the information processing device 1.

The storage unit 12 may be an external storage device connected to the information processing device 1. The information processing device 1 may be a multi-computer including multiple computers, or may be a virtual machine virtually constructed by software. The information processing device 1 is not limited to the above configuration, and may not include, for example, the display unit 14 and the operation unit 15.

In the information processing device 1 according to this embodiment, the processing unit 11 reads out and executes the program 12a stored in the memory unit 12, thereby realizing an image acquisition unit 11a, a feature point extraction unit 11b, a matching processing unit 11c, a pattern detection unit 11d, a state determination unit 11e, a display processing unit 11f, etc. as software functional units in the processing unit 11. Note that in this figure, functional units related to the process of detecting patterns from SEM images are shown as functional units of the processing unit 11, and functional units related to other processes are not shown.

The image acquisition unit 11a performs processing to acquire an SEM image (target image) taken by the scanning electron microscope 102 of a substrate that has been processed by the substrate processing apparatus 101, and a template image onto which the pattern to be detected is projected. The image acquisition unit 11a acquires the SEM image of the substrate taken by the scanning electron microscope 102 by communicating with the scanning electron microscope 102 via the communication unit 13.

The image acquisition unit 11a also displays an SEM image acquired from the scanning electron microscope 102 on the display unit 14, and receives a specification of a partial image area from the SEM image from the user, thereby acquiring this partial image area as a template image. The image acquisition unit 11a receives input of text information such as title information and explanatory information related to the acquired template image from the user, and stores the template image together with the received information in the template image storage unit 12b. The image acquisition unit 11a also displays a list of multiple template images stored in the template image storage unit 12b as candidates on the display unit 14, receives the selection of one of the displayed multiple candidates, and acquires the selected candidate as the template image. The user can appropriately select which method to use to specify the template image, for example on a menu screen, etc.

The feature point extraction unit 11b performs a process of extracting feature points for the SEM image and template image acquired by the image acquisition unit 11a. The feature point extraction process is a process of identifying characteristic points (pixels) such as edges or corners contained in an image, and can be performed using existing methods such as SIFT (Scale Invariant Feature Transformation) or AKAZE (Accelerated-KAZE). Each feature point extracted from an image by the feature point extraction unit 11b includes information on the position of the feature point in the image (e.g., x, y coordinates) and a numerical value (feature amount) indicating the characteristic of this point.

The matching processing unit 11c matches the feature points of the SEM image extracted by the feature point extraction unit 11b with the feature points of the template image. For example, the matching processing unit 11c determines which of the multiple feature points extracted from the SEM image each feature point extracted from the template image corresponds to by comparing the positions and feature amounts of the feature points. As a result of the matching processing, the matching processing unit 11c outputs information indicating which of the feature points of the SEM image each feature point of the template image corresponds to (matches).

The pattern detection unit 11d performs a process of detecting a pattern from the SEM image that is the same as the pattern transferred to the template image, based on the results of the matching process by the matching processing unit 11c. Note that a pattern that is "the same" as the pattern transferred to the template image does not mean that it is completely identical. Patterns that are the same in design but have different sizes or shapes due to each process of substrate processing are also included in the "same" pattern. First, the pattern detection unit 11d identifies an image area in the SEM image that is the same size as the template image and contains the most feature points that match the feature points of the template image. The pattern detection unit 11d extracts an image area from the SEM image that includes the image area identified in the SEM image and is larger than this image area (for example, an image area that is 1.2 times the size of the identified image area in both the vertical and horizontal directions).

Then, the pattern detection unit 11d causes the feature point extraction unit 11b to extract feature points and the matching processing unit 11c to match feature points between the template image and the image area extracted from the SEM image, and obtains the feature point matching results. Based on the results of the re-matching, the pattern detection unit 11d creates a bounding box that encloses the matched feature points in the extracted image area. The portion surrounded by this bounding box corresponds to the detected pattern, and the pattern detection unit 11d outputs information about the bounding box (for example, information such as the position of the bounding box in the original SEM image) as the result of pattern detection.

The pattern detection unit 11d can detect one pattern from the SEM image that is the same as the template image by the above process. The pattern detection unit 11d detects multiple patterns from the SEM image by repeatedly performing the same process on the same SEM image. At this time, the pattern detection unit 11d repeats the process by excluding the image area corresponding to the bounding box surrounding the pattern already detected in the SEM image from the target of the feature point extraction and feature point matching processes. In each repetition, the pattern detection unit 11d also calculates the similarity between the image area of the bounding box surrounding the detected pattern and the template image, and ends the repetition of pattern detection when the similarity falls below a predetermined threshold. Note that the calculation of the similarity may be performed by any method, and for example, if information such as the degree of matching is obtained in the feature point matching process, this may be used as the similarity.

The state determination unit 11e performs a process of determining the state of each pattern detected from the SEM image by the pattern detection unit 11d. For example, the state determination unit 11e measures the length and width of each pattern based on the information of the bounding box surrounding each pattern, and determines whether each pattern is normal (good) or abnormal (defective) depending on whether the measured length is within a predetermined range. For example, the state determination unit 11e can determine whether a pattern is normal or abnormal by regarding the pattern of the template image as a normal pattern or by accepting an input of an image of a normal pattern from a user, and determining whether the similarity between the normal pattern and the detected pattern exceeds a predetermined threshold. Note that the state determination of each pattern by the state determination unit 11e is not limited to the above, and any method and any state may be used to determine the pattern.

The display processing unit 11f performs processing to display information such as the results of pattern detection from the SEM image by the pattern detection unit 11d and/or the results of state determination of each pattern by the state determination unit 11e on the display unit 14. For example, the display processing unit 11f generates an image in which a bounding box surrounding each pattern detected by the pattern detection unit 11d is superimposed on the SEM image acquired from the scanning electron microscope 102, and displays the image on the display unit 14. In addition, for example, the display processing unit 11f notifies the user of an abnormal pattern by highlighting (for example, by changing the color or blinking) the bounding box corresponding to the pattern determined to be abnormal by the state determination unit 11e for each pattern superimposed on the SEM image.

<Pattern detection process>
3 is a schematic diagram for explaining an example of a method for acquiring a template image. The information processing device 1 according to the present embodiment displays an SEM image of a target substrate acquired from, for example, a scanning electron microscope 102 on the display unit 14, and accepts a user's designation of an area for the displayed SEM image. The information processing device 1 accepts a selection of a rectangular image area for the SEM image based on, for example, an operation on a mouse of the operation unit 15 (see the upper part of FIG. 3). The information processing device 1 acquires the rectangular image area selected by the user as a template image (see the lower left of FIG. 3). The information processing device 1 also treats an image obtained by removing the image area corresponding to the template image from the original SEM image as a target SEM image for detecting a pattern in subsequent processing (see the lower right of FIG. 3).

The information processing device 1 stores the template image selected based on the user's input in the template image storage unit 12b. Instead of acquiring a template image based on the user's input from an SEM image acquired from the scanning electron microscope 102, the information processing device 1 may, for example, display a list of the template images stored in the template image storage unit 12b, accept a selection from the user of one of the displayed template images, and use the selected template image for subsequent pattern detection processing. Note that the method by which the information processing device 1 acquires the template image may be appropriately selected by the user, for example, on a menu screen.

FIGS. 4 to 6 are schematic diagrams for explaining an example of pattern detection processing. The information processing device 1 acquires the SEM image and template image to be processed, and performs processing to extract feature points for each of the SEM image and template image. The upper part of FIG. 4 shows an image in which the extracted feature points are superimposed on the SEM image and template image, as an example of the feature point extraction result. Each feature point has coordinate information in the SEM image and template image, and feature amount information that indicates the characteristics of each point. The information processing device 1 can extract feature points from the SEM image and template image using an existing feature point extraction method such as SIFT or AKAZE.

Then, the information processing device 1 performs a matching process between the feature points of the SEM image and the feature points of the template image. In the matching process, the information processing device 1 determines which of the multiple feature points of the SEM image each feature point of the template image corresponds to, for example, based on the position and feature amount of each feature point. The lower part of Figure 4 shows an example of the result of the feature point matching process by displaying the SEM image and the template image side by side and connecting the feature points of the SEM image and the feature points of the template image with straight lines.

Based on the results of the matching process, the information processing device 1 identifies an image area from the SEM image that is the same size as the template image so as to include as many feature points of the SEM image as possible that match the feature points of the template image. The information processing device 1 then extracts an image area from the SEM image that is larger than this identified image area (for example, an image area that is expanded a predetermined amount in all directions from the identified image area). The upper part of Figure 5 shows an example of an image area extracted from the SEM image based on the results of the matching process, surrounded by a rectangular frame.

The information processing device 1 again performs the process of extracting feature points and the process of matching the extracted feature points on the image area extracted from the SEM image and the template image. The information processing device 1 further identifies an image area of the same size as the template image from this image area so as to include as many feature points as possible in the image area that match the feature points of the template image. The lower part of Figure 5 shows an example of an image area finally identified by the information processing device 1. The information processing device 1 regards the identified image area as an image area in which the same pattern as the template image is captured, and outputs the coordinates of the bounding box surrounding this image area as the pattern detection result.

After detecting one pattern from an SEM image, the information processing device 1 excludes the image area corresponding to the detected pattern from the original SEM image. The upper part of Figure 6 shows an example of an SEM image from which the detected area has been excluded. The information processing device 1 can detect multiple patterns from one SEM image by repeatedly performing the above-mentioned pattern detection procedure on the SEM image from which the image area of the detected pattern has been excluded. The lower part of Figure 6 shows an example of an SEM image from which pattern detection has been repeatedly performed and multiple image areas corresponding to the detected patterns have been excluded.

The information processing device 1 repeatedly detects patterns from the SEM image as described above, and each time a pattern is detected, calculates the similarity between the detected pattern and the pattern of the template image. When the similarity falls below a predetermined threshold, i.e., when the information processing device 1 determines that the detected pattern is not similar to the pattern of the template image, it ends the repeated pattern detection.

The information processing device 1 performs a state determination, such as the presence or absence of an abnormality, for each pattern detected from the SEM image. For example, the information processing device 1 can determine whether each pattern is normal or abnormal by measuring a distance such as the length or width for each detected pattern based on a bounding box and determining whether the measured distance is within a normal range. Furthermore, for example, the information processing device 1 can determine whether each pattern is normal or abnormal by calculating the similarity between the pattern of the template image and each detected pattern and determining whether the similarity exceeds a threshold value. The information processing device 1 may determine the state of a detected pattern using various other methods.

The information processing device 1 displays the pattern detection results from the SEM image and the state determination results for the detected patterns on the display unit 14. For example, the information processing device 1 displays an image in which a bounding box surrounding the detected pattern is superimposed on the target SEM image acquired from the scanning electron microscope 102, and by highlighting the bounding box corresponding to the pattern determined to be abnormal, the information processing device 1 notifies the user of the location of the patterns in the SEM image and the state of each pattern.

FIG. 7 is a flowchart showing an example of the procedure of the pattern detection process performed by the information processing device 1 according to the first embodiment. The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to the first embodiment communicates with the scanning electron microscope 102 via the communication unit 13, and acquires an SEM image of the target substrate photographed by the scanning electron microscope 102 (step S1).

The image acquisition unit 11a acquires a template image onto which the pattern to be detected is transferred from the SEM image by, for example, displaying the SEM image acquired in step S1 on the display unit 14 and accepting an image area selection operation for the SEM image on the operation unit 15 (step S2). Note that the image acquisition unit 11a may acquire a template image onto which the pattern to be searched is transferred by accepting, from the user, a selection of one template image from among multiple template images stored in the template image storage unit 12b. The image acquisition unit 11a stores the template image acquired in step S2 in the template image storage unit 12b (step S3).

The feature point extraction unit 11b of the processing unit 11 extracts feature points from the SEM image acquired in step S1 and the template image acquired in step S2 (step S4). The matching processing unit 11c of the processing unit 11 performs a feature point matching process based on the feature points extracted in step S4 to determine which feature points of the template image correspond to which feature points of the SEM image (step S5).

The pattern detection unit 11d of the processing unit 11 identifies an image area that is the same size as the template image and contains the most feature points that match the feature points of the template image based on the results of the matching process in step S5, and extracts an image area that is a constant multiple of the identified image area from the SEM image (step S6). The feature point extraction unit 11b re-extracts feature points from the image area extracted in step S6 (step S7). The matching processing unit 11c re-matches the feature points of the template image with the feature points of the image area re-extracted in step S7 (step S8). The pattern detection unit 11d creates a bounding box for the detected pattern based on the results of the re-matching in step S8, for example by creating a bounding box to surround the matched feature points (step S9).

The pattern detection unit 11d calculates the similarity between the pattern of the template image and the pattern surrounded by the bounding box created in step S9, and determines whether this similarity is equal to or less than a predetermined threshold (step S10). If the similarity exceeds the threshold (S10: NO), the pattern detection unit 11d deletes the area corresponding to the detected pattern from the SEM image to be processed (step S11), returns the process to step S4, and performs the processes of steps S4 to S10 on the SEM image from which the area of the detected pattern has been deleted.

If the similarity is equal to or less than the threshold (S10: YES), the state determination unit 11e of the processing unit 11 performs a state determination for each pattern detected from the SEM image, such as whether it is normal or abnormal (step S12). The display processing unit 11f of the processing unit 11 displays the detection result of the pattern indicated by the bounding box created in step S9 and the state determination result of each pattern performed in step S12 on the display unit 14 (step S13), and ends the process.

<Summary>
In the information processing system according to the present embodiment having the above configuration, a substrate that has been subjected to processing such as etching by the substrate processing apparatus 101 is photographed by the scanning electron microscope 102, and the information processing apparatus 1 performs processing for detecting a predetermined pattern formed on the substrate from the target image (SEM image) obtained by photographing. The information processing apparatus 1 acquires a template image on which the detection target pattern is projected, extracts feature amounts such as feature points from each of the target image and the template image, and detects the detection target pattern from the target image based on the comparison result between the feature amounts of the template image and the feature amounts of the target image. As a result, the information processing system according to the present embodiment is expected to detect various patterns from the target image without machine learning the detection target pattern in advance.

In addition, in the information processing system according to this embodiment, the information processing device 1 extracts feature points from each of the template image and the target image, matches the feature points of the template image with the feature points of the target image, and detects the target pattern from the target image based on the matching results. Existing methods such as SIFT can be used to extract feature points, and the information processing system according to this embodiment is expected to detect various patterns from the target image with higher accuracy without prior machine learning of the target pattern.

Furthermore, in the information processing system according to this embodiment, the information processing device 1 extracts an image area larger than the template image from the target image based on the result of matching feature points, re-extracts feature points from this image area, and matches the feature points of the template image with the feature points of the image area to detect the target pattern from this image area. As a result, the information processing system according to this embodiment is able to first extract an image area from the target image where the target pattern is likely to exist, and then detect the pattern from within this image area, which is expected to more accurately extract the target pattern from the target image.

In addition, in the information processing system according to this embodiment, the information processing device 1 extracts feature points from the target image excluding the detected pattern, matches the extracted feature points with feature points of the template image, extracts an image area based on the matching result, re-matches the feature points of the extracted image area with the feature points of the template image, and detects a pattern from the target image based on the re-matching result, thereby detecting multiple patterns from the target image. As a result, the information processing system according to this embodiment can detect multiple similar patterns when multiple similar patterns are placed in a single target image.

Furthermore, in the information processing system according to this embodiment, the information processing device 1 calculates the similarity between the pattern detected from the target image and the pattern of the template image, and ends the above-mentioned repeated process when the similarity falls below a threshold value. As a result, the information processing system according to this embodiment is expected to prevent a pattern with a low similarity from being detected as the target pattern, and to prevent erroneous detection of a pattern.

In addition, in the information processing system according to this embodiment, the information processing device 1 accepts from the user a selection operation of an image area in which the pattern to be detected is captured from the target image acquired from the scanning electron microscope 102, and sets the accepted image area as a template image. As a result, the information processing system according to this embodiment is expected to acquire, for example, one pattern from a target image in which multiple similar patterns are arranged, based on a user selection, and use it as a template image to detect a similar pattern from the target image.

Furthermore, in the information processing system according to this embodiment, the information processing device 1 stores a template image in the template image storage unit 12b, accepts a selection from multiple template images stored in the template image storage unit 12b, and detects a pattern from a target image based on the selected template image. As a result, in the information processing system according to this embodiment, when detecting a pattern similar to a previously detected pattern from another target image, the user can reuse the template image stored in the storage unit, which is expected to improve convenience.

Furthermore, in the information processing system according to this embodiment, the information processing device 1 performs a state determination for each pattern detected from the target image, for example, determining whether it is normal or abnormal. As a result, the information processing system according to this embodiment can be expected to provide the user with information regarding the state of each detected pattern, in addition to the detection results of the patterns contained in the target image.

In this embodiment, for example, in Figures 3 to 6, a schematic SEM image of one or more holes formed by etching a substrate in the vertical direction is shown, and the process performed by the information processing device 1 using the holes as the pattern to be detected is described, but the pattern to be detected is not limited to such holes. The pattern may be, for example, functional elements such as transistors or capacitors formed in multiple on the substrate, or various formations such as thin films. The same applies to the following embodiments.

[Embodiment 2]
In the second embodiment, a modified example of a repeating process for detecting a first pattern from an SEM image based on a template image and then detecting a second and subsequent patterns will be described. The information processing device 1 according to the second embodiment acquires an SEM image acquired from a scanning electron microscope 102 and a template image on which a detection target pattern is projected, and performs a process for detecting a first pattern from the SEM image based on a matching result of feature points of both images in the same procedure as the process by the information processing device 1 according to the first embodiment described above.

After detecting the first pattern from the SEM image, the information processing device 1 according to the second embodiment replaces the old template image with an image of the detected pattern (i.e., an image within the bounding box obtained as a result of the detection) as a new template image. The information processing device 1 according to the second embodiment detects the second and subsequent patterns from the SEM image by performing so-called template matching processing using this new template image.

Template matching is a technique in which an image area of the same size as a template image is extracted from a target image (SEM image), and if the similarity between the extracted image and the template image exceeds a threshold, it is determined that the pattern to be detected is captured in this image area. By moving the extraction position of the image area from the target image and repeating the same similarity judgment, and extracting image areas and judging the similarity at all positions in the target image, it is possible to detect one or more patterns from the target image.

FIG. 8 is a flowchart showing an example of the procedure of the pattern detection process performed by the information processing device 1 according to the second embodiment. The processing unit 11 of the information processing device 1 according to the second embodiment performs the following steps, similar to the procedure of the flowchart shown in FIG. 7: acquisition of an SEM image of the target substrate (step S1), acquisition of a template image (step S2), storage of the template image (step S3), extraction of feature points (step S4), feature point matching (step S5), extraction of an image area from the SEM image (step S6), re-extraction of feature points (step S7), re-matching of feature points (step S8), and creation of a bounding box for the detected pattern (step S9). Note that the process of steps S1 to S9 performed by the information processing device 1 according to the second embodiment is the same as the process of steps S1 to S9 shown in FIG. 7, and therefore some of the process of steps S1 to S9 are omitted in FIG. 8.

The pattern detection unit 11d of the processing unit 11 deletes the area corresponding to the pattern detected in steps S1 to S9 from the SEM image acquired in step S1 (step S31). The pattern detection unit 11d also replaces the image corresponding to the pattern detected in steps S1 to S9 with a new template image (step S32).

Next, the pattern detection unit 11d extracts an image area of the same size as the template image from the SEM image from which the area corresponding to the pattern detected in step S31 has been deleted (step S33). The pattern detection unit 11d calculates the similarity between the image area extracted in step S33 and the template image (step S34). The pattern detection unit 11d determines whether the similarity calculated in step S34 exceeds a predetermined threshold (step S35). If the similarity exceeds the threshold (S35: YES), the pattern detection unit 11d determines that the pattern to be detected is reflected in this image area, stores the position of the image area of this pattern in the SEM image (step S36), and proceeds to step S37. If the similarity does not exceed the threshold (S35: NO), the pattern detection unit 11d determines that the pattern to be detected is not reflected in this image area, and proceeds to step S37. The pattern detection unit 11d determines whether the process of extracting an image region and determining the similarity with the template image has been completed for all positions in the SEM image (step S37). If this process has not been completed for all positions (S37: NO), the pattern detection unit 11d returns to step S32, moves the extraction position, and repeats the processes of steps S32 to S37.

When these processes have been completed for all positions (S37: YES), the state determination unit 11e of the processing unit 11 performs a state determination for each pattern detected from the SEM image, such as whether it is normal or abnormal, based on the positions stored in step S36 (step S38). The display processing unit 11f of the processing unit 11 displays on the display unit 14 a bounding box surrounding each pattern detected from the SEM image and the results of the state determination for each pattern performed in step S38 (step S39), and ends the process.

In the information processing system according to the second embodiment configured as described above, the information processing device 1 updates the template image based on a pattern detected from the SEM image, extracts multiple image areas of the same size as the template image from the SEM image, calculates the similarity between the updated template image and each extracted image area, and detects the image area in which the target pattern is captured based on the calculated similarity. As a result, the information processing system according to this embodiment is expected to accurately detect these patterns from a target image in which, for example, multiple similar patterns are arranged.

Note that other configurations of the information processing system according to the second embodiment are similar to those of the information processing system according to the first embodiment, so similar parts are given the same reference numerals and detailed descriptions are omitted.

[Embodiment 3]
In the information processing system according to the third embodiment, the information processing device 1 presents a plurality of template image candidates to the user from the SEM image acquired from the scanning electron microscope 102 and acquires a template image by accepting a selection from the candidates. The information processing device 1 according to the third embodiment extracts feature points from the SEM image acquired from the scanning electron microscope 102 and extracts appropriate image regions from the SEM image based on the extracted feature points, thereby creating a plurality of template image candidates.

When multiple patterns of the same shape are repeatedly formed on a substrate, the multiple feature points that can be extracted from the SEM image may include a portion where similar feature point arrangements are repeated. The information processing device 1 according to the third embodiment searches for such repeated arrangements of similar feature points from among all the multiple feature points extracted from the SEM image. The information processing device 1 can select one of the repeated feature point arrangements based on the search results, extract a rectangular image area that includes the multiple feature points included in this arrangement, and use it as a candidate for the template image. The information processing device 1 may extract multiple candidates from one SEM image. Note that the method of extracting candidates by the information processing device 1 is not limited to the above method, and any method may be adopted.

The information processing device 1 displays, for example, on the display unit 14, a list of multiple candidates for template images that have been extracted and created from an SEM image. The information processing device 1 accepts a user operation at the operation unit 15 to select one of the multiple candidates displayed in the list, obtains the selected candidate as a template image, and performs processing to detect the pattern captured in this template image from the SEM image.

FIG. 9 is a flowchart showing an example of the procedure of the pattern detection process performed by the information processing device 1 according to the third embodiment. The procedure of the pattern detection process performed by the information processing device 1 according to the third embodiment is the same as that shown in FIG. 7 in which step S2 is replaced with step S51 to S52 in which the information processing device 1 presents candidates for the template image to the user from the SEM image and acquires the template image, instead of step S2 in which a template image is acquired based on a user's input. Since step S3 and onward are the same as those shown in FIG. 7 in the pattern detection process performed by the information processing device 1 according to the third embodiment, some of the processes in steps S3 to S13 are not shown in FIG. 9.

The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to the third embodiment acquires an SEM image of the target substrate photographed by the scanning electron microscope 102 (step S1). The feature point extraction unit 11b of the processing unit 11 extracts feature points from the SEM image acquired in step S1 (step S51). The image acquisition unit 11a performs processing such as searching for repetition of a specific arrangement of feature points based on the feature points of the SEM image extracted in step S2, and extracts an appropriate image area from the SEM image to create a template image candidate (step S52). The display processing unit 11f of the processing unit 11 displays one or more template image candidates extracted from the SEM image and created in step S52 on the display unit 14 (step S53). The image acquisition unit 11a accepts the selection of a template image from the user by receiving the selection of one of the candidates displayed in step S53 through the operation unit 15 (step S54).

The image acquisition unit 11a stores the template image acquired in step S54 in the template image storage unit 12b (step S3). The process from step S3 onwards is the same as that shown in the flowchart in FIG. 7, so a detailed description will be omitted.

In the information processing system according to the third embodiment configured as described above, the information processing device 1 extracts feature points from a target SEM image and creates a template image candidate from the SEM image based on the extracted feature points. The information processing device 1 displays the created candidates on the display unit 14 and accepts a selection operation from a plurality of candidates from the user, and performs pattern detection from the SEM image using the selected candidate as a template image. As a result, the information processing system according to this embodiment can assist the user in creating a template image, and is expected to improve user convenience.

Note that other configurations of the information processing system according to embodiment 3 are similar to those of the information processing systems according to embodiments 1 and 2, so similar parts are given the same reference numerals and detailed descriptions are omitted.

[Fourth embodiment]
In the information processing system according to the fourth embodiment, the information processing device 1 accepts a natural language input by a user and selects a template image from among a plurality of candidates based on the input natural language. The natural language for which input is accepted is, for example, a character string such as a word or a sentence in a language such as Japanese or English. In order to handle the natural language, the information processing device 1 according to the fourth embodiment uses an LLM (Large Language Model) that has been machine-learned in advance to interpret the natural language. The LLM may operate in the information processing device 1, or may operate in a device different from the information processing device 1, such as a remote server device.

In the information processing system according to the fourth embodiment, a user can input natural language such as "hole" or "mask" to the information processing device 1 to display candidate images depicting "hole" or "mask", and can determine the template image by selecting one of these candidates. The user can also cause the information processing device 1 to display candidate template images with any modifier added, such as "long and narrow hole" or "large mask".

The information processing device 1 according to the fourth embodiment extracts template image candidates from the SEM image acquired from the scanning electron microscope 102 by the method described in the third embodiment, selects a candidate corresponding to the input natural language from among the multiple candidates, and presents the selected candidates to the user in a list display. The information processing device 1 may also select a template image corresponding to the input natural language from among the template images stored in the template image storage unit 12b, and present the selected template images as candidates in a list display to the user.

The information processing device 1 converts, for example, a natural language input by a user into a feature vector using the LLM, and converts multiple images that are candidates for template images into feature vectors using a learning model (encoder) that has been machine-learned in advance. The information processing device 1 calculates the similarity between the feature vector of the input natural language and the feature vector of each candidate, and selects a predetermined number of candidates in descending order of similarity and presents them to the user, thereby being able to present candidates corresponding to the natural language input by the user to the user. The learning model that converts an image into a feature vector can be generated in advance by machine learning using learning data that associates, for example, a template image collected in advance with a natural language that expresses a pattern of this template image (or a feature vector converted from this natural language by the LLM). Note that the method of presenting candidates corresponding to natural languages to the user using the above-mentioned LLM is merely an example and is not limited to this, and any other method may be adopted.

FIG. 10 is a flowchart showing an example of the procedure of the pattern detection process performed by the information processing device 1 according to embodiment 4. The procedure of the pattern detection process performed by the information processing device 1 according to embodiment 4 is similar to the procedure of the pattern detection process performed by the information processing device 1 according to embodiment 1 shown in FIG. 7 except that step S2 of acquiring a template image based on a user's input is replaced with steps S71 to S77 of presenting template image candidates corresponding to natural language input from the user and acquiring a template image. Since steps S3 and onward of the pattern detection process performed by the information processing device 1 according to embodiment 4 are the same as those shown in FIG. 7, some of the processes of steps S3 to S13 are not shown in FIG. 10.

The image acquisition unit 11a of the processing unit 11 of the information processing device 1 according to the fourth embodiment acquires an SEM image of the target substrate photographed by the scanning electron microscope 102 (step S1). The feature point extraction unit 11b of the processing unit 11 extracts feature points from the SEM image acquired in step S1 (step S71). The image acquisition unit 11a performs processing such as searching for repetitions of a specific arrangement of feature points based on the feature points of the SEM image extracted in step S71, to create candidates for a template image (step S72).

The image acquisition unit 11a accepts the input of a natural language from the user using the operation unit 15, such as a keyboard (step S73). The image acquisition unit 11a inputs the accepted natural language into the LLM, thereby interpreting the natural language using the LLM (step S74). The image acquisition unit 11a selects one or more candidates corresponding to the input of the natural language from among the one or more candidates created in step S72 based on the result of the natural language interpretation by the LLM performed in step S74 (step S75). The display processing unit 11f of the processing unit 11 displays the one or more candidates selected in step S75 on the display unit 14 (step S76). The image acquisition unit 11a accepts the selection of a template image from the user by accepting the selection of one of the candidates displayed in step S76 through the operation unit 15 (step S77).

The image acquisition unit 11a stores the template image acquired in step S77 in the template image storage unit 12b (step S3). The process from step S3 onwards is the same as that shown in the flowchart in FIG. 7, so a detailed description will be omitted.

In the information processing system according to the fourth embodiment configured as described above, the information processing device 1 acquires natural language information on the pattern to be detected based on input from the user, and creates template image candidates from the SEM image according to the acquired natural language information. The information processing device 1 displays the created candidates on the display unit 14, accepts a selection operation from the user from among multiple candidates, and performs pattern detection from the SEM image using the candidate whose selection has been accepted as a template image. As a result, the information processing system according to the fourth embodiment is expected to further improve user convenience in creating template images.

In the fourth embodiment, candidates are selected according to natural language by using LLM, but the method of selecting candidates according to natural language is not limited to using LLM. For example, natural language such as names or words indicating the characteristics of the object depicted in the image can be added as tags to images that can be candidates, and corresponding tags can be searched for based on the natural language input by the user, and images with tags that match the search can be used as candidates for template images.

Note that other configurations of the information processing system according to embodiment 4 are similar to those of the information processing systems according to embodiments 1 to 3, so similar parts are given the same reference numerals and detailed descriptions are omitted.

The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present disclosure is indicated by the claims, not by the meaning described above, and is intended to include all modifications within the meaning and scope of the claims.

　The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations, regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multi-claim format), but this is not limited to this. They may also be written in a format in which a multiple claim cites at least one other multiple claim (multi-multi-claim).

1. Information processing device (computer)
REFERENCE SIGNS LIST 11 Processing section 11a Image acquisition section 11b Feature point extraction section 11c Matching processing section 11d Pattern detection section 11e State determination section 11f Display processing section 12 Storage section 12a Program 12b Template image storage section 13 Communication section 14 Display section 15 Operation section 99 Recording medium 101 Substrate processing apparatus 102 Scanning electron microscope

Claims (13)

A computer program for causing a computer to execute a process of detecting a predetermined pattern formed on a substrate from a target image obtained by capturing the substrate that has been processed by a substrate processing apparatus, the computer program comprising:
The computer includes:
A template image showing the pattern to be detected is obtained.
Extracting features from the template image and the target image, respectively;
a process of detecting a detection target pattern from the target image based on a comparison result between a feature amount of the template image and a feature amount of the target image. Extracting feature points from the template image and the target image, respectively;
Matching feature points of the template image with feature points of the target image;
The computer program according to claim 1 , further comprising: detecting a detection target pattern from the target image based on a result of the matching. extracting an image area from the target image that is larger than the template image based on the matching result;
Matching the feature points of the template image with the feature points of the image region;
detecting a detection target pattern from the image region based on a result of the matching;
3. A computer program according to claim 2. extracting feature points of the target image excluding the detected pattern, matching the extracted feature points with feature points of the template image, extracting an image region based on the matching result, matching the feature points of the template image with feature points of the image region, and detecting a pattern based on the matching result, thereby detecting a plurality of target patterns from the target image;
4. A computer program according to claim 3. Calculating a similarity between the pattern detected from the target image and the pattern captured in the template image;
When the similarity becomes equal to or less than a threshold, the repetition of detection is terminated.
5. A computer program according to claim 4. updating the template image based on the detected pattern;
Extracting a plurality of image regions from the target image;
detecting an image area, into which a detection target pattern is captured, from the plurality of image areas based on a similarity between the template image and each image area;
2. The computer program product of claim 1. Accepting a selection operation of an image area in which a detection target pattern is captured from the target image;
The received image region is set as the template image.
2. The computer program product of claim 1. Extracting features from the target image;
Extracting a plurality of template image candidates from the target image based on the extracted feature amount;
Accepting a selection operation from the extracted multiple candidates;
The accepted candidate is set as the template image.
2. The computer program product of claim 1. acquiring natural language information regarding the detection target;
extracting template image candidates from the target image according to the natural language information;
Accepting a selection operation from the extracted multiple candidates;
The accepted candidate is set as the template image.
2. The computer program product of claim 1. storing the template image in a storage unit;
accepting a selection operation from among the plurality of template images stored in the storage unit;
detecting a detection target pattern from the target image based on the received template image;
A computer program according to any one of claims 7 to 9. determining a state of a pattern detected from the target image;
2. The computer program product of claim 1. 1. An information processing method for detecting a predetermined pattern formed on a substrate from a target image captured by an information processing apparatus of the substrate that has been processed by the substrate processing apparatus, comprising:
The information processing device,
A template image showing the pattern to be detected is obtained.
Extracting features from the template image and the target image, respectively;
detecting a detection target pattern from the target image based on a comparison result between the feature amount of the template image and the feature amount of the target image;
Information processing methods. 1. An information processing apparatus comprising: a processing unit that performs processing to detect a predetermined pattern formed on a substrate from a target image obtained by capturing the substrate that has been processed by a substrate processing apparatus,
The processing unit,
A template image showing the pattern to be detected is obtained.
Extracting features from the template image and the target image, respectively;
detecting a detection target pattern from the target image based on a comparison result between the feature amount of the template image and the feature amount of the target image;
Information processing device.

Images