TW202530900A - Method for determining root causes of events of a semiconductor manufacturing process and for monitoring a semiconductor manufacturing process - Google Patents

Abstract

Described is a method for assessing a plurality of candidate actions for obtaining evidence data and relating to an assessment action of at least one manufacturing apparatus or system, the method comprising: obtaining at least one probabilistic model which relates said evidence data to an estimated probability of one or more root cause assessments of the manufacturing apparatus; determining, using the at least one probabilistic model, an estimated probability of one or more root cause assessments based on evidence data comprising additional evidence from one or more candidate actions which have not been performed; determining a reward based on the respective estimated probability of the one or more root cause assessments and an associated respective cost of said one or more candidate actions; and deciding on whether to perform any of said one or more candidate actions based on said reward.

Description

Method for determining the root cause of an event in a semiconductor process and for monitoring a semiconductor process

The present invention relates to a semiconductor manufacturing process, and more particularly to a method for determining the root cause of a yield-influencing issue on a substrate undergoing the process.

A lithography system is a machine designed to apply a desired pattern to a substrate. Lithography systems are used, for example, in the manufacture of integrated circuits (ICs). They project a pattern (often referred to as a "design layout" or "design") from a patterned device (e.g., a mask) onto a layer of radiation-sensitive material (resist) disposed on a substrate (e.g., a wafer).

To project patterns onto substrates, lithography equipment uses electromagnetic radiation. The wavelength of this radiation determines the minimum size of features that can be formed on the substrate. Typical wavelengths currently in use are 365 nm (i-line), 248 nm, 193 nm, and 13.5 nm. Compared to lithography equipment using radiation with a wavelength of, for example, 193 nm, lithography equipment using extreme ultraviolet (EUV) radiation with a wavelength in the 4 nm to 20 nm range (e.g., 6.7 nm or 13.5 nm) can be used to form smaller features on substrates.

Low- _k₁ lithography can be used to process features smaller than the classic resolution limit of the lithography tool. In this process, the resolution formula can be expressed as CD = _k₁ × λ/NA, where λ is the wavelength of the radiation used, NA is the numerical aperture of the projection optics in the lithography tool, CD is the "critical dimension" (usually the smallest feature size printed, but in this case, half the pitch), and _k₁ is an empirical resolution factor. Generally speaking, the smaller _k₁ , the more difficult it is to reproduce a pattern on the substrate that resembles the shape and dimensions planned by the circuit designer to achieve specific electrical functionality and performance. To overcome these difficulties, complex fine-tuning steps can be applied to the lithography projection tool and/or the design layout. These steps include, for example, but are not limited to, optimization of the NA, customized illumination schemes, use of phase-shift patterning devices, various optimizations of the design layout (such as optical proximity correction (OPC, sometimes also called "optical and process correction") in the design layout), or other methods generally defined as "resolution enhancement technology" (RET). Alternatively, a strict control loop for controlling the stability of the lithography equipment can be used to improve the reproduction of the pattern at low _k1 .

These stringent control loops are typically based on metrology data obtained by using metrology tools to measure the characteristics of the applied pattern or a metrology target representing the applied pattern. Typically, metrology tools are based on optical measurements of the position and/or size of the pattern and/or target. These optical measurements are essentially assumed to represent the quality of the integrated circuit process.

Lithography tools and/or related tools (deposition tools, exposure tools, metrology tools), collectively referred to herein as integrated circuit (IC) fabrication tools (e.g., any tool used in the IC fabrication process), or more generally as fabrication tools, can be extremely complex. Consequently, any nonideal performance (referred to herein as divergent behavior) can have many different possible root causes, with the specific root cause or causes being difficult to identify.

It is desirable to improve methods for identifying one or more root causes of divergent behavior in a process, such as an IC process.

The inventors' goal is to solve the above-mentioned shortcomings of the current state of the art.

In a first aspect of the present invention, a method for evaluating a plurality of candidate actions for obtaining evidence data and associated with an evaluation action for at least one manufacturing equipment or system is provided, the method comprising: obtaining at least one probability model that associates the evidence data with an estimated probability of one or more root cause evaluations of the manufacturing equipment; using the at least one probability model to evaluate a plurality of candidate actions for obtaining evidence data and associated with an evaluation action for at least one manufacturing equipment or system; At least one probability model determines an estimated probability of one or more root cause assessments based on evidentiary data including additional evidence from one or more candidate actions that have not yet been performed; determines a reward based on the respective estimated probability of the one or more root cause assessments and a respective cost associated with the one or more candidate actions; and determines whether to perform any of the one or more candidate actions based on the reward.

In this invention document, the terms "radiation" and "beam" are used to cover all types of electromagnetic radiation, including ultraviolet radiation (e.g., having a wavelength of 365, 248, 193, 157 or 126 nm) and EUV (extreme ultraviolet radiation, e.g., having a wavelength in the range of about 5 to 100 nm).

As used herein, the terms "reduction mask," "mask," or "patterning device" should be broadly interpreted to refer to a general patterning device that can be used to impart a patterned cross-section to an incident radiation beam, corresponding to the pattern to be produced in a target portion of a substrate; the term "light valve" may also be used in this context. In addition to classical masks (transmissive or reflective; binary, phase-shifting, hybrid, etc.), other examples of such patterning devices include: Programmable mirror arrays. Further information on such mirror arrays is provided in U.S. Patents Nos. 5,296,891 and 5,523,193, which are incorporated herein by reference. Programmable LCD arrays. Examples of such structures are given in U.S. Patent No. 5,229,872, which is incorporated herein by reference.

FIG1 schematically depicts a lithography apparatus LA. The lithography apparatus LA includes an illumination system (also referred to as an illuminator) IL configured to condition a radiation beam B (e.g., UV radiation, DUV radiation, or EUV radiation); a support structure (e.g., a mask stage) MT configured to support a patterning device (e.g., a mask) MA and connected to a first positioner PM configured to accurately position the patterning device MA according to certain parameters; a substrate stage (e.g., a wafer stage) WT configured to hold a substrate (e.g., a resist-coated wafer—wafer and substrate are synonymous and will be used interchangeably throughout this disclosure) W and connected to a second positioner PW configured to accurately position the substrate according to certain parameters; and a projection system (e.g., a refractive projection lens system). PS is configured to project the pattern imparted to the radiation beam B by the patterning device MA onto a target portion C of the substrate W (e.g., comprising one or more dies).

In operation, the illuminator IL receives a radiation beam from a radiation source SO, for example, via a beam delivery system BD. The illumination system IL may include various types of optical components for directing, shaping, and/or controlling the radiation, such as refractive, reflective, magnetic, electromagnetic, electrostatic, and/or other types of optical components, or any combination thereof. The illuminator IL may be used to condition the radiation beam B so as to have a desired spatial and angular intensity distribution in its cross-section at the plane of the patterning device MA.

The term "projection system" PS as used herein should be interpreted broadly to cover various types of projection systems, including refractive, reflective, catadioptric, synthetic, magnetic, electromagnetic, and electrostatic optical systems, or any combination thereof, as appropriate to the exposure radiation used or to other factors such as the use of an immersion liquid or the use of a vacuum. Any use of the term "projection lens" herein should be considered synonymous with the more general term "projection system" PS.

The lithography apparatus may be of a type in which at least a portion of the substrate is covered by a liquid having a relatively high refractive index (e.g., water) to fill the space between the projection system and the substrate, also known as immersion lithography. Further information on immersion technology is provided in U.S. Patent No. 6,952,253, which is incorporated herein by reference.

The lithography apparatus LA may also be of a type having two (dual-stage) or more substrate tables WT and, for example, two or more support structures MT (not shown). In such "multi-stage" machines, the additional tables/structures may be used in parallel, or preparatory steps may be performed on one or more tables while one or more other tables are being used to expose the design layout of the patterned device MA onto a substrate W.

In operation, a radiation beam B is incident on a patterning device (e.g., a mask MA) held on a support structure (e.g., a mask table MT) and is patterned by the patterning device MA. Having traversed the mask MA, the radiation beam B passes through a projection system PS, which focuses the beam onto a target portion C on a substrate W. With the aid of a second positioner PW and a position sensor IF (e.g., an interferometric device, a linear encoder, a 2-D encoder, or a capacitive sensor), the substrate table WT can be accurately moved, for example, to locate different target portions C in the path of the radiation beam B. Similarly, a first positioner PM and possibly another position sensor (not explicitly depicted in FIG. 1 ) can be used to accurately position the mask MA relative to the path of the radiation beam B. Mask alignment marks M1, M2 and substrate alignment marks P1, P2 may be used to align mask MA with substrate W. Although the substrate alignment marks are illustrated as occupying dedicated target portions, the marks may be located in spaces between target portions (these marks are referred to as scribe line alignment marks).

As shown in FIG2 , the lithography apparatus LA may form part of a lithography cell LC (sometimes also referred to as a lithocell or a (lithography) cluster), which often also includes equipment for performing pre-exposure and post-exposure processes on a substrate W. As is known, these include a spin coater SC for depositing a resist layer, a developer DE for developing the exposed resist, and cooling plates CH and bake plates BK, for example, for regulating the temperature of the substrate W (e.g., for regulating the solvent in the resist layer). A substrate handler or robot RO picks up substrates W from input/output ports I/O1 and I/O2, moves them between the various process equipment, and delivers the substrates W to a cassette LB of the lithography apparatus LA. The devices in the lithography unit, often collectively referred to as the coating and development system (track), are usually under the control of the coating and development system control unit TCU. The coating and development system control unit itself can be controlled by the supervisory control system SCS, which can also control the lithography equipment LA, for example, via the lithography control unit LACU.

To correctly and consistently expose substrates W exposed by lithography apparatus LA, the substrates need to be inspected to measure properties of the patterned structures, such as overlay errors between subsequent layers, line thickness, critical dimensions (CDs), etc. For this purpose, an inspection tool (not shown) may be included in lithography cell LC. If errors are detected, adjustments can be made to the exposure of subsequent substrates or other processing steps to be performed on substrate W, particularly if the inspection is performed before other substrates W from the same batch or lot are yet to be exposed or processed.

Inspection equipment (which may also be referred to as metrology equipment) is used to determine properties of a substrate W, and in particular, to determine how properties vary between different substrates W or how properties associated with different layers of the same substrate W vary between different layers. The inspection equipment may alternatively be configured to identify defects on the substrate W and may, for example, be part of the lithography cell LC, may be integrated into the lithography apparatus LA, or may even be a standalone device. Inspection equipment can measure properties on a latent image (the image in the resist layer after exposure), or on a semi-latent image (the image in the resist layer after the post-exposure bake step (PEB)), or on a developed resist image (where either the exposed or unexposed portions of the resist have been removed), or even on an etched image (after a pattern transfer step such as etching).

Typically, the patterning process in a lithography apparatus LA is one of the most critical steps in processing, requiring high accuracy in the sizing and placement of structures on the substrate W. To ensure this high accuracy, three systems can be combined in a so-called "holistic" control environment, as schematically depicted in FIG3 . One of these systems is the lithography apparatus LA, which is (actually) connected to a metrology tool MT (a second system) and to a computer system CL (a third system). The key to this "holistic" environment is to optimize the coordination between these three systems to enhance the overall process window and provide a tight control loop to ensure that the patterning performed by the lithography apparatus LA remains within the process window. A process window defines the range of process parameters (e.g., dose, focus, overlay) within which a particular process produces a defined result (e.g., a functional semiconductor device)—typically allowing process parameters in a lithography process or patterning process to be varied within this range.

The computer system CL can use (a portion of) the design layout to be patterned to predict which resolution enhancement techniques to use and perform computational lithography simulations and calculations to determine which mask layouts and lithography equipment settings achieve the maximum overall process window for the patterning process (depicted in FIG3 by the double white arrows in the first scale SC1). Typically, the resolution enhancement techniques are configured to match the patterning capabilities of the lithography equipment LA. The computer system CL can also be used to detect where within the process window the lithography equipment LA is currently operating (e.g., using input from a metrology tool MT) in order to predict whether defects may exist due to, for example, suboptimal processing (depicted in FIG3 by the arrow pointing to "0" in the second scale SC2).

The metrology tool MT may provide input to the computer system CL for accurate simulation and prediction, and may provide feedback to the lithography apparatus LA to identify, for example, possible drift in the calibration state of the lithography apparatus LA (depicted in FIG. 3 by the arrows in the third scale SC3).

The lithography apparatus LA is configured to accurately reproduce the pattern onto the substrate. The position and size of the applied features need to be within certain tolerances. Position errors can occur due to overlay errors (often referred to as "overlay"). Overlay is the error in placing a first feature of a first exposure period relative to a second feature of a second exposure period. The lithography apparatus minimizes overlay errors by accurately aligning each wafer to a reference before patterning. This is done by measuring the position of alignment marks on the substrate using alignment sensors. More information on the alignment process can be found in U.S. Patent Application Publication No. US20100214550, which is incorporated herein by reference. Pattern size calibration (e.g., CD) errors can occur, for example, when the substrate is not correctly positioned relative to the focal plane of the lithography apparatus. These focus position errors can be associated with unevenness of the substrate surface. Lithography apparatus minimizes these focus position errors by using a level sensor to measure the substrate surface topography prior to patterning. Substrate height correction is applied during subsequent patterning to ensure correct imaging (focusing) of the patterning device on the substrate. More information on level sensor systems can be found in U.S. Patent Application Publication No. US20070085991, which is incorporated herein by reference.

In addition to the lithography equipment LA and the metrology equipment MT, other processing equipment can also be used during IC production. An etching station (not shown) processes the substrate after exposing the pattern to the resist. The etching station transfers the pattern from the resist to one or more layers below the resist layer. Typically, etching is based on applying a plasma medium. The local etching characteristics can be controlled, for example, using temperature control of the substrate or using a voltage control ring to guide the plasma medium. More information about etching control can be found in International Patent Application Publication No. WO2011081645 and U.S. Patent Application Publication No. US 20060016561, which are incorporated herein by reference.

During fabrication processes such as lithography processes, for example, exposing structures on a substrate using a lithography system such as described herein (e.g., a system comprising a lithography apparatus and/or one or more other IC fabrication equipment), problems may arise that lead to unsatisfactory performance, which in turn may result in yield loss (non-functional devices). Any divergent or non-ideal behavior will have one or more root causes. Identifying the root cause is the first step in resolving any issues that arise during fabrication. It should be noted that divergent behavior in the context of this disclosure can include unexpected or unusually good behavior (e.g., determining the cause of such good behavior), rather than the more typical non-ideal behavior.

As the complexity of equipment used in semiconductor manufacturing increases, diagnostics becomes increasingly challenging. Many problems can occur, many of which have potential root causes that can exhibit the same behavior. Diagnostic tools and troubleshooting often require additional measurements to correctly identify the root cause of a problem at the machine or in the fabrication facility (fab). Field engineers perform diagnostics with incomplete data and dynamically decide whether to obtain additional data through diagnostic testing for machine diagnostics and/or additional tools or wafer metrology for fab diagnostics. These additional measurements are not routinely collected because they are expensive and their impact on improving root cause assessment is difficult to assess a priori due to the dependencies on already observed measurements. Tools are available to support engineers in making decisions, such as pattern recognition tools and diagnostic processes, but these tools are generally limited in their effectiveness.

Pattern recognition is often used to match the symptoms of a problem (e.g., a subset of error log entries) to known failure modes or past problems for which there is a database of potential diagnostic flows. In this way, the solution space is reduced. However, there is a large amount of contextual information that needs to be analyzed. A substrate typically goes through hundreds of different process steps during the manufacturing process. For each of these process steps, there may be dozens of possible tools/chambers that can be used for wafer processing. Furthermore, even for the same tool/chamber, tool characteristics can change and drift over time. There may be potential correlations between some of the contextual variables (e.g., due to wafer routing that is not completely random). This can reduce the accuracy of root cause analysis and increase time to insight because it requires manual oversight/filtering of irrelevant process steps based on domain knowledge. Therefore, pattern recognition reduces the solution space for a problem by matching its symptoms to known failure modes or symptoms of previously resolved problems. Pattern recognition is not an interactive tool; in this sense, it returns multiple potential solutions that match a symptom query. The candidate solutions returned may not contain the actual solution. Pattern recognition also does not provide recommendations on which actions should be performed to better identify the root cause.

Advanced machine learning methods such as Bayesian networks have also been proposed for diagnosis, but they also do not address the problem of recommending which measurements can be performed to achieve root cause estimation at the lowest cost.

A method and system for performing root cause diagnosis are described that dynamically recommend additional actions to be performed during the diagnostic process, such as what additional information should be collected to improve the root cause assessment. These recommendations can take into account the cost of collecting the additional information. For example, if it is deemed necessary to improve the root cause assessment as the upgrade progresses, additional information can be recommended. Decisions can be made based on cost-benefit considerations. Deciding whether to recommend further measurement can be formulated as a proactive feature acquisition problem over a forecast period. Thus, the method may (at least in part) aim to minimize the overall time to determine the root cause to achieve a certain accuracy (where the uncertainty value is below the uncertainty threshold), where the cost associated with the action may be a time cost (the time to perform a certain measurement). Other costs may also be considered, such as monetary costs, costs of additional tools, etc.

In one embodiment, a probabilistic model, such as a Bayesian model, can be queried by a reinforcement learning framework. The probabilistic model can be used to estimate the probability of a candidate root cause being the actual root cause, thereby providing an intermediate root cause estimate and an associated uncertainty measure that describes the uncertainty in that estimate. The probabilistic model can also infer unknown evidence (e.g., the results of a candidate action or measurement that has not yet been performed) and an associated uncertainty measure that describes the uncertainty in the inferred evidence.

The use of probabilistic models, such as the Bayesian model, ensures that interdependencies between different components and/or failure modes are modeled and accounted for (for example, a detected lens problem may have a dependency on one or more problems in other scanner components). These interdependencies are characteristic of the lithography equipment's behavior. The reinforcement learning framework enables data acquisition decisions based on the costs and benefits (rewards) of data acquisition. Over time, the framework's recommender (agent) refines its decision-making to identify the correct root cause at the lowest cost.

This approach uses machine learning models to perform root cause assessments using partial information while also proposing additional metrics to query within the context to dynamically refine the assessment. Optionally, the system can interact with field engineers or other experts and use their feedback to refine root cause estimation performance and/or decision making/action ranking.

FIG4 is a high-level flow chart describing a possible embodiment. A root cause estimation step 400 generates an intermediate root cause estimate 405. A measurement acquisition score is determined 410 based on the intermediate root cause estimate 405. A recommender 415 recommends a further action based on the measurement acquisition score and additional information, such as acquisition cost. The further action may be which subsequent measurement(s) should be acquired (if any). If further action (e.g., measurement) Y is recommended, the recommended action is executed 420. These root cause estimation steps 400 are executed again to generate an updated intermediate root cause estimate 405 based on the expanded information obtained from the recommended action 420. At a certain point, for example, when the intermediate root cause estimate is deemed sufficiently certain, the recommender 415 may not recommend further action N. This final intermediate root cause estimate 405 may then be finally determined 425 as an output root cause estimate 430. Optionally, the recommender's recommendation may be presented to a human expert 435 (e.g., a field engineer), who may provide feedback thereon to the recommender 415.

The root cause estimation step 400 may employ at least one machine learning model to estimate the (intermediate) root cause and its uncertainty using available information and measurements. Any suitable uncertainty measure may be used to quantify the uncertainty. In its evaluation, the model may use inferences (estimates) of any missing measurements (or infer missing measurements). Because diagnostic measurements often have dependencies on each other, the machine learning model may include at least one probabilistic model, such as at least one Bayesian network, that takes into account conditional dependencies among the features. Multiple such (e.g., probabilistic) machine learning models may be used, each modeling a respective submodule of the system.

The measurement acquisition score estimation step 410 can combine relevant quantities into a measurement acquisition score or measurement acquisition value (e.g., for each missing measurement or candidate action). Relevant quantities can include one or more of the following: known information or evidence (e.g., from measurements, logs, sensor values, test results, manual inspection, etc.); any estimated, inferred, or known information gain associated with the root cause assessment associated with the action (any suitable informativeness metric or information theory metric can be used to quantify information gain, such as an entropy-based metric); potential inferences/estimates/abstractions of missing values; uncertainty associated with such missing values; additional models; contextual information (e.g., the age or usage of the component). For example, if a missing measurement can be inferred with low uncertainty, it does not need to be measured. In another example, old and/or well-used components may be more likely to be associated with a root cause, and therefore, any measurements associated with such components may be considered more likely to provide relevant information.

The recommender 415 can combine the metric estimate scores for each candidate action (e.g., as a median positive reward) with other information from the environment, such as corresponding cost information (e.g., median negative reward), and recommend the next action. This recommendation can include ranking possible additional or candidate actions/metrics according to their scores. The action can be executed based on this ranking. For example, the highest-ranked recommendation can be acted upon without human intervention. Optionally, one or more of the highest-ranked recommendations can be provided to a human expert 435 who can make the final selection.

Expert 435 can be a human in the loop (field engineer) who receives recommendations and decides whether to approve them and/or which recommendations to approve. Human input can be fed back to recommender 415 to improve its decision making. This feedback can be a simple feedback command (e.g., undo, correct, incorrect) or more detailed feedback, such as changing the ranking of the recommendation or suggesting a different set of actions. After the root cause has been determined (e.g., after a divergent behavioral event), the human feedback can include the root cause itself. This component is optional because it requires additional design complexity and software interface; however, it can increase the performance of the recommender system.

In certain embodiments, the proposed method for root cause determination may utilize a probability model within a reinforcement learning framework. The probability model may include a probability graphical model and may implement the root cause estimation and measurement score determination steps described with respect to FIG. 4 . For example, the probability model may include a Bayesian network model. The root cause estimation and measurement score determination may further utilize appropriate simulation techniques using the probability model, such as Monte Carlo simulation. Therefore, the proposed embodiment may utilize a Bayesian network (BN) or other probability model as an automated reasoning tool to find the most likely root cause (RC).

The remainder of this disclosure assumes a BN-based model(s). However, this is not required, and any suitable probability model may be used that can determine the joint distribution of unobserved component health (root causes) and any one or more of the following, as appropriate: contextual information (e.g., component age), unobserved inferences or abstractions, and diagnostic evidence (logs, sensor values, test results, manual inspection, etc.). The joint distribution describes the distribution of all variables (i.e., observed and unobserved) of the system and is not limited to root causes.

Probabilistic graphical models or BN models can include multiple variables that do not necessarily need to be observed (for example, unobserved variables can include nodes that represent root causes). For example, the variables can be nodes (vertices) of a directed graph structure, such as a directed acyclic graph (DAG). The value of a node can depend on the value of its predecessor through a probability model. The DAG causes a factorization of the joint probability distribution of all variables in the model. For example, using Bayesian methods, an observed node/variable causes the posterior distribution over the unobserved nodes/variables by: a probability model of the values of one or more child nodes conditioned on its predecessor, and a prior distribution of the values of nodes without predecessors.

Posteriors (and Bayesian networks) can be used to simulate additional evidence, allowing possible or candidate (next) diagnostic actions to be ranked based on their expected changes to the root cause nodes of the network. As described, this additional evidence can include one or more of the following: estimated values of any unknown nodes, uncertainty, contextual information, and information gain. This additional evidence can be combined or summarized into a metric to obtain a score, which can be used to rank candidate diagnostic actions.

FIG5 is a network diagram illustrating a specific simplified example of a probability graphical model that can be used in a root cause estimation module according to embodiments disclosed herein. In this network, diamond-shaped nodes 500 represent potential or candidate root causes; the darker the node, the more likely that node is the root cause (in this example, the third and fifth diamond-shaped nodes 500 are equally likely to be the root cause). Square nodes A through F are evidence nodes that store information or evidence, such as the values and/or results of multiple diagnostic tests. The dark shading of evidence nodes A, B, D, and F indicates that a diagnostic action has been performed and the corresponding evidence has been inserted into the network. The evidence on nodes A, B, D, and F causes the posterior p ₁ on all other nodes (here, C and E) that do not have the specified value. Specifically, this is the bivariate posterior π at the evidence nodes C and E induced by marginalization.

To estimate quantities relevant to deciding on further actions (e.g., further acquisitions), the following algorithm can be performed: 1. Let _p1 be the posterior induced at the unspecified nodes of the network after observing all known evidence (i.e., evidence collected so far). 2. Let π be the posterior _p1 marginalized at the unspecified nodes for the outcome of the diagnostic test (candidate action). 3. Sample from the posterior to obtain an estimated outcome (one or more sampled values) for each diagnostic test or candidate action that has not yet been performed. 4. For each node in the unperformed diagnostic action, a. For each of the sampled values for that node, i. Insert the value of point a. into the node from step 2. ii. Compute the new posterior _p2 in the root cause node. iii. Evaluate a measure of the change in the _posterior p2 relative to _p1 (the posterior in the absence of additional evidence). This measure can be, in particular, the ℓ2 distance between _p1 and _p2 , or the aggregation of the mutual information between _p1 and _p2 at each node, or the aggregation of the information gain. b. Estimate the expected change in the metric from the sampled values across the nodes, for example using a Monte Carlo method. 5. Output any candidate actions (actions not taken) and the associated expected change in the posterior (e.g., a measure of uncertainty in the associated root cause estimate) based on the value estimated at point 4b.

For the recommender, a model-based reinforcement learning implementation is proposed, optionally with human feedback. This is only an example implementation and more traditional machine learning techniques can be used. For example, instead of learning a strategy to rank action plans according to their scores and costs, a heuristic can be constructed based on a cascade of threshold or root cause estimation models. However, reinforcement learning (RL) is a natural fit for this application. In particular, due to the (good) accuracy of the probabilistic model (Bayesian network) used to model root cause estimation, training such a model-based reinforcement learning is data-efficient.

The reinforcement learning framework can learn by continuously using a probabilistic model or BN to determine the probability of a root cause based on observed evidence, and then propose (e.g., rank) actions (e.g., measurement, intervention, etc.) to maximize or increase the probability of correct root cause inference in a cost-effective manner. The probabilistic model can be queried to provide a posterior based on the evidence and the reinforcement learning framework (optionally with the help of an engineer) to learn which actions related to evidence collection drive the posterior so that it improves the root cause estimate.

6 is a flow chart illustrating a method for root cause determination based on a reinforcement learning approach. The flow chart shows a recommender 600 to be described and a root cause estimator 620 already described (e.g., including a probability model or a BN model).

RL implementations can embody a Markov decision process (MDP), comprising an agent (recommender 600) interacting with an environment Env, a set of states S, and a set of actions A for each state, defining a state space. In a particular (e.g., current) state _St , the agent 600 determines the next action _At from the set of actions (candidate actions) in that state _St based on a policy that must be learned. When the agent 600 decides to change state, it receives a reward _Rt . The agent's goal is to maximize its total reward (e.g., by determining a policy that maximizes this reward).

The agent determines 610 the next action A _t based on the (current) state S _t from the environment Env. Such actions may include the agent 600 recommending candidate actions or diagnostic tests to be executed next in order to obtain additional data and/or including a ranking of possible candidate actions. Thus, the agent 610 acts as a recommender of proposed solutions. Intermediate rewards and information 625 may be obtained from the root cause estimation model 620, in particular: the intermediate or current root cause prediction E st and/or the associated uncertainty in the estimate and/or unobserved features. The root cause estimation model 620 receives all available information/measurements 615 for each state.

Possible actions at decision 610 may be i) obtaining any of Y unobserved features, i.e., expensive measurements Meas or a set thereof, or ii) stopping N and outputting the current prediction Est.

The state _St contains all the information 615 collected so far and newly acquired information, such as that acquired at the measurement Meas. The transition from one state to another is largely determined by the dependencies of the diagnostic measurements and their relationship to potential root causes (i.e., the conditional dependencies modeled by the Bayesian network 620).

The environment, Env, contains all possible states in the system and all state transitions. The environment interacts with agents that provide state updates and rewards.

The reward _Rt includes the positive reward for any proposed action determined by the root cause estimator (e.g., corresponding to the score achieved in Figure 4) and any negative reward or cost associated with the effectiveness of any action. At each state, the agent receives a negative reward (or cost) Cst when it achieves a measurement. In short, the cost can be the same for any additional measurement. However, if the cost depends on the actual cost of the action (e.g., in terms of time and/or any other considerations, such as monetary cost/extra tools required), then a better strategy can be determined. Some metrology actions may be associated with significantly higher costs than other metrology actions (e.g., a metrology action performed within the tool in question (e.g., a scanner) using one or more onboard sensors may be less expensive than a metrology action that requires additional tools or requires the tool to be offline and/or separated). Additional positive rewards 625 may be awarded based on any estimated improvement in the intermediate root cause assessment Est. Intermediate rewards may be determined, for example, for each new state and/or each time information is acquired. Such intermediate rewards may be based on information gain and improvements in the intermediate root cause assessment and may include additional positive rewards.

A final reward is determined after the process terminates, when the root cause condition has been resolved. This can be when the root cause is identified/estimated with a certain degree of certainty (e.g., an uncertainty below a critical uncertainty value). The final reward describes the agent's success in obtaining the correct root cause with the least cost (e.g., the minimum number of measurements or measurement time). In this way, a trade-off can be achieved between optimizing the effectiveness of the root cause estimation and minimizing the amount of information obtained.

Human expert 605 can also provide feedback (FB) (such as rewards) to agent 600 to indicate their satisfaction with the agent's recommendations. This can help agent 600 refine its strategy. A more effective way for humans to interact with the agent is to directly provide input to any action A _t in a specific state. Examples of human input to the agent could be "UNDO,""Correct,""Incorrect," or additional metrics about the agent's own decision (ranking).

Agent 600 is trained to learn the best policy for recommending actions (e.g., measurement acquisition). To learn the best policy, any state-of-the-art RL algorithm can be used (e.g., combined with human feedback FB, as appropriate). An example of a state-of-the-art algorithm that can be used is proximal policy optimization. In this case, the reward will include human feedback in addition to other information in any format. Examples of how human feedback can be included in the reward are using any distance estimate between human feedback and recommendations (e.g., Kullback-Leibler divergence), or even using human input as a supervisory marker for model updates or gradient training. This implementation can support irregular human input due to the support of Bayesian networks that provide intermediate rewards.

Human-agent interactions can be explained using human-understandable explanations of the agent's decisions. This explanation can be provided by interpretable approximations of complex decision functions (e.g., linear approximations, decision trees, etc.).

A practical implementation of the root cause estimator may use a plurality of Bayesian networks (e.g., one for each (sub)domain or (sub)function of the equipment), for which one or more strategies may be learned, e.g., learning a customer-specific optimal resolution strategy or specializing it by (sub)domain.

While the concepts disclosed herein are described in the context of root cause analysis, they are not limited thereto. The concepts disclosed herein can be applied to any device, system, or configuration evaluation activity where additional data with an associated cost is available for evaluation, the additional data is dependent, and a decision needs to be made regarding whether to obtain the additional data.

Although the concepts disclosed herein are disclosed with respect to a fabrication system, more particularly an IC fabrication system (e.g., including one or more of a lithography apparatus/scanner, a metrology apparatus, an etch apparatus, or a deposition apparatus), they may be applicable to any system.

Although specific reference may be made herein to the use of lithography equipment in IC manufacturing, it should be understood that the lithography equipment described herein may have other applications. Possible other applications include the fabrication of integrated optical systems, guide and detection patterns for magnetic resonance memory, flat panel displays, liquid crystal displays (LCDs), thin film magnetic heads, and the like.

Although specific reference may be made herein to embodiments of the present invention in the context of lithography equipment, embodiments of the present invention may be used in other equipment. Embodiments of the present invention may form part of mask inspection equipment, metrology equipment, or any equipment that measures or processes objects such as wafers (or other substrates) or masks (or other patterned devices). Such equipment may generally be referred to as lithography tools. Such lithography tools may utilize vacuum conditions or ambient (non-vacuum) conditions.

Although specific reference may be made above to the use of embodiments of the present invention in the context of photolithography, it will be appreciated that the present invention is not limited to photolithography and may be used in other applications, such as imprint lithography, where the context permits.

Although specific embodiments of the present invention have been described above, it should be understood that the present invention may be practiced in other ways than those described. The above description is intended to be illustrative rather than restrictive. Therefore, it will be apparent to those skilled in the art that modifications may be made to the present invention as described without departing from the scope of the claims set forth below.

400: Root Cause Estimation Step 405: Intermediate Root Cause Estimation 410: Step 415: Recommender 420: Step 425: Step 430: Output Root Cause Estimation 435: Human Expert 500: Diamond Node 600: Recommender/Agent 605: Human Expert 610: Decision/Agent 615: Available Information/Measurement 620: Root Cause Estimator/Root Cause Estimation Model/Bayesian Network 625: Intermediate Reward and Information A: Node A _t :Action B: Radiation beam/node BD: Beam delivery system BK: Bake plate C: Target part/node CH: Cooling plate CL: Computer system Cst: Negative reward D: Node DE: Display E: Node Env: Environment Est: Intermediate or current root cause prediction/intermediate root cause assessment F: Node FB: Feedback I/O1: Input/output port I/O2: Input/output port IF: Position sensor IL: Lighting System/Illuminator LA: Lithography Equipment LACU: Lithography Control Unit LB: Carrier LC: Lithography Unit M1: Mask Alignment Mark M2: Mask Alignment Mark MA: Patterning Device/Mask Meas: Measurement MT: Support Structure/Mask Stage/Metrology Tool P1: Substrate Alignment Mark P2: Substrate Alignment Mark PM: Primary Positioner PS: Projection System PW: Secondary Positioner RO: Substrate Handler/Robot _Rt : Reward SC: Spin Coater SC1: Primary Scale SC2: Secondary Scale SC3: Third Scale SCS: Supervisory Control System SO: Radiation Source _St : Status TCU: Coating and Development System Control Unit W: Substrate WT: Substrate Stage

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying schematic drawings, in which: Figure 1 depicts a schematic overview of a lithography apparatus; Figure 2 depicts a schematic overview of a lithography unit; Figure 3 depicts a schematic representation of overall lithography, illustrating the collaboration between three key technologies used to optimize semiconductor manufacturing; Figure 4 is a high-level flow chart illustrating a root cause estimation method according to one embodiment; Figure 5 is a specific example of a probability graph model that can be used in embodiments disclosed herein; and Figure 6 is a flow chart illustrating a reinforcement learning implementation of a root cause estimation method according to one embodiment.

400: Root cause estimation step

405: Intermediate Root Cause Estimate

410: Step

415: Recommender

420: Steps

425: Steps

430: Output root cause estimate

435:Human Expert

Claims (15)

A method for evaluating a plurality of candidate actions based on evidence data associated with an evaluation action for at least one manufacturing facility or system, the method comprising: obtaining at least one probability model that relates the evidence data to an estimated probability of one or more root cause evaluations for the manufacturing facility; using the at least one probability model to determine an estimated probability of one or more root cause evaluations based on evidence data including additional evidence from one or more candidate actions that have not yet been performed; determining a reward based on the respective estimated probabilities of the one or more root cause evaluations and a respective cost associated with the one or more candidate actions; and determining whether to perform any of the one or more candidate actions based on the reward. The method of claim 1, wherein the one or more candidate actions include performing at least one or more additional measurement and/or diagnostic actions. The method of claim 1, wherein the at least one probability model comprises at least one Bayesian network model. The method of claim 1, further comprising using the at least one probability model to determine an uncertainty associated with each of the one or more root cause assessments. The method of claim 1, further comprising using the at least one probability model to infer the evidence data for the one or more candidate actions that have not yet been executed to obtain inferred evidence data; and wherein the evidence data includes the inferred evidence data. The method of claim 1, wherein the evidence data further includes contextual information and/or information gain data. The method of claim 1, wherein the method is performed a plurality of times until the estimated probability of one or more root cause assessments is estimated with sufficient certainty, wherein each iteration is based on the evidentiary data, including any additional evidentiary data obtained as a result of a decision-making step in the previous one or more iterations. The method of claim 7, wherein each iteration comprises determining a median reward and a median estimated probability for one or more root cause assessments, and wherein the decision step is performed based on the median reward and the median estimated probability for the one or more root cause assessments. The method of claim 1, comprising obtaining a recommender; and using the recommender to perform the decision step of determining whether to perform any of the one or more candidate actions. The method of claim 9, wherein the recommender comprises an agent in a reinforcement learning framework, the recommender being operable to learn improvements in the decision over time based on the reward. The method of claim 10, wherein the reinforcement learning framework comprises a Markov decision process. The method of claim 9, wherein the recommender is operable to maximize the reward, wherein the cost includes a negative reward in the determination of the reward. The method of claim 9, wherein the decision and/or any output of the recommender is presented to a human expert for review and/or feedback. The method of claim 1, wherein the at least one probability model comprises a respective probability model for two or more components or subsystems of the at least one manufacturing apparatus or system. A computer program comprising program instructions operable to perform the method of claim 1 when run on a suitable device.