JP2014207110A - Observation apparatus and observation method - Google Patents

Abstract

The present invention generates and displays a good integrated image by performing appropriate integration processing on a plurality of images captured by a charged particle microscope. A plurality of images are captured, one or more ROIs included in the captured image are extracted and classified, and then the plurality of captured images are integrated and displayed. During integration, the integration method is controlled according to the ROI classification results. [Selection] Figure 1

Description

The present invention relates to an observation apparatus and an observation method.

  Charged particle microscopes have a much higher resolution than optical microscopes, and are widely used to clearly observe the fine structure of an object to be observed. A charged particle microscope irradiates a target sample with a charged particle beam, and detects the particles emitted from the target sample or transmitted through the target sample (the same kind of charged particles as the irradiated charged particles, or electromagnetic waves, photons). By detecting at, an enlarged image of the target sample is acquired.

  In particular, in semiconductor manufacturing processes, applications such as semiconductor wafer inspection, pattern dimension measurement, and pattern shape measurement include scanning electron microscopes, scanning ion microscopes, and scanning transmission. A charged particle microscope such as an electron microscope (Scanning Transmission Electron Microscope) is used. In these applications, the captured images are used to observe semiconductor patterns and defects, detect defects and analyze the causes of the defects, and measure the dimensions of the patterns.

  In charged particle microscopes, it is important to provide images that can be easily observed and analyzed. For this purpose, images that are easy to see the region (hereinafter referred to as ROI: Region Of Interest) to be observed and analyzed Need to be displayed. Examples of ROI include areas where defects on the semiconductor are shown, and areas where the pattern to be observed is shown. An image captured with a charged particle microscope may include multiple ROIs. In order to make the ROI easier to see, there are many cases where a plurality of captured images with different imaging conditions, imaging targets, imaging positions, etc. are acquired. The detector also includes secondary charged particle detectors, backscattered particle detectors, detectors that detect only specific particles and light, and detectors that detect particles whose energy or spin direction is within a specific range. There are several types, and the information obtained by each detector is different. In addition, since the information obtained varies depending on the position of the detector, many microscopes have a plurality of detectors of the same type or different types. If there are a plurality of detectors, a plurality of captured images can be acquired by one imaging.

In order to obtain an image with high visibility, a method of integrating a plurality of captured images has been proposed. For example, Patent Document 1 discloses that "a means for changing the focal point of a charged particle beam emitted from a charged particle source, a charged particle detector for detecting charged particles obtained at a charged particle beam irradiation site of the sample, A means for synthesizing a two-dimensional image of the sample viewed from the charged particle beam source direction based on a signal of a portion where the charged particle beam is focused among signals output from the charged particle detector is provided. A charged particle beam device "(summary).
Patent Document 2 states that “a sample is imaged under different imaging conditions to acquire a plurality of images of the sample, a degradation function of each image is generated for the acquired plurality of images, and the acquired plurality of images are acquired. A method (summary) is disclosed in which an image with improved resolution is generated using an image and a degradation function corresponding to each of the generated images, and the image with improved resolution is processed.

  A method for integrating captured images has also been proposed in order to improve ROI visibility. Patent Document 3 states that “the specific gravity of the matching result image 74 is high for known defect modes and normal portions, and the specific gravity of the detection image 71 can be increased otherwise, and a more natural review image 82 is generated” ( Paragraph 0043) “A composite image reflecting the information of the detected image 71 is realized by a pattern inspection apparatus or a morphing technique that is“ a technique of combining so as to maintain the correspondence between feature points of an image to be combined ”. A pattern inspection apparatus (paragraph 0044) is disclosed.

  Patent Document 4 describes a charge for the purpose of “capturing a high-quality image even in a region where a sufficient signal amount cannot be obtained from a sample, such as a lower layer region in a multilayer layer or a hole bottom of a hole pattern”. A method for imaging an image of a sample using a particle microscope device, wherein the gain of a detector of the charged particle microscope device is set to a first gain value, and the sample is scanned with a charged particle beam. A first image acquisition step of acquiring a first image, a gain of the detector is set to a second gain value different from the first gain value, and the sample is scanned with a charged particle beam; A second image acquisition step of acquiring a second image; and an image combination step of combining the first image and the second image using the first gain value and the second gain value. An image characterized by having Image method "is disclosed.

JP 2006-190693 A JP 2010-62106 A Republished WO2010 / 106837 JP 2012-226842 A

  However, if the target ROI looks very different for each ROI, the problem is that the appropriate integration method is different. In the electron beam pattern inspection apparatus targeted in Patent Document 3, only the defect is targeted as the ROI, and the electron beam pattern inspection apparatus captures an image with a low magnification and detects the defect. Etc. are not significantly different. In patent document 4, only a dark part is objected as ROI.

  On the other hand, for example, a charged particle microscope, such as a length measurement SEM or defect review SEM, that captures defects at a high magnification (for example, 100,000 times or more) and measures or observes the defect, the ROI that is targeted is visible for each ROI. May vary greatly. In some cases, it is possible to create an easy-to-see image using only the conventional method of weighted addition processing or image deformation by a morphing method, but depending on the ROI, it may be difficult to see such as unnatural appearance. In this way, when targeting a variety of ROIs that look very different, a different integration method is required.

  The present invention provides means for generating and displaying a good-looking image even in the above case.

  In the present invention, the above-described problem is solved by the following image display method and an observation apparatus equipped with the image display method.

  According to the present invention, it is possible to provide an observation apparatus and an observation method that provide an image in which each ROI is easy to see.

It is an example of a sequence for extracting and classifying ROI and integrating a plurality of captured images based on the classification result. 1 is a basic configuration of a charged particle microscope according to an embodiment of the present invention. It is an Example figure showing the advantage and subject of integrating a plurality of captured images. It is an Example figure which shows the example which integrates two picked-up images by weighted addition. It is an Example figure which shows the example which integrates two picked-up images by methods other than weighted addition. It is an Example figure which shows an example different from FIG.4, 5 which integrates two captured images. It is an Example figure of the process which unifies two picked-up images reflected in a different shape. It is an Example figure of the process which integrates two captured images from which magnification differs. It is an Example figure of the process integrated while performing shape deformation of ROI and its peripheral area | region. It is an Example figure of the process which integrates two images which imaged a different area | region. It is an Example figure which extracts ROI contained in two captured images. FIG. 12 is an embodiment diagram different from FIG. 11 showing ROI extraction processing. FIG. 2 is an example diagram showing a flow of ROI classification processing corresponding to step 103 in FIG. 1. It is an Example figure of the process which identifies the height relationship of the several target object which exists in ROI. It is an Example figure showing the process which identifies which area | region ROI belongs to among the upper layer pattern area | region, a lower layer area | region, and a pattern side wall area | region. It is an Example figure showing the process which controls an integration method according to a classification result. It is an Example figure showing the process which controls an integration method according to a classification result. It is an Example figure showing the user interface for setting the integration method for every classification result. It is an Example figure of the sequence which image | photographs the high magnification image of a defect automatically, and calculates | requires the integrated image. It is an Example figure showing the detail of the brightness adjustment process in FIG.

  The present invention relates to a scanning charged particle microscope that acquires an image by irradiating a sample with charged particles, and provides an image display method and apparatus for displaying an image with particularly high visibility. Embodiments according to the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing an example of a sequence for extracting and classifying ROI and integrating a plurality of captured images based on the classification result. First, in step 101, a plurality of captured images are acquired. Next, in step 102, ROI is extracted from the captured image, and in step 103, ROI is classified. Next, in step 104, based on the ROI classification result, a plurality of captured images are integrated, that is, for each pixel using information on the same pixel in the plurality of captured images or pixels in the vicinity thereof. A single image (integrated image) for which the brightness value is obtained is generated. Finally, in step 105, the integrated image is displayed.

  In many cases, the target ROI looks very different for each ROI. On the other hand, by classifying the ROI included in the captured image and then controlling the integration method according to the result, it is possible to integrate the ROI in a method suitable for each ROI and always provide an image that is easy to see the ROI. It becomes possible to do.

  FIG. 2 shows a basic configuration of a charged particle microscope according to an embodiment of the present invention. The charged particle microscope includes, for example, a charged particle image acquisition device 201, an input / output unit 221, a control unit 222, a processing unit 223, a storage unit 224, an ROI extraction unit 225, an ROI classification unit 226, an image integration unit 227, and the like. The In the charged particle image acquisition apparatus 201, a charged particle beam 203 is generated from a charged particle gun 202, and the charged particle beam 203 is focused on the surface of a sample 206 by passing through a condenser lens 204 and an objective lens 205. Next, an image is acquired by detecting particles generated from the sample 206 with the detector 208. The image is stored in the storage unit 224. A plurality of detectors 208 may be provided. Furthermore, the detector 208 may be a detector that detects different particles, such as a detector that detects electrons and a detector that detects electromagnetic waves, or the energy or spin direction is within a specific range. Or a detector that detects particles having different properties such as a secondary charged particle detector and a backscattered charged particle detector. A plurality of detectors that detect particles of the same property may be provided at different positions. When a plurality of detectors are provided, a plurality of images can be acquired usually by one imaging. A plurality of images may be acquired by changing the imaging conditions and imaging twice or more. The sample 206 is in contact with the stage 207. By moving the stage 207, an image at an arbitrary position of the sample can be acquired. Further, the charged particle beam can be scanned onto the sample by changing the direction of the charged particle beam 203 two-dimensionally by the beam deflector 209.

  The input / output unit 221 inputs an image capturing position and image capturing conditions, and outputs an image after integration. The control unit 222 controls the voltage applied to the charged particle gun 202, the adjustment of the focal positions of the condenser lens 204 and the objective lens 205, the movement of the stage 207, and the like as control of the imaging device. The control unit 222 also controls the input / output unit 221, the processing unit 223, the storage unit 224, the ROI extraction unit 225, and the ROI classification unit 226. The processing unit 223 performs various types of processing, for example, processing related to automatic focusing necessary for focusing the charged particle beam 203 on the surface of the sample 206. The storage unit 224 stores a plurality of captured images, images after integration, ROI extraction results, ROI classification results, various processing parameters, and the like. The ROI extraction unit 225 extracts ROI from the acquired plurality of captured images. The ROI classification unit 226 classifies each extracted ROI. The image integration unit 227 generates a single integrated image by integrating a plurality of captured images. At this time, the integration method is controlled based on the ROI classification result.

  FIG. 3 is an example diagram showing advantages and problems of integrating a plurality of captured images. The integrated image 303 is an example of an image obtained by integrating images (secondary charged particle images) 301 and 302 acquired by two secondary charged particle detectors. It is assumed that the image 301 and the image 302 are obtained by imaging the same position on the sample. These captured images may be images obtained by a single scan using a plurality of detectors arranged at different positions, or may be images captured by changing the imaging conditions. The image 301 shows a defect 311 and a pattern 314, but the defect 311 has a low contrast (in FIG. 3, the region surrounded by a dotted line indicates a low contrast). In the image 302, the same defect as 311 is shown in 312, but the contrast is higher than that of the image 301. On the other hand, the same pattern as 314 is shown in 315, and the contrast is low. I want to integrate these captured images and generate images with high contrast between defects and patterns. For this purpose, a method of weighting and adding the images 301 and 302 is suitable. Specifically, after extracting the defective area 312 from the image 302, the weight of the image 302 is increased for the defective area, and the weight of the image 301 is increased for the other areas, and weighted addition is performed. Just do it.

  The integrated image 306 is an example of an image obtained by integrating the shaded image (shadow image) 304 and the secondary charged particle image 305 acquired by the backscattered charged particle detector. Images 304 and 305 are obtained by imaging the same position on the sample. The secondary charged particle image 305 shows a pattern 322 and a scratch defect (scratches generated on the sample surface) 325, but the scratch defect 325 has low contrast and poor visibility. On the other hand, I would like to generate an integrated image that clearly shows scratch defects. Since the scratch defect can be clearly seen in the shadow image 304, this information may be integrated. However, the same method as when the integrated image 303 is generated, that is, the scratch defect area 324 is extracted, the weight of the image 304 is increased in that area, and the weight of the image 305 is increased in the other areas. The integration by cannot produce an appropriate image. An example of the integrated image obtained by such weighted addition is shown in 307. Although both the pattern 327 and the scratch defect 328 are clearly seen, the pattern 327 is not reflected in the scratch defect area, so that an unnatural image is obtained. In order to generate an image in which a pattern appears in the scratch defect area as in the integrated image 306, it is necessary to use an integration method that is not simple weighted addition. In view of the above, it is desirable to perform integration by weighted addition in the case of the captured images 301 and 302, and to perform integration other than weighted addition in the case of the captured images 304 and 305. In the present invention, a good integrated image is generated by selecting an integration method suitable for the type of ROI (in this example, a defective area).

  Next, an integration process corresponding to step 104 in FIG. 1 will be described with reference to FIGS. By showing using a specific example that the appropriate integration processing differs for each type of ROI, it is shown that an image with good image quality can be obtained if the integration processing can be controlled based on the ROI classification result. The ROI classification method will be described later with reference to FIGS. 13 to 15, and the integration processing control method will be described later with reference to FIGS. 16 to 18. 4 to 10, it is assumed that a process of acquiring a plurality of images and extracting and classifying ROI, which is the process of steps 101 to 103, is performed in advance.

  FIG. 4 is a diagram illustrating an example in which two captured images are integrated by weighted addition. Images 401 and 402 are the same as images 301 and 302 in FIG. When integrating, an image mainly used in the region other than ROI (hereinafter referred to as a base image) is determined. In this example, the image 401 is a base image. In this example, an area where the defect 412 is shown is ROI. In the following, the perimeter of the image is indicated by a thick frame in the base image, and the perimeter of the ROI is similarly indicated by a thick frame. As the ROI classification result, for example, a result of foreign matter on the pattern is obtained. The ROI has a low contrast in the image 401 and is high in the image 402. Therefore, in the integration process 404, first, in the integration ratio calculation process 405, the weight is set so that the weight of the image 402 is increased in the ROI and the weight of the image 401 is increased in the region other than the ROI. Add. Thereby, the integrated image 403 is obtained.

  There may be a plurality of integrated images. For example, an integrated image generated based on the image 401 and an integrated image generated based on the image 402 may be output. Further, there may be three or more captured images. Although an example in which weighted addition is performed in the load sum processing 406 has been shown, it is not limited to weighted addition. For example, a calculation method may be used in which a weighted addition is calculated after applying a function f (for example, a logarithmic function) to the brightness values of two images, and finally an inverse function of f is applied. The combined image does not have to be a gray scale image, and may be a color image. For example, the ROI may be displayed with a color for easy viewing, and the area other than the ROI may be displayed in grayscale. In this case, the integrated ratio calculation process 405 calculates the weight for each component (for example, Red, Green, Blue) representing the color image, and the load sum process 406 performs the weight addition process for each component.

  FIG. 5 is a diagram illustrating an example in which two captured images are integrated by a method other than weighted addition. Images 501 and 502 are the same as images 304 and 305 in FIG. The base image is 502. That is, the scratch defect 512 of the shadow image 501 is integrated with the secondary charged particle image 502. As the ROI classification result, for example, a result such as an uneven defect is obtained. In the integration process 509, first, information on the pattern 511 and the like is removed from the shadow image 501, and only the unevenness information representing the scratch defect 512 is extracted. In order to perform this extraction, first, processing 503 for estimating an image having no defect is performed, and information on the pattern 513 such as the image 504 is extracted. In this processing, for example, pattern information may be extracted using the secondary charged particle image 502, or may be extracted using an image obtained by imaging another place not including a defect, or comparison with design data. . After deleting the pixels in the ROI, estimation may be performed using techniques such as inpainting. Next, in a process 505 for extracting ROI difference information, a difference between the shadow image 501 and the image 504 is calculated, thereby obtaining an image 506 in which only unevenness information representing a scratch defect is shown as 514. Thereafter, the weighted addition of the secondary charged particle image 502 and the image 506 is performed by the load sum processing 507 to generate an integrated image 508. In the image 508, only the ROI unevenness information 516 is superimposed on the pattern 515, and natural integration is achieved. In order to selectively use the integration method by weighted addition shown in the example of FIG. 4 and the integration method shown in the example of FIG. 5, the defect is an opaque defect such as the defect 412 or is located below the defect such as the defect 512. Classification may be performed so that it can be determined whether the defect is a transparent defect in which the pattern is reflected. Note that the load sum process 507 corresponds to a process that combines the integration ratio calculation process 405 and the load sum process 406 in FIG. 4. Hereinafter, for simplification of description, it is assumed that the load sum process includes an integrated ratio calculation process.

  The images 501 and 502 in FIG. 5 may be images obtained by detecting particles having high energy and particles having low energy, for example. If the energy of the particles to be detected is different, the information contained in the particles changes, and the image may change greatly. Alternatively, an image in which different types of particles are detected may be used.

  FIG. 6 is an example diagram showing an example different from FIGS. 4 and 5 in which two captured images are integrated. Two images of the shadow image 601 and the secondary charged particle image 602 are integrated by processing 604 to obtain an image 603. As a result of ROI classification, for example, a defect such as a defect with low brightness in a shadow area is obtained. In the image 601, a pattern 611 is shown. A region 612 corresponds to a shadow portion of the pattern and has a lightness value lower than that of a similar region that is not a shadow. On the other hand, in the image 602, a defect 613 that is ROI is shown at a position corresponding to the shadow area of the image 601, and the brightness value of the defect is lower than that of the surrounding area. In the image 602, the shadow of the pattern is not visible. I want to merge two images, but when the light-dark relationship is reversed, it generally looks unnatural. Therefore, in order to perform integration while maintaining the light / dark relationship, first, the brightness adjustment processing 605 is performed on each of the images 601 and 602, and then the load sum processing 607 is performed. In the brightness adjustment processing 605, the brightness of each image is adjusted based on information on both the images 601 and 602. By adjusting the brightness before the load sum, in the image 603, the overall brightness balance is adjusted so that the brightness value of the defect 615 is further lowered while leaving a shadow portion 614 having a brightness value lower than that of the surrounding area. To do. In integration by only simple weighted addition, it is generally impossible to maintain a light-dark relationship while maintaining high contrast, but this can be achieved by combining with lightness adjustment processing.

  FIG. 20 is a diagram showing an example of the details of the brightness adjustment processing 605 in FIG. The same image or process as in FIG. 6 is denoted by the same number as in FIG. Hereinafter, when the same image or process is represented, the same number is assigned. Graphs 2001 and 2002 are lightness values in line segments AB of images 601 and 602, respectively. A broken line 2010 is an auxiliary line for explanation. The brightness value of the shadow area 2011 is lower than the brightness value of the right area. Further, the brightness value of the ROI 2012 of the secondary charged particle image is also lower than the brightness value of the region on the right side thereof. The brightness value of the region 2013 of the secondary charged particle image is higher than the brightness value of the same region of the shadow image. Graphs 2003 and 2004 are lightness values in the line segment AB after the lightness adjustment processing 605 is performed.

  From the results of ROI classification performed in advance, it is known that ROI is a defect with low brightness existing in the shadow area. However, since the brightness value of the shadow area of the shadow image is low, sufficient contrast cannot be obtained even if the ROI of the secondary charged particle image is integrated on the image 601 as it is. In lightness adjustment processing 605, therefore, the lightness value of the shadow region 2014 of the shadow image is adjusted to be high. Next, the brightness value of the ROI peripheral region 2017 of the secondary charged particle image is matched with the brightness value of the same region of the shadow image (that is, the shadow region 2014). In addition, the brightness value of the ROI 2015 is also lowered in order to keep the brightness difference between the ROI 2015 and the surrounding area constant. Further, in order to prevent an unnatural result in the load sum processing 607, the brightness value of the region 2016 or the like may be matched with the brightness value of the same region of the shadow image. A graph 2005 is a lightness value in the line segment AB with respect to the integrated image. As shown in the graph 2005, by performing the brightness adjustment processing 605, an integrated image is obtained in which the brightness value of the ROI 2018 is lower than the brightness value of the surrounding area while maintaining the contrast of the ROI and the surrounding area. be able to.

  The brightness value of each region of the image may vary greatly between the plurality of captured images. For example, when different types of detectors are used, the brightness values generally differ greatly because the types and characteristics of the particles detected by each detector are different. Also, the brightness value varies depending on the position of the detector, because the number of particles that reach each detector changes. When such a plurality of captured images are integrated, the weighted addition may reverse the brightness and may appear unnatural. On the other hand, in the present invention, the brightness relationship between ROI and the surrounding area of ROI is preserved and integrated by adjusting the brightness. As a result, integration is possible while suppressing the reversal of light and dark.

  FIG. 7 is a diagram illustrating an example of processing for integrating two captured images reflected in different shapes. Two images 701 and 702 are integrated by processing 704 to obtain an image 703. The object to be imaged is a multi-layer pattern including foreign matter. For example, the image 701 is a shadow image, and the image 702 is a secondary charged particle image. As the ROI classification result, for example, the result is that the ROI of 712 is a defect in the upper layer region, and the ROI of 713 is the lower region. In the image 701, the upper layer pattern 711 and the defect 712 are clearly shown, but sufficient contrast is not obtained in the lower layer. On the other hand, in the image 702, the lower layer region 713 is clearly shown, but the visibility of the upper layer pattern and the defect is poor. In the shadow image, a sufficient number of charged particles cannot be detected in the lower layer area compared to the secondary charged particle image, and the contrast tends to be low. Therefore, the secondary charged particle image may have higher visibility in the lower layer area. Many. I want to integrate the lower region of the image 702 into the image 701. However, the width of the lower layer region is different between the image 701 and the image 702, and the width of the image 701 is narrower. Therefore, first, a deformation amount calculation process 708 is performed using the image 701 and the image 702. The amount of deformation is calculated for each pixel or region. Next, a shape transformation process 705 is performed on the image 702 to generate an image 707 in which the width of the lower layer region is matched with the image 701. Thereafter, an integrated image 703 is generated by the load sum processing 706 of the images 701 and 707. In the deformation amount calculation processing 708 and the shape deformation processing 705, processing is performed so that the joint is natural not only at the boundary between the upper layer pattern and the lower layer region but also at the boundary between the lower layer region and the defect 712. In this embodiment, the shape of the image 702 is deformed to adjust the width of the lower layer area to the image 701. However, the image 701 may be deformed to adjust the width of the lower layer area to the image 702. .

  As in the examples of the images 701 and 702, what is reflected for each pixel may be different between a plurality of captured images due to image distortion. At the pattern boundary, the upper layer pattern information is dominant in the image 701, and the lower layer information is dominant in the image 702, so the width of the lower layer area appears to be different. For this reason, if the shape is not appropriately deformed, problems such as discontinuity of the boundary occur when the images are integrated. For this reason, it is necessary to transform not only the ROI but also the surrounding area. On the other hand, integration can be naturally performed even in the case of a captured image with image distortion.

  FIG. 8 is a diagram illustrating an example of processing for integrating two captured images having different magnifications. Consider processing 804 to integrate the defect 812 of the image 802 captured at a relatively low magnification and obtain an image 803 with respect to the image 801 captured at a relatively high magnification. First, the enlargement / reduction processing 805 enlarges or reduces the ROI or its surrounding area that is required in the subsequent integration. The enlargement ratio can be calculated from the imaging magnification of the images 801 and 802. Or you may calculate from what is reflected in both images, for example, the ratio of the size of the pattern 813 and the pattern 814, or the defect 811 and the defect 812. After that, an image 803 having the same magnification as the image 801 is generated by performing weighted addition of the image 801 and the enlarged or reduced image 806 by the load sum processing 807.

  In the present embodiment, the image 803 matched with the magnification of the image 801 is generated, but an image with a magnification different from both the image 801 and the image 802 may be generated. For example, in order to generate an image having a higher magnification than the image 801, the image 801 and 802 may be subjected to enlargement processing at a corresponding enlargement rate, and then weighted addition may be performed.

  FIG. 9 is a diagram illustrating an example of processing for integrating ROI and its surrounding area while deforming the shape. An image 901 is an image captured without being tilted, and an image 902 is a tilt image. A defect 911 is present on the side wall 913 of the pattern 912, and the defect is difficult to see in the image 901. In the tilt image 902, the side wall portion 915 corresponds to a wide area on the image, and the visibility of the defect 914 is better than that of the image 901. The two images are integrated by processing 904 to create an image 903 in which the defect 916 can be clearly seen. As a result of classification of ROI, for example, a result of foreign matter on the side wall is obtained. The image 901 is used as a base image so that the image 903 looks similar to the image 901 that is not tilted. In the integration process 904, first, after performing shape deformation for each of the images 901 and 902 by the shape deformation processes 905 and 906, a load sum process 909 is performed. In the image 901, since the side wall region is too narrow, it is difficult to generate an image in which the defect 914 is clear by an integration method in which weighted addition is performed after the size of the side wall portion of the image 902 is matched to the image 901. Therefore, the shape deformation process is also performed on the image 901 to widen the side wall region. Images 907 and 908 are the outputs of the shape transformation processes 905 and 906, respectively. In the image 907, the region of the side wall portion 918 is expanded with respect to the image 901, and the size of the defect 917 is also increased. However, since the image 907 is created from the image 901 by image processing, the image quality of the defective portion is poor. Therefore, the image quality of the defect is enhanced by weighted addition with the image 908. In the shape transformation process 906, the shape transformation is performed so that the position and area of the side wall 920 and the defect 919 of the image 908 are matched with the image 907. Note that misalignment often occurs between the images 901 and 902. For this reason, positional deviation correction may be performed as preprocessing for at least one of the images.

  When integrating an image that is not tilted and a tilted image, problems such as discontinuity of joints in the integrated image occur unless the shape is appropriately deformed as in the example of FIG. On the other hand, the integration can be naturally performed by deforming the shape of the ROI and the surrounding area as in this embodiment.

  Integrating an image that is not tilted and a tilted image is advantageous from the viewpoint of resolution. When a sample having a height that is higher than the depth of focus is imaged, a captured image that is in focus at all positions cannot be obtained, and an image with poor resolution is obtained at a certain position. In the tilt image, since the sample is imaged from an oblique direction, the larger the tilt angle, the higher the sample, and it becomes difficult to obtain an image with high resolution in focus at all positions. Therefore, by integrating images so that ROI uses tilted images and images that are not tilted in other regions, ROI has good visibility and high-resolution images in other regions. Can be generated.

  FIG. 10 is a diagram illustrating an example of processing for integrating two images obtained by imaging different areas. An image 1002 is an image obtained by capturing an area including a defect. The image 1001 is an image obtained by capturing a region different from the image 1002 including the same pattern as the image 1002. As the ROI classification result, for example, a result such as a foreign substance on the pattern is obtained. The image 1001 is acquired under imaging conditions different from those of the image 1002, and the amount of noise is relatively small. A speckled pattern seen in a region other than the ROI of the image 1002 indicates that a lot of noise is superimposed. The image 1001 may be, for example, a captured image on another chip in the semiconductor wafer or a captured image on the same chip. An image (golden image) in which the amount of noise is reduced by taking a plurality of patterns having the same shape and taking the addition average. The images 1001 and 1002 are integrated by processing 1004 to generate an image 1003 with a small amount of noise including ROI. The corresponding patterns are shown in the images 1001 and 1002, but misalignment occurs because different areas are imaged. Therefore, first, a position adjustment process 1005 is performed to correct the position of the image 1002 so as to match the image 1001. Thereafter, the load sum processing 1006 is performed. In the load sum processing 1006, a large weight is set for the image 1008 after alignment in ROI. In other areas, it is possible to generate a good image with a smaller amount of noise than the image 1001 by setting an appropriate weight for the images 1001 and 1002 in accordance with the amount of noise included in the images. For example, the weights corresponding to the images 1001 and 1002 are set so as to be inversely proportional to the square of the standard deviation of noise included in each image.

  Note that the amount of noise in the image 1001 may be equal to or greater than that of the image 1002. Also in this case, an image with less noise can be obtained by using both information of the images 1001 and 1002 in the region other than the ROI.

  Next, an ROI extraction process corresponding to step 102 in FIG. 1 will be described with reference to FIGS.

  FIG. 11 is a diagram illustrating an example of extracting ROI included in two captured images. The captured images 1101 and 1102 show a pattern, an elliptical elongated defect 1111, and a circular defect 1112. However, the defect 1111 shows the image 1101 more clearly, and the defect 1112 shows the image 1102 more clearly. ROI extraction processing 1105 is performed on these two images. This process includes an ROI type identification process 1106 and an area extraction process 1107. In the ROI type specifying process 1106, for example, the ROI is specified as a defective area and a lower layer area. This ROI type identification process may be performed based on information such as user information, design data, and process information. For example, when information indicating that the user is interested only in the lower layer area is input, the ROI may be determined only for the lower layer area. Further, the ROI type specifying process may be performed based on an optical image or the like. The type of ROI may be specified based on the captured image. Next, region extraction processing 1107 is performed. For example, when the ROI has a defect, this processing can be performed by comparing an image (reference image) obtained by imaging another region in which a similar pattern is captured with a captured image. When the ROI is a lower layer region, the region extraction can be performed by comparing the design data and the captured image, or extracting a region with low brightness from the captured image. Examples of ROI extraction results are shown in images 1103 and 1104. The two defects 1121 and 1122 included in the image are extracted as the images that are clearly visible. As in this example, in a plurality of captured images to be integrated, a plurality of ROIs may exist, and different regions are extracted for each captured image, such as a defect 1121 in the image 1103 and a defect 1122 in the image 1104. You may do it.

  FIG. 12 is an embodiment diagram different from FIG. 11 showing the ROI extraction processing. The ROI is extracted from the two captured images 1201 and 1202. Assume that ROIs to be extracted are the defect 1211 of the image 1201 and the lower layer region 1212 of the image 1202. In this embodiment, only the line segment AB in the image will be described. Graphs 1203 and 1204 are lightness values in line segments AB of images 1201 and 1202, respectively. A region extraction process 1205 is performed using this information. Comparing the graphs 1203 and 1204, the contrast between the upper layer region and the lower layer region is higher in the graph 1203. Therefore, the lower layer can be extracted more reliably by using the graph 1203. However, the width of the lower layer region of the image 1211 is narrower than that of the image 1202. Therefore, after the lower layer is roughly extracted using the image 1201, fine extraction is performed using the image 1202. A graph 1206 shows the ROI degree in the line segment AB. In FIG. 11, the ROI extraction result is an image having a binary value (that is, image data expressing only whether or not each pixel is ROI), but in the example of FIG. 12, the ROI degree is a continuous value from 0 to 1 Takes a value. If the ROI degree is a continuous value (or a discrete value with a fine step size), weighted addition processing using continuous weights can be performed in the subsequent integration processing, so that the ROI boundary looks more natural. Can be processed.

  FIG. 13 is a diagram showing an example of the flow of ROI classification processing corresponding to step 103 in FIG. In this process, the set of captured images and the ROI degree are used as an input 1301, and a classification result 1302 is output. In this example, an area having a high ROI degree is classified by performing processes 1311 to 1317. First, the pattern / defect determination process 1311 identifies whether the object shown in the ROI is a pattern or a defect. As a result, when it is determined as a defect, a defect type determination process 1312 is performed. The pattern / defect is determined by, for example, calculating the difference between an image (reference image) obtained by imaging another area in which a similar pattern is reflected and the captured image. If the difference is large, the pattern is determined as a defect. Can be done. Alternatively, a method such as machine learning can be used. A defect type determination method is disclosed in, for example, Japanese Patent Application Laid-Open No. 2000-99727. Further, the height relation determination processing 1313 identifies the height relation of a plurality of objects existing in the ROI and its surrounding area. An example of this identification method will be described later with reference to FIG. In transparency determination 1314, the transparency of the object in the ROI is determined. For example, the defect 311 shown in FIG. 3 has almost zero transparency, and the object under the defect is not reflected, whereas the uneven defect 324 is highly transparent. The transparency can be determined by performing, for example, processing 503 for estimating an image having no defect in FIG. 5 and comparing the result. In the dark part / bright part determination processing 1315, it is determined whether it is a dark part such as a lower layer area or a bright part such as an upper layer pattern according to the lightness value. In the shadow area inside / outside determination processing 1316, it is determined whether or not the ROI is inside the shadow area of the upper layer pattern in the vicinity. In a brightness difference determination process 1317 with the ROI peripheral area, a difference in representative brightness value with the ROI peripheral area is calculated. The representative brightness value may be the average of the brightness values or may be an amount obtained from the average of the brightness values and the standard deviation.

  In this way, by classifying ROIs based on a plurality of determination criteria, it is possible to perform appropriate integration processing for each type of ROI.

  The results of the processes 1311 to 1317 may be discrete values or continuous values. For example, the pattern / defect determination process 1311 returns a binary value, but the transparency determination process 1313 may be a continuous value representing the transparency. Further, each pixel or each small area in the ROI may be determined. The type of classification can vary depending on the subject. For example, the pattern / defect determination 1311 is unnecessary if the sample including the defect is not a target, and the shadow area inside / outside determination 1316 is not required if the image does not have a shadow. The classification items may be selected by the user through an interface described later, or may be automatically switched for each process.

  FIG. 14 is a diagram illustrating an example of processing for identifying the height relationship of a plurality of objects existing in the ROI. This will be described using an example of two images 1401 and 1402. In both images, regions 1412 and 1414 in which a defect is shown are defined as ROI. The ROI includes patterns 1411 and 1413 in addition to defects. Defects are foreign objects with almost zero transparency. First, ROI and surrounding area extraction processing 1403 is performed. This can be realized, for example, by extracting a region having a high ROI degree and a region in the vicinity thereof by using the ROI degree as a result of the ROI extraction process. Thereby, a region 1415 and a region 1416 can be extracted for both images. Next, processing 1405 for identifying the height relationship is performed. In the region 1415, since the pattern 1417 is not visible inside the defect region, it can be seen that the defect is on the upper side. On the other hand, in the region 1416, since the pattern 1418 appears to be connected inside the defect region, it can be seen that the defect is on the lower side. In this way, the determination can be made on the basis of information on whether or not the patterns appear to be connected.

  By classifying ROI using such identification, it is possible to perform integration that preserves information related to the height of the object, so that a natural image can be provided. For example, the integration process can be performed so that the defect appears to exist above the pattern in the image 1401 and the defect appears to exist below the pattern in the image 1402. .

  For identification of the height relationship, a method different from the example of FIG. 14 may be used. For example, if a shadow image can be used, the height relationship can be obtained by looking at how the shadow is added. Alternatively, the height relationship can be clarified using design data.

  FIG. 15 is a diagram illustrating an example of a process for identifying which region of ROI an upper layer pattern region (hereinafter referred to as an upper layer region), a lower layer region, and a pattern sidewall region (hereinafter referred to as a sidewall region) belongs to. An image 1501 is an image to be identified. An area of three defects A to C included in the image is defined as ROI. First, an upper layer / lower layer / side wall extraction process 1503 separates an image into an upper layer region, a lower layer region, and a sidewall region. Similar to the case of FIG. 14, this process can be performed using a shadow method and design data. Alternatively, a region having a relatively high brightness value can be identified as an upper layer region, and a region having a relatively low brightness value can be identified as a lower layer region. In the example of the image 1501, there is no side wall region, and the upper layer region and the lower layer region are separated as in the image 1502. Next, the ROI upper layer / lower layer / side wall identification processing 1504 identifies which region each ROI belongs to. In order to perform this process, it is only necessary to determine in which region each ROI exists using the image 1502. In the example of the image 1501, the defect A belongs to the upper layer region, and the defect B belongs to the lower layer region. The defect C belongs to both the upper layer region and the lower layer region.

  When the ROI is present in the lower layer region or the side wall region, in order to improve the visibility, it is often necessary to perform processing more advanced than the weighted addition processing. For example, when the ROI is present in the lower layer region, charged particles emitted from the sample are shielded by the upper layer pattern, making it difficult to reach the detector, and the contrast is often lowered. In order to realize ROI for such an image, not only weighted addition but also brightness adjustment is required. In addition, when the ROI exists in the side wall region, the side wall region is a small region in the image, and thus visibility is often not good. When the ROI is in the side wall region, the charged particle beam may be irradiated from an oblique direction (that is, tilted) instead of directly above the sample, and an image may be captured. At this time, the visibility of the ROI can be improved by integrating the tilt image with the image captured without tilting. However, before integration, it is necessary to appropriately expand and contract in order to associate the pixels of each image. If the ROI is present in the upper layer region, the lower layer region, or the sidewall region as in the embodiment of FIG. 15, an appropriate integration method can be selected according to the result.

  FIG. 16 is a diagram illustrating an example of processing for controlling the integration method according to the classification result. In this processing, a classification result 1601 and a set of captured images 1605 are input, and an integrated image 1607 is obtained. The relationship between the classification result and the integrated method to be applied is previously stored as data in the table 1603 or the like. First, in accordance with the given classification result 1601, which integration method is used is determined by the switch 1604. Also, parameters necessary for the integration process are determined by the parameter setting process 1602 according to the classification result. Also in the parameter setting process 1602, the relationship between the classification result and the parameter to be applied is previously stored as data such as a table, and a corresponding value is read according to the classification result. The parameter setting process 1602 may be performed using a captured image. For example, when it is desired to perform brightness value adjustment processing, it is often possible to set an appropriate value by using a target captured image, such as adjusting so that the average brightness values of a certain region are aligned. Finally, the image set 1605 is integrated by the integration process 1606.

  The classification result may include a continuous value. In this case, the switch process 1604 and the parameter setting process 1602 may include a process for calculating which method and parameter to use among data such as a table using a classification result including continuous values. Note that one or more ROIs are not always present, and depending on the captured image, there may be no ROIs. In that case, an appropriate process can be performed by returning a classification result indicating that there is no ROI and using an integration method suitable for that case.

  FIG. 17 is a diagram illustrating an example of processing for controlling the integration method according to the classification result. In FIG. 16, the processing for controlling the integration method for each classification result using data such as a table has been described, but in this embodiment, the integration method is controlled for each classification result based on a predetermined rule. In this example, it is determined by processing 1702-1705 whether ROI is a defect and whether it is above the pattern, whether the transparency is high, whether it is a shadow of the upper layer pattern, or whether the relative brightness is high, The integration method is changed according to the result. First, it is determined whether or not the pattern is above the pattern by the determination process 1702, and when it is determined that the pattern is not above the pattern, the load sum process 1711 with the ROI difference information as described in the embodiment diagram of FIG. I do. If it is determined that the pattern is located above the pattern, it is next determined whether or not the transparency is high by a determination process 1703. This process may be determined based on, for example, whether the transparency exceeds a certain threshold value. If it is determined that the transparency is high, processing 1711 is performed. On the other hand, if it is determined that the transparency is not high, it is determined by the determination process 1703 whether the ROI is in the shadow area of the upper layer pattern. When it is determined that it is not in the shadow area, the load sum processing 1712 as described in the embodiment diagram of FIG. 4 is performed. If it is determined that the area is in the shadow area, it is determined in the determination process 1705 whether or not the relative brightness with the ROI surrounding is high. When it is determined that the relative lightness is high, the load sum processing 1712 is performed. On the other hand, when it is determined that the relative lightness is low, a process 1713 for calculating the load sum is performed after the lightness adjustment as described in the embodiment diagram of FIG.

  FIG. 18 is a diagram illustrating an example of a user interface for setting an integration method for each classification result. In a tab 1801, settings relating to an integrated image generation method using each image as a base image can be switched. In this example, four captured images of a secondary charged particle image (SP image) and a backscattered charged particle image (BSP image) captured without tilting, and an SP image and a BSP image captured with tilting. Thus, a screen for setting an integration method for creating an integrated image using the SP image as a base image is displayed. If the tab 1801 is set as a BSP image, an integration method for generating an integrated image using the BSP image as a base image from the same four captured images can be designated. An arbitrary number of integrated images can be generated from the four captured images. Two integrated images using the SP image as a base image may be generated. The screen for setting the integration method includes a cell 1812 for specifying the target classification result, a cell 1813 for specifying the corresponding integration method, and a cell 1811 for specifying the priority of the setting. Further, there are a determination button 1821 for determining the setting and a clear button 1822 for returning the setting to the default value.

  In the row 1831, when the classification result is a scratch defect, the ROI difference in the BSP image (the description “sub (•)” in the cell represents the function for obtaining the ROI difference) is multiplied by a with respect to the SP image. The value obtained by adding the values is used as the integrated result in ROI. This processing corresponds to the integration method described in the embodiment diagram of FIG. a is a parameter, and a value separately designated by the user may be used, or may be automatically determined based on an evaluation index such that the contrast in ROI is a certain value or more. An integration method in a region other than the ROI is not described in the present embodiment diagram, but it may be set through an interface in the same manner, or may be controlled automatically. For example, in the case of line 1831, the SP image that is the base image may be used as it is in the region other than the ROI, and corresponding processing may be performed when shape deformation or brightness adjustment is necessary. In the row 1832, when the classification result is a dark part, the image in which the brightness adjustment is performed on the SP image (the description “contr (•)” in the cell represents a function for adjusting the brightness) and the BSP image The image with the higher average brightness value (the description "argmax (•)" in the cell represents the function that selects the image with the higher average brightness value) is used as the result of ROI integration. Represents what to do. In the 1831 row and the 1832 row, the 1831 row has a higher priority. Therefore, if there is a scratch defect in the dark part, the integration method specified in the 1831 row is applied. In the row 1833, when the classification result is in the pattern side wall region and the ROI is extracted from the tilt image, an image obtained by deforming the tilted SP image (the description “distort (•)” in the cell is the shape (Represents a function to be deformed) as an integrated result in ROI. At this time, as described in the embodiment diagram of FIG. 9, since the ROI may be easier to see if the shape of the base image is deformed, the shape of the base image is automatically deformed as necessary. In the row 1834, when the classification results are all other cases, the sum of the image obtained by multiplying the SP image by b and the image obtained by multiplying the ROI by 1-b is used as the integrated image at the ROI. b is a parameter. Thus, by providing an interface for designating an integration method for each classification result, the integration method can be controlled by a method desired by the user.

  The integration method may be provided with a function that can specify details as required. On the other hand, even when detailed designation is not performed, the processing may be performed to some extent automatically. For example, the parameter a in the line 1831 can be processed so as to automatically set an appropriate value without being specified as described above. Similarly, in the brightness adjustment in the 1832 line and the shape deformation in the 1833 line, if a specific parameter is not specified, the process can be automatically set.

  In addition, the interface screen has an interface for adding / deleting / modifying tags, an interface for increasing / decreasing the number of cell rows, and an interface for writing / reading settings to / from a file. Also good. Further, an interface for performing detailed settings such as parameters may be provided.

  FIG. 19 is an example of a sequence for automatically capturing a high-magnification image including a defect and obtaining an integrated image thereof. In order to automatically capture a high-magnification image including a defect, it is necessary to obtain the defect position with high accuracy. For this purpose, it is often necessary to perform preprocessing in which a low-magnification image is first taken and a defect position is obtained from the low-magnification image to obtain a defect position with high accuracy. Therefore, first, in step 1901, a low-magnification reference image 1911 is acquired. The reference image may be an image obtained by imaging a corresponding position on another chip in the semiconductor wafer, or may be an image obtained by imaging a position where a pattern having the same shape on the same chip exists. Next, a low-magnification defect image 1912 is acquired in step 1902, and a defect position is specified using the low-magnification reference image 1911 and the low-magnification defect image 1912 in step 1903. Thereafter, in step 1904, a high-magnification defect image is acquired. In step 1905, ROI is extracted, and in step 1906, ROI is classified. In step 1907, the images are integrated based on the classification result, and finally, in step 1908, the integrated image is displayed. A method for acquiring a high-magnification defect image is disclosed in, for example, Japanese Patent Laid-Open No. 9-139406. Before displaying the integrated image, the integrated image may be stored in a database or the like, and after repeating the processing of steps 1901 to 1907 for each defect position to be observed, the integrated image may be displayed collectively at the end. .

  When extracting the ROI in step 1905 or classifying the ROI in step 1906, not only the high-magnification defect image but also the low-magnification reference image 1911 or the multiplication defect image 1912 may be used. Further, when integrating images in step 1907, a plurality of high-magnification defect images may be integrated, or either one or both of the low-magnification reference image 1911 and the low-magnification defect image 1912 may be integrated with the high-magnification defect image. They may be integrated using only one or both of the low-magnification reference image 1911 and the low-magnification defect image 1912. Or you may integrate using the other image (Low magnification reference image, Low magnification defect image, or High magnification defect image) acquired when another defect position was imaged.

  As described in the above embodiments, the present invention solves the above problems by the following image display method and a charged particle microscope apparatus equipped with the image display method.

  (1) In the present invention, a plurality of images are captured, ROIs are extracted and classified from the captured images, and then the plurality of captured images are integrated and displayed. At this time, the integration method is controlled according to the ROI classification result.

  As a result, even if the appearance of the image differs greatly for each ROI, it is possible to automatically select and integrate the method suitable for each ROI, and always provide an image with good ROI image quality. It becomes possible.

  (2) Further, the present invention is characterized in that the height relationship of a plurality of objects existing in the ROI is identified in the ROI classification.

  Here, the closer to the charged particle source, the higher the position, and the farther away, the lower the position. For example, when the ROI includes a hole pattern, the hole bottom is at a relatively low position and the area around the hole is at a relatively high position. In the present invention, it is possible to provide a natural image in which information on the height of the object remains, by performing integration after identifying the height relationship of the object. As a specific example, let us consider a case where an ROI is a defect such as dust (hereinafter, foreign matter) in an image obtained by imaging an oxide film. When the foreign matter is on the oxide film, the pattern directly under the foreign matter is usually not visible in the region where the foreign matter is reflected. On the other hand, when the foreign substance is under the oxide film, the oxide film appears to rise in the ROI due to the foreign substance. In order to make a natural image, it is necessary to combine the images while leaving such features, and this can be done by classifying the ROI using the height relation.

  (3) Further, according to the present invention, in the classification of ROI, the ROI is identified as one of at least two of an upper layer pattern region, a lower layer region, and a pattern side wall region.

  When the ROI is present in the lower layer region or the pattern side wall region, in order to improve the visibility, it is often necessary to perform processing more advanced than the weighted addition processing. For example, when the ROI exists in the lower layer region, charged particles emitted from the sample are shielded by the upper layer pattern, so that it becomes difficult to reach the detector, and the contrast generally decreases. In order to realize ROI for such an image, not only weighted addition but also brightness adjustment is required. In addition, when the ROI exists in the pattern side wall region, the pattern side wall region often corresponds to a small region in the image, and thus visibility is often not good. When the ROI is in the pattern sidewall region, an image may be taken by irradiating a charged particle beam from an oblique direction (that is, tilted) rather than from directly above the sample. At this time, the visibility of the ROI can be improved by integrating the tilt image with the image captured without tilting. However, before performing integration, it is necessary to appropriately expand and contract the images in order to associate the pixels of each image. As described above, an appropriate integration method differs depending on whether the ROI exists in the upper layer pattern region, the lower layer region, or the pattern side wall region. However, according to the present invention, the region in which the ROI exists is classified and integrated. By controlling, appropriate integration methods can be applied.

  (4) In addition, the present invention is characterized in that, based on shape information between a plurality of captured images in the peripheral region of the ROI, the ROI and the peripheral region of the ROI are integrated after being deformed with respect to the captured image. To do.

  Among a plurality of captured images, what is reflected for each pixel may be different due to image distortion. An image that is not tilted and a tilted image are easy to understand. In addition, even in images captured by different types of detectors, for example, at the boundary of the pattern, information on the upper layer pattern is dominant in one image, and information on the lower layer region is dominant in the other image. There are many cases. In this case, the width of the lower layer area looks different depending on the image. In these cases, if the shape is not appropriately deformed in advance, when a plurality of captured images are integrated, problems such as discontinuities in the boundaries occur. At this time, it is necessary to transform not only the ROI but also the surrounding area. On the other hand, the present invention can naturally integrate even in the case of a captured image with image distortion.

  (5) Further, the present invention is characterized by preserving and integrating the ROI and the light-dark relationship between the ROI and the surrounding area.

  The brightness value of each region of the image may vary greatly between the plurality of captured images. For example, when different types of detectors are used, the brightness values generally differ greatly because the types and characteristics of the particles detected by each detector are different. Also, the brightness value varies depending on the position of the detector, because the number of particles that reach each detector changes. For this reason, with simple weighted addition, light and dark may be reversed and it may appear unnatural. On the other hand, in the present invention, the brightness relationship between the ROI and the surrounding area of the ROI is preserved by adjusting the brightness during the integration process. As a result, natural integration that suppresses the reversal of light and dark is possible.

  According to the present invention shown in the above (1) to (5), in the charged particle microscope, after capturing a plurality of images, extracting and classifying the ROI from the captured images, integrating the plurality of captured images. When displaying, by controlling the integration method according to the classification result, it is possible to provide a scanning charged particle microscope device that provides an image in which each ROI can be easily seen and its image display method.

Claims (12)

An image acquisition step of capturing a sample and acquiring a plurality of captured images;
An observation target region extraction step for extracting one or more observation target regions from the plurality of captured images acquired in the image acquisition step;
An observation area classification step for classifying the one or more observation areas extracted in the observation area extraction step;
An image integration step of generating an integrated image by integrating the plurality of captured images by an integration method determined based on the classification result of the observation target region classification step;
An observation method comprising: In the observation target region classification step, the one or more observation target regions are classified based on information on the heights of a plurality of objects existing in the observation target region,
The observation method according to claim 1, wherein the image integration step determines an integration method based on information on the height of the object. In the observation target region classification step, the observation target region is classified as to which of at least two regions of the upper layer pattern region, the lower layer region, and the pattern sidewall region belong,
The observation method according to claim 1, wherein the image integration step determines an integration method based on a classification result of the observation target region classification step.   The image integration step includes: observing the observation target region and the observation target region for at least one of the plurality of captured images based on shape information between the plurality of captured images in a peripheral region of the observation target region. The observation method according to claim 1, wherein the peripheral regions are integrated after being deformed.   5. The observation method according to claim 1, wherein the image integration step preserves and integrates a light-dark relationship between the observation target region and a peripheral region of the observation target region.   The observation method according to claim 1, further comprising a display step of displaying the integrated image generated by the image integration step. An image acquisition unit that captures a sample and acquires a plurality of captured images;
An observation target region extraction unit that extracts one or more observation target regions from the plurality of captured images acquired by the image acquisition unit;
An observation target region classification unit that classifies the one or more observation target regions extracted by the observation target region extraction unit;
An image integration unit that generates an integrated image by integrating the plurality of captured images by an integration method determined based on a classification result of the observation target region classification unit;
An observation apparatus comprising: The observation target region classification unit classifies the one or more observation target regions based on information on the heights of a plurality of objects existing in the observation target region,
The observation apparatus according to claim 7, wherein the image integration unit determines an integration method based on information on the height of the object. The observation target region classification unit classifies whether the observation target region belongs to at least two of the upper layer pattern region, the lower layer region, and the pattern sidewall region,
The observation apparatus according to claim 7, wherein the image integration unit determines an integration method based on a classification result of the observation target region classification unit.   The image integration unit, based on shape information between the plurality of captured images in a peripheral region of the observation target region, the observation target region and the observation target region for at least one of the plurality of captured images The observation apparatus according to claim 7, wherein the peripheral regions are integrated after being deformed.   The observation apparatus according to claim 7, wherein the image integration unit stores and integrates a light-dark relationship between the observation target region and a peripheral region of the observation target region.   The observation apparatus according to claim 7, further comprising a display unit that displays the integrated image generated by the image integration unit.

Images