WO2020179032A1 - Microscope and observation method - Google Patents

Abstract

This microscope is provided with: an illumination optical system which has a light flux splitting unit that splits light from a light source into a plurality of light fluxes, and which scans, in a plurality of directions of a sample, interference fringes generated by interference between at least three light fluxes among the plurality of light fluxes split by the light flux splitting unit; a detection optical system on which light from the sample is incident; a detection device including a plurality of detection units that detects light from the sample via the detection optical system; and an image processing unit that generates an image using the detection results from the two or more detecting units of the detection device.

Description

Microscope and observation method

The present invention relates to a microscope and an observation method.

A scanning microscope that detects fluorescence from a sample has been proposed (for example, see Non-Patent Document 1 below).

Confocal laser scanning microscope with spatiotemporal stimulated illumination, Peng Gao, G.M. Ulrich Nienhaus, Optics Letters, Vol. 41, No. 6, 1193-1196, 2016.3.15

According to the first aspect of the present invention, it has a luminous flux dividing portion that divides the light from the light source into a plurality of luminous fluxes, and is formed by interference of at least three or more luminous fluxes of the plurality of luminous fluxes divided by the luminous flux dividing portion. It includes an illumination optical system that scans the interference fringes to be generated in a plurality of directions of the sample, a detection optical system in which light from the sample is incident, and a plurality of detection units that detect light from the sample via the detection optical system. There is provided a microscope including a detection device and an image processing unit that generates an image using detection results of two or more detection units of the detection device.

According to the second aspect of the present invention, the light from the light source is divided into a plurality of luminous fluxes, and the interference fringes formed by the interference of at least three or more luminous fluxes of the plurality of luminous fluxes are scanned in a plurality of directions of the sample. The light from the sample is detected by a detection device including a plurality of detection units via the detection optical system in which the light from the sample is incident, and the detection results of two or more detection units of the detection device are obtained. And generating an image therewith is provided.

It is a figure which shows the microscope which concerns on 1st Embodiment, and the optical path of excitation light. It is a figure which shows the mask and polarizer concerning 1st Embodiment. It is a figure which shows the mask which concerns on 1st Embodiment, a polarizer, an interference fringe, and the polarization state of excitation light. It is a figure which shows the microscope and the optical path of fluorescence which concern on 1st Embodiment. It is a figure which shows the effective PSF at each position of the detection apparatus which concerns on 1st Embodiment. It is a figure which shows the 3D simulation result of the illumination shape and effective PSF of this embodiment which concerns on 1st Embodiment, and 3D simulation result of PSF of a conventional fluorescence microscope. It is a flowchart which shows the observation method which concerns on 1st Embodiment. It is a figure which shows the process of the image processing part of the microscope which concerns on 2nd Embodiment. It is a flowchart which shows the observation method which concerns on 2nd Embodiment. It is a figure which shows the process of the image processing part of the microscope which concerns on 3rd Embodiment. In 3rd Embodiment, it is a figure which shows the area|region of the frequency space used for component separation. It is a flowchart which shows the observation method which concerns on 3rd Embodiment. It is a flowchart which shows the observation method which concerns on 4th Embodiment. It is a figure which shows the area|region of the frequency space used for component separation which concerns on 5th Embodiment. It is a flowchart which shows the observation method which concerns on 5th Embodiment. It is a figure which shows the microscope which concerns on 6th Embodiment. It is a figure which shows an example of the phase modulation element which concerns on 6th Embodiment. It is a flowchart which shows the observation method which concerns on 6th Embodiment. It is a figure which shows the microscope which concerns on 7th Embodiment. It is a figure which shows the microscope which concerns on 7th Embodiment. It is a figure which shows the microscope which concerns on 8th Embodiment. It is a figure which shows the microscope which concerns on 9th Embodiment. It is a figure which shows the microscope which concerns on 10th Embodiment. It is a figure which shows the mask which concerns on 10th Embodiment, and the polarization state of excitation light. It is a figure which shows the microscope which concerns on 11th Embodiment. It is a figure which shows the polarization state of the excitation light which concerns on 11th Embodiment. It is a figure which shows the microscope which concerns on 12th Embodiment. It is a figure which shows the mask which concerns on 12th Embodiment. It is a figure which shows the microscope which concerns on 13th Embodiment. It is a figure which shows the illumination pupil which concerns on a modification. It is a figure which shows the illumination pupil which concerns on a modification. It is a figure which shows the microscope which concerns on the modification. It is a figure which shows the polarization adjustment part which concerns on the modification. It is a figure which shows the polarization adjustment part which concerns on the modification.

[First Embodiment]
The first embodiment will be described. FIG. 1 is a diagram showing a microscope and an optical path of excitation light according to the first embodiment. In the following embodiments, the microscope 1 will be described as being a scanning fluorescence microscope, but the microscope according to the embodiment is not limited to a scanning microscope or a fluorescence microscope. The microscope 1 includes a stage 2, a light source 3, an illumination optical system 4, a detection optical system 5, a detection device 6, and a control device CU. The microscope 1 operates as follows.

Stage 2 holds the sample S to be observed. The sample S is a cell or the like that has been fluorescently stained in advance. The sample S contains a fluorescent substance such as a fluorescent dye. The light source 3 emits excitation light L1 that excites the fluorescent substance contained in the sample S. The illumination optical system 4 scans the interference fringes L2 formed by the excitation light L1 in a plurality of directions (for example, the X direction and the Y direction) with respect to the sample S. The stage 2 can move the relative positions of the sample S and the illumination optical system 4 in the X direction and the Y direction in FIG. 1 in a range wider than the scanning range of the interference fringes L2 by the illumination optical system 4. The detection optical system 5 is arranged at a position where the fluorescence L3 (shown later in FIG. 4) from the sample S enters. The detection device 6 includes a plurality of detection units 6a (later shown in FIG. 4) that detect the fluorescence L3 from the sample S via the detection optical system 5. The control unit CU is, for example, a computer and includes a control unit CB and an image processing unit 7. The control unit CB controls the stage 2, the light source 3, the illumination optical system 4, the detection optical system 5, and the detection device 6. The image processing unit 7 generates an image (for example, a super-resolution image) by using the detection results of two or more detection units 6a of the detection device 6. Hereinafter, each part of the microscope 1 will be described.

The light source 3 includes a light source such as a laser element. The light source 3 generates coherent light in a predetermined wavelength band. The predetermined wavelength band is set to a wavelength band including the excitation wavelength of the sample S. The predetermined wavelength band may be set to a wavelength band in which the sample S is excited by multiphotons. The excitation light L1 emitted from the light source 3 is, for example, linearly polarized light. A light guide member such as an optical fiber 11 is connected to the emission port of the light source 3. The microscope 1 does not have to include the light source 3, and the light source 3 may be provided separately from the microscope 1. For example, the light source 3 may be provided interchangeably (attachable, removable) with the microscope 1. The light source 3 may be externally attached to the microscope 1 when observing with the microscope 1.

The illumination optical system 4 is arranged at a position where the excitation light L1 from the light source 3 is incident. Excitation light L1 is incident on the illumination optical system 4 from the light source 3 through the optical fiber 11. The optical fiber 11 may be a part of the illumination optical system 4 or a part of a light source device including the light source 3. The illumination optical system 4 includes a collimator lens 12, a λ/4 wave plate 13, a polarizer 14, a mask 15 (aperture member), a dichroic mirror 16, a relay optical system 17, and a scan in order from the light source 3 side to the sample S side. A unit 18, a lens 19, a lens 20, and an objective lens 21 are provided.

In the following explanation, the XYZ Cartesian coordinate system shown in FIG. In this XYZ orthogonal coordinate system, the X direction and the Y direction are directions perpendicular to the optical axis 21a of the objective lens 21, respectively. Further, the Z direction is a direction parallel to the optical axis 21a of the objective lens 21. The optical axis 21a of the objective lens 21 is included in the optical axis 4a of the illumination optical system 4. Further, in each of the X direction, the Y direction, and the Z direction, the same side as the arrow is referred to as a + side (for example, +X side), and the opposite side to the arrow is referred to as a − side (for example, −X side). Further, when the optical path is bent by reflection, the directions corresponding to the X direction, the Y direction, and the Z direction are indicated with subscripts. For example, the Xa direction, the Ya direction, and the Za direction in FIG. 1 are directions corresponding to the X direction, the Y direction, and the Z direction in the optical path from the collimator lens 12 to the dichroic mirror 16, respectively.

The collimator lens 12 converts the excitation light L1 emitted from the optical fiber 11 into parallel light. The collimator lens 12 is arranged, for example, so that the focal point on the same side as the light source 3 coincides with the light emission port of the optical fiber 11. In the following description, regarding the lens included in the illumination optical system 4, the focus on the same side as the light source 3 is referred to as the rear focus, and the focus on the same side as the sample S is referred to as the front focus.

The λ / 4 wave plate 13 changes the polarization state of the excitation light L1 to circularly polarized light. The polarizer 14 is, for example, a polarizing plate, and has a characteristic of transmitting linearly polarized light having a predetermined polarization direction. The polarizer 14 is arranged such that the light incident on the sample S is S-polarized (linearly polarized in the Y direction). The polarizer 14 is rotatable around the optical axis 12a of the collimator lens 12. The optical axis 12a of the collimator lens 12 is included in the optical axis 4a of the illumination optical system 4.

The mask 15 is a light beam splitting unit that splits the excitation light that excites the fluorescent material into a plurality of light beams. The illumination optical system 4 scans the sample S with the interference fringes L2 generated by the interference of three or more light beams among the plurality of light beams divided by the mask 15. The mask 15 is arranged at or near a position of a pupil conjugate plane P1 which is optically conjugate with the pupil plane P0 of the objective lens 21. The vicinity of the pupil conjugate surface optically conjugate with the pupil surface P0 of the objective lens 21 is a range in which the excitation light L1 can be regarded as a parallel ray in the region including the pupil conjugate surface. For example, when the excitation light L1 is a Gaussian beam, it can be regarded as a sufficiently parallel light beam within a range of about 1/10 of the Rayleigh length from the beam waist position. The Rayleigh length is given by πw02/λ, where λ is the wavelength of the pumping light L1 and w0 is the beam waist radius. For example, when the wavelength of the excitation light L1 is 1 μm and the beam waist radius is 1 mm, the Rayleigh length is about 3 m, and the mask 15 is within 300 mm near the pupil conjugate plane P1 optically conjugate with the pupil plane P0 of the objective lens 21. It may be arranged. The mask 15 may be arranged at or near the pupil plane P0.

The mask 15 has an opening 15a, an opening 15b, and an opening 15c through which the excitation light L1 passes. An interference fringe L2 is formed by the interference of the excitation light L1a passing through the opening 15a, the excitation light L1b passing through the opening 15b, and the excitation light L1c passing through the opening 15c. The mask 15 is rotatable around the optical axis 12a of the collimator lens 12. The mask 15 is fixed, for example, relative to the polarizer 14 and rotates integrally with the polarizer 14. The mask 15 and the polarizer 14 are rotated by the torque supplied from the drive unit 22. The mask 15 and the polarizer 14 do not have to rotate integrally, and each may rotate independently by the torque supplied from the independent drive unit.

FIG. 2A is a diagram showing a mask according to the first embodiment. The openings 15a, 15b, and 15c of the mask 15 are arranged symmetrically with respect to the optical axis 12a of the collimator lens 12 (see FIG. 1), for example. The openings 15a, 15b, and 15c of the mask 15 do not have to be symmetrically arranged with respect to the optical axis 12a of the collimator lens 12 (see FIG. 1). In the state of FIG. 2A, the opening 15a, the opening 15b, and the opening 15c are aligned in the Xa direction. FIG. 2B is a diagram showing the polarizer according to the first embodiment. The transmission axis 14a of the polarizer 14 is perpendicular to the direction in which the openings 15a, 15b, and 15c are arranged in the mask 15 (Xa direction in FIG. 2A) (FIG. 2B). Then, it is set parallel to the Ya direction). FIG. 2C is a diagram showing the pupil plane P0 of the objective lens 21. The region P0a, the region P0b, and the region P0c are regions on which the excitation light L1 enters, respectively. The parameters shown in FIG. 2C will be referred to later in the description of the image processing unit 7.

Returning to the description of FIG. 1, the dichroic mirror 16 has a property that the excitation light L1 is reflected and the fluorescence L3 (later shown in FIG. 4) from the sample S is transmitted. The excitation light L1 that has passed through the openings 15a, 15b, and 15c of the mask 15 is reflected by the dichroic mirror 16, the optical path is bent, and enters the relay optical system 17. The relay optical system 17 guides the excitation light L1 from the dichroic mirror 16 to the scanning unit 18. Although the relay optical system 17 is represented by one lens in the figure, the number of lenses included in the relay optical system 17 is not limited to one. Further, the relay optical system 17 may not be necessary depending on the distance of the optical system and the like. In addition, in each drawing, two or more lenses may be represented by one lens even in a portion other than the relay optical system 17.

The scanning unit 18 scans the sample S in the interference fringes L2 formed by the excitation light L1 in two directions, the X direction and the Y direction. The scanning unit 18 changes the position where the interference fringe L2 is formed by the excitation light L1 in two directions that intersect the optical axis 21a of the objective lens 21. The scanning unit 18 includes a deflection mirror 18a and a deflection mirror 18b. The tilts of the deflection mirror 18a and the deflection mirror 18b with respect to the optical path of the excitation light L1 are variable. The deflection mirror 18a and the deflection mirror 18b are a galvano mirror, a MEMS mirror, a resonant mirror (resonance type mirror), etc., respectively. The deflection mirror 18a and the deflection mirror 18b may be scanners.

The deflection mirror 18a changes the position where the excitation light L1 is incident on the sample S in the X direction. The deflection mirror 18b changes the position on the sample S where the excitation light L1 is incident in the Y direction. The scanning unit 18 is arranged so that, for example, the position conjugate with the pupil plane P0 of the objective lens 21 is the position of the deflection mirror 18a, the position of the deflection mirror 18b, or the position between the deflection mirror 18a and the deflection mirror 18b. It is desirable to be done. Alternatively, the position conjugate with the pupil surface P0 of the objective lens 21 may be on the light source 3 side from the position of the deflection mirror 18a, or on the sample S side from the position of the deflection mirror 18b. The scanning unit 18 may be configured such that the deflection mirror 18a changes the position where the excitation light L1 is incident on the sample S in the Y direction and the deflection mirror 18b changes in the X direction.

The excitation light L1 from the scanning unit 18 is incident on the lens 19. The lens 19 concentrates the excitation light L1 on the sample conjugate surface Sb optically conjugate with the sample surface Sa of the objective lens 21. The sample surface Sa is a surface that is arranged at a position near the front focal point or the front focal point of the objective lens 21 and is perpendicular to the optical axis 21a of the objective lens 21. An interference fringe is formed on the sample conjugate surface Sb due to the interference of the excitation light L1a passing through the opening 15a of the mask 15, the excitation light L1b passing through the opening 15b, and the excitation light L1c passing through the opening 15c. To be done.

The excitation light L1 that has passed through the sample conjugate surface Sb is incident on the lens 20. The lens 20 converts the excitation light L1 into parallel light. The excitation light L1 that has passed through the lens 20 passes through the pupil surface P0 of the objective lens 21. The objective lens 21 focuses the excitation light L1 on the sample surface Sa. The lens 20 and the objective lens 21 project the interference fringes formed on the sample conjugate surface Sb onto the sample surface Sa. Local interference fringes L2 are formed on the sample surface Sa.

The interference fringe L2 includes a bright portion having a relatively high light intensity and a dark portion having a relatively low light intensity. The direction in which the bright part and the dark part are lined up (X direction in FIG. 1) is appropriately referred to as the periodic direction D1 of the interference fringes L2. The periodic direction D1 of the interference fringe L2 corresponds to the direction in which the opening 15a, the opening 15b, and the opening 15c of the mask 15 are arranged (Xa direction in FIG. 1). When the drive unit 22 rotates the mask 15 around the Za direction, the direction in which the opening 15a, the opening 15b, and the opening 15c are aligned is rotated, and the periodic direction D1 of the interference fringe L2 is around the Z direction. Rotate. That is, the driving unit 22 is included in the fringe direction changing unit that changes the direction of the interference fringe L2. The drive unit 22 (stripe direction changing unit) is a direction in which three or more light fluxes are arranged on a surface perpendicular to the optical axis 4a of the illumination optical system 4 (for example, a light exit side surface of the mask 15) (hereinafter, referred to as a light flux dividing direction). ) Is changed. The light beam splitting direction is, for example, a direction in which the opening 15a, the opening 15b, and the opening 15c are arranged side by side, and the drive unit 22 changes the light beam splitting direction by rotating the mask 15.

Further, when the mask 15 rotates around the Za direction, the direction in which the excitation light L1 enters the sample S changes. The drive unit 22 rotates the polarizer 14 in conjunction with the mask 15 to change the direction of the transmission axis of the polarizer 14 and adjusts the excitation light L1 to be S-polarized and incident on the sample S. That is, the polarizer 14 and the driving unit 22 are included in the polarization adjusting unit that adjusts the polarization state of the excitation light L1 based on the direction of the interference fringes.

FIG. 3 is a diagram showing a mask, a polarizer, interference fringes, and a polarized state of excitation light according to the first embodiment. In FIG. 3A, the direction in which the opening 15a, the opening 15b, and the opening 15c of the mask 15 are arranged is the Xa direction. The transmission axis 14a of the polarizer 14 is in the Ya direction perpendicular to the Xa direction. In this case, in the excitation light L1 (see FIG. 1), the light flux that has passed through the opening 15a, the light flux that has passed through the opening 15b, and the light flux that has passed through the opening 15c are incident on the sample S, and the periodic direction D1. Interference fringe L2 is generated. The excitation light L1 incident plane is parallel to the XZ plane. The excitation light L1 when incident on the sample S is in the Y direction whose polarization direction D2 is perpendicular to the incident surface, that is, the excitation light L1 is incident on the sample S with S polarization.

In FIG. 3B, the direction in which the openings 15a, 15b, and 15c of the mask 15 are arranged is the direction obtained by rotating the Xa direction counterclockwise by 120°. The transmission axis 14a of the polarizer 14 is a direction in which the Ya direction is rotated counterclockwise by 120 °. The periodic direction of the interference fringes L2 is a direction forming 120 ° with respect to the X direction. The incident surface of the excitation light L1 is a surface obtained by rotating the XZ plane by 120 ° around the Z direction. The excitation light L1 when incident on the sample S is in a direction in which the polarization direction D2 is perpendicular to the incident surface, that is, the excitation light L1 is incident on the sample S with S polarization.

In FIG. 3C, the direction Xa in which the opening 15a, the opening 15b, and the opening 15c of the mask 15 are lined up is rotated by 240 ° counterclockwise. The transmission axis 14a of the polarizer 14 is a direction in which the Ya direction is rotated counterclockwise by 240 °. The periodic direction D1 of the interference fringe L2 is a direction forming 240 ° with respect to the X direction. The incident surface of the excitation light L1 is a surface obtained by rotating the XZ plane by 240 ° around the Z direction. The excitation light L1 when entering the sample S has a polarization direction D2 perpendicular to the incident surface, that is, the excitation light L1 enters the sample S as S-polarized light.

As described above, when the excitation light L1 is incident on the sample S with S-polarized light, the contrast of the interference fringes L2 is higher than when it is incident with P-polarized light. In FIG. 3, the periodic direction of the interference fringe L2 is changed in three ways in increments of 120 °, but the periodic direction of the interference fringe L2 is not limited to this example. The periodic direction of the interference fringes L2 corresponds to a direction in which the resolution can be improved (a direction in which the super-resolution effect can be obtained) in the image generated by the image processing unit 7 described later. The periodic direction of the interference fringe L2 is appropriately set so that a desired super-resolution effect can be obtained. For example, the periodic directions of the interference fringes L2 may be two or one at an angle of 90 ° to each other. Further, the mask 15 may be replaceable according to the magnification and NA (numerical aperture) of the objective lens 21 and the shape of the illumination pupil.

FIG. 4 is a diagram showing the microscope and the optical path of fluorescence according to the first embodiment. The detection optical system 5 forms an image of the fluorescence L3 generated in the sample S. The detection optical system 5 includes an objective lens 21, a lens 20, a lens 19, a scanning unit 18, a relay optical system 17, a dichroic mirror 16, an excitation light cut filter (barrier filter) 24, and an order of going from the sample S to the detection device 6. The lens 23 is included. The fluorescence L3 generated in the sample S passes through the objective lens 21, the lens 20, and the lens 19 in this order, and enters the scanning unit 18. The fluorescence L3 is descanned by the scanning unit 18 and enters the dichroic mirror 16 through the relay optical system 17. The dichroic mirror 16 has the property of transmitting the fluorescence L3. The fluorescence L3 that has passed through the dichroic mirror 16 enters the excitation light cut filter (barrier filter) 24. The excitation light cut filter (barrier filter) 24 has a property of blocking the excitation light L1 and transmitting the fluorescence L3. When the excitation light L1 can be sufficiently shielded by the dichroic mirror 16, the excitation light cut filter (barrier filter) 24 may not be provided. The fluorescence L3 that has passed through the excitation light cut filter (barrier filter) 24 enters the lens 23. The lens 23 concentrates the fluorescence L3 on the detection device 6.

The detection device 6 is an image sensor and includes a plurality of detection units 6a arranged two-dimensionally. The plurality of detection units 6a are arranged in two directions in the detection device 6. The plurality of detection units 6a are arranged in two directions, the Xb direction and the Yb direction. Each of the plurality of detection units 6a is a sensor cell including a photoelectric conversion element such as a photodiode, a pixel, or a photodetector. Each of the plurality of detection units 6a can detect the fluorescence L3. The detection unit 6a corresponds to, for example, one pixel, but a detection region (light receiving region) including a plurality of pixels may be used as one detection unit 6a.

The microscope 1 scans the interference fringes L2 on the sample surface Sa by the scanning unit 18, and the detection device 6 detects the fluorescence L3. For example, the microscope 1 illuminates the illumination area selected from the sample surface Sa with the interference fringe L2, and the detection device 6 detects the fluorescence L3 from the illumination area. The microscope changes the illumination area by the scanning unit 18 after the detection by the detection device 6 is completed. The microscope 1 acquires a fluorescence intensity distribution (measured value of the detection device 6) in a desired region by repeating a process of detecting fluorescence and a process of changing the illumination region.

The image processing unit 7 generates an image based on the detection result of the detection device 6 obtained as described above. Hereinafter, the processing executed by the image processing unit 7 will be described. In the mathematical formulas used in the following description, the coordinate system is described as a vector as appropriate. The coordinates on the sample surface Sa and the coordinates on the detection device 6 (hereinafter referred to as detector coordinates) are represented by a vector r=(x, y), and the corresponding wave number coordinates (coordinates after Fourier transform at r) are represented by a vector k=(k). _x, represented by _{k y).} Further, coordinates of the scanning destination of the scanning unit 18 (hereinafter, referred to as scanning coordinates) are represented by a vector r _s =(x _s , y _s ) and corresponding wave number coordinates (coordinates after Fourier transform at r _s ) are vectors. It is represented by k _s =(k _xs , k _ys ). In the following description, the wave number may be referred to as spatial frequency or frequency. The magnification of the optical system is assumed to be 1 for convenience of explanation, but any magnification may be used.

Assuming that the numerical aperture of the optical system including the objective lens 21 is NA, the wavelength of the excitation light L1 is λ _ex , and the wavelength of the fluorescence L3 is λ _em , the pupil radius k _NA ^ex of the objective lens 21 when the excitation light L1 is incident and The pupil radius k _NA ^em of the objective lens 21 when fluorescence enters is represented by the following equation (1). As is well known, since the electric field amplitudes of the pupil plane and the image plane are connected by the Fourier transform relationship, the coordinates of the pupil position may be represented by wave number coordinates. Each of k _NA ^ex and k _NA ^em indicates the value of the pupil radius when the pupil is expressed in wave number coordinates.

Here, various parameters will be described with reference to FIG. 2C. In FIG. 2C, the pupil plane P0 is represented by a wave number coordinate space (frequency space). The area inside the circle drawn by the dotted line shown in FIG. 2C is the pupil of the objective lens 21, and k _NA ^ex is the pupil radius of the objective lens 21. The region P0a, the region P0b, and the region P0c on which the excitation light L1 is incident are assumed to be circular here, but are not limited to circular. The radius of each of the region P0a, the region P0b, and the region P0c is σk _NA ^ex . σ is the ratio of the radius of the region P0a, the region P0b, or P0c to the pupil radius of the objective lens 21. The distance from the optical axis 21a of the objective lens 21 to the center of the region P0a is (1-σ)k _NA ^ex , but is not limited to this value. The distance between the center of the region P0a and the center of the region P0b is, for example, 2(1-σ)k _NA ^ex , but is not limited to this value. The center of the region P0c coincides with, for example, the center of the pupil plane P0 of the objective lens 21 (the optical axis 21a of the objective lens 21), but is not limited to this. The electric field intensity of the excitation light L1 is assumed to be uniform in the pupil plane of the objective lens 21, but the invention is not limited to this. The electric field intensity ill(r) of the excitation light on the sample surface Sa is expressed by the following equation (2).

Here, the vector k ₀ =(k ₀ , 0) represents the wave number vector of the illumination stripe, and k ₀ =(1−σ)k _NA ^ex . PSF _ill (r) is a point spread function (Point Spread Function) when the numerical aperture of the optical system is σNA. The distance between the interference fringes of ill (r) (distance from one bright part to the next bright part) is 1 / k ₀ = 1 / ((1-σ) k _NA ^ex ). In the following description, the interval of the interference fringes will be appropriately referred to as the fringe interval or the period of the interference fringes.

In the embodiment, the fluorescent substance contained in the sample S is excited by the excitation light L1, and the fluorescent substance L3 is emitted from the excited fluorescent substance. The detection device 6 receives the fluorescence L3 and captures an image of the fluorescent substance formed by the detection optical system 5. The detection device 6 captures an image of a fluorescent substance and acquires image data. In the following description, the size of the detection unit 6a of the detection device 6 (detection unit size) is smaller than the size corresponding to the cycle of the interference fringes L2 in the detection device 6 (the length on the detection device 6 corresponding to one cycle). Is small enough. For example, it is desirable that the size of the detection unit 6a is set to about λ _em / 4NA. When the magnification of the detection optical system is different from 1, the numerical aperture on the detection device 6 side of the detection optical system 5 is set to NAd, and for example, the size of the detection unit 6a is preferably set to about λ _em /4NAd. .. Alternatively, when the size of the detection unit 6a is d, the focal length of the lens 23 may be set such that the numerical aperture NAd of the detection optical system 5 on the detection device 6 side satisfies NAd<λ _em /4d. ..

Here, represents the distribution of the fluorescent substance in the sample S in Obj (r), represent the image data obtained by the detector 6 by I _(r, r _s). I(r,r _s ) is represented by the following equation (3).

* ^R in equation (3) is a convolution for r. Here, the PSF _det (r) is a detection PSF determined by the detection optical system 5 including the objective lens 21 and the fluorescence wavelength λ _em . Image data I _(r, r _s) is a detector coordinate r = (x, y) and the scan coordinates _{r s = (x s, y} s) 4 -dimensional data with the independent variables. When I(r,r _s ) is transformed, the following equation (4) is obtained.

^{* Rs} in the equation (4) is a convolution of _{r s. Further, PSF} eff (r, _{r s)} is the effective PSF which is defined by the following equation (5).

From the above equation (4), it can be seen that Obj(r _s ) image data can be obtained for each detection unit 6a of the detection device 6. Further, from the above formula (5), it can be seen that the shape of the effective PSF is different for each position (r) of the detection unit 6a of the detection device 6.

FIG. 5 is a diagram showing effective PSF in each detection unit of the detection device according to the first embodiment. In each graph of FIG. 5, the horizontal axis is the Xb direction of the detection device 6. The sample surface Sa is optically conjugate with the detection device 6, and the coordinates X of the sample surface Sa and the coordinates Xb of the detection device are associated with each other by an appropriate coordinate conversion. For example, when the magnification of the optical system is 1, X=Xb.

In FIG. 5A, the effective PSF (solid line) of each detection unit 6a is shown in one graph for the three detection units 6a whose coordinates in the Xb direction are different from each other. For example, the graph in the center of FIG. 5A shows the distribution Q1a (solid line) corresponding to the effective PSF of the detection unit 6a arranged at the position X1a. The graph on the left side of FIG. 5A shows the distribution Q1b (solid line) corresponding to the effective PSF of the detection unit 6a arranged at the position X1b. The graph on the right side of FIG. 5A shows the distribution Q1c (solid line) corresponding to the effective PSF of the detection unit 6a arranged at the position X1c.

The distribution Q2 corresponding to the dotted line in FIG. 5 is a distribution corresponding to the intensity distribution of the interference fringe L2 shown in FIG. The distribution Q2 corresponds to the electric field intensity ill (r) of the excitation light on the sample surface Sa (see the above equation (2)). The position where the intensity of the interference fringe L2 is maximum, that is, the peak position X2a, the peak position X2b, and the peak position X2c of the distribution Q2 can be obtained in advance by numerical simulation or the like.

Distribution Q2 includes distribution Q2a, distribution Q2b, and distribution Q2c, which are partial distributions. The distribution Q2a is a distribution in the range from the minimum position before the peak position X2a to the next minimum position. The distribution Q2b is a distribution in the range from the minimum position before the peak position X2b to the next minimum position. The distribution Q2c is a distribution in the range from the minimum position before the peak position X2c to the next minimum position.

The distribution Q3a, the distribution Q3b, and the distribution Q3c corresponding to the chain double-dashed line in FIG. 5 are distributions corresponding to the detection optical system 5 including the objective lens 21 and the detection PSF determined by the fluorescence wavelength λ _em . The detected PSF corresponds to PSF _det (r) such as in equation (3).

The distribution Q3a shown in the center graph of FIG. 5A is a distribution corresponding to the detection PSF of the detection unit 6a arranged at the position X1a among the plurality of detection units 6a. The distribution Q3a has a maximum (peak) at the position X1a where the detector 6a is arranged (for example, the center position of the light receiving region of the detector 6a). The position X1a is almost the same as the peak position X2a of the distribution Q2a corresponding to the intensity distribution of the interference fringe L2. The distribution Q1a corresponding to the effective PSF is a distribution obtained by multiplying the distribution Q2 corresponding to the intensity distribution of the interference fringes L2 and the distribution Q3a corresponding to the detected PSF of the detection unit 6a arranged at the position X1a.

In the graph in the center of FIG. 5A, the position X1a of the detection unit 6a, that is, the peak position of the detection PSF (the peak position of the distribution Q3a) is deviated from the peak position X2a of the distribution Q2a corresponding to the intensity distribution of the interference fringes L2. The amount is smaller than the predetermined value (for example, the shift amount is almost 0). In such a case, the effective PSF distribution Q1a has a single maximum (peak). In this case, the peak position of the distribution Q1a is almost the same as the position X1a of the detection unit 6a or the peak position X2a of the distribution Q2a corresponding to the intensity distribution of the interference fringes L2.

The distribution Q3b shown in the graph on the left side of FIG. 5A is a distribution corresponding to the detection PSF of the detection unit 6a arranged at the position X1b among the plurality of detection units 6a. The distribution Q3b reaches a maximum (peak) at the position X1b where the detection unit 6a is arranged (for example, the center position of the light receiving region of the detection unit 6a). The position X1b is displaced from the peak position X2b of the partial distribution Q2b including the position X1b in the distribution Q2 corresponding to the intensity distribution of the interference fringe L2. The distribution Q1b corresponding to the effective PSF is a distribution obtained by multiplying the distribution Q2 corresponding to the intensity distribution of the interference fringe L2 and the distribution Q3b corresponding to the detected PSF of the detection unit 6a arranged at the position X1b.

In the graph on the left side of FIG. 5A, the position X1b of the detection unit 6a, that is, the peak position of the detected PSF (peak position of the distribution Q3b) is deviated from the peak position X2b of the distribution Q2b corresponding to the intensity distribution of the interference fringe L2. The amount is larger than the predetermined value. In this case, the distribution Q1b of the effective PSF has two maxima (peaks). In this way, the peak of the effective PSF may be divided into two depending on the position of the detection unit 6a, and such a change in the shape of the effective PSF is called a collapse of the shape of the effective PSF. The beak with the strongest effective PSF is called the main lobe, and the other peaks are called the side lobes.

The peak position of the main lobe of the distribution Q1b of the effective PSF deviates from the center position (position X2a) of the detection device 6. As described above, it can be seen that the position of the main lobe of the effective PSF also shifts due to the relationship between the position (r) of the detection unit 6a of the detection device 6 and the position of the intensity distribution of the interference fringe L2. In the following description, the displacement of the main lobe of the effective PSF is appropriately referred to as the displacement of the effective PSF.

The distribution Q3c shown in the graph on the right side of FIG. 5A is a distribution corresponding to the detection PSF of the detection unit 6a arranged at the position X1c among the plurality of detection units 6a. The distribution Q3c has a maximum (peak) at the position X1c where the detector 6a is arranged (for example, the center position of the light receiving region of the detector 6a). The position X1c deviates from the peak position X2c of the partial distribution Q2c including the position X1c in the distribution Q2 corresponding to the intensity distribution of the interference fringes L2. The distribution Q1c corresponding to the effective PSF is a distribution obtained by multiplying the distribution Q2 corresponding to the intensity distribution of the interference fringes L2 and the distribution Q3c corresponding to the detected PSF of the detection unit 6a arranged at the position X1c.

In the graph on the right side of FIG. 5A, the position X1c of the detection unit 6a, that is, the peak position of the detection PSF (the peak position of the distribution Q3c) is deviated from the peak position X2c of the distribution Q2c corresponding to the intensity distribution of the interference fringes L2. The amount is larger than the predetermined value. In this case, the distribution Q1b of the effective PSF has two maximums (peaks), the shape of the effective PSF is deformed, and the position of the effective PSF is displaced.

In FIG. 5(B), the position of the detection unit 6a is different from that in FIG. 5(A). The distribution Q3d shown in the graph on the left side of FIG. 5B is a distribution corresponding to the detection unit 6a arranged at the position X1d among the plurality of detection units 6a. The distribution Q3d has a maximum (peak) at the position X1d where the detector 6a is arranged (for example, the center position of the light receiving region of the detector 6a). The position X1d is substantially the same as the peak position X2b of the partial distribution Q2b including the position X1d in the distribution Q2 corresponding to the intensity distribution of the interference fringe L2. In such a case, the distribution Q1d of the effective PSF has a single maximum (peak), and the peak position of the distribution Q1d is the peak position of the distribution Q2b corresponding to the position X1d of the detection unit 6a or the intensity distribution of the interference fringe L2. It becomes almost the same as X2b. That is, the shape of the effective PSF has not collapsed.

Further, the distribution Q3e shown in the graph on the right side of FIG. 5B is a distribution corresponding to the detection unit 6a arranged at the position X1e among the plurality of detection units 6a. The distribution Q3e has a maximum (peak) at the position X1e where the detector 6a is arranged (for example, the center position of the light receiving region of the detector 6a). The position X1e is substantially the same as the peak position X2c of the partial distribution Q2c including the position X1e in the distribution Q2 corresponding to the intensity distribution of the interference fringes L2. In such a case, the distribution Q1e of the effective PSF has a single maximum (peak), and the peak position of the distribution Q1e is the peak position of the distribution Q2c corresponding to the position X1e of the detection unit 6a or the intensity distribution of the interference fringe L2. It becomes almost the same as X2c. That is, the shape of the effective PSF has not collapsed.

In the present embodiment, the image processing unit 7 uses the detection result of the detection unit 6a selected from the plurality of detection units 6a based on the magnification of the detection optical system 5 and the period (strip interval) of the interference fringes L2. The image processing unit 7 selects the detection unit 6a from the plurality of detection units 6a based on the peak position of the interference fringe L2 (for example, the peak position X2a, the peak position X2b, and the peak position X2c in FIG. 5), and the selected detection unit 7. The detection result of the unit 6a is used. The peak position of the interference fringe L2 corresponds to, for example, the position where the intensity is maximum in the intensity distribution of the interference fringe L2 (for example, the center position of the bright portion).

The image processing unit 7 uses, for example, the detection result of the detection unit 6a arranged at the position X1a corresponding to the peak position X2a as the detection result corresponding to the peak position X2a in the center graph of FIG. For example, the peak position X2a, the peak position X2b, and the peak position X2c are obtained in advance by numerical simulation or the like and are stored in the storage unit in advance. The image processing unit 7 selects the detection unit 6a arranged closest to the peak position X2a among the plurality of detection units 6a based on the stored information on the peak position, and displays the detection result of the selected detection unit 6a. To use.

The image processing unit 7 may use only the detection result of the one detection unit 6a arranged at the position X1a as the detection result regarding the partial distribution Q2a including one peak in the intensity distribution of the interference fringe L2. The detection result of the detection unit 6a arranged at the position X1a and at least one detection unit 6a around the detection unit 6a may be used.

The image processing unit 7 uses, for example, the detection result of the detection unit 6a arranged at the position X1d corresponding to the peak position X2b as the detection result corresponding to the peak position X2b in the graph on the left side of FIG. 5B. .. Based on the magnification of the detection optical system 5 and the cycle of the interference fringes L2, the image processing unit 7 includes a plurality of detectors 6a whose positions match the partial distribution Q2b including one peak in the intensity distribution of the interference fringes L2. Select from the detection unit 6a. For example, the image processing unit 7 may detect the detection unit 6a (for example, the detection unit arranged at the position X1d) closest to the peak position X2b among the plurality of detection units 6a based on the stored information of the peak position. 6a) is selected. The image processing unit 7 uses the detection result of the selected detection unit 6a as the detection result regarding the distribution Q2b.

The image processing unit 7 may use only the detection result of the one detection unit 6a arranged at the position X1d as the detection result regarding the partial distribution Q2b including one peak in the intensity distribution of the interference fringe L2. The detection results of the detection unit 6a arranged at the position X1d and at least one detection unit 6a around the detection unit 6a may be used. In the effective PSF (PSF _eff ) corresponding to the distribution Q1d, the collapse of the shape of the effective PSF is reduced by matching the peak position X2b of the distribution Q2b with the position X1d of the detection unit 6a.

Further, the image processing unit 7 uses, for example, the detection result of the detection unit 6a arranged at the position X1e corresponding to the peak position X2c as the detection result corresponding to the peak position X2c in the graph on the right side of FIG. 5B. .. Based on the magnification of the detection optical system 5 and the period of the interference fringes L2, the image processing unit 7 includes a plurality of detectors 6a whose positions match the partial distribution Q2c including one peak in the intensity distribution of the interference fringes L2. Select from the detection unit 6a. For example, the image processing unit 7 may detect the detection unit 6a that is arranged closest to the peak position X2c (for example, the detection unit that is arranged at the position X1e) among the plurality of detection units 6a based on the stored information on the peak position. 6a) is selected. The image processing unit 7 uses the detection result of the selected detection unit 6a as the detection result regarding the distribution Q2c.

The image processing unit 7 may use only the detection result of the one detection unit 6a arranged at the position X1e as the detection result regarding the partial distribution Q2c including one peak in the intensity distribution of the interference fringe L2, You may use the detection result of the detection part 6a arrange|positioned at the position X1e, and at least one detection part 6a around this detection part 6a. In the distribution Q1e(PSF _eff ), the peak position X2c of the distribution Q2c and the position X1e of the detection unit 6a are matched with each other, whereby the collapse of the shape of the effective PSF is reduced.

The image processing unit 7 corrects the image position deviation (effective PSF _eff peak position or main lobe position deviation) for each detection unit with respect to the detection result of the detection unit 6a selected as described above. The positional deviation of the image for each detection unit can be acquired by theoretical calculation using various design values or from a captured image obtained by photographing a small object such as fluorescent beads with the detection device 6. By correcting such misalignment, the effective PSF of the images obtained by each of the selected detection units 6a can be made substantially the same. The PSF _eff of the image obtained in this way can be approximately regarded as equivalent to the PSF _eff of the detection unit (detection unit located on the optical axis) at the center position of the detection device 6. The PSF _eff of the detection unit at the center position (r = (0,0)) of the detection device 6 is represented by the following equation (6).

Focusing on the period direction of the interference fringes L2, that is, the k ₀ direction, it can be seen from Equation (6) that the smaller the period of the interference fringes L2, the narrower the full width at half maximum of PSF _{eff and} the better the resolution. That is, as the number of fringes (bright part) included in the periodic direction of the interference fringe L2 in the embodiment is larger, the full width at half maximum of PSF _eff is narrower and the resolution is better.

Although the improvement of the resolution in the XY plane (in the sample plane Sa) has been described above, in the present embodiment, the resolution in the Z direction (optical axis 4a direction) is also improved as compared with the conventional fluorescence microscope. In the present embodiment, the excitation light L1a that has passed through the opening 15a, the excitation light L1b that has passed through the opening 15b, and the excitation light L1c that has passed through the opening 15c interfere with each other, as shown in FIG. , Stripes are formed not only in the X direction but also in the Z direction. The resolution in the Z direction becomes better as the period of the stripes in the Z direction becomes smaller, and in the present embodiment, the resolution in the Z direction also improves. The illumination shape (interference fringes) on the XY plane for each position in the Z direction is shown in FIG.
FIG. 6B is a 3D simulation result of the effective PSF of this embodiment, and FIG. 6C is a 3D simulation result of the PSF of the conventional fluorescence microscope. Comparing FIG. 6B and FIG. 6C, it can be seen that the present embodiment has better resolution than the conventional fluorescence microscope in all XYZ directions. The same applies to other embodiments. In order to have superior resolution over conventional fluorescence microscopes, confocal fluorescence microscopes, etc., the number of bright parts included in the periodic direction of the interference fringes L2 (X direction in FIG. 6A) in the embodiment is 3 or more. Is desirable. The same applies to other embodiments.

The image processing unit 7 generates an image by adding the images having the same PSF _eff . The image processing unit 7 can generate the image I _SR (r _s ) having good resolution and S/N ratio because the PSF _{eff of the} images to be added are almost the same. Incidentally, when a wide range of the detection unit 6a for use in the generation of the image I _{SR (r s),} it is possible to increase the signal amount. Moreover, when narrowing the range of the detection unit 6a for use in the generation of the image I _{SR (r s),} it is possible to increase the sectioning capabilities.

As described by the above equation (6), the microscope 1 according to the embodiment improves the resolution of the interference fringes L2 in the periodic direction. The microscope 1 can improve the resolution three-dimensionally in the X direction, the Y direction, and the Z direction by changing the periodic direction of the interference fringe L2 and detecting the fluorescence from the sample S. Here, an example in which the cycle direction of the interference fringe L2 is changed by 90° will be described. To change the periodic direction of the interference fringes L2 by 90 °, the mask 15 and the polarizer 14 in the state of FIG. 2 are rotated by 90 ° around the Za direction.

The super-resolution image when the periodic direction of the interference fringes L2 is the X direction and I _{SRx (r s),} the super-resolution image when the periodic direction of the interference fringes L2 is the Y direction I _{SRy (r} s) And The image processing unit 7, by summing the I _{SRx (r s)} and I _{SRy (r s),} may generate a super-resolution image resolution is improved three-dimensionally. The image processing unit 7 may also generate a super-resolution image by the following processing.

The image processing section 7 performs Fourier transform super-resolution image I _{SRx (r s)} and a super-resolution image I _SRy the _{(r s),} respectively. Here, the Fourier-transformed super-resolution image I _SRx (r _s ) is represented by I ^to _SRx (k _s ). " ^~ " In the specification is a tilder in the mathematical formula. In addition, the Fourier-transformed super-resolution image I _SRy (r _s ) is represented by I 1 ^to _SRy (k _s ). For I 1 ^to _{SR_x} (k _s ), the cutoff frequency increases in the X direction and the Z direction, for example, by a factor of 2 as compared with a normal fluorescence microscope. Further, for I 1 ^to _SRy (k _s ), the cutoff frequency in the Y direction and the Z direction increases, for example, by a factor of 2 as compared with a normal fluorescence microscope. The image processing section 7 adds up the I ~ _{SRx (k s)} and I ~ _{SRy (k s).} This increases the cutoff frequency in the three directions (X direction, Y direction, and Z direction).

The shape of the added effective OTF may be distorted depending on the combination of the directions that change the periodic direction of the interference fringes L2. In this case, the image processing unit 7 may apply a frequency filter for correcting the shape of the effective OTF. Thus, it is possible to improve the effective resolution than when summing the I _{SRx (r s)} and I _{SRy (r s).} Further, as described with reference to FIG. 3, the illumination optical system 4 changes the periodic directions of the interference fringes L2 in three ways of 0 °, 120 °, and 240 °, and the detection device 6 changes each of the three periodic directions. Fluorescent L3 may be detected. The image processing unit 7 may generate a super-resolution image by using three detection results (for example, three images) detected by the detection device 6 in three different cycle directions. Further, the illumination optical system 4 changes the periodic direction of the interference fringes L2 in four or more ways, the detection device 6 detects the fluorescence L3 in each of the changed four or more periodic directions, and the image processing unit 7 The super-resolution image may be generated by using the four or more detection results detected by the detection device 6 in the four or more changed cycle directions. The method of generating a super-resolution image using a plurality of detection results obtained by changing the period direction of the interference fringes a plurality of times described above can also be used in the following embodiments.

Next, the observation method according to the embodiment will be described based on the configuration of the microscope 1 described above. FIG. 7 is a flowchart showing the observation method according to the embodiment. For each part of the microscope 1, refer to FIG. 1 or FIG. 4 as appropriate. In step S1, the illumination optical system 4 of FIG. 1 sets the angle of the scanning mirror. The illumination optical system 4 irradiates the excitation light as interference fringes at a position on the sample determined by the angle of the scanning mirror set in step S1. In step S2, the fluorescent substance of the sample is excited by the interference fringes of the excitation light. In step S3, the detection device 6 of FIG. 4 detects the fluorescence L3 from the sample S via the detection optical system 5.

In step S4, the control unit CB determines whether or not to change the angle of the scanning mirror. When it is determined that the processing of steps S1 to S3 has not been completed for a part of the scheduled observation region, the control unit CB determines to execute the angle change of the scanning mirror in step S4 (step S4; Yes). ). When the control unit CB determines to change the angle of the scanning mirror (step S4; Yes), the process returns to step S1 and the illumination optical system 4 sets the angle of the scanning mirror to the next scheduled angle. .. Then, the processes of steps S2 to S4 are repeated. In this way, the illumination optical system 4 scans the sample S two-dimensionally with the interference fringes of the excitation light L1.

When it is determined in step S4 that the processing in steps S1 to S3 has been completed for all of the scheduled observation regions, the control unit CB determines not to change the angle of the scanning mirror (step S4; No). .. When the control unit CB determines not to change the angle of the scanning mirror (step S4; No), the image processing unit 7 corrects the positional deviation of the image for each detection unit in step S5. The image processing unit 7 corrects the data obtained from at least one detection unit of the plurality of detection units based on the position of the detection unit. For example, the image processing unit 7 corrects the data obtained from the detection unit selected from the plurality of detection units based on the position of the detection unit. For example, the image processing unit 7 uses the data obtained from the first detection unit (for example, the detection unit arranged at the position X1d in FIG. 5B) of the plurality of detection units as the first detection unit. Correction is made based on the position (eg, position X1d). The image processing unit 7 also generates an image using the detection results of two or more detection units. For example, in step S6, the image processing unit 7 generates an image (for example, a super-resolution image) by adding the corrected images in step S5.

The positions of the plurality of detection units 6a of the detection device 6 may be set based on the cycle of the interference fringe L2 so as to match the peak (or maximum or bright point) position of the interference fringe L2. The detection device 6 may be preset so that the spacing between the detection units 6a and the fringe spacing of the interference fringes L2 match. The distance between the detection units 6a is the distance between the center of one detection unit 6a and the center of the detection unit 6a adjacent to the center. The fringe spacing of the interference fringe L2 is the spacing between the centerline of one bright portion and the centerline of the adjacent bright portion in the interference fringe L2. Here, the fringe spacing of the interference fringes L2 is 1 / k ₀ . When the fringe spacing of the interference fringes is 1/k ₀ , the spacing of the detection unit 6a of the detection device 6 is set to be substantially the same as P shown in the following formula (7).

In the above formula (7), the magnification of the detection optical system 5 including the objective lens 21 is set to 1. When the magnification of the detection optical system 5 is M _det , the interval of the detection unit 6a may be changed by the magnification, and the interval of the detection unit 6a may be M _det / k0. Alternatively, by using a part of the detection optical system 5 as a zoom lens, the distance between the detection units 6a can be matched with the period of the interference fringes L2. In this case, it is desirable that the lens 23 capable of changing only the magnification of the detection optical system 5 is a zoom lens. Further, the cycle of the interference fringes L2 may be adjusted so as to match the intervals between the plurality of detection units 6a of the detection device 6. For example, the period of the interference fringes L2 can be changed by changing the distance between the opening 15a and the opening 15b of the mask 15.

In the present embodiment, the microscope 1 may scan the interference fringes L2 two-dimensionally by scanning the interference fringes L2 in two directions parallel to the sample surface Sa, or may be parallel to the sample surface Sa. The interference fringes L2 may be scanned three-dimensionally by scanning the interference fringes L2 in two directions and in one direction perpendicular to the sample surface Sa. When the interference fringes L2 are three-dimensionally scanned, the process of scanning the interference fringes L2 in the two directions parallel to the sample surface Sa (hereinafter referred to as the two-dimensional process) is the same as the process described in the above embodiment. is there. The microscope 1 can generate, for example, a three-dimensional super-resolution image by repeating the two-dimensional processing by changing the position in the Z direction. Similarly to the embodiments described below, the microscope 1 may three-dimensionally scan the interference fringes L2.

[Second Embodiment]
The second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. In the present embodiment, the image processing unit 7 (see FIG. 4) filters the data in the frequency space to generate an image. The image processing unit 7 performs deconvolution on the data obtained from the detection device 6 to generate an image. The image processing unit 7 performs deconvolution and apodization for each detection unit 6a of the detection device 6 as the above-described filtering to generate an image. That is, the image processing unit 7 performs filtering including deconvolution on the data in the frequency space. In the following description, a series of processes of deconvolution and apodization may be collectively (generally) referred to as deconvolution.

FIG. 8 is a diagram showing processing of the image processing unit of the microscope according to the second embodiment. For each part of the microscope, refer to FIG. 1 or FIG. 4 as appropriate. FIG. 8 (A) shows the PSF before deconvolution, which is the same as FIG. 5 (A). In FIG. 8A, the distance between the detection units 6a of the detection device 6 (see FIG. 4) does not match the distance between the interference fringes L2. In this case, as described in the above equation (5), the effective PSF (solid line) of the image obtained for each detection unit 6a loses its shape depending on the position of the detection unit 6a. The effective PSF for each detection unit 6a can be obtained (estimated) by theoretical calculation from the design value or by photographing a small object such as fluorescent beads. The image processing unit 7 uses the thus obtained effective PSF to perform deconvolution so as to correct the shape collapse and the positional deviation of the effective PSF of the image obtained for each detection unit 6a.

Figure 8 (B) shows the PSF after deconvolution. In the central graph of FIG. 8B, the distribution Q4a is the effective obtained by deconvolving the effective PSF of the detection unit 6a arranged at the distribution Q1a, that is, the position X1a shown in the central graph of FIG. 8A. Corresponds to PSF. Here, the amount of deviation between the position X1a of the detection unit 6a and the peak position X2a of the distribution Q2a is smaller than a predetermined value, and the distribution Q4a corresponding to the effective PSF after deconvolution corresponds to the effective PSF before deconvolution. It is almost the same as the distribution Q1a.

In the graph on the left side of FIG. 8B, the distribution Q4b is obtained by deconvoluting the distribution Q1b shown in the graph on the left side of FIG. 8A, that is, the effective PSF of the detection unit 6a arranged at the position X1b. Corresponding to the effective PSF. In the graph on the right side of FIG. 8B, the distribution Q4c is obtained by deconvoluting the distribution Q1c shown in the graph on the right side of FIG. 8A, that is, the effective PSF of the detection unit 6a arranged at the position X1c. Corresponds to PSF. By such a series of processes (deconvolution), as shown in the three graphs of FIG. 8B, the effective PSF for each detection unit 6a becomes substantially the same. The image processing unit 7 uses the result of deconvolution to generate an image. Hereinafter, the processing of the image processing unit 7 will be described in more detail.

The image processing unit 7 converts the detection results of at least a part of the plurality of detection units 6a into data in the frequency space, and uses the conversion results to generate an image (for example, a super-resolution image). In the following description, data representing at least a part of the detection results of the plurality of detection units 6a in the frequency space is appropriately referred to as a component of the frequency space. The image processing unit 7 Fourier transforms at least a part of the detection results of the plurality of detection units 6a, and generates an image using the components of the frequency space obtained by the Fourier transform. When the above equation (4) is the Fourier transform for r _s, the following equation (8) is obtained.

I 1 ^to (r, k _s ) on the left side of the equation (8) is the Fourier transform of I (r, r _s ) with respect to r _s . Right side of the _OTF eff (r, _{k s) is, PSF eff (r, r s} ) the is obtained by Fourier transform for _{r s,} represents the effective OTF of each detector 6a of the detector 6. In addition, the right-hand side of ^Obj ~ _{(k s)} are those Obj a _{(r s)} obtained by Fourier transform on _{r s.}

There are various deconvolution methods such as the Wiener filter and Richardson-Lucy method. Here, the process using the Wiener filter will be described as an example, but the image processing unit 7 may execute deconvolution by other processes. The deconvolution of each detection unit by the Wiener filter is represented by the following formula (9).

In Expression (9), Obj ^to (r, k _s ) are distributions of the fluorescent substance estimated for each detection unit 6a of the detection device 6 (hereinafter, referred to as estimated fluorescent substance distributions). w is a Wiener parameter for suppressing noise. By this processing, the estimated fluorescent substance distributions Obj 1 ^to (r, k _s ) are substantially the same in the two or more detection units 6a of the detection device 6. That is, the above processing corrects the shape collapse and the positional deviation of the effective PSF for each detection unit 6a, and the effective PSFs for each detection unit 6a become substantially the same. The image processing unit 7, the process shown in the following equation ^(10), the apodization Obj ~ (r, _{k s),} adding the spectrum in the detection section 6a of the detection device 6, the super-resolution image _{I SR} ( to generate a r _s).

In Expression (10), A(k _s ) is an apodization function for suppressing the negative value of the image, and multiplying Obj ^˜ (r, k _s ) by A(k _s ) is called apodization. The functional form of A(k _s ) is designed to suppress the negative value of the image by theoretical calculation or simulation. _{Further, F} ^{ks -1} is the inverse Fourier transform relates _{k s.} The image processing unit 7 performs the inverse Fourier transform after adding the spectra of the detection units 6a, but may add the images after performing the inverse Fourier transform. In the processing of Expressions (9) and (10), the image processing unit 7 independently deconvolves each detection unit 6a and then adds the images to each detection unit 6a. The image processing section 7 may collectively deconvolute two or more detection sections 6a as in the following Expression (11).

FIG. 9 is a flowchart showing an observation method according to the second embodiment. The processing of steps S1 to S4 is the same as that of FIG. 7, and thus its description is omitted. In step S11, the image processing unit 7 Fourier transforms the detection result of each detection unit. In step S12, the image processing unit 7 executes deconvolution. In step S13, the image processing unit 7 executes apodization. Apodization may be part of the deconvolution process. In step S14, the image processing unit 7 adds the images for each detection unit 6a using the result of the deconvolution. In step S15, the image processing unit 7 performs an inverse Fourier transform on the first image (for example, Fourier image) obtained in step S14 to generate a second image (for example, super-resolution image).

Note that the image processing unit 7 may change the range of the detection unit 6a to be added, as described in the first embodiment. Further, the image processing unit 7 may improve the resolution one-dimensionally or two-dimensionally as described in the first embodiment.

[Third Embodiment]
A third embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 10 is a diagram showing processing of the image processing unit of the microscope according to the third embodiment. For each part of the microscope, refer to FIG. 1 or FIG. 4 as appropriate.

In the present embodiment, the image processing unit 7 (see FIG. 4) corrects the collapse of the shape of the effective PSF for each detection unit 6a by image processing different from that in the second embodiment. FIG. 10A is a PSF before image processing according to the present embodiment, which is the same as FIG. 5A. In FIG. 10A, the distance between the detection units 6a of the detection device 6 (see FIG. 4) does not match the distance between the interference fringes L2. In this case, as described in the above equation (5), the effective PSF (solid line) of the image obtained for each detection unit 6a is deformed depending on the position of the detection unit 6a. The image processing unit 7 sets the interference fringes so that the peak position of the partial distribution of the intensity distribution of the interference fringes L2 (for example, Q2b shown in the graph on the left side of FIG. 10A) matches the position of the detection unit 6a. The phase of the intensity distribution of L2 is effectively shifted by image processing. This process is appropriately called an image processing phase shift, and the phase shift amount is called an image processing phase shift amount.

FIG. 10B shows the effective PSF for each detection unit 6a after the image processing phase shift processing. In the graph on the left side of FIG. 10B, the distribution Q2f is subjected to image processing phase shift so that the peak position X2b of the distribution Q2b of FIG. 10A coincides with the position X1b of the detection unit 6a. Distribution. The peak position X2f of the distribution Q2f substantially coincides with the position X1b of the detection unit 6a. The distribution Q1f is a distribution corresponding to the effective PSF obtained from the distribution Q2f obtained by shifting the phase of the image processing phase and the detected PSF (distribution Q3b) of the detection unit 6a arranged at the position X1b. In the distribution Q1f, the collapse of the shape of the effective PSF is reduced.

In the graph on the right side of FIG. 10(B), the distribution Q2g has the phase of the distribution Q2 set to the image processing phase so that the peak position X2c of the distribution Q2c of FIG. 10(A) coincides with the position X1c of the detection unit 6a. This is the shifted distribution. The peak position X2g of the distribution Q2g substantially coincides with the position X1c of the detection unit 6a. The distribution Q1g is a distribution corresponding to the effective PSF obtained from the distribution Q2g obtained by phase-shifting the image processing phase and the detection PSF (distribution Q3c) of the detection unit 6a arranged at the position X1c. In the distribution Q1g, the collapse of the shape of the effective PSF is reduced.

By the image processing process as described above, the shape of the effective PSF (solid line) for each detection unit 6a is corrected so as to be substantially the same. The image processing unit 7 generates an image by using an image for each detection unit 6a having an effective PSF corrected to have substantially the same shape.

Hereinafter, the processing flow of the image processing unit 7 will be described in more detail. The image I(r,r _s ) obtained by the detection device 6 is represented by the above equation (3). Substituting the ill (r) shown in the above equation (2) into the equation (3), the following equation (12) is obtained.

In equation (12), φ indicates the initial phase of the interference fringe L2. The image processing unit 7 changes the phase of the interference fringe L2 by image processing according to the detector coordinates, and aligns the shapes of the effective PSFs. The microscope 1 acquires the four-dimensional image data I(r,r _s ) as described in Expression (3). The image processing unit 7 performs a four-dimensional Fourier transform on I(r,r _s ). The four-dimensional data in the frequency space obtained by the Fourier transform is represented by I 1 ^to (k, k _s ).

By performing Fourier transform on the equation (12) for r and rs, it can be seen that I 1 ^to (k, k _s ) are represented by the following equation (13).

OTF _det is the Fourier transform of PSF _det and represents the OTF of the detection optical system 5. OTF _ill is the Fourier transform of PSF _ill , and Obj ^˜ are the Fourier transform of Obj. Here ^{_{I 0 ~ (k, k s}} ), I +1 ~ (k, k s), I -1 ~ (k, k s), I +2 ~ (k, k s), I -2 ~ (k, k _s ) is defined by the above equation. ^{_{I 0 ~ (k, k s}} ) a zero-order ^{_{component, I +1 ~ (k, k}} s) +1 order ^{_{component, I -1 ~ (k, k}} s) -1-order _{component, I} ⁺² ~ _(k, k _s ) is called +second-order component, and I ₋₂ ^to (k, k _s ) is called −second-order component.

The cutoff frequency of OTF _det (k) is given by 2k _NA ^em . The cutoff frequency of OTF _ill (k) is given by 2σk _NA ^ex . Therefore, I ₀ ^to (k, k _s ) have a value only in the region AR1a of the zero-order component that satisfies the condition of Expression (14), and I ₊₁ ^to (k, k _s ) satisfy the condition of Expression (15). It has a value only in the area AR1b of the +1st-order component to be satisfied, and I ₋₁ ^to (k, k _s ) has a value only in the area AR1c of the −1st-order component satisfying the condition of Expression (16), and I ₊₂ ^to (K, k _s ) has a value only in the area AR1d of the +second-order component satisfying the condition of the formula (17), and I ₋₂ ^to (k, k _s ) satisfies the condition of the formula (18) −second-order It has a value only within the region AR1e of the component. The image processing unit 7, the I ^~ (k, _{k s),} from extracts information of any satisfying area of formula (18) from the following equation ^{(14), I ~ (k} , k s) I ₀ ^to (k, k _s ), I ₊₁ ^to (k, k _s ), I ₋₁ ^to (k, k _s ), I ₊₂ ^to (k, k _s ), I ⁻ ₂ ^to (k, k _{s ).} ) Get. From I ^~ (k, k _s ) to I ₀ ^~ (k, k _s ), I ₊₁ ^~ (k, k _s ), I _-1 ^~ (k, k _s ), I ₊₂ ^~ (k, k _s ), The process of separating I ₋₂ ^to (k, k _s ) is appropriately referred to as component separation.

FIG. 11 is a diagram showing a region of the frequency space used for component separation in the third embodiment. Here, a case where the opening 15a, the opening 15b, and the opening 15c of the mask 15 (see FIG. 2) are circular will be described. The opening of the mask 15 may have a shape other than a circular shape. The range of the area AR1a of the 0th order component, the area AR1b of the +first order component, the area AR1c of the −1st order component, the area AR1d of the +second order component, and the area AR1e of the −second order component is when the opening of mask 15 is circular. In any case where the opening of the mask 15 has a shape other than a circle, it can be obtained by numerical simulation, theoretical calculation, or the like based on the design value of the microscope 1. Alternatively, the region where each component has a value can be obtained by actually measuring the fluorescent sample.

FIG. 11A shows each region on the k _{xs -kys} plane. The 0th-order component area AR1a, the +1st-order component area AR1b, the −1st-order component area AR1c, the +second-order component area AR1d, and the −second-order component area AR1e are circular areas. The 0th-order component area AR1a, the +1st-order component area AR1b, the −1st-order component area AR1c, the +second-order component area AR1d, and the −second-order component area AR1e all have the same radius. The radius of the 0th-order component region AR1a is 2σk _NA ^ex . The region AR1a of the 0th-order component is a region centered on the origin. The area AR1b of the +1st order component, the area AR1c of the −1st order component, the area AR1d of the +second order component, and the area AR1e of the −second order component are areas on the axis of which the center is k _xs . The distance A1 between the center of the area AR1c of the −1st order component and the origin is k0=(1−σ)k _NA ^ex . The +1st-order component region AR1b is a region symmetric with respect to the -1st-order component region AR1c with respect to the 0th-order component region AR1a. The distance A2 between the center of the −second-order component area AR1e and the origin is 2k0. The + secondary component region AR1d is a region symmetric with respect to the second-order component region AR1e with respect to the zero-order component region AR1a.

FIG. 11B shows each region in the k _xs −kx plane. The 0th-order component area AR1a, the +1st-order component area AR1b, the −1st-order component area AR1c, the +second-order component area AR1d, and the −second-order component area AR1e are parallelogram areas, respectively.

The image processing unit 7 sets the region of the frequency space in the component separation based on the light intensity distribution of the excitation light in the sample S. For example, the image processing unit 7 sets a plurality of regions that do not overlap each other based on the electric field intensity ill(r) of the excitation light on the sample surface Sa as the light intensity distribution of the excitation light on the sample S. The plurality of regions described above include five or more regions that do not overlap with each other. For example, the plurality of areas include the area AR1a of FIG. 11 as the first area, the area AR1b of FIG. 11 as the second area, the area AR1c of FIG. 11 as the third area, the area AR1d of FIG. 11 as the fourth area, and The area AR1e of FIG. 11 is included as the fifth area. From the data in the frequency space, the image processing unit 7 includes data belonging to the first area (area AR1a), data belonging to the second area (area AR1b), third area (area AR1c), and fourth area (area AR1d). ), and the data belonging to the fifth area (area AR1e) are extracted to separate the components.

The image processing unit 7 includes I ₀ ^to (k, k _s ), I ₊₁ ^to (k, k _s ), I ₋₁ ^to (k, k _s ), I ₊₂ ^to (k, k _s ), I _−2. each ^{~ (k,} k _s) to, by performing an inverse Fourier transform of the four-dimensional, calculates the image data of real space. Hereinafter, representative of the image data obtained by inverse Fourier transform of ^{_{I 0 ~ (k, k s}} ) with _{I 0 (r, r s)} . _Further, representative of the image data obtained by inverse Fourier transform of _{^{I +1 ~ (k, k s}} ) with _{I +1 (r, r s)} . Further, representative of the image data obtained by inverse Fourier transform of ^{_{I -1 ~ (k, k s}} ) with _{I -1 (r, r s)} . Further, image data obtained by inverse Fourier transforming I ₊₂ ^to (k, k _s ) is represented by I ₊₂ (r, r _s ). Further, representative of the image data obtained by inverse Fourier transform of ^{_{I -2 ~ (k, k s}} ) in _{I -2 (r, r s)} . The image processing unit 7 includes I ₀ (r,r _s ), I ₊₁ (r,r _s ), I ₋₁ (r,r _s ), I ₊₂ (r,r _s ), and I ₋₂ (r,r ). For each r _s ), the calculation shown in the following equation (19) is performed.

In Expression (19), ψ(r) represents the image processing phase shift amount for each position r of the detection unit 6a of the detection device 6. The image processing phase shift amount ψ (r) is determined, for example, as follows. The image processing unit 7 calculates the positional deviation amount of the signal detected at the detector coordinate r. The image processing unit 7, for example, by simulation in _advance, by obtaining the peak position of the obtained function by the product of _{PSF det} (r + r _s) and _PSF ill _{(r s),} calculates the position deviation amount of the. Here, it can be considered that the position deviation of the effective PSF is proportional to the detector coordinate r, and the parameter indicating the degree of the position deviation is β, and the position deviation amount is represented by r/β. The value of β _may be calculated from the peak position of the function obtained by the product of the _{PSF det} (r + r _s) and _PSF ill _{(r s),} may be calculated by other numerical simulations. Further, a different β may be used for each detection unit 6a of the detection device 6. Once β is determined, the amount of image processing phase shift according to the detector coordinates is determined. Image processing the phase shift of the interference fringe L2 ψ (r) is the peak position of the function obtained by the product of _{PSF det} and (r + r _s) and _PSF ill _{(r s),} and the peak position of the interference fringes coincide Is decided. By such processing, the image processing phase shift amount becomes, for example, ψ (r) = -2πk ₀ · r / β−φ. The value of the initial phase φ may be a value measured in advance using fluorescent beads or a value estimated from an observed image. The image processing unit 7 determines the amount of phase conversion (image processing phase shift amount) based on the light intensity distribution of the excitation light in the sample S.
The image processing unit 7 calculates the sum of the calculation results of the five equations of the above equation (19) as shown in the following equation (20).

I′(r,r _s ) obtained from the calculation of the above equation (20) is an image in which the collapse of the shape of the effective PSF for each position r of the detection unit 6a is corrected, and the shape of the effective PSF is almost the same. Becomes

The image processing section _{7, I '(r, r s} ) with respect to correct the positional deviation of the effective PSF for each detected portion 6a of the detector 6. Thereby, the effective PSFs can be made substantially the same in the two or more detection units 6a of the detection device 6. The image processing unit 7, by adding the image for each detector 6a the effective PSF is corrected to almost the same, to produce a super-resolution image I _{SR (r s).} This series of processing is performed based on the equation (21).

Here, PH (r) is a pinhole function defined by the equation (22).

Increasing the value of r _PH can increase the signal amount, it is possible to increase the sectioning ability Lower values of r _PH.

In the present embodiment, the scan interval and the interval of the detection unit 6a of the detection device 6 may be set based on the cutoff frequency and the Nyquist theorem. The scan interval may be set to λ _ex / 8 NA or less in the periodic direction of the interference fringes. In addition, the scan interval may be set to λ _ex /4NA or less in the direction perpendicular to the periodic direction of the interference fringes. Further, the interval between the detection units 6a of the detection device 6 may be set to λ _em /4NA or less. When the magnification of the detection optical system is different from 1, the numerical aperture on the detection device 6 side of the detection optical system 5 is set to NAd, and for example, the size of the detection unit 6a is preferably set to about λ _em /4NAd. .. Alternatively, when the size of the detection unit 6a is d, the focal length of the lens 23 may be set such that the numerical aperture NAd of the detection optical system 5 on the detection device 6 side satisfies NAd<λ _em /4d. ..

FIG. 12 is a flowchart showing an observation method according to the third embodiment. The processes of steps S1 to S4 are similar to those of FIG. In step S21, the image processing unit 7 Fourier transforms at least a part of the detection results of the plurality of detection units 6a. In step S21, the image processing unit 7 performs a four-dimensional Fourier transform on I(r,r _s ). In step S22, the image processing unit 7 separates the components in the frequency space. The image processing unit 7 separates the components of the frequency space obtained by the Fourier transform into each region of the frequency space. In step S23, the image processing unit 7 performs an inverse Fourier transform on the separated components. In step S24, the image processing unit 7 executes the image processing phase shift process. The image processing unit 7 corrects the positional deviation of the effective PSF in step S25. In step S26, the image processing unit 7 generates an image (for example, a super-resolution image) by adding the images obtained by correcting the positional deviation in step S25.

As described above, the image processing unit 7 according to the present embodiment converts the phase of at least a part of the data obtained by the component separation to generate an image. In the above description, the image processing unit 7 executes the phase shift process on the data in the real space. That is, the image processing unit 7 uses, as the data obtained by the component separation, the data (the data on the real space) obtained by converting the data on the component separation (the data on the frequency space) into the data on the real space by the inverse Fourier transform. .. The image processing unit 7 may perform the phase shift process in the frequency space on the data in the frequency space in which the components have been separated.

[Fourth Embodiment]
A fourth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. In the present embodiment, the image processing unit 7 (see FIG. 4) performs the component separation described in the third embodiment, and then performs deconvolution on the separated components to generate an image.

Rewriting the formula (13), the following formula (23) is obtained.

In equation (23), OTF ₀ (k, k _s ), OTF ₊₁ (k, k _s ), OTF _-1 (k, k _s ), OTF ₊₂ (k, k _s ), and OTF- ₂ (k, k, _s ). k _s) is expressed by the following equation (24).

The image processing unit 7 includes OTF ₀ (k, k _s ), OTF ₊₁ (k, k _s ), OTF _-1 (k, k _s ), OTF ₊₂ (k, k _s ), and OTF- ₂ (k, k _s ). Deconvolution is performed using each estimated value of k _s ). There are various methods of deconvolution, such as the Wiener filter and the Richardson-Lucy method. Here, processing using a Wiener filter will be described as an example of deconvolution, but deconvolution using another method may be used. Regarding the above equation (23), deconvolution by the Wiener filter is represented by the following equation (25).

In the formula (25), A (k s ) is the apodization function for suppressing the negative value of the image. Further, w is a Wiener parameter for suppressing noise. F _ks ^-1 is the inverse Fourier transform for k _s . The image processing unit 7 generates an image using the result of the above deconvolution.

FIG. 13 is a flowchart showing the observation method according to the fourth embodiment. The processing of steps S1 to S4 is the same as that of FIG. 7, and the description thereof is omitted. In step S31, the image processing unit 7 Fourier transforms the detection result. Further, in step S32, the image processing unit 7 separates the components in the frequency space. In step S33, the image processing unit 7 performs deconvolution using the components separated by the process of step S32. In step S34, the image processing unit 7 performs apodization. In step S35, the image processing unit 7 performs an inverse Fourier transform on the data obtained by deconvolution and apodization. The image processing unit 7 generates an image using the data obtained by the inverse Fourier transform.

As described above, the image processing unit 7 according to the present embodiment performs component separation, deconvolution, and apodization in the frequency space, converts the data obtained by these processes into data in the real space, and forms an image. To generate. In the present embodiment, the image processing unit 7 may generate an image without performing the process of correcting the positional deviation by making the effective PSFs of the detection units 6a of the detection devices 6 substantially match.

In the present embodiment, the scan interval and the interval of the detection unit 6a of the detection device 6 may be set based on the cutoff frequency and the Nyquist theorem. The scan interval may be set to λ _ex /8NA or less in the periodic direction of the interference fringes. In addition, the scan interval may be set to λ _ex /4NA or less in the direction perpendicular to the periodic direction of the interference fringes. Further, the interval between the detection units 6a of the detection device 6 may be set to λ _em /4NA or less. When the magnification of the detection optical system is different from 1, the numerical aperture on the detection device 6 side of the detection optical system 5 is set to NAd, and for example, the size of the detection unit 6a is preferably set to about λ _em /4NAd. .. Alternatively, when the size of the detection unit 6a is d, the focal length of the lens 23 may be set so that the numerical aperture NAd of the detection optical system 5 on the detection device 6 side satisfies NAd<λ _em /4d. ..

The image processing unit 7 may set the range to be integrated with respect to the above k to the range of the entire space or to a part of the range of the entire space. Further, the image processing unit 7 is subjected to Fourier transform to perform I ₀ ^to (k, k _s ), I _{+ 1} ^to (k, k _s ), I _-1 ^to (k, k _s ), and I _{+ 2} ^to (k, k _s). ), and I ₋₂ ^to (k, k _s ) may be calculated by limiting the range of r. Further, the image processing unit 7 includes OTF ₀ (k, k _s ), OTF ₊₁ (k, k _s ), OTF _-1 (k, k _s ), OTF ₊₂ (k, k _s ), and OTF _-2 (k, k _s ). k, k _s ) may be data obtained in advance by measurement using fluorescent beads or numerical simulation using the design value of the microscope 1, or the result of detection of fluorescence from the sample S by the detection device 6. Data (eg, estimated values) obtained from the above may be used.

[Fifth Embodiment]
A fifth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. In the above-described embodiment, the microscope 1 scans the interference fringes L2 in two directions parallel to the sample surface Sa to scan the interference fringes L2 two-dimensionally and generate a two-dimensional image of the sample S. In the present embodiment, the microscope 1 scans the interference fringes L2 in two directions parallel to the sample surface Sa, and scans the interference fringes L2 in one direction perpendicular to the sample surface Sa, so that the three-dimensional image of the sample S is obtained. Generate an image.

The fifth embodiment extends the method for correcting the shape collapse of the effective PSF for each detection unit 6a by the image processing phase shift processing described in the third embodiment to image data obtained by three-dimensional scanning. The method is used. Of the processing methods described in the third embodiment, the processing that can be commonly used in the fifth embodiment will be omitted. In the fifth embodiment, image data obtained by three-dimensionally scanning the sample S with the interference fringes of the excitation light L1 is acquired, and the acquired image data is Fourier-transformed to be converted into data in the wave number space. The data is separated into components, and the separated data is transformed into data in the real space by inverse Fourier transform, and image processing phase shift processing is executed to correct the positional deviation of the image for each detection unit. A three-dimensional image of the sample S is generated by adding the images after the positional deviation correction.

The stage 2 holds the sample S to be observed, and the relative positions of the sample S and the illumination optical system 4 in the X direction, Y direction, and Z direction in FIG. 1 can be moved. The relative position between the stage 2 and the illumination optical system 4 is set to a predetermined position so that the region to be observed in the sample S is arranged on the sample surface Sa (region where the local interference fringes L2 are formed). Subsequently, the operations of steps S1 to S4 in the flowchart of FIG. 7 of the first embodiment are carried out, and the sample S is two-dimensionally scanned by the interference fringes of the excitation light L1. As a result, it is possible to acquire image data obtained by two-dimensionally scanning a specific region of the sample S. Subsequently, the relative position between the stage 2 and the illumination optical system 4 is moved in the Z direction by a predetermined amount, and the region to be observed in the sample S is changed in the Z direction. The sample S is two-dimensionally scanned by the interference fringes of the excitation light L1. By repeating this operation, it is possible to acquire image data obtained by three-dimensionally scanning the sample S with the interference fringes of the excitation light L1. This is the acquisition of so-called Z-stack images.

Image data I obtained a sample S when 3-dimensionally scanned by the interference fringes of the excitation light _{L1 (r, r s, z} s) is given by the following equation (26).

Here, it represents the Z-direction relative position of the stage 2 and the illumination optical system 4 by z _s, representing the (coordinates after the Fourier transform at z _s) the corresponding wave number coordinate in k _zs. r, and r _s are as described in the first embodiment. PSF _det (r, z) indicates a three-dimensional distribution of detected PSFs in the XYZ directions. Here, the detection PSF is determined by the detection optical system 5 including the objective lens 21 and the fluorescence wavelength λ _em . ill (r, z) represents the three-dimensional electric field intensity distribution of the excitation light in the XYZ direction in the vicinity of the sample surface Sa. The ZX cross section of ill (r, z) is as shown in FIG. 6 (a). Obj(r,z) represents the three-dimensional distribution of the fluorescent substance in the sample S in the XYZ directions. * ^R in equation (26) is a convolution for r. These are extensions of the three-dimensional scanning of the equation (3) described in the first embodiment. Image data _{I (r, r s, z} s) is in the 5-dimensional image data.

The image processing unit 7 performs a five-dimensional Fourier transform on r, r _s , and z _{s with} respect to Expression (26). The five-dimensional data of the frequency space obtained by the Fourier transform is represented by I ^~ (k, k _s , k _{z s} ). I ^to (k, k _s , k _zs ) are represented by the following equation (27).

ill 1 ^to ill represent Fourier transform of ill, OTF _det represents Fourier transform of PSF _det , and Obj 1 ^to Fourier transform of Obj.

ill ^~ (k, k _z ) is shown in FIG. 14 (A). This is a Fourier transform of the illumination shape shown in FIG. 6A. FIG. 14 (A) is an extension of FIG. 11 (A) described in the third embodiment to three dimensions, and like FIG. 11 (A), regions in which ill ^to (k, k _z ) have values. Is divided into multiple. As shown in FIG. ^{14 (B), ill ~ (} k, k z) AR 0 regions has a _{value, AR + 1x, + 1z,} AR + 1x, -1z, AR -1x, + 1z, AR -1x, -1z , AR _+2x , and AR _-2x .
Of the ill ^to (k, k _z ), the components having values in the area AR ₀ are ill ^to ₀ (k, k _z ), and the components having values in the areas AR _{+1x and +1z} are ill ^to _{+1x, +1z} (k, k _z ), the components having values in the regions AR _{+1x, −1z} are ill ^to _{+1x, −1z} (k, k _z ), and the components having values in the regions AR _{−1x, +1z} are ill ^to _{−1x, +1z} (k , K _z ), the components having values in the regions AR _{−1x, −1z} are ill ^to _{−1x, −1z} (k, k _z ), and the components having values in the region AR _+2x are ill ^to _+2x (k, k _{Let z} ), and let the components having a value in the region AR _-2x be ill ^to _-2x (k, k _z ).

From the above definition, ill ^~ (k, k _z ) can be written as the following expression (28).

Here, φx and φz are the initial phases of the illumination light.

Equation (29) is obtained by substituting equation (28) into equation (27).

^{_{I ~ 0 (k, k s}} , k zs) the zero-order _{^{component, I ~ + 1x, + 1z}} (k, k s, k zs) a + 1x + 1z ^{_{component, I ~ + 1x, -1z (}} k, k s, k zs) the + 1x-1z ^{_{component, I ~ -1x, + 1z (}} k, k s, k zs) the -1x + 1z ^{_{component, I ~ -1x, -1z (k}} , k s, k zs) the -1x-1z component, I ^{_{~ + 2x (k, k s}} , k zs) +2 order component, and ^I _~ -2x, called _{(k, k s, k zs} ) respectively and the minus second-order component.

The region where ill ^~ (k, k _z ) has a value is _limited to AR ₀ , AR _{+ 1x, + 1z} , AR _{+ 1x, -1z} , AR _{-1x, + 1z} , AR _{-1x, -1z} , AR _{+ 2x} , and AR _-2x . reflecting the will to be, the components _{^{I ~ 0 (k, k s}} , k zs), I ~ + 1x, + 1z (k, k s, k zs), I ~ + 1x, -1z (k, k s, k _zs ), I ^~ _{-1x, + 1z} (k, k _s , k _zs ), I ^~ _{-1x, -1z} (k, k _s , k _zs ), I ^~ _{+ 2x} (k, k _s , k _zs ), And I ^~ _-2x, (k, k _s , k _zs ) has a limited area. The region where each component has a value in the five-dimensional frequency space is when the opening of the mask 15 is circular, when the opening of the mask 15 has a shape other than circular, and when the number of openings of the mask 15 is 3 or more. Any of the above can be obtained by numerical simulation, theoretical calculation, etc. based on the design value of the microscope 1. Alternatively, the region where each component has a value can be obtained by actually measuring the fluorescent sample.

The image processing unit 7 extracts information from I 1 ^to (k, k _s , k _zs ) from a region where each component can have a value, and I ^to ₀ (k, k _s , k _zs ), I ^to _{+1x, +1z} (K, k _s , k _zs ), I ^~ _{+ 1x, -1z} (k, k _s , k _zs ), I ^~ _{-1x, + 1z} (k, k _s , k _zs ), I ^~ _{-1x, -1z} ( k, k _s , k _zs ), I ^~ _{+ 2x} (k, k _s , k _zs ), and I ^~ _-2x, (k, k _s , k _zs ) are obtained. As described in the third embodiment, this process is referred to as component separation. When executing the component separation, the region for extracting each component may be extracted from a region wider than the region in which each component may have a value obtained by numerical simulation, theoretical calculation, or may be extracted from a narrow region. Good. Further, even when there is an overlap between the components, if the overlap is small, the influence on the image processing result is small, and therefore the information of the region where the components overlap may be extracted.

^I _~ 0 extracted by performing component separation _{(k, k s, k zs} ), I ~ + 1x, + 1z (k, k s, k zs), I ~ + 1x, -1z (k, k s, k zs) , I ^~ _{-1x, + 1z} (k, k _s , k _zs ), I ^~ _{-1x, -1z} (k, k _s , k _zs ), I ^~ _{+ 2x} (k, k _s , k _zs ), and I ^~ _{For each of −2x} (k, k _s , k _zs ), five-dimensional real space data is calculated by performing a five-dimensional inverse Fourier transform on k, k _s , and k _zs .
_{^{I ~ 0 (k, k s}} , k zs) inverse Fourier transform results _I 0 of _{(r, r s, z s} ), I ~ + 1x, + 1z (k, k s, k zs) the inverse Fourier transform result of _{I + 1x, + 1z (r} , r s, z s), I ~ + 1x, -1z (k, k s, k zs) the inverse Fourier transform result of _{I + 1x, -1z (r,} r s, z s), The inverse Fourier transform results of I ^to _{−1x, +1z} (k, k _s , k _zs ) are I _{−1x, +1z} (r, r _s , z _s ), I ^to _{−1x, −1z} (k, k _s , k) inverse Fourier transform results _I -1x of _{zs), -1z (r, r} s, z s), I ~ + 2x (k, k s, the inverse Fourier transform results _I + 2x (r of _{k zs), r s, Let} z _s ), and the inverse Fourier transform result of I ^to _−2x (k, k _s , k _zs ) be I _−2x (r, r _s , z _s ).
I ₀ (r, r _s , z _s ), I _{+ 1x, + 1z} (r, r _s , z _s ), I _{+ 1x, -1z} (r, r _s , z _s ), I _{-1x, + 1z} (r, r) _{s, z s), I -1x} , -1z (r, r s, z s), I + 2x (r, r s, z s), and _I -2x _{(r, r s,} to each of the _{z s)} On the other hand, the operation shown in the equation (30) is performed.

In Expression (30), ψ(r) represents the image processing phase shift amount for each position r of the detection unit 6a of the detection device 6 as described in the third embodiment. The image processing phase shift amount ψ (r) is determined as follows, for example.
The image processing unit 7 calculates the positional deviation amount of the signal detected at the detector coordinate r. The image processing unit 7, for example, by simulation in advance, by obtaining the peak position of the distribution obtained by the product of the distribution that represents the envelope of _{PSFdet (r + r s, z} s) and ill _{(r s, z} s) , Calculate the above misalignment amount. Here, it can be considered that the position deviation of the effective PSF is proportional to the detector coordinate r, and the parameter indicating the degree of the position deviation is β, and the position deviation amount is represented by r/β. The value of β _{is, PSFdet (r + r s,} z s) and ill _{(r s, z} s) may be calculated from the peak position of the distribution obtained by the product of the distribution that represents the envelope of the other numerical It may be calculated by simulation or estimated from an observed image. Further, a different β may be used for each detection unit 6a of the detection device 6. Once β is determined, the amount of image processing phase shift according to the detector coordinates is determined. The image processing phase shift amount ψ(r) of the interference fringe L2 is the peak position of the function obtained by the product of PSFdet(r+r _s , z _s ) and the distribution representing the envelope of ill(r _s , z _s ), The peak positions of the interference fringes are determined so as to coincide with each other.
By such processing, the image processing phase shift amount becomes, for example, ψ(r)=−2πk ₀ ·r/β−φ _x . The values of the initial phase φ _x and the initial phase φ _z may be values measured in advance using fluorescent beads or values estimated from an observed image. The image processing unit 7 determines the amount of phase conversion (image processing phase shift amount) based on the light intensity distribution of the excitation light in the sample S.

The image processing unit 7 calculates the sum of the calculation results of the seven expressions of the above expression (30) as shown in the following expression (31).

I obtained from the calculation of the above formula _{(31) '(r, r} s, z s) are corrected collapse of the shape of the effective PSF for each position r of the detection part 6a, and substantially the same shape of the effective PSF It becomes an image that became. In this embodiment, it is possible to correct not only the initial phase φ _x of the interference fringe L2 in the X direction but also the initial phase φ _z of the Z direction. This makes it possible to correct, for example, the phase error of the interference fringe L2 caused by the aberration of the optical system or the error of the device in both the X direction and the Z direction.

The image processing unit 7 corrects the position shift of the effective PSF for each detection unit 6 a of the detection device 6 with respect to I′(r, r _s , z _s ). As a result, the effective PSF can be made substantially the same in the two or more detection units 6a of the detection device 6. The image processing unit 7 generates a three-dimensional super-resolution image I _SR (r _s , z _s ) by adding the images of the respective detection units 6 a whose effective PSFs have been corrected to be substantially the same. This series of processing is performed based on the equation (32).

Here, PH (r) is a pinhole function defined by the equation (22). The signal amount can be increased by increasing the value of r _{PH in} Expression (22), and the sectioning ability can be increased by decreasing the value of r _PH .

In the present embodiment, the scan interval and the interval of the detection unit 6a of the detection device 6 may be set based on the cutoff frequency and the Nyquist theorem. The scan interval may be set to λ _ex /8NA or less in the periodic direction of the interference fringes in the direction orthogonal to the optical axis. Further, the scan interval may be set to λ _ex / 4 NA or less in the direction perpendicular to the periodic direction of the interference fringes in the direction orthogonal to the optical axis. The scan interval may be set to λ _ex /(8(n−√(n ² −NA ² ))) or less in the periodic direction of the interference fringes in the optical axis direction. Here, n is the refractive index of the immersion liquid of the objective lens 21. Further, the interval between the detection units 6a of the detection device 6 may be set to λ _em /4NA or less. When the magnification of the detection optical system is different from 1, the numerical aperture on the detection device 6 side of the detection optical system 5 is set to NAd, and for example, the size of the detection unit 6a is preferably set to about λ _em /4NAd. .. Alternatively, when the size of the detection unit 6a is d, the focal length of the lens 23 may be set such that the numerical aperture NAd of the detection optical system 5 on the detection device 6 side satisfies NAd<λ _em /4d. ..

FIG. 15 is a flowchart showing an observation method according to the fifth embodiment. The processing of steps S1 to S4 is the same as that of FIG. 7, and the description thereof is omitted. The processing from step S24 to step S26 is the same as that in FIG. 12, and the description thereof is omitted. In step S41, the control unit CB sets the relative position of the interference fringe L2 and the sample S in the Z direction by the stage 2. In step S42, the control unit CB determines whether or not to change the relative positions of the interference fringes L2 and the sample S in the Z direction. The control unit CB determines to change the relative position of the interference fringe L2 and the sample S in the Z direction when it is determined that the processing of steps S41 to S4 has not been completed for a part of the scheduled observation region (( Step S42; Yes). When determining that the relative position of the interference fringe L2 and the sample S in the Z direction is changed (step S41; Yes), the control unit CB returns to the process of step S41, and the stage 2 causes the interference fringe L2 and the sample S to move in the Z direction. Set the relative position of. Then, the processes of steps S1 to S4 are repeated. In this way, the microscope 1 sets the relative position in the Z direction between the interference fringe L2 and the sample S by the stage 2, and the illumination optical system 4 sets the relative fringe of the excitation light L1 by the illumination optical system 4 at each set relative position. The sample S is three-dimensionally scanned by two-dimensionally scanning.

When it is determined that the processes of steps S41 to S4 have been completed for all of the scheduled observation regions, the control unit CB determines not to change the relative position of the interference fringe L2 and the sample S in the Z direction (step S42). ; No). When the control unit CB determines that the relative position of the interference fringe L2 and the sample S in the Z direction is not changed (step S42; No), the image processing unit 7 determines at least a part of the plurality of detection units 6a in step S43. The detection result is subjected to five-dimensional Fourier transform with respect to the detector coordinate r (two-dimensional), the scan coordinate r _s (two-dimensional), and z _s . In step S44, the image processing unit 7 separates the components in the frequency space. The image processing unit 7 separates the components of the five-dimensional frequency space obtained by the five-dimensional Fourier transform into regions of the frequency space. In step S45, the image processing unit 7 performs a five-dimensional inverse Fourier transform on the separated component with respect to the detector coordinate r (two-dimensional), the scan coordinate r _s (two-dimensional), and z _s .

[Sixth Embodiment]
The sixth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are appropriately designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 16 is a diagram showing a microscope according to a sixth embodiment. The microscope 1 according to this embodiment includes a phase modulation element 25 in the illumination optical system 4. The phase modulation element 25 is driven by the phase modulation element drive mechanism (drive unit 26), and the phase of the light and dark of the interference fringe L2 can be changed. In the present embodiment, the image processing unit 7 (see FIG. 4) acquires an image in which the phase of the interference fringe L2 is changed a plurality of times, and separates components using a plurality of images having different phases of the interference fringe L2. After that, the separated components are subjected to a series of processes following the image processing phase shift process described in the third embodiment to generate an image.

The phase modulation element 25 is, for example, a substantially disk-shaped glass plate, and can be rotated around the optical axis 4a by the torque supplied from the phase modulation element drive section (drive section 26). FIG. 17 is a diagram showing an example of the phase modulation element 25. 17(A) to 17(F) are views of the phase modulation element 25 seen from the Za direction in FIG. 16, and show a state in which one phase modulation element is rotated about the center Cn by mutually different angles. ing. The phase modulation element 25 is arranged so that the center Cn substantially coincides with the optical axis 4a of the illumination optical system 4 of FIG.

The thickness of the substantially disk-shaped phase modulation element 25 is different in the regions Ph0, Ph + 1, Ph + 2, Ph + 3, Ph + 4, Ph-1, Ph-2, Ph-3, and Ph-4, and is transmitted through each region. A phase difference is given between the luminous fluxes. The phase modulation element 25 gives a phase difference between the light rays transmitted through each region, and changes the phase of the light and dark of the interference fringes L2.

In the case of the phase modulation element 25 with the rotation angle shown in FIG. 17B, the excitation light L1a passing through the opening 15a, the excitation light L1b passing through the opening 15b, and the excitation light L1c passing through the opening 15c are both Since the region Ph0 is transmitted, no phase difference is added between the excitation light L1a, the excitation light L1b, and the excitation light L1c.

However, in the case of the phase modulation element 25 having the rotation angle shown in FIG. 17C, the excitation light L1a transmits the region Ph+1, the excitation light L1b transmits the region Ph-1, and the excitation light L1c transmits the region Ph0. Therefore, a predetermined first phase difference is added between the excitation light L1a, the excitation light L1b, and the excitation light L1c, respectively. For example, in the case of the phase modulation element 25 having the rotation angle shown in FIG. 17C, the pump light L1a has a phase difference of φ1 with respect to the pump light L1c, and the pump light L1b has a phase difference of −φ1 with respect to the pump light L1c. It may be added. Here, for example, π/5 [rad] may be selected as the value of φ1.

Similarly, in the case of the phase modulation element 25 having the rotation angle shown in FIG. 17D, the excitation light L1a transmits the region Ph+2, the excitation light L1b transmits the region Ph-2, and the excitation light L1c transmits the region Ph0. Therefore, a predetermined second phase difference is added between the excitation light L1a, the excitation light L1b, and the excitation light L1c, respectively. For example, in the case of the phase modulation element 25 having the rotation angle shown in FIG. 17D, the pump light L1a has a phase difference of φ2 with respect to the pump light L1c, and the pump light L1b has a phase difference of −φ2 with respect to the pump light L1c. It may be added. Here, for example, 2π/5 [rad] may be selected as the value of φ2.

Similarly, in the case of the phase modulation element 25 having the rotation angle shown in FIG. 17E, the excitation light L1a transmits the region Ph+3, the excitation light L1b transmits the region Ph-3, and the excitation light L1c transmits the region Ph0. Therefore, a predetermined third phase difference is added between the excitation light L1a, the excitation light L1b, and the excitation light L1c, respectively. For example, in the case of the phase modulation element 25 having the rotation angle shown in FIG. 17E, the pump light L1a has a phase difference of φ3 with respect to the pump light L1c, and the pump light L1b has a phase difference of −φ3 with respect to the pump light L1c. It may be added. Here, for example, 3π/5 [rad] may be selected as the value of φ3.

Similarly, in the case of the phase modulation element 25 having a rotation angle shown in FIG. 17 (F), the excitation light L1a transmits the region Ph + 4, the excitation light L1b transmits the region Ph-4, and the excitation light L1c transmits the region Ph0. Therefore, a predetermined fourth phase difference is added between the excitation light L1a, the excitation light L1b, and the excitation light L1c, respectively. For example, in the case of the phase modulation element 25 having the rotation angle shown in FIG. 17F, the pump light L1a has a phase difference of φ4 with respect to the pump light L1c, and the pump light L1b has a phase difference of −φ4 with respect to the pump light L1c. It may be added. Here, for example, 4π/5 [rad] may be selected as the value of φ4.

Therefore, by sequentially changing the rotation angle of the phase modulation element 25 from the state shown in FIG. 17B, the phase difference between the excitation light L1a, the excitation light L1b, and the excitation light L1c can be sequentially changed. , The light and dark phases of the interference fringes L2 can be changed.

Note that the embodiment is described assuming that the region through which the excitation light L1c passes is the region Ph0 regardless of the rotation angle of the phase modulation element 25. However, the phase modulation element 25 is configured so that the phase of the excitation light L1c is changed. May be. Further, the values of the phase differences φ1 to φ4 may be other than the values described above. As will be described later, the value of the phase difference and the number of times the phase difference is changed may be set so that the components can be separated.

Further, the respective regions Ph0, Ph+1 to Ph+4, Ph-1 to Ph-4 of the phase modulation element 25 are not limited to those in which the thickness of the phase modulation element 25 itself is different for each region as described above. , A thin film having a predetermined thickness may be formed in at least nine regions of a glass disk having a uniform thickness. It is also possible to arrange a plurality of glass discs having different thicknesses for each region and drive each of the plurality of glass discs. Further, an element such as a liquid crystal SLM or a MEMS mirror may be used as the phase modulation element. You may combine and use the phase modulation element from which a kind differs.

When the phase modulation element 25 adds a phase difference of Φ to the excitation light L1c with respect to the excitation light L1a and a phase difference of −Φ with respect to the excitation light L1c to the excitation light L1b, the sample surface Sa The electric field intensity ill(r) of the excitation light at is expressed by the following equation (33).

In the state shown in FIG. 17B, Φ=0, and in the state shown in FIG. 17C, Φ=φ1, and in the state shown in FIG. In the state shown, Φ=φ2, in the state shown in FIG. 17E, Φ=φ3, and in the state shown in FIG. 17F, Φ=φ4. Further, changing the phase of the interference fringe L2 by the phase modulation element 25 corresponds to changing the phase Φ without changing PSFill(r) on the right side of Expression (33). The fact that PSFill (r) does not change means that the envelope (envelope) of the intensity distribution of the interference fringes L2 does not change. Therefore, the phase modulation element 25 changes the light and dark phases of the intensity distribution of the interference fringes L2 without moving the envelope (envelope) of the intensity distribution of the interference fringes L2.

The microscope 1 scans the interference fringe L2 on the sample surface Sa by the scanning unit 18 when the rotation angle of the phase modulation element 25 is one of the states shown in FIGS. 17(B) to (F), and the detector 6 detects the fluorescence L3. To detect. For example, the microscope 1 illuminates the illumination region selected from the sample surface Sa with the interference fringes L2, and the detection device 6 detects the fluorescence L3 from the illumination region. The microscope 1 changes the illumination area by the scanning unit 18 after the detection by the detection device 6 is completed. The microscope 1 acquires a fluorescence intensity distribution (measured value of the detection device 6) in a desired region by repeating a process of detecting fluorescence and a process of changing the illumination region.

By substituting the equation (33) into the equation (3), the image data I(r, r _s ; Φ) can be obtained by the following equation (34) as in the case of the equation (12) of the third embodiment. Given in.

In Expression (34), in order to clarify the dependence of the phase Φ of the interference fringe L2, Φ is explicitly written as an argument and written as I(r,r _s ;Φ).

Two-dimensional Fourier transform (2D-FFT) is performed on the scan coordinates r _s (two-dimensional) for the image data I(r, r _s ; Φ) obtained by scanning the interference fringes L2 at the predetermined phase Φ. .. The data obtained by the Fourier transform is equivalent to equation (34) for scanning coordinates r _s below which is obtained by Fourier transform equation (35).

In the formula (35), a _{OTF'det (r, k s) =} e i2πksr OTFdet (k s). The equation (35) is the sum of five terms. As shown in the following equation (36), each term is converted into the 0th order component I 1 ^to ₀ (r, k _s ) and the +1st order component I ₁ ^to _{+1. (r, k s), -} 1 -order component ^{_{I ~ -1 (r, k s}} ), + 2 -order component ^{_{I ~ +2 (r, k s}} ), and -2-order component ^{_{I ~ -2 (r, k s}} ) Call. These components are essentially equivalent to the components described in the third embodiment.

In the present embodiment, as in the third embodiment, the ±first-order components and the ±second-order components in equation (36) are multiplied by a predetermined phase shift amount ψ to match the coordinates of the detection unit 6a. A correction process is performed to shift the phase of the interference fringe L2 in an image processing manner.

However, in this embodiment, unlike the third embodiment, since not performed a Fourier transform with respect to the detector coordinate r, the image data I obtained by the scanning of the interference fringes _{L2 (r, r s; φ} ) scan coordinate r of 0-order component obtained two-dimensional Fourier transform results for ^{_{s I ~ 0 (r, k}} s), + 1 -order component ^{_{I ~ +1 (r, k s}} ), - 1 -order component ^{_{I ~ -1 (r, k s}} ) , +2nd-order component I 1 ^to ₊₂ (r,k _s ) and −2nd-order component I 1 ^to ₋₂ (r,k _s ) respectively spread in the frequency space and overlap each other. It is difficult to separate clearly.

In the present embodiment, scanning is performed in a plurality of different phases of the interference fringes L2 to acquire image data. Based on the data obtained by Fourier transform the plurality of image data with respect to the scanning coordinates respectively, zero-order component ^{_{I ~ 0 (r, k s}} ), + 1 -order component ^{_{I ~ +1 (r, k s}} ), - 1 -order The components I ₁ ^to ₋₁ (r, k _s ), the +second-order components I ^to ₊₂ (r, k _s ) and the −second-order components I ^to ₋₂ (r, k _s ) can be separated from each other.

Here, it is assumed that images for five phases of 0, φ1, φ2, φ3, and φ4 are acquired for the phase Φ of the interference fringe L2. Then, the Fourier transform for the scan coordinate r _s of the image I(r, r _s ; Φ) obtained for each phase is represented by the following Expression (37).

When equation (37) is written in matrix format, it becomes equation (38).

Therefore, expression from the (38) 0-order component ^{_{I ~ 0 (r, k s}} ), + 1 -order component ^{_{I ~ +1 (r, k s}} ), - 1 -order component ^{_{I ~ -1 (r, k s}} ), + 2 -order The components I 1 ^to ₊₂ (r, k _s ) and the −2nd order components I 1 ^to ₋₂ (r, k _s ) can be obtained using the following equation (39).

From the above, 0th-order component I ^to ₀ (r, k _s ), + 1st-order component I ^to ₊₁ (r, k _s ), _-1st- order component I ^to _-1 (r, k _s ), + 2nd-order component I ^to ₊₂ In order to obtain (r, k _s ) and the −2nd order component I ^to ₋₂ (r, k _s ) (separate the components), φ1, φ2, φ3, and φ4 are inversed in the equation (39). It can be seen that it is sufficient to set the matrix so that it exists. For example, as described above, for example, φ1, φ2, φ3, and φ4 may be set to π/5, 2π/5, 3π/5, and 4π/5 [rad], respectively.

In this way, under the phase Φ of the interference fringes L2 of a plurality of types (five as an example), the plurality of images obtained by scanning the interference fringes L2 on the sample surface Sa by the scanning unit 18 are respectively obtained. Image is subjected to Fourier transform at scan coordinates r _s , and from the result of the Fourier transform, the 0th-order component I 1 ^to ₀ (r, k _s ) and the +1st-order component I ₁ ^to ₊₁ (r, k _s), - 1-order component ^{_{I ~ -1 (r, k s}} ), + 2 -order component ^{_{I ~ +2 (r, k s}} ), and -2-order component ^I _~ -2 (r, determine the _{k s)} (extract This is also called component separation.

Then, zero-order component ^I _~ obtained by the component separation _{0 (r, k s),} + 1 -order component _{^{I ~ +1 (r, k s}} ), - 1 -order component _{^{I ~ -1 (r, k s}} ), +2 order component _{^{I ~ +2 (r, k s}} ), and -2-order component _{^{I ~ -2 (r, k s}} ) for each, (wave number coordinate after the Fourier transform of the scan coordinates _{r s)} wavenumber coordinates k _s Performs an inverse Fourier transform (2D-inverse FFT) for.

^{_{I ~ 0 (r, k s}} ) of image data obtained by performing inverse Fourier transform on represented by _{I 0 (r, r s)} . ^{_{I ~ +1 (r, k s}} ) of image data obtained by performing inverse Fourier transform on represented by _{I +1 (r, r s)} . ^{_{I ~ -1 (r, k s}} ) of image data obtained by performing inverse Fourier transform on expressed in _{I -1 (r, r s)} . ^{_{I ~ +2 (r, k s}} ) of image data obtained by performing inverse Fourier transform on represented by _{I +2 (r, r s)} . ^{_{I ~ -2 (r, k s}} ) of image data obtained by performing inverse Fourier transform on the representative in _{I -2 (r, r s)} .

The thus obtained _{I 0 (r, r s)} , I +1 (r, r s), I -1 (r, r s), I +2 (r, r s), and _I -2 (r , R _s ) are I ₀ (r, r _s ), I ₊₁ (r, r _s ), I ₋₁ (r, r _s ), and I ₊₂ (r, r _s ) described in the third embodiment. , and _I -2 _(r, r _s) is equivalent to. Therefore, the image processing phase shift processing described in the third embodiment, the processing for correcting the positional deviation of the effective PSF for each detection unit 6a of the detection device 6, and the detection unit 6a in which the effective PSFs are corrected to be substantially the same. It is possible to apply a process of adding the images for each.

For I ₀ (r,r _s ), I ₊₁ (r,r _s ), I ₋₁ (r,r _s ), I ₊₂ (r,r _s ), and I ₋₂ (r,r _s ). , The operation of equation (19) is performed. The calculation of the formula (20) is performed on the calculation result of the formula (19). I′(r,r _s ) obtained from the calculation of the equation (20) becomes an image in which the collapse of the shape of the effective PSF for each position r of the detection unit 6a is corrected and the shape of the effective PSF is almost the same. ..

The image processing unit 7 corrects the position shift of the effective PSF for each detection unit 6 a of the detection device 6 with respect to I′(r, r _s ). Thereby, the effective PSFs can be made substantially the same in the two or more detection units 6a of the detection device 6. The image processing unit 7, by adding the image for each detector 6a the effective PSF is corrected to almost the same, to produce a super-resolution image I _{SR (r s).} This series of processing is performed based on the equation (21).

As described above, in the present embodiment, a plurality of images acquired by scanning the interference fringe L2 on the sample surface Sa by the scanning unit 18 under a plurality of (five as an example) the phase Φ of the interference fringe L2. Component separation is performed using the image processing phase shift processing for each detection unit 6a of the detection device 6, the positional deviation correction process of the effective PSF for each detection unit 6a of the detection device 6, and the effective PSF are corrected to be substantially the same. The processing of adding the images of each detection unit 6a is performed.

FIG. 18 is a flowchart showing an observation method according to the sixth embodiment. The processing of steps S1 to S4 is the same as that of FIG. 7, and the description thereof is omitted. The processing from step S24 to step S26 is the same as that in FIG. 12, and the description thereof is omitted. In step S51, the control unit CB sets the phase of the interference fringe L2. In step S52, the control unit CB determines whether to change the phase of the interference fringe L2. The control unit CB determines to change the phase of the interference fringe L2 when it is determined that the processes of steps S51 to S4 have not been completed for a part of the scheduled observation region (step S52; Yes). When determining that the phase of the interference fringe L2 is changed (step S52; Yes), the control unit CB returns to the process of step S51 and sets the phase of the interference fringe L2 again. Then, the processes of steps S1 to S4 are repeated. In this way, in the microscope 1, the illumination optical system 4 two-dimensionally scans the sample S with the plurality of interference fringes L2 of the excitation light L1.

When it is determined that the processes of steps S51 to S4 have been completed for all of the scheduled observation regions, the control unit CB determines not to change the phase of the interference fringe L2 (step S52; No). When the control unit CB determines that the phase of the interference fringe L2 is not changed (step S52; No), the image processing unit 7 scans at least a part of the detection results of the plurality of detection units 6a in step S53. Perform a two-dimensional Fourier transform on the coordinates r _s (two-dimensional). In step S54, the image processing unit 7 separates the components in the frequency space by, for example, solving the simultaneous equations represented by the equation (39). In step S55, the image processing section 7 performs to the separated components, a two-dimensional inverse Fourier transform of about scanning coordinates r _{s (two-dimensional).}

In the present embodiment, the microscope 1 may scan the interference fringes L2 two-dimensionally by scanning the interference fringes L2 in two directions parallel to the sample surface Sa, or may be parallel to the sample surface Sa. The interference fringes L2 may be scanned three-dimensionally by scanning the interference fringes L2 in two directions and in one direction perpendicular to the sample surface Sa. When the interference fringes L2 are three-dimensionally scanned, the process of scanning the interference fringes L2 in the two directions parallel to the sample surface Sa (hereinafter referred to as the two-dimensional process) is the same as the process described in the above embodiment. is there. The microscope 1 can generate, for example, a three-dimensional super-resolution image by repeating the two-dimensional processing by changing the position in the Z direction. Similarly to the embodiments described below, the microscope 1 may three-dimensionally scan the interference fringes L2.

The microscope 1 according to the embodiment has improved resolution in the periodic direction (X direction in FIG. 16) of the interference fringes L2. The microscope 1 can also improve the three-dimensional resolution of XYZ by changing the periodic direction of the interference fringe L2 and detecting the fluorescence from the sample S. The periodic direction of the interference fringe L2 can be changed by rotating the mask 15 in a desired direction. In this case, the rotation angle of the phase modulation element 25 may be changed according to the rotation angle of the mask 15.

In this embodiment, component separation is performed using a plurality of images acquired by scanning the interference fringe L2 on the sample surface Sa by the scanning unit 18 under the phase Φ of the plurality of interference fringes L2 (five as an example). Was executed. This zero-order component ^{_{I ~ 0 (r, k s}} ), + 1 -order component ^{_{I ~ +1 (r, k s}} ), - 1 -order component ^{_{I ~ -1 (r, k s}} ), + 2 -order component ^I _{~ +2} This is because the overlap of (r, k _s ) and the −2nd order component I ^to ₋₂ (r, k _s ) in the frequency space is large. However, for example, the respective overlaps of the +2nd-order component I 1 ^to ₊₂ (r,k _s ) and the _−2nd- order component I ^to ₋₂ (r,k _s ) in the frequency space may be sufficiently small, and the wavenumber space In some cases, the components can be separated by extracting data from the area where the +second-order component has a value and the area where the −second-order component has a value. In such a case, all the components can be separated even if the number of changes of the phase Φ of the interference fringe L2 is reduced.

Further, in performing the component separation, the component separation method described in the present embodiment and the component separation method described in the third embodiment may be appropriately combined. In this case, the number of times the phase Φ of the interference fringe L2 is changed may be determined according to the number of components to be separated.

For example, when it is desired to separate the +1x+1z component and the +1x-1z component described in the fifth embodiment without acquiring an image by three-dimensional scanning, the mixed component of the +1x+1z component and the +1x-1z component is separated from the other components. In order to separate the +1x+1z component and the +1x-1z component from the mixed component of the +1x+1z component and the +1x-1z component, the component separation described in the third embodiment is used. Good. In this case, using the two image data of the image data acquired with the first phase difference and the image data acquired with the second phase difference, from the mixed component of the +1x+1z component and the +1x-1z component to the +1x+1z component. The + 1x-1z component can be separated. Here, the first phase difference is a phase difference in which no phase difference is added between the excitation light L1a, the excitation light L1b, and the excitation light L1c. The second phase difference is such that the pump light L1a has a phase difference of π[rad] added to the pump light L1c, and the pump light L1b has a phase difference of π[rad] added to the pump light L1c. It is a phase difference. By such a method, the initial phase correction in the z direction described in the fifth embodiment can be performed on the image data obtained by the two-dimensional scanning.

[Seventh Embodiment]
The seventh embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 19 is a diagram showing a microscope according to a seventh embodiment. In the present embodiment, the detection device 6 includes a line sensor (line detector) in which a plurality of detection units 6a are arranged one-dimensionally. The plurality of detection units 6a are arranged in one direction in the detection device 6. The detection device 6 is arranged at a position optically conjugate with the sample surface Sa. The direction in which the plurality of detection units 6a are arranged (hereinafter referred to as the arrangement direction) is set to the direction corresponding to the periodic direction of the interference fringes L2. For example, in FIG. 19, the periodic direction of the interference fringes is the X direction, and the array direction of the plurality of detection units 6a is set to the Xb direction corresponding to the X direction.

The microscope 1 according to this embodiment includes a λ/2 wavelength plate 30 and an optical path rotating unit 31 that rotates the optical path around the optical axis. The λ/2 wave plate 30 rotates the polarized light passing through the optical path rotating unit 31 based on the rotation angle of the optical path by the optical path rotating unit 31. The optical path rotating unit 31 is arranged in the optical path between the mask 15 and the sample S in the illumination optical system 4. The optical path rotating unit 31 is arranged, for example, at a position where the excitation light L1 becomes substantially parallel light in the optical path of the illumination optical system 4. The optical path rotating unit 31 is arranged, for example, at a position where the excitation light L1 passes through in the illumination optical system 4 and the fluorescence L3 passes through in the detection optical system 5. The optical path rotating unit 31 is arranged in the optical path between the dichroic mirror 16 and the sample S, for example. The λ/2 wave plate 30 may be arranged on the same side as the sample S with respect to the optical path rotating unit 31, or on the side opposite to the sample S with respect to the optical path rotating unit 31 (for example, on the same side as the excitation light source). ).

The optical path rotating unit 31 is, for example, an image rotator such as a Dove prism. The optical path rotating unit 31 is provided rotatably around the optical axis of the illumination optical system 4. The optical path rotating unit 31 is driven by the driving unit 32 and rotates. When a Dove prism is used as the optical path rotating unit 31, when the Dove prism is rotated by θ° around the optical axis of the illumination optical system 4, the optical path on the light emission side (the sample S side) from the Dove prism is directed to the Dove prism. The optical path on the light incident side (light source 3 side) is rotated by 2×θ° around the optical axis of the illumination optical system 4. As a result, the incident surface of the excitation light L1 on the sample S rotates 2×θ° around the Z direction, and the periodic direction of the interference fringes L2 rotates 2×θ° around the Z direction. For example, when the periodic direction of the interference fringe L2 is changed by 90 °, the driving unit 32 rotates the optical path rotating unit 31 by 45 ° around the optical axis of the illumination optical system 4. As described above, the optical path rotating portion 31 is included in the fringe direction changing portion that changes the direction of the interference fringes with respect to the sample.

The λ/2 wave plate 30 is provided so as to be rotatable around the optical axis of the illumination optical system 4. The λ / 2 wave plate 30 rotates in conjunction with the optical path rotating portion 31. The λ / 2 wave plate 30 rotates by an angle determined based on the rotation angle of the optical path rotating portion 31. For example, the λ/2 wave plate 30 is fixed (for example, integrated) with the optical path rotating unit 31 and rotates together with the optical path rotating unit 31. In this case, the λ/2 wave plate 30 rotates by the same angle as the rotation angle of the optical path rotating unit 31.

When the λ/2 wave plate 30 is rotated by θ° around the optical axis of the illumination optical system 4, the polarization direction of the excitation light L1 is different from the polarization direction on the light incident side (the light source 3 side). Rotate 2 × θ ° around the optical axis of. As a result, the polarization state of the excitation light L1 when entering the sample S becomes S-polarized.

The optical path rotating unit 31 of FIG. 19 is also included in the image rotating unit. The image rotation unit rotates the image of the sample S (for example, the image of fluorescence from the sample S) with respect to the plurality of detection units 6a around the optical axis of the detection optical system 5. That is, the fringe direction changing portion and the image rotating portion include the optical path rotating portion 31 as the same member (optical member). The optical path rotating unit 31 is arranged at a position on the optical path of the illumination optical system 4 where fluorescence enters. The image rotating unit rotates the fluorescence image by the optical path rotating unit 31. The optical path rotation unit 31 adjusts the cycle direction of the interference fringes L2 with respect to the arrangement direction of the plurality of detection units 6a in the detection device 6. When the Dove prism is used as the optical path rotating unit 31, when the Dove prism is rotated by θ° around the optical axis of the illumination optical system 4, the cycle direction of the interference fringes L2 is rotated by 2×θ° around the Z direction. Then, the optical path of the fluorescence L3 from the sample S rotates −2×θ° on the light emitting side (detecting device 6 side) with respect to the light incident side (sample S side) on the Dove prism.

When the dub prism is rotated, the optical path of light toward the sample S through the dub prism is rotated, and the periodic direction of the interference fringe L2 with respect to the sample S changes. Further, the optical path of the light from the sample S to the detection device 6 via the dub prism rotates in the opposite direction to the optical path of the light toward the sample S by the same angle. Therefore, when the images of the plurality of detection units 6a (for example, line detectors) in the detection device 6 are projected onto the sample surface Sa through the detection optical system 5, the direction in which the plurality of detection units 6a are arranged and the periodic direction of the interference fringes. And always match even when the periodic direction of the interference fringes is changed by the Dove prism. Therefore, the detection device 6 can detect the fluorescence L3 in the same manner before and after the change of the periodic direction of the interference fringe L2.

As described in the sixth embodiment, the illumination optical system 4 may be provided with the phase modulation element 25.
The resulting image data I (x, _{r s)} is, for example, a three-dimensional data with detector coordinates x and scan coordinate _{r s = (x s, y} s) of the independent variable (in the case of 2D scans). Or, the resulting image data _{I (x, r s, z} s) , for example, with detector coordinates x, scan coordinate _{r s = (x s, y} s) to, and z directions of the scan coordinate _{z s} independently variable It is four-dimensional data (in the case of 3D scanning). In the first to sixth embodiments, the detector coordinates are two-dimensional because the detection device 6 includes a plurality of detection units 6a arranged in two dimensions, but in the seventh embodiment, the detection device Since 6 includes a plurality of detectors 6a arranged one-dimensionally, the detector coordinates are one-dimensional. In the image processing described in the first to sixth embodiments, even when the detector coordinates are one-dimensional, the processing relating to the detector coordinates is made one-dimensional (for example, Fourier transform for the detector coordinates in the third embodiment is It can also be applied to the seventh embodiment by performing it as a one-dimensional Fourier transform). In this way, the image processing unit 7 generates an image by the processing described in the first to sixth embodiments based on the detection result of the detection device 6.

In the first embodiment, the microscope 1 changes the cycle direction of the interference fringes L2 by rotating the mask 15 by the driving unit 22, but the interference fringes L2 are changed by the optical path rotating unit 31 (for example, the Dove prism). The cycle direction of may be changed. The fringe direction changing unit that changes the cycle direction of the interference fringes L2 may be different from both the drive unit 22 and the optical path rotating unit 31. For example, the stage 2 may be rotatably provided around the Z direction, and the rotation may change the direction of the interference fringes L2 with respect to the sample S. In this case, the stage 2 is included in the fringe direction changing portion that changes the direction of the interference fringe L2 with respect to the sample S.

The microscope 1 shown in FIG. 20 differs from that in FIG. 19 in the position where the optical path rotating unit 31 is provided. In FIG. 20, the fringe direction changing portion is the same as that of the first embodiment, and includes the mask 15 and the driving portion 22. The optical path rotating portion 31 also serves as a fringe direction changing portion and an image rotating portion in FIG. 19, but the optical path rotating portion 31 is provided separately from the fringe direction changing portion in FIG. 20. In FIG. 20, the optical path rotating unit 31 is arranged at a position in the optical path of the detection optical system 5 that does not overlap with the optical path of the illumination optical system 4. The optical path rotating unit 31 is arranged at a position where the excitation light L1 does not enter and the fluorescence L3 enters. The optical path rotating unit 31 is arranged in the optical path between the dichroic mirror 16 and the detection device 6.

In the microscope 1, the driving unit 22 changes the periodic direction of the interference fringes L2 by rotating the mask 15 and the polarizer 14. The drive unit 32 rotates the optical path rotation unit 31 by an angle determined based on the rotation angle of at least one of the mask 15 and the polarizer 14. In the microscope 1, the driving unit 32 rotates the optical path rotating unit 31 to align the direction of the image projected on the detection device 6 with respect to the direction in which the plurality of detecting units 6a are lined up.

[Eighth Embodiment]
The eighth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 21 is a diagram showing a microscope according to an eighth embodiment. In this embodiment, the microscope 1 includes a light blocking member 33. The light shielding member 33 is arranged at a position optically conjugate with the sample surface Sa or in the vicinity thereof. In FIG. 21, the detection device 6 is arranged at a position optically conjugate with the sample surface Sa, and the light shielding member 33 is arranged near the detection device 6. The light blocking member 33 may be arranged at a position conjugate with the sample surface Sa or in the vicinity thereof.

The light shielding member 33 has an opening 33a through which the fluorescence L3 passes, and shields the fluorescence L3 around the opening 33a. The opening 33a extends in the arrangement direction (Xb direction) of the plurality of detection units 6a in the detection device 6. The opening 33a is, for example, a rectangular slit. The light-shielding member 33 is arranged so that the long side of the opening 33a is substantially parallel to the arrangement direction of the plurality of detection units 6a. The light shielding member 33 may be variable in the size of the opening 33a and one or both of the shapes. For example, the light blocking member 33 may be a mechanical diaphragm having a variable light blocking area or a spatial light modulator (SLM). The dimension of the opening 33a and one or both of the shapes may be fixed.
As described in the sixth embodiment, the illumination optical system 4 may be provided with the phase modulation element 25.

The detection device 6 detects the fluorescence L3 that has passed through the opening 33a of the light shielding member 33. The image processing unit 7 generates an image based on the detection result of the detection device 6. The process performed by the image processing unit 7 may be any of the processes described in the first to sixth embodiments, as described in the seventh embodiment.

[Ninth Embodiment]
The ninth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 22 is a diagram showing a microscope according to a ninth embodiment. In the present embodiment, the microscope 1 includes a driving unit 22 and a driving unit 34. The drive unit 22 is similar to that of the first embodiment. The drive unit 22 rotates the mask 15 to change the cycle direction of the interference fringe L2. The driving unit 22 is included in the fringe direction changing unit that changes the direction of the interference fringe L2 with respect to the sample S.

In the present embodiment, the detection device 6 can rotate around the Zb direction. The drive unit 34 rotates the detection device 6 around the Zb direction. The drive unit 34 rotates the detection device 6 so that the arrangement direction of the detection units 6a in the detection device 6 corresponds to the cycle direction of the interference fringes L2. For example, when the drive unit 22 rotates the mask 15 by 90 °, the periodic direction of the interference fringe L2 changes by 90 °, so that the drive unit 34 rotates the detection device 6 by 90 °.

The drive unit 34 also rotates the light shielding member 33 so that the relative position between the detection device 6 and the light shielding member 33 is maintained. For example, the light blocking member 33 and the detection device 6 are integrated, and the driving unit 34 integrally rotates the light blocking member 33 and the detection device 6.

Note that the microscope 1 may include the optical path rotating unit 31 shown in FIG. 20, instead of rotating the detection device 6. Further, the microscope 1 may not include the light shielding member 33 as shown in FIG.

[Tenth Embodiment]
The tenth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 23 is a diagram showing a microscope according to the tenth embodiment. In the above embodiment, an example in which the illumination pupil is divided into three poles (three regions) on the pupil plane P0 (see FIG. 2C) has been described, but the illumination pupil may have other forms. Here, a form in which the illuminated pupil is divided into five poles (five regions) on the pupil surface will be described.

The illumination optical system 4 according to this embodiment includes a collimator lens 12, a λ/2 wavelength plate 35, a polarization separation element 36, a mirror 37, a mask 38 (aperture member), a mirror 39, and a mask on the light emission side of the optical fiber 11. 40 (opening member) and a polarization separating element 41 are provided. The illumination optical system 4 is similar to the first embodiment in the configuration from the dichroic mirror 16 to the objective lens 21.

The excitation light L1 emitted from the optical fiber 11 is converted into substantially parallel light by the collimator lens 12 and enters the λ/2 wavelength plate 35. The excitation light L1 that has passed through the λ/2 wave plate 35 includes excitation light L1d that is linearly polarized light in the first direction and excitation light L1e that is linearly polarized light in the second direction. The λ/2 wave plate 35 has its optical axis (fast axis, slow axis) direction set such that the light quantity of the pump light L1d and the light quantity of the pump light L1e have a predetermined ratio.

The pumping light L1 (the pumping light L1d and the pumping light L1e) that has passed through the λ/2 wavelength plate 35 enters the polarization separation element 36. The polarization separation element 36 has a polarization separation film 36a tilted with respect to the optical axis 12a of the collimator lens 12. The polarization separation film 36a has a characteristic that linearly polarized light in the first direction is reflected and linearly polarized light in the second direction is transmitted. The polarization separating element 36 is, for example, a polarization beam splitter prism (PBS prism). The linearly polarized light in the first direction is S polarized light with respect to the polarization separation film 36a. The linearly polarized light in the second direction is P-polarized light with respect to the polarization separation film 36a.

The S-polarized excitation light L1d for the polarization separation film 36a is reflected by the polarization separation film 36a and enters the mask 38 via the mirror 37. The P-polarized excitation light L1e for the polarization splitting film 36a passes through the polarization splitting film 36a and enters the mask 40 via the mirror 39. The mask 38 and the mask 40 are luminous flux dividing portions that divide the excitation light that excites the fluorescent substance into a plurality of luminous fluxes. The mask 38 and the mask 40 will be described later with reference to FIG. 24.

The excitation light L1d passing through the mask 38 and the excitation light L1e passing through the mask 40 are incident on the polarization separating element 41, respectively. The polarization separation element 41 has a polarization separation film 41a inclined with respect to the optical path of the excitation light L1d and the optical path of the excitation light L1e. The polarization separation film 41a has a characteristic that linearly polarized light in the first direction is reflected and linearly polarized light in the second direction is transmitted. The polarization separating element 41 is, for example, a polarization beam splitter prism (PBS prism). The linearly polarized light in the first direction is S polarized light with respect to the polarization separation film 41a. The linearly polarized light in the second direction is P polarized light with respect to the polarization separation film 41a.

The excitation light L1d is S-polarized with respect to the polarization separation membrane 41a, is reflected by the polarization separation membrane 41a, and is incident on the dichroic mirror 16. The excitation light L1e is P-polarized for the polarization separation film 41a, passes through the polarization separation film 41a, and enters the dichroic mirror 16. Note that one or both of the polarization separation element 36 and the polarization separation element 41 may not be the PBS prism. One or both of the polarization separating element 36 and the polarization separating element 41 may be a photonic crystal having different reflection and transmission between TE polarized light and TM polarized light.

FIG. 24 is a diagram showing the mask and the polarization state of the excitation light according to the tenth embodiment. In FIG. 24A, the Xc direction, the Yc direction, and the Zc direction are directions corresponding to the X direction, the Y direction, and the Z direction on the sample surface Sa (see FIG. 23 ), respectively. The mask 38 has an opening 38a, an opening 38b, and an opening 38c. The mask 38 is arranged at or near the pupillary conjugate plane. The opening 38a, the opening 38b, and the opening 38c are arranged in the Xc direction. The opening 38a, the opening 38b, and the opening 38c are, for example, circular, but may have a shape other than the circular shape.

24B, the Xd direction, the Yd direction, and the Zd direction are directions corresponding to the X direction, the Y direction, and the Z direction on the sample surface Sa (see FIG. 23), respectively. The mask 40 is arranged at or near the pupillary conjugate plane. The mask 38 or the mask 40 may be arranged on or near the pupil surface. The mask 40 has an opening 40a, an opening 40b, and an opening 40c. The opening 40a, the opening 40b, and the opening 40c are arranged in the Yd direction. The opening 40a, the opening 40b, and the opening 40c are, for example, circular, but may have a shape other than the circular shape.

In FIG. 24C, the region AR2a is a region where the excitation light L1d that has passed through the opening 38a of the mask 38 is incident on the pupil surface P0 of the objective lens 21. The area AR2b is an area on the pupil plane P0 where the excitation light L1d passing through the opening 38b of the mask 38 is incident. The region AR2e is a region on the pupil surface P0 where the excitation light L1d that has passed through the opening 38c of the mask 38 is incident. The arrows in the regions AR2a, AR2b, and AR2e indicate the polarization directions of the incident excitation light L1d. The area AR2a, the area AR2b, and the area AR2e are arranged in the X direction.

The excitation light L1d incident on the area AR2a, the excitation light L1d incident on the area AR2b, and the excitation light L1d incident on the area AR2e are each linearly polarized in the Y direction. The excitation light L1d incident on the area AR2a, the excitation light L1d incident on the area AR2b, and the excitation light L1d incident on the area AR2e have the same polarization direction and interfere with each other on the sample surface Sa (see FIG. 23). Due to this interference, interference fringes whose periodic direction is the X direction are formed on the sample surface Sa. The incident surface of the excitation light L1d on the sample surface Sa is the XZ plane, and the excitation light L1d is incident on the sample S as S-polarized light.

Further, in FIG. 24C, the region AR2c is a region on the pupil surface P0 where the excitation light L1e that has passed through the opening 40b of the mask 40 is incident. The region AR2d is a region on the pupil surface P0 where the excitation light L1e that has passed through the opening 40a of the mask 40 is incident. The region AR2f is a region on the pupil surface P0 where the excitation light L1e that has passed through the opening 40c of the mask 40 is incident. The arrows in the region AR2c, the region AR2d, and the region AR2f indicate the polarization direction of the incident excitation light L1e. The area AR2c, the area AR2d, and the area AR2f are arranged in the Y direction. Further, the region AR2e and the region AR2f may overlap, and for example, in FIG. 24C, the region AR2e and the region AR2f are the same region.

The excitation light L1e incident on the area AR2c, the excitation light L1e incident on the area AR2d, and the excitation light L1e incident on the area AR2f are each linearly polarized in the X direction. The excitation light L1e incident on the area AR2c, the excitation light L1e incident on the area AR2d, and the excitation light L1e incident on the area AR2f have the same polarization direction and interfere with each other on the sample surface Sa (see FIG. 23). Due to this interference, interference fringes whose periodic direction is the Y direction are formed on the sample surface Sa. The incident surface of the excitation light L1e with respect to the sample surface Sa is the YZ plane, and the excitation light L1e is incident on the sample S as S-polarized light.

Returning to the description of FIG. 23, an interference fringe L2 is formed on the sample surface Sa, which is a combination of an interference fringe due to the interference of the excitation light L1d and an interference fringe due to the interference of the excitation light L1e. Since the polarization directions of the excitation light L1d and the excitation light L1e are substantially orthogonal to each other, interference between the excitation light L1d and the excitation light L1e is suppressed.
As described in the sixth embodiment, the illumination optical system 4 may be provided with the phase modulation element 25.

The detection device 6 detects the fluorescence L3 from the sample S via the detection optical system 5. As described in the first embodiment, the detection device 6 is an image sensor in which a plurality of detection units 6a are arranged in two directions, an Xb direction and a Yb direction. The image processing unit 7 generates an image based on the detection result of the detection device 6 by any of the processes described in the first to sixth embodiments.

In the present embodiment, reflecting the fact that the illumination pupil is divided into five poles (five regions) on the pupil plane, interference fringes in the X direction and interference fringes in the Y direction simultaneously occur on the sample surface Sa. When the image processing described in the third embodiment, the fourth embodiment, and the sixth embodiment is applied to this embodiment, in this embodiment, the 0th order component, the +first order component in the X direction, and the −1 in the X direction. 9th order component, +second order component in X direction, -second order component in X direction, +first order component in Y direction, -first order component in Y direction, +second order component in Y direction, and second order component in Y direction By separating the components, the image processing phase shift processing or the deconvolution processing can be performed.

When the image processing described in the fifth embodiment is applied to the present embodiment, in the present embodiment, the 0th order component, the +1st order component in the X direction, the +1st order component in the Z direction, and the −1st order in the +1st order Z direction in the X direction. Component, −first order component in X direction, +first order component in X direction, −first order component in X direction, −first order component in X direction, +second order component in X direction, −second order component in X direction, +first order Z direction in Y direction Direction +first order component, Y direction +first order Z direction -1st order component, Y direction -1st order Z direction +first order component, Y direction -1st order Z direction -1st order component, Y direction +2 The image processing phase shift processing or the deconvolution processing can be performed by separating the second component and the 13th component of the −secondary component in the Y direction.
In any case, the region in which each component has a value can be determined by theoretical calculation or numerical simulation using the design value of the microscope 1 as in the third and fifth embodiments. .. Alternatively, the region where each component has a value may be obtained by actually measuring the fluorescent sample. Alternatively, as described in the sixth embodiment, the phase of the interference fringes L2 is changed a plurality of times to acquire an image of the sample S, and simultaneous equations are solved from a plurality of images having different phases of the interference fringes L2. The components may be separated with. The number of times the phase of the interference fringe L2 is changed may be determined according to the number of components to be separated.

[11th Embodiment]
The eleventh embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 25 is a diagram showing a microscope according to the eleventh embodiment. In the present embodiment, the microscope 1 includes the λ / 2 wave plate 30 and the optical path rotating portion 31 described with reference to FIG. The optical path rotating unit 31 is driven by the driving unit 32 and rotates around the optical axis of the illumination optical system 4. When the optical path rotating unit 31 rotates, the optical path of the excitation light L1d and the optical path of the excitation light L1e respectively rotate around the optical axis of the illumination optical system 4. As a result, the periodic direction of the interference fringe L2 formed on the sample surface Sa rotates around the Z direction.

FIG. 26 is a diagram showing the polarization state of the excitation light according to the eleventh embodiment. In FIG. 26A, the regions AR4a on which the excitation light L1d is incident on the pupil surface P0 are aligned in the X direction. Further, the regions AR4b on the pupil plane P0 on which the excitation light L1e is incident are arranged in the Y direction.

FIG. 26B corresponds to a state in which the Dove prism (optical path rotating unit 31 in FIG. 25) and the λ/2 wave plate 30 are rotated by 22.5° from the state in FIG. 26A. In FIG. 26B, the regions AR4a on which the excitation light L1d is incident on the pupil surface P0 are arranged in a direction rotated by 45 ° from the X direction. In this state, the periodic direction of the interference fringes of the excitation light L1d on the sample surface Sa is a direction rotated by 45° from the X direction. Further, the regions AR4b on the pupil plane P0 on which the excitation light L1e is incident are arranged in a direction rotated by 45° from the Y direction. In this state, the periodic direction of the interference fringes of the excitation light L1e on the sample surface Sa is a direction rotated by 45° from the Y direction.

Returning to the description of FIG. 25, in the present embodiment, the detection device 6 detects the fluorescence L3 from the sample S before and after the periodic direction of the interference fringe L2 is changed. The image processing unit 7 generates an image based on the detection result of the detection device 6 before the change of the periodic direction of the interference fringe L2 and the detection result of the detection device 6 after the change of the periodic direction of the interference fringe L2. The optical path rotating unit 31 may be arranged in the optical path between the dichroic mirror 16 and the detection device 6, as described with reference to FIG. As described in the sixth embodiment, the illumination optical system 4 may be provided with the phase modulation element 25.

[Twelfth Embodiment]
A twelfth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. In the eleventh embodiment, the microscope 1 changes the periodic direction of the interference fringes L2 by the optical path rotating unit 31, but the fringe direction changing unit that changes the periodic direction of the interference fringes L2 is also different from the optical path rotating unit 31. Good.

FIG. 27 is a diagram showing a microscope according to the twelfth embodiment. FIG. 28 is a diagram showing a mask according to the twelfth embodiment. In the present embodiment, the microscope 1 includes a driving unit 45 and a driving unit 46. The mask 38 is rotatable around the optical axis of the illumination optical system 4. The mask 38 is driven by the drive unit 45 to rotate (see FIG. 28A). In FIG. 28A, the mask 38 is rotated counterclockwise by 45°.

Moreover, the mask 40 is rotatable around the optical axis of the illumination optical system 4. The mask 40 is driven by the drive unit 46 to rotate (see FIG. 28B). The drive unit 46 rotates the mask 40 by the same angle as the drive unit 45 rotates the mask 38. In FIG. 28B, the mask 40 is rotated counterclockwise by 45°. As a result, the periodic direction of the interference fringes L2 on the sample surface Sa rotates 45° around the Z direction.

A λ/2 wavelength plate 48 is provided in the optical path between the polarization separation element 41 and the dichroic mirror 16. The λ / 2 wave plate 48 is driven by the drive unit 49 and rotates around the optical axis of the illumination optical system 4. The λ / 2 wave plate 48 and the driving unit 49 adjust each of the excitation light L1d and the excitation light L1e so as to be incident on the sample S with S polarization. As described in the sixth embodiment, the illumination optical system 4 may be provided with the phase modulation element 25.

[13th Embodiment]
The thirteenth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 29 is a diagram showing a microscope according to the thirteenth embodiment. In this embodiment, the microscope 1 includes a relay optical system 47. The relay optical system 47 is a part of the illumination optical system 4 and a part of the detection optical system 5. The relay optical system 47 is arranged in the scanning unit 18 in the optical path between the deflection mirror 18a and the deflection mirror 18b. The deflection mirror 18b is arranged at substantially the same position as the first pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21. The relay optical system 47 is provided so that a second pupil conjugate surface that is optically conjugate with the first pupil conjugate surface is formed between the relay optical system 47 and the relay optical system 17. The deflection mirror 18a is arranged at substantially the same position as the second pupil conjugate plane described above.

The scanning unit 18 is not limited to the above-mentioned form. For example, the stage 2 may include a Y stage that moves in the Y direction with respect to the objective lens 21, and the scanning unit 18 may include a Y stage instead of the deflection mirror 18b. In this case, the scanning unit 18 may scan the sample S with the excitation light L1 in the X direction by the deflection mirror 18a, and may scan the sample S with the excitation light L1 in the Y direction by moving the Y stage. In this case, the deflection mirror 18a may be arranged at substantially the same position as the pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21.

Further, the stage 2 may include an X stage that moves in the X direction with respect to the objective lens 21, and the scanning unit 18 may include an X stage instead of the deflection mirror 18a. In this case, the scanning unit 18 may scan the sample S with the excitation light L1 in the X direction by moving the X stage, and may scan the sample S with the excitation light L1 in the Y direction by the deflection mirror 18b. In this case, the deflection mirror 18b may be arranged at substantially the same position as the pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21.

Further, the stage 2 includes an X stage that moves in the X direction with respect to the objective lens 21 and a Y stage that moves in the Y direction with respect to the objective lens 21, and the scanning unit 18 includes the X stage and the Y stage described above. May be included. In this case, the scanning unit 18 may scan the sample S with the excitation light L1 in the X direction by moving the X stage, and may scan the sample S with the excitation light L1 in the Y direction by moving the Y stage. ..

In the above-described embodiment, the scanning direction for scanning the sample S with the interference fringes may be two directions of the X direction and the Y direction, or may be three directions of the X direction, the Y direction, and the Z direction. For example, the microscope 1 executes the 2D processing of scanning the sample S with the interference fringes in the X direction and the Y direction to obtain a 2D image. For example, at least one of the objective lens 21 and the stage 2 is moved to cause the interference fringes. The sample S may be three-dimensionally scanned with the interference fringes by changing the position in the Z direction at which is generated and repeating the 2D processing. The microscope 1 may three-dimensionally scan the sample S with interference fringes to acquire a plurality of 2D images having different positions in the Z direction and generate a 3D image (for example, Z-stack). When the sample S is three-dimensionally scanned with interference fringes, the illumination optical system 4 may scan in the X direction and the Y direction, and may scan in the Z direction by moving at least one of the objective lens 21 and the stage 2. Further, the illumination optical system 4 may scan the sample S three-dimensionally with interference fringes.

[Modification]
Hereinafter, a modified example will be described. The same reference numerals are given to the same configurations as those of the above-described embodiments, and the description thereof will be omitted or simplified. 30 and 31 are diagrams showing the illumination pupil according to the modified example.

The illumination pupil has 3 poles in FIG. 2 (C), 5 poles in FIG. 24 (C), and 4 poles in FIG. 30 (A). The regions AR5a to AR5d are regions in which the excitation light is incident on the pupil surface P0, respectively. In this case, the first interference fringes of the excitation light incident on the area AR5a and the excitation light incident on the area AR5b, the second interference fringes of the excitation light incident on the area AR5b and the excitation light incident on the area AR5c, and the area Third interference fringes of excitation light incident on AR5c and excitation light incident on area AR5a, fourth interference fringes of excitation light incident on area AR5a and excitation light incident on area AR5d, and incidence on area AR5b A fifth interference fringe between the excitation light and the excitation light incident on the area AR5d and a sixth interference fringe between the excitation light incident on the area AR5c and the excitation light incident on the area AR5d are formed. On the sample surface Sa, interference fringes that are a combination of the first interference fringes, the second interference fringes, the third interference fringes, the fourth interference fringes, the fifth interference fringes, and the sixth interference fringes are formed. The interference fringes include a cycle direction of the first interference fringe, a cycle direction of the second interference fringe, a cycle direction of the third interference fringe, a cycle direction of the fourth interference fringe, and a cycle direction of the fifth interference fringe. The periodic direction of the sixth interference fringes is the periodic direction, and the super-resolution effect in the XYZ directions can be obtained in one image acquisition. The illumination pupil may have 6 or more poles. The polarization direction of the excitation light may be a polarization direction other than the direction shown in FIG. 30(A), or circular polarization.

Also, the illumination pupil is circular in FIG. 2 and the like, but may have other shapes. In FIG. 30B, a region AR6 is a region on which the excitation light is incident. The area AR6 in FIG. 30B is an area surrounded by an arc that is a part of a circle centered on the optical axis 21a of the objective lens 21 and a curved line AR6c that is symmetrical with the arc AR6a.

In the case of the illumination pupil having the shape of FIG. 30(B), the resolution in the direction in which the interference fringes are not formed is better and the sectioning is better than that in the case of the circular illumination pupil.

In FIG. 31, the illumination pupil has a shape in which the illumination pupil having the shape shown in FIG. 30(B) has five poles. Even in the case of an illumination pupil having a shape other than a circle, the number of a plurality of regions (the number of poles) on which the excitation light is incident is set to an arbitrary number of 2 or more. In addition, the shape of one of the plurality of areas on the pupil plane P0 on which the excitation light is incident may be different from the shape of the other areas. Further, of the plurality of regions on the pupil plane P0 where the excitation light is incident, the size of one region may be different from the size of the other regions. Further, the plurality of regions in which the excitation light is incident on the pupil surface P0 may be arranged asymmetrically with respect to the optical axis 21a of the objective lens 21. Further, the polarization direction of the excitation light on the pupil surface P0 may be different or the same for each of a plurality of regions where the excitation light is incident.

The shape, dimensions, and arrangement of each pole of the illumination pupil can be realized, for example, by designing the shape, dimensions, and arrangement of the opening of the mask 15 shown in FIG. Further, the mask 15 may be a mechanical diaphragm having a variable light blocking area, a spatial light modulator (SLM), a Digital Mirror Device, or the like.

FIG. 32 is a diagram showing a microscope according to a modified example. In FIG. 32, the illumination optical system 4 includes a collimator lens 50, a λ/2 wavelength plate 51, a lens 52, a diffraction grating 53, a lens 54, and a mask 15 in this order from the optical fiber 11 to the dichroic mirror 16. The collimator lens 50 converts the excitation light L1 from the optical fiber 11 into substantially parallel light. The λ/2 wave plate 51 adjusts the polarization state of the excitation light L1 when entering the sample S. The lens 52 concentrates the excitation light L1 on the diffraction grating 53.

The diffraction grating 53 splits the excitation light L1 into a plurality of light beams by diffraction. The diffraction grating 53 is a light beam splitting unit that splits the excitation light that excites the fluorescent material into a plurality of light beams. The diffraction grating 53 is arranged at the focal point or a position near the focal point of the lens 52. That is, the diffraction grating 53 is arranged on the surface conjugate with the sample surface Sa or in the vicinity thereof. The vicinity of the focus is within the range of the depth of focus of the light condensed by the lens 52. For example, when the wavelength of the light condensed by the lens 52 is 1 μm and NA is 0.03, the depth of focus is about 1 mm, so the diffraction grating 53 may be arranged within 1 mm near the focus of the lens 52. The plurality of light beams include 0th-order diffracted light, +1st-order diffracted light, and -1st-order diffracted light. The lens 54 converts the 0th-order diffracted light, the +1st-order diffracted light, and the -1st-order diffracted light into substantially parallel lights. The mask 15 is provided so that at least a part of the 0th order diffracted light, at least a part of the +1st order diffracted light, and at least a part of the −1st order diffracted light pass through. In such a form, the light amount of the excitation light L1 that passes through the mask 15 can be increased. Further, a configuration may be adopted in which the mask 15 is not provided. Further, the diffraction grating 53 may be configured to be movable in the direction perpendicular to the optical axis, and the phase of the interference fringe L2 formed by the excitation light L1 may be modulated. In this case, the super-resolution image may be generated using the image processing described in the sixth embodiment.

33 and 34 are diagrams showing a polarization adjusting unit according to a modification, respectively. Although the optical path of the illumination optical system 4 is bent by the reflection member such as the dichroic mirror 16 shown in FIG. 1, in FIGS. 33 and 34, the illumination optical system 4 is developed so that the optical axis 4a becomes a straight line. Indicated. In FIGS. 33 and 34, the Z direction is a direction parallel to the optical axis 4a, and the X direction and the Y direction are directions perpendicular to the optical axis 4a, respectively.

In FIG. 33, the illumination optical system 4 includes a λ/4 wavelength plate 61, a mask 15, and a λ/4 wavelength plate 62. The excitation light L1 emitted from the optical fiber 11 is linearly polarized in the X direction and is incident on the λ/4 wavelength plate 61. A polarizer (for example, a polarizing plate) whose transmission axis is in the X direction may be provided in the optical path between the optical fiber 11 and the λ/4 wavelength plate 61.

The phase-advancing axis of the λ / 4 wave plate 61 is set in the direction in which the X direction is rotated counterclockwise by 45 ° when viewed from the + Z side. The excitation light L1 that has passed through the λ/4 wavelength plate 61 becomes circularly polarized light and enters the mask 15. The excitation light L1 that has passed through the openings 15a and 15b of the mask 15 is circularly polarized light and is incident on the λ/4 wavelength plate 62. The fast axis of the λ/4 wave plate 62 is set in a direction obtained by rotating the X direction clockwise by 45° when viewed from the +Z side. The excitation light L1 that has passed through the λ/4 wavelength plate 62 becomes linearly polarized light in the X direction and is applied to the sample.

As described in the first embodiment, the mask 15 is rotatably provided around the optical axis 4a. When the mask 15 rotates, the periodic direction of the interference fringes changes. For example, in the state of FIG. 33, the opening 15a and the opening 15b of the mask 15 are aligned in the Y direction, and the periodic direction of the interference fringes is the Y direction. When the mask 15 is rotated by 90 ° from the state of FIG. 33, the periodic direction of the interference fringes is rotated by 90 ° and becomes the X direction.

The λ/4 wave plate 62 is rotatable around the optical axis 4a. The λ/4 wave plate 62 is provided so as to rotate by the same angle as the mask 15. For example, the λ/4 wave plate 62 is integrated with the mask 15 and rotates integrally with the mask 15. For example, when the mask 15 is rotated by 90 °, the λ / 4 wave plate 62 is rotated by 90 ° and its phase advance axis becomes parallel to the phase advance axis of the λ / 4 wave plate 61. In this case, the excitation light L1 that has passed through the λ/4 wavelength plate 62 becomes linearly polarized light in the Y direction. The plane of incidence of the excitation light L1 on the sample surface is parallel to the periodic direction of the interference fringes, and the excitation light L1 entering the sample surface is linearly polarized light perpendicular to the periodic direction of the interference fringes. The sample surface is irradiated with S-polarized light.

As described above, the λ/4 wavelength plate 62 is included in the polarization adjusting unit that adjusts the polarization state of the excitation light when entering the sample. Such a polarization adjusting unit can reduce the loss of the light amount of the excitation light L1 as compared with the aspect described in FIG.

In FIG. 34, the illumination optical system 4 includes a polarizer 65, a mask 15, and a λ/2 wave plate 66. The excitation light L1 emitted from the optical fiber 11 is linearly polarized light in the X direction and is incident on the polarizer 65. The transmission axis of the polarizer 65 is set in the X direction. The excitation light L1 that has passed through the polarizer 65 is linearly polarized in the X direction and is incident on the mask 15. The excitation light L1 that has passed through the openings 15a and 15b of the mask 15 is linearly polarized light in the X direction and is incident on the λ / 2 wave plate 66. The fast axis of the λ/2 wave plate 66 is set in a direction obtained by rotating the X direction clockwise by 45° when viewed from the +Z side. The excitation light L1 that has passed through the λ / 2 wave plate 66 becomes linearly polarized light in the Y direction and irradiates the sample.

The mask 15 is rotatably provided around the optical axis 4a as described in the first embodiment. When the mask 15 rotates, the periodic direction of the interference fringes changes. For example, in the state of FIG. 34, the opening 15a and the opening 15b of the mask 15 are aligned in the X direction, and the periodic direction of the interference fringes is the X direction. When the mask 15 is rotated by 90 ° from the state of FIG. 33, the periodic direction of the interference fringes is rotated by 90 ° and becomes the Y direction.

The λ/2 wave plate 66 is rotatable around the optical axis 4a. The λ/2 wave plate 66 is provided so as to rotate by an angle half the rotation angle of the mask 15. For example, when the mask 15 rotates 90°, the λ/2 wave plate 66 rotates 45°. In this case, the excitation light L1 that has passed through the λ/2 wave plate 66 becomes linearly polarized light in the X direction. The plane of incidence of the excitation light L1 on the sample surface is parallel to the periodic direction of the interference fringes, and the excitation light L1 entering the sample surface is linearly polarized light perpendicular to the periodic direction of the interference fringes. The sample surface is irradiated with S-polarized light. As described above, the λ / 2 wave plate 66 is included in the polarization adjusting unit that adjusts the polarization state of the excitation light when it enters the sample. Such a polarization adjusting unit can reduce the loss of the light amount of the excitation light L1 as compared with the aspect described in FIG.

The polarization adjusting unit described in the above-described embodiment and modification is configured so that the illumination light becomes linearly polarized light immediately after passing through the polarization adjusting unit, but it does not have to be perfect linearly polarized light. Further, an additional polarizing element such as a λ/2 wavelength plate or a λ/4 wavelength plate may be added to the polarization adjusting unit to correct the change in the polarization state generated in the optical system on the way.

Note that the microscope 1 according to the embodiment may include an image sensor in which the detection device 6 includes an image sensor, and may include an image rotating portion for rotating the image of the sample S around the optical axis of the detection optical system 5. When the stripe direction is rotated, by rotating the image of the sample S, the stripe cycle and the position of the detection unit can be matched.

In the above-described embodiment, the image processing unit 7 includes, for example, a computer system. The image processing unit 7 reads out the image processing program stored in the storage unit and executes various processes according to the image processing program. The image processing program causes the computer to generate an image based on the detection result of the detection device 6. The detection result of the detection device 6 is that the light from the light source is divided into a plurality of light beams, and the interference fringes generated by the interference of at least three or more light beams of the plurality of light beams are scanned in a plurality of directions of the sample. The light from the sample is obtained by detecting the light from the sample through a detection optical system on which the light from the sample enters.

The technical scope of the present invention is not limited to the embodiments described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. In addition, the requirements described in the above-described embodiments can be combined as appropriate. In addition, to the extent permitted by law, the disclosure of all documents cited in the above-mentioned embodiments and the like shall be incorporated as part of the description in the main text.

DESCRIPTION OF SYMBOLS 1... Microscope, 3... Light source, 4... Illumination optical system, 5... Detection optical system, 6... Detection device, 6a... Multiple detection parts, 7... Image processing Part, L2... Interference fringe

Claims (28)

A plurality of samples have interference fringes formed by interference of at least three or more luminous fluxes of the plurality of luminous fluxes divided by the luminous flux dividing portion and having a luminous flux dividing portion for dividing the light from the light source into a plurality of luminous fluxes. An illumination optical system that scans in the direction of,
A detection optical system on which light from the sample is incident,
A detection device including a plurality of detection units that detect light from the sample via the detection optical system, and
A microscope including an image processing unit that generates an image using the detection results of two or more detection units of the detection device. The interference fringes have three or more bright portions in the periodic direction of the interference fringes.
The microscope according to claim 1. The illumination optical system has an objective lens,
The luminous flux dividing portion includes an opening member having a plurality of openings.
The aperture member is arranged in the pupil plane of the objective lens, in the vicinity of the pupil plane, in the pupil conjugate plane, or in the vicinity of the pupil conjugate plane.
The microscope according to claim 1 or 2. The light beam splitting unit includes a diffraction grating,
The diffraction grating is arranged at or near the position conjugate with the sample.
The microscope according to claim 1 or 2. The detection device includes a line sensor in which the plurality of detection units are arranged in one direction.
The microscope according to any one of claims 1 to 4. The detection device includes an image sensor in which the plurality of detection units are arranged in two directions.
The microscope according to any one of claims 1 to 4. A fringe direction changing portion for changing the direction of the interference fringes with respect to the sample is provided.
The microscope according to any one of claims 1 to 6. An image rotating unit for rotating an image of the sample with respect to the plurality of detecting units around the optical axis of the detection optical system is provided.
The microscope according to any one of claims 1 to 7. The image rotation unit is arranged in an optical path that does not overlap with the illumination optical system in the detection optical system,
The microscope according to claim 8. A fringe direction changing portion that changes the direction of the interference fringes with respect to the sample,
An image rotation unit that rotates the image of the sample with respect to the plurality of detection units around the optical axis of the detection optical system,
The striped direction changing portion and the image rotating portion are composed of the same member.
The microscope according to any one of claims 1 to 6. A polarization adjusting unit for adjusting the polarization state of the light when entering the sample,
The microscope according to any one of claims 1 to 10. The positions of the plurality of detection units are set based on the magnification of the detection optical system and the period of the interference fringes.
The microscope according to any one of claims 1 to 11. The image processing unit generates the image using the detection results of the detection unit selected from the plurality of detection units based on the magnification of the detection optical system and the period of the interference fringes.
The microscope according to any one of claims 1 to 12. The image processing unit corrects data obtained from at least one detection unit of the plurality of detection units based on the position of the detection unit.
The microscope according to any one of claims 1 to 13. The microscope according to any one of claims 1 to 11, wherein the image processing unit converts at least a part of the detection results of the plurality of detection units into data on a frequency space. The image processing unit converts at least a part of the detection results of the plurality of detection units into data on the frequency space by Fourier transform,
The microscope according to claim 15. The image processing unit performs filtering on the data in the frequency space to generate the image.
The microscope according to claim 15 or 16. The image processing unit separates the data on the frequency space into a plurality of regions of the frequency space to separate a plurality of components, and generates the image.
The microscope according to any one of claims 15 to 17. The image processing unit, based on the light intensity distribution of the interference fringes, separates the data on the frequency space into a plurality of regions of the frequency space,
The microscope according to claim 18. The plurality of areas are set so as not to overlap each other,
The microscope according to claim 18 or 19. The image processing unit converts at least a part of the detection results of the plurality of detection units into data on the frequency space by performing Fourier transform on scanning coordinates in the optical axis direction of the illumination optical system,
The microscope according to any one of claims 16 to 20. The illumination optical system includes a phase modulation element that changes the phase of the interference fringes.
The image processing unit generates the image by separating the data on the frequency space of the detection result detected by the detection unit when the interference fringes are in each of the plurality of phase states into a plurality of components. To do,
The microscope according to any one of claims 15 to 17. The image processing unit converts the phase of at least a part of the data obtained by separating the plurality of components to generate the image,
The microscope according to any one of claims 18 to 22. The image processing unit determines the amount of phase conversion based on the light intensity distribution of the interference fringes and the position of the detection unit,
The microscope according to claim 23. The microscope according to claim 23 or 24, wherein the image processing unit corrects the phase-converted data based on the position of the detection unit to generate the image. The image processing unit executes deconvolution on the data obtained from the detection device to generate the image.
The microscope according to any one of claims 15 to 22. The image processing unit executes the deconvolution on the data obtained from at least one detection unit of the plurality of detection units based on the position of the detection unit and the light intensity distribution of the interference fringes.
The microscope according to claim 26. By dividing the light from the light source into a plurality of luminous fluxes and scanning the interference fringes formed by the interference of at least three or more luminous fluxes of the plurality of luminous fluxes in a plurality of directions of the sample.
Through a detection optical system on which light from the sample enters, by a detection device including a plurality of detection units, detecting the light from the sample,
An observation method including generating an image using the detection results of two or more detection units of the detection device.

Images