JP2019120501A - Picture-taking system, image processing device, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To take a large number of photographs in which polarization conditions are different and a subject is changed in various directions, without performing work that burdens a user. A transillumination light source 101, a light source side optical element 102, a rotary table 104 that supports a subject 10 and changes the direction of the subject 10 with respect to an optical axis, a camera side optical element 105, and a camera 106. To place. The image capturing control apparatus 200 has a plurality of states in which the conditions such as the polarization state of light passing through the light source side optical element 102, the polarization state of light passing through the camera side optical element 105, and the orientation of the subject on the turntable 104 are combined. Control is performed so that the camera 106 sequentially performs shooting processing in each of a plurality of states. [Selection diagram] Figure 1

Description

  The present invention relates to an imaging system, an image processing apparatus, and an image processing method for taking a picture of a sample (sample) as a subject in order to observe the sample using polarized light. In the present specification, observing a sample using polarized light may be referred to as polarized light observation.

  Light is a kind of electromagnetic wave and has the property of "wave", and is classified into ultraviolet light, visible light, infrared light, etc. depending on the wavelength. Polarization is light vibrating in a specific direction, and can be artificially generated, for example, by transmitting light emitted from a predetermined light source to a polarizer.

  In polarized light observation, it may be possible to visualize structures that are invisible or hard to see with natural light or the naked eye. Under polarized light, an object that is glossy, black, transparent, or structural color passes birefringence and interference, and is colored or contoured to be emphasized, making it possible to observe effectively and easily.

  When stress is applied to a transparent material such as glass or polycarbonate and observed under polarized light, a strain causes a phase difference in the birefringent polarized light and interference fringes appear. This is called photoelasticity and is often used as an experimental method for analyzing the stress distribution of a material. It is a phenomenon known about 200 years ago.

As an existing product for polarization observation, a dedicated camera incorporating a polarizer is put to practical use. This is advantageous when shooting experiments and phenomena at high speed under polarized light, etc., but the cost is high and the need for dedicated hardware is limited.
(Refer to Photonic Lattice, Inc. http://www.photonic-lattice.com/ and Photoron, Inc. http://www.photron.co.jp/)

  Polarized light microscopes are often used to observe crystals and minerals. A polarized light microscope is capable of high resolution observation such as 100 μm or less, but it has a narrow field of view and can not simultaneously observe an entire image of a size equal to or more than a few centimeters of a mechanical part.

Although there are products for polarization observation inexpensively, which are composed only of a light source and a polarizing plate, the purpose is to simply view a sample placed on a table.
(Refer to http://www.mecan.co.jp/Visualization/strain_viewer/index.html)

  Patent Document 1 below discloses an apparatus for observing the internal strain of a mineral.

  Moreover, in the following patent document 2, the apparatus which can measure the defect and distortion inside a to-be-measured object is disclosed.

  Further, Patent Document 3 below discloses a technique for capturing an object from various fields of view using a camera without being limited to polarization observation.

  QuickTime VR (Virtual Reality) (registered trademark) is known as a technology for connecting a plurality of images in a three-dimensional manner and expressing a near-real view as a movie. There are two types of QuickTime VR, a QuickTime VR panorama that allows you to observe the surrounding field of view and space centered on the camera, and a QuickTime VR object movie that allows you to observe an object three-dimensionally. In addition, Web3D and HTML5 are known as a technology that can rotate and observe an object three-dimensionally on a Web browser.

  On the other hand, Patent Document 4 discloses a technique of using three-dimensional image data as an input and generating a highly convenient display image data file from a rendered image.

  In recent years, microcomputer boards such as Arduino and mbed and related books are commercially available, and methods for describing programs using these and controlling motors and electronic devices have become known.

  It is known to construct a user interface by describing a program using development tools such as OpenCV and Qt, and to perform various image processing on image data.

JP, 2009-103634, A JP, 2009-250911, A JP, 2015-012375, A Patent No. 5582584

  In polarized light observation, the appearance changes in various ways depending on the light source, the angle of the polarizing plate, the rotation of the subject, and the like. Since the interference color by polarization changes in correlation with the angle of the polarizing plate and the birefringence value and thickness of the subject, if the subject is observed from only a single direction under certain conditions, the entire color distribution can be obtained. Can not grasp In order to obtain useful image data that looks good in polarized light observation, it is a fact that it requires labor and know-how at the time of imaging.

  SUMMARY OF THE INVENTION In consideration of the above problems, the present invention provides an imaging system capable of easily and quickly obtaining a large number of image data useful for polarization observation, and image processing for processing image data obtained by this imaging system. An object of the present invention is to provide an apparatus and an image processing method.

In order to achieve the above object, the imaging system of the present invention is
A first light source,
A first optical element for changing light transmitted from the first light source to a first polarization state and transmitting the light;
A second optical element for changing light transmitted from the first optical element into a second polarization state and transmitting the light;
When a subject is not disposed on the optical axis between the first optical element and the second optical element, the light incident from the second optical element is received and converted into an electric signal. When the subject is disposed on the optical axis between the first optical element and the second optical element, the light from the second optical element including the light transmitted through the subject An imaging device configured to perform an imaging process of receiving incident light and converting it into an electrical signal and outputting the electrical signal as image data;
A rotating device configured to support the subject and to rotate the subject to change the orientation of the subject with respect to the optical axis;
A plurality of states combining conditions of the first polarization state, the second polarization state, and the direction of the subject by controlling the first optical element, the second optical element, and the rotation device. A photographing control device configured to control the photographing device to sequentially perform the photographing processing in each state of the plurality of states;
Have.

  Further, in order to achieve the above object, the image processing apparatus of the present invention is configured to perform image processing using a plurality of image data corresponding to each of the plurality of states obtained by the imaging system. It is done.

In addition, in order to achieve the above object, the image processing method of the present invention is
Reading a plurality of image data corresponding to each state of the plurality of states obtained by the imaging system from a predetermined storage device;
Performing image processing using the plurality of image data read from the predetermined storage device;
Have.

  According to the present invention, it is possible for a user to automatically and continuously shoot a large number of photographs in which the polarization conditions are different and the subject is changed in various directions by one operation without spending time and effort. Further, according to the present invention, the user can view an image of a subject photographed under various conditions, and can easily grasp various ways of viewing the subject under polarization. Further, according to the present invention, by processing image data obtained by photographing under various conditions, it is possible to calculate numerical values relating to the polarization characteristic of the subject, or to generate image data that makes it easy to grasp the polarization characteristic. It is possible to Further, according to the present invention, it is possible to compare an image photographed under various conditions with a visualized image which is digitized as a result of polarization calculation, so that a polarized image which has been inspected visually until now, a resin It is possible to understand the degree of forming strain such as

It is a block diagram showing an example of composition of an imaging system in an embodiment of the invention. It is a flowchart which shows an example of the process at the time of the calibration imaging | photography which concerns on the imaging | photography system in embodiment of this invention. It is a figure which shows a specific example of the calibration imaging | photography which concerns on the imaging | photography system in embodiment of this invention. It is a flowchart which shows an example of the process at the time of the sample imaging which concerns on the imaging system in embodiment of this invention. It is a figure showing a concrete example of sample photography concerning a photography system in an embodiment of the invention. It is a figure which shows typically an example of the use of the image data obtained by the imaging | photography system in embodiment of this invention. It is a figure which shows an example of the image produced | generated using the several image data obtained by the imaging | photography system in embodiment of this invention, and the birefringence phase difference obtained by the polarization calculation using several image data It is a figure which shows the birefringence phase difference visualization image to represent. It is a figure which shows an example of the image produced | generated using the several image data obtained by the imaging | photography system in embodiment of this invention, and is a main axis showing the main-axis direction obtained by polarization calculation using several image data. It is a figure which shows a direction visualization image. It is a figure which shows an example of the image produced | generated using the several image data obtained by the imaging | photography system in embodiment of this invention, and is a figure which shows the image image | photographed by making the spoon made from polystyrene into a to-be-photographed object. It is a figure for demonstrating the labeling process performed with respect to several image data obtained by the imaging | photography system in embodiment of this invention. It is a figure which shows an example of the transmitted illumination image obtained by the imaging | photography system in embodiment of this invention. It is a figure for demonstrating the case where gray scale conversion of the image data obtained by the imaging | photography system in embodiment of this invention is carried out, and also contrast is adjusted. It is a figure for demonstrating the case where the binarization process is performed with respect to the image data obtained by the imaging | photography system in embodiment of this invention, and the image obtained by the binarization process is used as mask information. It is a figure for demonstrating the image synthetic | combination processing using several image data obtained by the imaging | photography system in embodiment of this invention. It is a figure which shows an example of the two-dimensional arrangement | sequence format relevant to this invention. It is a figure which shows typically the state in which one display image data file was produced | generated by the several image data obtained by the imaging | photography system in embodiment of this invention being output to a moving image format. It is a figure which shows an example at the time of arrange | positioning the image image | photographed by each imaging | photography step shown in FIG. 5 in a two-dimensional array format. It is an explanatory view showing a schematic structure of a polarization characteristic measuring device by this related art. It is explanatory drawing which shows the incident Stokes vector which the calculating means of FIG. 18 calculates | requires. It is explanatory drawing which shows the after-transmission Stokes vector which the calculating means of FIG. 18 calculates | requires. It is explanatory drawing which shows an example of the sample installed in the measuring apparatus of FIG. In this related art, it is explanatory drawing which shows the Mueller matrix of the sample which the calculating means calculated. In this related art, it is explanatory drawing which shows the polarization state extracted from the element of a Mueller matrix. It is explanatory drawing which shows an example of the polarization characteristic measured using the measuring apparatus of FIG. It is an explanatory view showing a measurement result using this related art, and a measurement result using a general measurement (analysis) method. In this related art, it is explanatory drawing which shows an example (Table 1) of the azimuth angle of each optical element for calculating | requiring phase difference (delta) 2 of phase plate R2. In this related art, it is explanatory drawing which shows an example (Table 2) of the azimuth | direction angle of the optical element for Mueller matrix analysis. It is explanatory drawing which shows the comparison (Table 3) of the measurement result of this related art and the existing measuring device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an example of the configuration of a photographing system according to an embodiment of the present invention. The imaging system illustrated in FIG. 1 includes an imaging system that performs imaging using polarized light, an imaging control system that controls the operation of the imaging system, and image processing that processes image data obtained by imaging in the imaging system. It is divided roughly into systems. Hereinafter, each configuration of the imaging system, the imaging control system, and the image processing system will be described in order.

  An imaging system that performs imaging using polarized light includes a transmissive illumination light source 101, a light source side optical element 102, a rotating table 103, a reflective illumination light source 104, a camera side optical element 105, and a camera 106.

  In the imaging system, the light source side optical element 102 and the object 10 on the rotation table 103 (polarization observation) are on the path (the path L1 of light shown in FIG. 1) for the light emitted from the transmissive illumination light source 101 to reach the camera 106. Object to be performed, and the camera-side optical element 105 are arranged in this order. As described later, the subject 10 may not be placed on the rotary table 103.

  The light source side optical element 102, the subject 10 on the rotary table 103, and the camera side optical element 105 may be disposed on a path through which light emitted from the transmissive illumination light source 101 reaches the camera 106. The direction of light may be horizontal (direction perpendicular to the direction of gravity), may be the direction of gravity or the direction opposite to gravity, and may be any angular direction with respect to the direction of gravity. It may be

  Hereinafter, a path along which light emitted from the transmissive illumination light source 101 reaches the camera 106 may be referred to as an optical axis. The optical axis represents a path through which a part of a light beam propagated with a certain spread passes, and is not limited to, for example, a path through which a geometrically central portion of a cross section of the light flux passes.

  The transmissive illumination light source 101 is a light source that emits light toward the direction of the camera 106 (the optical axis direction). The transmissive illumination light source 101 is, for example, an array of LED (Light Emission Diode) panels that respectively emit visible light of each color of RGB (red, green, blue), and controls emission intensity of each color of RGB Thus, it is possible to emit light of any full color, not only RGB single color light. Moreover, since large-scale surface emitting illumination can be configured by arranging the LED panels, it is possible to realize, as the transmissive illumination light source 101, illumination that performs surface emitting with a large size of 384 mm × 384 mm, for example. By increasing the size of the transmissive illumination light source 101, it is possible to use a relatively large object 10 of about several tens of centimeters as a target for polarization observation. As described later, turning on / off and the light emission pattern of the transmissive illumination light source 101 can be controlled by the transmissive illumination light source control unit 201 of the photographing control system.

  The transmissive illumination light source 101 is a light source used when observing the subject 10 in polarization, and has a role as transmissive illumination disposed at a position to be a back light when the subject 10 is viewed from the camera 106 .

  The transmissive illumination light source 101 is not limited to a light source that emits visible light, and may be, for example, a light source that emits infrared light or ultraviolet light according to the application. In particular, when the subject 10 is a translucent or colored material that does not easily transmit visible light, it is effective to use a light source that emits infrared light as the transmissive illumination light source 101.

  The light source side optical element 102 is an optical element having a polarizer 102a and a quarter wavelength plate (QWP) 102b, and is disposed in the order of the polarizer 102a and the QWP 102b in the optical axis direction. When the polarizer 102a and the QWP 102b are, for example, film-like members, the polarizers 102a and the QWP 102b are arranged such that the direction of the normal to the film-like surface substantially matches the optical axis direction. The polarizer 102a is, for example, a linear polarization element, and can rotate the transmission axis orientation to a desired angle. The QWP 102 b is a quarter-wave plate that converts linearly polarized light into circularly polarized light, and can rotate the principal axis direction to a desired angle. As described later, the transmission axis orientation of the polarizer 102a and the main axis orientation of the QWP 102b can be controlled by the light source side optical element control unit 202 of the photographing control system.

  The rotary table 103 has a table (pedestal) on which the subject 10 is placed. For example, when the optical axis is in the substantially horizontal direction, the table surface on which the subject 10 is placed is arranged to be substantially horizontal. The subject 10 placed on the rotary table 103 is disposed on the optical axis connecting the light source 101 for transmissive illumination to the camera 106. As a result, the light emitted from the light source 101 for transmission illumination passes through the light source side optical element 102 and then passes through the subject 10 and further passes through the camera side optical element 105 to reach the camera 106. Since the positioning is performed, it is possible to observe, from the camera 106, the state of polarized light transmitted through the subject 10 sandwiched between the two optical elements 102 and 105. Further, the rotary table 103 is mounted with a motor, and is configured such that the table can be rotated with a direction perpendicular to the optical axis as a rotation axis. As a result, by rotating the rotary table 103 to a desired angle, it is possible to set the direction of the subject 10 placed on the rotary table to a desired direction.

  Note that it is also possible to operate the imaging system in a state where the subject 10 is not placed on the rotation table 103 as in calibration imaging which will be described later. As described later, the motor mounted on the rotary table 103 can be controlled by the rotary table control unit 203 of the imaging control system.

  Furthermore, a plurality of rotation tables 103 may be provided so that polarization observation of a plurality of subjects 10 can be performed simultaneously. For example, by arranging two rotary tables 103 sideways (that is, in a direction perpendicular to the optical axis) and placing different subjects 10 on the respective rotary tables 103, polarization observation of different subjects 10 can be performed. It can be done at the same time. In this case, the plurality of rotary tables 103 may be configured to be able to rotate synchronously, or may be configured to be capable of independently rotating.

  In this specification, a rotary table 103 configured to support the subject 10 with a table (base) as a device (rotation device) for arranging the subject 10 on the optical axis and changing the orientation of the subject 10 with respect to the optical axis. Is described in the case of using. However, the rotation device is not limited to the rotation table 103 as long as the rotation device has a configuration capable of arranging the subject 10 on the optical axis and changing the direction of the subject 10 with respect to the optical axis. The rotation device arranges the subject 10 on the optical axis and changes the direction of the subject 10 with respect to the optical axis by, for example, holding the subject 10 or inserting the needle-like member to fix the subject 10. It may be configured to In addition, as in the case of providing the plurality of rotation tables 103 described above, the rotation device arranges the plurality of subjects 10 on the optical axis and synchronizes the directions of the respective subjects 10 with the optical axis or is independent of each other. It may be configured to be able to change.

  The reflective illumination light source 104 is a light source for applying illumination (forward light) to the subject 10 from the camera 106 side, and has a role of reflective illumination. The reflective illumination light source 104 is provided outside the imaging range by the camera 106 so as not to disturb the imaging by the camera 106, and may be, for example, a ring shape (the optical axis and weight which can uniformly illuminate the object 10). (A shape in which the light sources are arranged along the circumferential direction of the optical axis). The light emitted from the reflective illumination light source 104 is reflected by the subject 10 and reaches the camera 106 (a light path L2 shown in FIG. 1). The reflective illumination light source 104 is desirably a white light source for clarifying the color and shape of the subject 10, but may be a visible light source of any color, an infrared light source, an ultraviolet light source, or the like. May be As described later, turning on / off of the reflective illumination light source 104 can be controlled by the reflective illumination light source control unit 204 of the photographing control system.

  The camera side optical element 105 is an optical element having the QWP 105 a and the polarizer 105 b. The QWP 105 a and the polarizer 105 b are film-like members, and the QWP 105 a and the polarizer 105 b are disposed in this order in the optical axis direction such that the direction of the normal to the film surface substantially matches the optical axis direction. The QWP 105 a is a quarter-wave plate that converts linearly polarized light into circularly polarized light, and can rotate the principal axis direction to a desired angle. The polarizer 105 b is, for example, a linear polarization element, and can rotate the transmission axis orientation to a desired angle. As described later, the main axis direction of the QWP 105a and the transmission axis direction of the polarizer 105b can be controlled by the camera-side optical element control unit 205 of the photographing control system.

  The camera 106 is an imaging device for imaging a predetermined imaging range, and can use a commercially available single-lens reflex digital camera. The camera 106 is installed at a place where the light transmitted through the camera side optical element 105 is incident, receives the light transmitted through the camera side optical element 105 and converts it into an electric signal, and is obtained at a predetermined shutter timing. It has a photographing function and the like for outputting an electrical signal as image data. Image data obtained by photographing with the camera 106 is stored in the storage device 400. For example, in the case where the transmissive illumination light source 101 emits infrared light or ultraviolet light, it is desirable that the camera 106 correspond to the light used by the transmissive illumination light source 101. As described later, photographing by the camera 106 can be controlled by the camera control unit 206 of the photographing control system.

  Further, the camera 106 is set at a camera platform capable of fine adjustment such as rotation or movement in the XYZ axial directions. Furthermore, the camera platform rests on a platform called a slider that can freely move back and forth, and is configured to be able to move closer to or away from the subject 10 when shooting a small / large subject 10 .

  When photographing with the camera 106, it is desirable to block external light to ensure sufficient darkness for polarized observation of the object 10. For example, each device of the photographing system (the light source 101 for transmissive illumination) It is desirable that the light source side optical element 102, the rotary table 103, the reflective illumination light source 104, the camera side optical element 105, and the camera 106) be installed in a dark room.

  On the other hand, a photographing control system for controlling the operation of the photographing system is constituted by a photographing control device 200 and a PC (Personal Computer: personal computer) 300.

  The imaging control apparatus 200 is connected to each device of the imaging system and the PC 300. The imaging control apparatus 200 can be equipped with a microcomputer board, and can centrally control the operation of each device of the imaging system. Specifically, in order to control the operation of each device of the imaging system, the imaging control device 200 controls the light source control unit 201 for transmission illumination, the light source side optical element control unit 202, the rotation table control unit 203, and the light source for reflection illumination A control unit 204, a camera-side optical element control unit 205, and a camera control unit 206 are included.

  The transmissive illumination light source control unit 201 represents a function of controlling the operation of the transmissive illumination light source 101. The transmissive illumination light source control unit 201 performs control of turning on / off of the transmissive illumination light source 101, control of emission color of the transmissive illumination light source 101 (control of light intensity of each LED panel emitting visible light of each color of RGB), and the like. It is possible. As an example, when the transmissive illumination light source control unit 201 causes the transmissive illumination light source 101 to emit RGB light, when the transmissive illumination light source 101 emits only R light (only red light having a known wavelength emits), the transmissive illumination light source It is possible to control three different states in the case of turning off the light 101.

  The light source side optical element control unit 202 has a function of controlling the operation of the light source side optical element 102. The light source side optical element 102 is composed of the polarizer 102a and the QWP 102b as described above, and the light source side optical element control unit 202 individually changes the transmission axis orientation of the polarizer 102a and the main axis orientation of the QWP 102b. Is possible. The light source side optical element control unit 202 can also control the presence or absence of the polarizer 102 a and the QWP 102 b (with and without the polarizer 102 a and the QWP 102 b).

  The light source side optical element control unit 202 only needs to change the setting relating to the light source side optical element 102 so as to make the light passing through the light source side optical element 102 into a desired polarization state (a plurality of polarization states). There are no particular limitations on the configuration for the light source and the number of patterns of desired polarization states. As an example, the polarizer 102a is fixed at a predetermined transmission axis orientation (for example, 45 °), and the QWP 102b is set to a first main axis orientation (for example, 0 °), a second main axis orientation (for example, 90 °), and further, QWP 102b It is possible to make 3 patterns of the state which does not arrange. In this case, for example, a state in which the QWP 102b of the first main axis orientation and the QWP 102b of the second main axis orientation are prepared and the QWP 102b of the first main axis orientation is disposed on the optical axis, the second main axis on the optical axis A mechanism may be provided to mechanically switch the state in which the QWP 102b in the orientation is arranged and the state in which the QWP 102b is not arranged on the optical axis. At this time, the light source side optical element control unit 202 controls the switching drive unit that mechanically switches the QWP 102 b to change the arrangement of the QWP 102 b, whereby the light passing through the light source side optical element 102 has three different polarization states. It is possible to realize.

  Here, with respect to the QWP 102b, a plurality of QWPs 102b having different main axis orientations are prepared and mechanically switched, but may be a mechanism that rotates the QWP 102b with the optical axis direction as a rotation axis.

  The rotary table control unit 203 represents a function of controlling the operation of the rotary table 103. The rotary table control unit 203 controls a motor that rotates the rotary table 103 to rotate the rotary table 103 so as to have a desired angle, whereby the subject 10 placed on the rotary table 103 is directed to the camera 106. It is possible to change to the desired orientation.

  The rotation table control unit 203 only needs to change the angle of the rotation table 103 so that the subject 10 has a desired orientation (a plurality of orientations) with respect to the camera 106. The numerical value and range of the angle of the rotation table 103 and the number of rotation steps are There is no particular limitation. As an example, by setting the range of the angle of the rotary table 103 to, for example, 120 ° and setting the number of rotation steps to 5 (five angles of 0 °, 30 °, 60 °, 90 °, 120 °), the subject 10 is different. It becomes possible to shoot from five directions (five pattern shooting directions).

  The reflective illumination light source control unit 204 represents a function of controlling the operation of the reflective illumination light source 104. The reflective illumination light source control unit 204 can perform control of turning on / off the reflective illumination light source 104, control of the light amount, and the like. As an example, the light source control unit 204 for reflective illumination can control two different states of two patterns when turning off the light source for reflective illumination 104 when emitting the light source for reflective illumination 104.

  The camera-side optical element control unit 205 represents a function of controlling the operation of the camera-side optical element 105. The camera side optical element 105 is composed of the QWP 105a and the polarizer 105b as described above, and the camera side optical element control unit 205 individually changes the main axis direction of the QWP 105a and the transmission axis direction of the polarizer 105b. Is possible. The camera-side optical element control unit 205 can also control the presence or absence of the QWP 105 a and the polarizer 105 b (with and without the QWP 105 a and the polarizer 105 b).

  The camera-side optical element control unit 205 only needs to change the setting related to the camera-side optical element 105 so that the light passing through the camera-side optical element 105 has a desired polarization state (a plurality of polarization states). There are no particular limitations on the configuration for obtaining and the number of patterns of desired polarization states.

  For the QWP 105a, for example, a mechanism may be provided to mechanically rotate the QWP 105a with the optical axis direction as the rotation axis while maintaining the direction of the normal to the film-like surface substantially coincident with the optical axis direction. At this time, the camera-side optical element control unit 205 can set the main axis direction of the QWP 105 a to a desired angle by controlling the rotation drive unit that mechanically rotates the QWP 105 a with the optical axis direction as the rotation axis. . In addition, for the polarizer 105b, for example, using a switching mechanism similar to that of the above-described polarizer 102a, a plurality of polarizers 105b (and a state in which the polarizers 105b are not arranged) are set. It is sufficient to provide a mechanism for switching the As an example, QWP 105a is made into four patterns which become four angles of 45 °, 67.5 °, 90 °, 112.5 ° in principal axis direction, and two angles of transmission axis azimuth 45 °, 135 ° in polarizer 105b And if it is set as three patterns of the state which does not arrange | position the polarizer 105b, it is possible to implement | achieve 12 different polarization states about the light which passes the camera side optical element 105 from these combination.

  Here, the QWP 105a has a mechanism for rotating the QWP 105a with the optical axis direction as the rotation axis, but it may be a mechanism for mechanically switching by preparing a plurality of QWPs 105a having different main axis orientations. In addition, with regard to the polarizer 105b, although a plurality of polarizers 105b having different transmission axis orientations are prepared and mechanically switched, the polarizer 105b may be rotated with the optical axis direction as a rotation axis.

  A camera control unit 206 represents a function of controlling the operation of the camera 106. The camera control unit 206 can cause the camera 106 to take a picture and output image data by instructing the camera 106 to take a picture, in particular, to release the shutter.

  In addition, the imaging control apparatus 200 can independently control the operation related to each device of the imaging system and control the operation related to each device related to the imaging system in an integrated manner through each control unit described above. is there. Specifically, the photographing control device 200 sets each state of the transmissive illumination light source 101, the light source side optical element 102, the rotary table 103, the reflective illumination light source 104, and the camera side optical element 105 to a predetermined state. When it becomes a predetermined state, it is possible to cause the camera 106 to take a picture. Hereinafter, this series of operations will be referred to as a photographing step.

  Furthermore, when a certain imaging step is completed, the imaging control apparatus 200 can control to change the state of a part of each device to perform another imaging step. Specifically, the photographing control device 200 determines the state of one or more devices of the transmissive illumination light source 101, the light source side optical element 102, the rotary table 103, the reflective illumination light source 104, and the camera side optical element 105. It is possible to change and to cause the camera 106 to take an image when the state is changed.

  The imaging control apparatus 200 controls the imaging step to be repeatedly performed as described above while changing the state of each device, thereby executing a plurality of imaging steps, and as a result, a plurality of imaging steps corresponding to the plurality of imaging steps. Image data can be stored in the storage device 400.

  In addition, it is possible to set a plurality of patterns as described above as the respective states of the light source for transmission illumination 101, the light source side optical element 102, the rotation table 103, the light source for reflection illumination 104, and the camera side optical element 105. The entire imaging system may be different by the number of combinations of these plural patterns. For example, as described above as an example, the light source 101 for transmissive illumination has three patterns, the light source side optical element 102 has three patterns, the rotary table 103 has five patterns, and the reflective light source 104 has two patterns Assuming that the camera-side optical element 105 can be in a 12-pattern state, the entire imaging system can be in different states in the number of combinations of these plural patterns (3 × 3 × 5 × 2 × 12 = 1080). . The imaging control apparatus 200 executes the imaging step for all combinations of the above or some desired combinations selected from the above combinations (only some combinations of the states of the respective devices). It is possible to control.

  Further, the PC 300 in the imaging control system is connected to an input device (not shown) such as a mouse or a keyboard and the display device 500, and a GUI (Graphical User) for inputting processing settings and instructions in the imaging control device 200. Interface: Functions as a graphical user interface. The PC 300 can perform setting and instructing of a photographing pattern (a control pattern of each device by the photographing control device 200), photographing start instruction, photographing stop instruction and the like to the photographing control device 200 based on input by the user. is there.

  The image processing system includes the PC 300, the storage device 400, and the display device 500. Although FIG. 1 illustrates that the same PC 300 is used in the imaging control system and the image processing system, different PCs 300 may be used.

  The PC 300 in the image processing system is connected to a storage device 400 storing image data, an input device (not shown) such as a mouse and a keyboard, and a display device 500, and reads out image data stored in the storage device 400. Can be displayed on the display device 500. The PC 300 in the image processing system has an image processing function. The PC 300 can read out the image data stored in the storage device 400, perform image processing, and display the image data after the image processing on the display device 500. The PC 300 can also perform polarization calculation based on the information included in the image data stored in the storage device 400. The image processing will be described in detail later.

  Note that a general-purpose PC can be used for the imaging control system and image processing system PC 300, and software (CPU (Central Processing Unit) designed to realize a desired operation (Central Processing Unit)) It is possible to execute a program described by a programming language. The storage device 400 is a data storage capable of storing image data obtained by photographing with the camera 106 and reading out image data by the PC 300 of the image processing system. For example, the storage device 400 may be a flash memory or a hard disk. It is possible to use auxiliary storage.

  Next, an example of the operation of the imaging system according to the embodiment of the present invention will be described. The operation of the imaging system according to the embodiment of the present invention is roughly classified into an imaging operation for imaging using polarized light and an image processing operation for processing image data obtained by imaging in the imaging system.

  First, an imaging operation for imaging using polarized light will be described. The imaging operation of performing imaging using polarized light includes calibration imaging in which imaging is performed with the subject 10 not mounted on the rotary table 103 and sample imaging in which imaging is performed with the subject 10 mounted on the rotational table 103. Including.

  FIG. 2 is a flowchart showing an example of processing at the time of calibration imaging according to the imaging system in the embodiment of the present invention. Each step shown in FIG. 2 represents a process performed by the imaging control apparatus 200 of FIG.

  The process relating to calibration imaging illustrated in FIG. 2 is started, for example, when the user inputs an imaging start instruction using the PC 300 and the PC 300 issues an imaging start instruction to the imaging control apparatus 200. The imaging control apparatus 200 reads a calibration imaging pattern parameter definition in which the operation of each device of the imaging system in calibration imaging is described (step S11). The calibration photographing pattern parameter definition may be incorporated in the program in advance, or may be set by the user using the PC 300.

  Calibration imaging is performed with the subject 10 not placed on the rotary table 103. Therefore, unlike the sample imaging to be described later, the imaging control apparatus 200 does not have to perform rotational drive control of the rotary table 103.

  The imaging control apparatus 200 starts loop processing of the imaging step based on the calibration imaging pattern parameter definition (step S12). The loop process of the imaging step is repeated until all calibration imaging based on the calibration imaging pattern parameter definition is completed. A loop process of the photographing step is a process of operating each of the light source for transmission illumination 101, the light source side optical element 102, the light source for reflection illumination 104, and the camera side optical element 105 according to the calibration photographing pattern parameter to make a specific state Step S13) A process of performing photographing with the camera 106 when each of the light source for transmission illumination 101, the light source side optical element 102, the light source for reflection illumination 104, and the camera side optical element 105 is in a specific state And).

  Note that it is also possible to interrupt calibration shooting in the middle, and the shooting control apparatus 200 ends the loop processing of the shooting step, for example, when receiving a shooting stop instruction from the user from the PC 300 (step S15).

  Since the loop process of the shooting step includes the process of controlling the operation of each device to change the state of each device and the process of shooting with the camera 106, the loop process of the shooting step is performed according to the calibration shooting pattern parameter definition. Every time the state is changed, shooting by the camera 106 is performed. As a result, a plurality of photographs (image data) with different combinations of the states of the respective devices can be obtained.

  Hereinafter, a specific example of calibration imaging will be described with reference to FIG. FIG. 3 is a diagram showing a specific example of calibration imaging according to the imaging system in the embodiment of the present invention.

  FIG. 3 shows the state of each device at each imaging step of calibration imaging. The imaging control apparatus 200 controls the operation of each device of the imaging system as follows, for example, in order to realize the imaging steps 1 to 10 shown in FIG. 3. Before the shooting step 1, the transmissive illumination light source 101 emits only R, the transmission axis azimuth angle of the polarizer 102a is set to 45 °, the QWP 102b is removed from the optical axis, and the reflective illumination light source 104 is extinguished. The principal axis orientation of the QWP 105a is 90 °, and the transmission axis orientation of the polarizer 105b is 45 °. Note that this state may be set before calibration imaging, or the imaging control apparatus 200 may control the operation of each device to create this state. The imaging control apparatus 200 causes the camera 106 to perform imaging in this state (imaging step 1). After the end of the photographing step 1, the photographing control device 200 changes the polarizer 105b to a polarizer whose transmission axis orientation is 135 °, and causes the camera 106 to perform photographing (photographing step 2).

  After completion of the imaging step 2, the imaging control device 200 changes the QWP 102b to a QWP of 90 ° in the main axis direction, changes the main axis of the QWP 105a to 45 °, and a polarizer in which the transmission axis direction of the polarizer 105b is 45 °. , And causes the camera 106 to shoot (shooting step 3). After the shooting step 3 ends, the shooting control device 200 changes the main axis direction of the QWP 105a to 67.5 °, and causes the camera 106 to shoot (shooting step 4). After the end of the photographing step 4, the photographing control device 200 changes the main axis direction of the QWP 105a to 90 °, and causes the camera 106 to perform photographing (photographing step 5). After the end of the photographing step 5, the photographing control device 200 changes the main axis orientation of the QWP 105a to 112.5 °, and causes the camera 106 to perform photographing (photographing step 6).

  After the completion of the photographing step 6, the photographing control device 200 changes the QWP 102b to a QWP having a main axis orientation of 0 °, changes the main axis orientation of the QWP 105a to 45 °, and causes the camera 106 to perform photographing (photographing step 7). After the end of the photographing step 7, the photographing control device 200 changes the main axis direction of the QWP 105a to 67.5 °, and causes the camera 106 to perform photographing (photographing step 8). After the end of the photographing step 8, the photographing control device 200 changes the main axis orientation of the QWP 105a to 90 °, and causes the camera 106 to perform photographing (photographing step 9). After the end of the photographing step 9, the photographing control device 200 changes the main axis direction of the QWP 105a to 112.5 ° and causes the camera 106 to perform photographing (photographing step 10).

  The above imaging steps 1 to 10 are performed continuously (automatically for the user) according to the calibration imaging pattern parameter definition. Further, in the photographing steps 1 to 10, the states of the entire photographing system (combination of the states of the respective devices) are different, and ten photographs (ten image data) different in the conditions of the whole photographing system are obtained. . At this time, it is desirable to associate the state of each device with the image data obtained by shooting with the camera 106. The format of the calibration shooting pattern parameter definition is not particularly limited, and it is possible to specify the state of each device by, for example, a numerical value or a function.

  FIG. 4 is a flowchart showing an example of processing at the time of sample shooting according to the shooting system in the embodiment of the present invention. Each step shown in FIG. 4 represents a process executed by the imaging control apparatus 200 of FIG.

  The process relating to sample imaging shown in FIG. 4 is started, for example, when the user inputs an imaging start instruction using the PC 300 and the PC 300 issues an imaging start instruction to the imaging control apparatus 200. The imaging control apparatus 200 reads a sample imaging pattern parameter definition in which the operation of each device of the imaging system in sample imaging is described (step S21). The sample shooting pattern parameter definition may be incorporated in the program in advance, or may be set by the user using the PC 300.

  The sample shooting is performed with the subject 10 placed on the rotary table 103. Therefore, unlike the calibration imaging described above, the imaging control apparatus 200 needs to perform rotational drive control of the rotary table 103. For example, when the user uses the PC 300 to input the parameters of the rotation table, the parameters of the rotation table are supplied from the PC 300 to the photographing control device 200 (step S22).

  The imaging control apparatus 200 starts the loop processing of the rotation table and the imaging step based on the sample imaging pattern parameter definition and the parameters of the rotation table (step S23). The loop process of the rotation table and the imaging step is repeated until all the sample imaging based on the sample imaging pattern parameter definition and the parameters of the rotation table is finished.

  The loop process of the rotary table is a process of setting the rotation angle of the rotary table 103 (that is, the direction of the subject 10 with respect to the camera) to a specific angle according to the parameters of the rotary table (step S24), and a loop process of the photographing step (step S25) And. Further, the loop process of the photographing step is a process of operating each of the light source for transmission illumination 101, the light source side optical element 102, the light source for reflection illumination 104, and the camera side optical element 105 in a specific state according to the sample photographing pattern parameters. (Step S26) A process of performing photographing with the camera 106 when each of the transmissive illumination light source 101, the light source side optical element 102, the reflective illumination light source 104, and the camera side optical element 105 is in a specific state And S27).

  Note that it is also possible to interrupt sample shooting on the way, and the shooting control apparatus 200 ends the loop process of the shooting step, for example, when receiving a shooting stop instruction from the user from the PC 300 (step S28).

  Loop processing of the rotation table includes processing of changing the angle of the rotation table and loop processing of the imaging step, and further, loop processing of the imaging step is processing of controlling the operation of each device to change the state of each device Since the process of performing imaging by the camera 106 is included, imaging by the camera 106 is performed each time the state of each device is changed according to the sample imaging pattern parameter definition and the parameters of the rotation table. As a result, a plurality of photographs (image data) with different combinations of the states of the respective devices can be obtained.

  Hereinafter, a specific example related to sample imaging will be described with reference to FIG. FIG. 5 is a diagram showing a specific example of sample imaging according to the imaging system in the embodiment of the present invention.

  FIG. 5 shows the state of each device at each imaging step of sample imaging. The imaging control apparatus 200 controls the operation of each device of the imaging system as follows, for example, in order to realize the imaging steps 1 to 10 shown in FIG. 5. Before the shooting step 1, the transmission illumination light source 101 emits only R, the transmission axis orientation of the polarizer 102a is set at 45 °, the main axis orientation of the QWP 102b is set at 90 °, and the rotation angle of the rotary table Is set to 0 °, the reflective illumination light source 104 is turned off, the main axis direction of the QWP 105a is 45 °, and the transmission axis direction of the polarizer 105b is 45 °. Note that this state may be set before sample photographing, or the photographing control device 200 may control the operation of each device to create this state.

  The imaging control apparatus 200 causes the camera 106 to perform imaging in this state (imaging step 1). After the end of the photographing step 1, the photographing control device 200 changes the main axis direction of the QWP 105a to 67.5 °, and causes the camera 106 to perform photographing (photographing step 2). After the end of the photographing step 3, the photographing control device 200 changes the main axis direction of the QWP 105a to 90 ° and causes the camera 106 to perform photographing (photographing step 3). After the end of the photographing step 3, the photographing control device 200 changes the main axis direction of the QWP 105a to 112.5 ° and causes the camera 106 to perform photographing (photographing step 4).

  After the completion of the photographing step 4, the photographing control device 200 changes the QWP 102b to a QWP having a main axis orientation of 0 °, changes the main axis orientation of the QWP 105a to 45 °, and causes the camera 106 to perform photographing (photographing step 5). After the end of the photographing step 5, the photographing control device 200 changes the main axis direction of the QWP 105a to 67.5 °, and causes the camera 106 to perform photographing (photographing step 6). After the end of the photographing step 6, the photographing control device 200 changes the main axis direction of the QWP 105a to 90 °, and causes the camera 106 to perform photographing (photographing step 7). After the end of the photographing step 7, the photographing control device 200 changes the main axis direction of the QWP 105a to 112.5 ° and causes the camera 106 to perform photographing (photographing step 8).

  After the completion of the imaging step 8, the imaging control device 200 causes the transmissive illumination light source 101 to emit RGB light, removes the QWP 102b from the optical axis, changes the principal axis orientation of the QWP 105a to 45 °, and transmits 135 for the polarizer 105b. The polarizer is changed to a degree of °, and the camera 106 performs photographing (photographing step 9). After the end of the photographing step 9, the photographing control device 200 changes the main axis direction of the QWP 105a to 67.5 °, and causes the camera 106 to perform photographing (photographing step 10). After the end of the photographing step 10, the photographing control device 200 changes the spindle orientation of the QWP 105a to 90 °, and causes the camera 106 to perform photographing (photographing step 11). After the end of the photographing step 5, the photographing control device 200 changes the main axis direction of the QWP 105a to 112.5 °, and causes the camera 106 to perform photographing (photographing step 12).

  After the end of the photographing step 12, the photographing control device 200 removes the polarizer 105b from the optical axis and causes the camera 106 to perform photographing (photographing step 13). After the end of the photographing step 13, the photographing control device 200 turns on the light source for reflection illumination 104 to make the camera 106 perform photographing (photographing step 14). After the end of the photographing step 14, the photographing control device 200 turns off the transmissive illumination light source and causes the camera 106 to perform photographing (photographing step 15). In the above shooting steps 1 to 15, the camera 106 and the subject 10 are fixed, and the subject 10 in the image captured in each of the shooting steps 1 to 15 is captured at the same position (the same pixel).

  After the end of the photographing step 15, the photographing control device 200 sets the rotation angle of the rotary table to 30 °, and causes the camera 106 to perform photographing under the same conditions as the photographing steps 1 to 15 (photographing step 16 to 30). ). After the end of the photographing step 30, the photographing control device 200 sets the rotation angle of the rotary table to 60 ° and makes the camera 106 perform photographing with the same conditions as the photographing steps 1 to 15 (photographing steps 31 to 45) ). After the end of the photographing step 45, the photographing control device 200 sets the rotation angle of the rotary table to 90 ° and makes the camera 106 perform photographing with the other conditions being the same as the photographing steps 1 to 15 (photographing step 46 to 60 ). After the completion of the imaging step 60, the imaging control apparatus 200 sets the rotation angle of the rotary table to 120 °, and causes the camera 106 to perform imaging under the same conditions as the imaging steps 1 to 15 (imaging steps 61 to 75). ).

  The above-described shooting steps 1 to 75 are performed as a continuous process (automatically for the user) in accordance with the sample shooting pattern parameter definition and the parameters of the rotation table. Further, in the photographing steps 1 to 75, the states of the entire photographing system (combination of the states of the respective devices) are different from one another, and 75 photographs (75 image data) different in the conditions of the photographing system as a whole are obtained. Be At this time, it is desirable to associate the state of each device with the image data obtained by shooting with the camera 106. The format of the sample shooting pattern parameter definition is not particularly limited as in the calibration shooting pattern parameter definition. For example, the state of each device can be designated by a numerical value or a function.

  Next, image processing related to image data obtained by shooting in a shooting system will be described. FIG. 6 is a view schematically showing an example of application of image data obtained by the imaging system according to the embodiment of the present invention.

  As described above, the photographing system in the embodiment of the present invention can obtain a large number of photographs (image data) in which the state of the entire photographing system is different. In particular, by changing the conditions of the light source side optical element 102 and the camera side optical element 105 arranged on the optical axis, various polarization states can be created, so the subject 10 looks different in various polarization states. You can get a lot of photos. Specifically, the imaging system according to the embodiment of the present invention can obtain an image for calibration (for example, can be acquired at imaging steps 1 to 10 in FIG. 3) and an image for polarization calculation (for example, imaging steps 1 to 4 in FIG. 5). 8, which can be acquired in polarized light visualization image (for example, can be acquired in photographing steps 9 to 12 in FIG. 5), reflected illumination image (for example, can be acquired in photographing step 13 in FIG. 5), reflected illumination + transmitted illumination image (for example, It is possible to obtain a transmitted illumination image (for example, obtainable at the shooting step 15 of FIG. 5), which can be obtained at the shooting step 14 of FIG.

  The PC 300 in the image processing system, for example, uses a luminance value of each pixel included in each of a plurality of image data to calculate a numerical value representing the polarization characteristic of an object corresponding to each pixel, and a polarization calculation unit And an image data generation unit that generates new image data using the numerical value representing the polarization characteristic of the subject corresponding to each pixel calculated by The polarization calculation unit performs polarization calculation using data included in each of the calibration image and the polarization calculation image (for example, the luminance value of each pixel (pixel) that constitutes the image), thereby the polarization of the subject 10 It is possible to obtain characteristics (birefringence retardation and principal axis orientation).

  In this polarization calculation, as one example, the following calculation is performed. For example, image data obtained by changing the condition of the polarizer 105b of the camera side optical element 105, focusing on a certain pixel of the image (for example, two image data obtained in the photographing steps 1 and 2 in FIG. 3) The phase difference of the polarizer 105 b is obtained from the luminance value of Next, from the phase difference of the polarizer 105 b and the luminance value of the image data (for example, the image data obtained in the photographing steps 3 to 10 in FIG. 3) photographed in the state where the subject 10 is not disposed While determining the polarization state on the light incident side of the optical element 105, the luminance of the image data of the same condition (for example, the image data obtained in the photographing steps 1 to 8 in FIG. 5) photographed in the state where the object 10 is arranged. Similarly for the value, the polarization state on the light incident side of the camera side optical element 105 is determined. Then, from the relationship between the polarization state according to the state in which the subject 10 is not arranged and the polarization state according to the state in which the subject 10 is arranged, the birefringence phase difference and the main axis direction which are the polarization characteristics of the object 10 are obtained. The polarization calculation for obtaining the polarization characteristic of the subject 10 is not limited to the method described above.

  Birefringence retardation and principal axis orientation obtained by polarization calculation are numerical data in pixel units. For example, for birefringence retardation, an image that is easy to visually recognize birefringence retardation obtained by polarization calculation by setting a color, brightness, and the like using a LUT (Look Up Table). It is possible to generate a (birefringence phase contrast visualization image). FIG. 7 is a view showing an example of an image generated using a plurality of image data obtained by the imaging system according to the embodiment of the present invention, and a compound obtained by polarization calculation using a plurality of image data It is a figure which shows the birefringent phase contrast visualization image showing a refractive phase difference. The image shown in FIG. 7 is taken in a state in which a polystyrene spoon is placed on each of the two rotary tables 103. The image shown in FIG. 7 is a color image, which is colored using the LUT to correspond to the value of the birefringence retardation at each pixel. As described above, it is possible to generate an image that is easy to visually grasp the birefringence phase difference of the subject 10 obtained by the polarization calculation by performing pseudo coloring on the numerical value of the birefringence phase difference. is there.

  Similarly, with regard to the main axis direction, it is possible to generate an image using a LUT, but a vector representing the main axis direction obtained by polarization calculation by averaging the main axis direction of an image region of an arbitrary number of pixels An image (principal axis visualized image) is drawn in each image area (a display element such as a thin line indicates the size of the main axis direction by the length of the line, and the direction of the main axis direction by the line direction) It is also possible. FIG. 8 is a view showing an example of an image generated using a plurality of image data obtained by the imaging system according to the embodiment of the present invention, and a main axis obtained by polarization calculation using a plurality of image data It is a figure which shows the main-axis direction visualization image which represents an azimuth | direction. The image shown in FIG. 8 is taken in a state where a polystyrene spoon is placed on each of the two rotary tables 103. In the image shown in FIG. 8, the image area is divided into square areas including a plurality of pixels. The average value of the magnitude and direction of the main axis direction in the pixels included in each area of the square is determined, and the average value of the main axis direction is represented by a thin line in each area, so that the main axis direction of the subject 10 obtained by the polarization calculation. It is possible to generate an image that is easy to grasp visually.

  In addition, the image for calibration, the image for polarization calculation, the image for polarized light, the image for reflected light, the image for reflected light and the light for transmitted light, the image for transmitted light, the image for birefringence phase difference visualization, and the image for visualization of main axis direction May be generated, or the process of superimposing images may be performed by calculating addition and subtraction of luminance values. The result obtained by the process of superimposing images can be used, for example, in applications such as binarization, labeling, region extraction, counting, clarification, and determination. In addition, when performing processing related to image data, for example, a specific region of interest (Region of Interest) is set, polarization calculation is performed only for pixels included in the region of interest, extraction and processing of an image, etc. May be

  Hereinafter, specific examples related to various image processing will be described.

  A digital image is a collection of pixels. For an 8-bit grayscale image, luminance values from 0 to 255 are stored in each pixel and arranged as a two-dimensional array. In the case of a color image, each color of RGB is represented by a value from 0 to 255, and either RGB value is taken as the luminance value of the pixel, for example, luminance value = INT (0.299 × (value of R) +0 Discard color information and convert it to a grayscale image by calculating the luminance value of each pixel based on an expression such as 587 × (value of G) + 0.114 × (value of B) +0.5) It is possible.

  FIG. 9 is a view showing an example of an image generated using a plurality of image data obtained by the photographing system according to the embodiment of the present invention, showing an image photographed with a polystyrene spoon as a subject It is. The image shown in FIG. 9 is taken in a state in which a polystyrene spoon is placed on each of the two rotary tables 103. The image shown in FIG. 9 is a color image, and a state in which an iridescent stripe pattern is visible on the spoon due to polarization is taken.

  In image analysis practice, the image areas of interest are usually limited. In this case, the region of interest is limited to the resin material portion of each of the two right and left spoons, and there is a case where it is desired to exclude the pedestal or background portion shown in the lower part in the image analysis.

  A typical image processing method used in such a case is binarization. Setting a threshold (for example, 100) for the luminance value of an 8-bit grayscale pixel results in darker pixels (pixels with luminance values from 0 to 99) and brighter pixels (0 to 99). It becomes possible to select pixels having luminance values, and it is possible to arbitrarily extract an image area of interest / an image area of interest. For example, if coloring is performed by setting the LUT only to the extracted image area, it is possible to obtain a clear image that is easy to see, which is useful for visualization applications such as when visual inspection is performed.

  FIG. 10 is a diagram for describing a labeling process performed on a plurality of image data obtained by the imaging system according to the embodiment of the present invention. In FIG. 10, an image (an image on the left side of FIG. 10) in which four regions of interest are present in an image area of 16 pixels × 15 pixels (image on the left of FIG. 10) and an image in a state where labeling processing is performed (image And the image on the right side of FIG.

  As shown in FIG. 10, when labeling processing is performed to assign different numbers in the program to each of the pixels in the region of interest, the area of each region of interest is determined by counting the pixels corresponding to each region of interest. Various digitization and automatic processing such as counting the number of regions and determining the presence or absence of a region of interest can be performed. Further, by assigning different colors to the respective numbers assigned by the labeling process, it is possible to generate an image which makes it easy to visually grasp the distribution of the area. The image after the labeling process (the image on the right in FIG. 10) is a color image, for example, black to pixel number 0, blue to pixel number 1, green to pixel number 2, orange to pixel number 3, orange Red is colored in each of the four pixels.

  Also, the binarization process of the image depends on the quality of the image data. FIG. 11 is a view showing an example of a transmitted illumination image obtained by the imaging system according to the embodiment of the present invention. As shown in FIG. 11, the brightness of the light transmitted through the resin of the spoon which is the subject 10 is close to the brightness of the light of the background part (the part different from the subject 10), so it corresponds to the spoon in the image. Each pixel has a bright value close to that of the pixel corresponding to the background. Therefore, it is not easy to extract only the spoon image area by simple binarization processing.

  On the other hand, FIG. 12 is a view for explaining the case where the image data obtained by the imaging system according to the embodiment of the present invention is subjected to grayscale conversion and the contrast is further adjusted. The image on the left side of FIG. 12 is taken in a state in which the first optical element 102 and the second optical element 105 are arranged to be in the state of cross nicol, and the background portion is dark. In an image captured in such a state, the pixel corresponding to the background portion has a dark luminance value as compared to the luminance value of the pixel corresponding to the spoon in the image. Therefore, by converting the image to grayscale (the center image in FIG. 12) and adjusting the contrast appropriately (the right image in FIG. 12), only the luminance value of the pixel corresponding to the spoon that is the subject 10 is high. It becomes easy to perform the binarization processing, and it becomes possible to generate an image in which the portion corresponding to the spoon is emphasized.

  The imaging system according to the embodiment of the present invention can perform visualization and image analysis by arbitrarily using a plurality of image data captured under different conditions. FIG. 13 is a view for explaining a case where image data obtained by the imaging system in the embodiment of the present invention is binarized and an image obtained by the binarization process is used as mask information. It is.

  The image on the left side of FIG. 13 is an image generated using a plurality of image data obtained by the imaging system according to the embodiment of the present invention, and a birefringence obtained by polarization calculation using a plurality of image data It is an image visualized by setting a LUT to the phase difference. The image on the left side of FIG. 13 is a color image, and the background portion is also colored, but the area other than the spoon which is the subject 10 is meaningless and has no interest. Therefore, it is possible to eliminate the background portion as shown on the right side of FIG. 13 by superimposing the image obtained by the above-described binarization processing as mask information. The image subjected to the mask processing has a valid numerical value only for each pixel in the region of interest corresponding to the subject 10. Therefore, processing such as obtaining the maximum value, the minimum value, the average value, and the standard deviation of the numerical values of each pixel corresponding to the subject 10 becomes easy, and the characteristics of the subject 10 can be easily evaluated.

  Moreover, the visualization process which used arbitrarily several image data image | photographed on different conditions is also possible. FIG. 14 is a diagram for describing an image combining process using a plurality of image data obtained by the imaging system according to the embodiment of the present invention. In addition, the image of FIG. 14 (a)-(e) is divided into nine area | regions, and each area | region respond | corresponds to each pixel or corresponds to the area | region containing several pixels. In the following, it is assumed that each area corresponds to each pixel.

  By means of polarization calculations, it is possible to calculate the value of the birefringence retardation for each pixel. The values of birefringence retardation are obtained by performing polarization calculation based on a plurality of calibration images and polarization calculation images. FIG. 14A shows numerical values of the birefringence retardation of each pixel obtained by the polarization calculation. By performing LUT processing on the numerical value of birefringence retardation of each pixel, it is possible to assign a color corresponding to the numerical value of birefringence retardation to each pixel, as shown in FIG. . FIG. 14 (b) is a color image, which is colored corresponding to the value of the birefringence retardation.

  On the other hand, the image in FIG. 14C is a reflection illumination image, and each pixel is a color image having color information (for example, a combination of numerical values of three colors of RGB). It is possible to obtain an image (grayed image) as shown in FIG. 14D by subjecting the reflected illumination image to a clay scale conversion.

  Here, for example, threshold processing is performed on the image shown in FIG. 14 (b), and the color information of the pixel having a numerical value of birefringence phase difference obtained by polarization calculation equal to or less than a predetermined threshold (eg, 100 or less) The colored image of FIG. 14 (c) and the gray of FIG. 14 (d) are maintained so that pixels larger than the predetermined threshold value are replaced (grayed) with the pixels of the grayed image of FIG. 14 (d). Composite with the Thereby, it is possible to generate an image as shown in FIG. The image in FIG. 14E is colored for pixels whose birefringence retardation value obtained by polarization calculation is less than a predetermined threshold value, and for pixels larger than the threshold value, the reflected illumination image is grayscale converted. It is an image having the luminance value obtained.

  Image processing in the PC 300 of the image processing system is not limited to the above-described processing, and image processing can be performed by any method. In addition to the above-described processing, for example, a function of generating a MIP (Max Intensity Projection) image obtained by projecting a pixel having the highest luminance value among pixels of a plurality of grayscale-converted images may be provided. In addition, it may have a clipping function of clipping only an area where a subject is shown, or a tiling function of arranging and combining an arbitrary image in the same image.

  Furthermore, the PC 300 of the image processing system has, for example, an N (N is an integer of 2 or more) dimensional array format for an image data group selected from a plurality of image data and new image data generated from a plurality of image data. A display image data file including an image data group in which a file name is attached to each image data included in the image data group and the image name is attached to each image data in accordance with a naming rule determined to represent the arrangement position in May be included in the display image data file generation unit.

  For example, a plurality of desired image data are selected from the image data obtained by the imaging system according to the embodiment of the present invention, the image data newly generated by the various image processing described above, etc. Image data is given a file name according to a certain naming rule based on a predetermined arrangement rule (for example, a two-dimensional arrangement format). Then, it is possible to generate a single display image data file including a plurality of image data by reading a series of image data with a file name and converting it into a moving image format according to a predetermined arrangement rule. . In this case, a plurality of image data are read. By compressing the data using H.264 and outputting it as a QuickTime (registered trademark) VR format (object movie), it is possible to generate one display image data file including a plurality of image data.

  Here, an example of an arrangement rule of a plurality of image data stored in the display image data file will be described. A plurality of selected image data are virtually arranged in the two-dimensional array format, for example, by being associated with the arrangement coordinates in the two-dimensional array format constituted by two array axes of Row and Column. . For example, as illustrated in FIG. 15, a two-dimensional array format (N × M two-dimensional array format) is set in which M images are arranged in Row and N images are arranged in Column as shown in FIG. Each image data is virtually arranged at each arrangement coordinate (Rm, Cn) (m, n is an integer, 1 ≦ m ≦ M, 1 ≦ n ≦ N). At this time, when the image included in the two-dimensional array format is reproduced and displayed in a suitable reproduction environment (for example, a QuickTime player) after generation, the display image data file is generated in the Row direction according to a predetermined user operation. Alternatively, it is arranged on the premise that the display is switched between the images adjacent in the column direction.

  In the reproduction environment of the display image data file, for example, when the user moves the mouse up and down or presses the cursor key corresponding to the upper and lower sides, the display is switched between the images adjacent in the row direction, and the mouse The display is switched between the images adjacent in the column direction by moving in the left-right direction or pressing the cursor keys corresponding to the left and right. In FIG. 15, the M-th row or the N-th column of Row in the two-dimensional array format may be set to be adjacent to the first row or the first column of Row.

  A plurality of image data virtually arranged in a two-dimensional array format as shown in FIG. 15 is output to a moving image format etc. according to a predetermined array rule, as schematically shown in FIG. As a result, they are stored in one display image data file. It is possible to use an arbitrary file format when storing a plurality of image data in the display image data file, but it becomes possible to apply a highly efficient codec by using the moving image format, and the file size becomes extremely high. It becomes possible to compress small.

  The two-dimensional array format illustrated in FIG. 15 is an example, and the present invention is not limited to this array. Also, the current QuickTime VR format has a two-dimensional array format, but the present invention is not limited to this, and even if a display image is arranged in an N (N is an integer of 3 or more) dimensional array format Good. In the two-dimensional array format, there are two array axes, and in the two-dimensional array format, there are four display images adjacent to each array axis. The QuickTime player switches the display of adjacent images along each array axis by associating the four directions with the up, down, left, and right movements of the mouse or the cursor keys in four directions. On the other hand, when the N-dimensional array format can be realized, there are N array axes in the N-dimensional array format, and 2 × N display images adjacent along each array axis in the N-dimensional array format. In this case, it is possible to realize desired image switching by preparing a player for switching display of adjacent images in the 2 × N direction in the reproduction environment.

  The information assigned to the two array axes of Row and Column can be appropriately determined by the user according to the selected plural image data and the application of the display image data file finally generated. . As an example, it is possible to define the alignment axis of Row as the orientation of the object with respect to the optical axis, and define the alignment axis of Column as a condition combining the light source element and the light source.

  For example, when using the image data taken at each photographing step shown in FIG. 5, it is assumed that five images are arranged in Row and fifteen images are arranged in Column, as shown in FIG. Is possible. When the display image data file is generated based on the virtual arrangement shown in FIG. 17, in the reproduction environment of the display image data file, the subject is switched under the same polarization condition by switching the adjacent images in the Row direction. By switching images adjacent to each other in the column direction, it is possible to realize a display that looks as if the polarization condition changes while the subject is facing the same direction.

  The image processing described above may be performed independently, or may be arbitrarily combined to generate a visualized image. In the case of combining the image processing, for example, an image may be generated in which a display element representing a colored retardation representing birefringence retardation obtained by polarization calculation and a principal axis orientation obtained by polarization calculation is drawn. .

  Furthermore, in particular, in the imaging system according to the embodiment of the present invention, the light source 101 for transmission illumination is not only an image for polarized light observation but, for example, in a state where the polarizer 105b of the camera side optical element 105 is removed from the optical axis. A transmission illumination image captured by emitting white light (RGB emission), and a reflection illumination image captured by emitting light (RGB emission) by the light source for reflected illumination 104 can be obtained. ing. The transmitted illumination image and the reflected illumination image are images that are excellent in color development, and are photographed in a state very close to a state visually recognized by humans. An image with high visibility can be generated by combining an image showing polarization characteristics obtained by polarization calculation, an image observed under polarization, etc. on the basis of such transmission illumination image and reflection illumination image. It is.

  As described above, in the present invention, the method of polarization calculation for obtaining the polarization characteristic of the subject 10 is not particularly limited. However, a patent application in which one of the inventors of the present invention is an inventor It is possible to use the methods of polarization measurement and polarization calculation described in the claims, specification and drawings according to the application No. 2017-98856). Hereinafter, an outline of technical ideas (hereinafter, referred to as present related technologies) useful for the present invention included in this patent application will be described. At the time of filing of the present application, this related art has not yet become a known art.

  Hereinafter, an outline of the present related technology will be described. In the following, vectors bolded in drawings and formulas are described using “[]”.

FIG. 18 is an explanatory view showing a schematic configuration of a polarization characteristic measuring apparatus according to the present related art. The illustrated measuring apparatus 901 includes a light source 911 for emitting light of a predetermined wavelength toward the sample 913, and a polarization modulation unit 912 for modulating light to be incident on the sample 913.
In addition, the measuring apparatus 901 subjects the light emitted from the sample 913 to predetermined polarization and the like, and enters the light emitted from the polarization analysis unit 914 and the polarization analysis unit 914 corresponding to the calculation described later, and outputs the predetermined electric signal. A detector 915 for converting and outputting the image data to the image data, and an operation means 916 for performing a predetermined operation using an output signal of the detector 915.

  As a specific measurement apparatus 901, for example, a collimator lens unit (not shown) is installed between the light source 911 and the polarization modulation unit 912, and between the collimator lens unit etc. and the polarization analysis unit 914. A sample stage (not shown) for mounting and fixing the sample 913 is installed, and an appropriate objective lens (not shown) is installed between the sample 913 and the polarization analysis unit 914, and between the polarization analysis unit 914 and the detector 915 A microscope (not shown) capable of infinity correction can be installed on the light source, and an optical waveguide etc. (not shown) can be appropriately provided between the light source 911 and the detector 915 described above to provide a microscope-type polarization measuring apparatus. It may be configured as

The light source 911 includes, for example, a quasi-monochromatic light emitter such as an LED that emits light having a wavelength of 780 nm, an optical filter, and the like, and is configured to emit light with a constant light intensity.
The polarization modulation unit 912 has a polarizer P1 and a phase plate R1, and the polarizer P1 is, for example, a linear polarization element, and is configured to be able to set and adjust the transmission axis orientation to a desired angle. For example, it is supported by a holder or the like. The phase plate R1 is, for example, a λ / 4 wavelength plate that converts linearly polarized light into circularly polarized light with respect to incident light having a wavelength of 633 nm, for example, a holder configured to be able to set and adjust the main axis azimuth to a desired angle. It is supported by
A sample stage (not shown) is placed between the polarization modulator 912 and the polarization analyzer 914, and the sample 913 is fixed at a predetermined position of the measuring apparatus 901 using this sample stage or the like.

The polarization analysis unit 914 has a phase plate R2 and a polarizer P2, and the phase plate R2 is, for example, a λ / 4 wavelength plate that converts linearly polarized light into circularly polarized light with respect to incident light of wavelength 633 nm. It is supported by, for example, a holder or the like, which is configured to be able to rotate to an angle for installation adjustment. The polarizer P2 is, for example, a linear polarization element, and is supported by, for example, a holder or the like configured to be able to set and adjust the transmission axis orientation to a desired angle.
The detector 915 is, for example, a CCD camera or the like provided with an imaging device capable of capturing an image or the like, a spectroscopic mechanism, or the like, and is configured to output a signal representing a captured color image.
In addition, the measuring apparatus 901 comprises a measurement optical system by the light source 911, the polarization modulation part 912, the polarization analysis part 914, the detector 915 etc. which were connected by the above-mentioned optical waveguide etc., for example. In particular, the inside of the measuring apparatus 901 or the place where the measuring apparatus 901 is installed needs to be a dark room environment so that light other than the light emitted from the polarization analysis unit 914 does not enter the detector 915.

A processor 916 inputs an image signal and the like output from the detector 915, and a processor that performs, for example, calculation of light intensity and the like and processing of acquired data included in the image signal, predetermined data and the like as necessary. The information processing apparatus is, for example, an information processing apparatus such as a personal computer, provided with a memory for storing and a display device for outputting and displaying the result of arithmetic processing and the like.
In order to enhance the measurement accuracy of the measurement apparatus 901, it is preferable to use a light source 911 having a good monochromaticity and a detector 915 having a good wavelength resolution.

The polarizer P1 and the phase plate R1 of the polarization modulation unit 912, and the phase plate R2 and the polarizer P2 of the polarization analysis unit 914 are rotatably supported by a holder or the like as described above. A mechanism unit that rotationally drives the holder or the like may be installed, and the operation of the mechanism unit may be controlled by, for example, the calculation unit 916. That is, the calculating means 916 provided with a processor, a memory, etc. is configured as control means for controlling the operation of each part of the measuring apparatus 901, or the calculating means 916 is included in the control means. The orientation of each of the above optical elements may be set or changed according to the data or in accordance with data set in advance. In addition, the operation of the light source 911, the detector 915 and the like may be controlled together by the above control means and the like.
Further, the calculation means 916 is configured to obtain values indicating the azimuth angle and the like of the optical elements such as the polarizer P1, the phase plate R1, the phase plate R2 and the polarizer P2, etc. For example, each of the above optical elements A holder or the like supporting the sensor is configured to obtain data representing a value such as an azimuth angle from an output signal of the sensor.

Next, the operation will be described.
here,
i) The polarization modulator 912
The transmission axis orientation of the polarizer P1 is θ _P1 ,
Principal axis orientation of phase plate R1 θ _R1 , phase difference (arbitrary) δ ₁ (δ ₁ ≠ 180 × n degrees)
ii) The polarization analyzer 914
Principal axis orientation of phase plate R2 θ _R2 , phase difference (arbitrary) δ ₂ (δ ₂ ≠ 180 × n degrees)
The transmission axis orientation of the polarizer P2 is θ _P2 ,
Define and explain.

The measurement of the polarization characteristic by the measuring apparatus 901 operates approximately as follows.
First, the phase difference [delta] ₂ of the phase plate R2 polarization analyzer 914 is measured by running the measurement optical system (Step1).
Then, using the data indicating the phase difference [delta] ₂ measured in Step1, we obtain the transfer matrix of the polarization analyzer 914 (4 × 4), and a light intensity inverse matrix and the detector 915 detects the transfer matrix The polarization characteristic (Stokes vector · 4 × 1) on the light incident side of the polarization analysis unit 914 is determined using (Step 2).
Further, in the operation of measuring the polarization characteristic of the sample 913, the Stokes vector [S _in ] obtained in the absence of the sample 913 and the Stokes vector [S _out ] when transmitted through the sample 913 are determined by the processing operation of Step 2. . From these, the Mueller matrix indicating the polarization characteristics of the sample 913 is obtained, and the polarization characteristics of the sample 913 are quantified using the elements of the Mueller matrix (Step 3).

Next, the operation of each step will be described.
Note that the operation processing described in Steps 1 and 2 below is a part of the operation processing performed in Step 3 described later.
<Step 1>
1-1. For example, with respect to the transmission axis azimuth theta _P1 of the polarizer P1 of the polarization modulation unit 912, the principal axis directions theta _R2 of the spindle orientation theta _R1 and the phase plate R2 of the phase plate R1, is set to be parallel or perpendicular orientation . At this time, the principal axis orientation of each phase plate may be set to either the fast axis or the slow axis.
1-2. For example, the main axis azimuth θ _R2 of the phase plate R2 of the polarization analysis unit 914 is rotated 45 × (2n + 1) degrees from the state set in the item 1-1 above.
1-3. The principal axis orientation θ _R2 of the polarization analysis unit 914 is set in the above item 1-2, and the transmission axis orientation θ _P2 of the polarizer _P2 of the polarization analysis unit 914 is the same as that of the polarizer P1 of the polarization modulation unit 912 The light intensity when it is set parallel to the transmission axis azimuth θ _P1 and the light intensity when it is set orthogonal are sequentially stored in the memory or the like of the calculation means 916.

1-4. Using the light intensity in the state (the azimuth angle) of each optical element having stored in item 1-3, and calculates the phase difference [delta] ₂ of the phase plate R2 polarization analyzer 914.
For example, the transmission axis azimuth theta _P1 = 0 ° of the polarizer P1, principal axis directions theta _R1 = 0 degrees phase plate R1, sets the spindle orientation theta _R2 = 45 degree phase plate R2, transmission axes of the polarizer P2 theta _The light intensity detected by the detector 915 at _{P 2} = 0 degrees (parallel to θ _{P 1} ) is I ₀ , and the light intensity detected by the detector 915 at transmission axis azimuth θ _{P 2} = 90 degrees (orthogonal to θ _{P 1} ) is I when a _90, a phase difference [delta] ₂ of the phase plate R2 is obtained by the following equation (1).

  In Step 1, the measuring apparatus 901 is in the same state as in the polarization measurement of the sample 913. For example, the magnification of each lens (not shown) is set to an optimum value. In the state where each part is set in this way, the orientation of each optical element (polarizer P1, phase plate R1, phase plate R2, polarizer P2) as described above without setting and fixing the sample 913 to the sample stage etc. For example, the above-described optical elements are set to the values shown in Table 1 of FIG. 26, the light emitted from the polarization analysis unit 914 is incident on the detector 915, and the light intensity is measured.

The detector 915 outputs, for example, RAW data as a signal indicating a photographed image, and the calculation unit 916 appropriately stores the RAW format image file output from the detector 915 in a storage unit such as a memory provided with itself. Do. That is, in Step 1, the image file stored in the storage unit is the one captured in the two states (azimuth angles of the respective optical elements) described in the above-mentioned item 1-3.
The calculating means 916 extracts the light intensity present in each pixel of the two stored (photographed in two states) image files, and the equation (1) described in the item 1-4. performing calculation for each pixel, and generates image data representing the two-dimensional distribution of the phase difference [delta] ₂ with the phase plate R2.

<Step 2>
2-1. The polarization state immediately before (on the light incident side) of the polarization analysis unit 914 is expressed as Stokes vector [S] = (S ₀ , S ₁ , S ₂ , S ₃ ), and detection is performed from Mueller matrix corresponding to each optical element described above. The equation for calculating the light intensity I _i in the case of using the device 5 is generated. In the measuring apparatus 901 illustrated here, it is defined as the following equation (2).

In equation (2),

2-2. Each optical elements constituting the polarization analyzer 914 (phase plate R2, polarizer P1) the azimuth angle of the set to an arbitrary value, by using the phase difference [delta] ₂ obtained in these azimuth angles and Step1, Stokes vector [ The upper coefficients A _i , B _i , C _i and D _i of S] are calculated.
2-3. Each angle of azimuth θ _R2 and azimuth θ _P2 of each optical element of polarization analysis unit 914 is arbitrarily set, and light intensity I _i after light of a certain incident polarization state passes through polarization analysis unit 914 is measured. The measured values are stored, for example, in the above-described memory or the like.
2-4. The angle setting of the azimuth θ _R2 and the azimuth θ _P2 of each optical element of the polarization analysis unit 914 is changed four times, and the operation processing described in the item 2-3 is repeated (i = 1 to 4).
Here, when the azimuth angle of the optical element is changed and the light intensity I _i is repeatedly measured, each angle of the azimuths θ _R2 and θ _P2 is arbitrarily set sequentially, but the above θ _a is not identical Set to

2-5. Coefficients A _i , B _i , C _i , D _i calculated in the above item 2-1 to item 2-4, Stokes vector [S] just before (polarization side) the polarization analysis unit 914, measured and stored The light intensity I _i is expressed as the following determinant (3).

From the above matrix equation (3), the transfer matrix A of the polarization analysis unit 914 whose coefficients are A _i , B _i , C _i and D _i are elements of the matrix, and its inverse matrix A ^-1 and each image file The Stokes vector [S] can be determined as shown in the following Equation (4), using the light intensity matrix I having the light intensity I _i of

In Step 2, the calculating means 916 of the measuring apparatus 901 determines an equation for calculating the light intensity I _i as described in the item 2-1 above, and as described in the item 2-2, each of the Stokes vectors [S] The coefficients A _i , B _i , C _i and D _i are calculated.
In addition, as described in item 2-3, the calculation unit 916 controls, for example, the above-described mechanism unit and the like for each optical element of the polarization analysis unit 914 to rotate a holder or the like that supports the optical element, The azimuth angle is set as shown in Table 2 of FIG. In addition, imaging at each azimuth angle is performed using the detector 915, and the light intensity I _i of each imaged image file is stored in the above-described memory or the like, and these operation processes are repeated as described in item 2-4. Then, the transfer matrix A and the inverse matrix A ⁻¹ are obtained, and the Stokes vector [S] indicating the polarization state immediately before the polarization analysis unit 914 is obtained.
The measuring apparatus 901 measures the polarization characteristic of the sample 913 by the operation processing of Step 3 described below. Step 3 is to perform operation processing (calculation processing) and the like described in Step 1 and Step 2 with respect to various azimuth angles and the like set in each optical element described later.

<Step 3>
3-1. State the measuring device 901 not including the sample 913, for example, the light source 911 is emitted as the state of the sample stage empty described above, the orientation theta _R1 of the phase plate R1 of the polarization modulating section 912 sets an arbitrary angle, above Step1 Also, the incident Stokes vector [S _inj ] is calculated by performing the arithmetic processing described in Step 2.
3-2. In the operation and arithmetic processing described in the item 3-1 above, the incident Stokes vector [S _inj ] is calculated by changing the condition (set angle of the azimuth θ _R1 ) of the polarization modulation unit 912 so that the polarization states do not overlap. _Are repeated four times (j = 1 to 4) to obtain incident Stokes vectors [S _in1 ], [S _in2 ], [S _in3 ], and [S _in4 ].

3-3. The light source 911 is caused to emit light in a state where the sample 913 is installed and fixed to the measuring device 901, and the sample 913 is irradiated with incident light under the same conditions as item 3-1 or item 3-2, and the light intensity detected in this state is The arithmetic processing described in Step 1 and Step 2 described above is performed to calculate a Stokes vector [S _outj ] after transmission through the sample. That is, the Stokes vectors after transmission [S _out1 ], [S _out2 ], [S _out3 ], [S _out4 ] for each of four azimuths θ _R1 set in the operation processing of item 3-2 (j = 1 to 4) Ask for
3-4. Here, when the polarization characteristic of the sample 913 is represented as Mueller matrix M, the relationship between the incident Stokes vector [S _inj ] and the post-transmission Stokes vector [S _outj ] can be expressed as the following formula (5) it can.

The incident Stokes vector [S _inj ] and the post-transmission Stokes vector [S _outj ] obtained by each operation process of item 3-1 to item 3-3 based on the above equation (5) are respectively 4 × 4 matrices If the elements are matrixed into a matrix S ′ _in and a matrix S ′ _out , they can be expressed as the following equation (6).

From the above equation (6), using the inverse matrix of the matrix S ′ _in consisting of each incident Stokes vector and the matrix S ′ _out consisting of each post-transmission Stokes vector, the elements of the Mueller matrix M of the sample 913 are It is calculated by the equation (7).

In Step 3, when calculating the incident Stokes vector [S _inj ] in a state where the sample 913 is not installed in the measuring device 901, the calculating means 916 of the measuring device 901 does not install the sample 913 as described in items 3-1 and 3-2. Incident polarization of the orientation (θ _R1 ) of the four patterns in the polarization modulation section 912} × {four elements (S ₀ , S ₁ , S ₂ , S ₃ ) = 16 elements indicating the polarization state immediately before the polarization analysis section 914 calculate. This is equivalent to the fact that the Stokes vector [S] related to the polarization analysis unit 914 described in Step 2 described above is composed of four elements (S ₀ , S ₁ , S ₂ , S ₃ ). Note that these calculations are performed for each pixel of the image file, as described in Step 1 described above.

FIG. 19 is an explanatory view showing an incident Stokes vector obtained by the calculation means 916 shown in FIG.
When the calculation means 916 obtains the incident Stokes vector [S _inj ], for example, any of the azimuth angles (θ _P1 , θ _R1 , θ _R2 , θ) shown in Table 2 of FIG. The transfer matrix A and the inverse matrix when the sample 913 is not installed in the measuring apparatus 901 by performing the processing operations of Step 1 and Step 2 as described in the item 3-1 with respect to the azimuth angle set in _P2 ) and set. A- ¹ is determined, and incident Stokes vectors [S _in1 ], [S _in2 ], [S _in3 ], and [S _in4 ] are determined using this. These incident Stokes vectors are each composed of four elements as shown in FIG.

Specifically, the arithmetic processing 16 is data indicating the values of the respective orientations θ _R2 and θ _P2 of the phase plate R2 and the polarizer P2 which have been set (for example, for the incident pattern 1 shown in Table 2 of FIG. 27) acquires, by using the phase difference [delta] ₂ obtained in these values and Step1, performs calculation of the equation (2). Further, the transfer matrix A is determined using the light intensities I _{1-1 to} I _4-1 acquired by the calculation of the equation (2), and the inverse matrix A ⁻¹ is determined.
Next, using the respective light intensities acquired in the incident pattern 1 and the inverse matrix A ⁻¹ , the elements (S _0inl , S _1inl , S _2inl ,) of the incident Stokes vector [S _in1 ] from the above equation (4) Calculate S _3inl ).
After that, as described in the item 3-2, the azimuth angle etc. of each optical element is set so that the polarization state does not overlap, for example, each setting shown in the incident patterns 2 to 4 shown in Table 2 of FIG. Also for the values, the same calculation processing as for incident pattern 1 is performed, and the four incident Stokes vectors [S _in1 ], [S _in2 ], [S in ₃ ], [S in ₃ ], and [S in ₃ ] that indicate the polarization states of the four patterns shown in FIG. _{in 4} ] or determine these vector elements.

FIG. 20 is an explanatory view showing a Stokes vector after transmission found by the calculation processing 16 of FIG.
When the arithmetic means 916 obtains the Stokes vector [S _outj ] after transmission, for example, any of the azimuth angles (θ _P1 , θ _R1 , θ _R2 , θ) shown in Table 2 of FIG. Transfer matrix A when the sample 913 is installed in the measuring apparatus 901 and light is incident on the azimuth angle set in _P2 ) and the operation processing of Step 1 and Step 2 as described in item 3-3 for the azimuth angle set. Also, the inverse matrix A ^-1 is obtained, and the transmitted Stokes vectors [S _out1 ], [S _out2 ], [S _out3 ], [S _out4 ] are obtained using this. Note that these post-transmission Stokes vectors each consist of four elements as shown in FIG.

FIG. 21 is an explanatory view showing an example of the sample 913 installed in the measuring apparatus 901 of FIG. The illustrated sample 913 is configured such that, for example, two types of retardation films are attached and fixed to a glass substrate, and the principal axis orientations of the respective retardation films are orthogonal to each other.
Specifically, when the post-transmission Stokes vector [S _outj ] is to be determined, for example, the sample 913 shown in FIG. The light emitted from the sample 913 is made incident on the detector 915 via the polarization analysis unit 914. At this time, azimuth angles and the like of the polarizer P1 and the phase plate R1 of the polarization modulation unit 912, and the azimuth angles of the phase plate R2 and the polarizer P2 of the polarization analysis unit 914 are the same as when the incident Stokes vector [S _inl ] is determined. For example, the values are set to the values shown in Table 2 of FIG. 27, and the calculation means 916 measures each light intensity when it is set to each value of the incident pattern 1, for example, and stores it in a memory or the like.

Thereafter, using the light intensity stored in the memory or the like, the same calculation processing as when the elements (S _0inl , S _1inl , S _2inl , S _3inl ) of the aforementioned incident Stokes vector [S _inl ] are determined is performed. When the sample 913 is set, the elements (S _{0out 1} , S _{1out 1} , S _{2out 1} , S _{3out 1} ) of the post-transmission Stokes vector (S _{0out 1} ) of (after transmission of the sample 913) are calculated.
Next, as in the case of _obtaining the incident Stokes vector [S _inl ] described above, the azimuth angle etc. of each optical element is set so that the polarization states do not overlap, for example, in the incident patterns 2 to 4 shown in Table 2. Also for each set value shown, the same calculation processing as for incident pattern 1 is performed, and four post-transmission Stokes vectors [S _out1 ], [S _out2 ], [S shown the polarization states of the four patterns shown in FIG. _out3 ], [S _out4 ], or these vector elements are determined.

The incident Stokes vectors [S _in1 ], [S _in2 ], [S _in3 ], [S _in4 ] and the post-transmission Stokes vectors [S _out1 ], [S _out2 ], [S _out3 ], [S _out4 ] _{thus obtained.} ] Is used to calculate the Mueller matrix M of the sample 913 by the operation of equation (7).
FIG. 22 is an explanatory drawing showing the Mueller matrix M of the sample 913 calculated by the calculation means 916. This figure shows the Mueller matrix M obtained using the measuring device 901 for the sample 913 shown in FIG. FIG. 22 (a) graphically shows the polarization state of each element constituting the Mueller matrix M, and FIG. 22 (b) shows the size or numerical value of each element of the Mueller matrix M.

  Next, operation means 916 discharges each polarization state from the elements of the calculated Mueller matrix M. Specifically, using the elements of the Mueller matrix M shown in the equation (6), the following equations are calculated to quantify the polarization characteristics of the sample 913.

  FIG. 23 is an explanatory drawing showing polarization states extracted from elements of the Mueller matrix. This figure shows the analysis result of the polarization characteristic of the sample 913 calculated using each of the above formulas, and FIG. 23 (a) shows the birefringence phase difference of the sample 913, and FIG. 23 (b) shows The principal axis orientation of the sample 913 is shown, and FIG. 23C shows the depolarization degree in the characteristic measurement of the sample 913.

FIG. 24 is an explanatory view showing an example of polarization characteristics measured using the measuring apparatus 901 of FIG. For example, when the polarization characteristic is measured using a phase plate with a design wavelength of 633 nm (λ / 4 plate with a phase difference of 90 degrees) as the sample 913, the light source 911 of the measuring apparatus 901 has a wavelength of 780 nm. Becomes approximately 75 degrees, and when the wavelength of the light source 911 and the design wavelength of the sample 913 are different as described above, an error of the degree shown in FIG. 24 occurs in the measurement result.
FIG. 25 is an explanatory view showing a measurement result using the related art and a measurement result using a general measurement (analysis) method. This figure shows the measurement (analysis) result when measuring the birefringence phase difference of the sample 913 (shown as a sample in the figure), and the analysis result (angle of phase difference) is shown on the vertical axis, the horizontal axis The phase difference (angle) of the sample 913 (sample) is shown in FIG. As for the measurement (analysis) results of the measurement apparatus using the general analysis method shown in this figure, the phase plate R2 (in this related art) provided in the apparatus and into which the light emitted from the sample is incident In the measuring apparatus 901, as the phase plate R2 of the polarization analysis unit 914), it is generally assumed that the phase plate R2 is an ideal λ / 4 wavelength plate within the range of 60 degrees to 120 degrees of phase difference. It is a calculation result of the birefringence phase difference using an analysis method.

The measurement (analysis) result of the general measurement device shown in FIG. 25 is that the error of the birefringence phase difference is about 10 to 20%. On the other hand, the measurement (analysis) result of the measuring apparatus 901 of this related art has an error within about 5% as shown in Table 3 of FIG.
From this, the measuring device 901 according to the related art performs the arithmetic processing using the correction based on the Mueller matrix M and the inverse matrix in the calculating unit 916, whereby the non-existent in the optical element etc. included in the measuring device 901. It is possible to eliminate the ideality and obtain a result closer to the original polarization characteristic of the sample 913. In other words, it is possible to correct the non-ideality of each optical element or the like and to eliminate the error of the polarization characteristic due to the optical element or the like.

In addition, since an error in polarization characteristics of an optical element or the like can be eliminated, the polarization characteristics of the sample 913 can be accurately measured even when the light wavelengths set for the sample 913 and each optical element or the like are different. be able to.
Further, the number of samplings when measuring the polarization characteristic can be suppressed to 16 and the time until calculation of the measurement result can be suppressed to a short time.

  The present invention can be applied to techniques relating to inspection and measurement for observing a sample (a sample) using polarized light, photographing technology for photographing a sample as a subject, polarization calculation technology for calculating polarization characteristics of a sample, and the like.

10 Subject 101 Light Source for Transmitting Illumination 102 Light Source Side Optical Element 102 a, 105 b Polarizer 102 b, 105 a QWP
DESCRIPTION OF SYMBOLS 103 Rotating table 104 Light source for reflective illumination 105 Camera side optical element 106 Camera 200 Imaging control device 201 Light source control part 202 for transmissive illuminations Light source side optical element control part 203 Rotation table control part 204 Reflective light source control part 205 Camera side optical element Control unit 206 Camera control unit 300 PC
400 storage device 500 display device 901 measurement device 911 light source 912 polarization modulation unit 913 sample 914 polarization analysis unit 915 detector 916 calculation means

Claims (16)

A first light source,
A first optical element for changing light transmitted from the first light source to a first polarization state and transmitting the light;
A second optical element for changing light transmitted from the first optical element into a second polarization state and transmitting the light;
When a subject is not disposed on the optical axis between the first optical element and the second optical element, the light incident from the second optical element is received and converted into an electric signal. When the subject is disposed on the optical axis between the first optical element and the second optical element, the light from the second optical element including the light transmitted through the subject An imaging device configured to perform an imaging process of receiving incident light and converting it into an electrical signal and outputting the electrical signal as image data;
A rotating device configured to support the subject and to rotate the subject to change the orientation of the subject with respect to the optical axis;
A plurality of states combining conditions of the first polarization state, the second polarization state, and the direction of the subject by controlling the first optical element, the second optical element, and the rotation device. A photographing control device configured to control the photographing device to sequentially perform the photographing processing in each state of the plurality of states;
Has a shooting system.   The photographing control device further controls the first light source to combine the on / off of the first light source, the first polarization state, the second polarization state, and the direction of the subject. The imaging system according to claim 1, wherein the imaging system is configured to create a plurality of states. And a second light source disposed closer to the imaging device than the subject,
The photographing control device further controls the second light source to turn on / off the first light source, turn on / off the second light source, the first polarization state, and the second polarization state. The imaging system according to claim 1 or 2, configured to create a plurality of states combining each condition of the direction of the subject. The first optical element has a configuration in which a polarizer and a wave plate are disposed in this order from the light source side,
The imaging control device is configured to set the first polarization state to a desired state by setting an azimuth angle of at least one or both of the transmission axis direction of the polarizer and the principal axis direction of the wave plate. The imaging system according to any one of claims 1 to 3. The first optical element includes a plurality of wavelength plates in which the main axis direction is set in advance,
The imaging control apparatus switches the plurality of wavelength plates, and any one of the plurality of wavelength plates is disposed on the optical axis, or any of the plurality of wavelength plates is on the optical axis 5. The imaging system according to claim 4, wherein the first polarization state is set to a desired state by setting the first polarization state not to be disposed. The second optical element has a configuration in which the light source, the wavelength plate, and the polarizer are arranged in this order,
The imaging control device is configured to set the second polarization state to a desired state by setting an azimuth angle of at least one or both of the principal axis direction of the wave plate and the transmission axis direction of the polarizer. The imaging system according to any one of claims 1 to 5. The wave plate of the second optical element is configured to be rotatable with the direction of the optical axis as a rotation axis,
The imaging control apparatus is configured to set the second polarization state to a desired state by rotating the wave plate of the second optical element to set the azimuth angle of the principal axis direction of the wave plate. The imaging system according to claim 6. The second optical element includes a plurality of polarizers having transmission axis orientations set in advance.
The imaging control device switches the plurality of polarizers, and any one of the plurality of polarizers is disposed on the optical axis, or any of the plurality of polarizers is disposed on the optical axis 8. The imaging system according to claim 6, wherein the second polarization state is set to a desired state by setting the second polarization state to a non-arranged state.   The first light source according to any one of claims 1 to 8, wherein the first light source is configured to emit light of a specific wavelength or light of a specific color obtained by combining light of a specific wavelength. Shooting system.   The imaging system according to claim 3, wherein the second light source is configured to emit light of a specific wavelength or a color obtained by combining the light of the specific wavelength.   When the subject is disposed on the optical axis, and when the subject is not disposed on the optical axis, the photographing is performed in each of the plurality of states. The imaging system according to any one of claims 1 to 10.   The rotation device is configured to support a plurality of the objects, and to change the directions of the objects with respect to the optical axis in synchronization or independently. The imaging system according to any one.   An image processing configured to perform image processing using a plurality of image data corresponding to each of the plurality of states obtained by the imaging system according to any one of claims 1 to 12. apparatus. A polarization calculation unit that calculates a numerical value representing the polarization characteristic of the subject corresponding to each of the pixels using the luminance value of each pixel included in each of the plurality of image data;
An image data generation unit for generating new image data using the numerical value representing the polarization characteristic of the subject corresponding to each pixel calculated by the polarization calculation unit;
The image processing apparatus according to claim 13, comprising:   The image data group selected from the plurality of image data and new image data generated from the plurality of image data is determined to represent the arrangement position in the N (N is an integer of 2 or more) dimensional array format According to a naming convention, a file name is given to each image data contained in the image data group, and a display image data file including the image data group in which the file name is given to each image data is generated for display The image processing apparatus according to claim 13, further comprising an image data file generation unit. A plurality of image data corresponding to each state of the plurality of states obtained by the imaging system according to any one of claims 1 to 12 is read from a predetermined storage device;
Performing image processing using the plurality of image data read from the predetermined storage device;
Image processing method.