JP2009294463A - Microscope system - Google Patents

Abstract

To provide a microscope system capable of automatically determining a staining method of a specimen necessary for faithfully reproducing a specimen color from an obtained specimen image.
A microscope 1 for magnifying and observing a specimen image, an imaging means 21 for capturing the specimen image obtained by the microscope and outputting the specimen image, and a tint discrimination means 2311 for discriminating the hue of the specimen image. The staining method determining means 2313 for determining the staining method of the specimen from the result of the color discrimination means and reference color distribution data prepared in advance, and the staining method determined by the staining method determining means. A microscope system 100 comprising color conversion matrix selection means 2306 for selecting a corresponding color conversion matrix, and color conversion means 2304 for performing color conversion of the observation image using the selected color conversion matrix. .
[Selection] Figure 2

Description

  The present invention relates to a microscope system.

Conventionally, a sample image magnified by a microscope is picked up by a digital camera, and the color component contained in the picked-up sample image is referred to a color conversion matrix corresponding to a staining method input in advance by a user, and this color conversion matrix is obtained. A digital camera for a microscope that reproduces a faithful color by using it has been proposed (for example, see Patent Document 1).
Japanese Patent Laid-Open No. 2003-230154

  However, the conventional apparatus has a problem that troublesome operations such as a user confirming and inputting a staining method at the time of specimen observation of a color conversion matrix corresponding to the staining method occur.

  In order to solve the above-described problems, the present invention discriminates the color of the specimen image, a microscope that magnifies and observes the specimen, an imaging means that captures an image of the specimen obtained by the microscope and outputs the specimen image Color determination means, staining method determination means for determining the staining method of the sample from the result of the color determination means and reference color distribution data prepared in advance, and the color determined by the staining method determination means A microscope system comprising color conversion matrix selection means for selecting a color conversion matrix corresponding to a staining method, and color conversion means for performing color conversion of the observation image using the selected color conversion matrix I will provide a.

  ADVANTAGE OF THE INVENTION According to this invention, the microscope system which can determine automatically the staining method of the sample required in order to reproduce the color of a sample faithfully from the acquired sample image can be provided.

  Hereinafter, a microscope system according to an embodiment of the present invention will be described with reference to the drawings. The following embodiments are merely for facilitating understanding of the invention, and excluding additions and substitutions that can be performed by those skilled in the art without departing from the technical idea of the present invention. It is not intended.

  FIG. 1 is a schematic configuration diagram of a microscope system according to an embodiment.

  In FIG. 1, an imaging device 2 connected to a microscope 1 illuminates a sample 13 placed on a stage 12 of the microscope 1 with an illumination device 14 and captures an observation image, which will be described later, with a camera unit 21. An observation image captured by the camera unit 21 is sent from the camera unit 21 to the CCU 23 (Camera Control Unit) via the cable 22. The objective lens 16 to be used can be exchanged by rotating the revolver 15, and the magnification of the observation image of the sample 13 can be changed.

  The CCU 23 performs later-described color correction and color conversion processing on the observation image sent from the camera unit 21, and displays a suitable observation image on the monitor 3 according to the staining method of the specimen 13. At this time, the user can instruct the image processing content to be executed by the CCU 23 by operating the mouse 5.

  Further, the microscope 1 is connected to the CCU 23 via the USB interface cable 18, and the CCU 23 obtains information on the microscope 1 by issuing a control command to the microscope 1 or replaces a color filter described later in the optical path of the microscope 1. Can perform actions. In addition, visual observation of the specimen image is possible through the eyepiece 17 of the microscope 1. In this way, the microscope system 100 is configured.

  In the microscope system 100, the specimen 13 is mainly colorless cells, and the cells are stained by dropping a selected staining solution so that the sample 13 can be clearly identified visually. Currently, various staining methods are known depending on the application, and the microscope system 100 is suitable for the staining method of the specimen 13 so that the observation image captured by the camera unit 21 can be observed as a suitable image on the monitor 3. Color correction and color conversion processing can be executed.

  Hereinafter, the observation image processing will be described with reference to FIG.

  FIG. 2 is a functional block diagram of the CCU 23 of the microscope system 100 according to the embodiment.

  In FIG. 2, sample image information in the RAW format digitized by the camera unit 21 is input to the correction processing block 2301 via the camera head interface unit 2301. The correction processing block 2302 performs various correction processes such as color balance correction, defect correction, shading correction, and γ value correction on the input observation image.

  Thereafter, the RGB pixel interpolation block 2303 converts the image data into an RGB format that can be displayed on the monitor 3, the color conversion block 2304 further performs color conversion, and the sample image is displayed on the monitor 3 via the monitor display control unit 2305.

  Further, the CPU 2306 issues an instruction to move the stage 12 to the microscope 1 via the USB control unit 2308 and moves the objective lens 16 to a position where the sample 13 is not present and the illumination light strikes, and the camera unit 21 displays an image of only the illumination light. Take an image with. After the captured image is acquired via the camera head interface unit 2301, the correction processing block 2302, the RGB pixel interpolation block 2303, and the color conversion block 2304, the CPU 2306 calculates the white balance. The CPU 2306 stores the calculated white balance value in the white balance value storage unit 2309. This value is used for correction such as color balance correction in the correction processing block 2302.

  Further, the CPU 2306 acquires the color temperature of the illumination light from the microscope 1 via the USB control unit 2308 and stores the acquired color temperature in the color correction circuit 2310. This color temperature information is used in the color discrimination described later. Thereafter, the CPU 2306 again instructs the microscope 1 to move the stage 12 via the USB control unit 2308, moves the objective lens 16 to a position where the sample 13 is located, and acquires the sample image by the camera unit 21.

  At this time, the operator places the stained specimen 13 on the stage 12 of the microscope 1 shown in FIG. 1, operates the input device 5, and inputs it to the CPU 2306 via the input device control unit 2321 and the menu display unit 2322. Thus, a color adjustment instruction operation or the like can be performed on the menu screen displayed on the monitor 3 via the monitor display control unit 2305.

  When the CPU 2306 receives a color adjustment start instruction from the menu display unit on the monitor 3 by the input device 5, the standard color conversion matrix stored in the color conversion matrix storage unit 2307 as the color conversion matrix used in the color conversion block 2304. 2307s is selected.

  The color correction circuit 2310 obtains an RGB value obtained by removing the influence of the color temperature from the RGB sample image output from the color conversion block 2304 by a known method, and inputs the RGB value to the color determination unit 2311.

The color discrimination unit 2311 converts the RGB value of the acquired sample image of one frame into a YCbCr value based on the following conversion equation group (1), discriminates the existing color, and outputs it to the staining method determination unit 2313.
Y = 0.2990R + 0.5870G + 0.1140B
Cb = -0.1687R-0.3313G + 0.5000B (1)
Cr = 0.5000R-0.4187G-0.0813B

  The staining method determination unit 2313 stores the YCbCr value in the color distribution data storage unit 2312. For example, the HE staining color distribution data 2312a, the Azan staining color distribution data 2312b, the one-Geeson staining color distribution data 2312c, and the Masson trichrome staining color. The staining method of the specimen 13 is determined from the specimen image by comparison with the color distribution data corresponding to each staining method such as the distribution data 2312d.

  Here, the processing of the color determination unit 2311 will be described with reference to FIGS. 3 to 5.

  FIG. 3 shows a diagram in which CbCr coordinates are divided into predetermined regions (16 divisions). FIG. 4 is a diagram showing an example of the number of pixels for the regions 1 to 16 shown in FIG. FIG. 5 is a diagram showing the weighting of each staining method for the regions 1 to 16 shown in FIG. 3 as an example of registered color distribution data 2312a to 23d.

As shown in FIG. 3, the color determination unit 2311 divides the CbCr data of the output YCbCr data into 16 regions i (1 to 16), and determines which region the CbCr pixels belong to 1 frame. Check all the pixels or the specified area. Since Cb and Cr each take a value of −128 to 128, each region can be classified by an angle θ formed by a line segment connecting the origin and a point on the circumference of the radius 128 with the X axis.
Region 1 is O ≦ θ <π / 8
Region 2 is π / 8 ≦ θ <2π / 8
Region 3 is 2π / 8 ≦ θ <3π / 8
Region 4 is 3π / 8 ≦ θ <4π / 8
...
Region 16 is 15π / 8 ≦ θ <16π / 8
It becomes.

In order to classify the output CbCr pixels, an angle θ formed by a line segment connecting the coordinate point of the pixel data and the origin may be obtained. Assuming that the output Cb pixel value is a and the Cr pixel value is b,
θ = tan ⁻¹ (b / a)
It becomes.

  All the output CbCr pixels are classified into each region based on the above formula, the number of pixels belonging to each region is counted, and a data table of region i and pixel number Xi shown in FIG. 4 is created.

  Next, a method for creating color distribution data 2312a stored in advance will be described. In this method, a region of a color that can exist for a certain staining method is specified and weighted.

  For example, it is assumed that the HE-stained image includes pixels having colors of regions 5, 10, and 12. The area 5 weighting Y55 is set to 0, and the value of Y5i is incremented by 1 for each area away from the position, and numbers 1 to 8 are assigned. Similarly, the areas 10 and 12 are also weighted to determine Y10i and Y12i. This is shown in FIG. The smallest of the weighting data (Y5i, Y10i, Y12i) regarding these three regions is defined as Yi of the staining method. An example of the created 2312a color distribution data is shown in FIG. In the same manner, color distribution data 2312b to 2312d are calculated and registered from the possible color areas specified for each staining method. Yi indicates the probability of a pixel that does not exist in the image captured by the corresponding staining method. In other words, the greater the difference from a possible color, the greater the value and the lower the probability of existence.

Determination of the staining method in the staining method determination unit 2313 is performed by calculating the following determination coefficient.
Determination coefficient = ΣXiYi

  Here, Xi is the presence rate of the pixels in the region i among the pixels of the sample image (see FIG. 4), and Yi is the weighting value of the region i in the color distribution data (see FIG. 5).

  The staining method determination unit 2313 performs determination coefficient calculation on all registered color distribution data 2312a, 2312b, 2312c, and 2312d, and the staining method of the color distribution data that provides the smallest determination coefficient stains the sample 13. Determine the staining method. The CPU 2306 receives the staining method information determined by the staining method determination unit 2313, and receives a color conversion matrix (HE staining color conversion matrix 2307a, Azan staining color conversion matrix 2307b, One-Geeson staining color conversion matrix 2303c, Masson trichrome staining color conversion matrix 2307d. The corresponding color conversion matrix is read from the color conversion matrix storage unit 2307 and set in the color conversion block 2304.

  In addition, the CPU 2306 performs color adjustment filter No storage unit 2314 (HE staining color adjustment filter No (2314a), azanine staining color adjustment filter No (2314b), One Gieson staining color adjustment filter No (2314c), Masson trichrome staining. The color adjustment filter No (2314d)) is read out from the color adjustment filter No. optimal for the determined staining method, and a color filter replacement instruction is issued to the color filter replacement device 19 of the microscope 1 via the USB control unit 2308. .

  As described above, the CCU 23 automatically performs color correction and color conversion processing corresponding to the staining method of the sample 13 by performing color correction and color conversion processing on the sample image picked up by the camera unit 21 and displays it on the monitor 3. Can be possible.

  Next, the color correction and color conversion processing flow will be described with reference to FIGS. FIG. 6 shows a color correction and color conversion processing flow. Hereinafter, each step will be described.

(Step S1)
The CPU 2306 issues a color temperature measurement instruction for the illumination device 14 to the microscope 1 via the USB control unit 2308.

(Step S2)
The microscope 1 receives the instruction from the CPU 2306 and measures the color temperature of the illumination light of the illumination device 14.

(Step S3)
The microscope 1 transmits the measured color temperature information to the CPU 2306 via the USB control unit 2308.

(Step S4)
The CPU 2306 acquires color temperature information from the microscope 1 and inputs it to the color correction circuit 2310.

(Step S5)
The CPU 2306 instructs the microscope 1 and the camera unit 21 to acquire a sample image of the sample 13. The camera unit 21 captures an image of the sample and outputs the sample image to the CCU 23.

(Step S6)
The sample image output from the camera unit 21 is input to the correction processing block 2302 via the camera head interface unit 2301. In the correction processing block 2302, various corrections such as color balance correction, defect correction, shading correction, and γ value correction are performed with reference to the white balance value (white balance value storage unit 2309) measured in advance, and RGB pixel interpolation is performed. The color is input to the color determination unit 2311 via the block 2303 and the color conversion block 2304. At this time, the CPU 2306 sends the standard color conversion matrix 2307 s from the color conversion matrix storage unit 2307 to the color conversion block 2304 and is used for color conversion.

(Step S7)
The color determination unit 2311 determines the color of the sample image for each pixel by performing the above-described YCrCb value processing and the like. At this time, the color temperature information measured in advance is used by the color determination unit 2311 via the color correction circuit 2310. The determination result is input to the staining method determination unit 2313, and the staining method determination unit 2313 compares the color determination result with each of the stained color distribution data 2312a to 2312d in the color distribution data storage unit 2012 to stain the sample 13. Decide how.

(Step S8)
The CPU 2306 selects a color conversion matrix corresponding to the staining method determined by the staining method determination unit 2313 from the color conversion matrix storage unit 2307 and sends it to the color conversion block 2304. For example, when the staining method determination unit 2313 determines that the HE staining color distribution data 2312a is the closest staining method, the HE staining color conversion matrix 2307a is selected from the color conversion matrix storage unit 2307 by the CPU 2306. Then, the color conversion of the specimen image is performed in the color conversion block 2304 based on the selected HE-stained color conversion matrix 2307a. The sample image color-converted by the color conversion block 2304 is displayed on the monitor 3 via the monitor display control unit 2305.

(Step S9)
The CPU 2306 selects a color adjustment filter No suitable for the staining method determined by the staining method determination unit 2313 from the color adjustment filter No storage unit 2314 and transmits it to the microscope 1 via the USB control unit 2308. For example, when the staining method determination unit 2313 determines that the HE staining color distribution data 2312a is the closest staining method, the CPU 2306 selects the HE staining color adjustment filter No (2314a) from the color adjustment filter No storage unit 2314.

(Step S10)
In the microscope 1, the color filter replacement device 19 inserts the color filter instructed by the CPU 2306 into the optical path of the illumination optical system. As a result, a color filter suitable for the staining method of the specimen 13 is set in the illumination optical system, and the color enhancement of the specimen image of the specimen 13 is performed.

  As described above, determination of the staining method of the specimen 13, color correction and color conversion processing of the specimen image are performed, and the color filter of the microscope 1 is exchanged.

  After that, by executing Step S3 to Step S10 again, the color of the specimen image can be made closer to the color corresponding to the staining method. The color conversion matrix used in step S6 at the time of re-execution is a color conversion matrix (for example, the HE staining color conversion matrix (2307a)) corresponding to the staining direction determined in step S8.

  Further, the re-execution from step S3 to step S10 is a process executed to better reproduce the color, and in the microscope system 100 according to the embodiment, it is sufficient to execute step S1 to step S10 once. Needless to say, it produces a great effect.

  As described above, in the microscope system 100 according to the embodiment, the CCU 23 automatically determines the staining method in which the specimen 13 is stained from the specimen image of the specimen 13 from the color information of the specimen image, and the corresponding color conversion matrix. And the sample image can be color-converted so as to have a color suitable for the staining method.

  In addition, the microscope system 100 according to the embodiment can set a color filter optimal for the staining method to the illumination light of the microscope 1 and can acquire a better specimen image.

1 is a schematic configuration diagram of a microscope system according to an embodiment. The functional block diagram of CCU of the microscope system 100 which concerns on embodiment is shown. The figure which divided the YCbCr coordinate into the predetermined area (16 divisions) is shown. The figure which shows an example of the pixel existing ratio with respect to the area | regions 1-16 shown in FIG. It is a figure which shows an example of the weighting value of a certain color distribution data with respect to the area | regions 1-16 shown in FIG. The color correction and color conversion processing flow is shown. An example of the weighting result table | surface at the time of color distribution data 2312a creation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Microscope 2 Imaging device 3 Monitor 5 Input device (mouse)
12 Stage 13 Specimen 14 Illumination Device 15 Revolver 16 Objective Lens 17 Eyepiece 18 USB Cable 19 Color Filter Replacement Device 21 Camera Unit 22 Cable 23 Camera Control Unit (CCU)
DESCRIPTION OF SYMBOLS 100 Microscope system 2301 Camera head interface part 2302 Correction process block 2303 RGB pixel interpolation block 2304 Color conversion block 2305 Monitor display control part 2306 CPU
2307 Color conversion matrix storage unit 2308 USB control unit 2309 White balance value storage unit 2310 Color correction circuit 2311 Tint determination unit 2312 Color distribution data storage unit 2313 Dyeing method determination unit 2314 Color adjustment filter No storage unit 2321 Input device control unit 2322 Menu Display section

Claims (8)

A microscope for magnifying the specimen,
Imaging means for capturing an image of the specimen obtained by the microscope and outputting the specimen image;
A tint discriminating means for discriminating the tint of the sample image;
Staining method determining means for determining the staining method of the specimen from the result of the color determination means and reference color distribution data prepared in advance;
Color conversion matrix selection means for selecting a color conversion matrix corresponding to the staining method determined by the staining method determination means;
A microscope system comprising: color conversion means for performing color conversion of the observation image using the selected color conversion matrix.   The microscope system according to claim 1, further comprising a color distribution data storage unit that stores the color distribution data.   The microscope system according to claim 1, further comprising a color conversion matrix storage unit that stores the color conversion matrix.   The microscope system according to claim 3, wherein the color conversion matrix storage unit includes a standard color conversion matrix. Color temperature detection means for detecting the color temperature of the illumination light of the microscope;
The microscope system according to any one of claims 1 to 4, further comprising color correction means for correcting the color of the specimen image with the detected color temperature. Color filter selection means for selecting a color filter corresponding to the selected color conversion matrix;
Color filter replacement means for inserting the selected color filter into the optical path of the microscope;
The microscope system according to claim 1, comprising:   The microscope system according to claim 6, further comprising a color filter number storage unit that stores a number of the color filter.   The microscope system according to claim 1, wherein the microscope includes bright field observation.