CN1993983A - Imaging apparatus and microscope apparatus using the same - Google Patents

Abstract

An imaging apparatus (1A) comprising an light detecting part (10) having a plurality of pixels; a charge transferring part (12) having 16 partial charge transferring parts (T01-T16); an A/D converting part (15) for converting signals from the charge transferring part (12) to digital data signals; and a digital signal processing part (20). The digital signal processing part (20) corrects the firstly outputted data signal of a signal sequence from a partial charge transferring part by use of a plurality of correction data signals including at least one of first and second correction data signals, the first correction data signal being included in that signal sequence, the second correction data signal being included in another signal sequence from a partial charge transferring part adjacent to the output terminal of the foregoing partial charge transferring part. In this way, there can be realized an imaging apparatus and a microscope apparatus using the same, the imaging apparatus being capable of effectively reducing the affection of abnormal output states occurring in the data signals.

Description

Imaging device and microscope device using the same

technical field

The present invention relates to an imaging device such as a CCD image sensor and a microscope device using the imaging device.

Background technique

The CCD sensor of an imaging device that obtains a one-dimensional or two-dimensional image is composed of a photodetector in which a plurality of pixels are arranged in an array, and a horizontal shift register that inputs charges from a plurality of vertical shift registers constituting the photodetector in parallel. In such a configuration, the charge generated in each pixel of the photodetector is transferred vertically through the vertical shift register and input to the corresponding unit of the horizontal shift register. The horizontal shift register transfers the charge input from the vertical shift register to the horizontal direction as the output direction, and reads the charge at the output terminal.

In addition, in order to read the charge generated in the photodetector at high speed, a multi-tap structure is used in which the horizontal shift register is divided into a plurality of partial shift registers, and the output terminals of charges are provided for each partial shift register. . Such a configuration in which a plurality of output terminals are provided is particularly effective when the number of pixels in the photodetection section increases, and the time required to read charges from all pixels in a common configuration becomes longer (see, for example, Patent Document 1, 2).

Patent Document 1: JP-A-2001-119010

Patent Document 2: JP-A-2003-198954

Patent Document 3: Japanese Unexamined Patent Application Publication No. 8-18779

Contents of the invention

In the imaging device having the above-mentioned multi-tap structure, output abnormalities such as output attenuation may occur in the data signals (first data of each tap) output from each output port first. Such an output abnormality is a cause of deterioration in image quality of the overall image data obtained by combining the signal trains of the data signals output from the plurality of output terminals.

As a method of reducing the influence of such data signal output abnormality, after obtaining image data converted from a data signal output from an imaging device, data correction is performed on a computer or the like that reads the image data using software. However, with this method, there are problems such as that since the correction process is performed after the image data is output from the imaging device, it takes time to obtain the final image data after data correction.

The present invention is to solve the above problems, and an object of the present invention is to provide an imaging device capable of effectively reducing the influence of an output abnormality occurring in a data signal, and a microscope device using the imaging device.

In order to achieve such an object, the imaging device according to the present invention is characterized in that it has (1) a photodetection unit having a plurality of pixels arranged in an array and outputting electric charges generated in corresponding to the light incident amount in the pixels, and (2) electric charges The transfer unit is arranged along one arrangement direction of the plurality of pixels relative to the photodetection unit, and has N partial charge transfer units divided in the arrangement direction, (3) the A/D conversion unit converts The signal generated by the charge output by the charge transfer unit is converted into a digital data signal and (4) the digital signal processing unit performs signal processing on the data signal output from the A/D conversion unit. (5) The partial charge transfer section preferably transfers the charge from the pixels located in the predetermined photodetection area of the photodetection unit to the output direction and outputs it from the output terminal, and at the same time, the digital signal processing unit transfers the charge from the partial charge transfer section The data signal output first in the signal sequence is used as the correction object, and other predetermined signals included in the signal sequence are used as the first correction data signal, which is sent from the output terminal side adjacent to the partial charge transfer part. The predetermined data signal included in other signal columns of the partial charge transfer section is used as the second correction data signal, and a plurality of correction data signals including at least one of the first correction data signal and the second correction data signal are used. The data signal undergoes data correction.

In the above-mentioned imaging device, the charge transfer unit for inputting in parallel the plurality of pixels of the photodetection unit and the charges from the plurality of vertical shift registers constituting the photodetection unit is divided into a plurality of partial charge transfer units each having an output terminal. Multi-tap configuration. Accordingly, it is possible to quickly read the charge generated by each pixel of the photodetection device.

In addition, in such a multi-tap structure, a digital signal processing unit using a digital signal processor (DSP: Digital Signal Processor) is provided for an output abnormality that occurs in the first data signal from each output terminal of a plurality of partial charge transfer parts. . Use this digital signal processing unit to perform data correction for abnormal output of digital signals. Thus, the final image data after data correction can be obtained conveniently and quickly. In addition, by using a plurality of data signals for correction including at least one of the first data signal for correction and the second data signal for correction, the influence of output abnormalities occurring in the data signals can be effectively reduced, and images with good image quality can be obtained. image data.

The microscope device related to the present invention is characterized in that the above-mentioned imaging device formed by the optical image of the sample as the observation object is obtained, and the light guide optical system includes an objective lens that makes the light from the sample incident, and guides the optical image of the sample into the imaging device. , and the image acquisition control unit controls the acquisition of the image of the sample formed by the imaging device.

In the above-mentioned microscope device, using the above-mentioned imaging device, data correction is performed for an output abnormality of the data signal in the digital signal processing unit with respect to the head signals respectively output from the respective output terminals of the plurality of partial charge transfer units. Thereby, when observing a sample, image data with good image quality can be obtained.

With the imaging device and the microscope device using the imaging device of the present invention, while adopting a multi-tap structure in which the charge transfer device is divided into a plurality of partial charge transfer devices, by using a predetermined correction method in the digital signal processing device, Performing data correction for output abnormality on the first output data signals in the signal trains from the respective output terminals of the plurality of partial charge transfer units can effectively reduce the influence of the output abnormality occurring in the data signals.

Description of drawings

FIG. 1 is a block diagram showing the configuration of a first embodiment of an imaging device.

FIG. 2 is a diagram showing data signals input to input FIFOs of the first to fourth DSPs.

FIG. 3 is a diagram showing data signals output from output FIFOs of the first to fourth DSPs.

FIG. 4 is a diagram showing an example of a data signal output from a charge transfer unit.

FIG. 5 is a diagram showing an example of image data used for shading correction of a data signal.

6 is a plan view showing an example of the configuration of a photodetection unit and a charge transfer unit.

FIG. 7 is a block diagram showing an example of the configuration of a digital signal processing device (DSP).

Fig. 8 is a block diagram showing the configuration of the first embodiment of the microscope device.

FIG. 9 is a block diagram showing the configuration of a second embodiment of the imaging device.

FIG. 10 is a diagram showing data signals input to a memory through EMIFs of the first to fourth DSPs.

FIG. 11 is a block diagram showing the configuration of a third embodiment of the imaging device.

FIG. 12 is a diagram showing a data signal input to a memory through an EMIF of a DSP.

Explanation of symbols

1A, 1B, 1C: imaging device, 10: photodetection unit, R01-R16: photodetection area, 11: storage unit, 12: charge transfer unit, part T01-T16 charge transfer unit, 13: storage unit, 14: charge Transmission part, part of T17~T32 charge transfer part, 15: A/D conversion part, C01~C16: A/D converter, 16a~16c data selection circuit (DS circuit), 17: RAM, 18: buffer, 19 : Control part, 20: Digital signal processing part, 21-29 Digital signal processing device (DSP), 3: Microscope device, 30: Sample stage, 35: XY stage driving part, 40: Light guiding optical system, 41: Objective lens, 45: optical system driving unit, 50: control device, 51: input device, 52: display device

Detailed ways

Hereinafter, preferred embodiments of the imaging device and the microscope device using the imaging device according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same reference numerals are used for the same elements, and overlapping descriptions are omitted. In addition, the dimensional ratios in the drawings do not necessarily match the contents of the description.

FIG. 1 is a block diagram showing the configuration of a first embodiment of the imaging device of the present invention. The imaging device 1A of the present embodiment includes a photodetection unit 10 , a charge transfer unit 12 , an A/D conversion unit 15 , and a digital signal processing unit 20 .

The photodetection unit 10 has a plurality of pixels arranged in a two-dimensional array, each of which has a photoelectric conversion function, and is a photodetection unit that outputs charges generated in accordance with the amount of light incident on the pixels. Here, for convenience of description, in the arrangement direction of the pixels of the photodetection unit, the direction from left to right is defined as the direction of the X-axis, and the direction from bottom to top is defined as the direction of the Y-axis. In addition, a plurality of pixels in a two-dimensional array in the photodetection unit 10 constitutes a plurality of shift registers whose charge transfer direction is in the vertical direction (the negative direction of the Y axis in FIG. 1 ).

As for the photodetection section 10 , in the lower part of the figure, along the direction of the X axis, which is one of the arrangement directions of a plurality of pixels, a charge transfer section 12 is provided. The charge transfer unit 12 transfers the charges output from the plurality of vertical shift registers constituting the photodetection unit 10 and then input in parallel to a predetermined output direction (horizontal direction, negative direction of the X axis in FIG. The output terminal outputs the horizontal shift register.

In the present embodiment, the charge transfer unit 12 has 16 partial charge transfer units T01 to T16 divided into a plurality in the X-axis direction. The partial charge transfer units T01 to T16 are each composed of a plurality of cells of the same number, and are arranged in order from left to right in the figure. In addition, as described above, the charge transfer direction of each of the partial charge transfer parts T01 to T16 is the negative direction of the X-axis, and the left end thereof is the output end of the charge.

In addition, for the partial charge transfer sections T01 - T16 in the charge transfer section 12 , the photodetection section 10 can be divided into corresponding 16 photodetection regions R01 - R16 . In this structure, the first partial charge transfer unit T01 of the charge transfer unit 12 transfers the charges output from the pixels located in the first photodetection region R01 of the photodetector 10 in the output direction by using the vertical registers. Output output. At this time, the signal output from the output terminal of the partial charge transfer unit T01 becomes a signal sequence, which is sequentially output and input in parallel from a plurality of vertical shift registers in the photodetection region R01 to a plurality of units of the partial charge transfer unit T01 The signal generated by the charge. For the second to sixteenth part charge transfer parts T02 to T16 of the charge transfer part 12 and the second to sixteenth photodetection regions R02 to R16 of the photodetection part 10, the structure is also the same as that of the first part charge transfer part T01 and the first part. The light detection region R01 is the same.

Sixteen A/D converters C01 to C16 are provided in the A/D conversion section 15 corresponding to the above-described partial charge transfer sections T01 to T16 . The analog signals generated by the charges from the photodetection section 10 output from the output terminals of the partial charge transfer sections T01 to T16 are converted into digital signals in the corresponding A/D converters in the A/D converters C01 to C16. data signal.

The data signals output from the A/D converters C01 to C16 are input to a digital signal processing unit 20 for performing predetermined signal processing on the data signals. In the present embodiment, the digital signal processing unit 20 is composed of four digital signal processing devices (DSP) 21 to 24 .

The first DSP 21 is provided corresponding to the first to fourth partial charge transfer units T01 to T04, and receives data signals from the A/D converters C01 to C04. The second DSP 22 is provided corresponding to the fifth to eighth charge transfer units T05 to T08, and receives data signals from the A/D converters C05 to C08. The third DSP 23 is provided corresponding to the ninth to twelfth charge transfer units T09 to T12, and receives data signals from the A/D converters C09 to C12. The fourth DSP 24 is provided corresponding to the 13th to 16th part charge transfer units T13 to T16, and receives data signals from the A/D converters C13 to C16. Furthermore, the signal processing operations of these DSPs 21 to 24 are controlled by the control section 19 including a CPU.

In addition, in FIG. 1 , between the A/D conversion unit 15 and the digital signal processing unit 20, three data selection (DS: Data Selector) circuits 16a to 16c are provided. The first DS circuit 16 a is provided between the A/D converters C04 and C05 and the first and second DSPs 21 and 22 . The second DS circuit 16b is provided between the A/D converters C08 and C09 and the second and third DSPs 22 and 23 . The third DS circuit 16c is provided between the A/D converters C12 and C13 and the third and fourth DSPs 23 and 24 .

The data signals from the A/D converters C01, C02, C03, and the DS circuit 16a are respectively input to the first to fourth ports (ports) of the input FIFO 21a of the first DSP21. The data signals from the DS circuit 16a, the A/D converters C06, C06, and the DS circuit 16b are respectively input to the first to fourth ports of the input FIFO 22a of the second DSP22.

The data signals from the DS circuit 16b, the A/D converters C10 and C11, and the DS circuit 16c are respectively input to the first to fourth ports of the input FIFO23a of the third DSP23. Data signals from the DS circuit 16c and the A/D converters C14, C15, and C16 are input to the first to fourth ports of the input FIFO 24a of the fourth DSP 24, respectively.

Here, each of the DS circuits 16a to 16c outputs the data signal input from the partial charge transfer unit via the A/D converter to the corresponding DSP. Also, as described below, a predetermined data signal is held, and the held data signal is output to another DSP as a correction data signal for data correction performed in the DSP.

The DSPs 21 to 24 perform necessary signal processing such as data correction on the data signals input from the corresponding A/D converters C01 to C16 and DS circuits 16a to 16c, respectively. In this way, the data signals subjected to the signal processing are output to the output FIFOs 21 b to 24 b of the DSPs 21 to 24 as the final image data obtained in the imaging device 1A.

Next, data correction of image data performed in the imaging device having the configuration shown in FIG. 1 will be described.

FIG. 2 shows an example of data signals input to the input FIFOs 21 a to 24 a of the first to fourth DSPs 21 to 24 . Hereinafter, the number of cells of each of the partial charge transfer units T01 to T16 is n (n is an integer of 2 or greater). In addition, for the signal sequence of n data signals output by the partial charge transfer unit T01, the first output data signal is denoted as 01-1, the second output data signal is denoted as 01-2, ..., and the last output data Signals are denoted as 01-n. The same method is used for the signal strings output from the other partial charge transfer units T02 to T16.

In addition, the number n of data signals output by the partial charge transfer section corresponds to the number of vertical shift registers in the corresponding photodetection area in the photodetection section. For example, the data signal 01-1 output first from the partial charge transfer unit T01 corresponds to the charge output from the vertical shift register at the left end of the photodetection region R01. In addition, the last output data signal 01-n corresponds to the charge output by the vertical shift register at the right end. Furthermore, as described above, the signal trains output from the partial charge transfer units T01 to T16 are input to the DSPs 21 to 24 as digitized data signals through the corresponding A/D converters C01 to C16.

In this embodiment, the second DSP 22 not only processes the signal strings from the partial charge transfer units T05 to T08, but also processes the partial charges from the different DSP 21 adjacent to its output side. Predetermined signals included in other signal trains of the transmission unit T04 are input as calibration data signals. Similarly, for the third and fourth DSPs 23 and 24, in addition to the signal trains from the partial charge transfer units T09 to T12 and T13 to T16 as processing targets, signals from the partial charge transfer units T08 and T12 are also included in other signal trains. A predetermined data signal is input as a calibration data signal.

Specifically, the first port of the first DSP 21 receives the signal sequence 01-1 to 01-n from the partial charge transfer unit T01. In addition, signal sequences 02-1 to 02-n from the partial charge transfer unit T02 are input to the second port. In addition, signal sequences 03-1 to 03-n from the partial charge transfer unit T03 are input to the third port. In addition, to the fourth port, the signal strings 04-1 to 04-n from the partial charge transfer unit T04 are input through the DS circuit. Here, in the first DSP 21 , there is no DSP adjacent to the output terminal side, and therefore, no correction data signal to be processed by a different DSP is input.

In the first port of the second DSP 22, the signal columns 05-1 to 05-n from the partial charge transfer part T05 are input through the DS circuit 16a, and the last output data in the signal sequence from the partial charge transfer part T04 The signal 04-n is input as a calibration data signal. In addition, signal sequences 06-1 to 06-n from the partial charge transfer unit T06 are input to the second port. In addition, the signal sequence 07-1 to 07-n from the partial charge transfer unit T07 is input to the third port. To the fourth port, the signal strings 08-1 to 08-n from the partial charge transfer unit T08 are input through the DS circuit 16b.

In the first port of the 3DSP23, the signal columns 09-1 to 09-n from the partial charge transfer part T09 are input through the DS circuit 16b, and the last output data in the signal sequence from the partial charge transfer part T08 The signal 08-n is input as a calibration data signal. In addition, signal sequences 10-1 to 10-n from the partial charge transfer unit T10 are input to the second port. Also, to the third port, the signal trains 11-1 to 11-n from the partial charge transfer unit T11 are input. To the fourth port, the signal trains 12-1 to 12-n from the partial charge transfer unit T12 are input through the DS circuit 16c.

In the first port of the fourth DSP 24, the signal columns 13-1 to 13-n from the partial charge transfer section T13 are input through the DS circuit 16c, and the last output data in the signal column from the partial charge transfer section T12 The signal 12-n is input as a calibration data signal. In addition, signal sequences 14-1 to 14-n from the partial charge transfer unit T14 are input to the second port. In addition, signal sequences 15-1 to 15-n from the partial charge transfer unit T15 are input to the third port. To the fourth port, the signal trains 16-1 to 16-n from the partial charge transfer unit T16 are input.

Among the data signals from the partial charge transfer sections T01 to T16 that are respectively input to the DSPs 21 to 24, among the data signals 01-1 to 16-1 that are first output from the partial charge transfer sections marked with oblique lines in FIG. 2 , output abnormalities such as output attenuation may occur. Such output abnormality is caused by, for example, pixel attenuation (pixel droop) caused by the structure of the device itself, or insufficient frequency band of the amplifier. For these reasons, in the imaging device 1A shown in FIG. 1 , the DSPs 21 to 24 perform data correction using these head data signals 01-1 to 16-1 as correction targets.

FIG. 3 is a diagram showing data signals after signal processing output from the output FIFOs 21 b to 24 b of the first to fourth DSPs 21 to 24 . First, in the first DSP21, for the data signal 01-1 from the partial charge transfer part T01, since there is no partial charge transfer part adjacent to the output terminal side, the second output data signal 01 in the signal column is -2 is used as a data signal for calibration. Then, data correction is performed in which the data signal 01-1 to be corrected is replaced by the data signal 01-2 for correction (01-1=01-2).

In addition, for the data signal 02-1 from the partial charge transfer unit T02, the data signal 02-2 output second in the signal row is used as the first correction data signal, and the data signal 02-2 from the adjacent output terminal side is used as the first correction data signal. The last output data signal 01-n of the signal sequence of the partial charge transfer unit T01 is used as the second correction data signal. Then, the data signal 02-1 to be corrected is replaced by the average of the first calibration data signal 02-2 and the second calibration data signal 01-n (02-1=(01-n+02-2)/2 ) data correction.

Data correction for data signals 03-1 and 04-1 in the first DSP21, data correction for data signals 05-1 to 08-1 in the second DSP22, data correction for data signals 09-1 to 08-1 in the third DSP23 The data correction of 12-1 and the data correction of the data signals 13-1 to 16-1 in the fourth DSP 24 are also performed in the same way as the data correction of the data signal 02-1 in the first DSP 21. Through the above data correction, the data signals shown in FIG. 3 are output from the output FIFOs 21 b to 24 b of the DSPs 21 to 24 as the final image data obtained by the imaging device 1A.

Effects of the imaging device according to the above-described embodiment will be described.

In the imaging device 1A shown in FIG. 1 , a multi-tap structure is adopted, that is, the transfer unit 12 that inputs charges from a plurality of vertical shift registers constituting the photodetection unit 10 in parallel is divided into a plurality of partial charge transfer units T01. ~ T16, respectively output charge from their respective output terminals. Accordingly, it is possible to quickly read the charge generated in each pixel in the photodetection regions R01 to R16 of the photodetector 10 .

In addition, in this multi-tap structure, for output abnormalities occurring in the first data signals 01-1 to 16-1 outputted from the output terminals located at the respective left ends of the plurality of partial charge transfer parts T01 to T16, a setting using In the digital signal processing unit 20 incorporating a DSP, data correction is performed in DSPs 21 to 24 of the digital signal processing unit 20 for output abnormalities of data signals. In this way, instead of performing data correction by software on the obtained image data after being output from the imaging device, by performing data correction in the DSPs 21 to 24 provided in the imaging device 1A, the final image data after data correction can be obtained simply and quickly.

Here, it is generally preferable to have a plurality of pixels arranged in an array for the configuration of the photodetection unit 10 used for an incident light image. That is to say, if the charge transfer section 12 as a horizontal shift register uses a multi-tap structure divided into a plurality of partial charge transfer sections (partial shift registers), for the structure of the photodetection section 10, regardless of whether the pixel is divided into two Whether it is an image sensor arranged in a one-dimensional array or a line sensor arranged in a one-dimensional array, the above-mentioned data correction method can be applied through the DSP. In addition, it may be an image sensor that performs TDI line sensor operation. In addition, any of the FT method, the FFT method, the IT method, and the FIT method may be used as the charge transfer method in the photodetection unit 10 .

In addition, the partial charge transfer section in the charge transfer section 12 is not limited to the above-mentioned structure having 16 partial charge transfer sections T01 - T16 . In general, the charge transfer section is preferably divided into N (N is an integer of 2 or more) partial charge transfer sections in the array direction. In addition, the number of cells in each partial charge transfer section can be appropriately set.

In the data correction method for the data signal implemented in the digital signal processing unit 20, it is preferable to include in the data signal which is output first in the signal sequence from the partial charge transfer unit and which is to be corrected in the DSP. Other predetermined data signals of the signal sequence are included in the signal sequence (other signal sequence) from the partial charge transfer section adjacent to the output end side (left side in FIG. 1 ) of the partial charge transfer section as the first correction data signal. ) is used as the second data signal for correction, and data correction is performed using the first data signal for correction and the second data signal for correction. According to this data correction method, the influence of output abnormality occurring in the data signal can be effectively reduced, and image data with good image quality can be obtained.

In this case, for the data signal to be corrected and the first data signal for correction selected from the signal sequence from the same partial charge transfer section, and the first correction data signal selected from the signal sequence from other adjacent partial charge transfer sections As for the selected second data signal for correction, one or more data signals can be selected individually depending on the specific data correction method. In addition, more generally, data correction may be performed using a plurality of correction data signals including at least one of the first correction data signal and the second correction data signal. With this data correction method, it is also possible to effectively reduce the influence of output abnormalities occurring in the data signal.

In particular, as shown in FIG. 2 and FIG. 3 , in the above-mentioned embodiment, for data signals 02-1 to 16-1 other than the data signal of 01-1 among the data signals to be corrected, DSPs 21 to 24 perform the following data correction: The second output data signals 02-2 to 16-2 in the signal column are used as the first calibration data signals, and the last output data signals 01-n to 15-n in the other signal columns are used as the second For the calibration data signal, replace the calibration target data signals 02-1 to 16-1 with the average of the first and second calibration data signals (for example: 02-1=(01-n+02-2)/2) . According to this replacement method, appropriate correction can be performed by performing simple correction processing on the data signal to be corrected.

In addition, as described above, for the data signal 01-1 to be corrected outputted by the partial charge transfer unit T01 that has no adjacent partial charge transfer unit on the output side, it is preferable to output the second data in the signal column. The signal 01-2 is used as a data signal for correction, and data correction is performed in which the data signal to be corrected (01-1=01-2) is replaced with the data signal for correction. Alternatively, data correction may be performed by using a plurality of data signals in the signal sequence as correction data signals.

Generally speaking, various methods can be used for the above-mentioned specific data correction method for the data signal. FIG. 4 is a schematic diagram of an example of a data signal output from a charge transfer unit. In this figure, the horizontal axis shows the number of the data signal along the X-axis direction (the position corresponding to the cell in the charge transfer section 12), and the vertical axis shows the data signal intensity I(x). For example, when the serial number of the data signal to be corrected is x=02-1, x-1 corresponds to the data signal 01-n, and X+1 corresponds to the data signal 02-2.

As a correction method for the data signal I(x), there is a method of taking the data signals of the preceding and following points in FIG. 3 as the first and second correction data signals as described above, and taking the sum and average thereof.

I(x)={I(x-1)+I(x+1)}/2

In addition, as another correction method, there is a method of taking the data signals of two points before and after each as the first and second correction data signals, and taking a weighted average of them using the coefficients a and b.

I(x)={(aI(x-2)+bI(x-1))+

(bI(x+2)+aI(x+2))}

/{2*(a+b)}

In addition, various setting methods such as using data signals of three points before and after each for data correction are possible.

Alternatively, it is also possible to use a correction method such as using a data signal I(x) to be corrected and a plurality of data signals I(x+1) and I(x+2) included in the same signal sequence as data signals for correction, Estimate the data signal I(x) from its slope

I(x)=I(x+1)

-{I(x+2)-I(x+1)}

In addition, for the method of calculating the corrected data signal using a plurality of data signals for correction, various methods other than the above-mentioned summing and averaging method, such as the least square method and spline interpolation (spline interpolation) can be used. law etc.

In addition, the digital signal processing unit 20 may be configured to perform shading correction (see, for example, Patent Document 3) in addition to the above-mentioned data correction. That is, in the imaging device 1A, there may be image non-uniformity caused by variations in sensitivity of the photodetection unit 10 itself, variations in luminance from the optical system, and the like. In this case, the shading correction is performed on all the data signals included in the data strings from the charge transfer unit 12 including the partial charge transfer units T01 to T16, thereby further improving the image obtained from the imaging device 1A. The quality of the data.

FIG. 5 is a diagram showing an example of image data used for shading correction of a data signal. In the shading correction, light is incident on the photodetector 10 in a general incident manner, and the distribution of the data signal intensity I(x) as shown in FIG. 5 is obtained. Next, the average value IO of the intensity is obtained, and the correction coefficient CC(x) is calculated by CC(x)=IO/I(x). The shading-corrected output data signal OUT(x) is obtained from the input data signal IN(x) by OUT(x)=IN(x)×CC(x).

For example, if the number of cells of the partial charge transfer unit is 256, 256 correction coefficients CC(0) to CC(255) are prepared in the DSP of the digital signal processing unit 20 for one partial charge transfer unit. At this time, if a correction coefficient is 16bit, the data volume is 16bit×256=512Bytes. Furthermore, the example shown here is an example in which shading correction is performed only in the x-axis direction for the charge from the photodetection unit 10, but two-dimensional shading correction may be performed on the x-axis and y-axis. In this case, the correction coefficient is represented by a two-dimensional array coefficient CC(x, y).

In the imaging device 1A shown in FIG. 1 , the digital signal processing unit 20 for performing signal processing such as data correction on the data signal output by the charge transfer unit 12 is composed of four parts respectively corresponding to the four partial charge transfer units. DSP21 ~ 24 form. In this way, by performing data correction using a plurality of DSPs, the time required for data correction in the imaging device 1A can be shortened. In this case, in general, the digital signal processing unit can be composed of M (M is an integer of 2 or more) DSPs to which signal trains from one or more partial charge transfer units are respectively input.

In addition, as described above for the second to fourth DSPs 22 to 24 in FIG. 2, in such a structure utilizing a plurality of DSPs, it is preferable for the DSPs to be processed as A predetermined data signal included in another signal sequence from a partial charge transfer unit of a different DSP adjacent to the DSP output terminal is input as a second correction data signal. In this manner, even when a plurality of DSPs are used for data correction of the data signal output from the charge transfer unit 12, appropriate data correction can be performed for each abnormality in the output of the data signal.

A specific configuration example of the imaging device 1A shown in FIG. 1 will be described.

FIG. 6 is a plan view of a configuration of a photodetection unit and a charge transfer unit used in the imaging device shown in FIG. 1 . In this configuration example, the photodetector 10 has 4096 cells in the horizontal direction (x-axis direction) and 70 lines (1ine) in the vertical direction (y-axis direction) to form a two-dimensional array structure of 4096×70 pixels. In addition, in the vertical direction, a group of three up and down respectively serves as a dummy (dummy) area 10b, and the inner 64 lines serve as an area 10a for light detection.

Corresponding to the photodetection unit 10, the charge transfer unit 12 is composed of 16 partial charge transfer units T01 to T16 of 256 cells each. In addition, on the side of each partial charge output terminal, a dummy cell (dummy cell) is provided as needed. As such a configuration, for example, in addition to the above-mentioned 256 active cells, four analog cells are connected to the output side, and a total of 260 cells are used as partial charge transfer units. In addition, in this configuration example, a storage unit 11 with 4096 cells and 70 lines is provided between the photodetection unit 10 and the charge transfer unit 12 similarly to the photodetection unit 10 .

Moreover, in the structure of FIG. 6 , above the photodetection unit 10, a storage unit 13 having the same structure as the storage unit 11 and the charge transfer unit 12 below, and a charge transfer unit having partial charge transfer units T17 to T23 are provided. 14. Thus, in this configuration example, the charge transfer direction can be set in either the upper direction or the lower direction in the vertical shift register of the photodetection unit 10 .

FIG. 7 is a block diagram showing an example of the configuration of a digital signal processor (DSP) used in the imaging device shown in FIG. 1 . The DSP 80 has a DSP core 80a for performing signal processing such as data correction, a program buffer 80b, a data buffer 80c, a memory 80d, and an EDMA (Enhanced Direct Memory Access) controller 80e. In addition, VP (Video Port: Video Port) 1, VP1, VP2, HPI (Host Port InterFace: Host Port Interface), and EMIF (External Memory InterFace: Extended Memory Interface) are connected to the EDMA controller 80e.

Among them, in the DSPs 21 to 24 of the digital signal processing unit 20 shown in FIG. 1 , the video ports of VP0 to VP2 are used as input FIFOs and output FIFOs. As such a configuration, for example, a configuration in which VP0 and VP1 are used as input FIFOs and VP2 is used as an output FIFO can be used.

Next, a microscope device using the imaging device shown in FIG. 1 according to the present invention will be described.

Fig. 8 is a block diagram showing the configuration of an embodiment of the microscope device of the present invention. The microscope device 3 is used to obtain a one-dimensional or two-dimensional image of the sample S. Here, the direction of the main optical axis in the microscope 3 is defined as the z-axis direction, and the direction of arrangement of pixels perpendicular to the main optical axis in the photodetection section of the imaging device 1A is defined as the x-axis direction and the y-axis direction (see FIG. 1). In addition, the sample S to be observed in the microscope device 3 may be, for example, a biological sample placed on the sample stage 30 .

The sample stage 30 is an XY stage movable in the x direction and the y direction (horizontal direction), and by driving the XY stage in the xy plane, the observation position of the sample S in the microscope 3 can be in the direction perpendicular to the main optical axis. Set and change the direction. In addition, the sample stage 30 is controlled by an XY stage drive unit 35 .

For the sample S on the sample stage 30 , in order to guide the light image of the sample S, a light guiding optical system 40 is provided above the stage 30 . The light guiding optical system 40 is composed of an objective lens 41 for making light from the sample S incident, and optical elements necessary for guiding and focusing the light image of the sample S. FIG. In addition, an imaging device 1A having the configuration of FIG. 1 is installed at a predetermined position on the optical path where the optical image of the sample S is introduced by the light guiding optical system 40 . This imaging device 1A is an imaging unit that acquires an image generated by an optical image of a sample S. As shown in FIG. In addition, these light guide optical systems 40 and the imaging device 1A are integrally fixed in a state where the main optical axis and the distance between the optical system 40 and the imaging device 1A are adjusted.

Furthermore, an optical system driving unit 45 is provided for these light guiding optical systems 40 and the imaging device 1A. The optical system drive unit 45 can be composed of, for example, a stepping motor or a piezoelectric actuator (piezo actuator), and drives the optical system 40 and the imaging device 1A fixed relative to the optical system 40 in the direction of the main optical axis. move in the z-axis direction. In such a configuration, the optical system driving unit 45 has a function of adjusting or changing the focus position of the sample S in the imaging device 1A and the light guide optical system in the direction of the optical axis. The optical system drive unit 45 is used to control the focus for obtaining an image of the sample S. As shown in FIG.

A control device 50 is provided for the sample stage 30, the light guide optical system 40, and the imaging device 1A. The control device 50 is a control unit that controls the acquisition of an image, and controls the acquisition of an image of the sample S by the imaging device 1A. In addition, it is preferable that the control device 50 has a focus control function, and performs acquisition of focus information including focus point positions for acquiring an image of the sample S, and performs focus control when acquiring an image of the sample S.

The control device 50 can be constituted by, for example, a computer including a CPU and necessary storage means such as a memory and a hard disk. In addition, an input device 51 and a display device 52 are connected to the control device 50 . The input device 51 is composed of, for example, a keyboard and a mouse connected to a computer, and is used to input necessary information and instructions when operating the microscope to obtain an image of the sample S. In addition, the display device 52 may be composed of, for example, a CRT display, a liquid crystal display, etc. connected to a computer, for displaying necessary information related to obtaining an image in the present microscope device 3 .

In the microscope 3 shown in FIG. 8 , the imaging device 1A shown in FIG. 1 is used. In this imaging device 1A, the digital signal processing unit 20 performs data correction for an output abnormality of the data signal for the data signal output first among the signal trains from the respective output terminals of the plurality of partial charge transfer units T01 to T16 . Thus, when observing the sample S, image data with good image quality can be obtained.

The structure of the imaging device of the present invention will be further described.

FIG. 9 is a block diagram showing the configuration of a second embodiment of the imaging device according to the present invention. The imaging device 1B of the present embodiment includes a photodetection unit 10 , a charge transfer unit 12 , an A/D conversion unit 15 , and a digital signal processing unit 20 . Among them, the configurations of the photodetection unit 10 , the charge transfer unit 12 and the A/D conversion unit 15 are the same as those in the first embodiment. In addition, the data correction method and the like at the time of signal processing performed by the digital signal processing unit 20 are also the same as those in the first embodiment described above.

The data signals output from the A/D converters C01 to C16 are input to a digital signal processing unit 20 for performing predetermined data processing on the data signals. In this embodiment, the digital signal processing unit 20 is composed of four DSPs 25 to 28 . In addition, in FIG. 9 , a RAM 17 is provided between the A/D conversion unit 15 and the digital signal processing unit 20 . The RAM 17 is a data storage unit that temporarily stores the data signal output from the A/D converter 15 . Signals from the A/D conversion unit 15 are temporarily stored in the RAM 17 . In this way, the necessary data signals are respectively input to the corresponding DSPs 25-28 in sequence. In this embodiment, data is input to and output from the memory 80d in the DSPs 25 to 28 through the EMIF in each DSP (see FIG. 7 ).

In this way, in the structure in which a data storage device such as RAM 17 is provided between the A/D converter 15 and the digital signal processing unit 20, the specific data correction method for the digital signal processing unit 20 and the specific structure Therefore, necessary data signals can be appropriately input from the A/D conversion unit 15 to the respective P25 to P28 constituting the data signal processing unit 20 . In particular, this configuration is advantageous in that the input configuration of the digital signal to the DSP of the digital signal processing section 20 has a large degree of freedom. For example, in the imaging device 1B having the configuration shown in FIG. 9 , various input configurations other than the data signal input configuration shown in FIG. 2 related to FIG. 1 can be used.

FIG. 10 is a diagram showing another example of data signals input to the memory through the EMIFs of the first to fourth DSPs 25 to 28. In this example, the same as the example in FIG. 2, for DSP26, 27, 28, in addition to the signal trains from the partial charge transfer parts T05~T08, T09~T12, T13~T16 as the processing target, will be from each The data signals 04-n, 08-n, and 12-n input last in the signal sequence T08, T12 of the partial charge transfer unit T04 are input as correction data signals. Furthermore, in DSPs 26 to 28, the data signal output from the partial charge transfer section last is inputted so as to be added to the first part of the signal sequence from the next partial charge transfer section. In this way, in the structure shown in FIG. 9, there are various setting modes for the data input structure of the DSP.

FIG. 11 is a block diagram showing the configuration of a third embodiment of the imaging device according to the present invention. The imaging device 1C of the present embodiment includes a photodetection unit 10 , a charge transfer unit 12 , an A/D conversion unit 15 , and a digital signal processing unit 20 . Among them, the configurations of the photodetection unit 10 , the charge transfer unit 12 and the A/D conversion unit 15 are the same as those in the first embodiment. In addition, the data correction method in the signal processing performed by the digital signal processing unit 20 is also the same as that in the first embodiment described above.

The data signals output from the A/D converters C01 to C16 are input to a digital signal processing unit 20 for performing predetermined signal processing on the data signals. In the present embodiment, the digital signal processing unit 20 is composed of one DSP 29 . In addition, in FIG. 1 , a buffer 18 is provided between the A/D conversion unit 15 and the digital signal processing unit 20 . The buffer 18 is a data storage unit that temporarily stores the data signal output from the A/D conversion unit 15 . The data signal from the A/D conversion unit 15 is temporarily stored in the buffer 18 . Next, the necessary signals are input to DSP29 in sequence.

In this way, when the digital signal processing unit 20 is constituted by one DSP 29 , the configuration of the imaging device 1C including the digital signal processing unit 20 can be simplified. Moreover, also in this structure, the input structure of the data signal input from the A/D conversion part 15 to the DSP of the internal digital signal processing part 20 is simplified. In addition, it is preferable to set the number of DSPs constituting the digital signal processing unit 20 in consideration of factors such as DSP processing capability and processing speed.

FIG. 12 is a schematic diagram of an example of a data signal input to the memory through the EMIF of the DSP29. In the configuration shown in FIG. 11 , data signals from all the partial charge transfer sections T01 to T16 constituting the charge transfer section 12 are input to the same DSP 29 . Therefore, also in this example, the signal strings from the first to sixteenth partial charge transfer units T01 to T16 are input to the DSP 29 as they are.

The imaging device and the microscope device using the imaging device according to the present invention are not limited to the above-described embodiments and structural examples, and various modifications are possible. For example, in FIG. 8 , the imaging device 1A shown in FIG. 1 is used as the imaging device in the microscope device 3 , but of course, imaging devices in other embodiments may also be used.

In addition, in the above-mentioned embodiment, the data correction is performed on the data signals output from a single photodetector, for example, data correction may be performed on the data signals from three photodetectors corresponding to RGB. Structure. In this case, data correction may be performed on all data signals from one of the three photodetectors first, and then data correction may be sequentially performed on the other two photodetectors. Alternatively, a configuration may be employed in which three data signals from corresponding pixels in the three photodetection sections are used as RGB signal groups, and data correction is performed sequentially on these signal groups.

Here, the imaging device has (1) a photodetection unit having a plurality of pixels arranged in an array and outputting charges generated in the pixels corresponding to the amount of incident light, and (2) a charge transfer unit arranged along the plurality of pixels with respect to the photodetection unit. set in one arrangement direction, and have N (N is an integer of 2 or more) partial charge transfer sections divided in the arrangement direction, (3) A/D conversion unit, which will transfer the charge according to the photodetection unit via the charge transfer unit The signal generated by the output charge is converted into a digital data signal and (4) a digital signal processing unit, which performs signal processing on the data signal output from the A/D conversion unit. (5) The partial charge transfer section preferably transfers the charge from the pixels located in the predetermined photodetection area of the photodetection unit to the output direction and outputs it from the output terminal, and at the same time, the digital signal processing unit transfers the charge from the partial charge transfer section The data signal output first in the signal sequence is used as the correction object, and other predetermined signals included in the signal sequence are used as the first correction data signal, which is sent from the output terminal side adjacent to the partial charge transfer part. The predetermined data signal included in other signal columns of the partial charge transfer section is used as the second correction data signal, and a plurality of correction data signals including at least one of the first correction data signal and the second correction data signal are used. The data signal undergoes data correction.

In addition, the digital signal processing unit may include M (M is an integer of 2 or greater) digital signal processing devices to which signal trains from one or more partial charge transfer units are respectively input.

In this case, to the digital signal processing device, it is preferable to input the signal train from the one or more partial charge transfer parts as the processing target, and also input the signal from the digital signal processing device adjacent to its output terminal and as a different digital signal processing device. The second correction data signal included in the other signal sequence of the partial charge transfer unit to be processed. Accordingly, even when a plurality of digital signal processing devices are used for data correction of the data signal, it is possible to perform appropriate data correction for an output abnormality of the data signal in each device.

Alternatively, the data signal processing unit may include one digital signal processing device that receives signal trains from the N partial charge transfer units.

In addition, a data storage unit may also be provided between the A/D conversion unit and the digital signal processing unit to temporarily store the data signal output by the A/D conversion unit. In this case, corresponding to the specific structure of the digital signal processing unit, necessary signals may be appropriately input from the A/D conversion unit to one or more digital signal processing devices constituting the digital signal processing unit.

In addition, it is preferable that the digital signal processing unit performs shading correction in which all data signals included in the signal sequence from the partial charge transfer section are corrected in addition to the above-mentioned data correction. Thereby, the image quality of the obtained image data can be improved.

Regarding the specific correction method for data correction of the data signal, for example, the following data correction can be used: the data signal output second in the signal column is used as the first correction data signal, and the data signal output last in the other signal columns is used as the first correction data signal. The data signal is used as the second data signal for correction, and the data signal to be corrected is replaced by the average of the first data signal for correction and the second data signal for correction.

In addition, when the digital signal processing unit does not have a partial charge transfer unit adjacent to the output end side of the partial charge transfer unit that outputs the data signal to be corrected, it is preferable to perform data correction such that the first The two output data signals are used as data signals for correction, and the data signal to be corrected is replaced with the data signal for correction. Alternatively, various methods other than these methods may be used as the data correction method for the data signal.

In addition, it is preferable that the microscope has: the above-mentioned imaging device that acquires the optical image formed by the sample as the observation object, the light guide optical system, including the objective lens that makes the light from the sample incident, and guides the optical image of the sample into the imaging device, and image acquisition. The control unit controls acquisition of an image of the sample formed by the imaging device.

Possibility of industrial use

The present invention can be used as an imaging device capable of effectively reducing the influence of an output abnormality occurring in a data signal, and a microscope using the imaging device.

Claims (9)

1. A camera, characterized in that it has: a photodetection unit having a plurality of pixels arranged in an array, and outputting charges generated in the pixels corresponding to the amount of incident light; a charge transfer unit disposed along an arrangement direction of the plurality of pixels relative to the photodetection unit, and having a partial charge transfer unit divided into N in the arrangement direction, wherein N is an integer of 2 or greater; an A/D conversion unit converting the signal generated according to the charge output from the photodetection unit via the charge transfer unit into a digital data signal; a digital signal processing unit that performs signal processing on the data signal output from the A/D conversion unit; The partial charge transfer unit transfers the charges from the pixels located in the predetermined photodetection area of the photodetection unit to the output direction and outputs them from the output terminal, and at the same time, the digital signal processing unit transfers the charges from the part The data signal output first in the signal sequence of the charge transfer unit is used as the calibration target, and other predetermined data signals included in the signal sequence are used as the first data signal for correction, and the data signal from the output terminal of the part of the charge transfer unit is used as the first correction data signal. The predetermined data signal included in the other signal column of the partial charge transfer part adjacent to the side is used as the second correction data signal, and the data signal including the first correction data signal and the second correction data signal is used. At least one of the plurality of calibration data signals is used for data calibration. 2. The imaging device according to claim 1, wherein: The digital signal processing unit has M digital signal processing devices respectively inputting signal trains from one or more partial charge transfer sections, where M is an integer of 2 or more. 3. The imaging device according to claim 2, wherein: To the digital signal processing device, in addition to the signal trains from the one or more partial charge transfer units to be processed, signals from a different digital signal processing device adjacent to its output terminal side are also input. The second correction data signal included in the other signal sequence of the partial charge transfer unit to be processed. 4. The imaging device according to claim 1, wherein: The digital signal processing unit has a digital signal processing device that inputs signal trains from the N partial charge transfer sections. 5. The imaging device according to any one of claims 1 to 4, wherein: Between the A/D conversion unit and the digital signal processing unit, a data storage unit for temporarily storing the data signal output from the A/D conversion unit is provided. 6. The imaging device according to any one of claims 1 to 5, wherein: The digital signal processing unit performs, in addition to the data correction, shading correction in which all data signals included in the signal sequence from the partial charge transfer unit are corrected. 7. The imaging device according to any one of claims 1 to 6, wherein: The digital signal processing unit performs the data correction on the data signal to be corrected as follows, using the second output data signal in the signal sequence as the first correction data signal. The last output data signal in the other signal sequence is used as the second calibration data signal, and the average value of the first calibration data signal and the second calibration data signal is used to replace the calibration target. data signal. 8. The imaging device according to any one of claims 1 to 7, wherein: The digital signal processing unit performs the data correction as follows when there is no partial charge transfer section adjacent to an output end side of the partial charge transfer section that outputs the data signal to be corrected. The second output data signal in the signal sequence is used as the correction data signal, and the correction target data signal is replaced by the correction data signal. 9. A microscope device, characterized in that it has: The imaging device according to any one of claims 1 to 8 for obtaining an image formed from an optical image of a sample to be observed, a light guiding optical system comprising an objective lens that makes light from the sample incident, and guides a light image of the sample into the imaging device, and The image acquisition control unit controls acquisition of an image of the sample formed by the imaging device.

Images