WO2012002787A1

WO2012002787A1 - Method and device for multi-spectral imaging by means of a digital rgb sensor

Info

Publication number: WO2012002787A1
Application number: PCT/LV2011/000003
Authority: WO
Inventors: Jānis SPĪGULIS; Dainis Jakovels; Uldis RUBĪNS
Original assignee: Institute of Solid State Physics University of Latvia
Current assignee: Institute of Solid State Physics University of Latvia
Priority date: 2010-06-29
Filing date: 2011-03-07
Publication date: 2012-01-05
Anticipated expiration: 2012-12-29
Also published as: LV14207A; EP2589225A1; LV14207B

Abstract

The invention relates to methods of digital colour image processing, in particular - to selection of images corresponding to more than three different spectral bands from a single RGB data set. In the proposed method of multi-spectral imaging, the object is illuminated simultaneously at several spectral bands, and the values of R_i-, G_i- un B_i-signals detected at each i-pixel of the image are identified. Further they are compared mutually and with externally determined signal discrimination level S that allows registering only one or two of the R-, G- and/or B-bands within the spectral sensitivity range of the sensor. In order to increase the number of available spectral images, the S-values are continuously variable up to the highest of all possible signal values of the R, G or B band, with condition that linear photo-response of the RGB sensor is ensured. Depending on the S-value, two situations are analyzed - if two colour band signals are registered simultaneously (i.e. B and G, G and R or B and R), or if the signals are registered only at one colour band - and after logical analysis the spectral interval of the pixel-registered radiation is identified. Further each spectral image is formed from the pixels or pixel groups that correspond to a particular selected spectral range. A device for multi-spectral imaging to implement this method comprises a multi-spectral light source, objective-equipped digital RGB sensor, RGB data set storage device, convertor for converting the RGB data into a set of spectral intensities in accordance with the selected signal discrimination level, selector of images for selecting the images related to each particular spectral band, and the output device, e.g. PC-monitor.

Description

Method and device for multi-spectral imaging by means of a digital RGB sensor

Technical field

Invention relates to methods of digital colour image processing, in particular - to selection of images corresponding to more than three different spectral bands from a single-shot RGB data set.

Background art

Analysis of multi-spectral image sets allows identifying the object zones with different spectral absorption or emission features and can be helpful in artwork expertise, satellite surveillance of Earth surface, clinical diagnostics of skin, etc. Digital sensors - two- dimensional photo matrices equipped with external spectral filters, e.g. a rotating disk with a set of different colour filters (E.C. Ruvolo et al, Proc. SPIE, Vol. 7548, 75480A, 2010), or spectrally tuneable acousto-optical or liquid crystal filters (http://www.usgs. gov/science/science.php?term=765) - are often used for obtaining the multi-spectral images.

Digital RGB sensors comprise integrated three colour (blue, green and red) spectral filters; they also can be used for multi-spectral imaging in various combinations with external spectral filters (US 7612822 B2, US 2009290124 Al , JP 2008136251 A). Another way of getting multi-spectral images is sequential illumination of the object by several light sources, each emitting at different spectral region (e.g. LEDs of different colours - WO 2008093988 Al) and taking an image at each spectral band of the illumination. The mentioned methods are useful, but their drawback is the necessity to take several consecutive images of the same object at different spectral bands. First, the process is time- consuming. Second, the object properties can change during this process, e.g. it can move - then either mistaken data are obtained, or additional procedures for image stabilisation should be included in the processing algorithm, so extending the processing time. Third, huge amount of unnecessary information on image details is being stored. Fourth, design of such equipment is complicated and expensive, since additionally to the RGB sensor with objective it comprises external filtering devices or sets of spectrally different illumination sources. Such drawbacks are avoided if methodology of single-shot RGB image multi-spectral analysis is applied. The image taken by a digital RGB sensor (e.g. CCD or CMOS, http:/ broadcastengineering.com/hdtv/ccd-cmos/) is transformed into colour format by determining first the numerical values Rj, G; un B; (i - the pixel number) of signals detected at this particular pixel in the red (R), green (G) and blue (B) spectral bands, and then adjusting for each pixel the colour that corresponds to the specific RiGjBj combination. Spectral sensitivities of the R-, G-, and B-channels are determined by absorption properties of the filtering coatings and of the photo-detector material, e.g. silicon. The specific spectral sensitivity curves of the three channels one can find in specifications of serially produced RGB sensors; they can also be measured experimentally.

It is possible to extract three spectrally selective sub-images for the R-, G- and B-channels, related to the red, green and blue light emission or absorption by the object, from the data set of a single-shot RGB colour image, and to perform multi-spectral analysis by specific manipulations with these sub-images, e.g. by division, extraction, summing, etc. (D. Kapsokalyvas et al, Proc. SPIE, Vol. 7548, 754808, 2010). The logical analysis of a single RGB data set additionally allows separating six narrower spectral bands if a certain discrimination level of the sensor output signals is applied, with condition that only two or one out of the three R, G, B values are registered at any working wavelength (J. Spigulis et al, Proc. SPIE, Vol. 7557, 75570M, 2010); this technical solution is the closest to the proposed invention.

Goal of this invention is to increase efficiency of multi-spectral imaging by increasing the number of images extracted from a single-shot RGB colour image data set and related to different spectral bands.

Disclosure of the invention To reach the goal, a new digital RGB sensor's output data processing method and a device implementing this method are proposed.

The method is characterized in that: (i) the discrimination level of the RGB output signals is variable instead of being fixed. It opens possibility to select more than the six known spectral intervals (additionally to the R-, G-, and B-bands), so increasing the number of multi- spectral images extracted from a single-shot colour image data set; (ii) the spectral selection is additionally being performed by using the crossing points of two spectral sensitivity curves (B-G, G-R or B-R) that correspond to fixed wavelengths. New spectral intervals are selected in the vicinity of these crossing points, with the conditions:

| Bi - Gi | < a (1)

and/or

I Gi - Ri I < ¾ (2)

and/or

I B; - R; I < c (3), where a, b and c are controllably adjustable fixed numbers. Description of the figures

Fig. 1 presents curves of the R-, G- and B-channel relative spectral sensitivities for a typical RGB sensor, and the way how a variable signal discrimination level S is used for additional selection of the spectral images;

Fig. 2 illustrates how the crossing points of the curves representing relative spectral sensitivities of the R-, G- and B-channels are being applied for additional selection of the spectral images;

Fig. 3 shows the set-up scheme of the proposed device for obtaining of multi-spectral images.

The essence of the method (i) is being illustrated in Fig. 1. The R-, G- and B-channel relative spectral sensitivity curves, provided by manufacturer or measured experimentally, are exploited, and their amplitudes are normalized so that the highest value of the registered signals does not exceed the maximum output signal of any channel, e.g. the number 255 in 8-bit system. The objective diaphragm or illumination intensity is adjusted so that the sensor operates linearly, i.e. the numerical values of the R;, Gj and B, signals are proportional to the intensity of the detected Optical signals (for instance, in the range between 0 and 255). Then a single-shot colour image is taken with digital data set comprising the Rj, Gj and Bj signal values for all pixels of the image. These values are compared to the value of variable discrimination level S_d , and only those exceeding the S_<j- level are being used in further analysis.

Depending on the S_d-value, the following situations may happen:

a) signals from two colour channels (R&G, R&B or G&B) are registered simultaneously at the i-pixel;

b) signals from only one colour channel (R, G, or B) are registered at the i-pixel.

The situation (a) means that the spectral range of radiation detected by the i-pixel is in the region where two spectral sensitivity curves are overlapping - see Fig. l . For instance, simultaneous registration of some B; and Gj values at the lowest discrimination level So means that the spectral range of incident radiation lies between λ and λ₁₄ (the wavelength corresponding to the points 6 and 14). Depending on which of the two values is higher (Bj or Gj), narrower spectral intervals can be specified - either λ₆ ... λιο (if Bj > Gj), or λιο ... λ₁₄ (if B; < Gi). However, simultaneous registration of some other B; and G; values at somewhat higher discrimination level S_! leads to conclusion that the spectral interval of incident radiation in this case is even narrower, between g and λι₂ - more specifically, either Xg ... λιο (if Bj > Gj) or λιο ... λ₁₂ (if Bj < Gj). Similarly, simultaneous registration of two values R; and Gj at discrimination level Si indicates to some other spectral interval of the incident radiation, in particular λ₁₇ ... λ₁₈ (if R; < Gj) or λ_]8 ... λΐ9 (if Ri > Gj), and so on.

The situation (b) relates to one of two other options. First, the registered spectral interval may correspond to some specific regions of the B, G or R sensitivity curves - λ₄ ... λ₁₀ (for B-channel at S₃), λιο ... λ]₆ (for G-channel at S₃), or λι₈ ... λ₂₁ (for R-channel at S₂). By increasing the discrimination level, the registered spectral interval narrows - for instance, at level S₄ only radiation of spectral band λ₅ ... λ₇ can be registered in the B-channel, and only the interval λπ ..· λι₅ can be registered in the G-channel. If the discrimination level reaches the peak value of the B-, G- or R-band, appearance of signal in this particular band means that monochromatic radiation of wavelength that corresponds to the band peak wavelength has been recorded in the respective pixel. Second, registration of signal only at one of the three colour bands at lower discrimination levels leads to conclusion that the registered spectral interval is out of the overlapping zones of any two bands. For instance, the signal recorded only in the B-channel at the discrimination level Si indicates to the spectral interval λ₂ ... λ₈, while that exceptionally in the G-channel - to the interval λι₃ ... λι₇ and that exceptionally in the R-channel - to the interval λΐ9 ... λ₂₂.

Consequently, the proposed solution that the discrimination level of RGB output signals is flexibly variable instead of being fixed is opening more options to select different specific spectral intervals for the needs of multi-spectral imaging.

The essence of method (ii) is being illustrated in Fig. 2. The three possible crossing points of the RGB spectral sensitivity curves have three fixed wavelengths. If there are pixels of the RGB image with identical values G; = B;, R; = G, or Rj = Bj, then the registered radiation by these particular pixels is monochromatic, with the wavelength of the corresponding crossing point. It is proposed to create new spectral intervals for multi- spectral imaging in the vicinity of these three crossing points by limiting the absolute values of differences between the values of signals that have been detected in the overlapping channels as follows: | Bj - Gi | < a and/or | G, - R, | < b and/or | B; - R; | < c. Following to these conditions, the spectral interval λ₃ ... λ_b can be extracted in the vicinity of the B-G crossing point, the interval λ_ε ... λ_ί - in the vicinity of the G-R crossing point, and the interval λ_ς ... λ_ά - in the vicinity of the B-R crossing point. The spectral intervals can be narrowed or expanded by the respective decrease or increase of the a, b and c values. The set-up scheme of the device for implementation of one or both of the above-mentioned methods is illustrated in Fig.3. The device comprises:

a multi-spectral light source (A), e.g. a set of light emitting diodes (LEDs) and/or laser diodes (LDs) that illuminates the object to be imaged (C), e.g. skin surface; an objective-supplied digital RGB sensor (D) that converts the object image into digital format by providing a specific set of the Rj, Gj and Bj values to each pixel of the image, and stores the whole RGB data set in the storage device (E); a converter (F) that converts the data of the RGB set into a set of spectral intensities accordingly to the selected discrimination level which is determined by the discriminator (H);

an image selector (I) that selects a number of spectral images from the RGB data set accordingly to the chosen spectral intervals and performs multi- spectral analysis by appropriate software

an output device (J), e.g. monitor of PC, that collects and displays the obtained multi-spectral imaging information.

Further processing of multi-spectral images is possible by means of the known methods.

Claims

A method of multi- spectral imaging by means of a digital RGB sensor using a single-shot RGB colour image data set with illumination of the object at several spectral bands simultaneously and identifying the R,-, Gj- and Bj-signal values registered at each "i"-pixel of the image with further their comparison mutually and with externally determined discrimination level S that allows registering only one or two of the R-, G- and/or B-bands, characterized in that aiming at increase of the number of available spectral images, the S-value is being continuously variable up to the highest possible of the signal values detectable at the R, G or B bands, provided that linearity of the sensor photo-response is ensured and, depending on the S-value, two situations are analysed - when i-pixel detects signals at two of the bands: B and G, or G and R, or B and R simultaneously, or only at one of them, wherein the spectral interval of the incident radiation detected at the particular i-pixel is identified in result of analysis.

The method according to claim 1, characterized in that additionally the crossing point(s) of two spectral sensitivity curves: B-G, or G-R, or B-R at fixed wavelength is/are being used for spectral selection, and/or new spectral intervals are extracted at the vicinity of these crossing points, provided that:

I Bi - Gi I < a , I G; - Rj I < b un/vai | Bj - Rj | < c,

where a, b and c are controllably variable fixed numbers.

The method according to claim 1 or 2, characterized in that at every or several of the identified spectral intervals or fixed wavelengths a single narrow-band spectral image is being formed of the pixels or pixel groups which have been selected as being consistent to this particular spectral interval or the fixed wavelength.

The method according to claim 3, characterized in that the narrow-band spectral images of the identified spectral intervals are being compared, divided, extracted and/or otherwise mutually manipulated. A device for multi-spectral imaging according to the method defined in claims 1-4 comprising a multi-spectral light source (A), an objective-equipped digital RGB sensor (D), a RGB data set storage device (E), a converter (F) for converting the RGB data into the set of spectral intensities accordingly to the chosen signal discrimination level, an image selector (I) for selecting the images belonging to the specified spectral interval, and an output device (J), e.g. a computer's monitor.