CN114414500B - Spectrum detection method, storage medium, electronic device, and apparatus - Google Patents

Abstract

The present application relates to a spectrum detection method, storage medium, electronic equipment and device. The method includes: obtaining, through an imaging device, a first image when the reference light source is on and a second image when the reference light source is off, wherein a difference between the shooting time of the first image and the shooting time of the second image The interval between the two is less than a preset threshold; a reference image is obtained according to the first image and the second image; and through the conversion model, the RGB intensity of each pixel on the reference image is converted into the corresponding spectral intensity, thereby obtaining the reference image. corresponding spectrogram. Wherein, the conversion model is based on the factory setting of the reference light source and undergoes pre-calibration processing. Wherein, the pre-calibration process of the conversion model is based on the simulation detection process of the imaging device. This facilitates spectral detection through existing hardware modules and functions.

Description

Spectral detection method, storage medium, electronic equipment and device

technical field

The present application relates to the field of Internet technologies, in particular to the technical field of intelligent terminals, and in particular, to a spectral detection method, a storage medium, an electronic device, and a device.

Background technique

With the development of smart terminal technology and the advancement of the manufacturing level of the semiconductor industry, smart terminal devices represented by mobile phones, tablet computers, etc. have more and more powerful processing capabilities and are also equipped with more and more powerful image acquisition and processing functions. For example The adoption of better cameras, etc., has also enabled smart terminal devices such as mobile phones to have richer uses, including uses beyond the traditional sense of using mobile phones to communicate and communicate. For example, smart phones and smart bracelets can be equipped with the function of physiological parameter measurement and used for health monitoring, for example.

Spectral detection technology and corresponding spectral analysis technology refer to identifying substances and determining chemical composition by detecting their spectra, and providing relevant analysis conclusions through appropriate processing. Spectral detection technology can detect material components without destroying the sample, and the results of spectral analysis are widely used in many fields such as biology, medicine, chemistry, food safety, and environmental testing. The equipment for spectral detection is called a spectrometer. With the technological progress in the miniaturization of spectrometers, the size of spectrometer equipment has been greatly reduced. For example, the volume of small spectrometers based on infrared spectral detection technology and digital light processing technology (Digital Light Processing, DLP) has been reduced to a pocket that can be put into clothing. middle. However, the current miniaturized spectrometer equipment is still too large to be integrated into the mobile phone, and the miniaturized spectrometer is difficult to integrate through the semiconductor process like other hardware modules of the mobile phone, that is, it must occupy a part of the limited space inside the mobile phone. , so it is not conducive to the promotion of spectral detection technology on intelligent terminal equipment. On the other hand, spectrometers or spectrometer chips based on micro-nano process and light detection array, although they can be prepared by semiconductor process, need to form an optical layer structure that meets special requirements in order to provide the necessary light modulation effect. For example, CN112510059B discloses a light modulation structure The refractive index must be controlled between 1 and 5, etc., so it is not conducive to the promotion of spectral detection technology on smart terminal equipment.

To this end, a spectral detection method, storage medium, electronic equipment and device are needed, which can make full use of the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers, thereby expanding the use of these smart terminal devices. In order to cover many fields such as biology, medicine, chemistry, food safety, and environmental detection based on the application of spectral detection technology, it is conducive to the promotion of spectral detection technology on intelligent terminal equipment.

SUMMARY OF THE INVENTION

In a first aspect, the embodiments of the present application provide a spectral detection method. The spectral detection method includes: obtaining, through an imaging device, a first image when a reference light source is on and a second image when the reference light source is off, wherein the shooting time of the first image and the second image are The interval between the shooting times is less than a preset threshold; a reference image is obtained according to the first image and the second image; and through the conversion model, the RGB intensity of each pixel on the reference image is converted into the corresponding spectral intensity, thereby obtaining Spectrogram corresponding to this reference image. Wherein, the conversion model is based on the factory setting of the reference light source and undergoes pre-calibration processing. Wherein, the pre-calibration process of the conversion model is based on the simulation detection process of the imaging device.

The technical solution described in the first aspect can make use of the existing hardware modules and functions of the smart terminal device, so it does not need to occupy space separately or to provide a specific optical layer structure and light modulation structure through a complex micro-nano process, and by setting sufficient A small preset threshold is used to limit the interval between the shooting time of the first image and the shooting time of the second image, so as to effectively exclude interference factors other than the reference light source, so that the conversion model based on the reference light source can be used. It realizes the accurate calculation of spectral detection results from the RGB intensity information contained in the reference image. In this way, the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers can be fully utilized, which is beneficial to the promotion of spectral detection technology on smart terminal devices.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that obtaining the reference image according to the first image and the second image includes: the first image and the second image A pixel-level subtraction operation is performed to obtain the reference image.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that the simulation detection process of the imaging device includes: the light emitted by a light source with a known emission spectrum is reflected by a reference object with a known reflection property After reflection, it is received by the imaging device.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that the factory setting of the reference light source includes the spectral distribution measured at the factory of the reference light source.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that obtaining the reference image according to the first image and the second image includes: obtaining the reference image in the first image and the second image respectively. A region of interest ROI is identified on the image, and pixel-level subtraction is performed on the pixel points in the ROI of the first image and the pixel points in the ROI of the second image to obtain the reference image.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that the exposure time for obtaining the first image by the imaging device can be adjusted.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that the adjustment of the exposure time is based on the dynamic interval of the imaging device, and the length of the exposure time is based on the relative The first image is obtained with reference to the intensity of ambient light of the light source and/or the imaging device.

According to a possible implementation manner of the technical solution of the first aspect, the embodiment of the present application further provides that the adjustment of the exposure time is also based on the preset threshold.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that the light emitted by the reference light source is filtered by an adjustable filter, and the adjustable filter is configured to The RGB components in the light emitted by the reference light source are selectively enhanced or attenuated according to the RGB components in the ambient light of the reference light source.

According to a possible implementation of the technical solution of the first aspect, the embodiment of the present application further provides that the tunable filter is configured to selectively enhance the spectral distribution of ambient light relative to the reference light source Or attenuate the RGB components in the light emitted by the reference light source, including: when the spectral distribution of the ambient light is dominated by blue light, the tunable filter is configured to enhance the light emitted by the reference light source. and attenuates other RGB components in the light emitted by the reference light source; when the spectral distribution of the ambient light is dominated by yellow light, the tunable filter is configured to enhance the reference The B component in the light emitted by the light source and attenuate the other RGB components in the light emitted by the reference light source.

In a second aspect, an embodiment of the present application provides a mobile device. The mobile device includes: an image capture device; an illumination device; and a processor. Wherein, the processor is configured to execute the spectral detection method according to any one of the first aspects and use the image acquisition device as the imaging device and the lighting device as the reference light source.

The technical solution described in the second aspect can make use of the existing hardware modules and functions of the smart terminal equipment, so it does not need to occupy space separately or to provide a specific optical layer structure and light modulation structure through a complex micro-nano process, and by setting enough A small preset threshold is used to limit the interval between the shooting time of the first image and the shooting time of the second image, so as to effectively exclude interference factors other than the reference light source, so that the conversion model based on the reference light source can be used. It realizes the accurate calculation of spectral detection results from the RGB intensity information contained in the reference image. In this way, the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers can be fully utilized, which is beneficial to the promotion of spectral detection technology on smart terminal devices.

According to a possible implementation of the technical solution of the second aspect, the embodiment of the present application further provides that the mobile device is a mobile phone, the image acquisition device is a camera on the mobile phone, and the lighting device is the Lights on your phone.

In a third aspect, embodiments of the present application provide a non-transitory computer-readable storage medium. The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the spectral detection method according to any one of the first aspects.

The technical solution described in the third aspect can make use of the existing hardware modules and functions of the smart terminal device, so it does not need to occupy space separately or to provide a specific optical layer structure and light modulation structure through a complex micro-nano process, and by setting enough A small preset threshold is used to limit the interval between the shooting time of the first image and the shooting time of the second image, so as to effectively exclude interference factors other than the reference light source, so that the conversion model based on the reference light source can be used. It realizes the accurate calculation of spectral detection results from the RGB intensity information contained in the reference image. In this way, the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers can be fully utilized, which is beneficial to the promotion of spectral detection technology on smart terminal devices.

In a fourth aspect, an embodiment of the present application provides an electronic device. The electronic device includes: a processor; and a memory for storing instructions executable by the processor. Wherein, the processor executes the executable instructions to implement the spectral detection method according to any one of the first aspects.

The technical solution described in the fourth aspect can utilize the existing hardware modules and functions of the intelligent terminal equipment, so it does not need to occupy space separately or to provide a specific optical layer structure and light modulation structure through a complex micro-nano process, and by setting enough A small preset threshold is used to limit the interval between the shooting time of the first image and the shooting time of the second image, so as to effectively exclude interference factors other than the reference light source, so that the conversion model based on the reference light source can be used. It realizes the accurate calculation of spectral detection results from the RGB intensity information contained in the reference image. In this way, the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers can be fully utilized, which is beneficial to the promotion of spectral detection technology on smart terminal devices.

In a fifth aspect, an embodiment of the present application provides a spectrum detection device. The spectral detection device includes: a reference light source; an imaging device for obtaining a first image when the reference light source is on and a second image when the reference light source is off, wherein the shooting time of the first image The interval between the shooting time of the second image and the second image is less than a preset threshold; the reference image generation module is used to obtain a reference image according to the first image and the second image; the conversion module is used to convert the model to the reference image. The RGB intensity of each pixel on the image is converted into the corresponding spectral intensity, thereby obtaining a spectral map corresponding to the reference image. Wherein, the conversion model is based on the factory setting of the reference light source and undergoes pre-calibration processing. Wherein, the pre-calibration process of the conversion model is based on the simulation detection process of the imaging device.

The technical solution described in the fifth aspect can utilize the existing hardware modules and functions of the intelligent terminal equipment, so it does not need to occupy space separately or to provide a specific optical layer structure and light modulation structure through a complex micro-nano process, and by setting sufficient A small preset threshold is used to limit the interval between the shooting time of the first image and the shooting time of the second image, so as to effectively exclude interference factors other than the reference light source, so that the conversion model based on the reference light source can be used. It realizes the accurate calculation of spectral detection results from the RGB intensity information contained in the reference image. In this way, the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers can be fully utilized, which is beneficial to the promotion of spectral detection technology on smart terminal devices.

According to a possible implementation of the technical solution of the fifth aspect, the embodiment of the present application further provides that the exposure time for obtaining the first image by the imaging device can be adjusted, and the adjustment of the exposure time is based on the imaging device and the length of the exposure time is based on the intensity of ambient light relative to the reference light source and/or the shooting distance at which the imaging device obtains the first image.

According to a possible implementation of the technical solution of the fifth aspect, the embodiment of the present application further provides that the spectral detection device further includes an adjustable filter for filtering the light emitted by the reference light source, the adjustable filter The filter is configured to selectively enhance or attenuate the RGB components of the light emitted by the reference light source according to the RGB components of the ambient light relative to the reference light source.

According to a possible implementation of the technical solution of the fifth aspect, the embodiment of the present application further provides that the tunable filter is configured to selectively enhance the RGB components in the ambient light relative to the reference light source Or attenuate the RGB components in the light emitted by the reference light source, including: when the spectral distribution of the ambient light is dominated by blue light, the tunable filter is configured to enhance the light emitted by the reference light source. and attenuates other RGB components in the light emitted by the reference light source; when the spectral distribution of the ambient light is dominated by yellow light, the tunable filter is configured to enhance the reference The B component in the light emitted by the light source and attenuate the other RGB components in the light emitted by the reference light source.

In a sixth aspect, an embodiment of the present application provides a mobile phone, wherein the mobile phone includes the spectral detection device according to any one of the fifth aspects, and the imaging device is a camera on the mobile phone, The reference light source is an illumination lamp on the mobile phone.

The technical solution described in the sixth aspect can utilize the existing hardware modules and functions of the smart terminal equipment, so it does not need to occupy space separately or to provide a specific optical layer structure and light modulation structure through a complex micro-nano process, and by setting sufficient A small preset threshold is used to limit the interval between the shooting time of the first image and the shooting time of the second image, so as to effectively exclude interference factors other than the reference light source, so that the conversion model based on the reference light source can be used. It realizes the accurate calculation of spectral detection results from the RGB intensity information contained in the reference image. In this way, the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers can be fully utilized, which is beneficial to the promotion of spectral detection technology on smart terminal devices.

According to a possible implementation of the technical solution of the sixth aspect, the embodiment of the present application further provides that the reference image generation module and the conversion module are implemented by a processing device on the mobile phone, or the reference image The generation module and the transformation module are provided separately from the processing device and are integrated into the spectral detection device.

Description of drawings

In order to describe the technical solutions in the embodiments of the present application or the background technology, the accompanying drawings required in the embodiments or the background technology of the present application will be described below.

FIG. 1 shows a schematic flowchart of a spectral detection method provided in an embodiment of the present application.

FIG. 2 shows a block diagram of a spectral detection apparatus provided by an embodiment of the present application.

FIG. 3 shows a block diagram of an electronic device used in the spectral detection method shown in FIG. 1 provided by an embodiment of the present application.

FIG. 4 shows a block diagram of a mobile phone with a spectrum detection function provided by an embodiment of the present application.

Detailed ways

In order to solve the technical problem of popularizing the spectral detection technology on the intelligent terminal equipment, the embodiments of the present application propose a spectral detection method, a storage medium, an electronic device and a device. Wherein, the spectrum detection method includes: obtaining a first image when the reference light source is turned on and a second image when the reference light source is turned off, through an imaging device, wherein the shooting time of the first image and the second image are obtained. The interval between the shooting times of the two images is less than a preset threshold; a reference image is obtained according to the first image and the second image; and the RGB intensity of each pixel on the reference image is converted into the corresponding spectral intensity through a conversion model, Thereby, a spectrogram corresponding to the reference image is obtained. Wherein, the conversion model is based on the factory setting of the reference light source and undergoes pre-calibration processing. Wherein, the pre-calibration process of the conversion model is based on the simulation detection process of the imaging device. The embodiments of the present application have the following beneficial technical effects: the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers can be fully utilized, thereby expanding the use of these smart terminal devices to cover applications based on spectrum detection technology. In many fields such as biology, medicine, chemistry, food safety, and environmental testing, it is beneficial to popularize spectral detection technology on intelligent terminal equipment.

The embodiments of the present application can be used in the following application scenarios, including but not limited to, spectral detection, spectral analysis, biological component detection, medical health, food safety, environmental detection, and the like.

The embodiments of the present application may be adjusted and improved according to a specific application environment, which is not specifically limited here.

In order to make those skilled in the art better understand the solutions of the present application, the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

FIG. 1 shows a schematic flowchart of a spectral detection method provided in an embodiment of the present application. As shown in Figure 1, the method includes the following steps.

Step S102: Obtain a first image when the reference light source is on and a second image when the reference light source is off, using an imaging device, wherein the difference between the shooting time of the first image and the shooting time of the second image is obtained. The interval between them is less than the preset threshold.

Step S104: Obtain a reference image according to the first image and the second image.

Step S106: Convert the RGB intensity of each pixel on the reference image into corresponding spectral intensity through the conversion model, thereby obtaining a spectrogram corresponding to the reference image; wherein, the conversion model is based on the factory setting of the reference light source and passed through Pre-calibration process; the pre-calibration process of the conversion model is based on the simulation detection process of the imaging device.

Please refer to the above steps S102 to S106. In step S102, by requiring the interval between the shooting time of the first image and the shooting time of the second image to be less than a preset threshold, and by setting a sufficiently small preset threshold, It is possible to make the difference between the first image and the second image almost all come from whether there is illumination of the reference light source. This makes it possible to obtain a reference image according to the first image and the second image in step S104, and the reference image is equivalent to an image obtained only under the illumination condition of the reference light source. In some embodiments, a relatively stable shooting environment can be arranged to reduce the requirement on the preset threshold, that is, reduce the requirement on the interval between the shooting time of the first image and the shooting time of the second image. For example, when the spectral detection method shown in Figure 1 is used in a food safety scenario, the food samples whose components need to be determined can be placed indoors with stable indoor lighting conditions, such as closing the curtains and using only indoor lighting equipment. It is beneficial to maintain the moment when the first image is taken and the moment when the second image is taken. Except whether the reference light source is in an on state or off state, other factors are basically the same, so that the corresponding operation in step S104 can be performed. Stage subtraction or other operations can be used to cancel these constant factors and retain only the influence of the reference light source. However, in other embodiments, it may be difficult to provide a relatively stable shooting environment, for example, a scene used for human medical health detection, such as a hospital, or a scene used for environmental detection, such as a sewage treatment site, etc., the requirements for these scenes are sufficiently small The interval between the shooting time of the first image and the shooting time of the second image, that is, the shooting speed or shutter speed of the imaging device has certain requirements. This point can be set in combination with the performance of the imaging device used. For example, the imaging device may be a professional high-speed camera and can provide a shutter speed of one thousandth of a second, or a maximum of two thousand images per second. . In general, the spectral detection method is mainly applied to smart terminal devices such as smart phones, and smart phones are generally equipped with cameras with better performance and provide fast shooting modes. Shutter shooting speed of 1/125 to 1/500, that is, 125 to 500 images per second can be captured. This shutter speed is sufficient to capture moving pedestrians, bicycles, etc. in everyday shooting situations. To sum up, depending on the interference factors other than the reference light source under specific shooting conditions, including ambient light, background light, reflected light and other light sources, a small enough preset threshold can be set to limit the shooting of the first image The interval between the time and the shooting time of the second image, so as to overcome the influence of these interference factors, so that the difference between the obtained first image and the second image almost all comes from whether there is illumination of the reference light source. factor.

In step S104, a reference image is obtained according to the first image and the second image. As mentioned above, the first image corresponds to the reference light source in an on state, and the second image corresponds to the reference light source in an off state, in addition to the state change of the reference light source, by setting a sufficiently small preset threshold, for example, in combination with specific shooting Conditions and interference factors can be equivalent to thinking that the first image represents R _F+A , where R represents the function of the RGB intensity of pixels on the image or the distribution of RGB intensity, F represents the reference light source, and A represents the reference light source except for the reference light source. Factors other than those that can affect RGB intensity (these are considered disturbances relative to the reference light source). Then the second image represents R _A , and it is assumed here that A in the second image, that is, the interference factor, is the same as or substantially consistent with A in the first image. Therefore, by performing appropriate operations on the first image and the second image, such as pixel-level subtraction, which is equivalent to performing an operation of R _F+A minus R _A , the reference image thus obtained represents R _F , that is to say, only the factor of F, which is the reference light source, is retained on the reference image.

In step S106, by converting the model, the RGB intensity of each pixel on the reference image is converted into corresponding spectral intensity, thereby obtaining a spectral map corresponding to the reference image. As mentioned above, the reference image represents the _RF , which is the RGB intensity of the pixels only affected by the reference light source. The transformation model can be denoted as T, while the spectral intensity distribution can be denoted as R(λ). By calculating R _F x T = R (λ), it is possible to obtain the corresponding spectral intensities of the pixels on the reference image according to the RGB intensity of the pixels of the reference image and the conversion model T. The spectrum obtained in this way is the spectrum detection result, which can be further used for analysis and processing to determine the substance composition, and is suitable for many fields such as biology, medicine, chemistry, food safety, and environmental detection based on the application of spectrum detection technology.

The key to the above-mentioned spectral detection method being able to provide good spectral detection results lies in the conversion model T, that is to say, the conversion model T must be able to achieve the conversion from the RGB intensity of the pixel point of the reference image to the corresponding spectral intensity accurately enough. And the factors that may interfere with the accuracy of the conversion model T, that is, the factors that may interfere with the performance of the above-mentioned spectral detection method, come from other light sources other than the reference light source. This is because other light sources other than the reference light source are often uncontrollable and have high randomness, such as natural light, ambient light, etc., which makes it impossible to establish an accurate enough conversion model to take these possible interference factors into account. As described above, the interval between the shooting time of the first image and the shooting time of the second image is defined by setting a sufficiently small preset threshold, and then the calculation is performed to effectively eliminate interference factors other than the reference light source. In addition, the conversion model T is based on the factory setting of the reference light source and undergoes a pre-calibration process; the pre-calibration process of the conversion model T is based on a simulated detection process of the imaging device. Here, the factory setting of the reference light source can be considered to represent the conversion model T of the reference light source when it leaves the factory, that is to say, the above-mentioned calculation can be completed with the help of the factory setting of the reference light source, so as to realize the transformation according to the pixels of the reference image. RGB intensities and transformation model T to obtain the corresponding spectral intensities of pixels on the reference image. Considering that the reference light source may face various conditions such as device aging and loss during the use process after leaving the factory, which deviates from the factory setting, it is necessary to determine the conversion model of the current reference light source through pre-calibration processing. In addition, even if the reference light source maintains the factory settings, there may be aging or wear of other components that may cause the external appearance of the reference light source to deviate from the factory settings. To this end, through the simulation detection process of the imaging device, that is, performing a spectral detection by simulating the current imaging device and the current reference light source, comparing the detection result with the reference result, and then performing appropriate processing, the conversion model T can be completed. pre-calibration.

It can be seen that the above-mentioned spectral detection method can use the existing hardware modules and functions of the intelligent terminal equipment, such as using the lighting equipment on the mobile phone as a reference light source and using the camera on the mobile phone as an imaging device, so it does not need to occupy space separately or not. A specific optical layer structure and a light modulation structure are provided through a complex micro-nano process, and the interval between the shooting time of the first image and the shooting time of the second image is defined by setting a sufficiently small preset threshold (for example, set It is determined as the shutter speed of shooting pedestrians in motion), and then the calculation is carried out to effectively eliminate the interference factors other than the reference light source, so that the conversion model of the reference light source can be used to accurately convert the RGB intensity contained in the reference image from the reference image. The information extrapolates the spectral detection results. The above-mentioned spectral detection method can make full use of the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers, thereby expanding the use of these smart terminal devices to cover biological, medical, Chemistry, food safety, environmental detection and other fields are conducive to the promotion of spectral detection technology on intelligent terminal equipment.

In a possible implementation manner, the reference image is obtained according to the first image and the second image. The method includes: performing a pixel-level subtraction operation on the first image and the second image to obtain the reference image. It should be understood that other suitable pixel-level operations or other operation manners can also be used, as long as interference factors other than the reference light source can be effectively excluded.

In a possible implementation, the analog detection process of the imaging device includes reflecting light emitted by a light source with a known emission spectrum by a reference object with a known reflection property before being received by the imaging device. In some embodiments, the reference object is a white test panel. As mentioned above, through the simulation detection process of the imaging device, that is, the current imaging device and the current reference light source are simulated to perform a spectral detection and compare the detection result with the reference result, and then perform appropriate processing to complete the conversion model. Pre-calibration of T. Here, through a reference object such as a white test plate, its own reflection properties can be pre-judged, so it can be estimated that the spectral detection result after the reflection of the reference object is the reference result, for example, it can be based on the known attenuation of the reference object. characteristics and the spectral intensity distribution of incident light to estimate the spectral intensity distribution of reflected light. Generally speaking, for a reference object with uniform or relatively simple reflection properties, such as a monochromatic test plate, the reflection properties are known or easily inferred. In this way, the pre-calibration of the conversion model T can be completed through the above-mentioned simulation detection process. It should be understood that, in addition to the white test plate, other reference objects of known reflective properties may be used, as long as their spectral detection results can be conveniently pre-estimated. Also consider using different colored test panels assembled together or other patterns or styles of test panels.

In a possible embodiment, the factory setting of the reference light source includes the spectral distribution of the reference light source determined at the factory. In this way, by measuring the spectral distribution of the reference light source at the time of shipment, the conversion model of the reference light source can be better estimated.

In a possible implementation manner, obtaining the reference image according to the first image and the second image includes: identifying a region of interest (ROI) on the first image and the second image, respectively; The reference image is obtained by performing pixel-level subtraction between the pixels in the ROI and the pixels in the ROI of the second image. Considering that the samples that need to be subjected to spectral analysis and detection in practical applications may only occupy a part of the area on the image, for example, the food sample to be detected in a food safety scenario is located in a part of the center of the image, so the ROI can be used to concentrate resources to analyze and process the ROI. In this way, the limited computing resources on smart terminal devices can be better utilized and energy consumption can be reduced.

In a possible implementation, the exposure time for obtaining the first image by the imaging device is adjustable. In some embodiments, the adjustment of the exposure time is based on the dynamic interval of the imaging device, and the length of the exposure time is based on the intensity of ambient light relative to the reference light source and/or the imaging device obtains the first The shooting distance of an image. In some embodiments, the adjustment to the exposure time is also based on the preset threshold. As mentioned above, when the reference light source corresponding to the first image is turned on, the dynamic range of the imaging device such as a camera can be fully utilized by adjusting the exposure time. If the exposure time is too long, the dynamic range will be too saturated. If the exposure time is too short, it may cause The signal is too weak and the signal-to-noise ratio is too high. By adjusting the exposure time of the first image, especially adjusting the exposure time according to the dynamic range, the signal-to-noise ratio can be improved, which helps to overcome the problems caused by the background light being too strong or the reference light source being too weak relative to the background light. Strong noise interference. In addition, in combination with practical requirements, the length of the exposure time can be adjusted more flexibly based on the intensity of ambient light relative to the reference light source and/or the shooting distance at which the imaging device obtains the first image. For example, in industrial inspection scenarios such as sewage inspection, it is generally necessary to perform spectral analysis and inspection on a large area of long-distance sewage. In this case, the imaging device obtains the first image with a long shooting distance, so it is suitable to adjust the exposure time. to improve the signal-to-noise ratio. For another example, indoors may face a situation where the background light is too strong, for example, the brightness of the indoor lighting is too high. In this case, the intensity of the ambient light relative to the reference light source is too high, so it is suitable to adjust the exposure time to overcome the effect of the too strong background light. interference. For another example, when the background light is not so strong and the shooting distance is relatively short, the exposure time can be adjusted to make full use of the dynamic range to achieve better detection results. In addition, the adjustment of the exposure time can also be based on the preset threshold. The function of the preset threshold mentioned above is to limit the interval between the moment when the first image is taken and the moment when the second image is taken, so as to suppress the influence of interference factors, Therefore, the exposure time can be adjusted in combination with the preset threshold to further suppress interference and improve the signal-to-noise ratio.

In one possible implementation, the light emitted by the reference light source is filtered by a tunable filter, the tunable filter being configured to selectively select according to RGB components in ambient light relative to the reference light source The RGB components in the light emitted by the reference light source are substantially enhanced or attenuated. Here, the filtering of the tunable filter will change the spectral distribution of the light emitted by the reference light source, for example, the filtering of the green filter will obtain a spectral distribution dominated by green light. Relative to the RGB components in the ambient light of the reference light source, it is the aforementioned interference factor that needs to be suppressed and will negatively affect the spectral detection effect. Therefore, the RGB components in the light emitted by the reference light source can be selectively enhanced or weakened according to the RGB components in the ambient light relative to the reference light source, thereby suppressing the influence of the ambient light and enhancing the influence of the reference light source , which can improve the signal-to-noise ratio and improve the detection effect. In some embodiments, the tunable filter is configured to selectively enhance or attenuate RGB components in light emitted by the reference light source according to a spectral distribution in ambient light relative to the reference light source, including: When the spectral distribution in the ambient light is dominated by blue light, the tunable filter is configured to enhance the R component or the G component in the light emitted by the reference light source and attenuate the light emitted by the reference light source. and other RGB components of other RGB components in the light. Therefore, when the RGB component in the ambient light is dominated by blue light, this situation can be the scene of using blue light or blue light-dominated lighting equipment in the factory workshop or laboratory. At this time, by enhancing the R component of the reference light source or The G component attenuates other components at the same time, and the signal-to-noise ratio can be improved. When the RGB component in the ambient light is dominated by yellow light, this situation is common in the semiconductor processing and manufacturing industry, where the ultra-clean room and other places generally use yellow light-based lighting equipment. At this time, by enhancing the B component of the reference light source At the same time, attenuating other components can improve the signal-to-noise ratio.

It should be understood that, the above method can be implemented by a corresponding executive body or carrier. In some exemplary embodiments, a non-transitory computer-readable storage medium stores computer instructions that, when executed by a processor, implement the foregoing method and any of the foregoing embodiments and implementations or a combination of them. In some exemplary embodiments, an electronic device includes: a processor; a memory for storing processor-executable instructions; wherein the processor executes the executable instructions to implement the above method and any of the above implementations examples, implementations, or a combination thereof.

In addition, the above method can be implemented by a mobile device or any suitable intelligent terminal device. For example, a mobile device includes: an image capture device; an illumination device; and a processor. Wherein, the processor is configured to execute the above-mentioned spectral detection method and use the image acquisition device as the imaging device and the lighting device as the reference light source. The mobile device can be a mobile phone, a tablet computer, or any suitable device or smart terminal device, as long as it has necessary components such as an imaging device and a reference light source. The image acquisition device may be any suitable device on the mobile device, for example, there may be multiple cameras or cameras with a shooting function on the mobile phone, any one of the cameras or cameras can be used as the imaging device. The lighting device may be a built-in back light of the mobile device, such as a mobile phone, or an additional or additionally provided lighting device, such as a matching lighting device, as long as the relevant details of the above-mentioned spectral detection method are met.

FIG. 2 shows a block diagram of a spectral detection apparatus provided by an embodiment of the present application. As shown in FIG. 2, the spectral detection device includes: a reference light source 210; an imaging device 220 for obtaining a first image when the reference light source 210 is on and a second image when the reference light source 210 is off image, wherein the interval between the shooting time of the first image and the shooting time of the second image is less than a preset threshold; the reference image generation module 230 is used to obtain a reference image according to the first image and the second image; conversion The module 240 is configured to convert the RGB intensity of each pixel point on the reference image into corresponding spectral intensity through a conversion model (not shown), so as to obtain a spectral map corresponding to the reference image. The conversion model is based on the factory settings of the reference light source 210 and undergoes pre-calibration processing. The pre-calibration process of the conversion model is based on the simulation detection process of the imaging device 220 . It should be understood that the spectral detection device can be understood as a part of the intelligent terminal equipment, and can also be understood as calling the intelligent terminal equipment by means of software such as instructions or programs or applications in an integrated or additional manner. The existing hardware modules are used to realize the corresponding functions, which are not specifically limited here. And, the reference light source 210 and the imaging device 220 are communicatively connected so that the imaging device 220 can obtain the first image and the second image and complete the analog detection process together with the reference light source 210 . The image obtained by the imaging device 220 is transmitted to the reference image generation module 230 , which is then transmitted to the conversion module 240 by the reference image generation module 230 . The transformation module 240 is also connected with the reference light source 210 for realizing the determination of the transformation model based on the factory setting and pre-calibration processing of the reference light source 210 .

The above-mentioned spectral detection device can use the existing hardware modules and functions of the intelligent terminal equipment, such as using the lighting device on the mobile phone as the reference light source 210 and using the camera on the mobile phone as the imaging device 220, so it does not need to occupy a separate space or pass through the complex. The micro-nano process is used to provide a specific optical layer structure and light modulation structure, and the interval between the shooting time of the first image and the shooting time of the second image is limited by setting a sufficiently small preset threshold (for example, set as Shooting the shutter speed of pedestrians in motion), and then perform calculations to effectively eliminate interference factors other than the reference light source, so that it can be accurately calculated from the RGB intensity information contained in the reference image based on the conversion model of the reference light source. Spectral detection results. The above-mentioned spectral detection device can make full use of the existing highly integrated hardware modules and functions of smart terminal devices such as mobile phones and tablet computers, so as to expand the use of these smart terminal devices to cover biological, medical, Chemistry, food safety, environmental detection and other fields are conducive to the promotion of spectral detection technology on intelligent terminal equipment.

In a possible implementation manner, the exposure time during which the imaging device 220 obtains the first image can be adjusted, the adjustment of the exposure time is based on the dynamic interval of the imaging device 220, and the length of the exposure time is based on The intensity of the ambient light relative to the reference light source 210 and/or the shooting distance at which the imaging device 220 obtains the first image. In this way, the signal-to-noise ratio can be improved, which helps to overcome the strong noise interference caused by the background light being too strong or the reference light source being too weak relative to the background light.

In a possible implementation, the spectral detection device further includes an adjustable filter (not shown) for filtering the light emitted by the reference light source 210, the adjustable filter is configured to The RGB components in the ambient light of the reference light source 210 are used to selectively enhance or weaken the RGB components in the light emitted by the reference light source 210 . In this way, the influence of ambient light can be suppressed while the influence of the reference light source 210 can be enhanced, so that the signal-to-noise ratio can be improved and the detection effect can be improved.

In a possible implementation, the tunable filter is configured to selectively enhance or weaken the light emitted by the reference light source 210 according to the RGB components in the ambient light relative to the reference light source 210 RGB components, including: when the spectral distribution in the ambient light is dominated by blue light, the tunable filter is configured to enhance the R component or the G component in the light emitted by the reference light source 210 and attenuate the reference other RGB components in the light emitted by the light source 210; when the spectral distribution in this ambient light is dominated by yellow light, the tunable filter is configured to enhance the B component in the light emitted by the reference light source 210 And attenuate other RGB components in the light emitted by the reference light source 210 . In this way, the influence of ambient light can be suppressed while the influence of the reference light source 210 can be enhanced, so that the signal-to-noise ratio can be improved and the detection effect can be improved.

FIG. 3 shows a block diagram of an electronic device used in the spectral detection method shown in FIG. 1 provided by an embodiment of the present application. As shown in FIG. 3 , the electronic device includes a main processor 302 , an internal bus 304 , a network interface 306 , a main memory 308 , an auxiliary processor 310 and auxiliary memory 312 , and an auxiliary processor 320 and auxiliary memory 322 . The main processor 302 is connected to the main memory 308, and the main memory 308 can be used to store computer instructions executable by the main processor 302, so that the spectral detection method shown in FIG. 1 can be implemented, including some or all of the steps, including the Any possible combination or combination of steps and possible substitutions or variations. The network interface 306 is used to provide network connectivity and to send and receive data over the network. Internal bus 304 is used to provide internal data exchange between main processor 302 , network interface 306 , auxiliary processor 310 , and auxiliary processor 320 . The auxiliary processor 310 is connected with the auxiliary memory 312 and provides auxiliary computing capability together, and the auxiliary processor 320 is connected with the auxiliary memory 322 and provides auxiliary computing capability together. The auxiliary processor 310 and the auxiliary processor 320 may provide the same or different auxiliary computing capabilities, including, but not limited to, computing capabilities optimized for specific computing needs, such as parallel processing capabilities or tensor computing capabilities, for specific algorithms or logical structures. Optimized computing capabilities such as iterative computing capabilities or graph computing capabilities. Auxiliary processor 310 and auxiliary processor 320 may include one or more processors of a particular type, such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., so as to provide Customized functions and structures. In some exemplary embodiments, the electronic device may not include an auxiliary processor, may include only one auxiliary processor, or may include any number of auxiliary processors, each of which has corresponding customized functions and structures, which are not detailed here. limited. The architecture of the two auxiliary processors shown in Figure 3 is illustrative only and should not be construed as limiting. In addition, the main processor 302 may include a single-core or multi-core computing unit for providing the necessary functions and operations of the embodiments of the present application. In addition, the main processor 302 and the auxiliary processor (such as the auxiliary processor 310 and the auxiliary processor 320 in FIG. 3 ) may have different architectures, that is, the electronic device may be a system based on a heterogeneous architecture, such as the main processor 302 It can be a general-purpose processor based on an instruction set operating system, such as a CPU, and the auxiliary processor can be a graphics processor GPU suitable for parallel computing or a dedicated accelerator suitable for neural network model-related operations. Auxiliary memory (for example, auxiliary memory 312 and auxiliary memory 322 shown in FIG. 3 ) can be used to cooperate with their corresponding auxiliary processors to implement customized functions and structures. The main memory 308 is used to store necessary instructions, software, configuration, data, etc., so as to cooperate with the main processor 302 to provide the necessary functions and operations of the embodiments of the present application. In some exemplary embodiments, the electronic device may not include auxiliary memory, may include only one auxiliary memory, or may include any number of auxiliary memories, which is not specifically limited herein. The architecture of the two auxiliary memories shown in FIG. 3 is merely illustrative and should not be construed as limiting. Main memory 308 and possibly secondary memory may include one or more of the following characteristics: volatile, non-volatile, dynamic, static, read/write, read only, random access, sequential access, location addressability, File addressability and content addressability, and can include random access memory (RAM), flash memory, read only memory (ROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only Memory (EEPROM), registers, hard disk, removable disk, recordable and/or rewritable compact disc (CD), digital versatile disc (DVD), mass storage media device or any other form of suitable storage medium. The internal bus 304 may include any one or a combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a bus utilizing any of a variety of bus architectures. processor or local bus. It should be understood that the structure shown in the electronic device shown in FIG. 3 does not constitute a specific limitation to the relevant apparatus or system. In some exemplary embodiments, the electronic device may include more details than the specific embodiments and the accompanying drawings. More or fewer components, or some components are combined, or some components are split, or have a different arrangement of components.

FIG. 4 shows a block diagram of a mobile phone with a spectrum detection function provided by an embodiment of the present application. As shown in FIG. 4 , the mobile phone includes a light 410 , a camera 420 , a processing device 430 and a memory 440 . The mobile phone shown in FIG. 4 may include the above-mentioned spectral detection device, and the imaging device 220 is the camera 420 on the mobile phone, and the reference light source 210 is the lighting lamp 410 on the mobile phone. Wherein, the reference image generation module 230 and the conversion module 240 are implemented by the processing device 430 on the mobile phone, or the reference image generation module 230 and the conversion module 240 are separately provided relative to the processing device 430 and integrated in the processing device 430. The spectroscopic detection device.

It should be understood that the processing device 430 on the mobile phone can operate the lighting lamp 410 and the camera 420 by running programs or instructions stored in the memory 440 to complete the above-mentioned spectral detection method, that is, the reference image generation module 230 and the conversion method. The module 240 is realized by the processing device 430 on the mobile phone. In other embodiments, the reference image generation module 230 and the conversion module 240 can be provided separately, for example, the functions of these modules can be provided separately by means of pluggable accessories, such as providing an integrated spectral detection device, As a whole, it can be connected to a mobile phone through a USB interface and provide extended spectral detection functions.

The specific embodiments provided herein may be implemented in any one or combination of hardware, software, firmware or solid state logic circuits, and may be implemented in conjunction with signal processing, control and/or special purpose circuits. The apparatus or apparatus provided by the specific embodiments of the present application may include one or more processors (eg, a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) ), etc.), these processors process various computer-executable instructions to control the operation of a device or apparatus. The device or apparatus provided by the specific embodiments of the present application may include a system bus or a data transmission system that couples various components together. A system bus may include any one or a combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or processing utilizing any of a variety of bus architectures device or local bus. The equipment or apparatus provided by the specific embodiments of the present application may be provided independently, may be a part of a system, or may be a part of other equipment or apparatus.

Embodiments provided herein may include or be combined with computer-readable storage media, such as one or more storage devices capable of providing non-transitory data storage. The computer-readable storage medium/storage device may be configured to hold data, programmers and/or instructions that, when executed by the processors of the apparatuses or apparatuses provided by the specific embodiments of the present application, cause these apparatuses Or the device realizes the relevant operation. Computer-readable storage media/storage devices may include one or more of the following characteristics: volatile, non-volatile, dynamic, static, read/write, read-only, random access, sequential access, location addressability, File addressability and content addressability. In one or more exemplary embodiments, the computer-readable storage medium/storage device may be integrated into the device or apparatus provided by the specific embodiments of the present application or belong to a public system. Computer readable storage media/storage devices may include optical storage devices, semiconductor storage devices and/or magnetic storage devices, etc., and may also include random access memory (RAM), flash memory, read only memory (ROM), erasable and programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Registers, Hard Disk, Removable Disk, Recordable and/or Rewritable Compact Disc (CD), Digital Versatile Disc (DVD), Mass storage media device or any other form of suitable storage media.

The above are the implementations of the embodiments of the present application. It should be noted that the steps in the methods described in the specific embodiments of the present application may be adjusted, combined and deleted in order according to actual needs. In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments. It can be understood that the structures shown in the embodiments of the present application and the accompanying drawings do not constitute a specific limitation on the relevant device or system. In other embodiments of the present application, the related device or system may include more or less components than the specific embodiments and drawings, or combine some components, or separate some components, or have different component arrangements. Those skilled in the art will understand that, without departing from the spirit and scope of the specific embodiments of the present application, various modifications or changes can be made to the arrangements, operations and details of the methods and devices described in the specific embodiments; Under the premise of applying the principle of the embodiment, several improvements and modifications can also be made, and these improvements and modifications are also regarded as the protection scope of the present application.

Claims (11)

1. a spectral detection method, is characterized in that, described spectral detection method comprises: Using the imaging device, obtain a first image of the same area where the same sample is located when the reference light source is turned on and a second image when the reference light source is turned off, wherein the shooting time of the first image and the time of the second image are obtained. The interval between shooting times is less than a preset threshold; performing a pixel-level subtraction operation on the first image and the second image to obtain the reference image; and Through the conversion model, the RGB intensity of each pixel on the reference image is converted into the corresponding spectral intensity, so as to obtain the spectral map corresponding to the reference image and obtain the spectral detection result of the same sample based on the spectral map, Wherein, the conversion model is based on the factory setting of the reference light source and is pre-calibrated to correct the deviation from the factory setting of the reference light source, Wherein, the pre-calibration processing of the conversion model is based on the simulation detection process of the imaging device, Wherein, the exposure time for obtaining the first image by the imaging device can be adjusted, the adjustment of the exposure time is based on the dynamic interval of the imaging device, and the length of the exposure time is based on the ambient light relative to the reference light source The intensity of the imaging device and the shooting distance of the first image obtained by the imaging device, the adjustment of the exposure time is also based on the preset threshold, Wherein, the light emitted by the reference light source is filtered by a tunable filter, and the tunable filter is configured to selectively enhance or weaken the reference light source according to RGB components in ambient light relative to the reference light source The RGB components in the light emitted by the light source, Wherein, when the spectral distribution of the ambient light is dominated by blue light, the tunable filter is configured to enhance the R component or the G component in the light emitted by the reference light source and attenuate the light emitted by the reference light source other RGB components in , Wherein, when the spectral distribution in the ambient light is dominated by yellow light, the tunable filter is configured to enhance the B component in the light emitted by the reference light source and attenuate the B component in the light emitted by the reference light source other RGB components, Wherein, the interval between the shooting time of the first image and the shooting time of the second image is limited by setting the preset threshold that is sufficiently small, so that the gap between the first image and the second image is almost all Whether there is illumination from the reference light source, thus effectively eliminating the interference factors other than the reference light source, Wherein, the spectral detection method is applied to a human medical and health detection scene or an environment detection scene, and the imaging device is a professional high-speed camera and can provide a shutter shooting speed of one thousandth of a second. 2 . The method according to claim 1 , wherein the analog detection process of the imaging device comprises reflecting the light emitted by a light source with a known emission spectrum by a reference object with a known reflection property and then receiving it by the imaging device. 3 . . 3 . The method according to claim 1 , wherein the factory setting of the reference light source includes a spectral distribution measured at the factory of the reference light source. 4 . 4. The method according to claim 1, wherein, performing a pixel-level subtraction operation on the first image and the second image to obtain the reference image, comprising: Identify a region of interest ROI on the first image and the second image, respectively, The reference image is obtained by performing a pixel-level subtraction operation on the pixel points in the ROI of the first image and the pixel points in the ROI of the second image. 5. A mobile device, wherein the mobile device comprises: image acquisition device; lighting fixtures; and processor, Wherein, the processor is configured to execute the spectral detection method according to any one of claims 1 to 4 and use the image acquisition device as the imaging device and the lighting device as the reference light source. 6 . The mobile device according to claim 5 , wherein the mobile device is a mobile phone, the image acquisition device is a camera on the mobile phone, and the lighting device is a light on the mobile phone. 7 . 7. A non-transitory computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which when executed by a processor implement the method according to any one of claims 1 to 4 spectral detection method. 8. An electronic device, characterized in that the electronic device comprises: processor; memory for storing processor-executable instructions; Wherein, the processor implements the spectral detection method according to any one of claims 1 to 4 by executing the executable instructions. 9. A spectral detection device, characterized in that the spectral detection device comprises: reference light source; An imaging device for obtaining a first image of the same region where the same sample is located when the reference light source is on and a second image when the reference light source is off, wherein the shooting time of the first image and the second image are The interval between the shooting times of the images is less than the preset threshold; a reference image generation module for obtaining a reference image by performing a pixel-level subtraction operation on the first image and the second image; and The conversion module is used to convert the RGB intensity of each pixel on the reference image into the corresponding spectral intensity through the conversion model, so as to obtain the spectral map corresponding to the reference image and obtain the spectral detection result of the same sample based on the spectral map, Wherein, the conversion model is based on the factory setting of the reference light source and is pre-calibrated to correct the deviation from the factory setting of the reference light source, Wherein, the pre-calibration processing of the conversion model is based on the simulation detection process of the imaging device, Wherein, the exposure time for obtaining the first image by the imaging device can be adjusted, the adjustment of the exposure time is based on the dynamic interval of the imaging device, and the length of the exposure time is based on the ambient light relative to the reference light source The intensity of the imaging device and the shooting distance of the first image obtained by the imaging device, the adjustment of the exposure time is also based on the preset threshold, Wherein, the spectral detection device further includes an adjustable filter for filtering the light emitted by the reference light source, the adjustable filter is configured to selectively select according to the RGB components in the ambient light relative to the reference light source to enhance or weaken the RGB components of the light emitted by the reference light source, Wherein, when the spectral distribution of the ambient light is dominated by blue light, the tunable filter is configured to enhance the R component or the G component in the light emitted by the reference light source and attenuate the light emitted by the reference light source other RGB components in , Wherein, when the spectral distribution in the ambient light is dominated by yellow light, the tunable filter is configured to enhance the B component in the light emitted by the reference light source and attenuate the B component in the light emitted by the reference light source other RGB components, Wherein, the interval between the shooting time of the first image and the shooting time of the second image is limited by setting the preset threshold that is sufficiently small, so that the gap between the first image and the second image is almost all Whether there is illumination from the reference light source, thus effectively eliminating the interference factors other than the reference light source, Wherein, the spectral detection method is applied to a human medical and health detection scene or an environment detection scene, and the imaging device is a professional high-speed camera and can provide a shutter shooting speed of one thousandth of a second. 10. A mobile phone, characterized in that the mobile phone comprises the spectral detection device according to claim 9, the imaging device is a camera on the mobile phone, and the reference light source is an illumination lamp on the mobile phone . 11. The mobile phone according to claim 10, wherein the reference image generation module and the conversion module are realized by a processing device on the mobile phone, or the reference image generation module and the conversion module are relative to each other. The processing device is provided separately and integrated with the spectral detection device.

Images