CN115204206B - Single-exposure multispectral computational imaging method, system and device based on metasurface - Google Patents
Single-exposure multispectral computational imaging method, system and device based on metasurface Download PDFInfo
- Publication number
- CN115204206B CN115204206B CN202210847527.2A CN202210847527A CN115204206B CN 115204206 B CN115204206 B CN 115204206B CN 202210847527 A CN202210847527 A CN 202210847527A CN 115204206 B CN115204206 B CN 115204206B
- Authority
- CN
- China
- Prior art keywords
- image
- optical
- super
- imaging
- hyperspectral
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/146—Methods for optical code recognition the method including quality enhancement steps
- G06K7/1482—Methods for optical code recognition the method including quality enhancement steps using fuzzy logic or natural solvers, such as neural networks, genetic algorithms and simulated annealing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/10544—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation by scanning of the records by radiation in the optical part of the electromagnetic spectrum
- G06K7/10821—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation by scanning of the records by radiation in the optical part of the electromagnetic spectrum further details of bar or optical code scanning devices
- G06K7/10831—Arrangement of optical elements, e.g. lenses, mirrors, prisms
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
- G06K7/1417—2D bar codes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Electromagnetism (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Toxicology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Automation & Control Theory (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Fuzzy Systems (AREA)
- Quality & Reliability (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
- Image Processing (AREA)
Abstract
本发明属于图像处理和计算成像技术领域，公开了基于超表面的单次曝光多光谱计算成像方法、系统及设备。所述方法包括：使用基于单层超表面结构，置于阵列成像传感器前方，经过单次曝光收集不同光谱通道的图像以及对应通道的光学点扩散函数PSF；利用获得的曝光图像和点扩散函数PSF信息，基于共轭梯度法来重构不同通道的成像图像；本发明的方法将单层的超表面结构作为光学成像系统前端，无需引入传统的透镜和滤波片，也不需要搭建复杂的光路系统，具有光学系统简单、小型轻便等优势；本发明方法通过共轭梯度算法对单次曝光采集到的多通道图像进行重建，可以精确求解“逆散射”问题，从而提高重构图像的质量。
The present invention belongs to the field of image processing and computational imaging technology, and discloses a single-exposure multi-spectral computational imaging method, system and device based on a metasurface. The method comprises: using a single-layer metasurface structure, placed in front of an array imaging sensor, collecting images of different spectral channels and the optical point spread function PSF of the corresponding channels through a single exposure; using the obtained exposure image and point spread function PSF information, reconstructing the imaging images of different channels based on the conjugate gradient method; the method of the present invention uses a single-layer metasurface structure as the front end of the optical imaging system, without the need to introduce traditional lenses and filters, and without the need to build a complex optical path system, and has the advantages of a simple optical system, small size and lightness; the method of the present invention reconstructs the multi-channel image collected by a single exposure through a conjugate gradient algorithm, and can accurately solve the "inverse scattering" problem, thereby improving the quality of the reconstructed image.
Description
Technical Field
The invention belongs to the technical field of image processing and computational imaging, and particularly relates to a single exposure multispectral computational imaging method, system and equipment based on a super surface.
Background
Conventional all-optical (lens) imaging systems map each point of an image sample onto a separate sensor pixel, directly producing a reliable aerial image. But this approach requires a complex optical device (a series of lenses or lens groups) and often fails to capture accurate spectral information without additional spectral filters or micro-lens arrays (micro arrays), making multispectral imaging more difficult.
In a lensless computational imaging system, a lensless optical system projects a blurred image of the target sample onto the sensor and reconstructs the target light field by solving the subsequent "backscatter" problem (by least squares fitting, etc.). However, the problem to be solved of "backscatter" is often ill-conditioned, sensitive to noise, and often requires high computational memory and power.
In recent years, the field of computational imaging has raised an "end-to-end" optimization method, i.e., a systematic method of optimizing by combining a front-end analog optical system with an image processing digital back-end, which can be used for multi-channel multi-spectral single-exposure computational imaging. The imaging system designed based on the method has better performance than the method of simple optical design and simple digital calculation imaging design. The end-to-end optimization method is introduced into the fields of full-wave vector nano photons and optical super-surface fronts, and has strong physical data acquisition and operation capacity.
Through the above analysis, the problems and defects existing in the prior art are as follows:
(1) The prior art reconstructs a target sample over multiple spectra, requiring multiple exposures.
(2) In the prior art, a complex light path system needs to be built, and the operation is inconvenient.
(3) The prior art has poor image reconstruction effect on the image acquired by single exposure, can not accurately solve the problem of 'back scattering', and has low reconstructed image quality.
Disclosure of Invention
In order to overcome the problems in the related art, the disclosed embodiments of the present invention provide a method, a system and a device for single-exposure multispectral calculation imaging based on a super surface.
The technical scheme is that the single exposure multispectral calculation imaging method based on the super surface comprises the steps of performing one-time exposure on images of different channels by an optimized super surface optical system, and collecting point spread functions PSF (point spreading function) of optical imaging systems of different channels;
In the image reconstruction stage, the original image is restored by using the conjugate gradient algorithm through the point spread functions PSF of different channels and imaging information on the sensor, and image restoration of a plurality of wave bands is obtained.
In one embodiment, the method for single-exposure multispectral imaging based on the super surface specifically comprises the following steps:
s1, constructing an image reconstruction model by utilizing a super-surface optical system and a conjugate gradient algorithm;
s2, randomly generating a series of two-dimensional code images as a training set, and training an image reconstruction model built in the step S1 to obtain optimal optical super-surface structure parameters and conjugate gradient algorithm parameters;
S3, after training is completed, the optimal optical super-surface structure and conjugate gradient algorithm parameters in the image reconstruction model are reserved and used as a final image reconstruction experimental model;
s4, inputting the multichannel image into the image reconstruction model in the step S3, and obtaining a restored image.
In one embodiment, in step S1, an image reconstruction model is built, the image v on the array sensor is defined as a roll-sum of a plurality of channel input images u and a Point Spread Function (PSF), and the image is restored by computing the imagingThe objective optimization function is
Where N is the number of channels, spectral channels, or scene distance.
In one embodiment, in step S2 to obtain an optimal optical super-surface structure, forward solution optimization is performed using a super-surface as an optical device, including the steps of:
1) Calculating a point spread function PSF of a subsurface using the subsurface as an optical imaging system front end
PSF=|ntff(t(p)·Ein)|2 (2);
Wherein, p is the structural parameter of the optical super surface, t (p) is the transmission coefficient of the optical super surface, E in is the incident electric field, ntff (·) represents the near field to far field conversion;
2) The image v on the array sensor is obtained by convolution of the point spread function PSF of the super surface and the real scene image u:
3) Reconstruction by solving the backscatter problem
Using Gihonofu regularization to process equation (4), we get
Where α is a constant to be optimized, and the convolution kernel operator is defined as
Substituting equation (3) into (5) and transforming to obtain
Solving equation (6) by conjugate gradient method to obtain reconstructed imageThe objective function L is calculated using equation (1).
In one embodiment, after calculating the objective function L, the derivative of the objective function L with respect to the super-surface structure p and the undetermined coefficient alpha of the reconstruction algorithm is calculated, so as to realize the optimization process of back propagation.
In one embodiment, the back propagation optimization process includes the steps of:
(i) Calculating derivatives
In connection with the derivative of equation (6), the above equation is replaced by:
Wherein,
Solving the lambda by using a conjugate gradient method;
(ii) Calculating derivatives
In connection with the derivative of equation (6), the above equation is replaced by:
and after the derivative of the objective function relative to the optimized parameter is obtained, the system is optimized by combining a gradient descent algorithm.
In one embodiment, the optimization of the system in combination with the gradient descent algorithm includes:
the updating method of parameters to be optimized comprises the following steps:
wherein P is a variable to be optimized, P and alpha are dynamic learning rates, and the specific value depends on a selected random gradient descent method and Adam algorithm.
Another object of the present invention is to provide a hyperspectral computed tomography system based on a hyperspectral surface, which implements the hyperspectral computed tomography method based on a hyperspectral surface, the hyperspectral computed tomography system based on a hyperspectral surface comprising:
The image reconstruction model building module is used for building an image reconstruction model by utilizing the super-surface optical system and the conjugate gradient algorithm, and training the image reconstruction model by utilizing the gradient descent method to obtain optimal super-surface structure parameters and conjugate gradient algorithm parameters;
the point spread function recording module is used for collecting multi-channel images penetrating through the optimized optical super surface by using the array sensor and recording point spread functions PSF (point spreading function) of different channels;
And the sample image reconstruction module is used for reconstructing a sample image by combining the optimized conjugate gradient algorithm with the point spread functions PSF of different channels.
It is a further object of the present invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the hyperspectral computed tomography method of single exposure based on a hypersurface.
It is another object of the present invention to provide an all-optical imager, wherein the all-optical imager performs the single-exposure multispectral imaging method based on a super surface.
By combining all the technical schemes, the invention has the advantages and positive effects that:
First, aiming at the technical problems in the prior art and the difficulty in solving the problems, the technical problems solved by the technical proposal of the invention are analyzed in detail and deeply by tightly combining the technical proposal to be protected, the results and data in the research and development process, and the like, and some technical effects brought after the problems are solved have creative technical effects. The specific description is as follows:
In combination with optimized optics and a reasonable computational imaging algorithm to reconstruct a target sample over multiple spectra, the detector can be of broad spectral response, but can be realized with only one exposure.
The method takes the single-layer super-surface structure as the front end of the optical imaging system, does not need to introduce a traditional lens (or lens group) and a filter, does not need to build a complex optical path system, and has the advantages of simple optical system, small size, portability and the like;
The method of the invention reconstructs the multichannel image acquired by single exposure through the conjugate gradient algorithm, and can accurately solve the problem of 'back scattering', thereby improving the quality of the reconstructed image.
Secondly, the technical scheme is regarded as a whole or from the perspective of products, and the technical scheme to be protected has the following technical effects and advantages:
The invention provides a method for realizing multispectral calculation imaging based on single exposure of an optical super surface, which comprises the following steps of (1) constructing an image reconstruction model by utilizing the optimal design of an optical super surface artificial microstructure and a conjugate gradient algorithm, as shown in figure 2. (2) Randomly generating a series of two-dimensional code pattern images as a training set for training the image reconstruction model built in the step (1) to obtain optimal optical super-surface structure parameters and free parameters in a conjugate gradient optimization algorithm, after training, reserving the optimal optical super-surface structure and conjugate gradient algorithm parameters in the image reconstruction model to serve as a final image reconstruction experimental model, and inputting the multichannel images into the reconstruction experimental model in the step (3) to obtain a recovery image. The method uses the single-layer super-surface structure as the analog front end of the optical imaging system, does not need to introduce a traditional lens group and a filter, has the advantages of simple optical system, small size, portability and the like, and can accurately solve the problem of 'back scattering' by reconstructing a multi-channel image acquired by single exposure through a reconstruction algorithm, thereby improving the quality of the reconstructed image.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flowchart of a single exposure multispectral computed imaging method based on a hypersurface provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of an image reconstruction model provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of image reconstruction training provided by an embodiment of the present invention;
FIG. 4 is a schematic diagram of a single exposure multispectral computed imaging system based on a hypersurface provided by an embodiment of the invention;
FIG. 5 is a diagram of an optimized subsurface structure provided by an embodiment of the present invention;
FIG. 6 is a graph of the results of three-wavelength channel reconstruction performed in accordance with an embodiment of the present invention;
FIG. 7 is a graph of results of three-depth channel reconstruction performed in accordance with an embodiment of the present invention;
in the figure, 1, an image reconstruction model building module, 2, a point spread function recording module and 3, a sample image reconstruction module.
Detailed Description
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The invention may be embodied in many other forms than described herein and similarly modified by those skilled in the art without departing from the spirit or scope of the invention, which is therefore not limited to the specific embodiments disclosed below.
1. Explanation of the examples:
the single exposure multispectral calculation imaging method based on the super surface comprises the steps of designing a super surface optical system and optimizing a conjugate gradient reconstruction algorithm. And the optimized super-surface optical system performs one-time exposure on images of different channels, and collects the point spread functions PSF of the optical imaging systems of the different channels. In the image reconstruction (image restoration) stage, the original image is restored by using the conjugate gradient algorithm through the point spread functions PSF of different channels and imaging information on the sensor, so that the image restoration of a plurality of wave bands is finally obtained.
As shown in fig. 1, the method for single-exposure multispectral calculation imaging based on the super surface provided by the embodiment of the invention comprises the following steps:
S101, constructing an image reconstruction model by using a super-surface optical system and a conjugate gradient algorithm, and training the image reconstruction model by using a gradient descent method to obtain optimal super-surface structure parameters and conjugate gradient algorithm parameters, as shown in figure 2.
S102, collecting multi-channel images transmitted through the optimized optical super-surface by using an array sensor, and recording point spread functions PSF (point spreading function) of different channels.
S103, reconstructing a sample image by combining the optimized conjugate gradient algorithm with the point spread functions PSF of different channels.
Example 1
The single exposure multispectral calculation imaging method based on the super surface provided by the embodiment of the invention comprises the following steps:
(1) Using a single-layer-based super-surface structure, placing the super-surface-based image sensor in front of an array imaging sensor, and collecting images of different spectrum channels and optical point spread functions PSF (point spreading function) of corresponding channels through single exposure;
(2) Using the obtained exposure image and the point spread function PSF information in (1), imaged images of different channels are reconstructed based on a conjugate gradient method.
Example 2
The single exposure multispectral calculation imaging method based on the super surface provided by the embodiment of the invention comprises the following steps:
(1) And constructing an image reconstruction model by using the super-surface optical system and a conjugate gradient algorithm. As shown in fig. 2.
(2) Randomly generating a series of two-dimensional code images as a training set for training the image reconstruction model built in the step (1) to obtain optimal optical super-surface structure parameters and conjugate gradient algorithm parameters;
(3) After training, the optimal optical super-surface structure and conjugate gradient algorithm parameters in the image reconstruction model are reserved and used as a final image reconstruction experimental model;
(4) The multichannel image is input into the reconstruction model in (3) to obtain a restored image.
Example 3
Further, in the practical optimization design, the image v on the array sensor can be regarded as the convolution sum of the multiple channel input images u and the Point Spread Function (PSF) according to the single exposure multispectral calculation imaging method based on the super surface provided in the embodiment 2 of the present invention. Restoring images by computational imagingThe objective optimization function is
Where N is the number of channels (spectral channels or scene distance).
In a practical design, embodiments of the present invention use a supersurface as the optical device, see fig. 3 for a specific optimization concept. The method comprises the following steps:
first, using a super-surface as an optical imaging system front end, a Point Spread Function (PSF) of the super-surface is calculated:
PSF=|ntff(t(p)·Ein)|2 (2)
Where p is the structural parameter of the optical supersurface, t (p) is the optical supersurface transmission coefficient, E in is the incident electric field, ntff (·) represents the near field to far field conversion (the electric field conversion of the supersurface's transmission plane electric field to the array sensor surface).
Then, the image v on the array sensor can be obtained by convolution of the PSF and the real scene image u:
Reconstruction by solving the backscatter problem
The present invention herein uses Gihonofu regularization to process equation (4) to obtain
Where α is a constant to be optimized, and the convolution kernel operator is defined as
To solve equation (5), equation (3) is substituted into (5) and transformed to obtain
Here, a reconstructed image can be obtained by solving equation (6) using the conjugate gradient methodThus, the objective function L can be calculated (see equation (1)).
Equations (1) - (6) give a forward solution basic process of end-to-end design, and then the derivative of the objective function L with respect to the super-surface structure p and the undetermined coefficient α of the reconstruction algorithm needs to be calculated, so as to implement a back-propagation optimization process.
The method specifically comprises the following steps:
(1) Calculating derivatives
In conjunction with the derivative of equation (6), the above equation can be written as:
Wherein,
The above equation may be solved for Λ using a conjugate gradient method.
(2) Calculating derivatives
In conjunction with the derivative of equation (6), the above equation can be written as:
Equations (7) - (11) give the inverse solution process for the end-to-end design. After the derivative of the objective function with respect to the optimization parameters is obtained, the system can be optimized by combining gradient descent algorithms, such as a random gradient descent method, an Adam algorithm and the like. The updating method of parameters to be optimized is as follows:
Wherein P is a variable to be optimized (namely P and alpha), beta is a dynamic learning rate, and the specific value depends on the selected algorithm.
Example 4
As shown in fig. 4, an embodiment of the present invention provides a hyperspectral computed-imaging system for performing the hyperspectral computed-imaging method for single exposure based on a hyperspectral surface, the hyperspectral computed-imaging system for single exposure based on a hyperspectral surface includes:
the image reconstruction model building module 1 is used for building an image reconstruction model by utilizing a super-surface optical system and a conjugate gradient algorithm, and training the image reconstruction model by utilizing a gradient descent method to obtain optimal super-surface structure parameters and conjugate gradient algorithm parameters;
a point spread function recording module 2 for collecting multi-channel images transmitted through the optimized optical super surface by using an array sensor and recording point spread functions PSF (point spreading function) of different channels;
and the sample image reconstruction module 3 is used for reconstructing a sample image by combining the optimized conjugate gradient algorithm with the point spread functions PSF of different channels.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
The content of the information interaction and the execution process between the devices/units and the like is based on the same conception as the method embodiment of the present invention, and specific functions and technical effects brought by the content can be referred to in the method embodiment section, and will not be described herein.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
2. Application examples:
Application example 1
The application embodiment of the invention provides a computer device, which comprises at least one processor, a memory and a computer program stored in the memory and capable of running on the at least one processor, wherein the steps in any of the method embodiments are realized when the computer program is executed by the processor.
Application example 2
The application embodiment of the present invention also provides a computer readable storage medium, where a computer program is stored, where the computer program can implement the steps in the above method embodiments when executed by a processor.
Application example 3
The application embodiment of the invention also provides an information data processing terminal which is used for providing a user input interface to implement the steps in the method embodiments when being implemented on an electronic device, and the information data processing terminal is not limited to a mobile phone, a computer and a switch.
Application example 4
The application embodiment of the invention also provides a server, which is used for realizing the steps in the method embodiments when being executed on the electronic device and providing a user input interface.
Application example 5
Embodiments of the present invention provide a computer program product which, when run on an electronic device, causes the electronic device to perform the steps of the method embodiments described above.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium can include at least any entity or device capable of carrying computer program code to a camera device/terminal device, a recording medium, a computer memory, a Read-only memory (ROM), a random access memory (RandomAccessMemory, RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc.
3. Evidence of example related effects:
By utilizing the optimization method mentioned in the embodiment, the invention realizes multi-wavelength and multi-depth image reconstruction, and experimental results show that:
FIG. 5 shows the optimized subsurface structure, a 3000X 3000 array of cylinders each with a radius between 0.02,0.15 (units: microns).
Fig. 6 shows the results of three wavelength channel image reconstruction, wherein the first column and the third column are the real image and the reconstructed image, respectively, and the corresponding pixel resolutions are 128×128. The second column is the image imaged on the sensor. Each row represents a channel, from the first to the third row, corresponding to wavelengths of 53 nm, 5538 nm and 583nm, respectively, and peak signal to noise ratios of 31dB,33dB and 29dB. The corresponding object distance is 2cm and the image distance is 2mm.
Fig. 7 shows three depth channel image reconstruction results, in which the first and third columns are the real image and the reconstructed image, respectively, and the corresponding pixel resolutions are 128×128. The second column is the image imaged on the sensor. Each row represents a channel, from first to third row, corresponding to 100um,120um and 140um of object distance, respectively, and corresponding to 30dB,28dB and 27dB of peak signal-to-noise ratio. The corresponding wavelength was 533nm with an image distance of 2mm.
While the invention has been described with respect to what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications, equivalents, and alternatives falling within the spirit and scope of the invention.
Claims (7)
1. The hyperspectral computed imaging method based on the single exposure of the super surface is characterized by comprising the steps of performing one-time exposure on images of different channels by an optimized super surface optical system, and collecting point spread functions PSF of optical imaging systems of different channels;
in the image reconstruction stage, restoring an original image by utilizing a conjugate gradient algorithm through point spread functions PSF of different channels and imaging information on a sensor, and obtaining image restoration of a plurality of wave bands;
the hyperspectral computed imaging method based on the single exposure of the super surface specifically comprises the following steps:
s1, constructing an image reconstruction model by utilizing a super-surface optical system and a conjugate gradient algorithm;
s2, randomly generating a series of two-dimensional code images as a training set, and training an image reconstruction model built in the step S1 to obtain optimal optical super-surface structure parameters and conjugate gradient algorithm parameters;
S3, after training is completed, the optimal optical super-surface structure and conjugate gradient algorithm parameters in the image reconstruction model are reserved and used as a final image reconstruction experimental model;
s4, inputting the multichannel image into the image reconstruction model in the step S3 to obtain a restored image;
in step S1, an image reconstruction model is built, the image v on the array sensor is defined as a roll-up sum of a plurality of channel input images u and a Point Spread Function (PSF), and the image is restored by computing the imaging The objective optimization function is
Wherein N is the number of channels, spectral channels or scene distance;
In the step S2, obtaining an optimal optical super-surface structure, using a super-surface as an optical device, performing forward solution optimization, including the following steps:
1) Calculating a point spread function PSF of a subsurface using the subsurface as an optical imaging system front end
PSF=|ntff(t(p)·Ein)|2 (2);
Wherein, p is the structural parameter of the optical super surface, t (p) is the transmission coefficient of the optical super surface, E in is the incident electric field, ntff (·) represents the near field to far field conversion;
2) The image v on the array sensor is obtained by convolution of the point spread function PSF of the super surface and the real scene image u:
3) Reconstruction by solving the backscatter problem
Using Gihonofu regularization to process equation (4), we get
Where α is a constant to be optimized, and the convolution kernel operator is defined as
Substituting equation (3) into (5) and transforming to obtain
Solving equation (6) by conjugate gradient method to obtain reconstructed imageThe objective optimization function L is calculated using equation (1).
2. The hyperspectral computed tomography method of claim 1 wherein the calculation of the target optimization function L is further performed by calculating the derivative of the target optimization function L with respect to the structural parameter p of the optical hypersurface and the constant α to be optimized to achieve the back propagation optimization process.
3. The hypersurface based single exposure multispectral computed imaging method of claim 2, wherein the back propagation optimization process comprises the steps of:
(i) Calculating derivatives
In connection with the derivative of equation (6), the above equation is replaced by:
Wherein,
Solving the lambda by using a conjugate gradient method;
(ii) Calculating derivatives
In connection with the derivative of equation (6), the above equation is replaced by:
and after the derivative of the target optimization function relative to the optimization parameter is obtained, the system is optimized by combining a gradient descent algorithm.
4. The hypersurface based single exposure multispectral computed imaging method of claim 3, wherein optimizing the system in combination with a gradient descent algorithm comprises:
the updating method of parameters to be optimized comprises the following steps:
Where β is the dynamic learning rate, and the specific value depends on the chosen random gradient descent method and adam algorithm.
5. A hyperspectral computed tomography system for carrying out the hyperspectral computed tomography method for single exposure on a hyperspectral surface as claimed in any one of claims 1 to 4 wherein the hyperspectral computed tomography system for single exposure on a hyperspectral surface comprises:
The image reconstruction model building module (1) is used for building an image reconstruction model by utilizing a super-surface optical system and a conjugate gradient algorithm, and training the image reconstruction model by utilizing a gradient descent method to obtain optimal super-surface structure parameters and conjugate gradient algorithm parameters;
A point spread function recording module (2) for collecting multi-channel images transmitted through the optimized optical super surface by using an array sensor and recording point spread functions PSF (point spreading function) of different channels;
And the sample image reconstruction module (3) is used for reconstructing a sample image by combining the optimized conjugate gradient algorithm with the point spread functions PSF of different channels.
6. A computer device, characterized in that the computer device comprises a memory and a processor, the memory storing a computer program, which when executed by the processor, causes the processor to perform the hyperspectral computed tomography method based on a hyperspectral single exposure as claimed in any one of claims 1 to 4.
7. An all-optical imager, wherein the all-optical imager performs the single-exposure multispectral computed imaging method based on the super surface of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210847527.2A CN115204206B (en) | 2022-07-19 | 2022-07-19 | Single-exposure multispectral computational imaging method, system and device based on metasurface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210847527.2A CN115204206B (en) | 2022-07-19 | 2022-07-19 | Single-exposure multispectral computational imaging method, system and device based on metasurface |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115204206A CN115204206A (en) | 2022-10-18 |
| CN115204206B true CN115204206B (en) | 2025-04-18 |
Family
ID=83582464
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210847527.2A Active CN115204206B (en) | 2022-07-19 | 2022-07-19 | Single-exposure multispectral computational imaging method, system and device based on metasurface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115204206B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102360498A (en) * | 2011-10-27 | 2012-02-22 | 江苏省邮电规划设计院有限责任公司 | Reconstruction method for image super-resolution |
| CN107507135A (en) * | 2017-07-11 | 2017-12-22 | 天津大学 | Image reconstructing method based on coding aperture and target |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7331523B2 (en) * | 2001-07-13 | 2008-02-19 | Hand Held Products, Inc. | Adaptive optical image reader |
-
2022
- 2022-07-19 CN CN202210847527.2A patent/CN115204206B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102360498A (en) * | 2011-10-27 | 2012-02-22 | 江苏省邮电规划设计院有限责任公司 | Reconstruction method for image super-resolution |
| CN107507135A (en) * | 2017-07-11 | 2017-12-22 | 天津大学 | Image reconstructing method based on coding aperture and target |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115204206A (en) | 2022-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Baek et al. | Single-shot hyperspectral-depth imaging with learned diffractive optics | |
| Hirsch et al. | Online multi-frame blind deconvolution with super-resolution and saturation correction | |
| US6483952B2 (en) | Super resolution methods for electro-optical systems | |
| Zeng et al. | Robust reconstruction with deep learning to handle model mismatch in lensless imaging | |
| DeWeert et al. | Lensless coded-aperture imaging with separable Doubly-Toeplitz masks | |
| Ng et al. | Mathematical analysis of super-resolution methodology | |
| Li et al. | Hybrid 2-D–3-D deep residual attentional network with structure tensor constraints for spectral super-resolution of RGB images | |
| Sun et al. | Improving the performance of computational ghost imaging by using a quadrant detector and digital micro-scanning | |
| Peng et al. | Residual pixel attention network for spectral reconstruction from RGB images | |
| Okamoto et al. | Simultaneous acquisition of spectral and spatial intensity distribution | |
| CN115220211B (en) | Microscopic imaging system and method based on deep learning and light field imaging | |
| Oktem et al. | High-resolution multi-spectral imaging with diffractive lenses and learned reconstruction | |
| CN112672054A (en) | Focusing method and device and electronic equipment | |
| Blocker et al. | Low-rank plus sparse tensor models for light-field reconstruction from focal stack data | |
| Degraux et al. | Multispectral compressive imaging strategies using Fabry–Pérot filtered sensors | |
| Yang et al. | Wide-field, high-resolution reconstruction in computational multi-aperture miniscope using a Fourier neural network | |
| CN115204206B (en) | Single-exposure multispectral computational imaging method, system and device based on metasurface | |
| CN112702486A (en) | A high-speed coherent imaging camera and phase recovery method based on electrically tunable lens | |
| CN117456018A (en) | Spectral image reconstruction method and device | |
| Gao et al. | Filter-based landweber iterative method for reconstructing the light field | |
| CN119863536A (en) | On-board spectral video reconstruction method and system based on global spectral coding | |
| CN113538285B (en) | Method, device, terminal and storage medium for determining spectral image | |
| Gundogan et al. | Computational spectral imaging with diffractive lenses and spectral filter arrays | |
| Le Bon et al. | Joint Fourier Disparity Layers unrolling with learned view synthesis for light field reconstruction from few-shots focal stacks | |
| Nguyen et al. | Multi-mask camera model for compressed acquisition of light fields |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |