CN113822812A - Image noise reduction method and electronic device - Google Patents
Image noise reduction method and electronic device Download PDFInfo
- Publication number
- CN113822812A CN113822812A CN202111081362.4A CN202111081362A CN113822812A CN 113822812 A CN113822812 A CN 113822812A CN 202111081362 A CN202111081362 A CN 202111081362A CN 113822812 A CN113822812 A CN 113822812A
- Authority
- CN
- China
- Prior art keywords
- image
- images
- fixed pattern
- noise
- pattern noise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Studio Devices (AREA)
Abstract
The application discloses an image noise reduction method and electronic equipment, and belongs to the field of image processing. The image noise reduction method comprises the following steps: acquiring a plurality of images acquired according to different exposure durations; calculating fixed pattern noise according to the multiple images and the exposure duration corresponding to each image; the acquired image is denoised by fixed pattern noise.
Description
Technical Field
The application belongs to the field of image processing, and particularly relates to an image denoising method and electronic equipment.
Background
In the night or dark environment, the imaging signal-to-noise ratio is low when the video and the image are shot, and the imaging definition, the visual perception and the image color are seriously influenced. The Noise in the image includes time-dependent temporal Noise, and time-independent Fixed Pattern Noise (FPN). The magnitude of the fixed mode noise is fixed, the imaging influence is small when the ambient illumination is high, but as the illumination is reduced, the signal is weakened, and the fixed mode noise ratio is gradually improved, so that the fixed mode noise becomes an important factor influencing the image quality of the night scene video. The fixed mode noise is noise brought by a sensor, is airspace noise, cannot be removed through time domain noise reduction, needs to be removed through airspace noise reduction or after calibration in advance, needs to calibrate a camera when leaving a factory, cannot be used for a camera which is not calibrated before leaving the factory, and is not enough in flexibility.
Disclosure of Invention
The embodiment of the application aims to provide an image noise reduction method and electronic equipment, and the problem that the fixed mode noise of an uncalibrated camera cannot be determined in the related art can be solved.
In a first aspect, an embodiment of the present application provides an image denoising method, including:
acquiring a plurality of images acquired according to different exposure durations;
calculating the fixed mode noise of the optical sensor according to the multiple images and the exposure duration corresponding to each image;
and reducing the noise of the image acquired by the optical sensor through fixed pattern noise.
In a second aspect, an embodiment of the present application provides an image noise reduction apparatus, including:
the first acquisition unit is used for acquiring a plurality of images acquired according to different exposure durations;
the computing unit is used for computing the fixed mode noise of the optical sensor according to the multiple images and the exposure duration corresponding to each image;
and the noise reduction unit is used for reducing noise of the image acquired by the optical sensor through fixed mode noise.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor, and when executed by the processor, the program or instructions implement the steps of the method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a readable storage medium, on which a program or instructions are stored, which when executed by a processor implement the steps of the method according to the first aspect.
In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the method according to the first aspect.
In the embodiment of the application, the images acquired at different exposure durations are acquired, and the fixed pattern noise is irrelevant to the light intensity, so that the different exposure durations do not influence the fixed pattern noise, and based on the fixed pattern noise, the fixed pattern noise in the images can be calculated, and then the images or videos shot by the subsequent cameras can be denoised through the calculated fixed pattern noise without calibrating the fixed pattern noise of the cameras in advance, so that the problem that the fixed pattern noise of the uncalibrated cameras cannot be determined in the related technology is solved.
Drawings
FIG. 1 is a schematic flow chart of an image denoising method according to an embodiment of the present application;
FIG. 2 is a block diagram illustrating a schematic structure of an image denoising method according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a neural network structure in an image denoising method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a neural network structure in another image denoising method according to an embodiment of the present application;
FIG. 5 is a block diagram illustrating a schematic structure of another image denoising method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a neural network structure in another image denoising method according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a neural network structure in another image denoising method according to an embodiment of the present application;
fig. 8 is a block diagram of an image noise reduction apparatus according to an embodiment of the present application;
fig. 9 is a block diagram of an electronic device according to an embodiment of the present application;
fig. 10 is a block diagram of a further electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present disclosure.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.
The image denoising method provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.
As shown in FIG. 1, the image denoising method provided by the embodiment of the application comprises the following steps 201-203:
The acquired image may be acquired by a light sensor. The light sensor is a sensor capable of sensing light intensity, and may be a differential sensor or an integral sensor, including but not limited to a Complementary Metal Oxide Semiconductor (CMOS) photosensitive element, a Charge-coupled device (CCD), a pulse sensor, and the like.
The optical sensor is a light sensing device of a camera, and the optical sensor senses light to generate an electrical signal. The electrical signals produced by the light sensors may generate an image. The generated image format may be an original image file Raw, for example.
The exposure time period is the time during which the shutter of the camera is open, the longer the open time period, the more light is sensed by the light sensor. The exposure time of each captured image can be controlled, in one example, the mobile phone can control the shutter opening time of the camera through the processor chip, and after shooting is performed for multiple times by using different exposure time lengths, multiple images are obtained.
In an alternative embodiment, the other parameters of the light sensor are unchanged except for the exposure time period; in another alternative embodiment, the analog gain of the photosensor can be varied in addition to the exposure time period. The analog gain is an amplification gain of the circuit of the optical sensor on the acquired electrical signal, and the analog gain can be controlled. The value of the analog gain may be recorded as each image is acquired, so that the analog gain may be used to directly remove the gain from the acquired image to remove the effect of the analog gain on the image pixel values.
Since the fixed pattern noise is independent of the light intensity and is a characteristic of the light sensor, the fixed pattern noise can be regarded as a constant in a plurality of images under different exposure time periods, and the change is the change of the image pixel value caused by the difference of the exposure time periods. Thus, the fixed pattern noise in the image can be estimated from the plurality of images and the known exposure time.
In one example, the fixed pattern noise may be solved by means of a system of column equations. Specifically, assume that an image including only fixed pattern noise is N, and the exposure time period is tkTime-acquired image KkK is 1, …, n, and X is assumed to be an ideal image from which the effect of fixed pattern noise is removedkThen, Xk+N＝KkWherein, additionally, there is X1:…:Xk:…:Xn＝t1:…:tk:…:tnThus, can be based on the known KkAnd tkSolve to Xk+ N, resulting in an image N of fixed pattern noise.
In another example, the fixed pattern noise may also be estimated by training a neural network model through which the fixed pattern noise is estimated.
Alternatively, in step 201, only one image may be acquired each time an image is acquired using one exposure duration, or a plurality of images may be acquired. When a plurality of images are acquired by using one exposure time length, an average image can be obtained for the plurality of images to obtain an average image corresponding to the exposure time length. Since the fixed pattern noise in the average image is an average of a plurality of images, the average image can carry more accurate fixed pattern noise.
Correspondingly, when the step 201 is executed to acquire a plurality of images acquired according to different exposure time lengths, the following steps 2011 to 2012 can be executed:
step 2011, acquiring m images acquired according to n different exposure durations, wherein m is larger than or equal to n, and at least one image is acquired in each exposure duration;
step 2012, generating an average image corresponding to the exposure duration according to the at least one image acquired for each exposure duration to obtain n average images.
Thus, when the fixed pattern noise is calculated according to the exposure time lengths corresponding to the plurality of images and each image in step 202, the fixed pattern noise may be calculated according to the exposure time lengths corresponding to the n average images and each average image.
The fixed pattern noise is additive noise, and after an image is subsequently acquired through the optical sensor, the fixed pattern noise can be directly subtracted from the acquired image, so that the image with the fixed pattern noise removed can be obtained.
In the embodiment of the present application, the architecture for implementing step 201 to step 203 may be as shown in fig. 2, and includes a sensor 102, a sensor control module 101, a fixed pattern noise estimation module 103, and a video image enhancement module 104.
The sensor 102, i.e. the light sensor described above, is used to capture an image. Taking the application of the image denoising method in the embodiment of the present application to a scene of a mobile phone as an example, the sensor 102 may be a sensor module of a camera in the mobile phone.
The sensor control module 101 may be a module for controlling the sensor 102 in the mobile phone, and for example, the sensor control module 101 may include a program or instructions stored in a memory of the mobile phone for controlling the sensor 102, and a hardware circuit portion of the mobile phone chip capable of executing the program or instructions.
The plurality of images acquired in step 201 may be a plurality of images acquired by controlling the sensor 102 through the sensor control module 101 at different exposure time periods. Alternatively, step 201 may be performed by a processor in the mobile phone, and after the sensor control module 101 acquires the image, the processor may acquire the acquired image.
The fixed pattern noise estimation module 103 is configured to calculate noise, and may execute step 202 to estimate and calculate the fixed pattern noise according to a plurality of images. Alternatively, the fixed pattern noise estimation module 103 may include a program or instructions stored in memory for estimating the fixed pattern noise.
The video image enhancement module 104 is used for denoising the acquired image. The fixed pattern noise calculated by the fixed pattern noise estimation module 103 may be sent to the video image enhancement module 104, so that the video image enhancement module 104 can perform noise reduction at least for the fixed pattern noise in the image. Optionally, the video image enhancement module 104 may also perform noise reduction on other noises, which is not limited in this embodiment. The video image enhancement module 104 may include a program or instructions stored in memory for removing fixed pattern noise.
After the fixed pattern noise estimation module 103 estimates the fixed pattern noise, the image acquired by the sensor 102 may be subjected to noise reduction processing on the fixed pattern noise in the image through the video image enhancement module 104, so as to improve the imaging quality of the video or the image under low illumination.
In the embodiment of the application, the images acquired at different exposure durations are acquired, and the fixed pattern noise is irrelevant to the light intensity, so that the different exposure durations do not influence the fixed pattern noise, and based on the fixed pattern noise, the fixed pattern noise in the images can be calculated, and then the images or videos shot by the subsequent cameras can be denoised through the calculated fixed pattern noise without calibrating the fixed pattern noise of the cameras in advance, so that the problem that the fixed pattern noise of the uncalibrated cameras cannot be determined in the related technology is solved. The image denoising method provided by the embodiment of the application can calculate the fixed mode noise only by collecting a few images, the processing time delay is low, and the quality of the subsequent shooting video or image can be improved without calibrating in advance.
In the embodiment of the present application, when step 202 is executed to calculate the fixed pattern noise according to the multiple images and the exposure duration corresponding to each image, an optional implementation manner is provided, and the fixed pattern noise may be calculated according to the following relationship:
relation 1: and the actual pixel value of each pixel point of the kth image in the plurality of images is equal to the ideal pixel value of the corresponding pixel point in the kth image plus the fixed pattern noise of the corresponding pixel point.
The k-th image is an image corresponding to the k-th exposure time period. Alternatively, in performing step 201, if multiple images are acquired for the k-th exposure time period, the k-th image is an average image of the k-th exposure time period.
The actual pixel value is the pixel value in the acquired image, and the ideal pixel value is the ideal value for removing the fixed pattern noise in the image. The actual pixel values are known, the ideal pixel values are unknown, and the fixed pattern noise is also unknown.
The fixed pattern noise of different images at the same pixel point is the same, and the same pixel point is the pixel point with the same position in different images. That is, for any two different images, the same coordinate position in the image is selected, and the fixed pattern noise of the pixel point is the same for the different images.
Relation 2: the ratio of ideal pixel values of the same pixel points in any two images is the ratio of the corresponding exposure time.
Because additive fixed pattern noise is superposed in the actual pixel value, the actual pixel value cannot be completely in direct proportion to the exposure time, and the ideal pixel value after the fixed pattern noise is removed is in direct proportion to the exposure time, so that for the pixel points at the same position, the ratio of the ideal pixel values of any two images is equal to the ratio of the exposure time of the corresponding images.
Therefore, based on the known conditions, the ideal pixel value of each pixel point of each image and the fixed pattern noise of the corresponding pixel point can be solved.
By way of example, based on the above relations 1 and 2, the following set of equations may be listed:
in the above formula, the first and second carbon atoms are,
the ideal pixel value of the pixel point with the coordinate positioned in (i, j) in the kth image is unknown;
the actual pixel value of the pixel point with the coordinate located at (i, j) in the kth image is known; n is a radical ofi,jThe fixed pattern noise of the pixel point with the coordinate located in (i, j) in any image is unknown; t is tkIs the exposure time period of the k-th image, is known. Solving the equation can obtain the ideal pixel value of each pixel point in each image and the fixed mode noise of the corresponding pixel point.
Further, when step 203 is executed to perform noise reduction on the acquired image by using fixed pattern noise, the following steps may be executed:
step 2031, combining the fixed pattern noise of each pixel point to obtain a fixed pattern noise image;
and step 2032, subtracting the fixed mode noise image from the acquired image to obtain a noise reduction image with the fixed mode noise removed.
That is, the fixed pattern noise N of each pixel pointi,jAnd combining the corresponding pixel positions into a matrix to obtain the fixed analog noise image N. Then for the subsequently acquired image P, the image P-N after the fixed analog noise is removed can be calculated.
Alternatively, if the analog gain of the subsequently acquired image is different from the analog gain used in acquiring the fixed analog noise image N, then the analog gain at the time of acquiring the image may be used to add the effect of the analog gain to the fixed analog noise image. For example, if the analog gain used in acquiring image P is r, the image after removing the fixed analog noise is calculated to be P-r × N.
Alternatively, if the analog gain is changed for each image in addition to the exposure time period at the time of acquisition, the gain may be removed before acquiring a plurality of images acquired according to different exposure time periods.
Specifically, before the step 201 is executed to acquire a plurality of images acquired according to different exposure durations, the analog gain when each image is acquired, and further, after the step 201 is executed to acquire a plurality of images acquired according to different exposure durations, the gain effect of the corresponding image is removed according to the analog gain of each image.
In removing the gain effect, the image may be divided by the parameter value of the analog gain. For example, for the k-th image, the image after the gain effect is removed is
akThe analog gain corresponding to the k-th image. After the gain effect is removed, the influence of the gain effect on the pixel value in the image can be removed, so that other parameters are the same except for different exposure time lengths, and thus, the fixed pattern noise can be estimated more accurately.
In another alternative embodiment, when step 202 is executed to calculate the fixed pattern noise according to the plurality of images and the exposure time duration corresponding to each image, the fixed pattern noise may be estimated by the trained first neural network model. The first neural network model may be trained through the training sample image group with the fixed pattern noise calibrated, which is not described herein again.
The structure of the first neural network model may be as shown in fig. 3 or fig. 4. As shown in FIG. 3, the input to the first neural network model is a plurality of images, I1～InAnd packaging and inputting the image into the model at one time, wherein the output is the fixed mode noise image N. As shown in FIG. 4, the first spiritThe input is an image through the network model, and the neural network model processes one frame of image I at a timetAnd inputting the t-th input image ItNetwork characteristics of, or network output image OuttReturned and next frame image It+1Processed together, the nth output Out of the neural networknI.e. the final fixed pattern noise image N.
Alternatively, in addition to the fixed pattern noise estimated by the first neural network model, other noise than the fixed pattern noise may be estimated by a trained second neural network model, wherein the first and second neural network models have the same structure but different model parameters. In this way, when step 203 is executed to perform noise reduction on the acquired image by the fixed pattern noise, the noise reduction on the acquired image by the fixed pattern noise and other noise may also be included.
Accordingly, for the architecture shown in fig. 2, if the video image enhancement module 104 includes removing other noise in addition to the fixed pattern noise, then the fixed pattern noise estimation module 103 and the video image enhancement module 104 are incorporated together into one large module, as shown in fig. 5, and the video noise reduction module 105 includes the fixed pattern noise estimation module 103 and the video image enhancement module 104. In this way, the fixed pattern noise estimation module 103 and the video image enhancement module 104 are incorporated into the video denoising module 105, and the fixed pattern noise estimation module 103 and the video image enhancement module 104 use neural network models with the same structure, so that they can be disposed on the same processor in the electronic device, and do not need to interact between different processors, so that the fixed pattern noise estimation module 103 and the video image enhancement module 104 do not occupy additional on-chip interaction communication resources.
The video noise reduction module 105 may use a model having the same structure as the first neural network model, i.e., a second neural network model. The second neural network model may be trained using the training sample image group with the corresponding noise calibrated, which is not described herein again.
The structure of the second neural network model and the structure of the first neural network model mayMay be different or the same. Taking the embodiment with the same structure as an example, the parameters of the first neural network model and the second neural network model are different, as shown in fig. 6 and 7, fig. 6 is the structure of the first neural network model, and fig. 7 is the structure of the second neural network model, it can be seen that the structures are the same, but the parameters of the models are different, and the parameters of the first neural network model can be expressed as (w)fpn,bfpn) The parameters of the second neural network model may be noted as (w)vdn,bvdn). Wherein, wfpnAnd wvdnRepresenting weight parameters in a neural network, bfpnAnd bvdnInitial value parameters in the neural network may be represented. The above parameters are merely used for exemplary illustration of parameters in the neural network model, and are not used for constituting a limitation of the present application.
Optionally, the image noise reduction method provided in the embodiment of the present application may be executed by default when an image or a video is captured each time, or may automatically select whether to execute the image noise reduction method according to actual scene illuminance, for example, the image noise reduction method may be executed in a scene with illuminance lower than a certain threshold.
In one embodiment, the fixed pattern noise may be stored in a buffer or a memory of the electronic device after being estimated once, so that the stored fixed pattern noise may be used for noise reduction when videos and images are taken subsequently.
It should be noted that, in the image noise reduction method provided in the embodiment of the present application, the execution subject may be an image noise reduction device, or a control module in the image noise reduction device for executing the image noise reduction method. The embodiment of the present application takes an image noise reduction device as an example to execute an image noise reduction method, and illustrates the image noise reduction device provided in the embodiment of the present application.
As shown in fig. 8, the image noise reduction apparatus provided in the embodiment of the present application includes a first obtaining unit 11, a calculating unit 12, and a noise reduction unit 13.
The first acquiring unit 11 is configured to acquire a plurality of images acquired according to different exposure durations;
the calculation unit 12 is configured to calculate fixed pattern noise according to the multiple images and the exposure duration corresponding to each image;
the noise reduction unit 13 is configured to reduce noise of the acquired image by fixed pattern noise.
In the embodiment of the application, the images acquired at different exposure durations are acquired, and the fixed pattern noise is irrelevant to the light intensity, so that the different exposure durations do not influence the fixed pattern noise, and based on the fixed pattern noise, the fixed pattern noise in the images can be calculated, and then the images or videos shot by the subsequent cameras can be denoised through the calculated fixed pattern noise without calibrating the fixed pattern noise of the cameras in advance, so that the problem that the fixed pattern noise of the uncalibrated cameras cannot be determined in the related technology is solved.
Alternatively, the calculation unit 12 may include:
a first calculation subunit configured to calculate the fixed pattern noise by a relationship:
the actual pixel value of each pixel point of the kth image in the multiple images is equal to the ideal pixel value of the corresponding pixel point in the kth image plus the fixed pattern noise of the corresponding pixel point; wherein, the fixed pattern noise of different images in the multiple images at the same pixel point is the same;
the ratio of ideal pixel values of the same pixel points in any two images is the ratio of the corresponding exposure time.
Further, the noise reduction unit 13 may include:
the combination subunit is used for combining the fixed mode noise of each pixel point to obtain a fixed mode noise image;
and the second calculating subunit is used for subtracting the fixed mode noise image from the acquired image to obtain a noise reduction image with the fixed mode noise removed.
Optionally, the apparatus may further include:
the second acquisition unit is used for acquiring the analog gain when each image is acquired before acquiring a plurality of images acquired according to different exposure durations;
and the removing gain unit is used for dividing each image by the analog gain after acquiring a plurality of images acquired according to different exposure durations to obtain the image with the gain effect removed corresponding to the image.
In an alternative embodiment, the first obtaining unit 11 may include:
the acquisition subunit is used for acquiring m images acquired according to n different exposure durations, wherein m is larger than or equal to n, and at least one image is acquired in each exposure duration;
the generating subunit is used for generating an average image corresponding to the exposure duration according to at least one image acquired by each exposure duration to obtain n average images;
accordingly, the calculation unit 12 may include:
and the third calculating subunit is used for calculating the fixed pattern noise according to the n average images and the exposure duration corresponding to each average image.
Alternatively, the calculation unit 12 may include:
an estimating subunit, configured to estimate a fixed pattern noise through the trained first neural network model;
accordingly, the apparatus may further include:
the estimation unit is used for estimating other noises except the fixed pattern noise through a trained second neural network model, wherein the first neural network model and the second neural network model have the same structure but different model parameters;
in addition, the noise reduction unit 13 is also used to reduce noise of the acquired image through fixed pattern noise and other noise.
The image noise reduction device in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device can be mobile electronic equipment or non-mobile electronic equipment. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and the non-mobile electronic device may be a server, a Network Attached Storage (NAS), a Personal Computer (PC), a television (televkskon, TV), a teller machine, a self-service machine, and the like, and the embodiments of the present application are not particularly limited.
The image noise reduction device in the embodiment of the present application may be a device having an operating system. The operating system may be an android (android) operating system, may be a kos operating system, and may also be another possible operating system, which is not specifically limited in the embodiment of the present application.
The image noise reduction device provided in the embodiment of the present application can implement each process implemented by the method embodiments of fig. 1 to fig. 7, and is not described herein again to avoid repetition.
Optionally, as shown in fig. 9, an electronic device 900 is further provided in this embodiment of the present application, and includes a processor 901, a memory 902, and a program or an instruction stored in the memory 902 and executable on the processor 901, where the program or the instruction is executed by the processor 901 to implement each process of the above-mentioned embodiment of the image denoising method, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.
Fig. 10 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
The electronic device 100 includes, but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010.
Those skilled in the art will appreciate that the electronic device 100 may further comprise a power source (e.g., a battery) for supplying power to various components, and the power source may be logically connected to the processor 1010 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The electronic device structure shown in fig. 10 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is not repeated here.
Wherein the processor 1010 is configured to perform the following steps:
acquiring a plurality of images acquired according to different exposure durations;
calculating fixed pattern noise according to the multiple images and the exposure duration corresponding to each image;
the acquired image is denoised by fixed pattern noise.
Alternatively, when the processor 1010 executes the step of calculating the fixed pattern noise according to the plurality of images and the exposure time corresponding to each image, the method may include calculating the fixed pattern noise by using the following relationship:
the actual pixel value of each pixel point of the kth image in the multiple images is equal to the ideal pixel value of the corresponding pixel point in the kth image plus the fixed pattern noise of the corresponding pixel point; wherein, the fixed pattern noise of different images in the multiple images at the same pixel point is the same;
the ratio of ideal pixel values of the same pixel points in any two images is the ratio of the corresponding exposure time.
Further, in an alternative embodiment, when performing denoising of the acquired image with fixed pattern noise, the processor 1010 may include performing the following steps:
combining the fixed mode noise of each pixel point to obtain a fixed mode noise image;
and subtracting the fixed mode noise image from the acquired image to obtain a noise reduction image with the fixed mode noise removed.
Optionally, before performing acquiring the plurality of images acquired according to different exposure time periods, the processor 1010 further performs the following steps:
acquiring the analog gain when each image is acquired;
after the processor 1010 executes the acquisition of the plurality of images acquired according to different exposure time lengths, the following steps are further executed:
and dividing each image by the analog gain to obtain the image with the gain effect removed from the corresponding image.
Optionally, when the processor 1010 performs acquiring a plurality of images acquired according to different exposure time periods, the following steps may be performed:
acquiring m images acquired according to n different exposure durations, wherein m is more than or equal to n, and each exposure duration acquires at least one image;
generating an average image corresponding to the exposure duration according to at least one image acquired by each exposure duration to obtain n average images;
accordingly, when the processor 1010 performs the calculation of the fixed pattern noise according to the plurality of images and the exposure time corresponding to each image, the following steps may be performed:
and calculating the fixed pattern noise according to the n average images and the exposure duration corresponding to each average image.
In yet another alternative embodiment, when the processor 1010 performs the calculating of the fixed pattern noise according to the plurality of images and the exposure time duration corresponding to each image, the method may include performing:
estimating fixed pattern noise through the trained first neural network model;
further, the processor 1010 is configured to perform the following steps:
estimating other noises except fixed pattern noises through a trained second neural network model, wherein the first neural network model and the second neural network model have the same structure but different model parameters;
accordingly, the processor 1010, when performing denoising of the acquired image by fixed pattern noise, may include performing the steps of:
the captured image is denoised by fixed pattern noise and other noise.
In the embodiment of the application, the images acquired at different exposure durations are acquired, and the fixed pattern noise is irrelevant to the light intensity, so that the different exposure durations do not influence the fixed pattern noise, and based on the fixed pattern noise, the fixed pattern noise in the images can be calculated, and then the images or videos shot by the subsequent cameras can be denoised through the calculated fixed pattern noise without calibrating the fixed pattern noise of the cameras in advance, so that the problem that the fixed pattern noise of the uncalibrated cameras cannot be determined in the related technology is solved.
It should be understood that in the embodiment of the present application, the input unit 1004 may include a Graphics Processor (GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes image data of a still picture or a video obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes a touch panel 10071 and other input devices 10072. The touch panel 10071 is also referred to as a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein. The memory 1009 may be used to store software programs as well as various data, including but not limited to application programs and operating systems. Processor 1010 may integrate an application processor that handles primarily operating systems, user interfaces, applications, etc. and a modem processor that handles primarily wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 1010.
The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or the instruction is executed by a processor, the program or the instruction implements each process of the above image denoising method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and so on.
The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement each process of the above embodiment of the image denoising method, and can achieve the same technical effect, and the details are not repeated here to avoid repetition.
It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as system-on-chip, system-on-chip or system-on-chip, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. An image noise reduction method, comprising:
acquiring a plurality of images acquired according to different exposure durations;
calculating fixed pattern noise according to the plurality of images and the exposure duration corresponding to each image;
and denoising the acquired image through the fixed pattern noise.
2. The method of claim 1, wherein the calculating the fixed pattern noise according to the exposure durations corresponding to the plurality of images and each image comprises:
the fixed pattern noise is calculated by the following relationship:
the actual pixel value of each pixel point of the kth image in the multiple images is equal to the ideal pixel value of the corresponding pixel point in the kth image plus the fixed pattern noise of the corresponding pixel point; wherein the fixed pattern noise of different images in the plurality of images at the same pixel point is the same;
the ratio of ideal pixel values of the same pixel points in any two images is the ratio of the corresponding exposure time.
3. The method of claim 2, wherein the denoising the captured image with the fixed pattern noise comprises:
combining the fixed mode noise of each pixel point to obtain a fixed mode noise image;
and subtracting the fixed mode noise image from the acquired image to obtain a noise reduction image with the fixed mode noise removed.
4. The image noise reduction method according to claim 1, further comprising, before acquiring the plurality of images acquired according to different exposure durations:
acquiring the analog gain when each image is acquired;
after acquiring a plurality of images acquired according to different exposure durations, the method further comprises the following steps:
and dividing each image by the analog gain to obtain an image with the gain effect removed from the corresponding image.
5. The method of claim 1, wherein the obtaining a plurality of images collected according to different exposure durations comprises:
acquiring m images acquired according to n different exposure durations, wherein m is more than or equal to n, and each exposure duration acquires at least one image;
generating an average image corresponding to the exposure duration according to at least one image acquired by each exposure duration to obtain n average images;
the calculating the fixed pattern noise according to the plurality of images and the exposure duration corresponding to each image comprises:
and calculating the noise of the fixed mode according to the n average images and the exposure duration corresponding to each average image.
6. The method of claim 1, wherein the calculating the fixed pattern noise according to the exposure durations corresponding to the plurality of images and each image comprises:
estimating fixed pattern noise through the trained first neural network model;
the method further comprises the following steps:
estimating other noise except the fixed pattern noise through a trained second neural network model, wherein the first neural network model and the second neural network model have the same structure but different model parameters;
the denoising the acquired image by the fixed pattern noise comprises:
and denoising the acquired image through the fixed pattern noise and the other noises.
7. An image noise reduction apparatus, comprising:
the first acquisition unit is used for acquiring a plurality of images acquired according to different exposure durations;
the calculation unit is used for calculating the fixed pattern noise according to the plurality of images and the exposure duration corresponding to each image;
and the noise reduction unit is used for reducing the noise of the acquired image through the fixed mode noise.
8. The image noise reduction device according to claim 7, wherein the calculation unit includes:
a first calculating subunit configured to calculate the fixed pattern noise by a relationship:
the actual pixel value of each pixel point of the kth image in the multiple images is equal to the ideal pixel value of the corresponding pixel point in the kth image plus the fixed pattern noise of the corresponding pixel point; wherein the fixed pattern noise of different images in the plurality of images at the same pixel point is the same;
the ratio of ideal pixel values of the same pixel points in any two images is the ratio of the corresponding exposure time.
9. An electronic device comprising a processor, a memory, and a program or instructions stored on the memory and executable on the processor, the program or instructions when executed by the processor implementing the steps of the image noise reduction method according to any one of claims 1 to 6.
10. A readable storage medium, on which a program or instructions are stored, which when executed by a processor, carry out the steps of the image noise reduction method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111081362.4A CN113822812B (en) | 2021-09-15 | 2021-09-15 | Image noise reduction method and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111081362.4A CN113822812B (en) | 2021-09-15 | 2021-09-15 | Image noise reduction method and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113822812A true CN113822812A (en) | 2021-12-21 |
| CN113822812B CN113822812B (en) | 2024-09-06 |
Family
ID=78914637
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111081362.4A Active CN113822812B (en) | 2021-09-15 | 2021-09-15 | Image noise reduction method and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113822812B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114418873A (en) * | 2021-12-29 | 2022-04-29 | 英特灵达信息技术(深圳)有限公司 | Dark light image noise reduction method and device |
| CN116721442A (en) * | 2023-05-19 | 2023-09-08 | 天津极豪科技有限公司 | Fingerprint collection image processing methods, electronic devices and storage media |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100237149A1 (en) * | 2009-03-17 | 2010-09-23 | Datalogic Scanning, Inc. | Systems and methods for compensating for fixed pattern noise |
| JP2011019155A (en) * | 2009-07-10 | 2011-01-27 | Hoya Corp | Fixed pattern noise removing unit, imaging unit, and electronic endoscope system |
| CN102055916A (en) * | 2009-11-03 | 2011-05-11 | 三星电子株式会社 | Methods of modeling an integrated noise in an image sensor and methods of reducing noise using the same |
| US20170154409A1 (en) * | 2015-12-01 | 2017-06-01 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| CN109889821A (en) * | 2019-03-19 | 2019-06-14 | 北京航空航天大学 | An AR Model-Based Column Fixed Pattern Noise Evaluation Method for CMOS Image Sensors |
| CN110574363A (en) * | 2017-11-22 | 2019-12-13 | 深圳市大疆创新科技有限公司 | Image noise calibration method and device, image noise reduction method and device, and image processing device |
-
2021
- 2021-09-15 CN CN202111081362.4A patent/CN113822812B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100237149A1 (en) * | 2009-03-17 | 2010-09-23 | Datalogic Scanning, Inc. | Systems and methods for compensating for fixed pattern noise |
| JP2011019155A (en) * | 2009-07-10 | 2011-01-27 | Hoya Corp | Fixed pattern noise removing unit, imaging unit, and electronic endoscope system |
| CN102055916A (en) * | 2009-11-03 | 2011-05-11 | 三星电子株式会社 | Methods of modeling an integrated noise in an image sensor and methods of reducing noise using the same |
| US20170154409A1 (en) * | 2015-12-01 | 2017-06-01 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| CN110574363A (en) * | 2017-11-22 | 2019-12-13 | 深圳市大疆创新科技有限公司 | Image noise calibration method and device, image noise reduction method and device, and image processing device |
| CN109889821A (en) * | 2019-03-19 | 2019-06-14 | 北京航空航天大学 | An AR Model-Based Column Fixed Pattern Noise Evaluation Method for CMOS Image Sensors |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114418873A (en) * | 2021-12-29 | 2022-04-29 | 英特灵达信息技术(深圳)有限公司 | Dark light image noise reduction method and device |
| CN114418873B (en) * | 2021-12-29 | 2022-12-20 | 英特灵达信息技术(深圳)有限公司 | Dark light image noise reduction method and device |
| CN116721442A (en) * | 2023-05-19 | 2023-09-08 | 天津极豪科技有限公司 | Fingerprint collection image processing methods, electronic devices and storage media |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113822812B (en) | 2024-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wei et al. | Physics-based noise modeling for extreme low-light photography | |
| CN111968066A (en) | Infrared image correction method, device and equipment and refrigeration infrared imaging system | |
| CN113538223B (en) | Noise image generation method, device, electronic device and storage medium | |
| WO2019071613A1 (en) | Image processing method and device | |
| CN110519485B (en) | Image processing method, device, storage medium and electronic device | |
| CN113099122A (en) | Shooting method, shooting device, shooting equipment and storage medium | |
| KR20110048922A (en) | Integrated noise modeling method of image sensor and noise reduction method using same | |
| CN112887629B (en) | Frequency detection method, frequency detection device, electronic equipment and storage medium | |
| WO2015142530A1 (en) | System for auto-hdr capture decision making | |
| CN113706402B (en) | Neural network training method, device and electronic equipment | |
| CN112734659A (en) | Image correction method and device and electronic equipment | |
| CN113163128A (en) | Image exposure method and electronic device | |
| CN114782280A (en) | Image processing method and device | |
| CN112422798A (en) | Photographing method and device, electronic equipment and storage medium | |
| CN113822812B (en) | Image noise reduction method and electronic device | |
| CN113592754A (en) | Image generation method and electronic equipment | |
| CN115278047A (en) | Shooting method, shooting device, electronic equipment and storage medium | |
| CN113132695A (en) | Lens shadow correction method and device and electronic equipment | |
| KR20090025385A (en) | Dark frame subtraction methods, computer program products and electronic devices | |
| CN113658118B (en) | Image noise level estimation method, device, electronic device and storage medium | |
| US8964055B2 (en) | Combining images based on position offset detection of a series of images | |
| CN112468794B (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN112508820A (en) | Image processing method and device and electronic equipment | |
| CN112367470B (en) | Image processing method and device and electronic equipment | |
| CN111951312B (en) | Image registration, image acquisition time registration method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |