CN114601437A - Heart rate detection method and device - Google Patents

Abstract

The embodiment of the present application discloses a heart rate detection method and device. The method includes: collecting a fingerprint image sequence; determining an initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of the fingerprint image in the fingerprint image sequence ; Perform time-frequency domain analysis on the initial signal sequence to obtain the heart rate. Through the solution of this embodiment, the heart rate detection is realized on the premise of the existing fingerprint identification optical imaging system, so that the detection accuracy is not easily disturbed by noise, and the application scenarios are broadened.

Description

Heart rate detection method and device

technical field

The embodiments of the present application relate to a heart rate detection technology, and in particular, to a heart rate detection method and device.

Background technique

Electrocardiography (ECG) is a technique that uses an electrocardiograph to record from the body surface the pattern of point activity changes produced by the heart during each cardiac cycle. Although this method has high accuracy, the instrument is expensive and requires professional operation, the equipment is cumbersome, and the usage scenarios are extremely limited.

Photoplethysmography (ppg) is a non-invasive detection method that monitors blood volume changes in living tissue by means of photoelectric means. When light of a certain wavelength is irradiated on the skin surface such as between the fingers, the light will be captured by the photoelectric receiver through transmission or reflection. During the whole process, the absorption of light by skin, muscle, blood, etc. is weakened, and the light intensity reaching the photoelectric receiver will be reduced. Among them, the weakening effect of skin, muscle, etc. on light intensity is constant, while the weakening effect of blood in blood vessels on light will show a pulsatile change with the beating of the heart. When the heart contracts, the blood volume in the blood vessels increases, the absorbed light intensity increases, and the light intensity received by the photoelectric receiver decreases accordingly. When the heart dilates, the opposite is true. By converting this light intensity into an electrical signal, the change in volume pulse blood flow can be obtained. This method requires the sensor to be in close contact with the fixed human body part, and has many restrictions on the user's usage methods and scenarios.

On the one hand, the heart rate detection in the prior art relies on special light sources and specific sensors, and additional modules need to be added to the hardware. On the other hand, only the time domain counting method is used to estimate the heart rate, and the accuracy of the results is easily affected by noise. solution.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a heart rate detection method and device, which can realize heart rate detection on the premise of an existing fingerprint identification optical imaging system, so that the detection accuracy is not easily disturbed by noise, and the application scenarios are broadened.

The embodiment of the present application provides a heart rate detection method, and the method may include:

Collect fingerprint image sequence;

Determine the initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence;

The heart rate is obtained by performing time-frequency domain analysis on the initial signal sequence.

In an exemplary embodiment of the present application, the collection of the fingerprint image sequence by the collection device may include:

The fingerprint area is irradiated by the light of the preset color from the screen light source;

Image acquisition is performed based on the optical signal returned from the fingerprint area to acquire the fingerprint image sequence.

In an exemplary embodiment of the present application, before determining the initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence, the method may further include:

The valid fingerprint area of each frame of fingerprint image in the fingerprint image sequence is determined.

In an exemplary embodiment of the present application, the determining the effective fingerprint area of each frame of fingerprint image in the sequence of fingerprint images includes: respectively performing the following operations for each frame of fingerprint image:

Screen out the area containing the fingerprint from the fingerprint image of this frame;

An area that meets the first condition is screened out from the area containing the fingerprint as the valid fingerprint area.

In an exemplary embodiment of the present application, the determining the initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence may include:

Calculate the initial signal corresponding to the current frame fingerprint image according to the effective fingerprint area of the current frame fingerprint image in the fingerprint image sequence;

The initial signal corresponding to the fingerprint image of the current frame and the initial signal corresponding to the fingerprint images of the previous N frames constitute the initial signal sequence.

In an exemplary embodiment of the present application, the calculation of the initial signal corresponding to the current frame fingerprint image according to the effective fingerprint area of the current fingerprint frame image in the fingerprint image sequence may include:

Data processing is performed on the pixel values of a plurality of the valid fingerprint regions in the fingerprint image of the current frame, and an initial signal corresponding to the fingerprint image of the current frame is obtained.

In an exemplary embodiment of the present application, after acquiring the initial signal sequence, the method may further include:

Perform preprocessing on the initial signal sequence in the time domain to obtain a time domain signal of the initial signal sequence;

Wherein, the preprocessing can include any one or more of the following:

Detrending, sliding filtering, bandpass filtering, and normalization.

In an exemplary embodiment of the present application, before performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate, the method may further include: judging the validity of the initial signal sequence.

In an exemplary embodiment of the present application, the validity judgment of the initial signal sequence may include:

extracting eigenvalues of preset features from the initial signal sequence;

comparing the eigenvalue with a preset standard eigenvalue;

When the eigenvalue conforms to the preset floating range of the standard eigenvalue, determining that the initial signal sequence is a valid signal;

When the characteristic value does not conform to the preset floating range of the standard characteristic value, it is determined that the initial signal sequence is an invalid signal.

In an exemplary embodiment of the present application, the preset features may include any one or more of the following: area, entropy, skewness, and kurtosis.

In an exemplary embodiment of the present application, the performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate may include:

Converting the initial signal sequence from a time domain signal to a first signal, wherein the first signal is a frequency domain signal or a time-frequency domain signal;

performing secondary frequency domain filtering and post-processing on the first signal to obtain a second signal;

According to the category of the second signal, the second signal is analyzed and counted to obtain the heart rate, where the second signal is a frequency domain signal or a time-frequency domain signal.

In an exemplary embodiment of the present application, when the type of the second signal is a frequency domain signal, the performing analysis and statistics on the second signal to obtain the heart rate may include:

Using a preset peak detection algorithm to obtain a peak within a preset heart rate range from the second signal;

Sort all the obtained peaks according to the peak size, obtain the highest peak with the largest peak, and calculate the confidence level of the highest peak;

When the confidence level is greater than or equal to the first threshold, the mode of the frequencies of all the peaks is obtained as the heart rate.

In an exemplary embodiment of the present application, the calculating the confidence of the highest peak may include:

Taking the ratio of the energy of the highest peak to the sum of the energy of all peaks except the highest peak as the confidence of the highest peak; or,

The ratio of the peak value of the highest peak to the peak value of the second highest peak is used as the confidence level of the highest peak.

In an exemplary embodiment of the present application, when the type of the second signal is a time-frequency domain signal, the performing analysis and statistics on the second signal to obtain the heart rate may include:

Perform frequency maximum detection on the second signal according to the time domain to obtain multiple frequency domain signals within a preset heart rate range;

calculating a confidence level of the plurality of frequency domain signals;

When the confidence level is greater than or equal to the second threshold, the mode of the frequencies of the plurality of frequency domain signals is obtained as the heart rate.

In an exemplary embodiment of the present application, the performing frequency maximum detection on the second signal according to the time domain to obtain multiple frequency domain signals within a preset heart rate range may include:

obtaining a time-frequency coordinate diagram corresponding to the second signal;

For each preset time point in the time axis, obtain a plurality of candidate heart rate frequency domain signals from all the frequency domain signals corresponding to the preset time point, and select the frequency response largest from the plurality of candidate heart rate frequency domain signals the frequency domain signal;

Accumulate a plurality of the preset time points, and record the frequency domain signal with the largest frequency response corresponding to each preset time point to obtain multiple frequency domains within the possible range of heart rate that vary along the time axis Signal.

In an exemplary embodiment of the present application, the calculating confidence levels of the multiple frequency domain signals may include:

The standard deviation of the plurality of frequency domain signals is calculated as the confidence level of the plurality of frequency domain signals.

In an exemplary embodiment of the present application, the method may further include:

When the confidence level is less than the preset threshold, discard the earliest obtained initial signal in the initial signal sequence, and add the initial signal corresponding to the next frame of fingerprint image to the initial signal sequence, so as to realize the detection of the initial signal. sequence update;

Acquire the updated time domain signal of the initial signal sequence, and perform time-frequency domain analysis on the updated time domain signal of the initial signal sequence to obtain the heart rate.

In an exemplary embodiment of the present application, when the type of the second signal is a time-frequency domain signal, the performing analysis and statistics on the second signal to obtain the heart rate may include:

Peak detection is performed on the second signal according to the time domain, and the heart rate is calculated according to the number of detected peaks and a preset heart rate calculation formula.

Embodiments of the present application further provide a heart rate detection device, which may include a processor and a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are executed by the processor, all The heart rate detection method described above.

Embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the heart rate detection method is implemented.

Compared with the related art, the embodiments of the present application may include: collecting a fingerprint image sequence; determining an initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence; The signal sequence is analyzed in the time-frequency domain to obtain the heart rate. Through the solution of this embodiment, the heart rate detection is realized on the premise of the existing fingerprint identification optical imaging system, so that the detection accuracy is not easily disturbed by noise, and the application scenarios are broadened.

Other features and advantages of the present application will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the present application. Other advantages of the present application may be realized and attained by the approaches described in the specification and drawings.

Description of drawings

The accompanying drawings are used to provide an understanding of the technical solutions of the present application, and constitute a part of the specification. They are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application.

FIG. 1 is a flowchart of a heart rate detection method according to an embodiment of the present application;

2 is a flowchart of a method for determining an initial signal sequence corresponding to a fingerprint image sequence according to an effective fingerprint area of each frame of fingerprint image in a fingerprint image sequence according to an embodiment of the present application;

FIG. 3 is a flowchart of a first solution for obtaining heart rate by performing time-frequency domain analysis on a time-domain signal of an initial signal sequence according to an embodiment of the present application;

4 is a flow chart of a second solution for obtaining heart rate by performing time-frequency domain analysis on a time-domain signal of an initial signal sequence according to an embodiment of the present application;

5 is a flowchart of a third solution for obtaining heart rate by performing time-frequency domain analysis on a time-domain signal of an initial signal sequence according to an embodiment of the present application;

FIG. 6 is a block diagram showing the composition of a heart rate detection apparatus according to an embodiment of the present application.

Detailed ways

This application describes a number of embodiments, but the description is exemplary rather than restrictive, and it will be apparent to those of ordinary skill in the art that within the scope of the embodiments described in this application can be There are many more examples and implementations. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Unless expressly limited, any feature or element of any embodiment may be used in combination with, or may be substituted for, any other feature or element of any other embodiment.

This application includes and contemplates combinations with features and elements known to those of ordinary skill in the art. The embodiments, features and elements that have been disclosed in this application can also be combined with any conventional features or elements to form unique inventive solutions as defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other inventive arrangements to form another unique inventive arrangement defined by the claims. Accordingly, it should be understood that any of the features shown and/or discussed in this application may be implemented alone or in any suitable combination. Accordingly, the embodiments are not to be limited except in accordance with the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

Furthermore, in describing representative embodiments, the specification may have presented methods and/or processes as a particular sequence of steps. However, to the extent that the method or process does not depend on the specific order of steps described herein, the method or process should not be limited to the specific order of steps described. Other sequences of steps are possible, as will be understood by those of ordinary skill in the art. Therefore, the specific order of steps set forth in the specification should not be construed as limitations on the claims. Furthermore, the claims directed to the method and/or process should not be limited to performing their steps in the order written, as those skilled in the art will readily appreciate that these orders may be varied and still remain within the spirit and scope of the embodiments of the present application Inside.

An embodiment of the present application provides a heart rate detection method. As shown in FIG. 1 , the method may include steps S101-S103:

S101. Collect a fingerprint image sequence;

S102, determining an initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence;

S103. Perform time-frequency domain analysis on the initial signal sequence to obtain a heart rate.

In the exemplary embodiment of the present application, the manner of collecting the fingerprint image sequence is not limited, for example, it can be collected by an off-screen camera. The solution of the embodiment of the present application can realize heart rate detection through an off-screen fingerprint heart rate estimation algorithm. The off-screen fingerprint heart rate estimation algorithm proposed by the embodiments of the present application mainly includes two modules: an off-screen camera module in the form of hardware and a signal processing module in the form of software. The off-screen camera module may include: an off-screen light source and an off-screen camera, the off-screen light source is used to illuminate the fingerprint area, and the camera collects the light signal returned by the user's finger to obtain a sequence of fingerprint images (ie, finger reflection imaging video); And use the signal processing module to detect and locate the fingerprint area of the fingerprint signal in the obtained fingerprint image sequence, perform denoising processing, and convert it into a digital signal. Specifically, the form of the digital signal is not limited in this application, and the digital signal can be Photoplethysmography ppg signal (referred to as ppg signal), and then through algorithm analysis to obtain the current heart rate estimate.

In the exemplary embodiment of the present application, the sensor data input required by the solution of the embodiment of the present application is relatively simple, and is only a sequence of fingerprint images collected by the under-screen camera, usually greater than 20fps.

In an exemplary embodiment of the present application, the collection of fingerprint image sequences may include:

The fingerprint area is irradiated by the light of the preset color from the screen light source;

Image acquisition is performed based on the optical signal returned from the fingerprint area to acquire the fingerprint image sequence.

In the exemplary embodiment of the present application, the light of the preset color emitted by the screen light source may include, but is not limited to, green light. Specifically, the screen light source is used to provide a light signal for fingerprint detection, and a built-in light source or an external light source may be used. light source. The fingerprint area is the area that the user's finger can press. This application does not limit the position of the fingerprint area on the detection device, ensuring that the screen light source emits light of a preset color and can illuminate the fingerprint area, and the light signal is scattered or reflected by the finger to collect The device can finally acquire the fingerprint image sequence.

In an exemplary embodiment of the present application, before determining the initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence, the method may include:

The valid fingerprint area of each frame of fingerprint image in the fingerprint image sequence is determined.

In an exemplary embodiment of the present application, the determining the effective fingerprint area of each frame of fingerprint image in the sequence of fingerprint images may include: respectively performing the following operations for each frame of fingerprint image:

Screen out the area containing the fingerprint from the fingerprint image of this frame;

An area that meets the first condition (for example, an area with a preset color) is selected from the area containing fingerprints as the valid fingerprint area.

In the exemplary embodiment of the present application, due to factors such as noise and finger posture affecting data validity in fingerprint collection, for the fingerprint image sequence collected in the fingerprint region, it is necessary to divide and filter out the effective fingerprint regions, that is, to effectively extract different fingerprint regions. The fingerprint features of the location are used to counteract the influence of the above factors, and are finally based on the effective fingerprint area. The above-mentioned effective fingerprint area refers to an area with a strong heart rate signal, which is more conducive to subsequent heart rate detection. In the exemplary embodiment of the present application, firstly, the area containing the fingerprint is screened out in the fingerprint image of the frame, that is, the area where the finger actually touches the touch screen; and then combined with the first condition, the effective fingerprint area is selected from the area containing the fingerprint. , Specifically, the first condition may include a finger pressing state and/or blood vessel distribution.

In the exemplary embodiment of the present application, the blood vessels are abundantly distributed on the finger, but the blood flow distribution varies. When the initial signal is the ppg signal, the information contained in the blood flow is used. There are different treatments depending on the degree of pressure: if the contact area is not enough when the pressure is too light, there is no fingerprint in the uncontacted area; when the pressure is heavy, the capillary blood flow in the area with the greatest pressure will be blocked (that is, the hair in some areas of the finger will be blocked. White has no blood color), and does not have the conditions for measuring the initial signal. Therefore, an area containing a fingerprint area (ie, an actual contact area) and an area that is rich in blood flow and is not blocked (can be judged by whether there is blood color) can be selected as the effective fingerprint area according to the actual situation.

In an exemplary embodiment of the present application, the determining the initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of the fingerprint image sequence in the fingerprint image sequence may include:

Calculate the initial signal corresponding to the current frame fingerprint image according to the effective fingerprint area of the current frame fingerprint image in the fingerprint image sequence;

The initial signal corresponding to the fingerprint image of the current frame and the initial signal corresponding to the fingerprint images of the previous N frames constitute the initial signal sequence.

In an exemplary embodiment of the present application, as shown in FIG. 2 , the detailed steps of the above solution may include S201-S204:

S201. Calculate the initial signal corresponding to the fingerprint image of the current frame.

In an exemplary embodiment of the present application, the calculating the initial signal corresponding to the fingerprint image of the current frame may include:

Data processing is performed on the pixel values of a plurality of the valid fingerprint regions in the fingerprint image of the current frame, and an initial signal corresponding to the fingerprint image of the current frame is obtained.

In the exemplary embodiment of the present application, the data processing may refer to weighted average, and weighted average of pixel values of multiple valid fingerprint regions is performed to obtain the sampling point value of the initial signal of the fingerprint image of this frame. In the weighted average, the weighted values of different effective fingerprint regions can be determined according to pre-experiments, for example, regions with strong heart rate signals have higher weights, and regions with lower heart rate signals have lower weights.

S202. Accumulate the initial signal corresponding to the fingerprint image of the current frame and the initial signals corresponding to the fingerprint images of the multiple frames before the fingerprint image of the current frame.

S203. Detect whether the number of initial signals has accumulated to the preset number N+1, when the number of initial signals has accumulated to the preset number N+1, go to step S204; when the number of initial signals has not accumulated to the preset number When N+1, acquire the fingerprint image of the next frame as the fingerprint image of the current frame, and return to step S201; N is a positive integer.

S204. Form the initial signal sequence from the accumulated N+1 initial signals.

In the exemplary embodiment of the present application, the ppg sampling point value of the fingerprint image of the current frame and the ppg sampling point value of the fingerprint images of the previous N frames form a set of ppg signals, that is, an initial signal sequence.

In the exemplary embodiment of the present application, the historical frame is relative to the current frame, and the previous N frames in the history mean that up to the current frame, N frames of fingerprint images have been accumulated excluding the current frame. For example, the number of the fingerprint image of the current frame is M, and the number of the fingerprint images of the previous N frames is [M-N, M-N+1, M-N+2,...,M-1].

In the exemplary embodiment of the present application, a sampling point value is obtained by weighted averaging of the effective fingerprint area of the fingerprint image of the current frame, and is combined with the signals obtained in the same manner in the previous N frames to obtain an initial signal sequence. Since multiple valid fingerprint regions are screened out in each frame of fingerprint image, it can be considered that there are multiple large valid fingerprint regions on a fingerprint image, and the pixels in each valid fingerprint region are required by the solution of the embodiment of the present application, and other non- Pixels in the valid fingerprint area are not needed. The ppg signal of the sampling point of the fingerprint image in this frame is also the weighted average of the pixel values in all valid fingerprint areas in the current fingerprint image, and a ppg value is obtained, that is, the ppg signal sampling value. When the sampled values of the ppg signal are connected into an array (forming a 1×(N+1) one-dimensional signal array), an initial signal sequence is generated, which is the original ppg signal.

In an exemplary embodiment of the present application, after acquiring the time domain signal of the initial signal sequence, the method may further include:

The initial signal sequence is preprocessed in the time domain to obtain a time domain signal of the initial signal sequence.

In an exemplary embodiment of the present application, the preprocessing may include any one or more of the following:

Detrending, sliding filtering, bandpass filtering, and normalization.

In the exemplary embodiment of the present application, through the preprocessing, a de-noised time domain signal can be obtained, which improves the quality of the heart rate data used to estimate the heart rate, thereby ensuring the accuracy of subsequent heart rate estimation.

In an exemplary embodiment of the present application, before performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate, the method may further include: judging the validity of the initial signal sequence.

In the exemplary embodiment of the present application, the solution of this embodiment can combat the problems of noise, finger gestures, etc. that affect the validity of data.

In an exemplary embodiment of the present application, the validity judgment of the initial signal sequence may include:

extracting eigenvalues of preset features from the initial signal sequence;

comparing the eigenvalue with a preset standard eigenvalue;

When the eigenvalue conforms to the preset floating range of the standard eigenvalue, determining that the initial signal sequence is a valid signal;

When the characteristic value does not conform to the preset floating range of the standard characteristic value, it is determined that the initial signal sequence is an invalid signal.

In an exemplary embodiment of the present application, the preset features may include any one or more of the following: area, entropy, skewness, and kurtosis.

In the exemplary embodiment of the present application, the area feature extracted is the closed area where the ppg sequence signal and the mean value intersect (that is, the curve corresponding to the ppg sequence signal intersects with the straight line corresponding to the ppg mean value calculated from the ppg sequence signal, area enclosed by curves and lines). Features such as entropy, skewness, and kurtosis directly process the entire initial signal sequence, and are classic feature indicators.

In an exemplary embodiment of the present application, the performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate may include:

Converting the initial signal sequence from a time domain signal to a first signal, wherein the first signal is a frequency domain signal or a time-frequency domain signal;

performing secondary frequency domain filtering and post-processing on the first signal to obtain a second signal;

According to the category of the second signal, the second signal is analyzed and counted to obtain the heart rate, where the second signal is a frequency domain signal or a time-frequency domain signal.

In the exemplary embodiment of the present application, the initial signal sequence is converted from a time domain signal to a first signal. Since the first signal is a frequency domain signal or a time-frequency domain signal, the second signal obtained after filtering and post-processing is also a frequency domain signal. Domain signal or time-frequency domain signal. According to the type of the signal, performing time-frequency domain analysis on the signal sequence to obtain the heart rate may include various schemes, and three embodiments are given below.

Option One

In an exemplary embodiment of the present application, as shown in FIG. 3 , when the type of the second signal is a frequency domain signal, the performing analysis and statistics on the second signal to obtain the heart rate may include: Steps S301-S303:

S301, using a preset peak detection algorithm to obtain the peak of the power spectrum signal (second signal) within a preset heart rate range;

S302. Sort all the obtained peaks according to the peak size, obtain the highest peak with the largest peak, and calculate the confidence level of the highest peak;

S303. When the confidence level is greater than or equal to a first threshold (which is a preset threshold), acquire the mode of the frequencies of all the peaks as the heart rate.

In the exemplary embodiment of the present application, the present application does not limit the time-frequency domain conversion method, and the time-domain signal of the denoised initial signal sequence may be converted into a frequency-domain signal through fast Fourier transform. Fast Fourier transform (FFT, fast fourier transform) is a classical algorithm, input time domain signal, after FFT, can obtain frequency domain signal.

In the exemplary embodiment of the present application, the obtained frequency domain signal may be subjected to secondary frequency domain filtering and post-processing to obtain a power spectrum signal with obvious peaks. Peak detection is used to obtain the peaks of the power spectrum signal within the possible range of the heart rate, and the confidence level of the highest peak is calculated by the ranking of the peaks and the ratio of the peak size.

In an exemplary embodiment of the present application, the calculating the confidence of the highest peak may include:

Taking the ratio of the energy of the highest peak to the sum of the energy of all peaks except the highest peak as the confidence of the highest peak; or,

The ratio of the peak value of the highest peak to the peak value of the second highest peak is used as the confidence level of the highest peak.

In an exemplary embodiment of the present application, the ratio of the energy of the highest peak to the sum of the energy of other peaks is a signal-to-noise ratio, and the signal-to-noise ratio can be used as the confidence of the highest peak.

In the exemplary embodiment of the present application, when the calculated confidence level is greater than or equal to a preset confidence level threshold (the confidence level threshold may be a first threshold), it may be considered that the initial signal sequence is not polluted by noise , the result is credible, and the frequency of the peak can be obtained. After conversion (for example, obtaining the mode of the frequencies of all the peaks), the main frequency corresponding to the input initial signal sequence, that is, the heart rate, can be obtained.

In an exemplary embodiment of the present application, the method may further include:

When the confidence level is smaller than a preset confidence level threshold (for example, the first threshold), discard the earliest obtained initial signal in the initial signal sequence, and acquire the initial signal corresponding to the next frame of fingerprint image and add it to the initial signal sequence, to realize the update of the initial signal sequence;

Acquire the updated time domain signal of the initial signal sequence, and perform time-frequency domain analysis on the updated time domain signal of the initial signal sequence to obtain the heart rate.

In the exemplary embodiment of the present application, when the calculated confidence level is less than a preset threshold, it means that the initial signal sequence is seriously polluted by noise (the signal confidence level calculated from the highest peak of the signal is the initial signal sequence If the standard is not qualified, it is not credible. If the standard is qualified, the process of obtaining heart rate according to the initial signal sequence can be entered), and the current result can be discarded and entered into the next frame of fingerprint image. On the basis of the accumulated initial signal sequence, continue to process the sampling point signal of the next frame of fingerprint image.

In the exemplary embodiment of the present application, each frame of fingerprint image can obtain one ppg sampling point, or obtain one ppg signal, and it is necessary to obtain N ppg signals continuously before forming an initial signal sequence, and then proceed to Signal processing, in the case of unqualified standards, you can go back to the initial step, obtain a new frame of fingerprint image in the fingerprint image sequence, calculate the ppg signal of this frame of fingerprint image, and add it to the end of the initial signal sequence as a new ppg signal. The oldest ppg signal at the head end of the signal sequence is eliminated, and subsequently the updated constant n-length initial signal sequence is subjected to processing such as denoising and validity judgment, and the heart rate is obtained according to the updated initial signal sequence.

Option II

In an exemplary embodiment of the present application, as shown in FIG. 4 , when the type of the second signal is a time-frequency domain signal, the second signal is analyzed and counted to obtain the heart rate, which may be Including steps S401-S403:

S401. Perform frequency maximum detection on the second signal according to the time domain to obtain multiple frequency domain signals within a preset heart rate range;

S402. Calculate the confidence level of the multiple frequency domain signals;

S403. When the confidence level is greater than or equal to a second threshold (which is a preset threshold), acquire a mode of frequencies of the multiple frequency domain signals as the heart rate.

In the exemplary embodiment of the present application, the denoised time-domain signal is decomposed using a time-frequency analysis method, and then the frequency-domain signal is subjected to secondary frequency-domain filtering and post-processing to obtain a "clean" time-frequency signal, The present application does not limit the time-frequency analysis method, and the time-domain signal can be converted into a time-frequency signal through short-time Fourier transform. In order to obtain higher time-frequency resolution and at the same time better distinguish random noise and signal, the present application can also use wavelet synchronous compression transform to perform time-frequency domain conversion. In addition, the present application performs frequency maximum detection on a "clean" time-frequency signal according to the time domain to obtain a signal within the possible range of the heart rate.

In the exemplary embodiment of the present application, the frequency maximum value detection is performed on the frequency domain signal after the frequency domain filtering and post-processing according to the time domain to obtain multiple frequency domain signals within the possible range of the heart rate, which may be include:

obtaining a time-frequency coordinate graph corresponding to the frequency-domain signal; the horizontal axis of the time-frequency coordinate graph is the time axis, the vertical axis is the frequency-domain axis, and each point in the time-frequency coordinate graph is the size of the frequency-domain signal;

For each preset time point in the time axis, obtain multiple heart rate candidate frequency domain signals from all frequency domain signals corresponding to the preset time point, and select a frequency from the multiple heart rate candidate frequency domain signals Response to the largest frequency domain signal;

Accumulate a plurality of the preset time points, and record the frequency domain signal with the largest frequency response corresponding to each preset time point, so as to obtain multiple frequency domain signals within the possible range of heart rate that vary along the time axis.

In an exemplary embodiment of the present application, the frequency response is the strength of the signal, not equal to the frequency.

In an exemplary embodiment of the present application, the calculating confidence levels of the multiple frequency domain signals may include:

The standard deviation of the plurality of frequency domain signals is calculated as the confidence level of the plurality of frequency domain signals.

In an exemplary embodiment of the present application, the method may further include:

When the confidence is less than the confidence threshold (second threshold), discard the earliest obtained initial signal in the initial signal sequence, and obtain the initial signal corresponding to the next frame of fingerprint image and add it to the initial signal sequence, implementing an update to the initial signal sequence;

Acquire the updated time domain signal of the initial signal sequence, and perform time-frequency domain analysis on the updated time domain signal of the initial signal sequence to obtain the heart rate.

In the exemplary embodiment of the present application, the processing scheme when the confidence level is less than the confidence level threshold is the same as the processing scheme in scheme 1, and details are not repeated here.

third solution

In an exemplary embodiment of the present application, as shown in FIG. 5 , when the type of the second signal is a time-frequency domain signal, the second signal is analyzed and counted to obtain the heart rate, which may be Including step S501:

S501. Perform peak detection on the second signal according to the time domain, and calculate the heart rate according to the number of detected peaks and a preset heart rate calculation formula.

In the exemplary embodiment of the present application, the denoised time-domain signal is decomposed using a time-frequency analysis method to obtain a frequency-domain signal, and then the frequency-domain signal is subjected to secondary frequency-domain filtering and post-processing to obtain a "clean" signal. time-frequency signal. The present application does not limit time-frequency analysis methods, such as converting time-domain signals into time-frequency signals by short-time Fourier transform. In addition, the present application performs peak detection on a "clean" time-frequency signal according to the time domain to calculate the number of peaks in the initial signal sequence.

In the exemplary embodiment of the present application, one peak and one trough in the waveform of the initial signal sequence may be used as a cycle, and the number of cycles, that is, the number of peaks in the initial signal sequence, may be calculated.

In an exemplary embodiment of the present application, the calculating the heart rate according to the detected number of peaks and a preset heart rate calculation formula includes: calculating the heart rate according to the following heart rate calculation formula:

HR=P/T*M;

Wherein, HR is the heart rate, P is the number of peaks, and M is a unit duration (for example, 1 minute).

This embodiment of the present application further provides a heart rate detection device 1, as shown in FIG. 6, which may include a processor 11 and a computer-readable storage medium 12, where instructions are stored in the computer-readable storage medium 12, when the instructions When executed by the processor 11, the heart rate detection method is implemented.

In the exemplary embodiments of the present application, any of the foregoing heart rate detection method embodiments are applicable to the heart rate detection apparatus embodiments, and details are not repeated here.

Embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the heart rate detection method is implemented.

In the exemplary embodiments of the present application, any of the foregoing heart rate detection method embodiments are applicable to this computer-readable storage medium embodiment, and details are not repeated here.

Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, functional modules/units in the systems, and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components Components execute cooperatively. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data flexible, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or may Any other medium used to store desired information and which can be accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media, as is well known to those of ordinary skill in the art .

Claims (20)

1. A method of heart rate detection, the method comprising:

collecting a fingerprint image sequence;

determining an initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence;

and analyzing the time-frequency domain of the initial signal sequence to obtain the heart rate.

2. The heart rate detection method of claim 1, wherein the capturing of the sequence of fingerprint images comprises:

emitting light with a preset color to irradiate the fingerprint area through a screen light source;

and acquiring an image based on the optical signal returned by the fingerprint area to obtain the fingerprint image sequence.

3. The heart rate detection method according to claim 1, wherein before determining the initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence, the method comprises:

determining an effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence.

4. The heart rate detection method of claim 3, wherein the determining the valid fingerprint region for each frame of fingerprint image in the sequence of fingerprint images comprises: the following operations are respectively executed for each frame of fingerprint image:

screening out an area containing the fingerprint from the frame of fingerprint image;

and screening out an area meeting a first condition from the areas containing the fingerprints as the effective fingerprint area.

5. The heart rate detection method according to claim 1, wherein the determining an initial signal sequence corresponding to the fingerprint image sequence according to the effective fingerprint area of each frame of fingerprint image in the fingerprint image sequence comprises:

calculating an initial signal corresponding to the current frame fingerprint image according to the effective fingerprint area of the current frame fingerprint image in the fingerprint image sequence;

and the initial signal corresponding to the fingerprint image of the current frame and the initial signal corresponding to the fingerprint image of the previous N frames form the initial signal sequence.

6. The heart rate detection method according to claim 5, wherein the calculating an initial signal corresponding to the current frame fingerprint image according to the effective fingerprint area of the current frame fingerprint image in the fingerprint image sequence includes:

and carrying out data processing on the pixel values of the effective fingerprint areas in the current frame fingerprint image to acquire an initial signal corresponding to the current frame fingerprint image.

7. The heart rate detection method of claim 1, wherein after acquiring the initial sequence of signals, the method further comprises:

preprocessing the initial signal sequence in a time domain to obtain a time domain signal of the initial signal sequence;

wherein the pretreatment comprises any one or more of the following:

detrending, sliding filtering, band pass filtering, and normalization.

8. The heart rate detection method according to claim 1, wherein before performing the time-frequency domain analysis on the initial signal sequence to obtain the heart rate, the method further comprises: and judging the validity of the initial signal sequence.

9. The heart rate detection method according to claim 8, wherein the determining the validity of the initial signal sequence includes:

extracting a characteristic value of a preset characteristic from the initial signal sequence;

comparing the characteristic value with a preset standard characteristic value;

when the characteristic value accords with a preset floating range of the standard characteristic value, judging that the initial signal sequence is an effective signal;

and when the characteristic value does not accord with the preset floating range of the standard characteristic value, judging that the initial signal sequence is an invalid signal.

10. The heart rate detection method according to claim 9, wherein the preset features include any one or more of: area, entropy, skewness, and kurtosis.

11. The heart rate detection method according to claim 1, wherein the performing time-frequency domain analysis on the initial signal sequence to obtain the heart rate comprises:

converting the initial signal sequence from a time domain signal to a first signal, wherein the first signal is a frequency domain signal or a time-frequency domain signal;

carrying out secondary frequency domain filtering and post-processing on the first signal to obtain a second signal;

and analyzing and counting the second signal according to the category of the second signal to obtain the heart rate, wherein the second signal is a frequency domain signal or a time-frequency domain signal.

12. The heart rate detection method according to claim 11, wherein when the category of the second signal is a frequency domain signal, the analyzing and counting the second signal to obtain the heart rate comprises:

obtaining a peak of the second signal within a preset heart rate range by using a preset peak detection algorithm;

sequencing all the obtained wave crests according to the size of the peak value, obtaining the highest wave crest with the largest peak value, and calculating the confidence coefficient of the highest wave crest;

when the confidence is larger than or equal to a first threshold, acquiring a mode of frequencies of all the peaks as the heart rate.

13. The method of heart rate detection according to claim 12, wherein the calculating the confidence level of the highest peak comprises:

taking the ratio of the energy of the highest peak to the sum of the energies of other peaks except the highest peak in all peaks as the confidence of the highest peak; or,

and taking the ratio of the peak value of the highest peak to the peak value of the second highest peak as the confidence coefficient of the highest peak.

14. The heart rate detection method according to claim 11, wherein when the category of the second signal is a time-frequency domain signal, the analyzing and counting the second signal to obtain the heart rate comprises:

carrying out frequency maximum value detection on the second signal according to a time domain to obtain a plurality of frequency domain signals within a preset heart rate range;

calculating confidence degrees of the plurality of frequency domain signals;

when the confidence is greater than or equal to a second threshold, acquiring a mode of frequencies of the plurality of frequency domain signals as the heart rate.

15. The heart rate detection method of claim 14, wherein the performing frequency maximum detection on the second signal according to the time domain to obtain a plurality of frequency domain signals within a preset heart rate range comprises:

acquiring a time-frequency coordinate graph corresponding to the second signal;

aiming at each preset time point in a time axis, acquiring a plurality of heart rate candidate frequency domain signals from all frequency domain signals corresponding to the preset time point, and selecting a frequency domain signal with the maximum frequency response from the plurality of heart rate candidate frequency domain signals;

and accumulating the preset time points, recording the frequency domain signal with the maximum frequency response corresponding to each preset time point, and obtaining a plurality of frequency domain signals which are changed along the time axis and are in the possible range of the heart rate.

16. The heart rate detection method of claim 14, wherein the calculating the confidence levels for the plurality of frequency domain signals comprises:

calculating a standard deviation of the plurality of frequency domain signals as a confidence of the plurality of frequency domain signals.

17. The method of heart rate detection according to claim 12 or 14, further comprising:

when the confidence coefficient is smaller than a preset threshold value, discarding the earliest obtained signal in the initial signal sequence, and adding a signal corresponding to the next frame of fingerprint image into the initial signal sequence to update the initial signal sequence;

and acquiring the updated time domain signal of the initial signal sequence, and performing time-frequency domain analysis on the updated time domain signal of the initial signal sequence to acquire the heart rate.

18. The heart rate detection method according to claim 11, wherein when the category of the second signal is a time-frequency domain signal, the analyzing and counting the second signal to obtain the heart rate comprises:

and performing peak detection on the second signal according to the time domain, and calculating the heart rate according to the number of detected peaks and a preset heart rate calculation formula.

19. A heart rate detection apparatus comprising a processor and a computer readable storage medium having instructions stored therein, wherein the instructions, when executed by the processor, implement the heart rate detection method of any one of claims 1-18.

20. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the heart rate detection method according to any one of claims 1-18.

Images