CN116211316A - Type identification method, system and auxiliary system for multi-lead ECG signals - Google Patents

Abstract

The application discloses a type identification method, a system and an auxiliary system of multi-lead electrocardiosignals, which are characterized in that firstly, multi-lead electrocardiosignals and patient characteristic information and electrocardiosignal type labels which are related to partial multi-lead electrocardiosignals in the multi-lead electrocardiosignals are acquired through a data acquisition module, then a data preprocessing module and a data set dividing module are sequentially input to respectively complete data preprocessing and data set dividing, then an electrocardiosignal self-supervision model is trained and stored through a model generating module, finally, when a service computing module receives a type identification request of the electrocardiosignals, the trained electrocardiosignal self-supervision model is automatically called, probability information corresponding to various set electrocardiosignal types is acquired based on data carried by the request, and meanwhile, an electrocardiosignal interpretation model built in an electrocardiosignal interpretation module can be called to interpret an electrocardiosignal. The method and the system can train and obtain the type recognition model of the multi-lead electrocardiosignal based on fewer label data, and improve the model recognition accuracy.

Description

Type identification method, system and auxiliary system for multi-lead ECG signals

technical field

The present application relates to the technical field of artificial intelligence, in particular to a type identification method, system and auxiliary system for multi-lead ECG signals.

Background technique

At present, the gold standard for the diagnosis of many cardiovascular diseases is based on imaging examinations, such as ultrasound, CT, MRI, and interventional angiography. cause radiation or trauma damage. Therefore, there is an urgent need for low-cost, convenient and safe inspection methods to deal with the current situation of a large number of cardiovascular patients.

Compared with imaging examinations, electrocardiogram (signal) examination has the advantages of non-invasiveness, simple operation, and cost-effectiveness. In recent years, deep learning has developed significantly in the field of ECG. There are many deep learning models that can assist in traditional ECG tasks such as identifying arrhythmias. Further studies have shown that deep learning can capture artificially difficult to identify in the ECG of patients with cardiovascular disease. The pattern can improve the accuracy of electrocardiogram type recognition. However, the current methods for type recognition based on ECG signals mainly use supervised deep learning models. The training of this type of model requires the use of a large number of ECG signals and patient ECG signal type label information (such as disease label information, etc.) associated with multi-lead ECG signals. On the one hand, a large amount of ECG data without patient ECG type label information is discarded; on the other hand, in many practical scenarios, there is not enough patient ECG type label information, so some ECG data A model for electrical signal type identification.

At present, there are many ECG signal type recognition systems based on deep learning that can better complete traditional ECG tasks such as arrhythmia identification. However, due to the lack of a large amount of multimodal label data that can be matched with ECG, and the lack of unsupervised learning methods for ECG signals, it is difficult for most institutions to establish a system based on ECG signals to identify adult congenital heart disease, valvular disease, coronary heart disease, and myocardial infarction. A deep learning model for ECG signal types such as heart disease. Therefore, at present, there are few systems that perform type recognition with high accuracy based on ECG signals.

Contents of the invention

The present application provides a multi-lead electrocardiographic signal type identification method, type identification system, and auxiliary system, which can obtain a type identification model based on less label data training and improve the accuracy of type identification.

In order to achieve the above object, the present application adopts the following technical solution: a method for identifying the type of a multi-lead ECG signal, comprising the following steps:

Data collection, collecting n multi-lead ECG signals, and patient characteristic information and ECG signal type labels associated with some of the n multi-lead ECG signals;

Data preprocessing, generating a multi-lead ECG data set D ₁ representing all ECG signals based on the n multi-lead ECG signals, the corresponding sample size is n; based on the part of the multi-lead ECG signals and The patient characteristic information and the ECG signal type label associated with the part of the multi-lead ECG signal generate an associated data set D ₂ , and the corresponding sample size is m, where m≤n;

dividing the data set, dividing the multi-lead ECG data set D ₁ into a multi-lead ECG training set D _1,train and a multi-lead ECG verification set D _1,vali , and dividing the associated data Set D ₂ is divided into associated data training set D _2,train , associated data verification set D _2,vali , associated data test set D _2,test ;

ECG self-supervised model framework construction, said model framework is based on the Transformer module, including cutter, double classification masker, encoder, decoder and classifier;

Model pre-training, initialize model parameters, and then the multi-lead ECG signal training set D _1,train , multi-lead ECG signal verification set D _1,vali , associated data training set D _2,train , associated data Verification set D _{2, vali} input model framework, self-supervised learning, penalty self-supervised learning, get pre-trained ECG self-supervised model;

Model fine-tuning, based on the associated data training set D _2,train and the associated data verification set D _2,vali fine-tuning the pre-trained ECG self-supervised model to complete the training of the model;

Model testing, based on the associated data test set D _2,test , testing the fine-tuned ECG model, evaluating the model effect, if the model evaluation result does not meet the preset requirements, adjusting the model parameters, and repeating the model pre-training and fine-tuning the model until the evaluation result of the model meets the preset requirements;

In the application stage, the collected multi-lead ECG signals and patient characteristic information are input into the trained model, and the probability information corresponding to various types of ECG signals is obtained.

Preferably, the patient characteristic information includes age, gender, and abnormality of the ECG signal, and the ECG signal type label represents whether the multi-modal data matching the part of the multi-lead ECG signal contains a certain type of ECG signal. Information on vascular diseases, the multimodal data is CT, ultrasound, contrast or nuclear magnetic data collected for the same patient, the cardiovascular diseases include adult congenital heart disease, valvular disease, coronary heart disease, cardiomyopathy, pulmonary vascular disease at least one.

Preferably, the model pre-training includes the following steps:

a. Randomly initialize the model parameters;

b. On the multi-lead ECG signal training set D _1,train and the multi-lead ECG signal verification set D _1,vali , the model-based cutter, dual classification masker, encoder, Decoder for self-supervised learning;

c. On the associated data training set D _2,train and the associated data verification set D _2,vali , perform penalty self-supervised learning based on the model's cutter, dual classification masker, encoder, decoder and classifier .

Preferably, the self-supervised learning of the model-based cutter, double classification masker, encoder, and decoder comprises the following steps:

Forward propagation, the ECG signal in the D1 _{, train} is passed through the cutter and the double classification masker in turn to obtain the self-training vector group and the transformed self-estimated vector group, the self-training vector group and a classification After the vectors are spliced, they pass through the encoder and the decoder in turn to output a set of prediction vectors as the estimated result of the transformed self-estimated vector group; wherein the classification vector is a preset learnable classification vector;

Parameter update, with the self-supervised loss function reflecting the error between the predicted vector and the transformed self-estimated vector group as the objective function, using an optimizer on D _{1, train} to update all available parameters in the encoder and decoder learning parameters;

Selecting the optimal first hyperparameter combination on the validation set D _1,vali minimizes the self-supervised loss function compared to the sum of all learnable parameters in the decoder.

Preferably, the transformation includes at least one of sampling dimensionality reduction, element-to-element power exponent, normalization within a vector, and classification according to a threshold, and the self-supervised loss function includes l ₊₁ loss, l ₊₂ loss, cross-entropy One of the loss functions; the first hyperparameter combination includes hidden dimensions and attention heads of the Transformer sub-block of the decoder in the model.

Preferably, the penalized self-supervised learning includes the following steps:

Forward propagation, the ECG signal in D _{2, train} is passed through the cutter and the double classification masker in turn to obtain the self-training vector group and the transformed self-estimated vector group, and the self-training vector group is spliced with a classification vector After passing through the encoder, output the coded self-training vector group and the coded classification vector, wherein the one classification vector is a preset learnable classification vector; the coded self-training vector group and the coded The classification vector enters branch one, and the encoded classification vector and the patient characteristic information in the D2 _{, train} enter branch two;

Branch 1, the decoder processes the input encoded self-training vector group to obtain a prediction vector, which is used to estimate the transformed self-estimation vector group;

Branch 2, the encoded classification vector and the patient characteristic information in the D _{2, train} are input into a classifier for processing to obtain the predicted probability of the ECG signal type;

Parameter update, with the penalty loss function as the objective function, use the optimizer to update all the learnable parameters in the encoder, decoder and classifier on D _2,train, the penalty loss function is about the prediction vector and the transformed self Estimate the self-supervised loss function of the vector group + λCrossEntropy, where CrossEntropy represents the cross-entropy loss of the predicted probability of the ECG type and the ECG type label, and λ is the hyperparameter ten;

Selecting the optimal hyperparameter λ on the verification set D _2,vali maximizes the selection metric for ECG type identification, and the selection metric includes one of AUC, F _β -score, and accuracy.

Preferably, the ECG self-supervised model includes:

A cutter for cutting each input ECG signal into a number of K rows and a number of columns of d _patch disjoint d _v sub-matrixes, and vectorizing the sub-matrices to obtain d v sub-matrices whose number of elements is d _v ECG signal full vector set {x _1,…, x _dv }, where d _patch is hyperparameter 1, K is the number of ECG leads;

The dual classification masker is used to receive the full vector set of ECG signals {x _1,…, x _dv }, and randomly extract T+T′ vectors with equal probability of non-replacement, where, T+T′≤d _v , the former T vectors constitute a self-training vector group, and the latter T′ vectors constitute an estimated vector group, T and T′ are hyperparameters three and hyperparameters four, respectively, and the output is a self-training vector group and a self-estimated vector group, and then the self-estimation The vector group is transformed to obtain the transformed self-estimated vector group;

The encoder is composed of a sequentially connected projection layer, a position embedding layer, and L Transformer sub-blocks whose hidden dimension is d _encoder and the attention head is h _encoder , where L is the hyperparameter five, and d _encoder is the super Parameter six, h _encoder is hyperparameter seven, in the pre-training phase, the input of the encoder is a self-training vector group and a classification vector, and the output is the encoded self-training vector group and classification vector; in the fine-tuning and testing phase, the encoding The input of the device is the full vector group of electrocardiographic signals and a classification vector, and the output is a group of encoded electrocardiographic signal full vector groups and a classification vector, wherein the classification vector is a learnable classification vector added artificially;

The decoder consists of a restoration layer, a position embedding layer, a Transformer sub-block whose hidden dimension is d _decoder , and the attention head is h decoder, and a fully _connected layer, where d _decoder is the hyperparameter eight, where h _decoder For hyperparameter nine, the decoder is only used in the pre-training stage, the decoder inputs the encoded self-training vector group and classification vector, and decodes the output prediction vector;

The classifier is composed of a fully connected layer and an activation layer. The input is the encoded classification vector and feature information, and the output is the predicted probability value of the ECG signal type.

Preferably, the data preprocessing includes the following steps:

a. Filter and denoise the ECG signal;

b. Normalize the ECG signal after filtering, so that the data range is between -1 and 1;

c. Fill the columns with a value of 0 for the standardized ECG signal to ensure that the number of columns after filling can divide the hyperparameter one, and obtain the ECG signal X _i , i=1,...n;

d. Perform min-max standardization processing on the numerical variables in the patient characteristic information, perform 0-1 coding on the categorical variables in the patient characteristic information, and obtain the patient characteristic information z _j of the ECG, j=1,...m ;

e. Obtain multi-lead ECG data set D ₁ and associated data set D ₂ , where D ₁ ={X _i :i=1,...,n} represents all ECG signals; D ₂ ={(X _{j ,} z _j , y _j ) :j=1,…,m} represents the associated data set, where X _j represents the multi-lead ECG signal, z _j represents the patient characteristic information associated with the multi-lead ECG signal, and y _j Represents the ECG signal type label associated with a multi-lead ECG signal.

Preferably, the model fine-tuning includes the following steps:

Forward propagation, the ECG signal in D _{2, train} is input into the cutter for processing, and the full vector group of the ECG signal is obtained, and then the full vector group of the ECG signal and the classification vector are spliced and input to the encoder for processing, and the encoded The electrocardiographic signal full vector group and the classification vector after encoding; The patient characteristic information input classifier in the classification vector after described encoding and D _2'train is processed, obtains the predictive probability of electrocardiographic signal type;

Parameter update, using the cross-entropy loss of the predicted probability of the ECG signal type and the ECG signal type label in D _{2, train} as the objective function, use the optimizer to update all available encoders and classifiers on D _{2, train} learning parameters;

Select the optimal second hyperparameter combination on the associated data verification set D _2,vali , so that the selection metric index of ECG type identification is the largest; the second hyperparameter combination includes The number of columns d _patch , the number of sub-matrices d _v after cutting the electrical signal of the center of the cutter, the number of vectors T of the self-training vector group, the number of vectors T' of the self-estimated vector group, and the The number of Transformer sub-blocks included in the encoder, the hidden dimension L of the Transformer sub-block in the encoder, and the attention head h _encoder of the Transformer sub-block in the encoder.

Preferably, the model testing includes the following steps:

On the associated data test set D _2,test , evaluate the effect of the model by selecting the metric, if the model evaluation result meets the preset requirements, the model is allowed to be used, if not, adjust the model parameters, repeat the model pre-training and fine-tuning steps , until the model evaluation results meet the preset requirements.

A type recognition system for multi-lead ECG signals, including a data acquisition module, a data preprocessing module, a data set division module, a model generation module, and a service calculation module, wherein

The data acquisition module is used to obtain training data, including n multi-lead ECG signals, and patient characteristic information associated with part of the multi-lead ECG signals in the n multi-lead ECG signals and ECG signal type label;

The data preprocessing module is used to generate a multi-lead ECG data set D ₁ representing all ECG signals based on the n multi-lead ECG signals, and the corresponding sample size is n; Combine the ECG signal and the patient characteristic information associated with the multi-lead ECG signal and the ECG signal type label, and eliminate the multi-lead ECG signal with missing patient characteristic information or ECG signal type label to generate an associated data set D ₂ , the corresponding sample size is m, where m≤n;

The data set division module is used to divide the multi-lead ECG data set _D1 into a multi-lead ECG training set D1 _,train and a multi-lead ECG verification set D1 _,vali , dividing the associated data set D ₂ into an associated data training set D _2,train , an associated data verification set D _2,vali , an associated data test set D _2,test ;

The model generation module is used to complete model training based on multi-lead ECG signals, feature information, label information and built model framework, and obtain and store a trained ECG self-supervised model;

The service computing module is used to receive a request for ECG signal type identification, and invoke a trained ECG self-supervised model to obtain probability information corresponding to various types of ECG signals.

Preferably, the model generation module includes a sample library, a model training engine and a model library, wherein the sample library is the multi-lead ECG data set D ₁ and the associated data set D ₂ sent by the receiving data set division module, And complete the storage; the model training engine is based on the multi-lead ECG signal data set and associated data set stored in the sample library, and completes the model training; the model library is used to store the trained ECG self-supervised model;

The service calculation module includes a service trigger engine and a model calculation engine; wherein, the service trigger engine is used to receive the ECG type identification request and the data carried in the request, and send them to the model calculation engine, wherein the request carried The data includes multi-lead ECG signals and patient characteristic information; the model calculation engine is used to call the trained ECG self-supervised model, based on the multi-lead ECG signals and patient characteristic information carried in the request, the obtained and set The probability information corresponding to various ECG signal types, and complete the result storage.

Preferably, the type identification system of the multi-lead ECG signal also includes a front-end interaction module and a dynamic monitoring module,

The front-end interaction module includes a recognition result presentation submodule and a label storage submodule; wherein the recognition result presentation submodule is used to provide visual prompts based on the probability information corresponding to various ECG signal types obtained by the service calculation module; The label storage sub-module is used to automatically obtain the final ECG signal type label generated during the application process and complete the storage;

The dynamic monitoring module includes a service monitoring and evaluation sub-module and a service update trigger engine; wherein, the service monitoring and evaluation sub-module is used to evaluate the model recognition effect in real time based on automatically accumulated type tags generated in the application process; the service update trigger engine , which is used to automatically trigger the update of the model and service when the model effect does not meet the preset requirements, so as to realize the dynamic optimization update of the model.

Preferably, the system for identifying the type of the multi-lead ECG signal also includes an ECG interpretation module, the ECG interpretation module has a built-in ECG interpretation model for interpreting the ECG, and the type of the ECG signal is identified as Arrhythmia condition.

An intelligent ECG auxiliary system, including the type recognition system of the multi-lead ECG signal, also includes a knowledge base, the knowledge base stores processing suggestions, when the type identification system of the multi-lead ECG signal gives After the type recognition result is obtained, the knowledge base is invoked, and the processing suggestions in the knowledge base that meet the preset conditions are output.

An electronic device, comprising: a processor;

The memory stores a program configured to implement the method for identifying the type of the multi-lead ECG signal when executed by the processor.

A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium stores instructions, it is characterized in that, when the instructions are executed by the processor, the processor executes the multi-lead electrocardiographic signal type identification method.

The present invention constructs an ECG self-supervised learning model, and can effectively use the ECG signal data that cannot be used in the prior art through the self-supervised learning method, improve the utilization rate of the ECG signal data, and appropriately reduce the sample size requirement for multi-modal data resources , promote the research on the relationship between mining ECG signals and various types of ECG signals (such as various types of diseases), promote type identification based on ECG signals, and potentially expand the application boundary of current ECG interpretation.

Description of drawings

Fig. 1 is the flow chart of the type identification method of the multi-lead electrocardiogram signal of embodiment 1 of the present invention;

Fig. 2 is the model pre-training flowchart of embodiment 1 of the present invention;

FIG. 3 is a model architecture diagram of Embodiment 1 of the present invention;

4 is a schematic diagram of a type identification system for a multi-lead ECG signal according to Embodiment 1 of the present invention;

5 is a schematic diagram of a model generation module in Embodiment 1 of the present invention;

6 is a schematic diagram of a type identification system for multi-lead ECG signals according to Embodiment 2 of the present invention;

7 is a schematic diagram of a type identification system for multi-lead ECG signals in Embodiment 2 of the present invention (with an ECG interpretation module);

FIG. 8 is a schematic diagram of an electronic device according to Embodiment 2 of the present invention.

Implementation

In order to make the purpose, technical means and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings.

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some, not all, embodiments of the application. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

The terms "comprising" and "having" in the description and claims of the present application and the above drawings and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include steps or units not explicitly listed or for these processes, methods, products, or Other steps or units inherent to equipment.

The technical solution of the present application will be described in detail below with specific examples. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

Embodiment 1;

As shown in Figure 1, a kind of type identification method of multi-lead ECG signal, comprises the following steps:

Data acquisition, collecting n multi-lead ECG signals, and patient characteristic information and ECG signal type label information associated with part of the multi-lead ECG signals in the n multi-lead ECG signals;

During the data acquisition process, not every multi-lead ECG signal can collect the associated patient ECG signal type label information. Some multi-lead ECG signals lack the patient ECG signal type label, and this part is missing The multi-lead ECG signal with the associated ECG signal type label information cannot be used in the existing supervised learning-based technology; and in this application, for the multi-lead ECG signal lacking the associated ECG signal type label information The ECG signal can also be applied to the training of the ECG self-supervised model; based on this, the patient feature information and the ECG signal type label collected in the data acquisition operation can be related to the m in the n multi-lead ECG signals 'The multi-lead ECG signals are associated, that is to say, for the collected n multi-lead ECG signals, only some of the multi-lead ECG signals have associated patient characteristic information and ECG signal type labels ;m'≤n;

Data preprocessing, generating a multi-lead ECG data set D ₁ representing all ECG signals based on the n multi-lead ECG signals, the corresponding sample size is n; based on part of the multi-lead ECG signals and The patient characteristic information associated with the multi-lead ECG signal and the ECG signal type label, the multi-lead ECG signal with missing ECG signal type label is eliminated, and the associated data set D ₂ is generated, and the corresponding sample size is m, where m≤ m'≤n; that is, in the multi-lead ECG data set D ₁ , there are m' multi-lead ECG signals with associated patient characteristic information and ECG signal type label information, and there are m in m' The multi-lead ECG signals are used to form the associated data set D ₂ ;

That is to say, after preprocessing the n multi-lead ECG signals collected in the aforementioned data collection step, n ECG signals X _i are obtained, where i=1,...n, n ECG signals form a multi-lead Joint ECG signal data set D ₁ ={X _i :i=1,...,n}, for m multi-lead ECG signals with associated ECG signal type labels in the collected n multi-lead ECG signals , preprocess the m multi-lead ECG signals and the patient characteristic information and ECG signal type label information associated with the m multi-lead ECG signals to obtain m ECG signals X _j and m preprocessed The final patient characteristic information z _j and m ECG signal type label information y _j form the associated data set D2={(X _j ,z _j ,y _j ):j=1,...,m}.

In this embodiment, m=8000, n=300000, and the number of m is much smaller than n.

dividing the data set, dividing the multi-lead ECG data set D ₁ into a multi-lead ECG training set D _1,train and a multi-lead ECG verification set D _1,vali , and dividing the associated data Set D ₂ is divided into associated data training set D _2,train , associated data verification set D _2,vali , associated data test set D _2,test ;

When dividing, the ECG signal data in D _{2, train} belongs to D _{1, train} , and the ECG signal data in D _{2, vali} belongs to D _{1, vali} . Among them, the probability of D _{2, train} , D _{2, vali} and D _{2, test} belonging to the same type of ECG signal type is almost the same. The divided multi-lead ECG signal training set D _1,train and multi-lead ECG signal verification set D _1,vali include a large number of missing associated patient characteristic information and ECG signals that cannot be used in the prior art ECG signal of type label information. Wherein, preferably, the multi-lead ECG signal training set D _1,train and the multi-lead ECG signal verification set D _1,vali are mutually disjoint, and the associated data training set D _2,train and the associated data verification set D _{2, vali} and associated data test set D _2,test are mutually disjoint.

ECG self-supervised model frame construction, as shown in Figure 3, said model is based on Transformer module, including cutter, double classification masker, encoder, decoder and classifier;

Model pre-training, initialize model parameters, and then the multi-lead ECG signal training set D _1,train , multi-lead ECG signal verification set D _1,vali , associated data training set D _2,train , associated data Verification set D _{2, vali} input model framework, self-supervised learning, penalty self-supervised learning, get pre-trained ECG self-supervised model;

Model fine-tuning, based on the associated data training set D _2,train and the associated data verification set D _2,vali fine-tuning the pre-trained ECG self-supervised model to complete the training of the model;

Model testing, based on the associated data test set D _2,test , test the fine-tuned ECG model, evaluate the model effect, if the model evaluation result does not meet the preset requirements, adjust the model parameters, repeat the model pre-training and model Steps of fine-tuning until the evaluation results of the ECG self-monitoring model meet the preset requirements;

In the application stage, the collected multi-lead ECG signals and associated patient characteristic information are input into the trained ECG self-supervised model, and the probability information corresponding to various types of ECG signals is obtained.

The present invention constructs an ECG self-supervised learning model. In the model pre-training, the self-supervised learning method utilizes ECG signal data that cannot be used in the prior art, such as the multi-lead ECG signal training set D _1,train , and the multi-lead ECG signal training set D 1,train . The electrocardiographic signal verification set D _{1, the ECG signal missing the ECG signal category label in vali} , takes into account the information of the ECG signal itself and the ECG signal category information corresponding to the ECG signal, and only uses a small amount of ECG signal type label information , such as the data in the associated data training set D _2,train , the associated data verification set D _2,vali , and the associated data test set D _2,test , the ECG self-supervised model for ECG signal type identification can be obtained, which can be used It is used for type identification based on ECG signals. The model provided by the present invention can identify various ECG signal types that are difficult for human eyes to detect, including ECG signal types associated with adult congenital heart disease, valvular disease, coronary heart disease, cardiomyopathy, and pulmonary vascular disease. Because the ECG signal type label information used is less, it provides a new solution to the problem for many medium-sized medical institutions/research laboratories with insufficient data resources.

It can be seen from the entire processing flow of the type identification method of the above-mentioned ECG signal, which specifically includes three parts: preparation of training data, training process of the ECG self-supervised model, and using the trained ECG self-supervised model to identify the type of ECG signal. identify. Among them, data collection, data preprocessing and data set division are used to prepare the training data, construct the structure of the ECG self-supervised model, model pre-training, model fine-tuning and model testing to form the complete training process of the ECG self-supervised model. A suitable ECG self-supervised model is obtained through training, and then the ECG self-supervised model can be used to conveniently and effectively identify the types of ECG signals. Next, the training process of the ECG self-supervised model is described in detail.

In the training of the ECG self-supervised model, the training data used includes not only the collected multi-lead ECG signals, but also patient characteristic information and ECG signal type label information associated with some multi-lead ECG signals.

Wherein, the patient characteristic information includes age, gender, abnormality of ECG signal, wherein, abnormality of ECG signal can be obtained by manual marking of ECG signal by experts, and the abnormality includes but not limited to ST-T change, Abnormal T wave, left ventricular high voltage, sinus bradycardia, abnormal Q wave, atrial fibrillation, complete right bundle branch block, sinus arrhythmia, ST segment changes, ventricular premature beats, abnormal P wave, incomplete Right bundle branch block, left axis deviation, first-degree atrioventricular block, sinus tachycardia, atrial premature beats without conduction, right ventricular hypertrophy, ventricular pacing rhythm, nonspecific intraventricular block, complete left Bundle branch block, left anterior fascicular block;

, the ECG signal type label represents whether the multimodal data matching the part of the multi-lead ECG signal contains information about a certain cardiovascular disease, that is, the label is a label that the ECG signal belongs to a certain type; Optionally, it can be manually judged by experts, wherein the cardiovascular diseases include adult congenital heart disease, valvular heart disease, coronary heart disease, cardiomyopathy, and pulmonary vascular disease. Valvular disease includes, but is not limited to, aortic stenosis, aortic regurgitation, mitral stenosis, mitral regurgitation. Congenital heart disease includes but not limited to atrial septal defect, ventricular septal defect. Cardiomyopathy includes, but is not limited to, hypertrophic cardiomyopathy. Coronary heart disease includes but is not limited to acute myocardial infarction and angina pectoris. Pulmonary vascular disease includes, but is not limited to, pulmonary hypertension and pulmonary embolism. For valvular disease, cardiomyopathy, coronary heart disease, and pulmonary vascular disease, the multimodal data that matches the patient's ECG signal is the CT, ultrasound, contrast-enhanced or MRI data collected by the same patient within 90 days before and after the ECG signal is collected. Data; for adult congenital heart disease, the multimodal data matching the patient's ECG signal is the ultrasound data of the same patient at any time.

As mentioned above, the training of the ECG self-supervised model includes building the model structure, model pre-training, model fine-tuning and model testing. In the following, combined with the built model structure, the three processes of model pre-training, model fine-tuning and model testing will be detailed respectively. describe.

As shown in Figure 2, wherein, the pre-training process may specifically include the following steps:

a. Randomly initialize the model parameters;

b. On the multi-lead ECG signal training set D _1,train and the multi-lead ECG signal verification set D _1,vali , the cutter, dual classification masker, encoder, and decoder based on the ECG self-supervised model conduct self-supervised learning;

As mentioned above, before training the self-supervised model, the training data for the ECG self-supervised model is prepared through data acquisition, data preprocessing and data set division. Among them, the multi-lead ECG signal training set D _1,train , the multi-lead ECG signal verification set D _1,vali , and the associated data training set D _{2, are obtained after data acquisition, data preprocessing and data set division. train} , associated data validation set D _2,vali and associated data test set D _2,test ;

c. On the associated data training set D _2,train and the associated data verification set D _2,vali , perform penalty self-supervised learning based on ECG self-supervised model cutter, dual classification masker, encoder, decoder and classifier .

The random initialization of model parameters in step a may include using xavier uniform initialization for all Transformer blocks, and using normal distribution initialization for other parameters.

The processing of the self-supervised learning of step b may specifically include the following steps:

Forward propagation, the ECG signals in D _{1, train} , are sequentially passed through the cutter and the double classification masker to obtain the self-training vector group and the transformed self-estimated vector group, and then, the self-training vector group and a classification vector Splicing (the specific splicing method can be similar to that in the vision transformer, for example, the classification vector is equivalent to a self-training vector in the self-training vector group and the self-training vector group is spliced) and then passed through the encoder and decoder in turn to obtain the output of the decoder A set of prediction vectors is used to estimate the transformed self-estimation vector group, that is to say, the prediction vector output by the decoder is used as the estimated value of the transformed self-estimation vector group, wherein the classification vector is preset (specifically can be artificially added) learnable classification vector; wherein, the classification vector is a vector with a length of K ×d _patch , and each of its components is a learnable (trainable) parameter; D _{1, ECG in train} The dimension of the signal is an integer multiple of K ×d _patch , where K is the number of ECG leads, and d _patch is hyperparameter one. The learnable parameter here can be regarded as a weight parameter of a neural network. During the training process of the model, it will be iteratively updated like other weight parameters in the neural network.

The parameters are updated to reflect the prediction vector (i.e., the estimated value of the transformed self-estimated vector set output by the decoder) and the actual value of the transformed self-estimated vector set (i.e., the transformed self-estimated vector set output by the dual classification masker The self-supervised loss function of the error between groups) is the objective function, and the optimizer is used to update all the learnable parameters in the encoder and decoder on D _1,train to reduce the actual selection of the predicted vector and the transformed self-estimated vector group The deviation between the values, and synchronously update the classification vector that needs to be input into the encoder and the self-training vector group splicing when the next iteration is trained; the optimizer used in this embodiment is the AdamW optimizer with the cosine learning rate scheduler, The base learning rate is 0.001, the weight decay is 0.05, the batch size is 256, the optimizer impulse β ₁ =0.9, β ₂ =0.95, the number of warm-up iterations is 40, and the total number of iterations is 400;

The aforementioned process of forward propagation and parameter update continues to iterate until the training termination condition is met (that is, the set maximum number of iterations is reached), then a training model is obtained, and the process of obtaining a training model after multiple iterations is called one-time The training process for self-supervised learning.

Next, select the optimal first hyperparameter combination on the verification set D _1,vali output by the double classification masker, (the first hyperparameter combination includes hyperparameter eight d _decoder , hyperparameter nine h _decoder ), Minimize the self-supervised loss function compared to the sum of all learnable parameters in the decoder. Here, the comparison result of the self-supervised loss function and the total amount of all learnable parameters in the decoder is selected as the optimal combination selection criteria of hyperparameters eight and hyperparameters nine, considering that the larger the total amount of all learnable parameters in the decoder is , the larger the scale of the decoder, the smaller the self-supervised loss function will be, and the model will be more accurate in estimating the transformation of the self-estimated vector group in the pre-training stage. However, the calculation amount of the model will increase accordingly. Furthermore, numerical experiments show that when the decoder grows much larger than the smaller magnitude of the self-supervised loss function, the model generalization decreases during the fine-tuning stage. Based on this, this application considers a balance between the self-supervised loss and the amount of decoder parameters. On the one hand, this balance can balance the self-supervised error and calculation amount in the pre-training stage, and on the other hand, it can help to obtain better generalization. strong model.

In addition, explain the specific operations of performing self-supervised learning training on the training set and selecting the optimal first hyperparameter combination on the verification set. Assuming that there are N groups of first hyperparameter combinations to be selected, for each first hyperparameter combination, use the training set D _{1, train} to perform the aforementioned complete training process of self-supervised learning (that is, the previous self-supervised learning in the aforementioned self-supervised learning To the process of propagating and parameter updating multiple iterations to generate a model), a corresponding model is obtained, and the value of the total amount of all learnable parameters in the decoder of the model is determined; for all first hyperparameter combinations, N models are obtained , the value of the total amount of all learnable parameters in the decoder corresponding to the N models can be different. For each of the N models, the signal of the verification set D _1,vali is input to the model, after processing, the value of the self-supervised loss function is obtained, and then compared with the total amount of all learnable parameters in the decoder of the model , to get the comparison result; select the smallest one from the comparison results of the N models, the first hyperparameter combination corresponding to the model is the optimal first hyperparameter combination, and use the selected model as the next step for penalty self-supervision The initial model for training.

In the above self-supervised learning process, the transformation of the self-estimated vector can include sampling dimensionality reduction, element-to-element power exponent, normalization within the vector, and classification according to a threshold. The self-supervised loss function can include l ₁ loss, l One of the ₂ loss, cross-entropy loss function, transformation, and self-supervised loss function are all existing common knowledge.

The hyperparameters in the ECG self-supervised model of this application include hyperparameters 1 to 10, and hyperparameters 1 to 10 will be described later.

Note that since the double classification masker is random in the extraction of the full vector set of the ECG signal, it is necessary to fix the self-training vector set and the self-estimated vector set corresponding to each ECG sample in the verification stage, that is, for each The ECG signal only uses the dual classification masker once to obtain the self-training vector group and the self-estimated vector group, and save them, and use these saved self-training vector groups and self-estimated vector groups to select the best The optimal first hyperparameter combination, namely hyperparameter eight and hyperparameter nine;

The penalized self-supervised learning of step c includes the following steps:

For forward propagation, the ECG signals in D _{2, train} are sequentially passed through the cutter and the double classification masker to obtain the self-training vector group and the transformed self-estimated vector group, wherein the self-training vector group is spliced with a classification vector ( The specific splicing method can be the same as the splicing method in self-supervised learning), and then input to the encoder for processing, and the encoder outputs the encoded self-training vector group and the encoded classification vector, wherein the classification vector is preset (specifically can be is the learnable classification vector of artificially added); the encoded self-training vector group and the encoded classification vector enter branch 1 for processing, and the encoded classification vector and patient feature information in D _{2, train} enter branch 2 for processing; In the process of punishing self-supervised learning, the transformation of the self-estimated vector group is the same as that in the self-supervised learning, which is a prior art.

Branch 1, a prediction vector is obtained through the decoder, which is used to estimate the transformed self-estimation vector group, that is, the prediction vector output by the decoder is used as the estimated value of the transformed self-estimation vector group;

Branch 2, the encoded classification vector and D _{2, the patient feature information in the train} are obtained through the classifier to obtain the corresponding preset prediction probabilities of various ECG signal types;

Parameter update, with the penalty loss function as the objective function, use the optimizer on D _{2, train} to update all the learnable parameters in the encoder, decoder and classifier, the penalty loss function is the self-supervised loss function + λCrossEntropy, where The self-supervised loss function is the same as the self-supervised loss function in the self-supervised learning process. CrossEntropy represents the cross-entropy loss of the prediction probability of the ECG signal type (that is, the disease prediction probability) and the ECG signal type label information, and λ is the hyperparameter dozens;

The optimizer used in this embodiment is an AdamW optimizer with a cosine learning rate scheduler, the basic learning rate is 0.001, the weight decay is 0.05, the batch size is 256, the optimizer impulse β ₁ =0.9, β ₂ =0.999, The number of warm-up iterations is 10, and the total number of iterations is 100;

λ is the penalty weight in the penalty loss function, which can take a preset value;

The process of forward propagation and parameter update is iterated continuously until the training termination condition is met (for example, the maximum number of iterations is reached), and then a training model is obtained. The process of obtaining a training model after multiple iterations is called one-time Penalizing the training process for self-supervised learning.

Next, select the optimal hyperparameter λ on the verification set D _2,vali , so that the selection metric index of ECG signal classification is the largest, and the selection metric index includes one of AUC, F _β -score, and accuracy . The value range of λ can be 0.00001-10, in this embodiment, the optional value of λ can be 0.05, 0.1 or 0.5;

Among them, AUC (Area Under Curve) is the area enclosed by the receiver operating characteristic curve (ROC) curve and the coordinate axis, and the definition of F _β -score is as follows:

，

,

Among them, β takes 0.5, 1 or 2, precision is the precision rate, which represents the proportion of the number of samples that actually belong to the corresponding type among the samples judged by the model to belong to a certain type of ECG signal, and recall is the recall rate, which represents the number of samples that actually belong to a certain type of ECG signal. Among the samples of one type of ECG signal, the proportion of the number of samples determined by the model to belong to the corresponding type of ECG signal; the accuracy rate refers to the accuracy rate of disease classification; the above measurable indicators are common knowledge.

In addition, explain the complete operation of punishing self-supervised learning training on the training set and selecting the optimal hyperparameters on the verification set. Similar to the aforementioned selection of the optimal first hyperparameter combination, assuming that there are M hyperparameters λ to be selected, for each value, the model finally selected in self-supervised learning is used as the initial model, and the training set is used D _{2, train} executes the aforementioned complete training process of penalty self-supervised learning (that is, the process of forward propagation and parameter update multiple iterations in the aforementioned penalty self-supervised learning), and obtains a corresponding model; for all λ Take the value to get M models, for each of the M models, input the ECG signal in the verification set D _2,vali to the cutter for processing to obtain the ECG full signal vector group, and then convert the ECG full signal vector group After splicing with the classification vector, input the encoder for processing, and obtain the encoded ECG full signal vector group and the encoded classification vector, wherein, according to the structure of the transformer, the encoded classification vector depends on the ECG signal full vector group; finally Input the encoded classification vector and the patient characteristic information in D _2,vali into the classifier, and obtain the predicted probability of the ECG signal type after processing, and then integrate all the predicted probabilities to determine the value of the selected measurement index for the ECG signal type ; Select the largest one from the selection metric values of the M models, and the value of the hyperparameter λ corresponding to the model is the optimal λ, and the selected model will be used as the initial stage of the model fine-tuning process in the next step Model.

As mentioned above, as shown in Figure 3, the ECG self-supervised model constructed in this application includes a cutter, a dual classification masker, an encoder, a decoder, and a classifier. Next, the specific functions of each component are described in detail. Introduction, and a detailed description of the ten hyperparameters:

A cutter for cutting each input ECG signal into a number of K rows and a number of columns of d _patch disjoint d _v sub-matrixes, and vectorizing the sub-matrices to obtain d v sub-matrices whose number of elements is d _v ECG signal full vector group {x ₁ ,…,x _dv }, where d _patch is hyperparameter 1, d _v is hyperparameter 2, and K is the number of ECG leads;

Optionally, K is 12, hyperparameter one d _patch is 10-200, and hyperparameter two d _v is 25-500. In this embodiment, K=12, d _patch =25, and d _v =200.

The dual classification masker is used to receive the full vector set of ECG signals {x ₁ ,…,x _dv }, and randomly extract T+T′ vectors with equal probability of no replacement, where, T+T′≤d _v , the former T vectors constitute a self-training vector group, and the latter T′ vectors constitute an estimated vector group, T and T′ are hyperparameters three and hyperparameters four, respectively, and the output is a self-training vector group and a self-estimated vector group; then the self-estimation The vector group is transformed to obtain the transformed self-estimated vector group;

Hyperparameter three T is 5-400, hyperparameter four T' is 5-400, in this embodiment, T=50, T'=100.

The encoder is composed of a sequentially connected projection layer, a position embedding layer, and L Transformer sub-blocks whose hidden dimension is d _encoder and the attention head is h _encoder , where L is the hyperparameter five, and d _encoder is the super Parameter six, h _encoder is hyperparameter seven, in the pre-training phase, the input of the encoder is a self-training vector group and a classification vector, and the output is the encoded self-training vector group and classification vector; in the fine-tuning and testing phase, the encoding The input of the device is the full vector group of electrocardiographic signals and a classification vector, and the output is a group of encoded electrocardiographic signal full vector groups and a classification vector, wherein the classification vector is a learnable classification vector added artificially;

Among them, the hyperparameter five L is 1-32, the hyperparameter six d _encoder is 128-1280, and the hyperparameter seven h _encoder is 3-16. In this embodiment, L=12, d _encoder =384, h _encoder =6 Or L=12, d _encoder =256, h _encoder =4.

The decoder consists of a restoration layer, a position embedding layer, a hidden dimension of d _decoder , an attention head of h _decoder , a Transformer sub-block, and a fully connected layer, where d _decoder is a hyperparameter eight, where h _Decoder is hyperparameter nine, and the decoder is only used in the pre-training stage. The decoder inputs the encoded self-training vector group and classification vector, and decodes and outputs the prediction vector;

The hyperparameter eight d _decoder is 128-1280, and the hyperparameter nine h _decoder is 4-10. In this embodiment, d _decoder =256, h _decoder =8 or d _decoder =128, h _decoder =4.

The classifier consists of a fully connected layer and an activation layer with sigmoid as the activation function. The input is the encoded self-training vector group output by the encoder, and the output is the predicted probability value of the ECG signal type.

In addition, the aforementioned data preprocessing can be performed according to the following steps:

a. Filter and denoise the ECG signal;

b. Normalize the ECG signal after filtering, so that the data range is between -1 and 1;

c. Fill the columns with a value of 0 for the standardized ECG signal to ensure that the number of columns after filling can divide the hyperparameter one, and obtain the ECG signal X _i , i=1,...n;

d. Perform min-max standardization processing on the numerical variables in the patient characteristic information, perform 0-1 coding on the categorical variables in the characteristic information, and obtain the patient characteristic information z _j of the ECG, j=1,...m;

e. Obtain multi-lead ECG data set D ₁ and associated data set D ₂ , where D ₁ ={X _i :i=1,...,n} represents all ECG signals; D ₂ ={(X _j ,z _j ,y _j ):j=1,…,m} means that after removing the multi-lead ECG signal with missing patient characteristic information or ECG signal type label, the remaining ECG signal and patient characteristic information, ECG signal Type label, where X _j is the remaining ECG signal after excluding the multi-lead ECG signal with patient characteristic information or ECG signal type label missing, z _j is patient characteristic information, and y _j is the ECG signal type label.

The multi-lead electrocardiographic signal is a numerical matrix of K×S, wherein K is the number of leads, and S is the number of sample points collected. First, the electrocardiographic signal is filtered and denoised; Standardize the electrical signal so that the data range is between -1 and 1; fill the column with a value of 0 for the standardized ECG signal to ensure that the number of filled columns can divide the hyperparameter one, and obtain the ECG signal X _i , i=1,...n; perform min-max standardization processing on the numerical variables in the feature information, and perform 0-1 encoding on the categorical variables in the feature information to obtain the artificial feature z _j of ECG, j=1, ...m, where the numerical variable means that the value is numerical data, such as age, the categorical variable is a name of the object category, and the value is categorical data, such as gender; obtain multi-lead ECG signal data set D ₁ , associated data set D ₂ , where D ₁ ={X _i :i=1,…,n} represents all ECG signal observations; D ₂ ={(X _j ,z _j ,y _j ):j=1 ,...,m} represent the remaining ECG signals, patient characteristic information, and ECG signal type label information after excluding multi-lead ECG signals with missing patient characteristic information or ECG signal type label information; where y _j is ECG Signal type label.

For example, i=1, X ₁ is a 12*5000 numerical matrix; j=1, z ₁ is a one-dimensional array, (0.5,1,1,1,0,1,1,1,0,1 ,1,1,0,1,1,1,0,1,1,1,0,1,0), where 0.5 means age, the second 1 means male, and the following 1 or 0 means something Whether there is any abnormality in the ECG signal; y ₁ =0, which means that there is no cardiovascular disease.

The model fine-tuning includes the following steps:

Forward propagation, the ECG signal in D _{2, train} is input to the cutter for processing, and the full vector group of the ECG signal is obtained, and then the full vector group of the ECG signal and the classification vector are spliced and input to the encoder for processing, and the encoded The full vector group of electrocardiographic signals and the classification vector after encoding; The classification vector after encoding and the patient feature information input classifier in D _{2, train} are processed, obtain the prediction probability of electrocardiographic signal type;

Parameter update, using the cross-entropy loss function between the predicted probability of ECG type and the ECG type label in D _{2, train} as the objective function, using the optimizer to update the encoder and classifier on D _{2, train} All learnable parameters in ;

The process of forward propagation and parameter update is iterated continuously until the training termination condition is met (for example, the maximum number of iterations is reached), and then a training model is obtained. The process of obtaining a training model after multiple iterations is called one-time Fine-tuned training process.

Next, select the second optimal hyperparameter combination on the associated data verification set D _2,vali (specifically including hyperparameter one d _patch , hyperparameter two d _v , hyperparameter three T, hyperparameter four T′ , hyperparameter five L, hyperparameter six d _encoder , hyperparameter seven h _encoder ), so that the selection index for ECG signal classification is the largest.

Wherein the cross-entropy loss function is common knowledge, and the selected metrics include one of AUC, F _β -score, and accuracy; the optimizer used in this embodiment is an AdamW optimizer with a cosine learning rate scheduler, based on The learning rate is 0.001, the weight decay is 0.05, the batch size is 256, the optimizer impulse β ₁ =0.9, β ₂ =0.999, the number of warm-up iterations is 5, and the total number of iterations is 50.

In addition, combined with the aforementioned pre-training, explain the complete operation of pre-training and fine-tuning. Assuming that there are X second hyperparameter combinations to be selected, for each hyperparameter combination, using the training set D _1,train and D _2,train and the verification set D _1,vali and D _2,vali , through pre-training The optimal first hyperparameter combination, the optimal hyperparameter ten and the corresponding model A after penalty self-supervised learning are obtained, and the model A after penalty self-supervised learning is used as the initial model, and the training set is used to execute A complete fine-tuning training process in the aforementioned fine-tuning (that is, the process of forward propagation and multiple iterations of parameter updating in the aforementioned fine-tuning to generate a model), obtains a corresponding model; for all second hyperparameter combinations, X models are obtained, and for For each of the X models, the signal of the verification set D _{2, vali} is input into the encoder and classifier of the model, and after processing, the value of the selection metric index for ECG signal classification is obtained; from the selection metric of the X models Select the largest one of the index values, the second hyperparameter combination corresponding to the model is the optimal second hyperparameter combination, and use the selected model as the model to be tested, and perform subsequent test processing;

Described model test comprises the following steps:

On the associated data test set D _2,test , evaluate the effect of the model by selecting metrics. When the model evaluation result meets the preset requirements, the model can be used. If not, adjust the model parameters and repeat the model pre-training and fine-tuning steps. , until the model evaluation results meet the preset requirements.

In the stage of fine-tuning and testing, the input of the encoder is a full vector group of ECG signals and a classification vector, and the output is a set of encoded full vector groups of ECG signals and a classification vector after encoding, wherein the classification vector is artificial The added learnable classification vector; the encoded classification vector enters the classifier together with the patient feature information, and the output is the predicted probability value of the ECG signal type; compare the predicted probability value of the ECG signal type with the real value, and select the measurement index Evaluate model performance. The preset requirements are: AUC above 0.9 or F1-score above 0.75 or accuracy above 80%. In this embodiment, the AUC is above 0.9.

As shown in Figure 4, a type recognition system for multi-lead ECG signals includes a data acquisition module, a data preprocessing module, a data set division module, a model generation module, and a service calculation module, wherein

The data acquisition module is used to obtain training data, including n multi-lead ECG signals, and patient characteristic information and ECG signal types associated with part of the multi-lead ECG signals in the multi-lead ECG signals Label;

The data preprocessing module is used to generate a multi-lead ECG data set D ₁ representing all ECG signals based on the n multi-lead ECG signals, and the corresponding sample size is n; based on the n multi-lead ECG signals signal and the patient characteristic information and ECG signal type label information associated with the multi-lead ECG signal, the multi-lead ECG signal with the patient characteristic information or ECG signal type label information is eliminated, and the associated data set D ₂ is generated, corresponding to The sample size is m, where m≤n;

The data set division module is used to divide the multi-lead ECG data set _D1 into a multi-lead ECG training set D1 _,train and a multi-lead ECG verification set D1 _,vali , dividing the associated data set D ₂ into an associated data training set D _2,train , an associated data verification set D _2,vali , an associated data test set D _2,test ;

The model generation module is used to complete model training based on multi-lead ECG signals, feature information, label information and built model framework, and obtain a trained ECG self-supervised model;

The service computing module is used to receive a request for ECG signal type identification, and invoke a trained ECG self-supervised model to obtain probability information corresponding to various types of ECG signals.

This type identification system is based on the ECG self-supervised model, utilizes the ECG signal data that cannot be used in the prior art through the self-supervised learning method, and can obtain various ECG signal types with only a small amount of multi-modal data ( For example, the probability information corresponding to various types of cardiovascular diseases (ECG signal types) can provide early screening for adult congenital heart disease, valvular disease, coronary heart disease, cardiomyopathy, pulmonary heart disease and other cardiovascular diseases that cannot be identified by existing technologies.

As shown in Figure 5, the described model generation module includes a sample library, a model training engine and a model library, wherein the sample library generates multi-lead ECG data sets based on the data acquisition module, data preprocessing module, and data set division module D ₁ and the associated data set D ₂ are completed and stored; the model training engine is based on the multi-lead ECG signal data set and associated data set stored in the sample library to complete the model training; the model library is used to store the trained ECG self-supervised model;

The service calculation module includes a service trigger engine and a model calculation engine; wherein the service trigger engine is used to realize the type identification request of the received ECG signal, and sends it to the data acquisition module; the data acquisition module is used for receiving the ECG signal The signal type identification request automatically collects the data required for model prediction corresponding to the request, including multi-lead ECG signals and feature information, and sends them to the model calculation engine; the model calculation engine is used to call the trained heart The electrical self-supervision model, based on the multi-lead ECG signal and feature information, obtains the probability information corresponding to the various types of ECG signal set, and completes the result storage.

Example 2

As shown in Figure 6, the type identification system of the multi-lead ECG signal described in this embodiment is basically the same as that of Embodiment 1, the difference is that it also includes a front-end interaction module and a dynamic monitoring module, and the front-end interaction module includes an identification The result presentation sub-module and the label storage sub-module; wherein, the identification result presentation sub-module is used for displaying the probability information corresponding to various ECG signal categories obtained by the service calculation module; The probability information corresponding to the ECG signal type determines the final ECG signal type label; in particular, the ECG signal type label generated during the model application process is dynamically updated to the sample library of the model generation module, and new data is continuously accumulated to facilitate follow-up Model update optimization and iteration;

The dynamic monitoring module includes a service monitoring and evaluation sub-module and a service update trigger engine; wherein, the service monitoring and evaluation sub-module is used to evaluate the model recognition effect in real time based on automatically accumulated ECG signal category label information generated during the application process; The service update trigger engine is used to automatically trigger the update of the model and service when the model effect does not meet the preset requirements, so as to realize the dynamic optimization update of the model.

The front-end interaction module provides clinicians with visual type recognition results to assist clinicians in diagnosis. At the same time, the ECG signal type labels generated during the model application process can be dynamically updated to the sample library of the model generation module in real time to continuously accumulate new data. , to facilitate subsequent model update optimization and iteration;

The dynamic monitoring module evaluates the model recognition effect in real time based on the probability information and label information of the ECG signal type. When the model effect does not meet the preset requirements, it automatically triggers the update of the model and service to realize the dynamic optimization and update of the model. The preset requirement is AUC 0.9 Above or F1-score above 0.75 or accuracy rate above 80%. In this embodiment, the AUC is above 0.9.

As shown in Figure 7, the type identification system of the multi-lead ECG signal also includes an ECG interpretation module, the ECG interpretation module has a built-in ECG interpretation model, which can interpret the ECG, and the identification category is heart rhythm abnormal situation.

The ECG interpretation model is the prior art, and the arrhythmia includes but not limited to sinus arrhythmia, atrial premature beat, ventricular premature beat, atrioventricular block, and atrial fibrillation.

The intelligent ECG auxiliary system includes the type identification system of the multi-lead ECG signal, and also includes a knowledge base, the knowledge base stores processing suggestions, when the type identification system of the multi-lead ECG signal gives the type After the recognition result, the knowledge base is invoked, and the processing suggestions in the knowledge base that meet the preset conditions are output. The preset condition is that the processing suggestion matches the ECG category identification result.

As shown in Figure 8, an electronic device includes: a processor;

The memory stores a program configured to implement the method for identifying the type of the multi-lead ECG signal when executed by the processor.

A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium stores instructions, and when the instructions are executed by a processor, the processor executes the method for identifying the type of the multi-lead ECG signal . The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

The flowcharts and block diagrams in the drawings of the present application show the architecture, functions and operations of possible implementations of the systems, methods and computer program products according to various embodiments disclosed in the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that includes one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the different figures. For example, two blocks shown connected in series may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a A combination of dedicated hardware and computer instructions.

Those skilled in the art can understand that various combinations and/or combinations can be made of the features described in the various embodiments and/or claims of the present application, even if such combinations or combinations are not explicitly recorded in the present application. In particular, without departing from the spirit and teaching of the present application, various combinations and/or combinations can be made of the features recorded in the various embodiments and/or claims of the present application, and all these combinations and/or combinations fall into The scope disclosed in this application.

In this paper, specific examples are used to illustrate the principles and implementation methods of the present application. The descriptions of the above examples are only used to help understand the method and core idea of the present application, and are not intended to limit the present application. For those skilled in the art, changes can be made in the specific implementation and application scope according to the ideas, spirit and principles of this application, and any modifications, equivalent replacements, improvements, etc. made by it should be included in this application within the scope of protection.

Claims (17)

1. The type identification method of the multi-lead electrocardiosignal is characterized by comprising the following steps of:

data acquisition, namely acquiring n multi-lead electrocardiosignals, and patient characteristic information and an electrocardiosignal type label which are associated with part of the multi-lead electrocardiosignals in the n multi-lead electrocardiosignals;

data preprocessing, generating a multi-lead electrocardiosignal data set D representing all electrocardiosignals based on the n multi-lead electrocardiosignals ₁ The corresponding sample size is n; generating an associated dataset D based on the partial multi-lead electrocardiograph signal and patient characteristic information and an electrocardiograph signal type tag associated with the partial multi-lead electrocardiograph signal ₂ The corresponding sample size is m, wherein m is less than or equal to n;

data set division, namely dividing the multi-lead electrocardiosignal data set D ₁ Divided into multiple lead electrocardiosignal training set D _1,train And multi-lead electrocardiosignal verification set D _1,vali The associated data set D ₂ Dividing into associated data training sets D _2,train Associated data verification set D _2,vali Associated data test set D _2,test ；

An electrocardiograph self-supervision model framework is constructed, and the model framework is based on a transducer module and comprises a cutter, a double-classification masker, an encoder, a decoder and a classifier;

model pre-training, initializing model parameters, and then training the multi-lead electrocardiosignal training set D _1,train Multi-lead electrocardiosignal verification set D _1,vali Training set D of associated data _2,train Associated data verification set D _2,vali Inputting a model framework, performing self-supervision learning, punishing the self-supervision learning, and obtaining a pre-trained electrocardiographic self-supervision model;

model fine tuning based on the associated data training set D _2,train Number of correlationsAuthenticated set D _2,vali Fine tuning the pre-trained electrocardiographic self-supervision model to finish the training of the model;

model testing, based on the associated data test set D _2,test Testing the trimmed electrocardiograph model, evaluating the model effect, if the model evaluation result does not meet the preset requirement, adjusting model parameters, and repeating the model pre-training and the model trimming until the model evaluation result meets the preset requirement;

In the application stage, the acquired multi-lead electrocardiosignals and the characteristic information of the patient are input into a trained model to obtain probability information corresponding to various set electrocardiosignal types.

2. The method of claim 1, wherein the patient characteristic information includes age, gender, and an electrocardiographic anomaly, the electrocardiographic type tag representing information whether multimodal data matching the partial multi-lead electrocardiographic signal contains a cardiovascular disease, the multimodal data being CT, ultrasound, contrast, or nuclear magnetic data acquired for the same patient, the cardiovascular disease including at least one of adult coronary heart disease, valvular disease, coronary heart disease, cardiomyopathy, pulmonary vascular disease.

3. The method according to claim 2, characterized in that the model pre-training comprises the steps of:

a. randomly initializing model parameters;

b. in the multi-lead electrocardiosignal training set D _1,train The multi-lead electrocardiosignal verification set D _1,vali Performing self-supervision learning on a cutter, a double-classification masker, an encoder and a decoder based on the model;

c. at the associated data training set D _2,train Associated data verification set D _2,vali And performing punishment self-supervision learning on the basis of the cutter, the double-classification masker, the encoder, the decoder and the classifier of the model.

4. A method according to claim 3, wherein the model-based cutter, dual class mask, encoder, decoder self-supervised learning comprises the steps of:

forward propagation of the D _1,train Sequentially passing through a cutter and a double-classification masker to obtain a self-training vector group and a transformed self-estimation vector group, splicing the self-training vector group and one classification vector, sequentially passing through an encoder and a decoder, and outputting a group of prediction vectors as estimation results of the transformed self-estimation vector group; wherein the classification vector is a preset learnable classification vector;

parameter updating, at D, with a self-monitoring loss function reflecting errors between the prediction vector and the transformed set of self-estimated vectors as an objective function _1,train Updating all the learnable parameters in the encoder and decoder using the optimizer;

in verification set D _1,vali The optimal first superparameter combination is chosen such that the self-supervising loss function is minimal compared to the total amount of all learnable parameters in the decoder.

5. The method of claim 4, wherein the transforming comprises at least one of sampling a dimension reduction, an element-to-element power exponent, a normalization within a vector, classifying according to a threshold, the self-supervising loss function comprising l ₁ Loss/l ₂ One of the loss and cross entropy loss functions; the first hyper-parameter combination includes a hidden dimension and an attention header of a transducer sub-block of the decoder in a model.

6. The method of claim 5, wherein said penalized self-supervised learning comprises the steps of:

forward propagation, D _2,train The electrocardiosignal in the heart is sequentially transmitted through a cutter and a double-classification masking device to obtain a self-training vector group and a transformed self-estimated vector group, and the self-training vector group and a classification vector are spliced and then transmitted through an encoderOutputting a coded self-training vector group and a coded classification vector, wherein one classification vector is a preset learnable classification vector; the encoded self-training vector group and the encoded classification vector enter branch one, the encoded classification vector and the D _2,train The patient characteristic information in the model is entered into a branch II;

the decoder processes the input self-training vector group after coding to obtain a predictive vector which is used for estimating the self-estimated vector group after transformation;

Branch two, the encoded classification vector and the D _2,train Inputting the patient characteristic information into a classifier for processing to obtain the prediction probability of the electrocardiosignal type;

parameter updating, taking penalty loss function as objective function, at D _2,train Updating all the learnable parameters in the encoder, the decoder and the classifier by using an optimizer, wherein the penalty loss function is a self-supervision loss function +lambda cross Entropy of a self-estimated vector group after prediction vector and transformation, wherein lambda is tens of super-parameters, and the cross Entropy represents the prediction probability of the electrocardiosignal type and the cross entropy loss of the electrocardiosignal type label;

in verification set D _2,vali The optimal super parameter lambda is selected to maximize the selection measurement index of electrocardiosignal type identification, wherein the selection measurement index comprises AUC and F _β -one of score, accuracy.

7. The method of claim 6, wherein the electrocardiographic self-monitoring model comprises:

a cutter for cutting each input electrocardiosignal into a number of columns K and a number of columns d _patch D of mutually exclusive _v Vectorizing the submatrices to obtain the element number d _v Is { x } of the global vector group of electrocardiosignals ₁ ,…,x _dv }, where d _patch The super parameter is one, K is the electrocardio lead number;

double-classification masker for receiving electrocardiosignal full-vector group { x } ₁ ,…,x _dv Equal probability random extraction of T+T 'vectors from which T+T'. Ltoreq.d is not put back _v The first T vectors form a self-training vector group, the last T 'vectors form an estimated vector group, T and T' are respectively a super parameter III and a super parameter IV, the output is the self-training vector group and a self-estimated vector group, and then the self-estimated vector group is transformed to obtain a transformed self-estimated vector group;

encoder consisting of projection layer, position embedding layer, and L hidden dimensions d connected in sequence _encoder The attention head is h _encoder Is formed by sequentially connecting transducer sub-blocks, wherein L is super-parameter five and d _encoder Is super parameter six, h _encoder In the pre-training stage, the input of the encoder is a self-training vector group and a classification vector, and the output is the encoded self-training vector group and classification vector; in the fine tuning and testing stage, the input of the encoder is an electrocardiosignal full vector group and a classification vector, and the input of the encoder is a group of coded electrocardiosignal full vector group and a classification vector, wherein the classification vector is a learnable classification vector added manually;

Decoder, composed of restoring layer, position embedding layer, 1 hidden dimension d _decoder The attention head is h _decoder And 1 full link layer, wherein d _decoder Eight is a super parameter, wherein h _decoder For super-parameter nine, the decoder is only used in the pre-training stage, the decoder inputs the encoded self-training vector group and classification vector, and decodes and outputs the prediction vector;

the classifier consists of a full connection layer and an activation layer, inputs the classification vectors after coding and characteristic information, and outputs a prediction probability value of the electrocardiosignal type.

8. The method according to any of claims 4-7, wherein the data preprocessing comprises the steps of:

a. filtering and denoising the electrocardiosignal;

b. the electrocardiosignals after the filtering treatment are standardized, so that the data range is between-1 and 1;

c. filling the standardized electrocardiosignal with a column with the value of 0 to ensure that the number of the filled column can be divided by the super parameter one to obtain the electrocardiosignal X _i ，i=1,...n；

d. Performing min-max standardization processing on the numerical variables in the patient characteristic information, and performing 0-1 coding on the classification variables in the patient characteristic information to obtain the patient characteristic information z of the electrocardio _j ，j=1,...m；

e. Acquisition of a Multi-lead electrocardiographic Signal dataset D ₁ Associated dataset D ₂ Wherein D is ₁ ={X _i I=1, …, n } represents all electrocardiographic signals; d (D) ₂ ={(X _j ,z _j ,y _j ) J=1, …, m } represents an associated dataset, where X _j Representing multi-lead electrocardiosignals, z _j Representing patient characteristic information associated with a multi-lead electrocardiograph signal, y _j Representing an electrocardiograph signal type tag associated with a multi-lead electrocardiograph signal.

9. The method of claim 8, wherein the model fine tuning comprises the steps of:

forward propagation, D _2,train Inputting the central electrocardiosignal into a cutter for processing to obtain an electrocardiosignal full-vector group, splicing the electrocardiosignal full-vector group with the classification vector, inputting the spliced electrocardiosignal full-vector group into an encoder for processing to obtain an encoded electrocardiosignal full-vector group and an encoded classification vector; sum D of the encoded classification vectors _2,train Inputting the patient characteristic information into a classifier for processing to obtain the prediction probability of the electrocardiosignal type;

parameter updating to predict probability and D for electrocardiosignal type _2,train Cross entropy loss of the electrocardiosignal type label in the center is taken as an objective function, and the cross entropy loss is taken as a D _2,train Updating all the learnable parameters in the encoder and the classifier by using the optimizer;

in the associated data verification set D _2,vali Selecting the optimal second super-parameter combination to maximize the selection measurement index of electrocardiosignal type identification; the second super-parameter combination includes the cutter Column number d of cut center electric signal _patch The number d of the submatrices after the central electric signal of the cutter is cut _v The number of vectors T of the self-training vector group, the number of vectors T' of the self-estimated vector group, the number of transducer sub-blocks included in the encoder, the hidden dimension L of the transducer sub-blocks in the encoder, the attention header h of the transducer sub-blocks in the encoder _encoder 。

10. The method of claim 9, wherein the model test comprises the steps of:

in the associated data test set D _2,test And finally, evaluating the model effect by selecting the measurement index, if the model evaluation result meets the preset requirement, allowing the model to be used, and if the model evaluation result does not meet the preset requirement, adjusting the model parameters, and repeating the model pre-training and fine tuning steps until the model evaluation result meets the preset requirement.

11. A type recognition system of multi-lead electrocardiosignals is characterized by comprising a data acquisition module, a data preprocessing module, a data set dividing module, a model generating module and a service calculating module, wherein the data acquisition module, the data preprocessing module, the data set dividing module, the model generating module and the service calculating module are arranged in the system

The data acquisition module is used for acquiring training data, and comprises n multi-lead electrocardiosignals, and patient characteristic information and electrocardiosignal type labels associated with part of the multi-lead electrocardiosignals in the n multi-lead electrocardiosignals;

The data preprocessing module is used for generating a multi-lead electrocardiosignal data set D representing all electrocardiosignals based on the n multi-lead electrocardiosignals ₁ The corresponding sample size is n; based on the n multi-lead electrocardiosignals, the patient characteristic information and the electrocardiosignal type label which are related to the multi-lead electrocardiosignals, the multi-lead electrocardiosignals of which the patient characteristic information or the electrocardiosignal type label is missing are removed, and a related data set D is generated ₂ The corresponding sample size is m, wherein m is less than or equal to n;

the data set dividing module is used for dividing the multi-lead electrocardiosignal dataSet D ₁ Divided into multiple lead electrocardiosignal training set D _1,train And multi-lead electrocardiosignal verification set D _1,vali The associated data set D ₂ Dividing into associated data training sets D _2,train Associated data verification set D _2,vali Associated data test set D _2,test ；

The model generation module is used for completing model training based on the multi-lead electrocardiosignal, the characteristic information, the label information and the built model frame, and obtaining and storing a trained electrocardio self-supervision model;

the service calculation module is used for receiving the electrocardiosignal type identification request and calling the trained electrocardiosignal self-supervision model to obtain probability information corresponding to the set electrocardiosignal types.

12. The system of claim 11, wherein the model generation module comprises a sample library, a model training engine, and a model library, wherein the sample library is a multi-lead electrocardiographic data set D from the received data set partitioning module ₁ Associated data set D ₂ And finishing the storage; the model training engine is used for completing model training based on a multi-lead electrocardiosignal data set and an associated data set stored in a sample library; the model library is used for storing a trained electrocardiographic self-supervision model;

the service computing module comprises a service triggering engine and a model computing engine; the service triggering engine is used for receiving the type identification request of the electrocardiosignal and the data carried by the request, and sending the request to the model calculation engine, wherein the data carried by the request comprises the multi-lead electrocardiosignal and the characteristic information of the patient; the model calculation engine is used for calling a trained electrocardio self-supervision model, obtaining probability information corresponding to various set electrocardio signal types based on multi-lead electrocardio signals carried by the request and the characteristic information of the patient, and completing result storage.

13. The multi-lead electrocardiographic signal type recognition system of claim 12 further comprising a front-end interaction module, a dynamic monitoring module,

The front-end interaction module comprises an identification result presentation sub-module and a label storage sub-module; the recognition result presentation submodule is used for carrying out visual prompt based on probability information corresponding to various electrocardiosignal types obtained by the service calculation module; the label storage sub-module is used for automatically acquiring a final electrocardiosignal type label generated in the application process and completing storage;

the dynamic monitoring module comprises a service monitoring evaluation submodule and a service update trigger engine; the service monitoring and evaluating sub-module is used for evaluating the model identification effect in real time based on the type label generated in the automatic accumulation application process; and the service update triggering engine is used for automatically triggering the update of the model and the service when the model effect does not meet the preset requirement, and realizing the dynamic optimization update of the model.

14. The system of claim 13, further comprising an electrocardiograph interpretation module, wherein the electrocardiograph interpretation module is embedded with an electrocardiograph interpretation model for interpreting an electrocardiograph and recognizing that the electrocardiograph signal is of an arrhythmia type.

15. An intelligent electrocardio-assisted system, which is characterized by comprising the type recognition system of the multi-lead electrocardiosignal as claimed in any one of claims 11 to 14 and a knowledge base, wherein the knowledge base stores processing suggestions, and when the type recognition system of the multi-lead electrocardiosignal gives a type recognition result, the knowledge base is called to output the processing suggestions meeting preset conditions in the knowledge base.

16. An electronic device, comprising: a processor;

a memory storing a program configured to implement the type recognition method of a multi-lead electrocardiographic signal according to any one of claims 1-10 when executed by the processor.

17. A non-transitory computer readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the type recognition method of a multi-lead electrocardiographic signal according to any one of claims 1-10.

Images