KR102186157B1 - Lung Sound Analysis Method and System Based on Neuro-fuzzy Network - Google Patents

Abstract

The present invention relates to a lung sound analysis method and system based on a neuro-fuzzy network, and more particularly, to extract lung sound feature data by converting lung sound data into two-dimensional lung sound data of time-frequency, and to extract the selected lung sound feature data into neuro- It relates to a neuro-fuzzy network based lung sound analysis method and system for analyzing lung sound data by analyzing based on a fuzzy network.

Description

Lung Sound Analysis Method and System Based on Neuro-fuzzy Network}

The present invention relates to a lung sound analysis method and system based on a neuro-fuzzy network, and more particularly, to extract lung sound feature data by converting lung sound data into two-dimensional lung sound data of time-frequency, and to extract the selected lung sound feature data into neuro- It relates to a neuro-fuzzy network based lung sound analysis method and system for analyzing lung sound data by analyzing based on a fuzzy network.

Lung sounds are sounds produced by the air flowing into and out of the bronchi and lungs by breathing. When auscultating the lungs, the doctor hears these sounds with a stethoscope and provides various important information about the condition of the lungs. Will be provided.

A stethoscope like this

The lung sound can be largely divided into breath sound and adventitious sound. It appears in the form of audible breathing sounds caused by human breathing and additional noise.

Breathing sound can be divided into alveolar breathing sound (Vesicular breath sound) and bronchial tubular sound (Bronchial tubular sound).

Alveolar respiration is heard in normal cases, it sounds like a weak wind, and a loud sound is heard during inhalation. Such a decrease in alveolar respiration is a case where there is a substance blocking sound conduction between the central airway and the chest wall, such as Pleural effusion, Pneumothorax, and Burla.

The bronchial breathing sound is loud, high-pitched, whistling-like, and exhalation and inhalation are almost the same. Such bronchial breathing sounds abnormal when there is a consolidation of the pulmonary parenchyma such as pneumonia, pulmonary edema, and pulmonary hemorrhage.

On the other hand, minor noises include Crackle, Wheeze, Stridor, Rhonchus, and Pleural friction rub. Among them, blister and pleural friction sounds are non-continuous sounds, and wheezing and grunt appear as continuous sounds.

There are largely fine crackle and coarse crackle, and fine bullous sound is relatively soft and short and slightly higher in pitch, which is the sound produced by the opening of a small airway that was blocked. It is mainly heard from the bottom, but can disappear by changing posture, which appears in diseases such as pulmonary fibrosis, where the lungs become hardened. Coarse vesicles sound more coarse and louder than fine vesicles and have a lower pitch. This is caused by airway secretion and may occur in chronic obstructive pulmonary disease (COPD), pneumonia, and congestive heart failure.

Wheezing is a continuous, long sound of 0.25 seconds or longer, and appears as a high-pitched whistle or flute-like sound, and occurs as the air passes rapidly through a narrowed airway. It has the characteristic that it is heard better when exhaled than when inhaled. It appears in a single tone in local bronchial narrowing, lung cancer, bronchial tuberculosis, foreign body, and tracheal stenosis, and in various tones in asthma, chronic obstructive pulmonary disease, and congestive heart failure.

The cramping appears as a rattle sound, such as a harsh, snoring sound, which is caused by a mucus secretion trapped in the proximal lower respiratory tract and vibrations as air passes through it. This occurs when there is a large amount of airway secretion.

The pleural friction sound is similar to the sound that occurs when the hairs rub against each other, and occurs when the parietal pleura and the intestinal pleura rub against each other. That is, it occurs when there is a problem with the pleura such as pleural inflammation or pleural tumor.

Conventionally, in order to detect the negative noise contained in the lung sound, a method of diagnosis by a doctor using a stethoscope or the like has been used, but there is a disadvantage that diagnosis through auscultation of the lung is possible only by a doctor with expertise. Therefore, there is a need to develop a technology for diagnosing the lung condition by automatically analyzing the lung sound.

The present invention converts lung sound data into two-dimensional lung sound data of time-frequency to extract lung sound feature data, and analyzes lung sound data by analyzing the selected lung sound feature data based on a neuro-fuzzy network. Its purpose is to provide an analysis method and system.

In order to solve the above problems, the present invention provides a lung sound analysis method performed by a lung sound analysis system including at least one processor and a memory, comprising: a lung sound input step of receiving lung sound data; A waste sound preprocessing step of converting the lung sound data into two-dimensional data of the lung sound of time-frequency; A lung sound feature extraction step of extracting two or more categories of lung sound feature data based on the converted two-dimensional lung sound data; A lung sound feature selection step of selecting lung sound feature data corresponding to the selected category from among the lung sound feature data of two or more categories; And a lung sound classification step of deriving classification of the lung sound data based on the learned model and the selected lung sound feature data. Including, the selected category is a process corresponding to the lung sound pre-processing step and the lung sound feature extraction step with respect to a plurality of learning waste sound data divided into two or more categories, extracting two or more categories of lung sound feature data, and the 2 A lung sound analysis method is provided in which a correlation degree according to the classification is determined by deriving a category of lung sound characteristic data meeting a preset criterion among the lung sound characteristic data of the above categories.

In the present invention, in the lung sound classification step, one or more selected lung sound feature data may be input to a fuzzy neural network based on a weighted fuzzy membership function, and a boundary sum of the weighted fuzzy membership function may be learned.

In the present invention, in the lung sound classification step, the classification of the lung sound data may be derived by comparing a boundary sum of the learned weighted fuzzy membership function with a preset value.

In the present invention, the selected category may be determined by deriving a category of lung sound characteristic data based on a Pearson Correlation Coefficient according to the classification among lung sound characteristic data of two or more categories of learning lung sound data.

In the present invention, the pre-processing step may convert the received lung sound data into a mel-spectrogram.

In the present invention, the preprocessing step may generate a divided mel-spectrogram by dividing the converted mel-spectrogram into a preset time length and a preset time interval.

In the present invention, the lung sound feature extraction step may extract feature data by applying convolution and pooling of a convolutional neural network to the two-dimensional lung sound data.

In the present invention, in the lung sound feature extraction step, feature data may be extracted by applying convolution and pooling of a VGG16 neural network model to the two-dimensional lung sound data.

In order to solve the above problems, in the present invention, a lung sound analysis system, comprising: a data input unit for receiving lung sound data; A data preprocessing unit converting the lung sound data into two-dimensional time-frequency lung sound data; A data feature extracting unit for extracting two or more categories of lung sound feature data based on the converted two-dimensional lung sound data; A feature selection unit for selecting lung sound feature data corresponding to the selected category from among the lung sound feature data of two or more categories; And a lung sound classification unit for deriving classification of the lung sound data based on the learned model and the selected lung sound feature data. Including, wherein the selected category is to extract two or more categories of lung sound feature data through the data preprocessing unit and the data feature extraction unit for a plurality of learning lung sound data divided into two or more categories, and A lung sound analysis system is provided in which a correlation degree according to the classification among lung sound characteristic data is determined by deriving a category of lung sound characteristic data meeting a preset criterion.

In the present invention, the lung sound classification unit may input one or more selected lung sound feature data into a fuzzy neural network based on a weighted fuzzy membership function, and learn a boundary sum of the weighted fuzzy membership function.

According to an embodiment of the present invention, feature data may be extracted through convolution and pooling by transforming the received lung sound data into 2D data.

According to an embodiment of the present invention, by dividing the two-dimensional lung sound data into a preset time length and a preset time interval, feature data may be extracted from the two-dimensional data of a standardized size to analyze the lung sound.

According to an embodiment of the present invention, by extracting feature data from learning data and selecting the feature data using Pearson's correlation coefficient, classification can be performed using feature data capable of easily classifying lung sound data.

According to an embodiment of the present invention, the accuracy of classification may be improved by classifying feature data extracted through convolution and pooling through a fuzzy neural network based on a weighted fuzzy membership function.

1 is a diagram schematically showing the configuration of a lung sound analysis system according to an embodiment of the present invention.
2 is a diagram schematically showing each step of a lung sound analysis method according to an embodiment of the present invention.
3 is a diagram schematically illustrating an example of lung sound data and two-dimensional lung sound data according to an embodiment of the present invention.
4 is a diagram schematically showing an example of a mel-spectrogram, which is two-dimensional lung sound data according to an embodiment of the present invention.
5 is a diagram schematically showing a method of generating a split-mel-spectrogram according to an embodiment of the present invention.
6 is a diagram schematically showing a method of extracting feature data according to an embodiment of the present invention.
7 is a diagram schematically showing data in a process of extracting feature data through a VGG16 neural network model according to an embodiment of the present invention.
8 is a diagram schematically showing an example of feature data extracted according to an embodiment of the present invention.
9 is a diagram schematically showing the structure of a fuzzy neural network (NEWFM) based on a weighted fuzzy membership function used in a lung sound classification step based on a neuro-fuzzy network according to an embodiment of the present invention.
10 is a diagram schematically showing a boundary sum (BSWFM) of a weighted fuzzy membership function learned by NEWFM used in a lung sound classification step according to an embodiment of the present invention.
11 is a diagram showing an example of a boundary sum (BSWFM) of a weighted fuzzy membership function according to classification in feature data according to an embodiment of the present invention.
12 is a block diagram illustrating an example of an internal configuration of a computing device according to an embodiment of the present invention.

In the following, various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, a number of specific details are disclosed to aid in an overall understanding of one or more aspects. However, it will also be appreciated by those of ordinary skill in the art that this aspect(s) may be practiced without these specific details. The following description and the annexed drawings set forth in detail certain illustrative aspects of one or more aspects. However, these aspects are exemplary and some of the various methods in the principles of the various aspects may be used, and the descriptions described are intended to include all such aspects and their equivalents.

Further, various aspects and features will be presented by a system that may include multiple devices, components and/or modules, and the like. It is also noted that various systems may include additional devices, components and/or modules, and/or may not include all of the devices, components, modules, etc. discussed in connection with the figures. It must be understood and recognized.

As used herein, “an embodiment,” “example,” “aspect,” “example,” and the like may not be construed as having any aspect or design described as being better or advantageous than other aspects or designs. . The terms'~part','component','module','system', and'interface' used below generally mean a computer-related entity, for example, hardware, hardware It can mean a combination of software and software, or software.

In addition, the terms "comprising" and/or "comprising" mean that the corresponding feature and/or element is present, but excludes the presence or addition of one or more other features, elements, and/or groups thereof. It should be understood as not.

In addition, terms including ordinal numbers such as first and second may be used to describe various elements, but the elements are not limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein including technical or scientific terms are commonly understood by those of ordinary skill in the art to which the present invention belongs. It has the same meaning as. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the embodiments of the present invention, an ideal or excessively formal meaning Is not interpreted as.

1 is a diagram schematically showing the configuration of a lung sound analysis system according to an embodiment of the present invention.

The lung sound analysis system 10 according to an embodiment of the present invention receives the lung sound data 1 and derives an analysis result 4 obtained by analyzing the lung sound data.

Referring to FIG. 1, a lung sound analysis system 10 according to an embodiment of the present invention includes a data input unit 100 for receiving lung sound data 1; A data preprocessing unit 200 for converting the lung sound data 1 into two-dimensional time-frequency lung sound data 2; A data feature extraction unit 300 for extracting two or more categories of lung sound feature data 3 based on the converted two-dimensional lung sound data 2; A feature selection unit 400 for selecting the lung sound feature data 3 corresponding to the selected category among the lung sound feature data 3 of two or more categories; And a lung sound classification unit 500 for deriving classification of lung sound data based on the learned model and the selected lung sound feature data 3. It may include. In this case, the selected category extracts two or more categories of lung sound feature data through the data pre-processing unit 200 and the data feature extracting unit 300 for a plurality of learning lung sound data classified into two or more categories, and The correlation according to the classification may be determined by deriving a category of lung sound characteristic data meeting a preset criterion among the two or more categories of lung sound characteristic data.

The lung sound analysis system 10 according to an embodiment of the present invention converts the lung sound data 1 received by each of these detailed configurations into two-dimensional lung sound data 2 to extract the lung sound feature data 3 , The extracted lung sound feature data (3) is selected and analysis is performed based on the selected lung sound feature data (3). In order to perform the analysis in this way, the lung sound analysis system 10 needs to perform setting for each detailed configuration through learning.

In an embodiment of the present invention, the feature selection unit 400 receives the learning lung sound data divided into two or more classifications by the data input unit 100, and receives the data preprocessor 200 and the data feature extractor 300. ) To extract the learning feature data, select one or more learning feature data having a large difference according to the classification from the extracted two or more learning feature data, and store it as the selection feature data.

2 is a diagram schematically showing each step of a lung sound analysis method according to an embodiment of the present invention.

In an embodiment of the present invention, the lung sound analysis method through the lung sound analysis system 10 as shown in FIG. 1 is an analysis setting step (S10) of performing the setting of the lung sound analysis system 10 based on the learning lung sound data. ; And a lung sound analysis step (S20) of analyzing lung sound data by the lung sound analysis system 10 on which the settings have been performed. It may include.

As described above, the lung sound analysis system 10 can accurately analyze the actually input lung sound data by receiving the learning lung sound data and performing the setting.

In an embodiment of the present invention, the analysis setting step (S10). A learning data input step (S100a) of receiving learning lung sound data divided into two or more categories; A learning data preprocessing step (S200a) of converting the learning lung sound data into time-frequency learning two-dimensional data; A learning data feature extraction step (S300a) of extracting learning feature data of two or more categories based on the converted two-dimensional learning data; And a feature selection step (S400a) of selecting one or more categories having a large difference according to the classification among the extracted learning feature data of two or more categories. Including, the lung sound analysis step (S20), the lung sound input step (S100b) receiving input of the lung sound data (1); A waste sound preprocessing step (S200b) of converting the lung sound data into two-dimensional data of the lung sound (2) of time-frequency; A lung sound feature extraction step (S300b) of extracting two or more categories of lung sound feature data (3) based on the converted two-dimensional lung sound data (2); And a lung sound characteristic selection step (S400b) of selecting the lung sound characteristic data 3 corresponding to the selected category among the lung sound characteristic data 3 of two or more categories. And a lung sound classification step (S500) of deriving a classification of the lung sound data 1 based on the learned model and the selected lung sound feature data 3; It may include.

As described above, setting needs to be performed in order for the lung sound analysis system 10 to receive and analyze the lung sound data 1 to derive the classification of the lung sound data 1. To this end, in an embodiment of the present invention, the setting is made by receiving learning lung sound data in the analysis setting step (S10) and performing learning.

At this time, the learning data input step (S100a), the learning data preprocessing step (S200a), and the learning data feature extraction step (S300a), which are processes in which feature data is extracted from the learning lung sound data, are characterized from the lung sound data (1). The lung sound input step (S100b), the waste sound preprocessing step (S200b), and the lung sound feature extraction step (S300b), which are processes in which data is extracted, are performed by the same method, respectively, from the learning feature data and the lung sound data. Lung sound feature data is extracted.

The learning lung sound data may include a plurality of lung sound data divided into two or more categories. For example, the classification may be a normal lung sound and a lung sound that includes crackle due to pneumonia, and the lung sound analysis system 10 is configured to include the actual normal lung sound and the lung sound of a patient with pneumonia. And the lung sounds of pneumonia patients can be classified.

The feature selection step (S400a) selects a category having a large difference according to the classification among the learning feature data categories extracted from the learning waste sound data, and classifies the lung sound using the category selected in the lung sound classification step (S500). By doing so, the accuracy of classification is improved.

3 is a diagram schematically illustrating an example of lung sound data and two-dimensional lung sound data according to an embodiment of the present invention.

3A shows a waveform of generally stored sound data. Data stored with sound, such as a lung sound, stores information on the amplitude of a sound waveform over time, and if this information is shown, it may be illustrated as shown in FIG. 3A.

However, since such amplitude data is not easy to analyze sound components, a method of analyzing frequency components through Fourier transform is used to separate and confirm sound components.

FIG. 3B shows a state in which information on frequency components over time by Fourier transform of sound data is shown as 2D data. As shown in (b) of FIG. 3, the input sound data is represented by a two-dimensional graph of time and frequency, and the intensity (dB) of sound at each frequency is represented by color.

As described above, in the case of time-frequency two-dimensional data, it is easy to extract sound features and is suitable for use in analysis. Therefore, in an embodiment of the present invention, the learning data pre-processing step (S200a) and the waste sound pre-processing step (S200b) performed by the data pre-processing unit 200 include the received learning lung sound data or lung sound data as time-frequency two-dimensional data. Is converted to.

4 is a diagram schematically showing an example of a mel-spectrogram, which is two-dimensional lung sound data according to an embodiment of the present invention.

In an embodiment of the present invention, the preprocessing step (S200b) converts the received lung sound data (1) into a mel-spectrogram, and the learning data preprocessing step (S200a) is the same as the preprocessing step (S200b). In this way, the learning lung sound data can be converted into a mel-spectrogram.

As shown in FIG. 3, in an embodiment of the present invention, the received learning lung sound data or lung sound data may be converted into two-dimensional time-frequency lung sound data. In this case, the two-dimensional lung sound data may be a mel-spectrogram. The mel-spectrogram refers to a spectrogram in which a unit of frequency is changed to a mel unit among the time-frequency spectrogram shown in FIG. 3B.

It is mainly the frequency of the sound that makes a person perceive the pitch of the sound. The sound of 1 kHz and 40 dB is defined as 1000 mel, and the unit representing the pitch of different frequency sounds relatively felt is the mel unit. The method of converting the frequency into Mel units follows the following equation.

4A and 4B illustrate the conversion of the input lung sound data into a mel-spectrogram according to the above equation.

5 is a diagram schematically showing a method of generating a split-mel-spectrogram according to an embodiment of the present invention.

In an embodiment of the present invention, in the preprocessing step (S200b), the converted mel-spectrogram is divided into a preset time length and a preset time interval to generate a divided mel-spectrogram, and the learning data preprocessing step (S200a) ) Can generate a split mel-spectrogram in the same manner as in the pretreatment step.

The lung sound data (1) or the learning lung sound data received through the lung sound input step (S100b) and the learning data input step (S100a) has a constant length measured by the lung sound as in (a) and (b) of FIG. It is not suitable for direct comparison with other lung sound data. Accordingly, in an embodiment of the present invention, the mel-spectrogram obtained from the lung sound data 1 or the learning lung sound data may be divided by a preset time length t _s and used as input data for analysis. As described above, by dividing the mel-spectrogram by a preset time length t _s , it is possible to acquire input data of a certain size and perform analysis, thereby increasing the accuracy of the analysis.

In this case, in the case of extracting one divided mel-spectrogram from one lung sound data, since the data of the unextracted region is not used for analysis, a plurality of divided mel-spectrograms can be extracted from one lung sound data. have. In this case, in an embodiment of the present invention, a plurality of divided mel-spectrograms may be extracted at a preset time interval t _d . Preferably, the preset time interval (t _d ) is equal to or smaller than the preset time length (t _s ) so that all of the lung sound data can be used for analysis.

FIG. 5 shows a state in which a split mel-spectrogram is generated from the mel-spectrogram as shown in FIG. 4(a).

First, as shown in (a) of FIG. 5, a divided mel-spectrogram of a predetermined time length (t _s ) as indicated by a box in the mel-spectrogram is generated.

Thereafter, as shown in (b) of FIG. 5, with a preset time interval t _d , a divided mel-spectrogram of a preset time length t _s as indicated by a box is generated.

Thereafter, as shown in (c) of FIG. 5, a divided mel-spectrogram of a preset time length (t _s ) as indicated by a box is generated again at a preset time interval (t _d ).

By repeating this process, it is possible to generate a plurality of divided mel-spectrograms having a constant size from the mel-spectrogram shown in FIG. 4A, and thus, a plurality of divided mel-spectrograms having a constant size. You can accurately analyze the lung sound through.

In an embodiment of the present invention, the preset time length (t _s ) may be 3 seconds and the preset time interval (t _d ) may be 1 second.

6 is a diagram schematically showing a method of extracting feature data according to an embodiment of the present invention.

According to an embodiment of the present invention, in the lung sound feature extraction step (S300b), feature data is extracted by applying convolution and pooling of a convolutional neural network to the two-dimensional lung sound data (2), and the learning data feature extraction In step S300a, learning feature data may be extracted based on the learning two-dimensional data in the same manner as in the lung sound feature extraction step S300b.

6 shows a state in which such convolution and pooling are applied to the two-dimensional lung sound data 2.

In FIG. 6, convolution and pooling blocks using a plurality of divided mel-spectograms generated in FIG. 5 as inputs can be confirmed.

Referring to FIG. 6, when the divided mel-spectrogram is input, a feature map is generated through a plurality of convolution blocks (Conv) and pooling blocks (P), and the size of the feature map is gradually reduced, while reducing the number of feature maps. Characteristics are extracted in a way that increases gradually.

Such a convolution block (Conv) extracts features by generating a feature map from an input, and the pooling block (P) reduces the size of the feature map to extract feature data. This structure is the same as a configuration in which a characteristic is extracted by repeatedly connecting a convolutional block and a pooling block in a general convolutional neural network (CNN).

As shown in FIG. 6, in the final pooling block P ₅ , a plurality of (N) characteristic maps having a size of 1×1 are derived. In the present invention, N feature data is extracted by flattening a plurality of feature maps derived as described above. In the present invention, characteristic data of N categories are extracted by using each of the N characteristic data extracted as described above as one category.

The convolution block (Conv) and the pooling block (P) shown in FIG. 6 are according to an embodiment of the present invention, and the present invention is not limited to the configuration shown in FIG. 6, and a number of conballs different from that of FIG. It may be configured to include a solution block (Conv) and a pooling block (P). For example, in FIG. 6, a feature map is derived including four convolution blocks (Conv), but in another embodiment of the present invention, a feature map is derived including five convolution blocks (Conv), and feature data is obtained. Can be extracted.

7 is a diagram schematically showing data in a process of extracting feature data through a VGG16 neural network model according to an embodiment of the present invention.

In an embodiment of the present invention, in the lung sound feature extraction step (S300b), 512 feature data is extracted by applying convolution and pooling of a VGG16 neural network model to the two-dimensional lung sound data (2), and the learning data feature In the extraction step, learning feature data may be extracted based on the learning two-dimensional data in the same manner as in the lung sound feature extraction step.

The convolution and pooling of the VGG16 neural network model has a structure similar to that shown in FIG. 6. However, in the VGG16 neural network model, there are 5 blocks, and a convolution layer and a pooling layer may be disposed in each block.

7 shows an example of output data in each layer in the VGG16 neural network model.

In the embodiment shown in FIG. 7, first, the divided mel-spectrograms 150, 150, and 3 having a size of 150 x 150 are input as input, and 64 feature maps are generated through the convolution 1 and convolution 2 layers of block 1. (150,150,64). Thereafter, a feature map of size 75 x 75 is generated through the pooling layer of block 1 (75,75,64), and 128 feature maps are generated through the convolution 1 and convolution 2 layers of block 2 (75, 75,128). Thereafter, through the pooling layer of block 2, a feature map of size 37 x 37 is created (37,37,128), and 256 feature maps are generated through the convolution 1, convolution 2, and convolution 3 layers of block 3 ( 37,37,256). After that, through the pooling layer of block 3, a feature map of size 18 x 18 is created (18, 18, 256), and 512 feature maps are generated through the convolution 1, convolution 2 and convolution 3 layers of block 4 ( 18,18,512). After that, through the pooling layer of block 4, a 9 x 9 feature map is generated (9,9,512), and 512 feature maps are generated through the convolution 1, convolution 2 and convolution 3 layers of block 5 ( 18,18,512). Thereafter, a 4 x 4 feature map is generated while passing through the pooling layer of block 5 (4,4,512).

The characteristic map of (4,4,512) derived through convolution and pooling in this way goes through two more pooling layers, respectively, the characteristic map of (2,2,512) and (1,1,512) is generated, and the finally derived By flattening the feature map of (1,1,512), 512 feature data are extracted.

8 is a diagram schematically showing an example of feature data extracted according to an embodiment of the present invention.

In an embodiment of the present invention, the feature selection step (S400a) is performed by deriving a category of lung sound feature data based on a Pearson Correlation Coefficient according to the classification among the learning feature data of two or more categories. You can decide which categories have been created.

8 shows an example of learning feature data of N categories extracted from the learning lung sound data according to the method described above. The learning lung sound data is classified into two classifications, and each classification includes three learning lung sound data T. That is, classification 1 includes the learning lung sound data of T1, T2, and T3, and the classification 2 includes learning lung sound data of T4, T5, and T6.

For each of the learning lung sound data, N categories of learning feature data were extracted through the learning data input step (S100a), the training data preprocessing step (S200a), and the learning data feature extraction step (S300a). For example, the learning lung sound data T1 is F ₁₁ , F ₁₂ , F ₁₃ , F ₁₄ . … N learning feature data for each of the N categories of F _1N were extracted. The average and variance of learning feature data of the N categories extracted as described above may be derived for each classification. For example, the average of feature data 1, the first category of category 1, is m ₁₁ , variance is σ ² ₁₁ , and the average of feature data 1, the first category of category 2, is m ₂₁ , and variance is σ ² ₂₁ . .

In the feature selection step S400a, the extracted learning feature data is selected. In an embodiment of the present invention, learning feature data may be selected based on the Pearson correlation coefficient.

에 대한 피어슨 상관계수

는 하기 식과 같이 계산될 수 있다.The Pearson correlation coefficient is a measure of whether two variables are correlated with each other. Two variables

Pearson correlation coefficient for

Can be calculated as follows.

는 두 변수

의 공분산이고,

는 각각

의 표준편차이다. 상기 식을 정리하면 하기와 같다.

Are two variables

Is the covariance of

Are each

Is the standard deviation of The above equation is summarized as follows.

Pearson's correlation coefficient calculated in this way has a value of [-1, 1], and if it is close to 1 or -1, it is considered that there is a correlation, and if it is 0, there is no correlation.

When the Pearson correlation coefficient is applied to the feature data of two classifications as shown in FIG. 8, when the feature data of the two classifications has a negative correlation, the Pearson correlation coefficient is lower than -1. Accordingly, in an embodiment of the present invention, the classification accuracy can be improved by selecting a category of feature data having a low Pearson correlation coefficient and using it to classify lung sound data. In an embodiment of the present invention, 20 categories of feature data may be selected based on the Pearson correlation coefficient among the feature data of 512 categories derived as shown in FIG. 7.

9 is a diagram schematically showing the structure of a fuzzy neural network (NEWFM) based on a weighted fuzzy membership function used in a lung sound classification step based on a neuro-fuzzy network according to an embodiment of the present invention.

In an embodiment of the present invention, in the lung sound classification step (S500), each of the selected lung sound feature data is input into a weighted fuzzy membership function-based Fuzzy Neural Network (NEWFM), and weighted Bounded Sum of Weighted Fuzzy Membership functions (BSWFM) can be learned. When a fuzzy neural network (NEWFM) based on a weighted fuzzy membership function is described with reference to FIG. 9, NEWFM is composed of three layers including an input layer, a hyper box layer, and a class layer. I can. At this time, the input layer may be composed of n input nodes into which input patterns having n characteristics are input, and the hyperbox layer includes n BSWFMs for n input nodes, and is connected to the class node. It may consist of m hyperbox nodes. In addition, the class layer may consist of p class nodes, each of which is connected to at least one hyperbox node.

Meanwhile, the h-th input pattern input to the input node may be expressed as follows.

At this time, class indicates the classification result, and F _h indicates n features (selected lung sound feature data) of the input pattern.

Each hyperbox node B _l is composed of n fuzzy sets, of which the i-th fuzzy set has a weighted fuzzy membership function (WFM) represented by B _l ⁱ as shown in FIG. 6.

10 is a diagram schematically showing a boundary sum (BSWFM) of a weighted fuzzy membership function learned by NEWFM used in a lung sound classification step according to an embodiment of the present invention.

As shown in Fig. 10, the weighted fuzzy membership function (WFM) B _l ⁱ is the original membership function for the continuous time change signal x(t) μ _l ⁱ ₁ , μ _l ⁱ ₂ , For μ _l ⁱ ₃ and the like, it represents the membership function given weights W _l ⁱ ₁ , W _l ⁱ ₂ , W _l ^i3, etc. In FIG. 10, WFM (w _l ⁱ ₁ , w _l ⁱ ₂ , w _l ⁱ ₃ ) given weights of 0.7, 0.8, and 0.3 for μ _l ⁱ ₁ , μ _l ⁱ ₂ , and μ _l ⁱ ₃ , respectively, is shown. Became.

On the other hand, the boundary sum (BSWFM) of the weighted fuzzy membership function may appear in a polygonal shape such as the thick line of FIG. 10, in which case BS _l ⁱ (F _i ), which is a BSWFM value for the weighted fuzzy membership function B _l ⁱ It can be expressed as the following equation.

In this case, F _i represents the i-th feature value of the input pattern F _h . In FIG. 10, BSWFM values for F _i located between v ₁ ⁱ ₂ and v ₁ ⁱ ₃ are shown.

11 is a diagram showing an example of a boundary sum (BSWFM) of a weighted fuzzy membership function according to classification in feature data according to an embodiment of the present invention.

As shown in FIG. 11, when the lung sound classification step S500 of the lung sound analysis step S20 according to an embodiment of the present invention is applied to the classification 1 and the classification 2, the N-th feature data of the lung sound data You can see the difference that is distinguished in the distribution of fuzzy values of.

Accordingly, in the lung sound classification step S500 in an embodiment of the present invention, classification of lung sound data may be derived by comparing the boundary sum (BSWFM) of the learned weighted fuzzy membership function with a preset value. Such a preset value can be set based on data obtained through a plurality of learning data, and in order to classify the input lung sound data, BSWFM, which is extracted from the lung sound data and selected, is learned by NEWFM. The classification of the lung sound data can be determined by comparing how much difference there is with the preset value. That is, the preset value may be set based on the result of learning the boundary sum of the weighted fuzzy membership function by inputting the lung sound feature data of the selected learning lung sound data of two or more categories into a fuzzy neural network based on a weighted fuzzy membership function, I can.

12 is a block diagram illustrating an example of an internal configuration of a computing device according to an embodiment of the present invention.

12, the computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, and an input/output subsystem ( I/O subsystem) 11400, a power circuit 11500, and a communication circuit 11600. In this case, the computing device 11000 may correspond to a lung sound analysis system.

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or nonvolatile memory. have. The memory 11200 may include a software module, an instruction set, or other various data necessary for the operation of the computing device 11000.

In this case, accessing the memory 11200 from another component such as the processor 11100 or the peripheral device interface 11300 may be controlled by the processor 11100.

The peripheral device interface 11300 may couple input and/or output peripheral devices of the computing device 11000 to the processor 11100 and the memory 11200. The processor 11100 may execute various functions for the computing device 11000 and process data by executing a software module or instruction set stored in the memory 11200.

The input/output subsystem 11400 may couple various input/output peripherals to the peripherals interface 11300. For example, the input/output subsystem 11400 may include a monitor, a keyboard, a mouse, a printer, or a controller for coupling a peripheral device such as a touch screen or a sensor to the peripheral device interface 11300 as needed. According to another aspect, the input/output peripheral devices may be coupled to the peripheral device interface 11300 without going through the input/output subsystem 11400.

The power circuit 11500 may supply power to all or part of the components of the terminal. For example, the power circuit 11500 may include a power management system, one or more power sources such as batteries or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator or power. It may contain any other components for creation, management, and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, the communication circuit 11600 may enable communication with other computing devices by transmitting and receiving an RF signal, also known as an electromagnetic signal, including an RF circuit, if necessary.

The embodiment of FIG. 12 is only an example of the computing device 11000, and the computing device 11000 omits some of the components shown in FIG. 12, further includes additional components not shown in FIG. 12, or 2 It can have a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor, in addition to the components shown in FIG. 12, and various communication methods (WiFi, 3G, LTE) in the communication circuit 1160 , Bluetooth, NFC, Zigbee, etc.) may include a circuit for RF communication. Components that can be included in the computing device 11000 may be implemented as hardware, software, or a combination of hardware and software including one or more signal processing or application-specific integrated circuits.

Methods according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. An application to which the present invention is applied may be installed on a user terminal through a file provided by the file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to the request of the user terminal.

In order to confirm the performance of the lung sound analysis system according to an embodiment of the present invention as described above, classification accuracy was derived using 199 data.

The 199 data included 171 normal lung sound data and 28 abnormal lung sound data. As shown in FIG. 5, the 199 data are converted into a mel-spectrogram, and divided into a preset time length and a preset time interval to generate a divided mel-spectrogram. As a result, 1774 normal segmented Mel-Spectrograms and 267 abnormal segmented Mel-Spectrograms were generated, and analysis was performed based on this.

Among them, 267 data were randomly selected from the normal segmented mel-spectrogram to reduce the bias between normal and abnormal data. Therefore, 267 normal split Mel-Spectrograms and 267 Abnormal Split Mel-Spectrograms were used for training and testing. Used for testing.

For this split mel-spectrogram, 512 feature data were extracted by applying convolution and pooling of the VGG16 neural network model as shown in FIG. 7, and 20 feature data were selected based on the Pearson correlation coefficient. The results of performing 5-fold and 10-fold learning on the 20 selected feature data, respectively, are shown in Table 1 below.

5-fold 10-fold Average 20 95.47% 96.22% 96.1%

According to an embodiment of the present invention, feature data may be extracted through convolution and pooling by transforming the received lung sound data into 2D data.

According to an embodiment of the present invention, by dividing the two-dimensional lung sound data into a preset time length and a preset time interval, feature data may be extracted from the two-dimensional data of a standardized size to analyze the lung sound.

According to an embodiment of the present invention, by extracting feature data from learning data and selecting the feature data using Pearson's correlation coefficient, classification can be performed using feature data capable of easily classifying lung sound data.

According to an embodiment of the present invention, the accuracy of classification may be improved by classifying feature data extracted through convolution and pooling through a fuzzy neural network based on a weighted fuzzy membership function.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved. Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (10)

A lung sound analysis method performed by a lung sound analysis system including at least one processor and a memory,
A lung sound input step of receiving lung sound data;
A waste sound preprocessing step of converting the lung sound data into two-dimensional data of the lung sound of time-frequency;
A lung sound feature extraction step of extracting two or more categories of lung sound feature data based on the converted two-dimensional lung sound data;
A lung sound feature selection step of selecting lung sound feature data corresponding to the selected category from among the lung sound feature data of two or more categories; And
A lung sound classification step of deriving classification of the lung sound data based on the learned model and the selected lung sound feature data; Including,
The selected categories are,
For a plurality of learning lung sound data classified into two or more categories, the lung sound feature data of two or more categories are extracted in the same process as the lung sound preprocessing step and the lung sound feature extraction step, and among the lung sound feature data of two or more categories of the learning lung sound data A Pearson Correlation Coefficient according to the classification is determined by deriving a category of lung sound feature data meeting a preset criterion.
The method according to claim 1,
The lung sound classification step,
A lung sound analysis method in which at least one selected lung sound feature data is input into a fuzzy neural network based on a weighted fuzzy membership function, and the boundary sum of the weighted fuzzy membership function is learned.
The method according to claim 2,
The lung sound classification step,
A lung sound analysis method that derives classification of lung sound data by comparing the boundary sum of the learned weighted fuzzy membership function with a preset value.
delete The method according to claim 1,
The waste negative pretreatment step,
Lung sound analysis method for converting the received lung sound data into a mel-spectrogram.
The method of claim 5,
The waste negative pretreatment step,
The converted mel-spectrogram is divided into a preset time length and a preset time interval to generate a divided mel-spectrogram.
The method according to claim 1,
The lung sound feature extraction step,
Lung sound analysis method for extracting feature data by applying convolution and pooling of a convolutional neural network to the two-dimensional lung sound data.
The method according to claim 1,
The lung sound feature extraction step,
Lung sound analysis method for extracting feature data by applying convolution and pooling of a VGG16 neural network model to the two-dimensional lung sound data.
As a lung sound analysis system,
A data input unit for receiving lung sound data;
A data preprocessing unit converting the lung sound data into two-dimensional data of the lung sound of time-frequency;
A data feature extracting unit for extracting two or more categories of lung sound feature data based on the converted two-dimensional lung sound data;
A feature selection unit for selecting lung sound feature data corresponding to the selected category from among the lung sound feature data of two or more categories; And
A lung sound classification unit for deriving classification of lung sound data based on the learned model and the selected lung sound feature data; Including,
The selected categories are,
Pearson's correlation according to the classification among the lung sound feature data of two or more categories is extracted through the data preprocessing unit and the data feature extraction unit for a plurality of learning lung sound data divided into two or more categories. A lung sound analysis system in which a Pearson Correlation Coefficient is determined by deriving a category of lung sound feature data meeting a preset criterion.
The method of claim 9,
The waste sound classification unit,
Lung sound analysis system that inputs one or more selected lung sound feature data into a fuzzy neural network based on a weighted fuzzy membership function, and learns the boundary sum of the weighted fuzzy membership function.

Images