CN109977406A - A kind of Chinese medicine state of an illness text key word extracting method based on sick position - Google Patents

Abstract

The invention discloses a method for extracting keywords of TCM condition text based on disease location, comprising the following steps: segmenting TCM condition text, and generating a TCM condition dictionary based on the word segmentation result of the TCM condition text; calculating the IDF of each word in the TCM condition dictionary value and TF value; according to the IDF value and TF value of the word, and the condition of the disease location in the word, the importance of the word is improved; according to the importance of each word, the m words in the top m are selected as the key of the text word. Considering that most of the text keywords in the TCM disease text are disease position words and disease words, the invention takes the disease position as the basis, and improves the extraction efficiency of keywords in the TCM disease text by weighting the disease position on the value of TF-IDF. accuracy.

Description

A method for extracting keywords from TCM disease text based on disease location

technical field

The invention belongs to the technical field of natural language processing, and in particular relates to a disease location-based method for extracting keywords from TCM disease text.

Background technique

Dialectical disease differentiation in TCM often adopts the method of trial and discord, comparison of aids and materials, and the four diagnostic methods of “looking, smelling, asking, and cutting” to diagnose and treat patients. Usually, the patient is asked about the location of the disease, the severity of the symptoms, and the presence or absence of the disease. With the continuous development of digital testing, TCM diagnosis and treatment often include Western medicine testing data, such as blood routine, urine routine and other data. Compared with general texts, such as People's Daily, online news texts, etc., Chinese medicine disease texts have the following characteristics:

1) The main components of sentences such as subject, predicate and object in TCM illness texts are not obvious, and even some of them are missing. In addition, the juxtaposition relationship in the sentence is obvious, such as "no tenderness, rebound tenderness", the correct understanding is "no tenderness", "no rebound tenderness";

2) Some western medicine testing data are often included in the texts of TCM illnesses. For example, data such as body temperature, these data bring some difficulties to algorithms based on text analysis;

3) There are many domain words in TCM disease texts. For example, the compound word "dry and wet rales" does not appear in ordinary texts;

4) The key semantic information of TCM disease text is mainly composed of words or phrases such as symptoms, disease location, whether symptoms are related or not, and the severity of symptoms.

Commonly used text keyword extraction algorithms are TF-IDF algorithm and TextRank algorithm. The TF-IDF algorithm calculates the product of the word frequency and the inverse document frequency to measure the importance of a word in the text, and then sorts in descending order according to the TF-IDF value of the word, and selects the most advanced words as text keywords. The TextRank algorithm draws on the idea of PageRank, and discovers keywords by constructing a word graph network, the core of which is to use the words in the text as the nodes of the graph. If the text window size is set to k, there is an undirected edge between the nodes whose distance between words is not greater than k. According to the word graph network, the importance of each word node is obtained by random walk, and the most important words are text keywords.

Whether it is the TF-IDF algorithm or the TextRank algorithm, there are the following problems: for each word in the text, before the algorithm is executed, each word has the same importance. The relationship between the words in each word and the content is not considered here. For TCM disease texts, most of the keywords should be disease words and symptom words in the text, that is to say, words containing disease locations are more likely to become keywords. For example, the word "diarrhea", which contains the word "abdomen", should be the key word in the text of TCM conditions.

SUMMARY OF THE INVENTION

The object of the present invention is to propose a method for extracting keywords from disease-position-based Chinese medicine disease texts, using disease-position as a feature of extracting keywords, and extracting keywords by calculating a weighted TF-IDF value.

In order to realize the above-mentioned purpose of the invention, the present invention provides the following technical solutions:

A disease location-based method for extracting keywords from TCM condition texts, comprising the following steps:

Segmentation of TCM condition text, and generate a TCM condition dictionary based on the result of TCM condition text segmentation;

Calculate the IDF value and TF value of each word in the TCM disease dictionary;

According to the IDF value and TF value of the word, and the condition of the disease location in the word, the importance of the word is improved;

According to the importance of each word, the m words in the top m are selected as the keywords of the text.

The method for extracting keywords in TCM disease text based on disease location provided by the present invention overcomes the problem of traditional text keyword extraction methods, such as TF-IDF or TextRank, that the importance of each word is the same. Most of the text keywords are disease location words and disease words. Based on disease location, by weighting the disease location on the value of TF-IDF, the accuracy of TCM disease text keyword extraction is improved.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that are used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative efforts.

Fig. 1 is a flow chart of the method for extracting keywords of TCM disease text based on disease location of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and do not limit the protection scope of the present invention.

As shown in FIG. 1 , the present embodiment provides a method for extracting keywords of TCM condition text based on disease location. The method is based on the disease location, and improves the TCM condition text by weighting the disease location on the value of TF-IDF. The accuracy of keyword extraction includes the following steps:

S101 , segmenting the text of TCM illnesses.

Specifically, when performing word segmentation of TCM disease text, for the disease text set to be segmented, according to the medical dictionary and stopWords, the disease text set is segmented, and the stop words are removed to obtain a word segmentation text set.

The disease text collection contains text cases representing TCM diagnosis. A medical dictionary is a dictionary that contains medicinal materials, prescriptions, and basic medical terms. The stop dictionary stopWords is a list of stop words related to the field of traditional Chinese medicine, such as words such as "patient" and "medical history".

S102, a TCM disease dictionary is generated based on the word segmentation result of the TCM disease text.

Wherein, the generation of the TCM disease dictionary based on the word segmentation result of the TCM disease text includes:

The words in the word segmentation text collection are counted, and the words whose frequency of occurrence belongs to the interval [α ₁ , α ₂ ] are added to the TCM disease dictionary.

S103, calculate the IDF value and TF value of each word in the TCM condition dictionary.

The IDF value refers to the inverse text frequency index. For the i-th word in the TCM condition dictionary, its IDF value is:

Among them, idf _i represents the IDF value of the ith word, |D| is the total number of TCM disease texts, and _ni is the number of TCM disease texts containing the ith word.

TF refers to the word frequency. Before calculating the TF value of the word, the disease texts belonging to the same department in the word segmentation text collection are spelled into a text D _{all, k} , and k represents the department index;

For the i-th word in the TCM condition dictionary, its TF value is:

Among them, tfi represents the TF value of the _ith word, and _ni represents the number of times the ith word appears in the text D _all,k .

S104, according to the IDF value and TF value of the word, and the condition that the word contains a disease location, improve the importance of the word.

When the word in the TCM disease dictionary contains the disease location, the importance of the word is increased. If the word does not contain the disease location, the importance of the word is not increased. The calculation formula of the importance is as follows:

Among them, word _i represents the importance of the i-th word in the TCM disease dictionary, α is used to adjust the importance of the disease location, and α≥0, w _j represents the weight of the j-th disease location, and the calculation formula is:

Among them, c _j represents the frequency of the jth disease site, and n represents the total number of disease sites.

In the present invention, the disease position not only includes the characters of various parts of the body, but also includes some basic disease characters, such as "pain" and "deficiency", and these disease characters represent the theoretical basis of TCM syndrome differentiation.

S105, according to the importance of each word, select m words in the top m positions as keywords of the text.

Specifically, the words may be arranged according to their importance, and the top m words are selected to be the keywords W={W ₁ , W ₂ , . . . , W _m } of the text D _all .

Experimental example

Suppose the disease text collection includes text A, text B, text C, text D, and text E:

Text A: The patient developed cough without obvious incentive 1 week ago, and was conscious and in good spirits. No tenderness, rebound tenderness, etc.

Text B: The patient presented with a cough without tenderness, rebound tenderness, etc.

Text C: Patient presents with cough.

Text D: Patient is sane.

Text E: Patient has elevated serum creatinine.

The text A, text B, text C, text D, and text E are calculated according to S101 to 105, wherein text A and text C are of the same type, text B and text D are of the same type, and text E is of a type.

After word segmentation and stop word removal in S101, the following text set D' is obtained, in which the word segmentation tool is used for word segmentation:

Text A: Coughing without obvious incentives 1 week ago, conscious but no tenderness, rebound tenderness

Text B: Coughing with nontender rebound tenderness

Text C: Coughing occurs

Text D: Consciousness

Text E: Elevated serum creatinine

Set α ₁ =1, α ₂ =10, according to the text set D' obtained in S101, after the screening of S102, a TCM disease dictionary X is generated, and the TCM disease dictionary X contains the words shown in Table 1:

Table 1 Dictionary X Generation

Numbering words frequency 1 Appear 3 2 cough 3 3 none 2 4 tenderness 2 5 rebound pain 2 6 consciousness 2 7 clear 2

Utilize S103 to calculate the IDF value of each word, wherein log base is taken natural logarithm e, concrete calculation result is as shown in table 2,

Table 2 IDF calculation results of dictionary X

Use S103 to calculate the TF value of each word. First, the disease texts of the same department in the text set D' are spliced into one text D _all , the texts A and C of the same type are merged into D _all-ac , the texts B and D are merged into D _all-bd , and the text E is D _all-e , and then, calculate the TF values of the words in the TCM condition dictionary X respectively, and the calculation results are shown in Table 3:

Table 3 Calculation results of TF values of the same type

According to S104, the importance of the word is increased. In the example, the disease location can select "pain" and "god". Among them, the initial results of the disease position weight are shown in Table 4:

Table 4 Initialization results of the sick position

Set α=10, according to S104, calculate the importance of the influence of the disease position in the word on the keyword, as shown in Table 5:

Table 5 word _i value calculation

According to S105, two words are selected to form keywords of the text, and the results are shown in Table 6:

Table 6 Text keyword results

The top2 keywords in D _all-ac are tenderness and rebound tenderness. The top2 keywords in D _all-bd can be any two combinations of tenderness, rebound tenderness, and consciousness. There are no keywords in D _all-e . The above examples only illustrate the calculation process of the invention and do not represent the actual situation.

The above-mentioned specific embodiments describe in detail the technical solutions and beneficial effects of the present invention. It should be understood that the above-mentioned embodiments are only the most preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, additions and equivalent substitutions made within the scope shall be included within the protection scope of the present invention.

Claims (6)

1. a kind of Chinese medicine state of an illness text key word extracting method based on sick position, comprising the following steps:

Feelings of curing the desease in text participle, and Chinese medicine state of an illness dictionary is generated based on Chinese medicine state of an illness text word segmentation result；

Calculate the IDF value and TF value of each word in Chinese medicine state of an illness dictionary；

According to including sick position situation in the IDF value of word and TF value and word, the different degree of word is promoted；

According to the different degree of each word, select m before coming m words for the keyword of text.

2. the Chinese medicine state of an illness text key word extracting method based on sick position as described in claim 1, which is characterized in that carrying out When Chinese medicine state of an illness text segments, for state of an illness text collection to be segmented, according to Medical Dictionary and deactivated dictionary, to state of an illness text Set is segmented, and removes stop words, obtains participle text collection.

3. the Chinese medicine state of an illness text key word extracting method based on sick position as claimed in claim 2, which is characterized in that the base Generating Chinese medicine state of an illness dictionary in Chinese medicine state of an illness text word segmentation result includes:

, there is frequency for satisfaction and belongs to section [α in word in statistics participle text collection₁,α₂] between word be added to Chinese medicine In state of an illness dictionary.

4. the Chinese medicine state of an illness text key word extracting method based on sick position as claimed in claim 2, which is characterized in that in I-th of word in feelings of curing the desease dictionary, IDF value are as follows:

Wherein, idf_iIndicate the IDF value of i-th of word, | D | for Chinese medicine state of an illness text sum, n_iFor comprising in i-th of word Feelings of curing the desease amount of text.

5. the Chinese medicine state of an illness text key word extracting method based on sick position as claimed in claim 2, which is characterized in that calculating Before the TF value of word, the state of an illness text for belonging to same department in text collection will be segmented respectively and is combined into a text D_all,k, k table Show that department indexes；

For i-th of word in Chinese medicine state of an illness dictionary, TF value are as follows:

Wherein, tf_iIndicate the TF value of i-th of word, n_iIndicate i-th of word in text D_all,kThe number of middle appearance.

6. the Chinese medicine state of an illness text key word extracting method based on sick position as claimed in claim 2, which is characterized in that work as Chinese medicine When in state of an illness dictionary in word including sick position, the different degree of word, the calculation formula of different degree are promoted are as follows:

Wherein, eord_iThe different degree of i-th of word in Chinese medicine state of an illness dictionary is represented, α is used to adjust disease position different degree, and α >=0, w_jThe weight for representing j-th of sick position, its calculation formula is:

Wherein, c_jIndicate the frequency that j-th of sick position occurs, n indicates sick position total quantity.