WO2021249311A1 - Named entity recognition method, recognition apparatus, and electronic apparatus - Google Patents

Abstract

Provided are a named entity recognition method, a recognition apparatus, and an electronic apparatus. The recognition method comprises: determining a field to which acquired text data to be recognized belongs; performing segmentation on the text data according to a pre-established entity dictionary of the field, and obtaining candidate entities by removing specific named entities included in the entity dictionary; and performing recognition on named entities in the candidate entities. The invention can improve the efficiency of recognizing named entities.

Description

Named entity identification method, identification equipment and electronic equipment

Cross-references to related applications

This disclosure requires the priority of a Chinese patent application filed with the Chinese Patent Office on June 10, 2020, the application number is 202010522433.9, and the application name is "A Chinese Named Entity Recognition Method, Recognition Device, and Electronic Equipment", and its entire contents Incorporated in this disclosure by reference.

Technical field

The present disclosure relates to the field of natural language processing, and in particular to a method for identifying named entities, identification equipment, and electronic equipment.

Background technique

Named entity recognition is a basic task of natural language processing, and it is an indispensable part of various natural language processing technologies such as information extraction, information retrieval, machine translation, and question answering systems.

According to different classification standards, named entity recognition can be divided into three types: entity type, time type, and number type. It can also be divided into three types: person name, place name, organization name, time, date, currency, and percentage. At present, the extraction of named entities is mainly carried out in the case of manually labeling the corpus. The entire recognition process has a large workload and low recognition accuracy.

It can be seen that the existing named entity recognition has a technical problem of low recognition efficiency.

Summary of the invention

In the first aspect, embodiments of the present disclosure provide a method for identifying named entities, including:

Determine the field of the acquired text data to be recognized;

Perform word segmentation on the to-be-recognized text data according to a pre-established entity dictionary in the field to obtain candidate entities, where the named entities in the entity dictionary are removed from the candidate entities;

Identify named entities among the candidate entities.

In a possible implementation manner, the identifying the named entity among the candidate entities includes:

According to the position of the candidate entity in the text data to be recognized, the named entity of the text data to be recognized is recognized from the candidate entities.

In a possible implementation manner, the segmentation of the text data to be recognized according to a pre-established entity dictionary to obtain candidate entities includes:

When the to-be-recognized text data is Chinese, divide the to-be-recognized text data into multiple word segments according to the stuttering word segmentation technology, and determine the part of speech of each word segmentation;

Extract named entities whose parts of speech are nouns from the multiple word segmentation;

From the named entities whose part of speech is a noun, remove named entities belonging to the entity dictionary to obtain candidate entities.

In a possible implementation manner, after the segmentation of the text data to be recognized according to the pre-established entity dictionary in the field to obtain candidate entities, the method further includes:

The entity extracted by the pre-established entity extraction rule is removed from the candidate entities.

In a possible implementation manner, after the segmentation of the text data to be recognized from the pre-established entity dictionary in the field to obtain candidate entities, the method further includes:

According to the context information of the text data to be recognized, an entity extraction rule for the candidate entity is determined and updated.

In the second aspect, the embodiments of the present disclosure also provide a named entity recognition device, including:

The determining unit is used to determine the field of the acquired text data to be recognized;

An obtaining unit, segmenting the text data to be recognized according to a pre-established entity dictionary in the field to obtain a candidate entity, and the named entity in the entity dictionary is removed from the candidate entity;

The identification unit is used to identify the named entity among the candidate entities.

In a possible implementation manner, the identification unit is specifically configured to:

According to the position of the candidate entity in the text data to be recognized, the named entity of the text data to be recognized is recognized from the candidate entities.

In a possible implementation manner, the obtaining unit is specifically configured to:

When the text data to be recognized is Chinese, the text data to be recognized is divided into multiple word segments according to the stuttering word segmentation technology, and the part of speech of each word segmentation is determined;

Extract named entities whose parts of speech are nouns from the multiple word segmentation;

From the named entities whose part of speech is a noun, remove named entities belonging to the entity dictionary to obtain candidate entities.

In a possible implementation manner, after the segmentation of the text data to be recognized according to the pre-established entity dictionary in the field to obtain candidate entities, the recognition device further includes:

The removing unit is used to remove the entities extracted by the pre-established entity extraction rules from the candidate entities.

In a possible implementation manner, after the segmentation of the text data to be recognized from the pre-established entity dictionary in the field to obtain candidate entities, the recognition device further includes:

The updating unit is used to determine and update the entity extraction rule for the candidate entity according to the context information of the text data to be recognized.

In the third aspect, the embodiments of the present disclosure also provide an electronic device for named entity recognition, including:

Memory and processor;

Wherein, the memory is used to store a program;

The processor is configured to execute the program in the memory and includes the following steps:

Determine the field of the acquired text data to be recognized;

Perform word segmentation on the to-be-recognized text data according to a pre-established entity dictionary in the field to obtain candidate entities, where the named entities in the entity dictionary are removed from the candidate entities;

Identify named entities among the candidate entities.

In a fourth aspect, the embodiments of the present disclosure also provide a computer-readable storage medium, including computer program instructions, which when run on a computer, cause the computer to execute the steps of the identification method described above.

Description of the drawings

FIG. 1 is a method flowchart of a named entity identification method provided by an embodiment of the disclosure;

2 is a method flowchart of step S102 in a method for identifying a named entity provided by an embodiment of the present disclosure;

3 is a structural block diagram of a named entity identification device provided by an embodiment of the disclosure;

FIG. 4 is a structural block diagram of an electronic device for named entity recognition provided by an embodiment of the disclosure.

detailed description

In order to make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. And if there is no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. Based on the described embodiments of the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative labor are within the protection scope of the present disclosure.

Unless otherwise defined, the technical or scientific terms used in the present disclosure shall have the usual meanings understood by those with ordinary skills in the field to which this disclosure belongs. The words "including" or "comprising" and other similar words used in the present disclosure mean that the element or item appearing before the word covers the element or item listed after the word and their equivalents, but does not exclude other elements or items.

The application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those of ordinary skill in the art will know that with the development of new application scenarios It appears that the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems.

As shown in FIG. 1, an embodiment of the present disclosure provides a method for identifying a named entity, including:

S101: Determine the domain of the acquired text data to be recognized;

In the specific implementation process, the text data to be recognized may be structured text data, unstructured text data, or semi-structured text data, which is not limited here. In addition, the field can be art, architecture, medical, transportation, etc.

S102: Perform word segmentation on the text data to be recognized according to the pre-established entity dictionary in the field to obtain candidate entities, and the named entities in the entity dictionary are removed from the candidate entities;

In the specific implementation process, taking the field of the text data to be recognized as the field of art as an example, the pre-established entity dictionary for this field can include entities such as painter's name, painting name, painting genre, painting theme, and artist's birthplace. . A pre-established entity dictionary in a certain field can be created based on structured text data in the field. Still taking the art field as an example, if the art field has data stored in a relational database such as mysql, correspondingly, the data in the relational database is structured entity data, which can be derived from known relationships The columns with the same content are extracted from the type database, and then merged and deduplicated to build the initial entity dictionary of the art field. For example, by extracting the columns representing painter names, aliases, and titles from the relational data, and fusing them to remove duplicates, an initial dictionary of painter names can be built, in which a painter name corresponds to an id. In addition, in the field of art, the entity can also be the name of the painting, the genre of the painting, the subject of the painting, the birthplace of the artist, and so on.

In the specific implementation process, according to the pre-established entity dictionary of the field of the text data to be recognized, word segmentation is performed on the text data to be recognized, so as to obtain candidate entities from which the named entities in the entity dictionary are removed. Still taking the field of art as an example, for example, the named entity of painter's name in the entity dictionary is removed, so that the named entity of painter's name is not included in the candidate entities. In addition, there may be one or more candidate entities, which are specifically determined according to actual applications, which are not limited here.

S103: Identify a named entity among the candidate entities.

In the specific implementation process, after the candidate entities are obtained, the named entities among the candidate entities are identified. In this case, the named entity recognized in the candidate entity and the named entity belonging to the entity dictionary in the text data to be recognized constitute the named entity of the text data to be recognized. The entire named entity process only needs to identify named entities from candidate entities, not all entities in all fields, and no need to recognize entities in the entity dictionary of the field to which the text data to be recognized belongs in advance. Thereby improving the efficiency of recognizing named entities.

In the embodiment of the present disclosure, step S103: identifying named entities in the candidate entities includes:

According to the position of the candidate entity in the text data to be recognized, the named entity of the text data to be recognized is recognized from the candidate entities.

In the specific implementation process, it can be based on the position of the candidate entity in the text data to be recognized, for example, the position of the word corresponding to the candidate entity in the sentence corresponding to the text data to be recognized, and then, according to its position, based on the field of expertise The personnel’s experience identifies the named entity of the text data to be recognized from the candidate entities.

In the embodiment of the present disclosure, as shown in FIG. 2, step S102: performing word segmentation on the text data to be recognized according to a pre-established entity dictionary in the field to obtain candidate entities, including:

S201: When the to-be-recognized text data is Chinese, divide the to-be-recognized text data into multiple word segments according to the stuttering word segmentation technology, and determine the part of speech of each word segmentation;

S202: Extract a named entity whose part of speech is a noun from multiple word segmentation;

S203: Remove the named entities belonging to the entity dictionary from the named entities whose part of speech is a noun to obtain candidate entities.

In the specific implementation process, the specific implementation process of step S201 to step S203 is as follows:

First, taking the text data to be recognized as Chinese as an example, the text data to be recognized is divided into multiple word segments according to the stuttering word segmentation technology, and the part of speech of each word segmentation is determined. It can be based on the stuttering word segmentation technology in python to segment the text data to be recognized, and divide the entire text corresponding to the text data to be recognized into non-repetitive words. Each word also corresponds to a part of speech, such as nouns, adjectives, and verbs. Wait. Then, a named entity whose part of speech is a noun is extracted from multiple word segmentation; then, from a named entity whose part of speech is a noun, the named entity belonging to the entity dictionary is removed to obtain a candidate entity. In the specific implementation process, the candidate entities are further determined by extracting named entities whose part of speech is nouns from multiple word segmentation. Since named entities are usually nouns, the efficiency of extracting candidate entities is improved, and the efficiency of named entities is improved. Recognition efficiency.

In the specific implementation process, while performing word segmentation on the text data to be recognized according to the stuttering word segmentation technology, it can be combined with the pre-built entity dictionary to segment words, ensuring that the structure of the text data to be recognized is closer to the pre-built entity dictionary. Therefore, the efficiency of extracting candidate entities is improved, and the efficiency of identifying named entities is improved.

To give a specific example, the text data to be recognized is a text introduction about the painter Da Vinci obtained from crawling on the Internet, specifically, “Da Vinci returned to Florence in 1500 and began to create the Mona Li "The Mona Lisa" used perspective and other painting methods. After that, Da Vinci went to Milan and continued to serve in the Milan court." After the word segmentation was performed according to the stammering technique, the part of speech was noun There are also "Nian", "Da Vinci", "Florence", "Mona Lisa", "Perspective", "Painting", "Method", "Milan", and "Palace". According to the pre-established location entity dictionary, it can be matched to "Florence" and "Milan" in the text data to be recognized. Remove "Florence" and "Milan" from all words whose part of speech is noun. Candidate entities include "Nian", "Da Vinci", "Mona Lisa", "Perspective", "Painting", and "Methods" ","palace". Subsequent identification of named entities among the candidate entities will reduce the scope of named entity identification and improve the efficiency of named entity identification.

In the embodiment of the present disclosure, in step S203: removing the named entity belonging to the entity dictionary from the named entity whose part of speech is the noun, and after obtaining the candidate entity, the method further includes:

The entities extracted by the pre-established entity extraction rules are removed from the candidate entities.

In the specific implementation process, after the text data to be recognized is segmented according to the pre-established entity dictionary in the field, and candidate entities are obtained, the candidate entities can be further screened, which can be the entities extracted by removing the pre-established entity extraction rules , Thereby further narrowing the scope of candidate entities, thereby improving the efficiency of named entity recognition. Still taking the above text introduction about Da Vinci as an example, the candidate entities obtained include "year", "Da Vinci", "Mona Lisa", "perspective", "painting", "method", After the "palace", according to the pre-established time extraction rule "number + year", matching to "1500", according to the pre-established painting name extraction rule "painting name", matching to "Mona Lisa", from the above candidates The "year" and "Mona Lisa" are removed from the entities, and named entities are further identified from the remaining entities, which further narrows the recognition scope of named entities and improves the efficiency of named entity recognition.

In the implementation of the present disclosure, in the process of determining candidate entities, in addition to using the above-mentioned first word segmentation, and then sequentially screening the noun part-of-speech entities according to the pre-established entity dictionary and the pre-established entity extraction rules to determine the candidate entity, After the word segmentation, the entities of the noun part of speech can be filtered according to the pre-established entity extraction rules and the pre-established entity dictionary in turn. Of course, it can also be after the word segmentation, based on the pre-established entity dictionary and the pre-established entity extraction at the same time Rules to filter the entities of the noun part of speech. In the specific implementation process, the process of determining the candidate entity can be selected according to the actual application, which is not limited here.

In the embodiment of the present disclosure, in step S102: performing word segmentation on the text data to be recognized according to a pre-established entity dictionary in the field, and obtaining candidate entities, the method further includes:

According to the context information of the text data to be recognized, the entity extraction rules for the candidate entities are determined and updated.

In the specific implementation process, after the candidate entity is obtained, the entity extraction rules for the candidate entity can be determined and updated according to the context information of the text data to be recognized. If the entity extraction rule has not been established previously, and the entity extraction rule is determined based on the context information of the text data to be recognized, the entity extraction rule is established. If there is a pre-established entity extraction rule before the candidate entity is obtained, and the entity extraction rule is different from the pre-established entity extraction rule after the candidate entity is obtained, that is, a new entity extraction rule, then the pre-established entity extraction rule is updated. Still taking the above text introduction about Da Vinci as an example, through the analysis of candidate entities, it is found that "Da Vinci" is a painter's name, but it is the same as "Da Vinci" in the pre-built painter name entity dictionary. If there is no match, a new extraction rule for extracting painter names can be established, and "·" is converted to "·", so as to update the pre-established painter name extraction rules, and then use the updated entity extraction rules to Extract entities, thereby narrowing the selection range of candidate entities, and improving the recognition efficiency of named entities.

In the embodiment of the present disclosure, the pre-established entity extraction rule may be that after obtaining a piece of unstructured text data, the entity extraction rule is established based on expert experience. Before obtaining any unstructured text data, no entity extraction rules have been established. With the subsequent identification of named entities to be recognized in text data, the entity extraction rules are constantly updated, and there are more and more entity extraction rules. With the increase of entity extraction rules, the selection of candidate entities can be further reduced, and the recognition of named entities can be further improved. efficient.

In addition, in the embodiments of the present disclosure, the pre-established entity dictionary can also be continuously updated based on more and more structured text data. There are more and more entity dictionaries. As the number of entity dictionaries increases, the candidate entities can be further reduced. The selection range of, further improve the efficiency of the recognition of named entities.

Based on the same inventive concept, as shown in FIG. 3, an embodiment of the present disclosure also provides a named entity recognition device, including:

The determining unit 10 is used to determine the field of the acquired text data to be recognized;

The obtaining unit 20 performs word segmentation on the text data to be recognized according to the pre-established entity dictionary in the field to obtain candidate entities, and the named entities in the entity dictionary are removed from the candidate entities;

The identification unit 30 is used to identify named entities among the candidate entities.

In the embodiment of the present disclosure, the identification unit 30 is specifically configured to:

According to the position of the candidate entity in the text data to be recognized, the named entity of the text data to be recognized is recognized from the candidate entities.

In the embodiment of the present disclosure, the obtaining unit 20 is specifically configured to:

When the text data to be recognized is Chinese, the text data to be recognized is divided into multiple word segments according to the stuttering word segmentation technology, and the part of speech of each word segmentation is determined;

Extract named entities whose parts of speech are nouns from multiple word segmentation;

From the named entities whose part of speech is noun, remove the named entities belonging to the entity dictionary to obtain candidate entities.

In the embodiment of the present disclosure, after performing word segmentation on the text data to be recognized according to a pre-established entity dictionary in the field, and obtaining candidate entities, the recognition device further includes:

The removing unit is used to remove the entities extracted by the pre-established entity extraction rules from the candidate entities.

In the embodiment of the present disclosure, after performing word segmentation on the text data to be recognized from the pre-established entity dictionary in the field to obtain candidate entities, the recognition device further includes:

The update unit is used to determine and update the entity extraction rule for the candidate entity according to the context information of the text data to be recognized.

Based on the same inventive concept, as shown in FIG. 4, an embodiment of the present disclosure also provides an electronic device for named entity recognition, including:

Memory 100 and processor 200;

Among them, the memory 100 is used to store programs;

The processor 200 is configured to execute a program in the memory, and includes the following steps:

Determine the domain of the acquired text data to be recognized;

According to the pre-established entity dictionary in the field, the text data to be recognized is segmented to obtain candidate entities, and the named entities in the entity dictionary are removed from the candidate entities;

Identify named entities among candidate entities.

Based on the same inventive concept, the embodiments of the present disclosure also provide a computer-readable storage medium, including a computer program. The computer program includes program instructions. When the program instructions are executed by an electronic device, the electronic device executes the naming provided in the above-mentioned embodiments. The identification method of the entity.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the device and module described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of modules is only a logical function division, and there may be other divisions in actual implementation, for example, multiple modules or components can be combined or integrated. To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or modules, and may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional modules in the various embodiments of the present disclosure may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium.

In the foregoing embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present disclosure are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to transmit to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

The technical solutions provided by the present disclosure are described in detail above. Specific examples are used in this disclosure to illustrate the principles and implementations of the present disclosure. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present disclosure. At the same time, for those of ordinary skill in the art, based on the ideas of the present disclosure, there will be changes in the specific implementation and scope of application. In summary, the content of this specification should not be construed as a limitation of the present disclosure.

Claims (12)

A method for identifying named entities, which includes: Determine the domain of the acquired text data to be recognized; Perform word segmentation on the to-be-recognized text data according to a pre-established entity dictionary in the field to obtain candidate entities, where the named entities in the entity dictionary are removed from the candidate entities; Identify named entities among the candidate entities. The method according to claim 1, wherein said identifying a named entity among said candidate entities comprises: According to the position of the candidate entity in the text data to be recognized, the named entity of the text data to be recognized is recognized from the candidate entities. The method according to claim 1, wherein said segmenting the text data to be recognized according to a pre-established entity dictionary to obtain candidate entities comprises: When the to-be-recognized text data is Chinese, divide the to-be-recognized text data into multiple word segments according to the stuttering word segmentation technology, and determine the part of speech of each word segmentation; Extract named entities whose parts of speech are nouns from the multiple word segmentation; From the named entities whose part of speech is a noun, remove named entities belonging to the entity dictionary to obtain candidate entities. The method according to any one of claims 1 to 3, wherein, after the segmentation of the text data to be recognized according to the pre-established entity dictionary of the field to obtain candidate entities, the method further comprises: The entity extracted by the pre-established entity extraction rule is removed from the candidate entities. The method according to any one of claims 1 to 3, wherein, after the segmentation of the text data to be recognized from a pre-established entity dictionary of the field to obtain candidate entities, the method further comprises : According to the context information of the text data to be recognized, an entity extraction rule for the candidate entity is determined and updated. A device for identifying named entities, which includes: The determining unit is used to determine the field of the acquired text data to be recognized; An obtaining unit, segmenting the text data to be recognized according to a pre-established entity dictionary in the field to obtain a candidate entity, and the named entity in the entity dictionary is removed from the candidate entity; The identification unit is used to identify the named entity among the candidate entities. The identification device according to claim 6, wherein the identification unit is specifically configured to: According to the position of the candidate entity in the text data to be recognized, the named entity of the text data to be recognized is recognized from the candidate entities. The identification device according to claim 6, wherein the obtaining unit is specifically configured to: When the to-be-recognized text data is Chinese, divide the to-be-recognized text data into multiple word segments according to the stuttering word segmentation technology, and determine the part of speech of each word segmentation; Extract named entities whose parts of speech are nouns from the multiple word segmentation; From the named entities whose part of speech is a noun, the named entities belonging to the entity dictionary are removed to obtain candidate entities. 8. The identification device according to any one of claims 6-8, further comprising: a removing unit configured to remove the entity extracted by the entity extraction rule established in advance from the candidate entity. 8. The recognition device according to any one of claims 6-8, further comprising: an update unit, configured to determine and update the entity extraction rule for the candidate entity according to the context information of the text data to be recognized. An electronic device for named entity recognition, which includes: Memory and processor; Wherein, the memory is used to store a program; The processor is configured to execute the program in the memory and includes the following steps: Determine the field of the acquired text data to be recognized; Perform word segmentation on the to-be-recognized text data according to a pre-established entity dictionary in the field to obtain candidate entities, from which named entities in the entity dictionary are removed; Identify named entities among the candidate entities. A computer-readable storage medium, which includes computer program instructions, which when run on a computer, cause the computer to execute the steps of the identification method according to any one of claims 1-5.

Images