KR102770519B1 - Method and Apparatus for Natural Language Processing using External Knowledge - Google Patents

Abstract

The present disclosure relates to a method and device for natural language processing using external knowledge. According to one embodiment of the present disclosure, a method for natural language processing using external knowledge can be provided, including: a step of obtaining an input text including a natural language expression; a step of generating external knowledge reference information for the input text by referring to a pre-generated external knowledge database; a step of inputting the input text and the external knowledge reference information into a pre-learned natural language processing model; and a step of obtaining an output of the natural language processing model including intent classification information corresponding to the input text and slot tagging information corresponding to the input text.

Description

{Method and Apparatus for Natural Language Processing using External Knowledge}

The present disclosure relates to a method and device for natural language processing using external knowledge.

Recent advances in natural language processing (NLP) technology have made it possible to provide services that provide natural conversations between human users and AI agents. These conversational AI services are being utilized in various technological fields and service fields, such as chatbots or voice recognition assistants.

In particular, the importance of task-oriented dialogue systems that aim to satisfy users' specific needs by using artificial intelligence agents is increasing, and the main tasks of such task-oriented dialogue systems are intent classification and slot tagging of users' input text.

In the case of a system based on a conditional random field (CRF) according to the prior art, a relatively high accuracy result value can be obtained by sequentially inferring the result value for each token in the slot tagging task and considering the linguistic context around each token, but there was a problem that the computation speed was slow.

In addition, in the case of systems based on non-autoregressive techniques according to conventional technology, there was a problem that although tasks could be performed at a relatively fast computational speed, the accuracy of the inferred results was low.

That is, since conventional natural language processing models that perform intent classification and slot tagging tasks inevitably have inversely proportional accuracy and computational speed, a new technology is needed that enables fast computation for performing the tasks while performing the tasks with high accuracy.

The background technology described above is technical information that the inventor possessed for deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be considered as publicly known technology disclosed to the general public prior to the application for the present invention.

The present disclosure provides a natural language processing method and device using external knowledge. The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems and advantages of the present disclosure that are not mentioned can be understood by the following description, and will be more clearly understood by the embodiments of the present disclosure. In addition, it will be understood that the problems and advantages to be solved by the present disclosure can be realized by the means and combinations thereof indicated in the patent claims.

As a technical means for achieving the above-described technical task, a first aspect of the present disclosure can provide a natural language processing method using external knowledge, including: a step of obtaining an input text including a natural language expression; a step of generating external knowledge reference information for the input text by referring to a pre-generated external knowledge database; a step of inputting the input text and the external knowledge reference information into a pre-learned natural language processing model; and a step of obtaining an output of the natural language processing model including intent classification information corresponding to the input text and slot tagging information corresponding to the input text.

A second aspect of the present disclosure may provide a natural language processing device using external knowledge, comprising: a communication module for performing communication with an external device; a memory having at least one program stored therein; and a processor operating by executing the at least one program, wherein the processor controls the communication module to obtain an input text including a natural language expression, generates external knowledge reference information for the input text by referencing a pre-generated external knowledge database, inputs the input text and the external knowledge reference information into a pre-learned natural language processing model, and obtains an output of the natural language processing model including intent classification information corresponding to the input text and slot tagging information corresponding to the input text.

A third aspect of the present disclosure can provide a computer-readable recording medium having recorded thereon a program for executing the method of the first aspect of the present disclosure on a computer.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to the problem solving means of the present disclosure described above, the intention classification task and the slot tagging task can be performed with high accuracy by referencing an external knowledge database.

In addition, according to the problem solving means of the present disclosure, the slot tagging task can be performed at a high computational speed by using a classifier that generates slot tagging information based on a non-self-regression technique.

In addition, according to the problem solving means of the present disclosure, the trade-off between computational speed and accuracy according to the prior art can be overcome by using a classifier that references an external knowledge database and generates slot tagging information based on a non-self-regression technique.

FIG. 1 is a schematic configuration diagram of a system including a user terminal, a natural language processing device, and an external knowledge database according to one embodiment.
FIG. 2 is an exemplary diagram for explaining a process of generating external knowledge reference information by referencing an external knowledge database according to one embodiment.
FIG. 3 is an exemplary diagram illustrating an expression, a type for the expression, and an expression-type pair included in an external knowledge database according to one embodiment.
FIG. 4 is an exemplary diagram for explaining a process of dividing an input text into a plurality of n-grams and searching for the plurality of n-grams in an external knowledge database according to one embodiment.
FIG. 5 is an exemplary diagram for explaining a process of matching a type to an expression based on a predetermined criterion according to one embodiment.
FIG. 6 is an exemplary diagram for explaining a natural language processing model according to one embodiment.
FIG. 7 is an exemplary diagram for explaining a model encoder included in a natural language processing model according to one embodiment.
FIG. 8 is an exemplary diagram illustrating a model decoder and classifier included in a natural language processing model according to one embodiment.
FIG. 9 is an exemplary diagram illustrating a second classifier that generates slot tagging information based on a non-self-regression technique according to one embodiment.
Figure 10 is an example of a method for processing natural language using external knowledge.
Fig. 11 is a block diagram of a natural language processing device according to one embodiment.

The advantages and features of the present invention, and the methods for achieving them, will become clear with reference to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms, and it should be understood to include all transformations, equivalents, and substitutes included in the spirit and technical scope of the present invention. The embodiments presented below are provided to ensure that the disclosure of the present invention is complete, and to fully inform a person having ordinary skill in the art to which the present invention belongs of the scope of the invention. In describing the present invention, if it is determined that a specific description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood to not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and/or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or by circuit configurations for a given function. Also, for example, the functional blocks of the present disclosure may be implemented by various programming or scripting languages. The functional blocks may be implemented by algorithms that are executed on one or more processors. In addition, the present disclosure may employ conventional techniques for electronic environment settings, signal processing, and/or data processing, etc. Terms such as "mechanism", "element", "means", and "configuration" may be used broadly and are not limited to mechanical and physical configurations.

In addition, the connecting lines or connecting members between components depicted in the drawings are only illustrative of functional connections and/or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that may be replaced or added.

The present disclosure will be described in detail with reference to the attached drawings below.

FIG. 1 is a schematic configuration diagram of a system including a user terminal, a natural language processing device, and an external knowledge database according to one embodiment.

Referring to FIG. 1, a system (100) according to one embodiment may include a user terminal (110), a natural language processing device (120), and an external knowledge database (130).

The user terminal (110) of the present disclosure is a terminal of a user who uses an artificial intelligence conversation service, and the user terminal (110) according to one embodiment may include a terminal that provides an interface to a user so that the user can input text.

The user terminal (110) according to one embodiment may be a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a PDA (personal digital assistant), a laptop, a media player, a micro server, a GPS (global positioning system) device, an e-book reader, a digital broadcasting terminal, a navigation device, a kiosk, an MP3 player, a digital camera, a home appliance, a device equipped with a camera, and other mobile or non-mobile computing devices. In addition, the user terminal (210) may be a wearable device such as a watch, glasses, a hair band, and a ring equipped with a communication function and a data processing function, but is not limited thereto.

Meanwhile, according to another embodiment, unlike that illustrated in FIG. 1, the system (100) may not include a user terminal (110), and the natural language processing device (120) may directly receive text input from the user.

The natural language processing device (120) of the present disclosure may mean any type of server that manages a web and/or app that can provide various services utilizing natural language processing, artificial intelligence conversation services, or destination-oriented conversation services, and may mean a server that manages homepages of various management platforms, etc. However, it is not limited thereto.

A natural language processing device (120) according to one embodiment may include a mobile electronic device having mobility, such as a computing device mounted on a vehicle, or may be implemented as a smartphone, tablet PC, PC, smart TV, PDA (personal digital assistant), laptop, media player, navigation, and other mobile electronic devices.

The external knowledge database (130) of the present disclosure is a database that stores external knowledge information that can be referenced in the process of the natural language processing device (120) performing natural language processing on input text. The external knowledge information according to one embodiment may be created in advance by the management entity of the natural language processing device (120), i.e., the management entity of a server providing an artificial intelligence conversation service. The structure and types of information stored in the external knowledge database (130) will be described later with reference to FIG. 3, etc.

The external knowledge database (130) may include random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory.

According to one embodiment, the external knowledge database (130) may include a storage medium that provides external knowledge information by performing communication with the natural language processing device (120) using a network outside of the natural language processing device (120), as illustrated in FIG. 1. For example, the external knowledge database (130) may be a cloud server that the natural language processing device (120) can access using a network.

Meanwhile, according to another embodiment, the external knowledge database (130) may include a storage medium that is included as a part of the natural language processing device (120) and can be directly accessed from the natural language processing device (120), unlike as shown in FIG. 1.

Each of the devices illustrated in FIG. 1 can communicate with each other and/or with other devices via a network. A network is a comprehensive data communication network that allows different entities to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile radio communication networks. For example, the network may include a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof. In addition, wireless communication may include, but is not limited to, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy, ZigBee, Wi-Fi Direct (WFD), ultra-wideband (UWB), infrared data association (IrDA), and near field communication (NFC).

According to one embodiment, the user terminal (110) can transmit an input text input by the user to the natural language processing device (120) by performing communication through a network, and the natural language processing device (120) can obtain the input text. Meanwhile, according to another embodiment, the natural language processing device (120) can include its own input interface for the user and can directly receive an input text from the user.

In addition, according to one embodiment, the natural language processing device (120) can refer to an external knowledge database (130) by performing communication through a network. The natural language processing device (120) can generate external knowledge reference information for an input text by referring to the pre-generated external knowledge database (130). At this time, the natural language processing device (120) can obtain a target pair, which is a type of data stored in the external knowledge database (130), by performing communication through a network.

Additionally, according to one embodiment, the natural language processing device (120) can perform a search for information related to the input text in an external knowledge database (130) by performing communication through a network. For example, the natural language processing device (120) can search for a plurality of n-grams for the input text in the external knowledge database (130).

In addition, according to one embodiment, the natural language processing device (120) may transmit a response to an input text generated based on intent classification information and slot tagging information to the user terminal (110) by performing communication through a network. At this time, the user terminal (110) may display the response to the input text to the user. Meanwhile, according to another embodiment, the natural language processing device (120) may include its own display unit and may display the response to the input text through the display unit.

FIG. 2 is an exemplary diagram for explaining a process of generating external knowledge reference information by referencing an external knowledge database according to one embodiment.

A natural language processing device (120) can obtain an input text (201) including a natural language expression. In the present disclosure, a natural language expression is an expression made up of a language used in human communication, and a natural language expression according to one embodiment can include expressions such as words and phrases.

As described above with reference to FIG. 1, a natural language processing device (120) according to one embodiment may obtain an input text (201) by performing communication with a user terminal (110) through a network, or obtain an input text (201) by directly receiving the input text (201) from a user.

Referring to FIG. 2, the natural language processing device (120) can generate external knowledge reference information (202) for input text (201) by referencing a parasitic external knowledge database (130).

According to one embodiment, the natural language processing device (120) can refer to the external knowledge database (130) by searching or retrieving external knowledge information related to the input text (201) from the external knowledge database (130).

According to one embodiment, the natural language processing device (120) can select and acquire external knowledge information that can be used in the process of generating a response to an input text (201) from among external knowledge information stored in the external knowledge database (130) by referencing the external knowledge database (130).

The external knowledge reference information (202) of the present disclosure means information that matches the type of natural language expression to the natural language expression included in the input text (201). According to one embodiment, the external knowledge reference information (202) may include information that matches the type of natural language expression to a character at a position corresponding to the natural language expression for each character constituting the input text (201).

FIG. 3 is an exemplary diagram illustrating an expression, a type for the expression, and an expression-type pair included in an external knowledge database according to one embodiment.

Referring to FIG. 3, an external knowledge database (130) according to one embodiment may include a plurality of expression-type pairs, each of which consists of an expression (320) and a type (310) for the expression. The expression (320) according to one embodiment may include a natural language expression, and may include words and phrases. For example, the expression (320) may include natural language expressions such as Seoul, Hong Gil-dong, and Seoul Bank.

A type (310) for an expression according to one embodiment may include a classification indicating which category the expression (320) belongs to. For example, the type (310) for an expression may be set as a label or tag used to classify what kind of object or concept the expression (320) indicates.

Meanwhile, the information required for the natural language processing process and the characteristics of the information may be different in a specific field or specialized area. The external knowledge database (130) according to one embodiment may be created differently depending on the purpose or field in which the natural language processing device (120) is used. That is, the external knowledge database (130) may be created differently depending on the domain, which is the problem area that the natural language processing device (120) is trying to solve.

A domain according to one embodiment may include at least one of various areas in which natural language processing using artificial intelligence can be usefully utilized, such as navigation, map services, reservation services, medical record analysis, medical conversation analysis, legal document analysis, stock market analysis, and/or real estate information analysis.

Meanwhile, according to one embodiment, a domain may be pre-selected before the input text (201) is acquired. According to one embodiment, an external knowledge database (130) utilized by the natural language processing device (120) among a plurality of databases may be selected based on the selected domain.

According to another embodiment, a type that is a target of reference in an external knowledge database (130) may be set based on a selected domain. For example, when a first domain is selected, external knowledge information with a region name and an organization name as types may be a target of reference, and when a second domain is selected, external knowledge information with a region name and a person name as types may be a target of reference.

According to one embodiment, the types of expressions (320) and types (310) for expressions stored in an external knowledge database (130) may be different depending on the domain, and even the same expressions may be mapped to different types depending on the domain.

Furthermore, even when the natural language processing device (120) is used in the same domain, external knowledge information that can be appropriately used may be adopted differently depending on the creator of the external knowledge database (130), so the expression (320) stored in the external knowledge database (130) and the type (310) for the expression may be different, and the same expression may be mapped to different types.

A type (310) for an expression according to one embodiment may include at least one type set based on a domain.

For example, if the domain is selected as a navigation or map service, the type (310) for the expression may be set to a place name and/or an organization name, etc.

For another example, if a domain is selected as a reservation service, the type (310) for the expression may be set to a region name, an organization name, and/or a person name.

As another example, if the domain is selected as medical record analysis or clinical conversation analysis, the type (310) for the expression may be set to a person's name, a body part, and/or a disease name.

According to one embodiment of the external knowledge database (130), the type (310) for expressions may include at least one of various noun types such as a region name, a person name, and an institution name. In the input text (201) that is the target of natural language processing, there are many cases where information with relatively high importance is revealed in noun expressions such as a region name, a person name, and an institution name. Therefore, by including at least one of various noun types such as a region name, a person name, and an institution name in the type (310) for expressions, the natural language processing device (120) may perform intent classification and/or slot tagging tasks with fast calculation and high accuracy.

A region name according to one embodiment may be a type for a characteristic region, a specific place, or a specific location expression. For example, an expression with a region name as a type may include names of cities such as Seoul and Busan, names of geographical areas such as the Han River and Namsan, and/or names referring to objects located at a specific address such as Building A, Company B, and Restaurant C. However, this is not limited thereto.

A person's name according to one embodiment may be a type for various forms of name expressions, such as real names, nicknames, stage names, etc. of a specific individual or a specific group. For example, expressions using a person's name as a type may include individual names such as Hong Gil-dong and John Smith, and group names such as BTS and The Beatles. However, this is not limited thereto.

The organization name according to one embodiment may be a type for the names of various types of organizational institutions, such as specific organizations, enterprises, groups, and government agencies. For example, an expression using the organization name as a type may include names of enterprises or organizations, such as OO Automobile and OO Electronics, and names of government agencies, such as OO Department and OO Agency. However, this is not limited thereto.

The expression-type pair of the present disclosure is external knowledge information stored in an external knowledge database (130), and is information indicating whether a specific expression corresponds to a certain type. The specific storage structure in which the expression-type pair is stored in the external knowledge database (130) may have various structures according to conventional technology or technology to be developed in the future.

For example, the external knowledge database (130) may store expression-type pairs based on at least one of a relational database structure, a NoSQL document-based structure, and a key-value database structure, but is not limited thereto.

According to one embodiment, the natural language processing device (120) can generate external knowledge reference information (202) by referencing an external knowledge database (130) and matching at least one of the expression-type pairs stored in the external knowledge database (130) to an input text (201).

For example, the natural language processing device (120) can obtain an input text (201) such as “Guide to Seoul Bank” and select and obtain an expression-type pair such as “Seoul Bank-organization name” by referencing an external knowledge database (130). The natural language processing device (120) can generate external knowledge reference information (202) by matching “Seoul Bank-organization name” to “Guide to Seoul Bank”.

As an example, the natural language processing device (120) can generate external knowledge reference information (202) in which type information of an institution name is assigned to “seo”, “ul”, “eun”, and “haeng”, and type information of no type is assigned to “eu”, “ro”, “an”, and “nae”.

FIG. 4 is an exemplary diagram for explaining a process of dividing an input text into a plurality of n-grams and searching for the plurality of n-grams in an external knowledge database according to one embodiment.

Referring to FIG. 4, according to one embodiment, the natural language processing device (120) may divide the input text (201) into a plurality of n-grams (410). An n-gram may be understood as a subset composed of n consecutive characters in a character set such as the input text (201).

For example, if the input text (201) is “Seoul Bank,” the natural language processing device (120) can separate “Seoul Bank” into “Seoul Bank” as 4-grams, “Seoul” and “Ulbank” as 3-grams, and “Seoul,” “Ulbank,” and “Bank” as 2-grams.

The natural language processing device (120) can search for multiple n-grams (410) in an external knowledge database (130) and select at least one target pair (420) from among expression-type pairs based on the search results.

For example, if the external knowledge database (130) stores external knowledge information as illustrated in FIG. 3 and the input text (201) is “Seoul Bank,” the natural language processing device (120) can search for “Seoul Bank,” “Seoul,” “Ulbank,” “Seoul,” “Ulbank,” and “Bank” in the external knowledge database (130).

At this time, the results searched from the external knowledge database (130) may be “Seoul Bank” and “Seoul,” and the natural language processing device (120) may select “Seoul Bank-organization name” and “Seoul-region name” as target pairs (420) from among the expression-type pairs stored in the external knowledge database (130).

The target pair (420) of the present disclosure may be composed of a target expression and a target type. The target expression according to one embodiment may include an expression included in the input text (201) and an expression stored in an external knowledge database (130). The target type according to one embodiment may include a type for the target expression.

In the example of the target pair (420) described above, “Seoul Bank” and “Seoul” are target expressions, and “organization name” and “region name,” which are types for each of the target expressions, can be understood as target types for each of the target expressions.

According to one embodiment, the natural language processing device (120) can generate external knowledge reference information (202) by matching a target type to a target expression included in an input text (201). For example, the natural language processing device (120) can generate external knowledge reference information (202) by matching a first target type, "organization name," to characters at positions corresponding to a first target expression, "Seoul Bank," among the characters constituting the input text (201), namely, "Seo," "Ul," "Eun," and "Haeng," and by matching a second target type, "region name," to characters at positions corresponding to a second target expression, "Seoul," namely, "Seo" and "Ul."

FIG. 5 is an exemplary diagram for explaining a process of matching a type to an expression based on a predetermined criterion according to one embodiment.

According to one embodiment, the target pair (420) may include a first pair and a second pair, and the first pair may be composed of a first expression and a first type, and the second pair may be composed of a second expression and a second type. According to one embodiment, the second type may be the same as or different from the first type. In this case, the first expression and the second expression may be understood as target expressions, and the first type and the second type may be understood as target types.

According to one embodiment, the first expression and the second expression may include overlapping characters. The second expression according to one embodiment may be different from the first expression and may include all characters that make up the first expression. For example, the first pair and the second pair may be "Seoul-region name" and "Seoul Bank-institution name", respectively. For another example, the first pair and the second pair may be "Seoul Bank-institution name" and "Seoul Bank Seoul Sales Department-region name", respectively.

In the process of the natural language processing device (120) matching target types such as the first type and the second type to target expressions such as the first expression and the second expression included in the input text (201), a predetermined criterion may be used in the matching process for characters that are included in duplicate in the first expression and the second expression.

For example, if the first pair and the second pair are "Seoul-region name" and "Seoul Bank-institution name", a predetermined criterion may be used in the process of matching the target type to "Seoul", which is a character included in the first and second expressions.

For another example, if the first pair and the second pair are "Seoul Bank-organization name" and "Seoul Bank Seoul Sales Department-region name", a predetermined criterion may be used in the process of matching the target type to "Seoul Bank", which is a character included in the first and second expressions.

Referring to FIG. 5, the natural language processing device (120) can generate external knowledge reference information (202) by matching at least one of the first type and the second type to the first expression based on a predetermined criterion. Meanwhile, the external information reference information (510, 520, 530, 540) according to the predetermined criterion illustrated in FIG. 5 can be understood as an example of the external information reference information (202) illustrated in FIG. 2. However, the predetermined criterion is not limited to the example described below with reference to FIG. 5.

External information reference information (510) according to the first criterion and external information reference information (520) according to the second criterion, as shown in FIG. 5, are examples in which only one type is matched to one character, and external information reference information (530) according to the third criterion and external information reference information (540) according to the fourth criterion are examples in which not only one type but also multiple types are matched to one character.

The examples below are explained under the assumption that the second expression is different from the first expression and includes all characters that make up the first expression, and in the examples below, the first pair, the second pair, the first expression, the second expression, the first type, and the second type can be understood as relative expressions.

For example, in the relationship between “Seoul Bank Seoul Sales Department” and “Seoul” or “Seoul Bank Seoul Sales Department” and “Seoul Bank”, “Seoul Bank Seoul Sales Department” can be understood as the second expression and “Seoul” and “Seoul Bank” can be understood as the first expression, and in the relationship between “Seoul Bank” and “Seoul”, “Seoul Bank” can be understood as the second expression and “Seoul” can be understood as the first expression.

Below, the process by which the natural language processing device (120) generates external information reference information (510) according to the first criterion will be described.

The first criterion may be to sequentially perform matching from the target expression with the longest string length to the shortest target expression, and if the previously matched type is different from the subsequently matched type, the previously matched type may be overwritten by the subsequently matched type.

For example, if the external knowledge database (130) stores external knowledge information as illustrated in FIG. 3 and the input text (201) is “Seoul Bank Seoul Sales Department,” the target pairs may be selected as “Seoul-region name,” “Seoul Bank-institution name,” and “Seoul Bank Seoul Sales Department-region name.”

The natural language processing device (120) can perform a first matching on the target expression "Seoul Bank Seoul Sales Department" to match the target type "regional name" to each character. Thereafter, the natural language processing device (120) can perform a second matching on the target expression "Seoul Bank" to overwrite the target type "organization name" to each character that constitutes "Seoul Bank". Thereafter, the natural language processing device (120) can perform a final matching on the target expression "Seoul" to overwrite the target type "regional name" to each character that constitutes "Seoul".

The natural language processing device (120) can sequentially perform the first matching, the second matching, and the final matching to generate external information reference information (510) according to the first criterion in which the target type “organization name” is matched to positions 3 and 4 among the nine characters that make up the input text (201) “Seoul Bank Seoul Sales Department” and the target type “region name” is matched to the remaining positions.

Below, the process by which the natural language processing device (120) generates external information reference information (520) according to the second criterion will be described.

A second criterion may be to match the target type to the target expression whose string length is longer among the first type and the second type.

For example, if the external knowledge database (130) stores external knowledge information as illustrated in FIG. 3, and the input text (201) is “Seoul Bank Seoul Sales Department,” the natural language processing device (120) can match the target type “regional name” for “Seoul Bank Seoul Sales Department,” which is the target expression with the longest string length among the target expressions “Seoul,” “Seoul Bank,” and “Seoul Bank Seoul Sales Department,” to all characters that make up “Seoul Bank Seoul Sales Department.”

That is, the natural language processing device (120) matches the target type "organization name" according to the target pair "Seoul Bank-organization name" to the target expression "Seoul Bank" included in the input text (201), rather than matching the target type "region name" according to the target pair "Seoul Bank Seoul Sales Department-region name", thereby generating external information reference information (520) according to the second criterion in which "region name" is matched to all characters constituting the input text (201), "Seoul Bank Seoul Sales Department."

Meanwhile, for another example, if the external knowledge database (130) stores external knowledge information as illustrated in FIG. 3, and the input text (201) is “Seoul Bank,” the natural language processing device (120) can match the target type “organization name” for the target expression “Seoul Bank”, which has a longer string length among the target expressions “Seoul” and “Seoul Bank,” to all characters that constitute “Seoul Bank.”

That is, the natural language processing device (120) can generate external information reference information (520) according to the second criterion in which “organization name” is matched to all characters constituting “Seoul Bank,” which is the input text (201), by matching the target type “region name” according to the target pair “Seoul-region name” to the target expression “Seoul” included in the input text (201), rather than matching the target type “organization name” according to the target pair “Seoul-region name.”

Below, the process by which the natural language processing device (120) generates external information reference information (530) according to the third criterion will be described.

A third criterion could be that the first expression matches both the first type and the second type.

For example, if the external knowledge database (130) stores external knowledge information as illustrated in FIG. 3 and the input text (201) is “Seoul Bank Seoul Sales Department,” the target pairs may be selected as “Seoul-region name,” “Seoul Bank-institution name,” and “Seoul Bank Seoul Sales Department-region name.”

The natural language processing device (120) can match the target type “organization name” to the target expression “Seoul Bank” according to the target pair “Seoul Bank-organization name”, and can match the target type “region name” to the target expression “Seoul Bank Seoul Sales Department” according to the target pair “Seoul-region name” and “Seoul Bank Seoul Sales Department”.

Finally, the natural language processing device (120) can generate external information reference information (530) according to the third criterion in which, among the nine characters composing the input text (201), “Seoul Bank Seoul Sales Department,” the target types “regional name” and “organization name” are matched at positions 1 to 4, and the target type “regional name” is matched at the remaining positions.

Meanwhile, as another example, if the external knowledge database (130) stores external knowledge information as shown in FIG. 3 and the input text (201) is “Seoul Bank,” the target pairs may be selected as “Seoul-region name” and “Seoul Bank-institution name.”

The natural language processing device (120) can match the target type “organization name” to the target expression “Seoul Bank” according to the target pair “Seoul Bank-organization name” and can match the target expression “Seoul” to the “region name” according to the target pair “Seoul-region name”.

Finally, the natural language processing device (120) can generate external information reference information (530) according to the third criterion in which the target types “region name” and “organization name” are matched at positions 1 and 2 among the four characters composing the input text (201) “Seoul Bank” and the target type “organization name” is matched at the remaining positions.

Below, the process by which the natural language processing device (120) generates external information reference information (540) according to the fourth criterion will be described.

The fourth criterion may be to match both the first type and the second type to the first expression, while reflecting the number of matches. That is, the fourth criterion may be understood as assigning target type weights to each character based on the frequency of matching.

For example, if the external knowledge database (130) stores external knowledge information as illustrated in FIG. 3 and the input text (201) is “Seoul Bank Seoul Sales Department,” the target pairs may be selected as “Seoul-region name,” “Seoul Bank-institution name,” and “Seoul Bank Seoul Sales Department-region name.”

The natural language processing device (120) can count the matching of the target type "region name" once in the target expression "Seoul" location according to the target pair "Seoul-region name", count the matching of the target type "organization name" once in the target expression "Seoul Bank" location according to the target pair "Seoul Bank-organization name", and count the matching of the target type "region name" once in the target expression "Seoul Bank Seoul Business Department" according to the target pair "Seoul Bank Seoul Business Department-region name".

Finally, the natural language processing device (120) can generate external information reference information (540) according to the fourth criterion in which, among the nine characters constituting the input text (201), “Seoul Bank Seoul Sales Department,” the target type “region name” is matched twice and “organization name” once at positions 1 and 2, the target type “region name” is matched once and “organization name” once at positions 3 and 4, the target type “region name” is matched twice and “organization name” zero times at positions 5 and 6, and the target type “region name” is matched once and “organization name” zero times at the remaining positions.

Fig. 6 is an exemplary diagram for explaining a natural language processing model according to one embodiment. The input text (601) and external knowledge reference information (602) illustrated in Fig. 6 may correspond to the input text (201) and external knowledge reference information (202) illustrated in Fig. 2.

Referring to FIG. 6, a natural language processing device (120) can input input text (601) and external knowledge reference information (602) into a pre-learned natural language processing model (600) and obtain an output of the natural language processing model (600).

According to one embodiment, the natural language processing device (120) can obtain an output of a natural language processing model (600) including intent classification information (631) corresponding to input text (601) and slot tagging information (632) corresponding to the input text.

For example, the natural language processing device (120) may input “Guide to Seoul Bank” as input text (601) into the natural language processing model (600) and input external knowledge reference information (602) about “Guide to Seoul Bank.” At this time, the intent classification information (631) acquired by the natural language processing device (120) from the natural language processing model (600) may indicate that the input text (601) is a sentence with the intent of “finding directions.”

In addition, the slot tagging information (632) obtained by the natural language processing device (120) from the natural language processing model (600) may indicate that the characters constituting “Seoul Bank” in the input text (601), “Guide to Seoul Bank,” or tokens corresponding to the characters, are labeled to a class called bank.

Meanwhile, slot tagging information (632) according to one embodiment may be implemented as a BIO expression that expresses a slot start position as “B-”, a slot inside position as “I-”, and a slot outside position as “O”, but is not limited thereto.

A natural language processing model (600) according to one embodiment may include a model encoder (610), a model decoder (620), and a classifier (630).

According to one embodiment, at least a part of at least one of the model encoder (610), the model decoder (620), and the classifier (630) may be implemented based on at least one or a combination of two or more of various artificial neural network models, such as a pre-net, a CBHG module, an autoencoder, a DNN (Deep Neural Network), a CNN (convolutional neural network), an RNN (Recurrent Neural Network), an LSTM (Long Short-Term Memory Network), and a BRDNN (Bidirectional Recurrent Deep Neural Network).

According to one embodiment, the natural language processing device (120) may input input text (601) and external knowledge reference information (602) into a model encoder (610) to obtain a language model hidden state (611) and external knowledge embedding (612) as outputs of the model encoder (610).

A model encoder (610) according to one embodiment can convert high-dimensional data input to the model encoder (610) into low-dimensional data and output it. At this time, the output of the model encoder (610) can represent the characteristics of the input data, and the output of the model encoder (610) can be transmitted to the model decoder (620).

In the present disclosure, the hidden state means an output of a specific layer or a specific module of a natural language processing model, and an output that is not a final output of the natural language processing model. The hidden state according to one embodiment is used as an input of the next layer or the next module, and the hidden state of the present disclosure includes a language model hidden state (611) and a decoder hidden state (621).

In addition, in the present disclosure, the external knowledge embedding (612) is an embedding extracted from external knowledge reference information (602), and the external knowledge embedding (612) according to one embodiment may include an embedding transformed so that meaningful features of the external knowledge reference information (602) can be expressed in a low-dimensional continuous vector space.

According to one embodiment, the natural language processing device (120) can input a language model hidden state (611) and external knowledge embedding (612) to a model decoder (620) to obtain a decoder hidden state (621) as an output of the model decoder (620).

A model decoder (620) according to one embodiment may obtain the output of the model encoder (610) as input and additionally extract or transform information. According to one embodiment, a decoder hidden state (621), which is an output of the model decoder (620), may include data representing meaningful features of the input text (601) for intent classification and slot tagging.

The natural language processing device (120) can input the decoder hidden state (621) to the classifier (630) and obtain the output of the natural language processing model (600) as the output of the classifier (630). The classifier according to one embodiment can generate intent classification information (631) by classifying the intent or purpose of the input text (601) from the decoder hidden state (621), and can generate slot tagging information (632) by labeling at least one slot with an appropriate class for the input text (601) from the decoder hidden state (621).

FIG. 7 is an exemplary diagram for explaining a model encoder included in a natural language processing model according to one embodiment.

Referring to FIG. 7, a model encoder (610) according to one embodiment may include a first processing unit (710) and a second processing unit (720). The first processing unit (710) according to one embodiment may include a tokenization layer (711) that tokenizes input text (601).

A tokenization layer (711) according to one embodiment may convert an input text (601) into a token having a predetermined unit by taking the input text (601) as input. For example, the tokenization layer (711) may convert the input text (601) "Seoul Bank Guide" into individual tokens corresponding to "Seo", "Ul", "Eun", "Haeng", "Eu", "Ro", "An", and "Nae". Meanwhile, a predetermined unit is not limited to a single letter.

A first processing unit (710) according to one embodiment may include a language model layer (712) that outputs a language model hidden state (611). The language model layer (712) according to one embodiment may include a language model trained to capture and extract contextual information for an input token sequence. The language model layer (712) according to one embodiment may output a language model hidden state (611) by capturing and extracting contextual information of a tokenized input text (601).

Meanwhile, the second processing unit (720) according to one embodiment may include an embedding layer (721) that inputs external knowledge reference information (602) and outputs external knowledge embedding (612).

An embedding layer (721) according to one embodiment can output an external knowledge embedding (612) having the same size as the language model hidden state (611). For example, the language model hidden state (611) and the external knowledge embedding (612) can be expressed as matrices having the same size.

As a specific example, the language model hidden state (611) and external knowledge embedding (612) can be expressed as a matrix in which one of the rows or columns is the total number of tokens of the tokenized input text (601) and the other of the rows or columns is a preset token-by-token vector size.

FIG. 8 is an exemplary diagram illustrating a model decoder and classifier included in a natural language processing model according to one embodiment.

Referring to FIG. 8, a natural language processing device (120) according to one embodiment may input a language model hidden state (611) and external knowledge embedding (612) into a model decoder (620) to obtain a decoder hidden state (621) as an output of the model decoder (620). The model decoder (620) according to one embodiment may include a transformer decoder.

According to one embodiment, the natural language processing device (120) can concatenate the language model hidden state (611) and the external knowledge embedding (612) and input them to the model decoder (620). According to one embodiment, the natural language processing device (120) can concatenate the language model hidden state (611) and the external knowledge embedding (612), which are matrices of the same size, and input the result of the matrix concatenation to the model decoder (620). For example, the natural language processing device (120) can perform matrix concatenation by concatenating the language model hidden state (611) and the external knowledge embedding (612), which are matrices of the same size, based on rows or columns.

Meanwhile, a classifier (630) according to one embodiment may include a first classifier (810) that generates intent classification information (631) based on a decoder hidden state (621) and a second classifier (820) that generates slot tagging information (632) based on the decoder hidden state (621).

The decoder hidden state (621) according to one embodiment may include data in which meaningful features for intent classification and slot tagging are extracted or transformed from the language model hidden state (611) and external knowledge embedding (612), as described above with reference to FIG. 6.

The first classifier (810) according to one embodiment can perform at least a part of the sentence classification task. The first classifier (810) according to one embodiment can generate intent classification information (631) by classifying a purpose or intent corresponding to an input text (601) using a decoder hidden state (621).

For example, if the input text (601) is “Directions to Seoul Bank,” the first classifier (810) can classify the purpose or intention of the input text (601) as “directions” by utilizing the features of the input text (601) indicated by the decoder hidden state (621).

Additionally, the second classifier (820) according to one embodiment can perform at least a part of the sequence labeling task. The second classifier (820) according to one embodiment can generate slot tagging information (632) by labeling at least one slot for the input text (601) to an appropriate class using the decoder hidden state (621).

For example, if the input text (601) is “Guide to Seoul Bank,” the second classifier (820) can label the class “bank” to the characters that make up “Seoul Bank” in “Guide to Seoul Bank” or the tokens corresponding to those characters by using the features of the input text (601) indicated by the decoder hidden state (621).

FIG. 9 is an exemplary diagram illustrating a second classifier that generates slot tagging information based on a non-self-regression technique according to one embodiment.

The second classifier (820) can label each character that constitutes the input text (601) or a token corresponding to the character with an appropriate class. According to one embodiment, the second classifier (820) can refer to the result value for at least one surrounding token when inferring the result value for a specific token. For example, the second classifier (820) can include a classifier based on a conditional random field (CRF).

When generating slot tagging information based on a conditional random field, the second classifier (820) can sequentially infer the result value for each token, thereby referencing the result value of another token, and can relatively largely consider the linguistic context around each token, thereby obtaining a result value with relatively high accuracy.

Meanwhile, referring to FIG. 9, the second classifier (820) according to one embodiment may generate slot tagging information based on a non-autoregressive technique. The non-autoregressive technique according to one embodiment may include non-autoregressive decoding.

According to one embodiment, the second classifier (820) can perform independent inference when inferring the result value for each token without referring to the result value for other tokens. For example, the second classifier (820) can include a classifier based on non-autoregressive. When generating slot tagging information based on non-autoregressive, the second classifier (820) can infer the result value for each token independently from the result value for other tokens.

A second classifier (820) according to one embodiment can generate slot tagging information (902) by labeling at least one slot for an input text (601) with an appropriate class using a decoder hidden state (901). The decoder hidden state (901) and slot tagging information (902) illustrated in FIG. 9 can be understood as examples for the decoder hidden state (621) and slot tagging information (632) illustrated in FIG. 6, respectively.

According to one embodiment, a second classifier (820) based on non-self-regression can obtain a decoder hidden state (901) as input and generate slot tagging information (902) at a high computational speed by processing in parallel at least a portion of each of the decoder hidden states (901) corresponding to each character or token of the input text (601).

Meanwhile, the decoder hidden state (901) includes the processing result of the model decoder (620) for the external knowledge embedding (612), so that slot tagging information (902) with high accuracy can be generated compared to the conventional non-self-regression technique that does not refer to external knowledge. Through this, the natural language processing model (600) can realize high accuracy and high operation speed at the same time.

Figure 10 is an example of a method for processing natural language using external knowledge.

Referring to FIG. 10, in step 1010, a natural language processing device (120) can obtain an input text including a natural language expression.

In step 1020, the natural language processing device (120) can generate external knowledge reference information for the input text by referencing a parasitic external knowledge database.

An external knowledge database according to one embodiment may include a plurality of expression-type pairs, each of which is composed of an expression and a type for the expression. At this time, the natural language processing device (120) may generate external knowledge reference information by matching at least one of the expression-type pairs to an input text by referencing the external knowledge database.

A type for an expression according to one embodiment may include at least one type set based on a selected domain.

According to one embodiment, the natural language processing device (120) can divide the input text into a plurality of n-grams. The natural language processing device (120) can search for the plurality of n-grams in an external knowledge database, and select at least one target pair from among the expression-type pairs based on the search result. At this time, the target pair can be composed of a target expression and a target type. The natural language processing device (120) can generate external knowledge reference information by matching the target type to the target expression included in the input text.

According to one embodiment, the target pair may include a first pair and a second pair, and the first pair may be composed of a first expression and a first type, and the second pair may be composed of a second expression and a second type. In this case, the second expression may be different from the first expression and may include all characters constituting the first expression. According to one embodiment, the natural language processing device (120) may generate external knowledge reference information by matching at least one of the first type and the second type to the first expression based on a predetermined criterion.

In step 1030, the natural language processing device (120) can input input text and external knowledge reference information into a pre-learned natural language processing model.

In step 1040, the natural language processing device (120) can obtain an output of a natural language processing model including intent classification information corresponding to the input text and slot tagging information corresponding to the input text.

A natural language processing model according to one embodiment may include a model encoder, a model decoder, and a classifier.

According to one embodiment, the natural language processing device (120) can input input text and external knowledge reference information into a model encoder to obtain a language model hidden state and external knowledge embedding as outputs of the model encoder. The natural language processing device (120) can input the language model hidden state and external knowledge embedding into a model decoder to obtain a decoder hidden state as outputs of the model decoder. The natural language processing device (120) can input the decoder hidden state into a classifier to obtain an output of a natural language processing model as outputs of the classifier.

A model encoder according to one embodiment may include a first processing unit and a second processing unit. The first processing unit according to one embodiment may include a tokenization layer that tokenizes an input text and a language model layer that outputs a language model hidden state, and the second processing unit according to one embodiment may include an embedding layer that inputs external knowledge reference information and outputs external knowledge embedding.

A classifier according to one embodiment may include a first classifier that generates intent classification information based on a decoder hidden state and a second classifier that generates slot tagging information based on the decoder hidden state.

A second classifier according to one embodiment may generate slot tagging information based on a non-autoregressive technique.

Fig. 11 is a block diagram of a natural language processing device according to one embodiment. Meanwhile, the device (1100) illustrated in Fig. 11 may correspond to the natural language processing device (120) illustrated in Fig. 1.

Referring to FIG. 11, the device (1100) may include a communication module (1110), a processor (1120), and a memory (1130). Only components related to the embodiment are illustrated in the device (1100) of FIG. 11. Accordingly, a person skilled in the art will understand that other general components may be further included in addition to the components illustrated in FIG. 11.

The communication module (1110) may include at least one component that allows the device (1100) to communicate with the user terminal (110), the external knowledge database (130), and/or other external devices via wired/wireless communication. For example, the communication module (1110) may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiving unit (not shown).

Memory (1130) is a hardware that stores various data processed within the device (1100) and can store a program for processing and controlling the processor (1120).

Memory (1130) may include random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory.

The processor (1120) controls the overall operation of the device (1100). For example, the processor (1120) can control the overall operation of the communication module (1110), the memory (1130), etc. by executing programs stored in the memory (1130). The processor (1120) can control the operation of the device (1100) by executing programs stored in the memory (1130).

The processor (1120) can control at least some of the operations of the device (1100) described above with reference to FIGS. 1 to 10.

For example, the processor (1120) may control the communication module to obtain an input text including a natural language expression, generate external knowledge reference information for the input text by referencing a pre-generated external knowledge database, input the input text and the external knowledge reference information into a pre-learned natural language processing model, and obtain an output of the natural language processing model including intent classification information corresponding to the input text and slot tagging information corresponding to the input text.

Meanwhile, a specific example of the operation of the processor (1120) is the same as that described above with reference to FIGS. 1 to 10. Therefore, a specific description of the operation of the processor (1120) is omitted below.

The processor (1120) may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

In one embodiment, the device (1100) may be an electronic device having mobility. For example, the device (1100) may include a computing device mounted on a vehicle. In this case, the 'vehicle' may mean any type of transportation means that has an engine and is used to move people or objects, such as an automobile, a bus, a motorcycle, a kickboard, or a truck. For another example, the device (1100) may be implemented as a smartphone, a tablet PC, a PC, a smart TV, a personal digital assistant (PDA), a laptop, a media player, a navigation, a device equipped with a navigation, and other mobile electronic devices.

In another embodiment, the device (1100) may be a server located outside the vehicle. The server may be implemented as a computer device or multiple computer devices that communicate over a network to provide commands, codes, files, content, services, etc.

In another embodiment, at least one operation performed in the device (1100) may be performed by at least some of a mobile electronic device, an electronic device embedded in a vehicle, and a server located outside the vehicle.

Embodiments according to the present disclosure may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. At this time, the medium may include a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions, such as a ROM, a RAM, a flash memory, and the like.

Meanwhile, the computer program may be one that is specifically designed and configured for the present disclosure or one that is known and available to those skilled in the art of computer software. Examples of computer programs may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

According to one embodiment, the method according to various embodiments of the present disclosure may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play StoreTM) or directly between two user devices. In the case of online distribution, at least a part of the computer program product may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or an intermediary server.

Unless there is an explicit description of the order or the contrary description for the steps constituting the method according to the present disclosure, the steps can be performed in any suitable order. The present disclosure is not necessarily limited by the description order of the steps. The use of all examples or exemplary terms (e.g., etc.) in this disclosure is merely for the purpose of describing the present disclosure in more detail and is not intended to limit the scope of the present disclosure due to the examples or exemplary terms, unless limited by the claims. In addition, those skilled in the art will appreciate that various modifications, combinations, and changes can be configured according to design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present disclosure should not be limited to the embodiments described above, and not only the scope of the patent claims described below but also all scopes equivalent to or equivalently modified from the scope of the patent claims are included in the scope of the spirit of the present disclosure.

Claims (11)

A step of obtaining an input text containing a natural language expression;
A step of generating external knowledge reference information for the input text by matching each of at least one target expression included in the input text with a target type corresponding to the target expression by referencing a parasitic external knowledge database;
A step of inputting the above input text and the above external knowledge reference information into a pre-learned natural language processing model; and
A step of obtaining an output of the natural language processing model including intent classification information corresponding to the input text and slot tagging information corresponding to the input text;
Including,
The above generating steps are:
A step of matching at least one of the first type and the second type to the duplicated character according to a preset criterion, based on the first expression and the second expression included in the input text including duplicated characters;
Including,
The above second expression is a natural language processing method using external knowledge, which is different from the above first expression. In paragraph 1,
The above external knowledge database includes a database storing a plurality of expression-type pairs, each of which consists of an expression and a type for the expression,
The above generating steps are:
a step of selecting at least one target pair among the above expression-type pairs; and
A step of matching the target type to each of the at least one target expression based on the selected target pair;
A method comprising: In the second paragraph,
A method in which the type to be referenced within the above external knowledge database is set based on a pre-selected domain. In the second paragraph,
The above selection steps are:
A step of dividing the above input text into multiple n-grams; and
A step of searching the plurality of n-grams in the external knowledge database and selecting at least one target pair from among the expression-type pairs based on the result of the search;
A method comprising: delete In paragraph 1,
The above natural language processing model includes a model encoder, a model decoder, and a classifier.
The step of obtaining the output of the above natural language processing model is:
A step of inputting the input text and the external knowledge reference information into the model encoder to obtain a language model hidden state and an external knowledge embedding as outputs of the model encoder;
A step of inputting the above language model hidden state and the external knowledge embedding into the model decoder to obtain a decoder hidden state as an output of the model decoder; and
A step of inputting the above decoder hidden state into the above classifier and obtaining the output of the natural language processing model as the output of the above classifier;
A method comprising: In paragraph 6,
The above model encoder includes a first processing unit and a second processing unit,
The first processing unit includes a tokenization layer that tokenizes the input text and a language model layer that outputs the language model hidden state.
A method wherein the second processing unit includes an embedding layer that takes the external knowledge reference information as input and the external knowledge embedding as output. In paragraph 6,
A method, wherein the classifier comprises a first classifier that generates the intent classification information based on the decoder hidden state and a second classifier that generates the slot tagging information based on the decoder hidden state. In Article 8,
A method wherein the second classifier generates the slot tagging information based on a non-autoregressive technique. A communication module that performs communication with external devices;
memory in which at least one program is stored; and
A processor operating by executing at least one program;
Including, but not limited to,
The above processor,
Controlling the communication module to obtain an input text containing a natural language expression,
By referencing a parasitic external knowledge database, external knowledge reference information for the input text is generated by matching each of at least one target expression included in the input text with a target type corresponding to the target expression, and
The above input text and the above external knowledge reference information are input into a pre-trained natural language processing model,
Obtaining an output of the natural language processing model including intent classification information corresponding to the input text and slot tagging information corresponding to the input text,
The above creation is,
Based on the first expression and the second expression included in the input text including duplicate characters, matching at least one of the first type and the second type to the duplicate characters according to a preset criterion,
The above second expression is a natural language processing device using external knowledge, which is different from the above first expression. A computer-readable recording medium having recorded thereon a program for executing the method according to claim 1 on a computer.