JP2022116979A - Text generation device, program and text generation method - Google Patents

Abstract

A technique is provided that accepts even unanalyzed sentences as input data and generates teacher data that enables efficient learning of a large amount of expressions that are not unnatural. Kind Code: A1 A text generation device includes a language understanding unit that analyzes a natural language text to model semantic information, an annotation estimation unit that models topic information of the text, and a A sentence generation unit that generates sentences using semantic information and topic information, and a sentence that evaluates the generated sentences and extracts and outputs sentences that are similar to natural language sentences and are not unnatural as teacher data. and an evaluation unit. [Selection drawing] Fig. 1

Description

The present invention relates to a sentence generation device, a program, and a sentence generation method.

In Patent Document 1, natural language processing is used to mechanically expand a small amount of annotated text data to obtain a large amount of annotated text data that is consistent with annotation labels and is not unnatural as text. There is a disclosure of a method for extending annotated text data that can be used.

JP 2019-185551 A

The technology described in Patent Document 1 requires that text data to be input have the context in which it appears (the genre of the article in which it appears) as an annotation label in advance. Considering that natural language processing requires learning nearly infinite language expressions, it is not sufficient.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technique for accepting even unanalyzed sentences as input data and generating teaching data that enables efficient learning of a large amount of natural expressions.

The present application includes a plurality of means for solving at least part of the above problems, and examples thereof are as follows. In order to solve the above problems, a sentence generation device according to an aspect of the present invention includes a language understanding unit that analyzes sentences in a natural language and models semantic information, and an annotation estimation unit that models topic information of the sentences. and a text generation unit that generates a text using the semantic information and the topic information related to the text in the natural language, and evaluates the generated text to determine whether it is similar to the text in the natural language and is inappropriate. and a sentence evaluation unit that extracts and outputs the unnatural sentence as teacher data.

Further, in the above sentence generation device, the language understanding unit divides each of the natural language sentence and the generated sentence into tokens to obtain a sentence semantic vector of each sentence, and the sentence evaluation unit obtains the sentence semantic vector of each sentence. The similarity may be determined by comparing the sentence semantic vectors of each sentence to calculate the degree of similarity.

Further, in the text generation device described above, the annotation estimation unit divides each of the natural language text and the generated text into tokens to obtain annotation vectors of each text, and the text evaluation unit obtains an annotation vector of each text. The similarity may be determined by calculating the degree of similarity by comparing the annotation vectors of the sentences.

Further, in the above sentence generation device, the sentence evaluation unit calculates the naturalness score of the generated sentence using a sentence naturalness evaluation model that has been trained to calculate the naturalness score of the sentence when the input sentence is received. It may be calculated and determined whether or not it is unnatural.

Further, in the above sentence generation device, the sentence evaluation unit outputs a similarity score or a naturalness score of each generated sentence to the natural language sentence as a graph when outputting the training data. It can be.

Further, in the sentence generation device described above, the graph may be a two-dimensional or more graph including the degree of similarity or the naturalness score as one of its axes.

Further, in the sentence generation device described above, the teacher data may be used for learning of a trained model that performs predetermined natural language dialogue processing.

Further, a text generation program according to the present invention is a program that causes a computer to function as a text generation device, wherein a processor of the computer is provided with a language understanding step of analyzing a natural language text and modeling semantic information, An annotation estimation step of modeling topic information of the sentence; a sentence generation step of generating a sentence using the semantic information and the topic information relating to the sentence in the natural language; and evaluating the generated sentence, and a sentence evaluation step of extracting and outputting the sentence similar to the natural language sentence and not unnatural as teacher data.

Further, a text generation method according to the present invention is a text generation method using a computer as a text generation device, wherein the computer is provided with a language understanding step of analyzing a text of a natural language to model semantic information; a sentence generation step of generating a sentence using the semantic information and the topic information relating to the sentence in the natural language; and evaluating the generated sentence to evaluate the natural language and a sentence evaluation step of extracting and outputting the sentence similar to the sentence of the language and not unnatural as teacher data.

According to the present invention, it is possible to provide a technique that accepts even unanalyzed sentences as input data and generates teacher data that enables efficient learning of a large amount of expressions that are not unnatural. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

It is a figure which shows the structural example of the sentence production|generation system which concerns on embodiment of this invention. It is a figure which shows the example of a data structure stored in an input sentence memory|storage part. FIG. 4 is a diagram showing an example data structure stored in a generated sentence storage unit; It is a figure which shows the hardware structural example of a sentence production|generation apparatus. It is a figure which shows the example of the flow of sentence amplification processing. FIG. 10 is a diagram showing an example of the flow of generated sentence evaluation processing; It is a figure which shows the input-output relationship of a sentence naturalness evaluation model. FIG. 10 is a diagram showing an example of an input/output screen;

One embodiment according to the present invention will be described below with reference to the drawings. In principle, the same members are denoted by the same reference numerals in all drawings for describing the embodiments, and repeated description thereof will be omitted. In addition, in the following embodiments, the constituent elements (including element steps, etc.) are not necessarily essential, unless otherwise specified or clearly considered essential in principle. Needless to say. In addition, when saying "consisting of A", "consisting of A", "having A", or "including A", other elements are excluded unless it is explicitly stated that only those elements are included. It goes without saying that it is not something to do. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc. of components, etc., unless otherwise specified or in principle clearly considered otherwise, the shape is substantially the same. It shall include things that are similar or similar to, etc.

In the following description, "input/output unit 140" and "display unit 150" may be one or more interface devices. The one or more interface devices may be at least one of the following:
- One or more I/O (Input/Output) interface devices. The I/O interface device is an interface device for at least one of the I/O device and the remote operation management device 10 . The I/O interface device for the operation management device 10 may be a communication interface device. The at least one I/O device may be any of a user interface device, eg, an input device such as a keyboard and pointing device, and an output device such as a display device.
- One or more communication interface devices. The one or more communication interface devices may be one or more of the same type of communication interface device (e.g., one or more NICs (Network Interface Cards)) or two or more different types of communication interface devices (e.g., NIC and It may be an HBA (Host Bus Adapter).

Also, in the following description, "memory" refers to one or more memory devices, which are examples of one or more storage devices, and may typically be a main memory device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

Also, in the following description, "storage unit" or "storage" may be memory or both of memory and permanent storage. In particular, the permanent storage device may be, for example, a HDD (Hard Disk Drive), an SSD (Solid State Drive), an NVME (Non-Volatile Memory Express) drive, or an SCM (Storage Class Memory).

Also, in the following description, a "processing unit" or "processor" may be one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may be another type of processor device such as a GPU (Graphics Processing Unit). At least one processor device may be single-core or multi-core. At least one processor device may be a processor core. At least one processor device is a circuit (for example, FPGA (Field-Programmable Gate Array), CPLD (Complex Programmable Logic Device) or ASIC (Application A processor device in a broad sense such as Specific Integrated Circuit) may also be used.

In addition, in the following description, the function may be described using the expression “yyy part”, but the function may be realized by executing one or more computer programs by a processor, or may be realized by executing one or more computer programs. It may be realized by the above hardware circuits (for example, FPGA or ASIC), or may be realized by a combination thereof. When a function is realized by executing a program by a processor, the defined processing is performed using a storage device and/or an interface device as appropriate, so the function may be at least part of the processor. good. A process described with a function as the subject may be a process performed by a processor or a device having the processor. Programs may be installed from program sources. The program source may be, for example, a program distribution computer or a computer-readable recording medium (for example, a non-temporary recording medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

Further, in the following explanation, the processing may be explained with the subject of "program" or "processing unit", but the processing explained with the program as the subject may be the processing performed by the processor or the device having the processor. . Also, two or more programs may be implemented as one program, and one program may be implemented as two or more programs.

In the following description, the expression "xxx table" may be used to describe information that provides an output for an input. It may be a learning model represented by a neural network, a genetic algorithm, or a random forest that generates . Therefore, the "xxx table" can be called "xxx information". Also, in the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table. may

Also, in the following description, the "text generation system" may be a system composed of one or more physical computers, or a system ( For example, it may be a cloud computing system). "Displaying" the display information by the text generation system may be displaying the display information on a display device of the computer, or the computer may transmit the display information to the display computer. (in the latter case the display information is displayed by a display computer).

In recent years, trained models that have learned the characteristics of data deeply using neural networks are used for recognition and analysis in various fields. On the other hand, in the target field of application, for example, NLP (Natural Language Processing), if sufficient teacher data (abnormal and normal) for learning cannot be prepared, it is difficult to achieve a practical level of recognition accuracy. However, since language cannot be converted directly to continuous value data, there is no method for automatically generating sentence data like image recognition, and a huge amount of time and labor costs are required to create training data.

For example, chatbots for chatting are realized by using the appearance probability distribution between words. However, sentences generated by chatbots are often of few types and have grammatical problems. In the present invention, by performing language understanding and annotation estimation on the input natural language sentence, according to the meaning and annotation ("annotation" according to the present invention in this specification means "topic"), It becomes possible to automatically generate more natural sentences for teachers.

In addition, the present invention is not limited to this, and automatically eliminates inappropriate teaching data from generated sentences to output high-quality teaching data. Specifically, the quality is measured by evaluating semantic similarity, annotation similarity, and naturalness. Eliminate unnatural sentences from training data.

In conventional teaching data creation, methods have been proposed to generate sentences by analyzing sentences and replacing words with a rule base or pattern matching in order to create the teacher data necessary for learning language processing. It should be added that a huge amount of time and labor is consumed in creating rules and patterns.

According to the text generating apparatus of the present invention, it is possible not only to automatically create a large amount of training data of the same meaning and annotation as the input text whose meaning and annotation have been analyzed, but also to automatically generate naturalness ( By automatically analyzing and evaluating grammatical correctness) and similarities in meaning and annotation, it is possible to create high-quality training data and contribute to improving the learning accuracy of artificial intelligence (trained models). .

FIG. 1 is a diagram showing a configuration example of a sentence generation system according to an embodiment of the present invention. The text generation system 1 includes a text generation device 100, which is a server device that operates in response to instructions for generating training data from the operation management device 10 via the network 50, and provides training data from the text generation device 100 via the network 50. and a learning processing device 200 for receiving. The learning processing device 200 uses teacher data to perform learning processing on a trained model that performs interactive processing.

In addition, the interactive processing to be learned by the learning processing device 200 is not limited to chatbots. It may also be interactive processing performed in various devices including, etc., processing of a summary tool that creates a summary of a large amount of sentences, and the like.

The network 50 is, for example, a wireless communication network that uses frequency bands such as 3.5 GHz (gigahertz) band, 4.5 GHz band, and 28 GHz band called LTE (Long Term Evolution) or 5G. The sentence generation device 100 and the learning processing device 200 are connected so as to be communicable with each other.

In addition, the learning processing device 200, the sentence generation device 100, and the operation management device 10 are connected via a wired network (LAN (Local Area Network), etc.) in consideration of overall communication quality, security, maintenance cost, etc. may be communicably connected to each other.

The operation management device 10 is an information processing device used to receive a text generation instruction to the text generation device 100 from a user. The operation management device 10 includes a processing unit 11 and a communication unit 13 , and the processing unit 11 includes a browser unit 12 .

The browser unit 12 instructs the sentence generation device 100 to generate teacher data. For example, the browser unit 12 has a function as a web browser and a screen interface as input assistance, accesses the web page of the sentence generation device 100 via the communication unit 13, and instructs the sentence generation device 100 to generate sentences and input information. Send text information.

The communication unit 13 communicates with other devices via the network 50 .

The text generation device 100 is an information processing device that receives input text information and training data generation instructions from the operation management device 10 and generates texts to create training data. Also, the sentence generation device 100 transmits the created teacher data to the learning processing device 200 . In addition, the sentence generation device 100 provides the operation management device 10 with an input/output screen relating to instruction data generation instruction as a web page.

For example, the text generation device 100 generates screen information having a text input area for receiving an input text from the operation management device 10 and displays it on the browser unit 12, or creates an output area showing the created teacher data in graph form. The screen information to be provided is generated and displayed on the browser unit 12 .

The sentence generation device 100 includes a storage unit 110 , a processing unit 120 , a communication unit 130 , an input/output unit 140 and a display unit 150 . The storage unit 110 includes an input sentence storage unit 111 and a generated sentence storage unit 112 .

FIG. 2 is a diagram showing an example data structure stored in an input sentence storage unit. The input sentence storage unit 111 contains an input sentence identifier 111a, an input sentence 111b, a sentence semantic vector 111c, and an annotation vector 111d. The input sentence identifier 111a is information for identifying an input sentence. The input sentence 111b is a natural language sentence used to create teacher data to be learned. The sentence semantic vector 111c is information indicating the characteristics of a sentence/word digitized using Word2Vec, Doc2Vec, or the like.

The annotation vector 111d is the degree of importance (weight) calculated and assigned by a predetermined method such as TF-IDF (Term Frequency - Inverse Document Frequency) for each token included in the text. is a set.

FIG. 3 is a diagram showing an example data structure stored in the generated sentence storage unit. The generated sentence storage unit 112 stores a generated sentence identifier 112a, a generated original sentence identifier 112b, a generated sentence 112c, a generated sentence semantic vector 112d, a semantic similarity 112e, a generated sentence annotation vector 112f, and an annotation similarity 112g. and are included.

The generated sentence identifier 112a is information for identifying a generated sentence. The generation source sentence identifier 112b is information for identifying the natural language sentence used for generating the generated sentence. The generated sentence 112c is a natural language sentence generated as teacher data to be learned. The generated sentence semantic vector 112d is information indicating the sentence/word characteristics digitized using Word2Vec, Doc2Vec, or the like. The semantic similarity 112e is information indicating the degree of similarity in meaning between the generated sentence 112c and the source sentence identified by the source sentence identifier 112b.

The generated sentence annotation vector 112f is a set of degrees of importance (weights) calculated and assigned by a predetermined method such as TF-IDF for each token included in the generated sentence. The annotation similarity 112g is information indicating the degree of annotation similarity between the generated sentence 112c and the source sentence identified by the source sentence identifier 112b.

Returning to the description of FIG. The processing unit 120 includes a language understanding unit 121 , an annotation estimation unit 122 , a text generation unit 123 and a text evaluation unit 124 .

The language understanding unit 121 analyzes natural language sentences to model semantic information. Specifically, the language understanding unit 121 divides each of the natural language sentences and the generated sentences into tokens and acquires the sentence semantic vector of each sentence. The sentence semantic vector is, as described above, information indicating the characteristics of a sentence/word digitized using Word2Vec, Doc2Vec, or the like. In this embodiment, the language understanding unit 121 divides sentences into tokens by a predetermined algorithm such as morphological analysis. The language understanding unit 121 then vectorizes each token. Note that the language understanding unit 121 may also vectorize the entire sentence.

The annotation estimation unit 122 models natural language topic information. Specifically, the annotation estimating unit 122 divides each of the natural language sentences and the generated sentences into tokens and acquires the annotation vector of each sentence. The generated text annotation vector is, as described above, a set of degrees of importance (weights) calculated and given to each token by a predetermined method such as TF-IDF. In this embodiment, the annotation estimation unit 122 divides the text into tokens by a predetermined algorithm such as morphological analysis. The annotation estimation unit 122 then vectorizes each token. Note that the annotation estimation unit 122 may also vectorize the entire sentence.

The sentence generation unit 123 generates sentences using semantic information (sentence semantic vector) and topic information (annotation vector) related to sentences in natural language. Specifically, the sentence generation unit 123 fixes a token having a high degree of importance (weight) in the topic information acquired in the natural language sentence (for example, a token with a top 15th percentile weight), and Masking (replacement) tokens that are not fixed. The sentence generation unit 123 generates multiple patterns of sentences by variously predictively transforming some or all of the masked tokens into similar expressions using a predetermined learned model.

The sentence evaluation unit 124 evaluates the generated sentences, extracts and outputs sentences that are similar to natural language sentences and that are not unnatural as teacher data. For example, the sentence evaluation unit 124 compares the sentence semantic vectors of each sentence to calculate the degree of similarity and determine the similarity. Further, for example, the text evaluation unit 124 compares the annotation vectors of each text to calculate the degree of similarity and determine the similarity. Further, for example, when receiving an input sentence, the sentence evaluation unit 124 uses a sentence naturalness evaluation model that has been trained to calculate the naturalness score of the sentence, and calculates the naturalness score of the generated sentence to check whether it is unnatural. determine whether or not

When determining whether or not it is unnatural using the naturalness score, the sentence evaluation unit 124, for example, selects sentences with high naturalness scores among the generated sentences (eg, the top 10 percentile of naturalness scores or a threshold 0.9 or more) may be determined as not unnatural. Alternatively, the naturalness score threshold may be changed to various values such as 0.75 and 0.8 depending on the characteristics of the system where the teacher data is used (such as for chat). This is because there are cases where the answer is given priority even if it is somewhat unnatural. Further, when outputting the training data, the sentence evaluation unit 124 can also output the degree of similarity or the naturalness score of each generated sentence to the natural language sentence as a graph.

The processing unit 120 also has a function as a web server that provides a predetermined web page to web browsers such as the browser unit 12 . The processing unit 120 receives an instruction to create training data and an instruction to delete sentences to be excluded from training data from the operation management apparatus 10 via a Web page. Then, the processing unit 120 transfers control to the language understanding unit 121, the annotation estimation unit 122, the text generation unit 123, and the text evaluation unit 124 according to the received instructions. The processing unit 120 also transmits the generated sentence (teaching data) to the learning processing device 200 via the communication unit 130 .

The communication unit 130 communicates with the operation management device 10 and the learning processing device 200 via the network 50 .

The learning processing device 200 is an information processing device that reads a large amount of teacher data in a specific field, performs deep learning, and realizes dialogue processing using natural language. In the present invention, the purpose of interactive processing is not particularly limited.

The learning processing device 200 includes a storage unit 210 , a processing unit 220 and a communication unit 230 . The storage unit 210 stores a teacher data storage unit 211 that stores teacher data. The processing unit 220 includes a learning execution unit 221 and a dialogue processing unit 222 . The learning execution unit 221 causes the dialog model used by the dialog processing unit 222 to learn the teacher data in the teacher data storage unit 211 . The dialogue processing unit 222 uses a trained dialogue model that has been deep-learned using the training data stored in the training data storage unit 211 to receive natural language input sentences in the form of texts or voices, and generate texts for responses. Alternatively, dialogue is performed by outputting sentences in natural language by voice. The communication unit 230 communicates with the sentence generation device 100 via the network 50 .

FIG. 4 is a diagram showing a hardware configuration example of the text generation device. The sentence generation device 100 is an information processing device such as a so-called personal computer or a server device, and transmits and receives information to and from other devices via the network 50 . Note that the sentence generation device 100 is not limited to a personal computer, and may be other devices such as a smart phone or a tablet device.

The sentence generation device 100 includes a processor 101 such as a CPU (Central Processing Unit), a memory 102 such as a RAM (Random Access Memory), a storage 103 such as a hard disk or an SSD (Solid State Drive), and a SIM card for LTE communication. etc., or a communication device 107 such as a NIC (Network Interface Card), and a bus connecting them.

The communication device 107 is a wireless communication device that performs wireless communication via an antenna. The communication device 107 performs wireless communication with other devices connected to the network 50 . Alternatively, in this embodiment, the communication device 107 may be a wired communication device that performs wired communication with another device via a network cable.

The language understanding unit 121, the annotation estimating unit 122, the text generating unit 123, and the text evaluating unit 124 described above are implemented by a program that causes the processor 101 to perform processing. This program is stored in memory 102 and executed by processor 101 upon execution.

Also, the input sentence storage unit 111 and the generated sentence storage unit 112 stored in the storage unit 110 are realized by the memory 102 and the storage 103 .

Also, the communication unit 130 is implemented by the communication device 107 . The input/output unit 140 and display unit 150 may be either an I/O interface device or a communication interface device as described above.

The above is an example of the hardware configuration of the sentence generation device 100 according to the present embodiment. However, the configuration is not limited to this, and may be configured using other hardware. Note that the sentence generation device 100 has well-known elements such as an OS, middleware, and applications (not shown).

[Description of Operation] Next, the operation of the text generating apparatus 100 according to this embodiment will be described.

FIG. 5 is a diagram showing an example flow of text amplification processing. The sentence amplification process is started when a teaching data generation instruction is received from the operation management device 10 .

First, the language understanding unit 121 reads one or more input sentences (step S001). The input sentence to be read is stored in the input sentence storage unit 111 . However, the text is not limited to this, and may be text received via the I/O interface or communication interface of the input/output unit 140 .

Then, the language understanding unit 121 performs the processing of steps S003 to S007 for each input sentence (steps S002 and S008).

The language understanding unit 121 divides the input sentence into tokens (step S003). Specifically, the language understanding unit 121 divides the input text into words separated by spaces into tokens according to the target language of the input text, or divides the words into predetermined tokens by performing morphological analysis or the like. This segmentation process may use other existing techniques. For example, if the input text is "The Nikkei Stock Average fell sharply on the Tokyo stock market on the 2nd. It was trending upward in the first half, but after it turned out that President Trump was infected with the new coronavirus in the second half, it fell into negative territory. On the other hand, there was a scene where it fell below 23,000 yen at one point.” In the case of “2 days / no / Tokyo stock market / at / Nikkei Stock Average / is / sharply declined /. / , / after / in / U.S. President Trump / was / infected / with / new coronavirus / / turned into negative territory /, / temporary / 20,000 3000 yen//There was a scene where I interrupted/.”

The language understanding unit 121 then calculates a sentence semantic vector of the input sentence (step S004). Specifically, as described above, the language understanding unit 121 uses Word2Vec, Doc2Vec, etc. to specify numerical information according to the characteristics of sentences/words, and vectorizes each token and the entire sentence.

The annotation estimation unit 122 then estimates the annotation vector of the input sentence (step S005). Specifically, as described above, the annotation estimating unit 122 uses IT-IDF or the like to identify information digitized according to the importance of tokens, and vectorizes each token and the entire sentence. For example, the annotation estimation unit 122 estimates a set of importance of each token as a multidimensional array 301 .

Then, the text generation unit 123 uses the annotation vector to mask tokens other than important tokens (step S006). Specifically, the sentence generation unit 123 fixes a token having a high degree of importance (weight) in the topic information acquired in the natural language sentence (for example, a token with a top 15th percentile weight), and Masking (replacement) tokens that are not fixed. For example, in masked sentence 302, significant tokens are kept intact, and non-critical tokens are labeled [MSK], indicating that tokens labeled [MSK] are masked tokens.

Then, the sentence generator 123 replaces the tokens other than the important tokens with similar words and amplifies them into one or more sentences (step S007). Specifically, the sentence generation unit 123 predictively converts part or all of the masked tokens into similar expressions using a predetermined trained model (not shown), thereby creating multiple patterns of sentences 303 . to generate For example, a sentence with a slightly different but similar meaning to the input sentence, such as "9:30 am Tokyo stocks fell sharply in the Nikkei Stock Average, resulting in 23,000...", is generated and amplified. The amplified generated sentence is stored in the generated sentence storage unit 112 .

The above is an example of the flow of the text amplification process. According to the sentence amplification process, it is possible to receive even unanalyzed sentences as input data and amplify sentences for generating a large amount of training data while maintaining a predetermined quality.

FIG. 6 is a diagram showing an example of the flow of generated sentence evaluation processing. The generated sentence evaluation process is started following the sentence amplification process. Alternatively, the generated sentence evaluation process may be started when one or more generated sentences are specified and an instruction is received from the operation management device 10 .

First, the language understanding unit 121 reads one or more generated sentences (step S101). The generated sentence to be read is stored in the generated sentence storage unit 112 . However, the text is not limited to this, and may be text received via the I/O interface or communication interface of the input/output unit 140 .

Then, the language understanding unit 121 performs the processing of steps S103 to S107 for each generated sentence (steps S102 and S108).

The language understanding unit 121 divides the generated sentence into tokens (step S103). Specifically, the language understanding unit 121 divides the generated text into words separated by spaces depending on the target language of the generated text to obtain tokens, or performs morphological analysis or the like to divide the words into predetermined tokens. This segmentation process may use other existing techniques.

The language understanding unit 121 then calculates a sentence semantic vector of the generated sentence (step S104). Specifically, as described above, the language understanding unit 121 uses Word2Vec, Doc2Vec, etc. to specify numerical information according to the characteristics of sentences/words, and vectorizes each token and the entire sentence.

The annotation estimation unit 122 then estimates the annotation vector of the generated sentence (step S105). Specifically, as described above, the annotation estimating unit 122 uses IT-IDF or the like to identify information digitized according to the importance of tokens, and vectorizes each token and the entire sentence.

Then, the text evaluation unit 124 calculates the degree of semantic similarity using the semantic vectors of the generated text and the generation source text (step S106). This semantic similarity can be calculated by using an existing technique for calculating the distance between semantic vectors.

Then, the text evaluation unit 124 calculates annotation similarity using the annotation vectors of the generated text and the generation source text (step S107). This annotation similarity can be calculated using an existing technique for calculating the distance between annotation vectors and the like.

Then, the sentence evaluation unit 124 deletes the generated sentences whose semantic similarity or annotation similarity is less than a predetermined level (step S109). It is assumed that the text evaluation unit 124 deletes a generated document in which either the degree of semantic similarity or the degree of annotation similarity is less than a predetermined value. A sentence that is the same as the sentence or has a different variation value may also be deleted. This is because duplicate sentences are likely to be wasted.

Then, the sentence evaluation unit 124 evaluates the naturalness of the remaining generated sentences, and deletes the generated sentences with a naturalness score less than a predetermined score (step S110). Specifically, the text evaluation unit 124 uses a text naturalness evaluation model that has been trained to calculate the naturalness score of the text for the generated text that has not been deleted in step S109, and evaluates the naturalness of the generated text. The naturalness score is calculated, and generated sentences with naturalness scores below a predetermined value (eg, below the bottom 90th percentile of naturalness scores, or below a threshold of 0.9) are deleted.

Then, the processing unit 120 transmits the remaining generated sentence as teacher data to the learning processing device 200 via the communication unit 130 (step S111).

The above is an example of the generated sentence evaluation process flow. According to the generated sentence evaluation process, it is possible to generate teacher data that can efficiently learn a large amount of sentences that are similar to the original sentence and are not unnatural.

FIG. 7 is a diagram showing the input/output relationship of the sentence naturalness evaluation model. In the generated sentence naturalness evaluation process performed in step S110 of the generated sentence evaluation process, the sentence naturalness evaluation model 402, which is a trained neural network or the like, is used to receive the input sentence 401 and calculate the naturalness score of the input sentence. 403 and the naturalness score 404 for each word are output. Such sentence naturalness evaluation model 402 can be used to generate unnatural sentences generated through a process of processing natural sentences (sentences in natural language such as newspaper articles) into unnatural expressions, and unprocessed sentences. It has been learned using sentences that remain natural.

FIG. 8 is a diagram showing an example of an input/output screen. The input/output screen 500 includes an input text reception area 501, an input text input instruction reception area 502, a token segmentation result display area 503, a text generation instruction reception area 504, a generated sentence display area 505, and an input text reception area 501. It includes an instruction receiving area 506, a graph drawing area 507 for plotting the generated sentence position 508, and an output instruction receiving area 510 for receiving input to be determined as teacher data.

In the graph drawing area 507, the position 508 of the generated sentence is plotted in a three-dimensional space in which the semantic similarity, the annotation similarity, and the naturalness score for each generated sentence with respect to the source sentence are orthogonal to each other, and the score 509 is plotted. Is displayed. In addition, the output instruction receiving area 510 receives an instruction input to set the generated sentence checked in the instruction receiving area 506 for inclusion in the teacher data as the teacher data.

Note that the graph drawing area 507 is not limited to this, and may draw a two or more-dimensional graph including similarity or naturalness score as one of its axes.

Although the text generation system 1 according to the embodiment has been specifically described above, the present invention is not limited to this embodiment, and it goes without saying that various modifications can be made without departing from the gist of the invention. do not have. For example, the operation management device 10 may be provided in the sentence generation device 100 or the learning processing device 200 instead of being an independent device.

Also, the learning processing device 200 may be incorporated as a part of the sentence generation device 100 instead of being an independent device.

Alternatively, the network 50 may also be realized by various wireless communication networks such as a mobile phone communication network or a local 5G network locally applied to the network.

In addition, in the above-described embodiment, the configuration is described in detail in order to explain the present invention in an easy-to-understand manner.

Further, each of the configurations, functions, processing units, etc. described above may be realized by hardware, for example, by designing a part or all of them using an integrated circuit. Further, the control lines and information lines indicate those considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. In practice, it may be considered that almost all configurations are interconnected.

Further, each configuration, function, processing unit, and the like described above may be implemented in a distributed system by executing a part or all of them on another device, for example, and performing integrated processing via a network.

Also, the technical elements of the above-described embodiments may be applied singly, or may be applied after being divided into a plurality of parts such as program parts and hardware parts.

The present invention has been described above with a focus on the embodiments.

1: sentence generation system, 10: operation management device, 11: processing unit, 12: browser unit, 13: communication unit, 50: network, 100: sentence generation device, 110: storage unit, 111: input sentence storage unit, 112 : generated sentence storage unit, 120: processing unit, 121: language understanding unit, 122: annotation estimation unit, 123: sentence generation unit, 124: sentence evaluation unit, 130: communication unit, 140: input/output unit, 150: display unit 200: learning processing device; 210: storage unit; 211: teacher data storage unit; 220: processing unit;

Claims (9)

a language understanding unit that analyzes natural language sentences and models semantic information;
an annotation estimation unit that models topic information of the sentence;
a text generation unit that generates a text using the semantic information and the topic information related to the text in the natural language;
a sentence evaluation unit that evaluates the generated sentences and extracts and outputs the sentences that are similar to the natural language sentences and that are not unnatural as training data;
A text generation device comprising: The text generation device according to claim 1,
The language understanding unit obtains a sentence semantic vector of each sentence by dividing each of the natural language sentence and the generated sentence into tokens,
The sentence evaluation unit compares the sentence semantic vectors of the sentences to calculate similarity and determine similarity.
A text generation device characterized by: The text generation device according to claim 1 or 2,
The annotation estimating unit divides each of the natural language sentences and the generated sentences into tokens to acquire annotation vectors of each sentence,
The text evaluation unit performs comparison between the annotation vectors of each of the texts to calculate similarity and determine similarity.
A text generation device characterized by: The text generation device according to any one of claims 1 to 3,
When an input sentence is received, the sentence evaluation unit calculates the naturalness score of the generated sentence using a sentence naturalness evaluation model that has been trained to calculate the naturalness score of the sentence, and determines whether or not it is unnatural. determine the
A text generation device characterized by: The text generation device according to any one of claims 1 to 4,
When outputting the training data, the sentence evaluation unit outputs a similarity score or a naturalness score of each generated sentence to the natural language sentence as a graph.
A text generation device characterized by: The text generation device according to claim 5,
The graph is a two-dimensional or more graph including the similarity or the naturalness score as one of the axes,
A text generation device characterized by: The text generation device according to any one of claims 1 to 6,
using the teacher data for learning a trained model that performs predetermined natural language dialogue processing;
A text generation device characterized by: A program that causes a computer to function as a sentence generation device,
to the processor of said computer;
a language understanding step that analyzes natural language sentences to model semantic information;
an annotation estimation step of modeling topic information of the sentence;
a text generation step of generating a text using the semantic information and the topic information relating to the text in the natural language;
a sentence evaluation step of evaluating the generated sentences and extracting and outputting the sentences similar to the natural language sentences and not unnatural as training data;
A sentence generation program characterized by executing A text generation method using a computer as a text generation device,
on said computer;
a language understanding step that analyzes natural language sentences to model semantic information;
an annotation estimation step of modeling topic information of the sentence;
a text generation step of generating a text using the semantic information and the topic information relating to the text in the natural language;
a sentence evaluation step of evaluating the generated sentences and extracting and outputting the sentences similar to the natural language sentences and not unnatural as training data;
A sentence generation method characterized by performing

Images