JP2018055670A - Similar sentence generation method, similar sentence generation program, similar sentence generation apparatus, and similar sentence generation system - Google Patents

Abstract

The present invention provides a similar sentence generation method capable of reducing the search cost for a database of language models and identifying a similar sentence with high accuracy. A similar sentence generation method inputs a first sentence, extracts one or more second phrases having the same meaning as the first phrase from a first database among a plurality of phrases constituting the first sentence, An N-gram value is calculated based on a context-dependent value corresponding to one or more second words based on the second database, and the first word is replaced with one or more second words in the first sentence. In the second sentence, one or more consecutive third phrases including the number of second phrases corresponding to the N-gram value are extracted, and the appearance frequency in the third database is calculated for the one or more third phrases, It is determined whether the calculated appearance frequency is equal to or higher than the threshold, and when it is determined that the calculated appearance frequency is equal to or higher than the threshold, one or more second sentences are adopted as similar sentences of the first sentence, Output to the device. [Selection] Figure 5

Description

  The present disclosure relates to a similar sentence generation method for generating a similar sentence from an original sentence, a similar sentence generation program, a similar sentence generation apparatus, and a similar sentence generation system including the similar sentence generation apparatus.

  In recent years, machine translation that translates sentences in a first language into sentences in a second language different from the first language has been researched and developed. To improve the performance of such machine translation, a large number of example sentences that can be used for translation are studied. A bilingual corpus that collects For this reason, one or a plurality of similar sentences (paraphrase sentences) similar to the original sentence are generated from one original sentence.

  For example, Patent Document 1 discloses a unified language conversion process in which an evaluation value is calculated using an evaluation function and an expression having the highest evaluation value is selected in order to determine whether or not the sentence is deformed in a predetermined pattern. A system is disclosed.

  Patent Document 2 discloses a natural language processing method in which points relating to activity are set as morphemes, the points are increased or decreased, and information is extracted from text based on the increased or decreased points.

  Patent Document 3 generates a new post-paraphrase example based on the pre-paraphrase example and post-paraphrase example specified by the user, and outputs a paraphrase sentence created by applying the difference to the analyzed sentence. A document processing apparatus is disclosed.

Japanese Patent No. 3932350 JP 2005-339043 A Japanese Patent No. 5060539

  However, in order to improve the performance of machine translation, it is preferable that there are many example sentences that can be used for translation. Further generation of similar sentences that can be used as example sentences requires further improvement.

  The present disclosure solves the above-described conventional problems, and can reduce the search cost for a database of language models and can identify similar sentences with high accuracy, a similar sentence generation method, a similar sentence generation program, and a similar sentence An object is to provide a generation device and a similar sentence generation system.

  A method according to an aspect of the present disclosure is a method of generating a similar sentence from an original sentence, and the first sentence is input, and one or more having the same meaning as the first phrase among a plurality of phrases constituting the first sentence Are extracted from the first database, the first database associates the phrase with a synonym of the phrase included in the first database, and the one or more second terms obtained based on the second database An N-gram value is calculated based on a context-dependent value corresponding to a phrase, and the second database associates the phrase with the context-dependent value corresponding to the phrase included in the second database, and the context-dependent value Indicates the degree to which the meaning of the phrase included in the second database depends on the context, and the one or more second sentences in which the first phrase is replaced with the one or more second phrases in the first sentence In the above -Extracting one or more consecutive third words / phrases including a number of the second words / phrases corresponding to a gram value, calculating an appearance frequency in a third database for the one or more third words / phrases, and Associates a phrase with the appearance frequency of the phrase included in the third database in the third database, determines whether the calculated appearance frequency is greater than or equal to a threshold, and the calculated occurrence frequency is greater than or equal to the threshold If it is determined that there is, the one or more second sentences are adopted as similar sentences of the first sentence and output to an external device.

  According to the present disclosure, it is possible to reduce the search cost for the language model database and to identify similar sentences with high accuracy.

It is a block diagram which shows an example of a structure of the similar sentence production | generation apparatus in Embodiment 1 of this indication. It is a figure which shows an example of a data structure of the replacement candidate dictionary shown in FIG. It is a figure which shows an example of a data structure of the context dependence rate dictionary shown in FIG. It is a figure which shows an example of a data structure of the language model database shown in FIG. It is a flowchart which shows an example of the similar sentence production | generation process by the similar sentence production | generation apparatus shown in FIG. It is a block diagram which shows an example of a structure of the similar sentence production | generation system in Embodiment 2 of this indication. It is a flowchart which shows an example of the similar sentence production | generation process containing the feedback data update process of the similar sentence production | generation system shown in FIG.

(Knowledge that became the basis of this disclosure)
As described above, it is preferable to increase the number of example sentences that can be used for translation for improving the performance of machine translation, and it is desired to automatically expand the amount of sentences based on a small amount of parallel corpus by generating similar sentences using phrase replacement. ing. When generating similar sentences using phrase replacement, there are cases where the success or failure of replacement depends on the context in selecting similar candidate sentences including expressions (phrases) to be replaced.

  For this reason, we want to enable learning and reflection of cases while considering context dependency by dynamic selection of replacement rules based on the language model, but how to select similar candidate sentences efficiently It is important to select candidate sentences.

  For example, when expanding a bilingual corpus by replacement (paraphrase) and generating similar candidate sentences, the paraphrase rule is “I can't speak” (1) “I can't speak”, (2) “I can't speak”, (3) If you replace it with any of the words "I can't speak English" and apply the above paraphrase rule to "I can't speak English", "I can't speak English", and "English can't speak" Three similar candidate sentences are generated.

  In this case, from the context, “I can't speak English” and “I can't speak English” can be adopted as similar sentences, but “English is a secret” is not an appropriate expression for Japanese, so it is adopted as a similar sentence. It cannot be done and will be rejected. In this way, even if the same paraphrase rule is applied, there are cases where a similar candidate sentence can be adopted as a similar sentence and a case where it cannot be adopted depending on the context.

  Conventional methods for discriminating adopted sentences that can be adopted as similar sentences and reject sentences that cannot be adopted are similarities in a distributed expression model using word vectors or sentence vectors, or language models (for example, N-gram language model). Judgment was made based on the appearance frequency of Specifically, by increasing the identification target area (search range) of the language model (for example, by increasing N of N-gram), it is determined whether or not it exists as an expression, and a paraphrase rule that depends on the context (Replacement rules) have been adopted and rejected.

  In addition, evaluation was performed by modeling the fluency of sentences using a language model. For example, if the language model is an N-gram language model, the score of translations and phrases using expressions included more in the database of the N-gram language model is increased, and the score of those that are not included is decreased. There is a technique that said. By applying this method, the score of similar candidate sentences is calculated, and “good sentences” (adopted sentences that can be adopted as similar sentences) or “bad sentences” (rejected sentences that cannot be adopted as similar sentences) are identified by threshold processing. Was.

  However, if the identification target area is enlarged, the data amount and calculation amount increase, and the data distribution becomes sparse. Therefore, in order to search all replacement candidates from the large identification target region, the data amount and calculation amount increase. To do. For example, in the case of 2-gram, there are about 80 million entries, but in case of 5-gram, there are about 800 million entries. If N of N-gram is increased, the amount of data and the calculation amount will increase dramatically. .

  In order to solve the above problem, in the present disclosure, for example, a context-dependent value storage unit that stores a plurality of replacement-candidate character strings and context-dependent values that indicate the degree of dependency of the replacement-candidate character strings on the context is stored. Whether to refer to a language model that includes words before and after the replacement candidate character string, depending on whether or not the replacement quality varies depending on the context in a similar sentence that varies depending on the context Decide whether or not.

  In other words, by determining the search range (identification target region) for the language model database according to the context-dependent value, and searching the language model database using the determined search range, replacement that is regarded as having a high context-dependent value Only candidate character strings are identified in a larger search area, and replacement candidate character strings having a low context-dependent value are identified in a smaller search area to balance search cost and identification accuracy.

  Further, in the conventional similar sentence generation method, distributed expressions and expressions that are not included in the language model cannot be identified in the first place, and are rejected. For example, if there is no sentence including the phrase “it is secret” in the training data, there is a problem that similar candidate sentences including “it is secret” cannot be identified and rejected.

  In order to solve the above problem, in the present disclosure, for example, when a replacement candidate character string having context dependency is input by an input from the outside (for example, feedback from a user or a predetermined device), a language model Update the database and the context-dependent value storage. In addition, when a new sentence expression is input, the context-dependent value of the word in the context-dependent value storage unit is changed according to the expression, and an N-gram or the like including the new sentence expression is partially constructed. Reflect new sentence expressions in the language model. In this way, by adding correct data, the appearance frequency of the language model including words before and after the replacement character string is adjusted, and the context-dependent value storage unit itself is also updated according to the external input.

  As described above, the identification accuracy can be improved by feeding back external knowledge and new knowledge and updating the language model database and the like. As a result, low-cost and accurate similar candidate sentences are identified, and high-efficiency and autonomous similar candidate sentences that can be updated and dealt with even in expressions that do not exist in the N-gram model database are identified. It can be carried out.

  Based on the above findings, the inventors of the present application have completed the present disclosure as a result of intensive studies on how to generate a similar sentence from the original sentence.

  A method according to an aspect of the present disclosure is a method of generating a similar sentence from an original sentence, and inputs a first sentence, and has a same meaning as the first phrase among a plurality of phrases constituting the first sentence. The second phrase is extracted from the first database, the first database associates the phrase with a synonym of the phrase included in the first database, and the one or more first phrases obtained on the basis of the second database. An N-gram value is calculated based on a context-dependent value corresponding to two phrases, and the second database associates the phrase with the context-dependent value corresponding to the phrase included in the second database, and the context-dependent The value indicates the degree to which the meaning of the word or phrase included in the second database depends on the context, and the one or more second words in which the first word or phrase is replaced with the one or more second words or phrases in the first sentence. In the sentence, -Extracting one or more consecutive third words / phrases including a number of the second words / phrases corresponding to a gram value, calculating an appearance frequency in a third database for the one or more third words / phrases, and Associates a phrase with the appearance frequency of the phrase included in the third database in the third database, determines whether the calculated appearance frequency is greater than or equal to a threshold, and the calculated occurrence frequency is greater than or equal to the threshold If it is determined that there is, the one or more second sentences are adopted as similar sentences of the first sentence and output to an external device.

  With this configuration, the first sentence is input, and one or more second phrases having the same meaning as the first phrase are extracted from the first database among the plurality of phrases constituting the first sentence. Associating a phrase with a synonym of a phrase included in the first database, calculating an N-gram value based on a context-dependent value corresponding to one or more second phrases obtained based on the second database; The second database associates the phrase with a context-dependent value corresponding to the phrase included in the second database, and the context-dependent value indicates the degree to which the meaning of the phrase included in the second database depends on the context, Extract one or more consecutive third words / phrases including the number of second words / phrases equivalent to the N-gram value in one or more second sentences in which the first word / phrase is replaced with one or more second words / phrases. And one or more third words And calculating the appearance frequency in the third database, the third database associates the phrase with the appearance frequency in the third database of the phrase included in the third database, and determines whether the calculated appearance frequency is equal to or greater than a threshold value, If it is determined that the calculated appearance frequency is greater than or equal to the threshold, one or more second sentences are adopted as similar sentences of the first sentence and are output to an external device. Only words / phrases are identified in a large search area, and second words / phrases with low context-dependent values can be identified in a small search area, reducing search costs for the third database, which is a language model database, and similar. Sentences can be identified with high accuracy.

  The first sentence is described in a first language, the first sentence is included in a bilingual corpus, and the bilingual corpus includes a plurality of pairs of a sentence described in the first language and a bilingual sentence described in the second language. In addition, when it is determined that the calculated appearance frequency is equal to or higher than the threshold, the one or more second sentences may be added to the parallel corpus as similar sentences to the first sentence.

  With such a configuration, a similar sentence can be added to the bilingual corpus.

  The third database includes an N-gram language model database, and N of the N-gram language model is determined as i (positive integer) according to the context-dependent value, and the third database is collated. Thus, the appearance frequency of the i-gram including the second word / phrase is obtained, and the one or more second sentences are set as similar sentences of the first sentence based on the appearance frequency of the i-gram including the second word / phrase. You may make it determine whether it employ | adopts.

  With such a configuration, N of the N-gram language model is determined as i (a positive integer) according to the context-dependent value, and the i- containing the second word is checked by collating the database of the N-gram language model. Since the appearance frequency of gram is obtained and it is determined whether or not one or more second sentences are adopted as similar sentences of the first sentence based on the obtained appearance frequency, i increases as the context-dependent value increases. By setting i to be smaller as the context-dependent value is smaller, an i-gram including the second word / phrase having a larger context-dependent value is used for a second word / phrase having a higher context-dependent value by using a wide identification target area. The frequency can be obtained with high accuracy and the appearance frequency of the i-gram including the second word / phrase having a small context-dependent value is reduced at a low cost by using a narrow identification target region for the second word / phrase having a low context dependence. And with high accuracy Can Mel, it is possible to identify similar sentence efficiently and accurately.

  Generated based on the one or more second sentences determined to be adopted as similar sentences of the first sentence, and a translated sentence obtained by translating the first sentence that generated the one or more second sentences in a second language Using the translated model, a predetermined translation target sentence is translated to create a translation result sentence, the translation result sentence is evaluated, and based on the evaluation result of the translation result sentence, the language of the translation target sentence and Feedback information including language information related to the language of the translation result sentence and evaluation information for the language information may be generated.

  With such a configuration, a translation model generated based on one or more second sentences determined to be adopted and a translated sentence obtained by translating the first sentence that generated one or more second sentences in the second language To create a translation result sentence by translating a predetermined translation target sentence, evaluate the created translation result sentence, and based on the evaluation result of the translation result sentence, the language of the translation target sentence and / or the translation result sentence Since feedback information including language information related to the language and evaluation information for the language information is generated, feedback information for learning and reflecting cases considering context dependency can be generated autonomously. .

  At least one of the first database, the second database, and the third database may be updated using the feedback information.

  With such a configuration, at least one of the first database, the second database, and the third database is updated using feedback information including language information and evaluation information. Can be reflected in at least one of the first database, the second database, and the third database, and can be applied to expressions that do not exist in the first database, the second database, and the third database before being updated, It is possible to identify autonomous similar sentences.

  When the feedback information includes the second word / phrase having context dependency, the second database and the third database may be updated.

  With such a configuration, when the feedback information includes a second word / phrase having context dependency, the second database and the third database are updated. It is possible to identify the similar sentence with high efficiency and autonomousness considering the context dependency.

  When the feedback information includes a new sentence expression, a context-dependent value of the second database may be changed according to the sentence expression.

  With this configuration, when the feedback information includes a new sentence expression, the context-dependent value of the second database is changed according to the new sentence expression. Similar sentences can be identified.

  When the feedback information includes a new sentence expression, the third database may be updated to include the sentence expression.

  With such a configuration, when the feedback information includes a new sentence expression, the third database is updated to include the new sentence expression. Therefore, it is possible to cope with a new sentence expression that does not exist in the third database before the update. Efficient and autonomous identification of similar sentences can be performed.

  In addition, the present disclosure can be realized not only as a similar sentence generation method that executes the characteristic processing as described above, but also a computer that causes a computer to execute the characteristic processing included in the similar sentence generation method. It can also be realized as a program. It can also be realized as a similar sentence generation device having a characteristic configuration corresponding to a characteristic process executed by the similar sentence generation method. Therefore, also in the following other aspects, the same effect as the above-described similar sentence generation method can be obtained.

  A program according to another aspect of the present disclosure is a program for causing a computer to function as a device that generates a similar sentence from an original sentence, and the first sentence is input to the computer to configure the first sentence. One or more second phrases having the same meaning as the first phrase among a plurality of phrases are extracted from the first database, and the first database associates phrases with synonyms of phrases included in the first database; An N-gram value is calculated based on a context-dependent value corresponding to the one or more second words obtained based on a second database, and the second database includes words and phrases included in the second database. The context-dependent value is associated with the context-dependent value, and the context-dependent value indicates the degree to which the meaning of the word included in the second database depends on the context. In one or more second sentences in which a first phrase is replaced with the one or more second phrases, one or more consecutive third phrases including a number of the second phrases corresponding to the N-gram value are extracted. And calculating an appearance frequency in a third database for the one or more third words, wherein the third database associates a word and an appearance frequency in the third database of a word included in the third database, It is determined whether the calculated appearance frequency is equal to or higher than a threshold value, and when it is determined that the calculated appearance frequency is equal to or higher than the threshold value, the one or more second sentences are adopted as similar sentences of the first sentence. And execute the process of outputting to an external device.

  An apparatus according to another aspect of the present disclosure is an apparatus that generates a similar sentence from an original sentence, and includes an input unit to which a first sentence is input, and a first phrase among a plurality of phrases that constitute the first sentence, A second word / phrase extraction unit that extracts one or more second words / phrases having the same meaning from the first database; and the first database associates words / phrases with synonyms of the words / phrases included in the first database; A calculation unit that calculates an N-gram value based on a context-dependent value corresponding to the one or more second words obtained based on the word, and the second database includes words and phrases included in the second database. The context-dependent value is associated with the context-dependent value, the context-dependent value indicates a context-dependent meaning of the phrase included in the second database, and the first phrase in the first sentence is the one In the second phrase above In one or more replaced second sentences, a third word / phrase extraction unit that extracts one or more consecutive third words / phrases including a number of the second words / phrases corresponding to the N-gram value; A calculation unit that calculates an appearance frequency in the third database, and the third database associates the phrase with an appearance frequency in the third database of the phrase included in the third database. A determination unit that determines whether the appearance frequency is greater than or equal to a threshold value, and when the calculated appearance frequency is determined to be greater than or equal to the threshold value, the one or more second sentences are set as similar sentences to the first sentence. And an output unit that outputs to an external device.

  A system according to another aspect of the present disclosure is a system that generates a similar sentence from an original sentence, and the device and the one or more second ones determined to be adopted as the similar sentence of the first sentence by the device. A translation result obtained by translating a predetermined translation target sentence using a translation model generated based on a sentence and a translation sentence obtained by translating the first sentence that has generated the one or more second sentences in a second language A translation unit that creates a sentence, an evaluation unit that evaluates the translation result sentence created by the translation unit, and a language of the translation target sentence and / or the translation result sentence based on the evaluation result of the evaluation unit A generating unit that generates feedback information including language information about the language and evaluation information for the language information;

  With such a configuration, the same effect as the above-described similar sentence generation method can be obtained, and at least one second sentence determined to be adopted as the similar sentence of the first sentence and the one or more second sentences Using the translation model generated based on the translation of the first sentence generated in the second language, a translation result sentence is created by translating a predetermined translation target sentence, and the created translation result sentence Feedback information including language information on the language of the translation target sentence and / or language of the translation result sentence and evaluation information on the language information is generated based on the evaluation result. It is possible to realize a similar sentence generation system capable of autonomously generating feedback information for learning and reflecting the considered case and autonomously learning and reflecting the case considering the context dependency.

  Needless to say, the above-described computer program can be distributed via a computer-readable non-transitory recording medium such as a CD-ROM or a communication network such as the Internet.

  Moreover, you may comprise as a system which disperse | distributed a part of component of the similar sentence production | generation apparatus or similar sentence production | generation system which concerns on one embodiment of this indication, and the other component to several computers.

  Note that each of the embodiments described below is for showing a specific example of the present disclosure. Numerical values, shapes, components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements. In all the embodiments, the contents can be combined.

  Hereinafter, each embodiment of the present disclosure will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram illustrating an example of a configuration of a similar sentence generation device according to Embodiment 1 of the present disclosure. A similar sentence generation device 1 shown in FIG. 1 generates a similar sentence from a replacement target sentence (original sentence). The similar sentence generation device 1 includes a replacement target sentence input unit 10, a replacement candidate extraction unit 11, a context dependency rate collation unit 12, a context dependency determination unit 13, a language model collation unit 14, a replacement determination unit 15, and a replacement result output unit 16. A replacement candidate dictionary 21, a context dependency rate dictionary 22, and a language model database 23.

  The replacement target sentence input unit 10 receives a predetermined operation input by the user, and outputs the replacement target sentence (first sentence) input by the user to the replacement candidate extraction unit 11. For example, a replacement target sentence such as “I can't speak English, please in Japanese” is input to the replacement target sentence input unit 10. Note that the language of the similar sentence generated by the similar sentence generation device 1 is not particularly limited to Japanese, and other languages such as English, Chinese, Korean, French, German, Italian, Portuguese, etc. Also good.

  The replacement candidate dictionary 21 is a replacement candidate storage unit that stores replacement examples of phrases / words / morphemes or the like as a dictionary, and one or a plurality of replacement candidate character strings that are replacement candidates for a replacement target portion to be replaced from the replacement target sentence. Is stored in advance. The replacement candidate dictionary 21 is an example of a first database in which words and phrases and synonyms of words included in the replacement candidate dictionary 21 are associated with each other.

  FIG. 2 is a diagram showing an example of the data configuration of the replacement candidate dictionary 21 shown in FIG. As shown in FIG. 2, the replacement candidate dictionary 21 stores a replacement target part (word / phrase) and a replacement candidate character string (synonymous word / phrase) in association with each other. For example, replacement candidate character strings such as “this is” and “this is” are stored in association with “this is” of the replacement target portion, and “speak” is associated with “cannot speak” of the replacement target portion. Replacement candidate character strings such as “No”, “Cannot speak”, “Secret” are stored.

  The replacement candidate extraction unit 11 replaces a replacement candidate character string (one or more second words) having the same meaning as the part to be replaced (first word) among a plurality of words constituting the sentence to be replaced (first sentence). Extracted from the candidate dictionary 21. Specifically, the replacement candidate extraction unit 11 divides the replacement target sentence input from the replacement target sentence input unit 10 in units of phrases / words / morphemes and the like, and replaces the divided phrases / words / morphemes and the like. A part is determined, a replaceable character string pair (replacement candidate character string) stored in association with the replacement target part is searched from the replacement candidate dictionary 21, and one or more replacement candidate character strings are extracted and replaced. Output to the context-dependent rate matching unit 12 together with the target sentence. For example, when the replacement target portion is “I can't speak”, the replacement candidate extraction unit 11 extracts replacement candidate character strings such as “I can't speak”, “I can't speak”, and “I am secret” from the replacement candidate dictionary 21. To do. Note that the method of dividing the replacement target sentence is not particularly limited to the above example, and various known methods can be used.

  The context-dependent rate dictionary 22 stores a context-dependent value indicating applicability (context-dependent) when replaced with a phrase / word / morpheme or the like as a dictionary in pairs of the phrase / word / morpheme and the numerical value. It is a dependence value storage unit. Specifically, the context dependency rate dictionary 22 stores in advance a plurality of data pairs in which a replacement candidate character string is associated with a context dependency rate pc indicating a degree to which the replacement candidate character string depends on the context. . The context-dependent rate dictionary 22 is an example of a second database that associates words and phrases with context-dependent values corresponding to the words included in the context-dependent rate dictionary 22, and the context-dependent values are included in the context-dependent rate dictionary 22. Indicates the degree to which the meaning of the given phrase depends on the context.

  FIG. 3 is a diagram showing an example of the data configuration of the context-dependent rate dictionary 22 shown in FIG. As shown in FIG. 3, in the context-dependent dictionary 22, for example, pc = 0.35 for the replacement candidate character string “Is”, pc = 0.05 for “Issuga”, “I cannot speak” ”Is stored in the context-dependent dictionary 22 in advance, such as“ pc = 0.25 ”,“ cannot speak ”, pc = 0.01,“ secret ”, pc = 0.75, and the like.

  Here, the context dependency rate pc is, for example, a value in the range of 0 to 1 that indicates that the candidate candidate character string is dependent on the context and thus the similar candidate sentence using the replacement candidate character string is rejected. . Note that the context-dependent value is not particularly limited to the above-described context-dependent rate pc, and can be changed in various ways. For example, another numerical value indicating the degree of the replacement candidate character string depending on the context is used, or the replacement candidate character string is used. It is also possible to classify the degree of dependence on the context (for example, classify the context dependence into a class of large, medium, small, etc.) and store which class it belongs to.

  The context dependency rate collation unit 12 searches the context dependency rate dictionary 22 for the context dependency rate pc of the replacement candidate character string, and extracts and extracts the context dependency rate pc stored in association with the replacement candidate character string. The context dependency rate pc is output to the context dependency determination unit 13 together with the replacement target sentence. For example, as the context dependency rate pc, 0.25 is extracted when the replacement candidate character string is “I can't speak”, 0.01 when “I can't speak”, and 0.75 when it is “secret”. .

  The context dependency determination unit 13 calculates an N-gram value based on the context dependency value corresponding to the replacement candidate character string (one or more second words / phrases) obtained based on the context dependency rate dictionary 22. Specifically, the context dependency determination unit 13 determines the identification target region of the language model database 23 to be referred to in order to determine the similar candidate sentence including the replacement candidate character string, from the value of the context dependency rate pc. The determination result is output to the language model matching unit 14 together with the replacement target sentence.

  Here, in the present embodiment, an N-gram language model database is used as the language model database 23, and the language model database 23 stores data in a table format in which language information and its appearance frequency are associated with each other. Is remembered. The language model database 23 is an example of a third database in which words and phrases are associated with the appearance frequencies of the words included in the language model database 23 in the language model database 23.

  FIG. 4 is a diagram showing an example of the data configuration of the language model database 23 shown in FIG. As shown in FIG. 4, the language model database 23 includes, for example, “234, 567, 890” for “English” and “12,345” for “English” as the language information and its appearance frequency. , 670 ”is stored in advance in the language model database 23 in the form of a table in association with“ 22,222,220 ”for“ English ”and“ 999,001 ”for“ I like English ”. ing. Further, for example, the appearance probability can be obtained based on the appearance frequency.

  The information stored in the language model database 23 is not particularly limited to the above example, and any information can be used as long as it is a table in which language information is associated with a value corresponding to its appearance frequency. Also good. Further, the language model of the language model database 23 is not particularly limited to the above-described N-gram language model, and other language models may be used.

  When the language model database 23 is an N-gram language model database, the context dependency determination unit 13 determines N (N-gram value) of the N-gram language model in the language model database 23 according to the context dependency rate pc. Is determined as i (a positive integer). Specifically, for example, the context dependency determination unit 13 classifies the context dependency rate pc into four classes, 0 ≦ pc ≦ 0.25 is class 1, and 0.25 <pc ≦ 0.5 is class 2. 0.5 <pc ≦ 0.75 is class 3, 0.75 <pc ≦ 1 is class 4, N-gram N (positive integer) is N = 4 in class 1, N = in class 2 5. N = 6 is determined for class 3 and N = 7 is determined for class 4.

  For example, in the case of “cannot speak” for the replacement candidate character string, the context dependency rate pc is 0.25 and belongs to class 1, and the context dependency determination unit 13 uses class 1 as the identification target area of the language model database 23. Determine a substantial N-gram or N = 4. Note that the criterion for determining the identification target area is not particularly limited to the above example, and various changes can be made. The identification target area may be directly expressed using the context-dependent rate pc. For example, if N = floor (k-log2 (pc)) (here, k is a constant) and the replacement candidate character string cannot be spoken, the context dependency rate pc is 0.25, and the constant k = 6 In this case, N = 4.

  The language model database 23 is not limited to the N-gram language model, and may be a database based on other language resources. For example, it may be a language model described in a distributed representation such as a real value or a vector, and can be constructed by combining arbitrary existing methods and existing data. In any case, the range for searching the database as the identification target area can be defined by an arbitrary variable, and the arbitrary variable can be determined according to the context dependency rate pc.

  The language model matching unit 14 replaces the replacement target part (first phrase) with the replacement candidate character string (one or more second phrases) in the replacement target sentence (first sentence) (one or more second sentences). ), Consecutive N-grams (one or more third words) including a number of replacement candidate character strings (second words) corresponding to the N-gram values are extracted, and N-grams (one or more third words) are extracted. The language model database 23 is calculated for the phrase).

  That is, the language model matching unit 14 searches the language model database 23 for the identification target data corresponding to the identification target region determined by the context dependency determination unit 13 and extracts the identification target data, thereby matching the replacement candidate character string. And generating pair data of linguistic information composed of phrases / words / morphemes and the like related to the replacement candidate character string and a value corresponding to the appearance frequency or appearance probability of the linguistic information, along with the replacement target sentence 15 is output.

  Specifically, the language model collation unit 14 uses the value of N given as the size of the identification target area referred to by the context dependency determination unit 13 and uses the N-gram (for example, a replacement candidate) from the language model database 23. If the character string belongs to class 1, the appearance frequency or appearance probability of 4-gram) is acquired, and the compared replacement candidate character string and the acquired appearance frequency or appearance probability are output to the replacement determination unit 15.

  The replacement determination unit 15 uses the pair data of the linguistic information composed of phrases / words / morphemes and the like obtained from the language model matching unit 14 and the value corresponding to the appearance frequency or appearance probability of the linguistic information. It is determined whether the replacement candidate character string is applied to the replacement target sentence or rejected, and the replacement result is output to the replacement result output unit 16 together with the replacement target sentence.

  As an example of the determination method, the replacement determination unit 15 determines whether the calculated appearance frequency is greater than or equal to a threshold value. Specifically, when the appearance frequency value of the j-th language information (j is an arbitrary integer) is nj and the predetermined threshold is Th, the replacement determination unit 15 performs nj for all j. If> Th, it is determined that the replacement candidate character string is to be applied to the replacement target sentence, and otherwise it is determined to be rejected.

  For example, using 4-gram as the N-gram, for the replacement candidate character string "I can't speak", the linguistic information and the frequency of appearance of the language information are "I can't speak English" 51,550 ”is“ 1,720 ”for“ I can't speak English ”,“ 530 ”for“ Japan because I ca n’t speak ”, and“ Japanese because I ca n’t speak ” If “3,220” is acquired and Th = 500, the appearance frequency is equal to or higher than the threshold Th for all of j = 1 to 4, and the replacement candidate character string “I cannot speak” is determined to be applicable. Is done.

  Note that the method of determining whether to apply or reject a replacement candidate character string to a replacement target sentence is not particularly limited to the above example, and various modifications are possible. Apply or reject is determined according to the distribution of nj. (For example, rejecting the lower 3% of the appearance frequency of 4-gram), determining whether to apply or reject depending on whether or not j where nj = 0 exists, or using nj The application or rejection may be determined according to the value calculated from the equation (1).

  When it is determined that the calculated appearance frequency is equal to or higher than the threshold, the replacement result output unit 16 replaces the replacement sentence (one or more second sentences) generated by the replacement candidate character string determined to be applied. Adopted as a sentence similar to the sentence (first sentence) and output to an external device. Specifically, the replacement result output unit 16 replaces the replacement target portion of the replacement target sentence with the replacement candidate character string determined to be applied by the replacement determination unit 15 based on the replacement result, and replaces the replacement candidate character determined to be applied. The replacement sentence generated by the column (the sentence after replacement) is adopted as a similar sentence, and the generated similar sentence is output to an external device (not shown).

  The replacement target sentence (first sentence) is described in a first language (for example, Japanese), the replacement target sentence (first sentence) is included in the parallel translation corpus, and the parallel translation corpus is described in the first language. The replacement result output unit 16 includes a plurality of pairs of sentences and a parallel translation written in a second language (for example, English), and the replacement result output unit 16 determines that the application is applied when the calculated appearance frequency is determined to be equal to or greater than a threshold value. The replacement sentence (one or more second sentences) generated by the replaced replacement candidate character string may be added to the parallel corpus as a similar sentence to the replacement target sentence (first sentence).

  Note that the configuration of the similar sentence generation device 1 is not particularly limited to an example in which dedicated functions are configured for each function as described above, and a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random). One or a plurality of computers or servers (information processing devices) having an Access Memory) and an auxiliary storage device, etc., install a similar sentence generation program for executing the above processing, and function as a similar sentence generation device You may comprise. The replacement candidate dictionary 21, the context dependency rate dictionary 22, and the language model database 23 are not particularly limited to the example provided inside the similar sentence generation device 1, and the replacement candidate dictionary 21 and the context dependency rate dictionary 22 are stored in an external server or the like. In addition, the language model database 23 may be provided, and the similar sentence generation device 1 may acquire necessary information via a predetermined network. This is the same in the other embodiments.

  Next, similar sentence generation processing by the similar sentence generation device 1 configured as described above will be described in detail. FIG. 5 is a flowchart illustrating an example of similar sentence generation processing by the similar sentence generation device 1 illustrated in FIG. 1. In the following processing, the application frequency / rejection determination of the replacement candidate character string is performed using the appearance frequency. However, the present invention is not particularly limited to this example. For example, the appearance probability may be used. This is the same in the other embodiments.

  First, in step S <b> 11, the replacement target sentence input unit 10 receives input of a replacement target sentence (original sentence) by the user, and outputs the input replacement target sentence to the replacement candidate extraction unit 11.

  Next, in step S12, the replacement candidate extraction unit 11 divides the replacement target sentence in units of phrases / words / morphemes, etc., determines a replacement target part from the divided phrases / words / morphemes, etc., and a replacement candidate dictionary The replacement candidate character string stored in association with the replacement target portion is extracted from 21 and output to the context dependency rate collating unit 12 together with the replacement target sentence.

  Next, in step S13, the context dependency rate collation unit 12 collates the context dependency rate dictionary 22, extracts the context dependency rate pc of the replacement candidate character string, and outputs it to the context dependency determination unit 13 together with the replacement target sentence. To do.

  Next, in step S14, the context dependency determination unit 13 refers to the context dependency by determining N of N-gram of the language model database 23 from the value of the context dependency rate pc of the replacement candidate character string. The language model length is determined, and the determined value of N is output to the language model matching unit 14 together with the replacement target sentence.

  For example, the sentence to be replaced is "I can't speak English, so please in Japanese", the replacement candidate string is "I can't speak", and the replacement candidate sentence is "I can't speak English If it is “Please in Japanese”, the context dependence determination unit 13 determines N = 4 as N of N-gram in the language model database 23.

  Next, in step S <b> 15, the language model collation unit 14 uses the value of N given as the size of the identification target area referenced from the context dependency determination unit 13, and the appearance frequency of the N-gram from the language model database 23. , And the collated replacement candidate character string and the acquired appearance frequency are output to the replacement determination unit 15 together with the replacement target sentence.

  For example, when replacing “I cannot speak” in the above sentence to be replaced with “I can't speak”, the language model matching unit 14 uses a surrounding 4-gram (for example, “ "I can't speak English", "I can't speak English", "I can't speak Japan", and "I can't speak Japanese") and collate with the language model database 23, and each 4-gram Occurrence frequency (for example, "I can't speak English" 51,550, "I can't speak English" 1,720, "I can't speak Japan" 530, "I can't speak Japanese" 3,220).

  Next, in step S <b> 16, the replacement determination unit 15 acquires the N-gram including the replacement candidate character string and the appearance frequency from the language model matching unit 14 and calculates the score of the replacement candidate character string.

  Next, in step S17, the replacement determination unit 15 applies the replacement candidate character string to the replacement target sentence by determining whether or not the score (appearance frequency) of the replacement candidate character string is equal to or greater than a predetermined threshold Th. It is determined whether to reject or reject, and the determination result is output to the replacement result output unit 16 together with the replacement target sentence.

  When it is determined in step S17 that the score (appearance frequency) of the replacement candidate character string is less than the predetermined threshold Th, in step S20, the replacement result output unit 16 rejects the replacement candidate character string and ends the process. .

  On the other hand, if it is determined in step S17 that the score (appearance frequency) of the replacement candidate character string is greater than or equal to the predetermined threshold Th, the replacement result output unit 16 replaces the replacement candidate character string with the replacement target sentence in step S18. Apply to the target part and create a replacement sentence that replaces the replacement target part of the replacement target sentence with a replacement candidate character string.

  Next, in step S19, the replacement result output unit 16 outputs the replacement sentence generated by the replacement candidate character string determined to be applied as a similar sentence, and ends the process.

  With the above processing, in the present embodiment, N of the N-gram language model is determined according to the context dependency rate pc, and N increases as the context dependency rate pc increases, and decreases as the context dependency rate pc decreases. It is set. Further, the N-gram appearance frequency including the replacement candidate character string is obtained by collating the language model database 23 using the determined N, and is generated from the replacement candidate character string based on the obtained appearance frequency. Since it is determined whether or not the replacement sentence is adopted as the similar sentence, the appearance frequency of the i-gram including the replacement candidate character string having a large context dependency rate pc is obtained with high accuracy using a wide identification target area. In addition, it is possible to obtain the appearance frequency of the N-gram including the replacement candidate character string having a small context-dependent rate pc at a low cost and with high accuracy using a narrow identification target region. As a result, the search cost for the language model database 23 can be reduced, and similar sentences can be identified with high accuracy.

(Embodiment 2)
FIG. 6 is a block diagram illustrating an example of a configuration of a similar sentence generation system according to the second embodiment of the present disclosure. The similar sentence generation system shown in FIG. 6 includes a similar sentence generation device 1a and a translation device 2.

  The similar sentence generation device 1a includes a replacement target sentence input unit 10a, a replacement candidate extraction unit 11, a context dependency rate collation unit 12, a context dependency determination unit 13, a language model collation unit 14, a replacement determination unit 15, and a replacement result output unit 16. , A data update unit 17, a replacement candidate dictionary 21, a context-dependent rate dictionary 22, and a language model database 23. The translation apparatus 2 includes a parallel corpus generation unit 31, a translation model generation unit 32, a translated sentence input unit 33, a machine translation unit 34, a translation result sentence output unit 35, a translation result evaluation unit 36, and a feedback data generation unit 37. .

  The similar sentence generation device 1a generates a similar sentence from the replacement target sentence (original sentence), and outputs the similar sentence determined to be adopted to the translation apparatus 2. The translation device 2 uses a translation model generated based on a similar sentence determined to be adopted by the similar sentence generation device 1a and a translated sentence obtained by translating the original sentence that generated the similar sentence in a predetermined language. The translation target sentence is translated to create a translation result sentence, and based on the evaluation result of the created translation result sentence, language information on at least one of the language of the translation target sentence and the language of the translation result sentence, and the language information Feedback information including evaluation information is generated and fed back to the similar sentence generation device 1a. The similar sentence generation device 1a updates at least one of the replacement candidate dictionary 21, the context dependency rate dictionary 22, and the language model database 23 based on the feedback information.

  Here, the similar sentence generation device 1a shown in FIG. 6 is different from the similar sentence generation device 1 shown in FIG. 1 in that the data update unit updates the data in the replacement candidate dictionary 21, the context dependency rate dictionary 22, and the language model database 23. 17 is added, and in addition to the input of the replacement target sentence, the replacement target sentence input unit 10a outputs the translated sentence of the input replacement target sentence (original sentence) to the translation device 2, and the other points are the same. Therefore, the same parts are denoted by the same reference numerals, and detailed description thereof is omitted.

  The replacement target sentence input unit 10a receives a predetermined operation input by the user, outputs the replacement target sentence input by the user to the replacement candidate extraction unit 11, and outputs a replacement result output unit from the replacement candidate extraction unit 11 for the subsequent replacement target sentence. The processing up to 16 is the same as the processing from the replacement candidate extraction unit 11 to the replacement result output unit 16 shown in FIG. 1, and the replacement result output unit 16 replaces the replacement candidate character string determined to be applied by the replacement determination unit 15. The replacement sentence (similar sentence) generated by the above is output to the bilingual corpus generation unit 31.

  The replacement target sentence input unit 10a receives a predetermined operation input by the user and translates the original sentence that has been input by the user and has generated the replacement sentence in a predetermined language, that is, a translation sentence (original sentence of the replacement target sentence). 2) is output to the bilingual corpus generation unit 31. For example, when the replacement sentence is created in Japanese (source language sentence) and the translation device 2 performs Japanese-English translation, the translation sentence is created in English (target language sentence). The source language sentence and the target language sentence are not particularly limited to the above example, and when the similar sentence generation device 1a generates an English similar sentence, English may be the source language sentence and Japanese may be the target language sentence. Also, other languages such as Chinese, Korean, French, German, Italian, Portuguese may be used.

  The parallel corpus generation unit 31 associates the replacement sentence output from the replacement result output unit 16 with the translation sentence of the replacement target sentence output from the replacement target sentence input unit 10a, generates a new parallel corpus, and generates a translation model. The data is output to the generation unit 32. The method for generating a bilingual corpus is not particularly limited to the above example, and a new bilingual corpus may be added to a bilingual corpus that has already been created, and various known methods can be used.

  The translation model generation unit 32 generates a translation model by predetermined learning using the new parallel translation corpus generated by the parallel translation corpus generation unit 31 and outputs the translation model to the machine translation unit 34. As a translation model generation method, various known methods can be used, and thus detailed description thereof is omitted.

  The translated sentence input unit 33 receives a predetermined operation input by the user and outputs the translation target sentence (source language sentence) input by the user to the machine translation unit 34. The machine translation unit 34 translates the translation target sentence using the translation model generated by the translation model generation unit 32, and outputs the translation result sentence (target language sentence) together with the translation target sentence to the translation result sentence output unit 35. . The translation result sentence output unit 35 outputs the translation result sentence together with the translation target sentence to the translation result evaluation unit 36 as a translation result.

  The translation result evaluation unit 36 evaluates the translation accuracy and quality of the translation result sentence (target language sentence) output from the translation result sentence output unit 35. Here, as an evaluation method of the translation result evaluation unit 36, the evaluation may be performed by a mechanical numerical index, or the manual evaluation result may be input to the translation result evaluation unit 36. The translation result evaluation unit 36 outputs evaluation information such as an evaluation value or an evaluation category as an evaluation result to the feedback data generation unit 37 in association with the translation result sentence (target language sentence) and / or the translation target sentence (source language sentence). To do.

  The feedback data generation unit 37 generates feedback data to be fed back to the similar sentence generation device 1a as feedback information based on the evaluation result output from the translation result evaluation unit 36, and outputs the feedback data to the data update unit 17. Here, the feedback data is pair data of arbitrary language information on the source language and / or target language side and value or state evaluation information related to the language information. Various data can be used as this feedback data, and the following data can be used.

  For example, when the translation result is bad, the translation result sentence (target language sentence) is corrected by the user or a predetermined translation result sentence correction device, and a better translation sentence is input, so that the input translation sentence and the original Pair data of a pair of language information with a translation target sentence (source language sentence) and evaluation information of a translation result state (bad) may be used as feedback data.

  In addition, when a translation target sentence (source language sentence) is corrected by a user or a predetermined translation target sentence correcting device, and a translation target sentence having a different expression is input for the same purpose, a better translation result sentence can be obtained. The language information of the pair of the original translation target sentence (source language sentence) and the translation target sentence (source language sentence) whose translation result was good, and evaluation information of the translation result state (good / bad binary value) Pair data may be used as feedback data.

  Also, one or a plurality of sentences close to the translation target sentence (source language sentence) are extracted from the bilingual corpus, and an evaluation value (for example, good) whether or not the user or a predetermined translated sentence evaluation device has failed as a source language. / Binary / bad) and assigns an evaluation value to the sentence close to the extracted source language sentence, and the pair data of the evaluation value and language information indicating a sentence close to the source language sentence as feedback data Also good.

  In addition, a plurality of translation result sentences are created by the machine translation unit 34, and a more appropriate translation result sentence is selected by the user or a predetermined translation sentence evaluation apparatus, and the selected translation result sentence and the selected translation result sentence are not selected. Pair data of language information of a pair with a translation result sentence and evaluation information indicating a selection result of these translation result sentences may be used as feedback data.

  The data update unit 17 is based on the feedback data generated by the feedback data generation unit 37 (pair data of the language information and the value or state evaluation information related to the language information), the replacement candidate dictionary 21, the context-dependent rate dictionary 22, and the language At least one database content of the model database 23 is updated.

  In addition, when the feedback data includes a replacement candidate character string having context dependency, the data update unit 17 updates the context dependency rate dictionary 22 and the language model database 23. Further, when the feedback data includes a new sentence expression, the data update unit 17 changes the value of the context dependency rate of the context dependency ratio dictionary 22 according to the sentence expression, and the language model so as to include the new sentence expression. The N-gram of the database 23 is partially constructed, and the language model database 23 is updated.

  Further, the data update unit 17 includes information on the source language side in the language information, and information registered in the replacement candidate dictionary 21, the context dependency rate dictionary 22, or the language model database 23 in the language information. Is included, the corresponding information in the replacement candidate dictionary 21, the context-dependent dictionary 22 or the language model database 23 is updated or added according to the corresponding feedback data value or state evaluation information, Or delete it.

  For example, when source language language information having positive (positive) values or state evaluation information is fed back, the data updating unit 17 adds a predetermined weight to the appearance frequency and increases the value of the appearance frequency. For example, the value including the language information in the language model database 23 is changed in the positive direction. On the other hand, when the language information on the source language side having negative (negative) values or state evaluation information is fed back, the data updating unit 17 updates the context dependency rate in a direction in which the context dependent rate increases. As a result, the value including the language information in the context-dependent dictionary 22 is changed in the negative direction.

  Also, the language information of the original translation target sentence (source language sentence) having a poor translation result and the translation target sentence (source language sentence) having a good translation result, and evaluation information of each translation result state (bad / good) In the case where the difference between the translation target sentences corresponding to the good state with respect to the original translation target sentence in the bad state is not registered in the replacement candidate dictionary 21, the data update unit 17 performs the difference corresponding to the good state. Is registered in the replacement candidate dictionary 21.

  In addition, when the pair data of the language information of the translation target sentence (original language sentence) whose translation result is bad and the evaluation information of the translation result state (bad) is fed back, the data update unit 17 has a bad translation result. The replacement candidate character string of the translation target sentence is deleted from the replacement candidate dictionary 21.

  Note that the configurations of the similar sentence generation device 1a and the translation device 2 are not particularly limited to the example in which dedicated functions are configured for each function as described above, and includes a CPU, a ROM, a RAM, an auxiliary storage device, and the like. A computer or a plurality of computers or servers (information processing devices) may be configured to install a program for executing the above processing and function as a similar sentence generation device or a translation device.

  Next, the similar sentence generation process including the feedback data update process by the similar sentence generation system configured as described above will be described in detail. FIG. 7 is a flowchart showing an example of a similar sentence generation process including a feedback data update process of the similar sentence generation system shown in FIG. Of the processes shown in FIG. 7, the same processes as those shown in FIG. 5 are denoted by the same reference numerals, and detailed description thereof is omitted.

  First, as similar sentence generation processing by the similar sentence generation device 1a, in step S11a, the replacement target sentence input unit 10a accepts input of a parallel translation sentence and a replacement target sentence corresponding to the original sentence by the user, and converts the parallel translation sentence into a parallel corpus generation unit. 31, and the replacement target sentence is output to the replacement candidate extraction unit 11. Note that the timing of outputting the parallel translation to the parallel corpus generation unit 31 is not particularly limited to the above example, and the replacement target sentence input unit 10a outputs the parallel translation to the parallel corpus generation unit 31 at the time of the processing in step S17. It may be.

  Next, in steps S12 to S17, processing similar to that in steps S12 to S17 shown in FIG. 5 is executed. In step S17, it is determined that the score (frequency of appearance) of the replacement candidate character string is less than the predetermined threshold Th. In step S20, the replacement result output unit 16 rejects the replacement candidate character string and ends the process.

  On the other hand, if it is determined in step S17 that the score (appearance frequency) of the replacement candidate character string is equal to or greater than the predetermined threshold Th, a process similar to that in step S18 shown in FIG. In S19, the replacement result output unit 16 outputs the replacement sentence (replacement sentence) generated by the replacement candidate character string determined to be applied by the replacement determination unit 15 to the bilingual corpus generation unit 31 by the similar sentence generation device 1a. The similar sentence generation process ends.

  Next, as feedback data update processing by the translation device 2 and the similar sentence generation device 1a, in step S21, the bilingual corpus generation unit 31 receives the replacement sentence output from the replacement result output unit 16 and the replacement target sentence input unit 10a. It associates with the output parallel translation sentence, generates a new parallel corpus, and outputs it to the translation model generation unit 32.

  Next, in step S <b> 22, the translation model generation unit 32 generates a translation model by learning using the new parallel corpus generated by the bilingual corpus generation unit 31 and outputs the translation model to the machine translation unit 34.

  Next, in step S <b> 23, the translated sentence input unit 33 receives input of the translation target sentence by the user, and outputs an arbitrary translation target sentence that the user desires to translate to the machine translation unit 34.

  Next, in step S24, the machine translation unit 34 translates the translation target sentence into the translation result sentence by the translation model generated by the translation model generation unit 32, and translates the translation result sentence together with the translation target sentence to the translation result sentence output unit 35. Output to.

  Next, in step S25, the translation result sentence output unit 35 outputs the translation result sentence together with the translation target sentence to the translation result evaluation unit 36.

  Next, in step S26, the translation result evaluation unit 36 evaluates the translation accuracy and quality of the translation result sentence output from the translation result sentence output unit 35, and the evaluation result includes an evaluation value or an evaluation category. The information is associated with the translation result sentence and output to the feedback data generation unit 37.

  Next, in step S <b> 27, the feedback data generation unit 37 generates feedback data from the evaluation result output from the translation result evaluation unit 36 and outputs it to the data update unit 17.

  Finally, in step S <b> 28, the data update unit 17 updates at least one database content of the replacement candidate dictionary 21, the context dependency rate dictionary 22, and the language model database 23 based on the feedback data generated by the feedback data generation unit 37. Then, the feedback data update process ends.

  Through the above processing, in the present embodiment, the translation result sentence obtained by translating a predetermined translation target sentence is evaluated using the translation model generated based on the replacement sentence determined to be adopted and the parallel translation sentence with respect to the original sentence. Based on the evaluation result, the feedback information including the language information about the language of the translation target sentence and / or the language of the translation result sentence and the evaluation information for the language information is generated. It is possible to autonomously generate feedback data for learning and reflecting the case in the similar sentence generation device 1a.

  In the present embodiment, the replacement candidate dictionary 21, the context dependency rate dictionary 22, and the language model database 23 are updated using feedback data including language information and evaluation information, so that context dependency is taken into consideration. The case can be reflected in the replacement candidate dictionary 21, the context dependency rate dictionary 22 and the language model database 23, and also for new sentence expressions that do not exist in the replacement candidate dictionary 21, the context dependency rate dictionary 22 and the language model database 23 before the update. A highly efficient and autonomous similar sentence that can be handled can be identified.

  The present disclosure can reduce the search cost for the language model database and can identify similar sentences with high accuracy. Therefore, the similar sentence generation method, the similar sentence generation program, and the similar sentence generation for generating the similar sentences from the original sentences. The present invention is useful for a device and a similar sentence generation system including the similar sentence generation device.

DESCRIPTION OF SYMBOLS 1, 1a Similar sentence production | generation apparatus 2 Translation apparatus 10, 10a Replacement target sentence input part 11 Replacement candidate extraction part 12 Context dependence rate collation part 13 Context dependence judgment part 14 Language model collation part 15 Replacement judgment part 16 Replacement result output part 17 Data update unit 21 Replacement candidate dictionary 22 Context-dependent rate dictionary 23 Language model database 31 Bilingual corpus generation unit 32 Translation model generation unit 33 Translated sentence input unit 34 Machine translation unit 35 Translation result sentence output unit 36 Translation result evaluation unit 37 Feedback data Generator

Claims (11)

A method for generating a similar sentence from an original sentence,
Enter the first sentence,
One or more second words / phrases having the same meaning as the first word / phrase are extracted from the first database among the plurality of words / phrases constituting the first sentence, and the first database includes the words / phrases and the words / phrases included in the first database And synonymous with
An N-gram value is calculated based on a context-dependent value corresponding to the one or more second words obtained based on a second database, and the second database includes words and phrases included in the second database. Correlating with the corresponding context-dependent value, the context-dependent value indicates the degree to which the meaning of the phrase included in the second database depends on the context;
In one or more second sentences in which the first phrase is replaced with the one or more second phrases in the first sentence, one or more consecutive ones or more including the number of the second phrases corresponding to the N-gram value Extract the third word of
Calculating an appearance frequency in a third database for the one or more third words, wherein the third database associates a word and an appearance frequency in the third database of a word included in the third database;
Determining whether the calculated appearance frequency is greater than or equal to a threshold;
When it is determined that the calculated appearance frequency is equal to or higher than the threshold, the one or more second sentences are adopted as similar sentences of the first sentence and output to an external device.
Method. The first sentence is written in a first language;
The first sentence is included in a bilingual corpus, and the bilingual corpus includes a plurality of pairs of sentences written in a first language and bilingual sentences written in a second language;
If it is determined that the calculated appearance frequency is equal to or higher than the threshold, the one or more second sentences are added to the parallel corpus as similar sentences of the first sentence.
The method of claim 1. The third database includes an N-gram language model database,
According to the context-dependent value, N of the N-gram language model is determined as i (positive integer),
By checking the third database, the occurrence frequency of the i-gram including the second word / phrase is obtained,
Determining whether to employ the one or more second sentences as similar sentences of the first sentence based on the appearance frequency of the i-gram including the second phrase;
The method according to claim 1 or 2. Generated based on the one or more second sentences determined to be adopted as similar sentences of the first sentence, and a translated sentence obtained by translating the first sentence that generated the one or more second sentences in a second language Using the translated model, create a translation result sentence by translating a predetermined translation target sentence,
Evaluating the translation result sentence;
Based on the evaluation result of the translation result sentence, generate feedback information including language information about the language of the translation target sentence and / or language of the translation result sentence, and evaluation information for the language information.
The method according to claim 1. Updating at least one of the first database, the second database, and the third database using the feedback information;
The method of claim 4. Updating the second database and the third database when the feedback information includes the second phrase having context dependency;
The method of claim 4. When the feedback information includes a new sentence expression, a context-dependent value of the second database is changed according to the sentence expression;
The method of claim 4. If the feedback information includes a new sentence expression, update the third database to include the sentence expression;
The method of claim 4. A program for causing a computer to function as a device that generates a similar sentence from an original sentence,
In the computer,
Enter the first sentence,
One or more second words / phrases having the same meaning as the first word / phrase are extracted from the first database among the plurality of words / phrases constituting the first sentence, and the first database includes the words / phrases and the words / phrases included in the first database. And synonymous with
An N-gram value is calculated based on a context-dependent value corresponding to the one or more second words obtained based on a second database, and the second database includes words and phrases included in the second database. Correlating with the corresponding context-dependent value, the context-dependent value indicates the degree to which the meaning of the phrase included in the second database depends on the context;
In one or more second sentences in which the first phrase is replaced with the one or more second phrases in the first sentence, one or more consecutive ones or more including the number of the second phrases corresponding to the N-gram value Extract the third word of
Calculating an appearance frequency in a third database for the one or more third words, wherein the third database associates a word and an appearance frequency in the third database of a word included in the third database;
Determining whether the calculated appearance frequency is greater than or equal to a threshold;
When it is determined that the calculated appearance frequency is equal to or higher than the threshold, the one or more second sentences are adopted as similar sentences of the first sentence and output to an external device.
A program that executes processing. An apparatus for generating a similar sentence from an original sentence,
An input unit for inputting the first sentence;
A second phrase extraction unit that extracts from the first database one or more second phrases having the same meaning as the first phrase among the plurality of phrases constituting the first sentence; the first database includes the phrase and the first phrase Corresponding to the synonym of the phrase contained in the database,
A calculation unit that calculates an N-gram value based on a context-dependent value corresponding to the one or more second words obtained based on a second database; and the second database is included in the words and the second database The context-dependent value corresponding to the phrase, the context-dependent value indicates the degree to which the meaning of the phrase included in the second database depends on the context,
In one or more second sentences in which the first phrase is replaced with the one or more second phrases in the first sentence, one or more consecutive ones or more including the number of the second phrases corresponding to the N-gram value A third word / phrase extraction unit for extracting the third word / phrase of
A calculation unit that calculates an appearance frequency in a third database for the one or more third words; and the third database associates a word and an appearance frequency in the third database of a word included in the third database;
A determination unit for determining whether the calculated appearance frequency is equal to or higher than a threshold;
When it is determined that the calculated appearance frequency is equal to or higher than the threshold value, the one or more second sentences are employed as similar sentences of the first sentence, and an output unit that outputs to an external device is provided.
apparatus. A system for generating a similar sentence from an original sentence,
An apparatus according to claim 10;
The one or more second sentences determined to be adopted as similar sentences of the first sentence by the device, and a translated sentence obtained by translating the first sentence that generated the one or more second sentences in a second language. A translation unit that translates a predetermined translation target sentence using a translation model generated based on the translation model, and creates a translation result sentence;
An evaluation unit for evaluating the translation result sentence created by the translation unit;
Based on the evaluation result of the evaluation unit, a generation unit that generates feedback information including language information on the language of the translation target sentence and / or language of the translation result sentence, and evaluation information on the language information,
system.

Images