CN113705251B - Training method of machine translation model, language translation method and equipment - Google Patents

Abstract

The embodiment of the application discloses a training method of a machine translation model, a language translation method and equipment, and relates to the field of machine translation of natural language processing. The method comprises the following steps: the data in the first bilingual parallel database is divided into source language data and target language data, an initial machine translation model is subjected to fine adjustment through the source language data to obtain a fine-adjusted machine translation model, the fine-adjusted machine translation model is applied to carry out translation tasks, the influence of language coverage deviation existing among data in different languages on the machine translation model can be eliminated, the performance of the machine translation model obtained through training by the method is improved, and translations with high translation quality and high fidelity can be obtained by applying the model.

Description

Machine translation model training method, language translation method and device

Technical Field

The present invention relates to the field of natural language processing of artificial intelligence, and in particular to a training method for a machine translation model, a language translation method and a device.

Background technique

Neural machine translation has risen rapidly in recent years. Compared with statistical machine translation, neural machine translation is relatively simple in terms of model. It mainly consists of two parts, an encoder and a decoder. The encoder represents the source language as a high-dimensional vector after a series of neural network transformations. The decoder is responsible for re-decoding (translating) this high-dimensional vector into the target language.

The training of neural machine translation models is inseparable from large-scale, high-quality bilingual parallel data. Bilingual parallel data is usually obtained by asking human translators to translate, and building large-scale bilingual parallel data requires huge human resources and time costs.

However, there are obvious differences in the content covered by bilingual parallel data from different languages, which is called language coverage bias. The fidelity of the translation obtained by the neural machine translation model is closely related to the language coverage bias. Therefore, the existence of language coverage bias in bilingual parallel data will affect the performance of the neural machine translation model trained with bilingual parallel data.

Summary of the invention

The embodiments of the present application provide a training method for a machine translation model, a language translation method and a device. The application of the training method for a machine translation model can eliminate the influence of the language coverage deviation between bilingual parallel sentence pairs originating from different languages in a bilingual parallel database on the machine translation model, thereby improving the performance of the machine translation model trained by the method.

In a first aspect, an embodiment of the present application provides a method for training a machine translation model, comprising: obtaining a bilingual parallel database, the bilingual parallel database comprising a plurality of groups of bilingual parallel sentence pairs, the bilingual parallel sentence pairs being content-aligned data consisting of source language data and target language data, the bilingual parallel database comprising a first bilingual parallel database;

Dividing the plurality of groups of bilingual parallel sentence pairs in the first bilingual parallel database into those derived from source language data and those derived from target language data, wherein the target language data in the bilingual parallel sentence pairs derived from the source language data are obtained by translating based on the source language data, and the source language data in the bilingual parallel sentence pairs derived from the target language data are obtained by translating based on the target language data;

A first machine translation model is trained by using the data derived from the source language, and the first machine translation model is used to translate the source language into the target language.

The step of dividing the plurality of bilingual parallel sentence pairs in the first bilingual parallel database into data derived from source language and data derived from target language includes:

Acquire parallel sentence pairs to be processed from the first bilingual parallel database;

According to the contents covered by the parallel sentence pairs to be processed, the data types of the parallel sentence pairs to be processed are determined, and the data types include the data derived from the source language and the data derived from the target language.

The step of determining the data type of the parallel sentence pairs to be processed according to the content covered by the parallel sentence pairs to be processed includes:

Determining a first probability that the source language data in the parallel sentence pair to be processed originates from a source language according to the source language data in the parallel sentence pair to be processed;

Determining a second probability that the target language data in the parallel sentence pair to be processed originates from the target language according to the content of the target language data in the parallel sentence pair to be processed;

The data type of the parallel sentence pair to be processed is determined according to a deviation between the first probability and the second probability.

The step of determining the data type of the parallel sentence pair to be processed according to the deviation between the first probability and the second probability specifically includes:

Determine a score of the parallel sentence pair to be processed according to a deviation between the first probability and the second probability, wherein the score is used to determine a data type of the parallel sentence pair to be processed;

When the score is greater than a target threshold, determining that the parallel sentence pair to be processed is derived from source language data;

When the score is less than the target threshold, it is determined that the parallel sentence pair to be processed is derived from the target language data.

Wherein, determining, according to the source language data in the parallel sentence pairs to be processed, a first probability that the source language data in the parallel sentence pairs to be processed are derived from a source language comprises: inputting the source language data in the parallel sentence pairs to be processed into a first language model to determine the first probability, wherein the first language model is used to determine the probability that the source language data in the parallel sentence pairs to be processed appear in the source language, and the first language model is a model obtained by training a source language monolingual database;

Determining a second probability that the target language data in the parallel sentence pair to be processed is derived from a target language according to the content of the target language data in the parallel sentence pair to be processed includes: inputting the target language data in the parallel sentence pair to be processed into a second language model to determine the second probability, wherein the second language model is used to determine the probability that the target language data in the parallel sentence pair to be processed appears in the target language, and the second language model is a model trained by a target language monolingual database.

Before training the first machine translation model using the source language data, the method further includes:

An initial machine translation model is trained using the bilingual parallel database to obtain the first machine translation model.

Before training the first machine translation model using the source language data, the method further includes:

Acquire a monolingual database, wherein the monolingual database includes a plurality of original texts in a target language;

Inputting each original text in the monolingual database into the second machine translation model to obtain a source language translation text corresponding to each original text, wherein the second machine translation model is obtained by training the first bilingual parallel database, and the second machine translation model is used to translate the target language into the source language;

Adding a plurality of groups of pseudo-parallel sentence pairs consisting of the plurality of original texts in the target language and the source language translation texts respectively corresponding to the plurality of original texts in the target language to the bilingual parallel database.

Before training the first machine translation model using the source language data, the method further includes:

Acquire a monolingual database, wherein the monolingual database includes a plurality of original texts in a source language;

Inputting each original text in the monolingual database into the third machine translation model to obtain a target language translation text corresponding to each original text, wherein the third machine translation model is obtained by training the first bilingual parallel database, and the third machine translation model is used to translate the source language into the target language;

Adding a plurality of groups of pseudo-parallel sentence pairs consisting of the plurality of original texts in the source language and the target language translation texts corresponding to the plurality of original texts in the source language to the bilingual parallel database.

In a second aspect, an embodiment of the present application provides a language translation method, comprising:

receiving data to be translated, wherein the data to be translated is source language data;

The data to be translated is input into a machine translation model to obtain target language data corresponding to the data to be translated, wherein the machine translation model is a first machine translation model trained by the method provided in the first aspect or various optional implementations of the first aspect.

In a third aspect, an embodiment of the present application provides a training device for a machine translation model, comprising:

An acquisition unit is used to acquire a bilingual parallel database, wherein the bilingual parallel database includes a plurality of groups of bilingual parallel sentence pairs, wherein the bilingual parallel sentence pairs are content-aligned data consisting of source language data and target language data, and the bilingual parallel database includes a first bilingual parallel database;

a dividing unit, configured to divide the plurality of groups of bilingual parallel sentence pairs in the first bilingual parallel database into groups derived from source language data and groups derived from target language data, wherein the target language data in the bilingual parallel sentence pairs derived from the source language data are obtained by translating based on the source language data, and the source language data in the bilingual parallel sentence pairs derived from the target language data are obtained by translating based on the target language data;

A training unit is used to train a first machine translation model using the data derived from the source language, wherein the first machine translation model is used to translate the source language into a target language.

In a fourth aspect, an embodiment of the present application provides a language translation device, including:

A receiving unit, configured to receive data to be translated, wherein the data to be translated is source language data;

A translation unit is used to translate the data to be translated into its corresponding target language data, the translation unit includes a machine translation model, and the machine translation model is a first machine translation model trained by the machine model training method provided in the first aspect or various optional implementations of the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer device, characterized in that it includes: one or more processors and one or more memories, wherein the one or more memories are respectively coupled to the one or more processors; the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions;

The processor is used to call the computer instruction to execute: a training method for a machine model as provided in the first aspect or various optional implementations of the first aspect.

In a sixth aspect, an embodiment of the present application provides a computer device, characterized in that it includes: one or more processors and one or more memories, wherein the one or more memories are respectively coupled to the one or more processors; the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions;

The processor is used to call the computer instruction to execute: the language translation method provided in the second aspect.

In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the training method of a machine translation model provided in the first aspect or various optional implementations of the first aspect is executed.

In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the language translation method as described in the second aspect is executed.

In a ninth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the first aspect or various optional implementations of the first aspect.

In a tenth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the language translation method provided in the second aspect above.

A method for training a machine translation model provided in an embodiment of the present application divides data in a first bilingual parallel database into data derived from a source language and data derived from a target language, fine-tunes an initial machine translation model using the data derived from the source language to obtain a fine-tuned machine translation model, and applies the fine-tuned machine translation model to perform translation tasks, thereby eliminating the influence of language coverage deviations between data in different languages on the machine translation model, thereby improving the performance of the machine translation model trained by the method, and applying the model can obtain translations with higher quality and fidelity.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1A is a schematic diagram of the structure of a computer system provided in an embodiment of the present application;

FIG1B is a flow chart of a method for training a machine translation model provided in an embodiment of the present application;

FIG2 is a schematic diagram of a result of a classification accuracy rate obtained by applying a method for classifying data types provided in an embodiment of the present application;

3-5 are schematic diagrams of translation performance results of several machine models obtained by training with divided data provided in embodiments of the present application;

FIG6 is a flow chart of another method for training a machine translation model provided in an embodiment of the present application;

7 is a schematic diagram of the translation performance results of a model obtained by training using a training method for a machine translation model provided in an embodiment of the present application;

FIG8 is a flow chart of another method for training a machine translation model provided in an embodiment of the present application;

FIG9 is a flow chart of another method for training a machine translation model provided in an embodiment of the present application;

FIG10 is a schematic diagram of the results of translation quality obtained by applying several machine translation models provided in an embodiment of the present application;

FIG11A is a schematic diagram of the structure of a training device for a machine translation model provided in an embodiment of the present application;

FIG11B is a schematic diagram of the structure of a language translation device provided in an embodiment of the present application;

FIG12 is a schematic diagram of the structure of a language translation device provided in an embodiment of the present application;

FIG13 is a schematic diagram of the structure of a training device for a machine translation model provided in an embodiment of the present application.

Detailed ways

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

First, for ease of understanding, before introducing the training method of the machine translation model provided in the embodiment of the present application, the following introduces the relevant terms involved in the invention example of the present application.

DL: Deep Learning is a branch of machine learning. It is an algorithm that attempts to use multiple processing layers containing complex structures or composed of multiple nonlinear transformations to perform high-level abstraction of data.

NN: Neural Network, a deep learning model in the field of machine learning and cognitive science that imitates the structure and function of biological neural networks.

DNN: Deep Neural Network, a neural network with a deeper network structure, and the core model in deep learning.

NMT: Neural Machine Translation, the latest generation of machine translation technology based on neural networks.

BLEU: A standard method for evaluating machine translation. The higher the value, the better the effect.

Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating/interactive systems, mechatronics, and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

Machine translation refers to a translation method that translates a sentence in a natural language (source language) into another natural language (target language) by a computer. Usually, machine translation is to translate a sentence in the source language into a sentence in the target language by a trained machine translation model. Exemplary, the source language is Chinese, the target language is English, and the sentence in the source language is "I am a student.", which is translated into "I am a student." by the machine translation model. Among them, the machine translation model can be trained by a large number of bilingual parallel sentence pairs.

Bilingual parallel sentence pairs are content-aligned data consisting of source language data and its corresponding target language data. Here, "content alignment" means that the content of the source language data and the content of the target language data have a translation relationship and consistent meaning.

In the embodiment of the present application, bilingual parallel sentence pairs can be divided into two categories, namely, those derived from source language data and those derived from target language data.

The source language data refers to the text first generated by the creator of the source language and then translated into the target language by human translators to form content-aligned bilingual parallel sentence pairs. In other words, the target language data in the bilingual parallel sentence pairs derived from the source language data is obtained by translation based on the source language data.

The target language data refers to the text first generated by the creator of the target language, and then translated into the source language by human translators in the reverse translation direction to form content-aligned bilingual parallel sentence pairs. In other words, the source language data in the bilingual parallel sentence pairs derived from the target language data is obtained by translation based on the target language data.

Language coverage bias: Data from different languages have obvious differences in the content they cover, which is called language coverage bias. For example, data from Chinese may include content with Chinese characteristics such as "Yunnan, Zhang San, Li Si, hairy crab, Bacheng", while data from English may include content with English characteristics such as "California, National Basketball Association, Birmingham".

With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

The machine translation model trained by the machine translation model training method provided in the embodiment of the present application can be used in the following scenarios:

(1) Machine Translation

In this application scenario, the machine translation model trained by the method provided in the embodiment of the present application can be used in applications supporting translation functions such as electronic dictionary applications, e-book applications, web browsing applications, social applications, and image and text recognition applications. When the above application receives the data to be translated, the machine translation model outputs the translation result according to the input data to be translated. Exemplarily, the data to be translated includes at least one of text type, picture type, audio type, and video type content.

(2) Dialogue Q&A

In this application scenario, the machine translation model trained by the method provided in the embodiment of the present application can be applied to smart devices such as smart terminals or smart homes. Taking the virtual assistant set in the smart terminal as an example, the automatic answer function of the virtual assistant is obtained by training the training method of the machine translation model provided in the embodiment of the present application and the translation model is implemented. The user asks the virtual assistant questions about translation. When the virtual assistant receives the questions input by the user (the questions input by the user can be implemented in the form of voice or text input), the machine translation model divides the questions input by the bureau and outputs the translation results. The smart device converts the translation results into voice or text and feeds them back to the user through the virtual assistant.

The above description only takes two scenarios as examples. The method provided in the embodiments of the present application can also be used in other application scenarios, such as extracting text summaries. The embodiments of the present application do not limit the specific application scenarios.

The training method of the machine translation model and the language translation method provided in the embodiments of the present application can be applied to a computer device with strong data processing capabilities. In some embodiments, the training method of the machine translation model and the language translation method provided in the embodiments of the present application can be applied to a personal computer, a workstation or a server, that is, machine translation and training of the machine translation model can be implemented by a personal computer, a workstation or a server.

A computer system provided in an embodiment of the present application is introduced below. Please refer to Figure 1A. Figure 1A is a structural diagram of a computer system. The computer system includes a database 10, a training device 11 and an execution device 12. The execution device 12 may include a first device 110 and a second device 120.

The database 10 includes bilingual parallel databases and monolingual databases in different languages, and the data therein can be used as sample data for training a machine translation model.

The training device 11 can be a server, workstation, personal computer or other device, and is used to train the machine translation model through the data obtained from the database 10. Specifically, the training device 11 can obtain a first bilingual parallel database, and then divide the multiple groups of bilingual parallel sentence pairs in the first bilingual parallel database into data derived from the source language and data derived from the target language, and then train the first machine translation model through the data derived from the source language, and the first machine translation model is used to translate the source language into the target language. In some embodiments, the training device can also translate the monolingual data in the monolingual database obtained from the database 10 through the machine translation model trained by the first bilingual parallel database to obtain pseudo-parallel sentence pairs, and add the obtained pseudo-parallel sentence pairs to the database 10.

The execution device 12 can store the machine translation model trained by the training device 11, and the first device 110 in the execution device 12 is transmitted to the server via a communication network.

The first device 110 is installed with an application supporting the translation function, which may be an electronic dictionary application, an e-book reading application, a web browsing application, a social application, etc. The first device 110 may be a terminal device such as a smart phone, a smart watch, a tablet computer, a laptop computer, an intelligent robot, etc.

The second device 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, content distribution networks, and big data and artificial intelligence platforms. In some embodiments, the second device 120 is a background server of an application in the first device 110.

In some embodiments, the language translation method provided in the embodiments of the present application can be executed by the second device 120. When the first device 110 obtains the data to be translated, it can be sent to the second device 120 through the communication network. After receiving the data to be translated, the second device 120 executes the above-mentioned language translation method to obtain a translation result. Further, the second device 120 sends the translation result to the first device 110, and the first device 110 displays the translation result through an application.

In other embodiments, the first device 110 and the second device 120 may be the same device, and the language translation method provided in the embodiment of the present application may also be executed by the device, which is not limited in the embodiment of the present application.

It should be understood that in some embodiments, the training device 11 and the execution device 12 may be the same device, which can perform machine translation model training and language translation work, and the embodiments of the present application do not limit the function of the device.

The solution provided in the embodiments of the present application relates to the natural language processing technology of artificial intelligence, which is specifically described by the following embodiments:

Please refer to FIG. 1B , which is a flow chart of a method for training a machine translation model provided in an embodiment of the present application. The method for training the machine translation model can be executed by the training device 11 in FIG. 1A above. The embodiment of the present application is described by taking the execution of the training device 11 in FIG. 1A above as an example. As shown in FIG. 1B , the method for training the machine translation model includes but is not limited to the following steps:

S1. Obtain a bilingual parallel database, where the bilingual parallel database includes a first bilingual parallel database.

The bilingual parallel database includes multiple sets of bilingual parallel sentence pairs. The bilingual parallel sentence pairs are content-aligned data consisting of source language data and target language data. That is, in the bilingual parallel database, each source language data has corresponding target language data.

In some embodiments, the bilingual parallel database is a first bilingual parallel database, which includes multiple groups of bilingual parallel sentence pairs, and the multiple groups of bilingual parallel sentence pairs are derived from source language data or from target language data, that is, one language data in each group of bilingual parallel sentence pairs is translated by a human translator.

In other embodiments, the bilingual parallel database includes a first bilingual parallel database and pseudo-parallel sentence pairs, wherein the pseudo-parallel sentence pairs are sentence pairs consisting of a first language original text in a translated monolingual database and a second language text obtained by translating the first language original text through a machine translation model.

S2. Divide the bilingual parallel sentence pairs in the first bilingual parallel database into those derived from source language data and those derived from target language data.

In one implementation, S2 may include but is not limited to the following steps:

S21. Obtain parallel sentence pairs to be processed from the first bilingual parallel database.

S22. Determine the data type of the parallel sentence pair to be processed according to the content covered by the parallel sentence pair to be processed, where the data type includes data derived from the source language and data derived from the target language.

Different languages cover different content distributions, that is, in the parallel sentence pairs to be processed, the content distributions covered by the source language data and the target language data are different. Therefore, the data type of the parallel sentence pairs to be processed can be determined by the deviation between the content distributions covered by the source language data and the target language data.

In a specific implementation, determining the data type of the parallel sentence pair to be processed according to the content covered by the parallel sentence pair to be processed may include the following process:

S221. Train a first language model using a source language monolingual database, where the first language model is used to determine the probability of source language data in a bilingual parallel sentence pair appearing in the source language.

S222: Input the source language data in the parallel sentence pairs to be processed into the first language model to obtain a first probability that the source language data in the parallel sentence pairs to be processed are derived from the source language.

S223. Train a second language model using a target language monolingual database, where the second language model is used to determine the probability of target language data in a bilingual parallel sentence pair appearing in the target language.

S224: Input the target language data in the parallel sentence pairs to be processed into a second language model to obtain a second probability that the target language data in the parallel sentence pairs to be processed are from the target language.

S225. Determine the data type of the parallel sentence pair to be processed according to the deviation between the first probability and the second probability.

In some implementations, S225 may include the following process: determining a score of the parallel sentence pair to be processed based on a deviation between the first probability and the second probability, the score being used to determine a data type of the parallel sentence pair to be processed; when the score is greater than a target threshold, determining that the parallel sentence pair to be processed is derived from source language data; when the score is less than the target threshold, determining that the parallel sentence pair to be processed is derived from target language data.

In other implementations, S225 may also be implemented in other ways, for example, by comparing the first probability with the second probability. When the first probability is greater than the second probability, it is determined that the parallel sentence pair to be processed is derived from the source language data. When the first probability is less than the second probability, the parallel sentence pair to be processed is derived from the target language data.

The following describes the principle of detecting the data type of the parallel sentence pairs to be processed according to the content distribution covered by different languages in S221-S224. Indicates the content distribution covered by the source language, Represents the content distribution covered by the target language. Given a parallel sentence pair <x, y> to be processed, the probability of it being covered by the source language or the target language can be expressed as:

The difference between the two probabilities above can be quantified using a score:

in because and It is only related to the language, but not to the parallel sentence pair <x, y> to be processed, so when the source language and the target language are given, c is a constant. It can be obtained that the parallel sentence pairs with higher scores have a higher probability of belonging to the data from the source language, and the parallel sentence pairs with lower scores have a higher probability of belonging to the data from the target language. In order to characterize and You can use the source language monolingual database and the target language monolingual database to train the language model based on the self-attention mechanism: the first language model With the second language model And use the probability of the language model to estimate and Specifically, The above score can be expressed as:

After the language model training is completed, the value of c can be determined through a small-scale bilingual parallel database of known data types.

It should be understood that the first language model and the second language model can be the above-mentioned language model based on the self-attention mechanism, or can be a language model based on any other architecture, such as a language model based on a recurrent neural network or a language model based on a convolutional neural network.

The language model is a language model trained by a large amount of sample data using the maximum likelihood estimation method. The principle of determining the probability that a language data comes from the language through the language model is as follows: a language model is usually constructed as the probability distribution p(s) of a string s, where p(s) actually reflects the probability that s appears as a sentence. The probability here refers to the likelihood of this combination of string components appearing in the training corpus. Assuming that the training corpus comes from human language, it can be considered that this probability is the probability of whether the input sentence is human language. The language model may include an input layer, a projection layer, a hidden layer, and an output layer. The string s consists of multiple words. Each word is represented in the form of a one-hot vector in the input layer. Each one-hot vector is converted into another word vector in the projection layer, connected to create a matrix e, and then the matrix is flattened and further converted into a hidden vector through the hidden layer. Finally, the probability distribution of the string s is calculated and output based on the hidden vector using the softmax function. The probability of a sentence (string) s can be the product of the probability of each word in the sentence appearing in the sentence, and the probability of the next word of the word appearing in the sentence can be predicted based on a word in the input sentence.

For example, the monolingual database is a Chinese monolingual database, and the training process of the language model is explained. Exemplarily, the process of training a language model with the sentence "I have a dream" as a sample data can be: first construct a target model, the target model includes an input layer, a projection layer, a hidden layer and an output layer; then "I have a dream" is input into the target model, through the input layer, the projection layer, the hidden layer and the output layer, and finally the model predicts the probability of the sentence "I have a dream". The model is guided to maximize this probability during the training process. In this way, each piece of data in the Chinese monolingual database is input into the target model, and after training, the optimized target model is obtained, which is the trained language model.

Exemplarily, as shown in FIG2, the data in the English-Chinese (En-Zh) parallel database, the English-Japanese (En-Ja) parallel database, and the English-German (En-De) parallel database are divided respectively by the method for determining the data type of the bilingual parallel sentence pair described in S2 above, and the division accuracy as shown in FIG2 is obtained. Among them, method FT is an existing text classification method based on convolutional neural network, method Ours is the method described in S2 in the embodiment of the present application, and the accuracy of dividing the data in the three databases is represented by the F1 value. The higher the F1 value, the higher the accuracy of data division.

After the data in the bilingual database is divided, the machine translation model can be trained with the divided data. Specifically, three machine translation models can be trained according to the data type: (1) Model 1 trained with data from the target language; (2) Model 2 trained with data from the source language; (3) Model 3 trained with the first bilingual database without distinguishing the data type.

For example, the data in the English-Chinese (En-Zh) parallel database, the English-Japanese (En-Ja) parallel database, and the English-German (En-De) parallel database are used as training samples to train the corresponding three machine translation models. The translation performance of each model is evaluated from three aspects: overall translation quality, translation fidelity, and translation fluency, and the results are shown in Figures 3 to 5.

As shown in Figure 3, the first column Data Origin represents the data type of the training sample for the machine translation model, Target represents the data derived from the target language, Source represents the data derived from the source language, and Both represents the use of all the data in the language parallel database without distinguishing the data type. Among them, starting from the second column, the bold in each column of BLEU values represents the highest score in the column, and the underlined score represents the second highest score in the column. The BLEU value represents the quality of the translation. The higher the value, the better the quality of the translation. Exemplarily, En-Zh represents English-Chinese translation, and the second column represents English as the source language and Chinese as the target language. Taking the second column as an example, the score 33.2 corresponding to Target represents the translation quality of the model translation obtained by training with data derived from Chinese, the score 36.5 corresponding to Source represents the translation quality of the model translation obtained by training with data derived from English, and the score 36.6 corresponding to Both represents the translation quality of the model translation obtained by training with all the data in the En-Zh bilingual parallel database without distinguishing between data derived from Chinese or English. It can be seen from the BLEU values in each column in Figure 3 that the translation quality of the model trained only using data from the source language and the model trained using all the data in the language parallel database without distinguishing the data type are very similar, and the translation quality of the model trained only using data from the source language has also achieved the highest score. It can be seen that the model trained only using data from the source language has better translation performance in terms of translation quality.

As shown in FIG4 , the first column Data Origin indicates the data type of the training sample of the machine translation model, Target indicates the target language data, Source indicates the source language data, and Both indicates that all the data in the language parallel database is used without distinguishing the data type. Among them, starting from the second column, the bold in the F-measure value of each column indicates the highest score of the column, and the underlined score indicates the second highest score of the column. The F-measure value indicates the fidelity of the translation. The higher the value, the higher the fidelity of the translation. Here, the first bilingual parallel database is the En-Zh bilingual parallel database as an example. For example, En→Zh indicates that English is the source language and Chinese is the target language, and En←Zh indicates that Chinese is the source language and English is the target language. The F-measure values of nouns, verbs, and adjectives or adverbs are determined respectively. It can be seen from FIG4 that the fidelity of the translation obtained by using only the model trained from the source language data has achieved the highest score many times. Therefore, in terms of the fidelity of the translation, the model trained using only the source language data performs best among the three models. The model trained with data from the target language performed poorly in terms of translation fidelity (the F-measure value was lower than that of the other two models). Moreover, as shown in the scores from the fourth to seventh columns in Figure 5, the model trained with all the data in the language parallel database without distinguishing the data type had a lower F-measure value than the model trained only with data from the source language, indicating that the indiscriminate use of additional models trained with data from the target language failed to improve the fidelity of the translation.

As shown in Figure 5, the fluency of translation is measured by the perplexity of the language model (i.e., PPL). The lower the PPL, the better the fluency. Diff refers to the relative change of the PPL corresponding to a model relative to the PPL corresponding to "Both". "No.Abs." means that no content words are abstracted, and "Count.Abs." means that all content words are abstracted with corresponding part-of-speech tags. Taking the WMT20En-Zh bilingual parallel database as an example, the performance of the three trained models in fluency is shown in Figure 5. When all content words in the text to be translated are abstracted with corresponding part-of-speech tags, the fluency of the translation of the model trained with the source language data is also very small compared with the translation fluency of the model trained with all the data in the bilingual parallel database without distinguishing data types, which is +1.4% and +2.2%. Therefore, it can be shown that when all content words are abstracted with corresponding part-of-speech tags, the model trained only with source language data also has a better performance in translation fluency.

It should be understood that the bilingual parallel sentence pairs in the bilingual parallel database are all translated by human translators from sentences in one language (first language) to sentences in another language (second language), and the first language of the bilingual parallel sentence pair is used as the original language of the bilingual parallel sentence pair (i.e., which language the bilingual parallel sentence pair originates from). Since the previous work ignored the influence of the original language of the data on the neural machine translation model, the original language information of the bilingual parallel sentence pairs in most large-scale bilingual parallel databases was lost in the process of data construction and collation. From the performance of the above-mentioned model trained with data in different original languages (derived from the source language data and derived from the target language data), it can be seen that since the original languages of a large number of bilingual parallel sentence pairs in the bilingual parallel database are different, the quality and fidelity of the translations translated by indiscriminately using data in different original languages to train the machine translation model are poor. Therefore, the bilingual parallel sentence pairs in the bilingual parallel database can be divided into those derived from the source language data and those derived from the target language data according to the original language of the bilingual parallel sentence pairs, and the machine translation model can be further trained for the divided data to improve the performance of the machine translation model.

S3. Training a first machine translation model by using data derived from a source language.

In the embodiment of the present application, training the first machine translation model by using source language data can be implemented by the following several implementation methods. The following specifically describes the processes of several implementation methods of S3.

Implementation method 1: Use the source language data divided in the first bilingual parallel database as training samples to train a first machine translation model. Specifically, use the source language data from the source language data as input and the target language data as output to train the first machine translation model.

Implementation method 2: First, a first machine translation model is trained with a first bilingual parallel database of unclassified data types, and then the trained first machine translation model is fine-tuned using the classified source language data.

For details, please refer to FIG. 6 , which is a flowchart of implementation method 2. FIG.

The above S3 may include the following steps in FIG6:

S301. Obtain a first machine translation model through training on a first bilingual parallel database.

The bilingual parallel sentence pairs in the first bilingual parallel database without data type division are used as training samples, the source language data in each group of bilingual parallel sentence pairs is used as input, and the target language data is used as output, and an initial machine translation model is trained, also called the first machine translation model.

S302: Fine-tune the first machine translation model using source language data.

The divided source language data from the source language data is used as input and the target language data is used as output to train the first machine translation model obtained through the training in S301 to obtain a fine-tuned first machine translation model.

It can be understood that fine-tuning the machine translation model is to further train the machine translation model through partial data.

The following describes the translation performance of the first machine translation model obtained by training the method described in implementation method 2. Exemplarily, the first machine translation models corresponding to the first machine translation models are fine-tuned with data from different languages, and the fine-tuned first machine translation model is used for translation, and the translation quality (BLEU value) is used to characterize the translation performance of the model. As shown in Figure 7, Figure 7 shows the BLEU values of the first machine model trained with the first bilingual database and the first machine translation model after fine-tuning in six translation directions (En-Zh, Zh-En, En-Ja, Ja-En, En-De, De-En, where En-Zh means English En is the source language and Chinese Zh is the target language). Among them, the values in the row corresponding to Baseline represent the BLEU values of the first machine model trained with the first bilingual parallel database used in the six translation directions, the values in the row corresponding to Tune represent the BLEU values of the model after the first machine models corresponding to the first machine models are fine-tuned using data from different languages in the six translation directions, and Average represents the average value of each row of values. It can be seen from the figure that in the six translation directions, the BLEU value of the model fine-tuned using the source language data is higher than the BLEU value of the first machine translation model without fine-tuning, indicating that the translation model trained by the method described in the above implementation method 2 can improve the quality of the translation.

Implementation method 3: Please refer to FIG. 8 , which is a flow chart of another method for training a machine translation model provided in an embodiment of the present application, wherein the above S3 may include the following steps in FIG. 8 :

S31a, obtaining a monolingual database, wherein the monolingual database includes a plurality of original texts in the target language.

S32a, input each original text in the target language in the monolingual database into the second machine translation model to obtain a source language translation text corresponding to each original text, and the second machine translation model is used to translate the target language into the source language.

Among them, the second machine translation model is a machine translation model trained by the first bilingual parallel database, and can be trained by inputting the target language data in each group of bilingual parallel sentence pairs in the first bilingual parallel database and outputting the corresponding source language data.

In some embodiments, after obtaining multiple target language original texts and their corresponding source language translation texts in a monolingual database, indication information, such as a label BT (meaning back translation), is added to each pseudo-parallel sentence pair consisting of the target language original text and its corresponding source language translation text. Back translation means translation from the target language to the source language.

S33a, adding multiple sets of pseudo-parallel sentence pairs consisting of multiple original texts in the target language and their corresponding source language translation texts in the monolingual database to the bilingual parallel database.

S34a, obtaining a first machine translation model through bilingual parallel database training.

S35a, fine-tuning the first machine translation model using the source language data. This process can refer to the relevant description of S302 in the above implementation mode 1, and will not be repeated here.

Implementation 4: Please refer to FIG. 9 , which is a flow chart of another method for training a machine translation model provided in an embodiment of the present application, wherein the above S3 may include the following steps in FIG. 9 :

S31b, obtaining a monolingual database, wherein the monolingual database includes a plurality of original texts whose language is a source language.

S32b, input each original text in the monolingual database in the source language into the third machine translation model to obtain a target language translation text corresponding to each original text, and the third machine translation model is used to translate the source language into the target language.

Among them, the third machine translation model is obtained by training the first bilingual parallel database, and can be trained by inputting the source language data in each group of bilingual parallel sentence pairs in the first bilingual parallel database and outputting the corresponding target language data.

In some embodiments, after obtaining multiple source language original texts and their corresponding target language translation texts in a monolingual database, indication information, such as a label FT (meaning forward translation), is added to each pseudo-parallel sentence pair consisting of the source language original text and its corresponding target language translation text. Forward translation means translation from the source language to the target language.

S33b, adding multiple pseudo-parallel sentence pairs consisting of multiple original texts in the source language and their corresponding target language translation texts in the monolingual database to the bilingual parallel database.

S34b. Obtain a first machine translation model through bilingual parallel database training.

S35b, fine-tuning the first machine translation model using the source language data. This process can refer to the relevant description of S302 in the above implementation mode 1, and will not be repeated here.

In implementation 3 and implementation 4, multiple sets of pseudo-parallel sentence pairs are determined based on the target language monolingual database and the machine translation model trained by the first bilingual parallel database, and the multiple sets of pseudo-parallel sentence pairs are added to the bilingual parallel database. In this way, the initial machine translation model is trained by the bilingual parallel database with the newly added pseudo-parallel sentence pairs. Compared with the initial machine translation model trained only by the first bilingual parallel database, the machine translation model can be fully trained by the features of the pseudo-parallel sentence pairs, thereby enhancing the performance of the machine translation model. Moreover, after adding the indication information for the pseudo-parallel sentence pairs, the performance of the first machine translation model trained by the bilingual parallel database is better.

Exemplarily, the following describes the translation performance (characterized by the translation quality BLEU value) of the machine translation model trained by the above implementation methods 2, 3, and 4 when English is the target language or source language. As shown in Figure 10, in the table, Monolingual means monolingual, Bilingual means bilingual; X→En means: X is the source language, English (En) is the target language, indicating translation from different source languages to English; En→X means: English (En) is the source language, X is the target language, indicating translation from English to different target languages; Tagging means adding tags to the pseudo-parallel data pairs obtained through monolingual data, such as (BT or FT); Fine-Tune means fine-tuning the initial machine translation model using data derived from the source language. The first row represents the translation quality (BLEU) of the first model trained by the first bilingual parallel database, the second row represents the translation quality of the second model obtained by fine-tuning the above-mentioned first model with source language data, the third row represents the translation quality of the third model trained by the first parallel database and pseudo-parallel sentence pairs without adding indication information (also called labels), the fourth row represents the translation quality of the fourth model trained by the first parallel database and pseudo-parallel sentence pairs with added indication information (also called labels), the fifth row represents the translation quality of the fifth model obtained by fine-tuning the above-mentioned third model with source language data, and the sixth row represents the translation quality of the sixth model obtained by fine-tuning the above-mentioned fourth model with source language data. Among them, the 3rd to 6th rows represent the translation quality of the model obtained by training the bilingual parallel database including pseudo-parallel sentence pairs obtained by applying the English (En) monolingual database, the BLUE value in the box 101 represents the translation quality of the machine translation model obtained by training the bilingual parallel database including pseudo-parallel sentence pairs obtained by the reverse translation monolingual database, the BLUE value in the box 102 represents the translation quality of the machine translation model obtained by training the bilingual parallel database including pseudo-parallel sentence pairs obtained by the forward translation monolingual database, and Ave represents the average value. It can be seen from the figure that the machine translation models (the second model, the fifth model, and the sixth model) fine-tuned by the source language data are respectively higher than the corresponding machine translation models (the first model, the third model, and the fourth model) that do not use the fine-tuning from the source language data.

The training method of the above-mentioned machine translation model divides the data in the first bilingual parallel database into data from the source language and data from the target language, fine-tunes the initial machine translation model through the data from the source language to obtain a fine-tuned machine translation model, and applies the fine-tuned machine translation model to perform translation tasks. This can eliminate the influence of the language coverage deviation between data in different languages on the machine translation model, thereby improving the performance of the machine translation model trained by this method. Application of this model can obtain translations with higher quality and fidelity.

The following describes a language translation method provided by an embodiment of the present application, which may include:

The computer device receives data to be translated, where the data to be translated is source language data.

The computer device inputs the data to be translated into the machine translation model to obtain the target language data corresponding to the data to be translated. The process may be that the machine translation model is first applied to perform word embedding processing on the input data to be translated to obtain a mapping intermediate vector; then the mapping intermediate vector is fitted according to the dependency relationship between the mapping intermediate vectors to obtain a fitted vector; finally, the fitted vector is decoded by the decoder in the machine translation model to obtain a decoded vector, thereby outputting a translation result. Among them, the machine translation model may be the first machine translation model trained by the training method of any one of the machine translation models of Figure 1B, Figure 6, Figure 8, or Figure 9. Among them, the relevant description of the machine translation model can refer to the description of Figure 1B, Figure 6, Figure 8, or Figure 9, which will not be repeated here.

Please refer to FIG. 11A , which is a schematic diagram of the structure of a machine translation model training device 1100 provided in an embodiment of the present application. The machine translation model training device can be applied to the training device 11 in FIG. 1A above, and the machine translation model training device can include:

An acquisition unit 1101 is used to acquire a bilingual parallel database, wherein the bilingual parallel database includes a plurality of groups of bilingual parallel sentence pairs, wherein the bilingual parallel sentence pairs are content-aligned data consisting of source language data and target language data, and the bilingual parallel database includes a first bilingual parallel database;

A division unit 1102 is used to divide the plurality of groups of bilingual parallel sentence pairs in the first bilingual parallel database into groups derived from source language data and groups derived from target language data, wherein the target language data in the bilingual parallel sentence pairs derived from the source language data are obtained by translating based on the source language data, and the source language data in the bilingual parallel sentence pairs derived from the target language data are obtained by translating based on the target language data;

The training unit 1103 is configured to train a first machine translation model using the data derived from the source language, wherein the first machine translation model is used to translate the source language into the target language.

Among them, the specific functional implementation of the division unit 1102 can refer to the relevant descriptions of S21, S221-S225 in the above S2; the specific functional implementation of the training unit 1103 can refer to the relevant descriptions of implementation methods 1-4 in the above S3, which will not be repeated here.

Please refer to FIG. 11B , which is a schematic diagram of the structure of a language translation device provided in an embodiment of the present application. The language translation device can be applied to the execution device 12 in FIG. 1A above, and the language translation device may include:

The receiving unit 1201 is used to receive data to be translated, where the data to be translated is source language data;

The translation unit 1202 is used to translate the data to be translated into its corresponding target language data. The translation unit 1202 includes a machine translation model, which is a machine translation model trained by any of the machine translation model training methods of FIG. 1B , or FIG. 6 , or FIG. 8 , or FIG. 9 .

Please refer to Figure 12, Figure 12 is a structural diagram of a language translation device provided by an embodiment of the present application. As shown in Figure 12, the language translation device 1000 can correspond to the second device 120 or the first device 110 in the computer system shown in Figure 1A above, and the language translation device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned language translation device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Wherein, the communication bus 1002 is used to realize the connection communication between these components. Wherein, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1004 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. 12 , the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

In the language translation device 1000 shown in FIG12, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to implement the translation task performed by the model trained by the training method of any machine translation model shown in FIG1B, or FIG6, or FIG8, or FIG9, and the training method is not described in detail here. In addition, the description of the beneficial effects of using the same method is not described in detail.

In addition, it should be noted that: the embodiment of the present application also provides a kind of computer-readable storage medium, and the computer program executed by the language translation device 1000 mentioned above is stored in the computer-readable storage medium, and the computer program includes program instructions, when the processor executes the program instructions, the above-mentioned language translation method can be executed, therefore, it will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For the technical details not disclosed in the computer-readable storage medium embodiment involved in the present invention, please refer to the description of the method embodiment of the present application.

Please refer to Figure 13, which is a structural diagram of a training device for a machine translation model provided in an embodiment of the present application. As shown in Figure 13, the training device 2000 of the machine translation model may include: a processor 2001, a network interface 2004 and a memory 2005. In addition, the training device 2000 of the above-mentioned machine translation model may also include: a user interface 2003, and at least one communication bus 2002. Among them, the communication bus 2002 is used to realize the connection communication between these components. Among them, the user interface 2003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 2003 may also include a standard wired interface and a wireless interface. The network interface 2004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 2004 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 2005 may also be at least one storage device located away from the aforementioned processor 2001. As shown in FIG. 13 , the memory 2005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

In the training device 2000 of the machine translation model as shown in FIG13 , the network interface 2004 can provide a network communication function; the user interface 2003 is mainly used to provide an input interface for the user, and can include a display screen, which can display the results obtained by the instructions executed by the processor 2001, such as displaying the progress of model training and displaying the translation results of the data to be translated; and the processor 2001 can be used to call the device control application stored in the memory 2005 to implement the training method of any of the machine translation models in FIG1B , or FIG6 , or FIG8 , or FIG9 , and the relevant description is not repeated here. In addition, the description of the beneficial effects of using the same method is not repeated.

It should be understood that the machine translation model training device 2000 described in the embodiment of the present application can execute the training method of any machine translation model in the above-mentioned FIG. 1B, or FIG. 6, or FIG. 8, or FIG. 9, which will not be described in detail here. In addition, the description of the beneficial effects of using the same method will not be repeated.

In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores the computer program executed by the training device 2000 of the machine translation model mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the training method of any one of the machine translation models of Figure 1B, or Figure 6, or Figure 8, or Figure 9, so the relevant description is not repeated here. In addition, the description of the beneficial effects of adopting the same method is not repeated. For the technical details not disclosed in the computer-readable storage medium embodiment involved in the present application, please refer to the description of the method embodiment of the present application.

A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

The above disclosure is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims (14)

1. A method for training a machine translation model, comprising: Acquire a bilingual parallel database, the bilingual parallel database comprising a plurality of groups of bilingual parallel sentence pairs, the bilingual parallel sentence pairs being content-aligned data consisting of source language data and target language data, the bilingual parallel database comprising a first bilingual parallel database; Dividing the plurality of groups of bilingual parallel sentence pairs in the first bilingual parallel database into those derived from source language data and those derived from target language data, wherein the target language data in the bilingual parallel sentence pairs derived from the source language data are translated by a human translator based on the source language data, and the source language data in the bilingual parallel sentence pairs derived from the target language data are translated by a human translator based on the target language data; Training a first machine translation model using the source language data, wherein the first machine translation model is used to translate the source language into a target language; The method of dividing the plurality of groups of bilingual parallel sentence pairs in the first bilingual parallel database into those derived from source language data and those derived from target language data comprises: obtaining parallel sentence pairs to be processed from the first bilingual parallel database; determining a first probability that the source language data in the parallel sentence pairs to be processed are derived from the source language according to the source language data in the parallel sentence pairs to be processed; determining a second probability that the target language data in the parallel sentence pairs to be processed are derived from the target language according to the content of the target language data in the parallel sentence pairs to be processed; determining a data type of the parallel sentence pairs to be processed according to a deviation between the first probability and the second probability; the data type comprises the data derived from the source language data and the data derived from the target language data. 2. The method according to claim 1, characterized in that the step of determining the data type of the parallel sentence pair to be processed according to the deviation between the first probability and the second probability specifically comprises: Determine a score of the parallel sentence pair to be processed according to a deviation between the first probability and the second probability, wherein the score is used to determine a data type of the parallel sentence pair to be processed; When the score is greater than a target threshold, determining that the parallel sentence pair to be processed is derived from source language data; When the score is less than the target threshold, it is determined that the parallel sentence pair to be processed is derived from the target language data. 3. The method according to claim 1 or 2, characterized in that determining, according to the source language data in the parallel sentence pairs to be processed, a first probability that the source language data in the parallel sentence pairs to be processed are derived from a source language comprises: inputting the source language data in the parallel sentence pairs to be processed into a first language model to determine the first probability, wherein the first language model is used to determine the probability that the source language data in the parallel sentence pairs to be processed appear in the source language, and the first language model is a model obtained by training a source language monolingual database; Determining a second probability that the target language data in the parallel sentence pair to be processed is derived from a target language according to the content of the target language data in the parallel sentence pair to be processed includes: inputting the target language data in the parallel sentence pair to be processed into a second language model to determine the second probability, wherein the second language model is used to determine the probability that the target language data in the parallel sentence pair to be processed appears in the target language, and the second language model is a model trained by a target language monolingual database. 4. The method according to any one of claims 1-2, characterized in that before training the first machine translation model using the source language data, the method further comprises: An initial machine translation model is trained using the bilingual parallel database to obtain the first machine translation model. 5. The method according to any one of claims 1-2, characterized in that before training the first machine translation model using the source language data, the method further comprises: Acquire a monolingual database, wherein the monolingual database includes a plurality of original texts in a target language; Inputting each original text in the monolingual database into a second machine translation model to obtain a source language translation text corresponding to each original text, wherein the second machine translation model is obtained by training the first bilingual parallel database, and the second machine translation model is used to translate the target language into the source language; Adding a plurality of groups of pseudo-parallel sentence pairs consisting of the plurality of original texts in the target language and the source language translation texts respectively corresponding to the plurality of original texts in the target language to the bilingual parallel database. 6. The method according to any one of claims 1-2, characterized in that before training the first machine translation model using the source language data, the method further comprises: Acquire a monolingual database, wherein the monolingual database includes a plurality of original texts in a source language; Inputting each original text in the monolingual database into a third machine translation model to obtain a target language translation text corresponding to each original text, wherein the third machine translation model is obtained by training the first bilingual parallel database, and the third machine translation model is used to translate the source language into the target language; Adding a plurality of groups of pseudo-parallel sentence pairs consisting of the plurality of original texts in the source language and the target language translation texts corresponding to the plurality of original texts in the source language to the bilingual parallel database. 7. A language translation method, characterized by comprising: receiving data to be translated, wherein the data to be translated is source language data; The data to be translated is input into a machine translation model to obtain target language data corresponding to the data to be translated, wherein the machine translation model is a first machine translation model trained by the method according to any one of claims 1 to 6. 8. A training device for a machine translation model, comprising: An acquisition unit is used to acquire a bilingual parallel database, wherein the bilingual parallel database includes a plurality of groups of bilingual parallel sentence pairs, wherein the bilingual parallel sentence pairs are content-aligned data consisting of source language data and target language data, and the bilingual parallel database includes a first bilingual parallel database; a dividing unit, configured to divide the plurality of groups of bilingual parallel sentence pairs in the first bilingual parallel database into data derived from source language data and data derived from target language data, wherein the target language data in the bilingual parallel sentence pairs derived from the source language data are translated by a human translator based on the source language data, and the source language data in the bilingual parallel sentence pairs derived from the target language data are translated by a human translator based on the target language data; A training unit, configured to train a first machine translation model using the data derived from the source language, wherein the first machine translation model is used to translate the source language into a target language; Wherein, when the division unit divides the multiple groups of bilingual parallel sentence pairs in the first bilingual parallel database into those derived from source language data and those derived from target language data, the division unit is specifically used to: obtain parallel sentence pairs to be processed from the first bilingual parallel database; determine a first probability that the source language data in the parallel sentence pairs to be processed are derived from the source language according to the source language data in the parallel sentence pairs to be processed; determine a second probability that the target language data in the parallel sentence pairs to be processed are derived from the target language according to the content of the target language data in the parallel sentence pairs to be processed; determine a data type of the parallel sentence pairs to be processed according to a deviation between the first probability and the second probability; the data type includes the data derived from the source language data and the data derived from the target language data. 9. A language translation device, comprising: A receiving unit, configured to receive data to be translated, wherein the data to be translated is source language data; A translation unit is used to translate the data to be translated into its corresponding target language data, wherein the translation unit includes a machine translation model, and the machine translation model is a first machine translation model trained by the method according to any one of claims 1 to 6. 10. A computer device, comprising: one or more processors and one or more memories, wherein the one or more memories are respectively coupled to the one or more processors; the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions; The processor is used to call the computer instruction to execute: the training method of the machine translation model as described in any one of claims 1-6. 11. A computer device, comprising: one or more processors and one or more memories, wherein the one or more memories are respectively coupled to the one or more processors; the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions; The processor is used to call the computer instruction to execute: the language translation method as claimed in claim 7. 12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the training method of the machine translation model according to any one of claims 1 to 6 is executed. 13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the language translation method according to claim 7 is executed. 14. A computer program product, characterized in that the computer program product comprises computer instructions, which, when executed by a processor, implement the training method for a machine translation model as described in any one of claims 1 to 6, or implement the language translation method as described in claim 7.