CN113821633A - Text processing method, device and computer storage medium - Google Patents

Abstract

The embodiment of the present application proposes a text processing method, device and computer storage medium, the method includes: obtaining text to be processed; calling a first classification network to encode and represent secondary categories, category matching parameters, and the to-be-processed text The semantic coding indicates that processing is performed to obtain the classification information of the secondary category of the text to be processed; based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, determine the The classification result of the processed text, the classification information of the first-level category of the text to be processed is obtained by processing the text to be processed through the second classification network, which can improve the accuracy of the hierarchical classification of the text.

Description

Text processing method, device and computer storage medium

technical field

The present application relates to the field of computer technology, and in particular, to a text processing method, device, and computer storage medium.

Background technique

Hierarchical Multi-Label Classification (HMC) is an important multi-classification task in the field of natural language processing/computer vision. The granularity of the lower level is finer. For example, the upper-level category is "games", and the lower-level categories are "mini games", "mobile games", and "end games".

At present, hierarchical classification is usually used as a basic multi-classification task, that is, the lower-level category is directly predicted, and the upper-level category is obtained by reverse inference through the lower-level category. The upper-level category is called "game", but direct prediction of the lower-level category often faces the problem of less training sample sets, and because the number of lower-level categories is too large, it is often impossible to classify accurately.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a text processing method, device, and computer storage medium, which can improve the accuracy of hierarchical classification.

On the one hand, an embodiment of the present application provides a text processing method, the method includes:

Get the pending text;

Invoking the first classification network to process the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed, to obtain the classification information of the secondary category of the text to be processed;

The classification result of the text to be processed is determined based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, and the classification information of the primary category of the text to be processed is obtained through the first It is obtained by processing the text to be processed by the binary classification network.

On the one hand, an embodiment of the present application provides a text processing apparatus, and the apparatus includes:

Get module, used to get the text to be processed;

a processing module, configured to call the first classification network to process the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed, to obtain the classification information of the secondary category of the text to be processed;

The processing module is further configured to determine the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed. The classification information of the category is obtained by processing the text to be processed through the second classification network.

In one aspect, an embodiment of the present application provides a computer device, including: a processor, a memory, and a communication interface, wherein the processor, the memory, and the communication interface are connected to each other, and the processor is adapted to execute a computer program , the memory stores a computer program, and when the computer program is executed by the processor, the above-mentioned text processing method is implemented.

In one aspect, an embodiment of the present application provides a computer storage medium, where a computer program is stored in the computer storage medium, and the computer program is adapted to be loaded by a processor and execute the above text processing method.

In one aspect, an embodiment of the present application provides a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer storage medium. The processor of the computer device reads the computer instructions from the computer storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned text processing method.

In the embodiment of the present application, the second classification network is first called to process the text to be processed to obtain the semantic coding representation of the to-be-processed text and the classification information of the first-level category of the to-be-processed text, and then the first classification network is called to classify the second-level category The coding representation, the category matching parameters and the semantic coding representation of the text to be processed are processed to obtain the classification information of the secondary category of the text to be processed, and finally based on the classification information of the secondary category and the primary category of the text to be processed The classification information can determine the classification result of the text to be processed; the text processing method can obtain the matching result between the text to be processed and the secondary category through the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed , according to the matching result, the secondary category of the text to be processed can be accurately obtained, so that the secondary category of the text can be obtained based on the matching method, and combined with the classification information of the primary category of the text to be processed, the text can be improved Accuracy of hierarchical classification.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic diagram of the architecture of a computer device provided by an exemplary embodiment of the present application;

2 is a schematic flowchart of a text processing method provided by an exemplary embodiment of the present application;

3 is a schematic flowchart of a text processing method provided by another exemplary embodiment of the present application;

4 is a schematic flowchart of a text processing method provided by another exemplary embodiment of the present application;

5 is a schematic flowchart of a text processing method provided by another exemplary embodiment of the present application;

6 is a schematic structural diagram of a text processing apparatus provided by an exemplary embodiment of the present application;

FIG. 7 is a schematic structural diagram of a computer device provided by another exemplary embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

It should be noted that the descriptions such as "first" and "second" involved in the embodiments of this application are only for description purposes, and should not be understood as indicating or implying their relative importance or implicitly indicating the indicated technology number of features. Therefore, technical features defined with "first" and "second" may expressly or implicitly include at least one of the features.

In order to achieve the purpose of improving the accuracy of the hierarchical classification of text, an embodiment of the present application proposes a text processing method based on cloud technology.

Artificial intelligence (Artificial Intelligence, AI) technology is a comprehensive discipline, involving a wide range of fields, both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, cloud storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

Natural Language Processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, the language that people use on a daily basis, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

Machine Learning (ML) is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other technologies.

With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields. The process of determining the classification result of the text in the embodiment of the present application involves technologies such as natural language processing and machine learning of artificial intelligence. Specifically, the following embodiments are used. Be explained.

In order to better understand the text processing method, device, and computer storage medium provided by the embodiments of the present application, the following first describes the architecture of the text processing system to which the embodiments of the present application are applicable. Please refer to FIG. 1. FIG. 1 is a schematic structural diagram of a text processing system provided by an exemplary embodiment of the present application. As shown in FIG. 1 , the text processing system may specifically include a terminal device 101 and a server 102, and the terminal device 101 and the server 102 are connected through a network, for example, through a wireless network connection.

The terminal equipment 101 is also referred to as a terminal (Terminal), user equipment (UE), access terminal, subscriber unit, mobile device, user terminal, wireless communication device, user agent, or user equipment. The terminal device can be a smart TV, a handheld device with wireless communication functions (such as a smart phone, a tablet computer), a computing device (such as a personal computer (PC), a vehicle-mounted device, a wearable device, or other smart devices, etc., but Not limited to this.

The server 102 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, Cloud servers for basic cloud computing services such as middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

As shown in FIG. 2, FIG. 2 is a schematic flowchart of a text processing method provided by an exemplary embodiment of the present application. The server 102 may call the semantic encoder included in the second classification network to process the input text to be processed to obtain the text to be processed. Processing the semantic coding representation of the text, and using the included classification layer to process the semantic coding representation of the text to be processed to obtain the classification information of the primary category of the text to be processed, and further, calling the first classification network to encode the secondary category Representation, category matching parameters and semantic coding representation of the text to be processed are processed to obtain the classification information of the secondary category of the text to be processed, based on the classification information of the secondary category and the classification information of the primary category of the text to be processed The classification result of the text to be processed is obtained. The method can obtain the matching result between the text to be processed and the secondary category through the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed. According to the matching result The secondary category of the text to be processed can be accurately obtained, so that the secondary category of the text can be obtained based on the matching method, and the classification information of the primary category of the text to be processed can be combined, and the accuracy of the hierarchical classification of the text can be improved. .

In one embodiment, the server 102 can use the video title as the text to be processed, and the user can perform hierarchical classification on the video title by inputting the video title on the terminal device 101 installed with the video playing client, thereby completing the video search task; the server 102 The product title can also be used as the text to be processed, and the product title can be classified hierarchically, so as to complete the product search task, and so on.

It can be understood that the schematic diagram of the architecture of the system described in the embodiments of the present application is to more clearly illustrate the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. , with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

A text processing method of the present application is described in detail below.

As shown in FIG. 3, FIG. 3 is a schematic flowchart of a text processing method provided by another exemplary embodiment of the present application. Taking the method applied to the server 102 in FIG. 1 as an example, the method may include the following steps :

S301. Acquire the text to be processed.

The to-be-processed text is the text to be classified, which may be a video title, a commodity title, and the like.

In one embodiment, the text to be processed may be short text with a length less than a preset number of characters (for example, 15 characters or 15 Chinese characters), or may of course be long text. This application does not limit the length of the text to be processed.

S302: Invoke the first classification network to process the secondary category coding representation, the category matching parameters, and the semantic coding representation of the text to be processed, to obtain classification information of the secondary category of the text to be processed.

Since the category itself also has semantic information, this application uses the word coding representation (such as word vector) of the secondary category to form the secondary category coding representation, and uses the secondary category coding representation to enhance the understanding of the secondary category. Recognition ability.

Specifically, the secondary category coding representation is a representation of a matrix structure composed of word coding representations (such as word vectors) of multiple secondary categories in a predefined category hierarchical structure, where the predefined category hierarchical structure includes Two-level categories: a first-level category and a second-level category, where the second-level category is the next-level category of the first-level category. The categories can be "mini game", "mobile game", "mobile phone", among which, if the word codes corresponding to "mini game", "mobile game" and "mobile phone" are respectively expressed as l1, l2, and l3, then the second level The category code is represented as [l1,l2,l3] or as

The semantic encoding representation of the text to be processed is to use the semantic encoder in the second classification network (such as CNN (Convolutional Neural Networks, convolutional neural network), LSTM+ (Long Short-Term Memory, long short-term memory), LSTM+ attention mechanism ( Attention)) is obtained by encoding the text to be processed.

The category matching parameters are obtained by adjusting the initialized matching parameters in the process of obtaining the first classification network and the second classification network. The initialized matching parameters are obtained by initializing the parameters in the predefined matrix structure. .

The server may use the secondary category coding representation to process the category matching parameters and the semantic coding representation of the text to be processed to obtain a matching result between the text to be processed and each secondary category in the predefined category hierarchy structure, The matching result can reflect the probability that the text to be processed belongs to each secondary category. The server can process the matching result between the text to be processed and each secondary category in the above-defined category hierarchy by invoking the second classification network, so as to obtain the classification information of the secondary category of the text to be processed, for example The matching result between the text to be processed and each secondary category is [60, 20, 10, 10], and the processing method is softmax (matching result), then the classification information of the predicted secondary category is [0.6, 0.2 ,0.1,0.1], assuming that the secondary categories are "mini games", "mobile games", "mobile phones", and "Latin dance", the classification information of the predicted secondary categories indicates that the text is predicted to be "mini games", The probabilities of "mobile game", "mobile phone", and "Latin dance" are 0.6, 0.2, 0.1, and 0.1, respectively. At this time, the second classification network includes a classification layer with the softmax function as the classification function.

In this embodiment, the server processes the semantic coding representation and the secondary category coding representation of the text to be processed through the category matching parameters, so that the matching result between the text to be processed and each secondary category can be obtained based on the matching method ( Similar to the matching score), and obtain the secondary category of the text to be processed based on the matching result, which can solve the problem of too few training samples and improve the accuracy of secondary category classification.

S303. Determine the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, where the classification information of the primary category of the text to be processed is: It is obtained by processing the text to be processed through the second classification network.

The classification information of the first-level category of the text to be processed is obtained by inputting the text to be processed into a second classification network, the second classification network includes a semantic encoder and a classifier, and the semantic encoder is used to obtain the semantic encoding features of the text to be processed , the classifier uses the semantic coding features of the text to be processed to obtain the classification information of the first-level category of the text to be processed.

The server can obtain the secondary category of the text to be processed based on the classification information of the secondary category. For example, the secondary category includes "mini game", "mobile game", and "mobile phone", and the classification information of the secondary category is [0.6 ,0.3,0.1], the secondary category of the text to be processed is "mini games", and the primary category of the text to be processed is obtained based on the classification information of the primary category, for example, the primary category includes "games", "Technology", the classification information of the first-level category is [0.8, 0.2], then the first-level category of the text to be processed is "games", and the combination of the first-level category and the second-level category of the pending text is regarded as the pending text. The classification result of the text, for example, the classification result of the text to be processed is "game/mini game".

In the embodiment of the present application, the second classification network is first called to process the text to be processed to obtain the semantic coding representation of the to-be-processed text and the classification information of the first-level category of the to-be-processed text, and then the first classification network is called to classify the second-level category The coding representation, the category matching parameters and the semantic coding representation of the text to be processed are processed to obtain the classification information of the secondary category of the text to be processed, and finally based on the classification information of the secondary category and the primary category of the text to be processed The classification information can determine the classification result of the text to be processed; the text processing method can obtain the matching result between the text to be processed and the secondary category through the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed , according to the matching result, the secondary category of the text to be processed can be accurately obtained, and at the same time, the accuracy of the hierarchical classification of the text can be improved by combining the classification information of the primary category of the text to be processed.

FIG. 4 is a schematic flowchart of a text processing method provided by another exemplary embodiment of the present application. Taking the method applied to the server 102 in FIG. 1 as an example, the method may include the following steps:

S401. Obtain a training sample set, a secondary category code representation, and an initialized matching parameter.

The training sample set includes multiple texts and the first-level category label and the second-level category tag of each text in the multiple texts, and the second-level category tag is the next-level category tag of the first-level category tag, as follows Table 1 shows a specific example of the training sample set. For example, the first-level category object of the text "jump, teach you how to score 600 points" is marked as "game", and the second-level category object is marked as "small game" ".

Table 1

The initialized matching parameters are obtained by initializing the parameters in the predefined matrix structure.

In one embodiment, obtaining the secondary category code representation includes the following steps:

(1) Determine multiple first-level categories and multiple second-level categories according to the predefined category hierarchy structure.

In order to realize that upper-level categories impose constraints on lower-level categories, the present application predefines a category hierarchy structure, and the predefined category hierarchy structure includes two-level categories: a first-level category and a second-level category, wherein the second-level category A category is a category at the next level of a first-level category. For example, a predefined category hierarchy can be defined as: the first-level category includes "sports", "games", "entertainment", and the second-level categories under "sports" include "football", "running", "games" The secondary categories under "Games" and "Mobile Games", and the secondary categories under "Entertainment" include "Music" and "Movies".

(2) Obtain word coding representation of each secondary category in multiple secondary categories.

Perform word encoding processing (such as word2vec, one-hot encoding, etc.) on each of the multiple second-level categories to obtain the word-encoded representation of each second-level category.

(3) Determine the coded representation of the secondary category according to the coded representation of each secondary category in the multiple secondary categories.

According to the word coding representation of each secondary category, the secondary category coding representation is obtained. For example, the word coding corresponding to the secondary category is represented as l1, l2, and l3, and the secondary category coding is represented as [l1, l2, l3] or expressed as

S402. Use the initialized first neural network to process each text included in the training sample set to obtain the semantic coding representation of each text and the classification information of the predicted first-level category.

In one embodiment, the initialized first neural network includes an initialized semantic encoder (eg, CNN, LSTM+, LSTM+Attention), and an initialized classifier including a classification layer with a softmax function as the classification function.

The server encodes the semantic encoder initialized for each text input included in the training sample set, obtains the semantic encoding representation of each text, and further classifies the input-initialized classifier for the semantic encoding representation of each text to obtain a prediction The classification information of the first-level category of the The classification information of the first-level category indicates that the probabilities of text prediction as "game", "technology", and "dance" are 0.8, 0.1, and 0.1, respectively.

S403. Use the initialized second neural network to process the secondary category coding representation, the initialized matching parameters, and the semantic coding representation of each text to obtain the predicted secondary category for each text Purpose classification information.

In one embodiment, the initialized second neural network includes a classification layer that uses a softmax function as a classification function, and encodes representations of secondary categories, initialized matching parameters, and each classification layer using the classification layer included in the initialized second neural network. When processing the semantic coding representation of each text to obtain the classification information of the predicted secondary category of each text, as shown in the following formula (1), the server can use the semantic coding representation of each text to multiply the initialized matching parameters Multiply by the secondary category code representation to obtain the matching result between each text and each secondary category in the above predefined category hierarchy.

Results=L1 _emb *W _ini *f _{l2_cls} (1)

为匹配结果，

为每个文本的语义编码表示，

为初始化的匹配参数，

为二级类目编码表示，l2_num为二级类目的数量，d×1为二级类目的词编码表示(该词编码表示为词向量)对应的维度，1×e为每个文本的语义编码表示对应的维度。in,

For matching results,

Semantic encoding representation for each text,

is the initialized matching parameter,

is the secondary category coding representation, l2_num is the number of secondary categories, d×1 is the dimension corresponding to the secondary category word coding representation (the word coding is represented as a word vector), and 1×e is the size of each text. The semantic encoding represents the corresponding dimension.

It can be understood that the matrix structure of the initialized matching parameters is e×d, so the corresponding dimension (1×e) needs to be represented according to the semantic encoding of each text, and the corresponding dimension (d) needs to be represented by the word encoding of the secondary category. ×1) to determine the matrix size corresponding to the initialized matching parameters, and because the initialized matching parameters are obtained by initializing the parameters in the predefined matrix structure, the size of the predefined matrix structure is based on each text. The semantic encoding represents the corresponding dimension, and the word encoding of the secondary category represents that the corresponding dimension is determined.

In one embodiment, as shown in the following formula (2), the server may also use the secondary category code representation multiplied by the initialized matching parameter multiplied by the semantic code representation of each text to obtain each text and the above-defined predefined Match results between each second-level category in the category hierarchy.

Results=f _{l2_cls} *W _ini *L1 _emb (2)

为匹配结果，

为每个文本的语义编码表示，

为初始化的匹配参数，

为二级类目编码表示，l2_num为二级类目的数量，1×d为二级类目的词编码表示(该词编码表示为词向量)对应的维度，e×1为每个文本的语义编码表示对应的维度。in,

For matching results,

Semantic encoding representation for each text,

is the initialized matching parameter,

is the secondary category coding representation, l2_num is the number of secondary categories, 1×d is the dimension corresponding to the secondary category word coding representation (the word coding is represented as a word vector), and e×1 is the size of each text. The semantic encoding represents the corresponding dimension.

Further, the server uses the classification layer included in the initialized second neural network to process the matching result between each text and each secondary category in the above-mentioned predefined category hierarchy to obtain the predicted secondary category. For example, the matching result is [60, 20, 10, 10], and the processing method is softmax (matching result), then the predicted classification information of the secondary category is [0.6, 0.2, 0.1, 0.1], assuming two The first-level categories are "mini games", "mobile games", "mobile phones", and "Latin dance", then the predicted classification information of the second-level categories indicates that the text is predicted to be "mini games", "mobile games", "mobile phones" , and the probabilities of "Latin dance" are 0.6, 0.2, 0.1, and 0.1, respectively.

S404. Based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label, the first The neural network, the second neural network and the initialized matching parameters are trained to obtain category matching parameters, a first classification network and a second classification network.

The server can use the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label of each text, to the first neural network, the second neural network and the second-level category label. The initialized matching parameters are trained to obtain category matching parameters, a first classification network and a second classification network.

In one embodiment, the server is training the first neural network, the second neural network and the initialized matching parameters to obtain the category matching parameters, the first classification network and the second classification network, which may include the following steps:

(1) Determine the first loss value of the first neural network based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category tag, and the second-level category tag of each text and the second loss value of the second neural network.

The server determines the first loss value of the first neural network according to the predicted classification information of the first-level category, the first-level category label, and the first loss function of the first neural network for each text. As shown in the following formula (3), it is the first loss function of the first neural network. When the first-level category is "game", "technology", and "dance", the predicted classification information of the first-level category is [ 0.8, 0.1, 0.1], when the first-level class target label is [1, 0, 0] (indicating that the text belongs to "game"), the first loss value of the first neural network is -log(0.8).

Among them, loss _cls1 is the first loss function of the first neural network, _yi is the first-level category label, a _i is the predicted classification information of the first-level category, and n is the number of first-level categories.

Further, the server determines the second loss value of the second neural network according to the predicted classification information of the secondary category, the secondary category label and the second loss function of the second neural network for each text. As shown in the following formula (4), it is the second loss function of the second neural network. When the second-level category is "mini game", "mobile game", "mobile phone", "Latin dance", the predicted second-level When the classification information of the category is [0.6, 0.2, 0.1, 0.1], and the secondary category label is [1, 0, 0, 0] (indicating that the text belongs to "mini games"), the second loss of the second neural network The value is -log(0.6).

Among them, loss _cls2 is the second loss function of the second neural network, y _j is the label of the secondary category, a _j is the classification information of the predicted secondary category, and m is the number of secondary categories.

(2) Determine the total loss value according to the first loss value and the second loss value.

In order to ensure the consistency of the classification results, that is, to ensure the consistency between the second-level category and the first-level category of the text (in this case, the second-level category is the category at the next level of the first-level category), this application determines the total The third loss value is introduced in the loss value, and the third loss value is obtained through the constraint loss function between the first-level category and the second-level category.

In one embodiment, the server determines the third loss value according to the first loss value, the second loss value, and a constrained loss function between the primary category and the secondary category.

Specifically, in order to ensure the consistency of the classification results, and because the first-level category is always easy to classify the second-level category, this application adds a constraint loss function between the first-level category and the second-level category to ensure that the text belongs to The probability of the first-level category is always greater than the probability of belonging to the second-level category. For example, the probability that the text is the first-level category "game" is 0.8, then the probability of the second-level category "mini game" is less than 0.8 and greater than or equal to 0. The constraint loss function is shown in the following formula (5):

loss _cls3 =max(0,λ+loss _cls2 -loss _cls1 ) (5)

Among them, λ is a custom parameter used to limit the error range of the first loss value and the second loss value, and loss _cls3 is a constraint loss function.

Further, the server may determine the total loss function according to the respective weight coefficients of the first loss function, the second loss function, and the constraint loss function, and input the first loss value, the second loss value, and the third loss value into the total loss function to obtain the total loss function. loss value. The total loss function is shown in the following formula (6):

Among them, q is the number of texts included in the training sample set, loss _all is the total loss function, λ ₁ , λ ₂ , λ ₃ are the respective weight coefficients of the first loss function, the second loss function and the constraint loss function, which are custom parameter.

It can be understood that when the second loss value is greater than the first loss value + λ, the probability that the text is a first-level category is less than that of the second-level category. For example, if λ is 0, the first-level category is "game", "Technology", "Dance", when the predicted classification information of the first-level category is [0.6, 0.2, 0.2], and the first-level category tag is [1, 0, 0] (indicating that the text belongs to "game"), the first The first loss value is -log(0.6); the secondary categories are "mini game", "mobile game", "mobile phone", "Latin dance", and the predicted classification information of the secondary category is [0.8, 0.1, 0.1 ,0], when the secondary category target is marked as [1,0,0,0] (indicating that the text belongs to a "mini game"), the second loss value is -log(0.8), and the probability that the text is a "game" at this time is 0.6, and the probability of being a "mini game" is 0.8. At this time, the second loss value (-log(0.8)) is greater than the first loss value (-log(0.6)), and the text is a first-level category ("game" ) is less than the probability of the second-level category (“mini game”), and because the probability that the text belongs to the first-level category is always greater than that of the second-level category, the classification is inaccurate, and the classification results are not guaranteed. The application can increase the total loss value by calculating the constraint loss function so that the third loss value at this time is a positive value, slow down the convergence speed of the total loss function to constrain the consistency of the classification results, and also make the subsequent training get The category matching parameters, the first classification network and the second classification network can achieve accurate hierarchical classification of text.

Through this embodiment, a constraint loss function is introduced into the total loss function, so that the total loss value can constrain the consistency of the classification results, and the upper and lower layer constraints in the category hierarchy structure are effectively utilized.

(3) Using the total loss value to adjust the network parameters of the first neural network, the network parameters of the second neural network and the initialized matching parameters, when the total loss value satisfies the convergence condition, the training obtains the category matching parameters, the first classification network and a second classification network.

The server uses the total loss value to adjust the network parameters of the first neural network, the network parameters of the second neural network, and the initialized matching parameters. When the total loss value satisfies the convergence condition, or the number of training times reaches the preset value (manually set) When , the adjustment of the network parameters of the first neural network, the network parameters of the second neural network and the initialized matching parameters is stopped, and the category matching parameters are obtained according to the adjusted initialized matching parameters. The network obtains the first classification network, and obtains the second classification network according to the adjusted second neural network.

In the embodiment of the present application, the category matching parameters are obtained by training the initialized first neural network, the initialized second neural network, and the initialized matching parameters. The first classification network and the second classification network can be used for Accurate hierarchical classification of text.

FIG. 5 is a schematic flowchart of a text processing method provided by another exemplary embodiment of the present application. Taking the method applied to the server 102 in FIG. 1 as an example, the method may include the following steps:

S501. Acquire the text to be processed.

S502. Invoke the semantic encoder of the second classification network to process the text to be processed to obtain a semantic encoding representation of the text to be processed.

The semantic encoder of the second classification network is used to obtain the semantic information of the text to be processed, and the semantic encoder can be CNN, LSTM, LSTM+Attention, etc.

A semantically encoded representation of the text to be processed can be obtained by inputting the text to be processed into the semantic encoder of the second classification network.

S503. Invoke the classifier of the second classification network to process the semantic coding representation of the text to be processed, and obtain classification information of the first-level category of the text to be processed.

The classifier of the second classification network includes a classification layer with the softmax function as the classification function, and the classification information of the first-level category of the text to be processed can be obtained by inputting the semantic coding representation of the text to be processed into the classification layer of the second classification network, For example, the first-level category is "game", "technology", and "dance", and the classification information of the first-level category is [0.8, 0.1, 0.1], then the classification information of the first-level category indicates that the text is predicted as "game", The probabilities of "technology" and "dance" are 0.8, 0.1, and 0.1, respectively.

S504. Use category matching parameters to process the secondary category coded representation and the semantic coding representation of the text to be processed, and determine a matching result between each secondary category and the text to be processed.

In one embodiment, when the server uses the category matching parameter to process the secondary category coding representation and the semantic coding representation of the text to be processed, as shown in the following formula (7), the server can use the semantic coding of the text to be processed. The representation is multiplied by the category matching parameter multiplied by the secondary category code representation to obtain the matching result between the text to be processed and each secondary category in the predefined category hierarchy.

Results=L1 _emb *W _end *f _{l2_cls} (7)

为匹配结果，

为待处理文本的语义编码表示，

为目录匹配参数，

为二级类目编码表示，l2_num为二级类目的数量，d×1为二级类目的词编码表示(该词编码表示为词向量)对应的维度，1×e为待处理文本的语义编码表示对应的维度。in,

For matching results,

is a semantically encoded representation of the text to be processed,

match arguments for directories,

is the coding representation of the secondary category, l2_num is the number of secondary categories, d×1 is the dimension corresponding to the word coding representation of the secondary category (the word coding is represented as a word vector), and 1×e is the size of the text to be processed. The semantic encoding represents the corresponding dimension.

In one embodiment, when the server uses the category matching parameter to process the secondary category coding representation and the semantic coding representation of the text to be processed, as shown in the following formula (8), the secondary category coding representation can be used to represent the multiplication The semantic coding representation of the text to be processed is multiplied by the category matching parameter to obtain a matching result between the text to be processed and each second-level category in the above-defined category hierarchy structure.

Results=f _{l2_cls} *W _end *L1 _emb (8)

为匹配结果，

为待处理文本的语义编码表示，

为类目匹配参数，

为二级类目编码表示，l2_num为二级类目的数量，1×d为二级类目的词编码表示(该词编码表示为词向量)对应的维度，e×1为待处理文本的语义编码表示对应的维度。in,

For matching results,

is a semantically encoded representation of the text to be processed,

match parameters for categories,

is the coding representation of the secondary category, l2_num is the number of secondary categories, 1×d is the dimension corresponding to the word coding representation of the secondary category (the word coding is represented as a word vector), and e×1 is the size of the text to be processed. The semantic encoding represents the corresponding dimension.

In this embodiment, the server processes the semantic coding representation and the secondary category coding representation of the text to be processed through the category matching parameters, so that the matching result between the text to be processed and each secondary category can be obtained based on the matching method ( similar to match scores).

S505: Invoke the first classification network to process the matching result between each secondary category and the text to be processed, and obtain the classification information of the secondary category of the text to be processed.

The first classification network includes a classification layer with the softmax function as the classification function, and the server obtains the second class of the text to be processed by inputting the matching result between each secondary category and the text to be processed into the classification layer of the first classification network. Purpose classification information, for example, the secondary category is "mini game", "mobile game", "mobile phone", "Latin dance", and the classification information of the secondary category is [0.6, 0.2, 0.1, 0.1], then the second The classification information of the first-level category indicates that the probabilities that the text is predicted to be "mini game", "mobile game", "mobile phone", and "Latin dance" are 0.6, 0.2, 0.1, and 0.1, respectively. In this embodiment, the server uses the first classification network to process the matching results (the first classification network is mainly used for normalization processing), and can obtain the classification information of the secondary category of the text to be processed, which can solve the problem of training The problem of too few sample sets can also improve the accuracy of secondary category classification.

S506. Determine the classification result of the text to be processed based on the classification information of the secondary category of the text to be processed and the classification information of the primary category of the text to be processed.

The server obtains the secondary category of the text to be processed based on the classification information of the secondary category, obtains the primary category of the text to be processed based on the classification information of the primary category, and combines the primary category and secondary category of the text to be processed. The item combination is used as the classification result of the text to be processed.

In the embodiment of the present application, the second classification network is first called to process the text to be processed to obtain the semantic coding representation of the text to be processed and the classification information of the primary category of the text to be processed, and then the category matching parameters encode the secondary category The representation and the semantic coding representation of the text to be processed are processed to obtain the matching result between each secondary category and the text to be processed, and the first classification network is called to process the matching result to obtain the secondary category of the text to be processed. Finally, the classification result of the text to be processed can be determined based on the classification information of the secondary category and the classification information of the primary category of the text to be processed; the text processing method encodes the secondary category through the category matching parameter. Processing with the semantic coding representation of the text to be processed can obtain the matching result between the text to be processed and the secondary category. According to the matching result, the secondary category of the text to be processed can be accurately obtained, so that the matching method can be used to obtain The secondary category of the text to be processed, combined with the classification information of the primary category of the text to be processed, can improve the accuracy of the hierarchical classification of the text.

The methods of the embodiments of the present application are described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of the present application, correspondingly, the devices of the embodiments of the present application are provided below. Please refer to FIG. 6. FIG. 6 is a schematic structural diagram of a text processing apparatus provided by an exemplary embodiment of the present application. The apparatus 60 may include:

an obtaining module 601, used for obtaining the text to be processed;

The processing module 602 is configured to call the first classification network to process the secondary category coding representation, category matching parameters and the semantic coding representation of the text to be processed, and obtain the classification information of the secondary category of the text to be processed ;

The processing module 602 is further configured to determine the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, and the primary category of the text to be processed. The target classification information is obtained by processing the text to be processed through the second classification network.

In one embodiment, the above obtaining module 601 is further configured to:

Determine a plurality of first-level categories and a plurality of second-level categories according to a predefined category hierarchy, and the second-level categories are the categories of the next level of the first-level categories;

obtaining the word code representation of each secondary category in the multiple secondary categories;

The secondary category coding representation is determined according to the word coding representation of each secondary category in the plurality of secondary categories.

In one embodiment, the above-mentioned processing module 602 is further configured to:

Use category matching parameters to process the secondary category coding representation and the semantic coding representation of the text to be processed, and determine a matching result between each secondary category and the text to be processed;

The first classification network is invoked to process the matching result between each secondary category and the text to be processed, to obtain the classification information of the secondary category of the text to be processed.

In one embodiment, the above-mentioned processing module 602 is further configured to:

invoking the semantic encoder of the second classification network to process the text to be processed to obtain a semantic encoding representation of the text to be processed;

The classifier of the second classification network is invoked to process the semantically encoded representation of the text to be processed to obtain the classification information of the primary category of the text to be processed.

In one embodiment, the above obtaining module 601 is further configured to:

Obtaining a training sample set, a secondary category code representation and initialization matching parameters, the training sample set includes a plurality of texts and a primary category label and a secondary category label of each text in the plurality of texts;

The above-mentioned processing module 602 is also used for:

Use the initialized first neural network to process each text included in the training sample set to obtain the semantic coding representation of each text and the classification information of the predicted first-level category;

The second-level category encoding representation, the initialized matching parameters, and the semantic encoding representation of each text are processed by using the initialized second neural network to obtain the predicted second-level category object for each text. Classified information;

Based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label, the first neural network , the second neural network and the initialized matching parameters are trained to obtain category matching parameters, a first classification network and a second classification network.

In one embodiment, the above-mentioned processing module 602 is further configured to:

The first neural network is determined based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label of each text The first loss value of and the second loss value of the second neural network;

determining a total loss value according to the first loss value and the second loss value;

Using the total loss value to adjust the network parameters of the first neural network, the network parameters of the second neural network and the initialized matching parameters;

When the total loss value satisfies the convergence condition, the category matching parameters, the first classification network and the second classification network are obtained through training.

In one embodiment, the above-mentioned processing module 602 is further configured to:

Determine the first loss value of the first neural network according to the predicted first-level category classification information of each text, the first-level category label, and the first loss function of the first neural network;

A second loss value of the second neural network is determined according to the predicted classification information of the secondary category of each text, the secondary category label, and the second loss function of the second neural network.

In one embodiment, the above-mentioned processing module 602 is further configured to:

determining a third loss value according to the first loss value, the second loss value and the constraint loss function between the primary category and the secondary category;

A total loss value is determined based on the first loss value, the second loss value, the third loss value, and the respective weight coefficients of the first loss function, the second loss function, and the constraint loss function .

In the embodiment of the present application, the second classification network is first called to process the text to be processed to obtain the semantic coding representation of the to-be-processed text and the classification information of the first-level category of the to-be-processed text, and then the first classification network is called to classify the second-level category The coding representation, the category matching parameters and the semantic coding representation of the text to be processed are processed to obtain the classification information of the secondary category of the text to be processed, and finally based on the classification information of the secondary category and the primary category of the text to be processed The classification information can determine the classification result of the text to be processed; the text processing method can obtain the matching result between the text to be processed and the secondary category through the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed , according to the matching result, the secondary category of the text to be processed can be accurately obtained, and at the same time, the accuracy of the hierarchical classification of the text can be improved by combining the classification information of the primary category of the text to be processed.

FIG. 7 is a schematic structural diagram of a computer device provided by an exemplary embodiment of the present application. The computer device 70 includes at least a processor 701 , a memory 702 and a communication interface 703 . Wherein, the processor 701, the memory 702 and the communication interface 703 may be connected by a bus or other means. Communication interface 703 may be used to receive or transmit data. A computer program is stored in the memory 702, and the computer program includes computer instructions. The processor 701 is used to execute computer instructions included in a computer program. The processor 701 (or called CPU (Central Processing Unit, central processing unit)) is the computing core and the control core of the computer device 70, which is suitable for implementing one or more computer instructions, specifically suitable for loading and executing one or more computer instructions. The instruction thus implements the corresponding method flow or corresponding function. In a specific implementation, the computer instructions in the memory 702 are loaded by the processor 701 and perform the following steps:

Get the pending text;

Invoking the first classification network to process the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed, to obtain the classification information of the secondary category of the text to be processed;

The classification result of the text to be processed is determined based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, and the classification information of the primary category of the text to be processed is obtained through the first It is obtained by processing the text to be processed by the binary classification network.

In one embodiment, the above-mentioned processor 701 is further configured to:

Determine a plurality of first-level categories and a plurality of second-level categories according to a predefined category hierarchy, and the second-level categories are the categories of the next level of the first-level categories;

obtaining the word code representation of each secondary category in the multiple secondary categories;

The secondary category coding representation is determined according to the word coding representation of each secondary category in the plurality of secondary categories.

In one embodiment, the above-mentioned processor 701 is further configured to:

Use category matching parameters to process the secondary category coding representation and the semantic coding representation of the text to be processed, and determine a matching result between each secondary category and the text to be processed;

The first classification network is invoked to process the matching result between each secondary category and the text to be processed, to obtain the classification information of the secondary category of the text to be processed.

In one embodiment, the above-mentioned processor 701 is further configured to:

invoking the semantic encoder of the second classification network to process the text to be processed to obtain a semantic encoding representation of the text to be processed;

The classifier of the second classification network is invoked to process the semantically encoded representation of the text to be processed to obtain the classification information of the primary category of the text to be processed.

In one embodiment, the above-mentioned processor 701 is further configured to:

Obtaining a training sample set, a secondary category code representation and initialization matching parameters, the training sample set includes a plurality of texts and a primary category label and a secondary category label of each text in the plurality of texts;

Use the initialized first neural network to process each text included in the training sample set to obtain the semantic coding representation of each text and the classification information of the predicted first-level category;

The second-level category encoding representation, the initialized matching parameters, and the semantic encoding representation of each text are processed by using the initialized second neural network to obtain the predicted second-level category object for each text. Classified information;

Based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label, the first neural network , the second neural network and the initialized matching parameters are trained to obtain category matching parameters, a first classification network and a second classification network.

In one embodiment, the above-mentioned processor 701 is further configured to:

The first neural network is determined based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label of each text The first loss value of and the second loss value of the second neural network;

determining a total loss value according to the first loss value and the second loss value;

Using the total loss value to adjust the network parameters of the first neural network, the network parameters of the second neural network and the initialized matching parameters;

When the total loss value satisfies the convergence condition, the category matching parameters, the first classification network and the second classification network are obtained through training.

In one embodiment, the above-mentioned processor 701 is further configured to:

determining a third loss value according to the first loss value, the second loss value and the constraint loss function between the primary category and the secondary category;

A total loss value is determined based on the first loss value, the second loss value, the third loss value, and the respective weight coefficients of the first loss function, the second loss function, and the constraint loss function .

In the embodiment of the present application, the second classification network is first called to process the text to be processed to obtain the semantic coding representation of the to-be-processed text and the classification information of the first-level category of the to-be-processed text, and then the first classification network is called to classify the second-level category The coding representation, the category matching parameters and the semantic coding representation of the text to be processed are processed to obtain the classification information of the secondary category of the text to be processed, and finally based on the classification information of the secondary category and the primary category of the text to be processed The classification information can determine the classification result of the text to be processed; the text processing method can obtain the matching result between the text to be processed and the secondary category through the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed , according to the matching result, the secondary category of the text to be processed can be accurately obtained, and at the same time, the accuracy of the hierarchical classification of the text can be improved by combining the classification information of the primary category of the text to be processed.

In the embodiments of the present application, the embodiments of the present application further provide a computer storage medium (Memory), where the computer storage medium is a memory device in the computer device 70 for storing programs and data. It can be understood that, the computer storage medium here may include both the built-in storage medium in the computer device 70 , and certainly also the extended storage medium supported by the computer device 70 . The computer storage medium provides storage space in which the operating system of the computer device 70 is stored. In addition, one or more computer instructions suitable for being loaded and executed by the processor 701 are also stored in the storage space, and these computer instructions may be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it can also be at least one memory located far away from the aforementioned processor. 701 computer storage medium.

One or more embodiments of the present application also provide a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer storage medium. The processor of the computer device reads the computer instructions from the computer storage medium, and the processor executes the computer instructions, so that the computer device performs the steps performed in the embodiments of the above methods.

The above examples only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (10)

1. a text processing method, is characterized in that, described method comprises: Get the pending text; Invoking the first classification network to process the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed, to obtain the classification information of the secondary category of the text to be processed; The classification result of the text to be processed is determined based on the classification information of the secondary category and the classification information of the primary category of the text to be processed, and the classification information of the primary category of the text to be processed is obtained through the first It is obtained by processing the text to be processed by the binary classification network. 2. The method according to claim 1, wherein the method further comprises: Determine a plurality of first-level categories and a plurality of second-level categories according to a predefined category hierarchy, and the second-level categories are the categories of the next level of the first-level categories; obtaining the word code representation of each secondary category in the multiple secondary categories; The secondary category coding representation is determined according to the word coding representation of each secondary category in the plurality of secondary categories. 3. The method according to claim 2, characterized in that, calling the first classification network to process the secondary category coding representation, category matching parameters, and the semantic coding representation of the text to be processed, to obtain the Classification information of the secondary category of the text to be processed, including: Use category matching parameters to process the secondary category coding representation and the semantic coding representation of the text to be processed, and determine a matching result between each secondary category and the text to be processed; The first classification network is invoked to process the matching result between each secondary category and the text to be processed, to obtain the classification information of the secondary category of the text to be processed. 4. The method according to any one of claims 1 to 3, wherein the determining the said Before the classification result of the text to be processed, the method further includes: invoking the semantic encoder of the second classification network to process the text to be processed to obtain a semantic encoding representation of the text to be processed; The classifier of the second classification network is invoked to process the semantically encoded representation of the text to be processed to obtain the classification information of the primary category of the text to be processed. 5. The method according to claim 1, wherein the method further comprises: Obtaining a training sample set, a secondary category code representation and initialization matching parameters, the training sample set includes a plurality of texts and a primary category label and a secondary category label of each text in the plurality of texts; Use the initialized first neural network to process each text included in the training sample set to obtain the semantic coding representation of each text and the classification information of the predicted first-level category; The second-level category encoding representation, the initialized matching parameters, and the semantic encoding representation of each text are processed by using the initialized second neural network to obtain the predicted second-level category object for each text. Classified information; Based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label, the first neural network , the second neural network and the initialized matching parameters are trained to obtain category matching parameters, a first classification network and a second classification network. 6 . The method according to claim 5 , wherein the classification information of the first-level category predicted based on the each text, the classification information of the predicted second-level category, the first-level category label and the secondary category label, train the first neural network, the second neural network and the initialized matching parameters to obtain category matching parameters, the first classification network and the second classification network, include: The first neural network is determined based on the predicted classification information of the first-level category, the predicted classification information of the second-level category, the first-level category label, and the second-level category label of each text The first loss value of and the second loss value of the second neural network; determining a total loss value according to the first loss value and the second loss value; Using the total loss value to adjust the network parameters of the first neural network, the network parameters of the second neural network and the initialized matching parameters; When the total loss value satisfies the convergence condition, the category matching parameters, the first classification network and the second classification network are obtained through training. 7 . The method according to claim 6 , wherein the classification information of the first-level category predicted based on the each text, the classification information of the predicted second-level category, the first-level category The label and the secondary class target label determine the first loss value of the first neural network and the second loss value of the second neural network, including: Determine the first loss value of the first neural network according to the predicted first-level category classification information of each text, the first-level category label, and the first loss function of the first neural network; A second loss value of the second neural network is determined according to the predicted classification information of the secondary category of each text, the secondary category label, and the second loss function of the second neural network. 8. The method according to claim 6 or 7, wherein the determining a total loss value according to the first loss value and the second loss value comprises: determining a third loss value according to the first loss value, the second loss value and the constraint loss function between the primary category and the secondary category; A total loss value is determined based on the first loss value, the second loss value, the third loss value, and the respective weight coefficients of the first loss function, the second loss function, and the constraint loss function . 9. A text processing device, wherein the device comprises: Get module, used to get the text to be processed; a processing module, configured to call the first classification network to process the secondary category coding representation, the category matching parameters and the semantic coding representation of the text to be processed, to obtain the classification information of the secondary category of the text to be processed; The processing module is further configured to determine the classification result of the text to be processed based on the classification information of the secondary category and the classification information of the primary category of the text to be processed. The classification information of the category is obtained by processing the text to be processed through the second classification network. 10 . A computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is adapted to be loaded by a processor and execute the text processing method according to any one of claims 1 to 8 .

Images