CN116502003A - Content audit model training method, device, equipment and storage medium - Google Patents

Abstract

The invention belongs to the technical field of computers, and discloses a content review model training method, device, equipment and storage medium. In the present invention, by obtaining the unlabeled training sample set and the artificially marked sample set, the number of samples in the unlabeled training sample set is greater than the number of samples in the artificially marked sample set; the initial self-supervising and auditing model is trained through the unlabeled training sample set to obtain content auditing Model; the second training of the content moderation model is carried out according to the manual labeling sample set. When training the content review model, a large amount of unlabeled data is used to train the initial self-supervised review model, so that the obtained content review model has strong generalization, and then the content review model is trained twice using the manually labeled sample set , to improve the review accuracy of the content review model, so that the final obtained model not only has the review accuracy but also has strong generalization, which can meet the needs of Internet content review scenarios.

Description

Content audit model training method, device, equipment and storage medium

technical field

The present invention relates to the field of computer technology, in particular to a method, device, equipment and storage medium for training a content review model.

Background technique

In Internet content review scenarios, there are various files and tasks that need to be reviewed. In order to cope with this scenario, the content review model used needs to have sufficient generalization, and the traditional means of enhancing the generalization of the model is Label a large amount of data to train the model. However, in the Internet content review scenario, it is often very difficult to obtain harmful samples, and it is impossible to label a large amount of data. Therefore, how to make the final obtained model generalizable in the Internet content review scenario has become A problem that urgently needs to be solved.

The above content is only used to assist in understanding the technical solution of the present invention, and does not mean that the above content is admitted as prior art.

Contents of the invention

The main purpose of the present invention is to provide a content review model training method, device, equipment and storage medium, aiming to solve the technical problem that it is difficult to obtain a large number of harmful samples in the Internet content review scene in the prior art, resulting in poor model generalization.

To achieve the above object, the present invention provides a method for training a content audit model, said method comprising the following steps:

Obtaining an unlabeled training sample set and a manually labeled sample set, wherein the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set;

training the initial self-supervised review model through the unlabeled training sample set to obtain a content review model;

Perform secondary training on the content review model according to the manually marked sample set.

Optionally, after the step of performing secondary training on the content review model according to the artificially marked sample set, the method further includes:

Use the content review model after the second training as the default content review model;

When receiving the file to be reviewed, obtain the file type corresponding to the file to be reviewed;

Obtaining content parsing rules corresponding to the file type;

Analyzing the file to be reviewed according to the content parsing rules to obtain the content to be reviewed;

Perform content audit on the content to be audited through the preset content audit model.

Optionally, before the step of using the content audit model after the secondary training as the preset content audit model, it may further include:

Obtaining a model verification sample set and a standard judgment result set corresponding to the model verification sample set;

Analyzing the samples in the model verification sample set through the content audit model after secondary training to obtain an audit judgment result set;

determining the model audit accuracy rate according to the standard judgment result set and the audit judgment result set;

If the model review accuracy rate is greater than or equal to the preset accuracy rate threshold, the step of using the content review model after the second training as the preset content review model is performed.

Optionally, the step of determining the model audit accuracy rate according to the standard judgment result set and the audit judgment result set includes:

If the accuracy of the model review is less than the preset accuracy threshold, the manual marking sample set is expanded, and the step of performing secondary training on the content review model according to the manual marking sample set is returned.

Optionally, the step of obtaining the file type corresponding to the file to be reviewed when receiving the file to be reviewed includes:

When receiving the file to be reviewed, read the file suffix of the file to be reviewed;

The file suffix is matched with the suffix set corresponding to each file type in the preset file type table to determine the file type corresponding to the file to be reviewed.

Optionally, the step of obtaining the content parsing rules corresponding to the file type includes:

Searching for corresponding parsing rules in the preset rule type mapping table according to the file type;

The number of rules to detect the found parsing rules;

If the number of rules is greater than the preset number, then obtain the rule priority corresponding to each parsing rule;

sorting the parsing rules from large to small based on the priority of the rules to obtain a sorting result;

The parsing rule ranked first in the sorting result is used as the content parsing rule corresponding to the file type.

Optionally, the step of performing content audit on the content to be audited through the preset content audit model includes:

performing a content analysis on the content to be audited through the preset content audit model, and obtaining a content analysis result;

extracting an audit result identifier from the content analysis result;

Determine whether the file to be reviewed has passed content review according to the review result identifier.

Optionally, the step of determining whether the file to be reviewed has passed the content review according to the review result identification includes:

Searching for the corresponding violation determination result in the preset result identification mapping table according to the audit result identification;

If the violation determination result is that there is no violation content, it is determined that the file to be reviewed has passed the content review.

Optionally, after the step of searching the corresponding violation determination result in the preset result identification mapping table according to the audit result identification, it further includes:

If the violation determination result is that there is violation content, extract the violation content information from the content analysis result;

Constructing a violation confirmation report based on the violation content information and the pending documents;

displaying the violation confirmation report to content reviewers, and receiving the violation confirmation result given by the content reviewer to the violation confirmation report;

If the confirmation result of the violation is a determination of violation, it is determined that the file to be reviewed does not pass the content review.

Optionally, after the step of presenting the violation confirmation report to the content reviewer and receiving the violation confirmation result fed back by the content reviewer to the violation confirmation report, the method further includes:

If the confirmation result of the violation is a misjudgment of violation, it is determined that the file to be reviewed has passed the content review;

Construct a positive labeled sample according to the file to be reviewed, the violation content information, and the violation confirmation result, and add the positive labeled sample to the manual labeled sample set.

Optionally, the step of obtaining an unlabeled training sample set and a manually labeled sample set includes:

Extract sample audit files from the preset sample file library;

Detecting whether there is a corresponding manual label in the sample audit file;

A manually labeled sample set is constructed according to the sample review files with corresponding manual labels, and an unlabeled training sample set is constructed according to the sample review files without corresponding manual labels.

In addition, in order to achieve the above object, the present invention also proposes a content audit model training device, the content audit model training device includes the following modules:

A sample acquisition module, configured to acquire an unlabeled training sample set and a manually marked sample set, wherein the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set;

A training module, which is used to train the initial self-supervised review model through the unlabeled training sample set to obtain a content review model;

A secondary training module is configured to perform secondary training on the content review model according to the artificially marked sample set.

Optionally, the secondary training module is also used to use the content review model after the secondary training as the preset content review model; when receiving the file to be reviewed, obtain the file type corresponding to the file to be reviewed; obtain A content analysis rule corresponding to the file type; analyzing the file to be reviewed according to the content analysis rule to obtain the content to be reviewed; performing content review on the content to be reviewed through the preset content review model.

Optionally, the secondary training module is also used to obtain a model verification sample set and a standard judgment result set corresponding to the model verification sample set; Analyze the sample to obtain an audit judgment result set; determine the model audit accuracy rate according to the standard judgment result set and the audit judgment result set; if the model audit accuracy rate is greater than or equal to the preset accuracy rate threshold, execute the A step of using the content review model after the secondary training as a preset content review model.

Optionally, the secondary training module is further configured to expand the artificially marked sample set if the accuracy of the model review is less than a preset accuracy threshold, and to perform the training according to the manually marked sample set. The content moderation model undergoes secondary training.

Optionally, the secondary training module is also used for reading the file suffix of the file to be reviewed when receiving the file to be reviewed; The suffix set is matched to determine the file type corresponding to the file to be reviewed.

Optionally, the secondary training module is also used to search for corresponding parsing rules in the preset rule type mapping table according to the file type; detect the number of found parsing rules; if the number of rules is greater than the preset If the quantity is set, the rule priority corresponding to each analysis rule is obtained; based on the rule priority, the analysis rules are sorted from large to small to obtain the sorting result; Content parsing rules corresponding to the above file types.

Optionally, the secondary training module is also used to perform content analysis on the content to be reviewed through the preset content review model to obtain a content analysis result; extract the review result identifier from the content analysis result; according to The audit result flag determines whether the file to be audited has passed the content audit.

In addition, in order to achieve the above object, the present invention also proposes a model training device, which includes: a processor, a memory, and a content review model training program stored on the memory and operable on the processor When the content audit model training program is executed by the processor, the steps of the above-mentioned content audit model training method are implemented.

In addition, in order to achieve the above object, the present invention also proposes a computer-readable storage medium, the computer-readable storage medium stores a content audit model training program, and when the content audit model training program is executed, the above-mentioned content is realized. Steps to audit the model training method.

In the present invention, by obtaining the unlabeled training sample set and the artificially marked sample set, the number of samples in the unlabeled training sample set is greater than the number of samples in the artificially marked sample set; the initial self-supervision audit model is trained through the unlabeled training sample set, and the content audit is obtained. Model; the second training of the content moderation model is carried out according to the manual labeling sample set. When training the content review model, a large amount of unlabeled data is used to train the initial self-supervised review model, so that the obtained content review model has strong generalization, and then the content review model is trained twice using the manually labeled sample set , improve the review accuracy of the content review model, so that the final obtained model not only has the review accuracy but also has strong generalization, which can meet the needs of Internet content review scenarios.

Description of drawings

Fig. 1 is a schematic structural diagram of an electronic device in a hardware operating environment involved in the solution of an embodiment of the present invention;

Fig. 2 is a schematic flow chart of the first embodiment of the content review model training method of the present invention;

Fig. 3 is a schematic flow chart of the second embodiment of the content review model training method of the present invention;

Fig. 4 is a schematic flow chart of the third embodiment of the content review model training method of the present invention;

Fig. 5 is a structural block diagram of the first embodiment of the content review model training device of the present invention.

The realization of the purpose of the present invention, functional characteristics and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Referring to FIG. 1 , FIG. 1 is a schematic structural diagram of a model training device in a hardware operating environment involved in the solution of an embodiment of the present invention.

As shown in FIG. 1 , the electronic device may include: a processor 1001 , such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Wherein, the communication bus 1002 is used to realize connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange different components.

As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a content review model training program.

In the electronic device shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in In the model training device, the electronic device calls the content review model training program stored in the memory 1005 through the processor 1001, and executes the content review model training method provided by the embodiment of the present invention.

An embodiment of the present invention provides a method for training a content review model. Referring to FIG. 2 , FIG. 2 is a schematic flowchart of a first embodiment of a method for training a content review model according to the present invention.

In this embodiment, the content audit model training method includes the following steps:

Step S10: Obtain an unlabeled training sample set and a manually labeled sample set, where the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set.

It should be noted that the execution subject of this embodiment may be the model training device, and the model training device may be an electronic device such as a personal computer or a server, or other devices that can realize the same or similar functions. This is not limited. In this embodiment and the following embodiments, the model training device is taken as an example to describe the content audit model training method of the present invention.

It should be noted that the unlabeled training sample set may be a set aggregated from a large number of unlabeled training samples, and the unlabeled training samples may be model training samples that have not been manually labeled. The manually labeled sample set may be a collection of a large number of manually labeled samples, and the manually labeled samples may be model training samples that have manually labeled violations in the samples. The number of samples in the unlabeled training sample set is much larger than the number of samples in the manually labeled sample set, for example: the number of samples in the unlabeled training sample set may be 100 times or more than the number of samples in the manually labeled sample set.

In the specific implementation, in order to facilitate the construction of the sample set and the training of the model, a large number of sample files can be collected and stored in the database, then the unlabeled training sample set and artificially labeled samples can be obtained at this time The set of steps can include:

Extract sample audit files from the preset sample file library;

Detecting whether there is a corresponding manual label in the sample audit file;

A manually labeled sample set is constructed according to the sample review files with corresponding manual labels, and an unlabeled training sample set is constructed according to the sample review files without corresponding manual labels.

It should be noted that the preset sample file library may be a preset database for storing sample files. However, manual labeling of samples requires a lot of labor costs, and it is difficult to obtain harmful samples. In order to reuse harmful samples, the collected sample audit files can be stored in the preset sample file library, and When manually labeling, the corresponding manual label is also stored in association with the sample review file. When it is necessary to build a sample set, the sample review file can be directly extracted from the preset sample file library, and the sample review is performed according to whether there is a corresponding manual label. Files are distinguished to construct a manually labeled sample set and an unlabeled training sample set.

Step S20: Using the unlabeled training sample set to train the initial self-supervised review model to obtain a content review model.

It should be noted that the initial self-supervision review model may be a model for content review constructed based on a self-supervision algorithm. Train the initial self-supervised audit model through a large amount of unlabeled data, use the method of comparative learning to let the model learn similar features between unlabeled data, and construct pseudo-labels of unlabeled data, which can make the model closer to the same type of data. , to shorten the distance between different data, so that the final content review model has a very powerful feature extraction ability, that is, has a very strong generalization.

Step S30: performing secondary training on the content review model according to the manually marked sample set.

It should be noted that after training the initial self-supervised review model through the unlabeled sample set, the obtained content review model already has very strong generalization, but at this time, the accuracy of the model for content review will be lower than If it is low, then the content review model can be retrained by manually marking the sample set at this time, so as to improve the accuracy of the content review model for content review.

In this embodiment, by obtaining an unlabeled training sample set and a manually labeled sample set, the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set; the initial self-supervised review model is trained through the unlabeled training sample set to obtain the content Review model: conduct secondary training on the content review model based on the manually marked sample set. When training the content review model, a large amount of unlabeled data is used to train the initial self-supervised review model, so that the obtained content review model has strong generalization, and then the content review model is trained twice using the manually labeled sample set , to improve the review accuracy of the content review model, so that the final obtained model not only has the review accuracy but also has strong generalization, which can meet the needs of Internet content review scenarios.

Referring to FIG. 3 , FIG. 3 is a schematic flowchart of a second embodiment of a method for training a content review model according to the present invention.

Based on the above-mentioned first embodiment, after the step S30 of the content audit model training method of this embodiment, it also includes:

Step S40: Use the content audit model after the secondary training as the default content audit model.

It should be noted that the second training of the content audit model can improve the accuracy of the model content audit on the basis of retaining the generalization of the model. Therefore, the content audit model after the second training can be used as the default The content audit model is used for auditing files to be audited.

Step S50: When the file to be reviewed is received, the file type corresponding to the file to be reviewed is obtained.

It should be noted that in the field of Internet content review, there are various files to be reviewed, and the analysis methods of different file types are completely different. Therefore, when receiving a file to be reviewed, it is necessary to first obtain the file file type.

In a specific implementation, in order to accurately determine the file type, step S50 in this embodiment may include:

When receiving the file to be reviewed, read the file suffix of the file to be reviewed;

The file suffix is matched with the suffix set corresponding to each file type in the preset file type table to determine the file type corresponding to the file to be reviewed.

It should be noted that different file suffixes correspond to different file types. For example, if the file suffix is txt, it means that the file is a text file; if the suffix is jpeg, it means that the file is a picture file.

In actual use, files with different suffixes can be divided into different file types according to the carrying content. One file type can correspond to multiple file suffixes. For example, the image type can correspond to various files such as jpeg, png, bmp, tif, etc. suffix. The preset file type table may include a mapping relationship between each file type and a file suffix, and the mapping management may be pre-set by the manager of the model training device. The suffix set corresponding to the file type may be a set formed by aggregating file suffixes corresponding to the file type.

It can be understood that the file type of the file to be reviewed can be quickly determined by obtaining the file suffix of the file to be reviewed and comparing the file suffix with the file suffix in the suffix set corresponding to each file type.

Step S60: Obtain the content parsing rules corresponding to the file type.

It should be noted that, in order to deal with a variety of files that need to be reviewed, different content analysis rules can be set for files of different file types, and the content analysis rules corresponding to the file type can be obtained by searching for the content analysis rules set in advance for the file type. rule.

In actual use, with the continuous development of technology, the way of parsing each file may also change continuously. There may be many different parsing methods for the same file type. In order to obtain the appropriate parsing rules as much as possible To analyze the file to be audited, step S60 in this embodiment may include:

Searching for corresponding parsing rules in the preset rule type mapping table according to the file type;

The number of rules to detect the found parsing rules;

If the number of rules is greater than the preset number, then obtain the rule priority corresponding to each parsing rule;

sorting the parsing rules from large to small based on the priority of the rules to obtain a sorting result;

The parsing rule ranked first in the sorting result is used as the content parsing rule corresponding to the file type.

It should be noted that the preset number can be set in advance by a manager of the model training device, and the preset number can be 1.

In actual use, if the number of rules is greater than the preset number, it means that there are multiple parsing rules corresponding to the file type. At this time, you can obtain the rule priority corresponding to the parsing rule, and then parse from type to small pair according to the rule priority Rules are sorted, and then the parsing rule ranked first is selected from the sorting results as the content parsing rule corresponding to the file type. Wherein, the preset rule type mapping table may include a mapping relationship between each parsing rule and a file type, and the mapping relationship may be preset by a manager of the model training device. The higher the priority of the rule, the more preferentially the parsing rule should be used, and the priority of the rule can be set in advance by the manager of the model training device.

It is understandable that selecting the content parsing rules corresponding to the file type in this way can ensure that the selected parsing rules with the highest priority preset by the management personnel of the model training equipment can ensure that the most appropriate parsing rules specified by the user can be obtained. Parse the file to be reviewed.

Step S70: Analyze the file to be reviewed according to the content analysis rules to obtain the content to be reviewed.

It can be understood that after the content analysis rules are obtained, the files to be reviewed can be analyzed according to the analysis method recorded in the content analysis rules, and the content recorded in the files to be reviewed can be read to obtain the content to be reviewed.

Step S80: Perform content audit on the content to be audited through the preset content audit model.

It should be noted that the content auditing of the content to be audited through the preset content auditing model can be performed by inputting the content to be audited as a model input parameter into the preset content auditing model for analysis, and then determining according to the content auditing results output by the preset content auditing model Whether the file to be reviewed has passed content review.

In actual use, in order to simplify the model input results, a fixed audit result identifier can be used to indicate the audit result, then step S80 in this embodiment may include:

performing a content analysis on the content to be audited through the preset content audit model, and obtaining a content analysis result;

extracting an audit result identifier from the content analysis result;

Determine whether the file to be reviewed has passed content review according to the review result identifier.

It should be noted that, through the content analysis of the content to be audited through the preset content audit model, and obtaining the content analysis results, the content to be audited can be input into the preset content audit model as model input data for content analysis, and the predicted content can be obtained at the end of the analysis. Set the content analysis results fed back by the content audit model. The content analysis result may be composed of an audit result identifier and violation content information, and extracting the audit result identifier from the content analysis result may be reading the audit result identifier in the content analysis result.

In actual use, the step of determining whether the file to be reviewed has passed the content review according to the review result identifier described in this embodiment may include:

Searching for the corresponding violation determination result in the preset result identification mapping table according to the audit result identification;

If the violation determination result is that there is no violation content, it is determined that the file to be reviewed has passed the content review.

It should be noted that the preset result identifier mapping table may include the mapping relationship between the audit result identifier and the violation judgment result. For example, if the audit result identifier is 1, the corresponding violation judgment result is that there is violation content; The corresponding violation judgment result is that there is no violation content.

It is understandable that the pre-set content review model is trained with manually labeled samples with illegal content in the second training, which can identify various pending documents with illegal content. If the violation judgment result is non-existent Violation content means that the preset content audit model determines that there is no illegal content in the file to be reviewed, and therefore, it can be determined that the file to be reviewed has passed the content review.

Furthermore, since some illegal content may not constitute a violation in some actual scenarios, such content may easily lead to misjudgment by the preset content review model. In order to avoid the impact of such misjudgment on the user who initiated the file review, After the step of searching the corresponding violation determination result in the preset result identification mapping table according to the audit result identification in this embodiment, it may further include:

If the violation determination result is that there is violation content, extract the violation content information from the content analysis result;

Constructing a violation confirmation report based on the violation content information and the pending documents;

displaying the violation confirmation report to content reviewers, and receiving the violation confirmation result given by the content reviewer to the violation confirmation report;

If the confirmation result of the violation is a determination of violation, it is determined that the file to be reviewed does not pass the content review.

It should be noted that some suspected illegal content needs to be analyzed in combination with semantics to determine whether it is really illegal. Such content may lead to misjudgment. In order to avoid such impact, when the preset content review model determines that there is illegal content, It is also possible to extract the violation content information from the content analysis results, and form a violation confirmation report based on the violation content information and the documents to be reviewed. The content reviewer will manually review and feed back the final violation confirmation result, and then determine the violation according to the final violation confirmation result. Whether the file to be reviewed has passed content review.

It is understandable that if the violation confirmation result is a violation determination, it means that the content reviewer confirms that the output result of the preset content review model is correct, therefore, it can be directly determined that the file to be reviewed does not pass the content review. Of course, according to actual needs, at this time, a corresponding manual label can be generated for the file to be reviewed according to the violation confirmation result, and then the manual label and the file to be reviewed can be associated and stored in the preset sample file library.

In the specific implementation, if the violation confirmation result is a violation misjudgment, it means that the preset content audit model has misjudged, and additional processing can be performed at this time. Therefore, the violation confirmation report is sent to the content audit After the step of displaying by personnel and receiving the violation confirmation result given by the content reviewer to the violation confirmation report, it may also include:

If the confirmation result of the violation is a misjudgment of violation, it is determined that the file to be reviewed has passed the content review;

Construct a positive labeled sample according to the file to be reviewed, the violation content information, and the violation confirmation result, and add the positive labeled sample to the manual labeled sample set.

It is understandable that if the confirmation result of the violation is misjudgment of violation, it means that the content reviewer believes that the default content review model has made a misjudgment, and the violation content information identified by the default content review model does not violate the rules. Therefore, it can be determined that the pending review The document passes the content review, and constructs a positive labeling sample based on the pending file, violation content information, and violation confirmation results, then adds the positive labeling sample to the manual labeling sample set, and reviews the preset content through the modified manual labeling sample set The model is further trained to reduce the probability of misjudgment of the preset content review model.

In this embodiment, the content audit model after secondary training is used as the preset content audit model; when the file to be audited is received, the file type corresponding to the file to be audited is obtained; the content analysis rule corresponding to the file type is obtained; Analyzing the file to be reviewed according to the content parsing rules to obtain the content to be reviewed; performing content review on the content to be reviewed through the preset content review model. Before performing content audit through the preset content audit model, the corresponding content analysis rules will be used to extract the content to be audited in the file to be audited according to the file type of the file to be audited. , the model does not need to parse the documents to be reviewed, which reduces the difficulty of model construction.

Referring to FIG. 4 , FIG. 4 is a schematic flowchart of a third embodiment of a method for training a content review model according to the present invention.

Based on the above-mentioned second embodiment, before the step S40 of the content audit model training method of this embodiment, it also includes:

Step S31: Obtain a model verification sample set and a standard judgment result set corresponding to the model verification sample set.

It should be noted that the model verification sample set may be a set aggregated from a large number of model verification samples. The model verification sample may be a sample used to verify the content review model after secondary training. The standard determination result set corresponding to the model verification sample set may be a set formed by aggregating manual review results corresponding to each model verification sample in the model verification sample set.

Step S32: Analyzing the samples in the model verification sample set through the content review model trained twice to obtain a review judgment result set.

It should be noted that by analyzing the samples in the model verification sample set through the content review model after the second training, the audit judgment result set can be obtained by inputting the order of the model verification samples in the model verification sample set into the content review after the second training The model is analyzed, and the output audit judgment results are aggregated to obtain the audit judgment result set.

Step S33: Determine the model audit accuracy rate according to the standard judgment result set and the audit judgment result set.

It should be noted that the determination of the model audit accuracy rate based on the standard judgment result set and the audit judgment result set can be obtained by obtaining the total number of results in the label judgment results, and then comparing the standard judgment result set with the results in the audit judgment result set to determine the audit accuracy. Determine the number of judgment results that are consistent with the standard judgment result set, and obtain the correct number, and then use the ratio of the correct number to the total number of results as the accuracy of the model review.

Step S34: If the model review accuracy rate is greater than or equal to the preset accuracy rate threshold, execute the step of using the content review model after the secondary training as the preset content review model.

It should be noted that the preset accuracy rate threshold may be preset by a manager of the model verification device. If the model review accuracy rate is greater than or equal to the preset accuracy rate threshold, it means that the model review accuracy rate has reached the standard and can be put into practical use. Therefore, the content review model after the second training can be used as the preset content review model.

In a specific implementation, if the model review accuracy rate is less than the preset accuracy rate threshold, it means that the review accuracy rate of the content review model obtained after the second training is not up to the standard, and at this time, the number of samples in the manually marked sample set may be too small Therefore, the manual marking sample set can be expanded, and the step of performing secondary training on the content review model according to the manual marking sample set can be returned to further train the content review model.

In this embodiment, by obtaining the model verification sample set and the standard judgment result set corresponding to the model verification sample set; and analyzing the samples in the model verification sample set through the content audit model after secondary training, the audit judgment result set is obtained; Analyze the samples in the model verification sample set through the content audit model after the second training to obtain the audit judgment result set; if the accuracy of the model audit is greater than or equal to the preset accuracy threshold, then execute the second The trained content moderation model is used as a step of the preset content moderation model. Before putting the content audit model into actual use, the audit accuracy of the content audit model after the second training will be verified according to the model verification sample set and the standard judgment result set corresponding to the model verification sample set. Only when the model audit accuracy of the audit model reaches the standard will the trained model be put into actual use, which ensures the actual use effect of the model.

In addition, an embodiment of the present invention also proposes a storage medium, on which a content audit model training program is stored, and when the content audit model training program is executed by a processor, the content audit model training method as described above is implemented. step.

Referring to FIG. 5 , FIG. 5 is a structural block diagram of the first embodiment of the content review model training device of the present invention.

As shown in Figure 5, the content audit model training device proposed by the embodiment of the present invention includes:

The sample acquisition module 10 is configured to acquire an unlabeled training sample set and a manually labeled sample set, where the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set.

A training module 20, used to train the initial self-supervised review model through the unlabeled training sample set to obtain the content review model;

The secondary training module 30 is configured to perform secondary training on the content review model according to the manually marked sample set.

In this embodiment, by obtaining an unlabeled training sample set and a manually labeled sample set, the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set; the initial self-supervised review model is trained through the unlabeled training sample set to obtain the content Review model: conduct secondary training on the content review model based on the manually marked sample set. When training the content review model, a large amount of unlabeled data is used to train the initial self-supervised review model, so that the obtained content review model has strong generalization, and then the content review model is trained twice using the manually labeled sample set , improve the review accuracy of the content review model, so that the final obtained model not only has the review accuracy but also has strong generalization, which can meet the needs of Internet content review scenarios.

Further, the secondary training module 30 is also used to use the content review model after the secondary training as the preset content review model; when receiving the file to be reviewed, obtain the file type corresponding to the file to be reviewed; obtain A content analysis rule corresponding to the file type; analyzing the file to be reviewed according to the content analysis rule to obtain the content to be reviewed; performing content review on the content to be reviewed through the preset content review model.

Further, the secondary training module 30 is also used to obtain the model verification sample set and the standard judgment result set corresponding to the model verification sample set; Analyze the sample to obtain an audit judgment result set; determine the model audit accuracy rate according to the standard judgment result set and the audit judgment result set; if the model audit accuracy rate is greater than or equal to the preset accuracy rate threshold, execute the A step of using the content review model after the secondary training as a preset content review model.

Further, the secondary training module 30 is also used to expand the artificially marked sample set if the accuracy rate of the model review is less than the preset accuracy rate threshold, and to update the artificially marked sample set according to the artificially marked sample set. The content moderation model undergoes secondary training.

Further, the secondary training module 30 is also used to read the file suffix of the file to be reviewed when receiving the file to be reviewed; The suffix set is matched to determine the file type corresponding to the file to be reviewed.

Further, the secondary training module 30 is also used to search for corresponding parsing rules in the preset rule type mapping table according to the file type; detect the rule quantity of the found parsing rules; if the rule quantity is greater than the preset If the quantity is set, the rule priority corresponding to each analysis rule is obtained; based on the rule priority, the analysis rules are sorted from large to small to obtain the sorting result; Content parsing rules corresponding to the above file types.

Further, the secondary training module 30 is also used to perform content analysis on the content to be audited through the preset content audit model to obtain content analysis results; extract audit result identifiers from the content analysis results; according to The audit result flag determines whether the file to be audited has passed the content audit.

Further, the secondary training module 30 is also used to look up the corresponding violation determination result in the preset result identification mapping table according to the audit result identification; if the violation determination result is that there is no violation content, then determine the The document to be reviewed has passed the content review.

Further, the secondary training module 30 is also used to extract the violation content information from the content analysis result if the violation determination result is that there is violation content; according to the violation content information and the pending file Build a violation confirmation report; display the violation confirmation report to content reviewers, and receive the violation confirmation result given by the content reviewer to the violation confirmation report; if the violation confirmation result is a violation determination, determine the The file to be reviewed does not pass content review.

Further, the secondary training module 30 is also used to determine that the file to be reviewed has passed the content review if the confirmation result of the violation is a misjudgment of violation; Construct a positive labeled sample based on the violation confirmation result, and add the positive labeled sample to the manual labeled sample set.

Further, the sample acquisition module 10 is also used to extract sample audit files from the preset sample file library; detect whether the sample audit files have corresponding manual tags; construct manual Label the sample set, and build an unlabeled training sample set based on the sample review files that do not have corresponding manual labels.

It should be understood that the above is only an example, and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can make settings according to needs, and the present invention is not limited thereto.

It should be noted that the workflow described above is only illustrative and does not limit the protection scope of the present invention. In practical applications, those skilled in the art can select part or all of them to implement according to actual needs. The purpose of the scheme of this embodiment is not limited here.

In addition, for technical details that are not exhaustively described in this embodiment, refer to the method for training a content review model provided by any embodiment of the present invention, which will not be repeated here.

Furthermore, it should be noted that in this document, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or system comprising a set of elements includes not only those elements, but also other elements not expressly listed, or elements inherent in such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system comprising that element.

The serial numbers of the above embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as a read-only memory (Read Only Memory) , ROM)/RAM, magnetic disk, optical disk), including several instructions to make a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the methods described in various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process conversion made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in other related technical fields , are all included in the scope of patent protection of the present invention in the same way.

The present invention also discloses A1, a content audit model training method, the content audit model training method includes the following steps:

Obtaining an unlabeled training sample set and a manually labeled sample set, wherein the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set;

training the initial self-supervised review model through the unlabeled training sample set to obtain a content review model;

Perform secondary training on the content review model according to the manually marked sample set.

A2. The content audit model training method as described in A1, after the step of performing secondary training on the content audit model according to the manually marked sample set, further comprising:

Use the content review model after the second training as the default content review model;

When receiving the file to be reviewed, obtain the file type corresponding to the file to be reviewed;

Obtaining content parsing rules corresponding to the file type;

Analyzing the file to be reviewed according to the content parsing rules to obtain the content to be reviewed;

Perform content audit on the content to be audited through the preset content audit model.

A3. The content audit model training method as described in A2, before the step of using the content audit model after the secondary training as the preset content audit model, it also includes:

Obtaining a model verification sample set and a standard judgment result set corresponding to the model verification sample set;

Analyzing the samples in the model verification sample set through the content audit model after secondary training to obtain an audit judgment result set;

determining the model audit accuracy rate according to the standard judgment result set and the audit judgment result set;

If the model review accuracy rate is greater than or equal to the preset accuracy rate threshold, the step of using the content review model after the second training as the preset content review model is performed.

A4. The content review model training method as described in A3, the step of determining the model review accuracy rate according to the standard judgment result set and the review judgment result set includes:

If the accuracy of the model review is less than the preset accuracy threshold, the manual marking sample set is expanded, and the step of performing secondary training on the content review model according to the manual marking sample set is returned.

A5. The content audit model training method as described in A2, the step of obtaining the file type corresponding to the file to be audited when the file to be audited is received includes:

When receiving the file to be reviewed, read the file suffix of the file to be reviewed;

The file suffix is matched with the suffix set corresponding to each file type in the preset file type table to determine the file type corresponding to the file to be reviewed.

A6. The method for training a content review model as described in A2, the step of obtaining the content parsing rules corresponding to the file type includes:

Searching for corresponding parsing rules in the preset rule type mapping table according to the file type;

The number of rules to detect the found parsing rules;

If the number of rules is greater than the preset number, then obtain the rule priority corresponding to each parsing rule;

sorting the parsing rules from large to small based on the priority of the rules to obtain a sorting result;

The parsing rule ranked first in the sorting result is used as the content parsing rule corresponding to the file type.

A7. The content audit model training method as described in A2, the step of performing content audit on the content to be audited through the preset content audit model, including:

performing a content analysis on the content to be audited through the preset content audit model, and obtaining a content analysis result;

extracting an audit result identifier from the content analysis result;

Determine whether the file to be reviewed has passed content review according to the review result identifier.

A8. The content audit model training method as described in A7, the step of determining whether the file to be audited has passed the content audit according to the audit result identification includes:

Searching for the corresponding violation determination result in the preset result identification mapping table according to the audit result identification;

If the violation determination result is that there is no violation content, it is determined that the file to be reviewed has passed the content review.

A9. The content audit model training method as described in A8, after the step of searching the corresponding violation determination result in the preset result identification mapping table according to the audit result identification, it also includes:

If the violation determination result is that there is violation content, extract the violation content information from the content analysis result;

Constructing a violation confirmation report based on the violation content information and the pending documents;

displaying the violation confirmation report to content reviewers, and receiving the violation confirmation result given by the content reviewer to the violation confirmation report;

If the confirmation result of the violation is a determination of violation, it is determined that the file to be reviewed does not pass the content review.

A10. The content review model training method described in A9, after the step of presenting the violation confirmation report to the content review personnel, and receiving the violation confirmation result fed back by the content review personnel to the violation confirmation report, Also includes:

If the confirmation result of the violation is a misjudgment of violation, it is determined that the file to be reviewed has passed the content review;

Construct a positive labeled sample according to the file to be reviewed, the violation content information, and the violation confirmation result, and add the positive labeled sample to the manual labeled sample set.

A11. The content review model training method described in any one of A1-A10, the step of obtaining an unlabeled training sample set and a manually marked sample set includes:

Extract sample audit files from the preset sample file library;

Detecting whether there is a corresponding manual label in the sample audit file;

A manually labeled sample set is constructed according to the sample review files with corresponding manual labels, and an unlabeled training sample set is constructed according to the sample review files without corresponding manual labels.

The present invention also discloses B12, a content review model training device, the content review model training device includes the following modules:

A sample acquisition module, configured to acquire an unlabeled training sample set and a manually marked sample set, wherein the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set;

A training module, which is used to train the initial self-supervised review model through the unlabeled training sample set to obtain a content review model;

A secondary training module is configured to perform secondary training on the content review model according to the artificially marked sample set.

B13, the content audit model training device as described in B12, the secondary training module is also used to use the content audit model after the secondary training as the preset content audit model; when receiving the file to be audited, obtain the The file type corresponding to the file to be reviewed; obtaining the content analysis rule corresponding to the file type; analyzing the file to be reviewed according to the content analysis rule to obtain the content to be reviewed; Content to be reviewed is subject to content review.

B14, the content review model training device as described in B13, the secondary training module is also used to obtain the model verification sample set and the standard judgment result set corresponding to the model verification sample set; through the content review after the secondary training The model analyzes the samples in the model verification sample set to obtain an audit judgment result set; determines the model audit accuracy rate according to the standard judgment result set and the audit judgment result set; if the model audit accuracy rate is greater than or equal to the expected If the accuracy rate threshold is set, the step of using the content audit model after the secondary training as the preset content audit model is performed.

B15. The content audit model training device as described in B14, the secondary training module is also used to expand the manually marked sample set if the accuracy of the model audit is less than the preset accuracy threshold, and according to The manually marked sample set performs secondary training on the content review model.

B16, the content audit model training device as described in B13, the secondary training module is also used to read the file suffix of the file to be audited when receiving the file to be audited; The suffix set corresponding to each file type in the file type table is matched to determine the file type corresponding to the file to be reviewed.

B17, the content audit model training device as described in B13, the secondary training module is also used to search the corresponding parsing rules in the preset rule type mapping table according to the file type; detect the rules of the found parsing rules number; if the number of rules is greater than the preset number, then obtain the rule priority corresponding to each analysis rule; sort the analysis rules based on the priority of the rules from large to small to obtain a sorting result; sort the result The parsing rule that ranks first in the file type is used as the content parsing rule corresponding to the file type.

B18. The content audit model training device as described in B13, the secondary training module is also used to perform content analysis on the content to be audited through the preset content audit model, and obtain content analysis results; from the content Extracting an audit result identifier from the analysis result; determining whether the file to be audited has passed the content audit according to the audit result identifier.

The present invention also discloses C19, a model training device. The model training device includes: a processor, a memory, and a content review model training program stored on the memory and operable on the processor. The content When the audit model training program is executed by the processor, the steps of the content audit model training method described above are implemented.

The present invention also discloses D20, a computer-readable storage medium, wherein a content audit model training program is stored on the computer-readable storage medium, and when the content audit model training program is executed, the above-mentioned content audit model training method is realized A step of.

Claims (10)

1. A content auditing model training method, is characterized in that, described content auditing model training method comprises the following steps: Obtaining an unlabeled training sample set and a manually labeled sample set, wherein the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set; training the initial self-supervised review model through the unlabeled training sample set to obtain a content review model; Perform secondary training on the content review model according to the manually marked sample set. 2. The content auditing model training method as claimed in claim 1, characterized in that, after the step of performing secondary training on the content auditing model according to the artificially marked sample set, further comprising: Use the content review model after the second training as the default content review model; When receiving the file to be reviewed, obtain the file type corresponding to the file to be reviewed; Obtaining content parsing rules corresponding to the file type; Analyzing the file to be reviewed according to the content parsing rules to obtain the content to be reviewed; Perform content audit on the content to be audited through the preset content audit model. 3. The content audit model training method as claimed in claim 2, characterized in that, before the step of using the content audit model after the secondary training as the preset content audit model, further comprising: Obtaining a model verification sample set and a standard judgment result set corresponding to the model verification sample set; Analyzing the samples in the model verification sample set through the content audit model after secondary training to obtain an audit judgment result set; determining the model audit accuracy rate according to the standard judgment result set and the audit judgment result set; If the model review accuracy rate is greater than or equal to the preset accuracy rate threshold, the step of using the content review model after the second training as the preset content review model is performed. 4. The content review model training method according to claim 3, wherein the step of determining the model review accuracy rate according to the standard judgment result set and the review judgment result set comprises: If the accuracy of the model review is less than the preset accuracy threshold, the manual marking sample set is expanded, and the step of performing secondary training on the content review model according to the manual marking sample set is returned. 5. The content audit model training method according to claim 2, wherein the step of obtaining the file type corresponding to the file to be audited when receiving the file to be audited comprises: When receiving the file to be reviewed, read the file suffix of the file to be reviewed; The file suffix is matched with the suffix set corresponding to each file type in the preset file type table to determine the file type corresponding to the file to be reviewed. 6. The content audit model training method according to claim 2, wherein the step of obtaining the content analysis rules corresponding to the file type comprises: Searching for corresponding parsing rules in the preset rule type mapping table according to the file type; The number of rules to detect the found parsing rules; If the number of rules is greater than the preset number, then obtain the rule priority corresponding to each parsing rule; sorting the parsing rules from large to small based on the priority of the rules to obtain a sorting result; The parsing rule ranked first in the sorting result is used as the content parsing rule corresponding to the file type. 7. The content audit model training method according to claim 2, wherein the step of performing content audit on the content to be audited through the preset content audit model comprises: performing a content analysis on the content to be audited through the preset content audit model, and obtaining a content analysis result; extracting an audit result identifier from the content analysis result; Determine whether the file to be reviewed has passed content review according to the review result identifier. 8. A content audit model training device, characterized in that the content audit model training device comprises the following modules: A sample acquisition module, configured to acquire an unlabeled training sample set and a manually marked sample set, wherein the number of samples in the unlabeled training sample set is greater than the number of samples in the manually labeled sample set; A training module, which is used to train the initial self-supervised review model through the unlabeled training sample set to obtain a content review model; A secondary training module is configured to perform secondary training on the content review model according to the artificially marked sample set. 9. A model training device, characterized in that the model training device comprises: a processor, a memory, and a content review model training program stored on the memory and operable on the processor, the content review When the model training program is executed by the processor, the steps of the content review model training method according to any one of claims 1-7 are realized. 10. A computer-readable storage medium, characterized in that, a content audit model training program is stored on the computer-readable storage medium, and when the content audit model training program is executed, any one of claims 1-7 is implemented. The steps of the content auditing model training method described above.