CN118714368A - Content security processing system - Google Patents

Abstract

The embodiment of the present application discloses a content security processing system, the system comprising: an application system for providing multiple uplink audio streams generated by the clients of multiple participant users of a target session respectively; a content audit auxiliary system for obtaining the correspondence between the multiple participant users and the voiceprint features according to the multiple uplink audio streams respectively, and merging the multiple uplink audio streams into a mixed audio stream for sending to a content security audit system for content security audit; a content security audit system for performing content security audit based on the mixed audio stream and generating an audit result; the content audit auxiliary system is also used to determine the illegal content and its location after receiving the audit result returned by the content security audit system, and to extract the target audio segment at the location of the illegal content from the mixed audio stream; the audit result is used to indicate whether the target session has a violation at the session granularity; the intercepted target audio segment is matched and judged with the voiceprint features of the multiple participant users respectively, and the illegal content is located to at least one of the target participant users according to the matching result. Through the embodiment of the present application, the illegal content can be located to a specific participant at a lower cost when multiple participant users are associated with the same session.

Description

Content security processing system

Technical Field

The present application relates to the technical field of content auditing, and in particular to a content security processing system.

Background Art

UGC (User Generated Content), also known as user-generated content, is the content that users create and display or provide to other users through the Internet platform. With the rapid development of the Internet, smart devices and various new services, the data on the Internet has exploded. Various forms of UGC content such as pictures, videos, posts, chats, live broadcasts, etc. have become an indispensable part of people's expression of feelings, recording events and daily work. However, these growing contents are also full of various uncontrollable risk factors, such as pornographic videos and pictures, political and violent terrorist content, various junk advertisements, etc. With the increasingly strict supervision of relevant departments, these are all tasks that need to be taken seriously and managed by various websites and platforms. Therefore, the content security review system came into being. This content security review system can control the security work related to the risks of UGC content on the Internet.

Specifically, the content security review system mainly reviews the content (text, pictures, audio, video) uploaded, published or shared by users on social platforms. Its main purpose is to filter out low-quality and vulgar illegal content, so as to produce high-quality content, prevent the reduction of user experience, and maintain a good content tone. For example, one of the review scenarios is to review the audio content during the live broadcast, that is, to determine whether the user's speech content during the live broadcast is illegal. If so, the corresponding application system can be prompted to handle the user. Similar scenarios include game scenarios, etc.

When conducting a security audit of the audio content during a live broadcast, the prior art usually conducts an audit in units of live broadcast rooms, that is, the audio stream generated in a specific live broadcast room is collected and then provided to the audit system for security audit. In actual applications, multiple anchors can speak in a live broadcast room. In addition, since the live broadcast system can also provide a "microphone" function, audience users can also speak in the live broadcast room, and so on. For this situation, the prior art processing method is to merge the multiple audio streams generated in the same live broadcast room into one, and then provide it to the audit system for security audit. Accordingly, the audit system can perform speech recognition, natural language understanding and other processing based on the synthesized audio stream to determine whether there is any illegal content.

Although this method can realize the content security review of live broadcast content, it can only judge whether there is any illegal content at the granularity of the live broadcast room. Correspondingly, if there is any illegal content, it means that the entire live broadcast room may be punished. However, in fact, it may be that only the speech of one of the users is in violation. At this time, it is unfair to punish the entire live broadcast room. Therefore, in this scenario, it is necessary to specifically locate the individual who violated the rules, rather than the entire live broadcast room. In order to achieve this goal, one solution is to provide the single-channel audio stream corresponding to each user who speaks during the live broadcast to the audit system for content security review. However, this will sharply increase the cost of content review, which is unbearable for most application systems. In addition, as the number of users participating in the speech during the live broadcast continues to increase, problems such as resource exhaustion on the audit system side will also arise.

Therefore, when multiple participant users are associated with the same session, how to locate the illegal content to specific individual participants at a lower cost becomes a technical problem that needs to be solved by technicians in this field.

Summary of the invention

The present application provides a content security processing system, which can locate illegal content to specific individual participants at a lower cost when multiple participant users are associated with the same session.

This application provides the following solutions:

A content security processing system, comprising:

An application system, for determining multiple uplink audio streams respectively generated by clients of multiple participant users of a target session;

A content audit auxiliary system, used for respectively obtaining the correspondence between the multiple participant users and the voiceprint features according to the multiple uplink audio streams, and merging the multiple uplink audio streams into a mixed audio stream for sending to a content security audit system for content security audit;

A content security audit system, used to perform content security audit based on the mixed audio stream and generate an audit result;

The content audit auxiliary system is also used to determine the illegal content and its location after receiving the audit result returned by the content security audit system, and to intercept the target audio segment at the location of the illegal content from the mixed audio stream; the audit result is used to indicate whether the target session has a violation at the session granularity; the intercepted target audio segment is matched and judged with the voiceprint features of the multiple participant users respectively, and according to the matching result, the illegal content is located to at least one of the target participant users.

A content security processing system, comprising:

An application system, for determining multiple uplink audio streams respectively generated by clients of multiple participant users of a target session;

The content audit auxiliary system obtains the correspondence between the multiple participant users and the voiceprint features based on the multiple uplink audio streams, and merges the multiple uplink audio streams into a mixed audio stream for sending to the content security audit system for content security audit; after receiving the audit result returned by the content security audit system, determines the illegal content and its location, and cuts out the target audio segment at the location of the illegal content from the mixed audio stream; the audit result is used to indicate whether the target session has a violation at the session granularity; the cut target audio segment is matched and judged with the voiceprint features of the multiple participant users, and according to the matching result, the illegal content is located to at least one of the target participant users.

A content security processing system, comprising:

A content audit auxiliary system, for obtaining the correspondence between the multiple participant users and the voiceprint features according to the multiple uplink audio streams respectively generated by the clients of the multiple participant users of the target session in the application system, and merging the multiple uplink audio streams into a mixed audio stream;

A content security audit system, used to perform a content security audit on the mixed audio stream sent by the content audit auxiliary system and generate an audit result;

The content audit auxiliary system is also used to, after receiving the audit results returned by the content security audit system, determine the illegal content and its location, and cut out the target audio segment at the location of the illegal content from the mixed audio stream; the audit results are used to indicate whether the target session has a violation at the session granularity; the intercepted target audio segment is matched and judged with the voiceprint features of the multiple participant users respectively, and according to the matching results, the illegal content is located to at least one of the target participant users.

According to the specific embodiments provided in this application, this application discloses the following technical effects:

Through the embodiment of the present application, an intermediate content audit auxiliary system can be provided between the application system and the content security audit system, wherein the application system is used to provide multiple uplink audio streams generated by the clients of multiple participant users of the target session respectively, and then, before the content security audit system is audited, the content audit auxiliary system can extract the voiceprint features of multiple participant users respectively according to the multiple uplink audio streams in the same session. When the audio stream is submitted for review, the multiple uplink audio streams can still be merged into a mixed audio stream to avoid the increase in the cost of content security audit. In addition, after receiving the audit result returned by the specific content security audit system, the content audit auxiliary system can also determine the illegal content and its location, and intercept the target audio clip at the location of the illegal content. In this way, the intercepted target audio clip can be matched and judged with the voiceprint features of the multiple participant users respectively, and then the illegal content can be located to at least one of the target participant users according to the matching results. In this way, the identification of violations at the user granularity can be realized without causing a sharp increase in costs, so that when punishment and other treatments are required, the individual who violates the law can be punished specifically without affecting other participant users in the same session.

Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic diagram of a system architecture provided in an embodiment of the present application;

FIG2 is a flow chart of a method provided in an embodiment of the present application;

FIG3 is a schematic diagram of a device provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of an electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

In the embodiment of the present application, in order to locate the illegal content to the specific participant user at a lower cost when multiple participant users are associated with the same session (Session, including live broadcast session, chat session, game session, etc.), a corresponding solution is provided. In this solution, an intermediate layer content audit auxiliary system can be established between the content security audit system and the specific application system (that is, the content production system, for example, the live broadcast system, the game system, etc.). Specifically, the application system can submit the multiple uplink audio streams generated in the specific session to the processing system. In addition to submitting the audio stream to the audit system for review, the processing system can also extract the voiceprint features of the corresponding participant user according to each uplink audio stream (wherein each specific user participating in the speech corresponds to an uplink audio stream). In a preferred manner, each uplink audio stream can also be recorded, so that multiple recording records can be saved for each participant user, and the corresponding relationship between the specific recording record and the specific session identifier, the generation time, etc. can also be recorded. This recording record can provide a more powerful reference basis in the process of further confirmation or punishment of the user in the future.

Specifically, when submitting the information to the audit system for review, the embodiment of the present application can still merge multiple uplink audio streams into one (or several) mixed audio streams, so that the audit system can perform content security review based on the mixed audio stream, without the need to perform content security review separately for the single uplink audio stream corresponding to each participant user.

When the audit system returns the audit result, it will first return to the processing system in the embodiment of the present application, and then the processing system will perform positioning processing before returning it to the specific application system. Among them, after receiving the audit result returned by the audit system, if the audit result shows that the audio stream submitted for review has violations, the processing system can determine the specific illegal content (for example, it may be a keyword or key sentence used by a user during a speech, etc.), and the audio data corresponding to the location of the illegal content (the audio data is also a very short audio clip, that is, a small audio clip corresponding to the above-mentioned keyword or key sentence spoken by the user). Afterwards, since the voiceprint feature information of the specific participant user can be extracted before, the audio data can be matched and judged with the voiceprint features of each participant user. If the voiceprint feature of a certain user is successfully matched, it can be determined that the participant user has violated the rules.

In this way, through the above scheme, since the audio clips containing the sounds of illegal content can be matched and judged according to the voiceprint characteristics of the participant user, the specific illegal content can be located to the specific participant user, so that the subsequent punishment and other processing such as banning can be carried out based on the illegal participant user, without affecting other participant users in the session. In addition, since the content security audit can still be conducted based on one or a few mixed audio streams after confluence, the illegal content can be located to the specific target participant user at a lower cost.

Specifically from the perspective of system architecture, as mentioned above, the embodiment of the present application can provide a content review assistance system. As shown in Figure 1, the system is located between a specific content security review system and an application system. The multiple uplink audio streams generated in the specific session of the application system can first be submitted to the content review assistance system of the embodiment of the present application. The system extracts voiceprint features and merges multiple audio streams into a mixed audio stream before submitting it to the review system for review. After receiving the review results, the audio clips containing illegal content can be identified based on the voiceprint features of each participant user, and the corresponding target participant user can be determined, thereby locating the illegal content to an individual.

The specific implementation scheme provided in the embodiments of the present application is described in detail below.

First, the embodiment of the present application provides a content review auxiliary processing method, referring to FIG2 , the method may include:

S201: Acquire multiple uplink audio streams generated in a target session, where the uplink audio streams are respectively generated by clients of multiple participant users of the target session.

Among them, the target session can be determined according to the situation in the specific application system. For example, in a live broadcast system, it can be a live broadcast session (one live broadcast room can correspond to one live broadcast session), in a game system, it can be a game session (one game "room" corresponds to one game session), in a communication system, it can also be a chat session, and so on. Specific sessions can include audio sessions and video sessions, etc. Of course, in the embodiment of the present application, the specific objects that need to be subject to content security review mainly refer to specific audio content. In the scenario of a video session, the specific application system can separate the audio stream component from the video stream and upload it to the auxiliary system described in the embodiment of the present application for content security review, etc.

In the same session, multiple participants can speak, and each user can generate an uplink audio stream through his client. The application system can provide multiple uplink audio streams corresponding to multiple participants to the audit processing system in the embodiment of the present application.

S202: Obtain corresponding relationships between the multiple participant users and voiceprint features according to the multiple uplink audio streams respectively.

After receiving multiple uplink audio streams, the voiceprint features of the corresponding participant users can be extracted according to each uplink audio stream. Specifically, since each uplink audio stream is associated with the identifier of the participant user, after receiving multiple uplink audio streams, the voiceprint features can be extracted from each audio stream respectively, and an association relationship can be established with the specific participant user identifier. Subsequently, based on this voiceprint feature, the identity of the speaker/speaker can be identified from the audio clip corresponding to the specific illegal content. Among them, the voiceprint feature specifically refers to the acoustic features of the speaker during the speech process, which refers to a set of acoustic description parameters extracted from the sound signal by a computer algorithm (mathematical method). There can be many specific algorithms for extracting voiceprint features, such as Gaussian mixture model (GMM), joint factor analysis (JFA), deep neural network method, etc., which will not be described in detail here.

In specific implementation, the voiceprint feature extraction of the participant user can be performed directly based on each audio stream. Alternatively, in another way, each uplink audio stream can be recorded separately, and then the voiceprint feature of the participant user can be extracted based on the recording record, and so on. In addition, in this recording method, the correspondence between each recording record and the identifier of the target session, the identifier of the participant user, and the time of generation can also be saved. It can be used as a basis for subsequent processing. For example, after locating the participant user corresponding to the specific illegal content through machine recognition, it can also be manually reviewed. At this time, the recording record of the corresponding time period can be provided to the manual review client as a basis for review. Alternatively, after a specific participant user is subsequently identified as having violated the rules, the recording record can also be provided to the application system, so that the application system can make a more accurate judgment based on the recording record before determining whether the participant user needs to be punished, and so on.

It should be noted here that in scenarios such as live broadcasting, although there may be multiple users speaking in the same live broadcast room session, some users may only speak occasionally, especially audience users, while users such as anchors may speak for a relatively long time, etc. That is to say, in some upstream audio streams, there may be voice signals only for part of the time, and in some upstream audio streams, there may be voice signals continuously, etc. When recording, the voice signal in the upstream audio stream can be detected. If a voice signal is detected, recording is performed, otherwise, recording is not required. Therefore, the same participant user can correspond to multiple recording records, corresponding to different time information. In addition, even for the case of continuous speaking by anchor users, when recording, it can be split into multiple different recording records. Therefore, for multiple participant users of a certain session, multiple recording records can be respectively corresponded, and each recording record can correspond to their own start time, end time and other information.

S203: Merge the multiple uplink audio streams into a mixed audio stream for sending to the audit system for content security audit.

In addition to obtaining the voiceprint features of each participant user from each uplink audio stream and recording, the processing system in the embodiment of the present application can also perform review processing. Specifically, in the embodiment of the present application, multiple uplink audio streams can still be merged into a mixed audio stream (for example, they can be merged into one mixed audio stream, etc.), and then, based on this mixed audio stream, an audit request can be sent to the audit system to perform a security audit on the content of the mixed audio stream.

Among them, in the specific implementation, since the mixed audio stream belongs to streaming data, the mixed audio stream can be first divided into multiple audio segments before uploading to the audit system. For example, every 12 seconds (or other time lengths) is an audio segment, and so on. In this way, the specific audit system can conduct content security audits based on such audio segments. There can be many ways for the specific audit system to conduct security audits. For example, in one way, voice recognition can be performed on the audio segment, that is, the voice signal is converted into text, and then, the relevant algorithms of natural language understanding are used to perform natural language understanding on the converted text to determine whether there is any illegal content, where the illegal content is mainly some keywords with sensitive words, or key sentences, etc.

S204: After receiving the audit result returned by the audit system, determine the illegal content and its location, and cut out the target audio segment at the location of the illegal content.

After the audit system performs a content security audit according to the received audit request, it can return the audit result to the processing system provided in the embodiment of the present application. Specifically, if the content security audit is applied for in units of the aforementioned segmented audio segments, the specific audit result can mainly include: the audio segments with specific violations. In other words, the processing system can know which specific audio segment or segments have violations.

However, since the specific illegal content is usually keywords, key sentences, etc., even if it is divided into audio segments, the specific illegal content usually only appears at a certain position in the specific audio segment. For example, an audio segment may be 12 seconds, and the illegal content is a keyword that only appears in the segment, a small audio segment between the 3rd and 5th seconds, etc. Only the audio segment corresponding to the specific illegal content can be matched with the voiceprint features of multiple participant users, and then determine which participant user specifically said the illegal content.

Therefore, in the specific implementation, after receiving the specific review results, the illegal content and the position of the illegal content in the target audio segment can also be determined, and according to the position of the illegal content in the target audio segment, the target audio segment can be cut out from the corresponding position in the target audio segment for subsequent voiceprint recognition.

Specifically, there are many ways to determine the illegal content and its location. For example, in one way, if the audit results returned by the audit system only include the audio segments with illegal content, that is, from the audit results returned by the audit system, we can only know which audio segments have illegal content, but we don’t know what the illegal content is and where it appears in the specific segment. In this case, the specific processing system can also identify the illegal content and its location from the audio segments with illegal content.

For example, in a specific implementation, the target audio segment may be subjected to speech recognition (which may include the specifically recognized text and the time information corresponding to the specific text content on the audio timeline), and then the speech recognition result may be matched with a preset word library. The word library may pre-store multiple keywords related to violations, so that if the speech recognition result hits a keyword in the word library, the keyword may be determined as the illegal content, and the position of the keyword in the target audio segment (that is, the start time and end time on the audio segment timeline) may be determined as the position of the illegal content in the target audio segment.

Alternatively, in another manner, the audit results returned by the audit system may include not only which audio segments contain violations, but also the specific illegal content contained in the specific audio segments with violations, and the specific location of the illegal content in the audio segments, etc. In this way, the specific illegal content and the location of the illegal content in the specific audio segments, etc. can be directly determined based on the results returned by the audit system.

S205: Match the intercepted target audio clips with the voiceprint features of the multiple participant users respectively, and locate the illegal content to at least one of the target participant users according to the matching results.

After the target audio clip corresponding to the sound position of the specific illegal content is intercepted, the intercepted target audio clip can be matched and judged with the voiceprint features of the multiple participant users respectively, so that the illegal content can be located to at least one of the target participant users according to the matching results. That is to say, although the multiple uplink audio streams in the same session are merged into one mixed audio stream when submitting for review, the audit result returned by the audit system can only determine whether there is a violation at the session granularity. However, through the determination of the specific illegal content and its location in the embodiment of the present application, and the matching of the audio clip at the location of the illegal content with the voiceprint features of multiple participant users, the specific illegal content can be located to a specific participant user (which can be one or more). In this way, even if the specific violation needs to be dealt with later, for example, punishment, banning, etc., only the individual can be punished, and other users in the same session will not be affected.

In specific implementation, since the punishment of users may be involved in the downstream link, the specific judgment result can be given more cautiously. For this reason, in specific implementation, after the illegal content is determined by the computer algorithm and the participant user who specifically generates the illegal content is located, the specific positioning result can be further confirmed by manual intervention. For example, specifically, after the illegal content is located to one of the target participant users, the audit result given by the audit system, the positioning result from the illegal content to the target participant user obtained by the processing system in the embodiment of the present application, and the recording record of the target participant user in the corresponding time period can be provided to the manual audit client, so as to further confirm the positioning result by manual audit. For example, it can be judged whether there is really illegal content and whether the corresponding participant user is the user identified by the algorithm by manually listening to the recording record, etc. Among them, a relatively long recording record can be provided to the manual audit client, so that when the manual audit is conducted, a more accurate judgment can be made in combination with the context of the specific speech content.

In addition, after locating the illegal content to one of the target participant users (it can also be after further manual review and confirmation), the review result, the positioning result, and the audio recording of the target participant user in the corresponding time period are provided to the corresponding application system so that the application system can determine whether to punish the target participant user. That is to say, in the embodiment of the present application, not only can the specific individual user who violated the law be determined, but also, when judging whether to punish the violator, not only can the conclusive review result or positioning result be used, but also the original sound data of the user during the speech at that time can be obtained in combination with the specific audio recording, and by listening to this audio recording, a more accurate judgment can be made.

In summary, through the embodiment of the present application, an intermediate processing system can be provided between the application system and the audit system, and the processing system can extract the voiceprint features of multiple participant users according to the multiple uplink audio streams in the same session. When the audio stream is submitted for review, the multiple uplink audio streams can still be merged into a mixed audio stream to avoid the increase in the cost of content security audit. However, after receiving the audit result returned by the specific audit system, the illegal content and its location can be determined, and the target audio segment at the location of the illegal content can be intercepted. In this way, the intercepted target audio segment can be matched and judged with the voiceprint features of the multiple participant users respectively, and then the illegal content can be located to at least one of the target participant users according to the matching results. In this way, the identification of violations at the user granularity can be realized without causing a sharp increase in costs, so that when punishment and other treatments are required, the individual who violates the law can be punished specifically without affecting other participant users in the same session.

It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

Corresponding to the aforementioned method embodiment, the embodiment of the present application further provides a content review auxiliary processing device, referring to FIG3 , which may include:

The audio stream acquisition unit 301 is used to acquire multiple uplink audio streams generated in the target session, where the uplink audio streams are respectively generated by clients of multiple participant users of the target session;

A voiceprint feature extraction unit 302 is used to obtain the corresponding relationship between the multiple participant users and the voiceprint features according to the multiple uplink audio streams respectively;

The audio stream mixing and review unit 303 is used to combine the multiple uplink audio streams into a mixed audio stream for sending to the review system for content security review;

The target audio segment interception unit 304 is used to determine the illegal content and its location after receiving the audit result returned by the audit system, and intercept the target audio segment at the location of the illegal content;

The voiceprint matching judgment unit 305 is used to match the intercepted target audio segment with the voiceprint features of the multiple participant users respectively, and locate the illegal content to at least one of the target participant users according to the matching results.

In a specific implementation, the device may further include:

The recording unit is used to record the multiple uplink audio streams respectively, and save the corresponding relationship between the recording record and the identifier of the target session, the identifier of the participant user, and the generation time, so as to be used as a basis for subsequent processing.

In addition, the device may also include:

The first recording result providing unit is used to provide the review result, the positioning result and the recording record of the target participant user in the corresponding time period to the manual review client after the illegal content is positioned to one of the target participant users, so as to further confirm the positioning result through manual review.

Furthermore, the device may further include:

The second recording result providing unit is used to provide the review result, the positioning result and the recording record of the target participant user in the corresponding time period to the corresponding application system after the illegal content is located to one of the target participant users, so that the application system can determine whether to process the target participant user.

In a specific implementation, the device may further include:

A segmentation unit, used for merging the multiple uplink audio streams into a mixed audio stream and then segmenting the mixed audio stream into a plurality of audio segments, so that the audit system can perform content security audit based on the audio segments;

The audit results returned by the audit system include: target audio segments with illegal content;

The target audio segment interception unit may specifically include:

An illegal content location determination subunit, used to determine illegal content and the location of the illegal content in the target audio segment;

The interception subunit is used to intercept the target audio segment from the corresponding position in the target audio segment according to the position of the illegal content in the target audio segment.

Specifically, the illegal content location determination subunit can be used to:

Performing speech recognition on the target audio segment, and matching the speech recognition result with a preset vocabulary;

If a keyword in the vocabulary is hit, the keyword is determined as the illegal content, and the position of the keyword in the target audio segment is determined as the position of the illegal content in the target audio segment.

Alternatively, in another manner, the audit result returned by the audit system may further include: the illegal content, and location information of the illegal content in the target audio segment;

At this time, the illegal content location determination subunit can be specifically used to:

The illegal content and the location of the illegal content in the target audio segment are determined according to the audit result returned by the audit system.

In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

And an electronic device, comprising:

one or more processors; and

A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

4 exemplarily shows the architecture of the electronic device, which may include a processor 410, a video display adapter 411, a disk drive 412, an input/output interface 413, a network interface 414, and a memory 420. The processor 410, the video display adapter 411, the disk drive 412, the input/output interface 413, the network interface 414, and the memory 420 may be communicatively connected via a communication bus 430.

Among them, the processor 410 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

The memory 420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 can store an operating system 421 for controlling the operation of the electronic device 400, and a basic input and output system (BIOS) for controlling the low-level operation of the electronic device 400. In addition, a web browser 423, a data storage management system 424, and a content audit auxiliary processing system 425, etc. can also be stored. The above-mentioned content audit auxiliary processing system 425 can be an application program that specifically implements the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 420 and is called and executed by the processor 410.

The input/output interface 413 is used to connect the input/output module to realize information input and output. The input/output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

The network interface 414 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

The bus 430 comprises a pathway for transmitting information between the various components of the device (eg, the processor 410, the video display adapter 411, the disk drive 412, the input/output interface 413, the network interface 414, and the memory 420).

It should be noted that, although the above device only shows a processor 410, a video display adapter 411, a disk drive 412, an input/output interface 413, a network interface 414, a memory 420, a bus 430, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

The above is a detailed introduction to the content review auxiliary processing method, device and electronic device provided by this application. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims (9)

1. A content security processing system, comprising:

An application system for providing multiple upstream audio streams respectively generated by clients of multiple participant users of a target session;

The content auditing auxiliary system is used for acquiring the corresponding relation between the plurality of participant users and the voiceprint features according to the multi-channel uplink audio streams respectively, and combining the multi-channel uplink audio streams into a mixed audio stream for sending to the content security auditing system to conduct content security auditing;

The content security auditing system is used for conducting content security auditing based on the mixed audio stream and generating auditing results;

The content auditing auxiliary system is also used for determining illegal contents and the positions of the illegal contents after receiving auditing results returned by the content security auditing system, and intercepting target audio fragments of the positions of the illegal contents from the mixed audio stream; the auditing result is used for indicating whether the target session has violations in session granularity; and respectively carrying out matching judgment on the cut target audio fragments and voiceprint characteristics of the plurality of participant users, and positioning the illegal contents to at least one target participant user according to a matching result.

2. The system of claim 1, wherein the system further comprises a controller configured to control the controller,

The content auditing auxiliary system is also used for recording the plurality of paths of uplink audio streams respectively and storing the corresponding relation between the recorded records and the identification of the target session, the identification of the participant user and the generation time so as to be used as the basis for subsequent processing.

3. The system of claim 1, wherein the system further comprises a controller configured to control the controller,

The content auditing auxiliary system is also used for providing the auditing result, the locating result and the record of the target participant user in the corresponding time period to the manual auditing client after locating the illegal content to one of the target participant users so as to further confirm the locating result in a manual auditing mode.

4. A system according to claim 2 or 3, wherein,

The content auditing auxiliary system is also used for providing the auditing result, the locating result and the record of the target participant user in the corresponding time period to the corresponding application system after locating the illegal content to one of the target participant users so that the application system can determine whether to process the target participant user.

5. The system of claim 1, wherein the system further comprises a controller configured to control the controller,

The content auditing auxiliary system is also used for splitting the mixed audio stream into a plurality of audio paragraphs after merging the plurality of uplink audio streams into the mixed audio stream;

The content security auditing system is specifically configured to perform content security auditing with the audio paragraph as a unit, so that the auditing result includes: a target audio paragraph for which offending content exists;

The content auditing auxiliary system is specifically used for determining illegal contents and the positions of the illegal contents:

determining illegal contents and the positions of the illegal contents in the target audio paragraphs, and cutting out the target audio fragments from the corresponding positions in the target audio paragraphs according to the positions of the illegal contents in the target audio paragraphs.

6. The system of claim 5, wherein the system further comprises a controller configured to control the controller,

The content auditing auxiliary system is specifically used for determining illegal contents and the positions of the illegal contents when the illegal contents are determined:

and performing voice recognition on the target audio paragraph, matching a voice recognition result with a preset word stock, and if a certain keyword in the word stock is hit, determining the keyword as the illegal content, and determining the position of the keyword in the target audio paragraph as the position of the illegal content in the target audio paragraph.

7. The system of claim 5, wherein the system further comprises a controller configured to control the controller,

The auditing result returned by the content security auditing system also comprises: the offending content and the position information of the offending content in the target audio clip;

The content auditing auxiliary system is specifically used for determining illegal contents and the positions of the illegal contents when the illegal contents are determined:

and determining illegal contents and positions of the illegal contents in the target audio fragments according to an auditing result returned by the auditing system.

8. A content security processing system, comprising:

An application system for determining multiple upstream audio streams respectively generated by clients of multiple participant users of a target session;

the content auditing auxiliary system acquires the corresponding relation between the plurality of participant users and the voiceprint features according to the multi-channel uplink audio streams respectively, and combines the multi-channel uplink audio streams into a mixed audio stream for sending to a content security auditing system for content security auditing; after receiving an auditing result returned by the content security auditing system, determining illegal contents and the positions of the illegal contents, and intercepting target audio fragments of the positions of the illegal contents from the mixed audio stream; the auditing result is used for indicating whether the target session has violations in session granularity; and respectively carrying out matching judgment on the cut target audio fragments and voiceprint characteristics of the plurality of participant users, and positioning the illegal contents to at least one target participant user according to a matching result.

9. A content security processing system, comprising:

The content auditing auxiliary system is used for acquiring the corresponding relation between the multiple participant users and the voiceprint features according to multiple uplink audio streams respectively generated by the clients of the multiple participant users of the target session in the application system, and combining the multiple uplink audio streams into a mixed audio stream;

The content security auditing system is used for conducting content security auditing on the mixed audio stream sent by the content auditing auxiliary system and generating auditing results;

the content auditing auxiliary system is also used for determining illegal contents and the positions of the illegal contents after receiving auditing results returned by the content security auditing system, and intercepting target audio fragments of the positions of the illegal contents from the mixed audio stream; the auditing result is used for indicating whether the target session has violations in session granularity; and respectively carrying out matching judgment on the cut target audio fragments and voiceprint characteristics of the plurality of participant users, and positioning the illegal contents to at least one target participant user according to a matching result.