CN117672211A - Dialogue processing method and device, electronic equipment and computer storage medium - Google Patents

Abstract

The disclosure provides a dialogue processing method, a dialogue processing device, electronic equipment and a computer readable storage medium, and relates to the technical field of voice processing. The method comprises the following steps: when the machine voice is played, user voice input of a user is stopped, the machine voice is played, the user voice and the machine voice are input into a recovery model, when the intention of the user for actually interrupting the machine voice is determined through the recovery model, timestamp information corresponding to the machine voice when the user voice is input is obtained, a recovery play node corresponding to the machine voice is determined according to the timestamp information, for example, the recovery play node is arranged at a preset pause position corresponding to the machine voice when the user voice is input, and the machine voice is played from the recovery play node; the embodiment of the disclosure can confirm the intention of the user, correct the situation that the user interrupts the voice by mistake, ensure the smoothness and continuity of voice recovery and improve the voice interaction effect.

Description

Conversation processing method, device, electronic equipment and computer storage medium

Technical field

The present disclosure relates to the field of speech processing technology, and in particular, to a dialogue processing method, device, electronic device and computer-readable storage medium.

Background technique

As the application of intelligent voice dialogue systems becomes more and more widespread, the intelligent experience of intelligent voice dialogue has increasingly become an important indicator of system performance. Whether it can effectively support users to interrupt the machine is one of the most effective ways to improve the experience.

In related technologies, after the intelligent voice dialogue system recognizes the user's input, it cannot detect whether the user actually intends to interrupt the speech. The user may just agree casually, or there may be people in the background talking, and the system will also stop broadcasting and there will be obvious interruptions. There is a sense of abruptness, and the system also has obvious lags; and the timing of the system to resume broadcasting is uncertain. It may start to resume from halfway through a word, such as "we", the system starts broadcasting from the word "we", and even worse In this case, it is possible to start broadcasting from the second half of the syllable of "人", resulting in poor voice interaction effect and poor user experience.

It should be noted that the information disclosed in the above background section is only used to enhance understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to those of ordinary skill in the art.

Contents of the invention

The present disclosure provides a dialogue processing method, device, electronic device and computer-readable storage medium, which at least to a certain extent overcomes the problem of poor voice interaction effect in related technologies.

Additional features and advantages of the disclosure will be apparent from the following detailed description, or, in part, may be learned by practice of the disclosure.

According to one aspect of the present disclosure, a dialogue processing method is provided, including: when there is a user voice input from a user while the machine voice is playing, and the user has no real intention to interrupt the machine voice, stopping the playback of the machine voice , and obtain the timestamp information corresponding to the machine voice when the user's voice is input; determine the recovery playback node corresponding to the machine voice according to the timestamp information; and play the machine voice from the recovery playback node.

In one embodiment of the present disclosure, when there is a user voice input from the user while the machine voice is playing, and the user has no real intention to interrupt the machine voice, the machine voice is stopped playing, and the machine voice is obtained. The timestamp information corresponding to the machine voice when the user's voice is input includes: inputting the user's voice and the machine voice to the recovery model, so as to determine whether the user actually interrupted the machine voice through the recovery model. intention of.

In one embodiment of the present disclosure, the training data of the recovery model includes: dialogue audio feature data, dialogue context data, dialogue turn data, and/or dialogue semantic data.

In one embodiment of the present disclosure, inputting the user voice and the machine voice to a recovery model, so as to determine whether the user has a real intention to interrupt the machine voice through the recovery model includes: The recovery model determines whether the user has a real intention to interrupt the machine voice based on the voice data, the user voice and the machine voice; wherein the voice data includes at least one of the following: guided speech content data, The playback progress data and voice confidence data of the machine voice.

In one embodiment of the present disclosure, the resuming playback node includes at least one of the following:

The starting position of the machine voice;

The position of the machine voice at the time of the user's voice input;

The corresponding sentence segment position of the machine voice during the user's voice input;

The preset pause position corresponding to the machine voice when the user's voice is input.

In one embodiment of the present disclosure, the machine voice includes: voice audio, text corresponding to the voice audio, and/or timestamp information corresponding to the voice audio, wherein the timestamp information includes: punctuation marks , sentence break position, and/or preset pause position.

In one embodiment of the present disclosure, when there is a user voice input from the user while the machine voice is playing, and the user has no real intention to interrupt the machine voice, the machine voice is stopped playing, and the machine voice is obtained. The timestamp information corresponding to the machine voice when the user's voice is input includes: when there is a user's voice input while the machine voice is playing, and the user has no real intention to interrupt the machine voice, immediately stop playing the machine voice Voice or slowly lower the volume of the machine voice.

In one embodiment of the present disclosure, the method further includes: when the pause of the user's voice exceeds a first time threshold, it indicates that the user's voice input is completed.

In one embodiment of the present disclosure, the method further includes: when the user has a real intention to interrupt the machine voice, generating a reply voice corresponding to the user's voice.

In one embodiment of the present disclosure, the method further includes: filtering silence data and noise data in the user's voice.

According to another aspect of the present disclosure, a dialogue processing device is also provided, including:

The acquisition module, when there is user voice input from the user when the machine voice is playing, and the user has no real intention to interrupt the machine voice, stops playing the machine voice, and obtains the user voice input of the machine voice. The corresponding timestamp information;

A determination module that determines the resumption playback node corresponding to the machine voice according to the timestamp information;

A recovery module that plays the machine voice from the recovery play node.

According to another aspect of the present disclosure, an electronic device is also provided, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions via Instructions to execute any of the above conversation processing methods.

According to another aspect of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the above conversation processing methods.

The dialogue processing method, device, electronic device and computer-readable storage medium provided by the embodiments of the present disclosure, when the machine voice is played and there is a user voice input from the user, the machine voice is stopped, and the user voice and the machine voice are input to the recovery model , and when it is determined through the recovery model that the user has no real intention to interrupt the machine voice, the timestamp information corresponding to the machine voice when the user's voice is input is obtained, and the recovery playback node corresponding to the machine voice is determined based on the timestamp information, for example, the recovery playback node is set Play the machine voice from the resume playback node at the preset pause position corresponding to the user's voice input; it can confirm the user's intention, correct the situation where the user accidentally interrupts the voice, and ensure the smoothness and coherence of voice recovery, improving voice quality. interactive effects.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and do not limit the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 shows a flow chart of a dialogue processing method in an embodiment of the present disclosure;

Figure 2 shows a flow chart of a method for determining user intention in an embodiment of the present disclosure;

Figure 3 shows a flow chart of yet another dialogue processing method in an embodiment of the present disclosure;

Figure 4 shows a schematic diagram of conversation recovery in an embodiment of the present disclosure;

Figure 5 shows a schematic diagram of a dialogue processing device in an embodiment of the present disclosure;

Figure 6 shows a schematic diagram of a dialogue processing system in an embodiment of the present disclosure;

7 shows a schematic diagram of an exemplary system architecture that can be applied to a dialogue processing method or dialogue processing device according to embodiments of the present disclosure; and

Figure 8 shows a structural block diagram of an electronic device in an embodiment of the present disclosure.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concepts of the example embodiments. To those skilled in the art. The described features, structures or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices.

This exemplary implementation will be described in detail below with reference to the drawings and examples.

First, embodiments of the present disclosure provide a dialogue processing method, which can be executed by any electronic device with computing processing capabilities.

Figure 1 shows a flow chart of a dialogue processing method in an embodiment of the present disclosure. As shown in Figure 1, the dialogue processing method provided in an embodiment of the present disclosure includes the following steps:

S102. When there is user voice input by the user when the machine voice is playing, and the user has no real intention to interrupt the machine voice, stop playing the machine voice and obtain the timestamp information corresponding to the machine voice when the user voice is input.

In one embodiment, the machine voice includes but is not limited to: voice audio, text corresponding to the voice audio, and/or timestamp information corresponding to the voice audio, etc., and the corresponding text, timestamp information, etc. are generated synchronously when synthesizing the voice audio; Among them, the timestamp information includes but is not limited to: the position of interrupting the played voice, punctuation mark, sentence segmentation position, and/or preset pause position. The sentence segmentation position can be the position where the sentence needs to be segmented based on semantics, etc., and the preset pause position is artificial Or automatically set the required pause position.

In one embodiment, data such as silence data and noise data in the user's voice are filtered.

In one embodiment, the user's voice and the machine's voice are input to the recovery model, so that the recovery model determines whether the user has a real intention to interrupt the machine's voice; the lack of real intention to interrupt the machine's voice includes but is not limited to: the user accidentally interrupts , the user does not change the conversation content, etc.

In one embodiment, the user's voice and machine voice after filtering data such as silence data and noise data are input to the recovery model, so that the recovery model can determine whether the user has a real intention to interrupt the machine voice.

In one embodiment, when there is user voice input from the user while the machine voice is playing, and the user has no real intention to interrupt the machine voice, the machine voice is immediately stopped playing, or the volume of the machine voice is slowly reduced until the pause in the user's voice exceeds The first time threshold represents the completion of the user's voice input.

In one embodiment, select and configure according to the usage scenario to immediately stop the machine voice broadcasting behavior or reduce the machine voice broadcasting volume.

In one embodiment, when the pause of the user's voice exceeds the first time threshold, it indicates that the user's voice input is completed. The first time threshold can be set automatically or manually according to the application scenario. For example, the first time threshold can be set to 600ms-800ms. .

In one embodiment, the recovery model determines whether the user has a real intention to interrupt the machine voice based on voice data, user voice, machine voice, etc.;

Among them, the voice data includes but is not limited to at least one of the following: guidance speech content data, machine voice playback progress data, and voice confidence data; guidance speech content data is data included in machine speech to guide user interaction; voice confidence The degree data is the recognition confidence corresponding to the user's voice and the machine's voice.

S104: Determine the resumption playback node corresponding to the machine voice according to the timestamp information.

In one embodiment, the recovery playback node includes but is not limited to at least one of the following:

The starting position of the machine voice;

The position of the machine voice during user voice input;

The sentence segment position corresponding to the machine voice during user voice input;

The preset pause position corresponding to the machine voice during user voice input.

In one embodiment, the resume play node can be set automatically or manually according to the scene. For example, the resume play node can be set to the sentence segment position corresponding to the machine voice when the user's voice input, and then a mapping table between the timestamp and the resume play node is established. , determine the corresponding resume playback node as the sentence segment position based on the timestamp information; the resume playback node can be set to the preset pause position corresponding to the machine voice during the user's voice input, and then a mapping table between the timestamp and the resume playback node is established. According to the time The stamp information A determines that the corresponding resume play node is the preset pause position B.

S106, play the machine voice from the recovery playback node.

In one embodiment, after the first round of user voice input ends, the user voice of the next round can be further detected and collected to further verify the user voice input of the first round.

In one embodiment, when resuming the playback of the machine voice, the volume of the machine voice can be restored directly, or the fade-in and fade-out method can be used for smooth recovery to ensure smooth and coherent voice recovery. The method of restoring the volume and the smooth transition time can be based on Requires automatic or manual configuration with the scenario.

In one embodiment, when the user truly intends to interrupt the machine voice, a reply voice corresponding to the user's voice is generated. The reply voice includes but is not limited to: "um", "received", etc., and then the corresponding voice is broadcast based on the user's voice. Systematic rhetoric with true intentions.

In the above embodiment, based on the recovery model, the user's intention is confirmed, the serious problem of accidental interruptions is corrected in the intelligent voice dialogue, the timestamp is maintained as the context of the voice dialogue, and the machine voice is used when handling user interruptions. During playback, the timestamp corresponding to the user's voice input is recorded. Multiple recovery strategies can be selected through the timestamp, and the appropriate recovery playback node can be determined to ensure smooth and coherent voice recovery, which can not only flexibly interrupt, but also It can detect false interruptions in time and restore machine voice in a friendly manner, improving the anthropomorphic experience of intelligent voice dialogue.

Figure 2 shows a flow chart of a user intention determination method in an embodiment of the present disclosure. As shown in Figure 2, the user intention determination method provided in an embodiment of the present disclosure includes the following steps:

S202, train the recovery model.

In one embodiment, the training data of the recovery model includes but is not limited to: dialogue audio feature data, dialogue context data, dialogue turn data, speech data, and/or dialogue semantic data, etc.; the dialogue audio features include but is not limited to short-term Zero-crossing rate, short-term energy, short-term autocorrelation function, short-term average amplitude, spectrum difference amplitude, spectrum centroid, spectrum width, Mel frequency cepstrum coefficient, etc.; the dialogue context is the context information of the interaction between machine voice and user voice, etc. ; Dialogue turn data includes but is not limited to: turn data such as signaling, confirming the end, and formal farewell; dialogue semantic data refers to the meaning of concepts represented by things in the real world corresponding to the data, etc.

S204: Input user voice and machine voice to the recovery model.

In one embodiment, the user's voice and the machine's voice are input to the recovery model, so that the recovery model determines whether the user has a real intention to interrupt the machine's voice; the lack of real intention to interrupt the machine's voice includes but is not limited to: the user accidentally interrupts , the user does not change the conversation content, etc.

In one embodiment, the user's voice and machine voice after filtering data such as silence data and noise data are input to the recovery model, so that the recovery model can determine whether the user has a real intention to interrupt the machine voice.

In one embodiment, the recovery model determines whether the user has a real intention to interrupt the machine voice based on voice data, user voice, machine voice, etc.

Among them, the voice data includes but is not limited to at least one of the following: guidance speech content data, machine voice playback progress data, and voice confidence data; guidance speech content data is data included in machine speech to guide user interaction; voice confidence The degree data is the recognition confidence corresponding to the user's voice and the machine's voice.

S206: Determine whether to resume the playback of the machine voice according to the recovery model judgment result.

One of the recovery model judgment results is to confirm the user's interruption intention. At this time, the current round of dialogue ends, and the user's voice is completely semantically analyzed and dialogue processed to generate new words for further interaction.

Another judgment result of the recovery model is that the user's intention to interrupt is denied. At this time, the current round of dialogue needs to be resumed, the machine voice is scrolled forward to the appropriate resume playback node, and the playback of the machine voice is resumed at the resume playback node.

In the above embodiment, based on the recovery model, the user's intention is confirmed, and based on the confirmation result, it is determined whether to further perform intention recognition or restore the conversation, correcting the serious problem of mistaken interruption in the intelligent voice conversation, when resuming the conversation. , by adding timestamps and scrolling forward from the interruption position to the appropriate resume playback node, this strategy has smoother and more coherent sounds, clearer expressions, and is closer to the real everyone-to-people dialogue experience.

Figure 3 shows a flow chart of yet another dialogue processing method in an embodiment of the present disclosure. As shown in Figure 3, the dialogue processing method provided in an embodiment of the present disclosure includes the following steps:

S302, synthesize and play machine voice.

In one embodiment, the machine voice is synthesized by a speech synthesis engine, or may be a recording made in advance; when the machine voice audio is generated, a timestamp of the corresponding text is generated simultaneously, and is maintained by the central control module as a context.

S304: When it is detected that there is user voice input by the user when the machine voice is played, obtain the timestamp information corresponding to the machine voice during the user voice input.

The interrupt function can be activated in combination with the voice endpoint detection module and/or noise reduction module in the central control module. After detecting the user's voice input, the central control module can immediately interrupt or reduce the volume of the machine voice, and The interruption timestamp corresponding to the user's voice input is detected in the context, and the interruption position is recorded through the timestamp, which is used to restore the interruption position during recovery.

In one embodiment, the user voice data is processed in real time, and the voice endpoint detection module and the noise reduction module are respectively called to filter the user voice data. If the voice endpoint detection module detects that the user voice data contains sound, the noise reduction module further detects it. If If the sound contains noise, it will be filtered out. If it does not contain noise, it will be determined to be input by the user, thereby improving the accuracy of the user's voice.

The voice endpoint detection module and noise reduction module are locally integrated lightweight modules with relatively fast response times. Using the voice endpoint detection module and noise reduction module to handle interruptions can respond to users faster, but their accuracy is susceptible to background noise. Therefore, after the user inputs the user voice, it is necessary to further collect enough information to further verify the user input of the user voice.

S306: Use the recovery model to detect whether the user has a real intention to interrupt the machine voice.

In one embodiment, the recovery model combines contextual information, integrates speech and semantic information, and detects whether the user has a real intention to interrupt the machine's speech.

S308, if so, this round of dialogue ends, and the central control module calls the semantic recognition engine and dialogue management service to perform complete semantic analysis and dialogue processing, and generate new words for further interaction.

S310, otherwise, resume the current round of dialogue. The central control module integrates the timestamp of the speech synthesis engine, rolls the speech forward to a suitable pause position, and starts to resume the playback of the machine voice at the pause.

In the above embodiment, a solution for rapid interruption, intent confirmation, and error recovery is provided. Fast voice breakpoint detection is used to interrupt the machine voice in time. After the user finishes speaking, a fast recovery model is used for confirmation. According to the confirmation As a result, it is determined whether to further perform intent recognition or resume dialogue. When resuming dialogue, scroll forward from the interruption position to the appropriate resume playback node by adding a timestamp. This strategy makes the sound smoother, more coherent, and more expressive. It is clearer and closer to the real conversation experience between people.

Figure 4 shows a schematic diagram of dialogue recovery in an embodiment of the present disclosure. The coordinate axis represents the direction of real-time generation of machine voice. Each square represents a time slice. When the machine voice asks for the user's name, the user may hear his or her own name. Confirm immediately. At this time, the machine voice will be interrupted immediately or the volume of the machine voice will be slowly reduced; for example, the machine voice can be: "Okay, are you Mr. Wang", and the user will reply when "Wang" is played, that is Corresponds to the user's voice input position.

If the user's reply is "yes", the recovery model will determine that the user has a real intention to interrupt the machine's voice based on the guidance content data, playback progress data, voice confidence and other parameters.

If the user's reply is "Ah~", the recovery model will judge that the user has no real intention to interrupt the machine's voice. At this time, the central control module rolls back the voice to the resume playback node position "you" based on the timestamp information, and starts from here. Repeat boot.

In the above embodiment, scroll forward from the interruption position to the appropriate resume playback node. This strategy makes the sound smoother and more coherent, the expression is clearer, and is closer to the real everyone dialogue experience, solving the problems that occur in the intelligent voice dialogue system. Accidental interrupt recovery problem.

Based on the same inventive concept, embodiments of the present disclosure also provide a dialogue processing device, such as the following embodiments. Since the problem-solving principle of this device embodiment is similar to that of the above-mentioned method embodiment, the implementation of this device embodiment can refer to the implementation of the above-mentioned method embodiment, and repeated details will not be repeated.

Figure 5 shows a schematic diagram of a dialogue processing device in an embodiment of the present disclosure. As shown in Figure 5, the dialogue processing device 5 includes: an acquisition module 501, a determination module 502 and a recovery module 503;

Acquisition module 501, when there is user voice input by the user when the machine voice is playing, and the user has no real intention to interrupt the machine voice, stops playing the machine voice, and obtains the timestamp information corresponding to the machine voice when the user voice is input;

The determination module 502 determines the resumption playback node corresponding to the machine voice according to the timestamp information;

The recovery module 503 plays the machine voice from the recovery play node.

In the above embodiment, based on the recovery model, the user's intention is confirmed, the serious problem of accidental interruptions is corrected in the intelligent voice dialogue, the timestamp is maintained as the context of the voice dialogue, and the machine voice is used when handling user interruptions. During playback, the timestamp corresponding to the user's voice input is recorded. Multiple recovery strategies can be selected through the timestamp, and the appropriate recovery playback node can be determined to ensure smooth and coherent voice recovery, which can not only flexibly interrupt, but also It can detect false interruptions in time and restore machine voice in a friendly manner, improving the anthropomorphic experience of intelligent voice dialogue.

Based on the same inventive concept, embodiments of the present disclosure also provide a dialogue processing system, such as the following embodiments. Since the problem-solving principle of this system embodiment is similar to that of the above-mentioned method embodiment, the implementation of this system embodiment can be referred to the implementation of the above-mentioned method embodiment, and repeated details will not be repeated.

Figure 6 shows a schematic diagram of a dialogue processing system in an embodiment of the present disclosure. As shown in Figure 6, the system includes: a central control module 601, a speech synthesis engine 602; a speech recognition engine 603; a semantic recognition engine 604; and a dialogue management service 605 ;The central control module 601 integrates various technologies and controls the dialogue logic;

The speech synthesis engine 602 synthesizes machine speech. When the machine speech audio is generated, the timestamp of the corresponding text is synchronously generated; the user speech and the machine speech are recognized through the speech recognition engine 603; the complete semantics is performed through the semantic recognition engine 604 and the dialogue management service 605. Analysis and dialogue processing to generate new words for further interaction.

The central control module 601 includes a voice endpoint detection module 6011, a noise reduction module 6012, and an undo recovery module 6013;

The voice endpoint detection module 6011 is used to identify the beginning and end of the silence period and the speech period from the sound signal. The application can only process valid speech, filter out excess silence, and save transmission bandwidth or computing power.

Noise reduction module 6012 is used to identify the beginning and end of noise and speech from the sound signal.

The undo recovery module 6013 continues to collect data after the machine voice is interrupted, further checks whether the interruption is correct or reasonable based on richer data, and restores incorrect interruptions.

Machine speech is synthesized by the speech synthesis engine 602 in the central control module 601. After detecting voice input through the voice endpoint detection module 6011 or noise reduction module 6012 in the central control module 601, the system can immediately stop the system's broadcasting behavior. You can also reduce the system's broadcast volume to produce a fade-in and fade-out effect, giving the user a feedback indicating that the system has learned that he is speaking. At the same time, when the user continues to speak beyond the second time threshold, it means that the user is expressing some complete intentions and the system will stop. The broadcast waits for the user's expression to be completed. When the user's pause exceeds the first time threshold, the system will consider the user's expression to be completed, and record the interruption timestamp corresponding to the user's voice input in the context, and record the location of the interruption through the timestamp. , used to restore the interrupted position during recovery.

In one embodiment, the first time threshold and the second time threshold can be set automatically or manually according to the scene and user habits. The second time threshold is generally about 1 second, and the first time threshold is generally 600ms-800ms.

In one embodiment, select and configure according to the usage scenario to immediately stop the system's machine voice broadcasting behavior, or reduce the system's machine voice broadcasting volume.

The recovery model is trained through dialogue audio feature data, dialogue context data, dialogue turn data, etc., and the recovery model is used to determine whether the user has a real intention to interrupt. If the user has a real intention to interrupt the machine voice, the system will add a The modal particles, such as "um" and "received", are used to broadcast the system's words corresponding to the true intention based on the user's voice; if the user has no real intention to interrupt, the system will naturally resume the broadcasting process.

Through the timestamp information of the speech synthesis engine 602, the system can calculate a resume playback node reported by the system since the last interruption, and resume broadcasting from the resume playback node, making the system more anthropomorphic.

In one embodiment, when broadcasting the system speech corresponding to the true intention or naturally resuming the broadcasting process, the playback volume can be restored. The restoration process can be a direct restoration or a smooth restoration and smooth transition using a fade-in and fade-out method. The time is automatically or manually configured according to needs and scenarios.

In the above embodiment, it is used in an intelligent voice dialogue scenario that supports interruption. One party to the conversation is a real person and the other is a voice robot. The real person and the machine speak alternately. When the machine speaks, the real person is allowed to interrupt the machine and obtain the right to speak. The time machine waits for the end of the real person's speech, and uses the recovery model to judge whether the previous interruption intention is correct or whether the real person's speech changes the conversation content. If the judgment result is that the interruption is mistaken or the conversation content does not change, the conversation will be restored, which can flexibly open the conversation. It can detect false interruptions in time, and can restore the machine voice in a friendly manner, ensuring smooth and coherent voice recovery, improving the user experience of interruption scenes in voice interaction, improving the anthropomorphic experience of intelligent voice dialogue, and ensuring that the system can be timely The response can correctly understand the user's intention, and the intelligent experience is better.

FIG. 7 shows a schematic diagram of an exemplary system architecture that can be applied to a dialogue processing method or dialogue processing apparatus according to embodiments of the present disclosure.

As shown in Figure 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704 and a server 705.

The network 704 is a medium used to provide communication links between the terminal devices 701, 702, 703 and the server 705, and may be a wired network or a wireless network.

Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and/or protocols. The network is usually the Internet, but can also be any network, including but not limited to Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless network, private network, or virtual private network). In some embodiments, technologies and/or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, you can also use conventional methods such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (Virtual Private Network, VPN), Internet Protocol Security (Internet Protocol Security, IPsec), etc. Encryption technology to encrypt all or some links. In other embodiments, customized and/or dedicated data communication technologies may also be used in place of or in addition to the above-described data communication technologies.

Terminal devices 701, 702, and 703 can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc., and can be used to display the playback progress of the machine voice, the judgment results of the recovery model, etc.

Optionally, the clients of the application programs installed in different terminal devices 701, 702, and 703 are the same, or the clients of the same type of application program based on different operating systems. Based on different terminal platforms, the specific form of the application client can also be different. For example, the application client can be a mobile phone client, a PC client, etc.

The server 705 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 701, 702, and 703. The background management server can analyze and process the received request and other data, and feed the processing results back to the terminal device; for example, it can obtain the timestamp information corresponding to the machine voice when the user inputs the voice; determine the machine voice correspondence based on the timestamp information resume playback node; play machine voice from the resume playback node; train the recovery model, etc.

Optionally, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, Cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smartphone, a tablet, a laptop, a desktop computer, etc., but is not limited thereto. The terminal and the server can be connected directly or indirectly through wired or wireless communication methods, which is not limited in this application.

Those skilled in the art will know that the number of terminal devices, networks and servers in Figure 7 is only illustrative, and there can be any number of terminal devices, networks and servers according to actual needs. The embodiments of the present disclosure do not limit this.

Those skilled in the art will understand that various aspects of the present disclosure may be implemented as systems, methods, or program products. Therefore, various aspects of the present disclosure may be embodied in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as "Circuits", "modules" or "systems".

An electronic device 800 according to this embodiment of the present disclosure is described below with reference to FIG. 8 . The electronic device 800 shown in FIG. 8 is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present disclosure.

As shown in Figure 8, electronic device 800 is embodied in the form of a general computing device. The components of the electronic device 800 may include, but are not limited to: the above-mentioned at least one processing unit 810, the above-mentioned at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

Wherein, the storage unit stores program code, and the program code can be executed by the processing unit 810, so that the processing unit 810 performs various exemplary methods according to the present disclosure described in the "Example Method" section of this specification. Implementation steps.

For example, the processing unit 810 can perform the following steps of the above method embodiment: when the machine voice is played and there is a user voice input of the user, stop playing the machine voice, input the user voice and the machine voice to the recovery model, and determine through the recovery model When the user has no real intention to interrupt the machine voice, obtain the timestamp information corresponding to the machine voice when the user's voice is input, and determine the resume playback node corresponding to the machine voice based on the timestamp information. For example, set the resume playback node when the machine voice is in the user's voice input. Play the machine voice from the resume playback node at the corresponding preset pause position during input.

The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 8201 and/or a cache storage unit 8202, and may further include a read-only storage unit (ROM) 8203.

Storage unit 820 may also include a program/utility 8204 having a set of (at least one) program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples, or some combination, may include the implementation of a network environment.

Bus 830 may be a local area representing one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or using any of a variety of bus structures. bus.

Electronic device 800 may also communicate with one or more external devices 840 (e.g., keyboard, pointing device, Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 800, and/or with Any device that enables the electronic device 800 to communicate with one or more other computing devices (eg, router, modem, etc.). This communication may occur through input/output (I/O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through a network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or can be implemented by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a terminal device, a network device, etc.) to execute a method according to an embodiment of the present disclosure.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, and the computer-readable storage medium may be a readable signal medium or a readable storage medium. Program products capable of implementing the above methods of the present disclosure are stored thereon. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the The terminal device performs the steps according to various exemplary embodiments of the present disclosure described in the above "Example Method" section of this specification.

For example, when the program product in the embodiment of the present disclosure is executed by a processor, the method implements the following steps: when the machine voice is played, there is a user voice input from the user, the machine voice is stopped, the user voice and the machine voice are input to the recovery model, and When it is determined through the recovery model that the user has no real intention to interrupt the machine voice, the timestamp information corresponding to the machine voice when the user inputs the voice is obtained, and the recovery playback node corresponding to the machine voice is determined based on the timestamp information. For example, the recovery playback node is set on the machine. The voice plays the machine voice from the resume play node at the preset pause position corresponding to the user's voice input.

More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard drives, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave carrying readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium that can send, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device.

Alternatively, program code embodied on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

When implemented, program code for performing operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and Includes conventional procedural programming languages—such as "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on. In situations involving remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as provided by an Internet service. (business comes via Internet connection).

It should be noted that although several modules or units of equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into being embodied by multiple modules or units.

Furthermore, although various steps of the methods of the present disclosure are depicted in the drawings in a specific order, this does not require or imply that the steps must be performed in that specific order, or that all of the illustrated steps must be performed to achieve the desired results. result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution, etc.

Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or can be implemented by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a mobile terminal, a network device, etc.) to execute a method according to an embodiment of the present disclosure.

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common common sense or customary technical means in the technical field that are not disclosed in the disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims (13)

1. A dialogue processing method, characterized by including: When there is user voice input from the user when the machine voice is playing, and the user has no real intention to interrupt the machine voice, stop playing the machine voice, and obtain the corresponding value of the machine voice when the user voice is input. timestamp information; Determine the resumption playback node corresponding to the machine voice according to the timestamp information; Play the machine voice from the resume play node. 2. The dialogue processing method according to claim 1, characterized in that when there is user voice input from the user when the machine voice is played, and the user has no real intention to interrupt the machine voice, the playback of the machine voice is stopped. Describing the machine voice and obtaining the timestamp information corresponding to the machine voice when the user's voice is input includes: The user's voice and the machine voice are input to the recovery model, so that the recovery model determines whether the user has a real intention to interrupt the machine voice. 3. The dialogue processing method according to claim 2, characterized in that the training data of the recovery model includes: dialogue audio feature data, dialogue context data, dialogue turn data, and/or dialogue semantic data. 4. The dialogue processing method according to claim 2, characterized in that the user's voice and the machine voice are input to a recovery model, so as to determine whether the user has truly interrupted the process through the recovery model. The intentions of the machine voice include: The recovery model determines whether the user has a real intention to interrupt the machine voice based on the voice data, the user voice and the machine voice; Wherein, the voice data includes at least one of the following: guided speech content data, playback progress data of the machine voice, and voice confidence data. 5. The dialogue processing method according to claim 1, characterized in that the resume playback node includes at least one of the following: The starting position of the machine voice; The position of the machine voice at the time of the user's voice input; The corresponding sentence segment position of the machine voice during the user's voice input; The preset pause position corresponding to the machine voice when the user's voice is input. 6. The dialogue processing method according to claim 1, characterized in that the machine voice includes: voice audio, text corresponding to the voice audio, and/or timestamp information corresponding to the voice audio, wherein the The timestamp information includes: punctuation marks, sentence segmentation positions, and/or preset pause positions. 7. The dialogue processing method according to claim 2, characterized in that when there is user voice input from the user when the machine voice is played, and the user has no real intention to interrupt the machine voice, the playback of the machine voice is stopped. Describing the machine voice and obtaining the timestamp information corresponding to the machine voice when the user's voice is input includes: When there is user voice input from the user while the machine voice is playing, and the user has no real intention to interrupt the machine voice, the machine voice is immediately stopped playing or the volume of the machine voice is slowly reduced. 8. The dialogue processing method according to claim 1, further comprising: When the pause of the user's voice exceeds the first time threshold, it indicates that the user's voice input is completed. 9. The dialogue processing method according to claim 1, further comprising: When the user has a real intention to interrupt the machine voice, a reply voice corresponding to the user's voice is generated. 10. The dialogue processing method according to claim 7 or 8, further comprising: filtering silence data and noise data in the user's voice. 11. A dialogue processing device, characterized in that it includes: The acquisition module, when there is user voice input from the user when the machine voice is playing, and the user has no real intention to interrupt the machine voice, stops playing the machine voice, and obtains the user voice input of the machine voice. The corresponding timestamp information; A determination module that determines the resumption playback node corresponding to the machine voice according to the timestamp information; A recovery module that plays the machine voice from the recovery play node. 12. An electronic device, characterized in that it includes: processor; and memory for storing executable instructions for the processor; Wherein, the processor is configured to execute the dialog processing method according to any one of claims 1 to 10 by executing the executable instructions. 13. A computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the dialogue processing method according to any one of claims 1 to 10 is implemented.