CN118803151A - Voice data processing method, device, equipment, medium and program product - Google Patents

Abstract

The present invention provides a voice data processing method, device, equipment, medium and program product, wherein the method comprises: obtaining uplink bandwidth data, determining a first candidate encoding format from a plurality of preset encoding formats according to the uplink bandwidth data, and determining a target encoding format based on the first candidate encoding format; sending a notification instruction to a call counterpart, and monitoring a reply message to the notification instruction sent by the call counterpart, wherein the notification instruction is used to indicate a switching of the encoding format; after receiving the reply message, encoding the voice data based on the target encoding format to obtain a voice data packet, wherein the voice data packet includes a flag bit reflecting the target encoding format, and the voice data packet is sent to the call counterpart. The present invention can realize a switching of encoding formats without additional overhead and provide a stable call experience by adaptively selecting an encoding format according to actual uplink bandwidth data during a call, and notifying the call counterpart of the change of the encoding format by adding a flag bit in the data packet.

Description

Voice data processing method, device, equipment, medium and program product

Technical Field

The present invention relates to the technical field of Internet of Things, and in particular to a voice data processing method, device, equipment, medium and program product.

Background Art

NB-IoT (Narrowband Internet of Things) focuses on the low power and wide coverage (LPWA) Internet of Things (IoT) market and is an emerging technology that can be widely used worldwide. It has the characteristics of wide coverage, multiple connections, low speed, low cost, low power consumption, and excellent architecture. NB-IOT uses the licensed frequency band and can be deployed in three ways: in-band, guard band, or independent carrier, to coexist with the existing network. At present, a large number of things have been connected to each other, and most of these connections are carried by short-range communication technologies such as Bluetooth and Wi-Fi, not by operator mobile networks. Home access control, doorbell and other systems have the requirements of multiple device connections, low speed, and low power consumption based on the family unit, as well as the characteristics of low usage frequency and low latency requirements, which are very compatible with the characteristics of NB-IoT. Therefore, many existing home smart devices use narrowband Internet of Things to achieve interconnection.

However, the stability of NB-IoT's uplink and downlink rates is still far from that of operators' mobile networks. Although the theoretical uplink rate is 16.9Kbps, the network speed is only about 3kbps under certain unstable network conditions. Due to the unstable network conditions, it is usually impossible to provide a stable call experience in this network environment.

Summary of the invention

The present invention provides a voice data processing method, device, equipment, medium and program product to solve the defect in the prior art that the narrowband Internet of Things network is unstable and cannot provide a stable call experience, and to dynamically select a voice encoding method according to the network situation to improve the stability of the call experience.

The present invention provides a voice data processing method, comprising:

Acquire uplink bandwidth data, determine a first candidate encoding format from a plurality of preset encoding formats according to the uplink bandwidth data, and determine a target encoding format based on the first candidate encoding format;

Sending a notification instruction to a call peer, and monitoring a reply message to the notification instruction sent by the call peer, wherein the notification instruction is used to instruct the coding format to switch;

After receiving the reply message, the voice data is encoded based on the target coding format to obtain a voice data packet, the voice data packet includes a flag bit reflecting the target coding format, and the voice data packet is sent to the call partner.

According to a voice data processing method provided by the present invention, the multiple preset coding formats include a Lyra coding format, and determining a first candidate coding format from the multiple preset coding formats according to the uplink bandwidth data includes:

When the uplink bandwidth data is lower than a first preset threshold, the Lyra encoding format is determined as the first candidate encoding format.

According to a voice data processing method provided by the present invention, encoding the voice data based on the target encoding format to obtain a voice data packet includes:

When the uplink bandwidth data is less than a second preset threshold, performing voice activation detection on the voice data to obtain a plurality of voice data segments and timestamps corresponding to each of the voice data segments, wherein the voice data segments include voice signals;

Encoding the plurality of voice data segments based on the target coding format to obtain a plurality of voice frames and a timestamp corresponding to each of the voice frames;

A voice data packet is generated based on a plurality of the voice frames, and the voice data packet includes a timestamp corresponding to each of the voice frames.

According to a voice data processing method provided by the present invention, before encoding the voice data based on the target encoding format, the method comprises:

Acquire collected data, perform feature extraction on the collected data, and obtain speech features, wherein the collected data is data obtained by a speech collection device through speech collection;

Inputting the speech features into a noise reduction model to obtain processing features output by the noise reduction model;

The processing feature is quantized to obtain the speech data.

According to a speech data processing method provided by the present invention, determining a target encoding format based on the first candidate encoding format includes:

Sending the first candidate encoding format to the call peer;

Acquire a first candidate decoding format sent by the call partner, and determine the target encoding format based on the first candidate encoding format and the first candidate decoding format.

A voice data processing method according to the present invention also includes:

Acquire device performance data, and determine a second candidate decoding format from a plurality of decoding formats based on the device performance data;

determining a target decoding format based on the second candidate decoding format;

The data packet sent by the call partner is decoded based on the target decoding format.

The present invention also provides a voice data processing device, comprising:

A network determination module, configured to obtain uplink bandwidth data, determine a first candidate encoding format from a plurality of preset encoding formats according to the uplink bandwidth data, and determine a target encoding format based on the first candidate encoding format;

A voice coding control module, used to send a notification instruction to a call peer and monitor a reply message sent by the call peer to the notification instruction, wherein the notification instruction is used to instruct the coding format to switch;

The voice encoding and decoding module is used to encode the voice data based on the target encoding format after receiving the reply message to obtain a voice data packet, the voice data packet including a flag reflecting the target encoding format, and send the voice data packet to the call partner.

The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned voice data processing methods is implemented.

The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the voice data processing method described in any one of the above is implemented.

The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the voice data processing method described above is implemented.

The present invention provides a voice data processing method, device, equipment, medium and program product, which obtains uplink bandwidth data, determines a first candidate encoding format from multiple preset encoding formats according to the uplink bandwidth data, determines a target encoding format based on the first candidate encoding format, determines the target encoding format, sends a notification instruction to the call peer, and after receiving a reply message from the call peer to the notification instruction, encodes the voice data based on the target encoding format to obtain a voice data packet, and inserts a flag bit in the voice data packet, which can reflect that the current encoding format has been switched to the target encoding format, and the call peer can switch to the decoding format corresponding to the target encoding format for decoding. The present invention can select the encoding format adaptively according to the actual uplink bandwidth data during the call process, so that when the uplink network rate is not good, a voice encoding technology that is more suitable for a low-bandwidth environment can be selected, and when the uplink network speed is good, a voice encoding technology that is more suitable for a high-bandwidth environment can be selected, and by adding a flag bit in the data packet to notify the call peer of the change of the encoding format, the encoding format switching without additional overhead can be achieved, providing a stable call experience.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a flow chart of a method for processing speech data provided by the present invention;

2 is a schematic diagram of the module architecture of the local terminal in the voice data processing method provided by the present invention;

3 is a schematic diagram of the module architecture of the call counterpart in the voice data processing method provided by the present invention;

4 is a diagram of heterogeneous control of encoding and decoding formats during a call in the voice data processing method provided by the invention;

5 is a schematic diagram of the process of switching encoding formats in the voice data processing method provided by the present invention;

6 is a schematic diagram of a frame fusion operation in the voice data processing method provided by the present invention;

FIG7 is a schematic diagram 1 of sound pickup enhancement in the voice data processing method provided by the present invention;

FIG8 is a second schematic diagram of sound pickup enhancement in the voice data processing method provided by the present invention;

9 is a schematic diagram of the structure of a voice data processing device provided by the present invention;

FIG. 10 is a schematic diagram of the structure of an electronic device provided by the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The following describes the voice data processing method provided by the present invention in conjunction with FIG. 1 to FIG. 8. As shown in FIG. 1, the method comprises the steps of:

S110, acquiring uplink bandwidth data, determining a first candidate coding format from a plurality of preset coding formats according to the uplink bandwidth data, and determining a target coding format based on the first candidate coding format;

S120, sending a notification instruction to the call peer, and monitoring a reply message to the notification instruction sent by the call peer;

S130. After receiving the reply message, encode the voice data based on the target coding format to obtain a voice data packet, which includes a flag reflecting the target coding format, and send the voice data packet to the call peer.

The method provided by the present invention can be executed by a narrowband Internet of Things device. As shown in Figure 2, the device may include a network judgment module, a voice coding control module, and a voice coding and decoding module. The network judgment module is used for bandwidth detection, and the voice coding control module is used to implement voice coding format switching and realize voice call coding mode selection under different network conditions. The voice coding module implements the coding function and can pre-set multiple coding formats, such as Lyra, G.711, AAC, ARM-WB, OPUS, etc.

Determining a first candidate encoding format from a plurality of preset encoding formats according to the uplink bandwidth data includes:

When the uplink bandwidth data is lower than a first preset threshold, the Lyra encoding format is determined as the first candidate encoding format.

Audio communication solutions for traditional low-power devices, such as intercom and fixed-line communication services, generally use G.711, AAC, and OPUS voice coding. Industry evaluation reports and actual tests show that at 3kbps, voice intelligibility has dropped significantly and is insufficient for normal voice communication.

Lyra is a voice coding technology developed by Google for 3kbps bandwidth environment. Under this bandwidth, its quality indicators are higher than traditional voice coding. It can greatly improve the voice intelligibility under this narrow band and realize normal voice communication.

The Lyra encoding format is not as effective as other traditional voice encodings in some high-bandwidth scenarios. In the method provided by the present invention, the Lyra encoding format is determined as the first candidate encoding format only when the uplink bandwidth data is lower than the first preset threshold value. When the uplink bandwidth is low, the encoding format that can also ensure voice intelligibility under low bandwidth can be used for encoding, thereby ensuring the stability of the call experience when the network fluctuates. When the uplink bandwidth is not lower than the first preset threshold value, traditional voice encoding that is more effective under high bandwidth can be used.

The optional coding formats include Lyra, G.711, AAC, ARM-WB, OPUS, etc. In a possible implementation, the first preset threshold may be 12 kbps, and the first candidate coding format may be determined according to the uplink bandwidth data by the following formula:

Wherein, Cs represents the first candidate coding format, v represents uplink bandwidth data, and η represents device performance data. The device performance data reflects the performance of the device. The larger the device performance data, the better the device performance.

In a possible implementation, the target encoding format is determined based on the first candidate encoding format, and the first candidate encoding format may be directly used as the target encoding format. Since the encoding is sent to the call peer end, the call peer end needs to use the corresponding decoding format for decoding to achieve a successful call. As shown in Figures 2 and 3, the terminal executing the method provided by the present invention and the call peer end communicating with it can both be provided with a media negotiation module to achieve consensus on the encoding and decoding formats supported by both parties.

Determining a target encoding format based on the first candidate encoding format includes:

Sending the first candidate encoding format to the call peer;

A first candidate decoding format sent by the call peer is obtained, and a target encoding format is determined based on the first candidate encoding format and the first candidate decoding format.

The first candidate decoding format sent by the call partner is the applicable encoding format selected by the call partner. The first candidate decoding format may be consistent with or inconsistent with the first candidate encoding format. When the first candidate encoding format and the first candidate decoding format are consistent, the target encoding format is determined to be the first candidate encoding format. When the first candidate encoding format and the first candidate decoding format are inconsistent, it means that the call partner does not support the first candidate encoding format, and the call partner selects the first candidate decoding format as the adaptive decoding format based on its own status. In this case, the encoding format corresponding to the first candidate decoding format can be used as the target encoding format.

In order to further ensure the success of the call, the method provided by the present invention performs format negotiation with the call partner after determining the first candidate encoding format to prevent the call partner from not supporting decoding of the first candidate encoding format, thereby ensuring the call success rate.

The Lyra format decoding performance requirements are very high. When tested on a 1GHZ low-power device, the decoding delay is over seconds, making real-time synchronization impossible. At the same time, the computing power of devices equipped with narrowband communications is lower, making this problem even more serious.

As shown in FIG4 , in the method provided by the present invention, the coding and decoding heterogeneous control method is adopted to reduce the computing power and realize the applicability to low-power devices, that is, the coding format of the uplink data and the decoding format of the downlink data may be inconsistent. Specifically, it includes:

Acquire device performance data, and determine a second candidate decoding format from a plurality of decoding formats based on the device performance data;

determining a target decoding format based on the second candidate decoding format;

Decode the data packets sent by the call peer based on the target decoding format.

Among them, the second candidate decoding format is a non-Lyra decoding format. Using the traditional decoding format instead of the Lyra decoding format to decode the downlink data can avoid the defect of Lyra format decoding having high performance requirements, achieve the effect of reducing the call computing power requirements and reducing power consumption, and achieve the applicability to low computing power and low power consumption devices.

In the method provided by the present invention, the Lyra decoding format is not set in the preset decoding format, and the second candidate decoding format is determined from multiple decoding formats based on the device performance data, which can be implemented by the following formula:

Cr＝common(η)

Wherein, Cr represents the second candidate decoding format, η represents device performance data, and the device performance data reflects the performance of the device. The larger the device performance data, the better the device performance.

In a possible implementation, the second candidate decoding format may be directly used as the target decoding format. Similarly, in order to ensure that the call peer can support the encoding format corresponding to the locally selected decoding format for encoding, in the method provided by the present invention, after determining the second candidate decoding format, determining the target decoding format based on the second candidate decoding format includes:

Sending the second candidate decoding format to the call peer;

Acquire a second candidate encoding format sent by the call peer, and determine a target decoding format based on the second candidate decoding format and the second candidate encoding format;

Decode the data packets sent by the call peer based on the target decoding format.

The process of determining the target decoding format based on the second candidate decoding format may be consistent with the process of determining the target encoding format based on the first candidate encoding format, and will not be described in detail herein.

After each negotiation with the other end of the call, the codec formats supported by the other end of the call can be recorded. When the first candidate encoding format is determined, and it can be determined based on the records that the other end of the call supports the corresponding decoding of the first candidate encoding format, the first candidate encoding can be directly used as the target encoding format. After determining the target encoding format, switch to the target encoding format to encode the voice data and send it to the other end of the call.

After determining the target encoding format, the target encoding format is switched to encode the voice data and sent to the call peer. Without negotiating with the call peer and directly determining the first candidate encoding as the target encoding format, the call peer will decode according to the previous decoding format by default, which will cause decoding failure. The method provided by the present invention, after determining the target encoding format, if the target encoding format is different from the current encoding format, sends a notification instruction to the call peer, and the notification instruction indicates that the encoding format switching has occurred.

Specifically, conventional SIP signaling has too much overhead under narrowband conditions, which will affect the stability of media speed transmission and reception. Even if the PB (protobuf) protocol with compression effect is used, it cannot guarantee that the voice call will not be interrupted. As shown in Figure 5, the method provided by the present invention uses the media switching notification instruction transmission in the media communication protocol RTCP to send a notification instruction to notify the call counterpart in advance to prepare for stream switching. After receiving the reply message of the call counterpart to the notification instruction, it is determined that the call counterpart has clearly indicated that the format will be switched. At this time, it switches to the target encoding format for encoding and sends the encoded data packet to the call counterpart.

Furthermore, the method provided by the present invention inserts a flag reflecting the target encoding format into the data packet after generating a voice data packet (taking the RTP data packet as an example, a flag reflecting the target encoding format can be inserted into the extension of the RTP packet). In this way, the call partner can further determine what format the received data packet needs to be decoded, thereby ensuring smooth switching of the encoding format.

In order to further improve the call quality in a worse network bandwidth environment, the method provided by the present invention encodes the voice data based on the target encoding format to obtain a voice data packet, including:

When the uplink bandwidth data is less than a second preset threshold, voice activation detection is performed on the voice data to obtain a plurality of voice data segments and a timestamp corresponding to each voice data segment, wherein the voice data segment includes a voice signal;

Encoding multiple voice data segments to obtain multiple voice frames and timestamps corresponding to each voice frame;

Encoding the multiple voice data segments based on the target coding format to obtain multiple voice frames and timestamps corresponding to the respective voice frames;

A voice data packet is generated based on multiple voice frames, and the voice data packet includes a timestamp corresponding to each voice frame.

In a traditional voice call, the collected voice data is encoded in real time. Even if there is a noise signal in the currently collected voice data, it will be encoded and transmitted. Although this can restore the real voice situation, it will cause more data transmission. When the uplink bandwidth data is less than the second preset threshold (the second preset threshold is less than the first preset threshold), that is, the uplink bandwidth data is very small, more data transmission will obviously cause the voice to be unsmooth.

As shown in FIG6 , the second preset threshold value may be 2 kpbs. In the method provided by the present invention, when the uplink bandwidth data is not less than the second preset threshold value, a normal data packet generation method is adopted, that is, after the voice data is collected, it is encoded in real time to obtain a voice frame, and each time a voice frame is obtained, an RTP data packet is generated and sent to the call counterpart. When the uplink bandwidth data is less than the second preset threshold value, a fused frame sending method is adopted, that is, after the voice data is collected, after the voice data length reaches the preset length value, a voice activation detection (VAD) is performed on it to obtain multiple voice data segments and the start time period of each voice data segment. The voice data segment is obtained by removing the noise part in the voice data after the voice activation detection. Multiple voice data segments are encoded to obtain multiple voice frames and the timestamps corresponding to each voice frame. The timestamps corresponding to the voice frames reflect the start time of the voice corresponding to the voice frame. A voice data packet is generated based on multiple voice frames, and the voice data packet includes the timestamps corresponding to each voice frame. After receiving the voice data packet, the call counterpart aligns the corresponding timestamps in each voice frame to ensure the overall voice synchronization.

The method provided by the present invention, through VAD detection, only sends the data related to the voice part. Although there will be a certain initial delay at the beginning, since the noise part is removed, only the data related to the voice part is sent and received, the transmission data volume is smaller, and in the process of accumulating voice data to a preset length value, it provides tolerance for bandwidth delay, and can achieve subsequent smooth speech.

In the actual home use environment, the device needs to have the ability to pick up sound remotely to ensure the user's call quality. However, the Lyra encoding format has obvious shortcomings in remote sound pickup processing. When the sound source exceeds a certain distance, the intelligibility of the Lyra encoding result decreases significantly, and the mechanical sound is serious. For low-power narrowband devices, cost is one of its important considerations, because the computing power of the device generally cannot support overly complex AI noise reduction and AGC algorithms. Therefore, it is inappropriate to improve the sound pickup ability of Lyra encoding by deploying more complex AI noise reduction algorithms and AGC algorithms on the device. In the method provided by the present invention, before encoding the voice data based on the target coding format, it includes:

Acquire collected data, perform feature extraction on the collected data, and obtain speech features, wherein the collected data is data obtained by a speech collection device through speech collection;

Input the speech features into the noise reduction model to obtain the processed features output by the noise reduction model;

The processed features are quantized to obtain speech data.

That is to say, in the method provided by the present invention, when the original data collected by the voice acquisition device is denoised, it is not like the traditional method shown in FIG7, directly using the collected voice (i.e., collected data) as the input of the denoising model, and then performing time domain conversion on the output data of the denoising model and then extracting features to obtain MFCC features, which are then quantized and converted into voice data for subsequent encoding. Instead, as shown in FIG8, feature extraction is first performed on the collected data to obtain MFCC voice features, and the voice features are used as the input of the denoising model. The denoising model outputs the processed MFCC features (i.e., processed features), and the processed features are quantized to obtain voice data for subsequent encoding. The denoising model can be an AI denoising model, and its loss function is:

Where g _b is the gain estimation parameter output by the model, is the real gain data, and γ is the perception parameter used to control the strength of noise suppression.

After testing and verification, on a 600Mhz low-power device, compared with the traditional method, the method provided by the present invention reduces the overall overhead by 15% by collecting speech features as the input of the noise reduction model, and increases the pickup distance by 1.5 meters while keeping the audio MOS unchanged. This effectively makes up for the defect of insufficient pickup distance of Lyra encoding, and does not require the deployment of more complex model algorithms, thereby increasing the pickup distance of low-power devices.

The following is a description of the speech data processing device provided by the present invention. The speech data processing device described below and the speech data processing method described above can be referred to in correspondence with each other. As shown in FIG9 , the speech data processing device provided by the present invention includes:

The network determination module 910 is used to obtain uplink bandwidth data, determine a first candidate encoding format from a plurality of preset encoding formats according to the uplink bandwidth data, and determine a target encoding format based on the first candidate encoding format;

The speech coding control module 920 is used to send a notification instruction to the call peer and monitor the reply message sent by the call peer to the notification instruction, where the notification instruction is used to instruct the coding format to switch;

The voice encoding and decoding module 930 is used to encode the voice data based on the target encoding format after receiving the reply message to obtain a voice data packet, which includes a flag reflecting the target encoding format, and send the voice data packet to the call partner.

FIG10 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG10 , the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the voice data processing method, the method comprising: obtaining uplink bandwidth data, determining a first candidate encoding format from a plurality of preset encoding formats according to the uplink bandwidth data, and determining a target encoding format based on the first candidate encoding format; sending a notification instruction to the call counterpart, and monitoring a reply message sent by the call counterpart to the notification instruction, the notification instruction is used to indicate the switching of the encoding format; after receiving the reply message, encoding the voice data based on the target encoding format to obtain a voice data packet, the voice data packet includes a flag reflecting the target encoding format, and sending the voice data packet to the call counterpart.

In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the voice data processing method provided by the above methods, the method including: obtaining uplink bandwidth data, determining a first candidate encoding format from multiple preset encoding formats according to the uplink bandwidth data, and determining a target encoding format based on the first candidate encoding format; sending a notification instruction to the call counterpart, and listening to a reply message sent by the call counterpart to the notification instruction, the notification instruction is used to indicate the switching of the encoding format; after receiving the reply message, encoding the voice data based on the target encoding format to obtain a voice data packet, the voice data packet including a flag reflecting the target encoding format, and sending the voice data packet to the call counterpart.

On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the voice data processing method provided by the above-mentioned methods, the method comprising: obtaining uplink bandwidth data, determining a first candidate encoding format from multiple preset encoding formats according to the uplink bandwidth data, and determining a target encoding format based on the first candidate encoding format; sending a notification instruction to the call counterpart, and listening to a reply message sent by the call counterpart to the notification instruction, the notification instruction being used to indicate a coding format switch; after receiving the reply message, encoding the voice data based on the target coding format to obtain a voice data packet, the voice data packet including a flag reflecting the target coding format, and sending the voice data packet to the call counterpart.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method of processing speech data, comprising:

Acquiring uplink bandwidth data, determining a first candidate coding format in a plurality of preset coding formats according to the uplink bandwidth data, and determining a target coding format based on the first candidate coding format;

Sending a notification instruction to a call opposite terminal, and monitoring a reply message which is sent by the call opposite terminal and is aimed at the notification instruction, wherein the notification instruction is used for indicating the switching of coding formats;

After receiving the reply message, encoding voice data based on the target encoding format to obtain a voice data packet, wherein the voice data packet comprises a flag bit reflecting the target encoding format, and the voice data packet is sent to the opposite call end.

2. The method according to claim 1, wherein the plurality of preset encoding formats includes a Lyra encoding format, and the determining a first candidate encoding format among the plurality of preset encoding formats according to the uplink bandwidth data includes:

And when the uplink bandwidth data is lower than a first preset threshold value, determining a Lyra coding format as the first candidate coding format.

3. The method for processing voice data according to claim 1, wherein the encoding the voice data based on the target encoding format to obtain a voice data packet comprises:

when the uplink bandwidth data is smaller than a second preset threshold value, voice activation detection is carried out on the voice data to obtain a plurality of voice data segments and time stamps corresponding to the voice data segments, wherein the voice data segments comprise voice signals;

Encoding the voice data segments based on the target encoding format to obtain a plurality of voice frames and corresponding time stamps of the voice frames;

And generating one voice data packet based on a plurality of voice frames, wherein the voice data packet comprises a time stamp corresponding to each voice frame.

4. The method according to claim 1, characterized in that before encoding the voice data based on the target encoding format, comprising:

Acquiring acquisition data, and performing feature extraction on the acquisition data to obtain voice features, wherein the acquisition data is data obtained by voice acquisition by a voice acquisition device;

Inputting the voice characteristics into a noise reduction model, and obtaining processing characteristics output by the noise reduction model;

and carrying out quantization processing on the processing characteristics to obtain the voice data.

5. The method of claim 1, wherein the determining a target coding format based on the first candidate coding format comprises:

Transmitting the first candidate coding format to the call opposite terminal;

And acquiring a first candidate decoding format sent by the call opposite terminal, and determining the target coding format based on the first candidate coding format and the first candidate decoding format.

6. The voice data processing method of claim 1, wherein the method further comprises:

Acquiring device performance data, determining a second candidate decoding format among a plurality of decoding formats based on the device performance data;

determining a target decoding format based on the second candidate decoding format;

and decoding the data packet sent by the opposite call terminal based on the target decoding format.

7. A voice data processing apparatus, comprising:

the network judgment module is used for acquiring uplink bandwidth data, determining a first candidate coding format in a plurality of preset coding formats according to the uplink bandwidth data, and determining a target coding format based on the first candidate coding format;

The voice coding control module is used for sending a notification instruction to a call opposite terminal and monitoring a reply message which is sent by the call opposite terminal and is aimed at the notification instruction, wherein the notification instruction is used for indicating the switching of coding formats;

and the voice encoding and decoding module is used for encoding voice data based on the target encoding format after receiving the reply message to obtain a voice data packet, wherein the voice data packet comprises a flag bit reflecting the target encoding format, and the voice data packet is sent to the opposite call end.

8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the speech data processing method according to any one of claims 1 to 6 when executing the program.

9. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the speech data processing method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program which, when executed by a processor, implements the speech data processing method according to any one of claims 1 to 6.