CN1094280C - Mute Detection Method in Internet Telephony - Google Patents

Abstract

A silence detection method in network telephone includes the following steps: when audio data of a current frame is collected, the amplitude value of the audio data is calculated to obtain short-time average energy, when the short-time average energy exceeds a speech energy critical value, the speech is detected, then the short-time average energy of the current frame passes through a low-pass filter to obtain long-time average energy of the frame and detect the long-time average energy, when the long-time average energy of continuous frames is lower than the critical value of silence energy, silence is detected, then the audio data of the current frame is collected, the conversion times of positive and negative signs of adjacent data are calculated, and once the zero-crossing rate exceeds the critical value, the noise is judged to be detected.

Description

Silence detection method in Internet telephony

Technical field

The invention relates to a mute detection method in Internet telephone.

The so-called silent detection method refers to the correct separation of voice and background noise through the various detection parts in the silent detection module in the Internet phone, so as to filter out redundant audio data, so as to obtain the best audio frequency within the limited network bandwidth. In addition, when it is detected that the voice data is not transmitted, it will transmit the silent frame data, which can save the transmission bandwidth compared with the voice data. At the same time, when the receiver receives the silent frame data, Background noise will also be played back, and the synchronization of speaking can be maintained. Secondly, there is a conversion module in the half-duplex speaking and listening mode. When receiving voice data on the network, it will automatically switch to answering mode. When the local When the voice is detected, it will automatically switch to the speaking mode, and a button is provided to facilitate the user to speak at any time.

Background technique

Generally, VoIP is an instant messaging system, but due to the limitation of network bandwidth, data transmission must be reduced as much as possible without damaging the voice quality. In addition, if the user uses a half-duplex Sound Blaster card, he cannot speak and answer at the same time. The conversation can only be achieved indirectly by switching the speaking/listening mode. In view of the above two problems, people began to learn from the mute technology in the field of speech recognition to filter out redundant voice data and realize the half-duplex speaking/listening mode. automatic conversion.

However, the mute technology is only used as an auxiliary means, and still has the following disadvantages:

1. Simple use of short-term average energy to detect whether there is silence, poor adaptability to the environment;

2. Simply filtering out the silent data without any processing may cause the asynchrony of the conversation;

3. Half-duplex automatically switches between speaking/listening mode, ignoring the initiative of the talker.

Therefore, in order to improve the above-mentioned shortcomings in the conventional technology, the present invention provides a method for detecting silence in Internet telephony.

Contents of the invention

The object of the present invention is to provide a method for detecting silence in the Internet phone, mainly through each detection part in the silence detection module on the Internet phone to correctly separate voice and background noise to filter out redundant audio data.

In order to achieve the stated purpose, the present invention provides a method for detecting silence in an Internet phone, the method is carried out by a silence detection module, and each detection part in the silence detection module includes voice detection, silence detection and noise detection, etc. The steps of the silent detection method are as follows. First, when the audio data of the current frame is collected, the short-term average energy is obtained by calculating the sum of its amplitude values. Once the short-term average energy exceeds the critical value of speech energy, it represents Speech is detected, and secondly, pass the short-term average energy of the current frame through a low-pass filter to obtain the long-term average energy of this frame and detect it. When the long-term average energy of consecutive frames is lower than the silence energy When the critical value is reached, silence is detected, and then the audio data of the current frame is collected, and the number of transformations of the positive and negative signs of the adjacent data is calculated (ie, the zero-crossing rate). Once the zero-crossing rate exceeds the critical value, that is It is determined that noise is detected, so the voice and background noise are separated through the three detections, so that the best call effect can be obtained within the limited network bandwidth.

The mute detection method in the Internet phone provided by the present invention, wherein, when the mute detection system of the transmitting party detects that the transmitting party has not transmitted voice data, it will transmit a silent frame data at the same time, and the silent frame data can save transmission compared with voice data At the same time, when the receiving party receives the silent frame data, the receiving party will also transmit the local background noise to the transmitting party. Therefore, the two parties will not feel that the conversation is out of sync due to the delay of network transmission.

In the method for detecting silence in the Internet phone provided by the present invention, there is a conversion module in the half-duplex speaking and listening mode, which will automatically convert to the answering mode when receiving voice data on the network, and when the local detection When it comes to voice, it will automatically switch to speaking mode, and it is equipped with function keys, so that users can speak at any time.

Description of drawings

In order to further understand and understand the purpose of the present invention, the characteristics of the shape structure device and its effect, the embodiment is described in detail as follows in conjunction with the accompanying drawings:

Fig. 1 is a schematic diagram of the hardware structure of the present invention.

Fig. 2 is a block diagram of the hardware structure of the full-duplex mute technology of the present invention.

Fig. 3 is a block diagram of the hardware structure of the half-duplex mute technology of the present invention.

Fig. 4 is a hardware structural diagram of the full-duplex recording end mute module of the present invention.

Fig. 5 is a flow chart of full-duplex recording data processing in the present invention.

Fig. 6 is a hardware structural diagram of the mute module of the half-duplex recording end of the present invention.

Fig. 7 is a flow chart of half-duplex recording data processing in the present invention.

Fig. 8 is a flowchart of speech detection in the present invention.

Fig. 9 is a flow chart of silence detection in the present invention.

Fig. 10 is a flowchart of noise detection in the present invention.

Fig. 11 is a hardware structural diagram of the full-duplex sound-emitting terminal mute module of the present invention.

Fig. 12 is a flow chart of full-duplex playback data processing in the present invention.

Fig. 13 is a hardware structural diagram of the half-duplex sound-emitting terminal mute module of the present invention.

Fig. 14 is a flow chart of half-duplex playback data processing in the present invention.

15A-15B are the general flowchart of the recording terminal of the present invention.

Fig. 16 is a general flow chart of the sound playback terminal of the present invention.

Detailed ways

Shown in Fig. 1 with reference to, the present invention is a kind of " mute detection method in network telephone ", and it comprises a personal computer 11, sound blaster card 12, microphone 13, loudspeaker 14, modem or network card 15 etc., wherein, microphone 13 is to convert the recorded sound signal into an electrical signal and input it into the Sound Blaster 12, and the speaker 14 converts the electrical signal output by the Sound Blaster 12 into a sound signal and sends it out, as shown in Fig. 2 and Fig. 3, General telephone call modes include full-duplex mode (as shown in Figure 2) and half-duplex mode (as shown in Figure 3). The so-called full-duplex mode means that both parties can talk and listen at the same time. It means that in the Internet telephone system, the Sound Blaster card is in the half-duplex working mode. At the current moment, only one of the recording or playback states can be performed, and the recording or playback cannot be performed at the same time. The silent detection technology of the present invention Applicable to full-duplex mode and half-duplex mode.

With reference to Fig. 2, shown in Fig. 3, it is the working block diagram that network telephone transmits and receives voice in full-duplex mode or half-duplex mode, and this full-duplex mode and half-duplex mode are both when transmitting voice. First, the recorded voice is transmitted to the mixer 21, the analog/digital conversion 22, the recording application program interface (recording API) 23, the recording end mute module 24, the modem or the network card 15 respectively through the microphone 13, and then through the network The voice is transmitted to the receiving party; while the full-duplex mode and the half-duplex mode receive voice, they first receive voice data through the network, and then the voice data passes through the modem or network card 15, the mute module at the playback end 25, receiving application program interface (receiving API) 26, digital/analog conversion 27, sound mixer 21, broadcast out by loudspeaker 14 again; In the terminal mute module 25, as for the half-duplex mode, its mute detection technology still includes a conversion module 28 to realize the automatic and forced conversion of the speaking/listening mode.

With reference to Fig. 4, shown in Fig. 5, be the hardware structural diagram and flow chart of full-duplex recording end mute module 24, at first, detect the audio data sampled by recording application program interface 23 earlier, if detect as voice data, then start Encoder 31 encodes voice data, and presses data selection switch 32 to transmit the encoded data to modem or network card 15, otherwise, if it is not voice data, close encoder 31, and press data selection switch 32 to mute Frame 33 data is sent to the modem or network card 15 .

With reference to Fig. 6, shown in Fig. 7, be the hardware structural diagram and flow chart of half-duplex recording end mute module 24, at first, detect the audio data sampled by recording application program interface 23 earlier, if detect as voice data, then start The encoder 31 encodes the voice data and transmits it to the modem or the network card 15. On the contrary, if the detection is not voice data, the encoder 31 is turned off, and the detection result is input to the conversion module 28 to start the speaking/listening mode conversion.

In the above-mentioned full-duplex and half-duplex recording end mute module 24, it has a mute detection module 34, and the present invention finds out the beginning and the beginning of the voice from the background noise by the mute detection module 34. Termination, and the silence detection module includes voice detection, silence detection and noise detection, etc.; as shown in Figure 8, the voice detection is used as the initial part of the detection voice (the beginning of a sentence or paragraph), and it uses short-term average Energy detection, first collect the audio data N of the current frame, calculate the amplitude value of the audio data N and pass short-term filtering to obtain the short-term average energy Se' of this frame, once the short-term average energy Se' exceeds When the speech energy Se reaches a critical value, it means that the speech is detected, and the audio data N starting from this frame are all regarded as speech data until silence is detected. 4

The speech detection is carried out according to the following formula:

First, calculate the speech energy Se of the current frame N is the number of voices per frame

Second, calculate the zero-crossing rate Sz of the current frame $\mathrm{Sz}=\underset{I=0}{\overset{N-1}{\mathrm{\Σ}}}\left|\mathrm{sgn}\right[\mathrm{new}\left(i\right)]-\mathrm{sgn}[\mathrm{new}(i-1)\left]\right|/2N$

Among them: sgn[Xin(i)]＝1 ， Xin(i)＞＝0

sgn[Xin(I)]＝-1 ， Xin(I)＜0

Then, after the above-mentioned speech energy is passed through a short-term filter, the short-term average energy Se is obtained,

Se=0.5Se'+0.5Se

Therefore, when (Se'>Et) and (Szmin<Sz<Szmax), it means that there is a speech signal at present, and the current state is set to the speaking state, and the program enters the silence detection; wherein, Et is the critical value of speech energy, and Szmin and Szmax are the lower limit and upper limit of the zero-crossing rate respectively.

With reference to shown in Figure 9, in the present invention, this silent detection is as the silent (sentence or the interval between paragraphs) of detection conversation gap, because the energy of silent is little and the duration is long, therefore adopt long-term average energy detection, it is Pass the short-term average energy Se' of the previous frame through a low-pass filter to obtain the long-term average energy Se' of this frame and perform detection, only when the long-term average energy Ss' of consecutive frames is lower than the silence energy When the critical Est is reached, it means that silence is detected. Therefore, the audio data N from this frame is regarded as background noise until the voice is detected. As for the number of consecutive detection frames, it is calculated from the pause time of normal conversation.

The silent detection is performed according to the following formula:

First, calculate the speech energy Se of the current frame N is the number of voices per frame

Secondly, after the above speech energy passes through the long-term filter, the long-term average energy Ss' is obtained

Ss'=0.9Ss'+0.1Ss

Then, when (Ss<Est) then CONUT++ (indicates that the technical device is progressively increased by 1);

When (Ss>Est) then CONUT=0

It means to detect silence, where Est is the critical value of silence energy, therefore, when CONUT=M, the current state is set as the answering state, and M is the number of frames required for silence.

As for the above-mentioned speech energy critical value and mute energy critical value are obtained by the following formula, first, when the user is not speaking or mute, the average value Ens of the microphone input energy is first measured, and when the user speaks and reads a sentence , the average value Ent of the microphone input energy is measured, so:

Speech energy threshold Et=Ens+0.5(Ent-Ens)

Silent energy critical value Est＝Ens+0.2(Ent-Ens)

Give an embodiment now, if each audio frequency data N=320, then through theoretical analysis and experimental result, can obtain speech energy critical value Et=250000, silent energy critical value Est=100000, and the critical value of zero-crossing rate Value, its lower limit Szmin=6, upper limit Szmax=36.

Please refer to shown in Fig. 10, among the present invention, introduced zero-crossing rate detection in noise detection, it is to collect the audio data of current one frame earlier, calculates the transformation number of its positive and negative sign of adjacent data (being zero-crossing rate ), when the zero-crossing rate is higher than the critical value, it is determined that noise is detected, and this frame of audio data is also regarded as background noise. Here is an example to illustrate how to calculate the number of positive and negative sign transformations of adjacent data, for example : This voice data is: 20, 50, 100, 40, 10, -30, -50, -10, 10, 60, 90, 50. Because the transformation of the positive and negative signs of the whole data is only twice, so its The zero-crossing rate is "2", and the critical value of the zero-crossing rate is obtained by statistical properties.

Therefore, by separating the voice and background noise through the above three detections, the best call effect can be obtained within the limited network bandwidth.

Referring to Figures 15A-15B, it is the general flowchart of the recording end, which integrates the full-duplex mute module, the half-duplex mute module and the mute detection module of the recording end. First use the voice detection method to determine the beginning of the voice, then use the silence detection method to determine the end of the voice, and finally use the noise detection method to filter out the noise mixed in the voice, so that the "pure" voice data is obtained. In the full-duplex mode, the voice data is encoded and transmitted through the network, and the silent frame data is transmitted to coordinate the synchronization of the conversation. In the half-duplex mode, the voice data is encoded and transmitted through the network, and at the same time, the speaking/listening mode is switched based on the presence or absence of voice data.

In the present invention, in the above-mentioned full-duplex recording terminal mute module 24, when it is detected that the transmitting party does not transmit voice data, a mute frame 33 data is transmitted at the same time, and the mute frame 33 data can save transmission frequency compared with voice data. Wide, so when the receiver receives the silence frame data, it means that the transmitter has not transmitted voice data, and the receiver can send the local background noise to the transmitter, so that the two parties will not have a conversation due to the delay of network transmission. Synchronous feeling; Therefore, with reference to shown in Fig. 11,12, be the hardware structural diagram and the flow chart of full-duplex sound-playing end mute module 25, at first, this network temporary test module 35 is to be responsible for monitoring modem or network card 15 received data, when monitoring to be coded data, i.e. voice data, promptly start decoder 36, and press data selection switch 32, the voice data after decoding is sent to receiving application program interface 26; Otherwise, if not When the coded data is detected, that is, the sender does not transmit voice data, the decoder 36 is turned off, and the data selection switch 32 is pressed to transmit the background noise 37 to the receiving API 26 .

With reference to Fig. 13, shown in 14, be the hardware structural diagram and flow chart of half-duplex sound-playing end mute module 25, at first, this network monitoring module 35 is to be responsible for monitoring the data that modem or network card 15 receive, when monitoring When the coded data is received, i.e. the voice data, the decoder 36 is started, and the decoded voice data is sent to the receiving API 26; otherwise, if the coded data is not detected, that is, the voice data is not transmitted, the decoder is closed 36. Input the detection result to the conversion module 28 to trigger the conversion of the listening/speaking mode.

With reference to shown in Fig. 16, it is the overall flow chart of the sound-playing end, which has integrated the full-duplex mute module and the half-duplex mute module of the sound-playing end. The mute module monitors whether the voice coded data is received at all times. In the full-duplex mode, the voice coded data is decoded and played back through the Sound Blaster card. In addition, the background noise is played back to coordinate the synchronization of the conversation; in the half-duplex mode, the voice After the coded data is decoded, it will be played back through the Sound Blaster card, and at the same time, the listening/speaking mode will be switched based on the presence or absence of speech coded data.

In the present invention, the above-mentioned conversion module 28 works in half-duplex mode, mainly to automatically or forcibly convert the speaking/listening mode. When the conversion module 28 starts, if the mute module 24 is at the recording end, the mute detection module When 34 detects voice data, keep speaking mode at once, otherwise, then automatically convert into answering mode; And if when the mute module 25 at the playback end, when network monitoring module 35 monitors voice data, keep answering mode immediately, Otherwise, it will automatically switch to the speaking mode. In addition, in the answering mode, if the user presses a function key 29, the conversion module 28 will immediately switch to the speaking mode.

To sum up, a combination of multiple detections can correctly separate speech and background noise to reduce data transmission. At the same time, special processing is performed on silent data to coordinate the synchronization of conversations. In addition, the half-duplex speaking/listening mode can be switched Freely, so that the user can speak at any time conveniently, so the present invention is actually an ideal silence detection technology.

The above descriptions are only some feasible embodiments of the present invention, but are not intended to limit the protection scope of the present invention. All equivalent implementations with other changes based on the content, features and spirit of the claims should be included in the protection scope of the present invention.

Claims (11)

1. the mute detection method in the networking telephone is used in the full duplex of the networking telephone and half-duplex call pattern correctly separating voice and background noise by each silence detection, and with the redundant voice data of filtering, described detection method is carried out as follows:

(1) detects the start-up portion of voice, when the voice data collected when former frame, by asking its amplitude and obtain short-time average energy, in case when short-time average energy surpasses the speech energy critical value, i.e. representative detects voice;

(2) detect the quiet of talk gap, to work as the short-time average energy that former frame calculates and pass through low pass filter, average energy and detecting when obtaining this frame long when average energy all is lower than quiet threshold amount of energy when consecutive numbers frame long, promptly detects quiet;

(3) detect the noise in talk gap, collect the voice data when former frame, calculate the number of transitions of its positive and negative symbol of adjacent data, i.e. the number of times of zero-crossing rate is in case when zero-crossing rate surpasses critical value, promptly judge to detect noise.

2. the mute detection method in the networking telephone as claimed in claim 1, wherein, this networking telephone when transmitting voice, earlier via microphone with the voice that are recorded to transfer to respectively mixer, mould/number conversion, recording API promptly record application program interface, the quiet module of recording end, modem or network card again by network with voice transfer to the recipient;

And wherein the quiet module of recording end at first detects the voice data of recording application program interface sampling, according to whether being voice data, determine to start encoder and carry out encoded speech data, still close encoder and the pressing data selector switch is sent to modem with quiet frame data.

3. the mute detection method in the networking telephone as claimed in claim 1, wherein, this networking telephone is when receiving voice, receive speech data by network earlier, again with this speech data via modem or network card, the quiet module of playback end, to receive API be application program interface, D/A switch, mixer, broadcast away by loud speaker again;

And wherein the quiet module of recording end at first network monitor module Monitoring Data machine whether be coded data, according to whether being that coded data starts decoder or closes decoder, if start decoder, then the pressing data selector switch is sent to the application program interface with the decoded speech data, if close decoder, then testing result input conversion module is triggered and listen/say mode switch.

4. the mute detection method in the networking telephone as claimed in claim 1, wherein, this speech detection is to adopt short-time average energy to detect, at first collect voice data N when former frame, this voice data N is asked its amplitude and also passes through filtering in short-term, and obtain the short-time average energy Se ' of this frame, when in case short-time average energy Se ' surpasses speech energy critical value Et, i.e. expression detects voice, and all be regarded as speech data from the voice data N that this frame begins, until detect quiet till.

5. the mute detection method in the networking telephone as claimed in claim 4, wherein, this speech energy critical value is to be obtained by following formula, at first, when the user is silent and quiet, record the mean value Ens of microphone intake earlier, when the user reads aloud one section sentence in a minute, record the mean value Ent of microphone intake, be thereby obtain the speech energy critical value

Et＝Ens+0.5(Ent-Ens)。

6. the mute detection method in the networking telephone as claimed in claim 1, wherein, this silence detection is that average energy detects when adopting length, at first, collect voice data N when former frame, with this voice data N ask its amplitude and and obtain the short-time average energy Se ' of this frame, at this moment, by low pass filter, and average energy Ss ' and detecting when obtaining this frame long is when having only when consecutive numbers frame long average energy Ss ' all to be lower than quiet threshold amount of energy Est with short-time average energy Se ', just expression detects quiet, therefore, all be regarded as background noise from the voice data N of this frame, till detecting voice.

7. the mute detection method in the networking telephone as claimed in claim 6, wherein, this quiet threshold amount of energy is to be obtained by following formula, at first, when the user is silent and quiet, record the mean value Ens of microphone intake earlier, when the user reads aloud one section sentence in a minute, record the mean value Ent of microphone intake, thereby obtain quiet threshold amount of energy

Est＝Ens+0.2(Ent-Ens)。

8. the mute detection method in the networking telephone as claimed in claim 2, wherein, in the quiet module of this full duplex recording end, when detecting the sender and not transmitting speech data, transmit quiet frame data simultaneously, these quiet frame data can be saved transmitting bandwidth than speech data, so when the recipient receives quiet frame data, promptly represent the sender not transmit speech data, the recipient can transmit local background noise to the sender, the retardance that makes both sides can not transmit because of network, and nonsynchronous sensation of talk is arranged.

9. the mute detection method in the networking telephone as claimed in claim 1, wherein, this networking telephone is in semiduplex mode, and it has a conversion module, makes half-duplex say/listen that pattern can change freely, can both speak at any time to make things convenient for the user.

10. the mute detection method in the networking telephone as claimed in claim 9, wherein, when the conversion module starts, if when the quiet module of recording end, when silence detection module one detects speech data, promptly keep the pattern of speaking, otherwise, then convert answer mode automatically to, and, when the network monitor module monitors speech data, promptly keep answer mode if when the quiet module of playback end, otherwise, then convert pattern in a minute automatically to.

11. the mute detection method in the networking telephone as claimed in claim 10, wherein, this conversion module also is provided with a function key, and when under answer mode, during as if user's pressing function key, the conversion module promptly forces to convert to the pattern of speaking.

Images