HK1254791B - Companding apparatus and method to reduce quantization noise using advanced spectral extension - Google Patents

Description

Companding device and method for reducing quantization noise using advanced spectrum extension

This application is a divisional application of the invention patent application with application number 201480008819.0, application date April 1, 2014, and invention name “Compression and expansion device and method for reducing quantization noise using advanced spectrum extension”.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent Application Nos. 61/809,028, filed April 5, 2013, and 61/877,167, filed September 12, 2013, which are hereby incorporated by reference in their entireties.

Technical Field

One or more embodiments relate generally to audio signal processing, and more particularly to reducing coding noise in audio codecs using compression/expansion (companding) techniques.

Background Art

Many popular digital audio formats utilize lossy data compression techniques that discard some of the data to reduce storage or data rate requirements. The application of lossy data compression not only reduces the fidelity of the source content (e.g., audio content), but it can also introduce perceptible distortion in the form of compression artifacts. In the context of audio coding systems, these acoustic artifacts are referred to as coding noise or quantization noise.

Digital audio systems utilize codecs (encoder-decoder components) to compress and decompress audio data according to defined audio file formats or streaming audio formats. Codecs implement algorithms that attempt to represent audio signals with the minimum number of bits while maintaining the highest possible fidelity. The lossy compression techniques typically used in audio codecs work on a psychoacoustic model of human auditory perception. Audio formats typically involve the use of time/frequency domain transforms (e.g., modified discrete cosine transforms - MDCTs) and use masking effects such as frequency masking or time masking so that certain sounds, including any apparent quantization noise, are hidden or masked by the actual content.

Most audio coding systems are frame-based. Within a frame, the audio codec typically shapes the coding noise in the frequency domain so that it becomes the most difficult to hear. Several current digital audio formats utilize frames of this long duration so that a frame can contain sounds of several different levels or intensities. Because coding noise is typically stable in level as the frame evolves, coding noise may be most audible during the low-intensity portion of the frame. This effect can manifest as pre-echo distortion, in which the silence (or low-level signal) before the high-intensity segment is drowned out by the noise in the decoded audio signal. This effect may be most significant in transient sounds or pulses from percussion instruments such as castanets or other sharp percussive sound sources. This distortion is typically caused by quantization noise introduced in the frequency domain and spread over the entire transform window of the codec in the time domain.

Current measures to avoid or minimize pre-echo artifacts include the use of filters. However, such filters introduce phase distortion and temporal smearing. Another possible solution involves using a smaller transform window, but this approach can significantly reduce frequency resolution.

The subject matter discussed in the Background section should not be considered prior art simply because it is mentioned in the Background section. Similarly, problems mentioned in or related to the subject matter in the Background section should not be considered to have been previously recognized in the prior art. The subject matter in the Background section merely represents different approaches, which may themselves be inventions.

Summary of the Invention

Embodiments are directed to a method for processing a received audio signal by extending the audio signal to an extended dynamic range via a process comprising: dividing the received audio signal into a plurality of time segments using a defined window shape, calculating a wideband gain in the frequency domain for each time segment using a non-energy-based average of a frequency domain representation of the audio signal, and applying a gain value to each time segment to obtain an extended audio signal. The gain value of the wideband gain applied to each time segment is selected to have the effect of amplifying segments of relatively high intensity and attenuating segments of relatively low intensity. For this method, the received audio signal comprises an original audio signal compressed from an original dynamic range via a compression process comprising: dividing the original audio signal into a plurality of time segments using a defined window shape, calculating a wideband gain in the frequency domain using a non-energy-based average of frequency domain samples of the original audio signal, and applying the wideband gain to the original audio signal. In the compression process, the gain value of the wideband gain applied to each time segment is selected to have the effect of amplifying segments of relatively low intensity and attenuating segments of relatively high intensity. The expansion process is configured to substantially restore the dynamic range of the original audio signal, and the wideband gain of the expansion process may be substantially the inverse of the wideband gain of the compression process.

In a system implementing a method for processing a received audio signal through an expansion process, a filter bank component can be used to analyze the audio signal to obtain its frequency domain representation, and the window shape defined for segmenting into multiple time segments can be the same as the prototype filter used for the filter bank. Similarly, in a system implementing a method for processing a received audio signal through a compression process, a filter bank component can be used to analyze the original audio signal to obtain its frequency domain representation, and the window shape defined for segmenting into multiple time segments can be the same as the prototype filter used for the filter bank. The filter bank in either case can be one of a QMF bank or a short-time Fourier transform. In this system, the received signal for the expansion process is obtained after the compressed signal is corrected by an audio encoder that generates a bitstream and a decoder that decodes the bitstream. The encoder and decoder can include at least a portion of a transform-based audio codec. The system can also include a component that processes control information received through the bitstream and determines the activation state of the expansion process.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following figures, like reference numerals are used to denote like elements. Although the following figures depict various examples, one or more implementations are not limited to the examples depicted in the figures.

FIG1 illustrates a system for compressing and expanding an audio signal in a transform-based audio codec, under one embodiment.

FIG. 2A illustrates an audio signal divided into a plurality of short time segments, under one embodiment.

FIG. 2B illustrates the audio signal of FIG. 2A after a broadband gain is applied to each short time segment, under one embodiment.

FIG3A is a flowchart illustrating a method of compressing an audio signal under one embodiment.

FIG3B is a flow chart illustrating a method of extending an audio signal under an embodiment.

FIG4 is a block diagram illustrating a system for compressing an audio signal, under one embodiment.

FIG5 is a block diagram illustrating a system for extending an audio signal, under an embodiment.

FIG6 illustrates the segmentation of an audio signal into multiple short time segments under one embodiment.

DETAILED DESCRIPTION

Systems and methods are described for implementing temporal noise shaping of quantization noise in audio codecs using companding techniques. These embodiments include implementing temporal shaping of quantization noise using a companding algorithm implemented in the QMF domain. The process includes encoder control of the desired level of companding at the decoder, as well as extending companding beyond monophonic applications to stereo and multichannel applications.

Aspects of one or more embodiments described herein can be implemented in an audio system for processing audio signals for transmission across a network comprising one or more computers or processing devices that execute software instructions. Any of the described embodiments can be used alone or in any combination together. Although the various embodiments may be inspired by various defects of the prior art that may be discussed or briefly mentioned in one or more places in the specification, the embodiments do not necessarily solve any of these defects. In other words, different embodiments may solve different defects that may be discussed in the specification. Some embodiments may only partially solve some of the defects that may be discussed in the specification or only one defect, and some embodiments may not solve any of these defects.

FIG1 illustrates, under one embodiment, a compression and expansion system for reducing quantization noise in a codec-based audio processing system. FIG1 illustrates an audio signal processing system built around an audio codec including an encoder (or "core encoder") 106 and a decoder (or "core decoder") 112. The encoder 106 encodes the audio content into a data stream or signal for transmission over a network 110, where it is decoded by the decoder 112 for playback or further processing. In one embodiment, the encoder 106 and decoder 112 of the codec implement a lossy compression method to reduce the storage and/or data rate requirements of the digital audio data, and such a codec can be implemented as MP3, Vorbis, Dolby Digital (AC-3), AAC or a similar codec. The lossy compression method of the codec generates coding noise, which typically has a steady level as the frame defined by the codec evolves. This coding noise is often most audible during the low-intensity portion of the frame. By providing a pre-compression step component 104 preceding the core encoder 106 of the codec and a post-expansion step component 114 operating on the output of the core decoder 112, system 100 includes components for reducing perceived coding noise in existing coding systems. Compression component 104 is configured to divide the original audio input signal 102 into a plurality of time segments using a defined window shape. A broadband gain is calculated and applied in the frequency domain using a non-energy-based average of the frequency domain samples of the original audio signal, where the gain value applied to each time segment amplifies segments of relatively low intensity and attenuates segments of relatively high intensity. This gain modification has the effect of compressing and significantly reducing the original dynamic range of the input audio signal 102. The compressed audio signal is then encoded in encoder 106, transmitted over network 110, and decoded in decoder 112. The decoded compressed signal is input to expansion component 114, which is configured to perform the inverse operation of pre-compression step 104 by applying an inverse gain value to each time segment, thereby expanding the dynamic range of the compressed audio signal back to the dynamic range of the original input audio signal 102. Thus, the audio output signal 116 includes an audio signal with the original dynamic range, while the coding noise is removed by the pre-step and post-step companding processes.

As shown in Figure 1, the compression component or pre-compression step 104 is configured to reduce the dynamic range of the audio signal 102 input to the core encoder 106. The input audio signal is divided into many short segments. The size or length of each short segment is a small fraction of the frame size used by the core encoder 106. For example, a typical frame size of the core encoder may be approximately 40 to 80 milliseconds. In this case, each short segment may be approximately 1 to 3 milliseconds. The compression component 104 calculates appropriate broadband gain values to compress the input audio signal on a segment-by-segment basis. This is achieved by modifying the short segments of the signal with an appropriate gain value for each segment. Relatively large gain values are selected to amplify segments of relatively low intensity, and small gain values are selected to attenuate segments of high intensity.

Fig. 2A illustrates an audio signal divided into multiple short time segments under one embodiment, and Fig. 2B illustrates the same audio signal after applying broadband gain by a compression component. As shown in Fig. 2A, audio signal 202 represents a transient or sound pulse that may be produced by a percussion instrument (e.g., castanets). As shown in the plot of voltage V versus time t, the signal is characterized by a peak in amplitude. Generally speaking, the amplitude of a signal is related to the sound energy or intensity of the sound and represents a measure of the sound power at any point in time. When processing audio signal 202 by a frame-based audio codec, the part of the signal is processed within a transform (e.g., MDCT) frame 204. Typical digital audio systems currently utilize frames of relatively long duration so that for sharp transients or short pulse sounds, a single frame can include both low-intensity and high-intensity sounds. Therefore, as shown in Fig. 1, a single MDCT frame 204 includes the pulse portion (peak) of the audio signal and a relatively large amount of low-intensity signals before or after the peak. In one embodiment, the compression component 104 divides the signal into a number of short time segments 206 and applies a broadband gain to each segment to compress the dynamic range of the signal 202. The number and size of each short segment can be selected based on application requirements and system constraints. Relative to the size of a single MDCT frame, the number of short segments can vary from 12 to 64 segments and can typically include 32 segments, but the embodiment is not limited thereto.

FIG2B illustrates the audio signal of FIG2A after applying a broadband gain to each short time segment, under one embodiment. As shown in FIG2B , the audio signal 212 has the same relative shape as the original signal 202, but the amplitude of the low-intensity segments has been increased by applying the amplification gain value, while the amplitude of the high-intensity segments has been reduced by applying the attenuation gain value.

The output of the core decoder 112 is the input audio signal (e.g., signal 212) with a reduced dynamic range, plus the quantization noise introduced by the core encoder 106. This quantization noise is characterized by a nearly uniform level over time within each frame. The expansion component 114 acts on the decoded signal to restore the dynamic range of the original signal. It uses the same short temporal resolution based on the short segment size 206 and inverts the gain applied by the compression component 104. Thus, the expansion component 114 applies a small gain (attenuation) to segments of the original signal that have low intensity and have been amplified by the compressor, while applying a large gain (amplification) to segments of the original signal that have high intensity and have been attenuated by the compressor. The quantization noise added by the core encoder and having a uniform temporal envelope is thus simultaneously shaped by the post-processor gain to approximately match the temporal envelope of the original signal. This process effectively makes the quantization noise less audible during quiet segments. While the noise may be amplified during high-intensity segments, it remains less audible due to the masking effect of the loud signals in the audio content itself.

As shown in Figure 2A, the companding process modifies discrete segments of the audio signal using individual gain values. In some cases, this can result in discontinuities at the output of the compression component, potentially causing problems in the core encoder 106. Similarly, discontinuities in gain at the expansion component 114 can result in discontinuities in the envelope of the shaped noise, potentially causing audible clicks in the audio output 116. Another problem with applying individual gain values to short segments of the audio signal stems from the fact that a typical audio signal is a mixture of many separate sources. Some of these sources may be stationary in time, while some may be transient. Stationary signals are typically time-constant in terms of statistical parameters, whereas transient signals are typically not. Given the broadband nature of transients, their presence in such a mixture is often more visible at higher frequencies. Gain calculations based on the signal's short-term energy (RMS) tend to be biased towards stronger low frequencies, resulting in them being dominated by stationary sources and appearing to be virtually constant over time. Consequently, these energy-based approaches are generally ineffective at shaping the noise introduced by the core encoder.

In one embodiment, system 100 calculates and applies gains at the compression and expansion components in a filterbank with short prototype filters to address potential issues associated with applying individual gain values. The signal to be modified (the original signal at compression component 104 and the output of core decoder 112 at expansion component 114) is first analyzed by the filterbank, and a broadband gain is applied directly in the frequency domain. The corresponding effect in the time domain is to naturally smooth the gain application according to the shape of the prototype filter. This resolves the discontinuity issue discussed above. The modified frequency-domain signal is then converted back to the time domain via the corresponding synthesis filterbank. Analyzing the signal with the filterbank provides access to its spectral content and allows calculation of gains that preferentially increase contributions due to high frequencies (or any spectral content that is weak), providing gain values that are not dominated by the strongest components in the signal. This addresses the issues associated with audio sources comprising a mixture of different sources, as discussed above. In one embodiment, the system calculates the gain using the p-norm of the spectral magnitude, where p is typically less than 2 (p<2). This enables greater emphasis on weak spectral content than when based on energy (p=2).

As mentioned above, described system comprises prototype filter and applies with smooth gain.Generally speaking, prototype filter is the basic window shape in the bank of filters, and it is modulated by sinusoidal waveform to obtain impulse response for the different sub-band filters in the bank of filters.For example, short time Fourier transform (STFT) is a bank of filters, and each frequency line of this conversion is the sub-band of bank of filters.Short time Fourier transform is realized by multiplying the signal by window shape (N sampling window), and window shape can be rectangular, Hann, Kaiser-Bessel derive (KBD) or some other shapes.The signal that is windowed then experiences discrete Fourier transform (DFT) operation, to obtain STFT.The window shape in this case is prototype filter.DFT is made up of sinusoidal basis function, and each sinusoidal basis function has different frequencies.Then, the window shape multiplied by sinusoidal function provides filter for the sub-band corresponding to this frequency.Because window shape is identical at all frequencies, it is called " prototype ".

In one embodiment, the system uses a QMF (quadrature modulated filter) bank for the filter bank. In a specific implementation, the QMF bank can have a 64-point window that forms a prototype. This window, modulated by cosine and sine functions (corresponding to 64 equally spaced frequencies), forms the subband filter for the QMF bank. After each application of the QMF function, the window shifts by 64 samples, meaning that the overlap between time segments in this case is 640-64=576 samples. However, although the window shape in this case spans ten time segments (640=10*64), the main lobe of the window (where its sample values are most significant) is approximately 128 samples long. Therefore, the effective length of the window is still relatively short.

In one embodiment, expansion component 114 is ideally inversely transformed by the gain applied by compression component 104. Although the gain applied by compression component can be transmitted to decoder by bitstream, this method typically can consume significant bit rate. In one embodiment, system 100 is directly from its available signal, i.e. the output of decoder 112, estimating the required gain of expansion component 114, which effectively does not require additional bits. The bank of filters at compression and expansion component place is selected to be identical, so as to calculate the gain that is inverse to each other. In addition, these bank of filters are time-synchronized so that any effective delay between the output of compression component 104 and the input of expansion component 114 is the multiple of the stride of the bank of filters. If the core encoder-decoder is lossless, and bank of filters provides perfect reconstruction, then the gain at compression and expansion component place will be each other's accurate inverse, thereby allowing accurate reconstruction of the original signal. But in practice, the gain applied by expansion component 114 is only an approximation of the inverse of the gain applied by compression component 104.

In one embodiment, the filter bank used in the compression and expansion component is a QMF bank. In a typical use case, the core audio frame may be 4096 samples long and overlap with adjacent frames by 2048 samples. At 48kHz, the length of such a frame is 85.3 milliseconds. In contrast, the QMF bank used may have a stride of 64 samples (which is 1.3 milliseconds in length), which provides fine temporal resolution for the gain. Moreover, QMF has a smooth prototype filter of 640 samples in length, ensuring that the gain applied varies smoothly over time. Analysis using this QMF filter bank provides a time-frequency flattened representation of the signal. Each QMF time slot is equal to the stride, and within each QMF time slot, there are 64 evenly spaced subbands. Alternatively, other filter banks such as the Short-Term Fourier Transform (STFT) can be used and still achieve this time-frequency flattened representation.

In one embodiment, compression component 104 performs a pre-processing step to scale the codec input. For this embodiment, _St (k) is a complex-valued filter bank sample at time slot t and frequency bin k. FIG6 illustrates the segmentation of an audio signal into a number of time slots for a range of frequencies under one embodiment. For the embodiment of FIG600, there are 64 frequency bins k and 32 time slots t, which produce multiple time-frequency blocks as shown (although not necessarily drawn to scale). The pre-compression step scales the codec input to become _S't (k)= _St (k)/ _gt . In this equation, is the normalized time slot average.

In the equation above, the expression is the mean absolute level/1-norm, and S ₀ is a suitable constant. The general p-norm is defined in this context as follows:

It has been shown that the 1-norm can give significantly better results than using the energy (rms/2-norm). The value of the exponential term γ is typically in the range of 0 to 1 and can be chosen to be 1/3. The constant _S0 ensures a reasonable gain value independent of the implementation platform. For example, it may be 1 when implemented in a platform where the absolute value of all _St (k) values may be constrained to be 1. It may be different in platforms where _St (k) may have different maximum absolute values. It can also be used to ensure that the average gain value of a large set of signals is close to 1. That is, it can be an intermediate signal value between the maximum signal value and the minimum signal value determined from a large corpus of content.

In a post-process performed by the expansion component 114, the codec output is expanded by the inverse gain applied by the compression component 104. This requires an exact or nearly exact replication of the filter bank of the compression component. In this case, the complex-valued samples of this second filter bank are represented. The expansion component 114 scales the codec output to become

In the above equation, is the normalized time slot average, given by:

as well as

Generally speaking, the expansion component 114 will use the same p-norm as used in the compression component 104. Thus, if the mean absolute level is used to define φ in the compression component 104, φ in the above equation is also defined using the 1-norm (p=1).

When complex filter banks (containing both cosine and sine basis functions) such as STFT or complex QMF are used in the compression and expansion components, the calculation of the magnitude of the complex subband samples or |S _t (k)| requires computationally intensive square root operations. This can be circumvented by approximating the magnitude of the complex subband samples in various ways, such as by calculating the sum of their real and imaginary parts.

In the above equation, the value K is equal to or less than the number of subbands in the filter bank. In general, any subset of the subbands in the filter bank can be used to calculate the p-norm. However, the same subset should be used at both the encoder 106 and the decoder 112. In one embodiment, the advanced spectral extension (A-SPX) tool can be used to encode the high-frequency portion of the audio signal (e.g., audio components above 6 kHz). Alternatively, it may be desirable to use only signals above 1 kHz (or similar frequencies) to guide noise shaping. In this case, only those subbands in the range of 1 kHz to 6 kHz can be used to calculate the p-norm, and thus the gain value. Furthermore, although the gain is calculated based on a subset of the subbands, it can still be applied to a different and potentially larger subset of the subbands.

As shown in Figure 1, the companding function for shaping the quantization noise introduced by the core encoder 106 of the audio codec is performed by two separate components 104 and 114 that perform certain pre-encoder compression functions and post-decoder expansion functions. Figure 3A is a flow chart illustrating a method for compressing an audio signal in a pre-encoder compression component under one embodiment, and Figure 3B is a flow chart illustrating a method for expanding an audio signal in a post-decoder expansion component under one embodiment.

As shown in FIG3A , the process 300 begins with the compression component receiving an input audio signal (302). The component then divides the audio signal into short time segments (304) and compresses the audio signal to a reduced dynamic range by applying a broadband gain value to each short segment (306). The compression component also implements some prototype filtering and QMF filter components to reduce or eliminate any discontinuities caused by applying different gain values to consecutive segments, as described above (308). In some cases, such as based on the type of audio content or certain characteristics of the audio content, the compression and expansion of the audio signal before and after the encoding/decoding stage of the audio codec may degrade rather than enhance the output audio quality. In such cases, the companding process can be turned off or modified to return different levels of companding (compression/expansion). Thus, the compression component determines, among other variables, the appropriateness of the companding function and/or the optimal level of companding required for the specific signal input and audio playback environment (310). This determination step 310 can occur at any practical point in the process 300, such as before the division 304 of the audio signal or the compression 306 of the audio signal. If companding is deemed appropriate, gain is applied (306), and the encoder then encodes a signal for transmission to the decoder (312) in accordance with the codec's data format. Certain companding control data, such as activation data, synchronization data, companding level data, and other similar control data, may be transmitted as part of the bitstream for processing by the expansion component.

FIG3B is a flow chart illustrating a method for extending an audio signal in an expansion component after a decoder under one embodiment. As shown in process 350, the decoder stage of the codec receives a bitstream of the encoded audio signal from the encoder stage (352). The decoder then decodes the encoded signal according to the codec data format (353). The expansion component then processes the bitstream and applies any encoded control data to turn off the extension or modify the extension parameters based on the control data (354). The expansion component uses a suitable window shape to divide the audio signal into time segments (356). In one embodiment, the time segments correspond to the same time segments used by the compression component. The expansion component then calculates an appropriate gain value (358) for each segment in the frequency domain and applies the gain value to each time segment to extend the dynamic range of the audio signal back to the original dynamic range or any other appropriate dynamic range (360).

Companding control

The compression and expansion components of the compandor of system 100 can be configured to apply the pre-processing and post-processing steps only at a certain time during the audio signal processing, or only for certain types of audio content. For example, companding may show benefits for transient signals of speech and music. However, for other signals such as stationary signals, companding may degrade the signal quality. Therefore, as shown in Figure 3A, a companding control mechanism is provided as block 310, and control data is transmitted from compression component 104 to expansion component 114 to coordinate the companding operation. The simplest form of this control mechanism is to turn off the companding function for blocks of audio samples where applying companding would degrade the audio quality. In one embodiment, the companding on/off decision is detected in the encoder and transmitted to the decoder as a bitstream element, so that the compressor and expander can be turned on/off in the same QMF time slot.

Switching between the two states typically results in a discontinuity in the applied gain, resulting in audible switching artifacts or clicks. Embodiments include mechanisms for reducing or eliminating these artifacts. In a first embodiment, the system allows the companding function to be switched off and on only at frames where the gain is close to 1. In this case, there is only a small discontinuity between turning the companding function on/off. In a second embodiment, a third weak companding mode, intermediate between the on and off modes, is applied in the audio frames between the on and off frames and signaled in the bitstream. The weak companding mode causes the exponential term γ to slowly transition from its default value during companding to 0, which is equivalent to no companding. As an alternative to the intermediate weak companding mode, the system can implement start and stop frames that smoothly fade into the no companding mode over a block of audio samples, rather than abruptly turning off the companding function. In another embodiment, the system is configured not to simply turn off the companding, but to apply an average gain. In some cases, the audio quality of pitch-smooth signals can be improved by applying a constant gain factor to an audio frame that is more similar to the gain factor of an adjacent frame with companding on than to a constant gain factor of 1.0 with companding off. This gain factor can be calculated by averaging all companding gains over a frame. Frames containing a constant average companding gain are thus signaled in the bitstream.

While the embodiments are described in the context of a monophonic audio channel, it should be noted that, in a straightforward extension, multiple channels can be processed by repeating the method separately for each channel. However, audio signals containing two or more channels present certain additional complexities that are addressed by the embodiment of the companding system of FIG1 . The companding strategy should depend on the similarity between the channels.

For example, in the case of stereo-panned transient signals, it has been observed that independent companding of individual channels can lead to audible image artifacts. In one embodiment, the system determines a single gain value for each time segment based on the subband samples of the two channels, and uses the same gain value to compress/expand both signals. This approach is generally suitable whenever two channels have very similar signals, where the similarity is defined, for example, by using cross-correlation. The detector calculates the similarity between the channels and switches between using individual companding of the channels or companding the channels together. Extension to multiple channels can be achieved by dividing the channels into channel groups using a similarity criterion and applying common companding to the groups. This grouping information can then be transmitted via the bitstream.

System Implementation

Fig. 4 is a block diagram illustrating a system for compressing an audio signal in conjunction with the encoder stage of a codec, under one embodiment. Fig. 4 illustrates a hardware circuit or system for implementing at least a portion of a compression method for the codec-based system shown in Fig. 3A. As shown in system 400, an input audio signal 401 in the time domain is input to a QMF filter bank 402. This filter bank performs an analysis operation that separates the input signal into multiple components, wherein each bandpass filter carries the frequency subbands of the original signal. Signal reconstruction is performed in a synthesis operation performed by a QMF filter bank 410. In the exemplary embodiment of Fig. 4, both the analysis and synthesis filter banks process 64 frequency bands. A core encoder 412 receives the audio signal from the synthesis filter bank 410 and generates a bitstream 414 by encoding the audio signal in a suitable digital format (e.g., MP3, AAC, etc.).

System 400 includes a compressor 406 that applies a gain value to each short segment into which the audio signal has been divided. This produces an audio signal with a compressed dynamic range, such as that shown in FIG2B . A companding control unit 404 analyzes the audio signal to determine whether or how much compression should be applied based on the type of signal (e.g., speech) or the characteristics of the signal (e.g., stationary versus transient) or other relevant parameters. The control unit 404 may include a detection mechanism to detect the temporal kurtosis characteristic of the audio signal. Based on the detected characteristics of the audio signal and some predefined criteria, the control unit 404 sends an appropriate control signal to the compressor 406 to disable the compression function or modify the gain value applied to the short segment.

In addition to companding, many other coding tools can also operate in the QMF domain. One such tool is A-SPX (Advanced Spectral Extension) shown in block 408 of Figure 4. A-SPX is a technique used to allow less important frequencies to be encoded with a coarser coding scheme than more perceptually important frequencies. For example, in A-SPX at the decoder end, QMF subband samples from lower frequencies can be repeated at higher frequencies, and the spectral envelope in the high-frequency band is then shaped using auxiliary information transmitted from the encoder to the decoder.

In a system where both companding and A-SPX are performed in the QMF domain, as shown in FIG4 , at the encoder, A-SPX envelope data for higher frequencies may be extracted from the uncompressed subband samples, and compression may be applied to lower frequency QMF samples corresponding to the frequency range of the signal encoded by the core encoder 412. At the decoder 502 of FIG5 , after QMF analysis 504 is performed on the decoded signal, an expansion process 506 is first applied, and an A-SPX operation 508 then reproduces the higher subband samples from the lower frequency expanded signal.

In this exemplary implementation, the QMF synthesis filter bank 410 at the encoder and the QMF analysis filter bank at the decoder 504 together introduce a 640-64+1 sample delay (~9 QMF time slots). The core codec delay in this example is 3200 samples (50 QMF time slots), so the total delay is 59 time slots. This delay is caused by embedding control data in the bitstream and using it at the decoder to synchronize the encoder compressor and decoder expander operations.

Alternatively, at the encoder, compression can be applied across the entire bandwidth of the original signal. A-SPX envelope data can then be extracted from the compressed subband samples. In this case, after QMF analysis, the decoder first runs the A-SPX tool to reconstruct the full-bandwidth compressed signal. An expansion phase is then applied to restore the signal to its original dynamic range.

In Figure 4, another tool that can operate in the QMF domain can be an advanced coupling (AC) tool (not shown). In an advanced coupling system, two channels are encoded into a mono downmix using additional parameterized spatial information that can be applied in the QMF domain at the decoder to reconstruct the stereo output. When AC and companding are used in conjunction with each other, the AC tool can be placed after the compression stage 406 at the encoder, in which case it would be applied before the expansion stage 506 at the decoder. Alternatively, AC side information can be extracted from the uncompressed stereo signal, in which case the AC tool would operate after the expansion stage 506 at the decoder. Hybrid AC modes can also be supported, in which AC is used above a certain frequency and discrete stereo is used below that frequency, or alternatively, discrete stereo is used above a certain frequency and AC is used below that frequency.

As shown in Figures 3A and 3B, the bitstream transmitted between the encoder and decoder stages of the codec includes certain control data. This control data constitutes auxiliary information that allows the system to switch between different companding modes. Switching control data (for turning companding on/off) may be combined with some intermediate states to add about 1 or 2 bits per channel. Other control data may include signals to determine whether all channels of a discrete stereo or multi-channel configuration will use a common companding gain factor, or whether they should be calculated independently for each channel. This data may require only a single additional bit per channel. Other similar control data elements and their appropriate bit weights may be used depending on system requirements and constraints.

Detection Mechanism

In one embodiment, a companding control mechanism is included as part of compression component 104 to provide control of companding in the QMF domain. Companding control can be configured based on a number of factors, such as the type of audio signal. For example, in most applications, companding should be turned on for speech signals and transient signals, or any other signal within the category of signals that are multimodal in time. The system includes a detection mechanism to detect the peakness of the signal to help generate appropriate control signals for the companding function.

In one embodiment, for a given core codec, the measure of temporal kurtosis TP(k) _frame is calculated over frequency bin k and is calculated using the following formula:

In the above equation, _St (k) is the subband signal, and T is the number of QMF time slots corresponding to one core encoder frame. In an exemplary implementation, the value of T can be 32. The temporal kurtosis calculated for each frequency band can be used to classify the sound content into two general categories: stationary music signals, and music transient signals or speech signals. If the value of TP(k) _frame is less than a defined value (e.g., 1.2), the signal in that subband of the frame is likely to be a stationary music signal. If the value of TP(k) _frame is greater than this value, the signal is likely to be a music transient signal or a speech signal. If the value is greater than an even higher threshold (e.g., 1.6), the signal is likely to be a pure music transient signal, such as castanets. Moreover, it has been observed that for naturally occurring signals, the temporal kurtosis values obtained in different bands are more or less similar, and this characteristic can be used to reduce the number of subbands for which temporal kurtosis values are to be calculated. Based on this observation, the system can implement one of the following two methods.

In a first embodiment, the decoder performs the following process. As a first step, it calculates the number of bands whose temporal kurtosis is greater than 1.6. As a second step, it then calculates the average of the temporal kurtosis values for the bands whose temporal kurtosis values are less than 1.6. If the number of bands found in the first step is greater than 51, or if the average value determined in the second step is greater than 1.45, then the signal is determined to be a music transient signal, and therefore compression and expansion should be turned on. Otherwise, it is determined to be a signal that compression and expansion should not be turned on. This detector will be turned off most of the time for speech signals. In some embodiments, speech signals will typically be encoded by a separate speech encoder, so this is generally not a problem. However, in some cases, it may be desirable to turn on the compression and expansion function for speech as well. In this case, a second type of detector may be preferred.

In one embodiment, this second type of detector performs the following process. As a first step, it counts the number of bands with a temporal kurtosis value greater than 1.2. In a second step, it then calculates the average of the temporal kurtosis values for bands with a temporal kurtosis less than 1.2. It then applies the following rules: if the result of the first step is greater than 55, turn on the companding, if the result of the first step is less than 15, turn off the companding; if the result of the first step is between 15 and 55 and the result of the second step is greater than 1.16, turn on the companding; and if the result of the first step is between 15 and 55 and the result of the second step is less than 1.16, turn off the companding. It should be noted that these two types of detectors describe only two examples of many possible solutions to detector algorithms, and that other similar algorithms may also or instead be used.

The companding control functionality provided by element 404 of FIG4 can be implemented in any suitable manner to allow companding to be used or not used based on certain operating modes. For example, companding is typically not used on the LFE (low frequency effects) channel of a surround sound system, and is also not used when A-SPX (i.e., non-QMF) functionality is not implemented. In one embodiment, the companding control functionality can be provided by a circuit or program executed by a processor-based element such as companding control element 404. The following is some exemplary syntax for a program snippet that can implement companding control, under one embodiment:

The sync_flag, b_compand_on[ch], and b_compand_avg flags or program elements may be approximately 1 bit in length, or any other length depending on system constraints and requirements. It should be noted that the program code illustrated above is an example of one method of implementing the companding control functionality, and that other programs or hardware components may be used to implement companding control according to some embodiments.

While the embodiments described thus far include a companding process for reducing quantization noise introduced by an encoder in a codec, it should be noted that aspects of this companding process can also be applied in signal processing systems that do not include an encoder and decoder (codec) stage. Furthermore, if the companding process is used in conjunction with a codec, the codec can be transform-based or non-transform-based.

Aspects of the systems described herein can be implemented in a suitable computer-based sound processing network environment for processing digital or digitized audio files. Portions of the adaptive audio system may include one or more networks comprising any desired number of individual machines, including one or more routers (not shown) for buffering and routing data transmitted between the computers. Such networks can be constructed based on a variety of different network protocols and can be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

One or more of the components, blocks, processes, or other functional components may be implemented by a computer program that controls the execution of a processor-based computing device of the system. It should also be noted that the various functions disclosed herein may be described in terms of their behavior, register transfers, logic components, and/or other characteristics using any number of combinations of hardware, firmware, and/or data and/or instructions contained in various machine-readable or computer-readable media. Computer-readable media containing such formatted data and/or instructions include, but are not limited to, various forms of physical (non-transitory), non-volatile storage media, such as optical, magnetic, or semiconductor storage media.

Unless the context clearly requires otherwise, throughout the specification and claims, the words "comprise," "comprising," and the like should be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is, in the sense of "including, but not limited to." Words using the singular or plural number also include the plural or singular number, respectively. Additionally, the words "herein," "below," "above," "below," and words of similar import refer to the entire application and not to any particular portion of the application. When the word "or" is used in reference to a list of two or more items, the word covers all of the following interpretations of the word: any item in the list, all items in the list, and any combination of items in the list.

Although one or more implementations have been described in terms of specific embodiments by way of example, it should be understood that one or more implementations are not limited to the disclosed embodiments. Rather, they are intended to cover various modifications and similar arrangements that would be apparent to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims (6)

1. A method for processing audio signals, comprising: Receives audio signals comprising multiple time segments. Determine the appropriate bandwidth gain for each audio signal time segment. The broadband gain is in the frequency domain and is based on the p-norm of the spectral amplitude of each time segment of the frequency domain representation of the audio signal, wherein the p-norm value is chosen to emphasize the weak spectral content of the audio signal relative to the strong spectral content of the audio signal, wherein the value of p in the p-norm is less than 2; and Apply a corresponding broadband gain to each time segment to obtain an extended audio signal. In this process, the application of broadband gain amplifies relatively high-intensity time segments and attenuates relatively low-intensity time segments. Each corresponding broadband gain for each time segment is calculated using subband samples from a subset of the subbands in the corresponding time segment. 2. The method of claim 1, wherein a first filter bank is used to analyze the audio signal to obtain a frequency domain representation. 3. The method of claim 2, wherein the first filter bank is one of a quadrature modulation filter (QMF) bank and a short-time Fourier transform. 4. A non-transitory computer-readable medium containing instructions that, when executed by one or more processors, perform the method of claim 1. 5. An apparatus for processing audio signals, comprising: The first interface is used to receive compressed audio signals that include multiple time segments. An expander that expands the compressed audio signal includes determining a corresponding broadband gain for each of the plurality of time segments and applying the corresponding broadband gain to each of the plurality of time segments to amplify relatively high-intensity time segments and attenuate relatively low-intensity time segments, wherein the broadband gain is in the frequency domain and represents the p-norm of the spectral amplitude of each of the plurality of time segments of the sample based on the frequency domain of the initial audio signal, wherein the value of p in the p-norm is less than 2, and wherein the p-norm value is selected to emphasize the weak spectral content of the audio signal relative to the strong spectral content of the audio signal; Each corresponding broadband gain for each time segment is calculated using subband samples from a subset of the subbands in each corresponding time segment. 6. The apparatus of claim 5 further comprises a first filter bank for analyzing the audio signal to obtain a frequency domain representation, and further wherein the first filter bank is one of a quadrature modulation filter (QMF) bank and a short-time Fourier transform.