CN1848690A - Multi-channel digital audio encoding device and method thereof - Google Patents

Abstract

A low bit-rate digital audio coding system includes an encoder that assigns a codebook to groups of quantization indices based on local characteristics of the quantization indices, such that the codebook application range is independent of quantization boundaries. The invention also includes a resolution filter bank or a tri-mode resolution filter bank that can be selectively switched between high and low frequency resolution modes or high, low and medium modes, e.g., when transients are detected in a frame. The resulting multi-channel audio signal has a greatly reduced bit rate for efficient transmission or storage. The decoder is essentially the inverse structure and method of the encoder, producing a restored audio signal that is not audibly distinguishable from the original signal.

Description

Multi-channel digital audio encoding device and method thereof

related application

This application claims priority to US Provisional Application 60/610,674, filed September 17,2004.

Background of the invention

The present invention generally relates to methods and systems for encoding and decoding multi-channel digital audio signals. More precisely, the present invention relates to a low-bit-rate digital audio coding system which greatly reduces the bit-rate of multi-channel audio signals for efficient transmission or storage, even for listening, while achieving transparent audio signal reproduction. Even test experts cannot distinguish the audio signal restored at the decoder from the original signal.

A multi-channel digital audio coding system usually includes the following elements: a time-frequency analysis filter bank, which produces a frequency representation of the incoming PCM (Pulse Code Modulation) samples, called subband samples or subband signals; a psychoacoustic model, whose A masking threshold below which quantization noise is less likely to be heard is calculated based on the auditory characteristics of the human ear; a global bit allocator that allocates bit resources to each group of subband samples so that the resulting quantization noise power is low multiple quantizers, which quantize the subband samples according to the assigned bits; multiple entropy encoders, which reduce the statistical redundancy in the quantization index; and finally a multiplexer, which quantizes The entropy encoding of the exponent and other auxiliary information are packaged into a complete bitstream.

For example, Dolby AC-3 uses a high frequency resolution MDCT (Modified Discrete Cosine Transform) filter bank with switchable window size to map input PCM samples into the frequency domain. Steady-state signals are analyzed with a window of 512 points, while transient signals are analyzed with a window of 256 points. The subband signals from the MDCT are represented as exponents/mantissas and then quantized. A forward-backward adaptive psychoacoustic model is employed to optimize quantization and reduce the bits required to encode bit allocation information. Entropy coding is not used in order to reduce the complexity of the decoder. Finally, the quantization index and other side information are multiplexed into a complete AC-3 bitstream. The frequency resolution of the adaptive MDCT configured in AC-3 is not well matched to the input signal characteristics, so its compression characteristics are greatly limited. The lack of entropy coding is another factor limiting its compression properties.

MPEG1&2 Layer III (MP3) uses a 32-band polyphase filter bank, where each subband filter is followed by an adaptive MDCT that switches between 6 and 18 points. An advanced psychoacoustic model is used to guide its bit allocation and scalar non-uniform quantization. Huffman codes are used to encode the quantization index and most of the other side information. The poor frequency isolation of hybrid filter banks greatly limits its compression properties and has high algorithmic complexity.

DTS Coherent Acoustics employs a 32-band polyphase filter bank to obtain a low-resolution frequency representation of the input signal. To compensate for poor frequency resolution, ADPCM (Adaptive Differential Pulse Code Modulation) is selectively used for each subband. Uniform scalar quantization is applied directly to the subband samples or to the prediction residual if ADPCM yields a good coding gain. Vector quantization can be selectively applied to high frequency subbands. Huffman codes can optionally be applied to scalar quantization indices and other side information. Because the structure of polyphase filter bank + ADPCM cannot provide good time and frequency resolution at all, its compression characteristics are very low.

MPEG 2 AAC and MPEG 4 AAC use an adaptive MDCT filter bank whose window size can be switched between 256 and 2048. The masking threshold generated by the psychoacoustic model is used to guide its scalar non-uniform quantization and bit allocation. Huffman codes are used to encode quantization indices and most other side information. Many other tools such as TNS (Temporal Noise Shaping), Gain Control (mixed filter banks similar to MP3), spectral prediction (linear prediction within subbands) are used to further enhance its compression properties, and this At the cost of greatly increasing algorithmic complexity.

Therefore, there remains a need for a low bit-rate audio coding system which greatly reduces the bit-rate of multi-channel audio signals for efficient transmission or storage, while also achieving transparent audio signal reproduction. The present invention fulfills this need and provides other related advantages.

Contents of the invention

In the following discussion, the terms "analysis/synthesis filter bank" and the like refer to devices or methods that perform time-frequency analysis/synthesis. It can include without limitation the following:

Unitary transformation;

Time-varying or time-invariant banks of critically sampled, uniform, or non-uniform bandpass filters;

● Harmonic or sine wave analyzer/synthesizer.

Polyphase filter banks, DFT (Discrete Fourier Transform), DCT (Discrete Cosine Transform), and MDCT are some of the widely used filter banks. The terms "subband signal or subband sample" etc. refer to the signal or sample coming out of the analysis filterbank and into the synthesis filterbank.

It is an object of the present invention to provide the same level of compression performance as in the prior art but with reduced algorithmic complexity for low bit rate coding of multi-channel audio signals.

This is done on the encoding end side by an encoder consisting of:

1) A framer, used to cluster the input PCM samples into quasi-stationary frames, whose size is an integer multiple of the number of subbands of the analysis filter bank, and whose time range is 2 to 50 ms.

2) A transient detector for detecting the presence of transients in the frame. One embodiment is based on taking a threshold of the subband distance measure obtained from the subband samples of the analysis filterbank in low frequency resolution mode.

3) A variable resolution analysis filter bank for converting the input PCM samples into subband samples, which can be implemented with one of the following:

a) A filter bank whose operation can be switched between high, medium and low frequency resolution modes. High frequency resolution modes are used for steady state frames, while medium and low frequency resolution modes are used for frames with transients. Within a transient frame, the low frequency resolution mode is used for the transient segment, while the intermediate resolution mode is used for the remainder of the frame. There are three types of frames under this architecture:

i) steady state frames processed by the filter bank only operating in high frequency resolution mode;

ii) Transient frames processed by the filter bank operating in medium and high temporal resolution modes;

iii) Slow transient frames processed by the filter bank operating only in intermediate resolution mode; two preferred embodiments are given as follows:

i) DCT implementation, wherein three levels of resolution correspond to three DCT block lengths;

ii) MDCT implementation, where three levels of resolution correspond to three MDCT block lengths or window lengths. Define multiple window types to bridge transitions between these windows.

b) a hybrid filter bank based on a filter bank whose operation can be switched between high and low resolution modes;

i) when there are no transients in the current frame, it switches to a high frequency resolution mode to ensure high compression performance for steady-state segments;

ii) When there is a transient in the current frame, it switches to a low frequency resolution/high time resolution mode to avoid the forward echo effect. This low frequency resolution mode is also followed by a transient clustering segmentation stage that divides the subband samples into stationary segments, then optionally followed in each subband by an arbitrary resolution filterbank or ADPCM, if If selected, can be used to provide the appropriate frequency resolution for each steady state segment.

Two examples are given, where one is based on DCT and the other on MDCT. Two examples of transient segments are given, one based on thresholding and the other based on the k-means algorithm, both using subband distance measurements.

2) Calculate the psychoacoustic model of the masking threshold.

3) An optional sum/difference encoder that converts subband samples in left and right channel pairs into sum/difference channel pairs.

4) An optional joint intensity encoder that extracts the intensity scale factor (steering vector) of the joint channel compared to the source channel, merges the joint channel into the source channel, and discards individual subbands in the joint channel sample.

5) A global bit allocator that allocates bit resources to groups of subband samples such that their quantization noise power is below the masking threshold.

6) A scalar quantizer that quantizes all subband samples with the step size provided by the bit divider.

7) An optional interleaver, which is selectively used to rearrange the quantization indices when there are transients in the frame in order to reduce the total number of bits.

8) entropy coder, it assigns the best codebook to multiple groups of quantization indexes from the codebook library based on the local statistical characteristics of the quantization index, comprising the following steps:

a) Assign the best codebook to each quantized index, thus essentially converting the quantized index into a codebook index.

b) Divide these codebook indexes into very large segments, and the segment boundaries define the application range of the codebook.

A preferred embodiment is:

c) Block the quantization indices into granules, each granule comprising a fixed number of quantization indices.

d) Determine the maximum codebook requirement for each block.

e) Assign the smallest codebook to a block that can accommodate its largest codebook requirement:

f) Eliminate those isolated small block regions whose codebook index is smaller than its neighbor's codebook index; those isolated small block regions corresponding to the codebook index of zero quantization index may not be processed in this way.

A preferred embodiment for encoding the range of application of an encoded codebook uses run-length codes.

9) An entropy encoder that encodes all quantization indices with a codebook and its range of application determined by the entropy codebook selector.

10) A multiplexer that packs all the entropy codes and side information of the quantization indices into one complete bitstream, constructed so that the quantization indices appear before the indices used for the quantization steps. This construction makes it unnecessary to pack the number of quantization units per transient segment into the bitstream, since it can be recovered from the unpacked quantization index.

The decoder of the present invention includes:

1) a demultiplexer for unpacking different codewords from the bitstream;

2) Quantization index codebook decoder, used to decode the entropy codebook used for quantization index and its application range from the bit stream;

3) an entropy decoder for decoding quantization indices from the bitstream;

4) An optional deinterleaver that selectively rearranges quantization indices when there are transients in the current frame;

5) Quantization unit number reconstructor, it reconstructs the quantization unit number of each transient segment from the quantization index with the following steps:

a) Find the largest subband with a non-zero quantization index for each transient segment;

b) Find the minimum critical frequency band that can accommodate this subband, which is the number of quantization units in this transient segment;

6) a step size unpacker, which unpacks the quantization step size of all quantization units;

7) An inverse quantizer, which reconstructs the subband samples from the quantization index and step size;

8) An optional joint intensity decoder that reconstructs the subband samples of the joint channel from the subband samples of the source channel using the joint intensity scale factor (steering vector);

9) An optional sum/difference decoder which reconstructs the subband samples of the left and right channels from the subband samples of the sum/difference channels;

10) A variable resolution synthesis filterbank that reconstructs audio PCM samples from subband samples, which can be performed by:

a) a synthesis filter bank capable of switching its operation between high, medium and low resolution modes;

b) a hybrid synthesis filter bank based on a synthesis filter bank that can be switched between high and low resolution modes;

i) When the bitstream indicates that the current frame is coded in low frequency resolution mode using a variable-resolution analysis filterbank, the synthesis filterbank is a two-stage hybrid filterbank, where the first stage is a Arbitrary resolution synthesis filter bank or an inverse ADPCM, while the second stage is a low frequency resolution mode of an adaptive synthesis filter bank switchable between high and low frequency resolution modes;

ii) When the bitstream indicates that the current frame was coded in high frequency resolution mode with a scalable resolution analysis filterbank, the synthesis filterbank is nothing but scalable resolution synthesis in high frequency resolution mode filter bank.

Finally, the present invention provides a low encoding delay mode that is enabled when the high frequency resolution mode of the switchable resolution analysis filter bank is disabled by the encoder, and the frame length is subsequently reduced to that at low frequency resolution The block length of the switchable-resolution filterbank in mode or an integer multiple thereof.

According to the invention, a method of encoding a multi-channel digital audio signal generally comprises the steps of creating PCM samples from the multi-channel digital audio signal and converting the PCM samples into subband samples. Multiple quantization indices with boundaries are created by quantizing the subband samples. The quantization indices are converted into codebook indices by assigning to each quantization index the smallest codebook that can accommodate the quantization indices in the pre-designed codebook library. The codebook index is clustered and encoded before creating an encoded data stream for storage or transmission.

In general, PCM samples are input into quasi-steady-state frames of duration between 2 and 50 milliseconds (ms). The masking threshold can be calculated using, for example, a psychoacoustic model. The bit allocator allocates bit resources into groups of subband samples such that the quantization noise power is below the masking threshold.

The conversion step includes using a resolution filter bank that selectively switches between high and low frequency resolution modes. To detect transients, the high frequency resolution mode is used when no transients are detected; however, when a transient is detected, the resolution filter bank is switched to low frequency resolution mode. With switching the resolution filter bank to low frequency resolution mode, the subband samples are divided into stationary segments. The frequency resolution of each steady-state segment is trimmed with an arbitrary resolution filter bank or adaptive differential pulse code modulation.

The quantization index can be rearranged to reduce the total number of bits when there are transients in the frame. A run-length encoder can be used to encode the application boundary of the optimal entropy codebook, and a clustering and partitioning algorithm can be used.

A sum/difference encoder may be used to convert subband samples in left and right channel pairs into sum/difference channel pairs. In addition, a joint intensity encoder can be used to compare the source channels to extract the intensity scale factor of the joint channel, merge the joint channels into the source channel, and discard all relevant subband samples in the joint channel.

Generally, the step of combining to create a complete bitstream is performed using a multiplexer before storing or sending the encoded digital audio signal to a decoder.

The method of decoding the audio data bitstream includes receiving the encoded audio data stream and depacketizing the data stream, such as by using a demultiplexer. The entropy codebook indices and their respective ranges of application are decoded. This may involve run-length and entropy decoders. They are also used to decode quantization indices.

When a transient is detected in the current frame, the quantization indices are rearranged, such as by using a deinterleaver. The subband samples are then reconstructed from the decoded quantization indices. Audio PCM samples are reconstructed from the reconstructed subband samples using a variable resolution synthesis filterbank switchable between low and high frequency resolution modes. When the data stream indicates that the current frame is coded in low frequency resolution mode with a switchable resolution analysis filterbank, the variable synthesis resolution filterbank is used as a two-stage hybrid filterbank, where the first stage consists of An arbitrary resolution synthesis filter bank or an inverse adaptive differential pulse code modulation, the second stage is a low frequency resolution mode of variable synthesis filter bank. The variable resolution synthesis filterbank operates in the high frequency resolution mode when the data stream indicates that the current frame is encoded in the high frequency resolution mode with the switchable resolution analysis filterbank.

A joint intensity decoder may be used to reconstruct the joint channel subband samples from the source channel subband samples using the joint intensity scale factor. Furthermore, a sum/difference decoder can be used to reconstruct left and right channel subband samples from the sum/difference channel subband samples.

The result of the present invention is a low bit-rate digital audio coding system which greatly reduces the bit-rate of multi-channel audio signals for efficient transmission, while still achieving a reproduction of the audio signal so transparent that it is difficult to compare it with the original Signal distinction.

Other features and advantages of the invention will become apparent from the following detailed description, taken with reference to the accompanying drawings, illustrating by way of example the principles of the invention.

Description of drawings

The following drawings serve to illustrate the invention. In these drawings:

Figure 1 is a schematic diagram describing the encoding and decoding of a multi-channel digital audio signal according to the present invention;

Figure 2 is a schematic diagram illustrating an exemplary encoder used in accordance with the present invention;

Fig. 3 is a schematic diagram of a variable resolution analysis filter bank with a filter bank of arbitrary resolution;

Fig. 4 is a schematic diagram of the analysis filter bank with the variable resolution of ADPCM;

Fig. 5 is the schematic diagram that is used for switchable MDCT window type according to the present invention;

Fig. 6 is a schematic diagram of a transient section according to the present invention;

7 is a schematic diagram of an application of a switchable filter bank with two resolution modes according to the present invention;

8 is a schematic diagram of an application of a switchable filter bank with three resolution modes according to the present invention;

Similar to FIG. 5, FIG. 9 is a schematic diagram of other window types for switchable MDCT with three resolution modes according to the present invention;

Figure 10 depicts a set of examples of switchable MDCT window sequences with three resolution modes according to the present invention;

Fig. 11 is a schematic diagram of determining the entropy codebook of the present invention compared with the prior art;

Fig. 12 is a schematic diagram of dividing the codebook index into very large segments or eliminating isolated small block regions of the codebook index according to the present invention;

Fig. 13 is a schematic diagram of a decoder equipped in the present invention;

Fig. 14 is a schematic diagram of a variable resolution synthetic filter bank with an arbitrary resolution filter bank according to the present invention;

Figure 15 is a schematic diagram of a variable resolution synthesis filter bank with inverse ADPCM; and

FIG. 16 is a schematic diagram of the structure of a bitstream when using a semi-hybrid filter bank or a switchable filter bank+ADPCM according to the present invention.

Figure 17 is a schematic illustration of the advantages of long windows for short-to-short transitions when dealing with transients only one frame apart.

FIG. 18 is a schematic diagram of the structure of a bitstream when a three-mode switchable filter bank is used according to the present invention.

Detailed ways

As shown in the accompanying drawings, for purposes of illustration, the present invention relates to a low bit-rate digital audio encoding and decoding system that greatly reduces the bit rate of a multi-channel audio signal for efficient transmission or storage, while also enabling transparent audio reproduction. That is, the bitrate of a multi-channel encoded audio signal is reduced by using a system with less algorithmic complexity, and even an audiologist cannot distinguish the restored audio signal from the original signal at the decoder end.

As shown in Figure 1, the encoder 5 of the present invention takes a multi-channel audio signal as input and encodes it into a bit stream, and greatly reduces the bit rate to be suitable for transmission or storage on a medium with limited channel capacity . As soon as the bitstream produced by the encoder 5 is received, the decoder 10 decodes it and reconstructs a multi-channel audio signal that even audiologists cannot distinguish from the original signal.

Inside the encoder 5 and decoder 10, the multi-channel audio signal is processed as discrete channels. That is, each channel is treated the same as other channels, unless joint channel coding 2 is explicitly specified. This is illustrated in Figure 1 with an extremely simplified encoder and decoder structure.

Utilizing this extremely simplified encoder structure, its encoding process is explained as follows. The audio signal from each channel is first decomposed into subband signals in the first stage 1 of the analysis filter bank. The subband signals from all channels are selectively sent to a joint channel encoder 2, which exploits the auditory characteristics of the human ear to reduce the bit rate by combining subband signals corresponding to the same frequency band from different channels. The subband signals, which can be jointly coded in 2, are then quantized and coded in 3. The quantization indices or their entropy encoding and side information from all channels are then multiplexed in 4 into a complete bitstream for transmission or storage.

On the decoding end, the bitstream is first demultiplexed in 6 into side information and quantization indices or its entropy encoding. The entropy codes are decoded in 7 (note: entropy decoding and demultiplexing of prefix codes such as Huffman codes are usually performed in one single step). The subband signal is reconstructed in 7 using the quantization index and the step size carried by the side information. If joint channel coding is used in the encoder, joint channel decoding is performed in 8 . The audio signal for each channel is then reconstructed in a synthesis stage 9 using the subband signals.

The extremely simplified encoder and decoder structures described above are used to separately illustrate the discrete nature of the encoding and decoding methods presented in the present invention. The encoding and decoding methods actually applied to each channel of an audio signal are very different and more complex. Unless otherwise stated, the methods are described below in the context of one channel of an audio signal.

Encoder

A general method of encoding one channel of an audio signal is depicted in Figure 2 as follows:

The framer 11 divides the input PCM samples of duration from 2 to 50 ms into quasi-stationary frames. The exact number of PCM samples in a frame must be an integer multiple of the maximum number of subbands of the different filter banks used in the variable resolution time-frequency analysis filter bank 13 . Assuming that the maximum number of subbands is N, then the number of PCM samples in a frame is

L＝k·N

Among them, k is a positive integer.

Transient analysis 12 detects the presence of transients in the current input frame and passes this information to variable resolution analysis group 13 .

Any known transient detection method can be used here. In one embodiment of the invention, an input frame of PCM samples is fed to the low frequency resolution mode of the variable resolution analysis filterbank. Let s(m,n) denote the output samples from this filterbank, where m is the subband index and n is the temporal index in the subband domain. In the following discussion, the term "transient detection distance" etc. refers to the distance measure defined for each time index:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>$

or

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; the\; s</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>$

Among them, M is the number of subbands of the filter bank. Other types of distance measurements can also be employed in a similar way. let ${\mathrm{E.}}_{\max }=\underset{\mathrm{no}}{\mathrm{Max}}\mathrm{E.}\left(\mathrm{no}\right)$ and ${\mathrm{E.}}_{\min }=\underset{\mathrm{no}}{\mathrm{Min}}\mathrm{E.}\left(\mathrm{no}\right)$ are the maximum and minimum values of this distance, if

$\frac{{\mathrm{E.}}_{\max }-{\mathrm{E.}}_{\min }}{{\mathrm{E.}}_{\max }+{\mathrm{E.}}_{\min }}>\mathrm{Threshold}$ (threshold)

A transient state is then declared, where the threshold can be set to 0.5.

The present invention uses a variable resolution analysis filter bank 13 . There are many known methods to implement variable resolution analysis filter banks. A prominent method is the use of filter banks whose operation can be switched between high and low frequency resolution modes, the high frequency resolution mode being used to process the steady state segments of the audio signal and the low frequency resolution mode being used to process transient states . However, the switching of resolution cannot happen arbitrarily in time due to theoretical and practical constraints. Instead, it usually occurs at frame boundaries, i.e. frames are processed in either high frequency resolution mode or low frequency resolution mode. As shown in Figure 7, for the transient frame 131, the filter bank has been switched to a low frequency resolution mode to avoid the forward echo effect. Because the transient 132 itself is very short, and the pre-transient 133 and post-transient 134 segments of the frame are much longer, the low frequency resolution mode's filter bank obviously does not match these steady state segments. This greatly limits the total coding gain that can be achieved for the entire frame.

The present invention proposes three methods to solve this problem. The basic idea is to provide a higher frequency resolution for the stationary majority of the transient frame within a switchable resolution structure.

Semi-hybrid filter bank

As shown in Figure 3, it is essentially a hybrid filter bank, including an analysis filter bank 28 with switchable resolution that can be switched between high and low frequency resolution modes, and in the low frequency resolution mode 24, followed by a transient clustering segmentation unit 25, and then an optional analysis filter bank 26 of arbitrary resolution in each subband.

When no transient is detected by the transient detector 12, the resolution-switchable analysis filter bank 28 enters a low temporal resolution mode 27, which ensures high frequency resolution for high audio signal coding gain, with strong tones portion.

When the transient detector 12 detects the presence of a transient, the resolution switchable analysis filter bank 28 enters the high temporal resolution mode 24 . This ensures that transients are processed with good time resolution to prevent forward echoes. The subband samples thus generated are divided into quasi-stationary segments by the transient clustering segmentation section 25 as shown in FIG. 6 . In the following discussion, the term "transient segment" and the like refer to these quasi-stationary segments. This is followed by an analysis filterbank 26 of arbitrary resolution in each subband equal to the number of subband samples per transient in each subband.

The switchable resolution analysis filter bank 28 can be implemented with any filter bank whose operation can be switched between high and low frequency resolution modes. One embodiment of the present invention employs a pair of DCTs, corresponding to low and high frequency resolutions, with small and large transform lengths, respectively. Assuming a transform length of M, the subband samples of a DCT of type 4 are obtained as:

$\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; cos</span>}<span\; class="notranslate">\; [</span>\frac{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; mM</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>$

where x(.) is the input PCM sample. Other forms of DCT can be used instead of Type 4 DCT.

Because DCT tends to cause blocking effects, a preferred embodiment of the invention is to use a Modified DCT (MDCT):

$\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; cos</span>}<span\; class="notranslate">\; [</span>\frac{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; mM</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>$

Among them, w(.) is the window function.

The window function must be power symmetric in each half window:

w ² (k)+w ² (Mk)=1 k=0, . . . , M-1

w ² (k+M)+w ² (2M-1-k)=1 k=0, . . . , M-1

In order to ensure ideal reconstruction.

Although any window that satisfies the above conditions can be used, only the following sine windows

$\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; \&PlusMinus;</span>\mathrm{<span\; class="notranslate">\; sin</span>}<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; )</span>\frac{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; ]</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}$

It has the good property that the DC component in the input signal is concentrated to the first transform coefficient.

In order to maintain ideal reconstruction when MDCT switches between high and low frequency modes or between long and short windows, the overlapping parts of long and short windows must have the same shape.

Depending on the transient characteristics of the input PCM samples, the encoder can choose a long window (shown as the first window 61 in Figure 5), switch to a sequence of short windows (shown as the fourth window 64 in Figure 5) , and returns. The long window 62 for the long-to-short transition and the long window 63 for the short-to-long transition in FIG. 5 are needed to bridge this type of switching. The short-to-short transition of the long window 65 in Figure 5 is useful when the two transients are very close but not close enough to warrant continuous application of the short window. The encoder needs to communicate to the decoder the type of window used for each frame, so that the same window is used to reconstruct the PCM samples.

The advantage of long windows for short-to-short transitions is that adjacent transients that are only one frame apart can be handled. As shown at the top 67 of Figure 17, the prior art MDCT can handle transients at least two frames apart. As shown at the bottom 68 of Figure 17, using this long window for short-to-short transitions can reduce this to one frame.

The present invention will then execute the transient segment 25 . By exploiting the change of binary function value from 0 to 1 or 1 to 0, a transient segment can be represented by a binary function or a cluster segmentation boundary indicating the location of the transient. For example, the quasi-steady-state segment in Figure 6 can be represented as follows:

$\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0,1,2,3,4</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 5,6,7,8,9</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 10,11,12,13,14,15,16</span>}\end{array}\right.$

Note that T(n) = 0 does not necessarily mean that the energy of the audio signal is high at time index n, and vice versa. The function T(n) is referred to throughout the following discussion as a "transient segment function" and the like. The information carried by this segment function must be passed directly or indirectly to the decoder. Run-length encoding that encodes zero-one run lengths is an efficient choice. For the specific example above, T(n) can be delivered to the decoder with run length codes of 5, 5 and 7. Run-length codes can also be entropy encoded.

The transient clustering segmentation part 25 can be realized by any known transient clustering segmentation method. In one embodiment of the present invention, transient clustering and segmentation can be accomplished by simply thresholding the transient detection distance.

Threshold can be set as

$\mathrm{<span\; class="notranslate">\; Threshold</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\frac{{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}$

Among them, k is an adjustable constant.

A more advanced embodiment of the present invention is based on the k-means clustering algorithm comprising the following steps:

1) The transient clustering and segmentation function T(n) is initialized, and the result obtained by the above-mentioned thresholding method is used.

2) The centroid of each class is calculated:

对于与T(n)＝0相关联的类；

For classes associated with T(n)=0;

对于与T(n)＝1相关联的类。

For the class associated with T(n)=1.

3) The transient clustering and segmentation function T(n) is assigned based on the following rules

4) Go to step 2.

The arbitrary resolution analysis filterbank 26 is essentially a transform such as DCT with a block size equal to the number of samples in each subband segment. Assuming that there are 32 subband samples in each subband in a frame and they are divided into (9, 3, 20), three transforms with block lengths of 9, 3 and 20 will be applied to the three subband segments respectively Each subband sample. In the following discussion, the term "subband segment" and the like refer to subband samples of a transient segment within a subband. The transformation of the last segment (9, 3, 20) of the m-th subband can be described using a DCT of type 4 as follows

$\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; 20</span>}}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 20</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; cos</span>}<span\; class="notranslate">\; [</span>\frac{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; 20</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>$

This conversion will increase the frequency resolution within each transient segment, so a good coding gain can be expected. In many cases, however, the coding gain is less than 1 or too small, and it is an advantageous decision to discard such transformation results and inform the decoder of this decision via side information. Due to the overhead associated with the side information, it can improve the overall coding gain if the decision whether to discard a conversion result is based on a set of subband segments, i.e. one bit is used for a set of subband segments instead of each subband segment Send this decision.

In the following discussion, the terms "quantization unit" and the like refer to a group of contiguous subband samples belonging to the same psychoacoustically critical band and transient. A quantization unit may be a good grouping of subband segments for the above decision making. If this is used, the total coding gain is calculated for all subband segments in the quantization unit. If the coding gain is greater than 1 or some other upper threshold, the conversion result is kept for all subband segments in the quantization unit. Otherwise, the result is discarded. Only one bit is required to convey this decision for all subband segments in the quantization unit to the decoder.

                                Switchable Filter Bank + ADPCM

As shown in FIG. 4, it is basically the same as in FIG. 3, except that the arbitrary resolution analysis filter bank 26 is replaced by ADPCM 29. The decision whether to apply ADPCM is again based on a set of subband segments, such as quantization units, in order to reduce the cost of side information. The set of subband segments may even share a set of prediction coefficients. Known methods of quantizing the prediction coefficients can be used here, including LAR (Log Area Ratio), IS (Inverse Sine), and LSP (Line Spectral Pair), for example.

Three-mode switchable filter bank

Unlike common switchable filter banks that only have high and low resolution modes, this filter bank can switch its operation between high, medium and low resolution modes. High and low frequency resolution modes are used for steady-state and transient frames, respectively, and follow the same class of principles as the dual-mode switchable filter bank. The main purpose of the intermediate resolution mode is to provide better frequency resolution to the steady state segments within the transient frame. Thus, within a transient frame, the low frequency resolution mode is used for the transient segment and the medium resolution mode is used for the remainder of the frame. Unlike the prior art, the switchable filter bank of the present invention operates in two resolution modes for a single frame of audio data. Intermediate resolution mode can also be used to process frames with smooth transients.

In the following discussion, the term "long block" etc. refers to a block of samples output by the filter bank at each time instant in high frequency resolution mode: the term "medium block" etc. refers to the filter bank output at medium frequency resolution A block of samples output at each moment in mode; the term "short block" etc. refers to a block of samples output at each moment of the filter bank in low frequency resolution mode. Three types of frames can be described using these three definitions as follows:

Steady state frames processed by the filter bank operating in a high frequency resolution mode, each of such frames typically comprising one or more long blocks;

Frames with transients processed by the filter bank operating in high and medium temporal resolution modes, each of which includes several medium blocks and several short blocks, the total number of samples of all short blocks is equal to one the total number of samples in the block;

• Filter banks operating in intermediate resolution mode to process frames with smooth transients, each of which includes several intermediate blocks.

The advantages of this new approach are shown in FIG. 8 . Figure 8 is essentially the same as Figure 7, except that many of the segments (141, 142 and 143) that were originally processed in the low frequency resolution mode in Figure 7 are now processed by the medium frequency resolution mode. Because these segments are steady state, the medium frequency resolution mode is clearly better matched than the low frequency resolution mode, so higher coding gain can be expected.

One embodiment of the present invention employs a triplet DCT with small, medium, and large block lengths, corresponding to low, medium, and high frequency resolution modes, respectively.

A preferred embodiment of the invention (without blocking) uses a triplet MDCT with small, medium and large block lengths. Due to the introduction of the intermediate resolution mode, the window types shown in Fig. 9 are provided in addition to the window types in Fig. 5 . These windows are described as follows:

· Medium window 151;

·Long window 152 for long-to-medium transition: as a long window, it bridges the transition from long window to medium window.

• Long window 153 for medium-to-long transition: as a long window, it bridges the transition from medium window to long window.

• Long window 154 for medium-to-medium transition: as a long window, it bridges the transition from a medium window to another medium window.

• Medium window 155 for medium to short transition: as a medium window, it bridges the transition from medium window to short window.

• Medium window 156 for short to medium transition: as a medium window, it bridges the transition from short window to medium window.

• Long window 157 for medium to short transition: as a long window, it bridges the transition from medium window to short window.

• Long window 158 for short to medium transition: as a long window, it bridges the transition from short to medium window.

NOTE: Similar to the long window 65 for the short-to-short transition in FIG. Transient for one frame.

Figure 10 shows some examples of window sequences. 161 illustrates the ability of this embodiment to handle slow transients with medium resolution 167, while 162 to 166 illustrate the assignment of fine temporal resolution 168 to transients and medium temporal resolution 169 to steady state segments within the same frame , and the ability to assign high frequency resolution 170 to steady state frames.

The usual sum/difference coding method 14 can be applied here. For example, a simple usage is as follows:

  Harmony channel = 0.5 (left channel + right channel)

  Harmony channel = 0.5 (left channel + right channel)

Common joint strength coding methods15 can be applied here. A simple way can be

• Replace the source channel with the sum of the source and joint channels.

Adjust it to the same energy level as the original source channel within the quantization unit

Subband samples of the joint channel within the quantization unit are discarded and only the quantization index of the scale factor (called "steering vector" or "scale factor" in the present invention) is passed to the decoder, which is defined as:

Non-uniform quantization of steering vectors such as logarithms will be used to match the auditory properties of the human ear. Entropy coding can be applied to the quantization indices of the steering vectors.

To avoid canceling effects of source and joint channels when they are nearly 180 degrees out of phase, polarity can be applied when they are summed to form the joint channel:

Harmony channel = source channel + polarity combined channel

Polarity must also be communicated to the decoder.

The psychoacoustic model 23 calculates a masking threshold for the current input frame of audio samples based on the auditory characteristics of the human ear, below which quantization noise is less likely to be heard. Any common psychoacoustic model can be used here, but the present invention requires its psychoacoustic model to output a masking threshold for each quantization unit.

The global bit allocator 16 globally allocates a frame of available bit resources to each quantization unit so that the quantization noise power in each quantization unit is below its respective masking threshold, and it controls each quantization by adjusting its quantization step size Quantization noise power of the unit. All subband samples within a quantization unit are quantized with the same step size.

All known bit allocation methods can be used here. One such method is the well-known Water Filling algorithm. Its basic idea is to find the quantization unit whose QNMR (quantization noise masking ratio) is the highest, and reduce the step size assigned to this quantization unit to reduce the quantization noise. It repeats this process until the QNMR of all quantization units is less than 1 (or any other threshold) or the current frame's bit resources are exhausted.

The quantization step itself must be quantized so that it can be packed into the bitstream. A non-uniform quantization such as logarithmic would be used to match the auditory properties of the human ear. Entropy coding can be applied to the quantization index of the step size.

The present invention quantizes all subband samples within each quantization unit 17 using the step size provided by the global bit allocation 16 . All linear or non-linear, uniform or non-uniform quantization schemes can be used here.

Interleave 18 can optionally be invoked only if there is a transient in the current frame. Let x(m,n,k) be the mth quasi-stationary segment and the kth quantization index in the nth subband. (m, n, k) is usually the order in which the quantization indices are arranged. The interleaving unit 18 reorders the quantization indices so that they are arranged as (n, m, k). The motivation for this is that the rearrangement of the quantized indices allows for encoding these indices with fewer bits than would be required if the indices were not interleaved. The decision whether to invoke interleaving needs to be passed to the decoder as side information.

In previous audio coding algorithms, the application range of the entropy codebook is the same as that of the quantization unit, so the entropy codebook is determined by the quantization index within the quantization unit (see the top of FIG. 11 ). So there is no room for optimization.

The present invention is quite different in this respect. It ignores the presence of quantization units when proceeding to codebook selection. Instead, it assigns the best codebook to each quantization index19, thus essentially converting the quantization index into a codebook index. It then divides these codebook indices into larger segments, and the segment boundaries define the extent to which the codebook applies. Obviously, the range of application of these codebooks is quite different from the range determined by the quantization unit. They are only based on the quality of the quantized index, so the selected codebook is more suitable for the quantized index. Therefore, fewer bits are required to convey the quantization index to the decoder.

The advantages of this approach over the prior art are illustrated in FIG. 11 . Let's look at the largest quantized exponent in the graph. It belongs to quantization unit d and with the previous method it is necessary to choose a large codebook which is obviously not optimal since most exponents in quantization unit d are much smaller. On the other hand, by using the new method of the present invention, the same quantization index is divided into segment C so that it shares a codebook with other quantization indices. Also, all quantization exponents in segment D are small, so a small codebook will be chosen. Therefore, fewer bits are required to encode the quantization index.

Referring now to FIG. 12, the prior art system only needs to communicate the codebook indices as side information to the decoder, since they apply to the same range as the predetermined quantization units. However, the method of the present invention needs to transmit the codebook application range as side information to the decoder in addition to the codebook index, since they are independent of the quantization unit. If not handled properly, this overhead can end up with more bits for the overall side information and quantization index. Therefore, dividing the codebook indices into large segments is quite critical to control this overhead, because a large segment means that a smaller number of codebook indices and their application ranges need to be transmitted to the decoder.

An embodiment of the present invention uses the following steps to accomplish this new scheme for codebook selection:

1) The quantized indices are divided into blocks, and each block includes P quantized indices.

2) Determine the maximum codebook requirement for each block. For symmetric quantizers, this is usually represented by the largest absolute quantization exponent within each block:

where I(.) is the quantitative index;

3) Assign the smallest codebook to the block that can accommodate the largest codebook requirement:

4) By raising the codebook index of those isolated small block areas whose codebook index is smaller than its neighbors to the minimum value of the codebook index of its neighbors, these isolated small block areas are removed. This is illustrated by maps 71 to 72 , 73 to 74 , 77 to 78 and 79 to 80 in FIG. 12 . Isolated small-block regions deep in the codebook index corresponding to zero quantization index can be eliminated from this process, since this codebook indicates that no code needs to be transmitted. This is depicted as a 75 to 76 mapping in FIG. 12 . This step significantly reduces the number of codebook indices that need to be transmitted to the decoder and its scope of application.

One embodiment of the present invention uses run-length codes to encode the codebook application range, and the run-length codes can also be encoded with entropy codes.

All quantization indices are coded 20 using the codebook determined by the entropy codebook selector 19 and their respective application ranges.

Entropy coding can be implemented with various Huffman codebooks. When the number of quantization levels in a codebook is small, multiple quantization indices are blocked together to form a large Huffman codebook. When the number of quantization levels is too large (for example, more than 200), recursive indexing is used. For this, a large quantization exponent q is denoted as

q＝m·M+r

where M is the modulus, m is the quotient, and r is the remainder. Only m and r need to be passed to the decoder. One or both of them can be encoded with Huffman codes.

Entropy coding can be implemented with various arithmetic codebooks. When the number of quantization levels is too large (for example, more than 200), the recursive index will also be used.

Other types of entropy coding can also be used for Huffman and arithmetic coding as described above.

It is also a good choice to directly pack all or part of the quantization index without entropy coding.

Because the statistical properties of the quantization indices differ significantly when the variable-resolution filterbank employs low and high-resolution modes, one embodiment of the present invention employs two entropy codebooks to encode the quantization indices in these two modes, respectively. A third library can be used for intermediate resolution modes, and it can also share this library with high or low resolution modes.

The present invention multiplexes 21 all quantization indices and other side information into a complete bit stream. Auxiliary information includes quantization step size, sampling rate, loudspeaker configuration, frame length, length of quasi-stationary segment, code of entropy codebook, etc. Other ancillary information such as timecodes can also be packed in the bitstream.

Prior art systems need to communicate the number of quantization units per transient to the decoder, since the unpacking of the quantization step size, the codebook of the quantization index, and the quantization index itself all depend on this. In the present invention, however, since the selection of the quantization index codebook and its application range is decoupled from the quantization unit by a special method of entropy codebook selection 19, the bitstream can be constructed in such a way that the quantization index can be Unpacked before the number of quantization units required. Once the quantization indices are unpacked, they can be used to reconstruct the number of quantization units. This will be interpreted in the decoder.

With the above considerations, an embodiment of the present invention uses a bitstream structure as shown in Figure 16 when using a semi-hybrid filter bank or switchable filter bank+ADPCM, which essentially includes the following parts:

Synchronization word 81: indicates the beginning of the audio data frame;

Frame header 82: includes relevant information of the audio signal, such as sampling rate, normal channel number, LFE (low frequency effect) channel number, speaker configuration, etc.;

Channels 1, 2, ..., N, 83, 84, 85: all audio data for each channel is packed here;

Ancillary data 86: includes ancillary data such as timecode;

Error detection 87: error detection code is inserted here to detect errors occurring in the current frame, so that the error handling program can start when a bitstream error is detected;

The audio data for each channel is also structured as follows:

Window type 90: indicates that a window such as that shown in Figure 5 is used for the encoder so that the decoder can use the same window;

• Transient Position 91: A frame only for transients, which indicates the position of each transient segment. If run-length codes are used, this is where the length of each transient segment is packed;

Interleaving Decision 92: One bit, only in the transient frame, indicating whether to interleave the quantization index of each transient segment so that the decoder knows whether to interleave the quantization index;

Codebook index and application range 93: It conveys all the information about the entropy codebook and the application range of the quantization index, which includes the following parts:

○Number of codebooks 101: transmit the number of entropy codebooks of each transient segment of the current channel;

○ Application range 102: transmit the application range of each entropy codebook according to the quantization index or block, and they can also be encoded with entropy codes;

○ Codebook index 103: send this index to the entropy codebook, they can be further encoded with entropy codes;

Quantization index 94: conveys the entropy codes for all quantization indices of the current channel;

• Quantization step size 95: The exponent is passed to the quantization step size for each quantization unit, which can also be encoded with an entropy code. As explained previously, the number of step indices, or the number of quantization units, will be reconstructed by the decoder from the quantization indices as shown in 49;

Arbitrary resolution filter bank decision 96: one bit for each quantization unit, it only occurs when the switchable resolution analysis filter bank 28 adopts low frequency resolution mode, indicating whether the decoder should make a quantization unit Perform arbitrary resolution filter bank reconstruction (51 or 55) for all subband segments within ;

• Sum/difference coding decision 97: One bit for one of the quantization units being sum/difference coded. It is optional and only appears when sum/difference encoding is used, it indicates whether the decoder is to perform sum/difference decoding47;

· Joint Strength Coding Decision and Steering Vector 98: It conveys information about whether the decoder is to perform joint strength decoding, it is optional and only used for quantization units of the joint channels that are jointly strength coded, and only appears in the encoding When the device adopts joint strength coding, it includes the following parts:

○ Decision 121: One bit per joint quantization unit, indicating to the decoder whether to perform joint channel decoding on the subband samples in the quantization unit;

○ Polarity 122: One bit per joint quantization unit, indicating the polarity of the joint channel relative to the source channel:

o Steering vector 123: one scaling factor per joint quantization unit, which can be entropy coded;

· Auxiliary data 99: includes auxiliary information such as dynamic range control.

When a three-mode switchable filter bank is used, the bitstream structure is essentially the same as above, except:

• Window Type 90: Indicates which window such as that shown in Fig. 5 and Fig. 9 is used for the encoder so that the decoder can use the same window. Note that for frames with transients, this window type only refers to the last window in the frame, since the rest of the windows can be inferred from this window type, the transient position, and the last window used in the last frame;

• Transient position 91 : only occurs in the case of frames with transients. It first indicates whether this frame has slow transients 171 or not. If not, it indicates the transient position according to the middle block 172 and then according to the short block 173;

• Arbitrary resolution filter bank decision 96: it is irrelevant and therefore not used.

decoder

The decoder of the present invention basically implements the inverse process of the encoder, which is shown in Figure 13 and explained below.

A demultiplexer 41 decodes the quantization index from the bitstream, as well as ancillary information such as quantization step size, sampling rate, loudspeaker configuration and time code, etc. When prefixed entropy codes such as Huffman codes are used, this step is a single step combined with entropy decoding.

The quantization index codebook decoder 42 decodes the entropy codebook of quantization indices and their respective ranges of application from the bitstream.

The entropy decoder 43 decodes quantization indices from the bitstream based on the entropy codebooks provided by the quantization index codebook decoder 42 and their respective application ranges.

De-interlacing 44 is selectively employed only when there are transients in the current frame. If the decision bit unpacked from the bitstream indicates that interleaving 18 was invoked in the encoder, the quantization index is deinterleaved. Otherwise, the quantized exponent is passed without modification.

The present invention reconstructs the number of quantization units from the non-zero quantization index of each transient segment 49 . Let q(m,n) be the quantization index for the nth subband of the mth transient (if there is no transient in the frame, then there is only one transient), find the Maximum subband with nonzero quantization index:

${\mathrm{<span\; class="notranslate">\; band</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; \}</span>$

Recall that quantization units are defined by critical bands in frequency and transients in time, so the number of quantization units per transient is the minimum critical band that can accommodate Band _max (m). Let the frequency band (Cb) be the largest subband of the Cbth critical frequency band, and the number of quantization units for each transient segment m can be expressed as follows:

$\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\underset{\mathrm{<span\; class="notranslate">\; Cb</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; Cb</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; band</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; Cb</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>{\mathrm{<span\; class="notranslate">\; band</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span>$

Quantization step unpacking 50 unpacks the quantization step of each quantization unit from the bitstream.

Inverse quantization 45 reconstructs the subband samples from the quantization indices using the individual quantization steps of each quantization unit.

If the bitstream indicates that joint intensity coding15 was invoked in the encoder, joint intensity decoding46 copies the subband samples from the source channels and multiplies them by the polarity and steering vectors to reconstruct the subband samples of the joint channel:

Joint Channel = Polarity Steering Vector Source Channel

If the bitstream indicates that the sum/difference encoding 14 was invoked in the encoder, the sum/difference decoder 47 reconstructs the left and right channels from the sum/difference channels. Corresponding to the sum/difference coding example explained in sum/difference coding 14, the left and right channels can be reconstructed as:

                                                              

                             

The decoder of the present invention incorporates a variable resolution synthesis filter bank 48 which is essentially the inverse of the analysis filter bank used to encode the signal.

If a three-mode switchable-resolution analysis filterbank is used for the encoder, the operation of its corresponding synthesis filterbank is uniquely determined and requires the same window sequence for the synthesis process.

If a semi-hybrid filter bank or switchable filter bank + ADPCM is used for the encoder, the decoding process is described as follows:

· If the bitstream indicates that the current frame was coded in high frequency resolution mode with switchable resolution analysis filterbank 28, then switchable resolution synthesis filterbank 54 enters high frequency resolution mode accordingly and Reconstruct the PCM samples from the samples (see Figures 14 and 15).

If the bitstream indicates that the current frame is coded in low frequency resolution mode with a switchable resolution analysis filterbank 28, the subband samples are first sent to an arbitrary resolution synthesis filterbank 51 (Fig. 14) or inverse ADPCM55 (FIG. 15), and depending on which of the encoders is used, then complete their respective synthesis processes. PCM samples are then reconstructed from these synthesized subband samples by a switchable resolution synthesis filterbank in low frequency resolution mode 53 .

Synthesis filter banks 52, 51 and 55 are the inverses of analysis filter banks 28, 26 and 29, respectively. Their structure and operational processing are uniquely determined by the analysis filterbank. Therefore, whatever analysis filterbank is used in the encoder, its corresponding synthesis filterbank must be used in the decoder.

Low Encoding Latency Mode

When the high-frequency resolution mode of the switchable-resolution analysis filterbank is disabled by the encoder, the frame length can then be reduced to the block length of the switchable-resolution filterbank in low-frequency mode or an integer multiple thereof, This yields a much smaller frame size, resulting in much smaller delays required for encoder and decoder operations. This is the low coding delay mode of the present invention.

Although several embodiments have been described in detail for the purpose of illustration, various modifications may be made without departing from the scope and spirit of the invention. Accordingly, the invention is limited only by the appended claims.

Claims (87)

1. A method for encoding and decoding a multi-channel digital audio signal, comprising the steps of: Cluster the input PCM samples into quasi-stationary frames; Convert PCM samples to subband samples; Block quantization of subband samples into numerous quantization indices; Provide pre-designed code book library; Based on the local characteristics of the quantization index, the codebook is assigned to multiple groups of quantization indices, so that the application range of the codebook is independent of the block quantization boundary; Encoding codebook indices and their respective domains of application; Create a complete encoded data stream; send the complete encoded data stream; receiving the encoded data stream and unpacking the data stream; Decode the quantization index from the data stream; reconstructing subband samples from the decoded quantization indices; and Reconstruct audio PCM samples from reconstructed subband samples. 2. The method of claim 1, wherein said codebook assigning step comprises: converting the quantization index into a codebook index by assigning to each quantization index the index of the smallest available codebook capable of accommodating said quantization index, and assigning the code Book index clustering partitioned into multiple application domains. 3. The method of claim 1, wherein the duration of the quasi-steady state frame is between 2 and 50 milliseconds. 4. The method of claim 1, wherein said step of converting comprises using a resolution filter bank selectively switchable between high and low frequency resolution modes. 5. The method of claim 4, including the step of detecting a transient, using the high frequency resolution mode when no transient is detected, and switching to the low frequency resolution mode when a transient is detected. 6. The method of claim 5, wherein the subband samples are clustered and partitioned into quasi-stationary segments as the resolution filter bank is switched to a low frequency resolution mode. 7. The method of claim 4, wherein the resolution filter bank is configured to include a long window capable of bridging transitions from short windows immediately to another short window in order to process transients only one long window apart. 8. The method of claim 1, wherein said converting step includes using a resolution filter bank selectively switchable between a high-resolution mode, a low-resolution mode, and an intermediate-resolution mode, so that in a single frame Use multiple resolutions. 9. The method of claim 8, wherein the resolution filter bank is configured to include a window that bridges an immediate transition from one shorter window to another shorter window so that processing is only separated by one such window transient state. 10. The method of claim 6, comprising trimming the frequency resolution of each steady state segment using an arbitrary resolution filter bank or Adaptive Differential Pulse Code Modulation (ADPCM). 11. The method of claim 1, including the step of calculating a noise masking threshold. 12. The method of claim 11, wherein the calculating step is performed using a psychoacoustic model. 13. The method of claim 1, wherein the step of creating a plurality of quantization indices comprises quantizing the subband samples using a quantization step provided by a bit allocator that allocates bit resources into groups of subband samples to Keep the quantization noise power below a masking threshold. 14. The method of claim 1, comprising converting subband samples of left and right channel pairs into sum and difference channel pairs. 15. The method of claim 14, wherein said transforming step is performed using a sum/difference encoder. 16. The method of claim 1, comprising: extracting an intensity scaling factor of the joint channel compared to the source channel, merging the joint channel to the source channel, and discarding all subband samples in the joint channel. 17. The method of claim 16, wherein said extracting and combining steps are performed using a joint intensity encoder. 18. The method of claim 1, comprising rearranging the quantization index to reduce the total number of bits when a transient is present in a frame. 19. The method of claim 1, comprising: providing a run length encoder for encoding the application range of the codebook. 20. The method of claim 1, comprising: applying a transient clustering segmentation algorithm when a transient is detected. 21. The method of claim 1, wherein said combining step is performed using a multiplexer. 22. The method of claim 1, wherein the encoded data stream includes codebook indices and their application ranges, including the number of codebooks, application ranges and codebook indices. 23. The method of claim 1, wherein the variable synthesis resolution filterbank is used as a secondary Hybrid filterbanks where the first stage consists of an arbitrary resolution synthesis filterbank or an inverse adaptive differential pulse code modulation (ADPCM) and the second stage is a low frequency resolution mode of variable synthesis filterbanks . 24. The method of claim 1, wherein when the data stream indicates that the current frame is coded in a high frequency resolution mode using a switchable resolution analysis filter bank, the variable resolution synthesis filter bank is encoded at a high frequency resolution rate mode operation. 25. The method of claim 1, wherein said step of unpacking the data stream is performed using a demultiplexer. 26. The method of claim 1, wherein said decoding step uses an entropy decoder to decode the entropy-encoded codebook and uses a run-length decoder to decode its respective range of applications from the data stream. 27. The method of claim 1, wherein said decoding step further comprises decoding the subband quantization indices from the data stream using an entropy decoder. 28. The method of claim 27, comprising reconstructing the number of quantization units using the decoded quantization index. 29. The method of claim 1, comprising rearranging the quantization indices when a transient is detected in the current frame. 30. The method of claim 29, wherein said rearranging step is performed using a deinterleaver. 31. The method of claim 1, comprising: reconstructing the subband samples of the joint channel from the subband samples of the source channels using the joint intensity scale factor. 32. The method of claim 31, wherein said reconstructing step is performed using a joint intensity decoder. 33. The method of claim 1, comprising reconstructing subband samples of the left and right channels from the sum and difference subband channels. 34. The method of claim 33, wherein said reconstructing step is performed using a sum/difference decoder. 35. A method for encoding a multi-channel digital audio signal comprising the steps of: Divide the input PCM samples into quasi-stationary frames; Convert PCM samples to subband samples; Create numerous quantization indices by creating block quantization boundaries in subband samples; Provide pre-designed code library; Based on the local characteristics of the quantization index, the codebook is assigned to multiple groups of quantization indices, so that the application range of the codebook is independent of the block quantization boundary; Encoding codebook indices and their respective domains of application; and Create a complete encoded data stream for storage or transmission. 36. The method of claim 35, wherein said step of assigning codebooks includes converting quantization indices to codebook indices by assigning to each quantization index the smallest available codebook that can accommodate the quantization indices. 37. The method of claim 36, wherein the duration of the quasi-steady state frame is between 2 and 50 milliseconds. 38. The method of claim 35, wherein said converting step comprises using a resolution filter bank selectively switchable between high and low frequency resolution modes. 39. The method of claim 38, including the steps of detecting a transient, and using the high frequency resolution mode when a transient is not detected; and switching to the low frequency resolution mode when a transient is detected. 40. The method of claim 39, wherein the subband samples are clustered and partitioned into stationary segments as the resolution filterbank is switched to a low frequency resolution mode. 41. The method of claim 40, comprising trimming the frequency resolution of each steady state segment using an arbitrary resolution filter bank or Adaptive Differential Pulse Code Modulation (ADPCM). 42. The method of claim 41, wherein the resolution filter bank is configured to include a long window for bridging transitions from a short window immediately to another short window in order to process transients only one long window apart. 43. The method of claim 35, wherein said converting step includes the step of using a resolution filter bank selectively switchable between high, low, and medium frequency resolution modes so that when a transient is detected Multiple resolutions can be used in a single frame. 44. The method of claim 43, wherein the resolution filter bank is configured to include a window for bridging transitions from one shorter window immediately to another shorter window, in order to process images separated by only one such window transient. 45. The method of claim 35, wherein the step of creating a plurality of quantization indices comprises: using a quantization step size provided by a bit allocator that allocates bit resources into groups of subband samples such that the quantization noise power is less than A masking threshold. 46. The method of claim 35, including a step of calculating a noise masking threshold. 47. The method of claim 46, wherein the calculating step is performed using a psychoacoustic model. 48. The method of claim 35, comprising converting subband samples of left and right channel pairs to sum and difference channel pairs. 49. The method of claim 48, wherein the converting step is performed using a sum/difference encoder. 50. The method of claim 35, comprising extracting an intensity scaling factor of the joint channel compared to the source channel, merging the joint channel to the source channel, and discarding all associated subband samples of the joint channel. 51. The method of claim 50, wherein the steps of extracting and combining are performed with a joint intensity encoder. 52. The method of claim 35, comprising rearranging the quantization indices to reduce the total number of bits when a transient is present in the frame. 53. The method of claim 35, comprising: providing a run length encoder for encoding application boundaries of the codebook. 54. The method of claim 35, comprising: applying a transient clustering segmentation algorithm when a transient is detected. 55. The method of claim 35, wherein the step of creating the complete data stream is performed using a multiplexer. 56. A method for encoding and transmitting a multi-channel digital audio signal comprising the steps of: Divide the input PCM samples into quasi-stationary frames; Convert PCM samples to subband samples with a resolution filter bank that is selectively switchable between high, low and medium frequency resolution modes, so that multiple resolutions can be used in a single frame when a transient is detected; Detect transients, use high frequency resolution mode when no transients are detected, switch to low or medium frequency resolution mode when transients are detected, where subband samples are divided into The steady state segments, and the frequency resolution of each steady state segment in that frame are adjusted with the low or medium frequency resolution mode in the same frame; Create multiple quantization indices by creating block quantization boundaries in subband samples; Provide pre-designed codebook library; Based on the local characteristics of the quantization index, the codebook is assigned to multiple groups of quantization indices, so that the application range of the codebook has nothing to do with the block quantization boundary; Codebook indices and their respective areas of application; and Use a multiplexer to create a complete data stream for storage or transmission. 57. The method of claim 56, wherein the codebook assigning step includes converting quantized indices to codebook indices by assigning to each quantized index the smallest available codebook that accommodates said index. 58. The method of claim 56, wherein the step of creating a plurality of quantization indices comprises using a step size provided by a bit allocator that allocates bit resources into groups of subband samples such that the quantization noise power of each subband is low based on the calculated masking threshold. 59. The method of claim 56, comprising: using a psychoacoustic model to calculate the masking threshold. 60. The method of claim 56, comprising converting subband samples in the left and right channels to the sum and difference channels with a sum/difference encoder. 61. The method of claim 56, comprising: using a joint intensity encoder to extract an intensity scale factor of the joint channel compared to the source channel, merging the joint channel into the source channel, and discarding all correlations in the joint channel Subband samples. 62. The method of claim 56, comprising: providing a run length encoder for encoding application boundaries of the codebook. 63. The method of claim 56, wherein the resolution filter bank is configured to include a window for bridging transitions from one shorter window immediately to another shorter window, so as to process images separated by only one such window transient. 64. A method for decoding an encoded audio data stream comprising the steps of: Receive an encoded audio data stream and unpack the data stream; Decode the quantization index from the data stream; reconstructing subband samples from the decoded quantization indices; and Reconstruct audio Pulse Code Modulation (PCM) samples from reconstructed subband samples using a variable resolution synthesis filterbank switchable between low and high frequency resolution modes; Wherein, when the data stream indicates that the current frame is coded with a switchable resolution analysis filter bank in low frequency resolution mode, the variable synthesis resolution filter bank is used as a two-stage hybrid filter bank, wherein the first stage consists of an arbitrary resolution synthesis filter bank or an inverse adaptive differential pulse code modulation (ADPCM), the second stage is a low frequency resolution mode of the variable synthesis filter bank; and Wherein, when the data stream indicates that the current frame is coded with the switchable resolution analysis filter bank in the high frequency resolution mode, the variable resolution synthesis filter bank operates in the high frequency resolution mode. 65. The method of claim 64, wherein the step of unpacking the data stream is performed with a demultiplexer. 66. The method of claim 64, wherein the step of decoding is performed with an entropy decoder for decoding the entropy codebook and a run-length decoder for decoding its respective application range from the data stream. 67. The method of claim 66, wherein the decoding step further comprises decoding the quantization index from the data stream with an entropy decoder. 68. The method of claim 67, comprising reconstructing the number of quantization units from the decoded quantization indices. 69. The method of claim 67, comprising rearranging the quantization indices when a transient is detected in the current frame. 70. The method of claim 69, wherein the step of rearranging is performed using a deinterleaver. 71. The method of claim 64, comprising: reconstructing the subband samples of the joint channel from the subband samples of the source channels using the joint intensity scale factor. 72. The method of claim 71, wherein the step of reconstructing is performed using a joint intensity decoder. 73. The method of claim 64, comprising reconstructing subband samples of the left and right channels from the sum and difference subband channels. 74. The method of claim 73, wherein the step of reconstructing is performed using a sum/difference decoder. 75. The method of claim 64, wherein the resolution filter bank is configured to include a window for bridging transitions from one short window immediate to another short window in order to process transients only one long window apart. 76. A method for decoding an encoded audio bitstream comprising the steps of: Receive an encoded audio data stream and unpack the data stream; Decode the quantization index from the data stream; reconstructing subband samples from the decoded quantization indices; and Reconstruct audio Pulse Code Modulation (PCM) samples from reconstructed subband samples by using a variable resolution synthesis filterbank switchable between low, medium, and high frequency resolution modes; wherein the variable resolution synthesis filterbank operates in the high frequency resolution mode when the data stream indicates that the current frame is encoded with the switchable resolution analysis filterbank in the high frequency resolution mode; and Among them, when the data stream indicates that the current frame is divided into clusters and these segments are coded with a switchable resolution analysis filter bank in low or medium frequency resolution mode, for each segment of the frame, the variable resolution The synthesis filterbank operates in low or medium frequency resolution mode accordingly. 77. The method of claim 76, wherein the step of unpacking the data stream is performed with a demultiplexer. 78. The method of claim 76, wherein the step of decoding is performed using an entropy decoder for decoding an entropy codebook and a run-length decoder for decoding its respective application range from the data stream. 79. The method of claim 78, wherein the decoding step further comprises decoding the quantization index from the data stream with an entropy decoder. 80. The method of claim 79, comprising reconstructing the number of quantization units from the decoded quantization indices. 81. The method of claim 79, comprising rearranging the quantization indices when a transient is detected in the current frame. 82. The method of claim 81, wherein the step of rearranging is performed using a deinterleaver. 83. The method of claim 76, comprising: reconstructing the subband samples of the joint channel from the subband samples of the source channels using the joint intensity scale factor. 84. The method of claim 83, wherein the step of reconstructing is performed with a joint intensity decoder. 85. The method of claim 76, comprising reconstructing the subband samples of the left and right channels from the sum and difference subband channels. 86. The method of claim 85, wherein the step of reconstructing is performed using a sum/difference decoder. 87. The method of claim 76, wherein the resolution filter bank is configured to include a window capable of bridging an immediate transition from one shorter window to another shorter window to facilitate processing of images separated by only one such window. transient.

Images