CN1532808A - Method and device for encoding and/or decoding audio data using bandwidth extension technology - Google Patents

Abstract

The present invention provides methods and devices for encoding and decoding audio data using bandwidth extension techniques. The method includes: bandwidth extension encoding audio data, outputting bandwidth limited audio data, and generating bandwidth extension information; arithmetically encoding the bandwidth limited audio data into a layered structure having a base layer and at least one enhancement layer to control bit rate ; and multiplexing the arithmetic-coded bandwidth-limited audio data and the bandwidth extension information.

Description

Method and device for encoding and/or decoding audio data using bandwidth extension technology

This application claims foreign priority from Application No. 2003-17978 filed with the Korean Intellectual Property Office on March 22, 2003, the disclosure of which is hereby incorporated by reference in its entirety.

technical field

The present invention relates to encoding and decoding of audio data, in particular to a method and device for encoding and/or decoding audio data using bandwidth extension technology.

Background technique

With the development of digital signal processing technology, audio signals are mainly stored and played as digital data. A digital audio memory and/or playback device samples and quantizes an analog audio signal, converts the analog audio signal into pulse code modulated (PCM) audio data as a digital signal, and stores the pulse code modulated (PCM) audio data on a file such as a compact disc ( CD), Digital Versatile Disc (DVD) or the like, so that the data on the information storage medium can be played back when the user wants to listen to PCM audio data. Compared to analog audio signal storage and/or reproduction methods employed by slow-rotation (LP) records, tapes, or the like, digital audio signal storage and/or reproduction methods greatly improve sound quality and significantly reduce damage caused by long-term storage. sound quality deteriorates. However, large amounts of digital data sometimes create storage and transmission problems.

To solve the above problems, a variety of compression techniques for reducing the amount of digital audio data are used. The Motion Picture Experts Group audio standard drafted by the International Organization for Standardization (ISO) or the AC-2/AC-3 technology of the Dalby noise reduction system adopts the method of reducing the amount of data by using a sound quality model, which effectively reduces the signal irrelevant to the signal characteristics. amount of data. That is to say, the MPEG audio standard and AC-2/AC-3 technology provide almost the same sound quality as CD only at the bit rate of 64Kbps-384Kbps, that is, 1/6-1/8 of the existing digital coding technology.

However, all of the above techniques follow a method of detecting, quantizing, and encoding digital data at an optimum of a fixed bit rate. Therefore, when transmitting digital data via a network, the transmission bandwidth is reduced due to network condition limitations. Then, the network disconnects and the network service stops. That is, when digital data is transformed into a smaller bit stream to fit a mobile device with limited storage capacity, re-encoding may be performed to reduce the amount of data. For this, considerable computation is required.

Therefore, the applicant filed Korean Patent Application No. 97-61298 "Audio Coding and/or Decoding Method Capable of Controlling Bit Rate Using Bit-Sliced Arithmetic Coding (BSAC) Technology" to the Korean Intellectual Property Office on November 19, 1997 and device", this application was authorized on April 17, 2002, Korean Patent Registration No. 261253. According to the BSAC technique, a bit stream encoded at a high bit rate can be transformed into a bit stream with a low bit rate. Since only a part of the bit stream can be used for reconstruction, even if the network is overloaded, the performance of the decoder is not good, or the user requires a low bit rate, only a part of the bit stream can be used to provide users with moderate sound quality services (even if the performance of the decoder is as bad as the low bit rate) ). However, at low bit rates, decoder performance inevitably degrades.

However, BSAC technology utilizes a modified discrete cosine transform (MDCT) to transform the audio signal, which severely degrades the sound quality produced by the lower layers. Since the frequency resolution of the MDCT is constant, the frequency resolution of the insensitive portion of the human ear becomes high in consideration of the sound quality model. Therefore, according to MDCT, the sound quality deteriorates from the enhancement layer to the lower layer.

Contents of the invention

The present invention provides an audio data encoding and/or decoding method and apparatus capable of controlling the bit rate of audio data, and can reproduce high-quality sound even if restoration is performed using only a part of the bit stream.

The present invention also provides an audio data encoding and/or decoding method and apparatus capable of controlling a bit stream so that high-quality sound can be produced from a lower layer.

According to an aspect of the present invention, a method of encoding audio data is provided. The method includes: bandwidth extension encoded audio data, outputting bandwidth-limited audio data, and generating bandwidth extension information; arithmetically encoding the bandwidth-limited audio data into a layered structure with a base layer and at least one enhancement layer to control bit rate; and multiplexing the arithmetically encoded bandwidth-limited audio data and bandwidth extension information.

The arithmetic coding includes: differential coding corresponding to auxiliary information of the base layer; bit partition coding corresponding to multiple quantized sampling values of the base layer; and repeating differential coding and bit partition coding for the next enhancement layer until coding is completed for multiple predetermined layers.

The arithmetic coding includes: differential coding corresponds to auxiliary information including scale factor information and coding model information of the base layer; referring to the coding model information, bit-segmented coding corresponds to a plurality of quantized sampling values of the base layer; and repeating the difference for the next enhancement layer Encoding and bit-splitting encoding until multiple predetermined layers are encoded.

Quantized sample values are preferably obtained by pseudo-wavelet transform of the audio data.

The encoded bandwidth-limited audio data and the bandwidth extension information are multiplexed in the following order, locating a portion of the encoded bandwidth-limited audio data corresponding to the base layer, locating the bandwidth extension information, and locating the portion corresponding to the remaining enhancement layer already Encode bandwidth-constrained audio data.

Optionally, the encoded bandwidth-limited audio data and the bandwidth extension information are multiplexed in the following order, locating the bandwidth extension information, locating the part of the encoded bandwidth-limited audio data corresponding to the base layer, and locating the part corresponding to the enhancement Layer part of encoded bandwidth-constrained audio data.

According to another aspect of the present invention, a method of decoding audio data is provided. The method includes: demultiplexing an input audio bitstream and sampling bandwidth-limited audio data encoded as a layered structure including a base layer and at least one enhancement layer and bandwidth extension information; arithmetic decoding at least a portion corresponding to the base layer the bandwidth-limited audio data of the bandwidth-limited audio data; based on the decoded portion of the bandwidth-limited audio data and referring to the bandwidth extension information, generating audio data in a frequency band at least in part not covered by the decoded portion of the bandwidth-limited audio data, and then The generated audio data is added to the decoded portion of the bandwidth-constrained audio data.

The audio data is generated in the frequency band portion so as to reach the boundary of the decoded portion of the bandwidth-limited audio data. The audio data is generated in the part of the frequency band so as to reach the boundary of the filter bank for the pseudo-wavelet transform. If the audio data does not reach the boundary of the filter bank used for the pseudo-wavelet transform, an overlapping portion of the bandwidth-limited decoded portion of the audio data and the resulting audio data is inserted.

The input audio bitstream is demultiplexed in the following order: data corresponding to the base layer is sampled from the input audio bitstream, bandwidth extension information is sampled from the input audio bitstream, and data corresponding to the remaining enhancement layers is sampled from the input audio bitstream.

Optionally, said input audio bitstream is demultiplexed in the following order: sampling bandwidth extension information from the input audio bitstream, sampling data corresponding to the base layer from the input audio bitstream, and sampling data corresponding to the remaining enhancements from the input audio bitstream layer data.

The arithmetic decoding includes: differential decoding corresponding to the auxiliary information of the base layer; bit-segmented decoding corresponding to multiple quantized sampling values of the base layer; and repeating the differential decoding and bit-segmented decoding for the next enhancement layer until the decoding is completed for multiple predetermined layers.

The arithmetic decoding includes: differentially decoding auxiliary information corresponding to the base layer including scale factor information and coding model information; referring to the coding model information, bit-segmented decoding corresponding to a plurality of quantized sampling values of the base layer; and repeating the difference for the next enhancement layer Decoding and bit-slicing decoding until multiple predetermined layers are decoded.

According to yet another aspect of the present invention, an apparatus for encoding audio data is provided. The apparatus includes: a bandwidth extension encoder for bandwidth extension encoding audio data, outputting bandwidth-limited audio data and generating bandwidth extension information; a fine-grain scalable encoder for converting the bandwidth-limited audio data Arithmetic coding is in a hierarchical structure including a base layer and at least one enhancement layer to control a bit rate; and a multiplexer for multiplexing the arithmetic coding bandwidth-limited audio data and bandwidth extension information.

The fine-grained scalable encoder differentially encodes side information corresponding to the base layer, bit-split encodes multiple quantized sample values corresponding to the base layer, and bit-split encodes side information and multiple quantized sample values corresponding to the next enhancement layer until Multiple predetermined layers complete the encoding.

The differential encoding of the fine-grained scalable encoder corresponds to auxiliary information including scale factor information and encoding model information of the base layer, referring to the encoding model information, bit-segmented encoding corresponds to multiple quantized sampling values of the base layer, and encoding corresponds to the next enhanced Auxiliary information of a layer including scale factor information and coding model information is coded until a plurality of predetermined layers are coded, and a plurality of quantized sample values corresponding to the next enhancement layer are bit-divided coded. The fine-grained scalable encoder preferably obtains quantized sample values by pseudo-wavelet transforming the audio data.

The multiplexer multiplexes the encoded bandwidth-limited audio data and the bandwidth extension information in the following order: locating a portion of the encoded bandwidth-limited audio data corresponding to the base layer, locating the bandwidth extension information, and locating the corresponding Part of the encoded bandwidth-limited audio data in the remaining enhancement layers.

According to yet another aspect of the present invention, an apparatus for decoding audio data is provided. The device comprises: a demultiplexer that demultiplexes an input audio bit stream and samples the bandwidth-limited audio data encoded into a layered structure with a base layer and at least one enhancement layer and bandwidth extension information; a fine-grained a scalable arithmetic decoder for decoding at least a portion of the bandwidth-restricted audio data corresponding to the base layer; and a bandwidth extension decoder for generating at least a portion of the bandwidth-restricted audio data based on the decoded portion of the bandwidth-restricted audio data and referring to the bandwidth extension information audio data within the frequency band covered by the decoded portion of the bandwidth-limited audio data, and the resulting audio data is then added to the decoded portion of the bandwidth-limited audio data.

The fine-grained scalable Huffman decoder differentially decodes side information corresponding to a base layer, bit-partitioned decodes a plurality of quantized sample values corresponding to a base layer, and decodes side information corresponding to a next enhancement layer until a plurality of predetermined layers are fully decoding, and bit-sliced decoding a plurality of quantized sample values corresponding to the next enhancement layer.

The demultiplexer demultiplexes the input audio bitstream in the following order: data corresponding to the base layer is sampled from the input audio bitstream, bandwidth extension information is sampled from the input audio bitstream, and data corresponding to the base layer is sampled from the input audio bitstream Data for the rest of the enhancement layers. Optionally, the demultiplexer may demultiplex the input audio bitstream in the following order: sampling the bandwidth extension information from the input audio bitstream, sampling data corresponding to the base layer from the input audio bitstream, and sampling the data corresponding to the base layer from the input audio bitstream The samples correspond to the data of the remaining enhancement layers.

Description of drawings

Features and other advantages of the present invention will become more apparent by describing in detail exemplary embodiments of the present invention with reference to the accompanying drawings, in which:

Fig. 1 is a block diagram of an encoding device according to the present invention;

Fig. 2 is a detailed block diagram of the encoding device shown in Fig. 1;

Fig. 3 is a block diagram of a decoding device according to the present invention;

Fig. 4 is a detailed block diagram of the decoding device shown in Fig. 3;

Figure 5 shows the structure of a bitstream output from a Fine Grain Scalable (FGS) encoder 2;

Fig. 6 shows the detailed structure of the auxiliary information shown in Fig. 5;

Figure 7 shows the structure of the bit stream output from the multiplexer 3 or input to the demultiplexer 7;

8 is a schematic diagram for explaining an arithmetic encoding and decoding method performed by an encoding and decoding device according to the present invention;

FIG. 9 is a schematic diagram for explaining bandwidth extension decoding performed by a bandwidth extension (BWE) decoder 9 in more detail;

Fig. 10 is a flowchart for illustrating an encoding method according to the present invention;

FIG. 11 is a flowchart illustrating a decoding method according to the present invention.

Detailed ways

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Fig. 1 is a block diagram of an encoding apparatus according to the present invention. As shown in Figure 1, the encoding device receives and encodes PCM audio data, and outputs the PCM audio data as an audio bit stream, and the encoding device includes a bandwidth extension (BWE) encoder 1 and a fine-grained scalable (FGS) encoder 2 and a multiplexer 3 .

The BWE encoder 1, BWE encodes PCM audio signals, outputs bandwidth-limited audio data, and generates BWE information. BWE encoding refers to a technique for receiving audio data, segmenting a portion of the audio data within a high frequency band, and generating side information necessary to restore the segmented portion of the audio data. Here, the rest of the audio data is referred to as "bandwidth limited audio data" and the side information is referred to as "BWE information". An example of BWE technology is Spectral Band Replication (SBR) technology developed from coding technology. Details of the SBR technology are disclosed in "Conference Proceedings 5560" at the 112th Audio Engineering Society Meeting, May 10-13, 2002.

The FGS encoder 2 encodes bandwidth-limited audio data into a layered structure having a base layer and at least one enhancement layer to control a bit rate. FGS encoding refers to a technique for encoding data into a multi-layer structure to control a bit rate, ie, provides FGS. The BSAC technique disclosed in Korean Patent Application No. 97-61298 is an example of FGS encoding. That is to say, the FGS encoder 2 differential encoding corresponds to the auxiliary information of the base layer, the bit-segmented encoding corresponds to a plurality of quantized sampling values of the base layer, and the differential encoding corresponds to the auxiliary information of the next enhancement layer until a plurality of predetermined layers are coded, and Bit-partitioned coding corresponds to multiple quantized sample values of the next enhancement layer. Here, the side information includes scale factor information and encoding model information, and quantized sample values are obtained by transforming and quantizing input audio data. The side information and quantized sample values will be described in detail below.

The multiplexer 3 multiplexes the bandwidth-limited PMC audio data encoded by the FGS encoder 2 and the BWE information produced by the BWE encoder 1 .

Fig. 2 is a detailed block diagram of the encoding device shown in Fig. 1 . As shown in FIG. 2 , the encoding device includes a BWE encoder 1 , a FGS encoder 2 and a multiplexer 3 . Blocks that perform the same functions as those in FIG. 1 use the same reference numerals and will not be repeated here.

Specifically, the FGS encoder 2 includes a pseudo-wavelet transform (PWT) unit 21 , a sound quality unit 22 , a quantization unit 23 and a FGS arithmetic coding unit 24 .

The PWT unit 21 receives PCM audio data in the time domain, and transforms the PCM audio data pseudo-wavelet into an audio signal in the frequency domain with reference to the sound quality model information provided by the sound quality unit 22 . The characteristics of the audio signal that can be perceived by humans, hereinafter referred to as the perceptual audio signal, do not differ much in the time domain. In contrast, considering the sound quality model, the characteristics of perceptual and non-perceptual audio signals in the frequency domain are quite different. Therefore, compression efficiency can be improved by allocating different numbers of bits to each frequency band. MDCT produces perceptual noise due to only slight frequency distortions produced by high-frequency resolution in the low-frequency band. Compared to MDCT, PWT provides stable sound quality even from lower layers with lower frequency bands due to moderate time/frequency resolution.

The sound quality unit 22 provides information about the sound quality model such as processing (attack) detection information to the PWT unit 21, packs the audio signal transformed by the PWT unit 21 into a sub-band audio signal, and utilizes the masking generated by the interaction between the sub-band signals. The effect computes a masked threshold for each subband and provides the masked threshold to the quantization unit 23 . The masking threshold represents the maximum power of an audio signal that a human cannot perceive due to the interaction between the audio signals. In this embodiment, the sound quality unit 22 calculates the masking threshold and the like for the stereo part using Binaural Masking Level Depression (BMLD).

The quantization unit 23 scalarizes each sub-band audio signal based on the corresponding scale factor information to reduce the quantization noise energy of each sub-band to be lower than the masking threshold provided by the sound quality unit 22, and then outputs the quantized sample value so that people can hear the sub-band Audio signal but no perceivable noise in it. That is, the quantization unit 23 quantizes the sub-band audio signal by a noise masking ratio (NMR), which represents the ratio of the noise generated by each sub-band to the masking threshold calculated by the sound quality unit 22, which is 0 dB or less in the full bandwidth. An NMR of 0 dB or less means that one cannot hear quantization noise.

The FGS arithmetic coding unit 24 codes quantized sample values and side information belonging to each layer into a hierarchical structure. The side information includes scale band information, coding band information, scale factor information, and coding model information corresponding to each layer. The proportional band information and the encoding band information may be packed into header information constituting each frame of the audio bitstream, and then transmitted to the decoding device. Optionally, the proportional band information and coded band information may be encoded and packaged as auxiliary information corresponding to each layer, and then transmitted to the decoding device. Also, since the proportional band information and the encoded band information are already stored in the decoding device, the proportional band signal and the encoded band information may not be transmitted to the decoding device.

In more detail, the FGS arithmetic coding unit 24 differentially encodes auxiliary information corresponding to the first layer including scale factor information and coding model information, and at the same time refers to the coding model information to bit-segment code and quantize the sample values corresponding to the first layer. Bit-segmented coding means the coding used in the above-mentioned BSAC, sequentially lossless coding the most significant bit, the next most significant bit... and the least significant bit. The second layer is treated the same as the first layer. That is, a plurality of predetermined layers are sequentially coded layer by layer. The first layer is called the base layer, and the remaining layers are called enhancement layers. The hierarchical structure will be described in more detail later.

The scale band information is necessary to perform quantization correctly depending on the frequency characteristics of the audio signal, and when the frequency domain is divided into bands and each band is assigned a correct scale factor, the scale band information notifies each of the corresponding scale bands layer. Therefore, each layer belongs to at least one proportional band. Each scale band is assigned a scale factor. The encoding band information is necessary to correctly perform encoding depending on the frequency characteristics of the audio signal, and when the frequency domain is divided into bands and each band is assigned a correct encoding model, the encoding band information notifies each of the corresponding encoding bands layer. The correct division of scale bands and coding bands is done by testing, and then the corresponding scale factors and coding models are determined.

Multiplexer 3 multiplexes the encoded bandwidth-limited audio data and BWE information in the following order: locate the encoded quantized sample value data corresponding to the base layer, locate the BWE information, and locate the encoded quantized sample value data corresponding to the remaining enhancement layers. Encodes quantized sample value data. The multiplexer 3 alternatively multiplexes the encoded bandwidth-limited audio data and the BWE information in the following order: locate the BWE information, locate the encoded quantized sample value data corresponding to the base layer, and locate the Encoded quantized sample value data.

FIG. 3 is a block diagram of a decoding device according to the present invention. As shown in FIG. 3 , the decoding device receives and decodes the audio bit stream, and then outputs the audio data. The decoding device includes a demultiplexer 7 , a FGS decoder 8 and a BWE decoder 9 .

Demultiplexer 7 demultiplexes an input audio bitstream into sample bandwidth limited audio data that has been encoded into layers having a base layer and at least one enhancement layer with BWE information therein structure. Here, bandwidth-limited audio data and BWE information are the same as described with reference to FIG. 1 . The FGS decoder 8 arithmetically decodes at least a portion of the bandwidth-limited audio data corresponding to the base layer. The layer that performs the decoding is related to network state, user selection, etc.

Based on the bandwidth-limited audio data portion arithmetically decoded by the FGS decoder 8 and with reference to the BWE information sampled by the demultiplexer 7, the BWE decoder 9 generates bandwidth-limited audio data in at least a portion not arithmetically decoded by the FGS decoder 8 The audio data within the frequency band covered by the audio data of the FGS decoder 8 is then added to the bandwidth-limited audio data that has been arithmetically decoded by the FGS decoder 8 .

Since the present invention employs PWT, the BWE decoder 9 goes through the following process. When decoding is performed using PWT, the division frequency is selected by determining the last point in the frequency domain during determination of bandwidth-limited audio data. Due to the low frequency resolution in the high frequency part, PWT cannot precisely limit the bandwidth according to the last point determined like MDCT. In the decoding process, the BWE decoder 8 arranges the core part generated by the FGS decoder 9 into the frequency domain, confirms the frequency bandwidth of the core part, and modifies and decodes the BWE part into an appropriate frequency bandwidth.

For example, let us assume that only 8 layers are reconstructed in a 16-layer bitstream encoded at a bit rate of 64kbps, corresponding to a frequency of 8.5kHZ for layer 8. In this case, the BWE decoder 8 has to reconstruct data in the frequency range of 8.5kHZ-15kHZ or wider. Due to the characteristics of the quadrature mirror filter (QMF), the BWE decoder 8 can adjust the frequency bandwidth based on the channel bandwidth of the quadrature mirror filter. When the nth frequency bandwidth of QMF is 8.3kHZ, the frequency components within the frequency bandwidth range of 8.3-8.5kHZ are contained in the core part or the BWE part. Therefore, the core part or the BWE part must be handled correctly.

The first way to deal with the core and BWE section is to remove frequency components in the 8.3-8.5kHZ frequency bandwidth range from the core. In this method, the FGS decoder 9 performs decoding in consideration of the bandwidth information of the BWE part. The second method is to use the QMF used in the BWE decoder 8 to filter the data of the core part, generate the QMF data by interpolation, and reconstruct the data of the core part by inverse quadrature mirror filtering of the QMF data.

As described above, even if the audio data decoded by the FGS decoder 8 is only baseband audio data, the BWE decoder 9 creates missing band audio data and fills it into the baseband audio data. Therefore, the quality of decoded audio data can be improved.

FIG. 4 is a detailed block diagram of the decoding device shown in FIG. 3 . As shown in FIG. 4 , the decoding device includes a demultiplexer 7 , a FGS decoder 8 and a BWE decoder 9 . The same reference numerals are used for the blocks that perform the same functions in FIG. 3 , and will not be repeated here.

Specifically, to control the bit rate, the FGS decoder 8 performs decoding up to a target layer, which is determined by network status, decoding device performance, user selection, and the like. The FGS decoder 8 includes a FGS arithmetic decoding unit 81 , an inverse quantization unit 82 and a PWT inverse transform unit 83 . The FGS arithmetic decoding unit 81 performs decoding up to the target layer of the audio bitstream. In more detail, based on encoding model information obtained by decoding side information including scale factor information and encoding model information corresponding to each layer, the FGS arithmetic decoding unit 81 arithmetically decodes encoded quantized sample values corresponding to each layer to obtain quantization sample value. The process of obtaining quantized sample values will be explained in detail below.

Scale band information and encoding band information can be obtained from the header information of the audio bitstream or side information of each decoding layer. Optionally, the decoding device may store proportional band information and coding band information in advance.

The inverse quantization unit 82 dequantizes and reconstructs quantized sample values of each layer based on scale factor information corresponding to each layer. The PWT inverse transform unit 83 frequency/time maps the reconstructed sample value, inversely transforms the mapped sample value into a time-domain PCM audio data, and outputs the time-domain PCM audio data.

The BWE decoder 9 includes a transformation unit 91 , a high frequency generation unit 92 , an adjustment unit 93 and a synthesis unit 94 . The transform unit 91 transforms the time domain PCM audio data output from the PWT inverse transform unit 83 into frequency domain data. The frequency domain data is called the low frequency part. The high frequency generation unit 92 creates a part not covered by the frequency domain data, that is, a high frequency part obtained by copying the low frequency part with reference to the BWE information and then adding the copied low frequency part to the frequency domain data, ie, the original low frequency part. The adjustment unit 93 adjusts the level of the high frequency part generated by the high frequency generation unit 92 using the envelope information included in the BWE information. The encapsulation information transmitted from the code point indicates the encapsulation information of the audio data corresponding to the high frequency part divided by the code point in the BWE encoding process. The synthesis unit 94 synthesizes the low frequency part output from the transformation unit 91 and the high frequency part output from the adjustment unit 93, and then outputs PCM audio data.

As described above, while the FGS decoder 8 decodes only the baseband audio data, the BWE decoder 9 reconstructs the missing band audio data and fills the missing band audio data into the baseband audio data. Therefore, the baseband audio data quality is improved.

FIG. 5 shows the structure of the bit stream output from the FGS encoder 2. As shown in FIG. As shown in Fig. 5, FGS encoder 2 encodes bitstream frames by mapping quantized samples and side information into a fine-grained scalable (FGS) hierarchical structure. That is, this frame has a hierarchical structure in which bitstreams of lower layers are included in bitstreams of enhancement layers. The auxiliary information necessary for each layer is encoded layer by layer.

The header area where the header information is stored is at the beginning of the bit stream, the information of the zeroth layer is packed, and the information of the first to the Nth layers of the enhancement layer is packed sequentially. The base layer ranges from the header area to the zeroth layer information, the first layer ranges from the header area to the first layer information, and the second layer ranges from the header area to the second layer information. Likewise, the highest enhancement layer ranges from the header area to the Nth layer information, that is, from the base layer to the Nth layer. Both side information and encoded data are stored as per-layer information. For example, side information 2 and encoded quantized sample values are stored as second layer information. Here, N is a natural number greater than or equal to "1".

FIG. 6 shows a detailed structure of the auxiliary information shown in FIG. 5 . As shown in Figure 6, side information and encoded quantized sample values are stored as arbitrary layer information. In the current embodiment, since the quantized sampling values have been arithmetically coded, the auxiliary information includes arithmetic coding model information, scaling factor information, channel auxiliary information and other auxiliary information. The arithmetic coding model information refers to index information of an arithmetic coding model used to encode or decode quantized sample values contained in a corresponding layer. The scalefactor information informs the corresponding layer of the quantization step size suitable for quantizing or dequantizing the audio data contained in the corresponding layer. Channel side information refers to channel-related information such as Mid/Side (M/S) stereo. Other auxiliary information is identification information indicating whether to use M/S stereo.

In this embodiment, the FGS encoder 2 of the encoding device differentially encodes auxiliary information including arithmetic coding model information and scale factor information. Since each scale band has a scale factor, to encode the scale factors, first the smallest scale factor among the scale factors belonging to the scale band is arithmetically encoded, and then the difference between the smallest scale factor and the other scale factors is arithmetically encoded. An arithmetic coding model and information within the allowed bit range corresponding to each coding band is coded according to the method of coding the quantization step size, ie differential coding.

In this embodiment, the FGS decoder 8 of the decoding device arithmetically decodes side information including arithmetic coding model information and scale factor information. Since each scale band has a scale factor, to decode the scale factors, first the smallest scale factor among the scale factors belonging to the scale band is arithmetically decoded, and then the difference between the smallest scale factor and the other scale factors is arithmetically decoded. An arithmetically encoded model and information within the allowed bit range corresponding to each encoding band is arithmetically decoded in the same manner as the scale factor.

FIG. 7 shows the structure of the bit stream output from the multiplexer 3 or input to the demultiplexer 7 . As shown in Figure 7, the zeroth layer, that is, the base layer encoded by the FGS encoder 2, is located at the beginning of the bit stream, the BWE information is after the zeroth layer, and the enhancement layers, namely the first layer, the second layer... and the Nth layer layer, after the BWE information. Although the decoding point only receives or decodes the base layer, the decoding point can create the missing layer audio data based on the decoded audio data of the base layer and refer to the BWE information.

FIG. 8 is a diagram for explaining an arithmetic encoding and decoding method performed by an encoding and decoding device according to the present invention. As shown in Figure 8, the dot matrix rectangular box represents the spectral lines that constitute the quantized sampling values, where A represents the line used to form the boundary between layers, and B represents the boundary line used to divide the spectral line to correspond to the terminal node of the PWT tree structure .

The PWT and/or inverse PWT used in the encoding and/or decoding method according to the present invention performs frequency transformation and/or inverse frequency transformation using a tree structure to represent frequencies closer to the state of the filter bank corresponding to the human ear. The last nodes of the tree structure correspond to arithmetic coding scale frequency bands respectively. Therefore, each last node corresponds to a scaling factor.

A coding band as a unit for transmitting arithmetic coding model information necessary for arithmetic coding can be determined in consideration of coding efficiency. For example, let's assume that the last node has the same scale band and code band. As shown in Figure 8, layers and final nodes are mapped. Since the data corresponding to the last node exists in the time domain of the same frequency band, the data corresponding to the last node is not divided in the process of dividing the layers.

The zeroth layer is fixed to encode band a, the first layer is fixed to encode band b, the second layer is fixed to encode band c, the third layer is fixed to encode band d, the fourth layer is fixed to encode Encoding is performed for the frequency band e, the fifth layer is fixed so that encoding is performed for the frequency band f, the sixth layer is fixed so that encoding is performed for the frequency band g, and the seventh layer is fixed so that encoding is performed for the frequency band h.

First, the quantized sampling value corresponding to the zeroth layer is arithmetically coded within the allowable bit range by using the corresponding coding model. The side information of the zeroth layer is arithmetically coded. The amount of bits is calculated when bit-slicing the quantized sample values of the zeroth layer. If the amount of bits exceeds the allowable bit range, the coding of the zeroth layer is stopped, and then the arithmetic coding of the first layer is started. When the allowed bit ranges of the first and second layers have additional bit parts, the uncoded quantized sample values of the zeroth layer are coded.

The quantized sample values corresponding to the first layer are encoded using the coding model corresponding to the first layer. Auxiliary information for the first layer of arithmetic coding. After encoding all quantized sample values of the first layer, the uncoded quantized sample values of the zeroth layer are encoded until the allowed bit range is reached, in case the allowed bit range of the first layer has an additional bit portion. When the allowed bit range is reached, the encoding of the first layer is stopped, and then the encoding of the second layer is started. This processing is performed up to the seventh layer, thereby completing the encoding of the seventh layer.

If all quantized sample values of each layer are encoded regardless of the allowed bit range, i.e. all quantized sample values of each layer are encoded even if the amount of encoded bits exceeds the allowed bit range, a part of the next The range of bits allowed by the layer. In this way, quantized sample values belonging to the next layer may not be coded. Therefore, if bit-rate scalable encoding is performed, ie, decoding is performed only on lower layers instead of all layers, quantized sample values within a predetermined frequency are not decoded. As a result, the decoded quantized sample values vary up and down the frequency band, causing a birdy effect that deteriorates the sound quality.

Since the decoding process calculates the amount of bits in accordance with the allowable bit range when the process reverse to the encoding process is performed, a time point at which decoding of a predetermined layer starts can be detected.

Encoding is performed on spectral lines from "msb" direction to "lsb" direction. Here, at the last node of the tree structure used for waveform transformation, the data bits on the same bit plane have to be coded together. For example, when the last node has the following quantized sample values,

00000000101010110101

11111100000000000000

00001100110000000110

Based on MDCT, quantized sample values are grouped into five 4x4 bit planes and coding is performed from left to right and top to bottom. However, based on PWT, all quantized sample values are taken as one bit plane and coding is performed based on N bits from most significant bit to least significant bit from lower frequency to higher frequency. The most significant bit "00000000101010110101" is encoded from left to right based on N bits, the subsequent bits "11111100000000000000" are encoded from left to right based on N bits, and the least significant bit "00001100110000000110" is encoded based on N bits. Here, N is an integer greater than or equal to "1". In particular, if N is 1, binary encoding is performed. Since arithmetic coding can assign bits to decimal positions, eg 0.001 bit, a large amount of information can be encoded with only a small number of bits when encoding one bit. That said, the coding efficiency is quite high. Huffman coding, another lossless coding, requires at least one bit per symbol, so arithmetic coding has poor coding efficiency.

FIG. 9 is a diagram for explaining BWE decoding performed by the BWE decoder 9 . As shown in FIG. 9 , the stripe part represents the data decoded by the FGS decoder 8 , and the dot matrix part represents the data created by the BWE decoder 9 . When all the data in the 1/4 part of the sampling frequency Fs belong to the base layer, Fig. 9(a) shows a situation where a decoding node only decodes baseband data, and Fig. 10(b), (c) and (d) show The case is shown where the FGS decoder 8 decodes data corresponding to the baseband and at least one enhancement layer. That is, the FGS decoder 8 can decode data to control the bit rate, while the BWE decoder 9 can create missing band data that the FGS decoder 8 cannot decode.

An encoding and decoding method according to a preferred embodiment of the present invention will be described based on the above structure.

FIG. 10 is a flowchart for explaining an encoding method according to the present invention. As shown in Fig. 10, in step 1001, an encoding device BWE encodes audio data, outputs bandwidth-limited audio data, and generates BWE information corresponding to the base layer. The BWE information of the base layer is necessary for creating missing band audio data based on the audio data belonging to the base layer using the decoding node, and includes encapsulation information. The encoding means encodes bandwidth-limited audio data into a layered structure having a base layer and at least one enhancement layer to control a bit rate. In more detail, in step 1002, the encoding device layer-by-layer pseudo-wavelet transforms the bandwidth-limited audio data, in step 1003, quantizes the bandwidth-limited audio data, and in step 1004, Huffman encodes the bandwidth-limited audio data And the bandwidth-limited audio data is packaged into a layered structure to control the bit rate. In step 1005, the encoding device multiplexes the bandwidth-limited audio data and BWE information, and then outputs an audio bitstream. In more detail, the encoding device multiplexes the encoded bandwidth-limited audio data and the BWE information in the following order: locating a portion of the encoded bandwidth-limited audio data corresponding to the base layer, locating the BWE information, and locating the remaining enhanced Part of the layer has encoded bandwidth-constrained data. Or multiplexed in the following order: locate the BWE information, locate the portion of the encoded bandwidth limited audio data corresponding to the base layer, and locate the portion of the encoded bandwidth limited audio data corresponding to the remaining enhancement layers.

FIG. 11 is a flowchart illustrating a decoding method according to the present invention. Referring to Fig. 11, in step 1101, the decoding device demultiplexes an input audio bit stream and samples bandwidth-limited audio data, which has been encoded to have a base layer and at least one enhancement layer and BWE information layered structure. That is, the decoding means demultiplexes the input audio bitstream in the following order: it samples data from the input audio bitstream corresponding to the base layer, BWE information and data corresponding to the remaining enhancement layers, it either samples from the input audio bitstream The BWE information in , the data corresponding to the base layer and the data corresponding to the rest of the enhancement layers. Then, the decoding means decodes at least a part of the bandwidth-limited audio data corresponding to the base layer to control the bit rate. In more detail, at step 1102, the decoding means performs arithmetic decoding up to the target layer, inverse quantization at step 1103, and pseudo-wavelet transform at step 1104 to obtain PCM audio data. In step 1105, based on the PCM audio data obtained in step 1104 and referring to the BWE information, the decoding device creates PCM audio data in at least a part of the frequency band not covered by the PCM audio data obtained in step 1104, and then converts the created PCM audio data into Audio data is added to the PCM audio data obtained in step 1104 .

As described above, the present invention provides a bit-scalable encoding and decoding method and device, which can obtain high-quality sound only by restoring part of the bit stream.

High FGS can be provided with a small amount of data based on arithmetic coding, and based on PWT, the frequency resolution can be the same as the human ear transmission function. Therefore, PWT-based coding has better time/frequency domain resolution than existing MDCT-based coding. Thus, high-quality sound can be generated from lower layers.

Although the invention has been described in detail with reference to exemplary embodiments, it is obvious that various changes in form and details of the invention can be made by persons skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims. Change.

Claims (23)

1. A method of encoding audio data, the method comprising: Perform bandwidth extension encoding on audio data, output bandwidth-limited audio data, and generate bandwidth extension information; Arithmetic coding of bandwidth-limited audio data into a layered structure with a base layer and at least one enhancement layer, thereby controlling the bit rate; The arithmetically encoded bandwidth-limited audio data and bandwidth extension information are multiplexed. 2. The method of claim 1, wherein arithmetic coding comprises: The differential encoding corresponds to the auxiliary information of the base layer; Bit-segmented coding corresponds to multiple quantized sample values of the base layer; And repeat the differential encoding and bit-splitting encoding for the next enhancement layer until the encoding is completed for a plurality of predetermined layers. 3. The method of claim 1, wherein arithmetic coding comprises: The differential encoding corresponds to the auxiliary information of the base layer including scale factor information and encoding model information; Referring to the coding model information, the bit-segmented coding corresponds to a plurality of quantized sampling values of the base layer; The differential encoding and bit-splitting encoding are repeated for the next enhancement layer until a number of predetermined layers have been encoded. 4. A method as claimed in claim 2 or 3, wherein the quantized sample values are obtained by pseudo-wavelet transform of the audio data. 5. The method of claim 1, wherein the encoded bandwidth-limited audio data and the bandwidth extension information are multiplexed in the following order: locating a portion of the encoded bandwidth-limited audio data corresponding to the base layer, locating the bandwidth extension information, and locate the portion of encoded bandwidth-limited data corresponding to the remaining enhancement layers. 6. The method of claim 1, wherein the encoded bandwidth limited audio data and the bandwidth extension information are multiplexed in the following order: locate the bandwidth extension information, locate the part of the encoded bandwidth limited audio corresponding to the base layer data, and locate the portion of encoded bandwidth-limited data corresponding to the remaining enhancement layers. 7. A method of decoding audio data, the method comprising: demultiplexing an input audio bitstream and sampling bandwidth-limited audio data encoded as a layered structure comprising a base layer and at least one enhancement layer, and bandwidth extension information; arithmetically decoding at least a portion of the bandwidth-limited audio data corresponding to the base layer; Based on the decoded portion of the bandwidth-limited audio data and with reference to the bandwidth extension information, audio data within at least a portion of a frequency band not covered by the decoded portion of the bandwidth-limited audio data is generated, and the generated audio data is then Fill in the decoded portion of bandwidth-constrained audio data. 8. A method as claimed in claim 7, wherein the audio data within the part of the frequency band is generated so as to reach the boundary of the bandwidth-limited encoded part of the audio data. 9. A method according to claim 8, wherein the audio data in this part of the frequency band is generated so as to reach the boundary of the filter bank used for the pseudo-wavelet transform. 10. The method of claim 8, wherein overlapping portions of the decoded portion of the bandwidth-limited audio signal and the resulting audio data are inserted if the audio data does not reach the boundary of the filter bank used for the pseudo-wavelet transform. 11. The method according to claim 7, wherein the input audio bitstream is demultiplexed in the following order: sampling data corresponding to the base layer from the input audio bitstream, sampling bandwidth extension information from the input audio bitstream, and sampling the input audio bitstream from Stream samples correspond to the data of the remaining enhancement layers. 12. The method according to claim 7, wherein the input audio bitstream is demultiplexed in the following order: sampling the bandwidth extension information from the input audio bitstream, sampling data corresponding to the base layer from the input audio bitstream, and sampling the data corresponding to the base layer from the input audio bitstream. Stream samples correspond to the data of the remaining enhancement layers. 13. The method of claim 7, wherein arithmetic decoding comprises: Differential decoding corresponds to the auxiliary information of the base layer; bit-sliced decoding of multiple quantized sample values corresponding to the base layer; The differential decoding and bit-sliced decoding are repeated for the next enhancement layer until decoding is complete for a number of predetermined layers. 14. The method of claim 7, wherein arithmetic decoding comprises: Differential decoding corresponds to auxiliary information of the base layer including scale factor information and encoding model information; Referring to the coding model information, the bit-segmented decoding corresponds to a plurality of quantized sample values of the base layer; And repeat the differential decoding and bit-splitting decoding for the next enhancement layer until the decoding is completed for a plurality of predetermined layers. 15. An apparatus for encoding audio data, the apparatus comprising: A bandwidth extension encoder, used for bandwidth extension encoding audio data, outputting bandwidth limited audio data and generating bandwidth extension information; a fine-grained scalable encoder for encoding bandwidth-limited audio data into a layered structure comprising a base layer and at least one enhancement layer to control the bit rate; A multiplexer for multiplexing the arithmetically coded bandwidth limited audio data and the bandwidth extension information. 16. The apparatus of claim 15, wherein the fine-grained scalable encoder differentially encodes side information corresponding to the base layer, bit-segmented encoding corresponds to multiple quantized sample values of the base layer, and bit-segmented encoding corresponds to the next enhancement layer The auxiliary information and multiple quantized sample values are coded until multiple predetermined layers are completed. 17. The apparatus according to claim 15, wherein the fine-grained scalable encoder differential encoding corresponds to auxiliary information including scale factor information and coding model information of the base layer, and the reference coding model information bit-segmented coding corresponds to multiple quantizations of the base layer The sampling value is coded corresponding to the auxiliary information of the next enhancement layer, including the scale factor information and the coding model information until the coding is completed for multiple predetermined layers, and bit-divided coding is corresponding to the auxiliary information of the next enhancement layer and a plurality of quantized sample values. 18. The apparatus according to claim 15, wherein the fine-grained scalable encoder obtains quantized sample values by pseudo-wavelet transforming the audio data. 19. The apparatus according to claim 15, wherein the multiplexer multiplexes the encoded bandwidth-limited audio data and the bandwidth extension information in the following order: Locate a portion of the encoded bandwidth-limited audio corresponding to the base layer data, locate the bandwidth extension information, and locate the portion of encoded bandwidth-limited data corresponding to the remaining enhancement layers. 20. An apparatus for decoding audio data, the apparatus comprising: A demultiplexer for demultiplexing an input audio bitstream and sampling bandwidth-limited audio data encoded into a hierarchical structure with a base layer and at least one enhancement layer and bandwidth extension information; a fine-grain scalable arithmetic decoder for decoding at least a portion of bandwidth-limited audio data corresponding to the base layer; a bandwidth extension decoder for generating audio data in a frequency band at least in part not covered by the decoded portion of the bandwidth-limited audio data based on the decoded portion of the bandwidth-limited audio data and with reference to the bandwidth extension information, and then The generated audio data is added to the decoded portion of the bandwidth-constrained audio data. 21. The apparatus of claim 20, wherein the fine-grained scalable Huffman decoder differentially decodes side information corresponding to the base layer, bit-partitioned decodes a plurality of quantized sample values corresponding to the base layer, and decodes values corresponding to the next enhancement The layer side information is fully decoded up to a number of predetermined layers, and bit-splitting decodes a number of quantized sample values corresponding to the next enhancement layer. 22. The apparatus according to claim 20, wherein the demultiplexer demultiplexes the input audio bitstream in the order of sampling data corresponding to the base layer from the input audio bitstream, sampling bandwidth extension information from the input audio bitstream, and Data corresponding to the remaining enhancement layers are sampled from the input audio bitstream. 23. The apparatus of claim 20, wherein the demultiplexer demultiplexes the input audio bitstream in the following order: sampling bandwidth extension information from the input audio bitstream, sampling data corresponding to the base layer from the input audio bitstream, And data corresponding to the remaining enhancement layers are sampled from the input audio bitstream.

Images