JP2008145716A - Voice signal processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To materialize high speed calculation processing of feature quantity of a voice signal which is included in a voice coded stream, by minimizing a decode processing amount of a variable length coded data of the voice coded stream (or, without decoding the variable length coded data). <P>SOLUTION: The voice coded stream which is obtained by subjecting the voice signal to frequency conversion and coding is stored in a buffer memory 11. A synchronization signal detection section 12 detects a synchronization signal in the voice coded stream. An auxiliary information extraction section 13 extracts frame length information expressing length from a detected synchronization signal to a following synchronization signal, and reads auxiliary information to desired auxiliary information (for example, global_gain for L-channel), and a feature quantity calculation section 15 calculates the feature quantity of the voice signal from the desired auxiliary information. A data skip section 14 skips data from the desired auxiliary information to the following synchronization signal. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an audio signal processing apparatus capable of calculating a feature amount of an audio signal related to an audio encoded stream from parameters (auxiliary information) included in the audio encoded stream.

  As for television broadcasting, analog broadcasting ended in 2011 and the transition to digital broadcasting is completed, but audio video equipment has been supported from analog broadcasting to digital broadcasting prior to that. Currently, DVD (Digital Versatile Disc) recorders receive various broadcasts (both analog broadcasts and digital broadcasts) such as BS (Broadcasting Satellite) digital / 110 degree CS (Communication Satellite) digital / terrestrial digital / terrestrial analog broadcasts. The number of models equipped with a tuner that can be used is increasing. Especially for digital broadcasting, an encoded stream can be recorded as it is on an HDD (Hard Disk Drive). In addition, audio signals in BS digital / 110 degree CS digital / terrestrial digital are encoded by MPEG (Moving Picture Experts Group) -2 AAC (Advanced Audio Coding) encoding method, multiplexed into a video encoded stream, and transmitted. Is done.

  In addition to supporting digital broadcasting, this technology captures the feature points of audio and video that are used for chapter-type functions (for example, functions for setting scene switching positions) and CM (Commercial) cut functions. However, in the side-by-side DVD recorder market, it has become important for differentiating from other companies. At that time, accompanying the above-described digital broadcasting, the auxiliary information included in the audio encoded stream is used without performing the process of decoding the audio encoded stream to an audio signal (a process in which high load and delay occur). Therefore, it is required to realize a function for capturing feature points of an audio signal.

On the other hand, in Patent Document 1 below, when an input audio signal having a plurality of contents in a time-division manner is frequency-decomposed and encoded together with a scale factor, the scale factor is extracted as an amplitude, A technique capable of detecting a change point of the content of an input audio signal based on a temporal change is disclosed.
JP2003-29772A (paragraphs 0015, 0016, 0064)

  However, in the MPEG-2 AAC encoding method, information called a scale factor is not used in the sense of a normalized value that fits a frequency signal within a certain range as described in Patent Document 1, It is just a parameter for scaling the frequency signal. Therefore, the scale factor is not closely related to the amplitude value of the audio signal, and there is a problem that even if the scale factor is extracted, this information cannot be used as the feature amount of the audio signal. Also, in the MPEG-2 AAC encoding method, a scale factor is encoded into an audio encoded stream using a variable length code. Therefore, in order to extract a scale factor, many variable length codes are decoded. There is a problem that it is necessary to spend a calculation amount.

  In order to solve the above problem, the present invention minimizes the decoding processing amount of the variable-length encoded data of the audio encoded stream (or avoids decoding the variable-length encoded data) It is an object of the present invention to provide an audio signal processing apparatus capable of realizing high-speed calculation processing of feature amounts of audio signals included in an audio encoded stream.

To achieve the above object, according to the present invention, the frame length information indicating the length of the frame and a plurality of other auxiliary information are sequentially arranged following the synchronization signal indicating the head of the frame. A coded audio stream storing means for storing a coded audio stream constituted by an arrangement of a plurality of the frames in units of the frames each having a header portion constituted by the above and an audio stream portion containing an audio signal; ,
Synchronization signal detection means for identifying the head of the frame by detecting the synchronization signal of the frame in the audio encoded stream stored in the audio encoded stream storage means;
In the audio encoded stream stored in the audio encoded stream storage means, the auxiliary information included in the header portion of the frame is sequentially read from the head of the frame specified by the synchronization signal detection means. An auxiliary information reading means;
Immediately after the auxiliary information set as desired auxiliary information is read by the auxiliary information reading means, the data amount from the synchronization signal of the frame to the desired auxiliary information and the frame length information are obtained. Based on the amount of data included in the frame, data skip means for controlling the start timing of the synchronization signal detection operation of the synchronization signal detection means by skipping the data up to the beginning of the next frame;
Feature quantity calculating means for calculating the feature quantity of the audio signal using the desired auxiliary information;
An audio signal processing apparatus is provided.

  Furthermore, according to the present invention, in addition to the above-described configuration, the audio in which quantization step information at the time of encoding the audio encoded stream is set as the desired auxiliary information read from the audio encoded stream A signal processing apparatus is provided.

  Furthermore, according to the present invention, in addition to the above-described configuration, the quantization step information includes scale factor information indicating an amount of enlargement / reduction at the time of encoding the audio encoded stream, and the desired auxiliary information The scale factor information is set as follows, and the data skip means controls to skip data from the scale factor information first read by the auxiliary information reading means to the beginning of the next frame. An apparatus is provided.

  Furthermore, according to the present invention, in addition to the above-described configuration, the feature amount calculating means is obtained by frequency-converting the sound signal as a feature amount for detecting a change in volume level of the sound signal. There is provided an audio signal processing apparatus configured to calculate an approximate number of bits obtained by taking a logarithm for each frequency spectrum using the auxiliary information read by the auxiliary information reading means.

  The present invention reduces the amount of decoding processing of variable length encoded data of an audio encoded stream (or avoids decoding of variable length encoded data), while Since the auxiliary information is extracted and the feature amount calculation process of the audio signal included in the audio encoded stream is performed based on the extracted auxiliary information, high-speed processing is possible. It is possible to calculate the feature amount of the audio signal in parallel while recording the audio encoded stream.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, an embodiment of the present invention will be described assuming an audio encoded stream encoded using the MPEG-2 AAC encoding method as an example of an audio encoded stream. Such an audio coding system is not limited to the MPEG-2 AAC coding system. The MPEG-2 AAC coding system is an international standard “MPEG-2 / Advanced Audio Coding (ISO / ISO) for audio information compression established by MPEG (Moving Picture Experts Group), which is a working group of ISO (International Organization for Standardization). IEC standard 13818-7) ”. Hereinafter, MPEG-2 AAC may be simply referred to as AAC.

  First, an example of the configuration of an audio signal processing device according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an example of the configuration of an audio signal processing device according to an embodiment of the present invention. The audio signal processing device illustrated in FIG. 1 includes a buffer memory 11, a synchronization signal detection unit 12, an auxiliary information extraction unit 13, a data skip unit 14, and a feature amount calculation unit 15.

  The buffer memory 11 has a function of temporarily storing an audio encoded stream in which an audio signal is frequency-converted and encoded. The audio encoded stream stored in the buffer memory 11 is appropriately read out by the synchronization signal detection unit 12, the auxiliary information extraction unit 13, and the data skip unit 14.

  In addition, the synchronization signal detection unit 12 detects a synchronization signal in the audio encoded stream with respect to the audio encoded stream obtained by encoding the audio signal after frequency conversion, and 1AAU (Audio Access unit: Audio Access Unit) ) To specify the starting point. The synchronization signal detection unit 12 receives the skip completion flag from the data skip unit 14 and starts the synchronization signal detection process. When the start point in the audio encoded stream is found by the synchronization signal detection unit 12, the synchronization completion flag becomes valid and is supplied to the auxiliary information extraction unit 13.

  Further, the auxiliary information extraction unit 13 has a function of specifying AAU from the start point specified by the synchronization signal detection unit 12 and extracting auxiliary information necessary for calculating the feature amount included in 1 AAU. The auxiliary information extraction unit 13 receives the synchronization completion flag from the synchronization signal detection unit 12 and starts the auxiliary information extraction process. When auxiliary information is extracted by the auxiliary information extraction unit 13, the extraction completion flag is validated and supplied to the data skip unit 14.

  The data skip section 14 has a function of skipping the remaining data (1AA remaining data) that has not yet been read and moving the data reading position to the position of the next synchronization signal. The data skip unit 14 receives the extraction completion flag from the auxiliary information extraction unit 13 and starts the data reading position movement process. Further, when the data read position is moved by the data skip unit 14, the skip completion flag is validated and supplied to the synchronization signal detection unit 12.

  Further, the feature amount calculation unit 15 has a function of calculating the feature amount of the audio signal using the auxiliary information extracted by the auxiliary information extraction unit 13 and outputting the calculated feature amount.

  Next, an example of the configuration of an AAC encoded stream that is one of the audio encoded streams will be described with reference to FIG. FIG. 2 is a diagram showing an example of a configuration of an AAC encoded stream used in the audio signal processing device according to the embodiment of the present invention. FIG. 2 shows an example of the configuration of an AAC encoded stream generated when a stereo audio signal is encoded.

  The configuration of the AAC encoded stream illustrated in FIG. 2 assumes an ADTS (Audio Data Transport Stream) format used in digital broadcasting. In FIG. 2, an adts_frame corresponding to 1 AAU is composed of adts_fixed_header, adts_variable_header, adts_error_check, and raw_data_block.

  The adts_fixed_header is used for information whose value does not change between frames, and a synchronization signal (syncword) exists at the head. Further, adts_variable_header is used for information whose value changes between frames, and includes information (frame_length) indicating the frame length of 1 AAU, information (adts_buffer_fullness) indicating transition of the decoder buffer, and the like. Further, adts_error_check is used for a CRC (Cyclic Redundancy Check) error check code.

  The raw_data_block is composed of units called elements. Elements constituting the raw_data_block include an L / R channel CPE (Channel Pair Element), a stuffing byte insertion FILL (Fill Element), and an END (Term Element) indicating the end of AAU. Note that FILL may not exist.

  In the CPE, there is information (common_window) representing a window function common to the L / R channels and information (individual_channel_stream) for each channel. In addition, individual_channel_stream includes information representing window function sequence processing (window_sequence), information representing band limitation (max_sfb), information representing a quantization step (global_gain), information representing a scaling parameter (scale_factor_data), quantization Information indicating data (spectral_data) exists.

  Note that the information (scale_factor_data) indicating the parameters for scaling and the information (spectral_data) indicating the quantized data are variable-length encoded by Huffman codes, and in the order from the beginning of the data, unlike the case of the fixed length. It is necessary to extract information. That is, for example, when variable length encoding is performed in the order of Lch scale_factor_data, Lch spectral_data, Rch scale_factor_data, and Rch spectral_data, in order to extract scale_factor_data of each channel, the Lch scale_factor_data is decoded. After the extraction, it is necessary to decode the Lch spectral_data in order to specify the read position of the Rch scale_factor_data and to decode the Rch scale_factor_data.

  The auxiliary information extraction unit 13 may extract auxiliary information including frame_length and Lch global_gain, and may not extract Rch global_gain. Further, the auxiliary information extracting unit 13 may further extract auxiliary information including Lch scale_factor_data and not extract Rch scale_factor_data. Then, the data skip unit 14 skips the data by a data amount obtained by subtracting the read data amount (for example, the bit number from the syncword to the Lch global_gain) from the value obtained by converting the frame_length into the bit number, It becomes possible to advance the reading position to the head of the next adts_frame. In this way, decoding of variable-length encoded data can be omitted, or the processing amount of data decoding can be minimized.

  The raw_data_block in the above format has an element configuration that differs depending on the format of the audio signal. For example, when elements are described using <>, elements included in an audio encoded stream related to stereo audio processed by the audio signal processing apparatus according to the embodiment of the present invention can be described as follows.

  Stereo audio: <CPE1> <FILL> <TERM> (CPE1 = L / R: L / R channel)

  For example, if there is no FILL for monaural audio or multi-channel audio (for example, 5.1ch), a single channel SCE (Single Channel Element) and a low frequency enhancement channel LFE (LFE (Low Frequency Enhancement) Channel Element) can be described as follows:

Monaural audio: <SCE1><TERM>
(However, SCE1 = C: Center channel)
Multi-channel audio: <SCE1><CPE1><CPE2><LFE><TERM>
(However, SCE1 = C: Center channel, CPE1 = L / R: L / R channel, CPE2 = Ls / Rs: Surround L / R channel, LFE = Low frequency emphasis channel)

  In the present invention, when the auxiliary information included in the element <SCE1> that appears first is acquired, not only stereo sound but also monaural sound and multi-channel sound can be supported.

  Also, the quantization formula and the inverse quantization formula used in the MPEG-2 AAC encoding scheme are expressed as the following formula (1) and formula (2), respectively.

  In the quantization formula (1) and the inverse quantization formula (2), mdct_line (k) indicates a frequency spectrum. Moreover, global_gain is information (quantization step information) indicating the above-described quantization step, and is used as a parameter for changing the quantization step of the entire frequency spectrum in order to keep the number of used bits within a predetermined number of bits. Scalefactor (sfb) is obtained by decoding scale_factor_data so that the frequency signal is scaled so that the quantization error caused by quantization or inverse quantization falls within the allowable range from the psychoacoustic viewpoint. It is used as a parameter. Therefore, even if only scalefactor (sfb) information is extracted, it cannot be said that there is a substantially proportional relationship with the amplitude value of the frequency signal.

  Note that, for example, the feature amount calculation unit 15 calculates the approximate number of bits obtained by taking the logarithm of each frequency spectrum obtained by frequency conversion using the auxiliary information, and decodes the calculated approximate number of bits. It is also possible to output as a feature amount for detecting a change in volume of the sound signal. An example of the calculation formula in this case is shown below.

  First, when the above-described quantization equation (1) is modified, the following equation (3) is obtained.

  The following equation (4) is obtained by calculating how many bits the quantized value is expressed without using Huffman coding.

  On the other hand, num_bit (k) in the above equation (4) takes an absolute value when the input audio signal is 16-bit PCM (Pulse Code Modulation) data, for example. A value of 15 bits from which (1 bit) is removed can be obtained. In addition, when considering that the quantized values are arranged in units of bits in the same way as the audio encoded stream, if the auxiliary information indicating how many bits the quantized values are expressed is not given, the original value It becomes difficult to take out. Therefore, 4 bits are used as auxiliary information for extracting the original quantized value so that values of 0 to 15 can be expressed. As a result, the above num_bit (k) is corrected to be as shown in the following equation (5).

  Then, when total_num_bit which is the sum of num_bit (k) of the frequency spectrum (that is, 1024 spectrum) is obtained from num_bit (k) obtained by the above expression (5), the following expression (6) is obtained. .

  Furthermore, when this total_num_bit is the number of used bits (used_bit), the following formula (7) in which used_bit, which is the number of used bits, is equal to total_num_bit is obtained.

  Then, the above equation (7) can be modified to obtain Σ (log2 (x)) as shown in the following equation (8).

  In the above formula (8), len represents the number of frequency spectra included in the frequency band number (sfb), and the last term is the amount of scalefactor in the entire frequency band. Further, there is a relationship of max_sfb × len = 1024. Further, the above equation (8) shows a calculation formula in the case of using auxiliary information extracted by decoding an encoded bitstream up to scale_factor_data, but when decoding of scale_factor_data is omitted, scalefactor (sfb ) Are all zero, and can be calculated by the following equation (9).

  Σ (log2 (x)) obtained by the above formulas (8) and (9) is an approximate number of bits. However, when the volume level of the audio signal is large, the amplitude level of the frequency spectrum is also large. The number of bits also indicates a large value, and when the volume level is small, the approximate number of bits also indicates a small value. That is, the approximate number of bits indicates a change corresponding to the amplitude level (volume) of the audio signal, and can be used as a feature amount for detecting a change in the volume of the audio signal.

  Also, since MPEG-2 AAC has an encoder buffer (decoder buffer), the length of 1 AAU is not constant even in the case of fixed rate encoding, and a variable length within the range where the encoder buffer does not fail is allowed. Has been. When the audio signal is silent (or close to silence), the amount of information included in the frame is small, and the value of frame_length is set small. That is, when a short frame length is indicated by the value of frame_length, the frame can be regarded as a frame including silence or a sound close to silence, and is used as a feature amount for detecting a silence portion of an audio signal. It is possible to use frame_length.

  Also, with fixed rate coding, many stuffing bytes are spent for bit rate adjustment in the silent portion of the audio signal. Therefore, if there is a sufficient amount of calculation processing and all variable-length codes of the encoded stream can be decoded, it is inserted as a stuffing byte in FILL as a feature quantity useful for detecting a silent portion of an audio signal. It is possible to use the number of bytes that are being used.

  Further, in other auxiliary information, max_sfb can be used as a feature amount representing the frequency bandwidth of the audio signal. Hereinafter, it will be described that max_sfb can be used as a feature amount representing the frequency bandwidth of an audio signal with reference to the relationship between max_sfb and the frequency bandwidth illustrated in FIG. 3.

  FIG. 3 is a diagram illustrating an example of a graph for explaining that max_sfb can be used as a feature amount representing the frequency bandwidth of an audio signal in the audio signal processing device according to the embodiment of the present invention. The vertical axis of the graph in FIG. 3 represents the amplitude level [dB], and the horizontal axis of the graph in FIG. 3 represents the frequency [Hz].

  As shown in FIG. 3, depending on the encoder used for encoding, the frequency bandwidth may be variable according to the set bit rate, and can be used at a low bit rate (for example, 128 kbps). Since the number of bits tends to be insufficient, the bandwidth limit is strengthened (equivalent to max_sfb = 42) to prevent the generation of annoying quantization noise, and at high bit rates (eg 256kbps), the number of usable bits is required. Since this is sufficient, the bandwidth limit is weakened (corresponding to max_sfb = 48) and the encoding is performed as faithfully as possible. max_sfb is a value indicating the band limitation, and therefore, the frequency bandwidth can be obtained in reverse from max_sfb. It is also possible to estimate the set bit rate from max_sfb or the frequency bandwidth, assuming that there is a correlation between the frequency bandwidth and the bit rate, and calculate the decoder buffer transition from adts_buffer_fullness and frame_length. It is also possible to ask. For example, if it is an instantaneous bit rate, it can be easily calculated from the value of frame_length by the following equation (10).

  Further, window_sequence, which is information representing the window function sequence processing, can be used as a feature amount for specifying a location where the stationary sound is switched to the non-stationary sound. FIG. 4 illustrates the relationship between the window_sequence value and the window function that can be used as a feature quantity for specifying the location where the stationary sound is switched to the non-stationary sound in the audio signal processing apparatus according to the embodiment of the present invention. FIG. FIG. 5 is a state transition diagram showing an example of state transition of the window function used in the audio signal processing apparatus according to the embodiment of the present invention.

  The long window (long (value = 0)) in FIG. 4 is a window function used in the steady state, and the start window (start (value = 1)) in FIG. 4 is used at the rise of sound. The window function, the short window in Fig. 4 (short (value = 2)) is the window function used in the transition state from the start of sound to the convergence, and the stop window in Fig. 4 (stop (value = 3)) is the sound. Is a window function used in the convergence of. For example, in the case of returning to silence after a sudden rise of sound from silence, such as when the castanets are struck, long (0) → start (1) → short (2) in the state transition diagram of FIG. The window function state changes in the order of stop (3) → long (0). Therefore, by monitoring the value of window_sequence, it is possible to grasp the tendency of the audio signal.

  The common_window is information representing a window function common to the L / R channels. When content close to monaural audio such as a news program is encoded, the number of bits is reduced using the information included in the common_window. Is done. Therefore, by detecting the change in information of this common_window, it is possible to grasp whether the content includes an audio signal close to stereo audio or an audio signal close to monaural audio. Alternatively, common_window can be used as a feature amount for identifying monaural sound.

  In the above-described embodiment, the description has been made on the assumption that the MPEG-2 AAC encoding method is used. However, MPEG-2 AAC SBR (Spectral Band Replication), MPEG-4 AAC, MPEG-4 AAC SBR, MPEG- 1. It is also possible to apply the present invention to layer 3. Further, each functional block of the audio signal processing apparatus shown in FIG. 1 can be realized by hardware and / or software. Further, the functions of the audio signal processing apparatus in the above-described embodiment may be realized by a computer by a program. This program may be read from a recording medium and loaded into a computer, or may be transmitted via a communication network and loaded into a computer.

  The present invention reduces the amount of decoding processing of variable length encoded data of an audio encoded stream (or avoids decoding of variable length encoded data), while Since the auxiliary information is extracted, and the feature amount calculation process of the audio signal included in the audio encoded stream is performed based on the extracted auxiliary information, it has the effect of enabling high-speed processing. The present invention is applicable to a technique for calculating a feature amount of an audio signal related to an audio encoded stream from parameters (auxiliary information) included in the encoded stream.

It is a block diagram which shows an example of a structure of the audio | voice signal processing apparatus in embodiment of this invention. It is a figure which shows an example of a structure of the AAC encoding stream used with the audio | voice signal processing apparatus in embodiment of this invention. It is a figure which shows an example of the graph for demonstrating that max_sfb can be used as a feature-value showing the frequency bandwidth of an audio | voice signal in the audio | voice signal processing apparatus in embodiment of this invention. FIG. 4 is a diagram for explaining the relationship between a window_sequence value and a window function that can be used as a feature amount for specifying a location where a stationary sound is switched to a non-stationary sound in the audio signal processing device according to the embodiment of the present invention. is there. It is a state transition diagram which shows an example of the state transition of the window function used in the audio | voice signal processing apparatus in embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Buffer memory 12 Synchronization signal detection part 13 Auxiliary information extraction part 14 Data skip part 15 Feature-value calculation part

Claims (4)

It includes a header portion configured by sequentially arranging frame length information indicating the length of the frame and a plurality of other auxiliary information following a synchronization signal indicating the beginning of the frame, and an audio signal An audio encoded stream storage means for storing an audio encoded stream composed of an array of a plurality of the frames, with the frame having an audio stream unit as a unit;
Synchronization signal detection means for identifying the head of the frame by detecting the synchronization signal of the frame in the audio encoded stream stored in the audio encoded stream storage means;
In the audio encoded stream stored in the audio encoded stream storage means, the auxiliary information included in the header portion of the frame is sequentially read from the head of the frame specified by the synchronization signal detection means. An auxiliary information reading means;
Immediately after the auxiliary information set as desired auxiliary information is read by the auxiliary information reading means, the data amount from the synchronization signal of the frame to the desired auxiliary information and the frame length information are obtained. Based on the amount of data included in the frame, data skip means for controlling the start timing of the synchronization signal detection operation of the synchronization signal detection means by skipping the data up to the beginning of the next frame;
Feature quantity calculating means for calculating the feature quantity of the audio signal using the desired auxiliary information;
An audio signal processing apparatus.   The audio signal processing apparatus according to claim 1, wherein quantization step information at the time of encoding the audio encoded stream is set as the desired auxiliary information read from the audio encoded stream.   The quantization step information includes scale factor information indicating an amount of enlargement / reduction at the time of encoding of the audio encoded stream, the scale factor information is set as the desired auxiliary information, and a data skip unit includes: 3. The audio signal processing apparatus according to claim 2, wherein control is performed so as to skip data from the scale factor information first read by the auxiliary information reading means to the head of the next frame.   The feature amount calculating means is a rough bit summed logarithmically for each frequency spectrum obtained by frequency-converting the sound signal as a feature amount for detecting a change in volume level of the sound signal. The audio signal processing device according to claim 1, wherein the number is calculated using the auxiliary information read by the auxiliary information reading unit.

Images