JP2014041240A - Time scaling method, pitch shift method, audio data processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To convert into a high quality sound in a phase continuous processing using a FFT system for achieving time scaling and a pitch shift.SOLUTION: An audio data processing device includes: a FFT section 21 for converting digital audio data into an amplitude and a phase for each frequency component; and a phase continuous processing section 33 which estimates a phase after time expansion using a differential value between a phase obtained by FFT converting the digital audio data again at execution timing different by a time expansion length from an execution timing of the FFT section 21 and the phase obtained by the FFT section 21 as a phase change amount, and performs phase continuous processing.

Description

  The present invention relates to a time scaling method, a pitch shift method, an audio data processing apparatus, and a program for performing time scaling or pitch shift of digital audio data.

  "Time scaling" that extends and compresses the length on the time axis without changing the pitch of the digital audio data, and "Pitch shift" that changes only the pitch of the digital audio data and does not change the length on the time axis As a method for realizing (key control), an FFT (Fast Fourier Transform) method is known. For example, Patent Document 1 describes a method for performing time scaling by changing the number of input samplings and the number of output samplings using an FFT method. Patent Document 2 discloses a method for correcting a start deviation of a transient (battering sound) when the input overlap sampling number and the output overlap sampling number are changed when performing time scaling using the FFT method. Have been described.

  However, when the FFT method is used, in the case of a percussion instrument sound (rhythm sound) with a sharp attack, the attack portion extends in the direction of the time axis, resulting in a deterioration in sound quality in which the sense of attack is lost. For example, when the time scaling is realized by changing the number of input / output samples using the FFT method as described above, the phase is discontinuous with the next FFT operation as it is with the original sound phase. Therefore, in order to prevent the phase from becoming discontinuous, a process for making the phase continuous (hereinafter referred to as “phase continuous process”) is required, and when the pitch shift is performed, the frequency shift is performed in the frequency domain. Therefore, the phase calculated by FFT cannot be made the phase after the frequency shift, and the phase continuation processing is required for each frequency component after the frequency shift, and as a result, the phase is completely different from the original sound. That is, in the conventional FFT method, the phase calculated by the FFT must be converted into a different value after the frequency shift, so that the phase relationship to be maintained in order not to lose the attack between the frequency components is lost. Loss of attack could not be prevented. In order to solve this problem, in Patent Document 3, when an attack sound is detected from the calculation result of the time change rate of the amplitude and / or phase, the phase reset process is performed using the FFT-converted phase itself instead of the phase continuous process. By doing, the feeling of attack is reproduced.

Special table 2007-519967 US Pat. No. 7,565,289 JP 2012-2858 A

However, the conventional FFT method has a problem that sound is deteriorated even in portions other than the attack portion. FIG. 12 is a conceptual diagram of phase continuation processing according to a conventional example. The upper part of the figure shows that 2048 samples of FFT data are input at an input overlap number N (N = (time expansion / contraction rate) × 512). The lower part of the figure shows that 2048 samples of IFFT data are output at intervals of the output overlap number 512 (fixed value). In the figure, “i” indicates the number of FFT operations, and “j” indicates the FFT frequency bin number. The continuous phase calculation formula according to the conventional example is as shown in (Formula B) in the middle of the figure. In (Formula B), “F _s ” indicates the sampling frequency. As shown in the continuous phase calculation formula, in the phase continuous processing according to the conventional example, the true frequency is estimated from the phase change amount, and the phase after time expansion / contraction is calculated from the time expansion / contraction length. For this reason, there are problems such as (1) accumulation of estimation errors, and (2) errors when frequency components are not independent, leading to sound quality degradation.

  In view of the above problems, the present invention provides a time scaling method, a pitch shift method, an audio data processing device, and a program that can be converted into high-quality sound when phase continuous processing is performed using the FFT method. With the goal.

  The time scaling method of the present invention includes a first frequency conversion step for converting digital audio data into amplitude and phase for each frequency component, and digital audio data from the execution timing of the first frequency conversion step by a time expansion / contraction length. Second frequency conversion step for converting the amplitude and phase for each frequency component at different execution timings, the phase obtained in the first frequency conversion step, and the phase difference value obtained in the second frequency conversion step And a phase estimation step of estimating a phase after time expansion and contraction.

  The audio data processing apparatus according to the present invention includes a first frequency conversion unit that converts digital audio data into amplitude and phase for each frequency component, and the digital audio data is subjected to time expansion and contraction length from the execution timing of the first frequency conversion unit. The difference between the phase obtained by the second frequency conversion means, the phase obtained by the first frequency conversion means, and the second frequency conversion means for converting the amplitude and phase for each frequency component at different execution timings Phase estimation means for estimating the phase after time expansion and contraction using the value as a phase change amount.

  According to these configurations, the difference value between the phase obtained in the first frequency conversion step (first frequency conversion means) and the phase obtained in the second frequency conversion step (second frequency conversion means). Conventional phase calculation that calculates the phase change after time expansion by estimating the true frequency from the amount of change in phase and calculating the phase after time expansion / contraction from the phase change amount Compared with processing, there are fewer error factors. For this reason, it is possible to prevent deterioration in sound quality due to phase continuous processing using the FFT method.

  In the time scaling method, the time expansion / contraction length is a length calculated based on a product of a time expansion / contraction ratio and an output overlap number.

  According to this configuration, when the output overlap number is fixed, a multiplication value of the time expansion ratio and the output overlap number can be calculated as the input overlap number. That is, by varying the number of input overlaps, the time expansion / contraction length (time scaling amount) can be varied.

  In the time scaling method described above, the phase for each frequency component is determined according to the processing results of the plurality of phase switching determination processes for performing different phase switching determination using the calculation result of the time change rate of the amplitude and / or phase. Phase reset processing that performs phase reset processing as the calculation result of the frequency conversion step itself, and the phase for each frequency component as a continuous change taking into account time expansion and contraction from the previous calculation result of the first frequency conversion step The phase continuation process is performed, the phase calculation process determination step for determining which phase calculation process is performed, and the phase reset process or the phase continuation process is performed according to the determination result of the phase calculation process determination step. Performing a phase calculation processing step, a first frequency conversion step, a second frequency conversion step, and a phase estimation Step is characterized in that it is executed when performing the phase continuous process.

  According to this configuration, it is possible to detect a steep rise in sound or the like by performing a plurality of phase switching determination processes for performing different phase switching determination using the calculation result of the time change rate of the amplitude and / or phase. Moreover, since appropriate phase calculation processing (either phase reset processing or phase continuation processing) is performed according to the processing results of the plurality of phase switching determination processing, it is possible to prevent a sense of attack from being lost. In other words, when a steep rise in sound is detected from the calculation result of the time change rate of the amplitude and / or phase, the phase reset process is performed using the FFT-converted phase itself, not the phase continuous process. It becomes possible to reproduce the feeling of attack. As a result, not only a long tone sound (melody sound) with a weak attack but also a percussion instrument sound (rhythm sound) with a sharp attack can be subjected to high-quality time scaling using the FFT method.

  In the time scaling method described above, the plurality of phase switching determination processes determine the presence or absence of an attack section for each different frequency band. In the phase calculation processing step, the attack section “present” is determined by the determination of the plurality of phase switching determination processes. ”Is performed, a phase reset process is performed. If it is determined that the attack part is“ none ”, a phase continuation process is performed.

  According to this configuration, since the presence / absence of the attack portion is determined for each different frequency band, the attack portion can be accurately detected.

  In the above time scaling method, each frequency component after the phase calculation process in the phase calculation process step is proportional to the time expansion ratio at the time of the frequency reverse conversion step for converting to digital audio data and the frequency reverse conversion process by the frequency reverse conversion step. Then, a time expansion / contraction calculation step of increasing / decreasing the number of digital audio data after frequency inverse transformation is further performed.

  According to this configuration, the time for extending and compressing the length on the time axis without changing the pitch of the digital audio data by executing the frequency reverse conversion step and the time expansion / contraction calculation step after the phase calculation processing step. Scaling can be realized.

  The pitch shift method of the present invention executes each step in the time scaling method described above, and a sampling rate conversion calculation step for changing the sampling frequency of the digital audio data to perform time expansion / contraction and pitch change. The time expansion / contraction length in each step of the scaling method and the time expansion / contraction length in the sampling rate conversion calculation step are offset, and only the pitch is changed.

  In the pitch shift of the frequency shift method using the conventional FFT in the frequency domain, the phase calculated by the FFT must be converted to a different value after the frequency shift, so that the attack between the frequency components is lost. Therefore, the phase relationship to be maintained is lost, so that the phase reset process cannot be performed correctly and the loss of the sense of attack cannot be prevented. On the other hand, in the configuration using the sampling rate conversion method, since the frequency shift is not performed in the frequency domain, the phase calculated by the FFT can be the phase of the pitch shift converted sound in the attack portion. Loss of attack feeling can be prevented. In addition, since there are few error factors in the frequency shift process, compared with the conventional FFT method that does not use the sampling rate conversion method, it is possible to prevent deterioration in sound quality other than the attack portion, and high-quality pit shift is possible.

  A program according to the present invention causes a computer to execute each step in the time scaling method described above.

  A program according to the present invention causes a computer to execute each step in the above-described pitch shift method.

  By executing these programs, it is possible to realize a time scaling method or a pitch shift method that can be converted into a high-quality sound when phase continuation processing is performed using the FFT method.

1 is a simplified block diagram of a playback apparatus according to a first embodiment and an audio data processing unit that is a part of the playback apparatus. FIG. It is a block diagram of an audio data processing unit according to the first embodiment. It is a flowchart which shows the pitch shift process by an audio data processing part. It is a flowchart which shows the phase calculation process which concerns on 1st Embodiment. It is a conceptual diagram of the phase continuous process which concerns on 1st Embodiment. It is a flowchart which shows the phase calculation process which concerns on 2nd Embodiment. It is a block diagram of the audio data processing part which concerns on 3rd Embodiment. It is a supplementary explanatory view showing the basis of the reference value. It is a supplementary explanatory view showing the basis of the reference value. It is a conceptual diagram of a peak phase maintenance process. It is a flowchart which shows the phase calculation process which concerns on 3rd Embodiment. It is a conceptual diagram of the phase continuation process which concerns on a prior art example and a modification.

  Hereinafter, a time scaling method, a pitch shift method, an audio data processing device, and a program according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the present embodiment, the case where the audio data processing device of the present invention is applied to a playback device such as a CD player will be exemplified.

[First Embodiment]
FIG. 1A is a simplified block diagram of the playback device 1. As shown in FIG. 1, the playback device 1 includes a playback unit 2, an audio data processing unit 3 (audio data processing device), a buffer memory 4, and an audio data output unit 5. The playback unit 2 reads and plays music / musical sound from a device such as a CD. The audio data processing unit 3 is constituted by a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), and buffers digital audio data (hereinafter simply referred to as “audio data”) reproduced by the reproduction unit 2. Digital signal processing is performed on the audio data stored in the memory 4 and read out from the buffer memory 4. The buffer memory 4 includes an input buffer memory 4 (hereinafter referred to as “input buffer 4a”) and an output buffer memory 4 (hereinafter referred to as “output buffer 4b”). The audio data output unit 5 outputs the audio data processed by the audio data processing unit 3 (audio data read from the output buffer 4b) to the outside (such as an output device having an amplifier and a speaker).

  FIG. 1B is a block diagram illustrating an example of the audio data processing unit 3. The audio data processing unit 3 in FIG. 1B includes a time scaling unit 11 as a main functional configuration. The time scaling unit 11 acquires audio data to be processed from the buffer memory 4 (input buffer 4a) and performs time scaling (time expansion / contraction conversion process). In the present embodiment, time scaling is performed using the FFT method.

  On the other hand, FIG. 1C is a block diagram showing another example of the audio data processing unit 3. The audio data processing unit 3 in FIG. 1C includes an SRC unit 12 and a time scaling unit 11 as main functional configurations. That is, the SRC unit 12 is added to the audio data processing unit 3 in FIG.

  The SRC unit 12 performs an SRC process for changing the sampling frequency of the audio data before or after the time scaling by the time scaling unit 11 (sampling rate conversion calculation step). SRC processing is a technique that is originally used to change the sampling period of digital audio data. However, sampling data newly obtained by performing SRC processing is time-expanded and pitched by keeping the sampling frequency as the original. Changes are made. In other words, the audio data processing unit 3 in FIG. 1C cancels the time expansion / contraction length of the audio data by the SRC unit 12 and the time scaling unit 11, so that only the pitch is changed without changing the length on the time axis. The pitch shift which changes can be realized. Hereinafter, a method of mainly performing a pitch shift by the audio data processing unit 3 shown in FIG. 1C will be described.

  FIG. 2 is a block diagram showing a detailed functional configuration of the audio data processing unit 3. As described above, the audio data processing unit 3 includes the SRC unit 12 and the time scaling unit 11. In the present embodiment, it is assumed that the SRC process is first performed and then time scaling is performed. The SRC unit 12 performs SRC processing on the audio data that is the original sound.

  On the other hand, the time scaling unit 11 includes an FFT unit 21, a phase calculation unit 22, an inverse FFT unit 23, and a time expansion / contraction calculation unit 24. The FFT unit 21 converts audio data into amplitude and phase for each frequency component (first frequency conversion step). That is, the sound in the time domain is converted into the frequency domain, and the amplitude and phase are obtained.

  The phase calculation unit 22 performs phase calculation processing according to the calculation result of the time change rate of the amplitude. Specifically, using the normalized amplitude difference value obtained by dividing the time change rate of the amplitude by the amplitude, a plurality of phase switching determination processes for performing different phase switching determination are performed, and a phase calculation process according to the determination result is performed. . The plurality of phase switching determination processes are processes for detecting an attack unit. As shown in the figure, the phase calculation unit 22 includes an attack detection unit 31, a phase reset processing unit 32, and a phase continuation processing unit 33. Further, the attack detection unit 31 includes an all frequency band detection unit 31a, a frequency band detection unit (A) 31b, and a frequency band detection unit (B) 31c.

  The total frequency band detection unit 31a performs a first phase switching determination process that is one of the plurality of phase switching determination processes described above, so that the total value of the normalized amplitude difference values is a predetermined threshold value L1 (hereinafter referred to as “the threshold value”). An attack that requires reset processing for all frequency bands if it is less than the high threshold value in the previous computation and greater than or equal to the high threshold value in the current computation. Part is detected. A specific example is a case where a percussion sound of a bass percussion instrument such as a bass drum is detected. In the case of bass percussion instruments, the attack part contains frequency components from the basic low frequency component that characterizes the pitch of the musical instrument to the considerably higher frequency range above, so the phase can be reset over almost the entire frequency band. Processing is required.

  Further, the frequency band detection unit (A) 31b performs the first phase switching determination process so that the total value of the normalized amplitude difference values is less than the above high threshold and the predetermined threshold L2 (hereinafter referred to as “low threshold”). If it is equal to or greater than (referred to as “threshold”) (where L1> L2), a second phase switching determination process is performed. The second phase switching determination process is a process of binarizing the normalized amplitude difference value for each frequency component with a low threshold and detecting an attack part for each frequency component with a high frequency limitation. Here, it is possible to detect a striking sound (attack) from the mid range to the high range.

  Further, the frequency band detection unit (B) 31c performs the third phase switching determination process when it is determined by the first phase switching determination process that the total value of the normalized amplitude difference values is less than the predetermined threshold L2. I do. The third phase switching determination process is a process of binarizing the normalized amplitude difference value for each frequency component with a high threshold and detecting the presence or absence of an attack portion for each frequency component. Here, it is possible to detect a hitting sound by a vocal or a stringed instrument.

  The phase reset processing unit 32, when an attack unit is detected in the all frequency band detection unit 31a, the frequency band detection unit (A) 31b, and the frequency band detection unit (B) 31c, A phase reset process (hereinafter referred to as “phase reset process”) is performed as the calculation result of the unit 21 itself.

  On the other hand, when the attack part is not detected in the all frequency band detection unit 31a, the frequency band detection unit (A) 31b, and the frequency band detection unit (B) 31c, the phase continuation processing unit 33 determines the frequency component (frequency bin). The phase continuation processing (hereinafter referred to as “phase continuation processing”) is performed on the assumption that each phase has changed continuously in consideration of time expansion and contraction from the previous calculation result of the FFT unit 21. As described above, the phase calculation unit 22 of the present embodiment selectively selects either the phase reset process or the phase continuation process according to the total value of the normalized amplitude difference values and the value for each individual frequency component. To do.

  By the way, the phase continuation processing unit 33 of the present embodiment performs the second FFT at the execution timing different from the FFT execution timing by the FFT unit 21 by the time expansion / contraction length (second frequency conversion step). Then, the difference value between the phase obtained by the FFT unit 21 and the phase obtained by the second FFT is used as a phase change amount, and the phase after time expansion / contraction is estimated (phase estimation step). That is, the phase change amount after time expansion / contraction is acquired directly from the pre-conversion audio data. Details will be described later with reference to FIG.

  The inverse FFT unit 23 converts each frequency component after the phase calculation processing by the phase calculation unit 22 into audio data (frequency reverse conversion step). That is, the amplitude and phase in the frequency domain are converted into sound in the time domain.

  The time expansion / contraction calculation unit 24 increases / decreases the number of data in proportion to the time expansion / contraction rate at the time of frequency inverse conversion processing by the inverse FFT unit 23 (time expansion / contraction calculation step). Specifically, the time expansion / contraction is performed so as to cancel out the time expansion / contraction length of the audio data by the SRC unit 12. The time expansion / contraction method is realized by shifting the time-domain audio data calculated by the inverse FFT unit 23 by the number of samples changed in proportion to the time expansion / contraction rate from the number of samples shifted during the FFT. The audio data after the arithmetic processing by the time expansion / contraction arithmetic unit 24 is output as converted sound.

  In the case of stereo reproduction, in this embodiment, the left and right sounds are processed independently in each unit (SRC unit 12, FFT unit 21, phase calculation unit 22, inverse FFT unit 23, and time expansion / contraction calculation unit 24).

  Next, the flow of the pitch shift process according to the first embodiment will be described with reference to the flowcharts of FIGS. 3 and 4. First, the audio data processing unit 3 performs an initialization process (FFT operation count i = 1, S01), and acquires audio data from the input buffer 4a (S02). Subsequently, SRC processing is performed by the SRC unit 12 (S03), and then time scaling after S04 is started.

  In time scaling, first, the input window function (Hanning window function) is multiplied (S04), and the i-th FFT is performed (S05). Further, the frequency component, that is, the FFT frequency bin number j is set to j = 0 (S06), and the phase amplitude calculation is performed (S07). As described above, S03 to S07 are processing steps by the FFT unit 21.

Subsequently, the audio data processing unit 3 performs phase calculation processing by the phase calculation unit 22 (S08). The phase calculation process will be described later with reference to FIG. When the audio data processing unit 3 finishes the phase calculation process, the amplitude and phase are converted into complex numbers (S09), and whether or not the FFT frequency bin number j has reached half of the FFT sample number n _FFT , that is, “j == It is determined whether or not “n _FFT / 2” has been reached (S10). If the number of FFT samples n has not reached half of the _FFT (S10: No), the FFT frequency bin number j is counted up (S11), and the process returns to S07. If the number of FFT samples n has reached half of the _FFT (S10: Yes), IFFT is performed using the complex complex data of the complex data as complex data of the negative frequency component of the other half (S12). As described above, S09 to S12 are processing steps by the inverse FFT unit 23.

  Subsequently, the audio data processing unit 3 multiplies the output window function (Hanning window function) (S13), multiplies the input overlap number by the time expansion ratio (time stretch ratio) to cancel the SRC ratio, and outputs the result. The pointer is moved (S14). Further, this is written into the output buffer 4b (added to the output buffer 4b, S15) and output as converted sound. As described above, S13 to S15 are processing steps by the time expansion / contraction calculation unit 24. In this embodiment, the output window function is also the same Hanning window as before the FFT, but it is not necessarily the same, and another window function may be selected.

  Thereafter, the audio data processing unit 3 moves the input pointer for the number of input overlaps (S16), and determines whether or not audio data remains in the input buffer 4a (S17). If audio data remains (S17: data present), the FFT operation count i is counted up (S18), and the process returns to S02. If no audio data remains (S17: no data), the pitch shift process ends.

  Next, the phase calculation process corresponding to S08 of FIG. 3 will be described with reference to FIG. The audio data processing unit 3 (phase calculation unit 22) first calculates an amplitude difference (S21) and obtains a normalized amplitude difference value (S22). That is, the normalized amplitude difference value is obtained by further dividing the amplitude time change rate by the amplitude. However, if the amplitude is 0 or very small, division may not be possible or the result of the division may not be appropriate, so the normalized amplitude difference value is also set to 0 as exception processing. Here, whether the total value of the i-th normalized amplitude difference value (indicated as “Σi” in FIG. 4) is a high threshold value or more, a low threshold value or more and less than a high threshold value, or a low threshold value or less. (S23, first phase switching determination process).

  Here, when the total value Σi of the i-th normalized amplitude difference value is equal to or greater than the high threshold (S23: equal to or greater than the high threshold), the total value Σi-1 of the i-1th normalized amplitude difference value is equal to or greater than the high threshold. (S24: No), phase reset processing is performed for all frequency bands (S30). If the sum value Σi-1 of the i-1th normalized amplitude difference value is equal to or higher than the high threshold (S24: Yes), the phase continuation process is performed (S31). That is, the total frequency band detection unit 31a determines that the attack unit is detected when the i-1th calculation binarization is 0 and the i-th calculation binarization is 1, and the phase reset processing unit 32 A phase reset process is performed using the phase for each frequency component as the calculation result itself of the FFT unit 21. If no attack portion is detected, the phase continuation processing unit 33 performs phase continuation processing on the assumption that the phase for each frequency component has changed continuously in consideration of time expansion and contraction from the previous calculation result of the FFT unit 21. Do.

  When the total value of the normalized amplitude difference values is not less than the low threshold value and less than the high threshold value (S23: not less than the low threshold value and less than the high threshold value), the normalized amplitude difference value for each frequency component is binarized with the low threshold value (S25). Further, it is further limited to the high frequency range (S26), it is determined whether or not the frequency-specific reset (A) is necessary (S27, second phase switching determination process). If it is determined that frequency-specific reset (A) is necessary (S27: Yes), phase reset processing is performed for each frequency component (S30), and if frequency-specific reset (A) is determined to be unnecessary (S27). : No), phase continuous processing is performed (S31). That is, the detection unit (A) 31b for each frequency band determines that the attack unit has been detected when the i-1th operation binarization is 0 and the i-th operation binarization is 1, and the phase reset processing unit The phase reset process by 32 is performed. If no attack portion is detected, the phase continuation processing by the phase continuation processing portion 33 is performed.

  Furthermore, when the total value of the normalized amplitude difference values is less than the low threshold value (S23: less than the low threshold value), the normalized amplitude difference value for each frequency component is binarized with the high threshold value (S28), and the frequency-specific reset (B ) Is determined (S29, third phase switching determination process). Here, when it is determined that frequency-specific reset (B) is necessary (S29: Yes), phase reset processing is performed (S30), and when frequency-specific reset (B) is determined to be unnecessary (S29: No), A continuous phase process is performed (S31). That is, the detection unit (B) 31c for each frequency band determines that the attack unit is detected when the i-1st calculation binarization is 0 and the i-th calculation binarization is 1, and the phase reset processing unit When the phase reset process by 32 is performed and an attack part is not detected, the phase continuous process by the phase continuous process part 33 is performed. The “phase calculation processing determination step” in the claims corresponds to S23 to S29, and the “phase calculation processing step” corresponds to S30 and S31.

Next, details of the phase continuation processing will be described. FIG. 5 is a conceptual diagram of phase continuation processing according to the first embodiment. The upper part of the figure shows that 2048 samples of FFT data are input at an input overlap number N (N = (time expansion / contraction rate) × 512). The lower part of the figure shows that 2048 samples of IFFT data are output at intervals of the output overlap number 512 (fixed value). In the figure, “i” indicates the number of FFT operations, and “j” indicates the FFT frequency bin number. Further, the continuous phase calculation formula according to the first embodiment is as shown in (Formula A) in the middle of the figure. In (Expression A), “ _in2 θ _{i, j} ” indicates the input phase calculated by the FFT executed at “i-th FFT2 execution timing” in FIG.

  The “i-th FFT execution timing (current)” in the upper part of the figure indicates the first FFT execution timing by the FFT unit 21. Similarly, the “i-th FFT2 execution timing” in the upper part of the figure indicates the second FFT execution timing by the phase continuation processing unit 33. Note that the second FFT may be performed before the attack detection by the attack detection unit 31 (before the phase calculation process determination step).

On the other hand, FIG. 12 is a conceptual diagram of phase continuation processing according to a conventional example. The continuous phase calculation formula according to the conventional example is as shown in (Formula B) in the middle of the figure. In (Formula B), “F _s ” indicates the sampling frequency.

  As is apparent from a comparison between FIG. 5 and FIG. 12, in the phase continuation processing according to the first embodiment, two FFTs are performed for each frame (as the i-th frequency conversion processing). Th FFT execution timing (current) "only). Further, the continuous phase calculation formula (Formula A) according to the first embodiment is a simple calculation formula as compared with the conventional example (Formula B). As described above, in the conventional example, the true frequency is estimated from the amount of change in the phase and the phase after time expansion / contraction is calculated from the time expansion / contraction length, whereas in the first embodiment, the calculation is not performed. The phase difference value of the two FFT calculation results is used as the phase after time expansion / contraction as it is.

  With reference to FIG. 5, the phase continuation process of this embodiment is further demonstrated. As shown in the upper part of the figure, the “i-th FFT execution timing (current)” is executed with a delay of the number of input overlaps (N) from the “(i−1) -th FFT execution timing” by the FFT unit 21. . The “i-th FFT2 execution timing” is executed with a delay of the number of output overlaps (512) from the “(i−1) -th FFT execution timing” by the FFT unit 21. That is, the “i-th FFT2 execution timing” is executed with a delay of the time expansion / contraction change length (512-N) from the “i-th FFT execution timing (current)”.

  In addition, as a user operation when performing time scaling, for example, when the playback speed can be changed between “0.5 times speed” and “2 times speed” (the time expansion ratio is 0.5 ≦ N / 512 ≦ 2). By changing the number of input overlaps between 256 ≤ N ≤ 1024, and a master tempo from "0.5 times speed" to "2 times speed" (without changing the pitch) To change the playback speed). Accordingly, the interval between the “i-th FFT2 execution timing” and the “i-th FFT execution timing (current)” does not change at 512 samples depending on the time expansion / contraction rate, and the “i-th FFT execution timing (current)” and the previous FFT execution are not changed. The timing “(i−1) -th FFT execution timing” varies.

  As described above, according to the first embodiment, the presence / absence of an attack unit is detected according to the normalized amplitude difference value, and the phase reset process is performed using the FFT calculation result itself according to the detection result. Therefore, it is possible to perform high-quality sound conversion.

  Further, in the phase continuation process executed when no attack portion is detected, the phase change amount is obtained from the phase difference value obtained by two FFTs, and the phase after time expansion / contraction is estimated from the phase change amount. Therefore, there are few error factors compared with the conventional phase calculation processing (see FIG. 12). For this reason, it is possible to prevent deterioration in sound quality due to phase continuous processing using the FFT method. In addition, the time expansion / contraction length of the time scaling unit 11 is calculated based on the time expansion / contraction rate (N / 512) and the multiplication value (N = input overlap number) of the output overlap number (512). By varying the number of laps (N), the time expansion / contraction length (time scaling amount) can be varied.

  Further, since the attack portion is detected using the normalized amplitude difference value obtained by dividing the time change rate of the amplitude by the amplitude, the attack portion can be detected accurately and reliably even when the volume of the original sound is small. Furthermore, since the phase switching determination process is performed three times for each different frequency band using the normalized amplitude difference value, the attack portion can be detected more reliably. For example, when the total value of the normalized amplitude difference values is equal to or higher than the high threshold value, it means that the frequency components to be phase-reset are spread over a wide range. Therefore, the first phase calculation process is performed for all frequency bands. By doing so, it is possible to reliably detect the attack portion. Further, when the total value of the normalized amplitude difference values is less than the high threshold value, the presence / absence of an attack portion is detected for each frequency band, so even a fine attack can be reliably detected.

  In addition, since the SRC unit 12 does not perform frequency shift in the frequency domain by using the sampling rate conversion method, the phase itself calculated by FFT can be used as the phase of the pitch shift converted sound in the attack portion. The loss of attack can be prevented by the reset process. In addition, since there are few error factors in the frequency shift process, compared with the conventional FFT method that does not use the sampling rate conversion method, it is possible to prevent deterioration in sound quality other than the attack portion, and high-quality pit shift is possible.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG. In the first embodiment, the attack unit is detected based on the normalized amplitude difference value obtained from the time change rate of the amplitude. However, in the present embodiment, the attack level is obtained based on the phase fault degree obtained from the time change rate of the phase. It is different in that the attack part is detected. Only differences from the first embodiment will be described below. In the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Moreover, the modification applied about the component similar to 1st Embodiment is applied similarly about this embodiment.

  The audio data processing unit 3 according to the present embodiment has a total frequency band detection unit 31a, a frequency band detection unit (A) 31b, and a frequency band detection unit (B) from the functional configuration of the first embodiment shown in FIG. 31c is omitted (not shown). With this configuration, the phase calculation unit 22 of the present embodiment performs phase calculation processing according to the calculation result of the time change rate of the phase (phase calculation step). Specifically, the attack detection unit 31 determines whether or not a phase fault has occurred using the phase tomographic degree that is the rate of time change of the phase, and performs a phase calculation process according to the determination result. That is, if a phase fault has occurred, it is determined that the attack portion is “present”, and the phase reset processing by the phase reset processing unit 32 is performed. If no phase fault has occurred, it is determined that the attack portion is “no”, and the phase continuation processing by the phase continuation processing unit 33 is performed.

  FIG. 6 is a flowchart showing phase calculation processing according to the second embodiment. Since the main flow of the pitch shift process shown in FIG. 3 is the same as that of the first embodiment, the illustration is omitted. The audio data processing unit 3 (phase calculation unit 22) of the present embodiment first calculates a second-order phase difference (S41) and calculates a phase tomography degree (S42). Here, the presence / absence of a phase fault (presence / absence of an attack portion) is determined according to whether or not the phase fault degree is equal to or greater than a predetermined threshold (S43). In other words, if the phase tomographic degree is equal to or greater than the predetermined threshold (S43: Yes), the phase reset processing unit 32 performs phase reset processing on the phase for each frequency component as the calculation result itself of the FFT unit 21 (S44). . In addition, when the phase slice degree is less than the predetermined threshold (S43: None), the phase for each frequency component is continuously considered by the phase continuation processing unit 33 in consideration of time expansion and contraction from the previous calculation result of the FFT unit 21. As a result of the change, phase continuation processing is performed (S45).

  As described above, according to the present embodiment, the attack portion can be accurately detected regardless of the volume of the original sound by using the time change rate of the phase. Further, since it is only necessary to branch into two processes depending on the presence or absence of a phase slice, high-quality time scaling and pitch shift can be realized with a small amount of calculation processing.

  In the above embodiment, the configuration of the audio data processing unit 3 is different from the functional configuration of the first embodiment shown in FIG. 2 in that the entire frequency band detection unit 31a, the frequency band detection unit (A) 31b, and the frequency band Although the separate detection unit (B) 31c is omitted, a configuration similar to the functional configuration of the first embodiment may be used. In this case, a plurality of phase switching discriminating processes for performing different phase switching discriminating using the phase tomography degree by the all frequency band detecting unit 31a, the frequency band detecting unit (A) 31b and the frequency band detecting unit (B) 31c. I do. Further, it is determined whether or not a phase fault has occurred by the plurality of phase switching determination processes, and a phase calculation process is performed according to the determination result. The flow of the phase calculation process in this example is the same as the phase calculation process of the first embodiment shown in FIG. 4 except that the “amplitude difference” in S21 is changed to “phase difference” and the “normalized amplitude difference value” in S22. Is not shown in FIG.

  Moreover, even if the attack part is detected by combining the attack part detection method using the normalized amplitude difference value of the first embodiment and the attack part detection method using the phase tomography degree of the second embodiment, good. According to this configuration, the detection accuracy of the attack part can be further improved.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to FIGS. In the present embodiment, a peak phase maintaining process is performed as the phase calculation process in addition to the phase reset process and the phase continuation process. The “peak phase maintaining process” refers to a process of maintaining the phase relationship between the spectrum peak of the frequency spectrum and the adjacent frequency band adjacent to the spectrum peak. Hereinafter, a method for determining a spectrum peak to be subjected to the “peak phase maintaining process”, a method for determining whether or not to maintain a phase relationship, and the like will be mainly described. Also in the present embodiment, the same components as those in the above-described embodiments are denoted by the same reference numerals, and detailed description thereof is omitted. Moreover, the modification applied about the component similar to said each embodiment is applied similarly about this embodiment.

  FIG. 7 is a block diagram of the audio data processing unit 3 according to the third embodiment. The audio data processing unit 3 of this embodiment adds a spectrum peak detection unit 34, a phase difference calculation unit 35, a phase determination unit 36, and a peak phase maintenance processing unit 37 to the functional configuration of the first embodiment shown in FIG. It has become the composition.

  The spectrum peak detection unit 34 detects a spectrum peak having a maximum amplitude from the frequency spectrum after FFT conversion by the FFT unit 21 (first frequency conversion step) (spectrum peak detection step). However, in the present embodiment, the spectrum peak (hereinafter referred to as “continuous sound peak”) that is not temporally maximized in amplitude but continued in time is detected. Specifically, the amplitude of the same frequency band according to the previous calculation result of the FFT unit 21 or the adjacent frequency band close to the same frequency band is maximum, and the amplitude according to the current calculation result of the FFT unit 21 is maximum. The spectrum peak is detected as a “continuation tone peak”. Note that the “proximity frequency band” used for continuous sound peak determination is an adjacent frequency band (two bins on both sides).

  The phase difference calculation unit 35 calculates the phase difference between the phase of the continuous sound peak detected by the spectrum peak detection unit 34 and the phase of the adjacent frequency band close to the continuous sound peak (phase difference calculation step).

  The phase determination unit 36 determines whether or not to maintain the phase relationship between the continuous sound peak and the adjacent frequency band according to the phase difference calculated by the phase difference calculation unit 35 (phase determination step). Specifically, when the phase difference is equal to or less than a predetermined threshold value from a reference value determined according to the input window function (see S04 in FIG. 3) applied before FFT by the FFT unit 21, the spectral peak and the adjacent frequency band It is determined that the phase relationship is maintained. Conversely, when the phase difference exceeds a predetermined threshold, it is determined that the phase relationship between the spectrum peak and the adjacent frequency band is not maintained. “When the phase difference exceeds a predetermined threshold” can be considered, for example, when another sound is present in the vicinity of the spectrum peak (when two sounds are close). In this case, since the relationship between the spectrum peak and the adjacent frequency band is low, deterioration in sound quality can be prevented by not maintaining the phase relationship. Note that the “reference value” for determining the phase difference is an antiphase (π) when the input window function is a Hanning window function. This point will be briefly described with reference to FIGS.

  FIGS. 8 and 9 show the frequency sample of the frequency bin “i−1”, the frequency sample of the frequency bin “i”, and the frequency sample of the frequency bin “i + 1” in the upper, middle, and lower stages, respectively. As shown by the dotted elliptical part in FIG. 8, assuming that the phases calculated by the FFT are the same, the phase is reversed at the center position (windowed peak position). Therefore, as shown in FIG. 9, in order to reproduce the windowed state (center maximum), if the phase is calculated so that the phase is matched at the center position, the FFT calculated phase has the bins on both sides of the frequency bin “i”. The phase is reversed (see dotted ellipse). That is, as shown in FIG. 8, when a window function having both ends “0” is used, the phase of the adjacent frequency band is opposite to the spectrum peak (continuous sound peak). For this reason, the phase determination unit 36 determines whether or not to maintain the phase relationship by setting the “reference value” as the opposite phase (π). If a window function whose both ends are not “0” is used as the input window function, a reference value is set according to the window function.

  Returning to the description of FIG. As a result of the determination by the phase determination unit 36, the peak phase maintenance processing unit 37 performs a peak phase maintenance process for the adjacent frequency band when it is determined that “the phase relationship is maintained”. More specifically, the phase continuation process is performed on the continuous sound peak, and the peak phase maintaining process is performed only on the adjacent frequency band. The first embodiment (see FIG. 5) can be applied to the phase estimation method of the continuous phase processing. On the other hand, the phase continuation processing unit 33 of the present embodiment performs the phase continuation processing similarly shown in the first embodiment when it is determined that “the phase relationship is not maintained” as a result of the determination by the phase determination unit 36.

  Here, the phase estimation method of the peak phase maintaining process will be described with reference to FIG. FIG. 10 is a conceptual diagram of the peak phase maintaining process. In the present embodiment, 3 bins (frequency bin “i-3”, frequency bin “i-2”, frequency bin “i−1”, frequency bin “i + 1”) on both sides of the continuous sound peak (frequency bin “i”), The peak phase maintaining process is performed with the frequency bin “i + 2” and the frequency bin “i + 3” in total 6 bins) as the adjacent frequency bands. In the figure, the horizontal direction indicates the time axis (seven squares for one column indicated by the vertical arrow indicate data for one FFT).

FIG. 10 shows that time scaling is performed by extending 3% time so that the state before the time expansion / contraction shown in the upper stage is changed to the state before the time expansion / contraction shown in the lower stage. In addition, a circular hollow arrow written on the horizontal arrow indicating the frequency bin “i” indicates a phase change estimated as a result of performing phase continuation processing only on the continuous sound peak (frequency bin “i”). Is shown. Further, the vertical arrow means that the phase change of the continuous sound peak is reflected on the adjacent frequency band (6 bins on both sides) in the peak phase maintaining process. Therefore, when the bin of j = i is a continuous sound peak, the peak phase calculated from complex data obtained from FFT is θ _i , and the phase after time expansion / contraction of the frequency bin “i” calculated by the phase continuation process is θ ′. When _i, the phase of adjacent frequency bin "i + 1" is calculated by _{"θ'i + 1 = θ'i +} (θ i + 1 -θ i) ". The same calculation is performed for the other five bins. As described above, in the peak phase maintaining process, the phase after time expansion / contraction is estimated in accordance with the phase difference between the previous continuous sound peak and the current phase continuous process.

  Next, a flow of phase calculation processing according to the third embodiment will be described with reference to FIG. Since the main flow of the pitch shift process shown in FIG. 3 is the same as that of the first embodiment, only different parts will be described. The audio data processing unit 3 (phase calculation unit 22) of the present embodiment first sets the FFT frequency bin number j to j = 1 (S51), and determines whether or not it corresponds to a continuous sound peak (S52). That is, it is determined whether or not the amplitude is the maximum at the same frequency bin or the adjacent bin at the previous calculation and the amplitude is the maximum at the current calculation. If it is determined that the peak corresponds to the continuous sound peak (S52: Yes), phase continuation processing is performed (S53).

  When it is determined that the peak does not correspond to the continuous sound peak (S52: No), it is determined whether or not a phase reset is necessary (S54). For this determination, the phase determination method of the first embodiment (see S23 to S29 in FIG. 4) can be applied. When it is determined that the phase reset is necessary (S54: Yes), the phase reset process is performed (S55). If it is determined that the phase reset is unnecessary (S54: No), it is determined whether or not the phase is in the vicinity of the continuous sound peak (S56). That is, it is determined whether or not each side of the continuous sound peak corresponds to a 3-bin adjacent frequency band.

  When it is determined that it does not correspond to the vicinity of the continuous sound peak (S56: No), phase continuation processing is performed (S53). If it is determined that the current position is in the vicinity of the continuous sound peak (S56: Yes), it is determined whether or not the continuous sound peak is a single sound (S57). That is, when the phase difference between the continuous sound peak and the adjacent frequency band is equal to or less than a predetermined threshold value from the reference value (reverse phase), it is determined that the continuous sound peak is a single sound. When it is determined that the continuous sound peak is a single sound (S57: single sound), a peak phase maintaining process is performed (S58). If it is determined that the continuous sound peak is a compound sound (S57: compound sound), phase continuation processing is performed (S53).

Note that after the phase calculation process (any one of S53, S55, and S58), the amplitude and phase are converted into complex numbers (S59), as in the process of the first embodiment shown in FIG. Thereafter, it is determined whether or not the FFT frequency bin number j has reached half of the FFT sample number n _FFT (S60). If not (S60: No), the FFT frequency bin number j is incremented. (S61), the process returns to S52. If it is reached (S60: Yes), the process proceeds to S12 in FIG.

  As described above, according to the present embodiment, when the phase reset process is not performed, according to the phase difference between the spectrum peak and the adjacent frequency band, either the phase continuous process or the peak phase maintaining process is performed. High quality time scaling is possible. That is, when the phase difference is equal to or smaller than the predetermined threshold value from the reference value, it means that the correlation is high. Therefore, the sound can be converted into a high-quality sound by maintaining the phase relationship. On the other hand, when the phase difference exceeds a predetermined threshold value from the reference value, it means that the correlation is low. Therefore, deterioration of sound quality can be prevented by not maintaining the phase relationship. In addition, it is determined whether or not the spectrum peak is continued in time, and only when it is continued in time, the adjacent frequency band is the target of the peak phase maintenance process. On the other hand, deterioration of sound quality due to maintaining the phase difference can be prevented.

  In the third embodiment, as the phase continuation processing, the second frequency conversion is performed by the phase continuation processing unit 33 (although the first frequency conversion is performed by the FFT unit 21), FFT is performed. In addition to the frequency conversion by the unit 21, the phase calculation unit 22 may perform frequency conversion twice.

  In the third embodiment, the same phase estimation method (see FIG. 5) as in the first embodiment is applied to the phase continuation processing, but the conventional phase estimation method (see FIG. 12) is applied. May be. That is, the true frequency may be estimated from the amount of phase change, and the phase after time expansion / contraction may be estimated from the time expansion / contraction length.

  Further, the phase determination method of the first embodiment (see S23 to S29 in FIG. 4) is applied to the determination of whether or not the phase reset is necessary (for the phase switching determination process). A plurality of phase switching determination processes may be performed using the sum of normalized amplitude difference values for left and right sounds. For example, when there is a volume difference between the left and right sounds of a stereo, the phases of the sounds generated from the same sound source will be scattered left and right unless they are reset simultaneously. For this reason, it is possible to synchronize the phase reset timings of the left and right sounds and prevent the disturbance of the sound image (localization) by performing the determination using the summed result.

  In addition, a plurality of phase switching determination processes may be performed using both the sum of normalized amplitude difference values for stereo left and right sounds and the normalized amplitude difference values for stereo left and right sounds. According to this configuration, by using the normalized amplitude difference values of the left and right sounds of the stereo, it is possible to more reliably prevent the disturbance of the sound image in consideration of the left and right volume differences.

  Further, as a modification, even when an attack portion is detected in the phase switching determination process, the phase continuation process may be performed on the continuous component in which the spectrum peak continues in time. According to this configuration, the continuation component in which the spectrum peak continues in time is excluded from the phase reset process, so that it is possible to make it difficult to interrupt the sound that is continuously sounding before and after the attack portion.

  When the total value Σi of the normalized amplitude difference values is equal to or higher than the high threshold and an attack portion is detected (S24: No in FIG. 4), only the low frequency component is delayed by a predetermined time. Phase reset processing may be performed. This is because the low frequency sound has a long period, so the phase is not stable at the timing of phase reset detected in the preprocessing, and the effect of the phase reset process is small, but the effect of the phase reset process is reduced by delaying the timing. This is because it can be increased. As a result, it is possible to improve the sound quality of a low-frequency sound that continues after the percussion sound in a bass percussion instrument, for example, the drumming of a bass drum.

  In each of the above embodiments, the audio data processing unit 3 performs the pitch shift (time scaling) while analyzing the audio data written to the buffer memory 4 along with the reproduction by the reproduction unit 2. These data may be read by reading the analyzed data. That is, it is good also as a structure which performs a pitch shift (time scaling) in real time, reproducing a music, or a structure which pitch-shifts (time scaling) the whole music or a part of music using the data analyzed beforehand. Also good.

  In addition, each component of the audio data processing unit 3 described above can be provided as a program. Further, the program can be provided by being stored in various recording media (CD-ROM, flash memory, etc.). That is, a program for causing a computer to function as each component of the audio data processing unit 3 and a recording medium recording the program are also included in the scope of the right of the present invention.

  Further, in each of the above embodiments, the case where the audio data processing unit 3 is applied to the playback device 1 is exemplified, but the present invention is applied to a DJ device such as a mixer device, various electronic musical instruments, and a computer (PC application, tablet terminal application). It may be applied. Moreover, application to a speech processing device having a function of changing the pitch, such as a karaoke device, a voice changer, and a speech synthesizer, is also useful. For example, by applying the present invention, in a DJ device that continuously plays different music pieces, when the keys of the music pieces to be played continuously are in a dissonant relationship, the harmonics are converted to keys having high affinity by pitch shift. The sound quality of the mix can be improved. In addition, in the karaoke apparatus, there is a function of changing the key according to the voice of the user, but in many cases, the sound source is set to MIDI, which is a driving sound, so that the key can be changed without degrading the sound quality. By applying the invention, it is possible to perform high-quality key conversion even when raw sound is used as a sound source.

  Furthermore, only time scaling can be applied, for example, when only the time axis length of the voice is changed without changing the key. For example, by applying the present invention to a DJ device that continuously plays different music, only the tempo of the music that is played continuously is changed, and the time scaling (master tempo) without changing the key (pitch) is improved. it can. In addition, in a device capable of recording and playing back sound, a fast listening function that does not change the key even when played back at high speed can be improved in sound quality. Other modifications can be made as appropriate without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Playback apparatus 2 ... Playback part 3 ... Audio data processing part 4 ... Buffer memory 4a ... Input buffer 4b ... Output buffer 5 ... Audio data output part 11 ... Time scaling part 12 ... SRC part 21 ... FFT part 22 ... Phase calculation part DESCRIPTION OF SYMBOLS 23 ... Inverse FFT part 24 ... Time expansion-contraction operation part 31 ... Attack detection part 32 ... Phase reset process part 33 ... Phase continuous process part 34 ... Spectral peak detection part 35 ... Phase difference calculation part 36 ... Phase determination part 37 ... Peak phase maintenance Processing part

Claims (9)

A first frequency conversion step for converting digital audio data into amplitude and phase for each frequency component;
A second frequency conversion step of converting the digital audio data into an amplitude and a phase for each frequency component at an execution timing different from the execution timing of the first frequency conversion step by a time expansion / contraction length;
A phase estimation step for estimating a phase after time expansion and contraction, using a phase difference value as a difference value between the phase obtained in the first frequency conversion step and the phase obtained in the second frequency conversion step. A time scaling method characterized in that it is executed.   The time scaling method according to claim 1, wherein the time expansion / contraction length is a length calculated based on a product of a time expansion / contraction ratio and an output overlap number. The phase for each frequency component is calculated in the first frequency conversion step according to the processing results of a plurality of phase switching determination processes for performing different phase switching determination using the calculation result of the time change rate of the amplitude and / or phase. Phase reset processing for performing phase reset processing as the result itself, and phase continuation as the phase for each frequency component has changed continuously in consideration of time expansion and contraction from the previous calculation result of the first frequency conversion step. A phase calculation process determining step for determining which phase calculation process to perform, and which phase calculation process to perform,
Depending on the determination result of the phase calculation process determination step, the phase calculation process step of performing the phase reset process or the phase continuation process,
3. The time scaling method according to claim 1, wherein the first frequency conversion step, the second frequency conversion step, and the phase estimation step are executed when the phase continuation processing is performed. The plurality of phase switching determination processing is for determining the presence or absence of an attack portion for each different frequency band,
In the phase calculation processing step, when it is determined that the attack part is “present” by the determination of the plurality of phase switching determination processes, the phase reset process is performed, and when it is determined that the attack part is “none”, the phase The time scaling method according to claim 3, wherein continuous processing is performed. Frequency inverse conversion step for converting each frequency component after the phase calculation processing by the phase calculation processing step into digital audio data;
The time expansion / contraction calculation step of increasing / decreasing the number of digital audio data after frequency inverse conversion in proportion to the time expansion / contraction ratio is further performed at the time of frequency inverse conversion processing by the frequency inverse conversion step. 5. The time scaling method according to 3 or 4. Each step in the time scaling method according to any one of claims 1 to 5,
Changing the sampling frequency of the digital audio data to perform a sampling rate conversion calculation step for performing time expansion and contraction and pitch change,
The pitch shift method characterized in that the time expansion / contraction length in each step of the time scaling method and the time expansion / contraction length in the sampling rate conversion calculation step are offset, and only the pitch is changed. First frequency conversion means for converting digital audio data into amplitude and phase for each frequency component;
Second frequency conversion means for converting the digital audio data into amplitude and phase for each frequency component at an execution timing that differs from the execution timing of the first frequency conversion means by a time expansion / contraction length;
A phase estimation means for estimating a phase after time expansion and contraction, using a difference value between the phase obtained by the first frequency conversion means and the phase obtained by the second frequency conversion means as a phase change amount; An audio data processing apparatus comprising:   The program for making a computer perform each step in the time scaling method of any one of Claim 1 thru | or 5.   The program for making a computer perform each step in the pitch shift method of Claim 6.

Images