WO2001067451A1

WO2001067451A1 - A method for reconstruction of audio signal

Info

Publication number: WO2001067451A1
Application number: PCT/FI2001/000213
Authority: WO
Inventors: Ismo Kauppinen; Jyrki Kauppinen
Original assignee: Individual
Current assignee: Individual
Priority date: 2000-03-07
Filing date: 2001-03-05
Publication date: 2001-09-13
Anticipated expiration: 2002-09-07
Also published as: FI20000523L; FI20000523A0; AU2001244235A1; FI117147B; EP1277208A1

Abstract

The invention relates to a method for the reconstruction of an area with errors (3, 22) in an audio signal (1, 21), in which method the impulse responses of signals are calculated from the area (23) before the area with errors (3, 22) and the area (24) after the area with errors. By using the impulse response calculated from the area before the area with errors, the signal is extrapolated from the beginning of the area with errors (3, 22) forward (s"), and by using the impulse response calculated from the area after the area with errors, the signal is extrapolated backward (s"") from the end of the area with errors, after which the area with errors (3, 22) is replaced by the linear combination (25) of signals (s", s"") obtained by extrapolation.

Description

A method for reconstruction of audio signal

The invention relates to a method according to the preambles of the independent claims set forth herein for the reconstruction of an audio signal and a computer software product.

The method according to the invention for reconstruction of audio signal is especially suitable for correcting errors of short duration in an audio signal, e.g. correcting errors in records when an LP record is stored in digital form.

A typical problem that occurs when an LP record is stored in digital form is the poor quality of the record as a result of scratches or other errors in it, whereby a mechanical error on the surface of the LP record, for example, causes an abrupt movement of the stylus of the record player.

The simplest way of correcting the error is editing, that is just cutting off the point of error. The empty point is removed by continuing the signal from the starting point of the error directly with a signal after the error. However, this often causes a jump in the signal because of the phase difference, which is detected as a new error in the signal. With this method it is possible to correct only errors including a few data. The erroneous signal can also be corrected by filtering off signals that are either above or below a predetermined frequency range. However, this will also deteriorate the quality of faultless signal areas.

Another known method of correcting errors is copying. In copying, signals are examined on both sides of the error. If the signals on both sides of the error resemble each other, a sequence of the signal of the required length is copied from around the point of error, which is then replaced to the point of error by the copied sequence. Another way of copying is to find elsewhere in the signal such a faultless point, which has a similar start and end as the sequences on both sides of the point of error. The drawback of the copying methods is that they easily create points of discontinuity in the signal, thus causing new errors. [1]

The gap left by the removed point of error can also be patched by interpolation. However, interpolation can only be used for correcting errors including a few data. [2] The patent publication EP 0336685 discloses a method for correcting the point of error in the signal by using an autoregressive model (AR model). In the method disclosed in the publication, the AR model is used over the whole area of data where the error is, whereby the same group of equations includes simultaneously the known data before and after the error data as well as the actual error data. Because the error data and the AR coefficients a_}- are in the same group of equations, they can interact with each other, whereby the solution is instable. The method disclosed in the publication also has the drawback that the solution requires large groups of equations, which makes calculation time-consuming. Another shortcoming of the solution is the fact that it is iterative.

The patent publication EP 0766247 also discloses a signal restoration method using an autoregressive model similar to the specification described above. The solution disclosed in this specification also has the problems of complicated calculation and iteration, whereby calculation is time-consuming and requires high calculation capacity.

A Japanese patent publication JP 63086163 discloses an iterative method based on linear prediction for the restoration of a missing signal. The impuls response of a faultless signal is not used in the method, as a result of which the error contained by the predicted signals increases exponentially the longer the sequence of missing signals is. In the method disclosed in the specification, an attempt was made to remove this error by comparing the last predicted signal to the signal of the first area with errors, whereby it was possible to estimate the size of the error. According to the specification, the signal derived by prediction is accepted when the detected error does not exceed a predetermined threshold value. If the error exceeds the threshold value, the erroneous value is replaced by a value obtained by interpolation.

In addition, S. Montresor et al. disclose a method in their publication [3] wherein impuls noise occurring in gramophone records is corrected by using an autoregressive model. The calculation of the coefficients used in the AR model described in the specification is very complicated, which reduces the usability of the model.

Other methods have also been suggested in the literature related to the art. For example, S.V. Vaseghi and R. Frayling-Cork disclose a method in their publication [4] for eliminating impulsive errors in the signals of LP records. In the method, the impulsive error is treated by creating an error model, which consists of the mean value of a number of similar real errors. The oscillating portion of an impulsive error in the signal can be cancelled by subtraction by means of the impulsive error model. In the case of a sharp peak, subtraction does not help, because the peak has caused irrecoverable damage to the signal. The erroneous data in the area damaged by the peak are corrected by the iterative interpolation method described in the publication [5]. Nevertheless, erroneous data are used in this method for calculating the corrective data.

Other known methods also have the problem that the length of the erroneous area is limited. The known methods can be used to correct an erroneous area, which contains some tens, or at the most a little over one hundred data points. In addition, the known methods have the drawback of the high calculation power required, because methods that include calculation are based on very complicated mathematical models, and they are mainly iterative.

It is the objective of the present invention to eliminate the above mentioned problems and to accomplish a better method for the reconstruction of an audio signal. The method according to the invention is accurate and simple as compared to the known methods.

The method according to the invention for the reconstruction of an audio signal is characterized in what is specified in the characterizing part of the independent claim enclosed herein.

According to the inventive method, the audio signal is typically reconstructed so that the impulse responses of the signals are at first calculated from the faultless area before and after the area with errors. After that, the signal is extrapolated forward from the beginning of the area with errors by using an impulse response calculated from the area before the area with errors, and backward from the end of the area with errors by using an impulse response calculated from the area after the area with errors. The erroneous signal is replaced by a linear combination of the signals obtained by extrapolation. When the signal is extrapolated from both directions of the erroneous area and the erroneous signal is replaced by the linear combination of the signals obtained by extrapolation, the formation of discontinuities at the ends of the area with errors is prevented. In addition, the frequency variations in the erroneous area are naturally changed by sliding due to the weighting function.

In an advantageous method according to the invention, the extrapolated signal is obtained as a convolution of a signal preceding the area with errors and its impulse response.

The method according to the invention applies direct extrapolation, in which a new signal point obtained by the extrapolation of one point is used for the calculation of the next missing signal point. Thus the method according to the invention can be reliably and accurately be used to replace even a long erroneous signal by a signal obtained by extrapolation.

In a preferred embodiment of the invention, the number of impulse response points used varies from 100 to 5000, typically between 500 and 2000 and is advantageously 1000 impulse response points. Thus the extrapolation is based on sufficient information about the strongest frequencies contained by the signal and their amplitude behaviour.

In an advantageous method according to the invention, the method is used for saving LP records in digital form. By means of the method according to the invention, errors in the LP record can be corrected in connection with digital saving in a quick and simple manner. The method according to the invention can also be used to correct errors contained by a speech signal, such as a speech signal transmitted by mobile stations and radios.

The method according to the invention is typically implemented with a computer software product, which can be directly loaded to the central memory of the computer or with a computer software product saved in a computer- readable medium, which computer software product contains program code elements for implementing the method according to the invention, when said computer software product is run in a computer. In this context, a computer software product means independent computer software or a part of computer software, which may comprise one or more program code elements. A program code element means an element that consists of one or more computer-readable instructions. In this context, a computer-readable medium means all mediums in which information, such as instructions, commands, instruction or command sequences or the like, which are readable by a computer, can be saved permanently or temporarily. Mediums like this include e.g. fixed disks, mass storages, cache memories, diskettes and CD-ROM discs.

A typical computer software product as mentioned above, which is saved in a computer-readable medium, comprises at least the following program code elements:

- a program code element for making the computer calculate an impulse response from the area before the area with errors,

- a program code element for making the computer calculate an impulse response from the area after the area with errors, - a program code element for making the computer extrapolate a signal forward from the beginning of the area with errors,

- a program code element for making the computer extrapolate a signal backward from the end of the area with errors,

- a program code element for making the computer calculate a linear combination of the signals obtained by extrapolation, and

- a program code element for making the computer replace the area with errors by the linear combination of the signals obtained by extrapolation, when the computer software product is run in the computer.

By means of the computer software product, the method according to the invention for the reconstruction of an audio signal can be used in a computer or as part of a device, which includes means for saving the computer software product so that by means of it the method can be automated by using automatic data processing.

An advantageous computer software product according to the invention also comprises a program code element for making the computer recognize an area or areas with errors in the signal. The recognition may be based e.g. on a procedure in which the program code element searches for areas of the audio _^signal in which the absolute value of the fourth derivate exceeds the squared mean value of the fourth derivate multiplied by a certain coefficient, and the program code element defines the areas found as areas with errors. By means of a program code element according to the embodiment, the search for erroneous areas can be automated and thus the reconstruction of the audio signal can be made a faster and more accurate process.

The greatest advantage of the method according to the invention is the fact that it enables the reconstruction of erroneous areas of thousands of data points reliably, quickly and accurately.

In addition, the invention has the advantage that the method requires less calculation capacity than many prior art methods, whereby the invention can be implemented with relatively simple means. Thus the manufacturing costs of apparatus for the method do not become unreasonably high.

In the following, the invention will be described in more detail with reference to the accompanying drawings, in which

Figure 1 illustrates the recognition of errors in an audio signal,

Figure 2 is a schematic diagram of correcting an error in an audio signal by linear prediction, in which the erroneus signal is at the top and the corrected signal is at the bottom,

Figure 3 is a schematic diagram of the linear weighting function,

Figure 4 is a schematic diagram of a general weighting function, and

Figs. 5a-5d show examples of the reconstruction of an audio signal by means of a computer sofware product, utilizing the method according to the invention.

Fig. 1 shows as an example the part of the original analogue recording, which is A/D converted in the memory of the computer as the audio signal 1. A mechanical error on the surface of an LP record causes an abrupt movement of the stylus of the record player. In the audio signal 1 this causes a quick change of amplitude values in time. Based on this, in order to locate the errors it is necessary to examine the rate of change per time of the signal data contained by the audio signal 1, or the derivative of the signal with respect to time. In practice, derivative d_j is obtained according to the definition of the derivative by calculating the difference of two consecutive signal data and dividing it by the distance between them

d_j = ^{SjH Sj} (Equation 1)

To be exact, this gives the derivative of the signal between these two data. By applying the derivative according to equation (1) several times in succession, the place of the derivative is shifted each time by half a sampling interval Δt backwards. By taking the absolute value of the derivative of the signal

(Equation 2)

we get a graph, which gives the higher positive value the higher the rate of change of the signal is. By differentiating the derivative graph of the signal further, we get a more distinct peak at the point of the signal where the rate of change is high. It has been proved in practical tests that the fourth derivative is good enough to discriminate even the smallest snaps or other similar deviations heard by the ear from the signal. By applying the derivative equation (1) four times in succession, we get the expression of the fourth derivative

^■ 6s_j +4s_JH -s_J+2 + 4s -_Sj_₂ j (Equation 3)

Δt

which is shifted forward in order to compensate for the shift of the derivative caused by discreteness. By drawing the absolute value signal 2 of the fourth derivate on the computer screen below the audio signal 1, the errors in the audio signal 1 can be located at the accuracy of one data. According to Fig. 1, the error searching method can also be applied so that the method searches for such areas 3 of the audio signal 1, in which the absolute value 2 of the fourth derivative exceeds the squared mean value 4 of the fourth derivative multiplied by a certain coefficient. The coefficient used can be selected by the user. Searching for the areas with errors can be realized by software in the computer or loadable to the main memory of the computer, which performs the above mentioned calculations when the software is run.

According to Fig. 2, the correction of a single error by linear prediction from an audio signal 21 in digital form is done by calculating at first the impulse response h from the faultless area 23 before the area with errors 22 and the impulse response h ' from the faultless area after the area with errors 22. After this, the signal 21 is extrapolated by using the impulse responses calculated above, i.e. the signal 21 is continued by linear prediction from the beginning of the area with errors 22 to the end by using the impulse response h, whereby a signal s ' predicted forward is obtained, and from the end of the area with errors 22 to the beginning by using the impulse response h', whereby a signal predicted backward s" is obtained. The erroneous signal is replaced by the weighted average 25 of the signals s ' and s ' ' predicted forward and backward.

The weighted average of signals s ' and s ' ' predicted from the part 23 before the area with errors 22 and the part 24 after it can be formed by means of several different weighting functions p(t). For the corrective data _j of the area with errors 22, an equation can be written:

S_j = p t)s_j + [l - p(t)]β_j" (Equation 4)

where s is the signal predicted forward, s "_j is the signal predicted backward, and the weighting function p(t) fulfils the condition 0 ≤ p(t) < 1. It would be simple to select a linear weighting function

p(t) = -^- (Equation 5)

where t_a is the starting point of the area with errors and ty is the ending point of the area with errors, as shown in Fig. 3. However, a linear weighting function is not optimal, because the accuracy of the prediction falls exponentially in proportion to the length of the prediction. Another extreme type of weighting function would be a function where the forward predicted signal s would be used only up to the mid-point of the area with errors 22, and after that only the backward predicted signal s' . The weighting function/?^ would then be a step function, which would change from 1 to 0 in the middle of the area with errors. However, this might cause a point of discontinuity in the middle of the area with errors 22, and that would again be a new error in the signal. The most advantageous weighting function found between the above mentioned cases is shown in Fig. 3. The weighting function can be written in a general form

(Equation 6)

where u = (Equation 7) Both extreme cases are obtained by selecting from the equation (6) n = 1, whereby a linear weighting is obtained, or n — ∞, whereby a step function is obtained. For example, if a frequency in the signal changes higher in the middle of the area of errors, the lower frequency is included in the portion predicted forward and the higher frequency in the portion predicted backward. Due to a weighting function where n > 1, the frequency changes in the middle as if naturally sliding.

Figs. 5a-5d show examples of the application of a computer software product in the reconstruction of an audio signal. According to what is shown in Fig. 5a, an error area 51 is detected by the eye in a stereo signal saved from an LP record.

According to what is shown in Fig. 5b, this area is selected for reconstruction by marking it with the cursor. According to what is shown in Fig. 5c, the Burg method is selected as the calculation method for impulse responses, and the lengths of the impulse responses and the amount of faultless data used for the calculation are selected. According to the selections mentioned above, the program calculates the reconstructive signal data and replaces the erroneous data by corrected data. The calculation and replacement operations are performed separately on both channels. Fig. 5d shows a reconstructed stereo signal.

The following mathematical model that will be explained in more detail here, is based on the fact that the signal s(t) is a convolution of itself and its impulse response h(t).

s(t) = h(t)* s(t) =

(Equation 8)

where t is time. When equation (8) is represented in the discrete form

s_J =Δt ∑_Jh_kS_j__k (Equation 9)

*=-_»

where h_k = hβΔt), the sampling frequency Δt = l/(2f_max) and f_max is the maximum frequency occurring in the signal s(t). In practice, a limited amount of signal data S_j, where j = 0, 1, 2, ..., N, is measured from the audio signal. Extrapolating the signal forward means calculating new signal data s_j, where/ = N+l, N+2, ..., from the previous signal values. This is not possible with a non- causal impulse response h_k, where k ^~ - ∞, ... , °°. Instead, this must be done with a causal impulse response h_k, where k - 1, 2, ..., M. Thus the discrete extrapolation equation of one point forward is

M s_j' = Δt^A_fc_-_;__A (Equation 10)

4=1

where s is the forward extrapolated signal data. The extrapolation equation of one point in discrete form backward is

M s = Δt∑ Σh_k's j+ (Equation 11)

where s "_j is the backward extrapolated signal data. By calculating new data by means of equation 10 or 11 and by setting the new data as the known signal data, the following data can be calculated by using equation 10 of 11 again. When the process described above is continued, the signal can be extrapolated without limits. The phase and amplitude information for the forward extrapolated signal comes from the faultless signal before the error, and the frequency and amplitude changing information comes from the impulse response obtained from the area preceding the error. The phase and amplitude information and the frequency and amplitude changing information of the backward extrapolated signal is obtained on the basis of the area after the error.

The impulse responses h_k and h '_k used in the calculation can be calculated for the known signal in many different ways. For example, in the case of a known signal it is possible to set s ) = S_j in the equations 10 and 11. In other words, the known signal S_j is calculated by means of M+ 1 previously known signal data. If there are 2M + 1 known data, it is possible to solve h '_k and h_k from the equations 10 and 11. Extra known data can also be included, whereby the matching of the smallest square sum is used in the calculation of the impulse response of M. The method described above is a so-called matrix method. Another method for the calculation of the impulse response is the Burg method [6, 7], which is in practice very suitable for audio signals because of efficient calculation and a good stability.

Thus the method according to the present invention can be applied to the reconstruction of an area of error as simplified so that if there are, for example, 2N known data s_y- (J = 1, 2, ..., 2N), before the area with errors, then m erroneous data s_y- j = 2N+1, 2N+2, ..., 2N+m), followed by 2N known data s₇- (j = 2N+ +l , 2N+m+2, ..., 4N+m),

N equations are formed from the 2Ndata before the area with errors

N

(Equation 1.1) _τ, = At∑h_kS_j__k j = N+l, N+2,

..., 2N

The impulse response h (k = 1, 2, 3, ..., N) is calculated from the previous equation group (equation 1.1).

In the same way, N equations are formed from the known data after the area with errors

(Equation 2.1) S_j = At∑h_ks_J+k j = 2N+m+l, k=l

2N+m+2, ..., 3N+m h '_k (k = 1, 2, 3, ..., N) is calculated from the previous equation group (equation 2.1).

After that, the erroneous data can be calculated forward by extrapolation, because the impulse responses h_k are known, (Equation 3.1) s) = At^Υ∑h_ks_J__k j = 2N+1, ...,

2N+m so that the extrapolated data S_j are used for calculating new, extrapolated data

(cf. equation 10 shown above).

In the same way, the erroneous data are calculated backward by extrapolation, because the impulse responses h _k are known.

(Equation 4.1) s] = Af∑ti_ks _j+k j = 2N+ ,

2N+W-1, ..., 2N+1 so that the extrapolated data s _j are used for the extrapolation of new data (cf . equation 11 shown above).

Finally, the erroneous data are obtained by a linear combination of the above, as has been stated above in equation 4

S_j = p(t)s_j' + [l - p(t)]s j = 2N+1 , 2N+2, ... , 2N+ The method according to the invention is thus very simple, and the missing erroneous data can be calculated by it by using only five different equations. In addition, the method according to the invention is not iterative. Furthermore, an autoregressive model is not necessarily needed in the method according to the invention, because the data s_j can be interpreted to include the noise as well, for using the impulse response h is a good way of modelling the signals.

The invention is not limited to the embodiment described as an example above, but on the contrary, it should be interpreted widely within the scope defined by the appended claims hereinafter.

References:

[1] L. Backman, Numerot ja aani, osa 2, Hifi 4/91, Helsinki Media Company Oy, 34-36, 1991.

[1] L. Backman, Numerot ja aani, osa 1, Hifi 1/91, Helsinki Media Company Oy, 28-30, 1991.

[3] S. Montresor, J. C. Valiere, M. Baudry, Detection et suppression de bruits impulsionnels appliques a la restauration d'enregistrements anciens, Colloque de Physique, Tome 51, C2, Vol. II, fevrier 1990, France, Premier Congres Franςais d'Acoustique, Lyon, France, 10-13 April 1990.

[4] S. V. Vaseghi, R. Frayling-Cork, Restoration of old gramophone recordings, Journal of the Audio Engineering Society, Volume 40, number i0, 791-801, 1992.

[5] S. V. Vaseghi, P. J. W. Rayner, Detection and suppression of impulsive noise in speech communication systems, Proc, IEE, Volume 137, pt I, 1990.

[6] S. Haykin, Nonlinear methods of spectral analysis, Springer- Verlag, Berlin, 1983.

[7] J. G. Proakis, D. G. Manolakis, "Digital Signal Prosessing", Third Edition, Prentice Hall, New Jersey, 1996.

Claims

1. A method for reconstruction of an area with errors (3, 22) in an audio signal (1, 21), characterized in that - impulse responses of signals are calculated from an area (23) before an area with errors (3, 22) and an area (24) after the area with errors,

- a signal is extrapolated forward (s ') from the beginning of the area with errors (3, 22), using an impulse response calculated from the area before the area with errors, and backward (s ' ') from the end of the area with errors using an impulse response calculated from the area after the area with errors,

- the area with errors (3, 22) is replaced by a linear combination (25) of signals (s', s' ') obtained by extrapolation.

2. A method according to claim 2, characterized in that the extrapolated signal is obtained as a convolution of a signal preceding the area with errors and its impulse response.

3. A method according to claim 1 or 2, characterized in that the method applies direct extrapolation, in which a signal point obtained by the extrapolation of one point is used for the calculation of the next missing signal point.

4. A method according to any one of the preceding claims, characterized in that the number of impulse response points used in the method varies from 100 to 5000, typically between 500 and 2000 and is advantageously 1000 impulse response points.

5. A method according to any one of the preceding claims, characterized in that the method is used for the restoration of signals saved in digital form from LP records.

6. A method according to any one of the preceding claims, characterized in that the method is used for the reconstruction of errors in audio signals, advantageously in radio and mobile station signals.

7. A computer software product, which can be directly loaded to the central memory of a computer, characterized in that the computer software product comprises program code elements for performing at least the phases of claim 1, when said computer software product is run in a computer.

8. A computer software product, which is saved in a computer-readable medium, characterized in that it comprises at least

- a program code element for making the computer calculate an impulse response from the area preceding the area with errors, - a program code element for making the computer calculate an impulse response from the area after the area with errors,

- a program code element for making the computer extrapolate a signal forward from the beginning of the area with errors,

9. A computer software product of claim 8, characterized in that it also comprises a program code element for making the computer recognize an area or areas with errors in the signal.