CN100382594C - Fast forward playing method for video and audio signal - Google Patents

Abstract

The invention relates to a video signal fast-forward playing method, which is a video signal fast-forward playing method using a time scaling algorithm (timescaling algorithm). The purpose of compressing the video signal is achieved by an interpolation-coefficient algorithm (inter-coefficient) of audio range limitation (angement) and slope calculation (slope calculation), and the quality of the played video signal is improved. The purpose of fast forward playing of the invention is to perform a time scaling operation on each video signal unit in a video signal data stream (data stream), so that a plurality of video signal units are compressed into one video signal unit according to a required compression ratio, and good sound quality can be maintained.

Description

Audio and video signal fast forward playback method

technical field

The invention relates to a video and audio signal fast-forward playback method using a time scaling algorithm, which improves the video and audio signal quality during fast-forward playback through audio range limitation and waveform slope calculation.

Background technique

Generally, when users use audio-visual materials such as CD, VCD, DVD or cassettes, they need to move forward or reverse from time to time when the audio-visual signal is played, especially to listen to or watch a certain multimedia clip repeatedly. Position, and play at a slow speed, there is a need for fast/slow forward or fast/slow reverse. Therefore, the known technologies have developed several solutions to the above requirements, such as sampling frequency and time scaling.

The concept of time scaling in the known technology can be described with reference to FIG. 1 . Among them, there is an input audio-visual signal M, which is divided into multiple windows, such as the first window 11, the second window 12, the third window 13, etc.; this window is the smallest unit in the audio signal, Windowing repeats the action at a predefined compression ratio. As shown in the figure, the audio-visual signal N is output, so that the first window 11 and the second window 12 of the input audio-visual signal M have a repetitive signal position P1, and the second window 12 and the third window 13 have a repetitive signal position P2. The repeated compression steps of adjacent windows achieve the purpose of time compression and fast-forwarding of video and audio signals.

For another embodiment of the time scaling method in the prior art, please refer to FIG. 2A to FIG. 2F , which are schematic diagrams showing the relationship between the sound wave and the time axis. In FIG. 2A there is a minimum wavelength Lmin and a maximum wavelength Lmax. Through a similarity detection step, the minimum wavelength Lmin is increased to the maximum wavelength Lmax to obtain a fundamental period Lp, as shown in FIG. 2B . Then, according to the fundamental period Lp, the original sound wave is divided into a first waveform A and a second waveform B, as shown in FIG. 2C . And it can be seen from FIG. 2C that the first waveform A is a waveform with a falling slope ( FIG. 2D ), and the second waveform B is a waveform with a rising slope ( FIG. 2E ). The mixed waveform (A+B) shown in FIG. 2F is obtained by adding the first waveform A and the second waveform B to replace the original waveforms A and B to obtain the result obtained by the time scaling correction method.

Please refer to FIG. 3A again, which is a schematic diagram of the data flow of the sampling frequency method shown in US Patent No. 6,424,789. This known technology is a method of changing the fast-forward or slow-forward sampling of image data according to the image content, and here only discusses the screening method of the video and audio signal data. The audiovisual signal data stream 30 as shown in the figure includes two groups of audiovisual signal segments (shot): the first audiovisual signal segment 31 and the second audiovisual signal segment 32; and each audiovisual signal segment is further divided into a plurality of audiovisual signal frames ( frame), such as video and audio signal frames F1, F2, F3, F4, . . . If you want to play the audio and video signal data at a slow speed, you need to add or copy additional audio and video signal frames according to the adjacent audio and video signal frames to make the played data stream longer; if you want to play fast, you need to follow a selection principle (This principle will not be described in detail here) The selected video and audio signal frame is deleted to shorten the length of the audio and video signal data stream. As shown in the figure, F2 and F4 in the first audio-visual signal segment 31 are discarded, and F2 and Fm' in the second audio-visual signal segment 32 are discarded, so as to achieve the purpose of fast/slow forward or reverse playback of the audio-visual signal.

FIG. 3B is a flow chart of the method of the known technology shown in FIG. 3A . When the audio-visual signal starts to play (step 301); the audio-visual device receives the audio-visual signal data stream (stream) from the playback source (step 302); the processor in the device, such as a digital signal processor (DSP), distinguishes the data stream into multiple segments (shot) (step 303); then the level of speed change is controlled by the user, such as the speed and quality of fast forward/slow forward (step 304); Classify the signal segments so that they are divided into a plurality of different audio-visual signal frames (step 305); use the appropriate sampling algorithm (sampling algorithm) described in this known technology to discard or duplicate some of the audio-visual signal frames, So as to achieve the purpose of fast forward or slow playback (step 306); then judge whether the last segment has been processed (step 307); ), and continue to classify each segment into different audio-visual signal frames according to the level (step 305); Recombined to become a modified audio-visual signal data stream (step 309 ), the newly modified audio-visual signal data stream is the audio-visual signal data for fast forward or slow playback required by the user.

However, the first embodiment in the above-mentioned known technology utilizes time scaling and compression (time-scalecompress) technology to achieve the purpose of fast-forwarding or reversing playback; The problem of discontinuity of video and audio signal data is caused, so popping sound or noise will appear especially when multi-tone (multi-tone) is played quickly. And the sampling frequency (sampling frequency) method of the second embodiment can cause the phenomenon of frequency conversion, so that the sound played out is an unusual sound, which usually becomes a sharp and high-frequency sound. Therefore, the present invention proposes a fast-forward playback method of video and audio signals developed through time scaling technology to solve the problem of fast forward or reverse in the known technology.

Contents of the invention

The main purpose of the present invention is to solve the known shortcoming of frequency conversion or popping sound when the audio-visual signal is fast-forwarded. Therefore, the present invention proposes a time scaling algorithm for fast-forward playback of audio-visual signals developed from time-scaling technology. The sound quality of fast-forward playback is improved by limiting the audio range and calculating the slope of the waveform.

The present invention provides a method for fast-forward playback of audio-visual signals, which is an inter-coefficient algorithm (inter-coefficient) for fast-forward playback of audio-visual signals developed from a time scaling technology (time scaling), to achieve compression of an audio-visual The purpose of the signal data flow, the step of this audio-visual signal fast-forward playing method comprises: Step 1, a plurality of audio-visual signal units are grouped and stored in at least one buffer memory; Step 2, set a plurality of indexes in this buffer memory ( index); step 3, setting a reference point, which is a reference point (alignment point) of the interpolation coefficient algorithm; step 4, starting the calculation of the interpolation coefficient algorithm by the reference point address, and obtaining a A new compressed audio-visual signal unit; Step 5, shifting the index of one of the buffer memories to the address of the next audio-visual signal to be compressed. And repeat steps 4 to 5 to complete the compression of video and audio signals, so as to achieve the purpose of fast-forward playback.

According to the above idea, wherein, at the beginning of the step, the video and audio signal data stream is divided into the plurality of audio and video signal units.

According to the above idea, wherein, the audio-video signal unit includes a plurality of sampled video-audio signals.

According to the above idea, wherein, in the step of grouping and storing the plurality of video and audio signal units into at least one buffer memory, grouping and storing the plurality of video and audio signal units into the at least one buffer memory is performed according to the required compression ratio or fast-forward playback speed.

According to the above idea, wherein, the step of calculating the reference point is to calculate a reference point from the sampled audio-visual signal of the audio-visual signal unit, use this reference point as the initial point, and then start to calculate point by point from the initial point at the front, and find Get the best reference value as the reference point for the next operation.

The present invention also provides a method for fast-forward playback of audio-visual signals, which is an interpolation coefficient algorithm for fast-forward playback of audio-visual signals developed from a time scaling technique, and is used to compress an audio-visual signal data stream, wherein the The steps of the audio-visual signal fast-forward playback method include: Step 1, dividing the audio-visual signal data stream into a plurality of audio-visual signal units; Step 2, storing the plurality of audio-visual signal units in groups into a first buffer memory and a second buffer memory; Step 3, setting a plurality of index points in the first buffer memory and the second buffer memory; Step 4, setting a reference point, which is a reference point for the interpolation coefficient algorithm; Step 5 , start the calculation of the interpolation coefficient algorithm from the reference point address, and obtain a new compressed audio-visual signal unit; Step 6, and shift the index of one of the buffer memories to the next address of the audio-visual signal that needs to be compressed; repeat the steps Step 5 to step 6 to complete the video signal compression, so as to fast-forward playback.

According to the above idea, wherein, the audio-video signal unit includes a plurality of sampled video-audio signals.

According to the above idea, wherein, in the step of grouping and storing the multiple audio-visual signal units into the first buffer memory and the second buffer memory, they are grouped into the buffer memory according to the required compression ratio or fast-forward playback speed middle.

According to the above idea, wherein, the formula for starting point-by-point calculation of the reference point is:

temp[i]+＝Buffer1[index1+i]×Buffer2[index2+j]

Wherein Buffer1[] is the address function of the first buffer memory, and Buffer2[] is the address function of the second buffer memory, wherein the variable index1+i represents the sampling audio-visual signal address in the audio-visual signal unit of the first buffer memory, and The variable index2+j represents the address of the sampled audio/video signal in the audio/video signal unit of the second buffer memory.

According to the above idea, wherein, the calculation formula of the interpolation coefficient algorithm is:

Buffer1[alignment+i]=(Buffer2[i]×i

+Buffer1[alignment+i]×unit-Buffer1[alignment+i]×i)/unit

Wherein Buffer1[] is the address function of the first buffer memory, Buffer2[] is the address function of the second buffer memory, wherein the variable alignment+i represents the reference point address in the audio-visual signal unit of the first buffer memory, and the variable i represents the address of the starting point in the video and audio signal unit of the second buffer memory.

Description of drawings

FIG. 1 is a schematic diagram of the concept of time scaling in the known technology;

2A to 2F are schematic diagrams showing the relationship between the sound wave and the time axis;

Fig. 3 A shows the schematic diagram of the sampling frequency method data flow of known technology;

Fig. 3B shows the flowchart of the sampling frequency method of known technology;

4A to 4C are schematic diagrams of the time scaling method for fast-forwarding of video and audio signals according to the present invention;

FIG. 5 is a flow chart of the steps of the fast-forward playback method of audio-visual signals according to the present invention.

Wherein, the reference signs are explained as follows:

11 First window opening 12 Second window opening 13 Third window opening

M input audio and video signal N output audio and video signal P1, P2 repeat signal position

30 AV signal data stream 31 First AV signal segment 32 Second AV signal segment

A The first waveform B The second waveform Lmax maximum wavelength

Lmin Minimum Wavelength Lp Basic Period 40 AV signal data flow

41 First buffer memory 42 Second buffer memory

401, 402, 403, 404 audio-visual signal unit

411, 421 memory blocks i401, i402 index points

Detailed ways

The present invention is a fast-forward playback method of audio-visual signals. In order to solve the disadvantages of using time scaling or sampling frequency and other methods when fast-forwarding audio-visual signals in the known technology, which will produce too high or sharp frequency conversion or popping sound during playback, an improved known method is proposed. The time scaling algorithm for fast-forward playback of video and audio signals developed from time scaling technology improves the sound quality of fast-forward playback through the audio range restriction and waveform slope calculation.

Please refer to FIG. 4A to FIG. 4C , which are schematic diagrams of the time scaling method for fast-forwarding audio and video signals according to the present invention.

Figure 4A shows a video and audio signal data stream (stream) 40, which includes a plurality of audio and video signal units 401, 402, 403 and 404, and each audio and video signal unit further includes a plurality of audio and video signal data stream minimum units, namely sampling AV signal (sample). The purpose of fast-forward playback in the present invention is to perform a time-scaling calculation on each audio-visual signal unit, so that multiple audio-visual signal units (units) can be compressed into one audio-visual signal unit according to the required compression ratio, and can maintain a good sound quality. For example, if it is played at double fast-forward, two audio-visual signal units will be compressed into a new audio-visual signal unit; if it is played at four times fast-forward, four audio-visual signal units will be compressed into a new audio-visual signal unit in the same way signal unit.

Please refer to the embodiment of double fast-forward playback shown in FIG. 4B , which includes a first buffer memory 41 and a second buffer memory 42 configured in memory. Store the two audio-visual signal units in the audio-visual signal data stream 40 as a group into the buffer memory in batches, such as the audio-visual signal units 401 and 402 are stored in the memory block 421 of the second buffer memory 42 (such as Buffer1 in formula 1), The video and audio signal units 403 and 404 are stored in the memory block 411 of the first buffer memory 41 (such as Buffer2 in Formula 1). In the embodiment of double fast-forward playback, the data length of the memory block 411 and the memory block 421 is the data length of two audio-visual signal units.

The index address (index) is defined in the buffer memory, such as setting the index point i401 in the first buffer memory 41, setting the index point i402 in the second buffer memory 42, and what the index value marks is the sample video and audio in the video and audio signal unit Signal (sample).

In order to avoid frequency conversion or popping during fast-forward playback, it is necessary to find the reference point of similar waveforms during compression. This reference point is the starting point when the inter-coefficient algorithm starts to operate. See formula one.

temp[i]+＝Buffer1[index1+i]×Buffer2[index2+j] (Formula 1)

In this formula, Buffer1[] is the address function of the first buffer memory 41, and Buffer2[] is the address function of the second buffer memory 42, wherein the variable index1+i represents the sample video signal in the video signal unit of the first buffer memory 41 (sample) address, and the variable index2+j represents the address of the sample video signal (sample) in the video signal unit of the second buffer memory 42 .

In fact, the algorithm for finding interpolation coefficients is to bring the values of the audio-visual signal units 401, 402, 403, and 404 into formula 1 to obtain a most similar figure. Taking FIG. 4B as an example, Buffer1 is provided to the video and audio signal units 401 and 402 , and Buffer2 is provided to the audio and video signal units 403 and 404 . After substituting into the formula, the maximum temp[i] is obtained, which means that point i is the most similar starting point of the two pairs of buffer memories, and we call this point the alignment. After the reference point is obtained, it is then substituted into formula 2 to obtain the value of a segment of audio-visual signal to be replaced.

Buffer 1[alignment+i]＝(Buffer2[i]×i

+Buffer 1[alignment+i]×unit-Buffer1[alignment+i]×i)/unit

(Formula 2) i＝0～unit

Wherein Buffer1[] is the address function of the first buffer memory, Buffer2[] is the address function of the second buffer memory, wherein the variable alignment+i represents the reference point address in the audio-visual signal unit of the first buffer memory, and the variable i represents the address of the starting point in the video and audio signal unit of the second buffer memory.

Since finding interpolation coefficients requires a lot of multiplications, a method of finding similarities by searching slopes and value ranges is proposed here, thereby reducing the complexity of operations. That is, take a point in the audio-visual signal units 403 and 404 as a comparison point, then take a starting index point i401 in 401 and 402, set a numerical difference range (range A), and compare whether it has the same slope from this starting point And whether the value difference is within range A; if not, search backward, up to index point i402. When the optimal reference point is found, we will substitute it into formula 2 to obtain a new segment of video and audio signal value. This method greatly reduces the amount of computation for finding the most similar waveform.

Figure 4C shows that the index point i401 in the first buffer memory 41 is shifted to the next set of audio-visual signal units to be compressed, and the second buffer memory 42 uses the reference point obtained by formula 2 as the starting point of the next compression step The starting point (as shown in the index point i402).

Finally, the result of the second buffer memory is output to obtain the compressed audio-visual signal required by the present invention to achieve the purpose of fast-forward playback of the audio-visual signal.

Shown in Fig. 5 is exactly the step flow chart of audio-visual signal fast-forward playing method of the present invention, and flow process is as follows:

Step S1: divide an audio-visual signal data stream into a plurality of audio-visual signal units (units) according to requirements;

Step S2: store the plurality of audio-visual signal units in groups according to the required compression ratio or fast-forward playback speed into at least one buffer memory (buffer), and the double compression ratio fast-forward playback as described in Figure 4A is two The audio-visual signal unit is used as a reference for group storage, and the buffer memory is the first buffer memory 41 and the second buffer memory 42 shown in FIG. 4B and FIG. 4C;

Step S3: set a plurality of index values (index) in the above-mentioned buffer memory to mark the address of the audio-visual signal therein, as shown in Figure 4B, there is a first index i401 in the first buffer memory 41, and a first index i401 in the second buffer memory 42 have a second index i402;

Step S4: Calculate a reference point through the sampled audio-visual signal (sample) marked by a plurality of audio-visual signal units in each buffer memory, and calculate this reference point to obtain the initial point of the interpolation coefficient algorithm (inter-coefficient);

Step S5: The initial point of the above steps is an alignment point of the interpolation coefficient method, which is calculated point by point from the initial point at the top, and the best alignment value is found by the interpolation coefficient algorithm as the benchmark for the next operation point (such as formula 2), and the first reference point is obtained by the rule of thumb, and then the next reference point is obtained by the interpolation coefficient algorithm;

The sample audio-visual signal (sample) in each audio-visual signal unit is sequentially substituted into formula 2, and the next reference point can be obtained by summing up at last;

Step S6: start the calculation of the interpolation coefficient method from the reference point address, that is, coordinate with each index address, a new compressed audio-visual signal unit can be obtained from the buffer memory, and output the obtained output result of fast-forward playback;

Step S7: judging whether the audio-visual signal compression is completed;

Step S8: If not completed, shift the first index of the first buffer memory to the address of the next audio-visual signal to be compressed (based on the compression ratio determined in step S2), and repeat the above steps S5 to S7 to complete Compress audio and video signals to achieve fast-forward playback;

Step S9: If the audio-video signal compression is completed, then end the steps of the method for fast-forwarding the video-audio signal.

In the above-mentioned steps, a plurality of audio-visual signal units are grouped and stored in the buffer memory according to the required compression ratio or fast-forward playback speed. If two audio-visual signal units are read, a new compressed audio-visual signal unit is output from the buffer memory. That is, fast-forward playback with twice the compression ratio; if every four audio-visual signal units are read and a new compressed audio-visual signal unit is output from the buffer memory, it is four-fold fast-forward playback.

To sum up, the present invention proposes a time scaling algorithm for fast-forward playback of audio-visual signals developed from time scaling technology (time scaling) in order to solve the known shortcomings of frequency conversion or popping sound when audio-visual signals are fast-forwarded. time scaling algorithm), in which the interpolation coefficient algorithm (inter-coefficient) for audio range limitation and waveform slope calculation achieves the purpose of audio and video signal compression, improves the sound quality of fast-forward playback, and can reduce the power consumption during compression calculation , It can reduce the usage rate of memory, and it really has the effect of killing multiple birds with one stone.

The above descriptions are only preferred feasible embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made by using the description of the present invention and the contents of the accompanying drawings are included in the scope of patent protection of the present invention. Inside.

Claims (10)

1. video-audio signal fast-forward play method, this method is developed and a next interpolation coefficient algorithm that is used for the video-audio signal fast-forward play by a time zoom technology, in order to compress a video-audio signal data flow, wherein, the step of this video-audio signal fast-forward play method comprises:

Step 1 deposits a plurality of video-audio signal unit packet at least one buffer storage in;

Step 2 is set a plurality of index points in this buffer storage;

Step 3 is set a reference point, and this reference point is a datum mark of this interpolation coefficient algorithm;

Step 4 begins the calculation of this interpolation coefficient algorithm by this datum mark address, draws a new compression video-audio signal unit;

Step 5 is moved to the next video-audio signal address that needs compression with one of them index point of this buffer storage;

Repeat this step 4 and compress to finish video-audio signal to this step 5, thus fast-forward play.

2. video-audio signal fast-forward play method as claimed in claim 1 is characterized in that, at the beginning of the step of this video-audio signal fast-forward play method begins, this video-audio signal data flow is divided into this a plurality of video-audio signals unit.

3. video-audio signal fast-forward play method as claimed in claim 1 is characterized in that, includes a plurality of sampling video-audio signals in this video-audio signal unit.

4. video-audio signal fast-forward play method as claimed in claim 1, it is characterized in that, these a plurality of video-audio signal unit packet are being deposited in the step of at least one buffer storage, are to divide into groups to deposit in this at least one buffer storage according to required compression ratio or fast-forward play speed.

5. video-audio signal fast-forward play method as claimed in claim 1, it is characterized in that, in described step 3 and step 4, calculate the step of this datum mark, be that sampling video-audio signal by the video-audio signal unit calculates a reference point, as initial point, begin pointwise from foremost initial point then and calculate with this reference point, find out the optimal criteria value, as the datum mark of computing next time.

6. a video-audio signal fast-forward play method is developed and a next interpolation coefficient algorithm that is used for the video-audio signal fast-forward play by a time zoom technology, and in order to compress a video-audio signal data flow, wherein, the step of this video-audio signal fast-forward play method comprises:

Step 1 is divided into a plurality of video-audio signals unit with this video-audio signal data flow;

Step 2 should a plurality of video-audio signal unit packet deposit one first buffer storage and one second buffer storage in;

Step 3 is set a plurality of index points in this first buffer storage and this second buffer storage;

Step 4 is set a reference point, and this reference point is a datum mark of this interpolation coefficient algorithm;

Step 5 begins the calculation of this interpolation coefficient algorithm by this datum mark address, draws a new compression video-audio signal unit; And

Step 6 moves to the next video-audio signal address that needs compression with one of them index bit of this buffer storage;

Repeat this step 5 and compress to finish video-audio signal to this step 6, thus fast-forward play.

7. video-audio signal fast-forward play method as claimed in claim 6 is characterized in that, includes a plurality of sampling video-audio signals in this video-audio signal unit.

8. video-audio signal fast-forward play method as claimed in claim 6, it is characterized in that, these a plurality of video-audio signal unit packet are being deposited in the step of this first buffer storage and this second buffer storage, are to divide into groups to deposit in this buffer storage according to required compression ratio or fast-forward play speed.

9. video-audio signal fast-forward play method as claimed in claim 6 is characterized in that, the formula that this datum mark begins pointwise calculating is:

temp[i]+＝Buffer1[index1+i]×Buffer2[index2+j]

Buffer1[wherein] be the address function of this first buffer storage, Buffer2[] be the address function of this second buffer storage, wherein variable i ndex1+i represents the sampling video-audio signal address in the video-audio signal unit of this first buffer storage, and variable i ndex2+j represents the sampling video-audio signal address in the video-audio signal unit of this second buffer storage.

10. video-audio signal fast-forward play method as claimed in claim 6 is characterized in that the computing formula of this interpolation coefficient algorithm is:

Buffer1[alignment+i]＝(Buffer2[i]×i

+Buffer1[alignment+i]×unit-Buffer1[alignment+i]×i)/unit

Buffer1[wherein] be the address function of this first buffer storage, Buffer2[] be the address function of this second buffer storage, wherein variable alignment+i represents the datum mark address in the video-audio signal unit of this first buffer storage, and variable i represents to be arranged in the starting point address of the video-audio signal unit of this second buffer storage, and unit is the number of above-mentioned a plurality of video-audio signals unit.

Images