TWI887845B - Audio codec device, method and computer program products - Google Patents

Abstract

An audio codec method for realistic restoration is shown. According to the method, a plurality of frames of original audio is compressed into compressed data, wherein a first piece of compressed data is further decoded and stored as prefetched feature data, to replace the first piece of compressed data in an audio restoring procedure.

Description

Speech coding and decoding device, method, and computer program product

The present invention relates to audio codec.

A voice frame is usually 320 points. One voice compression method is to use interleave compression technology, where adjacent original frames each provide part of the content, which is combined and compressed to generate compressed data.

Taking the encoding of N frames of original speech as an example, the second half of the first original frame FrameO_1, 160 points, is combined with the first half of the second original frame FrameO_2, 160 points, and compressed into compressed data (corresponding to the second original frame FrameO_2, numbered as the second compressed data Data_2). In particular, the first compressed data Data_1 corresponding to the first original frame FrameO_1 is obtained by combining the 160 points of the starting dummy content (which can be all 0) with the first 160 points of the first original frame FrameO_1. As for the second half of the last original frame FrameO_N, it is combined with the ending dummy content and compressed to form the (N+1)th compressed data Data_N+1.

In this way, N frames of original audio FrameO_1…FrameO_N are compressed into (N+1) pieces of compressed data Data_1…Data_N+1. When decoding, one more frame of content will be restored.

According to an implementation method of the present invention, a speech coding and decoding method is implemented, including: compressing multiple frames of original speech into multiple compressed data, wherein a first compressed data in the multiple compressed data is further decompressed to obtain and save a pre-fetched feature data, and the pre-fetched feature data is used to replace the first compressed data for speech restoration, wherein the first compressed data is generated by compressing a first original frame in the multiple frames of original speech, and the first original frame is the initial frame in the multiple frames of original speech.

In one implementation, the speech encoding and decoding method further includes: decompressing the plurality of frames to restore the speech, wherein the pre-fetched feature data is used to decompress a second compressed data in the plurality of compressed data. The second compressed data is generated by compressing the first original frame and a second original frame. The second original frame is inferior to the first original frame in the plurality of frames of original speech.

In one implementation, the number of frames of the multiple frames of original speech is N, which are numbered from the first original frame to the Nth original frame in time sequence. The multiple compressed data are N+1 records, which are numbered from the first compressed data to the (N+1)th compressed data in time sequence. The (N+1)th compressed data is obtained by compressing the Nth original frame. The number of frames of the multiple frames of restored speech is N, which are numbered from the first restored frame to the Nth restored frame in time sequence. The first restored frame is obtained by decompressing the second compressed data with the pre-fetched feature data. The Nth restored frame is obtained by decompressing the (N+1)th compressed data with an Nth feature data, wherein the Nth feature data is generated when decompressing an Nth compressed data.

The above concepts can be used to implement a computer program product, including program codes, which are executed by a computer to implement the aforementioned speech encoding and decoding method.

The above concept is further used to implement a speech coding and decoding device, including a speech coder and a speech decoder. The speech coder is used to compress a plurality of frames of original speech into a plurality of compressed data. The speech decoder is used to decompress the plurality of compressed data into a plurality of frames to restore the speech. The first frame in the plurality of frames of original speech is a first original frame. The speech coder compresses the first original frame to generate a first compressed data in the plurality of compressed data. The speech encoder further decompresses the first compressed data to obtain and save a pre-fetched feature data, and replaces the first compressed data with the pre-fetched feature data for the speech decoder to perform speech restoration.

In one implementation, the speech decoder decompresses a second compressed data in the plurality of compressed data using the pre-fetched feature data. The second compressed data is generated by the speech encoder compressing the first original frame and a second original frame. The second original frame is next to the first original frame in the plurality of frames of original speech.

The following is a specific example, and together with the attached diagrams, the content of the present invention is described in detail.

100: Voice encoding and decoding device

102: Voice encoder

104: Voice decoder

602: Computer program products

604: Calculator

606:Program code

608:Processor

610: Storage

Data_1…Data_N+1: compressed data

Feature_1: Pre-fetch feature data

Feature_i: Feature data

FrameO_1…FrameO_N: original frame

FrameR_1…FrameR_N: restore frame

S402…S420, S502…S510: Steps

FIG. 1 is a speech encoding and decoding device 100 implemented according to an embodiment of the present invention, including a speech encoder 102 and a speech decoder 104; FIG. 2 illustrates the operation of the speech encoder 102 in a timing diagram; FIG. 3 illustrates the operation of the speech decoder 104 in a timing diagram; FIG. 4 is a flow chart illustrating a speech encoding method according to an embodiment of the present invention; FIG. 5 is a flow chart illustrating a speech decoding method according to an embodiment of the present invention; and FIG. 6 is a block diagram illustrating a computer program product 602 implemented according to an embodiment of the present invention.

The following description lists various embodiments of the present invention to introduce the basic concepts of the present invention and is not intended to limit the content of the present invention. The actual scope of the invention should be defined in accordance with the scope of the patent application. The various functional blocks mentioned below can be implemented by a combination of hardware, software, and firmware, and may also include special circuits. The various functional blocks are not limited to being implemented separately, but can also be combined together to share certain functions.

FIG. 1 shows a speech encoding and decoding device 100 implemented according to an implementation method of the present invention, including a speech encoder 102 and a speech decoder 104. The speech encoder 102 is used to compress multiple frames of original speech (hereinafter referred to as original frames, numbered as the first original frame FrameO_1 to the Nth original frame FrameO_N in time sequence, N is a number) into multiple compressed data (numbered as the first compressed data Data_1 to the (N+1)th compressed data Data_N+1 in time sequence). The speech decoder 104 is used to decompress multiple frames of restored speech (hereinafter referred to as restored frames, numbered as the first restored frame FrameR_1 to the Nth restored frame FrameR_N in time sequence).

In particular, the speech encoder 102 not only performs compression, but also decompresses specific compression results.

The speech encoder 102 first combines the initial dummy content (e.g., fill 0) with the first original frame FrameO_1 and compresses it to generate the first compressed data Data_1. After obtaining the first compressed data Data_1, the speech encoder 102 further decompresses the first compressed data Data_1 to obtain and store a pre-fetched feature data Feature_1. As for the subsequent other original frames, the speech encoder 102 will not decompress them again after compressing them. For example, the second compressed data Data_2 is generated by the speech encoder 102 combining the first original frame FrameO_1 and a second original frame FrameO_2 and compressing them, but the speech encoder 102 does not decompress the second compressed data Data_2.

The pre-fetched feature data Feature_1 is reserved for the speech decoder 104. The speech decoder 104 decompresses the second compressed data Data_2 based on the pre-fetched feature data Feature_1 to obtain the first restored frame FrameR_1.

With this design, the voice decoder 104 can skip decompression of the first compressed data Data_1. Since the original content of the first compressed data Data_1 includes the initial dummy content, the decompression result (hereinafter referred to as FrameR_0) is meaningless. The voice decoder 104 skips decompression of the first compressed data Data_1, i.e. skips the meaningless restoration frame FrameR_0. If it is a sine wave signal, the meaningless restoration frame FrameR_0 will cause the restored sound to have repeated noise (e.g., bop...bop...bop...). In this case, the restored sound will not have the meaningless restoration frame FrameR_0, and the decompression restoration effect is good.

In one implementation, the speech encoder 102 deletes the first compressed data Data_1 after acquiring and saving the pre-fetched feature data Feature_1, so that the speech decoder 104 does not decompress the first compressed data Data_1. The pre-fetched feature data Feature_1 replaces the first compressed data Data_1 and is supplied to the speech decoder 104 for speech restoration.

As shown in the figure, starting from the second compressed data Data_2, the voice decoder 104 generates feature data Feature_i for decompression of each compressed data Data_i (i is an integer greater than 1), which is used for decompression of the next compressed data Data_i+1. The restored frames FrameR_2...FrameR_N are generated accordingly.

Under this architecture, the number of original audio frames is N (FrameO_1~FrameO_N), and the number of restored frames is also N (FrameR_1~FrameR_N), which is consistent with the number of original audio frames.

Figure 2 illustrates the operation of the speech encoder 102 in a timing diagram.

The speech encoder 102 combines the initial dummy content (e.g., 160 points of "0" value) with the first half 160 points of the first original frame FrameO_1, and compresses them into the first compressed data Data_1. The speech encoder 102 further decompresses the first compressed data Data_1, generates and stores the pre-fetched feature data Feature_1. Next, the speech encoder 102 allows two adjacent original frames to provide partial content, each combined, and compressed into compressed data. For example, the speech encoder 102 combines the second half 160 points of the first original frame FrameO_1 with the first half 160 points of the second original frame FrameO_2, and compresses them into the second compressed data Data_2, and the same processing is performed all the way to generate the Nth compressed data Data_N. Finally, the speech encoder 102 combines the second half 160 points of the Nth original frame FrameO_N with the ending dummy content (such as 160 points of "0" value) and compresses them into the (N+1)th compressed data Data_N+1. The diagram clearly shows that a decompression occurs in the encoding stage of this case (decoding the first compressed data Data_1 to generate the pre-fetched feature data Feature_1).

Figure 3 illustrates the operation of the speech decoder 104 in a timing diagram.

The speech decoder 104 first decompresses the second compressed data Data_2 according to the pre-fetched feature data Feature_1 generated in the encoding stage to generate the first restored frame FrameR_1 and the second feature data Feature_2. The second feature data Feature_2 will be used to decompress the next compressed data (Data_3, not shown in the figure). According to this rule, after the Nth compressed data Data_N is decompressed, in addition to generating the (N-1)th restored frame FrameR_N-1, it also generates the Nth feature data Feature_N. The speech decoder 104 decompresses the (N+1)th compressed data Data_N+1 based on the Nth feature data Feature_N to generate the final Nth restored frame FrameR_N.

Figure 4 is a flow chart illustrating a speech coding method according to one implementation method of the present invention.

Step S402 combines the initial virtual content with the first original frame FrameO_1, and compresses it in step S404 to obtain the first compressed data Data_1. Step S406 decompresses the first compressed data Data_1 to obtain the pre-fetched feature data Feature_1 and stores it. Step S408 sets the variable i to 2. Step S410 combines the two adjacent original frames FrameO_i-1 and FrameO_i, and compresses them in step S412 to obtain the i-th compressed data Data_i. Step S414 determines whether the original frame FrameO_i is the last frame. If not, step S416 increments the variable i. If yes, step S418 combines the original frame FrameO_i with the ending dummy content, and compresses it in step S420 to obtain the (N+1)th compressed data Data_N+1, and the process ends.

Figure 5 is a flow chart illustrating a voice decoding method according to one implementation method of this case.

Step S502 decompresses the second compressed data Data_2 based on the pre-fetched feature data Feature_1, generates the first restored frame FrameR_1, and collects the second feature data Feature_2. Step S504 sets the variable i to 2. Step S506 decompresses the (i+1)th compressed data Data_i+1 based on the i-th feature data Feature_i, generates the i-th restored frame FrameR_i, and collects the (i+1)th feature data Feature_i+1. Step S508 determines whether the i+1th compressed data Data_i+1 is the last compressed data. If yes, the process ends. If not, step S510 increments the variable i.

In this case, the voice encoding and decoding can be implemented not only as hardware but also as software.

Figure 6 is a block diagram illustrating a computer program product 602 implemented according to an implementation method of the present invention, which is loaded into a computer 604 as a program code 606 and executed by a processor 608 to implement the aforementioned speech encoding and decoding technology. The memory 610 can temporarily store the data used for the operation, including the original speech frame (FrameO_1~FrameO_N), pre-fetched feature data (Feature_1), compressed data (Data_1~Data_N+1), feature data (Feature_i, i>1), and speech restoration frame (FrameR_1~FrameR_N).

The above embodiments may be implemented in various ways. Any technology that decompresses at the beginning of encoding and uses pre-fetched feature data for decoding may fall within the technical scope of this case.

In other implementations, the coding details may be slightly different from the above. For example, the first original frame FrameO_1 may not be compressed by combining the virtual content at the beginning of half a frame, and there may be other variations; the Nth original frame FrameO_N may not be compressed by combining the virtual content at the end of half a frame, and there may be other variations. Two adjacent frames may not be combined with equal data amounts and then compressed, and there may be other variations.

Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Anyone familiar with this technology can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of the patent application attached hereto.

100: Voice codec device 102: Voice codec 104: Voice decoder Feature_1: Pre-fetched feature data Feature_i: Feature data FrameO_1…FrameO_N: Original frame FrameR_1…FrameR_N: Restored frame Data_1…Data_N+1: Compressed data

Claims (9)

A speech coding and decoding method, comprising: Compressing a plurality of frames of original speech into a plurality of compressed data, wherein the number of frames of the plurality of frames of original speech is N, which are numbered in time sequence as a first original frame and an Nth original frame, and the plurality of compressed data are N+1 pieces, which are numbered in time sequence as a first compressed data to an (N+1)th compressed data, and through a frame interleaving compression technique, the first compressed data is compressed into a plurality of compressed data. The data is generated by combining the initial virtual content and the partial content of the first original frame and recompressing them, the second compressed data to the Nth compressed data are each generated by combining the partial content provided by the adjacent original frames and recompressing them, and the (N+1)th compressed data is generated by combining the partial content of the Nth original frame and the ending virtual content and recompressing them; and After the first compressed data is generated, the first compressed data is further decompressed to obtain the feature data required for decompressing the second compressed data and saved as a pre-fetched feature data to replace the first compressed data for use in voice restoration, so that when the voice restoration decompresses a first restoration frame to an Nth restoration frame numbered in time sequence, the first restoration frame is obtained by decompressing the second compressed data with the pre-fetched feature data. The speech encoding and decoding method of claim 1 further includes: Decompressing the first restored frame to the Nth restored frame, wherein the second compressed data is decompressed with the pre-fetched feature data to generate the first restored frame and a second feature data for decompressing a third compressed data. The speech encoding and decoding method of claim 1 further includes: Replacing the first compressed data with the pre-fetched feature data so that decompression of the first compressed data no longer occurs. The speech encoding and decoding method of claim 1, wherein: The Nth restored frame is obtained by decompressing the (N+1)th compressed data with an Nth feature data, wherein the Nth feature data is generated when decompressing an Nth compressed data. A computer program product, comprising program codes, is executed by a computer to implement the speech encoding and decoding method as described in any one of claims 1 to 4. A speech encoding and decoding device, comprising: A speech encoder, used to compress a plurality of frames of original speech into a plurality of compressed data; and A speech decoder, used to decompress the plurality of compressed data into a plurality of frames to restore the original speech; Wherein: The number of frames of the plurality of frames of original speech is N, which are numbered from a first original frame to an Nth original frame in time sequence, and the plurality of compressed data are N+1 pieces, which are numbered from a first compressed data to an (N+1)th compressed data in time sequence; The speech encoder uses a frame interleaving compression technique, so that the first compressed data is generated by combining the initial virtual content and the partial content of the first original frame and recompressing them, and the second compressed data to the Nth compressed data are respectively generated by combining the partial content provided by the adjacent original frames and recompressing them, and the (N+1)th compressed data is generated by combining the partial content of the Nth original frame and the ending virtual content and recompressing them; After the first compressed data is generated, the speech encoder further decompresses the first compressed data to obtain the feature data required for decompressing the second compressed data, and saves it as a pre-fetched feature data to replace the first compressed data for speech restoration; and When the speech decoder performs speech restoration and decompresses a first restoration frame to an Nth restoration frame numbered in time sequence, the second compressed data is decompressed using the pre-fetched feature data to obtain the first restoration frame. As in claim 6, the speech encoding and decoding device, wherein: When the speech decoder decompresses the second compressed data with the pre-fetched feature data to generate the first restored frame, it also generates a second feature data for decompressing a third compressed data. The speech encoding and decoding device of claim 6 further includes: The speech codec replaces the first compressed data with the pre-fetched feature data so that the speech decoder does not decompress the first compressed data. As in claim 6, the speech encoding and decoding device, wherein: The Nth restored frame is obtained by the speech decoder decompressing the (N+1)th compressed data with an Nth feature data, wherein the Nth feature data is generated when the speech decoder decompresses an Nth compressed data.

Images