WO2013111976A1 - Graceful degradation-forward error correction method and apparatus for performing same - Google Patents

Description

Progressive degradation forward error correction method and apparatus for performing the same

The present invention relates to an error correction technique, and more particularly, to a progressive degradation forward error correction method that can be applied to a multimedia broadcast multicast service (MBMS) and an apparatus for performing the same.

In multimedia packet communication through a mobile communication network, packet loss may occur due to the characteristics of a mobile communication environment. For example, when a terminal passes through a shadow area such as a building or a tunnel, the terminal may not receive data, resulting in bursty packet loss.

As a countermeasure for such packet loss, a packet retransmission method using an automatic repeat request (ARQ) or a hybrid ARQ (HARQ) may be used. However, the quality of service may be deteriorated due to a delay due to packet retransmission. That is, delay is not a problem for non-real-time services such as the playback of pre-stored video, but for real-time services such as interactive video or video conferencing, the delay has a significant impact on service quality.

In consideration of the above situation, an application layer forward error correction (AL-FEC) method is used as a method for recovering a lost packet without retransmission.

The AL-FEC technique also increases redundancy to increase the coding rate as packet loss increases. Also, in order to increase redundancy, a rateless code such as a fountain code is used to determine a code rate adaptively to a channel at a receiving end, or the state of a channel is determined by a receiving application layer or a cross layer. The concept of layer) is introduced to use the method of controlling AL-FEC redundancy of the transmitter by feeding back from the lower layer to the transmitter.

However, even in the case of applying the AL-FEC technique, packets may be lost in a long period that cannot be recovered through AL-FEC, and in this case, lost packet recovery to provide seamless service with minimal delay I need a way.

SUMMARY OF THE INVENTION An object of the present invention for solving the above problems is to provide a progressive degradation forward error correction method capable of recovering continuous data loss while minimizing delay.

In addition, another object of the present invention is to provide a progressive degradation forward error correction apparatus for performing the progressive degradation forward error correction method described above.

Progressive deterioration forward error correction method according to an aspect of the present invention for achieving the above object of the present invention is to determine a coding unit data group consisting of a predetermined number of time intervals for the data stream continuously input in time Determining a reference data set to be used for generating a parity frame in each time interval of the coding unit data group, generating a parity frame using the determined at least one reference data set, and generating the parity frame And parity frame disposition according to a preset rule, in which at least one time interval among a plurality of time intervals including time intervals constituting the coding unit data group is disposed.

The determining of the reference data set may include comparing time lengths of reference data types having a highest reconstruction priority among preset data types included in each time interval of the coding unit data group, and each time. Determining the data size of the reference data type having the longest time length among the reference data types included in the interval as the size of the reference data set, and reconstructing preset in each time interval based on the determined size of the reference data set. The method may include determining the reference data set including at least one data type according to priority.

The generating of the parity frame may include constructing source data by concatenating the reference data set, and generating the parity frame by performing encoding on the source data.

In the generating of the parity frame by encoding the source data, the parity frame may be generated by performing RaptorQ-like encoding using the source data as an input.

In the generating of the parity frame, the parity frame may be generated according to a preset period configured in units of the reference data set.

The parity frame arrangement may include disposing the parity frame in a time interval before time intervals constituting the coding unit data group or at least one of time intervals constituting the coding unit data group according to a preset rule. It can be placed in the time interval of. The parity frame arrangement may include dividing the parity frame to generate two or more divided parity frames each having a specific length, and transferring the two or more divided parity frames to the time intervals constituting the coding unit data group. It may be disposed in time intervals of two or more time intervals of the time intervals constituting the coding unit data group.

In the generating of the divided parity frame, the size of the divided parity frame may be determined based on the number of reference data sets to be restored among the plurality of reference data sets and the size of each reference data set.

The determining of the coding unit data group may include determining a group of pictures (GoP) for each time interval having a preset length for pictures continuously input according to time, and in advance. The method may include determining the coding unit data group including the set number of picture groups.

The determining of the reference data set may include comparing the lengths of I pictures among picture types included in each of the plurality of picture groups constituting the coding unit data group, and including the pictures in the plurality of picture groups. Determining the length of the I picture having the longest length of the I picture, which is the length of the reference data set, and at least one picture according to a preset reconstruction priority in each of the picture groups based on the size of the reference data set determined. Determining the reference data set comprising a type.

In addition, the progressive degradation forward error correction method according to another aspect of the present invention for achieving the object of the present invention is a progressive degradation forward error correction method performed in the transmission device for transmitting the multimedia data, in advance with respect to the received data stream Determining a coding unit data group including a set number of time intervals, determining a reference data set to be used for generation of a parity frame in each time interval included in the determined coding unit data group, and determining the at least one reference criterion Generating a parity frame using a data set and transmitting the generated parity frame before transmitting data included in the preset coding unit data group.

The generating of the parity frame may include constructing source data by concatenating the reference data set, and generating the parity frame by performing RaptorQ-like encoding on the source data.

Here, in the transmitting of the generated parity frame before transmitting data included in the preset coding unit data group, the generated parity frame may be transmitted during a time interval in which the received data stream is buffered.

Here, the determining of the coding unit data group consisting of a predetermined number of time intervals for the received packet stream may include determining a time interval of the coding unit data group in the same time interval as the file division processing unit time interval of a specific protocol. You can decide.

Here, the determining of the coding unit data group consisting of a predetermined number of time intervals for the received packet stream may be performed at the same time interval as the file segment duration of the File Delivery over Unidirectional Transport (FLUTE) protocol. A time section of the coding unit data group may be determined.

Here, the progressive degradation forward error correction method may further include performing interleaving on a segment packet unit basis of a segment file included in a file segment section of the FLUTE protocol.

In addition, the progressive degradation forward error correction apparatus according to an aspect of the present invention for achieving another object of the present invention is a first encoding processing layer for performing a progressive degradation forward error correction processing on the input multimedia data, and progressive degradation forward A transport protocol processing layer that performs transport protocol processing on the data on which the error correction processing has been performed, and a second encoding processing layer that performs forward error correction processing of the application layer on the data on which the transport protocol processing has been performed.

Here, the first encoding processing layer may perform the progressive degradation forward error correction processing by classifying the type of the input multimedia data.

According to the progressive degradation forward error correction method described above and the apparatus for performing the same, the lost packet can be recovered even when a long packet of continuous intervals that cannot be recovered by the AL-FEC is lost, thereby allowing seamless Can provide services.

In addition, encoding gain can be improved by minimizing the size of redundant data added to recover lost data, thereby improving transmission efficiency.

In addition, the loss compensation efficiency can be improved by minimizing the delay time due to the progressive degradation forward error correction encoding and decoding, and the quality of service can be guaranteed.

1 is a conceptual diagram illustrating a first type GD-FEC method according to an embodiment of the present invention.

2 is a conceptual diagram illustrating a second type GD-FEC method according to an embodiment of the present invention.

3 is a conceptual diagram illustrating a third type GD-FEC method according to an embodiment of the present invention.

FIG. 4 shows a frame configuration timing diagram of the third type GD-FEC method shown in FIG. 3 in more detail.

5 is a conceptual diagram illustrating a fourth type GD-FEC method according to an embodiment of the present invention.

FIG. 6 shows in more detail a frame configuration timing diagram of the fourth type GD-FEC method shown in FIG. 5.

7 shows a header configuration of a parity frame used in a progressive degradation forward error correction method according to an embodiment of the present invention.

8 is a conceptual diagram illustrating a processing procedure of a progressive degradation forward error correction method according to an embodiment of the present invention.

9 shows a position in a 3GPP transport protocol structure that includes a forward error correction method according to an embodiment of the present invention.

10 illustrates an AL-FEC processing procedure processed after a gradual degradation forward error correction method is performed according to an embodiment of the present invention.

11 illustrates a method for minimizing a decoding delay of a progressive degradation forward error correction method according to an embodiment of the present invention.

12 illustrates a method for minimizing a decoding delay of a progressive degradation forward error correction method according to another embodiment of the present invention.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In the following description of the present invention, the same reference numerals are used for the same elements in the drawings and redundant descriptions of the same elements will be omitted.

In the progressive degradation forward error correction method according to an embodiment of the present invention, a method for generating an optimized parity frame for solving bursted packet loss and dispersing the effect of delay inevitably caused to not affect the service As a combination of a progressive degradation forward error correction technique and a payload formatting technique, the coding gain and packet loss compensation efficiency are improved.

In order to facilitate the description of the configuration and operation of the present invention, each of various types of progressive degradation forward error correction (hereinafter, referred to as 'GD-FEC') method according to an embodiment of the present invention. The form, timing diagram, encoding method, parity bit generation method, frame configuration and distribution method, and parity frame arrangement method in particular considering transmission delay and reception delay will be described.

In the following description of various types of GD-FEC methods described below, video data has been described by way of example. However, the GD-FEC method according to an embodiment of the present invention is not applied only to video data. That is, the GD-FEC method according to an embodiment of the present invention can be applied not only to video data but also to multimedia data such as audio, dynamic and interactive multimedia scenes (DIMS), and text.

First type GD-FEC

FIG. 1 is a conceptual diagram illustrating a first type GD-FEC method according to an embodiment of the present invention, and illustrates a timing diagram, encoding, parity frame configuration, distribution, and placement method of the first type GD-FEC method.

라 하고,

의 원소의 개수를 L이라 하면, L개의 GoP 별로 버스트 패킷 손실 발생시 복구하고자 하는 I 프레임들의 길이(또는 데이터 크기)를 비교하여 가장 긴 프레임을 선정하고, 선정된 I 프레임의 크기를 마크로 프레임(Macro frame, Mfr)의 크기로 정한다. 여기서, GoP는 복수의 픽처(또는, 프레임)로 구성된 그룹을 의미하며, 본 발명의 실시예들에서는 GoP가 4개의 프레임 시간(frame time)으로 구성된 것으로 가정하였다. I 프레임은 전후의 픽처(또는 프레임)와는 상관없이 하나의 픽처 내 정보만을 이용하여 부호화된 프레임을 의미한다. 또한, 마크로 프레임은 GD-FEC의 패리티 프레임을 생성시키기 위해 포함되는 비트들을 구성하는 소정 크기를 가지는 프레임 그룹을 의미한다.Referring to FIG. 1, a group of GoPs (Groups of Pictures) participating in one GD-FEC encoding is shown.

,

If the number of elements is L, the longest frame is selected by comparing the length (or data size) of I frames to be recovered when burst packet loss occurs for each L GoPs, and the size of the selected I frame is determined as a macro frame (Macro). frame, Mfr). Here, GoP means a group consisting of a plurality of pictures (or frames), and in the embodiments of the present invention, it is assumed that GoP is composed of four frame times. An I frame refers to a frame encoded using only information in one picture regardless of the preceding or following pictures (or frames). In addition, a macro frame refers to a frame group having a predetermined size constituting bits included to generate a parity frame of the GD-FEC.

를 구성하는 원소들인 각각의 GoP에서, I 프레임을 우선적으로 선정한 후 미리 설정된 우선 순위에 따라 보호하고자 하는 프레임 순서대로 마크로 프레임(Mfr)의 길이에 맞추어 각 GoP에 해당하는 마크로 프레임(Mfr)을 구성한다. 여기서, 우선 순위는 버스트 패킷 손실이 발생하는 경우 복원 우선 순위를 의미할 수 있고, 예를 들어, I, P, B 프레임 순으로 우선 순위가 설정될 수 있다. P 프레임은 픽처간의 순방향 예측 부호화를 통해 얻어지는 픽처를 의미하고, B 픽처는 픽처간 양방향 예측 부호화를 통해 얻어지는 픽처를 의미한다.Meanwhile,

In each GoP, which is the elements constituting, the I frame is first selected, and then the macro frame Mfr corresponding to each GoP is constructed according to the length of the macro frame Mfr in the order of frames to be protected according to a preset priority. do. Here, the priority may mean a restoration priority when burst packet loss occurs. For example, the priority may be set in the order of I, P, and B frames. The P frame refers to a picture obtained through forward prediction encoding between pictures, and the B picture refers to a picture obtained through bidirectional prediction coding between pictures.

Then, after concatenating the macro frames to form a source symbol, GD-FEC encoding is performed on the configured source symbols to obtain a parity frame (Pfr) as a result of the encoding.

In the first type GD-FEC, a parity frame generation procedure may be expressed by Equation 1 below.

Equation 1

내의 마크로 프레임(Mfr)들을 연접하여 구성한 소스 심볼을 RaptorQ-like 인코딩의 입력으로 하여 획득한 출력을

내의 각 GoP에 삽입하는 패리티 프레임(Pfr)을 생성하는 함수를 의미한다. 또한, F는 RaptorQ-like 인코딩 함수를 의미하고, i는 프레임 시간(frame time)을 의미한다. 패리티 프레임(Pfr)의 길이는 후술할 수학 정리 1 및 따름 정리 1에 의해 결정된다.In Equation 1, the operator '<~'

Output obtained by inputting RaptorQ-like encoding the source symbol formed by concatenating macro frames (Mfr) in the

It means a function to generate a parity frame (Pfr) to be inserted into each GoP. In addition, F means RaptorQ-like encoding function, i means frame time (frame time). The length of the parity frame Pfr is determined by the mathematical theorem 1 and the following theorem 1 to be described later.

의 각 해당 마크로 프레임을 연접하여 함수 F를 통해 발생되는 패리티 프레임(Pfr)의 생성 주기를 의미하고, 도 1에 도시한 바와 같이 G=1인 경우 매 GoP 마다 패리티 프레임(Pfr)이 발생된다. 여기서, 매개변수 L, D, G 값은 예를 들어 나타낸 것으로, 상기한 값들에 한정되는 것은 아니며, 전송 지연 또는 수신 지연 요구에 따라 다르게 설정될 수 있다. 또한, 본 발명의 실시예에에서는 패리티 프레임이 각 GoP 구간의 가장 마지막 프레임 시간에 위치하는 것으로 예를 들어 설명하였으나, 패리티 프레임의 위치가 각 GoP 구간의 가장 마지막 프레임 시간에 위치하는 것으로 한정되는 것은 아니다. 또한, 각 패리티 프레임을 생성하기 위해 선택되는 마크로 프레임의 선택 방법 역시 다양하게 구성될 수 있다.The parameters of the first type GD-FEC according to an embodiment of the present invention are L = 4, D = 1, G = 1. Here, D denotes the number of GoPs in which parity frames Pfr obtained as a result of GD-FEC encoding are divided. In the first type GD-FEC illustrated in FIG. 1, one parity frame (Pfr) is included in one GoP. ), So D = 1. Also, G is

Each of the corresponding macro frames in concatenated denotes a generation period of the parity frame Pfr generated through the function F. As shown in FIG. 1, when G = 1, the parity frame Pfr is generated every GoP. Here, the parameters L, D and G values are shown as an example, and are not limited to the above-described values, and may be set differently according to a transmission delay or a reception delay request. Also, in the embodiment of the present invention, for example, the parity frame is located at the last frame time of each GoP section, but for example, the position of the parity frame is limited to being located at the last frame time of each GoP section. no. In addition, a method of selecting a macro frame selected to generate each parity frame may also be variously configured.

Referring to Equation 1, the configuration of the first type GD-FEC shown in FIG. 1 will be described in more detail. First, each of four GoPs (GoP-2, GoP-3, GoP-4, and GoP-5) is included. After the concatenated four macro frames Mfr5, Mfr9, Mfr13, and Mfr17 are concatenated to form a source symbol to perform RaptorQ-like encoding (i.e., function F), the resulting parity frame (Pfr4) is framed. ) t4.

Meanwhile, since the first type GD-FEC assumes G = 1 as a parameter and one GoP consists of four frame times, the second parity frame Pfr8 is located at the frame time t8. As shown in Equation 1, source symbols are formed by concatenating four macro frames Mfr9, Mfr13, Mfr17, and Mfr21 included in four GoPs (GoP-3, GoP-4, GoP-5, and GoP-6), respectively. RaptorQ-like encoding is performed to generate a parity frame (Pfr8).

2nd type GD-FEC

FIG. 2 is a conceptual diagram illustrating a second type GD-FEC method according to an embodiment of the present invention, and illustrates a timing diagram, encoding, parity frame configuration, distribution, and placement method of the second type GD-FEC code.

Hereinafter, the configuration and method of the second type GD-FEC according to an embodiment of the present invention will be described with reference to FIG. 2.

As shown in FIG. 2, the parameters of the second type GD-FEC may be set to L = 4, D = 1, and G = 2, and one GoP may be configured with four frame times. Here, the meanings of the parameters L, D, and G and the method of generating the macro frame Mfr are the same as described above with reference to FIG. 1.

에 4개의 GoP가 포함되고, 4개의 GoP 각각에 포함된 마크로 프레임(Mfr)이 연접되어 소스 심볼로 구성된 후 RaptorQ-like 인코딩의 입력으로 제공되어 패리티 프레임(Pfr)을 생성하게 된다.The second type GD-FEC is L = 4, so one

4 GoPs are included, and macro frames (Mfr) included in each of the 4 GoPs are concatenated, composed of source symbols, and provided as inputs of RaptorQ-like encoding to generate parity frames (Pfr).

On the other hand, in the second type GD-FEC, since G = 2, one parity frame Pfr is generated for every two GoPs, and one parity frame Pfr generated because D = 1 is carried on one GoP.

In the second type GD-FEC, a parity frame generation procedure may be expressed by Equation 2.

Equation 2

In Equation 2, the meanings of the operators '<~', F, and i are the same as described in Equation 1.

As shown in Equation 2, the method of generating and applying the first parity frame Pfr 4 of the second type GD-FEC is the same as that of the first type GD-FEC, but the second type GD-FEC is G = 2. The generation method and application position of the second parity frame Pfr 12 are different from the first type GD-FEC.

That is, in the second type GD-FEC, since G = 2, the application position of the second parity frame Pfr 12 may be the frame time t12 at which GoP-3 is located, and the GoP- immediately after GoP-3. 4 macro frames (Mfr 13, Mfr 17, Mfr 21, and Mfr 25) included in each of the four GoPs (ie GoP-4, GoP-5, GoP-6, GoP-7) located consecutively from 4 Based on the parity frame Pfr 12 may be generated.

In the configuration of the second type GD-FEC shown in FIG. 2, the parameters L, D, and G values are shown as examples, and are not limited to the above values, and may be set differently according to a transmission delay or a reception delay request. have. In addition, in the embodiment of the present invention, for example, the parity frame is located at the last frame time of each GoP section, for example, but the position of the parity frame is limited to being located at the last frame time of each GoP section. no. In addition, a method of selecting a macro frame selected to generate each parity frame may also be variously configured.

3rd type GD-FEC

3 is a conceptual diagram illustrating a third type GD-FEC method according to an embodiment of the present invention, and illustrates a timing diagram, encoding, parity frame configuration, distribution, and placement method of the third type GD-FEC code. 4 shows the frame configuration timing diagram of the third type GD-FEC method shown in FIG. 3 in more detail.

Hereinafter, a third type GD-FEC method according to an embodiment of the present invention will be described in more detail with reference to FIGS. 3 and 4.

라 하고,

의 원소의 개수를 L이라 하면, L개의 GoP 별로 버스트 패킷 손실 발생시 복구하고자 하는 I 프레임들의 길이를 비교하여 가장 긴 프레임을 선정하고, 선정된 I 프레임의 크기를 마크로 프레임(Mfr)의 크기로 정한다.In FIG. 3, L denotes a source information input group to be GD-FEC encoded, and indicates a group of GoPs participating in one GD-FEC encoding.

,

If the number of elements is L, the longest frame is selected by comparing the lengths of I frames to be recovered in case of burst packet loss by L GoPs, and the size of the selected I frame is determined as the size of the macro frame (Mfr). .

에 포함된 GoP-3, GoP-4, GoP-5, GoP-6 중 가장 긴 길이를 가지는 I 프레임은 프레임 I41이 되고, 프레임 I41의 길이가 마크로 프레임(Mfr)의 크기로 결정된다.For example, when each GoP shown in FIG. 3 is configured as shown in FIG. 4, one GoP group

The I frame having the longest length among GoP-3, GoP-4, GoP-5, and GoP-6 included in the frame becomes frame I41, and the length of frame I41 is determined as the size of the macro frame Mfr.

를 구성하는 원소들인 각각의 GoP에서, I 프레임을 우선적으로 선정한 후 미리 설정된 우선 순위에 따라 보호하고자 하는 프레임 순서대로 마크로 프레임(Mfr)의 길이에 맞추어 각 GoP에 해당하는 마크로 프레임(Mfr)을 구성한다. 여기서, 우선 순위는 버스트 패킷 손실이 발생하는 경우 복원 우선 순위를 의미할 수 있고, 예를 들어, I, P, B 프레임 순으로 우선 순위가 설정될 수 있다.Also,

In each GoP, which is the elements constituting, the I frame is first selected, and then the macro frame Mfr corresponding to each GoP is constructed according to the length of the macro frame Mfr in the order of frames to be protected according to a preset priority. do. Here, the priority may mean a restoration priority when burst packet loss occurs. For example, the priority may be set in the order of I, P, and B frames.

For example, as shown in FIG. 4, the macro frame Mfr 9 is based on the length of the longest frame I41 among the I frames included in GoP-3, GoP-4, GoP-5, and GoP-6, respectively. When the size of is determined, the macro frames Mfr 13, Mfr 17, and Mfr having sizes corresponding to the macro frames Mfr 9 in the GoP-4, GoP-5, and GoP-6, respectively, according to the size of the macro frames Mfr 9 Configure 21.

That is, as illustrated in FIG. 4, the macro frame Mfr 13 configured as part of the I frame I41 and the P frame P41 may be determined in the GoP-4 according to the preset priority, and the I frame in the GoP-5. (I51), the macro frame Mfr 17 composed of a part of the P frame P51 and the B frame B51 can be determined. Also, in GoP-6, a macro frame Mfr 21 including an I frame I61 and an I frame P62 may be determined.

에 포함된 GoP들 각각에서 마크로 프레임이 결정되면, 결정된 마크로 프레임들을 연접하여 소스 심볼을 구성한 후, 구성된 소스 심볼을 대상으로 GD-FEC 부호화를 수행하여 그 결과로 분할 패리티 프레임(Divided Parity frame, DPfr)을 획득한다. 여기서, 분할 패리티 프레임은 L개로 구성되는 GoP 그룹의 마크로 프레임(Mfr)들을 연접하여 GD-FEC 부호화하여 발생된 패리티 프레임(Pfr)을 각 GoP의 적당한 위치(예를 들면, 각 GoP 구간의 마지막 프레임 시간)에 추가하기 위해 소정의 크기로 분할된 프레임을 지칭한다.As mentioned above

When the macro frame is determined in each of the GoPs included in the concatenated GoPs, the determined macro frames are concatenated to form a source symbol, and then a GD-FEC encoding is performed on the configured source symbols, resulting in a divided parity frame (DPfr). ). In this case, the divided parity frame is obtained by concatenating the macro frames (Mfr) of the GoP group consisting of L and parity frame (Pfr) generated by GD-FEC encoding to an appropriate position of each GoP (for example, the last frame of each GoP section). Frame divided into a predetermined size to add to the time).

Thereafter, the obtained divided parity frame DPfr is added to the corresponding GoP.

The generation and addition procedure of the split parity frame DPfr in the third type GD-FEC may be expressed as in Equation 3 below.

Equation 3

내의 마크로 프레임(Mfr)들을 연접하여 구성한 소스 심볼을 RaptorQ-like 인코딩의 입력으로 하여 획득한 출력을

내의 각 GoP에 삽입하는 분할 패리티 프레임(DPfr)을 생성하는 함수를 의미한다. 또한, 연산자 '//'는 RaptorQ-like 인코딩 결과로 생성된 패리티 프레임의 길이를 D 등분하여 분할 패리티 프레임(DPfr)를 생성하는 것을 의미한다. 또한, F는 RaptorQ-like 인코딩 함수를 의미하고, i는 프레임 시간(frame time)을 의미한다. 분할 패리티 프레임(DPfr)의 길이는 후술할 따름 정리(Corollary) 2에 의해 결정된다.In equation (3), the operator '<~'

Output obtained by inputting RaptorQ-like encoding the source symbol formed by concatenating macro frames (Mfr) in the

A function for generating a split parity frame (DPfr) to be inserted into each GoP. In addition, the operator '//' means generating a parity parity frame DPfr by dividing the length of the parity frame generated as a result of RaptorQ-like encoding. In addition, F means RaptorQ-like encoding function, i means frame time (frame time). The length of the split parity frame DPfr is determined by Corollary 2, which will be described later.

Parameters of the third type GD-FEC according to an embodiment of the present invention may be given as L = 4, D = 2, G = 2. Here, the meanings of the parameters D and G are the same as in FIG.

Meanwhile, in the third type GD-FEC, since G = 2, one parity frame Pfr is generated for every two GoPs, and one parity frame Pfr generated because D = 2 is divided into two parity frames DPfr. Each parity frame (DPfr) is divided into one GoP, so that one parity frame (Pfr) is located in two GoPs.

In the configuration of the third type GD-FEC shown in Figs. 3 and 4, parameter L, D, and G values are shown, for example, and are not limited to the above values. Can be set. In addition, in the embodiment of the present invention, for example, the divided parity frame is located at the last frame time of each GoP interval, for example, but the position of the divided parity frame is limited to be located at the last frame time of each GoP interval It doesn't happen. In addition, a method of selecting a macro frame selected to generate each parity frame may also be variously configured.

4th type GD-FEC

FIG. 5 is a conceptual diagram illustrating a fourth type GD-FEC method according to an embodiment of the present invention, and illustrates a timing diagram, encoding, parity frame configuration, distribution, and placement method of a fourth type GD-FEC code. 6 shows the frame configuration timing diagram of the fourth type GD-FEC method shown in FIG. 5 in more detail.

Hereinafter, a fourth type GD-FEC method according to an embodiment of the present invention will be described in more detail with reference to FIGS. 5 and 6.

라 하고,

의 원소의 개수를 L이라 하면, L개의 GoP 별로 버스트 패킷 손실 발생시 복구하고자 하는 I 프레임들의 길이를 비교하여 가장 긴 프레임을 선정하고, 선정된 I 프레임의 크기를 마크로 프레임(Mfr)의 크기로 정한다. 도 5에 도시한 제4 유형 GD-FEC에서는 매개 변수 L 및 5를 각각 5로 설정한 경우를 예를 들어 도시하였다. 그러나, 매개 변수 L 및 5의 값은 상기한 값들에 한정되는 것은 아니며, 전송 지연 또는 수신 지연 요구 등에 따라 다르게 설정될 수 있다.In FIG. 5, L denotes a source information input group to be GD-FEC encoded, and indicates a group of GoPs participating in one GD-FEC encoding.

,

If the number of elements is L, the longest frame is selected by comparing the lengths of the I frames to be recovered in case of burst packet loss by L GoPs, and the size of the selected I frame is determined as the size of the macro frame (Mfr). . In the fourth type GD-FEC shown in FIG. 5, the case where the parameters L and 5 are set to 5 is illustrated as an example. However, the values of the parameters L and 5 are not limited to the above values, and may be set differently according to a transmission delay or a reception delay request.

에 포함된 GoP-1, GoP-2, GoP-3, GoP-4, GoP-5 중 가장 긴 길이를 가지는 I 프레임은 프레임 I21이 되고, 프레임 I21의 길이가 마크로 프레임(Mfr)의 크기로 결정된다.When each GoP shown in FIG. 5 is configured as shown in FIG. 6, one GoP group

The I frame having the longest length among GoP-1, GoP-2, GoP-3, GoP-4, and GoP-5 included in the frame is frame I21, and the length of frame I21 is determined as the size of the macro frame (Mfr). do.

를 구성하는 원소들인 각각의 GoP에서, I 프레임을 우선적으로 선정한 후 미리 설정된 우선 순위에 따라 보호하고자 하는 프레임 순서대로 마크로 프레임(Mfr)의 길이에 맞추어 각 GoP에 해당하는 마크로 프레임(Mfr)을 구성한다. 여기서, 우선 순위는 버스트 패킷 손실이 발생하는 경우 복원 우선 순위를 의미할 수 있고, 예를 들어, I, P, B 프레임 순으로 우선 순위가 설정될 수 있다.Also,

In each GoP, which is the element constituting the, I frame is first selected, and then the macro frame Mfr corresponding to each GoP is constructed according to the length of the macro frame Mfr in the order of frames to be protected according to the preset priority. do. Here, the priority may mean a restoration priority when burst packet loss occurs. For example, the priority may be set in the order of I, P, and B frames.

For example, as shown in FIG. 6, the macro frame is based on the length of the longest frame I21 among the I frames included in GoP-1, GoP-2, GoP-3, GoP-4, and GoP-5, respectively. When the size of (Mfr 5) is determined, the macro having the size corresponding to the macro frame (Mfr 5) in each of GoP-1, GoP-3, GoP-4, and GoP-5 according to the size of the macro frame (Mfr 5). Frames Mfr 1, Mfr 9, Mfr 13 and Mfr 17.

That is, as illustrated in FIG. 6, the macro frame Mfr 1 configured as part of the I frame I11 and the P frame P11 may be determined in the GoP-1 according to the preset priority, and the I frame in the GoP-3. A macro frame Mfr 9 consisting of part of I31 and P frame P31 can be determined. Also, in GoP-4, a macro frame Mfr 13 consisting of a part of an I frame I41, a P frame P 41, and a B frame B41 can be determined. In a GoP-5, an I frame I51 and an I frame are determined. The macro frame Mfr 17 composed of I52 can be determined.

에 포함된 GoP들 각각에서 마크로 프레임이 결정되면, 결정된 마크로 프레임들을 연접하여 소스 심볼을 구성한 후, 구성된 소스 심볼을 대상으로 GD-FEC 부호화를 수행하여 그 결과로 분할 패리티 프레임(DPfr)을 획득한다.As mentioned above

When a macro frame is determined in each of the GoPs included in the concatenated GoPs, concatenating the determined macro frames to form a source symbol, then performing GD-FEC encoding on the configured source symbol, thereby obtaining a divided parity frame (DPfr). .

Thereafter, the obtained divided parity frame DPfr is added to the corresponding GoP.

In the fourth type GD-FEC, the generation and addition procedure of the divided parity frame DPfr may be expressed as in Equation 4.

Equation 4

내의 마크로 프레임(Mfr)들을 연접하여 구성한 소스 심볼을 RaptorQ-like 인코딩의 입력으로 하여 획득한 출력을

내의 각 GoP에 삽입하는 분할 패리티 프레임(DPfr)을 생성하는 함수를 의미한다. 또한, 연산자 '//'는 RaptorQ-like 인코딩 결과로 생성된 패리티 프레임의 길이를 (D-T) 등분하여 분할 패리티 프레임(DPfr)를 생성하는 것을 의미한다. 또한, F는 RaptorQ-like 인코딩 함수를 의미하고, T는 마크로 프레임(Mfr)의 연접 손실 발생시 복구 가능한 GoP의 수, 즉 복구 가능한 마크로 프레임(Mfr)의 수를 의미하며, i는 프레임 시간(frame time)을 의미한다. 분할 패리티 프레임(DPfr)의 길이는 후술할 따름 정리(Corollary) 2에 의해 결정된다.In equation (4), the operator '<~'

Output obtained by inputting RaptorQ-like encoding the source symbol formed by concatenating macro frames (Mfr) in the

A function for generating a split parity frame (DPfr) to be inserted into each GoP. In addition, the operator '//' means generating a parity parity frame DPfr by dividing the length of the parity frame generated as a result of RaptorQ-like encoding by DT. In addition, F means RaptorQ-like encoding function, T means the number of recoverable GoPs, i.e., the number of recoverable macro frames (Mfr) when concatenation loss of the macro frame (Mfr) occurs, i is the frame time (frame time). The length of the split parity frame DPfr is determined by Corollary 2, which will be described later.

In the following, the mathematical theorem is first defined in order to derive the theorem, followed by theorem 1 and theorem 2.

Math Theorem 1

Assuming the channel capacity C is B bits per channel usage, the rate R has a relationship of R = Bk / (k + m) ≦ C = B. Here, k means message length and m means parity length.

In addition, assuming that loss occurs as much as the α ratio of the message during the bearer transmission procedure, the length of the remaining portion of the message in which no loss occurs is (1-α) k, and a relationship of m> α k is established.

Proof of Math Theorem 1

The lost message in Math Theorem 1 can be interpreted as lost while passing through the Binary Erasure Channel (BEC). Therefore, an inequality as shown in Equation 5 is established, and thus an inequality as shown in Equation 6 must be established.

Equation 5

Equation 6

If m = αk, R '= 1, which results in a value larger than C', which is against the assumption. Therefore, the inequality shown in equation (6) holds only when m>? K.

Corollary 1

It can be seen that, in the case of a message having a length of k bits, lost by α according to theorem 1, the required length of parity is needed more than the lost bits. At this time, when performing loss recovery using a code close to the complete loss recovery code (for example, a RaptorQ code), the length of the parity may be configured to be close to the lost bit length. Thus, m = αk + ε holds for a sufficiently small amount ε.

Corollary 2

The length of the divided parity frame may be calculated as shown in Equation 7.

Equation 7

According to the above theorem 1 and the following theorem 1, the amount of parity bits should be set larger than the amount of information bits to be recovered. This means that if successive GoPs are lost and all frames are lost, more parity bits must be generated than the amount of bits in the I frame to recover even the bits of the I frame.

내에서 T개의 GoP가 모두 손실된 경우에는 D-T 개의 GoP를 이용하여 RaptorQ-like 디코딩을 수행하여 필요한 정보를 복원할 수 있기 때문에 최종적으로 손실을 복원하기 위해 필요한 분할 패리티 프레임의 길이는 수학식 7과 같이 주어진다.In conclusion, when the bits of αk are lost, the length of parity that can be recovered should be m = αk + ε, so in order to recover the lost T macro frames Mfr, [T · length (Mfr) + ε] parity of length is required. Also,

In the case where all T GoPs are lost in the network, RaptorQ-like decoding can be performed by using DT GoPs to restore the necessary information. Is given together.

FIG. 7 illustrates a header configuration of a parity frame used in a progressive degradation forward error correction method according to an embodiment of the present invention, and illustrates a header configuration of a parity frame generated through GD-FEC encoding.

Referring to FIG. 7, a header of a parity frame may include a Num frames field, a Frame nums field, and a Frame n Length (where n is a natural number) field.

In this case, the header of the parity frame may omit a duplicate field or may compress the header. In addition, other parameter information not included in the header of the parity frame may be transmitted through out of band signaling.

In the following, starting from the conceptual description about the processing sequence of the deterioration progresses forward error correction process according to the invention with reference to the transmission protocol structure of the 3GPP (3 ^rd Generation Partnership Project) will be explained by embodiment in sequence.

8 is a conceptual diagram illustrating a processing procedure of a progressive degradation forward error correction method according to an embodiment of the present invention.

Referring to FIG. 8, in general, multimedia data such as audio, video, dynamic and interactive multimedia scenes (DIMS), text, and the like are transmitted from the media device 810 to the transmission device 820, and then the network is transmitted from the transmission device 820. 830 is provided to the user terminal 840.

Here, the media device 810 may be a content server, for example, and the transmission device 820 may be configured, for example, with a broadcast multicast service center (BM-SC). In addition, the network 830 may be a mobile communication network supporting a multimedia broadcast and multicast service (MBMS).

The GD-FEC and AL-FEC methods according to an embodiment of the present invention can improve the reliability and robustness of multimedia data transmission, but the parity data for this is unnecessary redundancy in terms of the media device 810. Data. Therefore, the GD-FEC and AL-FEC methods should not be performed in the general-purpose media equipment 810 for providing multimedia, and the transmission equipment 820 may be provided with audio, video, DIMS, text, etc. provided from the media equipment 810. Perform GD-FEC and AL-FEC on multimedia data.

However, unlike AL-FEC, since the GD-FEC according to an embodiment of the present invention must distinguish I frames during encoding and decoding, data input to the transmission device 820 may be interleaved, encrypted, or the like. It must be processed before the data type or priority can be distinguished.

FIG. 9 illustrates a position where a progressive degradation forward error correction method according to an embodiment of the present invention is included in a 3GPP transport protocol structure, and illustrates an example of a protocol structure for performing MBMS transmission. In addition, Figure 10 shows an AL-FEC processing procedure that is processed after the gradual degradation forward error correction method is performed according to an embodiment of the present invention. The GD-FEC and AL-FEC methods shown in FIGS. 9 and 10 may be performed in transmission equipment.

As shown in FIG. 9, the GD-FEC encoding process according to an embodiment of the present invention may be performed in a layer that first processes input multimedia data among protocol layers.

That is, the GD-FEC encoding process according to an embodiment of the present invention may be performed at the uppermost layer of the protocol layer or at an upper layer before the layer performing at least a transmission function as shown in FIG. 9.

On the other hand, AL-FEC is performed at a layer just below the Realtime Transport Protocol (RTP) & Realtime Transport Control Protocol (RTP) and Secure RTP (SRTP) layers. Data input to the layer performing AL-FEC processing may be data that has already been interleaved and encrypted at the RTP or SRTP layer, and the layer processing AL-FEC is the same regardless of the type or priority of the data. Encoding is performed by grouping into symbols having priority.

That is, as shown in FIG. 10, the AL-FEC divides the input source file or stream S1010 into source blocks (S1020), configures source symbols in units of the divided source blocks (S1030), and constitutes a source symbol. After generating the parity symbol by performing encoding on (S1040), the encoded block is generated by encoding the packet corresponding to the source symbol and the packet corresponding to the parity symbol (S1050).

As described above, the transmission equipment for performing the GD-FEC method according to an embodiment of the present invention is the GD-FEC processing layer, the RTP & RTCP layer, the SRTP layer and the FEC layer, and the User Datagram Protocol (UDP) layer from the top. It may have a protocol hierarchy.

On the other hand, the decoding delay of the GD-FEC coded data may cause various problems in the system such as switching time delay.

Hereinafter, a method for minimizing a delay of a progressive degradation forward error correction method according to an embodiment of the present invention will be described.

11 illustrates a method for minimizing a decoding delay of a progressive degradation forward error correction method according to an embodiment of the present invention, which is a method of a GD-FEC method applied to a FLUTE (File Delivery over Unidirectional Transport) protocol for a streaming service. An example is shown.

In detail, referring to FIG. 12, when a source packet stream is received, the transmission equipment sets an encoding group including a predetermined number of time intervals according to a preset parameter (for example, L = 4) as described above ( S1110), after performing GD-FEC encoding on a packet (eg, an audio packet) included in each time interval for each set coding group, generating a reconstruction packet for recovering a lost packet when a packet is lost, and then generating The decompression packet is distributed and arranged in each time interval according to a preset rule.

Here, the reconstructed packets mean a parity frame, and as described above with reference to FIGS. 1 to 6, a reference data set to be used for GD-FEC encoding is first determined in each time interval according to a preset priority, and then RaptorQ is determined. The recovery packet can be generated using the input of the -link encoding.

In addition, after the received source packet stream is buffered in the transmission equipment, GD-FEC encoding may be performed on the received source packets that are buffered, and accordingly, transmission delay (or buffering delay) is performed by the time for performing GD-FEC. This can happen.

After performing GD-FEC encoding as described above, the transmission equipment first transmits a reconstruction packet obtained through performing GD-FEC encoding through a network (S1120).

The restoration equipment first receives a restoration packet transmitted from the transmission equipment (S1130). Here, the restoration equipment may receive a restoration packet after a predetermined time (that is, a transmission delay time through the network), and the restoration equipment may receive a restoration packet in which some restoration packets are lost.

Subsequently, the decompression device buffers the received decompression packet, and then decompresses the audio packet using the buffered decompression packet (S1140).

That is, in an embodiment of the present invention, the transmission equipment obtains a recovery packet by performing GD-FEC encoding during a buffered time interval for transmitting a packet stream, and restores the obtained recovered packet through a network during the buffering time interval. By first transmitting to the device, the reconstructing device reconstructs the packet first using the reconstructed packet so that even when the GD-FEC encoding is performed and the source packet is transmitted, only the network transmission delay occurs, and there is no delay for the GD-FEC decoding. do.

12 illustrates a method for minimizing a decoding delay of a progressive degradation forward error correction method according to another embodiment of the present invention, and shows another example of a GD-FEC method applied to the FLUTE protocol for a streaming service.

Referring to FIG. 12, in another embodiment of the present invention, a duration of a GD-FEC coding group is set equal to a FLUTE file segment duration.

Accordingly, GD-FEC encoding may be performed during the processing time of the FLUTE file segment, and thus no additional delay for GD-FEC encoding occurs.

In addition, in another embodiment of the present invention, since the GD-FEC decoding can be performed during the processing time of the FLUTE file segment, no additional delay due to the GD-FEC decoding occurs.

Meanwhile, in the embodiment of the present invention, it should be noted that performance of the GD-FEC may be improved by performing interleaving. An additional delay due to interleaving may be prevented by setting the interleaving time unit in the FLUTE segment packet unit.

In addition, the interleaving may be implemented by transmitting the FLUTE segment file according to a preset order in units of segment packet units, which are further divided for transmission, without separately processing the interleaving.

Although described with reference to the above embodiments, those skilled in the art will understand that various modifications and changes can be made without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

Claims (18)

Determining a coding unit data group including a preset number of time intervals for a data stream continuously input according to time; Determining a reference data set to be used for generating a parity frame in each time period of the coding unit data group; Generating a parity frame using the determined at least one reference data set; And Parity frame forward error correction comprising the step of arranging the generated parity frame in at least one time interval of a plurality of time intervals comprising the time intervals constituting the coding unit data group based on a predetermined rule Way. The method of claim 1, wherein the determining of the reference data set comprises: Comparing time lengths of reference data types having the highest reconstruction priority among preset data types included in each time period of the coding unit data group; Determining a data size of a reference data type having a longest length of time among the reference data types included in each time interval as the size of the reference data set; And And determining the reference data set including at least one data type according to a preset restoration priority in each time interval based on the determined size of the reference data set. Way. The method of claim 1, wherein generating the parity frame, Concatenating the reference data set to construct source data; And Generating the parity frame by performing encoding on the source data. The method of claim 3, wherein the encoding of the source data to generate the parity frame comprises: And generating the parity frame by performing RaptorQ-like encoding using the source data as input. The method of claim 1, wherein generating the parity frame, And generating the parity frame according to a predetermined period configured in units of the reference data set. The method of claim 1, wherein the parity frame arrangement step, According to a preset rule, the parity frame is disposed in a time interval before time intervals constituting the coding unit data group or in at least one time interval among the time intervals constituting the coding unit data group. Progressive degradation forward error correction method. The method of claim 1, wherein the parity frame arrangement step Dividing the parity frame to generate two or more divided parity frames each having a specific length; And The at least two split parity frames may be disposed in time intervals before the time intervals constituting the coding unit data group or in at least two time intervals of the time intervals constituting the coding unit data group. Progressive degradation forward error correction method. The method of claim 7, wherein generating the divided parity frame, And determining the size of the divided parity frame based on the number of reference data sets to be recovered among the plurality of reference data sets and the size of each reference data set. The method of claim 1, wherein the determining of the coding unit data group comprises: Determining a group of pictures (GoP) for each of the time intervals having a preset length for pictures continuously input according to time; And And determining the coding unit data group consisting of a preset number of picture groups. The method of claim 9, wherein the determining of the reference data set comprises: Comparing the length of an I picture among picture types included in each of a plurality of picture groups constituting the coding unit data group; Determining the length of the I picture having the longest length of the I picture included in each of the plurality of picture groups as the size of the reference data set; And And determining the reference data set including at least one picture type according to a preset reconstruction priority in each picture group based on the determined size of the reference data set. Way. In the progressive degradation forward error correction method performed in a transmission device for transmitting multimedia data, Determining a coding unit data group including a preset number of time intervals for the received data stream; Determining a reference data set to be used for generating a parity frame in each time period included in the determined coding unit data group; Generating a parity frame using the determined at least one reference data set; And And transmitting the generated parity frame before transmitting data included in the preset coding unit data group. The method of claim 11, wherein generating the parity frame, Concatenating the reference data set to construct source data; And Generating the parity frame by performing RaptorQ-like encoding on the source data. The method of claim 11, wherein transmitting the generated parity frame before transmitting data included in the preset coding unit data group comprises: And transmitting the generated parity frame during the time interval in which the received data stream is buffered. The method of claim 11, wherein the determining of the coding unit data group consisting of a predetermined number of time intervals for the received packet stream, And determining the time interval of the coding unit data group in the same time interval as the file division processing unit time interval of a specific protocol. The method of claim 11, wherein the determining of the coding unit data group consisting of a predetermined number of time intervals for the received packet stream, And determining a time interval of the coding unit data group in the same time interval as a file segment duration of the file delivery over unidirectional transport (FLUTE) protocol. The method of claim 15, wherein the progressive degradation forward error correction method, And performing interleaving on a segment packet unit basis for dividing a segment file included in the file segment section of the FLUTE protocol. A first encoding processing layer for performing progressive deterioration forward error correction (GD-FEC) on the input multimedia data; A transport protocol processing layer that performs transport protocol processing on data on which progressive degradation forward error correction processing has been performed; And And a second encoding processing layer that performs application layer forward error correction (AL-FEC) on the data on which the transmission protocol processing has been performed. The method according to claim 17, And the first encoding processing layer classifies the type of the input multimedia data to perform the progressive degradation forward error correction process.

Images