CN1969493B - Two-way communication method, device and system - Google Patents

Abstract

Two-way communication devices (101, 103) communicating through a transmission channel (102) each have an encoding section, decoding section and at least either a combination of an interleaver and a deinterleaver or a combination of an error correction encoding section and a decoding section. Each of the two-way communication devices can asymmetrically through transmission/reception set the interleaving unit and interleaving order, or the error-correction encoding method, error generation code generation unit, encoding rate, frame rate, and sampling frequency and communicate depending on the state of the transmission channel (102), the delay request of the user, and the quality request.

Description

Two-way communication method, device and system

technical field

The present invention relates to a two-way communication method, device, system and computer program. More specifically, the present invention relates to two-way communication suitable for suppressing deterioration of media quality due to data loss occurring on a transmission line and realizing stable quality communication in two-way communication of images, sounds, etc. via a network Methods, devices, systems and computer programs. In addition, the present invention also relates to a method, a device, and a program capable of setting a trade-off between quality and delay increase in both directions according to the state of the transmission line and the user's request for delay/quality. the

Background technique

In recent years, two-way communication called VoIP (Voice over IP, Voice over Internet) and TvoIP (TV over IP), which communicate coded data of images and sounds using packets via a network, has rapidly spread. There are several types of encoding schemes used for the above two-way communication. For example, as a video encoding method, a method of transmitting highly compressed encoded data based on inter-frame prediction is often used. In these methods, prediction parameters and prediction residual image data obtained by predictively encoding images from temporally preceding and following frames are encoded, thereby reducing the amount of information of moving image data highly correlated in the temporal direction. In addition, by efficiently compressing and encoding prediction residual image data by transform coding or quantization, communication can be performed in a small transmission band. the

As a typical example, there is a method using a compression encoding method such as MPEG (Moving Picture Experts Group)-1, MPEG-2, or MPEG-4. In these compression coding methods, inter-frame prediction by motion compensation is performed on an input image frame in units of fixed-sized rectangular areas called macroblocks. The compressed signal data is converted into a variable-length code by performing two-dimensional discrete cosine transform and quantization on the obtained motion vector and prediction residual image data. the

In addition, regarding sound, there are also G.722, G.729, AMR-NB (Adaptive MultiRate-Narrow Band, Adaptive Multi-Rate-Narrow Band), AMR-WB (Adaptive Multi-Rate-Wide Band, Adaptive Multi-Rate-Wide Band) , MPEG-4_AAC (Advanced Audio Codec, Advanced Audio Codec) and other sound (music) encoding methods. High-efficiency encoding is performed for each predetermined sampling amount, so that high-quality audio data can be transmitted in a small transmission band. the

There are many methods for distributing such encoded data of image or sound to an IP (Internet Protocol, Internet Protocol) network utilizing packet switching. the

In the future, it is expected to expand to a two-way communication system in a mobile communication system in which a mobile station is connected to a wireless base station via a wireless channel, wherein the mobile station is composed of a PHS (Personal Handyphone System), a portable phone, or a mobile phone using these Portable terminals used as communication means, etc. the

For the SIP (Session Initiation Protocol, Session Initiation Protocol)/SDP (Session Description Protocol, Session Description Protocol) described later, refer to Technical Document 1 (M.Handley, H.Schulzrinne, E.Schooler, J.Rosenberg, "SIP : Session Initiation Protocol", RFC2543, March 1999, Internet URL <http://www.ietf.org/rfc/rfc2543.txt>), and technical literature 2 (M. Handley, V. Jacobson, "SDP: Session Description Protocol ", RFC2327, April 1998, Internet URL <http://www.ietf.org/rfc/rfc2327.txt>). In addition, for the PTP described later, refer to Technical Document 3 (H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson, "PTR: A Transport Protocol for Real-Time Applications", RFC3550, January 2003, Internet URL <http://www.ietf.org/rfc/rfc3550.txt>). the

In conventional two-way communication, if a data error occurs on the transmission line or a transmission packet is lost, the received coded data cannot be decoded correctly, resulting in deterioration of the quality of the media. the

In addition, since it is two-way communication, it is necessary to control the delay below a certain value, so that it is difficult to request retransmission of lost data and it is also difficult to retransmit data. the

In addition, when inter-frame prediction is used in the image or sound encoding method, once the image or sound is disturbed, the disturbance will be propagated to subsequent frames. the

In this regard, the following method can be adopted, that is, the interleaving process is performed before transmission to the transmission line, so that the burst loss occurring on the wireless transmission line is dispersed in the interleaving processing unit so as not to cause continuous data loss, or it is also possible to use A method of recovering erroneous or lost data using error-correcting codes such as FEC (Forward Error Correction). However, if the above method is used as it is, it is difficult to use it for two-way communication as the delay increases. the

Therefore, the present invention is made in consideration of the above-mentioned problems, and its main purpose is to provide a setting based on a compromise between the media quality, delay, frequency band, and delay requirements required by each user, which can be as stable as possible. A method, device, system and program for two-way communication. the

Another object of the present invention is to provide a two-way communication method, device, system, and program that can meet user requirements for quality and delay even in asymmetrical uplink and downlink transmission lines. the

Still another object of the present invention is to provide a bidirectional communication method, device, system, and program that control quality stability and suppress increase in processing delay in accordance with fluctuating data error or loss states on transmission lines. the

Contents of the invention

In the first solution, the communication system of the present invention can also be constituted as follows: it includes: a unit for capturing media signals; a unit for encoding captured media data; an interleaving processing unit; a sending unit for sending to a transmission line; A receiving unit for receiving; a deinterleaving processing unit; a unit for decoding encoded data; a unit for replaying decoded media data; setting encoding settings in the encoding unit, interleaving processing settings, and sending settings in the sending unit and a unit that notifies at least one of the encoding configuration, the interleaving processing configuration, the transmission configuration, and the reception status of the encoded data, and the communication system encodes the captured media signal and then transmits it , receive the encoded data and replay it after decoding it. the

In the second solution, the communication system of the present invention can also be constituted as follows: it includes: a unit for capturing media signals; a unit for encoding captured media data; a unit for generating error correction code data from the encoded data; sending to the transmission line A unit for receiving from a transmission line; a unit for restoring encoded data using an error correction code when the received encoded data is erroneous or lost; a unit for decoding encoded data; a unit for replaying decoded media data A unit that sets the encoding setting in the encoding unit, the error correction code data generation processing setting, and the transmission setting in the transmitting unit; and notifies the encoding setting, the error correction code data generation processing setting, The unit of at least one of the transmission setting and the reception status of the encoded data, the communication system encodes the captured media signal and sends it, receives the encoded data and decodes it and then plays it back. the

In the third aspect, the communication system of the present invention is a two-way communication system including at least two communication devices communicating via a transmission line, wherein at least one of the two communication devices includes: an encoding unit for The input media is encoded; the interleaving unit interleaves the encoded data in a specified unit; and the sending unit outputs the interleaved encoded data to the transmission line;

At least another one of the two communication devices includes: a receiving unit for receiving the encoded data from the transmission line; a deinterleaving unit for rearranging the received interleaved encoded data into the original encoded data by a prescribed method data; a decoding unit that decodes the encoded data and outputs the decoded data; and a playback unit that plays back the decoded data. the

In the fourth aspect, the communication method of the present invention includes: a step of encoding the input media; a step of interleaving the encoded data in a prescribed unit; a step of outputting the interleaved encoded data; a step of receiving the encoded data; a step of deinterleaving the received coded data by a prescribed method; and a step of decoding and reproducing the coded data. the

In the fifth aspect, the computer program of the present invention causes the computer (processor) constituting the two-way communication device to execute: the processing of encoding the input media; the processing of interleaving the encoded data in predetermined units; and the output of the interleaved A process of encoding data; a process of receiving said encoded data; a process of deinterleaving the received encoded data by a prescribed method; and a process of decoding and reproducing the encoded data. the

In the present invention, the coded data to be sent and the coded data to be received can be configured such that at least one of the following A to F is different between them,

A) presence or absence of interweaving treatment;

B) unit of interleaving processing;

C) the order of interleaving;

D) bit rate;

E) frame rate; and

F) Sampling frequency. the

In the present invention, at least one of A to E of the coded data may also be set according to at least one of the available transmission line frequency band, the data loss mode on the transmission line, or the user's request input via the input unit:

A) presence or absence of interweaving treatment;

B) unit of interleaving processing;

C) the order of interleaving;

D) frame rate; and

E) Sampling frequency. the

In the present invention, it is also possible to notify the communication partner of at least one setting information in the following A～F through call connection processing:

A) presence or absence of interweaving treatment;

B) unit of interleaving processing;

C) the order of interleaving;

D) bit rate;

E) frame rate; and

F) Sampling frequency. the

In the present invention, it is also possible to obtain at least one setting information in the following A-F from the communication partner through call connection processing, so as to decode the received coded data, and the A-F are as follows:

A) presence or absence of interweaving treatment;

B) unit of interleaving processing;

C) the order of interleaving;

D) bit rate;

E) frame rate; and

F) Sampling frequency. the

In the present invention, it is also possible to send the arrival status of the received data including the loss rate of the received coded data and the burst length to the communication counterpart, and

When receiving the arrival status from the communication counterpart, reset at least one of the following A～F of the coded data to be sent according to the arrival status:

A) presence or absence of interweaving treatment;

B) unit of interleaving processing;

C) the order of interleaving;

D) bit rate;

E) frame rate; and

F) Sampling frequency. the

In the present invention, it is also possible to provide a serial number indicating the order of the encoded data, or to transmit the encoded data in a format whose order is known. the

In the present invention, may also include:

a step of generating error correction code data from said coded data in a prescribed unit;

The step of outputting at least a portion of at least one of encoded data and error correction code data;

The step of receiving at least a portion of at least one of encoded data and error correction code data;

When the received coded data is lost, the step of restoring the lost coded data from at least one of normally received coded data and error correction code data; and

The step of decoding encoded data. the

In the present invention, at least one of the following A to D may be different between the error correction code data to be sent and the error correction code data to be received,

A) error correction coding method;

B) presence or absence of error correction code data transmission;

C) the bit rate of the error correction code data; and

D) A unit for generating error correction code data. the

In the present invention, at least one of A to D of the error correction code data may be set according to at least one of the available transmission line frequency band, the data loss mode on the transmission line, or the user's request input through the input unit. item:

A) error correction coding method;

B) presence or absence of error correction code data transmission;

C) the bit rate of the error correction code data; and

D) A unit for generating error correction code data. the

In the present invention, at least one setting information in the following A to D can also be notified to the communication partner through call connection processing:

A) error correction coding method;

B) presence or absence of error correction code data transmission;

C) the bit rate of the error correction code data; and

D) A unit for generating error correction code data. the

In the present invention, it is also possible to obtain at least one setting information in the following A-D of the communication partner through call connection processing, so as to restore the coded data at the time of loss, and the A-D are as follows:

A) error correction coding method;

B) presence or absence of error correction code data transmission;

C) the bit rate of the error correction code data; and

D) A unit for generating error correction code data. the

In the present invention, it is also possible to send the arrival status of the received data including loss rate and burst length to the communication counterpart, and

When receiving the arrival status, at least one of the following A to D of the error correction code data is reset according to the arrival status:

A) error correction coding method;

B) presence or absence of error correction code data transmission;

C) the bit rate of the error correction code data; and

D) A unit for generating error correction code data. the

In the present invention, it is also possible to notify the corresponding relationship of the identifiers used to identify the coded data and the error correction code data to the communication counterpart, and

The corresponding identifier is provided for the coded data and the error correction code data and sent. the

Description of drawings

Fig. 1 is the schematic diagram of the system structure of the first and second embodiment of the present invention;

Fig. 2 is the structural representation of the two-way communication device of the first embodiment of the present invention;

Fig. 3 is a figure for explaining the processing of the two-way communication device of the first embodiment of the present invention;

Fig. 4 is the structural representation of the two-way communication device of the second embodiment of the present invention;

Fig. 5 is a figure for explaining the processing of the two-way communication device of the second embodiment of the present invention;

Fig. 6 is a schematic diagram of a modified example of the two-way communication device of the second embodiment of the present invention. the

Detailed ways

(first embodiment)

Preferred embodiments of the present invention will be described. The communication system of the first embodiment of the present invention includes: a unit (203) for capturing media signals; an encoding unit (204) for encoding captured media data; an interleaving processing unit (206); a sending unit (207) for sending to a transmission line ); a receiving unit (208) for receiving from a transmission line; a deinterleaving processing unit (209); a decoding unit (211) for decoding encoded data; and a unit (212) for replaying decoded media data, also including : a unit (202) that performs encoding setting in the encoding unit (204), interleaving processing setting in the interleaving processing unit (206), and transmission setting in the transmitting unit (207); and notifies the encoding setting, A unit (202) of at least one of the interleaving processing setting, the sending setting, and the reception status of encoded data, the communication system encodes the captured media signal and then sends it, and decodes the encoded data after receiving it and replay. the

Fig. 1 is a schematic diagram of the system structure of the first embodiment of the present invention. As shown in FIG. 1, the system of the first embodiment of the present invention includes a two-way communication device and a transmission line, and may also include a call connection server. In this embodiment, the two-way communication device 101, the two-way communication device 103, and the call connection server device 104 are connected to the transmission line 102 as an IP (Internet Protocol, Internet Protocol) network. The two-way communication devices 101 and 103 are terminals for sending and receiving coded data and call connection data through packet switching, and the call connection server device 104 is, for example, processing call connection data based on SIP (Session Initiation Protocol)/SDP (Session Description Protocol) A terminal that establishes communication between the two-way communication device 101 and the two-way communication device 103 (for SIP/SDP, refer to Non-Patent Documents 1 and 2). In the case where the two-way communication device 101 and the two-way communication device 103 know in advance the IP address of the other party's device, the receiving port number of the encoded data, and the encoding method or encoding option (option) of the media to be sent and received, there is no need to call the connection server device 104. the

FIG. 2 is a schematic structural diagram of the two-way communication devices 101 and 103 in FIG. 1 . The two-way communication devices 101, 103 of Fig. 2 have the same structure. Only one two-way communication device (reference number 201) is shown in FIG. 2 . As shown in FIG. 2, the two-way communication device 201 includes a setting and call connection processing unit 202, a media signal acquisition unit 203, a media encoding unit 204, a packetization processing unit 205, an interleaver 206, a sending unit 207, a receiving unit 208, A deinterleaver 209 , a payload extraction unit 210 , a media decoding unit 211 , and a playback unit 212 . the

For example, an image or sound signal from a camera not shown in the figure, a loudspeaker, etc. is captured after digital-to-analog conversion in the media signal capture unit 203 . Based on the two-way communication device set in the setting and call connection processing unit 202 and notified to the communication partner, or predetermined

·Frame rate,

·Sampling frequency,

• Quantize the number of bits to do the conversion. the

The media encoding unit 204 encodes the captured media data based on the encoding setting set in the setting/call connection processing unit 202 and notified to the two-way communication device of the communication partner, or Scheduled

·Encoding,

· Bit rate,

For example, the presence or absence of data segmentation or resynchronization flag in the MPEG-4 encoding method, or encoding options such as RVLC (Revers Variable Length Code, reversible long code), etc. the

The packetization processing unit 205 gives the coded coded data a serial number that can identify the sequence of the coded data and groups them according to the media data unit and payload format set in the setting and call connection processing unit 202 or predetermined. change. the

Here, for example, if packetization based on RTP (Real-Time Transport Protocol, Real-time Transport Protocol) is used, the sequence number contained in the RTP header can be used to represent the sequence of encoded data (for RTP, refer to Non-Patent Document 3). the

The packetized coded data is output from the transmission unit 207 to the transmission line 214 toward a destination address or port notified by the bidirectional communication device of the communication partner or to a predetermined destination. the

The two-way communication device 201 receives packets from the transmission line 214 through the receiving unit 208 at a port specified by the setting/call connection processing unit 202 or at a predetermined port. The deinterleaver 209 rearranges interleaved packets based on the sequence numbers assigned to the packets. the

The payload extraction unit 210 extracts coded data from the packets rearranged by the deinterleaver 209, and the media decoding unit 211 based on the information obtained by the setting and call connection processing unit 202 and notified by the two-way communication device of the communication partner, or predetermined

· Media encoding method,

· Encoding option to decode the extracted encoded data. the

In addition, it is based on the information obtained by the setting and call connection processing unit 202 and notified by the two-way communication device of the communication partner, or predetermined

·Frame rate,

·Sampling frequency,

• The number of qubits to replay the packet. the

FIG. 3 is an explanatory diagram for explaining the processing of the interleaver 206 and the deinterleaver 209 of this embodiment. The content of processing in interleaver 206 and deinterleaver 209 in FIG. 2 will be described with reference to FIG. 3 . the

As shown in FIG. 3 , an input stream 301 composed of coded data is interleaved by an interleaver 302 into, for example, a transmission stream 303 . The interleaving processing unit at this time is "9", and the interleaving processing is performed for every 9 coded data. A corresponding processing delay is thus generated. The unit of interleaving processing and the order of interleaving shown here are merely examples, and the processing unit and order are not limited to these set values. The transmission stream 303 is transmitted to the communication partner via the transmission line 304 . the

When a burst of data error or loss occurs on the transmission line 304 with respect to the transmission stream, the communication counterpart will receive a stream such as the reception stream 305 . Here, the coded data 7, 2, and 5 constituting the received stream 305 indicate that a data error or loss has occurred on the transmission line. the

The deinterleaver 306 rearranges the coded data into the original order to obtain the deinterleaved stream 307 . As a result, bursts of data errors/losses are dispersed, so that deterioration of media quality can be suppressed to be small compared to continuous data errors/losses. the

This is especially effective for audio-coded data, by dispersing the long-term deterioration of the sound quality caused by successive data errors/losses, making it less perceptible to the ear. the

Here, it is more effective if the interleave processing unit is set to be longer than the burst length of data error or loss occurring on the transmission line. That is, by increasing the interleave processing unit, it is possible to increase the width of the burst length that can cope with data errors or loss on the transmission line. However, if the interleave processing unit is increased, the processing delay will be increased. Therefore, when the quality required for the media to be reproduced differs from each other in a certain pair of two-way communication devices, delay can be reduced by setting an interleave processing size corresponding to the quality required by each communication partner. the

For example, in FIG. 1 , when the requirements for the quality of the media played back in the two-way communication device 101 are high, and the requirements for the quality of the media played back in the two-way communication device 103 are low, in the two-way communication device 101 Reduce the interleaving processing unit or do not perform the interleaving processing, and set the interleaving processing unit of the burst length that can cope with the data error/loss on the transmission line 102 in the bidirectional communication device 103, thereby meeting each other's media playback quality requirements, And can reduce the round-trip delay. the

In addition, it is best to change the setting of the interleaving processing unit according to the transmission line. For this reason, in

The case where different transmission lines are used for the uplink and downlink, or

· When the uplink and downlink have different data error/loss characteristics, the quality can be stabilized and the delay can be reduced by making the interleave processing unit asymmetric. the

Similarly, in order to reduce the delay, the frame rate of the captured image or the sampling frequency of the captured sound can also be increased in the media signal capture unit 203 . the

When the frame rate of the image is increased, the delay can be reduced although the image quality of each frame reproduced by the two-way communication device of the communication partner decreases if the bit stream does not change. the

In addition, for example, in the case of encoding sound with AAC, if the sampling frequency of the sound is increased, the number of encoded data units per unit of time will increase, thereby increasing overhead. Similarly, if the bit rate does not change, the allocation The bit rate for the medium is lowered, so that the sound quality of the playback on the communication partner's two-way communication device is lowered, but the delay can be reduced. the

Therefore, similar to the interleave processing unit, it is set by the setting and call connection processing unit 202 according to the quality requirements of the communication partner.

The frame rate of the image,

The sampling frequency of the sound,

By notifying these items to the two-way communication device of the communication partner, communication can be performed with a setting in which quality and delay requirements are compromised. the

In addition, by the receiving status transceiver unit 213, for example, using SR (Sender Report, sender report) or RR (ReceiverReport, receiver report) of PTCP (RTP Control Protocol, RTP control protocol), the encoded data received at the receiving unit 208

Data error/loss rate,

· Error/loss burst length

Statistical information, such as statistics, is notified to the two-way communication device of the communication partner, so that the interleaving processing unit can be changed during communication (for RTCP, refer to Non-Patent Document 3). the

Here, the change of the interleaving processing unit can be easily changed on the transmitting side by breaking the interleaving processing unit, and on the receiving side, the size of the interleaving processing unit can be increased by increasing the buffer of the interleaver. . However, when reducing the size of the interleave processing unit on the receiving side, it is necessary to change the size so as to suppress interruption of reproduced audio as small as possible. Therefore, if the last part of the buffer of the deinterleaver 209 is empty, it will be trimmed from that part, and if the buffer is filled with received data, it will be trimmed from the received data that is silent or close to the level of silent, thereby achieving Change the buffer size to suppress sound interruption. the

As a result, it is possible to effectively stabilize the quality of media and reduce delay even in transmission lines where data errors and loss patterns fluctuate. the

When a data error occurs on the transmission line 214, but neither data loss nor packet data order interleaving occurs, if the interleaving order of the interleaver 206, Alternatively, if the interleaving order is set in advance, it is not necessary to assign a sequence number to the packetization processing unit 205 . the

Also, in an encoding method such as AMR, interleaving/deinterleaving processing is sometimes included in the mechanism of payload format processing, and in this case the interleaver 206 and the deinterleaver 209 are unnecessary. In this case, the interleaving processing unit may be described as "interleaving=" in the a=fmtp attribute of the SDP, and notified to the communication partner. the

In addition, regarding the setting and call connection processing unit 202, the media signal capturing unit 203, the media encoding unit 204, the packetization processing unit 205, the interleaver 206, the transmitting unit 207, the receiving unit 208, and the deinterleaver in the two-way communication device 201, 209 , the payload extraction unit 210 , the media decoding unit 211 , and the playback unit 212 can of course realize their respective functions and processes through program control executed by the computer constituting the two-way communication device 201 . the

(second embodiment)

The communication system of the second embodiment of the present invention includes: a unit (203) for capturing media signals; a unit (204) for encoding the captured media data; a unit (403) for generating error correction code data from the encoded data; A sending unit (207) for sending; a receiving unit (208) for receiving from a transmission line; a unit (404) for restoring the data using an error correction code when the received coded data is wrong or lost; decoding the coded data The unit (211); and the unit (212) to playback the decoded media data, also includes: to the encoding setting in the encoding unit (204), the error correction code data generation processing setting, the sending unit (207 A unit (402) for setting the transmission setting in ); and notifying at least one of the encoding setting, the error correction code data generation processing setting, the transmission setting, and the reception status of the encoded data unit (402). The captured media signal is encoded and sent. The received encoded data is decoded and reproduced. the

In the following, only the differences between the second embodiment of the present invention and the above-mentioned first embodiment will be described in detail with reference to the accompanying drawings. the

Fig. 4 is a schematic structural diagram of a two-way communication device (101, 103 in Fig. 1 ) according to a second embodiment of the present invention. As shown in FIG. 4 , the bidirectional communication device 401 of this embodiment has an error correction coding unit 403 and an error correction decoding unit 404 to respectively replace the interleaver of the bidirectional communication device 201 in the first embodiment shown in FIG. 2 206 and deinterleaver 209. In the following, the difference between this embodiment and the first embodiment shown in FIG. 2 will be mainly described, and the description of the same parts will be appropriately omitted. the

The error correction encoding unit 403 generates error correction code data from the packetized encoded data based on the error correction encoding method and the error correction encoded data generation unit, and outputs it together with the encoded data from the transmission unit 207 to the transmission line 214. The error correction encoding method and the error correction encoding data generation unit are set by the setting and call connection processing unit 402 and notified to the two-way communication device of the communication partner, or are predetermined. Here, in the error correction coding method, such as LDPC (Low Density Parity Check, Low Density Parity Check) method or Reed-Solomon (Reed-Solomon) code, etc. are usually used, which is called FEC (Forward Error Correction, forward error correction) ) code. As for the error correction code, since the prior art is used, its detailed description is omitted here. the

FIG. 5 is an explanatory diagram for explaining the processing of the error correction coding unit 403 and the error correction decoding unit 404 in this embodiment. Next, with reference to FIG. 5 , the processing contents of the error correction coding unit 403 and the error correction decoding unit 404 in this embodiment shown in FIG. 4 will be described. Here, LDPC is taken as an example to describe the error correction coding method, but the error correction coding method is not limited to the LDPC method. the

As shown in FIG. 5 , an error correction code is generated from an input stream 501 composed of coded data by an error correction coding unit 502 , whereby the stream transmitted to the transmission line 504 becomes the transmission stream 503 . Here, the unit for generating the error correction code is "6", that is, the error correction coding is performed on every 6 pieces of coded data, so a corresponding processing delay is generated. the

In addition, in this embodiment, although 3 error correction code data units are generated from 6 coded data units, the number of error correction data for each error correction code generation unit is only an example and is not limited to this number. number. the

The transmission stream 503 is transmitted to the communication partner via the transmission line 504 . the

For example, when the transmission line 504 is a wireless transmission line, and a data error or loss of the transmission stream occurs, a stream such as the reception stream 505 will be received at the communication partner. Here, the coded data 3, 4, and 5 constituting the received stream 505 indicate that a data error or loss has occurred on the transmission line. the

The error correction decoder 506 restores the encoded data 3 , 4 , and 5 in which data errors/losses occurred from the correctly received encoded data and error correction code data. As a result, the effects of errors/losses on the transmission line are eliminated, enabling playback at the original media quality. the

If, for example, packetization based on RTP (Real-Time Transport Protocol, Real-Time Transport Protocol) is used in order to distinguish between media encoded data and error correction code data in the received stream 505, the payload type, SSRC, and SSRC contained in the RTP header (Synchronization Source identifier, synchronization source identifier), or CSRC (Contributing Source identifier, contribution source identifier) is used as an identifier, and different values are set for the encoded data and error correction code data of the media, and then the data type and By notifying the communication partner of the combination of values, the coded data and error correction code data of the medium can be identified in the error correction decoding unit 506 . the

FIG. 6 is a schematic diagram of a modified example of the present embodiment shown in FIG. 4 . As shown in FIG. 6 , in the two-way communication device 601 , media encoded data and error correction code data are respectively transmitted from the media code transmission unit 604 and the error correction code transmission unit 605 through separate sessions or separate transmission lines. In addition, the media code receiving unit 606 and the error correction code receiving unit 607 can respectively receive the two types of encoded data, and the error correction decoding unit 608 can restore the media encoded data affected by errors or losses. At this time, even if there is no identifier for identifying the media coded data and the error correction code data, the type of data can be identified according to the session or transmission line used for transmission and reception. the

In addition, among the error correction encoding methods, there is an encoding method in which only the error correction code data is transmitted without transmitting the encoded data of the medium. In this coding method, media coded data to be transmitted and received between the media code transmitting unit 604 and the media code receiving unit 605 is unnecessary. the

Here, it is effective to prepare a parity check matrix for generating error correction data by the LDPC method in accordance with a data error or loss pattern occurring on the transmission line. Also, the amount of error correction code data relative to the media coded data and the unit for generating the error correction code data are determined by the parity check matrix. the

The generation of error correction code data requires a processing delay corresponding to the generation unit of error correction code data. By increasing the generation unit, the recovery rate from data error/loss can be improved with a smaller data amount of error correction code data, but the delay is increased. Therefore, when the quality required for the media to be reproduced is different in a certain pair of two-way communication devices, delay can be reduced by setting a parity check matrix corresponding to the quality required by each communication partner. the

For example, in FIG. 1 , when the requirements for the quality of the media played back in the two-way communication device 101 are higher, and the requirements for the quality of the media played back in the two-way communication device 103 are lower, the two-way communication device 101 Set a parity check matrix that reduces the error correction code generation unit in , or do not perform error correction code data generation processing. In the two-way communication device 103, the parity check matrix that determines the appropriate error correction code data generation amount and generation unit is set according to the data error/loss pattern on the transmission line 102, thereby meeting mutual requirements for media playback quality. Requirements, and can reduce the round-trip delay. the

In addition, when the transmission frequency band is limited, the trade-off between the stability of the media quality and the quality level can be determined by changing the allocation of the bit rate in the coded and error-correcting code data of the media. This can also be set according to the requirements of the media being played back by the respective two-way communication devices. the

As described above, it is preferable that the parity check matrix is set differently according to the transmission line. Therefore, in the case where the uplink and downlink use different transmission lines, or the uplink and downlink have different data error/loss characteristics, the quality can still be stabilized by making the parity check matrix asymmetric and reduce latency. the

In this embodiment, the frame rate of the captured image and the sampling frequency of the captured sound are the same as those of the first embodiment described above. the

In addition, for example, SR (Sender Report) or RR (Receiver Report) of PTCP (RTP Control Protocol) is used to notify the communication partner of statistical information such as data error/loss rate and error/loss burst length of the encoded data received by the receiving unit 208. A two-way communication device, from which the error/loss characteristics of the transmission line can be grasped. In this way, by notifying the statistical information to the two-way communication device of the communication partner by the setting and call connection processing unit 202,

The error correction coding method can be changed during the communication process,

The check matrix can be changed during the communication process, that is, the bit rate and the generated unit of the error correction code data can be changed,

It is also possible to stop/restart transmission of error correction code data. the

For example, when the state of the transmission line is good and the coded data of the medium can be transmitted and received with a sufficiently good quality, the transmission of the error correction code data is stopped, and when the state of the transmission line deteriorates, the transmission of the error correction code data may be restarted. the

In addition, error correction code data can also be selectively sent, which can effectively stabilize media quality and reduce delay even in transmission lines where data error/loss patterns fluctuate. the

In the bidirectional communication device of this embodiment, the interleaving processing (interleaver 206) and deinterleaving processing (deinterleaver 209) explained in the first embodiment of FIG. 2 may also be combined. the

In addition, regarding the setting and call connection processing unit 402, the media signal acquisition unit 203, the media encoding unit 204, the packetization processing unit 205, the error correction encoding unit 403, the transmission unit 207, the reception unit 208, the correction Of course, the error decoding unit 404 , the payload extraction unit 210 , the media decoding unit 211 , and the playback unit 212 can realize their respective functions and processes through program control executed by a computer constituting the two-way communication device 401 . The two-way communication device 401 constitutes, for example, a voice communication device. the

In addition, regarding the setting and call connection processing unit 602, the media signal acquisition unit 203, the media encoding unit 204, the packetization processing unit 205, the error correction encoding unit 603, the media code transmission unit 604, and the error correction code in the two-way communication device 601, The sending unit 605, the media code receiving unit 606, the error correction code receiving unit 607, the error correction decoding unit 608, the payload extraction unit 210, the media decoding unit 211, and the playback unit 212 can also be configured by the two-way communication device 601 The program control executed in the computer realizes the respective functions and processes. The two-way communication device 601 constitutes, for example, a voice communication device. the

The present invention has been described above with reference to the above-mentioned embodiments, but the present invention is not limited to the structures of the above-mentioned embodiments, and includes various variations and modifications that can be made by those skilled in the art within the scope of the present invention. the

According to the present invention, two-way communication can be realized with as stable quality as possible based on a setting that compromises the media quality, delay, frequency band, and request for delay required by the user. the

In addition, according to the present invention, even in a transmission line with an asymmetrical uplink and downlink, it is possible to realize bidirectional communication that can meet user requirements for quality and delay. the

In addition, according to the present invention, it is possible to realize bidirectional communication in which the stability of quality is controlled in accordance with data errors/losses fluctuating on the transmission line and an increase in processing delay is suppressed. the

Claims (6)

1. A two-way communication method, characterized in that, comprising: a step of encoding the input media; the step of interleaving the coded data in specified units; The step of sending the coded data after interleaving; the step of receiving encoded data; a step of deinterleaving the received coded data by a prescribed method; and the step of decoding and replaying the encoded data, Among them, in the coded data to be transmitted, the unit of interleaving processing is changed so as to meet the quality of the medium required by the communication partner, Whenever the unit of the interleaving process is changed, changing the breakpoint of the interleaving unit at the transmitting side of said coded data, On the reception side of the coded data, the size of the buffer storing the coded data is made variable, the unit size of the interleaving process is increased by increasing the buffer, and the buffering is changed by suppressing interruption of a playback signal. The size of the device is reduced, thereby reducing the unit size of the interleaving process. 2. two-way communication method as claimed in claim 1, is characterized in that, said coded data includes voice coded data, On the receiving side of the coded data, if there is a free part in the buffer, the pruning is performed from the free part, and if the buffer is filled with the received coded data, then from the silent or near silent level reception The data starts to be cut, thereby suppressing the interruption of the sound. 3. A two-way communication device, characterized in that, comprising: a coding unit, which codes the input media and outputs it; an interleaving unit, which interleaves the coded data from the coding unit in a specified unit; a sending unit, for sending the coded data rearranged by the interleaving unit; a receiving unit, configured to receive the coded data; A deinterleaving unit rearranges the interleaved coded data received by the receiving unit into original coded data; a decoding unit that decodes the encoded data from the deinterleaving unit and outputs the decoded data; and a replay unit that replays the decoded data from said decoding unit, Among them, in the coded data to be transmitted, the unit of interleaving processing is changed so as to meet the quality of the medium required by the communication partner, As a unit for changing the unit of the interleaving process, there is a unit for changing a break point of the interleaving unit on the transmission side of the coded data, On the reception side of the coded data, there is a means for making the size of a buffer storing the coded data variable, increasing the unit size of the interleaving process by increasing the size of the buffer, and suppressing interruption of a reproduced signal to change the size of the buffer, thereby reducing the unit size of the interleaving process. 4. The two-way communication device according to claim 3, characterized by comprising a unit which provides a serial number indicating the sequence of the encoded data, or transmits the encoded data in a format in which the sequence of the encoded data is known. 5. A two-way communication system comprising at least two communication devices communicating via a transmission line, the two-way communication system is characterized in that, At least one of the two communication devices includes: an encoding unit, for encoding the input media; an interleaving unit, which interleaves the coded data in a specified unit; and a sending unit, outputting the interleaved coded data to the transmission line; At least one other of the two communication devices includes: a receiving unit that receives the encoded data from the transmission line; The de-interleaving unit rearranges the received coded data that has been interleaved into the original coded data by a prescribed method; a decoding unit that decodes the encoded data and outputs the decoded data; and a replay unit, replaying the decoded data, Wherein, in the coded data sent from one of the two communication devices to the other communication device, the unit of interleaving processing is changed to meet the quality of the media required by the communication partner, As a unit for changing the unit of the interleaving process, there is a unit for changing a break point of the interleaving unit on the transmission side of the coded data, On the reception side of the coded data, there is a means for making the size of a buffer storing the coded data variable, increasing the unit size of the interleaving process by increasing the size of the buffer, and suppressing interruption of a reproduced signal to change the size of the buffer, thereby reducing the unit size of the interleaving process 6. The two-way communication system of claim 5, wherein: said coded data includes voice coded data, On the receiving side of the coded data, there is a unit that cuts down from the free part if there is a free part in the buffer, and if the buffer is filled with received coded data, the unit Cutting is performed from received data at or near the level of silence, thereby suppressing interruption of sound. the

Images