JP3927443B2 - Moving picture transmission / reception system and moving picture transmission / reception method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving image transmission / reception system in which image data encoded using a compression algorithm is transmitted on a transmitting side through a packet transmission network, and a receiving side receiving the same decodes / decompresses and reproduces an original image signal, and The present invention relates to a transmission / reception method, and more particularly, to encoding and decoding processing in parallel processing by screen division.
[0002]
[Prior art]
When transmitting moving images over a network, a code for reducing the total amount of information of moving image data using some data compression algorithm is necessary to perform efficient and high-quality image transmission / reception within a limited communication band. It is widely performed to transmit after performing the conversion processing. In this case, an encoding processing circuit that compresses the image data on the transmission side and a decoding processing circuit that decompresses the compressed data on the reception side are required. However, with the current semiconductor technology, real-time compression / decompression processing is possible up to HDTV quality with about 1000 scanning lines within the range of hardware that can be made into one chip as an LSI. However, in order to transmit a high-resolution moving image such as a movie as a digital image over a network, it is necessary to transmit an image having a very high resolution exceeding 2000 × 2000 pixels in width and width. The required encoding transmission rate may exceed 100 Mbps.
[0003]
In order to transmit such an extremely high resolution moving image, the processing capability is insufficient only by using one image compression / encoding device and image expansion / decoding device. Prepare and process in parallel. For this parallel processing, it is necessary to multiplex the image data generated by a plurality of encoders on the transmission side, and distribute it to a plurality of decoders on the reception side.
[0004]
In a conventional moving image transmission / reception system, image data is transmitted through one transmission channel. Multiple data is multiplexed and transmitted in a single channel on the transmission side. A conventional configuration example is shown in FIG. On the transmission side, the image signal is divided by the frame division circuit 1A of the image data encoding unit 1, each region image is encoded by a plurality of encoders 1B, and the processing result is serially processed by the multiplexing circuit 1C as one piece of data. The data is converted into a series and stored as code data in the data storage circuit 2. At this time, since the encoded data output from the plurality of encoders on the transmission side has different data lengths depending on the contents of the divided images, it is serialized by embedding a delimiter identification code, or FIG. As described above, a method is employed in which a header storing the number of bytes of each data series to be separated is inserted at the head of the data series.
[0005]
When transmitting an image signal, it is necessary to consider transmission errors in the transmission path. In a moving image transmission / reception system using an IP packet network, UDP and TCP are mainly used as data transmission protocols. UDP supports multicast, but since the transmission protocol itself does not have an error correction function, if a transmission error occurs, the data is discarded and the decoding process is interrupted until the next moving image frame, or the application side It is necessary to perform error detection / correction and data retransmission processing. When an error detection and data retransmission function is required for the transmission protocol, data transmission by TCP is used. In this case, the error detection CRC is calculated by the error detection code addition circuit 3A in the preceding stage of the IP transmission unit 3, and is transmitted together with the encoded data by the IP transmission circuit 3B.
[0006]
In the case of data transmission using the TCP protocol, the TCP / IP packet data sequence received by the IP receiving circuit 4A of the IP receiving unit 4 is checked by the error detection / retransmission control circuit 4B for the presence of a transmission error. Requests the transmission side to retransmit the data packet for error correction. The image data decoding unit 5 re-decomposes the received image data series for each data series to be processed in parallel by the distribution circuit 5A using the previously inserted identification code or header, and inputs it to each decoder 5B. To do.
[0007]
Each decoder 5B performs decompression processing in parallel from the distributed encoded data. In the image buffer 5C, the moving image data expanded by each decoder is integrated into one frame image, and one complete image frame is formed and output. During the frame period, the decoding and image integration processing of the next frame is repeated, and continuous moving image signals are sequentially output.
[0008]
[Problems to be solved by the invention]
As described above, in order to encode a large-capacity image, parallel processing is performed by a plurality of encoding circuits. Conventionally, a data sequence of the plurality of encoders is transmitted so as to be transmitted through a single channel transmission path. Multiplexing into one data series was performed.
[0009]
In the image transmission / reception system using the image encoding / decoding processing by the parallel processing, high-speed image data multiplexing / separation processing is required. When a high-resolution image such as a movie is encoded and transmitted, a data transmission rate of 200 Mbps or more may be required, but detection of this identification code is a bottleneck in processing speed.
[0010]
When the header is used, the next header is not transmitted until one image frame has been read. However, since the main part of the next image data series is read based on this header information, there is one transmission path there. A time gap due to delay or the like occurs, leading to a decrease in throughput.
[0011]
In addition, when considering digital distribution of movies, data of 2 hours or more is continuously flowed, and the total amount of data is 1 Tbit or more. Therefore, an error that cannot be detected by normal TCP / IP transmission occurs. The probability of occurrence cannot be ignored. If a transmission error occurs in the identification code or the header portion, the data is not accurately distributed to each decoder, which causes a decisive malfunction in the encoding operation.
[0012]
An object of the present invention is to provide a large-capacity and high-resolution moving image transmission / reception system using a packet transmission network such as TCP / IP by parallel image transmission / reception processing using a plurality of encoders and decoders. This process is simplified to enable high-speed transmission of an encoded data sequence and transmission of an ultra-high resolution moving image.
[0013]
Another object of the present invention is to improve the tolerance of header transmission errors, so that in the transmission of very large capacity moving image data, there is much more error than in the case of transmission over a normal packet transmission network such as TCP / IP. It is possible to transmit a moving image with a small amount of motion.
[0014]
[Means for Solving the Problems]
In the moving image transmission / reception system according to the present invention, information required for decoding by data distribution / parallel processing on the reception side is multiplexed when multiplexing encoded data sequences processed in parallel by an encoding unit on a transmission path such as IP. The described header is created, and communication between sockets separate from the encoded data series is performed. In addition to the error correction / retransmission function based on TCP / IP and the like regarding the header, further addition of a separate error correction code and error detection / correction on the receiving side, or multiplexing transmission of duplicate copies of the header and majority determination on the receiving side I do.
[0015]
More specifically, in the present invention, the image data encoding unit divides the input image into a plurality of partial areas, compresses and encodes them in parallel, and multiplexes the data length of each encoder. A means for generating a table / header as content, a header separation / redundancy unit, separating a header and attaching an error correction code thereto, or copying a plurality of headers themselves, and a TCP / IP transmission unit A plurality of compressed image data obtained by the image data encoding unit is multiplexed and serialized, and the redundant header information that has been separated earlier is transmitted in a separate data path using another inter-socket communication. Means for transmitting toward the IP transmission path, and means for receiving and buffering the image data series received from the IP transmission path and the redundant header in parallel in the TCP / IP receiver, and the header In accordance with the means for detecting and correcting errors in the received header in the detection unit and the contents of the header effective in the image data decoding unit, the data is distributed to a plurality of decoders, When it is determined that the received header is not valid, the decoding process is stopped until the next valid header is received, and the latest image that has been successfully decoded is output. And a moving image transmission / reception system and a moving image transmission / reception method described below.
[0016]
(Video transmission / reception system)
(1) The moving image transmitting apparatus compresses and accumulates moving image signals for each frame, divides them and transmits them through a packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signals. In a moving image transmission / reception system that decodes and expands the moving image signal for reproduction,
The moving picture transmitting apparatus divides an image of each frame into a plurality of areas, independently encodes and compresses the data, collects the encoded data group into one stream, and stores data for each area. Means for calculating the header with the length as a table, and means for transmitting the header stream and the encoded data stream for packet transmission in parallel as separate streams ,
The moving image receiving apparatus receives the image data series and the redundant header in parallel, distributes the data to a plurality of decoders according to the contents of the header, and decodes the image data in parallel by the decoder. -It is characterized by an expansion process.
[0017]
(2) The moving image transmitting apparatus compresses and accumulates moving image signals for each frame, divides them and transmits them through a packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signals. In a moving image transmission / reception system that decodes and expands the moving image signal for reproduction,
The moving image transmitting apparatus divides the image of each frame into a plurality of regions, independently performs encoding / compression, collects the encoded data group into one stream, and sets the data length for each region. The header stream and the encoded data stream for packet transmission are transmitted in parallel as separate streams ,
The moving image receiving apparatus includes: means for receiving and buffering an image data sequence received through a network and a redundant header in parallel; means for performing error detection and error correction of a received header; and effective among the redundant headers If the header is determined to be a valid header, the data is distributed to a plurality of decoders, and the decoder decodes the image data in parallel. If it is determined that the received header is not valid, the decoder processing is And a means for stopping the operation of the decoder until a header is received and continuously outputting a recent image that has been successfully decoded.
[0018]
(3) The moving image transmitting apparatus transmits the data length table a plurality of times, and the moving image receiving apparatus takes a majority decision from the reception result, and based on the information of each data length of the result, the received image data string is obtained. It is characterized by having means for decomposing and performing image expansion processing in parallel using a decoding circuit corresponding to each data column.
[0019]
(4) The moving picture transmitting apparatus transmits the data length table with an error correction code added thereto, and the moving picture receiving apparatus obtains information on each data length from the header information of the error detection / correction result of the reception result. And a means for decomposing the received image data sequence and performing image decompression processing in parallel using a decoding circuit corresponding to the sequence for each data.
[0020]
(5) The moving image transmitting apparatus and moving image receiving apparatus are characterized in that moving image data and a header are transmitted and received as a TCP / IP stream, and include TCP error detection and retransmission control means.
[0021]
(Video transmission / reception method)
(6) The moving image transmitting apparatus compresses and stores the moving image signal for each frame, divides it and transmits it through a packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signal. In the moving picture transmission / reception method for decoding and expanding the moving picture signal and reproducing it,
The moving picture transmitting apparatus divides an image of each frame into a plurality of areas, independently performs encoding / compression, and collects the encoded data group into one stream to obtain data for each area. A process of calculating the header using the length as a table, and a process of transmitting the header stream and the encoded data stream to be transmitted in parallel as separate streams in parallel ,
The moving image receiving apparatus receives the image data series and the redundant header in parallel, distributes the data to a plurality of decoders according to the contents of the header, and decodes the image data in parallel by the decoder. -It is characterized by an expansion process.
[0022]
(7) The moving image transmitting apparatus compresses and accumulates the moving image signal for each frame, divides it and transmits it through the packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signal. In the moving picture transmission / reception method for decoding and expanding the moving picture signal and reproducing it,
The moving image transmitting apparatus divides the image of each frame into a plurality of regions, independently performs encoding / compression, collects the encoded data group into one stream, and sets the data length for each region. The header stream and the encoded data stream for packet transmission are transmitted in parallel as separate streams,
The moving image receiving apparatus includes: a process of receiving and buffering an image data sequence received through a network and a redundant header in parallel; a process of performing error detection and error correction of a received header; If the header is determined to be a valid header, the data is distributed to a plurality of decoders, and the decoder decodes the image data in parallel. If it is determined that the received header is not valid, the decoder processing is And the operation of the decoder is stopped until the header is received, and the latest image that has been successfully decoded is continuously output.
[0023]
(8) The moving image transmitting apparatus transmits the data length table a plurality of times, and the moving image receiving apparatus takes a majority decision from the reception result, and based on the information of each data length of the result, the received image data string is obtained. It is characterized by having a process of decomposing and performing image expansion processing in parallel using a decoding circuit corresponding to each data column.
[0024]
(9) The moving picture transmitting apparatus adds an error correction code to a data length table and transmits the data length table, and the moving picture receiving apparatus obtains information on each data length from the header information of the error detection / correction result of the reception result. In addition, the method includes a step of decomposing the received image data sequence and performing image expansion processing in parallel using a decoding circuit corresponding to the sequence for each data.
[0025]
(10) The moving image transmitting apparatus and moving image receiving apparatus transmit and receive moving image data and a header as a TCP / IP stream, and include a TCP error detection and retransmission control process.
[0026]
(Function)
In the moving image transmission / reception system and the moving image transmission / reception method according to the present invention, data sequences transmitted from a plurality of encoders are transmitted as they are through one transmission channel, and how much data is received by one reception operation on the receiving side. Whether the data is read or not is determined by referring to the header information sent through different channels, so that the same number of bytes can be written / read on the sending side and the receiving side, the buffer usage efficiency is high, and the data transmission throughput Will improve. At this time, since the header is not multiplexed in the data series, the header information can be read out simultaneously by the receiving circuit while the previous data series is received and transferred to the decoder. Therefore, the encoded data sequence can be transferred without interruption, and the throughput is improved. This tendency is particularly conspicuous when the TCP / IP transmission / reception circuit is configured by hardware and the receiving side error detection / correction / decoder control is performed by software. Further, since error detection is performed again for an error in header information that has passed without being detected by TCP / IP, the possibility of malfunction of the decoder on the receiving side can be greatly reduced.
[0027]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of a moving image transmission / reception system, a transmission apparatus, and a reception apparatus according to the present invention will be described below in detail with reference to FIGS.
[0028]
FIG. 1 shows a moving image data transmitting apparatus. Input moving image data is first divided into a plurality of areas for each frame by an area dividing circuit 11A of an image data encoding unit 11. As a division method, an image frame may be divided vertically and horizontally, or may be divided into strips vertically or horizontally.
[0029]
The divided image data are encoded in parallel by independent encoding circuits 11B, and are collected into one data series by the multiplexing circuit 11C. At this time, as in the example of FIG. 4, in the header generation circuit 11D, a table containing the number of bytes of encoded data generated from each encoding circuit is created and attached to the head of the data series, and then the hard disk or the like It is stored in the data storage circuit 12.
[0030]
When transmitting the encoded data sequence stored in the data storage circuit 12, the header separation / redundancy unit 13 reads out the encoded data stored in the data storage circuit 12, and the data separation circuit 13A reads the encoded data. The header information added to the head is extracted first.
[0031]
The header information redundancy circuit 13B makes the separated and extracted header information redundant. For this redundancy, an error correction code is added in order to increase resistance to transmission errors, or the header itself is transmitted a plurality of times, and the method of transmitting the odd number of 3 times or more so that the majority decision can be easily made on the receiving side. is there. In this case, the data amount of the header to be transmitted is three times or more. However, since the data amount is much smaller than that of the image data series, there is no problem in terms of transmission efficiency.
[0032]
The TCP / IP transmission unit 14 generates an image data sequence in which the redundant header and the encoded data are multiplexed in the socket multiplexing transmission circuit 14A, and the TCP error detection / retransmission control circuit 14B generates the image data sequence. Then, control is performed to transmit in parallel by communication between individual sockets using different socket numbers, and transmission is performed from the IP transmission circuit 14C.
[0033]
At the time of transmission, each group of IP packets obtained by dividing the header and the image data series is assigned a sequence number in accordance with the TCP protocol so that packet loss on the transmission path can be detected. For bit errors, CRC-32 is calculated and assigned to each IP packet.
[0034]
FIG. 2 shows a moving image data receiving apparatus. The TCP / IP receiving unit 15 performs CRC-32 matching determination on an IP packet in the TCP error detection / retransmission control circuit 15B for the image data series received by the IP receiving circuit 15A. And verify the TCP sequence number. Many transmission errors can be detected by these two checks. When an error is detected, retransmission is requested until the transmission side receives the error correctly. When correctly transmitted, the socket separation circuit 15C performs the socket separation and temporarily stores it in the socket buffer 15D.
[0035]
The header error detection unit 16 performs error detection and correction on the reception header made redundant by the header error detection and correction circuit 16A. For example, when the transmission side uses the method of sending the same header three times, error detection is realized by a simple majority circuit. Block transfer of data from the TCP / IP receiving unit to the image data decoding unit 17 based on the contents of the header recognized as valid by the header error detection unit 16, that is, information on the data length to be distributed to each decoding circuit I do.
[0036]
In the image data decoding unit 17, the distribution circuit 17A distributes the encoded data to each decoding circuit 17B. The data sent to the decoding circuit 17B is decompressed and reproduced into image blocks divided into strips. Is sent to the image integration buffer 17C, where it is reconstructed into an image signal of one frame and output.
[0037]
In the above configuration, received image signals without transmission errors are sequentially decoded by a plurality of decoding circuits operating in parallel. The decoded image has a one-to-one correspondence with the region image divided by the region dividing circuit 11A of the image data encoding unit 11 on the transmission side. Whether or not the image is similar to that of the image depends on the compression / decompression algorithm of the image used in the encoder and decoder.
[0038]
For example, when a lossless compression / decompression algorithm typified by differential encoding for encoding a difference between pixels is used, the same image signal is transmitted and received between the transmission side and the reception side as long as there is no transmission error.
[0039]
In general, an irreversible compression algorithm represented by JPEG having a high compression rate is widely used. However, a signal decoded on the receiving side includes some coding loss and image quality deteriorates. If a transmission error occurs in the encoded data sequence and the decoding process becomes abnormal, the encoder detects the error and discards the decoded data together with the processing results of the other decoders. Display the frame image again.
[0040]
【The invention's effect】
In the moving image transmission / reception system and the moving image transmission / reception method according to the present invention, each inter-socket communication identified by the socket signal has a buffer. Normally, the length of the image data sequence is larger than this buffer, so that the socket reception buffer is emptied by the amount transferred to the image data decoding unit on the reception side, and the subsequent packet arrives from the transmission side. become. On the other hand, since the capacity of the header is smaller than that of the buffer, when the end of the encoded data sequence of the previous frame is transmitted and the header information of the next frame is transmitted, The socket reception buffer is reached and held, so that it can be used immediately on the receiving side.
[0041]
The image data decoding unit on the receiving side can immediately analyze the next header when the entire encoded data sequence of the previous frame is decoded, so that the encoded data sequence of the next frame can be read immediately. Can be done. Compared to the case where the encoded data of the previous frame, the header of the next frame, and the encoded data of the next frame are sequentially transmitted, the variable length encoded data sequence is not affected by the delay of the transmission path. Can be transmitted at high speed.
[0042]
Further, by combining header redundancy and error detection / correction, the decoder operation can be greatly stabilized by slightly increasing the total data transmission amount. Regarding the header, the frequency of error occurrence can be extremely low. Both the encoded data series and the header perform a further error correction in addition to the TCP / IP error detection / retransmission function, thereby greatly increasing the rate of header transmission errors that can cause significant malfunctions on the decoding side. Can be reduced.
[0043]
Since the encoded data series is not protected by data more than the error correction function of TCP / IP, a transmission error may be mixed and the decoding process may become abnormal. In this case, the encoder By detecting an error and displaying the previous frame image in the image integration buffer again, it is possible to avoid noticeable image degradation. Also, since the header is protected, the boundary of the next encoded data sequence sent from the transmission side is not lost.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a moving image data transmission apparatus according to an embodiment of the present invention.
FIG. 2 is a configuration diagram of a moving image data receiving apparatus showing an embodiment of the present invention.
FIG. 3 is a configuration diagram of a conventional moving image data transmission apparatus.
FIG. 4 is an explanatory diagram of region division of a divided moving image frame, encoding thereof, and an encoded data stream.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1, 11 ... Image data encoding part 2, 12 ... Data storage circuit 3 ... IP transmission part 4 ... IP reception part 5 ... Image data decoding part 13 ... Header separation and redundancy part 14 ... TCP / IP transmission part 15 ... TCP / IP receiver 16 ... header error detector 17 ... image data decoder

Claims (10)

The moving image transmitting apparatus compresses and accumulates moving image signals for each frame, divides them and transmits them through a packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signals, In a video transmission / reception system that decodes and decompresses an image signal for playback,
The moving picture transmitting apparatus divides an image of each frame into a plurality of areas, independently encodes and compresses the data, collects the encoded data group into one stream, and stores data for each area. Means for calculating the header with the length as a table, and means for transmitting the header stream and the encoded data stream for packet transmission in parallel as separate streams ,
The moving image receiving apparatus receives the image data series and the redundant header in parallel, distributes the data to a plurality of decoders according to the contents of the header, and decodes the image data in parallel by the decoder. A moving image transmission / reception system characterized by performing decompression processing. The moving image transmitting apparatus compresses and accumulates moving image signals for each frame, divides them and transmits them through a packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signals, In a video transmission / reception system that decodes and decompresses an image signal for playback,
The moving image transmitting apparatus divides the image of each frame into a plurality of regions, independently performs encoding / compression, collects the encoded data group into one stream, and sets the data length for each region. And the header stream and the encoded data stream for packet transmission are transmitted in parallel as separate streams ,
The moving image receiving apparatus includes: means for receiving and buffering an image data sequence received through a network and a redundant header in parallel; means for performing error detection and error correction of a received header; and effective among the redundant headers If the header is determined to be a valid header, the data is distributed to a plurality of decoders, and the decoder decodes the image data in parallel. If it is determined that the received header is not valid, the decoder processing is Means for stopping the operation of the decoder until a header is received and continuing to output the latest image that has been successfully decoded;
A moving image transmission / reception system characterized by the above.   The moving image transmitting device transmits a data length table a plurality of times, the moving image receiving device takes a majority decision from the reception result, and based on the information of each data length of the result, decomposes the received image data sequence, 3. The moving image transmission / reception system according to claim 1, further comprising means for performing image expansion processing in parallel using a decoding circuit corresponding to each data column.   The moving image transmitting apparatus transmits an error correction code added to a data length table, and the moving image receiving apparatus receives from the header information of the error detection / correction result of the reception result based on the information of each data length. 3. The moving image transmission / reception system according to claim 1, further comprising means for decomposing the image data sequence and performing parallel image decompression processing using a decoding circuit corresponding to each data sequence.   5. The moving image transmitting apparatus and moving image receiving apparatus each transmit / receive moving image data and a header as a TCP / IP stream, and include a TCP error detection / retransmission control unit. The moving image transmission / reception system described in 1. The moving image transmitting apparatus compresses and accumulates moving image signals for each frame, divides them and transmits them through a packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signals, In a moving image transmission / reception method for decoding and expanding and reproducing an image signal,
The moving picture transmitting apparatus divides an image of each frame into a plurality of areas, independently performs encoding / compression, and collects the encoded data group into one stream to obtain data for each area. A process of calculating the header using the length as a table, and a process of transmitting the header stream and the encoded data stream to be transmitted in parallel as separate streams in parallel ,
The moving image receiving apparatus receives the image data series and the redundant header in parallel, distributes the data to a plurality of decoders according to the contents of the header, and decodes the image data in parallel by the decoder. A moving image transmission / reception method characterized by performing decompression processing. The moving image transmitting apparatus compresses and accumulates moving image signals for each frame, divides them and transmits them through a packet transmission network, and the moving image receiving apparatus receives the packet-transmitted moving image signals, In a moving image transmission / reception method for decoding and expanding and reproducing an image signal,
The moving image transmitting apparatus divides the image of each frame into a plurality of regions, independently performs encoding / compression, collects the encoded data group into one stream, and sets the data length for each region. And the header stream and the encoded data stream for packet transmission are transmitted in parallel as separate streams,
The moving image receiving apparatus includes: a process of receiving and buffering an image data sequence received through a network and a redundant header in parallel; a process of performing error detection and error correction of a received header; If the header is determined to be a valid header, the data is distributed to a plurality of decoders, and the decoder decodes the image data in parallel. If it is determined that the received header is not valid, the decoder processing is Stopping the operation of the decoder until a header is received, and continuing to output the latest image that has been successfully decoded,
A moving image transmission / reception method characterized by the above.   The moving image transmitting device transmits a data length table a plurality of times, the moving image receiving device takes a majority decision from the reception result, and based on the information of each data length of the result, decomposes the received image data sequence, 8. The moving image transmission / reception method according to claim 6, further comprising a step of performing image expansion processing in parallel using a decoding circuit corresponding to each data column.   The moving image transmitting apparatus transmits an error correction code added to a data length table, and the moving image receiving apparatus receives from the header information of the error detection / correction result of the reception result based on the information of each data length. 8. The moving image transmission / reception method according to claim 6, further comprising a step of decomposing the image data sequence and performing image expansion processing in parallel using a decoding circuit corresponding to the sequence for each data.   10. The moving image transmission device and the moving image reception device each transmit / receive moving image data and a header as a TCP / IP stream, and include a TCP error detection / retransmission control process. The moving image transmission / reception method described in 1.

Images