JPH11331839A - Apparatus and method for video transmission retransmission - Google Patents

Abstract

(57) [Summary] [Problem] In transmitting video data via a network and immediately reproducing the video data, the receiving side does not request retransmission of all data having an error in transmission, and important information is lost. The purpose is to avoid. SOLUTION: A transmitter 100 of a transmission apparatus examines position information of a screen at the time of reproducing video data. The low priority is added to the existing data, and the data is transmitted in descending order of importance. Receiver 200
Then, when a transmission error is detected, the importance of the data in which the error has occurred is checked, and whether to retransmit the data is determined according to the importance. A retransmission request is made for data having high importance. With this method, even if an error occurs in transmission, data with high importance is retransmitted, thereby preventing loss of data that may have important information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video transmission apparatus which retransmits data when data is lost due to communication in video transmission via a network.

[0002]

2. Description of the Related Art In recent years, the Internet has become widespread, and information has been frequently transmitted through networks. Various forms of information such as characters, images, and sounds can be exchanged. Video is one of them. And not only pre-recorded video,
For live broadcasting and monitoring, there is a growing demand for immediately transmitting and reproducing the captured video to the information receiving side.

[0003] Usually, TCP (Transm
ission Control Protocol) and UDP (User Datagram Proto
col) is used. The features of these two protocols will be briefly described.

[0004] TCP is connection-type communication.
Ensure reliable communication functions. A check sum for error detection is added to the header of the packet to be transmitted before transmission. When the receiving side detects an error, it automatically requests the transmitting side for retransmission. Therefore, highly reliable communication can be performed to guarantee correct data transmission. However, since a large transmission band is used to open a connection and retransmission processing is performed for all erroneous data, there is a possibility that data transmission will be delayed.

[0005] UDP is connectionless communication. Since it does not verify that the received data is as transmitted, it does not guarantee a reliable communication function. Therefore, errors or omissions occur in the data, resulting in low reliability. However, since there is no need to open or release a connection, high-speed transmission processing can be performed.

[0006] Since moving pictures are continuous data having a large size, they need to be sent at a high speed when reproduced in real time. When transmitting moving images over the Internet, UDP that can be transmitted at high speed is often used.

As an apparatus for transmitting continuous data, there is Japanese Patent Application Laid-Open No. 9-91314. In this method, the transmitting side attaches time stamp information indicating a time limit from when the data arrives to the packet to be transmitted to when the data is reproduced. When the receiving side detects an error in the data, it compares the time stamp with the system clock, makes a retransmission request only for a packet having a margin for the reception limit time stamped on the time stamp, and issues a retransmission request when the reception limit time is exceeded. If there is, a retransmission request is not made even if an error is detected.

[0008] These operations will be briefly described with reference to FIG. This continuous data transmission device includes a transmitter 100 for transmitting the video data 1 and a receiver 200 for receiving the video data 1.

The transmitter 100 includes a time stamp stamping section 141 for stamping a time stamp indicating the data arrival time limit,
A packet generation unit 142 for generating a packet with a time stamp added thereto, and a transmission / reception management unit 143 for transmitting / receiving a packet
And a retransmission management unit 144 that instructs the designated packet to be retransmitted when a retransmission request is received from the receiving side.

The receiver 200 includes a transmission / reception management unit 241 for transmitting / receiving a packet, a packet analysis unit 242 for detecting a packet error, and a time stamp comparison unit for comparing the data arrival time with the time limit indicated in the time stamp. Part 243
And a retransmission request unit 24 for determining whether to request a packet retransmission.
4 and a video reproducing unit 245 that returns the received data to the original data and reproduces it.

In the transmitter 100, when the video data 1 is read, the time stamp stamping section 141 calculates the arrival limit time Ts, and the packet generating section 142 pushes the time stamp stamping section 141.
, And extracts the time stamp information from the packet and adds it as header information of the packet. The packet generated by the transmission / reception management unit 143 is transmitted.

In the receiver 200, the transmission / reception management unit 241 receives the packet, and the packet analysis unit 242 performs error detection.
At the same time, the time stamp information is read and sent to the time stamp comparing unit 243. The time stamp comparing unit 243 determines a retransmission request based on the current time Sc indicated by the system clock, the arrival limit time Ts stamped on the packet, and the retransmission time Tr estimated in advance. If Ts ≦ Sc, no retransmission is performed. If Ts> Sc, make a further comparison. If Ts−Sc ≦ Tr, retransmission is not performed. If Ts-Sc> Tr, the retransmission request unit 244 is notified. In the retransmission request unit 244,
The transmission / reception management unit 241 generates a packet for requesting retransmission of the packet in which the error has been detected, and transmits the generated packet.

[0013]

However, in the prior art, when an error occurs in data transmission, retransmission is performed only when the arrival limit time Ts indicated by the time stamp stamped by the transmission side is reached even if retransmission is performed. Had made a request. For this reason, if the received time Sc has already exceeded the arrival limit time Ts indicated in the time stamp, or if retransmission is to be made to exceed the arrival limit time Ts, the packet is not retransmitted. When the receiver reproduces the video data without retransmitting the packet, it is not possible to correctly reproduce the video of the data portion of the lost packet. When such a part is near the center of the screen,
There is a problem that a video having important information for a viewer may not be reproduced when a scene of the video changes greatly.

If the transmission speed of the network is not constant, the reception of packets is continuously slowed down. If an error occurs in the packet transmission in such a case, the packet is discarded because the packet reception time Sc has exceeded the arrival limit time Ts. If a large number of erroneous packets occur continuously, video data for one frame may not be able to be received. As a result, there is a problem that the frame rate of the moving image is reduced.

According to the present invention, when video data is reproduced at the same time as the data generation time, a reduction in the video frame rate due to retransmission of erroneous data is prevented, and retransmission is requested for all erroneous data. However, an object of the present invention is to transmit data while avoiding a loss of important information.

[0016]

According to the present invention, there is provided a video data transmitting apparatus comprising: (1) position information indicating a position to be displayed on a screen at the time of reproduction; The degree of change of the video data at the same position as the video data to be transmitted in the frame immediately before the current frame. (3) Compare the frame being transmitted with the previous frame (4) When the transmitter reads a plurality of different images and transmits image data, the respective images are compared relatively and moved as an image. Is calculated to determine whether the correlation is large or small.

Based on the result, the importance of the data to be transmitted is determined. The transmitter transmits a packet which is a unit of the data to be transmitted. When the receiver detects an error in the data, the importance of the data is determined. Request retransmission depending on the degree. By taking this measure, when data having information considered to be important is lost, the data is retransmitted, thereby avoiding the loss of important screen information.

[0018]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The invention according to claim 1 of the present invention is intended to prevent a loss of important information when a data transmission error occurs during communication via a network.
A video transmission retransmission device characterized by adding importance to data based on the position information on the screen when the transmitter transmitting the video data displays the video data. This has the effect that the reception can determine whether or not retransmission should be performed when data is lost during communication.

According to a second aspect of the present invention, when a transmitter for transmitting video data and a receiver for receiving video data communicate via a network, the transmitter displays the video data. A video retransmission transmission device characterized by adding importance to data from the screen position information and transmitting the data, and when data is not transmitted correctly, the receiver determines a retransmission request based on the importance. Yes, if the receiving side loses data during communication, it makes a retransmission request according to the importance of the data, and retransmits data with high importance, so that important screen information is lost during video data playback Can be avoided.

According to a third aspect of the present invention, there is provided a data reading means for reading data, a position information inspection means for inspecting position information of a position on the screen at which video data is displayed, and a data transmission means. Transmission order determining means for determining the order of data, packet generating means for generating a packet which is a unit for transmitting data, data transmitting means for transmitting data, data receiving means for receiving data, and data not correctly transmitted. A video transmission retransmission device with retransmission processing means for retransmitting data when there is a retransmission request from the receiving side. In this case, the receiving side determines whether or not to retransmit, and retransmits important data.

According to a fourth aspect of the present invention, there is provided a data reading means for reading data, a position information checking means for checking position information on a position on the screen where video data is displayed, Transmission order determining means for determining the order of data, packet generating means for generating a packet which is a unit for transmitting data, data transmitting means for transmitting data, data receiving means for receiving data, and data not correctly transmitted. A transmitter having retransmission processing means for retransmitting data when there is a retransmission request from the receiving side, a packet analyzing means for checking an error of data which was not transmitted correctly, and a Retransmission determination means for determining whether to request retransmission,
A video transmission retransmission device having a receiver provided with a retransmission request unit for making a retransmission request and a data reproduction unit for reproducing received video data, and performing retransmission for data of high importance This has the effect of avoiding the loss of important screen information during video data reproduction.

According to a fifth aspect of the present invention, based on positional information indicating where video data is displayed on a screen, data which constitutes one frame of the video data is displayed on the screen. Distance setting means for setting how far away from the center, and importance determination means for determining importance so as to be distributed in a rectangle from threshold information describing a relationship between the distance from the center and the importance of data. It is a video transmission retransmission device, which can determine the importance distributed in a rectangle from the position information of the screen, and determines the importance according to the display screen because the video display screen is also a rectangle. Has an action.

According to a sixth aspect of the present invention, there is provided a buffer processing means for storing video data in a buffer, and a buffer processing means for storing the video data in a buffer which is one or more frames prior to the frame being transmitted. Data length comparing means for determining whether or not the video information of the data has changed by comparing the length of the data with reference to the data constituting the frame and the data displayed at the same position when displayed on the screen Video transmission retransmission device equipped with
This has the effect of determining the importance from the change in data length.

According to a seventh aspect of the present invention, there is provided a video transmission / retransmission apparatus including a frame comparison means for comparing the degree of change from one or more previous frames from previous frame information. This has the effect of determining importance from the degree of change between each other.

The invention according to claims 8 and 9 of the present invention provides a multi-screen in which a transmitting side reads a plurality of different video data and a receiving side displays a plurality of different video data on the same screen. This is a video transmission retransmission device equipped with a multi-screen management means for determining and managing the correlation between a large motion and a small motion relative to each other, and determines the importance from the video correlation. Having.

Hereinafter, each embodiment of the present invention will be described.
This will be described with reference to FIGS. It should be noted that the present invention is not limited to these embodiments at all, and can be implemented in various modes without departing from the gist thereof.

(Embodiment 1) In this embodiment, the transmitter 10
0 is a video that transmits video data with importance based on the position information on the display screen, and the receiver determines whether to request retransmission according to the importance of the data when an error occurs in data transmission. The transmission retransmission device will be described. FIG. 1 shows a configuration diagram of the present apparatus.

In FIG. 1, 1 is video data, 100 is a transmitter for transmitting data, and 200 is a receiver for receiving data. The transmitter 100 includes a data reading unit 101 that reads the video data 1, a position information checking unit 102 that checks the position information of the blocks that form the frame of the video data, and a transmission order determination that determines the order of the block to be transmitted from the position information. Part 10
3 and a packet generator 104 for generating a packet to be transmitted
A data transmitting unit 105 for transmitting data, a data receiving unit 106 for receiving data, and a retransmission processing unit 10 for retransmitting a designated packet when a retransmission request is received from the receiver.
Consists of seven.

The receiver 200 receives the data transmitted from the transmitter, the packet analyzer 202 checks the received packet for errors or omissions, and determines whether or not to make a retransmission request for the packet. Retransmission determination section 203
And a retransmission request unit 204 that makes a retransmission request for the packet whose retransmission request is determined by the retransmission determination unit, a data transmission unit 205 that transmits the data of the retransmission request, and a data reproduction unit 206 that reproduces the received video data. Be composed.

The following describes the video data 1 and packets handled in the present embodiment. In this embodiment, video data composed of frames is used, such as the DV or JPEG compression method. FIG. 2 shows a conceptual diagram of video data and packets handled in this embodiment.

First, video data and packets handled by the transmitter 100 will be described. A frame is a unit of a screen constituting a moving image (21). In the DV compression method, moving images are formed by displaying 30 frames continuously for one second. A frame encoded by compression is composed of a plurality of variable-length blocks (22). In this embodiment, importance is determined for each block. One block does not have information of a plurality of frames. In network communication, a packet, which is a unit of data to be transmitted, is formed by assembling a plurality of blocks to form a data portion and adding header information to the data portion (23).

The packet handled in this embodiment has, as header information, information such as the packet number, the number of the frame to be transmitted, the importance, and the total number of packets having the same importance among the frames to be transmitted (24).

The blocks constituting the data portion of the packet constitute the same frame, and are generated so as to be constituted by blocks having the same importance. Packets are not generated from blocks having different frame information and different importance. This is for the following two reasons. (1) This is because the video data 1 handled in the present embodiment performs transmission / reproduction processing in frame units. (2) Receiver 2
This is because it is determined whether or not a retransmission request is made for a packet in which an error has occurred in transmission based on the importance attached to the header information of the packet. Therefore, the importance is determined to be one for one packet. Then, the packets are transmitted in the order of importance. This is for the following two reasons. (1) If you send packets in order of importance,
When a retransmission request is determined for a packet having an error in transmission in the receiver 200, packets having numbers before and after the packet number in which the error has occurred are also arranged in order of importance. The importance of the missing packet can be determined by referring to the importance in the header information. (2) Since the receiver 200 determines that a packet with high importance is always retransmitted even if an error occurs in transmission, the last packet in the process of transmitting one frame of data with high importance is When the packet is retransmitted, the receiver waits for the start of video data reproduction until the lost packet is retransmitted, thereby preventing the data reproduction time from being delayed. The packet requested to be retransmitted by the receiver 200 includes a packet number, a frame number of data requested to be retransmitted, and a packet number requested to be retransmitted (25).

In the present embodiment, a block having information displayed at a position near the center of the screen when video data is displayed is assigned a high importance, and the lower the distance from the center, the lower the importance. This is because (1) generally, when generating video data, the focus is on the center of the screen, so the closer to the center, the more important information is included. (2)
This is because the viewer who views the video data has a longer time viewing the position closer to the center than the end of the screen. Figure 3
FIG. 1 shows a conceptual diagram of video data to which importance is assigned based on position information. In this example, one frame is composed of 100 blocks, and the level of importance is set to five levels such as 1 for the highest level and 5 for the lowest level. Therefore, the importance is 1 at a position near the center of the video display screen, and 5 at a position near the edge of the screen.

The flow of processing of the transmitter will be described. The transmitter 100 first reads the video data 1 with the data reading unit 101. Video data 1 read by the data reading unit 101
The position information checking unit 102 checks the information of the position displayed on the screen when the video data 1 is reproduced for each block. Then, the inspected position information is passed to the transmission order determination unit 103. The transmission order determination unit 103 determines the importance of the block based on the position information. In the video data reproduction screen, a block having information at a position close to the center is assigned a high importance, and the further away from the center, the lower importance is assigned to the block. The transmission order determination unit 103 collects the blocks whose transmission order is determined, and the packet generation unit 104 collects blocks having the same importance for one packet of data.
Add header information and generate as a packet to be transmitted.
Data transmitting section 105 transmits the packet generated by the packet generating section.

The operation of the receiver 200 will be described with reference to FIGS. FIG. 4 is a flowchart of a process up to retransmission of an erroneous packet, FIG. 5 is a flowchart of a process of determining the importance of an erroneous packet, and FIG. 6 is a conceptual diagram of a lost packet.
The receiver 200 receives the data transmitted from the transmitter 100 by the data receiving unit 201 (41). The data received by the data receiving unit 201 is analyzed by the packet analyzing unit 202. Here, it is determined whether or not there is a “transmission error” that the packet has not been sent in perfect form (42). In the example of FIG. 6, the packet number 10 is lost, indicating that a transmission error has occurred. Notify the determination result to the retransmission determination unit 203,
If it is determined that there is an error, the importance of the packet in which the error has occurred is determined (43). In the example of FIG. 5, the importance is determined with reference to the header information of the packet having a number before and after the packet number in which the error has occurred. In the case of FIG. 6, 1
Since the packet No. 0 is missing, first, the header information of the packet No. 9 is referred to (51). According to the header information,
It can be seen that the frame number 6 of the data of the ninth packet, the significance of the data is K1 = 1, and the total number N1 of the packets having the significance K1 in the frame number 6 is 10. The total number of packets having a packet number earlier than the number 10 and having the importance K1 among the already received packets is counted and compared with N1 (52). N1
If less than, the importance of this packet is determined to be K1
(53). If N1 has been reached, reference is made to the packet following the missing packet (54). In the case of (62), the 11th header information is referred to. According to the header information of the eleventh packet, the importance of the eleventh packet is K2 = 2, so the importance of the missing tenth packet is determined to be K2 (55). In the present embodiment, a threshold value of importance is determined in advance to determine whether to request retransmission. If the importance is determined in (43), it is compared with a threshold (44). If the importance determined is higher than the importance described in the threshold,
Retransmission determination section 203 notifies retransmission request section 204 (45). If the determined importance is lower than the threshold,
Compare the current time Tc and the time Tr required for retransmission with the playback start time Tp of the frame being processed in advance (46), and if retransmission is in time, perform retransmission in the same manner as above. Request (45). The retransmission request unit 204 generates a retransmission request packet from the packet number and the frame number requesting retransmission, and the data transmission unit 205 transmits the packet requesting retransmission. The time required for retransmission Tr and the reproduction start time Tp
Is determined in advance by the receiver 200, and the determination method is not specified.

When the receiver 200 detects a packet error or loss and requests retransmission, the data receiving unit 106 of the transmitter 100 receives the retransmission request packet and notifies the retransmission processing unit 107. The retransmission processing unit 107 determines, from the transmitted packets, a packet transmitted by the transmitter 100 and having an error. Then, it notifies the packet generation unit 104 of the packet number. The packet generation unit 104 regenerates the specified packet, and the data transmission unit 105 retransmits the specified packet.

The video data reproducing unit 206 of the receiver 200 reproduces a video after receiving one frame of video data.

As described above, according to the present embodiment, by assigning importance to data based on the position information indicating where the image is displayed on the screen when the transmitting side displays the image, the receiving side can lose data during communication. If it does, a retransmission request is made according to the importance of the data, and for example, the data at the center of the screen is always retransmitted, so that the loss of important screen information during video playback can be avoided, Its practical effect is great.

(Second Embodiment) In the first embodiment, importance is assigned to data based on the position information of the screen, and based on that, the receiver 200 determines a retransmission request for data in which an error has occurred. Here, a description will be given of an apparatus having a function of obtaining coordinates of a data on a display screen from position information of the screen and thereby determining importance of the data.

FIG. 7 shows a configuration of a transmitter of a video retransmission apparatus for determining importance from position information on a screen. In the present embodiment, in FIG. 7, 1 is video data,
100 is a transmitter for transmitting data, and 200 is a receiver for receiving data.

The transmitter 100 is different from the first embodiment in that a distance setting unit 108 for determining coordinates from the center by determining coordinates from the position information obtained by the position information checking unit 102, and the importance of the data to be transmitted. An importance determining unit 109 for determining the importance with reference to the threshold information 110 for determining the importance is added. Other configurations are the same as those of the first embodiment shown in FIG.

FIG. 8 shows an example of a screen on which importance is determined by the method described in the present embodiment. In the first embodiment, the importance of data is determined according to the distance from the center of the screen.
When the distance between the data display position and the center is calculated and the importance is determined in five steps as in the first embodiment, the range of data having the same importance is distributed in a circular shape as in (81). I do.
However, since the screen that normally displays video is rectangular,
If the importance is determined for the rectangle, the importance can be given according to the display screen (82). Therefore, in the present embodiment, a method for obtaining coordinates from position information on a display screen and determining importance in a rectangle will be described.

FIG. 9 shows a flow of processing for determining importance from position information. The distance setting unit 108 includes the position information inspection unit 1
From the position information acquired in 02, the distance from the center of the video display screen is obtained. The central point on the video playback screen
At (0,0), the coordinates of the display position of the video data to be inspected are acquired on the coordinates where the upper right end is (1,1) (91). FIG. 10 shows an example of acquiring coordinate axes and data coordinates. In this example,
The acquired coordinates are (0.5, 0.3). Then, the absolute values of the x and y coordinates of the obtained coordinates are compared (92). If the absolute value of the x coordinate is greater, the distance from the center is set to the absolute value of the x coordinate (93) .If the absolute value of the x coordinate is smaller, the distance from the center is the absolute value of the y coordinate. Set (94).

In the case of the example shown in FIG. 10, since the coordinates are (0.5, 0.3), the distance from the center is 0.5. Then, the importance is determined with reference to the threshold information 110 in which the correlation information between the distance from the center and the importance is described (95). As an example, a case is shown where the threshold value information 110 describes a correlation that the importance changes each time the ratio of the distance from the center expressed as a percentage increases by 20%. The distance from the center expressed as a percentage means that the distance from the center is
0 is 0% and 1 is 100%.

In the case of the example of FIG. 10, the distance from the center is 0.5
So the ratio is 50%. In the threshold information 110, if the distance from the center is less than 20%, the importance is 1, 20% or more.
If it is less than 40%, the importance is 2, if it is 40% or more and less than 60%, the importance is 3, if it is 60% or more and less than 80%, the importance is 4, and if it is 80% or more, it is importance Is defined as 5, In this case, in the example of FIG. 10, since the ratio is 50%, the importance is determined to be 3.

As described above, in this embodiment, it is possible to acquire the position information of the screen at the time of reproducing the video and easily determine the importance. In addition, by determining the distance from the center to be a rectangle, importance can be given according to the area ratio of the screen, and the practical effect is large.

(Embodiment 3) In Embodiment 1, the importance is determined based on the position information of the video display screen. In the third embodiment, when determining the importance of data, the importance is determined based on whether or not there is a change compared with the data at the same position in the frame (previous frame) one or more before the frame being transmitted (current frame). To determine. Whether or not there has been a change is determined by comparing the length of data at the same position in one or more previous frames. In DV compressed data, a unit of data constituting one frame of video is called a block. Two blocks having the same video information display the same video when reproduced, and have the same data length (block length).

Therefore, in this embodiment, the length of a block having the same position of video information at the time of video reproduction in one or more previous frames is compared with the length of a block in the frame being transmitted. It is assumed that the same video is displayed and there is no change in the video. If there is a change, unlike the video of the previous frame, even if the packet is lost, the previous frame may not be able to compensate for it, so the importance is increased. If it is determined that there has been no change, even if a packet is lost, the block of the previous frame can be supplemented. Therefore, in the present embodiment for reducing the importance of determining the retransmission request, The frame immediately before the frame was treated as the previous frame. Note that the previous frame is not limited to the immediately preceding frame. Further, in the present embodiment, the unit of data to be compared is a block, but this is not limited to a block.

FIG. 10 shows a video retransmission apparatus for determining the importance by comparing the block lengths. In the present embodiment, FIG.
In 11, 1 is video data, 100 is a transmitter for transmitting data, and 200 is a receiver for receiving data.

The transmitter 100 replaces the position information checking unit 102 of the first embodiment with a buffer processing unit 111 that buffers blocks of video data and deletes the contents of the buffer that is no longer required to be referenced. It is obtained by acquiring a block of the previous frame at the same position as the block of the current frame from the previous frame information secured in the buffer, and adding a block length comparison unit 112 for comparing the length. Other configurations are the same as those of the first embodiment shown in FIG.

The receiver 200 is obtained by adding the previous frame information 207 that supplements the data lost during reproduction to the first embodiment.

The processing flow of the transmitter 100 will be described. In the transmitter 100, the data reading unit 101 reads the video data 1 as in the first embodiment. The buffer processing unit 111 secures the read data in a buffer for each block,
Unnecessary frame information is sequentially deleted from the buffer. The data secured in the buffer by the buffer processing unit 111 is saved as previous frame information 113.

FIG. 13 shows the flow of processing for determining the importance by comparing the block lengths. The block length comparison unit 112 acquires the length L1 of the block whose importance is to be determined from the data of the current frame read by the data reading unit 101 (13
1). Further, from the previous frame information 113, the block length L0 of the block of the previous frame located at the same position on the display screen as the block whose importance is determined for the current frame is obtained.
(132). The block length L1 of the current frame is compared with the block length L0 of the previous frame (133), and if L0 ≠ L1, the higher importance is assigned (1
34), if L0 = L1, assign low importance (135). FIG. 12 shows a conceptual diagram of the block length comparison. Comparing the previous frame and the current frame, (121) shows the case where the block lengths are equal,
(122) shows a case where the block lengths are different. FIG.
Then, if it is determined that the video has changed,
This is an example in which the importance is determined to be 1 when it is determined that there is no change to 1 and the importance is determined to be 5.

Then, as in the first embodiment, a packet is generated by the packet generator 104, and the packet generated by the data transmitter 105 is transmitted. When a retransmission request is made, the data reception unit 106 receives a retransmission request packet from the receiver 200, and the retransmission processing unit 107 determines the designated packet and performs retransmission processing.

In receiver 200, as in the first embodiment, data receiving section 201 receives data, and packet analyzing section 202 analyzes whether an error or omission has occurred. When an error or omission occurs, the retransmission determination unit 203 determines a retransmission request based on the importance of data. The retransmission request unit 204 makes a retransmission request for data with high importance, and does not make a request for low data. After receiving one frame of packets,
The data reproducing unit 206 reproduces a video. If there is any missing video data, that is, data for which retransmission of the packet was not performed even if an error occurred in transmission, the data playback unit
The 206 acquires the data displayed at the same position in the previous frame from the previous frame information 207 previously stored in the buffer, and makes up for the missing data instead.
Further, the data reproducing unit 206 reserves data of the frame currently being processed in a buffer for the next frame.

As described above, in this embodiment, the video data of the frame being transmitted is compared with the data at the same position in the frame immediately before the frame being transmitted to determine whether or not the video has changed. And assign importance. Thus, when data is lost during communication, a retransmission request is made in accordance with the importance of the data, and loss of important screen information during video reproduction can be avoided, which has a large practical effect.

(Embodiment 4) In Embodiment 1, importance is determined based on position information. In the fourth embodiment, as a method of determining importance, in addition to position information, a frame being transmitted (current frame) is compared with a frame being transmitted one or more times before (current frame). hand,
The degree of importance is determined based on the degree of change as to whether the video of the entire frame has changed or the scene has changed. FIG. 14 shows a video retransmission apparatus in which the function of determining importance by comparing the degree of change of a frame is added to the first embodiment.

In this embodiment, the importance is set high when the scene change of the frame is large, and the importance is set low when the change is small. This is because if there is no change in the scene, it is highly likely that the video is similar to the video of the previous frame. Is made lower throughout the frame. When the scene changes, unlike the video of the previous frame, even if a packet is lost, the previous frame cannot compensate for the lost packet. Therefore, the importance of the entire frame is increased to avoid the loss of the packet.

As a method for measuring the degree of change between two frames, in the present embodiment, the function K = Y (F
Use 0, F1). This is a predetermined function, and returns the value of a variable K representing the degree of change between two frames using two pieces of frame information F0 and F1 as arguments. The function Y for determining the degree of change is not specified in the present embodiment.

In this embodiment, the frame immediately before the frame being transmitted is treated as the previous frame. In addition,
The previous frame is not limited to the previous frame.

In this embodiment, in FIG. 14, 1 is video data, 100 is a transmitter for transmitting data, and 200 is a receiver for receiving data.

The transmitter 100 includes a buffer processing unit 111 for storing the data read in the first embodiment in the buffer as previous frame information 113 and a frame comparing unit 114 for comparing the degree of change between the frame being processed and the previous frame. It has been added. Other configurations are the same as those of the first embodiment shown in FIG.
Is the same as Receiver 200 is the same as Embodiment 3 shown in FIG.

In the transmitter 100, the data reading unit 101 reads the video data 1 as in the first embodiment. Embodiment 3
Similarly to the above, the data restored by the buffer processing unit 111 is secured in the buffer for each block, and is stored as the previous frame information 113.

The position information inspection unit 102 and the frame comparison unit 114
FIG. 16 shows a flow of processing for acquiring importance from the position information and the degree of change of the frame in the transmission order determining unit 103.

The position information checking unit 102 checks the position information indicating where the data is displayed when the data is reproduced (161). The frame comparison unit 114 compares the previous frame with the frame being processed, and calculates a variable K = Y (F0, F1) representing the degree of change (162). FIG. 15 shows an example in which a scene is changed by comparing the previous frame with the current frame. If the change in the image is small as in (151), it is determined that there is no change in the scene, and the value of the degree of change K is small. If the image greatly changes as in (152), it is determined that the scene has changed, and the value of the degree of change K increases. In the transmission order determination unit 103, as in Embodiment 1, from the position information inspected by the position information inspection unit 102,
The importance is set (163). In the present embodiment,
A predetermined threshold value for determining whether the change is large or small according to the value of K is determined. This threshold value is referred to and compared with the degree of change K (164). If K is larger than the threshold, that is, if it is determined that the change is large, the level of importance set in (163) is reset to the highest value (165). If K is less than the threshold,
That is, if it is determined that there is little change, the level of importance set in (163) is analyzed (166), and if it is not the lowest level, the level of importance is reduced by 1 (167).

Then, as in the first embodiment, a packet is generated by the packet generation unit 104, and the packet is transmitted by the data transmission unit 105. When a retransmission request is issued, the data receiving unit 106 receives a packet from the receiver, and the retransmission processing unit 107 determines the designated packet and performs retransmission processing.

In receiver 200, data receiving section 201 receives data and packet analyzing section 202 analyzes whether an error or omission has occurred, as in the first embodiment. When an error or omission occurs, the retransmission determination unit 203 determines a retransmission request based on the importance of data. The retransmission request unit 204 makes a retransmission request for data with high importance, and does not make a request for low data. After receiving one frame, the data reproducing unit 206 reproduces the data as in the third embodiment.

As described above, in the present embodiment, the degree of change between the video data of the currently transmitted frame and the frame immediately before the currently transmitted frame is compared to determine whether or not the video scene has changed. Judge and assign importance. Thus, when data is lost during communication, a retransmission request is made in accordance with the importance of the data, and loss of important screen information during video reproduction can be avoided, which has a large practical effect.

(Fifth Embodiment) In the first embodiment, importance is determined based on position information. In the fifth embodiment, as a method of determining importance, in addition to position information, the transmitting side transmits a plurality of different video data, and the receiving side reads and reads a plurality of different video data on one screen. A plurality of video data are compared, and a change in a series of frames in each video is large or small, that is, a motion relatively large as a video in a screen displaying a plurality of videos is relatively large. The importance is determined by determining whether the movement is small or not.

In the present embodiment, in a screen (multi-screen) displaying a plurality of different video data, a video determined to have relatively large motion has a high importance, and a video determined to have little motion has a high importance. Give low importance. This is because, for a video with a large motion, the video changes greatly from frame to frame, so it can be said that the importance of each frame is high, and for a video with a small motion, the video changes little even if the frame is changed. Therefore, it can be said that the importance of the frame is low. Therefore, high importance is added to the entire frame for a video determined to be small in motion so that the frame rate is increased, and importance of the entire frame is added to a video determined to be small in motion. Lower.

The present embodiment employs a method of measuring the degree (image correlation) of a large motion or a small motion when relatively compared as a video among multi-screens displaying n-screen video. Then the function P = Q (D1, D2, ..., D
Use n). This is a predetermined function, and a function for calculating a correlation variable Pk representing a correlation between the image Dk and other images in the multi-screen with n pieces of image information D1, D2, ..., Dn as arguments. It is. Note that the video correlation function Q is not specified in the present embodiment.

FIGS. 17 and 18 show a screen (multi-screen) showing a plurality of different video data in the first embodiment.
A receiver and a transmitter of a video retransmission apparatus to which a function of determining importance by obtaining a correlation between videos as to whether or not one video has a motion compared to another video.

In the present embodiment, in FIG.
Video data by the compression method, 100 is a transmitter for transmitting data, and 300-N is a series of processes from reading one video data 1 to generating a packet and transmitting the packet. 300-1
1 to 300-N indicate that each process reads the N types of video data in the same configuration, and generates and transmits a packet. The transmitter 100 is obtained by adding a multi-screen management unit 115 that determines and manages correlation of video data in the process of transmitting video data to the first embodiment.

In FIG. 18, reference numeral 200 denotes a receiver for receiving data. 400-N is a flow of a series of processing for receiving and reproducing one transmitted video data. 400-1 to 4
From 00 to N, the same configuration is used to receive and reproduce N types of video data. Other configurations are the same as those of the first embodiment.

In the transmitter 100, the data reading unit 101 reads the video data 1 as in the first embodiment. Location information inspection unit 102, multi-screen management unit 115, and transmission order determination unit 103
FIG. 19 shows a flow of a process for determining the importance based on the correlation between the position information and the video in the multi-screen.

The position information inspecting unit 102 inspects the position information indicating where the image is displayed on the display screen of the video data (191). In the multi-screen management unit 115, when the transmitter 100 transmits a plurality of images, the relative motion of each image on the screen is compared, and a correlation function when the image Dk is compared with another image is Calculate Pk = Q (D1, D2, ..., Dn) (1
92). As an example of comparing images, a case will be described in which a transmitter reads images from four cameras as shown in FIG. 17 and compares the correlation between the magnitudes of motions in a multi-screen. Here, the movements of the four images are compared, and for example, the correlation functions P3 and P4 are set higher for sports images such as the camera 4 and images of moving vehicles such as the camera 3. In addition, low correlation functions P1 and P2 are set for a landscape image such as camera 1 and a still-life image such as camera 2. Transmission order determination unit 1
In 03, first, importance is set from the position information inspected by the position information inspecting unit 102 in the same manner as in the first embodiment (193).
In the present embodiment, a threshold value for determining the degree of importance given to data from the value of Pk obtained by multi-screen management section 115 is set in advance. The transmission order determination unit 103 refers to the threshold and compares the correlation function Pk set in (192) with the threshold (194). If Pk is larger than the threshold, that is, if it is determined that the screen has a large motion of the video, the level of importance set in (193) is reset to the highest value (195). If Pk is smaller than the threshold value, that is, if it is determined that the screen has little video motion, the level of importance set in (193) is analyzed (18).
6) If not the lowest level, lower the level of importance by 1 (197).

Then, as in the first embodiment, a packet is generated by the packet generation unit 104, and the packet is transmitted by the data transmission unit 105. When a retransmission request is issued, the data receiving unit 106 receives a packet from the receiver, and the retransmission processing unit determines the specified packet and performs retransmission processing.

In receiver 200, as in the first embodiment, data receiving section 201 receives data, and packet analyzing section 202 analyzes whether an error or omission has occurred. When an error or omission occurs, the retransmission determination unit 203 determines a retransmission request based on the importance of data. The retransmission request unit 204 makes a retransmission request for data with high importance, and does not make a request for low data. When receiving data for one frame, the data reproducing unit 206 reproduces the video.

As described above, in the present embodiment, importance is given to a plurality of videos to be transmitted by determining the video to be transmitted at a high frame rate and the one having a small influence by reducing the frame rate. Thus, when data is lost during communication, a retransmission request is made in accordance with the importance of the data, and loss of important screen information during video reproduction can be avoided, which has a large practical effect.

[0081]

As described above, according to the present invention, when a transmitter generates and transmits video data and a receiver immediately reproduces the video data, (1) data can be removed from data in which an error or loss occurs. Retransmission is performed according to the degree of importance, so loss of important information on the video screen can be prevented. (2) Because not all data is retransmitted for errors or omissions,
Can reduce network bandwidth consumption,
(3) From the position coordinates of the display screen, the distance from the center can be obtained to easily determine the importance of the video data block.
(4) The importance of a block of video data can be determined by comparing the degree of change between the previous frame and the frame being processed. (5) In a multi-screen for reproducing images from a plurality of cameras, an effect is obtained in which the importance of a block of image data can be determined by comparing the correlation between the images.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a video transmission and retransmission device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram of video data and a packet according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of video data with a degree of importance according to the first embodiment of the present invention;

FIG. 4 is a flowchart of a retransmission determination process according to the first embodiment of the present invention.

FIG. 5 is a flowchart of importance determination according to the first embodiment of the present invention.

FIG. 6 is a diagram illustrating an example of data having an error in transmission according to the first embodiment of the present invention.

FIG. 7 is a configuration diagram of a video transmission retransmission device according to a second embodiment of the present invention.

FIG. 8 is a diagram illustrating an example of video data with importance assigned according to the second embodiment of the present invention;

FIG. 9 is a flowchart of an importance determination process according to the second embodiment of the present invention.

FIG. 10 is a diagram illustrating an example of obtaining a distance from a center according to the second embodiment of the present invention.

FIG. 11 is a configuration diagram of a video transmission and retransmission device according to a third embodiment of the present invention.

FIG. 12 is a block length comparison example diagram according to the third embodiment of the present invention;

FIG. 13 is a flowchart of importance determination processing according to Embodiment 3 of the present invention.

FIG. 14 is a configuration diagram of a video transmission and retransmission device according to a fourth embodiment of the present invention.

FIG. 15 is a diagram illustrating a frame comparison example according to the fourth embodiment of the present invention.

FIG. 16 is a flowchart of an importance determination process according to the fourth embodiment of the present invention.

FIG. 17 is a configuration diagram of a transmitter of a video transmission / retransmission device according to a fifth embodiment of the present invention.

FIG. 18 is a configuration diagram of a receiver of a video transmission and retransmission device according to a fifth embodiment of the present invention.

FIG. 19 is a diagram illustrating an example of a video correlation comparison according to the fifth embodiment of the present invention.

FIG. 20 is a flowchart of an importance determination process according to the fifth embodiment of the present invention.

FIG. 21 is a configuration diagram of a conventional video transmission device.

[Explanation of symbols]

 1 Video data 100 Transmitter 101 Data reading unit 102 Location information inspection unit 103 Transmission order determination unit 104 Packet generation unit 105 Data transmission unit 106 Data reception unit 107 Retransmission processing unit 108 Distance setting unit 109 Importance determination unit 110 Threshold information 111 Buffer processing unit 112 Block length comparison unit 113 Previous frame information 114 Frame comparison unit 115 Multi-screen management unit 200 Receiver 201 Data reception unit 202 Packet analysis unit 203 Retransmission determination unit 204 Retransmission request unit 205 Data transmission unit 206 Data reproduction unit 207 Previous frame information

Claims (9)

[Claims] 1. When a data transmission error occurs, in order to prevent loss of important information, when transmitting video data, importance is added to the data based on position information of a screen when the video data is displayed. A video transmission method characterized by: 2. When a data transmission error occurs, in order to prevent loss of important information, the transmitting side transmits the data with the importance added from the position information of the screen when displaying the video data, and the receiving side transmits the data. A retransmission request is determined based on the importance added to the video transmission. 3. A data reading means for reading data,
Position information inspecting means for inspecting position information on where the video data is displayed on the screen, transmission order determining means for determining the order of data to be transmitted from the position information, and data for transmitting data according to the transmission order A video transmission apparatus comprising: a transmission unit; a data reception unit that receives data; and a retransmission processing unit that retransmits data when a retransmission request is received from a reception side for data that was not correctly transmitted. 4. A data reading means for reading data,
Position information inspecting means for inspecting position information indicating where on the screen the video data is displayed, transmission order determining means for determining the order of data to be transmitted from the position information, and data for transmitting data according to the transmission order Transmitting means, data receiving means for receiving data, retransmission processing means for retransmitting data when there is a retransmission request from the receiving side for data that was not correctly transmitted,
A retransmission requesting means for inspecting an error in data that was not transmitted correctly, a retransmission determining means for determining whether to request retransmission of data that was not transmitted correctly, and a retransmission requesting means for making a retransmission request And a data reproducing means for reproducing the received video data. 5. A distance for setting how far the data constituting one frame of the video data is away from the center of the screen based on position information indicating where the video data is displayed on the screen. 4. The apparatus according to claim 3, further comprising: setting means; and importance determining means for determining importance so as to be distributed in a rectangle from threshold information describing a relationship between the distance from the center and the importance of the data. The video transmission device according to claim 4, or the video transmission retransmission device according to claim 4. 6. A buffer processing means for inputting video data into a buffer on a transmission side, and data and a screen constituting a frame being transmitted being processed from frame information stored in the buffer one or more times earlier than the frame being transmitted. Data length comparing means for determining whether the video information of the data has changed by comparing the length of the data with reference to the data displayed at the same position when the data length is displayed, The video transmission apparatus according to claim 3, wherein the importance is determined from the change. 7. A frame comparing means for comparing whether or not the scene of the frame being transmitted and the scene of the immediately preceding frame have changed from frame information one or more preceding the frame currently being transmitted stored in the buffer. 5. The video transmission device according to claim 3, further comprising a function of determining the importance based on the degree of change of whether the scene has changed significantly or the video transmission retransmission device according to claim 4. 8. A multi-screen management means for reading a plurality of video data and judging and managing a correlation between a relatively large motion and a small motion in a plurality of videos, and
The video transmission device according to claim 3, further comprising a function of determining importance from a correlation between a plurality of videos. 9. A multi-screen management means for reading a plurality of different video data, judging and managing a correlation between a large motion and a small motion in a plurality of videos relative to each other. The video transmission / retransmission device according to claim 4, further comprising a function of determining importance from video correlation.