JP2004070712A - Data delivery method, data delivery system, split delivery data receiving method, split delivery data receiving device, and split delivery data receiving program - Google Patents

Abstract

An object of the present invention is to enable high-speed data distribution to be continued without interruption even when communication with some distribution servers is interrupted due to a failure during divided data distribution.
In response to a distribution request from a client, a plurality of distribution servers distribute distribution target data to a client, and another distribution server simultaneously distributes error correction information for the divided distribution data to the client. I do. Normally, the client 1 restores the data to be distributed by combining the data distributed without using the error correction information. However, data from some distribution servers cannot be received due to a failure, or the data is reduced to a predetermined transfer rate or less. If it does, the distribution target data is restored using the error correction information.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method and an apparatus for splitting and delivering data on a network, and more particularly to a data delivery method, a data delivery system, a split delivery data receiving method, a split delivery data receiving apparatus, and a split delivery data reception that enable robust data delivery. It is about the program.
[0002]
[Prior art]
When data is to be distributed at high speed over a network, a method is generally used in which a plurality of distribution servers are prepared to achieve load distribution and optimization of a distribution path. Further, a technique of dividing data into a plurality of servers and arranging the divided data to perform more advanced load distribution is also used (for example, JP-A-2002-32280).
[0003]
In particular, in the case of a network such as ADSL or CATV in which the receiving side band is configured with a line larger than the transmitting side band, the receiving speed is surely improved by simultaneously transmitting from a plurality of servers.
[0004]
On the other hand, as a means for reliably distributing data, a plurality of distribution servers holding data of the same contents are kept in a standby state, and when a failure occurs in a server used for distribution, switching to another distribution server is performed. Is used.
[0005]
In a LAN environment, a network RAID file is used as a method of transmitting and receiving data even if a failure occurs in some servers by arranging files in a plurality of fixedly installed servers to form a redundant configuration. A system has been devised.
[0006]
[Problems to be solved by the invention]
The above method of distributing data on a network is based on the assumption that all related distribution servers and networks are always in a normal state. If a failure occurs during data distribution, data distribution is performed immediately. Fails.
[0007]
In general data distribution, if it is determined that distribution has failed, re-distribution may be performed using a combination of normal servers. However, in the case of video and audio stream distribution where real-time performance is important, The problem of interruption occurs.
[0008]
In communication between end users such as Peer to Peer, sufficient reliability cannot be expected for both the server and the network. Therefore, it must be assumed that frequent failures occur.
[0009]
According to the present invention, when data is divided and distributed using a plurality of distribution servers on a network, the distribution servers and the network do not have sufficient reliability, and communication with some servers is disabled due to a failure during data distribution. It is an object of the present invention to be able to continue high-speed data distribution without interruption even when interruption occurs.
[0010]
[Means for Solving the Problems]
In order to solve the above problems, the present invention provides a plurality of distribution servers having means for retaining the same data on a network and partially distributing the data, means for partially requesting the plurality of distribution servers to distribute data, and A distribution method in a system including a client having a function of restoring original data by combining the partial data, wherein the distribution server has a means for generating error correction information, and the client is a data transmission target. Request to distribute the error correction information of the data to be distributed at the same time as making a partial distribution request to multiple distribution servers without duplication, and to distribute the data normally without using the error correction information. Combine and restore the data to be distributed, but if data cannot be received from some servers due to network failure or distribution server failure, or If it becomes less than the transfer speed determined Luo beforehand is characterized by restoring the distribution object data using the error correction information.
[0011]
By using the present invention, original distribution data can be obtained by combining only the divided data normally distributed from the distribution server. Also, even when the distribution data from some servers is interrupted, the missing part can be restored and the original distribution data can be obtained by combining the data distributed from the other servers with the error correction information. .
[0012]
Furthermore, even if the transfer speed is lower than the speed required for real-time distribution of moving images, etc. due to network failure or abnormal load on the server, etc., by combining the data distributed from other servers with the error correction information, the lack of And maintain the real-time property.
[0013]
Further, the present invention is characterized in that in the distribution method, parity is used as error correction information. Since error correction using parity requires a smaller amount of calculation than other error correction methods, robust data distribution can be performed without imposing a large load on the distribution server and the client.
[0014]
Further, according to the present invention, in the above-mentioned distribution method, a list server holding a list of distribution servers on a network is provided, and the client can receive data from some servers due to a network failure or a distribution server failure. Or, when the transfer speed falls below a predetermined value, the information of another distribution server is obtained from the list server, the distribution request destination is changed, and data reception is resumed, thereby returning to the normal state. And
[0015]
Although data distribution can be continued even if a failure occurs in some servers or networks, data distribution cannot be continued if a failure occurs in another part and exceeds the limit of the error correction function. Will be possible.
[0016]
To avoid this, at the same time that the client detects a failure, the client requests information of the replacement server from the list server, disconnects the connection with the failed server, and changes the distribution server to the replacement server based on the information obtained from the list server. change. By this operation, when the normal state is restored, even when a new failure occurs, it is possible to cope with the error correction function.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 is a network configuration diagram according to an embodiment of the present invention. Reference numeral 1 denotes a client that requests data distribution, 2 denotes a list server that holds a list of distribution servers, 3 to 7 denote distribution servers A to E that distribute the stored data, and 8 denote networks.
[0018]
To start data distribution, first, the client 1 connects to the list server 2 and transmits information (file name, ID, etc.) of data requested to be distributed. The list server 2 returns information on the distribution server group holding the data requested by the client 1.
[0019]
FIG. 2 shows an example of the content of the information sent from the list server 2. This information is based on the IP addresses of the distribution servers A3 to E7 and the number of other clients (connections) connected to the server. Number).
[0020]
The client 1 selects, for example, four distribution servers in ascending order of the number of connections. If there are multiple servers with the same load in the selection, the server is selected at random. Hereinafter, the description will be given on the assumption that the selected distribution servers are the distribution server A3, the distribution server B4, the distribution server C5, and the distribution server D6.
[0021]
In data distribution, distribution target data is considered to be divided into 32 KB blocks. The client 1 distributes the nth block to the distribution server A3, the (n + 1) th block to the distribution server B4, the n + 2th block to the distribution server C5, and the n, n + 1, n + 2th block to the distribution server D6. Requesting the distribution of parity data. When the distribution of the n, n + 1, n + 2th blocks is completed, the distribution of the n + 3, n + 4, n + 5th blocks is performed in the same manner. Thereafter, this operation is repeated until the distribution of all the data to be distributed is completed.
[0022]
The distribution server D6 generates parity data by calculating the exclusive OR of the blocks n, n + 1, and n + 2 as shown in FIG. If the size of the data does not become a multiple of 96 KB, the calculation is performed after supplementing 0 to make it a multiple of 96 KB.
[0023]
Data distribution is performed under the initiative of the client 1. Hereinafter, an example of the operation of the client 1 will be described in detail. FIG. 4 is a block diagram of the client 1. 11 is a control unit for instructing connection to the distribution server or the list server 2 and selecting a distribution server, 12 is a timer, 13 to 16 are server connection units A to D, 17 is a parity buffer in which the distributed parity data is written, Reference numerals 18 to 20 denote buffers A to C in which distributed data is written, and reference numerals 21 and 21 denote applications (programs) for reproducing moving images.
[0024]
In the embodiment of the present invention, the control unit 11 instructs the server connection unit A13 to connect to the list server 2 and obtain a list as shown in FIG. Upon receiving the list, the control unit 11 selects four distribution servers in ascending order of the number of connections, and instructs the server connection units A13 to D16 to connect. At the same time, the timer 12 is set in accordance with the transfer rate set by a request from the application 21 or directly specified by the user. Specifically, assuming that the designated transfer rate is mKbps, in this example, the size of one block is 32 KB and three blocks of data are transmitted at the same time, so that one byte is 8 bits. , 32 × 3 × 8 / msec = 768 / msec, and a value obtained by adding a margin time αsec in consideration of a change in the transfer speed.
[0025]
As described above, the server connection units A13 to C15 request each server to distribute n, n + 1, and n + 2 blocks of the distribution target data, and the distributed data is written to the buffers A18 to C20, respectively. The server connection unit D16 requests the distribution server D6 to distribute the parity data, and the distributed data is written to the parity buffer 17. Although the buffers A18 to C20 are actually one buffer, they are logically divided into three for convenience of explanation. When the reception of each block is completed, the server connection units A13 to C15 notify the control unit 11 of the reception completion.
[0026]
Hereinafter, an example of the operation of the control unit 11 will be described with reference to the flowchart of FIG. The control unit 11 waits until there is a notification of block data reception completion from the server connection units A13 to C15 or a signal from the timer 12 (step S1). When the notification or the signal from the timer 12 is detected, it is determined whether the notification is a notification of the completion of reception from any of the distribution servers A3 to C5 (step S2), and if the notification is from the distribution server, the distribution server A3 is determined. It is determined whether or not block distribution from all three distribution servers of distribution server C5 has been completed (step S3). When all the block distributions have been completed, the contents of the buffers A18 to C20 are transferred to the application 21 (step S8). If all block distributions have not been completed, the process returns to the standby state (step S1).
[0027]
If there is a signal from the timer 12 in the standby state in step S1, it means that the delivery has fallen below the designated transfer speed or a failure has occurred. Therefore, the control unit 11 determines whether the block distribution from three of the four distribution servers has been completed (step S4). Then, when block distribution from three of the four servers has been completed, a recovery operation is performed.
[0028]
In this recovery operation, assuming that the server whose distribution was not in time is the distribution server A3, the control unit 11 selects one of the unselected distribution servers having the smallest number of connections from the list received from the list server 2. (This is referred to as the distribution server E7), and the server connection unit A13 in charge of the distribution server A3 is instructed to change the connection destination to the distribution server E7 (step S5).
[0029]
Next, it is determined whether the parity data has been received (step S6). If the parity data has been received, the parity buffer 17 is used to supplement the data in the buffer A18 (step S7). That is, the control unit 11 calculates the exclusive OR of the parity data of the parity buffer 17 and the block data of the buffers B19 and C20, and writes the calculation result to the buffer A18. Then, the contents of the buffers A18 to C20 are transferred to the application 21 (step S8).
[0030]
It is the parity delivery server D6 that failed to deliver in time, and when the parity has not been received (when the determination in step S6 is NO), the contents of the parity buffer 17 are ignored and the application 21 The contents of A18 to C20 are transferred (step S8). In step S4, if the delivery from two or more servers is not in time (if the determination in step S4 is NO), the error is notified to the application 21 because the block data cannot be restored (error correction) (step S4). S9).
[0031]
The processing performed by the control unit 11 can be realized by a computer and a software program. The program is stored in an appropriate recording medium such as a computer-readable portable medium memory, a semiconductor memory, and a hard disk. , Can be executed by a computer by reading from there.
[0032]
FIG. 6 is a block diagram of the distribution server A3. Other distribution servers B4 to E7 are similarly configured. In the distribution server A3, reference numeral 30 denotes a distribution target data storage unit for storing distribution target data, 31 a distribution request receiving unit for receiving a distribution request from the client 1 via a network, and 32 a distribution request for analyzing the received distribution request. Analysis means, 33 is distribution target data dividing means for dividing the distribution target data into a plurality of blocks, 34 is division distribution data transmitting means for transmitting a part of the divided data to the client 1, and 35 is distribution target data divided into a plurality of blocks. Error correction information generating means for generating error correction information for the client 1, and error correction information transmitting means for transmitting the error correction information to the client 1.
[0033]
When the distribution request receiving unit 31 receives a distribution request from the client 1 via the network 8, the distribution request analyzing unit 32 analyzes the distribution request. In the distribution request, for example, the name of the data to be distributed (file name, etc.), the divided block size of the data to be distributed, the transfer speed, and the number of blocks from which block of the divided data should be transferred (by the number of distribution servers). Transfer) or error correction information is transferred. If the divided block size, transfer speed, and how many blocks to transfer the data to be distributed are fixed in the system in advance, it is unnecessary to specify such information in the distribution request.
[0034]
When distribution of the distribution target data is requested instead of the error correction information, the distribution target data dividing unit 33 reads the distribution target data from the distribution target data storage unit 30 and divides the distribution target data by the designated division block size. The divided distribution data transmitting unit 34 transmits, to the client 1, a block in which the own device is responsible for distribution of the divided data.
[0035]
When the distribution of the error correction information is requested, the error correction information generating means 35 generates the error correction information for the blocks transmitted by the other n distribution servers by, for example, the method described with reference to FIG. The information transmitting unit 36 transmits the error correction information to the client 1.
[0036]
【The invention's effect】
By using the present invention, even when the distribution data from some servers is interrupted or the transfer speed is reduced, the lost portion can be restored and the data distribution can be continued while maintaining the real-time property. In particular, by using parity as error correction information, robust data distribution can be performed without imposing a large load on the server and the client.
[0037]
In addition, since the distribution server can be changed dynamically while data is being distributed, even if a failure occurs in some servers or networks, data distribution can be performed even if another failure occurs. Can be continued.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a network configuration according to an embodiment of the present invention.
FIG. 2 is a diagram showing an example of the content of information sent from a list server.
FIG. 3 is a diagram illustrating generation of parity data by a parity distribution server.
FIG. 4 is a block diagram of a client according to the embodiment of the present invention.
FIG. 5 is a diagram illustrating an example of an operation processing flow of a control unit of a client.
FIG. 6 is a block diagram of a distribution server according to the embodiment of the present invention.
[Explanation of symbols]
1 client 2 list server 3 distribution server A
4 Distribution server B
5 Distribution server C
6 distribution server D
7 Distribution server E
8 Network 11 Control unit 12 Timer 13 Server connection unit A
14 Server connection part B
15 Server connection part C
16 Server connection part D
17 Parity buffer 18 Buffer A
19 Buffer B
20 Buffer C
21 Application 30 Distribution target data storage unit 31 Distribution request reception unit 32 Distribution request analysis unit 33 Distribution target data division unit 34 Divided distribution data transmission unit 35 Error correction information generation unit 36 Error correction information transmission unit

Claims (7)

A data distribution method having a plurality of distribution servers and distributing distribution target data to a data receiving device via a network,
The distribution server divides the distribution target data into a plurality of blocks,
The distribution target data of n (n ≧ 2) blocks successively successive from the head are shared one block at a time, and distributed to the data receiving devices by the n distribution servers,
A data distribution method, wherein the error correction information on the n blocks of data to be distributed is simultaneously distributed to a data receiving device by another distribution server. The data distribution method according to claim 1,
A data distribution method, wherein a parity of the data to be distributed of the n blocks is used as the error correction information. A data distribution system having a plurality of distribution servers and distributing data to be distributed to a data receiving device via a network,
Each of the delivery servers,
Means for retaining the same data to be distributed,
Means for dividing the distribution target data into a plurality of blocks;
Means for generating error correction information for n (n ≧ 2) blocks that are successively consecutive from the beginning of the divided distribution target data;
Means for sharing and delivering one specified block of n blocks of data to be delivered sequentially from the beginning in response to a delivery request from the data receiving apparatus, or delivering the error correction information A data distribution system characterized by the following. A divided distribution data receiving method for receiving distribution target data divided into a plurality of blocks from a plurality of distribution servers,
Of the divided distribution target data, the distribution target data of n (n ≧ 2) blocks that are sequentially consecutive from the top are distributed to n distribution servers one block at a time and received from the n distribution servers. And
At the same time, error correction information on the n blocks of data to be distributed is received from another distribution server,
When n blocks of distribution target data are received from the n distribution servers, they are combined into continuous n blocks of reception data,
When the distribution target data from some distribution servers cannot be received due to a failure among the distribution target data from the n distribution servers, or the distribution target data from some distribution servers within a predetermined time. Is not received, the data to be delivered of the block that could not be received is restored using the error correction information received from the other delivery server, and the received block and the restored block are combined to continue. A divided distribution data receiving method, characterized by using n blocks of received data. 5. The method according to claim 4, wherein:
When the distribution target data from some distribution servers cannot be received due to a failure among the distribution target data from the n distribution servers, or the distribution target data from some distribution servers within a predetermined time. When the distribution request data is not received, the distribution request destination is changed to another new distribution server having a distribution function of the distribution target data, and the distribution target data of the block for which some of the distribution servers are in charge of distribution is changed. , Receiving data from another new distribution server. A divided distribution data receiving device that receives distribution target data divided into a plurality of blocks from a plurality of distribution servers,
n data receiving means for respectively receiving n blocks of distribution target data divided and distributed from n distribution servers without duplication,
A buffer for storing n blocks of distribution target data received by the n receiving units;
Means for receiving error correction information for the n blocks of distribution target data from a distribution server different from the n distribution servers;
A buffer for storing received error correction information,
Time monitoring means for monitoring whether data to be distributed has been received within a predetermined time;
When the n blocks of data to be distributed are received within a predetermined time, the data is combined to restore a series of n blocks of data to be distributed, and data from some distribution servers cannot be received due to a failure. Or a control unit for performing control to restore the data to be distributed using the error correction information when the transfer rate becomes equal to or less than a predetermined transfer rate. A divided distribution data receiving program for causing a computer of a data receiving apparatus to execute the divided distribution data receiving method according to claim 4 or 5.

Images