JP2001034517A - File replication system - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide a file replication system capable of suppressing the influence of the performance of a hard disk and improving the processing performance of file replication. SOLUTION: A master computer 1 has a master side replication execution unit 1a, and a master side hard disk device 4
From the master file 6 and read the data of the read master file 6 from the network 3.
And sequentially transmitted to the slave computer 2 side. The slave computer 2 is a slave-side replication execution unit 2
a, and the data of the master file 6 transmitted from the master computer 1 and the data of the slave file 7 read from the slave hard disk device 5 are compared on a byte-by-byte basis on the slave computer 2. The data of the detected difference is written to the slave file 7 held in the slave hard disk device 5.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a distributed processing system having a plurality of computers, and more particularly to a file replication system for maintaining files of the same contents in a plurality of computers.

[0002]

2. Description of the Related Art Conventionally, a copy (hereinafter, referred to as "slave file") of a file (hereinafter, referred to as "slave file") held by one computer (hereinafter, referred to as "master computer") is transferred to another computer (hereinafter, "slave computer"). A file replication system is known that is operated to maintain consistency between a master file and a slave file.

In such a conventional file replication system, in order to maintain consistency between a master file and a slave file, file replication for matching the contents of the slave file with the contents of the master file is periodically performed. In the file replication process, the master file is sent from the master computer to the slave computer via the network, and the contents of the slave file held by the slave computer are updated. Specifically, (1) the master file is read from the hard disk device of the master computer,
A series of processings of (2) transmitting a master file from the master computer side to the slave computer side via the network and (3) writing the contents of the master file to the slave file held in the hard disk device of the slave computer are performed. .

[0004]

However, in the above-described file replication processing, since all data of the master file is written to the slave file, the processing performance is significantly reduced due to the performance of the hard disk device. There's a problem. The factors that determine the processing performance of file replication include:
There is network performance in addition to the performance of the hard disk device. With the improvement of the network performance, the situation in which the performance of the hard disk device becomes a main bottleneck is increasing.

[0005] The present invention has been made in view of the above points, and it is an object of the present invention to provide a file replication system capable of suppressing the influence of the performance of a hard disk and improving the processing performance of file replication. I do.

[0006]

SUMMARY OF THE INVENTION The present invention provides a master computer, an external storage device connected to the master computer, and a master-side external storage device for holding a master file.
A slave computer connected to the master computer via a communication path, and an external storage device connected to the slave computer, the slave external storage device holding a slave file corresponding to the master file, When performing file replication for matching the contents of the slave file with the contents of the master file, the data of the master file and the data of the slave file are read from the master-side external storage device and the slave-side external storage device, respectively. Comparing the data of the master file with the data of the slave file, and writing the data of the difference detected by the comparison processing to the slave file held in the slave-side external storage device. Replication To provide a system.

In the present invention, it is preferable that a predetermined range of data including the data of the different portion detected by the comparison processing be written to a slave file held in the slave external storage device.

According to the present invention, the data of the master file transmitted from the master computer is compared with the data of the slave file read from the external storage device on the slave side, and the data of the different part detected by the comparison processing is compared. Is written to the slave file held in the slave-side external storage device. Therefore, although the read amount from the slave-side external storage device increases, the write amount to the slave-side external storage device can be reduced, so that the read performance of the external storage device is much higher than the write performance, and By taking advantage of the fact that file update locations tend to be concentrated at specific locations, the processing performance of file replication can be significantly improved.

Further, according to the present invention, when a difference is detected between the data of the master file and the data of the slave file by comparison processing, in addition to the data of the difference, the data of a predetermined range including the data is included. Is written to the slave file held in the slave-side external storage device, so that the number of comparison processes and the number of accesses to the slave-side external storage device (the number of times of writing) And the number of read operations) can be effectively reduced, so that the performance of file replication can be further improved.

[0010]

Embodiments of the present invention will be described below with reference to the drawings.

First Embodiment First, a first embodiment of a file replication system according to the present invention will be described with reference to FIGS.

FIG. 1 is a block diagram showing an overall configuration of a first embodiment of a file replication system according to the present invention. As shown in FIG. 1, the file replication system includes a master computer 1 and a master computer 1.
And a slave computer 2 connected to the slave computer 2 via a network (communication path) 3. Of these, master computer 1
Is connected to a master hard disk device (master external storage device) 4, and the master hard disk device 4 holds a master file 6. A slave hard disk device (slave external storage device) 5 is connected to the slave computer 2, and the slave hard disk device 5 holds a slave file 7 corresponding to the master file 6.

Here, the master computer 1 has a master-side replication execution unit 1a, reads data of the master file 6 from the master hard disk device 4, and transfers the read data of the master file 6 via the network 3. The data can be sequentially transmitted to the slave computer 2 side. Further, the slave computer 2 has a slave-side replication execution unit 2a, and stores the data of the master file 6 transmitted from the master computer 1 and the data of the slave file 7 read from the slave-side hard disk device 5 on the slave computer 2. , And the data of the different part detected by the comparison processing can be written (overwritten) to the slave file 7 held in the slave hard disk device 5.

Next, the operation of the first embodiment of the present invention having such a configuration will be described.

In FIG. 1, when performing file replication for matching the contents of the slave file 7 with the contents of the master file 6, the master-side replication execution unit 1 a of the master computer 1 transmits the data of the master file 6 from the master-side hard disk device 4. Is read, and the read data of the master file 6 is sequentially transmitted to the slave computer 2 via the network 3.

In this state, the slave-side replication execution unit 2a of the slave computer 2 performs processing as shown in FIGS. Here, FIG. 2 shows the slave-side replication execution unit 2 according to the first embodiment of the present invention.
FIG. 3 is a conceptual diagram for explaining an example of a file updating method according to the first embodiment of the present invention. Note that, here, for simplicity of explanation, a case in which reading and writing of data from the network 3 and the slave file 7 are performed in byte units will be described as an example. It is of course possible to read and write all at once.

As shown in FIGS. 2 and 3, the slave-side replication execution unit 2a of the slave computer 2 first substitutes 0 for a variable i representing an offset position from the start of a file (step 101).

Next, the value of the variable i is compared with the file size (the number of bytes) N (step 102). If the value of the variable i is not less than N, the processing is stopped (step 10).
3).

On the other hand, if the value of the variable i is smaller than N, the first one-byte data is read out from the byte string transmitted via the network 3 (step 10).
4) Substitute the read data for a variable x (step 105).

Next, 1-byte data stored at the offset position i of the slave file 7 is read (step 106), and the read data is substituted for a variable y (step 107).

Thereafter, the value of the variable x is compared with the value of the variable y. If they are the same, the value of the variable i is increased by 1 and the process returns to step 102 (step 109).

On the other hand, when the value of the variable x is different from the value of the variable y, the value of the variable x is written to the offset position i of the slave file 7 (step 110).
The value of the variable i is increased by 1 and the process returns to step 102 (step 109).

As described above, according to the first embodiment of the present invention, the master file 6 transmitted from the master computer 1
Is compared with the data of the slave file 7 read from the slave hard disk device 5 on the slave computer 2, and the different bytes detected by this comparison processing are compared with the slave file 7 held in the slave hard disk device 5. I try to write to. Therefore,
Although the amount of data read from the slave hard disk device 5 increases, the amount of data written to the slave hard disk device 5 can be reduced. Therefore, the read performance of the hard disk device is much higher than the write performance, and the file is updated. By taking advantage of the fact that locations tend to be concentrated at specific locations, the processing performance of file replication can be significantly improved.

In the above-described first embodiment, the comparison between the data in the master file 6 and the data in the slave file 7 is performed on the slave computer 2. However, the present invention is not limited to this. The data of the slave file 7 read from the side hard disk device 5 is transmitted from the slave computer 2 to the master computer 1 via the network 3, and the transmitted data of the slave file 7 and the read data from the master side hard disk device 4 are read. The data of the output master file 6 is compared on the master computer 1, and the data of the difference detected by the comparison processing is transmitted from the master computer 1 to the slave computer 2 via the network 3 and the slave The data may be written to the slave file 7 held in the hard disk device 5. This makes it possible to appropriately cope with a case where the bandwidth of the network 3 from the master computer 1 to the slave computer 2 is smaller than the bandwidth of the network 3 from the slave computer 2 to the master computer 1.
The same operation and effect as those of the first embodiment can be obtained.

In the above-described first embodiment, the master-side replication execution section 1a and the slave-side replication execution section 2a can be realized as programs operating on a CPU (Central Processing Unit). Such a program is stored in an internal storage device such as a memory or a hard disk on the master computer 1 or the slave computer 2, a flexible disk or a CD-ROM.
The functions described above are realized by being stored in various computer-readable recording media such as an external storage device, and sequentially read out from the CPU on the master computer 1 or the slave computer 2 and executed.

Second Embodiment Next, a second embodiment of the file replication system according to the present invention will be described with reference to FIGS. 1, 4 to 7. According to the second embodiment of the present invention, in addition to the difference data detected by the comparison processing, a predetermined range of data including the data is written to the slave file 7 held in the slave hard disk device 5. Except for this point, other parts are substantially the same as those of the first embodiment shown in FIGS. In the second embodiment of the present invention, the same portions as those in the above-described first embodiment are denoted by the same reference numerals, and detailed description is omitted.

In the second embodiment of the present invention, FIG.
When the slave-side replication execution unit 2a of the slave computer 2 compares the data of the master file 6 and the data of the slave file 7 on the slave computer 2, the data of the master file 6 and the slave file 7 are compared.
Are compared in order from the start end to the end, and when a difference is detected by this comparison processing, in addition to the data of the difference portion, the data from the position of the data to a position advanced by size M from the position of the data is compared. Writing to the slave file 7 held in the slave hard disk device 5 can be performed. Note that the value of M that defines the update range can be a fixed value smaller than the size of the slave file 7.

In FIG. 1, when performing file replication for matching the contents of the slave file 7 with the contents of the master file 6, the master-side replication execution unit 1 a of the master computer 1 transmits the data of the master file 6 from the master-side hard disk device 4. Is read, and the read data of the master file 6 is sequentially transmitted to the slave computer 2 via the network 3.

In this state, the slave-side replication execution unit 2a of the slave computer 2 performs processing as shown in FIGS. Here, FIG. 4 shows the slave-side replication execution unit 2 according to the second embodiment of the present invention.
FIG. 5 is a conceptual diagram for explaining an example of a file updating method according to the second embodiment of the present invention. Note that, here, for simplicity of explanation, a case in which reading and writing of data from the network 3 and the slave file 7 are performed in byte units will be described as an example. It is of course possible to read and write all at once.

As shown in FIGS. 4 and 5, the slave-side replication execution unit 2a of the slave computer 2 first substitutes 0 for a variable i representing an offset position from the beginning of a file (step 201).

Next, the value of the variable i is compared with the file size (the number of bytes) N (step 202). If the value of the variable i is not less than N, the processing is stopped (step 20).
3).

On the other hand, when the value of the variable i is smaller than N, the first one-byte data is read from the byte string transmitted via the network 3 (step 20).
4) Substitute the read data into a variable x (step 205).

Next, 1-byte data stored at the offset position i of the slave file 7 is read (step 206), and the read data is substituted for a variable y (step 207).

Thereafter, the value of the variable x is compared with the value of the variable y. If they are the same, the value of the variable i is increased by 1 and the process returns to step 202 (step 209).

On the other hand, when the value of the variable x is different from the value of the variable y, (M-1) bytes of data are further transferred from the head of the byte string transmitted via the network 3. Reading (step 210), the read data is converted from x '[1] to x' [M-1] of the array x '.
And assigns the value of the variable x to the array x '[0] (step 211).

Then, from x '[0] to x' of the array x '
M bytes of data up to [M-1] are collectively written to the offset position i of the slave file 7 (step 2).
12), increase the value of the variable i by M, and return to step 202 (step 213).

As described above, according to the second embodiment of the present invention, the data of the master file 6 and the slave file 7
Are compared on the slave computer 2 with the data of the master file 6 and the data of the slave file 7 in order from the start end to the end. When a difference is detected by this comparison processing, In addition to the data of the different part, the data from the position of the data to the position advanced by the size M is written to the slave file 7 held in the slave hard disk device 5.
Therefore, even when a large number of updated portions of the slave file 7 occur, the number of times of the comparison process and the number of times of access to the slave side hard disk device 5 (the number of times of writing and the number of times of reading) can be effectively reduced. Therefore, the performance of file replication can be further improved.

In the above-described second embodiment, the value of M defining the update range is fixed.
The present invention is not limited to this, and the value of M may be made variable according to the update frequency of the slave file 7. Specifically, for example, the value of M can be increased as the update frequency of the slave file 7 increases, and the value of M can be decreased as the update frequency of the slave file 7 decreases.

In the above-described second embodiment, when a difference is detected by the comparison processing, in addition to the data of the difference, the size M is determined from the position of the data.
Although the data up to the position advanced by only one is written in the slave file 7 held in the slave hard disk device 5, the present invention is not limited to this, and any file updating method can be adopted.

Specifically, for example, as shown in FIG. 6, the data of the master file 6 and the data of the slave file 7 are sequentially compared from the start end to the end, and when a difference is detected by this comparison processing, The data from the position of the data to the end position is added to the slave file 7 held in the slave-side external hard disk drive 5 in addition to the data of the difference, without performing further comparison processing on the file. Can be written.

As shown in FIG. 7, a file of size N is regarded as a sequence of blocks of size M in advance, and the data of the master file 6 and the slave file 7 are stored.
Is compared for each block of size M, and when a difference is detected by this comparison processing, the block is subjected to the data in addition to the data of the difference without further comparison processing. The data from the position to the last position in the block can be written to the slave file 7 held in the slave hard disk device 5.

[0042]

As described above, according to the present invention, the amount of writing to the slave-side external storage device can be reduced, so that the read performance of the external storage device is much higher than the write performance, and By taking advantage of the fact that file update locations tend to be concentrated at specific locations, the processing performance of file replication can be significantly improved. When a difference between the master file data and the slave file data is detected by the comparison process, in addition to the difference data, a predetermined range of data including the data is held in the slave external storage device. By writing to a slave file, the number of times of comparison processing and the number of times of access to the slave-side external storage device (the number of times of writing and the number of times of reading) can be effectively reduced even when a large number of updated portions of the slave file occur. Therefore, the performance of file replication can be further improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of a file replication system according to first and second embodiments of the present invention.

FIG. 2 is a flowchart for explaining the operation of the first embodiment of the file replication system according to the present invention.

FIG. 3 is a conceptual diagram for explaining an example of a file updating method in the first embodiment of the file replication system according to the present invention.

FIG. 4 is a flowchart for explaining the operation of the second embodiment of the file replication system according to the present invention.

FIG. 5 is a conceptual diagram illustrating an example of a file update method in a second embodiment of the file replication system according to the present invention.

FIG. 6 is a conceptual diagram for explaining a modification of the file updating method shown in FIG. 5;

FIG. 7 is a conceptual diagram for explaining another modified example of the file updating method shown in FIG. 5;

[Explanation of symbols]

Reference Signs List 1 master computer 1a master-side replication execution unit 2 slave computer 2a slave-side replication execution unit 3 network (communication path) 4 master-side hard disk device (master-side external storage device) 5 slave-side hard disk device (slave-side external storage device) 6 master File 7 Slave file

Claims (7)

[Claims] A master computer; an external storage device connected to the master computer, the master external storage device holding a master file; a slave computer connected to the master computer via a communication path; An external storage device connected to the slave computer, comprising: a slave-side external storage device that holds a slave file corresponding to the master file; and performing file replication for matching the content of the slave file with the content of the master file. When performing, the data of the master file and the slave file are read from the master-side external storage device and the slave-side external storage device, respectively, and the read master file data and the read slave file data are compared. And the comparison process detects A file replication system, wherein the data of the different part is written to a slave file held in the slave external storage device. 2. The data of the master file read from the external storage device on the master side is transmitted from the master computer side to the slave computer side via the communication path,
The transmitted master file data and the slave file data read from the slave external storage device are compared on the slave computer, and the difference data detected by the comparison process is compared with the slave data. 2. The file replication system according to claim 1, wherein writing is performed to a slave file held in the external storage device. 3. The slave file data read from the slave external storage device is transmitted from the slave computer side to the master computer side via the communication path, and the transmitted data of the slave file is transmitted. And the data of the master file read from the external storage device on the master side are compared on the master computer, and the data of the difference detected by the comparison process is transmitted from the master computer side via the communication path. 2. The method according to claim 1, wherein the data is transmitted to the slave computer and written into a slave file held in the slave external storage device.
The described file replication system. 4. The file replication system according to claim 1, wherein a predetermined range of data including the data of the different portion detected by said comparison processing is written to a slave file held in said slave external storage device. . 5. The comparison processing compares the data of the master file and the data of the slave file in order from the start end to the end, and when a difference is detected by the comparison processing, the data of the difference part is added. 2. The file replication system according to claim 1, further comprising: writing data up to a position advanced by a predetermined size from the position of the data to a slave file held in the slave-side external storage device. 6. The comparison process compares the data of the master file and the data of the slave file in order from the start end to the end, and when a difference is detected by the comparison process, the data of the difference portion is added. 2. The file replication system according to claim 1, wherein data from the data position to the end position is written to a slave file held in the slave-side external storage device. 7. The comparison processing compares the data of the master file with the data of the slave file for each block of a predetermined size, and when a difference is detected by the comparison processing, adds the data of the difference. 2. The file replication system according to claim 1, wherein data from the data position to the last position in the block is written to a slave file held in the slave external storage device.