KR20060073724A - Method and device for storing and downloading duplicate files by using file information - Google Patents

Abstract

A method and apparatus for storing and downloading duplicate files using information of a file.

According to an embodiment of the present invention, a method of storing a duplicate file by using information of a file includes receiving check data for at least a portion of a first file from a client, and having the same check data as the received check data. Retrieving a second file, if the second file exists, comparing the first file with the second file from the client to extract a same portion and a different portion and the same portion of the first file Refers to the second file and includes storing only the different portion of the first file.

Duplicate File, File Server, CRC, Segment

Description

Method and apparatus for storing and downloading duplicated file using different file information}

1 is an exemplary view showing a case where a conventional duplicated large file is stored for each user.

2 is an exemplary diagram in which a plurality of identical files are stored as one file according to an embodiment of the present invention.

3 is an exemplary view showing a configuration of a client and a file server according to an embodiment of the present invention.

4 is an exemplary view illustrating a process of processing some duplication of two files according to an embodiment of the present invention.

5 is an exemplary diagram in which different parts of a duplicate file are stored in a file server according to an embodiment of the present invention.

6 is a flowchart illustrating a process in which a file server receives a file from a client according to an embodiment of the present invention.

7 is a sequence diagram showing operations between a client and a server according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

500: client 520: segment generator

530: check file generation unit 950: file information unit

960: file storage unit 1000: file server

A method and apparatus for storing and downloading duplicate files using information of a file.

Recently, with the increase of multimedia technology and the commercialization of mass storage media, the size of multimedia files and the storage capacity of digital devices are steadily increasing. In addition, in the past, the method of using a company's storage device that provides a large amount of storage space is gradually moving away from storing the file on a personal digital device. However, many multimedia files stored in a large storage space are often duplicated. In particular, in the case of a multimedia file, the user is unlikely to manipulate or change the file, which reduces the storage space efficiency in that the same multimedia file is likely to be stored by different users at similar time points.

Programs that find duplicate files in personal digital storage devices that users store are currently in circulation. It checks for the type of files stored on your personal computer and finds duplicate files.

However, in the storage service, where various users store files, searching through a large number of individual files and searching for duplicates takes tremendous time, so it is difficult to apply them to a file storage server that provides a real storage service. . In addition, since the file storage server providing the storage service is not configured on a single computer, but is configured by connecting computers having a plurality of mass storage devices, it is impossible to inspect each storage device.

In providing storage services, storing duplicate files is not only a waste of storage space. FIG. 1 shows a case in which duplicated large files are stored, and storage efficiency and network efficiency are reduced.

1 is an exemplary view showing a case where a conventional duplicated large file is stored for each user.

A user 10, 20, or 90 using the file server 1000 uploads the same file Movie_nemo.avi. In general, in the case of a multimedia file, attention is focused on the same content at a specific point in time, so that a plurality of multimedia files having the same content are often stored in the file server 1000. As a result, since the same files 201, 202, and 209 are stored in the file server 1000, unnecessary space is wasted, and since the same files are repeatedly uploaded, the network load is increased.

Therefore, there is a need for a method of avoiding duplication of files uploaded to a large file server, efficiently processing duplicate files, and reducing storage space and network load.

The technical problem to be achieved by the present invention is to increase the efficiency of the storage space by not storing the overlapping portion of the file.

Another technical problem to be achieved by the present invention is to reduce the load on the network by limiting the upload of duplicate files.

The objects of the present invention are not limited to the above-mentioned objects, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

A method and apparatus for storing and downloading duplicate files using information of a file.

According to an embodiment of the present invention, a method of storing a duplicate file by using information of a file includes receiving check data for at least a portion of a first file from a client, and having the same check data as the received check data. Retrieving a second file, if the second file exists, comparing the first file with the second file from the client to extract a same portion and a different portion and the same portion of the first file Refers to the second file and includes storing only the different portion of the first file.

According to another aspect of the present invention, there is provided a method of storing a duplicate file using information of a file, the method comprising: receiving check data of at least some files of a first filegroup from a client, the same check as the received check data Retrieving a second filegroup having data, comparing the first file of the first filegroup with the second file of the second filegroup from the client if the second filegroup exists; and The first file, which is identical to the second file, refers to the second file, and the first file, which is not the same as the second file, is stored.

According to an embodiment of the present invention, a method of downloading a duplicate file by using information of a file may include receiving information about a first file to be downloaded from a client, wherein the second file having the same portion as the first file may be selected. Searching for existence, if the second file exists, the same part of the first file and the second file is extracted from the second file and sent to the client, and the first file and the second file The different portion of the includes extracting from the first file and sending it to the client.

The file server according to an embodiment of the present invention is a transceiver for receiving check data for at least a portion of a first file from a client, searching for a second file having the same check data as the received check data, and receiving the received data. If the file information unit storing the check data and the second file exist, comparing the first file and the second file from the client and extracting the same and different parts, the same part refers to the second file. And the different portion includes a file storage portion for storing using a portion of the first file.

Prior to the description, the meaning of terms used in the present specification will be briefly described. Nevertheless, it should be noted that the terminology is used to limit the technical spirit of the present invention unless it is explicitly described as limiting the present invention as an explanation of the present specification.

-CRC (Cyclic Redundancy Checking)

CRC is one of the methods for checking if there is an error in the data transmitted on the communication link. The transmitter applies a 16-bit or 32-bit polynomial to the block of data to be transmitted, and adds the resulting code to the block. The receiver applies the same polynomial to the data and compares the result with that sent by the sender. If the two match, the data has been successfully received, otherwise the sender is requested to retransmit the data block. The ITU-T has established a standard for 16-bit polynomials used to obtain codes to be added to transmission blocks. The 16-bit CRC guarantees 99.998% detection for all possible errors, including when two bits fail at the same time. This degree of detection assurance is estimated to be sufficient for data block transmission of 4 KB or less, and 32-bit CRC is used for further large-scale transmission. Both the Ethernet and Token ring protocols also use 32-bit CRCs. A slightly less complicated and less error-prone method is the checksum method. The checksum method counts the bits in a transmission unit so that the receiver can verify that the same number of bits have arrived. If the calculation is correct, it is considered smoothly received without errors. Checksum calculation and verification services are provided at both the TCP and UDP communication layers.

- file

A file is usually a unit for handling data. Therefore, one multimedia content, one copyrighted work, etc. are usually composed of one or more files. In the present specification, as a main embodiment, one file exists as one meaningful data, but is not limited thereto. Segments to be described later may exist as a single file. Especially, in the case of a file server, a file that was originally one may be divided into several segments according to a predetermined size or a predetermined criterion to check whether a file is duplicated. Can be saved as a file. Therefore, a file server includes both a single file and a multi-segment file.

-Check file

The check file is a kind of check data. Check data can be obtained through CRC, checksum for a file or part of a file, and can also include the file size, file name, file type, and so on. Check file means to save the check data in the form of a file in the file server. When stored in the form of a file, the file server may check whether the files are identical by using the check file as the check data when uploading. In the present specification, since the check data is partially stored in the database, and part of the check data is stored in the form of a file, the check data is referred to as an embodiment of the check data. However, the present invention is not limited thereto, and the check data may not be stored in the form of a file but may be generated by calculating at every upload moment. It can also be stored in a database for searching.

2 is an exemplary diagram in which a plurality of identical files are stored as one file according to an embodiment of the present invention.

Many users upload the Movie_nemo.avi file to the file server 1000. Here, only one user's Movie_nemo.avi file 210 is stored, and the other user is notified that the file is stored. A counter 291 is used to indicate that N users have stored a Movie_nemo.avi file stored in the file server 1000. Thereafter, when K of N users requests deletion of the file, the file server 1000 does not delete the file directly, but changes the counter 291 to N-K. If this number is zero, there is no user who has stored this file anymore, so it is deleted from the file server 1000. If someone tries to save the same file again, increment the counter 291 by one.

3 is an exemplary view showing a configuration of a client and a file server according to an embodiment of the present invention.

The term '~ part' used in this embodiment, that is, '~ module' or '~ table' means a hardware component such as software, FPGA or ASIC, and the module performs certain functions. However, modules are not meant to be limited to software or hardware. The module may be configured to be in an addressable storage medium and may be configured to play one or more processors. Thus, as an example, a module may include components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, procedures, subroutines, and the like. , Segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and modules may be combined into a smaller number of components and modules or further separated into additional components and modules. In addition, the components and modules may be implemented to play one or more CPUs in a device or secure multimedia card.

The client 500 is installed in the above-described user's computer 10, 20, 90, and uploads and downloads a file. The client 500 includes a transceiver 510 for largely transmitting and receiving files and data, a segment generator 520 for dividing one file into segments according to a predetermined size or a predetermined criterion, and one file or a segment. A check file generating unit 530 for generating a check file, and a file storage unit 560 in which the file is stored.

The transceiver 510 transmits and receives data and files to and from a file server. Send and receive files over wired and wireless networks. The file to be transmitted and received may be one file or may be a segmented file.

The segment generator 520 divides the file according to a predetermined length or a predetermined criterion. This is because it may take time to perform the entire file to check for duplicate files, so that the file can be divided by a certain size. If a large file is already divided into a predetermined size, it may be transmitted without going through the segment generator 520.

The check file generating unit 530 extracts information from a file or a segment that is a part of the file, and generates a check file for examining whether the file or the segment is identical. The generation method of the check file may be the above-described CRC method or checksum method. In addition, a check file can be created based on the information, size, and the like of a file or a segment.

The file storage unit 560 means a medium in which a file to be uploaded is stored or a file to be downloaded is stored. Storage of large files is made through a hard disk, but in recent years, it has been diversified into a flash memory, a USB storage device, and the like.

The client 500 transmits the information of the file to be uploaded to the file server 1000 before uploading the file to the file server 1000. As information to be transmitted, the identity of a file can be examined. For example, the file can be a check file, a file size, a file name, etc. for the file or the first segment of the file. If the multimedia content exists as a single file, the segment generator 520 divides the segment into segments, and then the check file generator 530 generates a check file. The transmitter / receiver 510 transmits file information, for example, a check file, a file size, a file name, and the like, to the file server 1000 so that the same file is stored in the file server 1000. Using the received information, the file server 1000 stores only the information on the file to be uploaded without receiving the file when the file is the same file. If the part is a different file, the client 500 is requested to receive only the different part. The method of examining the different parts will be described later.

The file server 1000 includes a transceiver 910, a segment generator 920, a check file generator 930, a file information unit 950, and a file storage unit 960.

The transceiver 910 transmits and receives files and data with the client 500. The client 500 may transmit and receive data and files via a wired or wireless network. When the client 500 transmits a file, the segment generator 920 generates and stores the segment as a segment. If there is no file duplicated with the file uploaded by the client 500, the client 500 may transmit the file without dividing the file into segments. The segment generator divides the received file into segments so that the received file can be reviewed as a duplicate.

The check file generator 930 generates a check file for each segment. When duplicate files are not stored in the file server 1000, check files may be generated for all segments. On the other hand, if there is a duplicate file, only the segments corresponding to the different parts are stored, and a check file for the segment is generated. After that, when the user wants to download the file, the user can distinguish the segment of the original file from the segment different from the original file. This will be described in detail with reference to FIG. 4.

The file information unit 950 stores information about which file has what information. The file may include a part of the check file of the file, and may include information necessary for examining the identity of the file such as the file size and the file name. Also, if the original file and the same or some different file are uploaded based on this original file, the original file can be found. If there is another file that refers to the original file, the original file will not be deleted. Can be controlled. Information about files can be stored using databases, information files, and so on.

The file storage unit 960 is a storage medium for storing a file. It is a medium suitable for storing a large amount of files, and the file storage unit 960 may be distributed and installed in various systems. The file information unit 950 may provide information for knowing which file is stored in the file storage unit 960 of the system.

In the embodiment of FIG. 3, both the file server and the client have a check file generator, and present a method of storing the check file. However, this corresponds to an embodiment, and check data can be generated and compared when uploading or downloading. However, it is preferable to store the check data in the form of a file in consideration of the upload speed, and it is also possible to use a searchable part such as a database even if the file is not in the form of a file.

4 is an exemplary view illustrating a process of processing some duplication of two files according to an embodiment of the present invention.

If the multimedia files are all the same, only one file can be uploaded to manage how many users have stored this file through data such as a database or a file. However, in order to know whether a file is the same, there may be a method of checking each bit of the file, and there is a method of checking whether the file is the same by comparing information representing the file. Examining each bit of the file is time consuming due to its size in the case of multimedia files.

Therefore, comparing the information that represents the file may take less time. The information indicating the file may be a file size, a file name, or a specific part of the file, or a pattern of bits or a CRC check value constituting the specific part.

In Fig. 4, the pattern of the bits of the file is checked and the result values thereof are compared to examine the identity of the file. 101 is a file to be uploaded by User 1, and is stored in a personal computer or a digital device. This file is reviewed as a non-duplicate file in the file duplicate check and stored as an original file on the file server. The way in which the file is stored is divided into one or more segments (hereinafter, referred to as 'segments') (201, 202, 203, etc.). These segments 201, 202, 203, etc. mean that the file is divided into predetermined sizes according to one embodiment of the present invention. Each segment is stored on a file server. And check files (301, 302, etc.) indicating each segment can be further generated. The checkfile shows the characteristics of the segment and is the basis for distinguishing the two segments. As shown in FIG. 4, check file 1 301 is a file representing the characteristics of segment 1 201, and check file 2 302 is a file representing the characteristics of segment 2 202. In general, a check file is generated through a CRC, a checksum method, or the like.

The next user, User 2, wants to upload 102 files. To check whether the 102 file is the same as another file stored in the server, the check file 311 of the first segment 211 of the 102 file is sent to the file server. The file server compares the received check file 311 and the stored check file 301 and finds that the two segments 201 and 211 are the same segment. Also, if the size of the 102 file is similar or similar to the size of the file uploaded by User 1, the two files are likely to be duplicated. Therefore, 101 files composed of segments such as 201 and 202 are used as original files, and only the non-overlapping portions are stored.

Since the server does not need to store the 211 segment that is the same as the 201 segment, the server does not receive the 201 segment and compares the checkfile 312 and the stored checkfile 302 of the next segment 212. Again, the same result is obtained, so the second segment 212 is not stored. As a result of receiving and examining the check file 313 of the third segment 213, it is found that the two check files 303 and 313 are different. This means that the two segments 203 and 213 are different and thus receive and store the 213 segment. Then, a check file 313 for the 213 segment is generated. Thereafter, by comparing the segments of the 102 file by the above-described method, the segment can be stored only when the check file is different, thereby avoiding the storage of the duplicated segment, thereby improving efficiency.

Then, when the user 2 wants to download the 102 file again, the user downloads the segment file such as 201 or 202 as an original, and also downloads the segment 213 stored in a different part from the original file.

4 stores different parts and check files thereof. On the other hand, the same part, for example, segment 1 (211), segment 2 (212) should refer to the original segment segment 1 (201), segment 2 (202), so this information can be stored. Reference information can be stored in a file format or stored in a database.

In the embodiment of Fig. 4, the client sends a check file before uploading the file and compares it with other check files stored in the server. Since the check file is smaller than the segment file represented by the check file, the amount of data transmitted and received through the network is relatively small. In addition, when the same or only some segments are different, only the check file is continuously transmitted and received and compared, and only different segments are received according to the result, so that a user uploading a duplicate file has a high upload speed. Not only the efficiency of storage space but also network transmission can be saved.

In the embodiment of FIG. 4, one segment may be a predetermined size, for example, 64 Kbytes or 128 Kbytes, and the check file may be 4 bytes or 8 bytes. The size of the checkfile varies with the size of the segment.

In addition, the segment may be divided into segments in the actual client and the file may be stored, and the client may divide and transmit the segment in segments. The check file can also be generated by the client and sent in the form of check data for each segment. In the present specification, a file includes a concept of a file stored in a storage medium, and also includes a concept of a series of bitstrings that are located as data in a predetermined area of a memory.

In addition, even if a file is viewed from the user's point of view, the file may be divided into several segments and stored in the form of a file. In this case, what is recognized as one file from the user's point of view means one file group, and segments constituting the file group may each be one file. In this case, the segment transmitted by the client may be one file. Similarly, segments stored on file servers can also exist as independent files. In particular, it refers to the original file, and for some different files, each segment can be stored as a separate file. In this case, files are uploaded and downloaded from the user's point of view, and detailed segments of the file group exist as files in the server and the client.

There are two methods for selecting segments for comparison, two segments for selecting a segment at the same position in two files and a segment at a position separated by a predetermined number of bits in a specific region. In the former case (segment selection at the same position), the overall size of the file is the same, and it is applicable when some areas are modified. The latter case (segment selection separated by a certain offset from a specific area) is applicable when certain parts have been added or deleted compared to the original file. For example, the source contains segments 1, 2, 3, 4, and 5, the file to upload is only segments 1, 3, 4, and 5, and segment 2 is deleted. Does not need to be uploaded because it already exists in the original file. However, if you select a segment that is in the same location, compare segment 2 of the original with segment 3 of the uploaded file, compare segment 3 of the original with segment 4 of the uploaded file, and segment 4 of the original file and segment 5 of the uploaded file. By comparing them, we can recognize them as different segments. Therefore, segments 4 and 5 can be judged as the offset in segment 3, respectively, and can be compared.

In an exemplary embodiment of the present invention, each segment may be compared by comparing two files, or a check file of each segment may be compared. Of course, these files can be directly compared to examine the identity of the files, but there is a possibility that a large amount of time is required when directly comparing files in a large multimedia file. It is desirable to.

The check file may be generated according to the CRC method described above. In addition to the CRC scheme, the bits of a file or segment can be calculated to yield a result that is non-overlapping or unlikely to overlap. If the checkfiles containing these values are the same, it can be determined that the files or segments have the same values.

5 is an exemplary diagram in which different parts of a duplicate file are stored in a file server according to an embodiment of the present invention.

The file server 1000 includes a file information unit 950 and a file storage unit 960. Of course, in addition to the above, as shown in FIG. 3, there is a transceiver for transmitting and receiving a file and communicating with a client.

The file information unit 950 has information about a stored file. The file identifier, the location where the file is stored, the full checkfile of the checkfiles for the file, the checkfile for the first segment of the file, the size of the file, information about the original file if it is original or not original, and the reference count (counter ), And the like. The file server 1000 may search the information of the file information unit 950 to examine whether the file to be uploaded is a duplicate file. If the first check file is the same and the file size is the same or the entire check file is the same as the stored file, the file information unit 950 indicates that the file is the same and does not store the file.

On the other hand, if the first check file and the file size are the same, but the entire check file is different, it means that some segments of the two files are different, as shown in Figure 4 can be stored only different segments. As shown in the file storage unit 960, FILE_0002 stores a segment, a check file, and an entire check file different from FILE_0001. In addition, the counter field of FILE_0001 can be set to 2 to indicate that FILE_0002 originates from FILE_0001.

Afterwards, when the user wants to download FILE_0002, the file information unit 950 learns that FILE_0002 uses FILE_0001 as an original. Therefore, the overlapped portions (segment 1, segment 3, etc.) are downloaded from FILE_0001, and the different portions (segment 2). Should be downloaded from FILE_0002.

On the other hand, when the user of FILE_0001 deletes his file, the counter of FILE_0001 is replaced with 1 instead of immediately deleting the file. The reason for replacing 1 is that the user of FILE_0002 needs FILE_0001 to download FILE_0002. After that, when the user deletes FILE_0002, the counter of FILE_0002 is decreased by 1 to 0, and the counter of the original file FILE_0001 is also reduced to 1 by 0. In the batch execution program or the deletion program, the file whose counter is 0 can be deleted from the file storage unit 960.

In FIG. 5, a case in which FILE_0002 has one file as an original file is shown. However, this is an embodiment of the invention and may have more than one original file. For example, if FILE_0003 is the same as the second segment of FILE_0002, FILE_0002 may have the second segment refer to FILE_0003. Alternatively, if FILE_0003 is the original file such as FILE_0002 and FILE_0001 is the same file as FILE_0002, FILE_0002 may be stored using FILE_0003 as the source rather than FILE_0001. This can be applied when storing several similar files with only a few differences.

6 is a flowchart illustrating a process in which a file server receives a file from a client according to an embodiment of the present invention.

The file server 1000 receives a file upload request from the client 500 (S101). The client 500 receives a file to be uploaded or a check file for a part of the file (S102). The check file may be a file generated by the above-described CRC, checksum method, and may include information on the size of the file to be uploaded. In addition, the file name or content type may be included together. In addition, it may include predetermined identifier information for identifying the multimedia content. Such identifier information is used to manage the copyright of the digital file and can be used when the multimedia content can be distinguished by this identifier. The file server 1000 receiving the check file including the information about the file searches whether the original file having the same check file as the check file exists (S103). The file information unit 950 of FIG. 5 may be used for searching.

As a result, if the original file exists (S111), only a different portion is received and stored (S113 to S142). Information about the file to be uploaded is stored in the file information unit 950 (S113). This is to allow the user to continue uploading because the user may stop the upload depending on the network or computer environment. Of the information on the file to be uploaded, information about the original file may be stored, such as storing a specific file identifier in the original field of FIG. 5. In addition, the information of the original file stored in the file information unit 950 is corrected (S114). This is to indicate that if the original file is deleted during upload, the original file is referenced by another file. In operation S115, a file to be uploaded to the client or a check file for a part of the file is requested. This may be requested from the file server 1000, or the check file may be sequentially transmitted from the client for comparison with the original file. The file server 1000 compares the received check file with the check file of the original file (S116). If the comparison result is the same check file (S131), it is checked whether the check file is the check file for the last segment or the last part of the file (S141). If it is the last, the file information unit 950 records that the upload is completed and ends (S142).

If it is not the end of the file, a check file for the next segment or the next part of the file is requested, and the processes S115 to S141 described above are performed.

On the other hand, if the check file is different in step S131, since the two segments are different, the check file is stored (S133). The client requests the client to transmit the file or part of the file indicated by the check file (S134). The received file or part of the file is stored (S135). The flow proceeds to step S141 described above. If not, the check file of the next segment is received and the above-described steps are performed.

On the other hand, if it is determined in step S111 that the original file does not exist, the file information unit 950 stores the information of the newly uploaded file (S121). In this case, it indicates that the file to be uploaded is the original. The client requests to send the upload file (S122). When receiving the upload file (S123), the file server generates a check file to use this file as an original file in the future (S124). The check file may be generated by receiving all the files in a batch. The check file may be continuously stored in segment units in the middle of receiving the file to generate a check file for the segment.

The flowchart of FIG. 6 adopts a method of receiving a different file portion from a client when the original file exists, compared to the check file of the original file.

In addition, when there are a large number of check files for a file to be uploaded, the client may transmit all the check files, notify the client of only the different parts of the file server, and receive only the corresponding segments.

7 is a sequence diagram showing operations between a client and a server according to an embodiment of the present invention.

The client 500 transmits file information (S201). The file information includes a check file, file size, file name, and the like for checking the identity. Alternatively, the check file including the information may be transmitted. The transmission / reception unit 910 receiving the information checks through the file information unit 950 whether an original file having this information exists (S205). As a result of the review, if the original file exists, it informs that the original file exists (S208). In addition, the transmitter / receiver 910 notifies the client 500 that only a different part is uploaded through the check file because the original file exists (S210).

The client 500 transmits a check file of a segment constituting the file to transmit a file or a part of the file that is not duplicated to the server. When the file to be uploaded by the client 500 is composed of a plurality of small files, each file may be a segment. The check file of the first segment is transmitted (S213). The transceiving unit 910 receiving the check file transmits the check file to the file storage unit 960 (S215). The received check file is compared with the check file of the original file (S217). If the comparison results in the same checkfile, there is no need to receive the file represented by the checkfile or a segment that is part of the file. Therefore, the comparison result is transmitted to the transceiver unit (S220). In addition, the transceiver 910 requests to transmit the check file of the second segment, which is the next segment (S222). The client S500 transmits a check file of the second segment (S225). Again, the transceiver 910 transmits the check file to the file storage unit 960 (S227). The file storage unit 960 compares the check file of the second segment of the original file (S229). As a result, if the two check files are not the same, the result is transmitted to the transceiver (S232). The transmitter / receiver requests the second segment to transmit the second segment because the second check file is not the same (S234). The client transmits a second segment (S237). The transceiver 910 transmits this segment to the file storage 960 (S239). The file storage unit 960 stores the segment and stores the check file for the segment (S241).

After that, the client and the file server store the different segments by examining whether they overlap each other. When the client transmits the check file of the last segment (S261), the transceiver 910 transmits the check file to the file storage 960 and compares it with the original. As a result, if the comparison result is transmitted to the transceiver 910 (S268), the transceiver 910 notifies the client 500 that the upload is finished (S270). Of course, if the check file of the last segment is different from the original, the process shown in S234 to S241 is performed. The file information unit 950 modifies or stores the information of the upload file and modifies the information of the original file (S272).

Those skilled in the art will appreciate that the present invention can be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. The scope of the present invention is indicated by the scope of the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and the equivalent concept are included in the scope of the present invention. Should be interpreted.

By implementing the present invention it is possible to increase the efficiency of the storage space by storing only the non-overlapping portions of the file.

By implementing the present invention it is possible to reduce the load on the network by limiting the upload of duplicate files.

Claims (24)

Receiving check data for at least a portion of the first file from the client; Searching for a second file having the same check data as the received check data; If the second file exists, comparing the first file with the second file from the client and extracting a same part and a different part; And And the same portion of the first file refers to the second file, and only storing different portions of the first file. The method of claim 1, The extracting step And comparing the check data for the first area of the first file with the check data for the second area of the second file. And a position occupied by the first region in the first file and a position occupied by the second region in the second file are the same. The method of claim 1, The extracting step And comparing the check data for the first area of the first file with the check data for the second area of the second file. The first region and the second region have the same bitstring in the first file and the second file and have the same offset as the common third region, and the duplicated file is stored using information of the file. . The method of claim 1, The storing step And storing check data for a portion of the first file. The method of claim 1, And the check data includes data obtained through a CRC or a checksum scheme for at least a portion of the first file. The method of claim 1, The check data includes information on the size of the first file, the method of storing a duplicate file using the information of the file. Receiving check data for at least some files of the first filegroup from the client; Searching for a second file group having the same check data as the received check data; Comparing the first file of the first filegroup with the second file of the second filegroup from the client when the second filegroup exists; And Storing the duplicate file using information of the file, the first file being identical to the second file referring to the second file, and storing the first file that is not equal to the second file. How to. The method of claim 7, wherein The comparing step And comparing the check data of the first file with the check data of the second file. The method of claim 7, wherein The storing step And storing the check data for the first file when the first file is stored. The method of claim 7, wherein And the check data includes data obtained through a CRC or a checksum method for the first file group. The method of claim 7, wherein And the check data includes information on the size of the first filegroup. Receiving information about a first file to download from the client; Searching whether a second file having the same portion as the first file exists; If the second file exists, the same portion of the first file and the second file is extracted from the second file and transmitted to the client, and different portions of the first file and the second file are stored in the first file. And extracting from the file and transmitting the extracted file to the client. The method of claim 12, And determining the same or different parts by referring to the check data for the first file. The method of claim 13, And the check data includes data obtained through a CRC or a checksum scheme for at least a portion of the first file. The method of claim 13, The check data includes information about the size of the first file, the method of downloading a duplicate file using the information of the file. The method of claim 12, The first file or the second file comprises a file group comprising two or more files, the method of downloading a duplicate file using the information of the file. A transceiver for receiving check data on at least a portion of the first file from a client; A file information unit searching for a second file having the same check data as the received check data and storing the received check data; And If the second file exists, compare the first file and the second file from the client to extract the same and different portions and store the same portion to refer to the second file, wherein the different portion is A file server comprising a file storage unit for storing using a portion of the first file. The method of claim 17, The transceiver receives information about a first file to download from the client, The file information unit searches whether the second file having the same portion as the first file exists by using the received information. When the second file is present, the transceiver unit extracts the same portion of the first file and the second file from the second file stored in the file storage unit and transmits the same to the client. And a different portion of the second file is extracted from the first file stored in the file storage and sent to the client. The method of claim 17, The file information unit compares and extracts check data for the first area of the first file and check data for the second area of the second file. And a position occupied by the first region in the first file and a position occupied by the second region in the second file are the same. The method of claim 17, The file information unit compares and extracts check data for the first area of the first file and check data for the second area of the second file. And the first region and the second region have a common bitstring for the first file and the second file and have the same offset as a third region that exists. The method of claim 17, The file storage unit stores the check data for a portion of the first file. The method of claim 17, And the check data includes data obtained via CRC or checksum scheme for at least a portion of the first file. The method of claim 17, And the check data includes information about the size of the first file. The method of claim 17 or 18, And the first file or the second file consists of a filegroup comprising one or more files.

Images