CN105635264B - A File System Based on Network Game Application - Google Patents

Abstract

The invention discloses a file system based on an online game application, which includes a storage module, a search module, a unified download platform module and a download sharing module, wherein the storage module is used to store file information of the file system; the search module It is used to retrieve corresponding file information from the storage module; the unified download platform module is used to receive the download request of the corresponding file sent by the game client in one or more client process environments, and control the search module to retrieve the corresponding file information according to the download request. The file information; the download sharing module is used to analyze whether the download address is the same download address when the unified download platform module receives the download request, and returns the shared information of the downloaded file when it is the same download address. The invention makes the resource loading of the game faster, makes the resource file safer, increases the difficulty of illegally obtaining the resource, and makes the effect of the game file version upgrade on the player smaller and more transparent.

Description

A File System Based on Network Game Application

technical field

The invention relates to the field of Internet game applications, in particular to a file system based on network game applications.

Background technique

In recent years, with the rapid development of online games, the number of fully installed client games has gradually decreased. Being able to quickly enter the game has become a stepping stone for the game to go online. For this reason, the game client needs to be able to dynamically download the resources required for the game process, which brings about a series of related problems.

During the running of online game programs, a large number of files need to be read. In the Windows system, if these files are directly placed in the Windows file directory, not only the access speed is slow, the system cache cannot be used efficiently, nor can it be compressed to save space, but it is also not conducive to resources. security, it is easy for resource files to be stolen and modified.

Therefore, large-scale client games have a requirement, that is, file packaging, compression, encryption and reading. Ensure security and stability first, then focus on speed, and finally consider compression and file changes and updates.

At present, most of the high-end games at home and abroad have packaged their resource files, but it is rare that they can be dynamically updated online. It seems that some newly emerging online games such as Dragon Gun and Ba Dao in China have adopted similar technology. There are fewer applications in foreign games, probably because of the large network bandwidth in foreign countries, and it is not time-consuming to download the entire game. Regardless of whether it is domestic or foreign, there is currently no open source code available for download.

Therefore, the current technology manufacturers are actively developing new file systems. This file system needs to be suitable for the integrated management of a large number of client resource files, and has less strict security requirements, online download requirements, and file data without runtime modification. Where required. Based on this system, the client is easier to install and move, reducing the exposure of resource files makes data more secure and difficult to steal and tamper, using compression to reduce data size and hard disk reading time consumption, supporting dynamic download of required resources from the network, and when part When resources are updated, there is no need to re-download the data that has been cached by the client. In the future, it can also transparently optimize the performance of the application layer by continuously optimizing the internal implementation of the system.

Contents of the invention

In order to solve the above problems, the object of the present invention is to provide a file system based on online game applications, improve the speed of game loading resources, allow multiple processes to simultaneously read and download resources for the same package, and allow online background synchronization on the server. resources, and guarantee the security of resources to a certain extent.

The technical scheme that the present invention solves its problem adopts is:

A preferred embodiment of the present invention provides a file system based on online game applications, at least including a storage module, a search module, a unified download platform module, and a download sharing module, wherein:

The storage module is used to store file information of the file system;

The search module is used to retrieve corresponding file information from the storage module;

The unified download platform module is used to receive the download request of the corresponding file sent by the game client, and control the search module to retrieve the corresponding file information according to the download request;

The download sharing module is used to analyze whether the download address is the same download address when the unified download platform module receives the download request, and return the sharing information of the downloaded file if it is the same download address.

Further, the file system also includes:

The compression and encryption module is used to compress or/and encrypt the file information stored in the storage module. When the compression and encryption module compresses the corresponding file information, only the compressed files can reach the set compression ratio. File information is compressed.

Further, the storage module stores the corresponding file information in a multi-level tree, and generates a tree index file at the same time, and the tree index file is stored separately or together with the corresponding file information.

Further, the file system also includes:

The compression and encryption module is used to compress or/and encrypt the file information stored in the storage module. When the compression and encryption module compresses the corresponding file information, the generated compressed package includes:

a main document, said main document comprising a header, a file data area and a tree information area; or

a main document and a tree index file, the main document includes a header and a file data area, and the tree index file includes a header and a tree index file;

The header includes at least the length and version information of the file header, and the location information of the corresponding tree information area is stored in the header, the file data area stores the corresponding file information, and the tree information area stores the corresponding tree index file.

Further, the unified download platform module includes:

The multi-process processing unit is used for receiving the processes sent by the game client, sorting the processes, and processing the processes in sequence;

The information identification unit is used to read the configuration file under the process directory when the multi-process processing unit processes a certain thread, obtain the quantity, path and download address of the corresponding file information according to the configuration information, and return to the multi-process processing unit.

Further, the unified download platform module also includes an online download identification unit, which is used to identify whether the process received by the multi-process processing unit is an online download process, and when it is identified as an online download process, set a high priority weight for the corresponding process.

Further, the unified download platform module also includes:

The process management unit is used to uniformly manage the shared information and download traffic in the download when multiple processes are downloaded at the same time. The shared information includes the information of multiple game clients downloading the same file and the same client repeatedly downloading the same file.

Further, the file system also includes an online publishing module, and the online publishing module includes:

The update package generation unit is used to generate a complete data package that needs to be updated by recreating or updating;

The separation tree unit is used to perform separation tree index file operation on the complete data packet to obtain the corresponding tree index file;

The uploading unit is used for uploading the tree index file together with the data package to the download server after renaming it.

The beneficial effects of the present invention are:

The present invention adopts a file system based on online game application, which is implemented on the bottom layer of the game, allowing multiple processes to read and download resources for the same package at the same time, allowing the online background to synchronize resources on the server, and enabling the game to load resources more quickly. Faster, making resource files more secure, increasing the difficulty of illegally obtaining resources, and making the impact of game file version upgrades on players smaller and more transparent.

Description of drawings

The present invention will be further described below in conjunction with accompanying drawing and example.

Fig. 1 is a schematic structural diagram of a file system in a preferred embodiment of the present invention;

Fig. 2 is a schematic diagram of the structural composition of the unified download platform module of the present invention;

Fig. 3 is the processing schematic diagram of unified download platform module and process of the present invention;

Fig. 4 is a schematic diagram of the specific internal structure and workflow of the file system of the present invention;

Fig. 5 is a schematic diagram of thread relationship and processing flow of the file system of the present invention.

Detailed ways

The following embodiments of the present invention only represent specific explanations of the present invention, and the present invention is not limited to the application of certain systems or games. The game mentioned in the present invention is not aimed at a specific game, and the application of the present invention is a specific description of the technical solution in the present invention.

The file system of the present invention provides the FPC library and its online version FPCO (File Pack with Compression Online) library and DownloadClient library (Client of DownloadService unified download service client) library to upper-level programmers, and the DownloadServer program is attached (Server of DownloadService unified download service platform) and file package management tools, as well as a console sample program and a visual demonstration program based on the .Net platform C++/CLI language. For the specific implementation of the file system in the present invention, refer to the specific description in the following embodiments.

Example 1:

As a preferred embodiment of the present invention, with reference to shown in Figure 1, the file system of the present invention at least includes a storage module 100, a search module 101, a unified download platform module 102, and a download sharing module 103, wherein:

The storage module 100 is used for storing file information of the file system;

The search module 101 is used to retrieve corresponding file information from the storage module 100;

The unified download platform module 102 is used to receive the download request of the corresponding file sent by the game client, and control the search module 101 to retrieve the corresponding file information according to the download request;

The download sharing module 103 is used to analyze whether the download address is the same download address when the unified download platform module 102 receives the download request, and return the sharing information of the downloaded file if it is the same download address.

The above system structure constitutes the most basic component unit of the present invention. The download requests of all processes sent by the game client are processed by the unified download platform module 102, and the download flow of the processes and the allocation of resources can be reasonably and orderly Control to avoid blockage and other abnormal conditions.

In the preferred embodiment of the present invention, a compression and encryption module 104 is added to the file system, which is used to compress or/and encrypt the file information stored in the storage module 100. When the compression and encryption module 104 compresses the corresponding file information , only compress the file information of the compressed file that can reach the set compression ratio. Here, the set compression ratio can be set according to the actual application, and the present invention is not limited to a single fixed value, and its value can also be dynamically set according to different compressed files. The compression and encryption scheme adopted in the present invention can be the conventional scheme adopted at present, and can also be a high-level compression and encryption method. In the file system of the present invention, the mode adopted for compression and encryption has no substantial impact on the realization of the present invention. influences.

Generally, binary files with good scalability have a file header (packet header), which is used to indicate the length of the file header and the file version, etc. Using the package header can enhance fault tolerance and version compatibility. In order to be compatible with the different requirements of FPC and FPCO for file tree structure information (referred to as tree information), the file tree structure information can be stored in the main file together with the file data, or can be separated and saved as a file (called a tree index file) . In the header you can indicate which file the tree information is stored in. Since the length of the main document may grow dynamically, the tree index file must be used when using FPCO.

For this reason, in a preferred embodiment of the present invention, the storage module 100 stores the corresponding file information in the form of a multi-level tree, and generates a tree index file at the same time, and the tree index file is stored separately or together with the corresponding file information . Regardless of whether the tree index file is stored separately or with the corresponding file information, when the compression and encryption module compresses the corresponding file information, the generated compressed package includes two types:

The compressed package only includes the main file, and the main file includes a header, a file data area and a tree information area; or

The compressed package includes a main document and a tree index file, the main document includes a header and a file data area, and the tree index file includes a header and a tree index file;

The header includes at least the length and version information of the file header, and the location information of the corresponding tree information area is stored in the header, the file data area stores the corresponding file information, and the tree information area stores the corresponding tree index file.

It can be seen from the above description of the storage module 100 that the present invention adopts the structure in the Windows file system, that is, a multi-level file tree system. Correspondingly, the paths in the storage module 100 support forward and backward slashes and are not case-sensitive. Correspondingly, the attributes of the file information include file name, file size, modification time, compression information, parent directory index, storage information, etc., and the information of each directory includes path, file/directory number, parent directory index, and can also be Including the Hash of the directory path, etc. Correspondingly, the search module 101 supports operations such as traversing subdirectories and subfiles under the specified directory, judging whether the file directory of the specified path exists, and can also obtain information such as the name, size, and modification date of the specified file.

In a preferred embodiment of the present invention, as shown in FIG. 2, the unified download platform module 102 includes:

The multi-process processing unit 200 is used to sort the processes sent by the receiving game client, and process the processes in sequence;

The information identification unit 201 is used to read the configuration file under the process directory when the multi-process processing unit 200 processes a certain thread, obtain the quantity, path and download address of the corresponding file information according to the configuration information, and return to the multi-process processing unit ;

The online download unit 202 is used to identify whether the process received by the multi-process processing unit is an online download process, and when it is identified as an online download process, set a high priority weight for the corresponding process;

The process management unit 203 is used to uniformly manage the shared information and download traffic during downloading when multiple processes are downloaded at the same time. The shared information includes information about multiple game clients downloading the same file and the same client repeatedly downloading the same file.

The multi-process processing unit 200, the information identification unit 201, the online download unit 202 and the process management unit 203 are all specific implementation processes of the unified download platform module 102 of the present invention, and the present invention realizes the file system through the unified download platform module 102 unified planning and management. One is the planning and management of multi-threads. The received multi-threads are uniformly processed by the multi-process processing unit 200. When sorting, they can be sorted according to the time priority method, or the first-in-first-out stack method, or other methods, strictly The arrangement of the restrictions has no significant meaning to the realization of the present invention, therefore, adopting other ways to realize the management of the process is also a specific way or method that can realize the present invention. The second is planning and management when processing a certain thread. When the present invention processes a certain thread, the information identification unit 201 can read the configuration file under the process directory to obtain the address and quantity of the files that the process needs to download. Wait for the information, after that, just search from the storage module 100 through the search module 101. The third is the planning and management of the online download process. Since the online download process is directly related to the user's experience of the game, its application must be processed with priority. The present invention uses the most conventional method to identify the online process. Give a higher priority, when it needs to be pointed out, the present invention only specifically elaborates a way to realize fast download of online process, for other processing methods, such as exclusive processing of online download process, etc. within the achievable range. The fourth is the planning and management of simultaneous downloading of multiple processes. For simultaneous downloading of multiple processes, the present invention must take into account the download speed of different processes and reasonably control the download flow. For example, the corresponding download software (Xunlei, BT, etc.) can Set the corresponding download speed, so as to avoid affecting other normal Internet access of users, the process management unit 203 can manage multiple processes at the same time according to the specific requirements of users. Sometimes the content of files downloaded by multiple processes is the same, such as among Internet cafes, the same user It is possible to download repetitive files (large-scale games are mostly group games, which require multi-person groups, and a single user in the game often triggers the same process to download the same file, or triggers different processes to download the same file), the invention can identify the download Repeated downloading is avoided for processes with the same content. Similarly, different processes may download the same file at the same time, and the process management unit 203 may also allow multiple processes to download the same file at the same time.

Referring to FIG. 3 , multiple games or multiple processes of one game are connected to a unified download platform module to download resources. Multiple processes of the same game can share the downloaded content, that is, the resource files requested by process A, and process B does not need to download repeatedly. When the present invention is implemented, multiple games can use the same download platform process to achieve overall load control. Due to the different directories of different games, the package files are independent of each other, and the URLs can be configured separately, and there will be no conflict even if the names are the same. In terms of expansibility, after expansion, the file system of the present invention can completely realize download sharing at the LAN level, effectively improve the experience of Internet cafe users, and can also use P2P transmission to increase the download speed. The above specific implementation is based on the description of the specific content of FIG. 2 above.

The update of the online game is usually an online update. In order to further enhance the user experience, the file system of the present invention also includes an online publishing module, and the online publishing module includes:

The update package generation unit is used to generate the complete data package xxx.fpc that needs to be updated by recreating or updating;

The separation tree unit is used to perform separation tree index file operation on the complete data package xxx.fpc to obtain the corresponding tree index file xxx.fpci;

The upload unit is used to upload the tree index file together with the data package to the HTTP download server after the tree index file is renamed (after the name is changed to xxx.server.fpci).

When the data package is generated by updating, the files with the same size and modification time are regarded as unchanged files and do not need to be updated.

If the version update is not a critical update, there is no need to kick all players on the server from the client. At this time, the existing server files cannot be replaced directly, otherwise there will be a version mismatch. The invention connects clients of different versions to different directories, and deletes resource files of old versions when appropriate. When the version is updated, the configuration file will be updated at the same time, and the client that has not been restarted will continue to connect to the old data source.

When the game client uses online resources, download the latest tree index file x.server.fpci when the game is started for the first time or when the version is updated. After that, create a package or update the tree structure to get x.fpc and x.fpci, and then , the FPCO library loads the package document, and gradually downloads data to x.fpc and modifies xxx.fpci during use.

In order to further illustrate the invention and fully realize the specific application of the present invention, after the files in the storage module 100 are compressed and compressed by the encryption module 104, the space size of the whole package should be less than 1G. 500M is appropriate; the size of a single file in the package should not exceed 16 digits (binary), that is, 65535. When a single file is larger than 16 digits, it will be unpacked. (The number of file resources of common small and medium-scale MMORPG games is less than 3000, and the number of file resources of large-scale MMORPGs is less than 30000.)

Referring to Figure 4, the FPCO layer shown in the figure is a derived class of FPC, which can provide more interfaces. The game process and the unified download platform process will read the same configuration file under the game process directory, and determine the number and path of package files and the corresponding download address according to the configuration. Multiple processes of the same game will share resources; the package files of different games are independent of each other and will not conflict.

The underlying FPC library uses memory-mapped file sharing for the same package file, and the unified download platform process writes the downloaded data into the package file, and the game process can directly read the latest content without IO or inter-process notification.

The game logic uses the "Download Client Library" to connect to the download platform and obtain the resource reading interface.

The download client library handles the connection to the download platform, manages multiple package documents and provides an interface for obtaining package objects.

The FPC library is responsible for reading the package document data.

Download the server-side library to handle package document initialization (when the package document does not exist), game process connection and disconnection, request object creation and release, resource data update, and notify the game process to trigger a request completion event.

The download server library calls the FPCO layer, and the FPCO layer calls the cURL download library through callback.

After the resource arrives, the FPC library is responsible for writing the package document data.

Below, the present invention further explains the file system of the present invention from the perspective of threads, referring to the relationship and processing flow of each thread shown in Figure 5:

1) First, for a resource requirement thread (a thread that needs to read file data), after judging that the resource needs to be downloaded, apply for a request object (which can be explicitly created or wrapped in a blocking read call);

2) The request processing thread creates an empty request object and returns it to the resource demand thread;

3) After the resource demand thread sets the request information (a request can contain multiple file requests), it sends it to the request processing thread in the unified download platform process for processing, and obtains an event object Mutex, which can be detected at any time or wait for the Mutex in a synchronous blocking manner;

4) After the request processing thread checks the request information (and thinks it needs to be downloaded), it applies to the document space manager for the storage space of the resource file, and uses the download thread pool to download the data.

5) After the download thread finishes downloading, notify the request processing thread that a resource file has been downloaded, and if the game process has a registration window file request completion notification message, then send the message;

6) The request processing thread enumerates all requests related to this resource file, and updates their progress. If all resource files in the request have been downloaded, the event object Mutex of the request is set; if the game process has a registration window request completion notification message, then send the message;

7) After the resource demand thread finds that the event is processed, it can release the request object and use the file resource;

8) After the request processing thread finds that the request object is released, it destroys the request object.

Example 2:

The following code will demonstrate how to start the download service, find MapFile/1.wbf.ter and request a download.

#include "DownloadClient.h"

#include "FilePackOnline.h"

using namespace FilePack;

using namespaceDownloadService;

using namespaceDownloadService::DownloadClient;

#include <stdio.h>

#include <conio.h>

//ErrExit prompts an error, wait for the key to exit the program

#defineErrExit(text) do{ printf(text); returngetch(); }while(0)

intmain(intargc, wchar_t* argv[]){

//①Connection service

CDownloadClientclient(123/*game number*/); //game number

if(!client.Connect()) ErrExit("Failed to connect to the service\n");

//② Load package document

std::vector<CFilePackOnline*>packols = client.LoadPacks(654321/*package document password*/);

if(packols.empty() || !packols[0] || !packols[0]->IsOpened())ErrExit("package document loading error\n");

//③Find the MapFile/1.wbf.ter file object and determine whether it needs to be downloaded

SFile* file = packols[0]->GetFile(L"MapFile/1.wbf.ter"/*file address*/); //get file object

if(!file) ErrExit("Resource file not found\n");

if(CFilePack::IsFileStored(file)) ErrExit("The file has been downloaded\n");

//④Create and send a request based on a single file object

Query* query = client.CreateAndSendFileQuery(file, 100/*request priority*/, 0/*file priority*/);

//⑤Wait for the download request to complete

while(true){

//Every time 0.5s is still processing the file download, output a line of log:

boolprocessing = client.IsProcessingQuery(query, 0/*wait 0ms*/);//Test whether it is processing

while(processing){

printf("The download is not completed, continue to wait for up to 0.5s\n");

processing = client.IsProcessingQuery(query,500);//Whether it is being processed, wait up to 0.5s

}

//⑥The download request is completed, judge success or failure, and further process

if(query->Files[0].Downloaded){//You can also directly judge whether it is readable: CFilePack::IsFileStored(file)

printf("File downloaded successfully\n");

client.ReleaseQuery(query); //Release the request object after use

break; //Regardless of active release, disconnection or process end will be released

}

elseif(query->Files[0].Failed){

printf("File download failed, try again\n");

client.SendQuery(query); //Send the request again and re-download

}

}

//⑦Disconnect and close the package, and all unreleased requests will be released

client.CloseConnection(); //Actually, as long as the process exits, the system will properly release resources without intentionally disconnecting

return getch();

}

Therefore, the file system according to the present invention can provide the following functions for the host software (that is, the software calling the file system):

1. Provide a way for the host software to read resource files (read-only but not modified when the host software is running);

2. Let the host software run and start as soon as possible before the non-urgently needed resource files are in place (downloaded to the local hard disk), and continuously download the necessary urgently needed resource files during the running process (that is, postpone the non-urgently needed resource files) Data downloading, in the field of online games, a 500MB game that took 20 minutes to download can now be started in 30 seconds);

3. For the host software and its corresponding version files that need to be upgraded, provide support for file-level accuracy (any resource file needs to be updated, you only need to download it and do not need to be forced to download other files). Methods of operation and maintenance management;

4. When there are multiple host software processes running at the same time, when downloading resource files, the downloaded data will not be lost, and the network bandwidth will not be wasted by repeated downloads;

5. Ensure that the resource files that have been successfully completed are still available even if the download error or program error crashes or power failure occurs during the download process;

6. Provide transparent compression and encryption functions to host software writers, improve the security of resource files and download speed to a certain extent, and save hard disk storage space to a certain extent (especially for text files such as xml, the compression effect is remarkable);

7. For in-place (downloaded to local hard disk) files, provide possible faster reading speed than normal (non-optimized) reading of Windows files.

The above descriptions are only preferred embodiments of the present invention, and the present invention is not limited to the above-mentioned embodiments, as long as they achieve the technical effects of the present invention by the same means, they should all belong to the protection scope of the present invention.

Claims (7)

1. A file system, characterized in that, the file system includes a storage module, a search module, a unified download platform module, a download sharing module, and an online release module, wherein: The storage module is used to store file information and data of the file system; The search module is used to retrieve the corresponding file information and storage location from the storage module for reading; The unified download platform module is used to receive the download request of the corresponding file sent by the game client, and download the corresponding file according to the download request; The download sharing module is used to analyze whether the download address is the same download address when the unified download platform module receives the download request, and returns the shared information of the downloaded file when it is the same download address; The online publishing module includes: an update package generation unit, which is used to generate a complete data package that needs to be updated by recreating or updating; a separation tree unit, which is used to perform a separation tree index file operation on the complete data package, and obtain the corresponding tree Index file; uploading unit, for renaming the tree index file and uploading it to the download server together with the data package; The system detects whether the data package is a critical update, if it is not a critical update, then keeps all players online, and the system connects different versions of the client to different directories so that the client that has not been restarted continues to connect to the old data source. 2. The file system according to claim 1, wherein the file system further comprises: The compression and encryption module is used to compress or/and encrypt the file information stored in the storage module. When the compression and encryption module compresses the corresponding file information, only the compressed files can reach the set compression ratio. File information is compressed. 3. The file system according to claim 1, wherein the storage module stores the corresponding file information in the form of a multi-level tree, and generates a tree index file at the same time, and the tree index file is stored separately or with the corresponding file information is stored together. 4. The file system according to claim 3, wherein the file system further comprises: The compression and encryption module is used to compress or/and encrypt the file information stored in the storage module. When the compression and encryption module compresses the corresponding file information, the generated compressed package includes: a main document, said main document comprising a header, a file data area and a tree information area; or a main document and a tree index file, the main document includes a header and a file data area, and the tree index file includes a header and a tree index file; The header includes at least the length and version information of the file header, and the location information of the corresponding tree information area is stored in the header, the file data area stores the corresponding file information, and the tree information area stores the corresponding tree index file. 5. The file system according to claim 1, wherein the unified download platform module comprises: The multi-process processing unit is used for receiving the processes sent by the game client, sorting the processes, and processing the processes in sequence; The information identification unit is used to read the configuration file under the process directory when the multi-process processing unit processes a certain thread, obtain the quantity, path and download address of the corresponding file package according to the configuration information, and return to the multi-process processing unit. 6. The file system according to claim 5, wherein the unified download platform module also includes an online download identification unit for identifying whether the process received by the multi-process processing unit is an online download process, and is identified as an online download process. When downloading a process, download related information for the corresponding process management. 7. The file system according to claim 5 or 6, wherein the unified download platform module further comprises: The process management unit is used to uniformly manage the shared information and download traffic in the download when multiple processes are downloaded at the same time. The shared information includes the information of multiple game clients downloading the same file and the same client repeatedly downloading the same file.