CN103810114A - Method and device for distributing storage space - Google Patents

Abstract

The present invention relates to a method and device for allocating storage space, comprising: receiving a request message for querying file storage information sent by an application program, wherein the request message for querying file storage information includes file information; searching for a file corresponding to the file information, And search for the identification number corresponding to the file; analyze the identification number, and obtain the storage information of the storage block occupied by the file on the memory according to the result of analyzing the identification number; send the storage information to the application program information, so that the application program classifies the storage blocks according to the storage information, and enables the application program to allocate the classified storage blocks to the read and write requests according to the continuity of the read and write requests of the application program . This enables applications to understand the continuity of storage space, which can effectively improve the speed of reading and writing disk data by applications.

Description

Method and device for allocating storage space

technical field

The invention relates to the field of mobile communication, in particular to a method and device for allocating storage space.

Background technique

The main bottleneck of disk database operation is the read and write speed of database data. The reason why it is difficult to improve the read and write speed of database data is that the database does not know the continuity of files, and functions such as pre-reading, write cache, and continuous writing are affected.

In the prior art, when the database needs to perform read and write processing, it first sends a read and write request message to the kernel, and after the kernel receives the read and write request message, it starts the disk to perform read and write processing. The cache returns the data to be read; when the database writes data, the data to be written is directly written to the disk without knowing the continuity of the storage space, which easily leads to the inconsistency of the storage location of the relevant data on the disk Continuous, which seriously affects the read and write speed of the disk database.

Contents of the invention

The embodiment of the present invention provides a method and device for allocating storage space, which solves the problem of low efficiency of reading and writing disk data by application programs.

In the first aspect, a method for allocating storage space is provided, and the method includes:

receiving a request message for querying file storage information sent by an application program, wherein the request message for querying file storage information includes file information;

Find the file corresponding to the file information, and find the identification number corresponding to the file;

Analyzing the identification number, and obtaining the storage information of the storage block occupied by the file on the memory according to the result of analyzing the identification number;

sending the storage information to the application, so that the application classifies the storage blocks according to the storage information, and makes the application classify the classified blocks according to the continuity of the read and write requests of the application A memory block is allocated to the read and write requests.

According to the first aspect, in a first possible implementation manner of the first aspect, the file information is specifically a file name, and the searching for the file corresponding to the file information and searching for the identification number corresponding to the file includes :

Check whether the file name is legal, if the file name is legal, then find whether the file corresponding to the file name exists, if the file exists, then search for the identification number corresponding to the file according to the file name.

According to the first aspect, in a second possible implementation manner of the first aspect, the file information is specifically a file handle, and the searching for the file corresponding to the file information and searching for the identification number corresponding to the file includes :

Find the file corresponding to the file handle from the process management module of the operating system according to the file handle, and find the identification number corresponding to the file.

According to the first aspect or any one of the above two implementation manners of the first aspect, in the third implementation manner of the first aspect, the file system parses the identification number, and according to the As a result, obtaining the storage information of the storage block occupied by the file on the storage includes:

Analyzing the identification number, and according to the result of analyzing the identification number, obtaining information on all storage blocks occupied by the file on the storage, where the information on all storage blocks includes information on multiple groups of storage blocks, wherein each The information of the group of storage blocks includes: the starting address information of the group of storage blocks and the number of storage blocks contained in the group of storage blocks.

According to the third possible implementation manner of the first aspect, in the fourth possible implementation manner of the first aspect, the sending the storage information to the application program, so that the application program The information classifies the storage blocks, and the application program allocates the classified storage blocks to the read and write requests according to the continuity of the application program's read and write requests, including:

sending the storage information to the application program, so that the application program determines the total size of each group of storage blocks according to the number of storage blocks contained in each group of storage blocks and the size of the preset storage block, And according to the total size of each group of storage blocks, classify the groups of storage blocks, and allocate the classified storage blocks to the read and write requests according to the continuity of the read and write requests of the application.

In a second aspect, a device for allocating storage space is provided, the device comprising: a receiving unit, a search unit, a processing unit, and a sending unit;

The receiving unit is configured to receive a query file storage information request message sent by an application program, and the query file storage information request message includes file information;

The searching unit is configured to search for the file corresponding to the file information received by the receiving unit, and search for the identification number corresponding to the file;

The processing unit is configured to analyze the identification number searched by the search unit, and obtain the storage information of the storage block occupied by the file on the memory according to the result of analyzing the identification number;

The sending unit is configured to send the storage information obtained by the processing unit to the application program, so that the application program can classify the storage blocks according to the storage information, and enable the application program to classify the storage blocks according to the The continuity of the read and write requests of the application program allocates the classified storage blocks to the read and write requests.

According to the second aspect, in a first possible implementation manner of the second aspect, the search unit is specifically configured to:

Check whether the file name is legal, if the file name is legal, then find whether the file corresponding to the file name exists, if the file exists, then search for the identification number corresponding to the file according to the file name.

According to the second aspect, in a second possible implementation manner of the second aspect, the search unit is specifically configured to:

Find the file corresponding to the file handle from the process management module of the operating system according to the file handle, and find the identification number corresponding to the file.

According to the second aspect or any one of the above two implementation manners of the second aspect, in the third implementation manner of the second aspect, the processing unit is specifically configured to:

Analyzing the identification number, and according to the result of analyzing the identification number, obtaining information on all storage blocks occupied by the file on the storage, where the information on all storage blocks includes information on multiple groups of storage blocks, wherein each The information of the group of storage blocks includes: the starting address information of the group of storage blocks and the number of storage blocks contained in the group of storage blocks.

According to a third possible implementation manner of the second aspect, in a fourth possible implementation manner of the second aspect, the sending unit is specifically configured to:

sending the storage information to the application program, so that the application program determines the total size of each group of storage blocks according to the number of storage blocks contained in each group of storage blocks and the size of the preset storage block, And according to the total size of each group of storage blocks, classify the groups of storage blocks, and allocate the classified storage blocks to the read and write requests according to the continuity of the read and write requests of the application.

The method and device for allocating storage space provided by the embodiments of the present invention receive a request message for querying file storage information sent by an application program, the request message for querying file storage information includes file information; search for a file corresponding to the file information, and Find the identification number corresponding to the file; analyze the identification number, and obtain the storage information of the storage block occupied by the file on the memory according to the result of analyzing the identification number; send the storage information to the application program so that the application program classifies the storage blocks according to the storage information, and the application program allocates the classified storage blocks to the read and write requests according to the continuity of the read and write requests of the application program. This enables the application to understand the continuity of the storage space, and uses the storage space according to the characteristics of the read and write requests, thereby effectively improving the read and write speed of the application to the disk data.

Description of drawings

Fig. 1 is the Posix file system schematic diagram that the present invention provides;

FIG. 2 is a flowchart of a method for allocating storage space provided by Embodiment 1 of the present invention;

FIG. 3 is a schematic diagram of obtaining storage space information by an application program provided by Embodiment 1 of the present invention;

FIG. 4 is a flowchart of a method for allocating storage space provided by Embodiment 2 of the present invention;

FIG. 5 is a schematic diagram of an apparatus for allocating storage space provided by Embodiment 3 of the present invention.

Detailed ways

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

In the embodiment provided by the present invention, the Posix file system is taken as an example for illustration. Refer to the schematic diagram of the Posix file system provided by the present invention shown in FIG. The module is a part of the built-in function of the file system, a part of the file system interface, which can directly obtain the physical location of the file; add the system call interface of the file physical location at the operating system layer, for example, two kinds of system call interfaces can be defined: 1) intGetFileBlock_name (const char*pathname, char*buf, int size); 2) int GetFileBlock_fd (const int fd, char*buf, int size); among them, the parameter pathname in the first interface form indicates that the operating system sends to the file system The path of the file, the path includes the file name of the file, the parameter buf points to the first address of each storage block group returned by the file system to the application program, and the parameter size indicates the number of storage blocks in each group of storage blocks returned, the above The storage block can be a disk block; the data block traversal module, according to the physical identification number corresponding to the file on the storage block, uses the disk management rules of the file system to analyze the above physical identification number, and returns all the storage blocks occupied by the above file on the storage ; The file handle management module, according to whether the file handle fd contains the file handle in the process management module, finally finds the corresponding physical identification number of fd on the storage block.

FIG. 2 is a flowchart of a method for allocating storage space provided by Embodiment 1 of the present invention. The subject of execution of the method may be a file system. As shown in Figure 2, the method specifically includes:

S210. Receive a request message for querying file storage information sent by an application program, where the request message for querying file storage information includes a file name.

Referring to the schematic diagram of the application program obtaining the storage space information provided by the first embodiment of the present invention shown in Fig. 3, in Fig. 3, first add a system call interface in the system call layer: int GetFileBlock_name (const char * pathname, char * buf, int size) or int GetFileBlock_fd (const int fd, char*buf, int size), the application or specifically the database sends a request message for querying file storage information to the file system by calling the interface function of the above operating system (through the parameter pathname), the The request message for querying file storage information includes file path information, and the file path information includes a file name, wherein the file corresponding to the above file name may be a data file or a log file.

S220. Check that the file name is legal, and if the file name is legal, search whether the file corresponding to the file name exists, and if the file exists, search for the identification number corresponding to the file according to the file name.

Referring to Figure 3, after the namespace module of the file system receives the path information of the file sent by the above-mentioned application or database, it obtains the file name in the path information, for example, the file name is a character string; then the namespace module checks Whether the file name is legal; when the above file name is legal, the namespace module searches locally according to the above file name to see if the file corresponding to the file name is included. The name is sent to the metadata index module of the file system, and the metadata index module searches for the identification number corresponding to the file corresponding to the file name according to the file name.

S230. Analyze the identification number, and acquire storage information of a storage block occupied by the file on the storage according to a result of analyzing the identification number.

Specifically, the metadata indexing module of the file system parses the identification number, and according to the result of parsing the identification number, obtains information of all storage blocks occupied by the file on the storage, and the information of all storage blocks The information includes information of multiple groups of storage blocks, wherein the information of each group of storage blocks includes: start address information of the group of storage blocks and the number of storage blocks included in the group of storage blocks.

For example, the above-mentioned memory is a disk, and the storage block is a disk block divided by the disk, then the identification number obtained above is the inode number of the file on the disk; the above-mentioned file system is an ext3 file system, according to the ext3 file system Management rules, the inode number can be mapped to the array i_block[EXT3_N_BLOCKS], the array includes the above inode number, the first index value and the second index value, wherein the inode number is an integer number, which uniquely identifies the The disk block address of some data; the first index value is a first-level pointer, which points to the start address of a group of continuous disk blocks storing part of the data in the above file on the disk; the second index value is a second-level pointer, This pointer can point to a plurality of different first-level pointers, and each first-level pointer points to the starting address of a group of continuous disk blocks storing part of the data in the above-mentioned file on the disk; specifically, according to the above-mentioned first index value and With the second index value, the information of multiple groups of disk blocks can be obtained, wherein, the information of each group of disk blocks includes: the starting address information of the group of disk blocks and the number of disk blocks contained in the group of disk blocks. The information of the group disk blocks constitutes the information of all disk blocks.

S240. Send the storage information to the application program, so that the application program classifies the storage blocks according to the storage information, and enables the application program to classify the storage blocks according to the continuity of the read and write requests of the application program After the storage block is allocated to the read and write requests.

Specifically, the application determines the total size of each group of storage blocks according to the number of storage blocks included in each group of storage blocks and the preset size of the storage blocks, and according to the total size of each group of storage blocks, Classifying the groups of storage blocks, and assigning the classified storage blocks to the read and write requests according to the continuity of the read and write requests of the application.

Referring to Figure 3, after the file system obtains the information of each group of disk blocks, it returns the information of each group of disk blocks to the application program or database, and the storage engine in the application program or database classifies the returned groups of disk blocks , specifically, the disk blocks divided by the disk logically have a fixed size, for example, it can be 4KB, according to the number of disk blocks contained in the group of disk blocks in the information of each group of disk blocks and the above-mentioned disk block preset The total size of each group of disk blocks can be obtained; the above-mentioned storage engine classifies each group of disk blocks according to the total size of each group of disk blocks. For example, there are three groups of disk blocks in total: the first group of disk blocks The information includes: the first address is 200KB, including 2 disk blocks in total, then the total size of the first group of disk blocks is 8KB; the information of the second group of disk blocks includes: the first address is 512KB, including 16 disk blocks in total, then the first The total size of the second group of disk blocks is 64KB; the information of the third group of disk blocks includes: the first address is 768KB, including 128 disk blocks in total, then the total size of the third group of disk blocks is 512KB; then the above disk blocks can be divided There are three types: the size of the first type is 8KB, the size of the second type is 64KB, and the size of the third type is 512KB. The application program or the database allocates the classified storage blocks to the read and write requests according to the continuity of the read and write requests. Specifically, since there are many random reads and writes of the data files in the database, and the data pages of the data files are relatively small, priority is given to Allocate disk blocks from small disk block groups, for example: allocate disk blocks from the first type of disk group. Therefore, when the data page in the data file may be larger than the minimum allocation unit of the file system, this file system can be pre-aligned to avoid spanning blocks; when the log files in the database are continuously read and written and the data blocks are relatively large, the disk blocks are allocated from the large disk block groups first, for example, the disk blocks are allocated from the third type of disk block groups; when the disk blocks are insufficient When , large disk blocks are allowed to be split into small disk blocks, or small disk blocks are merged into logically continuous large disk blocks for use, but at this time, users need to be prompted that the continuity of data files or log files is insufficient.

FIG. 4 is a flowchart of a method for allocating storage space provided by Embodiment 2 of the present invention. The subject of execution of the method may be a file system. As shown in Figure 4, the method specifically includes:

S410. Receive a request message for querying file storage information sent by an application program, where the request message for querying file storage information includes a file handle.

See Figure 3, first add a system call interface in the system call layer: int GetFileBlock_name (const char*pathname, char*buf, int size) or int GetFileBlock_fd (const int fd, char*buf, int size), application or specific Send a file handle to the file system for the database by calling the interface function of the above-mentioned operating system (via the parameter pathname), where the file corresponding to the file handle can be a data file or a log file.

S420. Find the file corresponding to the file handle from the process management module of the operating system according to the file handle, and find the identification number corresponding to the file.

For example, referring to Fig. 3, after the file system receives the file handle fd sent by the above-mentioned application program or database, it searches whether the file is contained in the task structure of the current task managed by the process management module of the operating system according to the file handle Handle fd, here, the file handles of all open files in the application are recorded in the task structure, where the relationship between files and file handles is one-to-many, that is, one file corresponds to multiple file handles, according to one of the file The handle can uniquely find a file; specifically, when the file system finds the file handle fd from the task structure, it can find the relevant information of the file corresponding to fd from the process management module according to the fd, for example: , the relevant information of the file can be recorded in the file structure of the process management module, and the file structure includes the parameter address_space, wherein the inode field of the parameter address_space records the identification number corresponding to the above file on the memory, therefore, According to the above file handle, the identification number corresponding to the file can be found.

S430-S440 are the same as steps S230-S240.

FIG. 5 is a schematic diagram of an apparatus for allocating storage space provided by Embodiment 3 of the present invention, and the apparatus is used to execute the method shown in FIG. 2 or FIG. 4 . As shown in FIG. 5 , the apparatus includes: a receiving unit 501 , a searching unit 502 , a processing unit 503 and a sending unit 504 .

The receiving unit 501 is configured to receive a request message for querying file storage information sent by an application program, where the request message for querying file storage information includes file information.

The searching unit 502 is configured to search for the file corresponding to the file information received by the receiving unit 501, and search for the identification number corresponding to the file.

Optionally, the search unit 502 is specifically configured to: check whether the file name is legal, if the file name is legal, then search whether the file corresponding to the file name exists, if the file exists, then according to the Find the identification number corresponding to the file by using the file name.

Optionally, the search unit 502 is specifically configured to: search for the file corresponding to the file handle from the process management module of the operating system according to the file handle, and find the identification number corresponding to the file.

The processing unit 503 is configured to analyze the identification number searched by the search unit 502, and obtain the storage information of the storage block occupied by the file on the memory according to the result of analyzing the identification number.

Optionally, the processing unit 503 is specifically configured to: analyze the identification number, and obtain information of all storage blocks occupied by the file on the storage according to a result of analyzing the identification number, and all storage blocks The information includes the information of multiple groups of storage blocks, wherein the information of each group of storage blocks includes: the starting address information of the group of storage blocks and the number of storage blocks included in the group of storage blocks.

The sending unit 504 is configured to send the storage information acquired by the processing unit 503 to the application program, so that the application program classifies the storage blocks according to the storage information, and enables the application program to classify the storage blocks according to the The continuity of the read and write requests of the application program allocates the classified storage blocks to the read and write requests.

Optionally, the sending unit 504 is specifically configured to: send the storage information to the application program, so that the application program can store the information according to the number of storage blocks included in each group of storage blocks and the number of preset storage blocks. Size, determine the total size of each group of storage blocks, and classify each group of storage blocks according to the total size of each group of storage blocks, and classify according to the continuity of the read and write requests of the application program After the storage block is allocated to the read and write requests.

The device provided in the embodiment of the present invention is implanted with the method provided in Embodiment 1 of the present invention. Therefore, the specific working process of the device provided in the present invention will not be repeated here.

In summary, the above file system can export the disk grouping information used by the file, that is, it can export the logical storage space, so that the application program or database can know the continuity of the storage space corresponding to the file, and support the application program or database according to the continuity of the storage space. Use the buddy algorithm to manage storage space, ensure the physical continuity of files, and realize the purpose of optimizing continuous reading and writing of database log files and random reading and writing of small blocks of data files.

It should be noted that, in the embodiment of the present invention, a disk file system is used as an example for illustration, but it is not limited to a disk file system. This solution can also be used for SSDs, memory file systems, or scenarios with strict continuity requirements.

Professionals should further realize that the units and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the relationship between hardware and software Interchangeability. In the above description, the composition and steps of each example have been generally described according to their functions. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Protection scope, within the spirit and principles of the present invention, any modification, equivalent replacement, improvement, etc., shall be included in the protection scope of the present invention.

Claims (10)

1. A method for allocating storage space, characterized in that the method comprises: receiving a request message for querying file storage information sent by an application program, wherein the request message for querying file storage information includes file information; Find the file corresponding to the file information, and find the identification number corresponding to the file; Analyzing the identification number, and obtaining the storage information of the storage block occupied by the file on the memory according to the result of analyzing the identification number; sending the storage information to the application, so that the application classifies the storage blocks according to the storage information, and makes the application classify the classified blocks according to the continuity of the read and write requests of the application A memory block is allocated to the read and write requests. 2. The method according to claim 1, wherein the file information is specifically a file name, and the searching for the file corresponding to the file information and the identification number corresponding to the file include: Check whether the file name is legal, if the file name is legal, then find whether the file corresponding to the file name exists, if the file exists, then search for the identification number corresponding to the file according to the file name. 3. The method according to claim 1, wherein the file information is specifically a file handle, and the searching for the file corresponding to the file information and the identification number corresponding to the file include: Find the file corresponding to the file handle from the process management module of the operating system according to the file handle, and find the identification number corresponding to the file. 4. The method according to any one of claims 1-3, wherein the analyzing the identification number, according to the result of analyzing the identification number, obtains the storage of the storage block occupied by the file on the memory. Information includes: Analyzing the identification number, and according to the result of analyzing the identification number, obtaining information on all storage blocks occupied by the file on the storage, where the information on all storage blocks includes information on multiple groups of storage blocks, wherein each The information of the group of storage blocks includes: the starting address information of the group of storage blocks and the number of storage blocks contained in the group of storage blocks. 5. The method according to claim 4, characterized in that, sending the storage information to the application program, so that the application program classifies the storage blocks according to the storage information, and makes the application program classify the storage blocks according to the The continuity of the read and write requests of the application program, assigning the classified storage blocks to the read and write requests includes: sending the storage information to the application, so that the application determines the total size of each group of storage blocks according to the number of storage blocks contained in each group of storage blocks and the size of the preset storage blocks, And according to the total size of each group of storage blocks, classify the groups of storage blocks, and allocate the classified storage blocks to the read and write requests according to the continuity of the read and write requests of the application. 6. A device for allocating storage space, characterized in that the device comprises: a receiving unit, a search unit, a processing unit and a sending unit; The receiving unit is configured to receive a query file storage information request message sent by an application program, and the query file storage information request message includes file information; The searching unit is configured to search for the file corresponding to the file information received by the receiving unit, and search for the identification number corresponding to the file; The processing unit is configured to analyze the identification number searched by the search unit, and obtain the storage information of the storage block occupied by the file on the memory according to the result of analyzing the identification number; The sending unit is configured to send the storage information acquired by the processing unit to the application program, so that the application program can classify storage blocks according to the storage information, and enable the application program to classify storage blocks according to the The continuity of the read and write requests of the application program allocates the classified storage blocks to the read and write requests. 7. The device according to claim 6, wherein the search unit is specifically configured to: Check whether the file name is legal, if the file name is legal, then find whether the file corresponding to the file name exists, if the file exists, then search for the identification number corresponding to the file according to the file name. 8. The device according to claim 6, wherein the search unit is specifically configured to: Find the file corresponding to the file handle from the process management module of the operating system according to the file handle, and find the identification number corresponding to the file. 9. The device according to any one of claims 6-8, wherein the processing unit is specifically configured to: Analyzing the identification number, and obtaining information on all storage blocks occupied by the file on the storage according to the result of analyzing the identification number, the information on all storage blocks includes information on multiple groups of storage blocks, wherein each The information of the group of storage blocks includes: the starting address information of the group of storage blocks and the number of storage blocks contained in the group of storage blocks. 10. The device according to claim 9, wherein the sending unit is specifically used for: sending the storage information to the application program, so that the application program determines the total size of each group of storage blocks according to the number of storage blocks contained in each group of storage blocks and the size of the preset storage block, And according to the total size of each group of storage blocks, classify the groups of storage blocks, and allocate the classified storage blocks to the read and write requests according to the continuity of the read and write requests of the application.

Images