CN114816260B - A parallel file storage system and method - Google Patents

Abstract

The invention relates to the technical field of file storage, in particular to a parallel file storage system and a parallel file storage method, wherein the parallel file storage system comprises a parallel file storage system, the output end of the parallel file storage system is electrically connected with a parallel file storage optimizing module, the output end of the parallel file storage optimizing module is electrically connected with a parallel memory module, the output end of the parallel file storage optimizing module is electrically connected with a hierarchical memory module, the output end of the parallel file storage optimizing module is electrically connected with an adding buffer module, a cross access memory comprises a low-level cross module and a high-level cross module, and continuous addresses of the cross access parallel memories in the low-level cross module are distributed in adjacent different modules, so that multi-module parallel access can be realized, the memory bandwidth is improved, and access addresses in a high-level cross program are sequentially distributed to each module.

Description

A parallel file storage system and method

The present invention relates to the technical field of file storage, and in particular to a parallel file storage system and method.

Background Art

With the continuous expansion of computer application fields, the amount of information processed is increasing, and the I/O devices of modern computers are also increasing. Therefore, improving the speed of memory access has become an urgent task, and a large number of files need to be stored in computer terminals.

The parallel file system is an optimized high-performance file system provided by the object storage service. It provides millisecond-level access latency, TB/s-level bandwidth, and millions of IOPS. It can quickly process high-performance computing (HPC) workloads. The speed between the CPU and storage is often mismatched, and data access often becomes a computing bottleneck. This problem can be solved by increasing the speed on the one hand, and optimizing the storage structure on the other.

Most of the current parallel file storage systems and methods use dual-port memories to achieve the purpose of parallel file storage, but the storage access cycle is long, resulting in slow parallel file storage and frequent read errors during the reading process.

To this end, a parallel file storage system and method are proposed.

Summary of the invention

The object of the present invention is to provide a parallel file storage system and method to solve the problem mentioned in the above background technology that most of the current parallel file storage systems and methods use dual-port memories to achieve the purpose of parallel file storage, but the storage access cycle is long, resulting in slow parallel file storage and frequent reading errors during the reading process.

To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a parallel file storage system and method, comprising a parallel file storage module, the output end of the parallel file storage system is electrically connected to a parallel file storage optimization module, one output end of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other output end of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other output end of the parallel file storage optimization module is electrically connected to an additional buffer module, one output end of the parallel memory module is electrically connected to a memory word length increase module, and the other output end of the parallel memory module is electrically connected to a storage body number increase module, and the output ends of the memory word length increase module and the storage body number increase module are electrically connected to a parallel access memory.

Preferably, the parallel memory modules are divided into space parallel type and time parallel type.

Preferably, a dual-port memory is provided at one end of the parallel memory module, and a multi-memory module parallel memory is provided at the other end of the parallel memory module.

Preferably, the multi-storage module parallel storage is composed of multiple storage bodies.

Preferably, the other output end of the added buffer module is electrically connected to various buffers.

Preferably, the various buffers are set as general registers.

Preferably, the storage body number increasing module includes a cross-access memory and a conflict-free access memory.

Preferably, the cross-access memory includes a low-order cross-access module and a high-order cross-access module.

Compared with the prior art, the present invention has the following beneficial effects:

1. A parallel file storage module, wherein the output end of the parallel file storage system is electrically connected to a parallel file storage optimization module, one output end of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other output end of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other output end of the parallel file storage optimization module is electrically connected to an additional buffer module, one output end of the parallel memory module is electrically connected to a memory word length increase module, and the other output end of the parallel memory module is electrically connected to a memory body number increase module, the output ends of the memory word length increase module and the memory body number increase module are electrically connected to a parallel access memory, and the parallel memory module is divided into a spatial parallel type and a temporal parallel type. Parallel type, one end of the parallel memory module is provided with a dual-port memory, and the other end of the parallel memory module is provided with a multi-memory module parallel memory, the multi-memory module parallel memory is composed of a plurality of memory bodies, the other end of the output of the added buffer module is electrically connected to various buffers, and the various buffers are set as general registers. The memory body number increase module includes a cross-access memory and a conflict-free access memory, and the cross-access memory includes a low-order cross-module and a high-order cross-module. The parallel memory module, the hierarchical storage module and the added buffer module are used as the main method for optimizing the storage in the parallel file storage optimization module, and the parallel memory module is used to increase the storage access of the parallel file in advance. In addition, by using layered storage modules and adding buffer modules at the branch, the online data in the layered storage module is further layered, and the data information storage is guaranteed under a more scientific concept. It can be used as an independent high-performance storage system for enterprise application environments with frequent I/O access, and the added buffer module connected in series can be used to transmit and temporarily store data, participate in arithmetic and logical operations, and save the results of operations, greatly increasing the computing power of the server and avoiding storage congestion. In addition, in conjunction with the memory word length increase module and the storage body number increase module in the parallel memory module, the storage access volume is quickly increased, the parallel access memory is improved, the storage efficiency of parallel files is increased, and in conjunction with the storage The number of bodies increases the cross-access memory and conflict-free access memory in the module. The odd and even address parts of the storage body are separated by the cross-access memory to compensate for the relatively slow read and write speeds of memories such as DRAM. The cross-storage disperses continuous information into each block, thereby improving the read and write storage efficiency. The cross-access memory includes a low-order cross-module and a high-order cross-module. The cross-access parallel memory continuous addresses in the low-order cross-module are distributed in different adjacent modules, which can realize multi-module parallel access and improve the memory bandwidth. The access addresses in the high-order cross program are sequentially allocated to each module. When a module is working, other modules are not working, and parallel file storage is regulated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a diagram of a central control system of the present invention;

FIG2 is a system diagram of a parallel memory module of the present invention;

FIG. 3 is a system diagram of a module for increasing the number of storage bodies according to the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

Please refer to Figures 1 to 3, the present invention provides a parallel file storage system and method technical solution:

A parallel file storage module, the output end of the parallel file storage system is electrically connected to a parallel file storage optimization module, one end of the output of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other end of the output of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other end of the output of the parallel file storage optimization module is electrically connected to an added buffer module, one end of the output of the parallel memory module is electrically connected to a memory word length increase module, and the other end of the output of the parallel memory module is electrically connected to a memory body number increase module, the output ends of the memory word length increase module and the memory body number increase module are electrically connected to a parallel access memory, and the parallel memory module is divided into a spatial parallel type and a temporal parallel type. Type, one end of the parallel memory module is provided with a dual-port memory, and the other end of the parallel memory module is provided with a multi-memory module parallel memory, the multi-memory module parallel memory is composed of a plurality of memory bodies, the other end of the output of the added buffer module is electrically connected to various buffers, and the various buffers are set as general registers. The memory body number increase module includes a cross-access memory and a conflict-free access memory, and the cross-access memory includes a low-order cross-module and a high-order cross-module. The parallel memory module, the hierarchical storage module and the added buffer module are used as the main method for optimizing the storage in the parallel file storage optimization module, and the parallel memory module is used to increase the storage access of the parallel file in advance. In addition By adopting hierarchical storage modules and adding buffer modules at the branch, the online data in the hierarchical storage module is further hierarchical. Under a more scientific concept, the data information storage is guaranteed. It can be used as an independent high-performance storage system for enterprise application environments with frequent I/O access. The added buffer module connected in series can be used to transmit and temporarily store data, participate in arithmetic and logical operations, and save the operation results, which greatly increases the computing power of the server and avoids storage congestion. In addition, in conjunction with the memory word length increase module and the storage body number increase module in the parallel memory module, the storage access volume is quickly increased, the parallel access memory is improved, the storage efficiency of parallel files is increased, and in conjunction with the storage body number increase module, the storage word length increase module and the storage body number increase module in the parallel memory module are used to increase the storage word length increase module and the storage body number increase module in the parallel memory module. The cross-access memory and conflict-free access memory in the number-increasing module are used to separate the odd and even address parts of the storage body through the cross-access memory to compensate for the relatively slow read and write speeds of memories such as DRAM. The cross-storage disperses continuous information into various blocks, thereby improving the read and write storage efficiency. The cross-access memory includes a low-order cross-module and a high-order cross-module. Through the cross-access parallel memory in the low-order cross-module, the continuous addresses are distributed in different adjacent modules, which can realize multi-module parallel access and improve the memory bandwidth. The access addresses in the high-order cross program are sequentially assigned to each module. When a module is working, other modules are not working, and parallel file storage is regulated.

Usage: When users perform parallel file storage, they will first transfer and control the file to the parallel file storage optimization module, and use the parallel memory module to increase the storage access of parallel files. In addition, by using the layered storage module and adding the buffer module at the branch, the online data in the layered storage module is further layered. Under a more scientific concept, the storage of data information is guaranteed. It can be used as an independent high-performance storage system for enterprise application environments with frequent I/O access. The added buffer module connected in series can be used to transmit and temporarily store data, participate in arithmetic and logical operations, and save the results of operations, greatly increasing the computing power of the server and avoiding storage congestion. Through the cross-access parallel memory continuous addresses in the low-order cross module, the continuous addresses are distributed in different adjacent modules, which can realize multi-module parallel access and improve the memory bandwidth. The access addresses in the high-order cross program are sequentially allocated to each module. When a module is working, other modules do not work, and parallel file storage is regulated. In addition, with the memory word length increase module and the storage body number increase module in the parallel memory module, the storage access volume is quickly increased, the parallel access memory is improved, and the storage efficiency of parallel files is increased.

The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims (3)

1. A parallel file storage system, comprising a parallel file storage module, wherein the output end of the parallel file storage module is electrically connected to a parallel file storage optimization module, one output end of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other output end of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other output end of the parallel file storage optimization module is electrically connected to an additional buffer module, the parallel memory module is divided into a spatial parallel type and a temporal parallel type, one output end of the parallel memory module is electrically connected to a memory word length increasing module, and the other output end of the parallel memory module is electrically connected to a memory body number increasing module, the memory word length increasing module and the memory body number increasing module are electrically connected to each other. The output end of the module is electrically connected to a parallel access memory. The storage body number increasing module includes a cross access memory and a conflict-free access memory. The cross access memory includes a low-order cross module and a high-order cross module. The odd address and even address parts of the storage body are separated by the cross access memory. The cross storage disperses the continuous information into each block. The continuous addresses are distributed into different adjacent modules by the cross access parallel memory in the low-order cross module. The access addresses in the high-order cross program are sequentially allocated to each module. A dual-port memory is provided at one end of the parallel memory module, and a multi-storage module parallel memory is provided at the other end of the parallel memory module. The multi-storage module parallel memory consists of multiple storage bodies. 2. A parallel file storage system according to claim 1, characterized in that: the other end of the output of the added buffer module is electrically connected to various buffers. 3. A parallel file storage system according to claim 2, characterized in that: the various buffers are set as general registers.