CN1305265C - Asynchronous remote mirror image method based on load selfadaption in SAN system - Google Patents

Abstract

The asynchronous remote mirroring method based on load self-adaptation in the SAN system belongs to the field of network storage technology, and is characterized in that: it is composed of hosts, interconnected devices including host adapter cards, switches and data links, I/O nodes and disk arrays thereof, In the storage area network SAN composed of the mirror I/O node of the network hard disk, that is, the Mirror I/O node and its disk array, the SCSI target simulator of the I/O node can be based on the size of the I/O load , take two different asynchronous mirroring methods to reduce the command response time, and at the same time when the mirroring relationship is initially established, or after the disk fails and the disk is replaced, the synchronization between the disks is carried out to ensure the consistency of the data, and the target simulation All error conditions of each sub-module of the device can be analyzed and processed differently. It can reduce the response time of write commands on the premise of ensuring data security and consistency, and the greater the I/O load, the more obvious the effect.

Description

Asynchronous Remote Mirroring Method Based on Load Adaptation in SAN System

technical field

An asynchronous remote mirroring method based on load self-adaptation in a SAN system belongs to the technical field of network storage.

Background technique

As a key technology of SAN (Storage Area Network), remote data mirroring makes full use of the characteristics of long-distance connection and unified storage of the middle and bottom network (FC or Ethernet) of SAN, ensuring that important data is protected from regional physical disasters (such as: fire, The availability after floods and large-scale power failures, etc.) provides strong support for data disaster recovery.

At present, the remote mirroring solution of the SAN system mostly adopts the method of synchronous mirroring. Synchronous mirroring means that all write operations to mirrored disks or logical volumes will send commands/data to the mirrored disk (or logical volume) pair at the same time, and will be notified only after the write commands of the mirrored disk pair (or logical volume) are completed The write command issues program command completion. The most notable feature of this synchronous data mirroring method is that the data consistency is very good, but it also has the following shortcomings:

1. Due to the need to wait for all the local write commands and remote write commands to be completed, the command response time of the system increases. Some data show that when the system I/O load is relatively heavy, the command response time increases exponentially.

2. Synchronous mirroring requires high bandwidth, and it needs to be implemented with the support of high-speed networks such as Fiber Channel using the FCP protocol, which leads to a relatively simple implementation method of synchronous mirroring and high costs for the entire system. The products of some companies use the low-speed network as the remote mirroring channel, but need to use the bridging conversion from FCP to iSCSI, which is expensive and has poor performance.

The asynchronous remote mirroring method based on load self-adaptation solves the above shortcomings and maintains good data consistency.

Contents of the invention

The object of the present invention is to provide a method for asynchronous remote mirroring based on load self-adaptation in a SAN (Storage Area Network) system to improve data security and under the premise of maintaining good data consistency, according to the system Depending on the load situation, two different asynchronous write protocols are dynamically used for asynchronous mirroring to reduce the impact of system load on system performance. Compared with synchronous mirroring, it has certain system performance advantages.

The present invention is characterized in that the mirror I/O node that provides network hard disk for above-mentioned I/O node is Mirror In the storage area network SAN that I/O node and disk array thereof constitute, the small computer interface of I/O node is the SCSI target emulator to carry out the following steps successively according to the module structure of setting to realize the method described in the present invention:

(1) The SCSI target simulator has:

The mirror relationship initialization sub-module is provided with: user-specified mirror relationship interface ProcWriteUserCommand(); mirror pair automatic restoration interface STML_Read_Mirror_Map(), STML_Restore_Mirror_Map(); abnormal exit processing interface Handle_Abnormal_Shutdown();

Synchronization sub-module, receiving mirror relationship initialization sub-module request, equipped with: disk synchronization thread Sync_disk_thread(); disk full synchronization interface, receiving disk synchronization thread command Full_Sync_disks(); bitmap-based synchronization interface, receiving synchronization thread command Bitmap_Sync_disk() ;

The real-time storage sub-module is provided with: command processing thread STML_Handle_Cmnd_thread (); read command processing interface, receive command processing thread command STML_Handle_Read_Command (); write command processing interface, receive command processing thread command STML_Handle_Write_Command () mirror write command processing interface, STML_Handle_Mirror_Write_Command ( );

The dual-protocol self-adaptive data replication sub-module receives the request of the image writing command processing interface in the real-time storage sub-module, and is provided with: self-adaptive asynchronous protocol switching function and data copy thread STML_Protocol_change(), STML_data_mover_thread(); using asynchronous writing protocol A Write command interface, receive the command of the above-mentioned adaptive asynchronous protocol switching function and data copy thread, STML_Handle_Async_A_Command(); adopt the write command interface of asynchronous write protocol B, receive the command of the above-mentioned adaptive asynchronous protocol switching function and data copy thread, STML_Handle_Async_B_Command( );

The data change recording sub-module receives the request of the dual-protocol self-adaptive data replication sub-module, and is equipped with: bitmap recording interface STML_Bitmap_Write(), bitmap reading interface STML_Bitmap_Read();

The error handling analysis sub-module receives the requests of the above-mentioned mirror relationship initialization sub-module, synchronization sub-module, real-time storage sub-module, dual-protocol adaptive data replication sub-module and data change record sub-module respectively in real time, and is provided with: error handling thread STML_ERROR_Handle_thread( );

The system exits the sub-module, if the processing command from the error processing and analysis sub-module is not received, the system exits normally; if the processing command from the error processing and analysis sub-module is received, the system saves the error scene and records, and exits abnormally;

(2) The SCSI target emulator performs the following steps in sequence according to the read and write commands from the host:

(2.1) The SCSI target emulator of the I/O node receives read and write commands from the host.

(2.2) The SCSI target simulator judges the command type. If it is a read command, the command is sent to the local disk for execution, and when the command is executed, the host is notified to inform the execution of the command;

Otherwise, follow the steps below to execute the write command;

Determine the type of load according to the predetermined threshold value of the write command queue length of the SCSI target emulator, that is, the I/O load:

If the I/O load of the I/O node is less than the load threshold, the load is light, and the asynchronous write protocol A is executed: the I/O node sends the write command issued by the host to the local disk and the remote mirror I/O respectively. The O node performs processing respectively, and after the I/O node receives the confirmation that the execution of the local write command is completed, it returns the operation end command to the host.

If the I/O load of the I/O node is greater than the load threshold, the load is heavy, and the asynchronous write protocol B is executed: the I/O node sends the write command issued by the host to the local disk for processing, and uses the bitmap Record the data block changed by the command; after the I/O node is confirmed that the local write command is executed, it returns the operation end command to the host. The changed data block is automatically read from the local disk by the adaptive asynchronous protocol data replication thread according to the load condition of the I/O node or at regular intervals, and sent to the mirror I/O node for asynchronous mirror write operation.

(3) The asynchronous writing protocol A described in turn comprises the following steps:

1. The write command is sent from the host to the I/O node;

2. The I/O node copies the write command and sends it to the mirror I/O node;

3. After the I/O node processes the local write command, it returns the result to the host;

4. The mirror I/O node write command returns.

(4) The asynchronous writing protocol B includes the following steps in turn:

1. The write command is sent from the host to the I/O node;

2. After the I/O node processes the write command, it returns the result to the host;

3. Adaptive asynchronous protocol data replication thread reads the changed data block and writes it to the mirror I/O node;

4. The mirror I/O node write command returns.

Figure 7 is a block diagram of its program.

Proof of use: Compared with the commonly used synchronous mirroring method, the asynchronous remote mirroring method based on load adaptation has a certain improvement in the command response time, and the performance improvement range will increase with the increase of the system load.

Description of drawings

Figure 1: Hardware structure diagram of massive network storage system TH-MSNS

Figure 2: I/O path diagram of TH-MSNS storage system based on Fiber Channel transmission protocol FCP

Figure 3: Hardware structure diagram of TH-MSNS system I/O node data remote mirroring

Figure 4: Schematic diagram of asynchronous write protocol A

Figure 5: Schematic diagram of asynchronous write protocol B

Fig. 6: the structural diagram of each submodule of the present invention in the standard simulator module

Fig. 7: the program flow diagram of dual-protocol self-adaptation of the present invention

Detailed ways

The asynchronous remote mirroring method based on load self-adaptation proposed by the present invention is designed and implemented based on the massive network storage system TH-MSNS independently developed by Tsinghua University. TH-MSNS is a SAN architecture, but it is different from the usual SAN architecture. In the usual SAN structure, the storage system is basically implemented by the Fiber Channel array controller. The Fiber Channel array controller is an integrated Fiber Channel chip, cheap Redundant Disk Array (RAID) chip, Small Computer System Interface (SCSI) Or fiber channel interface chip and single board computer (SBC) with embedded CPU and memory; it is connected to fiber switch and hub (HUB) through fiber channel interface, and connected to SAN; the back end is connected to SCSI through SCSI interface or fiber channel interface Hard disk or fiber channel hard disk (using FC-AL interface); generally speaking, the fiber channel array controller provides its own firmware (Firmware), which is responsible for the connection of fiber channel, RAID disk setting and other functions. The SAN storage device based on the array controller (Controller) has disadvantages such as poor scalability, poor compatibility, non-openness, and high price.

The most significant difference between the TH-MSNS storage system and the usual SAN structure is that it uses a software system to replace the method of controlling storage I/O operations that usually uses a fiber channel array controller, avoiding the expensive price of hardware control, At the same time, an efficient software storage control method is used to obtain maximum performance and flexibility and is applicable to various underlying transmission protocols. Such as FCP, iSCSI protocol, etc.

The target device of TH-MSNS storage system is a complete general server (I/O processing node). The I/O node provides a dedicated storage network for the server cluster, and is connected to different host adapter cards (such as FC HBA, Ethernet HBA) according to different needs to receive the package sent by the host through the network (Fibre Channel, Ethernet, etc.) An information frame or packet containing SCSI commands and user data. The role of the HBA is to unpack the received information frame or data packet, restore it to SCSI commands and user data, and then hand it over to the SCSI target simulator (Target Simulator Module) for processing. The SCSI target emulator is a software module running on the I/O node. Its function is to process, queue, and encapsulate the SCSI commands and user data transmitted by the HBA, and then pass the commands to the SCSI subsystem for processing. And return the execution status of the command returned by the SCSI subsystem to the host according to the original path. As shown in Figure 1.

All sub-modules mentioned in the asynchronous remote mirroring method based on load self-adaptation run in the SCSI target simulator of the I/O node. See Figure 2.

Similar to the structure of the I/O node, by adding a Mirror I/O node to the optical fiber network, it forms a storage dual-node cluster together with the I/O node. Use storage clusters to provide redundant storage paths to achieve data replication. The difference between this design and the common cluster system with equal structure is that the two nodes are not on the same layer, but use the newly added node to provide the network hard disk for the original single I/O node, which is used for TH - Data remote mirroring of I/O nodes of MSNS system. The corresponding mirror design structure is shown in Figure 3.

In the structure diagram of remote mirroring of I/O node data in the TH-MSNS system, it can be seen that the front-end host is the initiator, and the back-end I/O node is the target. In the figure above, the I/O requests and data of ordinary storage enter the I/O node through the thin line from ① or ②, through the switch and ③, and are submitted by the I/O node to the loaded RAID subsystem for implementation . The newly added mirroring data link adopts a thick line path, at this time, the I/O node acts as the initiator, and the mirroring I/O node works in the target mode. The mirrored data starts from the HBA2 card of the I/O node, passes through ④, the switch, and ⑤, and then submits it to the mirrored I/O node for implementation.

The categories and brief functions of the above sub-modules are shown in the table below: module name module function Mirroring relationship initialization submodule When the system is started, it is responsible for the recovery process of the mirrored pair, and the mirrored pair is restored from the site (mirrored relationship table) retained when the system was stopped last time; and a user interface is provided when the system is running, and the mirrored disk pair is specified by the user. Real-time storage submodule It is responsible for handing over the specific read and write operations to the SCSI subsystem for processing, and is responsible for analyzing the execution results of the commands. If an error is found, the error command is handed over to the error handling analysis sub-module for processing. Data change record sub-module Record specific disk read and write events. When a write operation occurs, a corresponding recording operation is performed using a storage bitmap (Bitmap) to record the change information of the disk medium. Dual-protocol adaptive data replication submodule According to the load situation of the system, two different asynchronous writing protocols (asynchronous writing protocol A and asynchronous writing protocol B) are dynamically used to perform different operations to complete the asynchronous mirroring process. Synchronization submodule When the mirror relationship is initially established, or after a disk failure occurs and a new disk is used, synchronization between disks is performed to ensure data consistency. Error Handling Analysis Submodule Provides an interface for error handling for all modules. Contains an error handling thread that analyzes and handles all error conditions differently. System exit submodule Responsible for ensuring that when the system exits normally, the data of the I/O node and the mirror I/O node are completely consistent. And, when the system is abnormally terminated, the on-site preservation and recording work.

The sub-modules of the SCSI target simulator will be described in detail below, as shown in Figure 7

1. Mirroring relationship initialization sub-module. When the system is started, the mirror pair recovery process is carried out through this module, and the mirror pair recovery is carried out from the site (mirror relationship table) retained when the system was stopped last time; and the user can dynamically specify the mirror disk through the user interface provided by the module when the system is running right. After the mirror relationship is established, the synchronization sub-module is automatically called to perform corresponding synchronization processing.

2. Real-time storage sub-module and data change recording sub-module. The real-time storage sub-module is responsible for handing over the specific read and write operations to the SCSI subsystem for processing, and is responsible for analyzing the execution results of the commands. If an error is found, the error command is handed over to the error handling analysis sub-module for processing. The data change recording sub-module records specific disk read and write events. If there is a write operation, it means that the disk media has been changed. While the system does not affect the occurrence of the write operation, it stores the bitmap (Bitmap) for corresponding records. For the purpose of minimization, only changed disk block addresses are recorded.

3. Dual-protocol adaptive data replication sub-module. Dual-protocol adaptive data replication is divided into two cases, that is, following asynchronous writing protocol A and asynchronous writing protocol B to perform different operations. When the system load is light, asynchronous write protocol A is adopted, that is, when the real-time storage submodule performs disk write operations, a copy of the command and data is sent to the mirror disk for write operations; Protocol B ensures that the real-time storage sub-module is not affected. While real-time storage occurs in the storage system, another adaptive asynchronous protocol data replication thread is responsible for the asynchronous mirroring process based on asynchronous write protocol B (see Figure 5). This thread obtains the time slice from the system CPU, and reads out the data blocks changed in step 2 recorded by the Bitmap from the mirroring source at regular intervals or according to the system load, and copies them to the mirroring destination disk to complete the asynchronous mirroring process.

4. The synchronization sub-module needs to synchronize between disks to ensure data consistency when the mirror relationship is initially established after the system starts, or after a disk fails and a new disk is replaced. The synchronization module is a separate thread, which obtains the operation lock before the specific mirroring occurs, and completely copies the data from the mirroring source to the mirroring destination disk; for the case where the original mirroring relationship is interrupted and rebuilt, after the synchronization thread obtains the operation lock , an incremental synchronization can be performed based on all the Bitmap records during the period when the mirror relationship expires.

5. The error handling analysis sub-module, the error handling module has an error handling thread, and all the above modules have the entrance to the error handling thread. When an error occurs, the error handling thread acquires the operation lock, analyzes the exception and performs intelligent processing. If it is a hardware failure, an alarm is issued, and at the same time, the log record in the abnormal situation is started for recovery.

6. The system exits the sub-module, and exits the real-time storage module before the system exits. If the system exits normally, the asynchronous mirroring module obtains the operation lock, and writes all the changed disk blocks marked in the Bitmap to the mirroring destination disk, and then performs the entire Suspension of the system; if the system exits in an emergency due to a partial failure, the Bitmap content in the memory will be saved in the configuration file of the system disk before the emergency exit, that is, the data accuracy of the image source is guaranteed while the storage bitmap is preserved. At the same time, set the emergency exit safety bit, so that incremental synchronization can be performed after checking the safety bit when the system is rebuilt; if the system fails and fails, there is no safety bit, and the mirror pair needs to be completely synchronized after the system is rebuilt.

The present invention is a SAN asynchronous remote mirroring method based on a unified storage medium for duplication and dynamically adopting two asynchronous writing protocols based on load conditions. It uses a storage bitmap (Bitmap) to perform asynchronous mirroring storage by a unified mirroring monitoring program. Disaster handling and synchronization operations. And because it adopts the asynchronous writing protocol, it can be applied to some low-speed networks. Therefore, in addition to being applicable to the FCP protocol as the underlying transmission protocol, it can also be applied to iSCSI protocol, InfiniBand and other protocols. SAN-based dual-protocol adaptive asynchronous remote mirroring contains the following steps in turn:

(1) The real-time storage module provides a management interface to configure the disk mapping table during the mirroring process:

The mirror storage module obtains the operation lock from the host, and performs the actual data read and mirror write operations. The read operation only needs to be completed on the I/O node, and the write operation needs to be completed asynchronously on the I/O node and the mirror I/O node;

(2) The disaster recovery processing module is called by the monitoring program when the mirror storage module device, that is, the I/O node or the mirror I/O node reads and writes errors: the disaster recovery processing module switches the mirror cluster system, and the normal working storage device Continue to provide full data services, while enabling logging;

(3) After the damaged device is updated, the re-synchronization sub-module acquires the operation lock and performs background synchronization of data; after the synchronization is completed, the disaster recovery processing module notifies the mirror monitoring program to switch the system from the disaster recovery module to the mirror storage mode during normal operation .

The above-mentioned disaster recovery processing operation belongs to the prior art.

After the mirror relationship is established, the data synchronization between disks is mainly divided into two stages to achieve

(1) The first stage is the whole disk copy stage between mirror disks. At this stage, we first use the mechanism of snapshot technology to create a static copy of the source disk for full-disk replication between mirror disks. For the write operation to the source disk during copying, first record the data block that needs to be changed by the write command (only need to set a bit in the corresponding Bitmap, indicating that the data block has been changed), and then use the COW (Copy On Write) algorithm , the original copy of the data block modified by the cache write operation is used to ensure the consistency and continuity of the entire disk replication.

(2) The second stage is to read the modified data block from the source disk and write it into the mirror disk to achieve complete data synchronization. At this stage, we use a separate synchronous thread to complete the above data reading and writing. For the write operations to the source disk that occurred at this stage, we also use the Bitmap table to record the changed data blocks, and hand them over to the synchronization thread to process in order. The asynchronous mirroring process begins.

Implementation mechanism of asynchronous mirroring process

1. The read operation process in the process of asynchronous mirroring is relatively simple, and the operation steps are the same as those of synchronous mirroring. Contains the following steps in order:

(1) The HBA of the I/O node receives the protocol packet (which can be FCP packet, iSCSI packet or InfiniBand packet, etc.) encapsulating SCSI commands and user data sent by the Host.

(2) The HBA analyzes the protocol, unpacks the data, and takes out SCSI commands and user data.

(3) The HBA hands over the SCSI command and user data to the target simulation module (Target Simulator Module) for processing.

(4) The target simulation module sends SCSI commands directly to the SCSI middle layer, and allocates data buffers for commands, and coordinates the interaction between modules and commands.

(5) The SCSI middle layer issues SCSI commands to the inexpensive redundant disk array SCSI Raid subsystem, and completes the actual reading of data;

(6) Regardless of whether it is successful or not, the result of the I/O request is returned to the Host in the same way.

2. The write operation during the asynchronous mirroring process has two different processing methods according to the I/O load of the I/O node. Note: The I/O load of the I/O node can be measured according to the length of the write command queue of the target analog module.

When the I/O load of the I/O node is light, the processing method adopts the asynchronous writing protocol A (see FIG. 4 ). Under this protocol, the I/O node sends the write command issued by the host to the local disk and the remote mirror I/O node for processing respectively. Unlike the synchronous write protocol, the I/O node receives the local write command After confirming that the command is completed, it returns to the host that the operation is complete, without waiting for the completion of the write command of the mirror disk. Contains the following steps specifically (see accompanying drawing 5):

(1) The HBA of the I/O node receives the protocol packet (which can be FCP packet, iSCSI packet or InfiniBand packet, etc.) encapsulating SCSI commands and user data sent by the Host.

(2) The HBA analyzes the protocol, unpacks the data, and takes out SCSI commands and user data.

(3) The HBA hands over the SCSI command and user data to the target simulation module (Target Simulator Module) for processing.

(4) The target simulation module (Target Simulator Module) copies the received SCSI command and data, and stores the command into the historical command queue for use by the error handling module when an error occurs;

(5) SCSI command after copying and data a copy are sent to local SCSI-RAID (SCSI Redundant Disk Array) subsystem to realize writing through SCSI middle layer;

(6) Another copy of the copied SCSI command and data is used for mirroring operation. It is sent to the mirror I/O node through the SCSI intermediate layer. The corresponding processing flow of the mirror I/O node is similar to that of the I/O node.

(7) After the local write operation command ends, it does not need to wait for the end of the write operation of the mirror I/O node, and returns the execution result to the Host node through the original path.

(8) After the write operation of the mirror I/O node is completed, the result is returned to the I/O node. The I/O node performs different processing according to the returned result. If the operation is successful, the previously enqueued command will be dequeued, indicating that the command has been executed. If the operation is unsuccessful, the error handling module is triggered to perform error handling operations such as retry.

When the I/O load of the I/O node is heavy, the processing method adopts the asynchronous writing protocol B (see FIG. 3 ). Under this protocol, the I/O node sends the write command issued by the host only to the local disk for processing, and uses the Bitmap to record the data block changed by the command. After the I/O node is confirmed that the execution of the local write command is completed, it returns the end of the operation to the host. The changed data block is automatically read from the local by the asynchronous data replication process according to the system load or timing, and sent to the mirror I/O node for execution to perform asynchronous mirroring. (See Attachment 5)

Specifically, it contains the following steps:

(1) The HBA of the I/O node receives the protocol packet (which can be FCP packet, iSCSI packet or InfiniBand packet, etc.) that encapsulates the SCSI command and user data sent by the Host;

(2) The HBA analyzes the protocol, unpacks the data, and takes out SCSI commands and user data.

(3) The HBA hands over the SCSI command and user data to the target simulation module (Target Simulator Module) for processing.

(4) The SCSI request is analyzed in the Target Simulator Module, and the data block changed by the command is recorded by using the storage bitmap (Bitmap), and then the SCSI command and data are sent to the local SCSI-RAID (SCSI Redundant Disk Array) through the SCSI middle layer The subsystem implements data writing;

(5) After the local write operation command is completed, the execution result is directly returned to the Host node through the original route.

(6) The asynchronous data replication process automatically reads the data blocks set in Bitmpap (the data blocks changed by the write command) from the local in the original order according to the load of the system or timing, and sends them to the mirror I/O node for execution, performing asynchronous mirror image.

(7) After the write operation of the mirror I/O node is completed, the result is returned to the I/O node. The I/O node performs different processing according to the returned result. If the operation is successful, the corresponding setting in the Bitmap is cleared, indicating that the data block has been mirrored. If the operation is unsuccessful, the error handling module is triggered to perform error handling operations such as retrying.

In the disaster recovery processing operation, when the physical disk device of the module to which the I/O node belongs is damaged, when the SCSI_RAID driver detects a hot-plug event or determines that a non-hot-plug event fails according to the return value of the read and write operation, the I/O According to the data structure of the mapping between the local disk and the network disk, the O node sets the state of the corresponding disk as Defunct, and under the control of the I/O node, transfers the operation request for the failed node to the corresponding mirror disk, and at the same time utilizes the storage The bitmap (Bitmap) records subsequent requests based on the failed disk, and is used for data resynchronization after the original physical disk is recovered.

In the disaster recovery processing operation, when the I/O node is damaged, the optical fiber switch will send a failure warning to the management node, and the management node will start the disaster recovery operation: open the software shielding of the mirror node to the host, make the mirror node visible to the host, All requests sent to the I/O node are completed by the mirror node, and log records are established at the same time, so that after the failed I/O node rejoins, data synchronization is accelerated. Once the I/O node rejoins the two-node cluster storage system, the management node initiates the process of re-establishing data and services.

In the disaster recovery operation, when the mirror I/O node fails, the original dual-node architecture needs to be switched to the single-node architecture, and the I/O node continues to provide data services, while the active I/O node creates a storage bitmap (Bitmap ) data block change record, so that after the failed mirror I/O node rejoins, the data synchronization process can be accelerated.

The resynchronization is completely performed in the background, so that the system automatically switches from the storage mode provided by a single node to the dual-node working mode.

The system proposed according to the method of the present invention is characterized in that it comprises the following equipment:

Host (Host): used to build a cluster system, provide network users with highly available network services, or provide high-performance computing capabilities;

Interconnected devices: including host adapter cards (such as FC HBA, iSCSI HBA, etc.), switches (such as FC Switch, iSCSISwitch, etc.) and data links;

I/O node (I/O node): provides unified network storage services for the host cluster system;

Mirror I/O node (Mirror I/O node): Provide data mirroring storage space for I/O nodes and provide uninterrupted storage services for the cluster system when a storage disaster occurs on the I/O node.

Test Results

The average command response time is a very important index to measure the service quality and performance of the system. Here, we use the asynchronous remote mirroring method based on load adaptation and the commonly used synchronous mirroring method to test, and compare the command response time accordingly. The hardware environment of the test is the mass network storage system TH-MSNS of Tsinghua University. The I/O node uses a 32-bit Itanium 2.4GHZ dual-CPU server with 2GB of memory and an operating system of Linux (Kernel 2.4.18-5). The storage subsystem adopts 3410 SCSI RAID card of adapetc company, and a disk cabinet composed of 14 73GB 10000 rpm SCSI disks of Seagate company. The underlying protocol is FCP, using 2Gb/s fiber channel. The test tool is the iometer of Inter Company, and the reading method is sequential reading. Because the mirroring operation is aimed at the write command, and the execution of the read command is only completed locally, there is no impact on system performance. So in the test, we only tested the response time of the two different methods in the case of 100% write commands. The test data is shown in the table below: block size average response time Asynchronous mirroring (ms) Synchronous mirroring (ms) 64KB 1.718 1.795 128KB 3.482 3.573 192KB 5.357 5.335 256KB 7.083 7.202 512KB 13.573 14.469 1024KB 24.797 28.610

The test data shows that the asynchronous remote mirroring method based on load adaptation has a certain improvement in command response time compared with the commonly used synchronous mirroring method. It should be pointed out that the range of performance improvement increases with the increase of the system load. The greater the load on the system, the greater the performance improvement of the load-adaptive-based asynchronous remote mirroring method compared to the commonly used synchronous mirroring method. In addition, this test data is obtained based on the optical fiber network test. The transmission delay of the optical fiber network is very small. If it is based on other underlying transmission networks (such as: Ethernet), the command response time using this method can be greatly improved compared to synchronous mirroring. improve.

The present invention has the following characteristics:

(1) Asynchronous remote mirroring based on unified storage is adopted to achieve data security and high availability, and under the premise of maintaining good data consistency, different asynchronous writing protocols are dynamically adopted according to the load of I/O nodes , effectively shortening the command response time of the system, especially in the case of high load, it has a greater system performance advantage compared with synchronous mirroring.

(2) Make full use of the long-distance connection capability of SAN and the characteristics of unified storage, and use the secondary structure of the two-node cluster to provide mirroring, and use the flexible feature of software control of I/O nodes to continue to provide data when any node fails. Data service, and provides a background data resynchronization mechanism after the system is rebuilt, that is, the disaster recovery function;

(3) The asynchronous remote mirroring method can be applied to various underlying transmission protocols such as FCP protocol, iSCSI protocol, InfiniBand protocol, etc. It has good flexibility, and when implementing the iSCSI protocol, it can make full use of existing resources and reduce system costs. .

(4) All software runs in the core state of the operating system, implemented through the kernel module, which reduces the memory copy of the user state and the core state, and improves efficiency;

(5) Asynchronous remote mirroring can be used in combination with synchronous remote mirroring (for different disks or logical volumes) to meet different requirements of different applications.

Claims (3)

1, in the SAN system based on the asynchronous remote mirror method of loaded self-adaptive, adopt asynchronous mirror image mode, it is characterized in that: be in the storage area network SAN that consists of of Mirror I/O node and disk array thereof at the mirror image I/O node that network hard disc is provided by main frame, the InterWorking Equipment that comprises host adaptor, switch and data link, I/O node and disk array thereof, for above-mentioned I/O node, the minicomputer interface of I/O node is that the scsi target simulator is carried out successively following steps according to the modular structure of setting and realized method of the present invention:

(1) the scsi target simulator is provided with:

Mirror initializes submodule, is provided with: mirror image is to automatic restoration interface STML_Read_Mirror_Map (), STML_Restore_Mirror_Map (); User's designated mirror concerns interface ProcWriteUserCommand (); Unusually withdraw from Processing Interface Handle_Abnormal_Shutdown ();

Submodule receives mirror and initializes the submodule request synchronously, is provided with: disk synchronizing thread Sync_disk_thread (); The disk fully synchronous interface, reception of magnetic disc synchronizing thread order Full_Sync_disks (); Based on the sync cap of bitmap table, receive synchronizing thread order Bitmap_Sync_disk ();

The real-time storage submodule is provided with: command process thread STML_Handle_Cmnd_thread (); The read command Processing Interface receives command process order thread STML_Handle_Read_Command (); The write order Processing Interface receives command process order thread STML_Handle_Write_Command (); Mirror-write command process interface, STML_Handle_Mirror_Write_Command ();

Two protocol self-adapting data Replica submodules, receive the request of mirror-write command process interface in the real-time storage submodule, be provided with: self adaptation asynchronous protocol switching function and data Replica thread STML_Protocol_change (), STML_data_mover_thread (); Adopt the write order interface of asynchronous write agreement A, receive the order of above-mentioned self adaptation asynchronous protocol switching function and data Replica thread, STML_Handle_Async_A_Command (); Adopt the write order interface of asynchronous write agreement B, receive the order of above-mentioned self adaptation asynchronous protocol switching function and data Replica thread, STML_Handle_Async_B_Command ();

The data change record sub module receives the request of two protocol self-adapting data Replica submodules, is provided with: bitmap table record interface STML_Bitmap_Write (), bitmap table fetch interface STML_Bitmap_Read ();

Mistake Treatment Analysis submodule, receive respectively in real time the above-mentioned image relation and initialize submodule, the synchronously request of submodule, real-time storage submodule, two protocol self-adapting data Replica submodule and data change record sub module, be provided with: mistake processing threads STML_ERROR_Handle_thread ();

System withdraws from submodule, and when being responsible for guaranteeing normally withdrawing from system, the data of I/O node and mirror image I/O node are in full accord, and when system exception is ended, on-the-spot preservation and writing task;

(2) the scsi target simulator is according to carrying out successively following steps from the read write command of main frame:

(2.1) the scsi target simulator of I/O node receives the read write command from main frame;

(2.2) the scsi target simulator is judged command type: if read command then is dealt into local disk to order and carries out, Wait Order is finished, and just notifies main frame to inform that command execution is complete;

Otherwise, carry out according to the following steps write order;

According to being scheduled to be located at the weight that scsi target simulator write order queue length is the threshold determination I/O node load of I/O load:

If the I/O load of I/O node is less than load threshold, then belong to the lighter situation of I/O node load, just carry out that write order that asynchronous write agreement A:I/O node sends main frame is respectively transferred to local disk and long-range mirror image I/O node is processed respectively, after the I/O node is obtaining the complete affirmation of local write command execution, namely to the END instruction of main frame return;

If the I/O load of I/O node is greater than load threshold, then belong to the heavier situation of I/O node load, just carry out write order that asynchronous write agreement B:I/O node sends main frame and only transfer to local disk and process, and the data block of utilizing the bitmap table record order to change; The I/O node is after obtaining that the local write command execution is complete and must confirming, namely to the END instruction of main frame return; The data block of more correcting one's mistakes, then by self adaptation asynchronous protocol data Replica thread according to I/O node load situation or timing automaticly read from local disk, mail to mirror image I/O node and carry out the asynchronous mirroring write operation.

2, in the SAN according to claim 1 system based on the asynchronous remote mirror method of loaded self-adaptive, it is characterized in that: described asynchronous write agreement A comprises following steps successively:

(1), write order is sent to the I/O node by main frame;

(2), the I/O node copies write order and mails to mirror image I/O node;

(3), the complete local write order of I/O node processing, the result is returned main frame;

(4), mirror image I/O node write order returns.

3, in the SAN according to claim 1 system based on the asynchronous remote mirror method of loaded self-adaptive, it is characterized in that: described asynchronous write agreement B comprises following steps successively:

(1), write order is sent to the I/O node by main frame;

(2), the complete write order of I/O node processing, the result is returned main frame;

(3), self adaptation asynchronous protocol data Replica thread reads the data block of change, writes mirror image I/O node;

(4), mirror image I/O node write order returns.

Images