[go: up one dir, main page]

WO2017088664A1 - Data processing method and apparatus for cluster file system - Google Patents

Data processing method and apparatus for cluster file system Download PDF

Info

Publication number
WO2017088664A1
WO2017088664A1 PCT/CN2016/105219 CN2016105219W WO2017088664A1 WO 2017088664 A1 WO2017088664 A1 WO 2017088664A1 CN 2016105219 W CN2016105219 W CN 2016105219W WO 2017088664 A1 WO2017088664 A1 WO 2017088664A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage node
request
read
data
file system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/105219
Other languages
French (fr)
Chinese (zh)
Inventor
张勤
李璐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen China Blog Imformation Technology Co Ltd
Original Assignee
Shenzhen China Blog Imformation Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen China Blog Imformation Technology Co Ltd filed Critical Shenzhen China Blog Imformation Technology Co Ltd
Publication of WO2017088664A1 publication Critical patent/WO2017088664A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604Improving or facilitating administration, e.g. storage management
    • G06F3/0605Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • G06F16/1824Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0643Management of files
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present invention relates to the field of data processing technologies, and in particular, to a data processing method and apparatus for a cluster file system.
  • cluster file systems have become a new trend in the development of computer technology.
  • the cluster file system provides a large-capacity file system shared storage application, such as CIFS, by concatenating a plurality of single independent hosts into a systematic whole and relying on the storage area network. Multiple concurrent user operations and big data transfers within the storage area network are implemented.
  • cluster nodes also increases the likelihood of node downtime or service failure. For example, when the client reads and writes a large file to the cluster, if the service node that accepts the read/write request fails, even if the service can switch to other nodes in time, since there is no data cache in the memory of other nodes, then the client The machine's read and write operations will still be interrupted.
  • the main object of the present invention is to provide a data processing method and apparatus for a cluster file system, which aims to solve the technical problem of interrupting the read and write operations of the client due to the failure of the cluster service node.
  • the present invention provides a data processing apparatus for a cluster file system, and the data processing apparatus of the cluster file system includes:
  • a storage module configured to: when the storage node receives the file read or write request, and the storage node where the storage module is located is the primary storage node, storing the request information in the read or write request into the cache Area;
  • a synchronization module configured to synchronize the request information to a slave storage node in the cluster file system
  • a data processing module configured to read data in the storage area according to the read request and the request information in the buffer area, or to the storage area according to the write request and the request information in the buffer area Write data in .
  • the present invention further provides a data processing method for a cluster file system, where the data processing method of the cluster file system includes:
  • the storage node determines whether data corresponding to the read or write request requires verification
  • the storage node acquires the verification information input by the user;
  • the storage node When the verification information matches the pre-stored verification information, the storage node reads data in its storage area according to the read request and the request information in the buffer area, or according to the write request and the The request information in the buffer area writes data to its storage area;
  • the storage node stores the request information in the read or write request into its buffer area
  • the storage node synchronizes the request information to a slave storage node in the cluster file system
  • the service request is not a read or write request, then responding to the service request.
  • the present invention further provides a data processing method for a cluster file system, where the data processing method of the cluster file system includes:
  • the storage node When the storage node receives a file read or write request, and the storage node is a primary storage node, the storage node reads the storage area according to the read request and the request information in the cache area. Data, or writing data to its storage area according to the write request and the request information in the buffer area;
  • the storage node stores the request information in the read or write request into its buffer area
  • the storage node synchronizes the request information to a slave storage node in the cluster file system.
  • the data processing method and apparatus of the cluster file system provided by the present invention, when the storage node receives a file read or write request, and the storage node is a primary storage node, the storage node according to the read request and the The request information in the buffer area reads data in its storage area, or writes data to its storage area according to the write request and the request information in the buffer area, and the storage node reads the data. Or the request information in the write request is stored in its cache area, and the storage node synchronizes the request information to the slave storage node in the cluster file system. After the primary storage node fails, the slave storage node may be in the cache area according to the cache node.
  • the service request information takes over the client's read or write process without interruption, ensuring the stability of the cluster file system service.
  • FIG. 1 is a schematic diagram of functional modules of a first embodiment of a data processing apparatus of a cluster file system according to the present invention
  • FIG. 2 is a schematic diagram of functional modules of a data processing apparatus of a cluster file system according to the present invention for processing other services such as reading and writing;
  • FIG. 3 is a schematic diagram of functional modules of a second embodiment of a data processing apparatus of a cluster file system according to the present invention.
  • FIG. 4 is a schematic diagram of a refinement function module of the detection module of FIG. 3;
  • FIG. 5 is a schematic diagram of functional modules of a third embodiment of a data processing apparatus of a cluster file system according to the present invention.
  • FIG. 6 is a schematic flowchart of a first embodiment of a data processing method of a cluster file system according to the present invention.
  • FIG. 7 is a schematic flowchart of a data processing method of a cluster file system according to the present invention for processing other services such as reading and writing data;
  • FIG. 8 is a schematic flowchart of a second embodiment of a data processing method of a cluster file system according to the present invention.
  • FIG. 9 is a schematic diagram of a refinement process of detecting a link connection state and an operation state of a primary storage node in FIG. 8;
  • FIG. 10 is a schematic flowchart diagram of a third embodiment of a data processing method of a cluster file system according to the present invention.
  • the present invention provides a data processing apparatus for a cluster file system.
  • FIG. 1 is a schematic diagram of functional modules of a first embodiment of a data processing apparatus of a cluster file system according to the present invention.
  • the data processing device of the cluster file system includes:
  • the data processing module 10 is configured to: when the storage node receives the file read or write request, read the data in the storage area according to the read request and the request information in the buffer area, or according to the writing The request and the request information in the buffer area write data to the storage area;
  • a storage module configured to store request information in the read or write request into its buffer area
  • the storage module 20 may acquire the configuration parameters on the control node at the initial startup, perform an initialization operation, and create the buffer area in each storage node. It can be understood that the configuration parameter may be set by the user or may be used during server manufacturing. Entered by the manufacturer, and the configuration parameters can be modified at any time and forwarded by the control node to all storage nodes.
  • the configuration parameter includes: a workgroup name, which is used to define a workgroup name of the cluster file system; a server name, which is used to define a name of each storage node of the cluster file system; and a maximum number of connections, which is used to define the most The number of clients that are allowed to access the cluster file system at the same time; the unified storage path is used to define the root directory of the cluster file system, that is, the same directory address or file can be accessed through any node address; the redundant network interface is used to define the cluster.
  • Each storage node of the file system is used as a redundant network interface, that is, a network card; a redundant network address is used to define a network address of each storage node of the cluster file system used as a redundant network interface (ie, an IP corresponding to the network card), Multiple addresses can be set; the redundant host address is used to define the public network address that the cluster file system provides access to the client, and only one can be set; the data cache size is used to define the storage file cache on each storage node of the cluster file system.
  • the memory size of the data For example, the user can set the storage area of the storage node to occupy 20 GB of storage space, and the storage space occupied by the storage area cannot exceed the maximum storage space of the storage node server.
  • the synchronization module 30 is configured to synchronize the request information to a slave storage node in the cluster file system.
  • the control node when the system initiates initialization, sends a message to the server that is set as the primary storage node, and controls the primary storage node to start the program, and the other storage nodes do not start the service program, and can only receive and A request information for storing a read or write request synchronized by the primary storage node, and only the primary storage node synchronizes the read or write request to all slaves in the cluster file system upon receiving a data read or write request
  • the storage node stores the requested information of the read or write request received from the storage node into its buffer area without any other operations.
  • the request information of the request After the primary storage node receives the data read or write request from the client, the request information of the request: the process number of the client data read or write, the I/O offset of the process, Synchronize to all slave storage nodes.
  • the storage node may determine the type of the service request when receiving the service request, and perform corresponding processing according to the type of the server request, that is, the data processing apparatus of the cluster file system further includes:
  • the detecting module 40 is configured to detect a type of the service request when receiving a service request, and determine whether the service request is a read or write request;
  • the response module 50 is configured to respond to the service request if the service request is not a read or write request;
  • the data processing module 10 is further configured to: if the service request is a read or write request, read data in the storage area according to the read request and the request information in the buffer area, or according to the The write request and the request information in the buffer area write data to the storage area.
  • the detecting module 40 detects the service type invoked by the request, and when the request invokes the data reading or writing service, the synchronization module 30 sends the requested request information to the
  • the primary storage node network interface is further sent by the network interface of the primary storage node to the secondary storage node corresponding to the node address obtained from the control node, and the received synchronization request is stored in the buffer area by the storage module 10 of the storage node .
  • the primary storage node not only receives the data read or write request, but also may receive other types of service requests, such as configuration parameter modification requests, and the response module 50 responds directly to the service request.
  • the response module 50 responds directly to the service request.
  • the storage module 20 does not store the service request into its buffer area, and the synchronization module 20 does not synchronize the service request to all the slave storage nodes. .
  • the primary storage node when the primary storage node may not be able to provide data reading or writing services to the client due to system upgrade or maintenance, etc., the primary storage node selects a normal working state. From the storage node, the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.
  • the primary storage node after receiving the data operation request, the primary storage node synchronizes the requested information to all the slave storage nodes, and when the storage node needs to be switched, any one of them is in a normal working state.
  • the slave storage node can take over the client's data operation process, provide the client with uninterrupted data operation services, and improve the availability of the cluster file system.
  • the data processing apparatus of the cluster file system further includes:
  • the detecting module 40 is further configured to detect a link connection state and an operating state thereof;
  • the detecting module 40 includes:
  • the link detecting unit 41 is configured to periodically send the first detection data packet to the control node;
  • the read/write detecting unit 42 is configured to determine whether the reading and writing are normal when the response packet is received within a preset time interval;
  • the determining unit 43 is configured to determine that the operation is normal when the reading and the writing are normal, determine the operation failure when the reading and the writing are abnormal, and determine that the response data packet is not received within the preset time interval.
  • the link connection is faulty.
  • the primary storage node selects a secondary storage node in a normal working state.
  • the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.
  • the switching module 60 is configured to: when detecting a link connection failure or an operation failure, select a primary storage node from the storage node in a normal working state, and mark the selected address of the secondary storage node as a primary storage node address;
  • An update module 70 configured to send the marked primary node address to the control node and the selected secondary storage node, where the control node updates the saved primary node address by using the received primary node address update And the selected storage node switches the working state to the state of the primary storage node when receiving the address of the primary node.
  • the switching module 60 selects a slave storage node in a normal working state, marks the selected address of the slave node as the address of the new primary storage node, and the update module 70 sends the address.
  • the control node will forward the subsequent client request to the new primary storage node, the selected secondary storage node will switch to the working state of the primary storage node, and the new primary storage node can read its cache area.
  • the internal data reads or writes the process information, directly takes over the corresponding process, and does not interrupt the data operation process of the client.
  • the performance of the slave storage node may be substantially the same, or the performance difference may be large. Therefore, when the primary storage node performs the handover, if all the performances of the slave storage nodes are similar, the switching module 60 may randomly select one to be in normal operation.
  • the slave storage node of the state is used as the primary storage node. If the performance of all the slave storage nodes is different, the node with higher performance in the normal working state may be selected as the new primary storage node according to the performance level of the storage node.
  • the data processing device of the cluster file system in this embodiment periodically detects the link connection state of the primary storage node and the service program running state, discovers the fault of the primary storage node in time, and then switches the primary storage node to provide uninterrupted support for the client.
  • Data read and write services ensure the stability of cluster file system data processing services.
  • the present invention further provides a third embodiment of the data processing apparatus of the cluster file system based on the first or second embodiment.
  • the data processing apparatus of the cluster file system also includes:
  • a determining module 80 configured to determine, when a file read or write request is received, whether data corresponding to the read or write request needs to be verified;
  • the primary storage node when the primary storage node receives the file read or write request, it detects the access level of the file requested to be accessed. If the file is a shared file, it can be accessed by all users. There is no need to verify the identity rights of the user. If the file is a private file, that is, the file can only be accessed by a user with permission, the identity permission of the user needs to be verified to determine whether the file can be issued. Client access to the request.
  • the obtaining module 90 is configured to obtain verification information input by the user when the data corresponding to the read or write request needs to be verified;
  • the primary storage node When the file requested by the client is a private file, that is, when authentication is required, the primary storage node needs to obtain information input by the user and verify the authority of the user. At this point, the user may have logged in, and the primary storage node only needs to traverse the information forwarded by the control node, grab the data with the user name and password key or handle, and then obtain the information input by the user, and perform the user's authority. If the user has not logged in at this time, the primary storage node sends a message to the control node, notifying the control node to send a reminder to the client or controlling the client to pop up the login interface. After the user inputs the verification information, the control node inputs the user. The verification information is forwarded to the primary storage node for verification.
  • the data processing module 10 is further configured to: when the verification information matches the pre-stored verification information, read data in the storage area according to the read request and the request information in the buffer area, or according to the The write request and the request information in the buffer area write data to its storage area.
  • the data processing apparatus of the cluster file system obtains the operation authority of the data after receiving the data operation request, and obtains the verification information input by the user if the data requires the authority verification, in the user When the input verification information matches the pre-stored verification information, the corresponding data operation is performed to improve the security of the cluster file system data.
  • the invention further provides a data processing method for a cluster file system.
  • the data processing method of the cluster file system includes:
  • Step S10 when the storage node receives the file read or write request, the storage node reads the data in the storage area according to the read request and the request information in the buffer area, or according to the write The incoming request and the request information in the buffer area write data to its storage area;
  • Step S20 the storage node stores the request information in the read or write request into its buffer area
  • the storage node may acquire the configuration parameters on the control node at the initial startup, perform an initialization operation, and create the cache area in the storage node. It can be understood that the configuration parameter may be set by the user, or may be set by the manufacturer during server manufacture. Enter.
  • configuration parameters are described in the first embodiment of the data processing of the cluster file system, and details are not described herein again.
  • the control node when the system initiates initialization, sends a message to the server that is set as the primary storage node, and controls the primary storage node to start the program, and the other storage nodes do not start the service program, and can only receive and A request information for storing a read or write request synchronized by the primary storage node, and only the primary storage node synchronizes the read or write request to all slaves in the cluster file system upon receiving a data read or write request
  • the storage node stores the requested information of the read or write request received from the storage node into its buffer area without any other operations.
  • the request information of the request After the primary storage node receives the data read or write request from the client, the request information of the request: the process number of the client data read or write, the I/O offset of the process, Synchronize to all slave storage nodes.
  • Step S30 the storage node synchronizes the request information to a slave storage node in the cluster file system.
  • the primary storage node After receiving the data read or write request, the primary storage node responds to the request and provides a data read or write service to the client according to the request information.
  • the primary storage node not only receives the data read or write request, but also may receive other types of service requests, such as configuration parameter modification requests, and directly respond to the request, as described above.
  • the corresponding parameter is requested to be modified accordingly, but the service request is not stored in its cache area, and the service request is not synchronized to all the slave storage nodes.
  • the storage node may determine the type of the service request when the service request is received, and perform corresponding processing according to the type of the server request, that is, the data processing method of the cluster file system further includes:
  • Step S40 When the storage node receives the service request, detecting a type of the service request.
  • Step S50 determining whether the service request is a read or write request
  • the service request is a read or write request, executing, by the storage node, reading data in a storage area according to the read request and request information in the buffer area, or according to the write request And a step of writing data in the storage area to the request information in the buffer area;
  • the data processing method of the cluster file system further includes:
  • Step S60 if the service request is not a read or write request, responding to the service request.
  • the primary storage node After receiving the service request, the primary storage node detects the service type invoked by the request, and sends the requested request information to the primary storage node network interface when the request invokes the data read or write service. Then, the network interface of the primary storage node sends the slave storage node corresponding to the node address acquired from the control node, and the synchronization request received from the storage node is stored in its buffer area.
  • the primary storage node when the primary storage node may not be able to provide data reading or writing services to the client due to system upgrade or maintenance, etc., the primary storage node selects a normal working state. From the storage node, the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.
  • the data processing method of the cluster file system in this embodiment after receiving the data operation request, the primary storage node synchronizes the requested information to all the slave storage nodes, and when the storage node needs to be switched, any one is in a normal working state.
  • the slave storage node can take over the client's data operation process, provide the client with uninterrupted data operation services, and improve the availability of the cluster file system.
  • the primary storage node when the primary storage node provides services to the client, a failure may occur, and the primary storage node needs to periodically detect the link connection status and the service program running status, so as to timely find the fault and perform corresponding processing, based on the first
  • An embodiment provides a second embodiment of the data processing method of the cluster file system of the present invention.
  • the data processing method of the cluster file system further includes:
  • Step S70 When the storage node is a primary storage node, the storage node detects a link connection state thereof and an operation state of the storage node.
  • the step of detecting, by the storage node, its link connection state and the running state of the storage node includes:
  • Step S71 the storage node periodically sends a first detection data packet to the control node
  • Step S72 Receive a second response data packet that is sent back by the control node based on the detection data packet:
  • Step S73 when receiving the response data packet within a preset time interval, determining whether the reading and writing of the storage node are normal;
  • Step S74 when the reading and writing of the storage node are normal, determining that the storage node is operating normally;
  • Step S75 determining that the storage node is faulty when the storage node reads and writes an abnormality
  • Step S76 When the response data packet is not received within the preset time interval, determine that the link connection of the storage node is faulty.
  • the primary storage node when the primary storage node detects that it is faulty and cannot continue to provide data reading or writing services to the client, the primary storage node selects a secondary storage node that is in a normal working state, and the selected slave will be selected. The storage node switches to the primary storage node and takes over the data read or write process of the client.
  • Step S80 when the acquiring storage node detects its link connection failure or the storage node running fault in real time, the storage node selects a primary storage node from the storage node in a normal working state, and selects the selected storage node.
  • the address from the storage node is marked as the primary storage node address;
  • Step S90 sending the marked primary node address to the control node and the selected secondary storage node, wherein the control node updates the saved primary node address by using the received primary node address, and selects When the storage node receives the address of the primary node, the working state is switched to the state of the primary storage node.
  • a slave storage node in a normal working state is selected, the selected address of the slave service node is marked as the address of the new primary storage node, and the address is sent to the control node, and the control node will The subsequent client request will be forwarded to the new primary storage node, and the selected secondary storage node will switch to the working state of the primary storage node, and the new primary storage node can read the data in the buffer to read or write.
  • the process information directly take over the corresponding process, will not interrupt the client's data operation process.
  • the performance of the slave storage node may be substantially the same, or the performance difference may be large. Therefore, when the primary storage node performs the handover, if the performance of all the slave storage nodes is almost the same, the slaves in the normal working state may be randomly selected. As the primary storage node, if the performance of all the secondary storage nodes is different, the node with higher performance in the normal working state can be selected as the new primary storage node according to the performance of the storage node.
  • the data processing method of the cluster file system in this embodiment periodically detects the link connection state of the primary storage node and the service program running state, discovers the fault of the primary storage node in time, and then switches the primary storage node to provide uninterrupted support for the client.
  • Data read and write services ensure the stability of cluster file system data processing services.
  • the present invention further provides a third embodiment of the data processing method of the cluster file system based on the first or second embodiment.
  • the The data processing method of the cluster file system further includes the steps of:
  • Step S100 the storage node determines whether data corresponding to the read or write request needs to be verified
  • the primary storage node when the primary storage node receives the file read or write request, it detects the access level of the file requested to be accessed. If the file is a shared file, it can be accessed by all users. There is no need to verify the identity rights of the user. If the file is a private file, that is, the file can only be accessed by a user with permission, the identity permission of the user needs to be verified to determine whether the file can be issued. Client access to the request.
  • Step S110 when the data corresponding to the read or write request needs to be verified, the storage node acquires the verification information input by the user;
  • the primary storage node When the file requested by the client is a private file, that is, when authentication is required, the primary storage node needs to obtain information input by the user and verify the authority of the user. At this point, the user may have logged in, and the primary storage node only needs to traverse the information forwarded by the control node, grab the data with the user name and password key or handle, and then obtain the authentication information input by the user, and the authority to the user. If the user has not logged in yet, the primary storage node sends a message to the control node, notifying the control node to send a permission verification reminder to the client or controlling the client to pop up the login interface. After the user inputs the verification information, the control node will The verification information input by the user is forwarded to the primary storage node for verification.
  • Step S120 determining whether the verification information matches the pre-stored verification information
  • step S10 When the verification information matches the pre-stored verification information, executing the storage node to read data in the storage area according to the read request and the request information in the buffer area, or according to the write request and The step of writing the request information in the buffer area to the data area thereof, that is, step S10.
  • the data processing method of the cluster file system acquires the operation authority of the data after receiving the data operation request, and if the data requires the authority verification, the verification information input by the user is obtained, and the user inputs the When the verification information matches the pre-stored verification information, the corresponding data operation is performed to improve the security of the cluster file system data.
  • the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better.
  • Implementation Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk,
  • the optical disc includes instructions for causing a terminal device (which may be a mobile phone, a computer, a cloud server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)

Abstract

A data processing apparatus for a cluster file system. The data processing apparatus for a cluster file system comprises: a data processing module, for reading, when a storage node receives a file reading or writing request, data in a storage area according to the reading request and request information in a buffer, or writing data into the storage area according to the writing request and request information in the buffer; a storage module, for storing the request information in the reading or writing request into the buffer; and a synchronization module, for synchronizing the request information to a slave storage node in a cluster file system. Also provided is a data processing method for a cluster file system. After a master storage node fails, the apparatus and method enable a slave storage node to take over a reading or writing process of a client without interruption according to service request information in the buffer, thus ensuring the stability of service of a cluster file system.

Description

集群文件系统的数据处理方法和装置  Data processing method and device for cluster file system

技术领域Technical field

本发明涉及数据处理技术领域,尤其涉及一种集群文件系统的数据处理方法和装置。The present invention relates to the field of data processing technologies, and in particular, to a data processing method and apparatus for a cluster file system.

背景技术Background technique

近年来,集群文件系统已经成为计算机技术发展的新趋势。集群文件系统通过将多个单一独立的主机有机的结合串联成一个系统性的整体,并依托存储区域网络,对外提供一个大容量文件系统的共享存储应用,如CIFS。实现了存储区域网络内的多并发用户操作和大数据传输。In recent years, cluster file systems have become a new trend in the development of computer technology. The cluster file system provides a large-capacity file system shared storage application, such as CIFS, by concatenating a plurality of single independent hosts into a systematic whole and relying on the storage area network. Multiple concurrent user operations and big data transfers within the storage area network are implemented.

虽然如此,集群节点的增加同时也增加了节点宕机或服务故障的可能性。比如,当客户机在向集群中读写一个较大文件时,如果受理该读写请求的服务节点故障,即使服务能及时切换到其他节点,由于其他节点的内存中并没有数据缓存,那么客户机的读写操作还是会中断。Nonetheless, the increase in cluster nodes also increases the likelihood of node downtime or service failure. For example, when the client reads and writes a large file to the cluster, if the service node that accepts the read/write request fails, even if the service can switch to other nodes in time, since there is no data cache in the memory of other nodes, then the client The machine's read and write operations will still be interrupted.

发明内容Summary of the invention

本发明的主要目的在于提供一种集群文件系统的数据处理方法和装置,旨在解决由于集群服务节点故障,使客户端读写操作中断的技术问题。The main object of the present invention is to provide a data processing method and apparatus for a cluster file system, which aims to solve the technical problem of interrupting the read and write operations of the client due to the failure of the cluster service node.

为实现上述目的,本发明提供的一种集群文件系统的数据处理装置,所述集群文件系统的数据处理装置包括:In order to achieve the above objective, the present invention provides a data processing apparatus for a cluster file system, and the data processing apparatus of the cluster file system includes:

存储模块,用于在存储节点接收到文件读取或写入请求,且所述存储模块所在的存储节点为主存储节点时,将所述读取或写入请求中的请求信息存入其缓存区;a storage module, configured to: when the storage node receives the file read or write request, and the storage node where the storage module is located is the primary storage node, storing the request information in the read or write request into the cache Area;

同步模块,用于将所述请求信息同步至集群文件系统中的从存储节点;a synchronization module, configured to synchronize the request information to a slave storage node in the cluster file system;

数据处理模块,用于根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据。a data processing module, configured to read data in the storage area according to the read request and the request information in the buffer area, or to the storage area according to the write request and the request information in the buffer area Write data in .

本发明进一步提供一种集群文件系统的数据处理方法,所述集群文件系统的数据处理方法包括:The present invention further provides a data processing method for a cluster file system, where the data processing method of the cluster file system includes:

在所述存储节点接收服务请求时,检测所述服务请求的类型;Detecting a type of the service request when the storage node receives a service request;

判断所述服务请求是否为读取或写入请求;Determining whether the service request is a read or write request;

若所述服务请求为读取或写入请求,则所述存储节点确定所述读取或写入请求对应的数据是否需要验证;If the service request is a read or write request, the storage node determines whether data corresponding to the read or write request requires verification;

在所述读取或写入请求对应的数据需要验证时,所述存储节点获取用户输入的验证信息;When the data corresponding to the read or write request needs to be verified, the storage node acquires the verification information input by the user;

在所述验证信息与预存的验证信息匹配时,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据;When the verification information matches the pre-stored verification information, the storage node reads data in its storage area according to the read request and the request information in the buffer area, or according to the write request and the The request information in the buffer area writes data to its storage area;

所述存储节点将所述读取或写入请求中的请求信息存入其缓存区;The storage node stores the request information in the read or write request into its buffer area;

所述存储节点将所述请求信息同步至集群文件系统中的从存储节点;The storage node synchronizes the request information to a slave storage node in the cluster file system;

若所述服务请求不是读取或写入请求,则响应所述服务请求。If the service request is not a read or write request, then responding to the service request.

本发明进一步提供一种集群文件系统的数据处理方法,所述集群文件系统的数据处理方法包括:The present invention further provides a data processing method for a cluster file system, where the data processing method of the cluster file system includes:

在存储节点接收到文件读取或写入请求,且所述存储节点为主存储节点时,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据;When the storage node receives a file read or write request, and the storage node is a primary storage node, the storage node reads the storage area according to the read request and the request information in the cache area. Data, or writing data to its storage area according to the write request and the request information in the buffer area;

所述存储节点将所述读取或写入请求中的请求信息存入其缓存区;The storage node stores the request information in the read or write request into its buffer area;

所述存储节点将所述请求信息同步至集群文件系统中的从存储节点。The storage node synchronizes the request information to a slave storage node in the cluster file system.

本发明提出的集群文件系统的数据处理方法和装置,在存储节点接收到文件读取或写入请求,且所述存储节点为主存储节点时,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据,所述存储节点将所述读取或写入请求中的请求信息存入其缓存区,所述存储节点将所述请求信息同步至集群文件系统中的从存储节点,在主存储节点故障后,从存储节点可根据其缓存区内的服务请求信息无间断的接管客户端的读取或写入进程,保证集群文件系统服务的稳定性。The data processing method and apparatus of the cluster file system provided by the present invention, when the storage node receives a file read or write request, and the storage node is a primary storage node, the storage node according to the read request and the The request information in the buffer area reads data in its storage area, or writes data to its storage area according to the write request and the request information in the buffer area, and the storage node reads the data. Or the request information in the write request is stored in its cache area, and the storage node synchronizes the request information to the slave storage node in the cluster file system. After the primary storage node fails, the slave storage node may be in the cache area according to the cache node. The service request information takes over the client's read or write process without interruption, ensuring the stability of the cluster file system service.

附图说明DRAWINGS

图1为本发明集群文件系统的数据处理装置第一实施例的功能模块示意图;1 is a schematic diagram of functional modules of a first embodiment of a data processing apparatus of a cluster file system according to the present invention;

图2为本发明集群文件系统的数据处理装置处理读写外其他服务时的功能模块示意图;2 is a schematic diagram of functional modules of a data processing apparatus of a cluster file system according to the present invention for processing other services such as reading and writing;

图3为本发明集群文件系统的数据处理装置第二实施例的功能模块示意图;3 is a schematic diagram of functional modules of a second embodiment of a data processing apparatus of a cluster file system according to the present invention;

图4为图3中检测模块的细化功能模块示意图;4 is a schematic diagram of a refinement function module of the detection module of FIG. 3;

图5为本发明集群文件系统的数据处理装置第三实施例的功能模块示意图;5 is a schematic diagram of functional modules of a third embodiment of a data processing apparatus of a cluster file system according to the present invention;

图6为本发明集群文件系统的数据处理方法第一实施例的流程示意图;6 is a schematic flowchart of a first embodiment of a data processing method of a cluster file system according to the present invention;

图7为本发明集群文件系统的数据处理方法处理数据读写外其他服务时的流程示意图;7 is a schematic flowchart of a data processing method of a cluster file system according to the present invention for processing other services such as reading and writing data;

图8为本发明集群文件系统的数据处理方法第二实施例的流程示意图;8 is a schematic flowchart of a second embodiment of a data processing method of a cluster file system according to the present invention;

图9为图8中主存储节点检测链路连接状态和运行状态的细化流程示意图;9 is a schematic diagram of a refinement process of detecting a link connection state and an operation state of a primary storage node in FIG. 8;

图10为本发明集群文件系统的数据处理方法第三实施例的流程示意图。FIG. 10 is a schematic flowchart diagram of a third embodiment of a data processing method of a cluster file system according to the present invention.

本发明目的实现、功能特点及优点将结合实施例,参照附图做进一步说明。The object, features, and advantages of the invention will be further described in conjunction with the embodiments.

具体实施方式detailed description

应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

本发明提供一种集群文件系统的数据处理装置。The present invention provides a data processing apparatus for a cluster file system.

参照图1,图1为本发明集群文件系统的数据处理装置第一实施例的功能模块示意图。 Referring to FIG. 1, FIG. 1 is a schematic diagram of functional modules of a first embodiment of a data processing apparatus of a cluster file system according to the present invention.

在本实施例中,所述集群文件系统的数据处理装置包括 :In this embodiment, the data processing device of the cluster file system includes:

数据处理模块10,用于在存储节点接收到文件读取或写入请求时,根据所述读取请求以及所述缓存区中的请求信息读取存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向存储区中写入数据;The data processing module 10 is configured to: when the storage node receives the file read or write request, read the data in the storage area according to the read request and the request information in the buffer area, or according to the writing The request and the request information in the buffer area write data to the storage area;

存储模块20,用于将所述读取或写入请求中的请求信息存入其缓存区;a storage module 20, configured to store request information in the read or write request into its buffer area;

存储模块20可在初次启动时获取控制节点上的配置参数,进行初始化操作,在各个存储节点创建所述缓存区,可以理解的是,所述配置参数可由用户自行设置,也可在服务器制造时由厂家录入,且所述配置参数可在任何时间进行修改,并由控制节点转发至所有存储节点。The storage module 20 may acquire the configuration parameters on the control node at the initial startup, perform an initialization operation, and create the buffer area in each storage node. It can be understood that the configuration parameter may be set by the user or may be used during server manufacturing. Entered by the manufacturer, and the configuration parameters can be modified at any time and forwarded by the control node to all storage nodes.

在本实施例中,所述配置参数包括:工作组名称,用于定义集群文件系统的工作组名称;服务器名称,用于定义集群文件系统各个存储节点的名称;最大连接数,用于定义最多允许同时访问集群文件系统的客户机数量;统一存储路径,用于定义集群文件系统的根目录,即通过任一节点地址都能访问到同一目录地址或文件;冗余网络接口,用于定义集群文件系统的各个存储节点用做冗余的网络接口,即网卡;冗余网络地址,用于定义集群文件系统的各个存储节点用做冗余网络接口的网络地址(即该网卡对应的IP),可以设置多个;冗余主机地址,用于定义集群文件系统面向客户端提供访问的公共网络地址,仅可设置一个;数据缓存区大小,用于定义集群文件系统各个存储节点上用于存放缓存数据的内存大小。例如用户可以设置存储节点的缓存区占用20GB的存储空间,所述缓存区占用的存储空间不得超过存储节点服务器的内存最大存储空间。In this embodiment, the configuration parameter includes: a workgroup name, which is used to define a workgroup name of the cluster file system; a server name, which is used to define a name of each storage node of the cluster file system; and a maximum number of connections, which is used to define the most The number of clients that are allowed to access the cluster file system at the same time; the unified storage path is used to define the root directory of the cluster file system, that is, the same directory address or file can be accessed through any node address; the redundant network interface is used to define the cluster. Each storage node of the file system is used as a redundant network interface, that is, a network card; a redundant network address is used to define a network address of each storage node of the cluster file system used as a redundant network interface (ie, an IP corresponding to the network card), Multiple addresses can be set; the redundant host address is used to define the public network address that the cluster file system provides access to the client, and only one can be set; the data cache size is used to define the storage file cache on each storage node of the cluster file system. The memory size of the data. For example, the user can set the storage area of the storage node to occupy 20 GB of storage space, and the storage space occupied by the storage area cannot exceed the maximum storage space of the storage node server.

同步模块30,用于将所述请求信息同步至集群文件系统中的从存储节点.The synchronization module 30 is configured to synchronize the request information to a slave storage node in the cluster file system.

在本实施例中,控制节点在系统启动初始化时,会向被设置为主存储节点的服务器发送消息,控制所述主存储节点启动程序,而其他存储节点不会启动服务程序,只能接收并存储主存储节点同步的读取或写入请求的请求信息,只有主存储节点在接收到数据读取或写入请求时,将所述读取或写入请求同步至集群文件系统中的所有从存储节点,从存储节点仅仅将接收到的所述读取或写入请求的请求信息存入其缓存区,而不进行其他任何操作。In this embodiment, when the system initiates initialization, the control node sends a message to the server that is set as the primary storage node, and controls the primary storage node to start the program, and the other storage nodes do not start the service program, and can only receive and A request information for storing a read or write request synchronized by the primary storage node, and only the primary storage node synchronizes the read or write request to all slaves in the cluster file system upon receiving a data read or write request The storage node stores the requested information of the read or write request received from the storage node into its buffer area without any other operations.

在主存储节点接收到客户端的数据读取或写入请求后,会将所述请求的请求信息:如客户端数据读取或写入的进程号、所述进程的I/O偏移量,同步至所有从存储节点。After the primary storage node receives the data read or write request from the client, the request information of the request: the process number of the client data read or write, the I/O offset of the process, Synchronize to all slave storage nodes.

参照图2,存储节点可在接收到服务请求时,确定服务请求的类型,根据服务器请求的类型进行相应的处理,即所述集群文件系统的数据处理装置还包括:Referring to FIG. 2, the storage node may determine the type of the service request when receiving the service request, and perform corresponding processing according to the type of the server request, that is, the data processing apparatus of the cluster file system further includes:

检测模块40,用于接收服务请求时,检测所述服务请求的类型,判断所述服务请求是否为读取或写入请求;The detecting module 40 is configured to detect a type of the service request when receiving a service request, and determine whether the service request is a read or write request;

响应模块50,用于若所述服务请求不是读取或写入请求,则响应所述服务请求;The response module 50 is configured to respond to the service request if the service request is not a read or write request;

所述数据处理模块10,还用于若所述服务请求是读取或写入请求,则根据所述读取请求以及所述缓存区中的请求信息读取存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向存储区中写入数据。The data processing module 10 is further configured to: if the service request is a read or write request, read data in the storage area according to the read request and the request information in the buffer area, or according to the The write request and the request information in the buffer area write data to the storage area.

在主存储节点接收到服务请求后,检测模块40会检测所述请求调用的服务类型,在所述请求调用数据读取或写入服务时,同步模块30才将所述请求的请求信息发送至主存储节点网络接口,再由主存储节点的网络接口发送至从控制节点获取到的节点地址对应的从存储节点,并由从存储节点的存储模块10将接收到同步请求存入其缓存区内。After the primary storage node receives the service request, the detecting module 40 detects the service type invoked by the request, and when the request invokes the data reading or writing service, the synchronization module 30 sends the requested request information to the The primary storage node network interface is further sent by the network interface of the primary storage node to the secondary storage node corresponding to the node address obtained from the control node, and the received synchronization request is stored in the buffer area by the storage module 10 of the storage node .

可以理解的是,主存储节点不仅仅会接收到数据读取或写入请求,也有可能会接收到其他类型的服务请求,如配置参数修改请求,此时响应模块50会响应所述服务请求直接响应所述请求,如对所述请求对应的参数进行相应的修改,但该存储模块20不会将服务请求存储至其缓存区内,且同步模块20不会将服务请求同步至所有从存储节点。It can be understood that the primary storage node not only receives the data read or write request, but also may receive other types of service requests, such as configuration parameter modification requests, and the response module 50 responds directly to the service request. In response to the request, if the parameter corresponding to the request is modified correspondingly, but the storage module 20 does not store the service request into its buffer area, and the synchronization module 20 does not synchronize the service request to all the slave storage nodes. .

可以理解的是,在本实施例中,由于系统升级或维护等原因,所述主存储节点可能无法继续为客户端提供数据读取或写入服务时,主存储节点会选择一处于正常工作状态的从存储节点,将被选取的从存储节点切换为主存储节点,接管客户端的数据读取或写入进程。It can be understood that, in this embodiment, when the primary storage node may not be able to provide data reading or writing services to the client due to system upgrade or maintenance, etc., the primary storage node selects a normal working state. From the storage node, the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.

本实施例提出的集群文件系统的数据处理装置,主存储节点在接收到数据操作请求后,将所述请求的信息同步至所有从存储节点,在需要切换存储节点时,任一处于正常工作状态的从存储节点均可接管客户端的数据操作进程,为客户端提供无间断的数据操作服务,提高集群文件系统的可用性。In the data processing apparatus of the cluster file system according to the embodiment, after receiving the data operation request, the primary storage node synchronizes the requested information to all the slave storage nodes, and when the storage node needs to be switched, any one of them is in a normal working state. The slave storage node can take over the client's data operation process, provide the client with uninterrupted data operation services, and improve the availability of the cluster file system.

进一步地,在主存储节点为客户端提供服务时,有可能发生故障,则主存储节点需定时检测其链路连接状态和服务程序运行状态,以便及时发现故障,并作出相应处理,则基于第一实施例提出本发明集群文件系统的数据处理装置第二实施例,参照图3,所述集群文件系统的数据处理装置还包括:Further, when the primary storage node provides services to the client, a failure may occur, and the primary storage node needs to periodically detect the link connection status and the service program running status, so as to timely find the fault and perform corresponding processing, based on the first An embodiment of the present invention provides a second embodiment of the data processing apparatus of the cluster file system of the present invention. Referring to FIG. 3, the data processing apparatus of the cluster file system further includes:

所述检测模块40,还用于检测其链路连接状态和运行状态;The detecting module 40 is further configured to detect a link connection state and an operating state thereof;

参照图4,所述检测模块40包括:Referring to FIG. 4, the detecting module 40 includes:

链路检测单元41,用于定时向所述控制节点发送第一检测数据包;The link detecting unit 41 is configured to periodically send the first detection data packet to the control node;

接收所述控制节点基于所述检测数据包反馈的第二响应数据包:Receiving, by the control node, a second response data packet based on the detection data packet feedback:

读写检测单元42,用于在预设时间间隔内接收到所述响应数据包时,判断读取以及写入是否正常;The read/write detecting unit 42 is configured to determine whether the reading and writing are normal when the response packet is received within a preset time interval;

判定单元43,用于在读取以及写入正常时,判定运行正常,在读取以及写入异常时,判定运行故障,以及在预设时间间隔内未接收到所述响应数据包时,判定链路连接故障。The determining unit 43 is configured to determine that the operation is normal when the reading and the writing are normal, determine the operation failure when the reading and the writing are abnormal, and determine that the response data packet is not received within the preset time interval. The link connection is faulty.

在本实施例中,在检测模块40检测到所述主存储节点出现故障,无法继续为客户端提供数据读取或写入服务时,主存储节点会选择一处于正常工作状态的从存储节点,将被选取的从存储节点切换为主存储节点,接管客户端的数据读取或写入进程。In this embodiment, when the detecting module 40 detects that the primary storage node is faulty and cannot continue to provide data reading or writing services to the client, the primary storage node selects a secondary storage node in a normal working state. The selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.

切换模块60,用于检测到链路连接故障或运行故障时,在处于正常工作状态的从存储节点中选取主存储节点,将选取的所述从存储节点的地址标记为主存储节点地址;The switching module 60 is configured to: when detecting a link connection failure or an operation failure, select a primary storage node from the storage node in a normal working state, and mark the selected address of the secondary storage node as a primary storage node address;

更新模块70,用于将标记的所述主节点地址发送至控制节点以及选取的所述从存储节点,其中,所述控制节点采用接收到的所述主节点地址更新保存的所述主节点地址,且选取的所述存储节点接收到所述主节点地址时,将工作状态切换为主存储节点状态。An update module 70, configured to send the marked primary node address to the control node and the selected secondary storage node, where the control node updates the saved primary node address by using the received primary node address update And the selected storage node switches the working state to the state of the primary storage node when receiving the address of the primary node.

在需要切换存储节点时,切换模块60会选取一处于正常工作状态的从存储节点,将所述被选取的从服务节点的地址标记为新的主存储节点的地址,更新模块70将其地址发送至控制节点,控制节点将会把之后的客户端请求转发至新的主存储节点,所述被选取的从存储节点会切换为主存储节点工作状态,新的主存储节点可以读取其缓存区内的数据读取或写入进程信息,直接接管相应的进程,不会使客户端的数据操作进程中断。When the storage node needs to be switched, the switching module 60 selects a slave storage node in a normal working state, marks the selected address of the slave node as the address of the new primary storage node, and the update module 70 sends the address. To the control node, the control node will forward the subsequent client request to the new primary storage node, the selected secondary storage node will switch to the working state of the primary storage node, and the new primary storage node can read its cache area. The internal data reads or writes the process information, directly takes over the corresponding process, and does not interrupt the data operation process of the client.

可以理解的是,从存储节点的性能可能大致相同,也可能性能差异较大,所以在主存储节点进行切换时,若所有从存储节点性能相差无几,则切换模块60可以随机选取一处于正常工作状态的从存储节点作为主存储节点,若所有从存储节点的性能差异较大,则可按照从存储节点的性能高低顺序选择一处于正常工作状态的性能较高的节点作为新的主存储节点。It can be understood that the performance of the slave storage node may be substantially the same, or the performance difference may be large. Therefore, when the primary storage node performs the handover, if all the performances of the slave storage nodes are similar, the switching module 60 may randomly select one to be in normal operation. The slave storage node of the state is used as the primary storage node. If the performance of all the slave storage nodes is different, the node with higher performance in the normal working state may be selected as the new primary storage node according to the performance level of the storage node.

本实施例提出的集群文件系统的数据处理装置,定时检测主存储节点的链路连接状态和服务程序运行状态,及时发现主存储节点的故障,进而切换主存储节点,为客户端提供无间断的数据读写服务,确保集群文件系统数据处理服务的稳定性。The data processing device of the cluster file system in this embodiment periodically detects the link connection state of the primary storage node and the service program running state, discovers the fault of the primary storage node in time, and then switches the primary storage node to provide uninterrupted support for the client. Data read and write services ensure the stability of cluster file system data processing services.

进一步地,为提高集群文件系统的安全性,基于第一或第二实施例本发明还提出集群文件系统的数据处理装置的第三实施例,参照图5,所述集群文件系统的数据处理装置还包括:Further, in order to improve the security of the cluster file system, the present invention further provides a third embodiment of the data processing apparatus of the cluster file system based on the first or second embodiment. Referring to FIG. 5, the data processing apparatus of the cluster file system Also includes:

确定模块80,用于在接收到文件读取或写入请求时,确定所述读取或写入请求对应的数据是否需要验证;a determining module 80, configured to determine, when a file read or write request is received, whether data corresponding to the read or write request needs to be verified;

在本实施例中,主存储节点接收到文件读取或写入请求时,会检测所述被请求访问的文件的访问级别,若所述文件为共享文件,即可以被所有用户访问时,则不需要对用户的身份权限进行验证,若所述文件为私有文件,即所述文件仅仅可以被有许可权限的用户访问,则需要对用户的身份权限进行验证,确定所述文件是否可被发出访问请求的客户端访问。In this embodiment, when the primary storage node receives the file read or write request, it detects the access level of the file requested to be accessed. If the file is a shared file, it can be accessed by all users. There is no need to verify the identity rights of the user. If the file is a private file, that is, the file can only be accessed by a user with permission, the identity permission of the user needs to be verified to determine whether the file can be issued. Client access to the request.

获取模块90,用于在所述读取或写入请求对应的数据需要验证时,获取用户输入的验证信息;The obtaining module 90 is configured to obtain verification information input by the user when the data corresponding to the read or write request needs to be verified;

在客户端请求访问的文件为私有文件,即需要验证时,主存储节点需要获取用户输入的信息,对用户的权限进行验证。此时,用户可能已进行登录操作,主存储节点只需要对控制节点转发的信息进行遍历,抓取带有用户名和密码关键字或句柄的数据,进而获取用户输入的信息,对用户的权限进行验证;若此时用户尚未登录,主存储节点会向控制节点发送消息,通知控制节点向客户端发送提醒或者控制客户端弹出登录界面,在用户输入验证信息后,由控制节点将所述用户输入的验证信息转发至主存储节点进行验证。When the file requested by the client is a private file, that is, when authentication is required, the primary storage node needs to obtain information input by the user and verify the authority of the user. At this point, the user may have logged in, and the primary storage node only needs to traverse the information forwarded by the control node, grab the data with the user name and password key or handle, and then obtain the information input by the user, and perform the user's authority. If the user has not logged in at this time, the primary storage node sends a message to the control node, notifying the control node to send a reminder to the client or controlling the client to pop up the login interface. After the user inputs the verification information, the control node inputs the user. The verification information is forwarded to the primary storage node for verification.

所述数据处理模块10,还用于在所述验证信息与预存的验证信息匹配时,根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据。The data processing module 10 is further configured to: when the verification information matches the pre-stored verification information, read data in the storage area according to the read request and the request information in the buffer area, or according to the The write request and the request information in the buffer area write data to its storage area.

本实施例提出的所述集群文件系统的数据处理装置,在接收到数据操作请求后,会获取所述数据的操作权限,若所述数据需要权限验证,则获取用户输入的验证信息,在用户输入的验证信息与预存的验证信息匹配时,才执行相应的数据操作,提高集群文件系统数据的安全性。The data processing apparatus of the cluster file system according to the embodiment obtains the operation authority of the data after receiving the data operation request, and obtains the verification information input by the user if the data requires the authority verification, in the user When the input verification information matches the pre-stored verification information, the corresponding data operation is performed to improve the security of the cluster file system data.

本发明进一步提供一种集群文件系统的数据处理方法。The invention further provides a data processing method for a cluster file system.

参照图6,所述集群文件系统的数据处理方法包括:Referring to FIG. 6, the data processing method of the cluster file system includes:

步骤S10,在存储节点接收到文件读取或写入请求时,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据;Step S10, when the storage node receives the file read or write request, the storage node reads the data in the storage area according to the read request and the request information in the buffer area, or according to the write The incoming request and the request information in the buffer area write data to its storage area;

步骤S20,所述存储节点将所述读取或写入请求中的请求信息存入其缓存区;Step S20, the storage node stores the request information in the read or write request into its buffer area;

存储节点可在初次启动时获取控制节点上的配置参数,进行初始化操作,在存储节点创建所述缓存区,可以理解的是,所述配置参数可由用户自行设置,也可在服务器制造时由厂家录入。The storage node may acquire the configuration parameters on the control node at the initial startup, perform an initialization operation, and create the cache area in the storage node. It can be understood that the configuration parameter may be set by the user, or may be set by the manufacturer during server manufacture. Enter.

在本实施例中,所述配置参数参照集群文件系统的数据处理第一实施例所述,在此不再赘述。In this embodiment, the configuration parameters are described in the first embodiment of the data processing of the cluster file system, and details are not described herein again.

在本实施例中,控制节点在系统启动初始化时,会向被设置为主存储节点的服务器发送消息,控制所述主存储节点启动程序,而其他存储节点不会启动服务程序,只能接收并存储主存储节点同步的读取或写入请求的请求信息,只有主存储节点在接收到数据读取或写入请求时,将所述读取或写入请求同步至集群文件系统中的所有从存储节点,从存储节点仅仅将接收到的所述读取或写入请求的请求信息存入其缓存区,而不进行其他任何操作。In this embodiment, when the system initiates initialization, the control node sends a message to the server that is set as the primary storage node, and controls the primary storage node to start the program, and the other storage nodes do not start the service program, and can only receive and A request information for storing a read or write request synchronized by the primary storage node, and only the primary storage node synchronizes the read or write request to all slaves in the cluster file system upon receiving a data read or write request The storage node stores the requested information of the read or write request received from the storage node into its buffer area without any other operations.

在主存储节点接收到客户端的数据读取或写入请求后,会将所述请求的请求信息:如客户端数据读取或写入的进程号、所述进程的I/O偏移量,同步至所有从存储节点。After the primary storage node receives the data read or write request from the client, the request information of the request: the process number of the client data read or write, the I/O offset of the process, Synchronize to all slave storage nodes.

步骤S30,所述存储节点将所述请求信息同步至集群文件系统中的从存储节点。Step S30, the storage node synchronizes the request information to a slave storage node in the cluster file system.

主存储节点在接收到数据读取或写入请求后,会响应所述请求,根据所述请求信息,为客户端提供数据读取或写入服务。After receiving the data read or write request, the primary storage node responds to the request and provides a data read or write service to the client according to the request information.

可以理解的是,主存储节点不仅仅会接收到数据读取或写入请求,也有可能会接收到其他类型的服务请求,如配置参数修改请求,此时直接响应所述请求,如对所述请求对应的参数进行相应的修改,但不会将服务请求存储至其缓存区内,且不会将服务请求同步至所有从存储节点。It can be understood that the primary storage node not only receives the data read or write request, but also may receive other types of service requests, such as configuration parameter modification requests, and directly respond to the request, as described above. The corresponding parameter is requested to be modified accordingly, but the service request is not stored in its cache area, and the service request is not synchronized to all the slave storage nodes.

参照图7,存储节点可在接收到服务请求时,确定服务请求的类型,根据服务器请求的类型进行相应的处理,即所述集群文件系统的数据处理方法还包括:Referring to FIG. 7, the storage node may determine the type of the service request when the service request is received, and perform corresponding processing according to the type of the server request, that is, the data processing method of the cluster file system further includes:

步骤S40,在所述存储节点接收服务请求时,检测所述服务请求的类型;Step S40: When the storage node receives the service request, detecting a type of the service request.

步骤S50,判断所述服务请求是否为读取或写入请求;Step S50, determining whether the service request is a read or write request;

若所述服务请求为读取或写入请求,则执行所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据的步骤;And if the service request is a read or write request, executing, by the storage node, reading data in a storage area according to the read request and request information in the buffer area, or according to the write request And a step of writing data in the storage area to the request information in the buffer area;

所述步骤S50之后,所述集群文件系统的数据处理方法还包括:After the step S50, the data processing method of the cluster file system further includes:

步骤S60,若所述服务请求不是读取或写入请求,则响应所述服务请求。Step S60, if the service request is not a read or write request, responding to the service request.

在主存储节点接收到服务请求后,会检测所述请求调用的服务类型,在所述请求调用数据读取或写入服务时,才将所述请求的请求信息发送至主存储节点网络接口,再由主存储节点的网络接口发送至从控制节点获取到的节点地址对应的从存储节点,并由从存储节点将接收到同步请求存入其缓存区内。After receiving the service request, the primary storage node detects the service type invoked by the request, and sends the requested request information to the primary storage node network interface when the request invokes the data read or write service. Then, the network interface of the primary storage node sends the slave storage node corresponding to the node address acquired from the control node, and the synchronization request received from the storage node is stored in its buffer area.

可以理解的是,在本实施例中,由于系统升级或维护等原因,所述主存储节点可能无法继续为客户端提供数据读取或写入服务时,主存储节点会选择一处于正常工作状态的从存储节点,将被选取的从存储节点切换为主存储节点,接管客户端的数据读取或写入进程。It can be understood that, in this embodiment, when the primary storage node may not be able to provide data reading or writing services to the client due to system upgrade or maintenance, etc., the primary storage node selects a normal working state. From the storage node, the selected storage node is switched from the storage node to the primary storage node, taking over the data read or write process of the client.

本实施例提出的集群文件系统的数据处理方法,主存储节点在接收到数据操作请求后,将所述请求的信息同步至所有从存储节点,在需要切换存储节点时,任一处于正常工作状态的从存储节点均可接管客户端的数据操作进程,为客户端提供无间断的数据操作服务,提高集群文件系统的可用性。The data processing method of the cluster file system in this embodiment, after receiving the data operation request, the primary storage node synchronizes the requested information to all the slave storage nodes, and when the storage node needs to be switched, any one is in a normal working state. The slave storage node can take over the client's data operation process, provide the client with uninterrupted data operation services, and improve the availability of the cluster file system.

进一步地,在主存储节点为客户端提供服务时,有可能发生故障,则主存储节点需定时检测其链路连接状态和服务程序运行状态,以便及时发现故障,并作出相应处理,则基于第一实施例提出本发明集群文件系统的数据处理方法第二实施例,在本实施例中,参照图8,所述集群文件系统的数据处理方法还包括:Further, when the primary storage node provides services to the client, a failure may occur, and the primary storage node needs to periodically detect the link connection status and the service program running status, so as to timely find the fault and perform corresponding processing, based on the first An embodiment provides a second embodiment of the data processing method of the cluster file system of the present invention. In this embodiment, referring to FIG. 8, the data processing method of the cluster file system further includes:

步骤S70,在所述存储节点为主存储节点时,所述存储节点检测其链路连接状态和所述存储节点的运行状态;Step S70: When the storage node is a primary storage node, the storage node detects a link connection state thereof and an operation state of the storage node.

参照图9,所述存储节点检测其链路连接状态和所述存储节点的运行状态的步骤包括:Referring to FIG. 9, the step of detecting, by the storage node, its link connection state and the running state of the storage node includes:

步骤S71,所述存储节点定时向所述控制节点发送第一检测数据包;Step S71, the storage node periodically sends a first detection data packet to the control node;

步骤S72,接收所述控制节点基于所述检测数据包反馈的第二响应数据包:Step S72: Receive a second response data packet that is sent back by the control node based on the detection data packet:

步骤S73,在预设时间间隔内接收到所述响应数据包时,判断所述存储节点的读取以及写入是否正常;Step S73, when receiving the response data packet within a preset time interval, determining whether the reading and writing of the storage node are normal;

步骤S74,在所述存储节点的读取以及写入正常时,判定所述存储节点运行正常;Step S74, when the reading and writing of the storage node are normal, determining that the storage node is operating normally;

步骤S75,在所述存储节点的读取以及写入异常时,判定所述存储节点运行故障;Step S75, determining that the storage node is faulty when the storage node reads and writes an abnormality;

步骤S76,在预设时间间隔内未接收到所述响应数据包时,判定所述存储节点的链路连接故障。Step S76: When the response data packet is not received within the preset time interval, determine that the link connection of the storage node is faulty.

在本实施例中,主存储节点检测到其出现故障,无法继续为客户端提供数据读取或写入服务时,主存储节点会选择一处于正常工作状态的从存储节点,将被选取的从存储节点切换为主存储节点,接管客户端的数据读取或写入进程。In this embodiment, when the primary storage node detects that it is faulty and cannot continue to provide data reading or writing services to the client, the primary storage node selects a secondary storage node that is in a normal working state, and the selected slave will be selected. The storage node switches to the primary storage node and takes over the data read or write process of the client.

步骤S80,在所述获取存储节点实时检测到其链路连接故障或所述存储节点运行故障时,所述存储节点在处于正常工作状态的从存储节点中选取主存储节点,将选取的所述从存储节点的地址标记为主存储节点地址;Step S80, when the acquiring storage node detects its link connection failure or the storage node running fault in real time, the storage node selects a primary storage node from the storage node in a normal working state, and selects the selected storage node. The address from the storage node is marked as the primary storage node address;

步骤S90,将标记的所述主节点地址发送至控制节点以及选取的所述从存储节点,其中,所述控制节点采用接收到的所述主节点地址更新保存的所述主节点地址,且选取的所述存储节点接收到所述主节点地址时,将工作状态切换为主存储节点状态。Step S90, sending the marked primary node address to the control node and the selected secondary storage node, wherein the control node updates the saved primary node address by using the received primary node address, and selects When the storage node receives the address of the primary node, the working state is switched to the state of the primary storage node.

在需要切换存储节点时,选取一处于正常工作状态的从存储节点,将所述被选取的从服务节点的地址标记为新的主存储节点的地址,将其地址发送至控制节点,控制节点将会把之后的客户端请求转发至新的主存储节点,所述被选取的从存储节点会切换为主存储节点工作状态,新的主存储节点可以读取其缓存区内的数据读取或写入进程信息,直接接管相应的进程,不会使客户端的数据操作进程中断。When the storage node needs to be switched, a slave storage node in a normal working state is selected, the selected address of the slave service node is marked as the address of the new primary storage node, and the address is sent to the control node, and the control node will The subsequent client request will be forwarded to the new primary storage node, and the selected secondary storage node will switch to the working state of the primary storage node, and the new primary storage node can read the data in the buffer to read or write. Into the process information, directly take over the corresponding process, will not interrupt the client's data operation process.

可以理解的是,从存储节点的性能可能大致相同,也可能性能差异较大,所以在主存储节点进行切换时,若所有从存储节点性能相差无几,则可以随机选取一处于正常工作状态的从存储节点作为主存储节点,若所有从存储节点的性能差异较大,则可按照从存储节点的性能高低顺序选择一处于正常工作状态的性能较高的节点作为新的主存储节点。It can be understood that the performance of the slave storage node may be substantially the same, or the performance difference may be large. Therefore, when the primary storage node performs the handover, if the performance of all the slave storage nodes is almost the same, the slaves in the normal working state may be randomly selected. As the primary storage node, if the performance of all the secondary storage nodes is different, the node with higher performance in the normal working state can be selected as the new primary storage node according to the performance of the storage node.

本实施例提出的集群文件系统的数据处理方法,定时检测主存储节点的链路连接状态和服务程序运行状态,及时发现主存储节点的故障,进而切换主存储节点,为客户端提供无间断的数据读写服务,确保集群文件系统数据处理服务的稳定性。The data processing method of the cluster file system in this embodiment periodically detects the link connection state of the primary storage node and the service program running state, discovers the fault of the primary storage node in time, and then switches the primary storage node to provide uninterrupted support for the client. Data read and write services ensure the stability of cluster file system data processing services.

进一步地,为提高集群文件系统的安全性性,基于第一或第二实施例本发明还提出集群文件系统的数据处理方法的第三实施例,参照图10,所述步骤S10之前,所述集群文件系统的数据处理方法还包括步骤:Further, in order to improve the security of the cluster file system, the present invention further provides a third embodiment of the data processing method of the cluster file system based on the first or second embodiment. Referring to FIG. 10, before the step S10, the The data processing method of the cluster file system further includes the steps of:

步骤S100,所述存储节点确定所述读取或写入请求对应的数据是否需要验证;Step S100, the storage node determines whether data corresponding to the read or write request needs to be verified;

在本实施例中,主存储节点接收到文件读取或写入请求时,会检测所述被请求访问的文件的访问级别,若所述文件为共享文件,即可以被所有用户访问时,则不需要对用户的身份权限进行验证,若所述文件为私有文件,即所述文件仅仅可以被有许可权限的用户访问,则需要对用户的身份权限进行验证,确定所述文件是否可被发出访问请求的客户端访问。In this embodiment, when the primary storage node receives the file read or write request, it detects the access level of the file requested to be accessed. If the file is a shared file, it can be accessed by all users. There is no need to verify the identity rights of the user. If the file is a private file, that is, the file can only be accessed by a user with permission, the identity permission of the user needs to be verified to determine whether the file can be issued. Client access to the request.

步骤S110,在所述读取或写入请求对应的数据需要验证时,所述存储节点获取用户输入的验证信息;Step S110, when the data corresponding to the read or write request needs to be verified, the storage node acquires the verification information input by the user;

在客户端请求访问的文件为私有文件,即需要验证时,主存储节点需要获取用户输入的信息,对用户的权限进行验证。此时,用户可能已进行登录操作,主存储节点只需要对控制节点转发的信息进行遍历,抓取带有用户名和密码关键字或句柄的数据,进而获取用户输入的验证信息,对用户的权限进行验证;若此时用户尚未登录,主存储节点会向控制节点发送消息,通知控制节点向客户端发送权限验证提醒或者控制客户端弹出登录界面,在用户输入验证信息后,由控制节点将所述用户输入的验证信息转发至主存储节点进行验证。When the file requested by the client is a private file, that is, when authentication is required, the primary storage node needs to obtain information input by the user and verify the authority of the user. At this point, the user may have logged in, and the primary storage node only needs to traverse the information forwarded by the control node, grab the data with the user name and password key or handle, and then obtain the authentication information input by the user, and the authority to the user. If the user has not logged in yet, the primary storage node sends a message to the control node, notifying the control node to send a permission verification reminder to the client or controlling the client to pop up the login interface. After the user inputs the verification information, the control node will The verification information input by the user is forwarded to the primary storage node for verification.

步骤S120,判断所述验证信息与预存的验证信息是否匹配;Step S120, determining whether the verification information matches the pre-stored verification information;

在所述验证信息与预存的验证信息匹配时,执行所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据的步骤,即步骤S10。When the verification information matches the pre-stored verification information, executing the storage node to read data in the storage area according to the read request and the request information in the buffer area, or according to the write request and The step of writing the request information in the buffer area to the data area thereof, that is, step S10.

本实施例提出的集群文件系统的数据处理方法,在接收到数据操作请求后,会获取所述数据的操作权限,若所述数据需要权限验证,则获取用户输入的验证信息,在用户输入的验证信息与预存的验证信息匹配时,才执行相应的数据操作,提高集群文件系统数据的安全性。The data processing method of the cluster file system provided in this embodiment acquires the operation authority of the data after receiving the data operation request, and if the data requires the authority verification, the verification information input by the user is obtained, and the user inputs the When the verification information matches the pre-stored verification information, the corresponding data operation is performed to improve the security of the cluster file system data.

通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,云端服务器,空调器,或者网络设备等)执行本发明各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes instructions for causing a terminal device (which may be a mobile phone, a computer, a cloud server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

以上仅为本发明的优选实施例,并非因此限制本发明的专利范围,凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本发明的专利保护范围内。The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Claims (20)

一种集群文件系统的数据处理装置,其特征在于,所述集群文件系统的数据处理装置包括 : A data processing device for a cluster file system, wherein the data processing device of the cluster file system comprises: 数据处理模块,用于在存储节点接收到文件读取或写入请求时,根据所述读取请求以及所述缓存区中的请求信息读取存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向存储区中写入数据a data processing module, configured to: when the storage node receives the file read or write request, read the data in the storage area according to the read request and the request information in the buffer area, or according to the write request And requesting information in the buffer area to write data to the storage area 存储模块,用于将所述读取或写入请求中的请求信息存入缓存区;a storage module, configured to store the request information in the read or write request into a buffer area; 同步模块,用于将所述请求信息同步至集群文件系统中的从存储节点。And a synchronization module, configured to synchronize the request information to a slave storage node in the cluster file system. 如权利要求 1 所述的集群文件系统的数据处理装置,其特征在于,所述集群文件系统的数据处理装置还包括:Claims 1 The data processing device of the cluster file system, further comprising: 检测模块,用于接收服务请求时,检测所述服务请求的类型,判断所述服务请求是否为读取或写入请求;The detecting module is configured to: when receiving the service request, detect a type of the service request, and determine whether the service request is a read or write request; 响应模块,用于若所述服务请求不是读取或写入请求,则响应所述服务请求;a response module, configured to respond to the service request if the service request is not a read or write request; 所述数据处理模块,还用于若所述服务请求是读取或写入请求,则根据所述读取请求以及所述缓存区中的请求信息读取存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向存储区中写入数据。The data processing module is further configured to: if the service request is a read or write request, read data in the storage area according to the read request and the request information in the buffer area, or according to the The write request and the request information in the buffer area write data to the storage area. 如权利要求 1 所述的集群文件系统的数据处理装置,其特征在于,所述集群文件系统的数据处理装置还包括:Claims 1 The data processing device of the cluster file system, further comprising: 确定模块,用于在接收到文件读取或写入请求时,确定所述读取或写入请求对应的数据是否需要验证;a determining module, configured to determine, when a file read or write request is received, whether data corresponding to the read or write request needs to be verified; 获取模块,用于在所述读取或写入请求对应的数据需要验证时,获取用户输入的验证信息;An obtaining module, configured to obtain verification information input by the user when the data corresponding to the read or write request needs to be verified; 所述数据处理模块,还用于在所述验证信息与预存的验证信息匹配时,根据所述读取请求以及所述缓存区中的请求信息读取存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向存储区中写入数据。The data processing module is further configured to: when the verification information matches the pre-stored verification information, read data in the storage area according to the read request and the request information in the buffer area, or according to the writing The incoming request and the request information in the buffer area write data to the storage area. 如权利要求3所述的集群文件系统的数据处理装置,其特征在于,所述确定模块还用于,检测所述读取或写入请求对应的文件的访问级别,以确定所述读取或写入请求对应的数据是否需要验证,其中,在所述文件为共享文件时,确定所述读取或写入请求对应的数据不需要验证,在所述文件为私有文件时,确定所述读取或写入请求对应的数据需要验证。The data processing apparatus of the cluster file system according to claim 3, wherein the determining module is further configured to detect an access level of the file corresponding to the read or write request to determine the read or Whether the data corresponding to the write request needs to be verified, wherein when the file is a shared file, determining that the data corresponding to the read or write request does not need to be verified, and when the file is a private file, determining the read The data corresponding to the fetch or write request needs to be verified. 如权利要求2所述的集群文件系统的数据处理装置,其特征在于,所述集群文件系统的数据处理装置还包括:The data processing device of the cluster file system according to claim 2, wherein the data processing device of the cluster file system further comprises: 所述检测模块,还用于检测链路连接状态和运行状态;The detecting module is further configured to detect a link connection state and an operating state; 切换模块,用于在检测到链路连接故障或运行故障时,在处于正常工作状态的从存储节点中选取主存储节点,将选取的所述从存储节点的地址标记为主存储节点地址;a switching module, configured to select a primary storage node from a storage node in a normal working state when detecting a link connection failure or an operation failure, and mark the selected address of the secondary storage node as a primary storage node address; 更新模块,用于将标记的所述主存储节点地址发送至控制节点以及选取的所述从存储节点,其中,所述控制节点采用接收到的所述主存储节点地址更新保存的所述主存储节点地址,且选取的所述存储节点接收到所述主存储节点地址时,将工作状态切换为主存储节点状态。And an update module, configured to send the marked primary storage node address to the control node and the selected secondary storage node, wherein the control node updates the saved primary storage by using the received primary storage node address The node address, and when the selected storage node receives the address of the primary storage node, the working state is switched to the state of the primary storage node. 如权利要求5所述的集群文件系统的数据处理装置,其特征在于,所述检测模块包括:The data processing device of the cluster file system according to claim 5, wherein the detecting module comprises: 链路检测单元,用于定时向所述控制节点发送第一检测数据包以及接收所述控制节点基于所述检测数据包反馈的第二响应数据包;a link detecting unit, configured to periodically send a first detection data packet to the control node, and receive a second response data packet that is sent back by the control node based on the detection data packet; 读写检测单元,用于在预设时间间隔内接收到所述响应数据包时,判断读取以及写入是否正常;a read/write detection unit, configured to determine whether the reading and writing are normal when the response data packet is received within a preset time interval; 判定单元,用于在读取以及写入正常时,判定运行正常,在读取以及写入异常时,判定运行故障,以及在预设时间间隔内未接收到所述响应数据包时,判定链路连接故障。a determining unit, configured to determine that the operation is normal when reading and writing are normal, determining an operation failure when reading and writing an abnormality, and determining the chain when the response data packet is not received within a preset time interval The road connection is faulty. 如权利要求1所述的集群文件系统的数据处理装置,其特征在于,所述存储模块还用于在所述存储节点初次启动时,获取控制节点上的配置参数并进行初始化操作,创建缓存区。The data processing device of the cluster file system according to claim 1, wherein the storage module is further configured to: when the storage node is first started, acquire configuration parameters on the control node and perform an initialization operation to create a buffer area. . 一种集群文件系统的数据处理方法,其特征在于,所述集群文件系统的数据处理方法包括:A data processing method for a cluster file system, the data processing method of the cluster file system includes: 在所述存储节点接收服务请求时,检测所述服务请求的类型;Detecting a type of the service request when the storage node receives a service request; 判断所述服务请求是否为读取或写入请求;Determining whether the service request is a read or write request; 若所述服务请求为读取或写入请求,则所述存储节点确定所述读取或写入请求对应的数据是否需要验证;If the service request is a read or write request, the storage node determines whether data corresponding to the read or write request requires verification; 在所述读取或写入请求对应的数据需要验证时,所述存储节点获取用户输入的验证信息;When the data corresponding to the read or write request needs to be verified, the storage node acquires the verification information input by the user; 在所述验证信息与预存的验证信息匹配时,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据;When the verification information matches the pre-stored verification information, the storage node reads data in its storage area according to the read request and the request information in the buffer area, or according to the write request and the The request information in the buffer area writes data to its storage area; 所述存储节点将所述读取或写入请求中的请求信息存入其缓存区;The storage node stores the request information in the read or write request into its buffer area; 所述存储节点将所述请求信息同步至集群文件系统中的从存储节点;The storage node synchronizes the request information to a slave storage node in the cluster file system; 若所述服务请求不是读取或写入请求,则响应所述服务请求。If the service request is not a read or write request, then responding to the service request. 如权利要求8所述的集群文件系统的数据处理方法,其特征在于,所述集群文件系统的数据处理方法还包括:The data processing method of the cluster file system according to claim 8, wherein the data processing method of the cluster file system further comprises: 在所述存储节点为主存储节点时,所述存储节点检测其链路连接状态和所述存储节点的运行状态;When the storage node is a primary storage node, the storage node detects a link connection state thereof and an operation state of the storage node; 在所述获取存储节点实时检测到其链路连接故障或所述存储节点运行故障时,所述存储节点在处于正常工作状态的从存储节点中选取主存储节点,将选取的所述从存储节点的地址标记为主存储节点地址;When the acquiring storage node detects its link connection failure or the storage node running fault in real time, the storage node selects a primary storage node from the storage node in a normal working state, and selects the selected secondary storage node. Address is marked as the primary storage node address; 将标记的所述主存储节点地址发送至控制节点以及选取的所述从存储节点,其中,所述控制节点采用接收到的所述主存储节点地址更新保存的所述主存储节点地址,且选取的所述存储节点接收到所述主存储节点地址时,将工作状态切换为主存储节点状态。Sending the marked primary storage node address to the control node and the selected secondary storage node, wherein the control node updates the saved primary storage node address by using the received primary storage node address, and selects When the storage node receives the address of the primary storage node, the working state is switched to the state of the primary storage node. 如权利要求9所述的集群文件系统的数据处理的方法,其特征在于,所述存储节点检测其链路连接状态和所述存储节点的运行状态的步骤包括:The method of data processing of a cluster file system according to claim 9, wherein the step of detecting, by the storage node, the link connection status and the running status of the storage node comprises: 所述存储节点定时向所述控制节点发送第一检测数据包;The storage node periodically sends a first detection data packet to the control node; 接收所述控制节点基于所述检测数据包反馈的第二响应数据包:Receiving, by the control node, a second response data packet based on the detection data packet feedback: 在预设时间间隔内接收到所述响应数据包时,判断所述存储节点的读取以及写入是否正常;When receiving the response data packet within a preset time interval, determining whether the reading and writing of the storage node are normal; 在所述存储节点的读取以及写入正常时,判定所述存储节点运行正常,在所述存储节点的读取以及写入异常时,判定所述存储节点运行故障;When the reading and writing of the storage node are normal, determining that the storage node is operating normally, and determining that the storage node is faulty when the storage node reads and writes abnormally; 在预设时间间隔内未接收到所述响应数据包时,判定所述存储节点的链路连接故障。When the response packet is not received within the preset time interval, it is determined that the link connection of the storage node is faulty. 如权利要求8所述的集群文件系统的数据处理的方法,其特征在于,所述集群文件系统的数据处理的方法还包括:The method for processing data of the cluster file system according to claim 8, wherein the method for processing data of the cluster file system further comprises: 在所述存储节点初次启动时,所述存储节点获取控制节点上的配置参数并进行初始化操作,创建缓存区。When the storage node is initially started, the storage node acquires configuration parameters on the control node and performs an initialization operation to create a buffer area. 如权利要求8所述的集群文件系统的数据处理的方法,其特征在于,所述存储节点确定所述读取或写入请求对应的数据是否需要验证的步骤包括:The method of data processing of the cluster file system according to claim 8, wherein the step of determining, by the storage node, whether the data corresponding to the read or write request requires verification comprises: 检测所述读取或写入请求对应的文件的访问级别,以判断所述读取或写入请求对应的数据是否需要验证,其中,在所述文件为共享文件时,判定所述读取或写入请求对应的数据不需要验证,在所述文件为私有文件时,判定所述读取或写入请求对应的数据需要验证。Detecting an access level of the file corresponding to the read or write request, to determine whether data corresponding to the read or write request needs to be verified, wherein when the file is a shared file, determining the read or The data corresponding to the write request does not need to be verified. When the file is a private file, it is determined that the data corresponding to the read or write request needs to be verified. 如权利要求8所述的集群文件系统的数据处理的方法,其特征在于,所述存储节点获取用户输入的验证信息的步骤包括:The method for processing data of a cluster file system according to claim 8, wherein the step of the storage node acquiring the verification information input by the user comprises: 在用户已经进行登录操作时,所述存储节点对控制节点转发的信息进行遍历,抓取带有用户名和密码关键字或句柄的数据以获取用户输入的验证信息;When the user has performed the login operation, the storage node traverses the information forwarded by the control node, and grabs the data with the user name and the password key or the handle to obtain the verification information input by the user; 在用户未进行登录操作时,所述存储节点向所述控制节点发送消息,并接收所述控制节点反馈的验证信息,其中,所述控制节点在接收到所述消息时,向客户端发送提醒或者控制客户端弹出登录界面,并将用户输入的所述验证信息发送至所述存储节点。When the user does not perform the login operation, the storage node sends a message to the control node, and receives the verification information fed back by the control node, where the control node sends a reminder to the client when receiving the message. Or controlling the client to pop up the login interface, and sending the verification information input by the user to the storage node. 一种集群文件系统的数据处理方法,其特征在于,所述集群文件系统的数据处理方法包括:A data processing method for a cluster file system, the data processing method of the cluster file system includes: 在存储节点接收到文件读取或写入请求时,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据;When the storage node receives the file read or write request, the storage node reads data in its storage area according to the read request and the request information in the cache area, or according to the write request and The request information in the buffer area writes data to its storage area; 所述存储节点将所述读取或写入请求中的请求信息存入其缓存区;The storage node stores the request information in the read or write request into its buffer area; 所述存储节点将所述请求信息同步至集群文件系统中的从存储节点。The storage node synchronizes the request information to a slave storage node in the cluster file system. 如权利要求14所述的集群文件系统的数据处理方法,其特征在于,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据的步骤之前还包括:The data processing method of the cluster file system according to claim 14, wherein the storage node reads data in the storage area according to the read request and the request information in the buffer area, or according to the Before the step of writing the request and the request information in the buffer area to the data in the storage area, the method further includes: 在所述存储节点接收服务请求时,检测所述服务请求的类型;Detecting a type of the service request when the storage node receives a service request; 判断所述服务请求是否为读取或写入请求;Determining whether the service request is a read or write request; 若所述服务请求为读取或写入请求,则执行所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据的步骤;And if the service request is a read or write request, executing, by the storage node, reading data in a storage area according to the read request and request information in the buffer area, or according to the write request And a step of writing data in the storage area to the request information in the buffer area; 所述判断所述服务请求是否为读取或写入请求的步骤之后,所述集群文件系统的数据处理方法还包括:After the step of determining whether the service request is a read or write request, the data processing method of the cluster file system further includes: 若所述服务请求不是读取或写入请求,则响应所述服务请求。If the service request is not a read or write request, then responding to the service request. 如权利要求14所述的集群文件系统的数据处理的方法,其特征在于,所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据的步骤之前,所述集群文件系统的数据处理方法还包括步骤:The method of data processing of a cluster file system according to claim 14, wherein the storage node reads data in a storage area thereof according to the read request and request information in the buffer area, or according to Before the step of writing the request and the request information in the buffer area to the data in the storage area, the data processing method of the cluster file system further includes the following steps: 在存储节点接收到文件读取或写入请求时,所述存储节点确定所述读取或写入请求对应的数据是否需要验证;When the storage node receives the file read or write request, the storage node determines whether the data corresponding to the read or write request needs to be verified; 在所述读取或写入请求对应的数据需要验证时,所述存储节点获取用户输入的验证信息;When the data corresponding to the read or write request needs to be verified, the storage node acquires the verification information input by the user; 在所述验证信息与预存的验证信息匹配时,执行所述存储节点根据所述读取请求以及所述缓存区中的请求信息读取其存储区中的数据,或者根据所述写入请求以及所述缓存区中的请求信息向其存储区中写入数据的步骤。When the verification information matches the pre-stored verification information, executing the storage node to read data in the storage area according to the read request and the request information in the buffer area, or according to the write request and The step of requesting information in the buffer area to write data into its storage area. 如权利要求16所述的集群文件系统的数据处理的方法,其特征在于,所述存储节点确定所述读取或写入请求对应的数据是否需要验证的步骤包括:The method for processing data of a cluster file system according to claim 16, wherein the step of determining, by the storage node, whether the data corresponding to the read or write request requires verification comprises: 检测所述读取或写入请求对应的文件的访问级别,以判断所述读取或写入请求对应的数据是否需要验证,其中,在所述文件为共享文件时,判定所述读取或写入请求对应的数据不需要验证,在所述文件为私有文件时,判定所述读取或写入请求对应的数据需要验证。Detecting an access level of the file corresponding to the read or write request, to determine whether data corresponding to the read or write request needs to be verified, wherein when the file is a shared file, determining the read or The data corresponding to the write request does not need to be verified. When the file is a private file, it is determined that the data corresponding to the read or write request needs to be verified. 如权利要求14所述的集群文件系统的数据处理方法,其特征在于,所述集群文件系统的数据处理方法还包括:The data processing method of the cluster file system according to claim 14, wherein the data processing method of the cluster file system further comprises: 在所述存储节点为主存储节点时,所述存储节点检测其链路连接状态和所述存储节点的运行状态;When the storage node is a primary storage node, the storage node detects a link connection state thereof and an operation state of the storage node; 在所述获取存储节点实时检测到其链路连接故障或所述存储节点运行故障时,所述存储节点在处于正常工作状态的从存储节点中选取主存储节点,将选取的所述从存储节点的地址标记为主存储节点地址;When the acquiring storage node detects its link connection failure or the storage node running fault in real time, the storage node selects a primary storage node from the storage node in a normal working state, and selects the selected secondary storage node. Address is marked as the primary storage node address; 将标记的所述主存储节点地址发送至控制节点以及选取的所述从存储节点,其中,所述控制节点采用接收到的所述主存储节点地址更新保存的所述主存储节点地址,且选取的所述存储节点接收到所述主存储节点地址时,将工作状态切换为主存储节点状态。Sending the marked primary storage node address to the control node and the selected secondary storage node, wherein the control node updates the saved primary storage node address by using the received primary storage node address, and selects When the storage node receives the address of the primary storage node, the working state is switched to the state of the primary storage node. 如权利要求18所述的集群文件系统的数据处理的方法,其特征在于,所述存储节点检测其链路连接状态和所述存储节点的运行状态的步骤包括:The method for processing data of a cluster file system according to claim 18, wherein the step of detecting, by the storage node, the link connection status and the running status of the storage node comprises: 所述存储节点定时向所述控制节点发送第一检测数据包;The storage node periodically sends a first detection data packet to the control node; 接收所述控制节点基于所述检测数据包反馈的第二响应数据包:Receiving, by the control node, a second response data packet based on the detection data packet feedback: 在预设时间间隔内接收到所述响应数据包时,判断所述存储节点的读取以及写入是否正常;When receiving the response data packet within a preset time interval, determining whether the reading and writing of the storage node are normal; 在所述存储节点的读取以及写入正常时,判定所述存储节点运行正常,在所述存储节点的读取以及写入异常时,判定所述存储节点运行故障;When the reading and writing of the storage node are normal, determining that the storage node is operating normally, and determining that the storage node is faulty when the storage node reads and writes abnormally; 在预设时间间隔内未接收到所述响应数据包时,判定所述存储节点的链路连接故障。When the response packet is not received within the preset time interval, it is determined that the link connection of the storage node is faulty. 如权利要求14所述的集群文件系统的数据处理的方法,其特征在于,所述集群文件系统的数据处理的方法还包括:The method of data processing of the cluster file system according to claim 14, wherein the method for processing data of the cluster file system further comprises: 在所述存储节点初次启动时,所述存储节点获取控制节点上的配置参数并进行初始化操作,创建缓存区。When the storage node is initially started, the storage node acquires configuration parameters on the control node and performs an initialization operation to create a buffer area.
PCT/CN2016/105219 2015-11-26 2016-11-09 Data processing method and apparatus for cluster file system Ceased WO2017088664A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510847053.1 2015-11-26
CN201510847053.1A CN105511805B (en) 2015-11-26 2015-11-26 The data processing method and device of cluster file system

Publications (1)

Publication Number Publication Date
WO2017088664A1 true WO2017088664A1 (en) 2017-06-01

Family

ID=55719825

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/105219 Ceased WO2017088664A1 (en) 2015-11-26 2016-11-09 Data processing method and apparatus for cluster file system

Country Status (2)

Country Link
CN (1) CN105511805B (en)
WO (1) WO2017088664A1 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704201A (en) * 2017-09-11 2018-02-16 厦门集微科技有限公司 Data storage handling method and device
CN109508317A (en) * 2018-10-31 2019-03-22 武汉光谷联众大数据技术有限责任公司 A kind of Large Volume Data and service management system
CN109587221A (en) * 2018-11-09 2019-04-05 平安科技(深圳)有限公司 Large data sets group managing means, device, storage medium and computer equipment
CN110674095A (en) * 2019-09-27 2020-01-10 浪潮电子信息产业股份有限公司 A CTDB cluster expansion method, device, device and readable storage medium
CN110716692A (en) * 2018-07-13 2020-01-21 浙江宇视科技有限公司 Reading performance improving method and device, storage node and data reading method
CN111314129A (en) * 2020-02-13 2020-06-19 上海凯岸信息科技有限公司 High-availability architecture based on file type storage service
CN111506459A (en) * 2020-04-22 2020-08-07 深圳前海微众银行股份有限公司 Data import method, device, device and computer storage medium
CN112131229A (en) * 2020-09-29 2020-12-25 中国银行股份有限公司 Block chain-based distributed data access method and device and storage node
CN112395165A (en) * 2020-11-27 2021-02-23 中电科技(北京)有限公司 Method and equipment for acquiring resource utilization rate of server and server
CN112468330A (en) * 2020-11-13 2021-03-09 苏州浪潮智能科技有限公司 Method, system, equipment and medium for setting fault node
CN112671905A (en) * 2020-12-23 2021-04-16 广州三七互娱科技有限公司 Service scheduling method, device and system
CN112737962A (en) * 2020-12-24 2021-04-30 平安科技(深圳)有限公司 Storage service request processing method and device, computer equipment and storage medium
CN112988905A (en) * 2021-04-27 2021-06-18 北京沃丰时代数据科技有限公司 Node memory synchronization method and device for cluster deployment
CN113590040A (en) * 2021-07-29 2021-11-02 郑州阿帕斯数云信息科技有限公司 Data processing method, device, equipment and storage medium
CN114189547A (en) * 2022-02-14 2022-03-15 北京安盟信息技术股份有限公司 SSL tunnel fast switching method under cluster
WO2023109381A1 (en) * 2021-12-16 2023-06-22 中移(苏州)软件技术有限公司 Information processing method and apparatus, and storage medium

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105511805B (en) * 2015-11-26 2019-03-19 深圳市中博科创信息技术有限公司 The data processing method and device of cluster file system
CN106254103B (en) * 2016-07-28 2019-08-16 北京国电通网络技术有限公司 A method and device for dynamic configuration of RTMP cluster system
CN106598762B (en) * 2016-12-29 2020-04-17 上海理想信息产业(集团)有限公司 Message synchronization method
CN108512753B (en) 2017-02-28 2020-09-29 华为技术有限公司 Method and device for transmitting messages in cluster file system
CN107168649B (en) * 2017-05-05 2019-12-17 南京城市职业学院 method and device for data distribution in distributed storage system
CN107707620B (en) * 2017-08-30 2020-09-11 华为技术有限公司 Method and device for processing IO (input/output) request
CN109543204B (en) * 2017-09-22 2022-09-13 南京理工大学 Electric heating integrated analysis method for semiconductor device under human body electrostatic action
CN108023772B (en) * 2017-12-07 2021-02-26 海能达通信股份有限公司 Abnormal node repairing method, device and related equipment
CN110099084B (en) * 2018-01-31 2021-06-15 北京易真学思教育科技有限公司 Method, system and computer readable medium for ensuring storage service availability
CN108829720B (en) * 2018-05-07 2022-01-14 麒麟合盛网络技术股份有限公司 Data processing method and device
CN109213507A (en) * 2018-08-27 2019-01-15 郑州云海信息技术有限公司 A kind of upgrade method and server
CN110474981A (en) * 2019-08-13 2019-11-19 中科天御(苏州)科技有限公司 A kind of software definition dynamic security storage method and device
CN110868323B (en) * 2019-11-15 2022-07-22 浪潮电子信息产业股份有限公司 Bandwidth control method, device, equipment and medium
CN111371865B (en) * 2020-02-26 2023-02-24 上海达梦数据库有限公司 Client connection relation adjusting method, system and node
CN113051349A (en) * 2021-04-02 2021-06-29 广东美电贝尔科技集团股份有限公司 Data synchronization method for duty system
CN113259092A (en) * 2021-04-04 2021-08-13 余绍祥 Document distributed encryption system
CN116644039B (en) * 2023-05-25 2023-12-19 安徽继远软件有限公司 Automatic acquisition and analysis method for online capacity operation log based on big data
CN119652911B (en) * 2024-12-05 2025-10-31 天翼云科技有限公司 System operation method, apparatus, device, storage medium, and program product

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110055494A1 (en) * 2009-08-25 2011-03-03 Yahoo! Inc. Method for distributed direct object access storage
CN102035862A (en) * 2009-09-30 2011-04-27 国际商业机器公司 Configuration node fault transfer method and system in SVC cluster
US20130073717A1 (en) * 2011-09-15 2013-03-21 International Business Machines Corporation Optimizing clustered network attached storage (nas) usage
CN105511805A (en) * 2015-11-26 2016-04-20 深圳市中博科创信息技术有限公司 Data processing method and device for cluster file system

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4643543B2 (en) * 2006-11-10 2011-03-02 株式会社東芝 Storage cluster system with cache consistency guarantee function
US8875146B2 (en) * 2011-08-01 2014-10-28 Honeywell International Inc. Systems and methods for bounding processing times on multiple processing units
CN103634269B (en) * 2012-08-21 2017-04-19 中国银联股份有限公司 A single sign-on system and a method
US9189510B2 (en) * 2013-02-26 2015-11-17 Facebook, Inc. System and method for implementing cache consistent regional clusters
CN103207841B (en) * 2013-03-06 2016-01-20 青岛海信传媒网络技术有限公司 Based on data read-write method and the device of key-value pair buffer memory
CN103207894A (en) * 2013-03-14 2013-07-17 深圳市知正科技有限公司 Multipath real-time video data storage system and cache control method thereof
CN104361030A (en) * 2014-10-24 2015-02-18 西安未来国际信息股份有限公司 Distributed cache architecture with task distribution function and cache method
CN104580432A (en) * 2014-12-23 2015-04-29 上海帝联信息科技股份有限公司 Memcached system, memory cache data provision method and device, memory cache data maintenance method and device as well as cluster maintenance method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110055494A1 (en) * 2009-08-25 2011-03-03 Yahoo! Inc. Method for distributed direct object access storage
CN102035862A (en) * 2009-09-30 2011-04-27 国际商业机器公司 Configuration node fault transfer method and system in SVC cluster
US20130073717A1 (en) * 2011-09-15 2013-03-21 International Business Machines Corporation Optimizing clustered network attached storage (nas) usage
CN105511805A (en) * 2015-11-26 2016-04-20 深圳市中博科创信息技术有限公司 Data processing method and device for cluster file system

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704201A (en) * 2017-09-11 2018-02-16 厦门集微科技有限公司 Data storage handling method and device
CN107704201B (en) * 2017-09-11 2020-07-31 厦门集微科技有限公司 Data storage processing method and device
CN110716692A (en) * 2018-07-13 2020-01-21 浙江宇视科技有限公司 Reading performance improving method and device, storage node and data reading method
CN110716692B (en) * 2018-07-13 2022-11-25 浙江宇视科技有限公司 Reading performance improvement method, device, storage node and data reading method
CN109508317A (en) * 2018-10-31 2019-03-22 武汉光谷联众大数据技术有限责任公司 A kind of Large Volume Data and service management system
CN109508317B (en) * 2018-10-31 2023-06-09 陕西合友网络科技有限公司 High-capacity data and service management system
CN109587221A (en) * 2018-11-09 2019-04-05 平安科技(深圳)有限公司 Large data sets group managing means, device, storage medium and computer equipment
CN110674095B (en) * 2019-09-27 2022-06-10 浪潮电子信息产业股份有限公司 CTDB cluster expansion method, device and equipment and readable storage medium
CN110674095A (en) * 2019-09-27 2020-01-10 浪潮电子信息产业股份有限公司 A CTDB cluster expansion method, device, device and readable storage medium
CN111314129A (en) * 2020-02-13 2020-06-19 上海凯岸信息科技有限公司 High-availability architecture based on file type storage service
CN111506459A (en) * 2020-04-22 2020-08-07 深圳前海微众银行股份有限公司 Data import method, device, device and computer storage medium
CN112131229A (en) * 2020-09-29 2020-12-25 中国银行股份有限公司 Block chain-based distributed data access method and device and storage node
CN112468330A (en) * 2020-11-13 2021-03-09 苏州浪潮智能科技有限公司 Method, system, equipment and medium for setting fault node
CN112468330B (en) * 2020-11-13 2022-12-06 苏州浪潮智能科技有限公司 A method, system, device and medium for setting up a faulty node
CN112395165A (en) * 2020-11-27 2021-02-23 中电科技(北京)有限公司 Method and equipment for acquiring resource utilization rate of server and server
CN112671905A (en) * 2020-12-23 2021-04-16 广州三七互娱科技有限公司 Service scheduling method, device and system
CN112737962A (en) * 2020-12-24 2021-04-30 平安科技(深圳)有限公司 Storage service request processing method and device, computer equipment and storage medium
CN112737962B (en) * 2020-12-24 2023-06-02 平安科技(深圳)有限公司 Storage service request processing method and device, computer equipment and storage medium
CN112988905B (en) * 2021-04-27 2021-08-10 北京沃丰时代数据科技有限公司 Node memory synchronization method and device for cluster deployment
CN112988905A (en) * 2021-04-27 2021-06-18 北京沃丰时代数据科技有限公司 Node memory synchronization method and device for cluster deployment
CN113590040A (en) * 2021-07-29 2021-11-02 郑州阿帕斯数云信息科技有限公司 Data processing method, device, equipment and storage medium
CN113590040B (en) * 2021-07-29 2024-03-19 郑州阿帕斯数云信息科技有限公司 Data processing method, device, equipment and storage medium
WO2023109381A1 (en) * 2021-12-16 2023-06-22 中移(苏州)软件技术有限公司 Information processing method and apparatus, and storage medium
CN114189547B (en) * 2022-02-14 2022-05-03 北京安盟信息技术股份有限公司 SSL tunnel fast switching method under cluster
CN114189547A (en) * 2022-02-14 2022-03-15 北京安盟信息技术股份有限公司 SSL tunnel fast switching method under cluster

Also Published As

Publication number Publication date
CN105511805A (en) 2016-04-20
CN105511805B (en) 2019-03-19

Similar Documents

Publication Publication Date Title
WO2017088664A1 (en) Data processing method and apparatus for cluster file system
WO2016068622A1 (en) Terminal device and method of controlling same
WO2018233370A1 (en) Mirror synchronization method, system, device and computer readable storage medium
WO2013131444A1 (en) Content sharing method, terminal, server, and system, and computer storage medium
WO2015120774A1 (en) Network access method and apparatus applied to mobile application
WO2019128174A1 (en) Method for playing audio, smart television and computer-readable storage medium
WO2013017004A1 (en) Method, system, client and server for scanning file
WO2018068411A1 (en) Remote debugging method for smart television and remote debugging system for smart television
WO2018028121A1 (en) Method and device for managing storage space of data partition
WO2017206883A1 (en) Application processing method and apparatus, storage medium, and electronic device
WO2018076812A1 (en) Data request response method and device, storage medium, server and system
WO2019218441A1 (en) Request processing method and apparatus, device, and storage medium
WO2018076811A1 (en) Data sharing method, device, system, storage medium and electronic device
WO2016058258A1 (en) Terminal remote control method and system
WO2019205272A1 (en) Virtual machine service providing method, device and equipment and computer readable storage medium
WO2018014567A1 (en) Method for improving performance of virtual machine, and terminal, device and computer-readable storage medium
WO2016000560A1 (en) File transmission method, file transmission apparatus, and file transmission system
WO2018053963A1 (en) Method and apparatus for upgrading system of smart television
WO2018120680A1 (en) Virtual disk backup system, method, apparatus, service host and storage medium
WO2018076875A1 (en) Backup data synchronisation method, apparatus, storage medium, electronic device, and server
WO2017024805A1 (en) File delivery method, apparatus, and system
WO2023033588A1 (en) System for controlling data flow in virtualization terminal, and method thereof
WO2021241849A1 (en) Electronic device for performing edge computing service, and operating method of electronic device
WO2018076870A1 (en) Data processing method and apparatus, storage medium, server, and data processing system
WO2018006581A1 (en) Smart-television playback method and apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16867888

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16867888

Country of ref document: EP

Kind code of ref document: A1