CN106155566A - For storing method, the Apparatus and system of data - Google Patents

Abstract

The present invention relates to methods, devices and systems for storing data. The method includes: receiving a data operation request for instructing to store target data, wherein the data operation request includes a data classification of the target data; according to a storage strategy representing a correspondence between multiple data classifications and multiple storage systems, determine the A storage system corresponding to the data classification of the target data; and storing the target data in the determined storage system. In the embodiment of the present invention, data is classified and different storage systems are used to store data of different data classifications, so as to meet different storage requirements of users for data of different classifications.

Description

Method, device and system for storing data

technical field

The present invention relates to the field of information technology, in particular to a method, device and system for storing data.

Background technique

With the explosive growth of global data scale, data storage technology is becoming more and more important in this era of big data. Therefore, how to better implement data storage has become a hot spot of people's attention.

Common storage systems may include centralized storage systems and distributed storage systems. As a traditional storage technology, centralized storage systems have been widely used in enterprise-level applications and some key applications. For example, commonly used centralized storage systems include network attached storage (Network Attached Storage, NAS), storage area network (Storage Area Network, SAN), and so on. Due to the use of controllers and management software based on specific hardware, the centralized storage system can achieve high performance and high reliability, but the cost is relatively high.

With the development of cloud computing and big data-related technologies, distributed storage systems have emerged. In a distributed storage system, Directly Attached Storage (DAS) is usually used as a storage medium, and a Distributed File System (Distributed File System, DFS) is used to manage distributed data files or nodes. Therefore, compared with the centralized storage system, the cost of the distributed storage system is greatly reduced, but the performance is also reduced accordingly.

Currently, users (for example, enterprises or public institutions, etc.) will always use one of these two storage systems to store data. However, with the rapid increase of data services, users have various types of data, for example, some are important data, and some are less important data, so there are also various requirements for data storage. For example, a user may hope that data storage can achieve both high performance and low cost. In this case, it will be difficult to meet the different storage requirements of the user by using one of the two storage systems.

Contents of the invention

Considering the above-mentioned problems in the prior art, embodiments of the present invention provide a method, device and system for storing data, which can meet users' different storage requirements for different categories of data.

An embodiment of the present invention provides a method for storing data, including: receiving a data operation request indicating to store target data, wherein the data operation request includes a data classification of the target data; A storage strategy representing correspondence between multiple data categories and multiple storage systems, determining a storage system corresponding to the data category of the target data; and storing the target data in the determined storage system.

Wherein, the data classification is represented by a user ID or a data source, and the storage policy includes at least one of the following correspondences: a correspondence between multiple user IDs and the multiple storage systems, and multiple data sources Correspondence with the multiple storage systems.

Wherein, the multiple storage systems include a centralized storage system and a distributed storage system.

Wherein, the method further includes: receiving another data operation request indicating to perform a specified operation on the target data, wherein the another data operation request includes the data classification of the target data, and the specified operation includes One of a read operation, an update operation, and a delete operation; according to the storage policy, search for a storage system corresponding to the data classification of the target data; and execute the target data in the found storage system The specified operation.

Another embodiment of the present invention provides an apparatus for storing data, including: a receiving module, configured to receive a data operation request indicating to store target data, wherein the data operation request includes the target Data classification of data; a determination module, configured to determine a storage system corresponding to the data classification of the target data according to a storage strategy representing a correspondence relationship between multiple data classifications and multiple storage systems; an operation module, used to convert all The above target data is stored in the determined storage system.

Wherein, the data classification is represented by a user ID or a data source, and the storage policy includes at least one of the following correspondences: a correspondence between multiple user IDs and the multiple storage systems, and multiple data sources Correspondence with the multiple storage systems.

Wherein, the multiple storage systems include a centralized storage system and a distributed storage system.

Wherein, the receiving module is further configured to receive another data operation request indicating to perform a specified operation on the target data, wherein the other data operation request includes the data classification of the target data, and the specified The operation includes one of a read operation, an update operation, and a delete operation; the determination module is further configured to search for a storage system corresponding to the data classification of the target data according to the storage policy; and the operation module , further configured to perform the specified operation on the target data in the found storage system.

Another embodiment of the present invention provides a system for storing data, including: a plurality of storage systems; and a control device, configured to, when receiving data indicating to store target data and including the target data When a classified data operation request is made, a storage system corresponding to the data classification of the target data is determined and the target data is stored in the determined storage system.

Wherein, the data classification is represented by a user ID or a data source, and the storage policy includes at least one of the following correspondences: a correspondence between multiple user IDs and the multiple storage systems, and multiple data sources Correspondence with the multiple storage systems.

Wherein, the multiple storage systems include a centralized storage system and a distributed storage system.

Wherein, the control device is further configured to, when receiving another data operation request indicating to perform a specified operation on the target data and including the data classification of the target data, search for a data classification related to the target data The corresponding storage system performs the specified operation on the target data in the found storage system, where the specified operation includes one of a read operation, an update operation, and a delete operation.

In the embodiment of the present invention, data is classified and different storage systems are used to store data of different data classifications, so as to meet different storage requirements of users for data of different classifications.

Description of drawings

Other features, features, advantages and benefits of the present invention will become more apparent from the following detailed description in conjunction with the accompanying drawings.

FIG. 1 is a schematic diagram of a system for storing data according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of an example of a form of a storage policy according to an embodiment of the present invention.

Fig. 3 is a flowchart of a method for storing data according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of an apparatus for storing data according to an embodiment of the present invention.

Fig. 5 is a schematic diagram of a control device according to an embodiment of the present invention.

detailed description

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a schematic diagram of a system for storing data according to an embodiment of the present invention. As shown in FIG. 1 , a system 100 for storing data includes a control device 110 and multiple storage systems.

In a specific implementation manner, the multiple storage systems may include a centralized storage system and a distributed storage system, for example, the centralized storage system 120 and the distributed storage system 130 shown in FIG. 1 . For example, the centralized storage system 120 may be NAS or SAN. The distributed storage system 130 may use DAS as a storage medium, and use DFS to manage distributed data files or nodes. However, the embodiments of the present invention are not limited thereto. For example, multiple storage systems may include a greater number of storage systems, and may also include other storage systems than the centralized storage system and the distributed storage system.

The control device 110 can interact with the centralized storage system 120 and the distributed storage system 130 . For example, the control device 110 may be connected to the centralized storage system 120 and the distributed storage system 130 in a wired or wireless manner, so as to communicate with the centralized storage system 120 and the distributed storage system 130 .

Various data transmission protocols can be used between the control device 110 and the centralized storage system 120 and the distributed storage system 130 . For example, if the centralized storage system 120 is NAS, NFS or Common Internet File System (Common Internet File System, CIFS) protocol can be used between the control device 110 and the NAS.

If the centralized storage system 120 is a SAN, the data transmission protocol to be used may be determined according to the type of the SAN. For example, for a Fiber Channel (Fiber Channel, FC) SAN, a dedicated FC protocol can be used between the control device 110 and the NAS; and for an Internet Protocol (Internet Protocol, IP) SAN, the Internet can be used between the control device 110 and the NAS Small Computer System Interface (Internet Small Computer System Interface, ISCSI) protocol.

The transmission protocol between the control device 110 and the distributed storage system 120 usually depends on the DFS used to organize the DAS. For example, when the DFS is Hadoop DFS (HDFS), an HTTP-based HDFS protocol can be used.

In addition, the control device 110 may also be connected to a device used by the user (for example, a computer, etc.) in a wired or wireless manner, and communicate with the device used by the user.

For example, control device 110 may be connected to computers used by various departments in an enterprise. Applications can be installed on these computers, and can communicate with the control device 110 through the installed applications. Computers in various departments may generate different types of data, such as financial data, management data, business data, and so on. When these data need to be stored in a storage system, the applications installed on these computers may send a data operation request to the control device 110 to request to store the data in a certain storage system.

The application can send a data operation request to the control device 110 through a variety of file transfer protocols. For example, the transfer protocol between the application and the control device 110 can include REST API, File Transfer Protocol (File Transfer Protocol, FTP), network file system ( Network File System, NFS), etc.

Specifically, the control device 110 is configured to determine the storage system corresponding to the data classification of the target data and store the target data when receiving a data operation request indicating to store the target data and including the data classification of the target data stored in the identified storage system.

Specifically, the control device 110 may receive a data operation request for instructing to store the target data, where the data operation request includes the data classification of the target data; according to the storage policy indicating the correspondence between multiple data classifications and multiple storage systems , determining a storage system corresponding to the data classification of the target data; and storing the target data in the determined storage system.

For example, the storage policy may be configured in the control device 110 by the user in advance. In addition, users can also update storage policies to adapt to users' ever-changing requirements for storage solutions. For example, the storage policy may be a user through a web user interface (Web User Interface) or an application programming interface (Application Programming Interface, API) based on a representational state transfer (Representational State Transfer, REST) protocol (which may be referred to as a REST API) to configure.

In a specific implementation manner, the data classification may be represented by a user identifier or a data source. The storage policy may include at least one of the following correspondences: correspondences between multiple user identifications (UserIdentification, User ID) and multiple storage systems, and correspondences between multiple data sources and multiple storage systems.

Correspondingly, the above-mentioned data operation request may include a user identifier or a data source identifier of the target data.

In this way, the control device 110 can query the storage policy according to the user ID or data source ID in the data operation request, so as to determine the storage system corresponding to the user ID or data source ID of the target data, and then store the target data in the determined in the storage system.

It can be seen from the above that in the embodiment of the present invention, data is classified and data of different data classifications are stored using different storage systems, so as to meet different storage requirements of users for data of different classifications.

For example, in the case of multiple storage systems including a centralized storage system and a distributed storage system, the user can combine the storage requirements with the characteristics of the two storage systems when configuring the storage policy, so as to meet the different storage requirements of the user. For example, real-time data requires high storage performance, while for other data, users want to achieve low storage costs. For the centralized storage system, it has high storage performance, and the distributed storage system has low cost. Then, the user can specify in the storage strategy to store real-time data in the centralized storage system and store other data in the distributed storage system. in the storage system.

In another specific implementation manner, the control device 110 may also be configured to, when receiving another data operation request indicating to perform a specified operation on the target data and including the data classification of the target data, search for the data classification related to the target data. The corresponding storage system performs a specified operation on the target data in the found storage system, wherein the specified operation includes one of a read operation, an update operation and a delete operation.

Specifically, the control device 110 may support read, update and delete operations on target data. Therefore, the control device can be understood as a transparent interface for data access, supporting CRUD (create/read/update/delete) operations.

After the target data is stored, the storage address of the target data may be cached, or the storage address of the target data may not be cached. In the case of caching the storage address of the target data, the control device 110 can obtain the cached storage address without querying the storage policy after receiving a data operation request from the application for requesting an operation other than storage on the target data. address to perform operations on the target data in the corresponding storage system. If the storage address of the target data is not cached, after the control device 110 receives a data operation request from the application for requesting to perform operations other than storage on the target data, it can query the storage policy to determine the location of the target data. In the storage system corresponding to the data classification, corresponding operations are performed on the target data in the determined storage system. For example, operations such as read, update, or delete may be performed on the target data in the determined storage system.

The storage strategy will be described below. The above storage policy can be expressed in various forms.

As an example, the correspondence between multiple user identities and multiple storage systems can be directly represented by the correspondence between multiple user identities and multiple storage system identities, and the correspondence between multiple data sources and multiple storage systems can be directly It is represented by the corresponding relationship between multiple data source identifiers and multiple storage system identifiers.

Or, as another example, the concept of a "role" could be defined. The roles may include VIP (Very Important Person, VIP) roles and common roles. The correspondence between multiple user IDs and multiple storage systems can be represented by multiple user IDs, roles, and multiple storage system IDs. For example, the corresponding relationship can be defined by the correspondence between multiple user IDs and multiple roles and multiple It is represented by the corresponding relationship between roles and identifiers of multiple storage systems. The corresponding relationship between multiple data sources and multiple storage systems can be represented by multiple data source IDs, multiple roles, and multiple storage system IDs. For example, the corresponding relationship can be represented by multiple data source IDs and multiple The corresponding relationship between roles and the corresponding relationship between multiple roles and multiple storage system identities are represented.

The form of the storage policy will be described in detail below in combination with specific examples. It should be understood that this example is only intended to help those skilled in the art better understand the embodiments of the present invention, rather than limit the scope of the present invention. FIG. 2 is a schematic diagram of an example of a form of a storage policy according to an embodiment of the present invention. In FIG. 2 , the centralized storage system and the distributed storage system are still taken as examples for illustration.

As shown in FIG. 2 , the storage policy may include correspondences between multiple roles and multiple storage systems. For example, a VIP role corresponds to a centralized storage system, while a common role corresponds to a distributed storage system.

The storage policy may also include correspondences between multiple user IDs and multiple roles. For example, as shown in Figure 2, user IDs G001-032 and G001-021 can correspond to VIP roles, while user ID G002-001 can correspond to ordinary roles, then the data from user IDs G001-032 and G001-021 Can correspond to a centralized storage system, while data from user ID G002-001 can correspond to a distributed storage system.

In addition, the storage policy may also include correspondences between multiple data sources and multiple roles. For example, data sources may include enterprise resource planning (Enterprise Resources Planning, ERP), customer relationship management (Customer Relationship Management, CRM), filing systems, and the like. As shown in Figure 2, ERP can correspond to a VIP role, CRM can correspond to a VIP role, and the filing system can correspond to a common role. Then, both the data from ERP and the data from CRM can correspond to the centralized storage system, and the data from the filing system can correspond to the distributed storage system.

It should be understood that although it is described above that the storage policy is configured according to the data user or data source, the user may also configure the storage policy according to other factors to meet different storage requirements.

Referring now to FIG. 3 , it is a flowchart of a method for storing data according to one embodiment of the present invention. For example, the method in FIG. 3 may be executed by the control device 110 in FIG. 1 above.

As shown in Figure 3, the method includes the following steps:

Step 310, receiving a data operation request for instructing to store the target data, wherein the data operation request includes the data classification of the target data.

Step 320: Determine the storage system corresponding to the data classification of the target data according to the storage policy indicating the correspondence between multiple data classifications and multiple storage systems.

Step 330, storing the target data in the determined storage system.

In a specific implementation manner, the multiple storage systems may include a centralized storage system and a distributed storage system.

In another specific implementation manner, the above-mentioned data classification can be represented by user ID or data source, and correspondingly, the storage policy can include at least one of the following correspondences: correspondence between multiple user IDs and multiple storage systems , and, the correspondence between multiple data sources and multiple storage systems.

In another specific implementation manner, the method may further include the following steps: receiving another data operation request indicating to perform a specified operation on the target data, wherein the other data operation request includes the data classification of the target data, the specified operation Including one of a read operation, an update operation and a delete operation; according to a storage policy, searching for a storage system corresponding to the data classification of the target data; and performing a specified operation on the target data in the found storage system.

Referring now to FIG. 4 , it is a schematic diagram of an apparatus for storing data according to an embodiment of the present invention. The apparatus 400 shown in FIG. 4 can be implemented by software, hardware (such as an integrated circuit or DSP, etc.) or a combination of software and hardware, and can be installed in the control device 110 .

As shown in FIG. 4 , the apparatus 400 may include a receiving module 410 , a determining module 420 and an operating module 430 .

The receiving module 410 is configured to receive a data operation request for instructing to store target data, wherein the data operation request includes data classification of the target data. The determining module 420 is configured to determine the storage system corresponding to the data classification of the target data according to the storage policy indicating the correspondence between multiple data classifications and multiple storage systems. The operation module 430 is used for storing the target data in the determined storage system.

In a specific implementation manner, data classification may be represented by user identifiers or data sources, and correspondingly, the storage policy may include at least one of the following correspondences: correspondences between multiple user identifiers and multiple storage systems, and , the correspondence between multiple data sources and multiple storage systems.

In another specific implementation manner, the multiple storage systems may include a centralized storage system and a distributed storage system.

In another specific implementation manner, the receiving module 410 may also be configured to receive another data operation request indicating to perform a specified operation on the target data, where the other data operation request includes the data classification of the target data , specifying that the operation includes one of a read operation, an update operation, and a delete operation. The determination module 420 can also be used to search for a storage system corresponding to the data category of the target data according to the storage policy; and the operation module 430 can also be used to perform a specified operation on the target data in the found storage system.

Referring now to FIG. 5 , which is a schematic diagram of a control device according to one embodiment of the present invention. As shown in FIG. 5 , the control device 500 may include a memory 510 for storing executable instructions and a processor 520 connected to the memory 510 , where the processor 520 may execute the operations performed by the various modules of the aforementioned apparatus 400 .

The embodiment of the present invention also provides a machine-readable medium on which executable instructions are stored, and when the executable instructions are executed, the machine implements the operations of the processor 520 .

Those skilled in the art should understand that various variations and modifications can be made to the above-disclosed embodiments without departing from the essence of the invention. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims (14)

1. the method being used for storing data, comprising:

Receiving the data operation request storing target data for instruction, wherein, described data are grasped Make request and include the data classification of described target data；

According to the storage strategy representing the classification of multiple data and the corresponding relation of multiple storage systems, determine with The data corresponding storage system of classification of described target data；And

In storage system determined by described target data is stored in.

2. method according to claim 1, wherein,

Described data classified use ID or data source represent,

Described storage strategy includes at least one in following corresponding relation: multiple ID with described many The corresponding relation of individual storage system, and, multiple data source is corresponding with the plurality of storage system closes System.

3. method according to claim 1 and 2, wherein,

The plurality of storage system includes centralised storage system and distributed memory system.

4. method according to claim 1 and 2, also includes:

Receive for indicating another data operation request performing assigned operation to described target data, wherein, Another data operation request described includes the data classification of described target data, and described assigned operation includes reading Extract operation, the one updating in operation and deletion action；

According to described storage strategy, search the storage system corresponding with the classification of the data of described target data； And

In the storage system finding, described assigned operation is performed to described target data.

5. the device being used for storing data, comprising:

Receiver module, for receiving the data operation request storing target data for instruction, its In, described data operation request includes the data classification of described target data；

Determining module, for according to representing depositing of the classification of multiple data and the corresponding relation of multiple storage systems Storage strategy, determines the storage system corresponding with the classification of the data of described target data；

Operation module, in storage system determined by being stored in described target data.

6. device according to claim 5, wherein,

Described data classified use ID or data source represent,

Described storage strategy includes at least one in following corresponding relation: multiple ID with described many The corresponding relation of individual storage system, and, multiple data source is corresponding with the plurality of storage system closes System.

7. the device according to claim 5 or 6, wherein,

The plurality of storage system includes centralised storage system and distributed memory system.

8. the device according to claim 5 or 6, wherein,

Described receiver module, is additionally operable to reception and performs the another of assigned operation for instruction to described target data One data operation request, wherein, another data operation request described includes that the data of described target data are divided Class, described assigned operation includes read operation, the one updating in operation and deletion action；

Described determining module, is additionally operable to, according to described storage strategy, search the data with described target data Classify corresponding storage system；And

Described operation module, is additionally operable to perform institute to described target data in the storage system being found State assigned operation.

9. a control equipment, comprising:

Memory；And

Processor, for performing the operation included by any one in claim 1-4.

10. a machine readable media, is stored thereon with executable instruction, when described executable instruction quilt During execution so that machine performs the operation included by any one in claim 1-4.

11. 1 kinds for storing the system of data, comprising:

Multiple storage systems；And

Control equipment, carries out storing and include described target to target data for instruction for working as to receive

During the data operation request that the data of data are classified, determine the data classification phase with described target data

In corresponding storage system storage system determined by described target data is stored in.

12. systems according to claim 11, wherein,

Described data classified use ID or data source represent,

Described storage strategy includes at least one in following corresponding relation: multiple ID with described many The corresponding relation of individual storage system, and, multiple data source is corresponding with the plurality of storage system closes System.

13. systems according to claim 11 or 12, wherein,

The plurality of storage system includes centralised storage system and distributed memory system.

14. systems according to claim 11 or 12, wherein,

Described control equipment, is additionally operable to receiving for instruction to described target data execution assigned operation And including described target data data classification another data operation request when, search with described target The data of data classify corresponding storage system and in the storage system being found to described target Data perform described assigned operation, and wherein, described assigned operation includes read operation, updates operation and delete One in division operation.