CN102938784A - Method and system used for data storage and used in distributed storage system - Google Patents
Method and system used for data storage and used in distributed storage system Download PDFInfo
- Publication number
- CN102938784A CN102938784A CN2012104411761A CN201210441176A CN102938784A CN 102938784 A CN102938784 A CN 102938784A CN 2012104411761 A CN2012104411761 A CN 2012104411761A CN 201210441176 A CN201210441176 A CN 201210441176A CN 102938784 A CN102938784 A CN 102938784A
- Authority
- CN
- China
- Prior art keywords
- data
- memory node
- stored
- different
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 82
- 238000013500 data storage Methods 0.000 title claims abstract description 25
- 230000006870 function Effects 0.000 claims description 42
- 230000008569 process Effects 0.000 claims description 32
- 230000009471 action Effects 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 21
- 238000004321 preservation Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 4
- 238000007726 management method Methods 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 239000012467 final product Substances 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a system used for data storage and used in a distributed storage system. The method comprises the steps of determining a data category of data to be stored; looking up a preset storage rule corresponding to the data category of the data to be stored, wherein the preset storage rule comprises the number of copies and different storage location information; generating data copies of the data to be stored according to the number of copies; and storing data copies at storage nodes corresponding to the different storage location information respectively. According to the method and the system, different numbers of data copies are generated for different data categories for storage, so that the safety, the expansibility, the operability and the like of the stored data are guaranteed, the occupancy of system resources is avoided, and the reliability of the storage system is effectively improved.
Description
Technical field
The application relates to technical field of data processing, particularly a kind of date storage method and system.
Background technology
Storage system refers in computer by the equipment (hardware) of the various memory devices of depositing program and data, control assembly and management information scheduling and the system that algorithm (software) forms, along with the application of mass data storage need to, distributed memory system, being about to data disperses to be stored in the server of many platform independent, share the storage load owing to can utilizing multiple servers, improve to a certain extent reliability and the storage efficiency of system, therefore become the main development mode of data storage.
But, along with storage system scale in the systems such as HLRS, data center and cloud stores service center is increasing, the number of servers comprised is more and more, reliability remains one of major issue that distributed memory system faces.
In prior art, a kind of reliability method that improves distributed memory system is to adopt the Redundant Arrays of Inexpensive Disks mode, by the redundancy backup characteristic, guarantees.Be about to a plurality of disks and be combined into a big hard disk by different compound modes, thereby the memory property higher than single hard disk is provided.But because the distributed memory system reliability not only depends on the reliability of disk array, also relate to the stability problem of storage server, storage networking etc., and the large-scale distributed storage system often polytype storage mode of Integrated using is stored, therefore still exist corrupted data, data to be difficult to the integrity problems such as expansion, the data recovery is slow, operability is poor.
In a word, need at present the urgent technical problem solved of those skilled in the art to be exactly: a kind of active data storage means of the proposition how can innovate, with the low problem of reliability that solves distributed memory system in prior art.
Summary of the invention
The application's technical problem to be solved is to provide a kind of date storage method that is applied to distributed memory system, in order to solve the low technical problem of data storing reliability in prior art.
The application also provides a kind of data-storage system that is applied to distributed memory system, in order to guarantee said method implementation and application in practice.
In order to address the above problem, the application discloses a kind of date storage method that is applied to distributed memory system, comprising:
Determine the data category of data to be stored;
Search the default storage rule corresponding to data category of described data to be stored, described default storage rule comprises copy number and different stored position informations;
According to described copy number, generate the data trnascription of described data to be stored;
Described data trnascription is stored into respectively in the memory node that different stored position informations is corresponding.
Preferably, described data to be stored are specially file to be stored, and the data category of described definite data to be stored comprises:
Obtain the filename suffix of file to be stored;
According to described filename suffix, determine the data category of described file to be stored.
Preferably, described by described data trnascription store into respectively different stored position informations respectively corresponding memory node comprise:
Call the file system of described distributed memory system, described data trnascription is stored into respectively in memory node corresponding to different stored position information difference.
Preferably, the described file system of calling described distributed memory system, store respectively described data trnascription into different memory node corresponding to memory node address and comprise:
Determine the file system type of described distributed memory system;
Search the configuration interface function corresponding to described file system type of preservation;
By the described file system of described configuration interface function call, described data trnascription is stored into respectively in memory node corresponding to different memory node addresses.
Preferably, described by described data trnascription store into respectively different stored position informations respectively corresponding memory node comprise:
Determine described different memory node corresponding to stored position information difference, and the file system of each memory node application;
Search the configuration interface function corresponding to type difference of different file system;
By file system corresponding to described different configuration interface function calls, described data trnascription is stored into respectively in the memory node that different stored position informations are corresponding.
Preferably, described by described data trnascription, store into respectively in the memory node that different stored position informations is corresponding after, described method also comprises:
When receiving data access request, determine the target data classification of request access according to described data access request;
Search target storage rule corresponding to described target data classification;
According to the stored position information in described target storage rule, determine the target memory node corresponding to different target data trnascription difference of described target data;
According to the action type of described data access request, process the target data copy in described each target memory node.
Preferably, the target data copy in described each memory node of described processing comprises:
Judge that whether described each memory node is effective memory node, obtains the first judged result;
According to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
Preferably, described action type is specially the Data Update operation, and after processing the target data copy in the effective memory node in described memory node, described method also comprises:
Whether the target data copy that judges described memory node is updated successfully, and obtains the second judged result;
According to described the second judged result, determine that the memory node be updated successfully is effective memory node, upgrading failed memory node is the invalid storage node.
In order to address the above problem, disclosed herein as well is a kind of data-storage system be applied in distributed memory system, comprising:
The data category determination module, for determining the data category of data to be stored;
The rule searching module, for default storage rule corresponding to data category of searching described data to be stored, described default storage rule comprises copy number and different stored position informations;
The copy generation module, for according to described copy number, generate the data trnascription of described data to be stored;
Data memory module, for storing respectively described data trnascription into memory node that different stored position informations is corresponding.
Preferably, described data category determination module comprises:
Acquisition module, for obtaining the filename suffix of file to be stored;
Classification is determined submodule, for according to described filename suffix, determines the data category of described file to be stored.
Preferably, described data memory module, specifically for calling the file system of described distributed memory system, stores described data trnascription respectively in the memory node that different stored position informations is corresponding.
Preferably, described data storage mould comprises:
The system type determination module, for determining the file system type of described distributed memory system;
The first function is searched module, for configuration interface function corresponding to described file system type of searching preservation;
The first sub module stored, for by the described file system of described configuration interface function call, store described data trnascription respectively in memory node corresponding to different memory node addresses.
Preferably, described data memory module comprises:
The node determination module, for determining described different memory node corresponding to stored position information difference, and the file system of each memory node application;
The second function is searched module, for searching the configuration interface function corresponding to type difference of different file system;
The second sub module stored, for by file system corresponding to described different configuration interface function calls, store described data trnascription respectively in the memory node that different stored position informations are corresponding.
Preferably, described system also comprises:
Data demand module, for receiving data access request, determine the target data classification of request access according to described data access request;
Goal rule is searched module, for searching target storage rule corresponding to described target data classification;
The destination node determination module, according to the stored position information in described target storage rule, determine the memory node corresponding to different target data trnascription difference of described target data;
Processing module, for the action type according to described data access request, process the target data copy in described each target memory node.
Preferably, described processing module comprises:
Whether the first judge module, be effective memory node for judging described each memory node, obtains the first judged result;
Process submodule, for according to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
Preferably, described action type is specially the Data Update operation, and described system also comprises:
Whether the second judge module, be updated successfully for the target data copy that judges described memory node, obtains the second judged result;
Node effect determination module, for according to described the second judged result, determine that the memory node be updated successfully is effective memory node, and upgrading failed memory node is the invalid storage node.
Preferably, described system also comprises:
Subscriber Interface Module SIM, for receiving user's default storage rule update request, upgrade described default storage rule, and preserve the default storage rule after upgrading.
Compared with prior art, the application comprises following advantage:
In this application, at first determine the data category of data to be stored, and search the default storage rule that this data category is corresponding, generate a plurality of data trnascriptions of these data to be stored according to the copy data in default storage rule, and data trnascription is stored into respectively in the memory node that stored position information in this default storage rule is corresponding, the data trnascription that generates targetedly different numbers by the data category to different is stored, thereby both guaranteed the fail safe of storage data, autgmentability, the problems such as operability, avoided again taking system resource, thereby effectively improved the reliability of storage system.
Certainly, arbitrary product of enforcement the application might not need to reach above-described all advantages simultaneously.
The accompanying drawing explanation
In order to be illustrated more clearly in the technical scheme in the embodiment of the present application, in below describing embodiment, the accompanying drawing of required use is briefly described, apparently, accompanying drawing in the following describes is only some embodiment of the application, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
Fig. 1 is a kind of flow chart that is applied to the date storage method embodiment 1 of distributed memory system of the application;
Fig. 2 is a kind of flow chart that is applied to the date storage method embodiment 2 of distributed memory system of the application;
Fig. 3 is a kind of flow chart that is applied to the date storage method embodiment 3 of distributed memory system of the application;
Fig. 4 is a kind of structural representation that is applied to the data-storage system embodiment 4 of distributed memory system of the application;
Fig. 5 is a kind of structural representation that is applied to the data-storage system embodiment 5 of distributed memory system of the application;
Fig. 6 is a kind of structural representation that is applied to the data-storage system embodiment 6 of distributed memory system of the application.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present application, the technical scheme in the embodiment of the present application is clearly and completely described, obviously, described embodiment is only the application's part embodiment, rather than whole embodiment.Embodiment based in the application, those of ordinary skills are not making under the creative work prerequisite the every other embodiment obtained, and all belong to the scope of the application's protection.
The application can be used in numerous general or special purpose calculation element environment or configuration.For example: personal computer, server computer, handheld device or portable set, plate equipment, multi-processor device, comprise distributed computing environment (DCE) of above any device or equipment etc.
The application can describe in the general context of the computer executable instructions of being carried out by computer, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract data type, program, object, assembly, data structure etc.Also can in distributed computing environment (DCE), put into practice the application, in these distributed computing environment (DCE), be executed the task by the teleprocessing equipment be connected by communication network.In distributed computing environment (DCE), program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
One of main thought of the application can comprise, at first determine the data category of data to be stored, and search the default storage rule that this data category is corresponding, generate a plurality of data trnascriptions of these data to be stored according to the copy data in default storage rule, and data trnascription is stored into respectively in the memory node that stored position information in this default storage rule is corresponding, the data trnascription that generates targetedly different numbers by the data category to different is stored, thereby both guaranteed the fail safe of storage data, autgmentability, the problems such as operability, avoided again taking system resource, thereby effectively improved the reliability of storage system.
With reference to figure 1, show a kind of flow chart that is applied to the date storage method embodiment 1 of distributed memory system of the application, can comprise the following steps:
Step 101: the data category of determining data to be stored.
Distributed memory system is a kind of cluster storage system, data can be disperseed to store in different storage servers, by file system, data is managed, and file system refers to for the method for the file on hyperdisk or subregion and data structure.
Wherein, these data to be stored can refer to metadata or file data.File data and metadata are two kinds of storage data modes that file system is managed, and file data comprises catalogue data and real data, and metadata is the data for the organization and management file data.
In actual applications, as a kind of possible situation that realizes, described data to be stored can refer to metadata.
In the present embodiment, data to be stored, when being stored, are at first determined its data category, and described data category refers to for distinguishing the type feature of different pieces of information.Described data category can be divided according to the file format of data to be stored, file type, size etc.Such as can partitioned image according to different file types, the data category such as animation, sound, text, webpage, can divide the data categories such as temporary file, backup file, help file according to file format, can divide the data categories such as large file, small documents according to file size, can also be divided according to the importance of data, such as source code, document etc. can be divided into the importance data.
As a kind of possible implementation, when described data to be stored are file to be stored, determine that the data category of data to be stored can specifically comprise:
Obtain the filename suffix of file to be stored.
According to described filename suffix, determine the data category of described file to be stored.
According to the filename suffix of file to be stored, can determine the data category of this file to be stored.According to the filename suffix, can draw the information such as file format, file type of these data to be stored, and then can determine the data category of data to be stored.For example, the filename suffix is: " .voc ", and " .wav " is audio files, and " .htm ", " .html " refers to web page files, " .txt ", " .doc " is text etc.And for example, the filename suffix is " .tmp ", and " .syd ", be temporary file, and " .bak " " .old " is backup file etc.Then can analyze according to the filename suffix data category that draws file to be stored.
Certainly, all right data category corresponding to pre-stored different filename suffix of system, thus can directly search the data category that obtains these data to be stored.
Step 102: search the default storage rule that described data category is corresponding, described default storage rule comprises copy number and different stored position informations.
In the embodiment of the present application, preserve storage rule for different data category correspondences in advance, described storage rule can comprise copy number and different stored position informations, certainly can also comprise other storage information.Copy number refers to the copy amount that data to be stored can generate, and stored position information can refer to the memory node for the data storage that distributed memory system can provide, and it can comprise the information such as address, sign of memory node.
The storage rule that different data categories is corresponding different, its copy number and stored position information are also different.Such as the data to be stored that are temporary file for data category,, in order to reduce performance loss, its copy number can be set to one, and stored position information comprises a memory node.And for example data category is the importance data, as source code, document etc., its copy number can arrange a plurality of, corresponding each copy can arrange a plurality of different memory nodes and be stored, stored position information also comprises a plurality of, and memory address wherein or memory node sign can be according in the available memory nodes of distributed memory system.The data that are and for example large file for data category, its copy number can arrange less, and, for the data of small documents, copy data can arrange more a plurality of.
Therefore, in the present embodiment, can be to different storage rules should be arranged according to different data categories, thus can be stored according to the characteristic of data itself.
Wherein, storage rule corresponding to described different pieces of information classification, the update request of all right User, upgraded storage rule.
Step 103: according to described copy number, generate the data trnascription of described data to be stored.
Data trnascription is and the on all four data of data to be stored, and it can be the copy data of data to be stored, and data to be stored and copy data all are called the data trnascription of data to be stored.
According to the copy number in determined default storage rule, can the ghost number data trnascription of these data to be stored.
Step 104: described data trnascription is stored into respectively in the memory node that different stored position informations is corresponding.
Can determine for storing the different memory nodes of data trnascription according to different stored position informations, thereby data trnascription can be stored into respectively in different memory nodes.
Store data trnascription in different memory nodes, guaranteed the extensibility of data, and can, so that during the corrupted data of a certain memory node, still can use the data in other memory nodes to carry out normal data access operation.Due to its storage mode difference of data to be stored of different pieces of information classification, therefore increase the operability of data, and avoided some data to the taking of system resource, improved the reliability of data.
In the present embodiment, data category according to data to be stored, can determine the default storage rule that data to be stored are corresponding, thereby storage rule that can be default according to this, the generated data copy and by the data trnascription respective stored in different memory nodes, the data trnascription that generates targetedly different numbers for different data categories is stored, thereby the problems such as fail safe, autgmentability, operability of storage data had both been guaranteed, avoided again the taking of system resource, thereby effectively improved the reliability of storage system.
With reference to figure 2, show a kind of flow chart that is applied to the date storage method embodiment 2 of distributed memory system of the application, can comprise the following steps:
Step 201: the data category of determining data to be stored.
Wherein, when data to be stored are file to be stored, can at first obtain the filename suffix of file to be stored.According to described filename suffix, determine the data category of described file to be stored.
Step 202: search the default storage rule that described data category is corresponding, described default storage rule comprises copy number and different stored position informations.
Step 203: according to described copy number, generate the data trnascription of described data to be stored.
The operation of step 201 ~ step 203 can, referring to the step 101 of above-described embodiment 1 ~ step 103, not repeat them here.
Step 204: call the file system of described distributed memory system, described data trnascription is stored into respectively in the memory node that different stored position informations is corresponding.
In distributed memory system, usually by file system, data are managed, file system is method and the data structure of operating system for the file on hyperdisk or subregion, be a kind of on disk the method for constituent act, in distributed memory system, file system is to be distributed in different clusters.In this application, call the file system in storage system, use file system to realize data trnascription is stored into respectively in different memory nodes.
In computer network, data to be stored are normally produced by consumer process, therefore described data to be stored can be the data of obtaining from consumer process by Virtual File System, and Virtual File System is an interface layer between physical file system and consumer process service.Data to be stored, after above-mentioned processing, store in the different memory nodes of physical file system management.
Wherein, when carrying out data storage and data access, the algorithm that different file adopts or function are different.Therefore in the embodiment of the present application, corresponding configuration interface function can be set for different file system types, according to different file system types, can select corresponding configuration interface function to realize storage and the access of data.Therefore, calling the file system of distributed memory system, described data trnascription stored into respectively in memory node corresponding to different memory node addresses and can be specially:
Determine the file system type of described distributed memory system;
Search the configuration interface function corresponding to described file system type of preservation;
By the described file system of described configuration interface function call, described data trnascription is stored into respectively in memory node corresponding to different memory node addresses.
When carrying out data access, described method can also comprise:
Step 205: when receiving data access request, determine the target data classification of the target data of request access according to described data access request.
Data access request comprises data read request and Data Update request, and Data Update can comprise the operations such as writing, revise, deleting of data or increase.
According to the target data of data access request request access, can at first determine the target data classification of this target data.
Step 206: search target storage rule corresponding to described target data classification.
According to default storage rule corresponding to different pieces of information classification, can search target storage rule corresponding to this target data classification.
Step 207: according to the target storage position information in described target storage rule, determine the memory node corresponding to different target data trnascription difference of described target data.
Because storage rule comprises copy number and different stored position informations, therefore can, according to the target storage position information in the target storage rule, determine the memory node corresponding to different target data trnascription difference of the target data of this request access.
Step 208: according to the action type of described data access request, process the target data copy in described each memory node.
This action type also can comprise read operation, upgrade operation, wherein upgrades operation and can comprise the operations such as data modification, deletion or increase again.According to different action types, can to the target data copy in each memory node, be processed simultaneously.
Due to different reasons, the memory node of the storage data trnascription of described distributed memory system may be invalid, therefore can at first judge at the target data copy of processing each memory node whether it is effective memory node, if, processed the data trnascription of its preservation, therefore, the target data copy in described each memory node of this processing can comprise:
Judge that whether described each memory node is effective memory node, obtains the first judged result;
According to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
Wherein, when data access request is the Data Update request, be action type while being Data Update, to being treated to of data copy: the target data copy in each memory node is all upgraded, thereby can realize the synchronous renewal of data trnascription in different memory nodes.
After the target data copy to each memory node is all upgraded, may be failed owing to upgrading operation, the data trnascription that upgrades failed memory node is different from the data trnascription of other memory nodes, and in order not affect the access of follow-up data, therefore described method can also comprise:
Whether the target data copy that judges described memory node is updated successfully, and obtains the second judged result.
According to described the second judged result, determine that the memory node be updated successfully is effective memory node, upgrading failed memory node is the invalid storage node.
Wherein, when data access request is data read request, be that action type is while being read operation, because the memory node of storing data trnascription may be invalid memory node, therefore the target data copy of processing in each memory node can be specifically: read successively the target data copy in each memory node, until a certain memory node reads successfully, can stop read operation.
In the present embodiment, data category according to data to be stored, can determine the default storage rule that data to be stored are corresponding, thereby storage rule that can be default according to this, the generated data copy and by the data trnascription respective stored in different memory nodes, the data trnascription that generates targetedly different numbers for different data categories is stored, thereby the problems such as fail safe, autgmentability, operability of storage data had both been guaranteed, avoided again the taking of system resource, thereby effectively improved the reliability of storage system.When carrying out data access, can synchronously process the data in different memory nodes, guaranteed the consistency of data, further improved the reliability of storage system.
With reference to figure 3, show a kind of flow chart that is applied to the date storage method embodiment 3 of distributed memory system of the application, can comprise the following steps:
Step 301: the data category of determining data to be stored.
Wherein, when data to be stored are file to be stored, can at first obtain the filename suffix of file to be stored.According to described filename suffix, determine the data category of described file to be stored.
Step 302: search the default storage rule that described data category is corresponding, described default storage rule comprises copy number and different stored position informations.
Step 303: according to described copy number, generate the data trnascription of described data to be stored.
The operation of step 301 ~ step 303 can, referring to the step 101 of above-described embodiment 1 ~ step 103, not repeat them here.
Step 304: determine described different memory node corresponding to stored position information difference, and the file system of each memory node application.
Step 305: configuration interface function corresponding to type difference of searching different file system;
Step 306: by file system corresponding to described different configuration interface function calls, described data trnascription is stored into respectively in the memory node that different stored position informations are corresponding.
File system due to possibility Integrated using number of different types in distributed memory system, form system complicated, isomery, manage respectively the data in different memory nodes, because its algorithm adopted of dissimilar file system or function are different, therefore, at first determine the file system of different memory nodes and the application of each memory node.Thereby the configuration interface function that the different file type that can search pre-save is corresponding, configuration interface function call this document system corresponding to file system type by each memory node application, be stored to data trnascription in memory node.
Step 307: when receiving data access request, determine the target data classification of request access according to described data access request.
Step 308: search target storage rule corresponding to described target data classification.
Step 309: according to the stored position information in described target storage rule, determine the memory node corresponding to different target data trnascription difference of described target data;
Step 310: according to the action type of described data access request, process the target data copy in described each memory node.
Wherein, the target data copy in processing described each memory node can be specially:
Judge that whether described each memory node is effective memory node, obtains the first judged result;
According to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
Wherein, when data access request is the Data Update request, be action type while being Data Update, to being treated to of data copy: the target data copy in each memory node is all upgraded, thereby can realize the synchronous renewal of data trnascription in different memory nodes.
After the target data copy to each memory node is all upgraded, may be failed owing to upgrading operation, the data trnascription that upgrades failed memory node is different from the data trnascription of other memory nodes, and in order not affect the access of follow-up data, therefore described method can also comprise:
Whether the target data copy that judges described memory node is updated successfully, and obtains the second judged result.
According to described the second judged result, determine that the memory node be updated successfully is effective memory node, upgrading failed memory node is the invalid storage node.
Wherein, when data access request is data read request, be that action type is while being read operation, because the memory node of storing data trnascription may be invalid memory node, therefore the target data copy of processing in each memory node can be specifically: read successively the target data copy in each memory node, until a certain memory node reads successfully, can stop read operation.
In the present embodiment, data category according to data to be stored, can determine the default storage rule that data to be stored are corresponding, thereby storage rule that can be default according to this, the generated data copy and by the data trnascription respective stored in different memory nodes, the data trnascription that generates targetedly different numbers for different data categories is stored, thereby the problems such as fail safe, autgmentability, operability of storage data had both been guaranteed, avoided again the taking of system resource, thereby effectively improved the reliability of storage system.When carrying out data access, can synchronously process the data in different memory nodes, guaranteed the consistency of data, further improved the reliability of storage system.
For aforesaid each embodiment of the method, for simple description, therefore it all is expressed as to a series of combination of actions, but those skilled in the art should know, the application is not subject to the restriction of described sequence of movement, because according to the application, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in specification all belongs to preferred embodiment, and related action and module might not be that the application is necessary.
Corresponding with the method that a kind of date storage method embodiment 1 be applied in distributed memory system of above-mentioned the application provides, referring to Fig. 4, the application also provides a kind of data storage device embodiment 1 be applied in distributed memory system, and in the present embodiment, this device can comprise:
Data category determination module 401, for determining the data category of data to be stored.
These data to be stored can refer to metadata or file data.File data and metadata are two kinds of storage data modes that file system is managed, and file data comprises catalogue data and real data, and metadata is the data for the organization and management file data.
Described data category refers to for distinguishing the type feature of different pieces of information.Described data category can be divided according to the file format of data to be stored, file type, size etc.
Wherein, when described data to be stored are file to be stored, described data category determination module 401 can comprise:
Classification is determined submodule 4012, for according to described filename suffix, determines the data category of described file to be stored.
According to the filename suffix, can draw the information such as file format, file type of these data to be stored, and then can determine the data category of data to be stored.
In the embodiment of the present application, preserve storage rule for different data category correspondences in advance, described storage rule has comprised copy number and different stored position informations.Copy number refer to that data to be stored can generate copy amount, stored position information can refer to the memory node for the data storage that distributed memory system can provide, it can comprise the information such as address, sign of memory node.
Store data trnascription in different memory nodes, guaranteed the extensibility of data, can, so that during the corrupted data of a certain memory node, still can use the data in other memory nodes to carry out normal data access operation.Due to its storage mode difference of data to be stored of different pieces of information classification, therefore increase the operability of data, and avoided some data to the taking of system resource, effectively improved the reliability of data.
Wherein, described system can also comprise a Subscriber Interface Module SIM, for receiving user's default storage rule update request, upgrades described default storage rule, and preserves the default storage rule after upgrading.
In the present embodiment, system is according to the data category of data to be stored, can determine the default storage rule that data to be stored are corresponding, thereby storage rule that can be default according to this, the generated data copy and by the data trnascription respective stored in different memory nodes, the data trnascription that generates targetedly different numbers for different data categories is stored, thereby the problems such as fail safe, autgmentability, operability of storage data had both been guaranteed, avoided again the taking of system resource, thereby effectively improved the reliability of storage system.
Corresponding with the method that a kind of date storage method embodiment 2 be applied in distributed memory system of above-mentioned the application provides, referring to Fig. 5, the application also provides a kind of data storage device embodiment 2 be applied in distributed memory system, and in the present embodiment, this device can comprise:
Data category determination module 501, for determining the data category of data to be stored.
When data to be stored are file to be stored, described data category determination module can comprise:
Acquisition module, for obtaining the filename suffix of file to be stored;
Classification is determined submodule, for according to described filename suffix, determines the data category of described file to be stored.
Use file system that described data trnascription is stored into respectively in memory node corresponding to different stored position information difference
Wherein, when carrying out data storage and data access, the algorithm that different file adopts or function are different.Therefore in the embodiment of the present application, corresponding configuration interface function can be set for different file system types, according to different file system types, can select corresponding configuration interface function to realize storage and the access of data.Therefore, described data memory module 504 can comprise:
The system type determination module, for determining the file system type of described distributed memory system;
The first function is searched module, for configuration interface function corresponding to described file system type of searching preservation;
The first sub module stored, for by the described file system of described configuration interface function call, store described data trnascription respectively in memory node corresponding to different memory node addresses.
Data access request comprises data read request and Data Update request, and Data Update can comprise the operations such as writing, revise, deleting of data or increase.
According to the target data of data access request request access, can at first determine the target data classification of this target data.
Goal rule is searched module 506, for searching target storage rule corresponding to described target data classification.
According to default storage rule corresponding to different pieces of information classification, can search target storage rule corresponding to this target data classification.
Destination node determination module 507, according to the stored position information in described target storage rule, determine the memory node corresponding to different target data trnascription difference of described target data.
Because storage rule comprises copy number and different stored position informations, therefore can, according to the target storage position information in the target storage rule, determine the memory node corresponding to different target data trnascription difference of the target data of this request access.
Due to difference, the memory node of the storage data trnascription of described distributed memory system may be invalid, and therefore, described processing module can comprise:
Whether the first judge module, be effective memory node for judging described each memory node, obtains the first judged result;
Process submodule, for according to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
Wherein, when data access request is the Data Update request, be action type while being Data Update, to being treated to of data copy: the target data copy in each memory node is all upgraded, thereby can realize the synchronous renewal of data trnascription in different memory nodes.After the target data copy to each memory node is all upgraded, may be failed owing to upgrading operation, the data trnascription that upgrades failed memory node is different from the data trnascription of other memory nodes, and in order not affect the access of follow-up data, described system also comprises:
Whether the second judge module, be updated successfully for the target data copy that judges described memory node, obtains the second judged result;
Node effect determination module, for according to described the second judged result, determine that the memory node be updated successfully is effective memory node, and upgrading failed memory node is the invalid storage node.
Wherein, when data access request is data read request, be that action type is while being read operation, because the memory node of storing data trnascription may be invalid memory node, therefore the target data copy of processing in each memory node can be specifically: read successively the target data copy in each memory node, until a certain memory node reads successfully, can stop read operation.
The described data-storage system of the present embodiment can be connected with the file system of distributed memory system by interface layer, and is connected with user process layer by Virtual File System, thereby realizes that the data of consumer process store in file system.This data-storage system can be connected with dissimilar file system.Because the described data-storage system of the present embodiment is by using file system to carry out the data storage, on software architecture, described data-storage system is stacked on file system, therefore it also can be called the topmost paper system of storage system, and the physical file system of described storage system own is lower floor's file system.
In the present embodiment, data category according to data to be stored, can determine the default storage rule that data to be stored are corresponding, thereby storage rule that can be default according to this, the generated data copy and by the data trnascription respective stored in different memory nodes, the data trnascription that generates targetedly different numbers for different data categories is stored, thereby the problems such as fail safe, autgmentability, operability of storage data had both been guaranteed, avoided again the taking of system resource, thereby effectively improved the reliability of storage system.When carrying out data access, can synchronously process the data in different memory nodes, guaranteed the consistency of data, further improved the reliability of storage system.
Corresponding with the method that a kind of date storage method embodiment 3 be applied in distributed memory system of above-mentioned the application provides, referring to Fig. 6, the application also provides a kind of data storage device embodiment 3 be applied in distributed memory system, and in the present embodiment, this device can comprise:
Data category determination module 601, for determining the data category of data to be stored;
File system due to possibility Integrated using number of different types in distributed memory system, form system complicated, isomery, manage respectively the data in different memory nodes, because its algorithm adopted of dissimilar file system or function are different, therefore, described data memory module can comprise:
The second function is searched module 6042, for searching the configuration interface function corresponding to type difference of different file system;
The second sub module stored 6043, for by file system corresponding to described different configuration interface function calls, store described data trnascription respectively in memory node corresponding to different stored position informations difference.
Goal rule is searched module 606, for searching target storage rule corresponding to described target data classification;
Destination node determination module 607, according to the stored position information in described target storage rule, determine the memory node corresponding to different target data trnascription difference of described target data;
Wherein, described processing module can comprise:
Whether the first judge module, be effective memory node for judging described each memory node, obtains the first judged result;
Process submodule, for according to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
Wherein, when action type is specially the Data Update operation, described system also comprises:
Whether the second judge module, be updated successfully for the target data copy that judges described memory node, obtains the second judged result;
Node effect determination module, for according to described the second judged result, determine that the memory node be updated successfully is effective memory node, and upgrading failed memory node is the invalid storage node.
Wherein, when data access request is data read request, be that action type is while being read operation, because the memory node of storing data trnascription may be invalid memory node, therefore the target data copy of processing in each memory node can be specifically: read successively the target data copy in each memory node, until a certain memory node reads successfully, can stop read operation.
In the present embodiment, data category according to data to be stored, can determine the default storage rule that data to be stored are corresponding, thereby storage rule that can be default according to this, the generated data copy and by the data trnascription respective stored in different memory nodes, the data trnascription that generates targetedly different numbers for different data categories is stored, thereby the problems such as fail safe, autgmentability, operability of storage data had both been guaranteed, avoided again the taking of system resource, thereby effectively improved the reliability of storage system.When carrying out data access, can synchronously process the data in different memory nodes, guaranteed the consistency of data, further improved the reliability of storage system.
It should be noted that, each embodiment in this specification all adopts the mode of going forward one by one to describe, and what each embodiment stressed is and the difference of other embodiment that between each embodiment, identical similar part is mutually referring to getting final product.For device class embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, relevant part gets final product referring to the part explanation of embodiment of the method.
Finally, also it should be noted that, in this article, relational terms such as the first and second grades only is used for an entity or operation are separated with another entity or operating space, and not necessarily requires or imply between these entities or operation the relation of any this reality or sequentially of existing.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby make the process, method, article or the equipment that comprise a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or also be included as the intrinsic key element of this process, method, article or equipment.In the situation that not more restrictions, the key element limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment that comprises described key element and also have other identical element.
Being divided into various unit with function while for convenience of description, describing above device describes respectively.Certainly, when implementing the application, can realize the function of each unit in same or a plurality of software and/or hardware.
As seen through the above description of the embodiments, those skilled in the art can be well understood to the mode that the application can add essential general hardware platform by software and realizes.Understanding based on such, the part that the application's technical scheme contributes to prior art in essence in other words can embody with the form of software product, this computer software product can be stored in storage medium, as ROM/RAM, magnetic disc, CD etc., comprise that some instructions are with so that a computer equipment (can be personal computer, server, or the network equipment etc.) carry out the described method of some part of each embodiment of the application or embodiment.
Above a kind of search method, the Apparatus and system that the application is provided is described in detail, applied specific case herein the application's principle and execution mode are set forth, the explanation of above embodiment is just for helping to understand the application's method and core concept thereof; Simultaneously, for one of ordinary skill in the art, the thought according to the application, all will change in specific embodiments and applications, and in sum, this description should not be construed as the restriction to the application.
Claims (17)
1. a date storage method that is applied to distributed memory system, is characterized in that, comprising:
Determine the data category of data to be stored;
Search the default storage rule corresponding to data category of described data to be stored, described default storage rule comprises copy number and different stored position informations;
According to described copy number, generate the data trnascription of described data to be stored;
Described data trnascription is stored into respectively in the memory node that different stored position informations is corresponding.
2. method according to claim 1, is characterized in that, described data to be stored are specially file to be stored, and the data category of described definite data to be stored comprises:
Obtain the filename suffix of file to be stored;
According to described filename suffix, determine the data category of described file to be stored.
3. method according to claim 1, is characterized in that, described by described data trnascription store into respectively different stored position informations respectively corresponding memory node comprise:
Call the file system of described distributed memory system, described data trnascription is stored into respectively in memory node corresponding to different stored position information difference.
4. method according to claim 3, is characterized in that, the described file system of calling described distributed memory system stores respectively described data trnascription into different memory node corresponding to memory node address and comprises:
Determine the file system type of described distributed memory system;
Search the configuration interface function corresponding to described file system type of preservation;
By the described file system of described configuration interface function call, described data trnascription is stored into respectively in memory node corresponding to different memory node addresses.
5. method according to claim 1, is characterized in that, described by described data trnascription store into respectively different stored position informations respectively corresponding memory node comprise:
Determine described different memory node corresponding to stored position information difference, and the file system of each memory node application;
Search the configuration interface function corresponding to type difference of different file system;
By file system corresponding to described different configuration interface function calls, described data trnascription is stored into respectively in the memory node that different stored position informations are corresponding.
6. according to the described method of claim 1 ~ 5 any one, it is characterized in that, described by described data trnascription, store into respectively in the memory node that different stored position informations is corresponding after, described method also comprises:
When receiving data access request, determine the target data classification of request access according to described data access request;
Search target storage rule corresponding to described target data classification;
According to the stored position information in described target storage rule, determine the target memory node corresponding to different target data trnascription difference of described target data;
According to the action type of described data access request, process the target data copy in described each target memory node.
7. method according to claim 6, is characterized in that, the target data copy in described each memory node of described processing comprises:
Judge that whether described each memory node is effective memory node, obtains the first judged result;
According to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
8. method according to claim 7, is characterized in that, described action type is specially the Data Update operation, and after processing the target data copy in the effective memory node in described memory node, described method also comprises:
Whether the target data copy that judges described memory node is updated successfully, and obtains the second judged result;
According to described the second judged result, determine that the memory node be updated successfully is effective memory node, upgrading failed memory node is the invalid storage node.
9. the data-storage system be applied in distributed memory system, is characterized in that, comprising:
The data category determination module, for determining the data category of data to be stored;
The rule searching module, for default storage rule corresponding to data category of searching described data to be stored, described default storage rule comprises copy number and different stored position informations;
The copy generation module, for according to described copy number, generate the data trnascription of described data to be stored;
Data memory module, for storing respectively described data trnascription into memory node that different stored position informations is corresponding.
10. system according to claim 9, is characterized in that, described data category determination module comprises:
Acquisition module, for obtaining the filename suffix of file to be stored;
Classification is determined submodule, for according to described filename suffix, determines the data category of described file to be stored.
11. system according to claim 9, is characterized in that, described data memory module, specifically for calling the file system of described distributed memory system, stores described data trnascription respectively in the memory node that different stored position informations is corresponding.
12. system according to claim 11, is characterized in that, described data storage mould comprises:
The system type determination module, for determining the file system type of described distributed memory system;
The first function is searched module, for configuration interface function corresponding to described file system type of searching preservation;
The first sub module stored, for by the described file system of described configuration interface function call, store described data trnascription respectively in memory node corresponding to different memory node addresses.
13. system according to claim 9, is characterized in that, described data memory module comprises:
The node determination module, for determining described different memory node corresponding to stored position information difference, and the file system of each memory node application;
The second function is searched module, for searching the configuration interface function corresponding to type difference of different file system;
The second sub module stored, for by file system corresponding to described different configuration interface function calls, store described data trnascription respectively in the memory node that different stored position informations are corresponding.
14. according to the described method of claim 9 ~ 13 any one, it is characterized in that, described system also comprises:
Data demand module, for receiving data access request, determine the target data classification of request access according to described data access request;
Goal rule is searched module, for searching target storage rule corresponding to described target data classification;
The destination node determination module, according to the stored position information in described target storage rule, determine the memory node corresponding to different target data trnascription difference of described target data;
Processing module, for the action type according to described data access request, process the target data copy in described each target memory node.
15. system according to claim 14, is characterized in that, described processing module comprises:
Whether the first judge module, be effective memory node for judging described each memory node, obtains the first judged result;
Process submodule, for according to described the first judged result, when memory node is effective memory node, process the target data copy in described effective memory node.
16. system according to claim 14, is characterized in that, described action type is specially the Data Update operation, and described system also comprises:
Whether the second judge module, be updated successfully for the target data copy that judges described memory node, obtains the second judged result;
Node effect determination module, for according to described the second judged result, determine that the memory node be updated successfully is effective memory node, and upgrading failed memory node is the invalid storage node.
17. system according to claim 9, is characterized in that, described system also comprises:
Subscriber Interface Module SIM, for receiving user's default storage rule update request, upgrade described default storage rule, and preserve the default storage rule after upgrading.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012104411761A CN102938784A (en) | 2012-11-06 | 2012-11-06 | Method and system used for data storage and used in distributed storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012104411761A CN102938784A (en) | 2012-11-06 | 2012-11-06 | Method and system used for data storage and used in distributed storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102938784A true CN102938784A (en) | 2013-02-20 |
Family
ID=47697652
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012104411761A Pending CN102938784A (en) | 2012-11-06 | 2012-11-06 | Method and system used for data storage and used in distributed storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102938784A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103618769A (en) * | 2013-11-15 | 2014-03-05 | 华为技术有限公司 | Redundancy configuration method, system and related device |
| CN104035925A (en) * | 2013-03-04 | 2014-09-10 | 深圳市腾讯计算机系统有限公司 | Data storage method and device and storage system |
| CN104298726A (en) * | 2014-09-26 | 2015-01-21 | 惠州市亿能电子有限公司 | BMS data storage system and method based on database |
| CN104461779A (en) * | 2014-11-28 | 2015-03-25 | 华为技术有限公司 | Distributed data storage method, device and system |
| CN104881581A (en) * | 2015-05-28 | 2015-09-02 | 成都艺辰德迅科技有限公司 | IoT (Internet of Things) data high-efficiency analysis method |
| CN105025306A (en) * | 2015-06-30 | 2015-11-04 | 中国农业大学 | Video storage method and system based on virtual management distributed file system |
| WO2016065592A1 (en) * | 2014-10-30 | 2016-05-06 | 华为技术有限公司 | Method, device and apparatus for obtaining reliability configuration strategy |
| CN105608167A (en) * | 2015-12-18 | 2016-05-25 | 金蝶软件（中国）有限公司 | KDFS system-based file backup method and system |
| CN106227769A (en) * | 2016-07-15 | 2016-12-14 | 北京奇虎科技有限公司 | Date storage method and device |
| CN106484712A (en) * | 2015-08-27 | 2017-03-08 | 北京易车互联信息技术有限公司 | The date storage method of distributed file system and device |
| CN108121500A (en) * | 2016-11-30 | 2018-06-05 | 深圳市中兴微电子技术有限公司 | The access method and device of a kind of data |
| CN108388406A (en) * | 2018-01-10 | 2018-08-10 | 链家网（北京）科技有限公司 | Data processing method and device |
| CN108920095A (en) * | 2018-06-06 | 2018-11-30 | 深圳市脉山龙信息技术股份有限公司 | A kind of data store optimization method and apparatus based on CRUSH |
| CN109660597A (en) * | 2018-11-14 | 2019-04-19 | 武汉盛华伟业科技股份有限公司 | A kind of well site Remote Decision-making support system and method |
| CN109933289A (en) * | 2019-03-15 | 2019-06-25 | 深圳市网心科技有限公司 | A storage copy deployment method, system, electronic device and storage medium |
| CN110096301A (en) * | 2019-05-08 | 2019-08-06 | 深信服科技股份有限公司 | A kind of hot upgrade method of storage system, system and electronic equipment and storage medium |
| CN112050549A (en) * | 2017-05-31 | 2020-12-08 | 日立环球生活方案株式会社 | Household appliance system, household appliance diagnosis method and terminal device |
| CN112783445A (en) * | 2020-11-17 | 2021-05-11 | 北京旷视科技有限公司 | Data storage method, device, system, electronic equipment and readable storage medium |
| CN113835616A (en) * | 2020-06-23 | 2021-12-24 | 华为技术有限公司 | Applied data management method, system and computer device |
| CN115167773A (en) * | 2022-07-13 | 2022-10-11 | 谭乃星 | Data processing method and data processing device |
| CN117453153A (en) * | 2023-12-26 | 2024-01-26 | 柏科数据技术（深圳）股份有限公司 | File storage method, device, terminal and medium based on flush rule |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1954318A (en) * | 2004-05-20 | 2007-04-25 | 国际商业机器公司 | Methods, systems and programs for maintaining a namespace of filesets accessible to clients over a network |
| CN1952940A (en) * | 2006-11-24 | 2007-04-25 | 北京神舟航天软件技术有限公司 | Method for calling relational database-based database abstraction layer by application layer |
| CN101944124A (en) * | 2010-09-21 | 2011-01-12 | 卓望数码技术（深圳）有限公司 | Distributed file system management method, device and corresponding file system |
| CN102414692A (en) * | 2009-04-24 | 2012-04-11 | 微软公司 | Dynamic placement of replica data |
-
2012
- 2012-11-06 CN CN2012104411761A patent/CN102938784A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1954318A (en) * | 2004-05-20 | 2007-04-25 | 国际商业机器公司 | Methods, systems and programs for maintaining a namespace of filesets accessible to clients over a network |
| CN1952940A (en) * | 2006-11-24 | 2007-04-25 | 北京神舟航天软件技术有限公司 | Method for calling relational database-based database abstraction layer by application layer |
| CN102414692A (en) * | 2009-04-24 | 2012-04-11 | 微软公司 | Dynamic placement of replica data |
| CN101944124A (en) * | 2010-09-21 | 2011-01-12 | 卓望数码技术（深圳）有限公司 | Distributed file system management method, device and corresponding file system |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104035925A (en) * | 2013-03-04 | 2014-09-10 | 深圳市腾讯计算机系统有限公司 | Data storage method and device and storage system |
| CN104035925B (en) * | 2013-03-04 | 2018-07-17 | 深圳市腾讯计算机系统有限公司 | Date storage method, device and storage system |
| CN103618769A (en) * | 2013-11-15 | 2014-03-05 | 华为技术有限公司 | Redundancy configuration method, system and related device |
| CN104298726A (en) * | 2014-09-26 | 2015-01-21 | 惠州市亿能电子有限公司 | BMS data storage system and method based on database |
| CN104298726B (en) * | 2014-09-26 | 2018-07-03 | 惠州市亿能电子有限公司 | A kind of BMS data-storage systems and its method based on database |
| WO2016065592A1 (en) * | 2014-10-30 | 2016-05-06 | 华为技术有限公司 | Method, device and apparatus for obtaining reliability configuration strategy |
| CN104461779B (en) * | 2014-11-28 | 2018-02-23 | 华为技术有限公司 | A kind of storage method of distributed data, apparatus and system |
| CN104461779A (en) * | 2014-11-28 | 2015-03-25 | 华为技术有限公司 | Distributed data storage method, device and system |
| CN104881581A (en) * | 2015-05-28 | 2015-09-02 | 成都艺辰德迅科技有限公司 | IoT (Internet of Things) data high-efficiency analysis method |
| CN105025306A (en) * | 2015-06-30 | 2015-11-04 | 中国农业大学 | Video storage method and system based on virtual management distributed file system |
| CN105025306B (en) * | 2015-06-30 | 2018-05-11 | 中国农业大学 | Video storage method and system based on virtual management distributed file system |
| CN106484712A (en) * | 2015-08-27 | 2017-03-08 | 北京易车互联信息技术有限公司 | The date storage method of distributed file system and device |
| CN105608167B (en) * | 2015-12-18 | 2019-10-08 | 金蝶软件（中国）有限公司 | A kind of file backup method and system based on KDFS system |
| CN105608167A (en) * | 2015-12-18 | 2016-05-25 | 金蝶软件（中国）有限公司 | KDFS system-based file backup method and system |
| CN106227769A (en) * | 2016-07-15 | 2016-12-14 | 北京奇虎科技有限公司 | Date storage method and device |
| CN106227769B (en) * | 2016-07-15 | 2019-11-26 | 北京奇虎科技有限公司 | Date storage method and device |
| WO2018099106A1 (en) * | 2016-11-30 | 2018-06-07 | 深圳市中兴微电子技术有限公司 | Method and device for accessing data, and computer readable storage medium |
| CN108121500A (en) * | 2016-11-30 | 2018-06-05 | 深圳市中兴微电子技术有限公司 | The access method and device of a kind of data |
| CN112050549B (en) * | 2017-05-31 | 2022-06-14 | 日立环球生活方案株式会社 | Home appliance system, diagnostic method for home appliance, and terminal device |
| CN112050549A (en) * | 2017-05-31 | 2020-12-08 | 日立环球生活方案株式会社 | Household appliance system, household appliance diagnosis method and terminal device |
| CN108388406A (en) * | 2018-01-10 | 2018-08-10 | 链家网（北京）科技有限公司 | Data processing method and device |
| CN108388406B (en) * | 2018-01-10 | 2021-07-02 | 链家网（北京）科技有限公司 | Data processing method and device |
| CN108920095B (en) * | 2018-06-06 | 2021-06-29 | 深圳市脉山龙信息技术股份有限公司 | Data storage optimization method and device based on CRUSH |
| CN108920095A (en) * | 2018-06-06 | 2018-11-30 | 深圳市脉山龙信息技术股份有限公司 | A kind of data store optimization method and apparatus based on CRUSH |
| CN109660597A (en) * | 2018-11-14 | 2019-04-19 | 武汉盛华伟业科技股份有限公司 | A kind of well site Remote Decision-making support system and method |
| CN109933289A (en) * | 2019-03-15 | 2019-06-25 | 深圳市网心科技有限公司 | A storage copy deployment method, system, electronic device and storage medium |
| CN109933289B (en) * | 2019-03-15 | 2022-06-10 | 深圳市网心科技有限公司 | Storage copy deployment method and system, electronic equipment and storage medium |
| CN110096301A (en) * | 2019-05-08 | 2019-08-06 | 深信服科技股份有限公司 | A kind of hot upgrade method of storage system, system and electronic equipment and storage medium |
| CN113835616A (en) * | 2020-06-23 | 2021-12-24 | 华为技术有限公司 | Applied data management method, system and computer device |
| CN112783445A (en) * | 2020-11-17 | 2021-05-11 | 北京旷视科技有限公司 | Data storage method, device, system, electronic equipment and readable storage medium |
| CN115167773A (en) * | 2022-07-13 | 2022-10-11 | 谭乃星 | Data processing method and data processing device |
| CN117453153A (en) * | 2023-12-26 | 2024-01-26 | 柏科数据技术（深圳）股份有限公司 | File storage method, device, terminal and medium based on flush rule |
| CN117453153B (en) * | 2023-12-26 | 2024-04-09 | 柏科数据技术（深圳）股份有限公司 | File storage method, device, terminal and medium based on flush rule |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102938784A (en) | Method and system used for data storage and used in distributed storage system | |
| KR102240557B1 (en) | Method, device and system for storing data | |
| US10936423B2 (en) | Enhanced application write performance | |
| CN103959264B (en) | Using deduplication in a storage cloud to manage immutable redundant files | |
| US10127243B2 (en) | Fast recovery using self-describing replica files in a distributed storage system | |
| RU2598991C2 (en) | Data recovery client for moveable client data | |
| WO2019166940A2 (en) | Transactional operations in multi-master distributed data management systems | |
| US10909086B2 (en) | File lookup in a distributed file system | |
| CN110914814B (en) | Cognitive file and object management for distributed storage environments | |
| JP7431490B2 (en) | Data migration in hierarchical storage management systems | |
| CN111201763A (en) | Security based on file system content | |
| US20170270149A1 (en) | Database systems with re-ordered replicas and methods of accessing and backing up databases | |
| CN116848517A (en) | Cache indexing using data addresses based on data fingerprints | |
| CN111400350B (en) | Configuration data reading method, system, electronic device and storage medium | |
| CN109710454A (en) | A kind of cloud host snapshot method and device | |
| US9684668B1 (en) | Systems and methods for performing lookups on distributed deduplicated data systems | |
| US20140114918A1 (en) | Use of proxy objects for integration between a content management system and a case management system | |
| CN109558082B (en) | Distributed file system | |
| US11687416B2 (en) | Data backup optimization | |
| US10678813B2 (en) | Scaling quorum based replication systems | |
| US11416468B2 (en) | Active-active system index management | |
| CN108694102A (en) | A kind of data manipulation method, equipment, system and medium based on Nexus services | |
| Shabeera et al. | A novel approach for improving data locality of mapreduce applications in cloud environment through intelligent data placement | |
| CN112084141B (en) | A method, device, equipment and medium for expanding full-text search system | |
| US11121981B1 (en) | Optimistically granting permission to host computing resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130220 |