CN101814042A - Data asynchronous replication method and device thereof - Google Patents
Data asynchronous replication method and device thereof Download PDFInfo
- Publication number
- CN101814042A CN101814042A CN201010120004.5A CN201010120004A CN101814042A CN 101814042 A CN101814042 A CN 101814042A CN 201010120004 A CN201010120004 A CN 201010120004A CN 101814042 A CN101814042 A CN 101814042A
- Authority
- CN
- China
- Prior art keywords
- write request
- cycle
- data block
- data
- cycle write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000010076 replication Effects 0.000 title claims abstract description 73
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000000737 periodic effect Effects 0.000 claims 1
- 238000011084 recovery Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 19
- 230000005540 biological transmission Effects 0.000 description 5
- 230000002950 deficient Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003760 hair shine Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明实施例提供一种数据异步复制方法及装置。该数据异步复制方法包括：在T周期接收到访问本地源卷的写请求时，确定T周期写请求访问的数据块中、与T-1周期写请求访问的数据块中，是否存在相同的数据块；若存在相同的数据块，从本地源卷中将相同的数据块中数据复制到本地快照区；将T周期写请求的数据写入本地源卷，并在T周期写请求的数据块访问记录信息中记录T周期写请求访问的数据块的标识；在T周期，根据T-1周期写请求的数据块访问记录信息和T周期写请求的数据块冲突记录信息，从本地源卷及本地快照区，读取T-1周期写请求的数据并复制到备份存储阵列。本发明实施例提高了本地应用系统的性能，减小了容灾系统的RPO。
Embodiments of the present invention provide a data asynchronous replication method and device. The data asynchronous replication method includes: when a write request for accessing a local source volume is received in a T period, determining whether there is the same data in the data block accessed by the T period write request and in the data block accessed by the T-1 period write request block; if the same data block exists, copy the data in the same data block from the local source volume to the local snapshot area; write the data requested by the T cycle write to the local source volume, and access the data block requested by the T cycle write The record information records the identification of the data block accessed by the T cycle write request; in the T cycle, according to the data block access record information of the T-1 cycle write request and the T cycle write request data block conflict record information, from the local source volume and the local In the snapshot area, the data requested by the T-1 period write is read and copied to the backup storage array. The embodiment of the present invention improves the performance of the local application system and reduces the RPO of the disaster recovery system.
Description
Technical field
The present invention relates to field of computer technology, relate in particular to a kind of data asynchronous replication method and device.
Background technology
Disaster tolerance system generally comprises the local production center and long-distance disaster center, is provided with the back-up storage array that data in the local storage array of the production center are backed up in the long-distance disaster in the heart.When disaster tolerance system runs into catastrophic event, utilize geographic separation to come local storage array to be recovered by data in the back-up storage array.Wherein, performance index estimating disaster tolerance system are (RecoveryPoint Object is hereinafter to be referred as RPO), after RPO is illustrated in the disaster generation, and the length of disaster tolerance system obliterated data time.
Data backup mode between local storage array and the back-up storage array comprises synchronization replication pattern and asynchronous replication pattern.In the asynchronous replication pattern, when local storage array receives local application system write request, earlier the data of write request are write the source book of local storage array, and to local application system return finish the write request acknowledge message after, the data with write request send to the back-up storage array again.The asynchronous replication pattern with respect to the synchronization replication pattern, has reduced the influence of write request to local application system performance, and data transmission distance is unrestricted substantially.At present, data asynchronous remote transmission implementation method is based on the snapping technique realization.Snapshot is periodically done to the source book of local storage array in the production center, obtains the snapshot district of source book; And periodically the data in snapshot district are copied to back-up storage array in the far-end disaster recovery center.
The inventor is in the process that realizes the embodiment of the invention, find that there is following defective in prior art: existing data asynchronous replication method has increased the time delay of local application system write request, thereby reduced the performance of local application system, caused the RPO of disaster tolerance system to increase.
Summary of the invention
The embodiment of the invention provides a kind of data asynchronous replication method and device, in order to the long defective of time delay that solves local application system write request in the prior art, has improved the performance of local application system, has reduced the RPO of disaster tolerance system.
The embodiment of the invention provides a kind of data asynchronous replication method, comprising:
When the T cycle receives the write request of the local source book of visit, sign according to the data block of the data block Visitor Logs information of T-1 cycle write request and the visit of T cycle write request, determine in the data block of T cycle write request visit, with the data block of T-1 cycle write request visit in, whether have identical data block; The described T-1 cycle is the last one-period in described T cycle; The data block Visitor Logs information of described T-1 cycle write request is used to write down the sign of the data block of T-1 cycle write request visit;
If there is identical data block, data copy to local express according to the district in will be described from the described local source book identical data block that writes in the T-1 cycle, and write down the sign of described identical data block in the data block conflict recorded information of T cycle write request; The data block conflict recorded information of described T cycle write request is used for writing down the data block of T cycle write request visit and the sign of the data block identical block that described T-1 cycle write request is visited;
The data of T cycle write request are write described local source book, and the sign of the data block that the described T of record cycle write request is visited in the data block Visitor Logs information of T cycle write request;
In the described T cycle, according to the data block Visitor Logs information of described T-1 cycle write request and the data block conflict recorded information of described T cycle write request, shine the district from described local source book and described local express, read the data of described T-1 cycle write request and copy to the back-up storage array.
The embodiment of the invention provides a kind of data asynchronous replication device, comprising:
The conflict determination module, be used for when the T cycle receives the write request of the local source book of visit, sign according to the data block of the data block Visitor Logs information of T-1 cycle write request and the visit of T cycle write request, determine in the data block of T cycle write request visit, with the data block of T-1 cycle write request visit in, whether have identical data block; The described T-1 cycle is the last one-period in described T cycle;
Write preceding replication module, be used for if there is identical data block, from described local source book, data in the described identical data block that writes in the T-1 cycle are copied to local express according to the district, and in the data block conflict recorded information of T cycle write request the sign of the described identical data block of record;
The write data module is used for the data of T cycle write request are write described local source book, and the sign of the data block that record T cycle write request is visited in the data block Visitor Logs information of T cycle write request;
First replication module, be used for the cycle at described T, according to the data block Visitor Logs information of described T-1 cycle write request and the data block conflict recorded information of described T cycle write request, shine the district from described local source book and described local express, read the data of described T-1 cycle write request and copy to the back-up storage array.
Embodiment of the invention data asynchronous replication method and device, before the write request of handling the T cycle, pass through the data block visit information of T-1 cycle write request earlier, determine that there are the conflicting data piece in current period write request and T-1 cycle write request, only data in the conflicting data piece are write preceding replicate run.Then, the data with T cycle write request write in the local source book again, the data block visit information of record T cycle write request.When the back-up storage array of far-end duplicates the data of T-1 cycle write request, read data in the colliding data piece according to the data block conflicting information of T cycle write request; And, from local source book, read other data of T-1 cycle write request according to the data block visit information of T-1 cycle write request.Because the embodiment of the invention only exists T cycle write request and T-1 cycle write request in the conflicting data piece data to write preceding replicate run, relatively the total data of T-1 cycle write request is write preceding replicate run, saved the running time, therefore shortened the time delay of current period write request, thereby improved the performance of local application system, reduced the RPO of disaster tolerance system.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do one to the accompanying drawing of required use in embodiment or the description of the Prior Art below introduces simply, apparently, accompanying drawing in describing below is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
The data asynchronous replication method flow diagram that Fig. 1 a provides for first embodiment of the invention;
Duplicate the process flow diagram of the data of T-1 cycle write request to the back-up storage array in the T cycle in the data asynchronous replication method that Fig. 1 b provides for first embodiment of the invention;
The processing flow chart of T cycle write request in the data asynchronous replication method that Fig. 2 a provides for second embodiment of the invention;
Duplicate the process flow diagram of the data of T-1 cycle write request to the back-up storage array in the T cycle in the data asynchronous replication method that Fig. 2 b provides for second embodiment of the invention;
Begin to duplicate the process flow diagram of the data of T-1 cycle write request in the data asynchronous replication method that Fig. 3 provides for third embodiment of the invention to the back-up storage array from the T-1 cycle;
Begin to duplicate the process flow diagram of the data of T cycle write request in the data asynchronous replication method that Fig. 4 provides for fourth embodiment of the invention to the back-up storage array from the T cycle;
The processing flow chart of the data asynchronous replication method T cycle write request that Fig. 5 a provides for fifth embodiment of the invention;
Duplicate the process flow diagram of the data of T cycle write request to the back-up storage array in the data asynchronous replication method that Fig. 5 b provides for fifth embodiment of the invention;
The data asynchronous replication apparatus structure synoptic diagram that Fig. 6 provides for sixth embodiment of the invention;
The data asynchronous replication apparatus structure synoptic diagram that Fig. 7 provides for seventh embodiment of the invention;
The data asynchronous replication apparatus structure synoptic diagram that Fig. 8 provides for eighth embodiment of the invention;
The data asynchronous replication apparatus structure synoptic diagram that Fig. 9 provides for ninth embodiment of the invention.
Embodiment
For the purpose, technical scheme and the advantage that make the embodiment of the invention clearer, below in conjunction with the accompanying drawing in the embodiment of the invention, technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that is obtained under the creative work prerequisite.
The inventor finds that the snapshot district of source book has been created in the production center in the process that realizes the embodiment of the invention.When receiving the write request of local application system, write in data before the source book of local storage array write request, working method according to snapshot, need earlier the legacy data in the source book, in the write request corresponding data piece all to be copied to the snapshot district of source book, and then the data of write request are write source book.Therefore,, can produce the extra preceding replicate run of writing, thereby the time delay that has increased write request increases for the write request of local application system.
The data asynchronous replication method flow diagram that Fig. 1 a provides for first embodiment of the invention, as shown in Figure 1, present embodiment comprises:
Step 11: when the T cycle receives the write request of the local source book of visit, sign according to the data block visit of the data block Visitor Logs information of T-1 cycle write request and T cycle write request, determine in the data block of T cycle write request visit, with the data block of T-1 cycle write request visit in, whether have identical data block;
Wherein, the T-1 cycle is the last one-period in T cycle; The data block visit information of T-1 cycle write request is used to write down the sign of the data block of T-1 cycle write request visit;
It is to be noted: data block Visitor Logs information, and follow-up data block conflict recorded information specifically can be the binary bitmap that records the data block sign, each corresponding data block in the visit bitmap.The embodiment of the invention all is example explanation data block Visitor Logs information and data block conflict recorded information with the bitmap.Wherein, data block Visitor Logs information is the visit bitmap, and data block conflict recorded information is the conflict bitmap.
When the T cycle receives the write request of that local application system sends, the local source book of visit, the visit bitmap when reading T-1 cycle write request and visiting local source book.The visit bitmap of T-1 cycle write request is used to write down the sign of the data block of T-1 cycle write request visit.For example, the data block of T-1 cycle write request visit is 1,2,10 and 11, recording data blocks 1,2,10 and 11 in the visit bitmap, can with in the visit bitmap the 1st, the 2nd, the 10th and the 11st be labeled as 1 respectively, all the other positions are 0.
In the data block of visiting with T cycle write request in the data block of determining the visit of T-1 cycle write request, whether there is identical data block, need the data block sign of corresponding data position in the visit bitmap of T-1 cycle write request and the visit of T cycle write request is compared, to find out identical data block.
Step 12: if there is identical data block, data copy to local express according to the district in the data block that writes in the T cycle that will be identical from local source book, and write down the sign of identical data block in the data block conflict recorded information of T cycle write request;
Wherein, the data block of T cycle write request conflict recorded information, be used for writing down the visit of T-1 cycle write request data block, with data block that T cycle write request is visited in the sign of identical data block.
When having identical data block in the write request of determining the T cycle and the data block that the write request in T-1 cycle is visited, before the data with the write request in T cycle write local source book, with data in the data block identical in the local source book, copy to local express according to the district earlier from local source book.Be the preceding replicate run of writing in the present embodiment, only T-1 cycle and T cycle write request existed that data copy to local express according to distinguishing in the conflicting data piece.
With T-1 cycle and T cycle write request exist in the conflicting data piece data copy to local express according to the district after, in the conflict bitmap of T cycle write request in the data block of record T-1 cycle write request visit, with data block that T cycle write request is visited in identical data block.
Step 13: the data of T cycle write request are write local source book, and the sign of the data block that record T cycle write request is visited in the data block Visitor Logs information of T cycle write request;
Step 14: according to the data block Visitor Logs information of T-1 cycle write request and the data block conflict recorded information of T cycle write request, shine the district, read the data of T-1 cycle write request and copy to the back-up storage array from local source book and local express.
Enter T week after date, need the data of T-1 cycle write request are copied to the back-up storage array.Because the T cycle may visit data block identical in the local source book with the T-1 cycle, promptly in step 13, T-1 cycle and T cycle write request have been existed in the conflicting data piece data to copy to the local express photograph and distinguished.Therefore, need conflict bitmap according to T cycle write request, from local express according to reading the T-1 cycle the district and there are data in the conflicting data piece in T cycle write request.Other data block (data block in the data block of T-1 cycle write request visit, except that identical data block) for the T-1 cycle access needs to read from local source book, and copies to the back-up storage array.
Duplicate the process flow diagram of the data of T-1 cycle write request to the back-up storage array in the T cycle in the data asynchronous replication method that Fig. 1 b provides for first embodiment of the invention.Shown in Fig. 1 b, step 14 specifically can be:
Step 141:, determine the data block of T-1 cycle write request visit according to the visit bitmap of T-1 cycle write request;
For example, record data block 2,7,11 and 13 in the visit bitmap of T-1 cycle write request, then data block 2,7,11 and 13 is the data block of T-1 cycle write request visit.
Step 142: according to the conflict bitmap of T cycle write request, determine in the data block of T-1 cycle write request visit, with the data block of T cycle write request visit in, whether have identical data block;
For example, in the conflict bitmap of T cycle write request, record data block 13, then illustrate in the data block of T cycle write request visit, with the data block of T-1 cycle write request visit in, have identical data block 13, promptly data have write local express according in the district in the data block 13 of T-1 cycle write request.
Step 143: if there is identical data block, then according to distinguishing, read data in the identical data block, copy to the back-up storage array from local express;
As above, when from local express photograph district, reading T-1 cycle write request, the data in the data block 13.
Step 144: from local source book, read the data of the T-1 cycle write request the data in identical data block, copy to the back-up storage array.
Because, there is not recording data blocks 2,7 and 11 in the visit bitmap in T cycle, the data in the data block 2,7 and 11 in the local source book during cycle at T then are described, also be the data of T-1 cycle write request, therefore distinguish data in the read block 2,7 and 11 from local source book.
In the present embodiment, the time span in T cycle is set short more, then the number of T write request in the cycle is few more, then the number of times of the data block conflict of two interior write requests of cycle is few more, and then the number of times that the data block of conflict is duplicated before writing is also few more, thereby make under the high more situation of period frequency the performance impact when having reduced to local application system write request.
Embodiment of the invention data asynchronous replication method, before the write request of handling the T cycle, by the data block visit information of T-1 cycle write request, determine that there are the conflicting data piece in current period write request and T-1 cycle write request earlier, only data in the conflicting data piece are write preceding replicate run.Then, the data with T cycle write request write in the local source book again, the data block visit information of record T cycle write request.When the back-up storage array of far-end duplicates the data of T-1 cycle write request, read data in the colliding data piece according to the data block conflicting information of T cycle write request; And, from local source book, read other data of T-1 cycle write request according to the data block visit information of T-1 cycle write request.Because the embodiment of the invention only exists T cycle write request and T-1 cycle write request in the conflicting data piece data to write preceding replicate run, relatively the total data of T-1 cycle write request is write preceding replicate run, saved the running time, therefore shortened the time delay of current period write request, thereby improved the performance of local application system, reduced the RPO of disaster tolerance system.
Duplicate the process flow diagram of the data of T-1 cycle write request to the back-up storage array in the T cycle in the data asynchronous replication method that the processing flow chart of T cycle write request in the data asynchronous replication method that Fig. 2 a provides for second embodiment of the invention, Fig. 2 b provide for second embodiment of the invention.Shown in Fig. 2 a and Fig. 2 b, present embodiment comprises:
Step 21:, receive the write request of the local source book of visit in the T cycle;
Step 22: the visit bitmap that reads T-1 cycle write request;
For example, in the T cycle, write request has been visited the data block 2,7,11 and 13 in the local source book.Data block 2,7,11 and 13 has been recorded in the visit bitmap of T-1 cycle write request.
Whether record a data block of T cycle write request visit in the visit bitmap of step 23:T-1 cycle write request;
If record a data block of T cycle write request visit in the visit bitmap of T-1 cycle write request, this data block that the visit of T cycle write request is described is identical with a data block of T-1 cycle write request visit, and promptly the T cycle conflicts with the data block existence of T-1 cycle write request visit.Before the data with T cycle write request write local source book, need earlier data in the conflicting data piece to be write preceding replicate run, promptly from local source book, copy to local express according in the district, with the T cycle when far-end back-up storage array sends the data of T-1 cycle write request, according to the district, read the T cycle and there are data in the conflicting data piece in T-1 cycle write request from local express.Therefore, if there is conflict, then first execution in step 24 is to step 25, and execution in step 26 is to step 27 again; Otherwise directly execution in step 26 is to step 27.
For example, in the T cycle, write request will be visited the data block 1,12,13 and 16 in the local source book.Recorded data block 13 in the visit bitmap of T-1 cycle write request, then data block 13 is that T cycle and the visit of T-1 cycle write request exist the conflicting data piece.
Step 24: data in this data block in the local source book are write preceding replicate run;
Replicate run is about to data in the data block of this data bit correspondence in the local source book before writing, and is written to local express according in the district.
Step 25: this data block of record in the conflict bitmap of T cycle write request;
After the replicate run, write down this data block in the conflict bitmap of T cycle write request before writing, the data block of representing this data bit correspondence is that T cycle and T-1 cycle write request exist the conflicting data piece.For example, with conflicting data piece 13 copy to local express according to the district, and in the conflict bitmap of T cycle write request recording data blocks 13.
Step 26: the data of T cycle write request are write in this data block of local source book;
From the above, there is the conflicting data piece in data block 13 for T cycle and the visit of T-1 cycle write request, and conflicting data piece 13 is copied to local express according to after distinguishing, and the data of T cycle write request are write in the data block 13 of local source book.
Step 27: this data block of record in the visit bitmap of T cycle write request;
After the write request data of the data block correspondence of T cycle write request visit are write source book, in the visit bitmap of T cycle write request, this data block of record T cycle write request visit.
After the execution in step 27, all write in the local source book if judge the data of T cycle write request, then execution in step 28; Otherwise, return step 23 and continue to carry out, all write in the local source book until the data of T cycle write request.
Step 28:, send the data of T-1 cycle write request to the back-up storage array in the T cycle.
Wherein, duplicate the detailed process of the data of T-1 cycle write request to the back-up storage array, shown in Fig. 2 b:
Step 281 a: data block that writes down in the visit bitmap of visit T-1 cycle write request;
For example, record data block 2,7,11 and 13 in the visit bitmap of T-1 cycle write request, promptly data block 2,7,11 and 13 is the data block of T-1 cycle write request visit, reads a data block 2 or 7 or 11 or 13 from the visit bitmap of T-1 cycle write request.
Step 282: in the conflict bitmap of T cycle write request, whether record this data block;
In the conflict bitmap of T cycle write request, whether record this data block, promptly determine in the data block of T-1 cycle write request visit, with the data block of T cycle write request visit in, whether all have this data block.If record this data block in the conflict bitmap of T cycle write request, illustrate that T-1 cycle write request also visited this data block, then execution in step 283, otherwise execution in step 284.
For example, in the conflict bitmap of T cycle write request, record data block 13, then illustrate in the data block of T cycle write request visit, with the data block of T-1 cycle write request visit in, have identical data block 13.
Step 283:, according to distinguishing, read data in this data block from local express if record this data block;
As above, in the conflict bitmap of T cycle write request, record data block 13, in step 24, data in the data block 13 of T-1 cycle write request visit are write local express according in the district so, need from local express according to data in the read block 13 district.
Step 284:, from local source book, read data in this data block if do not write down this data block;
As above, in the conflict bitmap of T cycle write request, there is not recording data blocks 2,7 and 11, the data in the data block 2,7 and 11 in the local source book during cycle at T then are described, also be the data of T-1 cycle write request, then distinguish data in the read block 2,7 and 11 from local source book.
Step 285: data in this data block are copied to the back-up storage array.
Will be from local source book or local express according to the data that read in the district, copy to the back-up storage array.
Repeated execution of steps 281 is to step 285, and the data block that writes down in the visit bitmap of T cycle write request was all visited, and the data of T-1 cycle write request are all copied to the back-up storage array, so that the data in the local source book are backed up.
Present embodiment data asynchronous replication method, before the write request of handling the T cycle, by the visit bitmap of T-1 cycle write request, determine that there are the conflicting data piece in T cycle write request and T-1 cycle write request earlier, only data in the conflicting data piece are write preceding replicate run.Then, the data with T cycle write request write in the local source book again, the visit bitmap of record T cycle write request.At T in the cycle, when the back-up storage array of far-end sends the data of T-1 cycle write request, read data in the colliding data piece according to the conflict bitmap of T cycle write request; And, from local source book, read other data of T-1 cycle write request according to the visit bitmap of T-1 cycle write request.Because present embodiment only exists T cycle write request and T-1 cycle write request in the conflicting data piece data to write preceding replicate run, relatively the total data of T-1 cycle write request is write preceding replicate run, saved the running time, therefore shorten the time delay of T cycle write request, thereby improved the performance of local application system.
Begin to duplicate the process flow diagram of the data of T-1 cycle write request in the data asynchronous replication method that Fig. 3 provides for third embodiment of the invention to the back-up storage array from the T-1 cycle.In first embodiment and second embodiment, enter T week after date, the data of T-1 cycle write request are duplicated in beginning to the back-up storage array.And in the present embodiment, the data of T-1 cycle write request are all write local source book after, from the T-1 cycle, just duplicate the data of T-1 cycle write request to the back-up storage array.If the total data of T-1 cycle write request is not copied to the back-up storage array,, continue to duplicate data remaining, T-1 cycle write request to the back-up storage array entering T week after date in the T-1 cycle.
Therefore, as shown in Figure 3, before the step 11 or the step 21 among second embodiment of first embodiment, also comprise step 10A and step 10B; Step 14 can be step 14A.Specific as follows:
Since the T-1 cycle when the back-up storage array duplicates the data of T-1 cycle write request.At this moment, T cycle write request does not also take place, and promptly the data of T-1 cycle write request all are stored in the local source book.Therefore, directly according to the data block Visitor Logs information of T-1 cycle write request, for example, the visit bitmap of T-1 cycle write request reads the data of T-1 cycle write request from local source book.
Might the total data of T-1 cycle write request not copied to the back-up storage array in the T-1 cycle,, also continue to duplicate data remaining, T-1 cycle write request to the back-up storage array entering T week after date.Therefore, the T-1 cycle when the back-up storage array sends the data of T-1 cycle write request, need be in the data block Visitor Logs information of T-1 cycle write request, remove the sign that has copied to data block in the back-up storage array, with the T cycle when the back-up storage array duplicates remaining data block, the conflicting information of record and remaining data piece only in the data block of T cycle write request conflict recorded information.
Step 10B specifically can comprise: a; In the data block Visitor Logs information of T-1 cycle write request, definite sign that has copied to the data block of back-up storage array; B; In the data block Visitor Logs information of T-1 cycle write request, remove the sign of the data block that has copied to the back-up storage array.
Wherein, the remaining data piece of T cycle write request visit does not also copy to the data block of back-up storage array in the data block for the visit of T cycle write request.
Present embodiment data asynchronous replication method, after the data of T-1 cycle write request write in the local source book, promptly begin to duplicate the data of T-1 cycle write request, rather than wait for and to enter T week after date, duplicate the data of T-1 cycle write request again to the back-up storage array to the back-up storage array.Therefore, present embodiment in first embodiment and second embodiment, has reduced the number of times of the data block conflict of two adjacent periods write requests, has further shortened the time delay of write request thus, and then has improved the performance of local application system to a greater extent.
Begin to duplicate the process flow diagram of the data of T cycle write request in the data asynchronous replication method that Fig. 4 provides for fourth embodiment of the invention to the back-up storage array from the T cycle.In the present embodiment, similar to the method for the 3rd embodiment, after the data of current period write request all write local source book, begin the data that the back-up storage array duplicates the current period write request from current period, the difference of the embodiment of the invention and the 3rd embodiment is, the current period of the 3rd embodiment is the T-1 cycle, and present embodiment is made as the T cycle in order more clearly to describe the relation of copy step and step before with current period.For further specifying the reason that the embodiment of the invention has effectively reduced " number of times of the data block conflict of two adjacent periods write requests ", the method of " after the data of T cycle write request all write local source book, from T cycle begin the data that the back-up storage array duplicates T cycle write request " is described in the present embodiment.
As shown in Figure 4, present embodiment comprises:
Step 41 in the present embodiment is to step 43, and is similar to step 13 to step 11 among first embodiment.The difference of the present embodiment and first embodiment is, among first embodiment, entering T week after date, just beginning to duplicate the data of T-1 cycle write request to the back-up storage array, thereby can know by inference and entering T+1 week after date, just begin to duplicate the data of T cycle write request to the back-up storage array; And in the present embodiment, in step 43, write the data of T cycle write request in the local source book after, promptly begin to duplicate the data of T cycle write request to the back-up storage array from the T cycle.
Step 41: when the T cycle receives the write request of the local source book of visit, sign according to the data block of the data block Visitor Logs information of T-1 cycle write request and the visit of T cycle write request, determine in the data block of T cycle write request visit, with the data block of T-1 cycle write request visit in, whether have identical data block;
When the T cycle receives the write request of that local application system sends, the local source book of visit, the visit bitmap when reading T-1 cycle write request and visiting local source book.The visit bitmap of T-1 cycle write request is used to write down the sign of the data block of T-1 cycle write request visit.For example, the data block of T-1 cycle write request visit is 1,2,10 and 11, recording data blocks 1,2,10 and 11 in the visit bitmap, can with in the visit bitmap the 1st, the 2nd, the 10th and the 11st be labeled as 1 respectively, all the other positions are 0.
For in the data block of determining the visit of T-1 cycle write request with the data block of T cycle write request in, whether there is identical data block, need the data block sign of corresponding data position in the visit bitmap of T-1 cycle write request and the visit of T cycle write request is compared, to find out identical data block.
Step 42: if there is identical data block, from local source book, data in the identical data block are copied to local express according to the district, and the sign of the data block that record is identical in the data block conflicting information of T cycle write request;
When having identical data block in the write request of determining the T cycle and the data block that the write request in T-1 cycle is visited, before the data with the write request in T cycle write local source book, with data in the data block identical in the local source book, the local express that copies to local source book from local source book is according to the district earlier.Be the preceding replicate run of writing in the present embodiment, only T-1 cycle and T cycle write request existed that data copy to local express according to distinguishing in the conflicting data piece.
With T-1 cycle and T cycle write request exist in the conflicting data piece data copy to local express according to the district after, in the conflict bitmap of T cycle write request in the data block of record T-1 cycle write request visit, with data block that T cycle write request is visited in identical data block.
Step 43: the data of T cycle write request are write local source book, and the sign of the data block that record T cycle write request is visited in the data block visit information of T cycle write request;
Step 44: after the data of T cycle write request write local source book, data block visit information according to T cycle write request, perhaps according to the data block visit information of T cycle write request and the data block conflicting information of T+1 cycle write request, shine the district from local source book and local express, the data that read T cycle write request copy to the back-up storage array, and remove the sign of the data block that has copied to the back-up storage array in the data block visit information of T cycle write request.
First kind of situation is: the data in the T cycle to back-up storage array transmission T cycle write request promptly after the write request in T cycle is finished dealing with, begin the data of this write request are transmitted.
If in the T cycle, according to the visit bitmap of T cycle write request, the data that read T cycle write request from local source book copy to the back-up storage array, and remove the record of the data block that has copied to the back-up storage array in the visit bitmap of T cycle write request.
For example, at T in the cycle, after writing the data (T cycle write request has been visited data block 2,7,11,13 and 16 in the local source book) of T cycle write request visit in the local source book, visit bitmap according to T cycle write request, beginning is to the data of back-up storage array transmission T cycle write request, data in the promptly local source book data block 2,7,11,13 and 16.
Second kind of situation is: if in the T cycle, all data with T cycle write request all do not copy to the back-up storage array, then enter all after dates in next period T+1 in T cycle, continue the data of remaining T cycle write request are copied to the back-up storage array.
If in the T+1 cycle, according to the visit bitmap of T cycle write request and the conflict bitmap of T+1 cycle write request, shine the district from local source book and local express, the data that read T cycle write request copy to the back-up storage array, and remove the record of the data block that has copied to the back-up storage array in the visit bitmap of T cycle write request.
From the above, in the cycle, if sent data in local source book data block 2,7 and 11, then after the T+1 cycle enters, continue the data with T remaining write request of cycle at T, promptly data in the data block 13 and 16 copy to the back-up storage array.And visited local source book data block 1,11,12 and 16 in the write request of T+1 during the cycle, wherein data block 11,16 is: T cycle and T+1 cycle write request conflicting data piece.And at T during the cycle, data have copied to and have led the ground storage array in the local source book data block 11, when therefore duplicating the data in T cycle in the T+1 cycle, only from local express according to data in the read block 16 district, the data from local source book in the read block 13.
In the present embodiment, the time span in T cycle is set short more, then the number of T write request in the cycle is few more, then the number of times of the data block conflict of two interior write requests of cycle is few more, and then the number of times that the data block of conflict is duplicated before writing is also few more, thereby make under the high more situation of period frequency the performance impact when having reduced to local application system write request.
Present embodiment data asynchronous replication method, after the data of T cycle write request write in the local source book, promptly begin to duplicate the data of T cycle write request, rather than wait for and to enter T+1 week after date, duplicate the data of T cycle write request again to the back-up storage array to the back-up storage array from the T cycle.Therefore, present embodiment in first embodiment and second embodiment, has reduced the number of times of the data block conflict of two adjacent periods write requests, has further shortened the time delay of write request thus, and then has improved the performance of local application system to a greater extent.
Duplicate the process flow diagram of the data of T cycle write request to the back-up storage array in the data asynchronous replication method that the processing flow chart of T cycle write request in the data asynchronous replication method that Fig. 5 a provides for fifth embodiment of the invention, Fig. 5 b provide for fifth embodiment of the invention.Shown in Fig. 5 a and Fig. 5 b, present embodiment comprises:
Following steps 51 to step 57 is the treatment scheme of T cycle write request, and is similar to step 27 to step 21 among Fig. 2 a, can be referring to the description of the corresponding embodiment of Fig. 2 a.
Step 51:, receive the write request of the local source book of visit in the T cycle;
Step 52: the visit bitmap that reads T-1 cycle write request;
Whether record a data block of T cycle write request visit in the visit bitmap of step 53:T-1 cycle write request;
If record a data block of current period write request visit in the visit bitmap of T-1 cycle write request, illustrate that current period conflicts with the data block existence of T-1 cycle write request visit.If there is conflict, then first execution in step 54 is to step 55, and execution in step 56 is to step 57 again; Otherwise directly execution in step 56 is to step 57.
Step 54: data in the data block of this data bit correspondence in the local source book are write preceding replicate run;
Step 55: this data block of record in the conflict bitmap of T cycle write request;
Step 56: the data of T cycle write request are write in this data block of local source book;
For example, there is the conflicting data piece in data block 13 for T cycle and the visit of T-1 cycle write request, and conflicting data piece 13 is copied to local express according to after distinguishing, and the data of T cycle write request are write in the data block 13 of local source book.
Step 57: this data block of record in the visit bitmap of T cycle write request;
After the execution in step 57, judge whether the data of T cycle write request all write in the local source book, continue to carry out that all write in the local source book until the data of T cycle write request, then execution in step 58 otherwise return step 53.
Step 58: in the visit bitmap of T cycle write request, record T cycle write request visit data block after, duplicate the data of T cycle write request to the back-up storage array.
Particularly, handle the write request of current period, and to back-up storage array copy data, these two operations can be carried out simultaneously; That is: step 51 can be finished separately by two threads respectively with step 58 to step 57.Wherein, duplicate the detailed process of the data of sending T cycle write request to the back-up storage array, shown in Fig. 5 b:
Step 580: read a data block in the visit bitmap of T cycle write request;
Step 581: judge whether be in the T cycle this moment;
Be in the T cycle if judge in the step 581, then execution in step 582 this moment.
Step 582: from local source book, read data in this data block;
As if be in T in the cycle when the back-up storage array transmits data, owing to also the data of T+1 cycle write request are not write in the local source book this moment, therefore the data of T cycle write request all are stored in the local source book at this moment.Therefore, in T cycle during to the data of strange land transmission T cycle write request, the data block of the T cycle write request visit of directly writing down in the visit bitmap according to T cycle write request reads data in this data block from local source book.
In the T cycle, from local source book, read in this data block after the data, elder generation's execution in step 583: data in this data block are copied to the back-up storage array, and execution in step 586 again: remove this data block in the visit bitmap of T cycle write request, execution in step 587 then.
Step 587: whether the data of judging T cycle write request all duplicate, if then duplicate end, continue to carry out otherwise return step 580.
If in step 581, judge and be in the T+1 cycle this moment, then execution in step 584.
Step 584: judge in the conflict bitmap of T+1 cycle write request whether record this data block;
The T+1 cycle when the back-up storage array sends the data of T cycle write request since this moment T+1 cycle write request if visited the data block that T cycle write request was visited, then can in the conflict bitmap of T+1 cycle write request, write down this data block.Therefore, need conflict bitmap, judge whether data block transmitted has write local express according to the district from local source book according to T+1 cycle write request.
If step 584 is judged: do not write down this data block in the conflict bitmap of T+1 cycle write request, then execution in step 582: read data in this data block from local source book;
If step 584 is judged: if record this data block in the conflict bitmap of T+1 cycle write request, then execution in step 585.
Step 585: according to distinguishing, read data in this data block from local express;
After the execution in step 585, first execution in step 583 back execution in step 586.
Step 586: in the visit bitmap of T cycle write request, remove this data block.
Data in this data block are copied to after the back-up storage array, in the visit bitmap of T cycle write request, remove this data block, so that at T+1 in the cycle,, directly data are write this data block if when T+1 cycle write request has access to this data block.The data of the T cycle write request of this data block store have copied to the back-up storage array and have backed up in the local source book, therefore, need not write down this data block in the conflict bitmap of T+1 cycle write request.
Step 587: whether the data of judging T cycle write request all duplicate.
In the such scheme, after execution in step 585 or step 586, execution in step 583, step 586 and step 587 successively.If step 587 is judged the data of T cycle write request and all duplicated, then duplicate end; Otherwise returning step 580 continues to carry out.
Embodiment of the invention data asynchronous replication method, before the write request of handling the T cycle, by the data block visit information of T-1 cycle write request, determine that there are the conflicting data piece in current period write request and T-1 cycle write request earlier, only data in the conflicting data piece are write preceding replicate run.Then, the data with T cycle write request write in the local source book again, the data block visit information of record T cycle write request.When the back-up storage array of far-end duplicates the data of T-1 cycle write request, read data in the colliding data piece according to the data block conflicting information of T cycle write request; And, from local source book, read other data of T-1 cycle write request according to the data block visit information of T-1 cycle write request.Because the embodiment of the invention only exists T cycle write request and T-1 cycle write request in the conflicting data piece data to write preceding replicate run, relatively the total data of T-1 cycle write request is write preceding replicate run, saved the running time, therefore, present embodiment, in first embodiment and second embodiment, reduced the number of times of the data block conflict of two adjacent periods write requests, further shorten the time delay of write request thus, and then improved the performance of local application system to a greater extent.
The data asynchronous replication apparatus structure synoptic diagram that Fig. 6 provides for sixth embodiment of the invention, as shown in Figure 6, present embodiment comprises: conflict determination module 61, write before replication module 62 and the write data module 63 and first replication module 64.
When conflict determination module 61 receives the write request of the local source book of visit in the T cycle, sign according to the data block of the data block Visitor Logs information of T-1 cycle write request and the visit of T cycle write request, determine in the data block of T cycle write request visit, with the data block of T-1 cycle write request visit in, whether have identical data block.Wherein, the T-1 cycle is the last one-period in T cycle; The data block Visitor Logs information of T-1 cycle write request is used to write down the sign of the data block of T-1 cycle write request visit; The sign of the data block of T cycle write request visit is used to write down the sign of the data block of T cycle write request visit.
The working mechanism of above-mentioned each module does not repeat them here referring to the description of Fig. 1 a and the corresponding embodiment of Fig. 1 b.
As shown in Figure 7, on the basis of Fig. 6, first replication module 64 specifically can comprise: the first replicon module 641 and the second replicon module 642.The structural representation of the data asynchronous replication device that Fig. 7 provides for seventh embodiment of the invention.
The first replicon module 641 is according to the data block conflicting information of T cycle write request, shine the district from local express, read in the data block of T cycle write request visit in the data block with the visit of T-1 cycle write request, data in the identical data block, and copy to the back-up storage array.The second replicon module 642, be used for data block recorded information according to T-1 cycle write request, data block conflict recorded information with T cycle write request, from local source book, read data in the data block of T-1 cycle write request visit, in the data block except that identical data block, and copy to the back-up storage array.
The working mechanism of above-mentioned each module does not repeat them here referring to the description of Fig. 2 a and the corresponding embodiment of Fig. 2 b.
Embodiment of the invention data asynchronous replication device, before write data module 63 is handled the write request in T cycle, conflict determination module 61 is by the data block Visitor Logs information of T-1 cycle write request, determine T cycle write request and there is the conflicting data piece in T-1 cycle write request, before 62 of replication modules are write data in the conflicting data piece before writing after the replicate run, write data module 63 writes the data of T cycle write request in the local source book, the sign of the data block of note and the visit of record T cycle write request.When first replication module 64 duplicates the data of T-1 cycle write request to the back-up storage array of far-end, read data in the colliding data piece according to the data block of T cycle write request conflict recorded information; And, from local source book, read other data of T-1 cycle write request according to the data block Visitor Logs information of T-1 cycle write request.Because the embodiment of the invention only exists T cycle write request and T-1 cycle write request in the conflicting data piece data to write preceding replicate run, relatively the total data of T-1 cycle write request is write preceding replicate run, saved the running time, therefore shortened the time delay of current period write request, thereby improved the performance of local application system, reduced the RPO of disaster tolerance system.
The data asynchronous replication apparatus structure synoptic diagram that Fig. 8 provides for eighth embodiment of the invention, as shown in Figure 8, on the basis of the corresponding embodiment of Fig. 6, present embodiment also comprises: second replication module 81 and renewal logging modle 82.
In the 6th embodiment and the 7th embodiment, enter T week after date, the data of T-1 cycle write request are duplicated in beginning to the back-up storage array.And in the present embodiment, the data of T-1 cycle write request are all write local source book after, from the T-1 cycle, just duplicate the data of T-1 cycle write request to the back-up storage array.If the total data of T-1 cycle write request is not copied to the back-up storage array,, continue to duplicate data remaining, T-1 cycle write request to the back-up storage array entering T week after date in the T-1 cycle.
The data asynchronous replication apparatus structure synoptic diagram that Fig. 9 provides for ninth embodiment of the invention as shown in Figure 9, upgrades logging modle 82 and comprises: determines sign submodule 821 and remove sign submodule 822.
Determine sign submodule 821, be used for data block Visitor Logs information, definite sign that has copied to the data block of back-up storage array in T-1 cycle write request; Remove sign submodule 822, be used for data block Visitor Logs information, remove the sign of the data block that has copied to the back-up storage array in described T-1 cycle write request.
The working mechanism of above-mentioned each module does not repeat them here referring to the description of the corresponding embodiment of Fig. 3.
Present embodiment data asynchronous replication device, after the data of T cycle write request write in the local source book, the data of T cycle write request are duplicated in i.e. second replication module, 81 beginnings to the back-up storage array, rather than wait for and to enter T+1 week after date, duplicate the data of T cycle write request again to the back-up storage array.Therefore, present embodiment in the 6th embodiment and the 7th embodiment, has reduced the number of times of the data block conflict of two adjacent periods write requests, has further shortened the time delay of write request thus, and then has improved the performance of local application system to a greater extent.
One of ordinary skill in the art will appreciate that: all or part of step that realizes said method embodiment can be finished by the relevant hardware of programmed instruction, aforesaid program can be stored in the computer read/write memory medium, this program is carried out the step that comprises said method embodiment when carrying out; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CD.
It should be noted that at last: above embodiment only in order to technical scheme of the present invention to be described, is not intended to limit; Although with reference to previous embodiment the present invention is had been described in detail, those of ordinary skill in the art is to be understood that: it still can be made amendment to the technical scheme that aforementioned each embodiment put down in writing, and perhaps part technical characterictic wherein is equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution break away from the spirit and scope of various embodiments of the present invention technical scheme.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101200045A CN101814042B (en) | 2010-03-05 | 2010-03-05 | Data asynchronous replication method and device thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101200045A CN101814042B (en) | 2010-03-05 | 2010-03-05 | Data asynchronous replication method and device thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101814042A true CN101814042A (en) | 2010-08-25 |
| CN101814042B CN101814042B (en) | 2011-12-07 |
Family
ID=42621303
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101200045A Expired - Fee Related CN101814042B (en) | 2010-03-05 | 2010-03-05 | Data asynchronous replication method and device thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101814042B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102306115A (en) * | 2011-05-20 | 2012-01-04 | 成都市华为赛门铁克科技有限公司 | Asynchronous remote copying method, system and equipment |
| WO2014071588A1 (en) * | 2012-11-08 | 2014-05-15 | 华为技术有限公司 | Data replication method, storage controller and system |
| CN108733514A (en) * | 2018-05-24 | 2018-11-02 | 郑州云海信息技术有限公司 | Method, system and the equipment that snapshot is realized between a kind of storage of isomery |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7318135B1 (en) * | 2003-07-22 | 2008-01-08 | Acronis Inc. | System and method for using file system snapshots for online data backup |
| CN101334748A (en) * | 2007-06-27 | 2008-12-31 | 株式会社日立制作所 | Asynchronous remote replication system and its control method |
-
2010
- 2010-03-05 CN CN2010101200045A patent/CN101814042B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7318135B1 (en) * | 2003-07-22 | 2008-01-08 | Acronis Inc. | System and method for using file system snapshots for online data backup |
| CN101334748A (en) * | 2007-06-27 | 2008-12-31 | 株式会社日立制作所 | Asynchronous remote replication system and its control method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102306115A (en) * | 2011-05-20 | 2012-01-04 | 成都市华为赛门铁克科技有限公司 | Asynchronous remote copying method, system and equipment |
| CN102306115B (en) * | 2011-05-20 | 2014-01-08 | 华为数字技术（成都）有限公司 | Asynchronous remote copying method, system and equipment |
| WO2014071588A1 (en) * | 2012-11-08 | 2014-05-15 | 华为技术有限公司 | Data replication method, storage controller and system |
| CN104025062A (en) * | 2012-11-08 | 2014-09-03 | 华为技术有限公司 | Data replication method, storage controller and system |
| CN104025062B (en) * | 2012-11-08 | 2016-11-23 | 华为技术有限公司 | Data copy method and storage control and system |
| CN108733514A (en) * | 2018-05-24 | 2018-11-02 | 郑州云海信息技术有限公司 | Method, system and the equipment that snapshot is realized between a kind of storage of isomery |
| CN108733514B (en) * | 2018-05-24 | 2022-07-08 | 郑州云海信息技术有限公司 | Method, system and equipment for realizing snapshot between heterogeneous storages |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101814042B (en) | 2011-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8250033B1 (en) | Replication of a data set using differential snapshots | |
| CN102929750A (en) | Nonvolatile media dirty region tracking | |
| CN101819543A (en) | Snapshot data processing method, system and storage device | |
| CN114896641B (en) | Data verification method, device, electronic device and computer readable storage medium | |
| CN101561773A (en) | Method for recovering disk data and device thereof | |
| US20150286653A1 (en) | Data Synchronization Method and Data Synchronization System for Multi-Level Associative Storage Architecture, and Storage Medium | |
| CN103617096A (en) | Storage data copying method, equipment and system | |
| CN101661415A (en) | Method for memorizing snapshot data and system for memorizing snapshot | |
| CN106844597A (en) | The method and its device of a kind of quick deletion dependent snapshot target volume | |
| CN115145697A (en) | Database transaction processing method and device and electronic equipment | |
| US8818943B1 (en) | Mirror resynchronization of fixed page length tables for better repair time to high availability in databases | |
| US20060277376A1 (en) | Initial copy system | |
| CN102567214A (en) | Method for quickly initializing bitmap page management by RAID5 | |
| CN101814042B (en) | Data asynchronous replication method and device thereof | |
| CN111159117A (en) | A low-overhead file operation log collection method | |
| CN103986771A (en) | High-availability cluster management method independent of shared storage | |
| JP5445504B2 (en) | Data replication apparatus, data replication control method, and data replication control program | |
| US9032169B2 (en) | Method for high performance dump data set creation | |
| CN109542860B (en) | Service data management method based on HDFS and terminal equipment | |
| CN104331344A (en) | Data backup method and device | |
| WO2003065220B1 (en) | Method and system for providing direct access recovery using seekable tape device | |
| US6854038B2 (en) | Global status journaling in NVS | |
| CN104407932A (en) | Data backup method and device | |
| US8799597B2 (en) | Data copying | |
| JP2015079474A (en) | Data backup method and system for supporting the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee |
Owner name: HUAWEI DIGITAL TECHNOLOGY (CHENGDU) CO., LTD. Free format text: FORMER NAME: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES CO., LTD. |
|
| CP01 | Change in the name or title of a patent holder |
Address after: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Patentee after: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. Address before: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Patentee before: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220907 Address after: No. 1899 Xiyuan Avenue, high tech Zone (West District), Chengdu, Sichuan 610041 Patentee after: Chengdu Huawei Technologies Co.,Ltd. Address before: 611731 Qingshui River District, Chengdu hi tech Zone, Sichuan, China Patentee before: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20111207 |