CN1520562A

CN1520562A - System and method for a caching mechanism for a central synchronization server

Info

Publication number: CN1520562A
Application number: CNA028127234A
Authority: CN
Inventors: 乌韦・汉斯曼; 乌韦·汉斯曼; ・黑珀尔; 斯特凡·黑珀尔; ・施托贝尔; 托马斯·施托贝尔; 斯・海因茨曼; 道格拉斯·海因茨曼; 逖姆・普拉卡亚萨; 阿普拉逖姆·普拉卡亚萨
Original assignee: International Business Machines Corp
Current assignee: International Business Machines Corp
Priority date: 2001-06-30
Filing date: 2002-06-25
Publication date: 2004-08-11
Also published as: US20040153576A1; WO2003005206A2; TW579463B; KR100592647B1; US7539778B2; WO2003005206A3; KR20040010677A; AU2002313483A1; EP1407387A2

Abstract

The present invention discloses a new synchronization architecture for data synchronization between different clients using a central synchronization server connected to a back-end data store, said back-end data store further providing a cache which permanently buffers incoming updates into a persistent store by assigning a unique cache Identification . Write conflicts between a sync server writing a new entry to a cache and updates replicated from the back-end to the cache can be resolved using a blocking mechanism based on the cache ID, such that during a sync session, back-end updates are blocked as long as incoming updates from a client with the same cache ID are not completely written into the cache. The invention is preferably adapted to a synchronization structure with a large number of clients connected to a central synchronization server, while the blocking of the back-end data store, the connection and transmission to the back-end data store is minimal.

Description

The system and method that is used for the buffer memory mechanism of central synchronization server

Technical field

The present invention relates to the method and system of the data sync between the different clients, relate in particular to the data sync that central synchronization server that use links to each other with back-end data stores through network is carried out between mobile client.

Background technology

Can be defined in the consistance that keeps data between the different clients synchronously, for example, the consistance that keeps data between notebook calendar and the PDA(Personal Digital Assistant) calendar, and the data of these clients are stored with different data layouts with different identifiers.Synchrodata also can comprise the complete computer program or the part of these programs.

Usually, by using central synchronization server, data can exchange (see figure 1) between different clients.Central synchronization server can locally be stored all data (promptly when using the private data form), these data usually by the operator with high capacity (for example, Yahoo) uses, perhaps can directly visit the back-end data stores as DB2, Domino or Exchange.The disadvantage that direct visit had to back-end data stores is, synchronizing process may expend time in very much, and this depends on the portfolio between central synchronization server and the back-end data stores.This is very disadvantageous for the user of mobile phone, because they are necessary for the tie-time defrayment with back-end data stores.

United States Patent (USP) 5,974,238 have described the device of the dynamic synchronization between the data that are stored in hand-held computer and principal computer, wherein hand-held computer and principal computer each all have a plurality of data acquisitions that comprise the set of at least one common data, each computing machine has the copy of common data set.Hand-held computer has processor, communication port and data synchronization engine.Data synchronization engine has pseudo-buffer and one or more mark that is connected to pseudo-buffer.When data are written into primary memory, and/or when relevant pseudo-buffer marked invalid, data will be carried out synchronously.By strictly observing one group of agreement, obtain the unanimity of data, because system always knows who has data, and who has the copy of data, and who changes data.Data synchronization engine is resolved any difference in the copy and is made the identical copies that stores the common data set in principal computer and the hand-held computer.

This prior art patent does not relate to the synchronization structure that uses back-end data stores.Therefore, the above-mentioned problem of mentioning that relates to back-end data stores can not be handled and solve by this patent.

Summary of the invention

Therefore the object of the present invention is to provide a kind of new method and system, be used for using the central synchronization server that links to each other with back-end data stores but having avoided the data sync of synchronization structure of the shortcoming of prior art scheme.

This purpose realizes by the scheme of independent claims.Further preferred embodiment is as described in the dependent claims.

The invention discloses central synchronization server that use links to each other with back-end data stores and between different clients, carry out the new synchronization structure of data sync, described back-end data stores also provides buffer, and the renewal that this buffer will import into by the unique buffer sign of appointment (ID) forever is buffered in the permanent storage.Write the new record item to writing to conflict and to use obstruction mechanism to solve between the synchronization server of buffer and the renewal that copies to buffer from the rear end based on buffer ID, make during simultaneous session, as long as the renewal of importing into same buffer ID from client does not write in the buffer fully, the rear end is upgraded and is just got clogged.In further preferred embodiment of the present invention, time tag is buffered mechanism and additionally composes to each renewal, thereby allows to have only when the time sign is new, just buffer update in buffer.The renewal that is buffered in this buffer can be by buffer memory mechanism with different mode, and for example batch mode or continuous mode come and go between back-end data stores and client and duplicate.The present invention preferably is suitable for such synchronization structure, and it has a large amount of clients and is connected to central synchronization server, and the obstruction of Back end data storer, to the connection of back-end data stores be transmitted as minimum.The separating and use buffer to allow to handle higher load of central synchronization server and back-end data stores than direct access back end data stores, because buffer memory mechanism can the batch processing update request also only send to back-end data stores with batch processing, this is than all more effective with the mode that back-end data stores is got in touch at each update request.

Description of drawings

The present invention will be described in more detail in conjunction with the accompanying drawings, wherein:

Fig. 1 show the present invention can based on the simplification synchronization structure;

Fig. 2 shows preferably can be by SyncML communication protocol of the invention process;

Fig. 3 shows the prior art synchronization structure of direct access back end data stores;

Fig. 4 shows the synchronization structure that uses invention buffer memory mechanism;

Fig. 5 shows the preferred embodiment that uses the synchronization structure of the present invention of the buffer memory mechanism that provides time tag as shown in Figure 4;

Fig. 6 shows to be connected to through central synchronization server and upgrades synchronous method step between the client of buffer;

Fig. 7 shows the method step that utilizes buffer that the renewal of back-end data stores is duplicated;

Fig. 8 shows the method step that utilizes back-end data stores that the renewal that occurs in the buffer is duplicated;

Fig. 9 shows expansion buffer memory of the present invention mechanism.

Embodiment

Use between the different clients of central synchronization server synchronously based on the synchronous protocol that generally includes the following step:

Presynchronization: for prepare to carry out actual synchronously, some action that must take before can actual synchronization.These actions are divided into following classification: authentication, authorize and definite client end capacity.Authentication guarantees that server is the server of its requirement, and client is the client of its requirement.Whether the authorization check client is allowed to carry out the action of being asked (for example, deletion is upgraded or only set up the new record item).At last, server determines that capacity of equipment (for example, the size of largest buffer) flows to the data stream of client with optimization.

Synchronously: this is the part of exchange synchrodata.Between two synchronous sides, the local ident of all data recording all is mapped to the known Global ID of both sides.Therefore each side has a mapping table that local ident is mapped to Global ID.So, have only entry that upgraded, new or deletion to be exchanged.If both sides upgrade identical data recording item, will produce conflict.This update inconsistency can be solved with different modes: attempt merging and upgrade, the duplicated record item allows an entry have precedence over another entry, or just report conflict simply, so that the user can solve.

The back is synchronous: in the synchronizing process of back, carry out all cleaning works, for example upgrade mapping table, report unsolved conflict or the like.

Widely used synchronous protocol is SyncML.SyncML provides host-host protocol based on XML synchronously for what be independent of host-host protocol.Each synchronization message all is XML files.Common SyncML system wherein uses B (for example personal digital assistant) and just sends synchrodata to the SyncML framework by its client synchronization engine as shown in Figure 2.The SyncML framework with API Calls (for example upgrade, create ...) become effective SyncML file with data-switching and send it to server.At server side, the SyncML framework receives this document, and it is analyzed, and will order then with data and send to server sync engine, and server sync engine then engages in the dialogue with application A (for example, the Lotus Notes calendar on the notebook).

In Fig. 3, provided the synchronization structure of direct access back end data stores in the prior art.Customer end A and B can be mobile client (for example, mobile phone or personal digital assistant), and itself and central synchronization server (synchronization server (SynchServer), particularly synchronization engine (SyncEngine)) have wired or wireless connection.Renewal or new record item from mobile client send to synchronization engine through the SyncML adapter.Renewal is directly read and write to central synchronization server at back-end data stores.Communication between central synchronization server and the back-end data stores can be that Java JDBC calls, and ODBC calls, SOAP or HTTP request, or private communication protocol.Back-end data stores can be the relational database of similar Oracle or IBM DB2, the non-relational database of perhaps similar Microsoft Exchange or LotusDomino.Back-end data stores links to each other with central synchronization server through Intranet or Internet.This may produce bottleneck in following state:

With being connected at a slow speed of back-end data stores;

The read/write of carrying out on back-end data stores is to operate slowly;

Back-end data stores can't well be regulated;

Back-end data stores is not supported the data recording of fine granulation or the obstruction of piece.

Fig. 4 has provided basic synchronization structure of the present invention.Compare with synchronization structure (see figure 3) of the prior art, the present invention provides permanent (permanent) buffer 24 basically and has been buffered to renewal in the buffer 24 and will makes the renewal of buffer memory and buffer memory mechanism that each

client

2,4 is synchronous.This permanent buffer 24 can be the relational database as Oracle or IBMDB2, also can call by for example JDBC to conduct interviews.Buffer memory mechanism preferably includes cache monitor device 20, back end monitor 18 and buffer management component.What cache monitor device 20 control client of 16 from the buffer to the back-end data stores was upgraded duplicates.What back end monitor 18 control rear end of 24 from the back-end data stores to the buffer was upgraded duplicates.Buffer management parts 23 are central synchronization server 6 (synchronization server), and cache monitor device 20 and back end monitor 18 are provided for visiting the interface of the renewal of buffer memory.Buffer management parts 23 also provide creates buffer ID, buffer ID is mapped to rear end ID and rear end ID is mapped to the function of buffer ID.When central synchronization server 6 was just writing buffer 24 with renewal, cache monitor device 20 and back end monitor 18 received the message that contains the buffer ID that is assigned to the renewal for the treatment of buffer memory at least.Mapping table 26 by means of the buffer ID that contains rear end ID and appointment, as long as be not buffered to fully in the buffer 24 from the renewal with same buffer ID of client, back end monitor 18 is just blocked each and is imported renewal buffering of 24 from the back-end data stores to the buffer into.Preferably called (beginSync) at the synchronous startup of cache monitor device 20 and opened, and called (endSync) by same EOS and close at cache monitor device 20 by the synchronizing process of central synchronization server startup.During simultaneous session, cache monitor device 20 does not allow to duplicate and has a renewal that is buffered to the buffer ID that the beginSync in the buffer 24 calls.

Fig. 5 shows further preferred embodiment of the present invention.Compare with the basic synchronization structure shown in Fig. 4, this preferred embodiment also comprises (persistent) storer 22 forever.Permanent storage 22 replaces the buffer management parts 23 among Fig. 4, and the mapping between rear end ID and the buffer ID is provided, and guarantees that the renewal that only has newer time tag just is written in the buffer 24.For example customer end A 2 is opened the simultaneous session with central synchronization server 6 (synchronization server), and is sent to and comprises local general identifier (LUID) less, version number and want the renewal 1 of data updated.Data recording on the LUID unique identification client data storer.Central synchronization server 6 has mapping table 10, is used for the LUID (comprising client id and database URI) of client is mapped to buffer ID, and buffer ID is used for identifying update request on buffer 24.Client uses count-up counter as time tag, because many

mobile clients

2,4 are not supported time tag in its data-carrier store.Synchronization engine 12 (synchronization engine) part of synchronization server 6 knows that the version number's mapping 8 that will upgrade is to the simultaneous session time tag.The renewal that synchronization engine 12 just will have simultaneous session time tag and buffer ID sends to permanent storage 22.Permanent storage 22 is compared the time tag of the renewal in the time tag of this renewal and the buffer.If the time tag of update request is newer, then this update request is buffered in the buffer 24.If update request is not new, then this update request is left in the basket.Simultaneously, the renewal in the back-end data stores 16 occurs, and back end monitor 18 is just by comprising time tag, rear end ID and want the renewal 19 of data updated to call permanent storage 22.Permanent storage 22 is mapped to the buffer ID that is assigned with rear end ID, time tag relatively, and if to import time of renewal into newer, upgrade buffer 24.The result of these two renewals as shown in Figure 6.

How Fig. 6 makes from the renewal of the client the inventive method step by the data sync in central synchronization server and the buffer if showing.

Client, mobile phone for example, request 2 is carried out simultaneous session with synchronization server (synchronization server).Before renewal can be sent to central synchronization server, must finish the authentication between client and the central synchronization server.If authentication success 4, then central synchronization server is accepted synchronizing process, and client can send its renewal 6.

Upgrade and preferably to comprise version number, LUID and want data updated.Central synchronization server receives 8 and upgrades, and utilizes mapping table version number to be converted to the time tag of being assigned.In next step, utilize the mapping table of the buffer ID that contains LUID and its appointment that the LUID that upgrades is shone upon 10.If LUID does not comprise 12 in the register map table, permanent storage is created 14 new buffer ID and is upgraded mapping table with it.If this LUID has been contained in the register map table, central synchronization server request 16 permanent storagies cushion renewal in buffer.Then, the time tag of the renewal with same buffer ID that has cushioned in the time tag of renewal and the buffer is imported in

permanent storage inspection

18,20 into.If it is newer to import the time tag of renewal into, then imports into and upgrade the old renewal that covers in 22 buffers.If time tag is older, then imports renewal into and be left in the basket 21.

Fig. 7 shows the method step how renewal from back-end data stores is replicated in buffer.Suppose that being applied in of access back end data stores upgrade entry, for example secretary can produce its person in charge's calendar data in back-end data stores renewal in the back-end data stores.Each record upgrades and to comprise rear end ID, time tag and want data updated.New data more can be sent to back end monitor according to following modes:

Back-end data stores monitor is to the back-end notified each new record item or renewal.Back end monitor is initiated read request for receiving to upgrade.This pattern is called as propelling movement (push) pattern, perhaps

Back end monitor periodically starts the inquiry at back-end data stores.Back-end data stores is returned the rear end ID of renewal or new record item.By means of this rear end ID, back end monitor starts the visit to those renewals.This pattern is called as poll (polling) pattern.

Back end monitor upgrades and the Back end data format conversion is become to comprise the

buffer data layout

4,6 of rear end ID and time tag from the rear end retrieval.Back end monitor will be upgraded and send 8 to permanent storage, and this permanent storage is mapped to rear end ID according to mapping table the buffer ID of appointment.If rear end ID is not contained in the mapping table, permanent storage is created new buffer ID 12 and the rear end ID of new buffer ID. and appointment is inserted in the mapping table 14.Then permanent storage imports inspection into the time tag of already present renewal (if there is) in the time tag of renewal and the buffer.

If the introduction time sign is newer, then the renewal in the buffer is capped 20.If this time tag is older, then imports renewal into and be left in the basket 18.

Fig. 8 shows the method for updating step that how occurs in the duplicate cache device in back-end data stores by synchronization server (synchronization server).According to method step as shown in Figure 6, in buffer, upgrade entry.Permanent storage is notified the item of the new record in 4 buffers to the cache monitor device now.The cache monitor device uses the appointment rear end ID of 6 mapping tables selection corresponding to the buffer ID of new record.If this buffer ID does not have the rear end ID of appointment in mapping table, the cache monitor device is activated to create new rear end entry.Originally, the cache monitor device will contain the data layout of 10 one-tenth back-end systems of buffer data-switching of time tag and entry data, connect with back-end data stores, and establishment 12 contains time tag in back-end data stores, more the new record item of new data and rear end ID.New rear end ID is sent to permanent storage 16 through the cache monitor device, and this permanent storage inserts this new rear end ID in the mapping table of the rear end ID that contains buffer ID and respective assigned.If this buffer ID has the rear end ID of appointment in mapping table, then notify 20 cache monitor devices in the rear end, to start this renewal entry.The cache monitor device will upgrade 22 one-tenth Back end data forms of buffer Data Format Transform of entry, and the renewal entry that contains time tag and rear end ID obtains carrying out 24 in back-end memory.

Fig. 9 shows expansion buffer memory mechanism.Here applications 10 is through blocker (interceptor) 12 access back end data stores 16.Blocker 12 is understood the buffer 24 of synchronization server, and at first checks this buffer before back-end data stores 16 retrieve data.This has solved the read/write conflict problem, and still from the viewpoint of applications, this is a kind of compromise consideration of low performance certainly.

In a word, compare with immediate prior art, advantage of the present invention is as follows:

The variation each time of back-end data stores all copies in the buffer immediately.When existing buffer to duplicate, do not block synchronization server.When synchronization server when buffer carries out write operation, the renewal of buffer and back-end data stores is all by total blockage.In order to obtain peak performance, can use sight at reality, for example batch processing or drip (trickle) pattern is carried out meticulousr adjusting to buffer update strategy and rear end update strategy.

If desired, the read/write conflict of applications can solve by using blocker.

Claims

1. A system for synchronizing data between different clients (2, 4) by using a central synchronization server (6) having a connection to said clients (2, 4) and a The connection of the end data storage (16), wherein the client has a program to generate the data to be synchronized, and the synchronization engine for synchronizing with the central synchronization server, wherein the back-end data storage (16) has a connection with the external application ( 15) data connection between to allow updating of said data in said backend data store (16), wherein said system is characterized by the following components:

a cache (24) for permanently caching updates of said backend data storage (16) and said clients (2, 4), and

A caching mechanism component for controlling and executing caching of updates to said cache (24), and replicating cached updates to said respective clients and said backend data store (16).

2. The system of claim 1, wherein said cache mechanism has a cache management component (23) that provides caching of said client in said cache (24) by assigning a unique cache identification (ID). (2, 4) update function, and copy updates from said cache (24) to said client (2, 4), said caching mechanism also has a backend monitor (18), where in a sync session Meanwhile, if the update from the client with the same cache ID is not completely written into the cache (24), the backend monitor (18) blocks updating the cache from the backend data storage to the cache .

3. The system as claimed in claim 1, wherein said caching mechanism has a persistent storage (22) which provides the function that only if the time stamp has This function caches updates of said client or said backend data store (16) in said cache (24) when the time stamp of updates for the same cache ID is new.

4. The system of claim 3, wherein the persistent storage (22) further provides a function of mapping cache IDs to backend IDs.

5. The system of claim 2, wherein said cache mechanism has a cache monitor (20) that copies updates from said cache (24) to a backend data store ( 16).

6. The system of claim 2, wherein said caching mechanism has a back-end monitor (18) that runs between said cache (24) and back-end data storage (16) in batch or continuous trickle mode. Copy updates between.

7. The system of claim 1, wherein the cache (24) and the backend data store (16) are databases.

8. The system of claim 1, wherein the clients (2, 4) are mobile clients.

9. The system of claim 1, wherein the synchronization protocol is SyncML.

10. A method for performing update synchronization between clients using the system according to claims 1-9, comprising the steps of:

receiving an update from said client via said client's sync engine (2) comprising a version number, LUID and update data,

mapping said updated version number to an assigned synchronization time stamp (8) using a mapping table,

mapping said updated LUIDs to assigned buffer IDs using a mapping table (10),

caching updated data comprising a time stamp and a buffer ID into said buffer only if said updated time stamp is newer than an updated time stamp in said buffer having the same buffer ID (20),

mapping said updated cache IDs to assigned backend IDs using a mapping table (6),

converting the updated buffer data format into the data format of the backend data store (22),

Initiate an update (24) of said backend data storage containing update data, time stamp and backend ID.

11. A method for performing update synchronization between clients using the system according to claims 1-9, comprising the steps of:

receiving an update from said client via said client's sync engine, comprising a version number, LUID and update data,

mapping said updated version number to an assigned synchronization timestamp using a mapping table,

mapping said updated LUID to an assigned buffer ID using a mapping table,

caching updated data comprising a timestamp and a cache ID into said cache only if said updated timestamp is newer than an updated timestamp in said cache with the same cache ID,

If the operation of mapping the cache ID to the backend ID is unsuccessful, generate a new backend ID,

converting the updated buffer data format into the data format of the backend data store,

start the updated new entry in the backend data store, the new entry includes new entry data, time stamp and backend ID,

Insert the new backend ID into the mapping table.

12. A method as claimed in claim 10 or 11, wherein the synchronization protocol used between the client and the central synchronization server is SyncML, and the updates received by the central synchronization server are represented as XML files .

13. A method of replicating updates from said backend data store to said cache using the system of claims 1-9, comprising the steps of:

receiving by said central synchronization server from said backend data store an update comprising update data, a time stamp and a backend ID,

converting the backend data format of the backend update into a cache data format,

Use the mapping table to map the backend ID to the cache ID,

Caching the update from the backend data store to the cache only if the timestamp is newer than an update cached in the cache with the same cache ID.

14. The method of claim 13, wherein during a synchronization session, if updated data from the client with the same cache ID has not been completely written into the cache, blocking data from the backend memory to the update cache of the cache.

15. The method of claim 13, further comprising the step of:

generate a new cache ID when the mapping from backend ID to cache ID was unsuccessful, and

Insert the new cache ID into the mapping table.

16. A computer program product stored in the memory of a digital computer, comprising software code portions for performing the method according to claims 10 to 15 when said product is run on the computer.