SyncML: an introduction, its potential, its problems

April 8, 2009

This article was contributed by Nick Jenkins

Users of PIM (Personal Information Manager) software, such as Evolution, Kontact, or Chandler, tend to accumulate more information than just email. Typically, this is data such as notes, tasks, calendars and contacts, collectively known as "personal information". Keeping all of this information synchronized between the desktop, mobile devices and the web is difficult, but the SyncML standard may be able to help.

SyncML (Synchronization Markup Language) is a standard for synchronizing information. SyncML allows different kinds of devices (cell phones, portable music players, desktops, etc.) to synchronize various contact and scheduling information so that each device is kept up to date. It can also synchronize with a web service so that a forgotten phone number, for example, could be retrieved at an internet café.

SyncML is currently maintained by the OMA (Open Mobile Alliance) under the official name OMA-DS (Data Synchronization). The specification is available free of charge.

Usually there is a SyncML server, which provides a central master copy of a user's personal information. All of the user's SyncML clients will then synchronize against this central server. This central server communicates using the platform-independent SyncML standard. This avoids the combinatorial explosion of every device having to know how to talk to every other device in a device-specific custom format.

For selecting a SyncML server, the options are either to use a pre-existing public SyncML server, which is typically less effort, or to set up a SyncML server for private use. Two examples of popular public SyncML servers are ScheduleWorld and MyFunambol beta. ScheduleWorld is based on Funambol, but forked from the GPL-licensed version quite a while ago, with no code being contributed back. Since then, its author has invested a lot of work into improving it, and in the process, rewriting the contact and calendar support from scratch. These aspects are what ScheduleWorld deserves praise for, but because the code changes are kept private it has to be considered a closed-source non-distributable server.

Alternatively, there are open-source SyncML servers for users who prefer to set up their own local SyncML servers, such as Funambol, or for a complete list, see the Wikipedia article. For Funambol, the software does not seem to have been packaged by many distributions, and is surprisingly large at 170 MB for a distribution-independent .bin file, which includes its own installation of Tomcat. Deploying Funambol as part of an existing Tomcat installation is not recommended and is unsupported.

There are also open-source SyncML connectors available for connecting Linux PIMs to SyncML servers. Examples include SyncEvolution, and its user-friendly Genesis-Sync panel applet. For users wanting a quick HOWTO, this message on the Evolution mailing list outlines how to synchronize Evolution on Ubuntu with a Nokia N-series or E-series phone, using a public SyncML server.

The benefits of SyncML are that it has the potential to do for personal information what IMAP does for email; that is, make it live "in the cloud," and be remotely accessible, modifiable, and synchronized between a wide variety of devices. The idea of being able to view and modify personal information anywhere on multiple devices or on the web, and have it all synchronized together, with an update made on any one device replicated to all the others, is quite compelling.

A further strength of SyncML is that many cell phones, including all Nokia N-series and E-series phones, most Sony Ericsson phones, and most Motorola phones, have built-in SyncML clients - which means there is no need to install extra software on these phones to synchronize personal information. Add-on software is available for most other phones, including BlackBerry and Windows Mobile - see the ScheduleWorld wiki for example configuration information.

A final strength is that all of the SyncML clients can have a local cache of information, so that even when the Internet is not available users can still access and update their data. Two-way syncing then ensures that those updates will be propagated at the next synchronization.

Some of the weaknesses of SyncML are partially traceable back to its early adopters - namely, a consortium of various cell phone manufacturers. The protocol itself is data format agnostic. So for all but the simplest uses, like verbatim copying, the client and server need to agree on a common format for items. Due to its history, this format is often vCard 2.1 and vCalendar 1.0. The more capable iCalendar 2.0 format used by all desktop PIMs is often not or only partially supported.

A good SyncML server takes the capabilities and quirks of its clients into account when exchanging data with them. For example, a photo associated with a contact can be preserved when receiving an update of that contact from a client which cannot store photos. Less capable servers themselves drop some information because their internal data model is more limited than that of the clients they exchange data with. Client implementations can be poor, including crashes when sent something unexpected.

Further weaknesses are that most of the syncing interfaces seem to assume there is exactly one contacts folder, one calendar, one notes folder, and one tasks folder. This one-folder-only assumption makes sense on a cell phone, but it does not hold for a desktop PIM. For example, it is quite common to have multiple tasks folders and multiple calendars - so this assumption means that not everything is being replicated, which reduces the power of synchronization.

In addition, support for SyncML is not yet built into PIMs, in the same way that IMAP comes "as standard" in email clients. However, even having it integrated into a single PIM would be very useful for people who wanted the same data seamlessly synchronized between a laptop and a desktop, or a work machine and a home machine, and who ran the same PIM at both ends. The most desirable though would be to have complete synchronization between different PIMs.

A final weakness is that most SyncML User-Interfaces require the user to manually initiate the two-way sync with the server. Instead, it would be easier for the user if there was a set-and-forget option, where the SyncML client would sync only when it needed to; namely when either when the server push-notified the client of pending changes that it had not yet received, or when the client uploaded changes in the background made by the user on that device as the user made them.

In summary, it is currently possible to synchronize personal information between a mobile phone and PIM software using open-source with SyncML, and it currently works quite well, albeit with some limitations. However, SyncML or a successor to it, has the potential to be so much more powerful; it could be the next logical step beyond IMAP by providing seamless automatic synchronization of all personal information between multiple PIM clients. This would enable users to easily access and update all of their personal information, wherever they were, irrespective of whether there was Internet-access or not. There is certainly hope for further developments in this area.

The author gratefully thanks Patrick Ohly, the author of SyncEvolution, for his invaluable assistance in writing this article.

Index entries for this article
GuestArticles	Jenkins, Nick

to post comments

SyncML: an introduction, its potential, its problems

Posted Apr 9, 2009 9:57 UTC (Thu) by stevan (guest, #4342) [Link] (6 responses)

Thanks for a good summary of the issues. I have deployed syncml on over 50 Palm Treos using the Synthesis utility against eGroupware's syncml capability, with pretty good results, except when it came to the clock changes, when calendar entries were often mucked about, especially recurring events. Syncml, as a piece of the building blocks that make up the service, may or may not have been to blame, but that was the one flaw that made syncing calendars fairly tense twice a year. Hunting down the issue, whether it was on the Palms, the Synthesis syncml client, the syncml code on eGroupware or the eGroupware calendar was a nightmare, never fully resolved.

But the deployment itself was extremely trivial, that's for sure.

Stevan

SyncML: an introduction, its potential, its problems

Posted Apr 9, 2009 11:47 UTC (Thu) by pohly (guest, #6319) [Link] (5 responses)

Stevan, most likely the calendar problems you experienced came from insufficient support for time zone information in one of the involved components. As mentioned in the article, this is not a problem of SyncML itself but of the data format used for exchanging calendar events. Support for time zones is improving, but much more slowly that I had hoped. Synthesis added support for it in their 3.x engine; not sure about eGroupware.

Many of the traditional SyncML clients only supported UTC or local time. Neither one nor the other are sufficient when dealing with recurring events in many different time zones - in other words, a multi-national enterprise scenario.

Here's a more detailed analysis of this problem: http://www.estamos.de/blog/2008/06/30/calendar-time-stamp...

SyncML: an introduction, its potential, its problems

Posted Apr 9, 2009 15:00 UTC (Thu) by stevan (guest, #4342) [Link]

Thanks for the info - it always had the feeling of being a problem on the verge of being solved. I left that company a while ago, and they subsequently went the crackberry route, I hope this issue wasn't used as the excuse (when the real reason, as we all know, is because suits aren't suits without their latest crackberry.) I thought using syncml with a variety of devices like that a great example of the scalability and robustness of a Free software-based infrastructure, and although stack can get a little wobbly at times, I'd be happy to put that solution in anywhere.

SyncML: an introduction, its potential, its problems

Posted Apr 10, 2009 7:58 UTC (Fri) by fb (guest, #53265) [Link] (3 responses)

The fact that the standard decided to not support an important feature seems first and foremost a flaw in the standard to me.

Couldn't this just be solved by using XML datetime format for transferring the dates?

SyncML: an introduction, its potential, its problems

Posted Apr 10, 2009 10:15 UTC (Fri) by Los__D (guest, #15263) [Link] (2 responses)

Couldn't this be solved in the (IMHO) proper way; always using UTC, letting clients do the conversion?

SyncML: an introduction, its potential, its problems

Posted Apr 11, 2009 2:54 UTC (Sat) by rganesan (guest, #1182) [Link] (1 responses)

> Couldn't this be solved in the (IMHO) proper way; always using UTC,
> letting clients do the conversion?

Not really. Suppose the client is in a timezone which has daylight savings in effect and set a recurring calendar item. What should happen when daylight savings is turned off in Winter? The user set the meeting at a specific hour local time not UTC. You can make the client automatically split the recurrent meeting into two both in UTC, but that causes other problems. Server has to be timezone aware to handle these kinds of cases.

SyncML: an introduction, its potential, its problems

Posted Apr 11, 2009 8:21 UTC (Sat) by Los__D (guest, #15263) [Link]

Ah, of course. DST coming along to screw things up again! :)

SyncML: an introduction, its potential, its problems

Posted Apr 10, 2009 10:23 UTC (Fri) by fb (guest, #53265) [Link]

Nice to learn something about SyncML!

It seems the standard was built for corporate users, with the server-client architecture. To me, it seems obvious that phones should be able to sync data to each other. I guess everybody expects the hundreds of millions of "regular" mobile phones users to just lose their contacts when they get a new phone.

I have recently transfered contacts from a Samsung ?SGH-350? (a 3 year old samsung), and had to input that into a Nokia X-Press Music. Both phones had SyncML support, but now I understand that it could never work because I lacked a server. The mess increased as we had no Outlook installed (either Samsung's or Nokia's suite expected Outlook to be present). I solved it by writing some python to convert between the CSV formats. Worth mentioning that the Samsung refused to talk to Linux (i.e. Gnome Conduit).

In any case, that experience convinced me of how badly supported this kind of syncing still is.

SyncML: an introduction, its potential, its problems

Posted Apr 14, 2009 22:39 UTC (Tue) by gvegidy (guest, #5063) [Link] (3 responses)

IIRC, Nokia recently announced that they will move their Outlook<->phone
synchronization on N- an E-Series to ActiveSync.

Didn't they use some kind of SyncML for this job before? Wouldn't that mean
that at least for Nokia- and Windows-phones it would make more sense to
look at an ActiveSync-based way to sync, e.g. Z-push?

SyncML: an introduction, its potential, its problems

Posted Apr 15, 2009 8:09 UTC (Wed) by pohly (guest, #6319) [Link] (2 responses)

Are the specs for ActiveSync publicly available? Can it be implemented without paying royalties to Microsoft on a per-copy basis?

I looked and so far have neither found the specs nor how much it would cost to get and implement them. Considering that, ActiveSync doesn't look like a suitable alternative to SyncML for open source solutions.

Technically I cannot tell either whether it has advantages over SyncML, exactly because I haven't seen the protocol definition. I suspect that as soon as a variety of less-capable devices used it, it would run into the same issues as SyncML. Poor or inconsistent support for PIM data itself is not a problem of the protocol used to exchange it.

SyncML: an introduction, its potential, its problems

Posted Apr 17, 2009 19:11 UTC (Fri) by filipjoelsson (guest, #2622) [Link] (1 responses)

There is an ActiveSync <-> SyncML bridge. I used it years ago with an iPAQ, and it worked ok (don't remember the name). A lot better than latter day smartphones, at least for me. The bridge let me sync with Evolution too, so no servers were needed either.

SyncML: an introduction, its potential, its problems

Posted Apr 19, 2009 11:55 UTC (Sun) by pohly (guest, #6319) [Link]

What you describe sounds like ActiveSync for locally connected devices. There's also ActiveSync for over-the-air synchronization. My (as I said, unfortunately very uniformed) impression is that these two share the name, but not necessarily the same protocol.

Was this bridge open source? Any chance of finding the name again? I know that Synchronica had (has?) a closed-source bridge between ActiveSync and SyncML.