18 July 2008

Apple iPhone 2.0: The real story behind MobileMe Push Mail and Jabber/XMPP Chat

So those of you who anticipated a Jabber/XMPP chat client on the iPhone (and iPod Touch) after TUAW rumoured that 'a new XMPP framework has been spotten(sic) in the latest iPhone firmware' back in April were close... but no cigar. Same applies for those who hypothesised about P-IMAP or IMAP IDLE being used by MobileMe for push mail.

The real story, as it turns out, is that Jabber (the same open protocol behind many instant messaging networks including Google Talk) is actually being used for delivering push mail notifications to the iPhone. That's right, you heard it here first. This would explain not only why the libraries were curiously private (in that they are not exposed to developers) but also why IMAP IDLE support only works while Mail.app is open (it's a shame because Google Apps/Gmail supports IMAP IDLE already).

While it's in line with Apple's arguments about background tasks hurting user experience (eg performance and battery life), cluey developers have noted that the OS X (Unix) based iPhone has many options to safely enable this functionality (eg via resource limiting) and that the push notification service for developers is only a partial solution. It's no wonder though with the exclusive carrier deals which are built on cellular voice calls and SMS traffic, both of which could be eroded away entirely if products like Skype and Google Talk were given free reign (presumably this is also why Apple literally hangs onto the keys for the platform). If you want more freedom you're going to have to wait for Google Android, or for ultimate flexibility one of the various Linux based offerings. We digress...

So without further ado, here's the moment we've all been waiting for: a MobileMe push mail notification (using XMPP's pubsub protocol) from aosnotify.mac.com:5223 over SSL:


<message from="pubsub.aosnotify.mac.com" to="[email protected]/5e60ad2e47da9fca36de59244f25c9b1cd8e0cb8" id="/protected/com/apple/mobileme/samnsofi/mail/[email protected]__3gK4m">
<event xmlns="http://jabber.org/protocol/pubsub#event">
<items node="/protected/com/apple/mobileme/samnsofi/mail/Inbox">
<item id="5WE7I82L5bdNGm2">
<plistfrag xmlns="plist-apple">
<key>maild</key>
<string>E1B537</string>
</plistfrag>
</item>
</items>
</event>
<x xmlns="jabber:x:delay" stamp="2008-07-18T01:11:11.447Z"/>
</message>

<message from="pubsub.aosnotify.mac.com" to="[email protected]/5e60ad2e47da9fca36de59244f25c9b1cd8e0cb8" id="/protected/com/apple/mobileme/samnsofi/mail/[email protected]__NterM">
<event xmlns="http://jabber.org/protocol/pubsub#event">
<items node="/protected/com/apple/mobileme/samnsofi/mail/Inbox">
<item id="8ATABX9e6satO6Y">
<plistfrag xmlns="plist-apple">
<key>maild</key>
<string>544FE17</string>
</plistfrag>
</item>
</items>
</event>
<headers xmlns="http://jabber.org/protocol/shim">
<header name="pubsub#subid">3DEpJ055dXgB2gLRTQYvW4qGh91E36y2n9e27G3X</header>
</headers>
</message>
I'll explain more about the setup I used to get my hands on this in another post later on. So what's the bet that this same mechanism will be used for the push notification service to be released later in the year?

10 comments:

  1. great article .. but not that great , at least not for me. Because I have been struggling for about 2 weeks to get my hackintosh up and running, and now I find out that there is no darn XMPP library available for coders arround the world. So I cant write my app for iphone and make part of my great system putting together Androids, MIDPs and ... sigh ... iPhones. Or can I ?

    I am all newb to the green Apple world. Rumours say that you can run code other than objective-c on an iphone. Is there any chance I could port some C / C++ library to iphone, and then call some functions I need from the allmighty objective-c ?

    Thanks,
    kellogs

    ReplyDelete
  2. Did you try to send such messages to simulate push e-mail on a non-MobileMe address ?

    I'd love to mod the Dovecot LDA to get push mail on my iPhone :-)

    ReplyDelete
  3. So what I was hoping for was that the notifications would specify which account to poll (eg MobileMe or Yahoo!). I'd intended then to sit in the middle of the SSL connection and send notifications for 'Google Apps', but now I see it uses XMPP pubsub it seems likely the iPhone will only want to hear about events it's actually listening for.

    That's not the news you wanted, but it's not the news I wanted either... wouldn't be a problem with more open platforms like Android.

    Sam

    ReplyDelete
  4. Have you had any luck capturing traffic during iPhone <-> MobileMe sync for calendar, contacts, or bookmarks? I'd like to add iPhone support to the open-source dotmac project (now dotmobileus) but don't have any packet captures to base it off of.

    ReplyDelete
  5. Once I worked out that this was how it worked for mail I assumed it was the same story for contacts and calendar. I did tinker with it a bit but didn't take any traces I'm afraid. Fortunately all is not lost; it's not that difficult to re-capture the traces now that there's a process for doing so.

    Sam

    ReplyDelete
  6. Hey, after the update to OS X version 10.5.6 MobileMe also pushes contacts and calenders to the Mac (previously push only worked on iPhone/iPod touch devices). When i used Wireshark to sniff the traffic i could see that there's connections over port 5223 on the Mac, too.
    So yes, it seems like MobileMe also uses XMPP pubsub for pushing contacts and calenders, at least on the Mac.

    ReplyDelete
  7. Thanks for the feedback - that makes a lot of sense... they would want to have good reason for introducing another mechanism for pushing through updates.

    Sam (off to update to 10.5.6)

    ReplyDelete
  8. Hi! This is a great article. I am wondering how you figured this out. What tools did you use to get the XML?

    ReplyDelete
  9. What did you use to get the XML? I am wondering if Me relies on XMPP pub for all synchronization. Has anyone figured out what it uses on PCs?

    ReplyDelete
  10. Interesting question re: PCs - one would think they would try to use the same services for different devices (there's little reason not to). A simple test would be to run strings over the installed executables and grep for 'xmpp' - if you get results then you can be pretty sure that it is... if not... hard to say.

    Check out the followup post HOWTO: Reverse engineer the iPhone protocols.

    Sam

    ReplyDelete

Note: only a member of this blog may post a comment.