jerakeen.org

I really miss Dashboard. It was an effort to display some context around whatever person you were interacting with at any given moment - look at an email from Paul, or open an IM chat with him and you’d see things that he’d blogged or uploaded to Flickr recently. Genius. From the screenshots, it looks practically magic, tying into incoming SMS messages, IM conversations, the RSS feed reader, etc.

Alas, I never had a fully working Dashboard setup locally, mostly because applications had to actively participate in the process - they sent things called ‘cluepackets’ to the dashboard application containing hints about the current context. Because of this design, every app involved needed its source code patched and a recompile. This was a complete pain. Obviously, had everything gone to plan, the patches would have been merged and everyone would have been happy. I presume that Dashboard failed because the bootstrapping process was so hard that no-one used it.

Anyway, inspired by both Dashboard and Aaron‘s obsession with the address book, I’ve had a stab at doing it again, but worse.

Shelf

Shelf will look at the current foreground application, and try to figure out if what you’re looking at corresponds to a person in your Address Book. Then it’ll tell you things about them.

Update 2008/01/08: I have downloadable versions of Shelf now. Go to the project page and download one.

It’s for MacOS. Because on MacOS, I have OSA - I can interrogate most (well-written) applications about their state in a beautiful, language-agnostic and fast manner. I can ask Mail.app for the email address of the current mail. I can ask Safari what the URL of the foreground window is. I can ask Adium for the account details of the current chat. I can ask NetNewsWire for the homepage URL of the current subscription. And I can ask the system what app is in the foreground. I can also interrogate the system address book via the Cocoa bindings for same and find out what users have got that email address, or URL, or AIM screen name. And then I can take all the other information about them in their address book entry, and figure out some context. Oh, and the thing’s written in Ruby, because the Ruby scripting bridge is a thing of serious beauty and should be played with by everyone.

Good thing

So, advantages. I don’t have the bootstrapping problem, because most MacOS applications already have enough of a scripting interface that I can extract information from them. Firefox is proving to be a serious problem, alas, but I’ve hit no other apps I can’t get something useful out of.

Once I have an Addressbook record as context, I can update the interface with a picture of the person and their name/company (direct from the address book, so easy). As a ‘will this work?’ experiment, I’m parsing every referenced URL in the address book card for RSS feeds, and displaying those as context. And (because I work there) I have special-case Dopplr support that tells me where the person is in the world and where they’re going next. This means that when someone IMs me, a window pops up and tells me where they are, when they’re back, and what they’ve blogged recently. Awesome.

The system address book is great - it has multiple email address and URLs for people, so I’m indicating things like Dopplr username by just putting the url to my traveller page in my address book entry. I can parse the username out later and use it to call the API with. This has the advantage that if I visit my Dopplr page in Safari, hey, wow, that URL is in the address book, and it knows that it’s me again. Flickr is the next obvious choice for special-casing, but the principle extends to anything.

Bad thing

Disadvantages. Firstly, urgh, I’m polling. Every 2 seconds, I ask the system for the foreground application, then ask that application (if I know how) for context. This is probably a little heavy (is it? I’m guessing..). Secondly, I have to do explicit work for every app out there. The huge advantages of Dashboard’s cluepacket approach over mine were that packets were pushed instantly on a change of context, and that a new application was responsible for sending its own cluepackets.

Actually, this is easy. My app should have a ‘change context’ OSA method that other applications can call. Smart apps can tell me when their context changes, and I’ll just poll everyone else. Once I’ve taken over the world, everyone will be pushing messages to me, and I can deprecate the poll interface. Genius.

Recently, most of the crazy apps I’ve put here have been labelled as ‘proof of concept’. This one is different. This one probably won’t even build on your computer. I’m putting things up here as a was of musing about technique. For instance, Dashboard had a far better design than this app. It had a nice pipeline thing going for it, whereas I just have a class per foreground application, this class must produce an Address Book record, then I just interrogate every context producer for information and display it. This is silly - if I’m looking at Paul’s Flickr photos page, I don’t need my app showing me the thumbnails again, I might be much more interested in where he is right now. Hell, in a perfect world, it would work out the dates of the photos I’m looking at, and show me where he was at that time.

Future

Clever things I could (and want to) do:

• If the foreground URL doesn’t belong to a user, look for hcard markup in the source HTML and try to derive a person from that. Right now, for instance, I’ll only recognise your Flickr page as belonging to you if it’s one of the URLs against your address book card. But Flickr pages are marked up with enough hcard that I should be just able to figure it out.

• More intelligence around context - as above, if I’m looking at a blog of a friend, I want to see other things, not their blog again.

• Remembering connections - if I figure out a local person from a Flickr page via hcard markup rather than an Address Book URL, why not remember their Flickr username and display their photos when they email me?

Many of these features are difficult, mostly because of my core design right now - I derive an Address Book entry from the current application, then derive context from that entry. This hampers cleverness somewhat - I really need to pass around a lot more information about how I derived this person, and keep a local cache of conclusions about them. Maybe the person isn’t in my address book - I get email from people I don’t know! But their email address might correspond to a Gravatar so I could show a picture of them. Maybe the mail has some URLs in the .sig and I could find their blog. Maybe they’ve commented on my blog in the past and I’d like links to the comments. Likewise, if I find, via hcard in the source of a page, that a page is about someone I know, should I update Address Book and add URLs for them? Probably not a good idea. So I need a local store of connections as well.

Now what?

I don’t know. It’s very tempting to rewrite the thing in Python before it gets any more complex. Partially this is because the Ruby feedparser dependencies are a bugger, but mostly it’s because I don’t want my python sk1llz to atrophy down to nothing. Recently everything I do is in Ruby, and I don’t like that. Shelf also desperately needs some work done to make it asynchronous, and cache things - when I look at an email right now, it’ll hang for 5 minutes while it goes off and fetches 20 RSS feeds, every time I change the person I’m looking at. Not exactly pleasant. But the ‘find out about a person’ is really just a trivial example of the sort of things you can do once you know who they are. The ‘derive context from current machine state’ side of things is much more interesting.

Right, Shelf has now reached version 0.0.6 - download it (there are newer versions out now - get those). It’s good enough that I’m running it full time now. Thanks to Mark Fowler, it can now pull clues from Firefox, which is a relief. I’ve also added Address Book and iChat support, although the iChat stuff is a little hokey - it assumes you’re not using tabbed chats, and that you speak English. Sorry. The iChat AppleScript dictionary is lousy.

Musings

It’s been suggested that I could work out twitter feed and Flickr photostream URLs about people based on their name / nick / email. I’m currently shying away from deriving too many things about a person magically. For instance, I could work out (and cache, obviously) a Flickr username for a person from their email address. Quite apart from the horrible privacy implications of sending the email addresses of everyone you read mail from to Flickr, I just don’t like the approach. I’d much rather encourage a rich address book with lots of data in it. This has the side-effect that Shelf will also recognise my Flickr page as belonging to me.

Amazingly, Shelf is still fun to work on. Hence version 0.0.10. Read the release notes.

Well, as of next month, Facebook developers can no longer put things in a feed unless it’s going in as a direct result of an action that the owner of that feed performed on a facebook page.

Now, this is clearly spun as trying to prevent ‘you’ve been given a beer’ / ‘you’ve been bitten by a zombie’, etc messages. Not letting apps belonging to another person put entries into my feed is a good thing. But the documentation page says

“

Note that you should not be using a session_key for a user who is not actively using your application when calling this method.

”

Now, I read this as, you now can’t use a background process to put entries into a user’s feed using a session_id. You can only put things into their feed if they’re looking at your app while you’re doing it. And this is irritating. It completely breaks all the apps I use that syndicate the outside world into my Facebook profile - Twitter, delicious, CPAN, Dopplr, etc.

It seems that this is an effort to stop users like me, who just want to use existing services, but centralize update notification for the friends I have that don’t want to obsessively subscribe to 20 RSS feeds. Naturally, facebook don’t like me, because I don’t see, and therefore click, their ads. Forcing me to visit the site to tell my friends anything ‘fixes’ this.

Maybe I’m paranoid. Maybe I’m mis-reading ‘actively’, and they’re ok with installed apps using session_ids of users who aren’t logged in and using that app at this precise moment. I hope so.

The ‘state of the art’ for iPhone apps is a single URL, serving a static page with lots of JavaScript and Ajax. Clicking (touching, whatever) things loads in fragments and changes the page. IUI works like this, Hahlo works like this, the Facebook app works like this (hence IUI‘s behaviour).

To preserve state, normal usage is to use the URL fragment to add bookmarkability and history. And sometimes, this actually works. But given that the point is to pretend to be a native iPhone application, it’s wrong. You should be storing state in cookies.

Justification: Native iPhone apps act like you never quit them. Even if they get closed by the system at some point, they’ll come back to the state you left them in. Bookmarking a rich web application should act the same - I want to bookmark the application, and have it open in the state that I left it, not the state that I bookmarked it in. So you should update the ‘current state’ in a local cookie every time to navigate somewhere, and respect that state when you next visit the application. Combine this with the 1.1.3 firmware’s webclips thing and you can almost pretend to be native.

Which is why I expect the much-anticipated iPhone SDK to be nothing more than ‘local web applications’. Give developers a little bit of local storage (you know, like webkit just got), a way of promoting a bookmark to the home screen (we have that one now) and a way of storing some HTML and JS on the phone, and Jobs can claim he’s given us an SDK. And he’ll be right.

Me? I’d be happy with that. It solves all the sandboxing, security, ‘bring down the network’, etc problems. And it will keep people from jailbreaking the phone trivially. The only alternative I can see is installation of signed apps only, with the iTunes Media Store as the single point of installation. Which would suck more. But there will be howls of outrage.