The Pennyworth Project

Just another WordPress site

Genkanban and MarcoPolo

The recent Mac Developer Roundtable podcast featured a good deal of information and discussion of context-sensitive interfaces and applications on the Mac platform (start listening around 50:00). I was surprised to hear one of the panel members, David Symonds, talk about a product on the Mac that sounded extremely similar to my own work. I feared that I’d reinvented the wheel and someone had build what I wanted before I even realized it.

The product in question is MarcoPolo, a context-sensing application that uses sensors to detect the user’s “context” and take actions accordingly. The interface design is eerily similar to my own and there is quite a deal of overlap of functionality between the two apps. However, there are a few significant differences:

1. MarcoPolo uses a rule-based system to make sense of observations. MarcoPolo does not learn from user-provided labels, but relies upon the user to define rules that define a given context. Genkanban uses an ID3 decision tree (or a naive Bayes classifier) to accomplish the same goal.

2. MarcoPolo seems to have a more system-oriented action functionality than Genkanban. Recall that Gankanban can initiate an arbitrary action using either distributed notifications (and listening apps) or executing AppleScripts on context changes. MarcoPolo includes a number of specific actions that are oriented toward configuring the local system for the new context. This includes setting up the network, configuring the default printer, etc. MarcoPolo includes a ShellScript action type that allows shell scripts to be executed on context changes. This can conceivably encompass Genkanban’s AppleScript functionality. (Update: ActionScripts can be directly executed using the “Open” action.)

3. MarcoPolo seems oriented toward the idea that location is largely the same as context. Genkanban takes a bigger picture view and explicitly looks at location, activity, and social contexts as discrete elements of context. This finer-grained formulation allows Genkanban to take specific actions based on changes in the specific components. For example, I only care about activity when updating my Adium status message. I probably only care about location when deciding whether to activate the system screensaver.

That said, both Genkanban and MarcoPolo are more similar than different. The way I see things, components like Genkanban’s machine learner can be replaced with MarcoPolo’s rule-based system without much fuss (and vice versa). Genkanban’s sensors work much like MarcoPolo’s, and there’s probably no good reason that a common sensor interface couldn’t be be written so both systems’ sensors are interchangeable. I couldn’t tell if MarcoPolo is issuing distributed notifications on context changes, but this may be another element that can be standardized.

Posted by admin at 11:29 am on January 28, 2008 . Filed under Other Apps,Pennyworth.

Genkanban, the context-sensing application for MacOS X

As I mentioned in my previous post, I am currently creating a context-sensing application for the Mac that will serve as the basis of my research into other context-aware applications. At the moment, that application is Genkanban.

According to my Japanese-English dictionary, the word genkanban is the closest Japanese equivalent to the English word concierge. Like a hotel concierge, Genkanban’s purpose is to watch and learn from you so that it can relay it’s nuanced understanding of your context to other parties that perform actions for you. In this case, Genkanban quietly runs on your computer, observing you. When your context changes (such as a task switch), Genkanban detects the change, calculates your new context, and relays that information on to other applications. For example, if I am switch from programming to writing, Genkanban can instruct iTunes to play something a bit more mellow and update my IM client’s status message accordingly.

Predictions

Before I go any further describing the application, let me first define what I mean by context. The concept of context is easily understood, but sometimes difficult to define concretely. My own understanding of context is informed by the following definitions:

“By context, we mean environmental information that is part of an application’s operating environment and that can be sensed by the application.” Anid K. Dey (The Context Toolkit)

“Context-aware computing promises applications that sense an environment, model situations, and act appropriately.” James Fogarty (Subtle Toolkit)

“The circumstances that form the setting for an event and in terms of which it can be fully understood and assessed.” New Oxford American Dictionary

When Genkanban senses context, it is trying to identify a particular model of the world that surrounds a particular situation. Context is the evidence, and the situation is the conclusion. However, situation can be a complex item in itself, and to maintain my sanity, Genkanban predicts the location, activity, and social context properties of the current situation.

Returning to the application itself, it began life as a Mac implementation of the ideas expressed in James Fogarty’s Subtle Toolkit. At the core of Subtle is a machine learner that receives information from observers, and labels from its user. The machine learner treats the information from observers as features and the label as a classification. Using a collection of features and classifications, the machine learner constructs a model that can predict classifications from unlabeled lists of features. (For more information about the underlying idea behind classification, see the Wikipedia article on statistical classification.)

Observations

To train the machine learner, Genkanban implements a simple user interface that can be invoked as needed (the current prediction is incorrect) or on a fixed interval (“ask me about my context every X minutes”). This small interface collects labels and submits them to the learner to refine the learner’s mode. Users are not limited to a fixed set of labels, and can add new labels as needed.

Training Interface

Genkanban also adds some important functionality to basic ideas expressed in Subtle. Subtle used a relatively constrained interprocess communication (IPC) model where applications could synchronously query the user’s context using XML-RPC calls to a service running on the user’s local machine. Genkanban implements an asynchronous broadcast scheme where it automatically broadcasts a distributed notification of context changes that any application can listen to and adjust itself accordingly. (I’m using MacOS X’s NSDistributedNotification mechanism to accomplish this.) This approach eliminates any need for an application to poll for context, but applications using this IPC method still must be compiled to listen to for the notifications.

Genkanban also implements an action-oriented approach to disseminating context. Using the built-in AppleScript functionality, users can define actions as AppleScript files that are executed when the system predicts a new context. This is crucial for making third-party applications context-aware. The screenshot below illustrates a script that is used to set the status message of the Adium IM client when the local user’s activity changes. Genkanban ships with similar scripts for controlling other applications. These scripts are intended to be used as is, or as starting points for users wishing to customize the scripts for their own environment.

AppleScript Action

Like Subtle, Genkanban includes sensors that can observe various aspects of the local user’s environment. These sensors include wireless network detection, running applications, open websites, connected devices (USB and Bluetooth), and application-specific sensors such as current chat partner or current calendar event. Genkanban also extends this basic model by allowing other applications to contribute observations using the distributed notification approach. Currently, a Caller-ID application (Cidney, to be discussed in a future post) tells Genkanban when the phone is in use. A home automation application (Shion) tells Genkanban when local lights and appliances have been switched on or off. Finally, Genkanban implements an AppleScript interface where users can write their own sensors as scripts and use these custom sensors just as if they were compiled into the application.

Finally, the last major feature of Genkanban is its logging abilities. Since this is a research project for me, I need to collect information about its users to determine how it’s being used, if it’s accurate, and so on. Genkanban can be configured to log snapshots of the current environment at set intervals. (This functionality is disabled by default.) A logged snapshot consists of all of the observations at that point in time, the active predictions at that time, and an optional screenshot. For privacy purposes, users can elect to delete specific logs or disable logging for a fixed period of time.

Logger

This concludes the discussion about what Genkanban is. In a future post, I’ll describe how to use it.

Posted by admin at 11:28 am on January 27, 2008 . Filed under Pennyworth.

The Context Blog leaves the drydock

Welcome to The Context Blog. I began this weblog on a bit of a whimsy when I realized that I didn’t have a good place to discuss some of my ongoing work and thoughts in the area of context-aware applications and adaptive user interfaces. This area is intended to serve as a half-way point between the chaos inside my head and more formal avenues such as conferences, journals, and books.

A bit about me: I am graduate student at Northwestern University in the Media, Technology, and Society MA/Ph.D. program in the School of Communication. My main research interest is conceptualizing, describing, implementing, and deploying new kinds of applications and user interfaces in both traditional desktop and ubiquitous computing environments.

At the moment, I am working on creating context-aware applications on MacOS X. These applications currently include a context-sensing and learning toolkit, a home automation application, and a “smart” CallerID-based notification system for traditional landline phones. Over the next few weeks, I’ll discuss these specific pieces of software in more detail.

In addition to the technology, I am also interested in studying the impact of these technologies in everyday situations and observing how “smarter” applications change (or not) users’ interactions with both their own computers and other users. Will making applications more knowledgeable about their users make them more effective and usable? I think so, but I’m still far from able to make that case definitively, and this humble weblog will chronicle that journey.

In closing, thanks for tuning in and I hope that you’ll stick around for the ride.

Posted by admin at 12:31 am on . Filed under Meta.