Work on the assistive technologies (Task Views & Shion + Pennyworth) has been steady and I’m hoping to release some updates to Pennyworth client applications soon.

Not too much has changed with Pennyworth proper. After some consideration and given some future projects coming down the pipeline, I’ve begun planning out Pennyworth 2.

In short, PW2 will be a re-implementation of the Pennyworth architecture designed to work on multiple platforms and devices. The plan is to implement the core of the system in cross-platform C and use native languages and tools to wrap the core in a user interface that “fits” on the platform.

I’m taking this approach (as opposed to the older approach of rewriting the system by scratch for each platform) for a couple of reasons:

1. As a solo developer, I don’t have the time or energy to manage and support full ports to each platform in each respective language. Assuming that the average size of a Pennyworth port is N lines of code, and I port it to P platforms, I need to manage N * P lines of code. This is too much code to maintain and continuously debug.

2. While some code will necessarily be unique to each port, I believe that the bulk will be portable across platforms. If 25% of the Pennyworth code is generic in this sense, the amount of code to manage becomes 0.25N + 0.75P By keeping N somewhat constant and only growing the P, the software development side of things will scale better as I add support for each platform.

The current Pennyworth implementation has approximately 10000 lines of code. Of that code, about a third (0.33) is a good candidate for the platform-neutral core. Using the assumption that other implementations will have a similar amount of code in the same proportion, the project will scale in the following way:

1 port: 10000 LOC (both old approach & new approach)

2 ports: 20000 LOC (old) vs. 16700 LOC (new)

3 ports: 30000 LOC (old) vs. 23400 LOC (new)

4 ports: 40000 LOC (old) vs. 30100 LOC (new)
(The 4th port is almost free when compared to the older system.)

5 ports: 50000 LOC (old) vs. 36800 LOC (new)

Since I have more 3 ports that I would like to implement, this change makes sense.

3. I also believe that sharing the core code will allow me to implement a more robust and efficient system across platforms. By including resource-constrained devices (such as mobile phones) to the mix, it easier to notice inefficient and unstable code.

To be honest, this effort is in its early phases and I probably won’t have anything ready for testing until later this year. However, I do think that the short-term pain will result in long term gain as I develop a framework and approach for getting Pennyworth on more devices.

Thank you for your patience.

The application is called “Task Views” and it attempts to solve the problem that modern workers face when managing their tasks and todos. There are a variety of great task management applications (desktop and web alike), but the realities of the modern workplace is that we find ourselves using a variety of solutions to attack the same problem: getting things done.

In my own personal life, I use FogBugz to track software tasks, a Basecamp site to collaborate with clients, and OmniFocus to handle everything else. The problem with splitting things up in this way is that it leads to tunnel vision. When I’m working in FogBugz, I’ll plow through a variety of software issues, while forgetting to pick up groceries on the way home. When I’m in OmniFocus, I find it easy to overlook urgent tasks for my Basecamp client. I get things done, but often find my triumph short-lived as I discover that I didn’t do everything I needed to do, simply because I forgot to look at my other task applications.

Task Views solves this problem by offering a set of unified views that transcend individual systems. The program accomplishes this using conduits. A conduit is a bit of code that pulls tasks from another system, normalizes the fields, and imports it into the current system.

Task Views currently talks to OmniFocus, Things, iCal, Basecamp, Entourage, and FogBugz. (Support for other systems is in the works.) The conduits fetch task information and dump it into a local database.

The program provides a very basic list view that allows users to quickly identify and view individual tasks. However, as the image above shows, this list can grow really long, really quickly. To alleviate this problem, Task Views provides a filtering function that allows users to select subsets of the database:

Users can construct elaborate filters by using simple Boolean rule-based logic.

The list view shows the main weakness of the current crop of task management applications. Almost without exception, all of them use a list metaphor to display what needs doing. Even on the largest monitors, tasks scroll off the visible screen rather quickly.

A cloud view provides a simple way of displaying more content in a more space-effective manner. Where as the list above shows less than a hundred items, the cloud view shows over seven hundred. However, the items are so small, that this view is clearly suboptimal.

Task Views allows the user to bind the size of the text to a numeric or date value associated with the tasks. In the screenshot above, the date the tasks were created is bound to the text size. Tasks created farther in the past appear larger than more recently created items.

Categorical and text values may be bound to the color of the text items. In the screenshot above, I’ve bound a variety of colors to various projects. Color makes salient which projects need more attention than others.

In addition to the list and cloud views, Task Views also provides a histogram view of the task space.

The view above shows the number of tasks associated with particular projects. As the cloud view demonstrated, I have many tasks still associated with the Jarvis project.

The histogram view also provides a useful temporal perspective of the task space. In the image above, the height of the bars represent how many tasks were created on a given day. This could as easily show how many tasks were completed within a particular duration in order to better understand personal productivity patterns.

Not pictured above are the task update notifications. Task Views periodically refreshes itself and when a task changes, the application will alert you using an unobtrusive Growl notification. This is useful for discovering that a coworker has elevated the priority of particular item without taking the time to inform you.

In upcoming releases, I plan to add new views that draw upon the data visualization research and literature.

The Context Angle

Since this blog is focused on context-aware computing, there is a context angle here. I speculate that while Task Views provides a variety of views and configurations, not all of them will be appropriate in all cases. A large histogram view may be useful for when the user is planning their week, but a smaller cloud view my be more appropriate throughout the working day. Task Views is explicitly designed to be dynamically reconfigured using systems like Pennyworth. Currently, the context-sensitive portions of the application are

1. The active filter: The list of visible tasks may vary with the current activity. For example, when programming, show the incomplete items from the FogBugz database. When collaborating, show items from Basecamp. When planning the upcoming week, default to OmniFocus.

2. The active view: A larger histogram is useful when explicitly planning. When doing other things, a more ambient cloud view may be more effective.

3. Individual view configurations: Date histograms may be more effective when planning, while categorical histograms may be more effective when triaging. In some cases, it may be useful to bind the size of the text in a cloud view to the date, in other cases, priority.

At the moment, I’m in the process of prototyping and testing this system with users with ADHD. If you’re interested in testing it yourself, post a note in the comments and I’ll get back to you. The software isn’t quite ready for release yet, but I am interested in recruiting more testers.

From an e-mail I sent out earlier today:

You’re receiving this mass message because I’ve been in touch with you about the Windows port of the Pennyworth system (or you belong to one of the NU mailing lists).

I wanted to let you know that while I’m several days late, the Windows port of the Pennyworth architecture is now online at

http://pennyworth.aetherial.net/jarvis/ 

Jarvis represents my initial efforts at bringing the context-sensing approach embodied in Pennyworth to the platform(s) that the rest of the world is using. :-)

This is a very early release and at the moment, it can observe you, learn context models from your corrections, and predict your context in real-time. That’s about all that it does at the moment. In next several weeks, I will be adding new features and functionality in order to achieve feature parity with Pennyworth.

The current missing features include:

• The ability to modify other applications based upon context changes.
• The ability to solicit context information from other applications.
• The rules interface for overriding the machine learner.
• An interface that exposes the learned context model for inspection.
• Application auto-updating.
• Sharing context to networked clients (including Pennyworth).
• The ability to tweak or reset the machine learners.

My main goal in releasing this initial version is to begin to identify any brittleness in the system that may be a result of any of the combinations of hardware and software that you use. In other words, this is the “test and see if it crashes on others’ machine” release. If all goes well and the current set of functionality is stable and responsive on your systems, I’ll move onto the next steps of adding the missing functionality above.

I’m quite excited to be working on this and I’m looking forward to receiving your feedback on Jarvis. Ideally, I’ll be able to turn your feedback around quickly and do a series of releases every couple of weeks between now and June. By June, I hope to have iterated the system enough times that it’s as stable and full of features as Pennyworth. Once I’ve achieved that goal, I’ll be in touch with some of you to talk about how we can make Jarvis talk with your applications in order to start building a smart context-aware Windows environment.

Thanks for your patience, and please let me know what you think.

Please check it out if you’re planning on submitting bugs for Pennyworth or related applications. It’ll help me resolve issues and get fixes back to you much more quickly.

This release includes minor bug fixes, a new detailed report, support for user-specified date ranges, and a hot key (Command-Control-P) for quickly updating manual streams.

Prerequisites

MacOS X 10.5 “Leopard”

Source

There is not a source distribution available yet, but if you’d like the source code, it is available via Subversion at

The source requires Xcode 3 to build.

Bugs, suggestions, or comments?

Please post any feedback in the comments below or send it to pennyworth@aetherial.net. This is early release software, so there will be bugs, and the application will only improve when problems are identified and submitted.

The Pennyworth Punch Clock (PPC) is a time tracking utility for monitoring the predictions produced by Pennyworth (activity, location, social context) along with other relevant events (active applications, current websites). PPC uses a combination of automatic and manual logging to create useful reports about how you spend your time.

The software is available on the Pennyworth website. I don’t have many instructions for getting started posted, but I will be working on those this weekend. However, if you have any questions or comments, please let me know via e-mail or in the comments below. This is an early release of an application that has a lot of potential, so I appreciate all the feedback that I receive.

I’ve gone ahead and added the following to my PPC to-do list:

1. Application Stream: This will be an optional time stream (enable in the preferences) that can be turned on to track which applications are active at any given slice of time.

2. URL Stream: This will be an optional time stream that tracks the current URL of the Safari and Firefox web browsers. I’ll probably break this into two time streams, one for each browser.

3. Rename the “Statistics” tab to “Reports”: I plan on implementing this as an en export option that spits out your time streams as an XML file. Scripts and other programs can be run on this file to generate specific reports of how time has been spent. I’ll include some options for dropping such scripts in the application support folder for running the reports straight from the interface.

4. AppleScript support: Initially this will be implemented as two main classes of commands: reading and writing. Reading will allow calling scripts to get a full lists of slices in a given time slice (“tell app ‘PPC’ to return every slice of the timestream named ‘Foo’). I’m also planning on implementing a feature where calling AppleScripts can log slices to streams (“tell app ‘PPC’ to log ‘Eating’ to timestream named ‘Foo’).

Fortunately (with the exception of the reporting functionality), I’ve already written most of this code for other projects. I’m going to keep plugging away and see if I can have an early beta version released sometime next week.

One of my major tasks over the summer is to conduct a directed search for the killer app that really lets Pennyworth shine. I’m following the Growl model of releasing the system as a service that is accessible to other applications, and I need to find Pennyworth’s Adium.

While I wait for a project proposal to snake through the appropriate review boards (a cost of doing business in academia), I’ve been thinking about applications that I could release to the general public that helps me explore this space to begin to understand what people want and need. I am happy to say that I’ve been working on a fairly simple application that readers of this weblog might find useful.

Pennyworth Punch Clock (PPC) is a time tracking application that uses Pennyworth to help you recall and analyze how, where, and with who you spend your time with. PPC does this by using the idea of time streams and time slices. A time stream is a timeline that describes the temporal properties of one aspect of your person. For example, a location stream might look like “home, train, school, park, school, train, home”. The elements of the stream are called time slices. In other words, a stream is an ordered collection of slices.

By default Pennyworth exposes three time streams: Activity, Location, and Social Context. PPC listens for changes in these streams and logs them automatically. However, you will likely prefer to define other streams that you populate automatically: projects, official roles, billing rates, phone calls, and so on. To be honest, I expect that my notion of useful time streams differs greatly from the next person, and PPC is designed to accommodate these differences by being as general as possible.

The application is still in its early stages of development, but I wanted to publish a few screenshots of the work in progress to spark a conversation about who might consider using this software and how.

This first window shows the main time log. Time streams are in the upper half of the split pane and slices logged to each stream are visible in the lower half. In this screenshot, I have the “Activity” stream selected. The logs may be filtered by time as shown with the pulldown containing various options for narrowing or expanding the scope of the log.

In user-created streams (“Activity” is not one of them), you can log new slices to a stream by clicking the “Update Selected Stream” button. This invokes a small dialog where you can select a previously-used slice name or enter a new one.

In the preferences tab, you can create or remove new user-defined streams in addition to setting the standard update checking options. In this screen, I’ve created a “Project” stream that I can use to tag time periods when I’m working on different projects. If you click the magnifying glass button, the “Projects” time stream details appear, including an associated color and a list of saved values:

This is some of the more mundane aspects of the interface, and I’d be interested in learning more from you about what you might find useful in an application such as this. Personally, I plan on using this application to accomplish the following tasks:

• Create a finer-grained log of how I spend my time. In particular, I am interested in tracking how much time I spend on different projects for different “clients”.

• Better understand how I spend my time throughout the day. I’d like to have some hard data about how much time I spend dealing with e-mail, idly surfing the web, and so on in order to be able to identify and fix particular problem areas. I’d also like to better understand when I am doing these things (during lunch, late afternoon, etc.).

• Generate better time estimates for getting things done. At the moment, I don’t have enough information about how long it takes to do something like set up a web site, write a 10 page paper, etc. to come up with anything better than off-the-cuff predictions about how long something similar will take in the future.

Once I get the statistical and reporting features of the application in decent shape, I plan on thinking about how to make this application proactive in the sense of providing prompts to the user. For example, I may set up a built in prompt that nags me when the system detects I’m playing games, but I haven’t allocated time for gaming. Another prompt would be one that encourages me to do a little exercise and stretch after programming for a long stretch to reduce the chances of RSI.

I am interested in hearing from any interested readers out there. If you had a convenient and flexible tool for tracking and tagging how you spend your time, what would you do with it?

First of all, you can query the system for current predictions and observations:

tell application "Pennyworth"
  value of prediction named "Location"
end tell

might return something like Home. Likewise,

tell application "Pennyworth"
  value of observation named "Desktop Observer (Running)"
end tell

might return something like Safari.

Secondly, if you use other apps that call AppleScripts in response to events, you can now instruct the system to log the observation using

tell application "Pennyworth"
  log "My Observation" from observer "My Custom Observer" with duration 10
end tell

This is equivalent to an internal observer logging observations. The “from observer” and “with duration” parameters are optional. The system will choose reasonable defaults if left empty.

This functionality has been requested from a few users, so I’m glad that it is now working. If you have any comments or questions, please post them in the comments below.

The product in question is MarcoPolo, a context-sensing application that uses sensors to detect the user’s “context” and take actions accordingly. The interface design is eerily similar to my own and there is quite a deal of overlap of functionality between the two apps. However, there are a few significant differences:

1. MarcoPolo uses a rule-based system to make sense of observations. MarcoPolo does not learn from user-provided labels, but relies upon the user to define rules that define a given context. Genkanban uses an ID3 decision tree (or a naive Bayes classifier) to accomplish the same goal.

2. MarcoPolo seems to have a more system-oriented action functionality than Genkanban. Recall that Gankanban can initiate an arbitrary action using either distributed notifications (and listening apps) or executing AppleScripts on context changes. MarcoPolo includes a number of specific actions that are oriented toward configuring the local system for the new context. This includes setting up the network, configuring the default printer, etc. MarcoPolo includes a ShellScript action type that allows shell scripts to be executed on context changes. This can conceivably encompass Genkanban’s AppleScript functionality. (Update: ActionScripts can be directly executed using the “Open” action.)

3. MarcoPolo seems oriented toward the idea that location is largely the same as context. Genkanban takes a bigger picture view and explicitly looks at location, activity, and social contexts as discrete elements of context. This finer-grained formulation allows Genkanban to take specific actions based on changes in the specific components. For example, I only care about activity when updating my Adium status message. I probably only care about location when deciding whether to activate the system screensaver.

That said, both Genkanban and MarcoPolo are more similar than different. The way I see things, components like Genkanban’s machine learner can be replaced with MarcoPolo’s rule-based system without much fuss (and vice versa). Genkanban’s sensors work much like MarcoPolo’s, and there’s probably no good reason that a common sensor interface couldn’t be be written so both systems’ sensors are interchangeable. I couldn’t tell if MarcoPolo is issuing distributed notifications on context changes, but this may be another element that can be standardized.