The Pennyworth Project

Just another WordPress site

Project Update

It’s been a few months since my last update, so I wanted to post a quick note to let everyone know what’s going on.

Work on the assistive technologies (Task Views & Shion + Pennyworth) has been steady and I’m hoping to release some updates to Pennyworth client applications soon.

Not too much has changed with Pennyworth proper. After some consideration and given some future projects coming down the pipeline, I’ve begun planning out Pennyworth 2.

In short, PW2 will be a re-implementation of the Pennyworth architecture designed to work on multiple platforms and devices. The plan is to implement the core of the system in cross-platform C and use native languages and tools to wrap the core in a user interface that “fits” on the platform.

I’m taking this approach (as opposed to the older approach of rewriting the system by scratch for each platform) for a couple of reasons:

1. As a solo developer, I don’t have the time or energy to manage and support full ports to each platform in each respective language. Assuming that the average size of a Pennyworth port is N lines of code, and I port it to P platforms, I need to manage N * P lines of code. This is too much code to maintain and continuously debug.

2. While some code will necessarily be unique to each port, I believe that the bulk will be portable across platforms. If 25% of the Pennyworth code is generic in this sense, the amount of code to manage becomes 0.25N + 0.75P By keeping N somewhat constant and only growing the P, the software development side of things will scale better as I add support for each platform.

The current Pennyworth implementation has approximately 10000 lines of code. Of that code, about a third (0.33) is a good candidate for the platform-neutral core. Using the assumption that other implementations will have a similar amount of code in the same proportion, the project will scale in the following way:

1 port: 10000 LOC (both old approach & new approach)

2 ports: 20000 LOC (old) vs. 16700 LOC (new)

3 ports: 30000 LOC (old) vs. 23400 LOC (new)

4 ports: 40000 LOC (old) vs. 30100 LOC (new)
(The 4th port is almost free when compared to the older system.)

5 ports: 50000 LOC (old) vs. 36800 LOC (new)

Since I have more 3 ports that I would like to implement, this change makes sense.

3. I also believe that sharing the core code will allow me to implement a more robust and efficient system across platforms. By including resource-constrained devices (such as mobile phones) to the mix, it easier to notice inefficient and unstable code.

To be honest, this effort is in its early phases and I probably won’t have anything ready for testing until later this year. However, I do think that the short-term pain will result in long term gain as I develop a framework and approach for getting Pennyworth on more devices.

Thank you for your patience.

Posted by admin at 12:59 pm on July 11, 2009 . Filed under Meta,Other Apps,Task Views.

Introducing Task Views

Over the past several weeks, I’ve been Twittering about a new application that has been under construction. I’m happy to now lift the sheet and share what I’ve been working on.

The application is called “Task Views” and it attempts to solve the problem that modern workers face when managing their tasks and todos. There are a variety of great task management applications (desktop and web alike), but the realities of the modern workplace is that we find ourselves using a variety of solutions to attack the same problem: getting things done.

Read more…

Posted by admin at 10:51 pm on March 25, 2009 . Filed under Other Apps,Pennyworth,Task Views.

Introducing Jarvis

It’s been a while since I’ve updated this weblog. I’ve had a busy couple of months, and the following message explains (some of) what I’ve been up to.

From an e-mail I sent out earlier today:

You’re receiving this mass message because I’ve been in touch with you about the Windows port of the Pennyworth system (or you belong to one of the NU mailing lists).

I wanted to let you know that while I’m several days late, the Windows port of the Pennyworth architecture is now online at 

Jarvis represents my initial efforts at bringing the context-sensing approach embodied in Pennyworth to the platform(s) that the rest of the world is using. 🙂

This is a very early release and at the moment, it can observe you, learn context models from your corrections, and predict your context in real-time. That’s about all that it does at the moment. In next several weeks, I will be adding new features and functionality in order to achieve feature parity with Pennyworth.

The current missing features include:

  • The ability to modify other applications based upon context changes.
  • The ability to solicit context information from other applications.
  • The rules interface for overriding the machine learner.
  • An interface that exposes the learned context model for inspection.
  • Application auto-updating.
  • Sharing context to networked clients (including Pennyworth).
  • The ability to tweak or reset the machine learners.

My main goal in releasing this initial version is to begin to identify any brittleness in the system that may be a result of any of the combinations of hardware and software that you use. In other words, this is the “test and see if it crashes on others’ machine” release. If all goes well and the current set of functionality is stable and responsive on your systems, I’ll move onto the next steps of adding the missing functionality above.

I’m quite excited to be working on this and I’m looking forward to receiving your feedback on Jarvis. Ideally, I’ll be able to turn your feedback around quickly and do a series of releases every couple of weeks between now and June. By June, I hope to have iterated the system enough times that it’s as stable and full of features as Pennyworth. Once I’ve achieved that goal, I’ll be in touch with some of you to talk about how we can make Jarvis talk with your applications in order to start building a smart context-aware Windows environment.

Thanks for your patience, and please let me know what you think.

Posted by admin at 6:07 pm on February 2, 2009 . Filed under Jarvis,Other Apps.

How to report bugs

I ran across Steven Frank’s recommendations for submitting bug reports a couple of days ago.

Please check it out if you’re planning on submitting bugs for Pennyworth or related applications. It’ll help me resolve issues and get fixes back to you much more quickly.

Posted by admin at 5:53 am on September 8, 2008 . Filed under Meta,Other Apps,Pennyworth,Pennyworth Punch Clock.

Pennyworth Punch Clock 1.0b3

Pennyworth Punch Clock 1.0b3 is now available for download.

This release includes minor bug fixes, a new detailed report, support for user-specified date ranges, and a hot key (Command-Control-P) for quickly updating manual streams.


MacOS X 10.5 “Leopard”


There is not a source distribution available yet, but if you’d like the source code, it is available via Subversion at

The source requires Xcode 3 to build.

Bugs, suggestions, or comments?

Please post any feedback in the comments below or send it to This is early release software, so there will be bugs, and the application will only improve when problems are identified and submitted.

Posted by admin at 6:34 pm on July 30, 2008 . Filed under Other Apps,Pennyworth,Pennyworth Punch Clock.

Pennyworth Punch Clock 1.0b1

After a decent amount of work, I’m happy to announce the release of the first application designed specially for Pennyworth: the Pennyworth Punch Clock.

The Pennyworth Punch Clock (PPC) is a time tracking utility for monitoring the predictions produced by Pennyworth (activity, location, social context) along with other relevant events (active applications, current websites). PPC uses a combination of automatic and manual logging to create useful reports about how you spend your time.

The software is available on the Pennyworth website. I don’t have many instructions for getting started posted, but I will be working on those this weekend. However, if you have any questions or comments, please let me know via e-mail or in the comments below. This is an early release of an application that has a lot of potential, so I appreciate all the feedback that I receive.

Posted by admin at 3:23 pm on July 24, 2008 . Filed under Other Apps,Pennyworth Punch Clock.

Updated thoughts (and to-do’s) regarding the Pennyworth Punch Clock

Thanks everyone who sent in comments and suggestions. They were immensely helpful.

I’ve gone ahead and added the following to my PPC to-do list:

1. Application Stream: This will be an optional time stream (enable in the preferences) that can be turned on to track which applications are active at any given slice of time.

2. URL Stream: This will be an optional time stream that tracks the current URL of the Safari and Firefox web browsers. I’ll probably break this into two time streams, one for each browser.

3. Rename the “Statistics” tab to “Reports”: I plan on implementing this as an en export option that spits out your time streams as an XML file. Scripts and other programs can be run on this file to generate specific reports of how time has been spent. I’ll include some options for dropping such scripts in the application support folder for running the reports straight from the interface.

4. AppleScript support: Initially this will be implemented as two main classes of commands: reading and writing. Reading will allow calling scripts to get a full lists of slices in a given time slice (“tell app ‘PPC’ to return every slice of the timestream named ‘Foo’). I’m also planning on implementing a feature where calling AppleScripts can log slices to streams (“tell app ‘PPC’ to log ‘Eating’ to timestream named ‘Foo’).

Fortunately (with the exception of the reporting functionality), I’ve already written most of this code for other projects. I’m going to keep plugging away and see if I can have an early beta version released sometime next week.

Posted by admin at 5:37 am on July 15, 2008 . Filed under Other Apps,Pennyworth.

Introducing Pennyworth Punch Clock

One of the simultaneously exciting and frustrating things working on Pennyworth is the system’s potential. On one hand, I think that the Pennyworth system has within it the right stuff to really make computing more convenient, effective, and less frustrating. On the other hand, I can’t really demonstrate how this vision “works” without some sort of obvious application. Pennyworth by itself is able to take action on context changes using the built-in AppleScript functionality, but regular people don’t seem to really get this.

One of my major tasks over the summer is to conduct a directed search for the killer app that really lets Pennyworth shine. I’m following the Growl model of releasing the system as a service that is accessible to other applications, and I need to find Pennyworth’s Adium.

While I wait for a project proposal to snake through the appropriate review boards (a cost of doing business in academia), I’ve been thinking about applications that I could release to the general public that helps me explore this space to begin to understand what people want and need. I am happy to say that I’ve been working on a fairly simple application that readers of this weblog might find useful.

Pennyworth Punch Clock (PPC) is a time tracking application that uses Pennyworth to help you recall and analyze how, where, and with who you spend your time with. PPC does this by using the idea of time streams and time slices. A time stream is a timeline that describes the temporal properties of one aspect of your person. For example, a location stream might look like “home, train, school, park, school, train, home”. The elements of the stream are called time slices. In other words, a stream is an ordered collection of slices.

By default Pennyworth exposes three time streams: Activity, Location, and Social Context. PPC listens for changes in these streams and logs them automatically. However, you will likely prefer to define other streams that you populate automatically: projects, official roles, billing rates, phone calls, and so on. To be honest, I expect that my notion of useful time streams differs greatly from the next person, and PPC is designed to accommodate these differences by being as general as possible.

The application is still in its early stages of development, but I wanted to publish a few screenshots of the work in progress to spark a conversation about who might consider using this software and how.

Pennyworth Punch Clock: Main Screen

This first window shows the main time log. Time streams are in the upper half of the split pane and slices logged to each stream are visible in the lower half. In this screenshot, I have the “Activity” stream selected. The logs may be filtered by time as shown with the pulldown containing various options for narrowing or expanding the scope of the log.

In user-created streams (“Activity” is not one of them), you can log new slices to a stream by clicking the “Update Selected Stream” button. This invokes a small dialog where you can select a previously-used slice name or enter a new one.

Pennyworth Punch Clock: Preferences

In the preferences tab, you can create or remove new user-defined streams in addition to setting the standard update checking options. In this screen, I’ve created a “Project” stream that I can use to tag time periods when I’m working on different projects. If you click the magnifying glass button, the “Projects” time stream details appear, including an associated color and a list of saved values:

Pennyworth Punch Clock: Stream Editor

This is some of the more mundane aspects of the interface, and I’d be interested in learning more from you about what you might find useful in an application such as this. Personally, I plan on using this application to accomplish the following tasks:

  • Create a finer-grained log of how I spend my time. In particular, I am interested in tracking how much time I spend on different projects for different “clients”.

  • Better understand how I spend my time throughout the day. I’d like to have some hard data about how much time I spend dealing with e-mail, idly surfing the web, and so on in order to be able to identify and fix particular problem areas. I’d also like to better understand when I am doing these things (during lunch, late afternoon, etc.).

  • Generate better time estimates for getting things done. At the moment, I don’t have enough information about how long it takes to do something like set up a web site, write a 10 page paper, etc. to come up with anything better than off-the-cuff predictions about how long something similar will take in the future.

Once I get the statistical and reporting features of the application in decent shape, I plan on thinking about how to make this application proactive in the sense of providing prompts to the user. For example, I may set up a built in prompt that nags me when the system detects I’m playing games, but I haven’t allocated time for gaming. Another prompt would be one that encourages me to do a little exercise and stretch after programming for a long stretch to reduce the chances of RSI.

I am interested in hearing from any interested readers out there. If you had a convenient and flexible tool for tracking and tagging how you spend your time, what would you do with it?

Posted by admin at 4:27 pm on July 14, 2008 . Filed under Other Apps,Pennyworth.

New AppleScript features coming in 1.0b5

It took me long enough, but I’m happy to report that the next version of my context sensing toolkit will include some new functionality provided via AppleScript.

First of all, you can query the system for current predictions and observations:

tell application "Pennyworth"
  value of prediction named "Location"
end tell

might return something like Home. Likewise,

tell application "Pennyworth"
  value of observation named "Desktop Observer (Running)"
end tell

might return something like Safari.

Secondly, if you use other apps that call AppleScripts in response to events, you can now instruct the system to log the observation using

tell application "Pennyworth"
  log "My Observation" from observer "My Custom Observer" with duration 10
end tell

This is equivalent to an internal observer logging observations. The “from observer” and “with duration” parameters are optional. The system will choose reasonable defaults if left empty.

This functionality has been requested from a few users, so I’m glad that it is now working. If you have any comments or questions, please post them in the comments below.

Posted by admin at 4:43 pm on February 19, 2008 . Filed under Other Apps,Pennyworth.

Genkanban and MarcoPolo

The recent Mac Developer Roundtable podcast featured a good deal of information and discussion of context-sensitive interfaces and applications on the Mac platform (start listening around 50:00). I was surprised to hear one of the panel members, David Symonds, talk about a product on the Mac that sounded extremely similar to my own work. I feared that I’d reinvented the wheel and someone had build what I wanted before I even realized it.

The product in question is MarcoPolo, a context-sensing application that uses sensors to detect the user’s “context” and take actions accordingly. The interface design is eerily similar to my own and there is quite a deal of overlap of functionality between the two apps. However, there are a few significant differences:

1. MarcoPolo uses a rule-based system to make sense of observations. MarcoPolo does not learn from user-provided labels, but relies upon the user to define rules that define a given context. Genkanban uses an ID3 decision tree (or a naive Bayes classifier) to accomplish the same goal.

2. MarcoPolo seems to have a more system-oriented action functionality than Genkanban. Recall that Gankanban can initiate an arbitrary action using either distributed notifications (and listening apps) or executing AppleScripts on context changes. MarcoPolo includes a number of specific actions that are oriented toward configuring the local system for the new context. This includes setting up the network, configuring the default printer, etc. MarcoPolo includes a ShellScript action type that allows shell scripts to be executed on context changes. This can conceivably encompass Genkanban’s AppleScript functionality. (Update: ActionScripts can be directly executed using the “Open” action.)

3. MarcoPolo seems oriented toward the idea that location is largely the same as context. Genkanban takes a bigger picture view and explicitly looks at location, activity, and social contexts as discrete elements of context. This finer-grained formulation allows Genkanban to take specific actions based on changes in the specific components. For example, I only care about activity when updating my Adium status message. I probably only care about location when deciding whether to activate the system screensaver.

That said, both Genkanban and MarcoPolo are more similar than different. The way I see things, components like Genkanban’s machine learner can be replaced with MarcoPolo’s rule-based system without much fuss (and vice versa). Genkanban’s sensors work much like MarcoPolo’s, and there’s probably no good reason that a common sensor interface couldn’t be be written so both systems’ sensors are interchangeable. I couldn’t tell if MarcoPolo is issuing distributed notifications on context changes, but this may be another element that can be standardized.

Posted by admin at 11:29 am on January 28, 2008 . Filed under Other Apps,Pennyworth.