tlogger: Capture click-stream web browsing logs

If you’re reading this, chances are that you’ve heard about the web browsing study I’m doing for my master’s thesis. If not, you might want to check out the summary of my talk at Mozilla, the responses to the talk from Jono and Boriss, or just check out the posts under the “research” category.

Since my talk, a few people have contacted me to ask about exactly how I did the logging, and did I notice this, or did people do that. Unfortunately, I can’t release my raw data, but I decided to do the next-best thing: release my logging tool. I humbly present tlogger for your consideration:

tlogger is a Firefox extension for capturing click-stream web browsing logs. In other words, it collects data about how the browser is used. Mainly it records navigation events and tab events, as well as the UI actions that cause those events. It’s roughly similar to the Spectator extension, but with a few key differences:

• it’s compatible with Firefox 2 and 3

• it doesn’t submit ANY data automatically, to anyone. Everything stays on in your profile directory, in a human-readable format.

• URLs are obfuscated on a per-user basis. From the log file, someone can see when the user revisits a site or a URL, but there is no way to determine what the actual URL is. It’s also not possible to make comparisons between users.

• it can log a few things that Spectator can’t, like when javascript on a web page changes window.location.href.

The source code is managed on GitHub at http://github.com/pdubroy/tlogger/. For the impatient, you can install the latest version of the extension or grab a snapshot of the repository. In addition to the source code for the Firefox extension, the git repository contains tools for analyzing the log files generated by tlogger.

tlogger is useful for anyone who needs real data about how people use Firefox. Of course, it’s perfect if you’re doing a field study on web browser usage, but it’s also useful for prototyping new UI features for Firefox. Liz Blankenship has already used it for her tabviz project, and discovered some interesting things about her own web browsing habits.

Enjoy! If you find it useful, or if you have any questions, send ‘em my way (email to pat, at the domain dubroy.com). If you make changes, send me a pull request on GitHub.

My Tab Study: Apropos Links

There’s been lots of interest in the talk I gave at Mozilla last week on the early results of my web browsing study. I’m starting to realize that I’m far from the only one thinking about this stuff. Here are some interesting things I came across in the last week:

Andy Edmonds pointed me to an informal survey done by Dave Munger at Cognitive Daily on how many tabs people use. Dave found that most people had only 2-4 tabs open, and that younger people were likely to have more tabs open. But my favourite part was his finding that if you know who Jonathan Ive, Leo Laporte, and Esther Dyson are — you’re likely to have more tabs open.

Liz Blankenship told me about her project on Tab Visualization for an infoviz course at the University of Michigan. Their goal is to help browser users who tend to have “too many” tabs open at once make sense of the information overload they experience. I’m looking forward to seeing what they come up with.

Also, I realized that I haven’t mentioned anything yet about Mozilla’s Test Pilot project, other than a brief mention in my last post. Test Pilot is a Mozilla Labs program that will allow people to do studies like mine on a massive scale. The goal is to have 1% of Firefox users opt-in to being participants in these kinds of studies. My study had 22 participants. Think hundreds of thousands, or even millions. Pretty cool. I can’t wait to see where it goes.