tlogger: Capture click-stream web browsing logs

February 13, 2009

If you’re reading this, chances are that you’ve heard about the web browsing study I’m doing for my master’s thesis. If not, you might want to check out the summary of my talk at Mozilla, the responses to the talk from Jono and Boriss, or just check out the posts under the “research” category.

Since my talk, a few people have contacted me to ask about exactly how I did the logging, and did I notice this, or did people do that. Unfortunately, I can’t release my raw data, but I decided to do the next-best thing: release my logging tool. I humbly present tlogger for your consideration:

tlogger is a Firefox extension for capturing click-stream web browsing logs. In other words, it collects data about how the browser is used. Mainly it records navigation events and tab events, as well as the UI actions that cause those events. It’s roughly similar to the Spectator extension, but with a few key differences:

  • it’s compatible with Firefox 2 and 3

  • it doesn’t submit ANY data automatically, to anyone. Everything stays on in your profile directory, in a human-readable format.

  • URLs are obfuscated on a per-user basis. From the log file, someone can see when the user revisits a site or a URL, but there is no way to determine what the actual URL is. It’s also not possible to make comparisons between users.

  • it can log a few things that Spectator can’t, like when javascript on a web page changes window.location.href.

The source code is managed on GitHub at http://github.com/pdubroy/tlogger/. For the impatient, you can install the latest version of the extension or grab a snapshot of the repository. In addition to the source code for the Firefox extension, the git repository contains tools for analyzing the log files generated by tlogger.

tlogger is useful for anyone who needs real data about how people use Firefox. Of course, it’s perfect if you’re doing a field study on web browser usage, but it’s also useful for prototyping new UI features for Firefox. Liz Blankenship has already used it for her tabviz project, and discovered some interesting things about her own web browsing habits.

Enjoy! If you find it useful, or if you have any questions, send ‘em my way (email to pat, at the domain dubroy.com). If you make changes, send me a pull request on GitHub.