Dubroy.com/blog

For the past few months, I’ve been knee-deep in data from the tabbed browsing study that I conducted late last year. Now that I’m finishing up my thesis, I figured it’s about time that I share some of my findings. In this post, I’ll talk about one of the quantitative questions I was trying to answer in my study: how many tabs do people use?

Measures

The first thing we need to do is to be a bit more precise with the question. What does it mean to “use” multiple tabs or windows, and how should it be measured? There are few possible answers.

We could simply count the number of tabs that a person creates. One problem with that is that when Firefox starts up, at least one tab is created (and possibly more, if it is restoring a session), so someone who opens and closes their browser frequently would have a high number of tab creation events. To eliminate this effect, we only count tabs that are created after browser startup. We also ignore the first tab that is created in any window.

Another thing that’s interesting to measure is the number of concurrent tabs or windows that a person typically has open. For example, two people might create a similar number of tabs, but one of them might have an email client, an RSS reader, and another tab playing music open at all times. It seems reasonable to say that this person uses tabs more heavily than the other person. We decided to measure the number of windows and tabs that were open whenever a navigation event occurred. For consistency, we ignored all navigation events caused by Firefox’s session restore feature when calculating this measure, although it didn’t have a very significant effect overall.

A third measure of tab usage that might be interesting to look at is the number of tab switches that a person performs, but I’ll address that in a later post. For now, we’ll concentrate on how many tabs.

Study Details

First, I guess I should mention a little bit about how the study was conducted. Here was the recruitment email we sent out to family, friends, and colleagues:

We are seeking participants (at least 18 years old) for a research study exploring how people use web browsers.

If you use Mozilla Firefox for several hours a day, and often use multiple tabs or windows, then you are a candidate for the study. If you choose to participate, you’ll install a Firefox extension that will log various actions, e.g. clicking on a link, visiting a bookmark, opening a new tab, and clicking the back button. Don’t worry, the names and addresses of the web sites that you visit will NOT be revealed to researchers.

The study will last for two weeks. During this time, you’ll take part in five short interviews (approx. 30 minutes each) that will be arranged at your convenience. In addition, you’ll be asked to record some brief notes several times during the course of each day (again, at your convenience) and to email your log file to the researchers at the end of the day. Participating in the study will take about 4-5 hours in total over the two week period.

We also put posters up with similar wording across the University of Toronto campus. 21 people completed the study, and 1 person started it but dropped out early.

Unlike many other studies like this that have been done, our study participants had a variety of ages, professions, and technical skill levels. One participant was younger than 20, 14 were between the ages of 20 and 29, four were 30-39, and two were 50-59. Only 6 participants came from a computer science or engineering background, while others had studied education, environmental science, business, and psychology. Six of the participants were full-time students (either undergraduate or graduate), and 15 were working in some kind of office environment where they spent most of their time on the computer.

Results

Tab Creation

First lets take a look at the tab creation rate. Now, keep in mind that these are only tabs that are created after browser startup, and the first tab in any window is not counted. On the X axis is the “tab creation rate”: the ratio of tabs created to navigation actions. (A navigation action is anything that changes the URL of the page, even if it doesn’t result in a top-level page load.) The height of the bars represents the number of participants with a given tab creation ratio.

Interesting…we’ve got a very clear bi-modal distribution. More than half of our participants (13/21) are clustered around the 0.04 mark. In other words, these people create about 4 tabs for every 100 navigation actions. The rest of the participants are loosely centered around 0.14, meaning that they create about 3 times as many tabs as the other people. In fact, we have two participants who are even higher, creating (respectively) 17 and 22 tabs per 100 navigation actions.

Unsurprisingly, the four highest tab creation rates belong to the four participants with Computer Science and/or programming backgrounds. The other four participants in the high part of the distribution are not your typical “techie” types: one is trained as a civil engineer, and the others have backgrounds in communications, humanities, and marketing.

Concurrent Tabs

Another interesting thing to look at is the number of tabs that people tend to have open at any given time. Again, we used navigation actions as our increment of measurement. On the X axis is the number of tabs open when a navigation action occurs, and the height of the bars it the total number of navigation actions that occurred with the that number of tabs open.

Not too surprising — the most common number of tabs to have open is one, with a pretty steady descent down to 9. It flattens out and hits a valley at 13, but then rises slightly again for a second peak at 16. So, again we see a slight bi-modality to the distribution. But if we take the same set of “tab power users” from the first graph, we see that they have roughly the same profile as all the participants put together. In fact, the peak at 16 is almost entirely caused by only two of the power users: participants 14 and 20.

One possible explanation for the bi-modal distribution here is tab bar scrolling. At a typical screen size on a laptop or desktop, the tab bar can fit about 9-13 tabs without scrolling. Many people probably try to avoid having the tab bar scroll, but once you get to the point where it starts scrolling, there’s little additional cost to having more tabs open. So it may be the P14 and P20 are the only ones comfortable with keeping so many tabs open that the tab bar is scrolling.

Take a look at the median and max number of tabs that each participant had open:

(Note that there’s no participant 8…he’s the one that dropped out.)

Participants 14 and 20 definitely stick out — it’s easy to see why they contributed so much to the yellow portions of the previous chart. Participant 14 by far the highest median number of tabs open with 17, while no other participant had a median higher than 6. Participant 20 actually edges out P14 for max tabs though, with 42. (For once it really is the answer.) Forty-two — that’s a lot of tabs! The tab bar would be scrolling two are three times over at this point. He actually commented on this, saying “Now I am opening tabs up from Digg and they are appearing at the end of my massive list. This is truly a bad way to browse.”

P19 is an interesting one…a median of only one tab open, but a max of 27! Participant 2 is similar, but not nearly as extreme: a median of 4 and a max of 20. Even if these two preferred not to have too many tabs open most of the time, they weren’t afraid of opening up lots of tabs when they needed to.

In general though, it looks like most people don’t go far beyond where the tab bar starts to scroll. I’ve marked a gray line on the graph at 13 tabs. At a resolution of 1280×1024, this is the point at which the tab bar starts scrolling (on my computer, at least). We see one person who maxes out at 13, and a few more who max out at 14. In all, there are 9 people whose maximum is between 10 and 14.

Conclusions

There are a few things we can take away from this. First, we saw that people who use tabs heavily can create 2 to 3 times as many tabs as other users. It’s not obvious what the cause for the bi-modality in the distribution is though. From the second and third graphs, we see that having 10 or 11 tabs open is not that uncommon, even for people who aren’t “power users”. And the third chart also show us that even people who don’t have many tabs open on average can sometimes have spikes of a large number of tabs.

Let me know if you have any questions or feedback. Have I explained things clearly? Are there any other numbers you think I should take a look at? Leave me a comment or send me an email. I’ll be posting some other interesting results from my study over the next couple of weeks.

This morning I read an article about CloudKick, a new Y Combinator startup that provides a nice interface for managing cloud computing resources on Amazon EC2 and Slicehost. It turned out to be a good lesson in how paying attention to basic search engine optimization (SEO) techniques can also give you usability benefits.

After reading the article, I was looking for a bit more information on CloudKick. When I Googled searched Google™ for ‘cloudkick’, here’s what I saw:

The CloudKick home page doesn’t appear until #5, and #4 is actually a separate page from their site! Not the best experience for someone searching for more information about the company. It seems odd that their official web site doesn’t appear until #4, almost as if it’s not really the official web site. And why does the contact page appear before the home page?

Notice anything else? The URL on the #4 result has www in front, but the one in #5 doesn’t. I’m by no means an SEO expert, but I know a thing or two — and I could tell that because links to their site weren’t using a canonical URL, Google was treating it as two different sites. This causes two problems: first, Google doesn’t know that the contact page is actually a subpage of the main web site, and second, it dilutes the PageRank, because Google thinks that these two results represent two different sites.

So I sent a quick tweet to CloudKick co-founder Alex Polvi to let him know about the problem and to suggest that they change cloudkick.com to be a 301 redirect to www.cloudkick.com (or vice versa). That lets Google know that both URLs refer to the same site.

And what do you know — they made the change, and just a few hours later, here are the results I get:

Looks a lot better, doesn’t it?

The moral of the story: you may not think you need to bother with search engine optimization, but paying attention to even the basics can give you a big usability win. Make it easier for Google to understand your site, and you make it easier for your potential users as well.

In the next month or two, I’ll be finishing up my master’s degree. After that, I’m looking for full-time, part-time, or contract work doing programming and/or interaction design.

What I can do

Whenever possible, I like to code in Python. It’s been my language of choice for more than 6 years. Lately I’ve also been writing lots of Javascript, especially for the Firefox extension I wrote for my master’s. (I’m pretty handy with HTML and CSS too.)

I also know my way around Java: I worked on the Eclipse team for a short time, and later worked on IBM’s Java VM for more than four years, where I wrote some heavy-duty C and C++ code. I’m also happy to dive into other languages when I need to: Ruby, PHP, shell scripting, etc.

For the past two years, I’ve been working on my master’s degree in Computer Science (with a focus on human-computer interaction) at the University of Toronto. For my master’s thesis, I did a field study examining how people use tabs in Firefox, which I recently spoke about at the Mozilla Mountain View office. For one of my courses, I also did some interaction work on the One Laptop per Child XO laptop.

And, of course, I’ve been writing about programming and HCI on this blog for more than two years.

For more details about me, check out my about page, or get in touch and I can send you a full CV.

What I’m looking for

For a full-time job, I’m looking for something where I can do interaction design and also write lots of code. Ideally I’d like to work on a public-facing product or site that I use and love.

For part-time and contract work, I’m up for pretty much anything that’s interesting and challenging.

Get in touch

If you think you’ve got something I might be interested in, send me an email @ pat at dubroy.com.

I ran across an interesting post on programming.reddit called Anecdote Driven Development, or Why I Don’t Do TDD. The article focused on testing, but what I found most interesting was the part about how long a method or function should be:

I recently wrote some code for Class::Sniff which would detect “long methods” and report them as a code smell. […] Ben Tilly asked an embarrassingly obvious question: how do I know that long methods are a code smell?

I threw out the usual justifications, but he wouldn’t let up. He wanted information and he cited the excellent book Code Complete as a counter-argument. I got down my copy of this book and started reading “How Long Should A Routine Be” (page 175, second edition). The author, Steve McConnell, argues that routines should not be longer than 200 lines. Holy crud! That’s waaaaaay to long. If a routine is longer than about 20 or 30 lines, I reckon it’s time to break it up.

Regrettably, McConnell has the cheek to cite six separate studies, all of which found that longer routines were not only not correlated with a greater defect rate, but were also often cheaper to develop and easier to comprehend.

I’d never heard this before, but this is great, because it verifies what I’ve believed for a long time…

That which obscures my code is bad

Last year I wrote a post called If this is Object Calisthenics, I think I’ll stay on the couch where I argued (among other things) that making your methods as short as possible is NOT a good idea. My justification was that it just makes the code more complicated: “That which obscures my code is bad.” But this is even better…actual empirical evidence.

I don’t have a copy of Code Complete, so I did a bit more research to see if I could find the actual studies. I found a good summary here (links added by me):

McConnell cites the findings of several studies of the correlation between the size of routines and the cost and/or fault rate of routines. Some findings which favor longer routines are:

• Routine size is inversely correlated with errors, up to 200 lines of code. [Basili and Perricone, 1984]
• Larger routines (65 lines of code or more) are cheaper to develop per line of code. [Card, Church, and Agresti, 1986; Card and Glass, 1990]
• Routines with fewer than 143 source statements (including comments) had 23% more errors per line of code than larger routines. [Selby and Basili, 1991]
• Routines averaging 100 to 150 lines of code need to be changed least. [Lind and Vairavan, 1989]

Hmmm. It looks like the studies are all about 20-25 years old. I wonder if — or how — the results would apply now. I took a quick look at the papers (the ones I could get my hands on), and the programming languages used were: Fortran [Card ‘86], Pascal and Fortran [Lind ‘89], and a mix of custom languages (one being PL/1-like) and assembly [Selby ‘91].

Does anyone know of any more current results? (Greg?) It would be interesting to see if this can be shown with more modern languages. But intuitively, it makes sense. In her book Software Engineering: Theory and Practice, Joanne Atlee summarizes it nicely:

Card and Glass (1990) point out that the design complexity really involves two aspects: the complexity within each component and the complexity of the relationships among components.

By making your methods shorter, you’re just trading one kind of complexity for another.

Update: In the comments, Stephane Vaucher pointed to a much more recent study (from 2002): The Optimal Class Size for Object-Oriented Software. They point out that the conclusion that shorter methods are more error-prone is misleading, at best:

The observed phenomenon of smaller components having a higher fault density is due to an arithmetic artifact. First, note that the above conclusions were drawn based exclusively on examination of the relationship between fault density versus size […] However, by definition, if we model the relationship between any variable X and 1/X, we will get a negative association as long as the relationship between size and faults is growing at most linearly.

Another way of putting it is: short methods may have more defects per line, but they still have fewer defects overall. There may be a justification for not making methods too short, but these studies do not provide one.

The one sure thing is that the more code you write, the more bugs you will have.

If you’re reading this, chances are that you’ve heard about the web browsing study I’m doing for my master’s thesis. If not, you might want to check out the summary of my talk at Mozilla, the responses to the talk from Jono and Boriss, or just check out the posts under the “research” category.

Since my talk, a few people have contacted me to ask about exactly how I did the logging, and did I notice this, or did people do that. Unfortunately, I can’t release my raw data, but I decided to do the next-best thing: release my logging tool. I humbly present tlogger for your consideration:

tlogger is a Firefox extension for capturing click-stream web browsing logs. In other words, it collects data about how the browser is used. Mainly it records navigation events and tab events, as well as the UI actions that cause those events. It’s roughly similar to the Spectator extension, but with a few key differences:

• it’s compatible with Firefox 2 and 3

• it doesn’t submit ANY data automatically, to anyone. Everything stays on in your profile directory, in a human-readable format.

• URLs are obfuscated on a per-user basis. From the log file, someone can see when the user revisits a site or a URL, but there is no way to determine what the actual URL is. It’s also not possible to make comparisons between users.

• it can log a few things that Spectator can’t, like when javascript on a web page changes window.location.href.

The source code is managed on GitHub at http://github.com/pdubroy/tlogger/. For the impatient, you can install the latest version of the extension or grab a snapshot of the repository. In addition to the source code for the Firefox extension, the git repository contains tools for analyzing the log files generated by tlogger.

tlogger is useful for anyone who needs real data about how people use Firefox. Of course, it’s perfect if you’re doing a field study on web browser usage, but it’s also useful for prototyping new UI features for Firefox. Liz Blankenship has already used it for her tabviz project, and discovered some interesting things about her own web browsing habits.

Enjoy! If you find it useful, or if you have any questions, send ‘em my way (email to pat, at the domain dubroy.com). If you make changes, send me a pull request on GitHub.

There’s been lots of interest in the talk I gave at Mozilla last week on the early results of my web browsing study. I’m starting to realize that I’m far from the only one thinking about this stuff. Here are some interesting things I came across in the last week:

Andy Edmonds pointed me to an informal survey done by Dave Munger at Cognitive Daily on how many tabs people use. Dave found that most people had only 2-4 tabs open, and that younger people were likely to have more tabs open. But my favourite part was his finding that if you know who Jonathan Ive, Leo Laporte, and Esther Dyson are — you’re likely to have more tabs open.

Liz Blankenship told me about her project on Tab Visualization for an infoviz course at the University of Michigan. Their goal is to help browser users who tend to have “too many” tabs open at once make sense of the information overload they experience. I’m looking forward to seeing what they come up with.

Also, I realized that I haven’t mentioned anything yet about Mozilla’s Test Pilot project, other than a brief mention in my last post. Test Pilot is a Mozilla Labs program that will allow people to do studies like mine on a massive scale. The goal is to have 1% of Firefox users opt-in to being participants in these kinds of studies. My study had 22 participants. Think hundreds of thousands, or even millions. Pretty cool. I can’t wait to see where it goes.

Earlier this week, I visited the Mozilla office in Mountain View and presented some initial results from the web browsing study that I’m doing for my master’s thesis. The (all-meat-no-filler) title of my talk was “How Do People Use Tabs?” It went really well — everyone seemed to be interested to hear my results, and as I expected, they asked lots of great questions and gave me some good ideas for my further analysis.

I dropped the ball and didn’t post my slides anywhere before the talk. Boriss and Jono have already blogged about the talk and linked to my slides, and since it’s already generated quite a bit of discussion, I thought I’d add a bit more detail here.

I was hoping to have a video to post, but that didn’t work out. So instead, I decided to try something different: I’ve written up the talk inline with the slides. It’s transcribed from memory, but I think it’s pretty close to the actual talk that I gave. Let me know what you think of this format — is it worthwhile?

You can also grab the full slides of the talk in PDF.

How do people use tabs?

For those who haven’t read my about page yet, I’m a master’s student in Computer Science at the University of Toronto, focusing on Human-Computer Interaction. This talk was about the research I’m doing for my master’s thesis. If I can boil it down to five words, it’s “how do people use tabs?”

To give you an idea of how I got here: I have a love/hate relationship with tabs. On one hand, I find tabs to be amazingly useful, and I don’t think I could ever go back to using a browser that doesn’t support tabs. But on the other hand, I find I run into a lot of problems with tabs. For one, tabs make it much harder to use the back button. Instead of one trail of history, you now have several — one for each tab. If you’ve got 5 or 10 tabs open, and you’re trying to find a page that you were looking at just a few minutes ago, you might not remember what tab you were in when you were looking at it. That makes it really tough to find the page you’re looking for.

Another problem with tabs is that they subvert the traditional task management mechanisms of the OS. Exposé, for example, is a really useful feature on OS X. If you are looking for a particular tab, like GMail — if that tab is not the selected tab in the browser window, you’re not going to find it in Exposé. The same is true of the Window taskbar.

Tabs also force you to make a premature commitment. What I mean is that every time I click on a link, I have to decide whether I want to open it in a new tab or in the current tab. Sure, it’s an easy decision, but it’s still something I have to think about every time I click on a link. And that’s something we all do pretty often.

Finally, we probably all run into the problem sometimes that there are just too many tabs open. They clutter up the screen, the tab bar starts scrolling, and it takes an effort to clean things up. It’s a pain.

So, as I was looking for topics for master’s thesis, I started thinking — could I make something better? Could I come up with something that gives all the advantages of tabs, and eliminates some of these problems?

But as I was sketching up concepts, I started to realize that I really didn’t have a good idea of how or why people use tabs. I knew what I did, and I could ask my friends what they did. But I’m a programmer, and a lot of my friends are programmers, and we all know that programmers are not exactly what we’d call a representative sample.

So I looked at the literature. I had no trouble finding academic papers that looked at how people use web browsers, but surprisingly, I found hardly any mention of tabs. What I found was a big focus on revisitation. A couple papers on revisitation were published recently at CHI — one in 2008, and another in 2007 (and a good summary here). One of them didn’t mention tabs at all, and the other had only a brief mention of how tabs might change revisitation behaviour. I thought this was funny, because in my mind, tabs are highly related to revisition. If I have a page open in a tab, it’s because I want to go do something else, and then eventually come back to that page. To me, tabs offer another kind of revisitation, so surely they warrant more than just a cursory mention?

It seemed to me like a nice opportunity for my research to fill an significant gap in the literature. So I decided that I would do a study to investigate how people use tabs, and how tabs are related to revisitation.

In November and Decmeber of last year, I conducted a field study with 22 participants who each participated for two weeks. Unlike many studies done in HCI, these were not 22 CS graduate students — I really tried to get a variety of people to participate. In terms of age, most of my participants were in their 20s, but I had a few people in their 30s, one in his 40s, and another in his 50s. And in the study, only 6 of the 22 participants came from a CS or engineering background. The others were quite varied: I had some students from the social sciences, a high-school teacher, a professor, and some administrators from the university.

I wanted to gather both quantitative and qualitative data. So, I wanted to know things like:

• how many tabs to people have open, on average
• what percentage of links are opened in a new tab, vs. in the current tab?
• is use of the back button or other history mechanisms correlated to tab usage?

…et cetera. But I also wanted to know why people did things the way they do. And I wanted to learn what people use tabs for, what things they like and dislike, and what problems they run into.

Now, gathering quantitative data is fairly easy. Firefox is pretty easy to instrument, so I built a small logging extension that captured all the data I was interested in:

• Tab events: when a tab is opened, closed, moved, and switched to
• Navigation events: load start, changes to the URL, and load events
• Causes of the navigation events: clicking on a link, using the back button, etc.

(This extension is actually quite similar to the Spectator extension, but for a few reasons, I couldn’t actually use Spectator.)

So that gave me my quantitative data. What about the qualitative data? How would I collect that?

Well, that was actually a really tough question. I tried out a few different ideas, piloting them on some of my friends. In the end, what I settled on was this: periodically during the day, my extension would show a not-too-obtrusive notification asking the user to record a short diary entry about what they are doing right now. An interesting thing about this technique was that I didn’t actually get that much interesting information from these diary entries, but they were actually useful in another way. I interviewed each participant 2 - 4 times over the course of the study, and the diary entries served as a memory trigger, a kind of anchor to bring them back to a particular time or event. And then I could ask them about that event, things like: Why do you think you opened this page in a new tab? Were you still using this tab, or were you done with it? And through these interviews, I was able to collect a lot of really interesting data about how people use tabs, what purposes they serve for them, and why they do things in a particular way.

I completed the data collection before Christmas, and for the past few weeks have been starting to do analysis. I’d like to share some of my initial results with you. Keep in mind that these are very early results — they are far from conclusive, but it looks like there are some really interesting things in here.

One thing is that I’ve heard lots about what people are using tabs for. A lot of these things aren’t that suprising — they are probably things that you do as well — but it was nice to hear about them from other people, especially people who aren’t programmers.

Several people mentioned using tabs instead of the back button. For example, with a Google search, a lot of people will go and open several links in new tabs, and then go and peruse those tabs, and see which ones give the information that they need. Without tabs, they said that they’d click on a link, check it out, go back to the search results page, click on another link, et cetera.

They also mentioned using tabs as lightweight bookmarks. For example, you might look up a recipe for something you want to make for dinner, and instead of bookmarking it, just leave the tab open for a few hours until you are actually making dinner.

Similarly, many people said that they use tabs as reminders. One participant said that at the end of the day, she scans all of her open tabs, and can quickly figure out if there’s anything left to do before she leaves.

Of course, tabs allow people to multitask, to have several things on the go at once. Quite a few people reported keeping a tab open to Pandora or an internet radio station to listen to music while they are working.

And another somewhat obvious one is that tabs are useful for comparison. But what was interesting is that almost everybody said that tabs are better than multiple windows for this. It’s not clear exactly why, but people said that it’s just quicker and easier to switch between tabs than to switch between multiple windows.

Another question I wanted to answer was, what are the advantages of using tabs? This was kind of funny, but I kept hearing people say things like, “it’s just right there.” It seemed that the whole visual and spatial aspect of tabs was something that people really found helpful. People also mentioned that they liked having a visual browsing history. A couple people even told me that this helped prevent procrastination! If they were working on something, and then they went off an a little sojourn through Wikipedia, then the first couple of tabs would still be there in the top left, reminding them of what they’re supposed to be doing.

When compared to the back button, most people reported that using tabs is easier and faster. Some of them couldn’t quite put a finger on why; but others mentioned that with the back button, they don’t know how far back a page is going to be. But with tabs, they know exactly what they’re going to get when they click on the tab.

People also seemd to distrust the back button in a way. They said they weren’t always sure that they’d be able to find the page that they’re looking for, or whether the back button would even do what they intended (”Some sites don’t really agree with the back button,” said one person). Tabs, on the other hand, felt much more certain.

And of course, tabs allow new browsing strategies that weren’t possible before. For example, you can go through a bunch of links and open up several of them in tabs, and then go and investigate them. This can be handy if you’re in the middle of reading an article, but see an interesting link that you’d like to check out.

As for my quantitive results, what I have done so far is only very basic analysis, basically grepping the files and looking for the frequency of certain events. So take these early results with a heavy grain of salt.

One of the first things I looked at was the frequency of tab switching. As my benchmark, I used the number of link click events. Since I can’t (yet) accurately determine the number of actual navigation events, this seemed like the next best thing. And previous studies have shown that link clicks account pretty reliably for about 45% of all navigation events [Note: in the talk, I believe I said 50%].

What I found is that the median number of tab switches was roughly 1 for every 2 link clicks. This is interesting, because it would mean that tab switching is the second-most frequent thing that people do in their browser (link clicks are the most frequent, besides typing, pointing, and scolling).

But, I also found that in 5 of the 22 participants, tab switching was actually more frequent than clicking on links. And for all but two of my participants, tab switching was more frequent than clicking on the back button.

This is interesting, because up to now it’s been assumed that the primary thing that people do in their browser is click on links. And this may still be true (for some people), but tab switching is a close second. This means that the browser is used both for navigation, but also as a task-management tool.

Another thing I wanted to look into was how often people choose to open a link in a new tab. In doing so, I noticed something interesting: 6 of the people in my study never opened a link in a new tab, and 3 others did so less than 10 times. These people still used tabs quite a bit. Maybe they never felt the need to open a link in a new window, but I think it’s more likely that they didn’t know they could even do that. One person actually described to me a long work-around. If she was on a page with two links that she would have wanted to open in new tabs, she would copy the URL of the page, open up a new tab, and paste the URL in the new tab. Then, she would follow a different link in each one of the tabs. She did this so that she could compare between the two sites. The thing is, she was telling me that this was something she liked about tabs — that should could compare between two pages. So, even with the amount of work she was putting in, tabs were a win for her. Clearly she would benefit from knowing how to open up a link in a new tab.

The conclusion I make from these numbers is that opening a link in a new tab is not very discoverable. The only way you would find out about it is if someone told you about the magic Ctrl-Tab shortcut, or if you happened to right-click on a link. But that isn’t very easy to discover.

Finally, I noticed something interesting about the use of the back button. Previous studies on web page revisitation have shown that link clicks have pretty steadily accounted for about 45% of all navigation actions [in my presentation, I originally said 50% –ed.]. The back button seems to be accounting for less and less. In papers by Catledge & Pitkow (from 1994) and Tauscher & Greenberg (from 1995-96), the back button accounted for about 32 - 36% of navigation actions. Hartmut Obendorf and his co-authors, in their study published at CHI 2007 but conducted in 2004-05, they found that the back button only accounted 14% of all navigation actions. [A good summary of the different findings in all 3 studies can be found here]

[Note: I’ve corrected a few of the numbers here. In my presentation I believe I said that link click accounted for 50% of navigation actions, and that the earlier studies had shown the back button accounted for about 30%. I was slightly off.]

In my data, I’m seeing that the median number of back events is about 1 for every 50 link clicks, and for 9 people, it was less than 1 in 100. I don’t have an exact number of navigation events yet, but assuming that link clicks are relatively stable at about 45%, then back events would be less than 1%! In fact, 7 participants in my study used the back button less than once per day. And these people were among the heaviest users of tabs. Now these are just rough numbers, but even if it’s off by as much as a factor of 2 (which is unlikely), the conclusion here is that the back button is becoming irrelevant for a large class of users.

What’s still not clear is exactly why people are using the back button so much less. It might be that they don’t need it as much when they use tabs, or maybe that it’s harder to use when they use tabs. Or maybe it’s for other reasons entirely. [There were a couple of interesting suggestions about this from the Mozilla folks. It might be that a lot of sites are better designed these days and provide a way of going “back” without using the back button. Or, maybe that it’s because many people are using web applications like GMail, in which it’s not clear what happens sometimes when you press the back button.]

As I’ve said, these results are based on my intial, fairly basic analysis. I’m planning on digging a lot deeper. With the qualitative data, I’m continuing to analyze and code it, hoping to eventually come up with a kind of “theory of tabs.”

My quantitive analysis has been very basic so far. I’m currently working on tools to help me analyze the logs in much greater depth. (Yes, tools that are even more sophisticated than ‘grep -c’!) My plan is to measure a bunch of obvious things, such as the number of tabs that a person has open, the time that they spend on a tab, the portion of links that are opened in a new tab, etc.

I’m also planning on looking at tab switching as a kind of revisit — when you switch away from a tab and switch back to it, that’s revisiting the page, even though it doesn’t cause a navigation action — and comparing these results to papers on revistation that I mentioned earlier.

But I’m also looking for ideas. There are lots of interesting patterns that I might be able to find, but I won’t find them unless I know what to look for. Please leave a comment below and let me know your ideas.

These are just a few of the lessons that I’ve learned from conducting this study, and they might be helpful to anyone who’s thinking of doing a similar study.

First of all, I found that there are lots of people out there who are passionate Firefox users, who would love to be able to help out the project in any way. People would say to me, “I love Firefox! I’d love to participate in your study.” Even when I told them that I wasn’t affiliated with Mozilla, they were still really interested.

I also found that many people aren’t that concerned about someone seeing the web sites that they visit. Now, the people I talked to are a biased sample, because anyone who was really concerned about their privacy obviously wouldn’t be interested in the study and wouldn’t have gotten in touch with me at all. But out of all the people who contacted me, almost no one had any concerns about how the data was being collected. Now, I didn’t have access to any of their personal information — all URLs were obfuscated on a per-user basis, and no other personally-identifiable data was collected — but still, I was surprised how little concern most people showed. I was very conscientious about explaining exactly how I was protecting their privacy, but most people didn’t seem to care that much. A few even offered to send the full list of the sites that they visited, and even capture a video of them during web browsing. I had to pass on these offers though, as they were outside the scope of my study.

I’ve found that qualitative studies are quite hard — sometimes difficult to design, and definitely difficult to do data analysis on.

Finally, I found that the extensibility of Firefox is a double-edged sword. While it made it really easy to instrument the browser to record my data, the possibility of all kinds of other plugins being installed really complicates the log analysis.

And of course, please feel free to leave your comments below.

Lately, I’ve been finding Ubiquity to be pretty handy. But honestly, I only use a few of the commands on a regular basis: tinyurl, map, and define. I use Ubiquity in these cases because it’s significantly faster and easier than what I’d normally have to do. On the other hand, I don’t really find it easier to use Ubiquity to do a Google search than to just open up a new tab and hit Ctrl-K.

What I find cool about Ubuity is not that it’s “a command line for the web”, but that it provides a much simpler way to extend Firefox. To reach its full potential, I think it needs to move beyond the command line — which is why I was glad to see Aza Raskin (one of Ubiquity’s creators) blogging about making Ubiquity more mouse-oriented (see here and here).

The Concept

In Aza’s proposal, the main way Ubiquity would be accessed by the mouse is through a “badge” that appears beside the selected text, as in the photo below:

When the badge is clicked on, the page slides to the left to reveal the Ubiquity pane. Take a look at Aza’s video to see what I mean:

Mouse-Based Ubiquity from Aza Raskin on Vimeo.

Thoughts

The badge-with-selected-text behaviour can be incredibly annoying (e.g. Snap previews), but it can also be done well (e.g., nytimes.com). As implemented in Aza’s prototype, it’s very subtle, so I think it could work.

But there also needs to be a way to bring up Ubiquity without selecting some text. And whatever that is — a toolbar button, a context menu item, etc. — it’s going to be something completely different from this badge. It’s already a problem that Ubiquity doesn’t integrate into your regular workflow, and having two separate ways of accessing Ubiquity would only make things worse, in my opinion.

As for the slide-over effect, I’m not sure I understand the advantage over a translucent overlay like Ubiquity currently uses. And I think that if Ubiquity is invoked with the mouse, it should look as similar as possible to when it’s invoked with the keyboard, providing a smooth transition from mouse-based use to keyboard-based use.

Alternative Ideas

One possibility is for Ubiquity to have a dedicated toolbar button, like many extensions do. When the user clicks the button, the Ubiquity pane pops up or slides out. If we want to call attention to Ubiquity when text is selected, the button could change it’s icon in a subtle way, getting more colourful, or even glowing or pulsing.

There are other ways Ubiquity could be hooked into the Firefox UI. A Ubiquity bar could replace the search bar (since Ubiquity can do search anyways). Or maybe Ubiquity deserves its own pseudo-tab? See the sketches below:

Panel Layout

The real hard part about a mouse-based Ubiquity is figuring out how commands would be accessed using the mouse. Since Ubiquity is essentially a command-line interface, it can easily scale to handle dozens or hundreds of commands. It’s harder to imagine how that can be done with a mouse-based interface.

To complicate things, there’s no obvious way of grouping Ubiquity commands. Do you group them by verb? e.g. search, post, convert, translate? Or do you group them by site? Or…? And how do new commands fit into that picture? Overall, I think trying to fit commands into a static hierarchy is a losing proposition.

A Ubiquity user needs to be able to access all of their commands, but it doesn’t need to be especially quick to access any arbitrary command. Most people probably have 5 - 10 commands that they use most frequently (it would be nice to have some real numbers on this — Test Pilot anyone?). Those commands should be quick to access. For other commands, I think it’s okay if it’s not as quick.

I don’t have a complete design in mind, but I’ve sketched up a few ideas:

Any thoughts?

Writing software is incredibly hard. Every programmer knows this. The software we write is complex, unreliable, and difficult to maintain. And this isn’t a new thing — the term “the software crisis” was coined in 1968.

The thing about software is that it’s remarkably easy to write a program that mostly works. And it’s difficult to tell the difference between a quick hack and a stable, reliable, and robust system, because the software development process produces almost no visible artifacts.

When you look at a building, it’s easy to get a quick sense of how well-built it is. Which of the two buildings below would you rather be in during a heavy storm?

Besides the program itself, the only visible artifact of the software development process is the source code. And that is only viewed by the programmers, through the tiny lens of the text editor. What if we could make the entire process more visible?

I’m thinking of visualizations along the lines of the comparison of system calls in Linux/Apache and Windows/IIS that I posted a while back. But this is just one idea. What other ways could we visualize Code Smells? (Maybe we could actually smell them!)

Aside from visualizing various aspects of the source code itself, could we show how well tested a piece of software is? We could show how many tests were run recently, what their results were, how good the code coverage is, the number of crashes encountered in the field, etc. With projectors and LCD displays being so cheap these days, there’s no reason a development team couldn’t have a few displays dedicated to these kinds of visualizations.

What do you think? Could this help improve the quality of software? What if companies openly published these kinds of things?

Since I’m doing a field study on how people use tabs in Firefox, you can imagine that I spend a lot of time thinking some of the smallest details of the Firefox user experience. One thing that’s been on my mind lately is the session restore feature. You know, when you start Firefox, and it asks you if you’d like to restore your windows and tabs from last time? That’s session restore.

It’s definitely a handy feature. I often use tabs like lightweight bookmarks, leaving tabs open to a page that I am planning to come back to. (And in my field study I’ve learned that lots of other people do this too.) If your browser crashes, or the Firefox process sustains collateral damage in a kill(1)ing spree, it’s a relief not to lose all the tabs your were saving.

The Problem

But the interface for session restore has always bugged me a bit. First, it uses modal dialog boxes, which are generally a bad idea. One of great things about Firefox 3 is that it eliminated a lot of the modal dialog boxes (e.g. “Do you want to remember this password?”) in favour of non-modal messages in the notification bar (see Alex Faaborg’s post about this from last year).

Not only does it use modal dialog boxes, but it’s asking me a question that’s usually unrelated to what I’m trying to do. “Do you want Firefox to save your tabs for next time?” I’m probably closing my browser for a reason, but I have no idea whether or not I’ll need these tabs next time. And asking me when I start up might not be the right time either…I’m starting my browser because I have something to do, and I can’t remember what I had open before, so how should I know whether to restore or not?

Either you’re part of the problem, or…

So I’m thinking — what if we got rid of these questions altogether? What if Firefox always remembered what windows and tabs you had open? But you might not want 15 tabs loading every time you start Firefox up.

A while back, Aza proposed making the new tab screen more useful. His proposal included a separate screen for restoring recently-closed tabs and windows, but it’s kind of hidden in his design. Most of the space in his design is taken up by contextual actions, but when you’re just starting the browser, these aren’t as relevant. Here’s a quick mockup of what a similar screen might look like on startup (mouse over to see the notes):

There are a few things to mention here. First of all, recently-closed windows are accessed from within a history list. Lately, I’ve been thinking that so much of what we do in the browser is revisiting pages that we’ve been to before, so a time-based view makes a lot of sense. Unfortunately, the history is pretty much hidden in most browsers. So you can see some inspiration here from Google Chrome, which presents the history like a regular web page. I think this makes a lot of sense, because it lets you use the same behaviours that you use on the web, whereas a separate history window forces you to learn a new UI.

Recently-closed windows are presented in a way that looks somewhat like they actually appeared in the window, maintaining the tab ordering. It probably needs to be made a bit more obvious than in this mockup, but you get the picture. Of course you could go a step further here, and make it look almost exactly like a screenshot of the tab bar. This mockup only shows one recently-closed window, but you can imagine having more than one, and they would appear in the history at the time that they were closed. I’ve also incorporated recently-closed tabs into this page. That’s currently available as a menu item under History->Recently Close Tabs, but to me, it’s always seemed kind of tacked-on there. You could also imagine using some of the space here for bookmarks.

What do you think? If you have any thoughts, please leave a comment.