My Talk at Mozilla

January 29, 2009 ⋅ 22 Comments »

Earlier this week, I visited the Mozilla office in Mountain View and presented some initial results from the web browsing study that I’m doing for my master’s thesis. The (all-meat-no-filler) title of my talk was “How Do People Use Tabs?” It went really well — everyone seemed to be interested to hear my results, and as I expected, they asked lots of great questions and gave me some good ideas for my further analysis.

I dropped the ball and didn’t post my slides anywhere before the talk. Boriss and Jono have already blogged about the talk and linked to my slides, and since it’s already generated quite a bit of discussion, I thought I’d add a bit more detail here.

I was hoping to have a video to post, but that didn’t work out. So instead, I decided to try something different: I’ve written up the talk inline with the slides. It’s transcribed from memory, but I think it’s pretty close to the actual talk that I gave. Let me know what you think of this format — is it worthwhile?

You can also grab the full slides of the talk in PDF.

How do people use tabs?

For those who haven’t read my about page yet, I’m a master’s student in Computer Science at the University of Toronto, focusing on Human-Computer Interaction. This talk was about the research I’m doing for my master’s thesis. If I can boil it down to five words, it’s “how do people use tabs?”

To give you an idea of how I got here: I have a love/hate relationship with tabs. On one hand, I find tabs to be amazingly useful, and I don’t think I could ever go back to using a browser that doesn’t support tabs. But on the other hand, I find I run into a lot of problems with tabs. For one, tabs make it much harder to use the back button. Instead of one trail of history, you now have several — one for each tab. If you’ve got 5 or 10 tabs open, and you’re trying to find a page that you were looking at just a few minutes ago, you might not remember what tab you were in when you were looking at it. That makes it really tough to find the page you’re looking for.

Another problem with tabs is that they subvert the traditional task management mechanisms of the OS. Exposé, for example, is a really useful feature on OS X. If you are looking for a particular tab, like GMail — if that tab is not the selected tab in the browser window, you’re not going to find it in Exposé. The same is true of the Window taskbar.

Tabs also force you to make a premature commitment. What I mean is that every time I click on a link, I have to decide whether I want to open it in a new tab or in the current tab. Sure, it’s an easy decision, but it’s still something I have to think about every time I click on a link. And that’s something we all do pretty often.

Finally, we probably all run into the problem sometimes that there are just too many tabs open. They clutter up the screen, the tab bar starts scrolling, and it takes an effort to clean things up. It’s a pain.

So, as I was looking for topics for master’s thesis, I started thinking — could I make something better? Could I come up with something that gives all the advantages of tabs, and eliminates some of these problems?

But as I was sketching up concepts, I started to realize that I really didn’t have a good idea of how or why people use tabs. I knew what I did, and I could ask my friends what they did. But I’m a programmer, and a lot of my friends are programmers, and we all know that programmers are not exactly what we’d call a representative sample.

So I looked at the literature. I had no trouble finding academic papers that looked at how people use web browsers, but surprisingly, I found hardly any mention of tabs. What I found was a big focus on revisitation. A couple papers on revisitation were published recently at CHI — one in 2008, and another in 2007 (and a good summary here). One of them didn’t mention tabs at all, and the other had only a brief mention of how tabs might change revisitation behaviour. I thought this was funny, because in my mind, tabs are highly related to revisition. If I have a page open in a tab, it’s because I want to go do something else, and then eventually come back to that page. To me, tabs offer another kind of revisitation, so surely they warrant more than just a cursory mention?

It seemed to me like a nice opportunity for my research to fill an significant gap in the literature. So I decided that I would do a study to investigate how people use tabs, and how tabs are related to revisitation.

In November and December of last year, I conducted a field study with 22 participants who each participated for two weeks. Unlike many studies done in HCI, these were not 22 CS graduate students — I really tried to get a variety of people to participate. In terms of age, most of my participants were in their 20s, but I had a few people in their 30s, one in his 40s, and another in his 50s. And in the study, only 6 of the 22 participants came from a CS or engineering background. The others were quite varied: I had some students from the social sciences, a high-school teacher, a professor, and some administrators from the university.

I wanted to gather both quantitative and qualitative data. So, I wanted to know things like:

• how many tabs to people have open, on average
• what percentage of links are opened in a new tab, vs. in the current tab?
• is use of the back button or other history mechanisms correlated to tab usage?

…et cetera. But I also wanted to know why people did things the way they do. And I wanted to learn what people use tabs for, what things they like and dislike, and what problems they run into.

Now, gathering quantitative data is fairly easy. Firefox is pretty easy to instrument, so I built a small logging extension that captured all the data I was interested in:

• Tab events: when a tab is opened, closed, moved, and switched to
• Navigation events: load start, changes to the URL, and load events
• Causes of the navigation events: clicking on a link, using the back button, etc.

(This extension is actually quite similar to the Spectator extension, but for a few reasons, I couldn’t actually use Spectator.)

So that gave me my quantitative data. What about the qualitative data? How would I collect that?

Well, that was actually a really tough question. I tried out a few different ideas, piloting them on some of my friends. In the end, what I settled on was this: periodically during the day, my extension would show a not-too-obtrusive notification asking the user to record a short diary entry about what they are doing right now. An interesting thing about this technique was that I didn’t actually get that much interesting information from these diary entries, but they were actually useful in another way. I interviewed each participant 2 - 4 times over the course of the study, and the diary entries served as a memory trigger, a kind of anchor to bring them back to a particular time or event. And then I could ask them about that event, things like: Why do you think you opened this page in a new tab? Were you still using this tab, or were you done with it? And through these interviews, I was able to collect a lot of really interesting data about how people use tabs, what purposes they serve for them, and why they do things in a particular way.

I completed the data collection before Christmas, and for the past few weeks have been starting to do analysis. I’d like to share some of my initial results with you. Keep in mind that these are very early results — they are far from conclusive, but it looks like there are some really interesting things in here.

One thing is that I’ve heard lots about what people are using tabs for. A lot of these things aren’t that suprising — they are probably things that you do as well — but it was nice to hear about them from other people, especially people who aren’t programmers.

Several people mentioned using tabs instead of the back button. For example, with a Google search, a lot of people will go and open several links in new tabs, and then go and peruse those tabs, and see which ones give the information that they need. Without tabs, they said that they’d click on a link, check it out, go back to the search results page, click on another link, et cetera.

They also mentioned using tabs as lightweight bookmarks. For example, you might look up a recipe for something you want to make for dinner, and instead of bookmarking it, just leave the tab open for a few hours until you are actually making dinner.

Similarly, many people said that they use tabs as reminders. One participant said that at the end of the day, she scans all of her open tabs, and can quickly figure out if there’s anything left to do before she leaves.

Of course, tabs allow people to multitask, to have several things on the go at once. Quite a few people reported keeping a tab open to Pandora or an internet radio station to listen to music while they are working.

And another somewhat obvious one is that tabs are useful for comparison. But what was interesting is that almost everybody said that tabs are better than multiple windows for this. It’s not clear exactly why, but people said that it’s just quicker and easier to switch between tabs than to switch between multiple windows.

Another question I wanted to answer was, what are the advantages of using tabs? This was kind of funny, but I kept hearing people say things like, “it’s just right there.” It seemed that the whole visual and spatial aspect of tabs was something that people really found helpful. People also mentioned that they liked having a visual browsing history. A couple people even told me that this helped prevent procrastination! If they were working on something, and then they went off an a little sojourn through Wikipedia, then the first couple of tabs would still be there in the top left, reminding them of what they’re supposed to be doing.

When compared to the back button, most people reported that using tabs is easier and faster. Some of them couldn’t quite put a finger on why; but others mentioned that with the back button, they don’t know how far back a page is going to be. But with tabs, they know exactly what they’re going to get when they click on the tab.

People also seemd to distrust the back button in a way. They said they weren’t always sure that they’d be able to find the page that they’re looking for, or whether the back button would even do what they intended (“Some sites don’t really agree with the back button,” said one person). Tabs, on the other hand, felt much more certain.

And of course, tabs allow new browsing strategies that weren’t possible before. For example, you can go through a bunch of links and open up several of them in tabs, and then go and investigate them. This can be handy if you’re in the middle of reading an article, but see an interesting link that you’d like to check out.

As for my quantitive results, what I have done so far is only very basic analysis, basically grepping the files and looking for the frequency of certain events. So take these early results with a heavy grain of salt.

One of the first things I looked at was the frequency of tab switching. As my benchmark, I used the number of link click events. Since I can’t (yet) accurately determine the number of actual navigation events, this seemed like the next best thing. And previous studies have shown that link clicks account pretty reliably for about 45% of all navigation events [Note: in the talk, I believe I said 50%].

What I found is that the median number of tab switches was roughly 1 for every 2 link clicks. This is interesting, because it would mean that tab switching is the second-most frequent thing that people do in their browser (link clicks are the most frequent, besides typing, pointing, and scolling).

But, I also found that in 5 of the 22 participants, tab switching was actually more frequent than clicking on links. And for all but two of my participants, tab switching was more frequent than clicking on the back button.

This is interesting, because up to now it’s been assumed that the primary thing that people do in their browser is click on links. And this may still be true (for some people), but tab switching is a close second. This means that the browser is used both for navigation, but also as a task-management tool.

Another thing I wanted to look into was how often people choose to open a link in a new tab. In doing so, I noticed something interesting: 6 of the people in my study never opened a link in a new tab, and 3 others did so less than 10 times. These people still used tabs quite a bit. Maybe they never felt the need to open a link in a new window, but I think it’s more likely that they didn’t know they could even do that. One person actually described to me a long work-around. If she was on a page with two links that she would have wanted to open in new tabs, she would copy the URL of the page, open up a new tab, and paste the URL in the new tab. Then, she would follow a different link in each one of the tabs. She did this so that she could compare between the two sites. The thing is, she was telling me that this was something she liked about tabs — that should could compare between two pages. So, even with the amount of work she was putting in, tabs were a win for her. Clearly she would benefit from knowing how to open up a link in a new tab.

The conclusion I make from these numbers is that opening a link in a new tab is not very discoverable. The only way you would find out about it is if someone told you about the magic Ctrl-Tab shortcut, or if you happened to right-click on a link. But that isn’t very easy to discover.

Finally, I noticed something interesting about the use of the back button. Previous studies on web page revisitation have shown that link clicks have pretty steadily accounted for about 45% of all navigation actions [in my presentation, I originally said 50% —ed.]. The back button seems to be accounting for less and less. In papers by Catledge & Pitkow (from 1994) and Tauscher & Greenberg (from 1995-96), the back button accounted for about 32 - 36% of navigation actions. Hartmut Obendorf and his co-authors, in their study published at CHI 2007 but conducted in 2004-05, they found that the back button only accounted 14% of all navigation actions. [A good summary of the different findings in all 3 studies can be found here]

[Note: I’ve corrected a few of the numbers here. In my presentation I believe I said that link click accounted for 50% of navigation actions, and that the earlier studies had shown the back button accounted for about 30%. I was slightly off.]

In my data, I’m seeing that the median number of back events is about 1 for every 50 link clicks, and for 9 people, it was less than 1 in 100. I don’t have an exact number of navigation events yet, but assuming that link clicks are relatively stable at about 45%, then back events would be less than 1%! In fact, 7 participants in my study used the back button less than once per day. And these people were among the heaviest users of tabs. Now these are just rough numbers, but even if it’s off by as much as a factor of 2 (which is unlikely), the conclusion here is that the back button is becoming irrelevant for a large class of users.

What’s still not clear is exactly why people are using the back button so much less. It might be that they don’t need it as much when they use tabs, or maybe that it’s harder to use when they use tabs. Or maybe it’s for other reasons entirely. [There were a couple of interesting suggestions about this from the Mozilla folks. It might be that a lot of sites are better designed these days and provide a way of going “back” without using the back button. Or, maybe that it’s because many people are using web applications like GMail, in which it’s not clear what happens sometimes when you press the back button.]

As I’ve said, these results are based on my intial, fairly basic analysis. I’m planning on digging a lot deeper. With the qualitative data, I’m continuing to analyze and code it, hoping to eventually come up with a kind of “theory of tabs.”

My quantitive analysis has been very basic so far. I’m currently working on tools to help me analyze the logs in much greater depth. (Yes, tools that are even more sophisticated than ‘grep -c’!) My plan is to measure a bunch of obvious things, such as the number of tabs that a person has open, the time that they spend on a tab, the portion of links that are opened in a new tab, etc.

I’m also planning on looking at tab switching as a kind of revisit — when you switch away from a tab and switch back to it, that’s revisiting the page, even though it doesn’t cause a navigation action — and comparing these results to papers on revistation that I mentioned earlier.

But I’m also looking for ideas. There are lots of interesting patterns that I might be able to find, but I won’t find them unless I know what to look for. Please leave a comment below and let me know your ideas.

These are just a few of the lessons that I’ve learned from conducting this study, and they might be helpful to anyone who’s thinking of doing a similar study.

First of all, I found that there are lots of people out there who are passionate Firefox users, who would love to be able to help out the project in any way. People would say to me, “I love Firefox! I’d love to participate in your study.” Even when I told them that I wasn’t affiliated with Mozilla, they were still really interested.

I also found that many people aren’t that concerned about someone seeing the web sites that they visit. Now, the people I talked to are a biased sample, because anyone who was really concerned about their privacy obviously wouldn’t be interested in the study and wouldn’t have gotten in touch with me at all. But out of all the people who contacted me, almost no one had any concerns about how the data was being collected. Now, I didn’t have access to any of their personal information — all URLs were obfuscated on a per-user basis, and no other personally-identifiable data was collected — but still, I was surprised how little concern most people showed. I was very conscientious about explaining exactly how I was protecting their privacy, but most people didn’t seem to care that much. A few even offered to send the full list of the sites that they visited, and even capture a video of them during web browsing. I had to pass on these offers though, as they were outside the scope of my study.

I’ve found that qualitative studies are quite hard — sometimes difficult to design, and definitely difficult to do data analysis on.

Finally, I found that the extensibility of Firefox is a double-edged sword. While it made it really easy to instrument the browser to record my data, the possibility of all kinds of other plugins being installed really complicates the log analysis.

And of course, please feel free to leave your comments below.