I mentioned it off-hand in a previous post, but thought I should mention it again:
If you’re interested in web browser user experience, take a look at Browser Bits, my little tumblelog/mini-blog thingy all about browser UX.
Last week, I went to Atlanta for CHI 2010 to present my paper A Study of Tabbed Browsing Among Mozilla Firefox Users. For those who couldn’t be there, or just don’t feel like reading a 10-page academic paper, here’s a transcript-by-memory of the talk.
If you want a Cole’s Notes version, just check out the summary.
I have a problem with tabs. On one hand, I find them amazingly useful, and don’t think I could ever go back to using a browser that doesn’t support tabs. But on the other hand, my browser usually looks like the one above. I have so many tabs open that the tab bar is scrolling. The tab titles are too small to read. I have some pages open in duplicate tabs. It’s generally just a big mess.
When I spoke to my friends and colleagues, I realized that I wasn’t the only one who has this problem. In order to fix the problem, I think it’s important that we understand how and why people use tabs today.
I did some research, and although I could find several studies that had looked at web browsing behaviour (e.g., Catledge & Pitkow, Cockburn et al., and Obendorf et al.), I couldn’t find very much data about how people use tabs. So I decided to do a study myself.
The study was conducted with 21 participants over the course of approximately two weeks each. In our call for participation, we asked for “Mozilla Firefox users who often use multiple tabs or windows.” You’ll notice that I said “tabs OR windows”, and up to this point I haven’t mentioned anything about windows.
In many ways, tabs and windows serve the same purposes. You can open up links in multiple windows, and switch between them in any order, just like you can with tabs. Although we were most interested in how people use tabs, we felt that there was also a benefit to studying people who use multiple windows. However, all but one of our participants showed a strong preference for tabs.
In this study, we gathered both quantitative as well as qualitative data. The quantitative data was gathered through a Firefox extension I wrote called tlogger. Tlogger records click-stream logs–essentially, all network events, along with everything interesting the user does: clicking links, using the back button, opening, closing, and switching tabs, etc. In order to get a natural representation of their browsing behaviour, we felt that it was important that the actual URLs they visited were not visible to us. They were anonymized so that we could tell whether two URLs were from the same site or different sites, but we couldn’t see what the sites were.
The qualitative data was gathered in two ways. First, tlogger would periodically prompt the participants to record a short diary entry, explaining what they are doing, what tabs they have open, and why they have them open. Second, we interviewed each participant 2-4 times over the course of the study. In these interviews, we asked some prepared questions, and also just generally talked to the participants about how they use tabs. Also, if we noticed any interesting comments in their diary entries, we also asked them for further detail on those.
Rather than presenting the quantitative and qualitative results separately, we’ll take a more holistic approach, and look at the results from three angles. First, we’ll compare our participants’ usage of tabs to their usage of windows, and show that all but one of them showed a strong preference for tabs. We’ll also discuss why they preferred tabs. Second, we’ll take a closer look at the tab usage of our participants, and talk about a subset who used tabs much more frequently than the others–the so-called “tab power users.” We’ll talk about the reasons our participants gave us for using tabs, and talk about one key difference between the tab power users and the other participants. Finally, we’ll take a look at the use of tabs as a revisitation mechanism–as a way of getting back to pages that have previously been seen. Tabs offer a kind of revisitation that is similar to, but different from, the browser’s conventional revisitation mechanisms (the back button, bookmarks, and the history).
Before we get into the results, let’s talk about how we measured tab and window usage. We could take a time-based approach, and look at the number of tabs created per hour. The problem with this is that it’s difficult to tell exactly when a person is using their browser, versus when it is just sitting idle.
Instead, our unit of measure is a navigation action. A navigation action is considered to be any time the URL changes. We measure the tab creation rate as the number of tabs created per navigation action. The window creation rate is defined similarly. I should note that we only count additional tabs and windows that are created. We don’t count the first window that is opened when the browser starts up, nor the first tab that is created in a window.
Ok, with that out of the way–let’s take a look at tab and window usage.
As you can see, most participants used tabs a lot more than they used windows. Only one participant–Participant 10–used windows more than tabs. But let’s take a closer look at the window usage.
There are several ways that a window can be created. Sometimes, it’s purposely by the user, such as by hitting Ctrl-N or right-clicking on a link and selecting “Open Link in New Window”. Other times, the web site itself causes the new window to be opened.
So what if we only look at windows that were created intentionally by the user?
When we do that, window usage drops off pretty dramatically–only three participants show any real window usage to speak of. Now, to be fair, let’s do the same with tab usage.
We see a pretty substantial drop-off for some users, but overall, there’s still a lot of intentional tab usage. Participant 10 still used more windows than tabs, but only slightly. Everyone else used a lot more tabs than windows. *Despite the fact that our call for participation asked for people who either used multiple tabs or multiple windows, what we got was mostly people who preferred to use tabs.
We asked our participants why they preferred to use tabs instead of windows. The most commonly cited reasons were that tabs are cleaner, less cluttered, or more organized than using multiple windows (cited by 10/21 participants) and that tabs are easier to access and switch between (mentioned by 7 participants).
Some other reasons that were only mentioned by a few participants:
Now that we’ve established that our participants used tabs much more than they used windows, the rest of our results will focus on their tab usage. In the following chart, the tab creation rates of our participants are plotted as a histogram:
You can see that we have a bi-modal distribution. We’ve got 13 participants clustered around the 0.04 mark, i.e. four tabs created per 100 navigation actions. And then we’ve got another 8 participants loosely clustered around 0.14, or 14 tabs per 100 navigation actions. We called this second group of people the “tab power users”. They created about 3 times as many tabs as the others.
Although I didn’t mention it in my talk, if you’re interested in a closer look at how many tabs people used, you should take my earlier post: How many tabs do people use? (Now with real data!).
In the interviews, we asked our participants why they used tabs. In what situations are they useful? What are the advantages of using tabs?
The most commonly cited reason (by 17/21 participants) was reminding:
The second most commonly cited reason (by 14/21 participants) was the ability to open links in the background.
For example, 10 participants mentioned that on a search results page, they like to open several of the results in new tabs, and then go look at them. Some people mentioned doing something similar on news sites like Google News or Digg–they’d scan through the list of articles, open any interesting articles in new tabs, and then go and read the articles.
One interesting thing that we noticed was that several people mentioned that opening links in a new tab had become habitual:
In fact, it turned out that every single one of the tab power users, and none of the other users, reported that opening links in a new tab had become habitual. And this was something that we learned after we identified the tab power users. So this is likely a key difference between the tab power users and the other participants.
Why else did people use tabs? Here are some of the other reasons people mentioned:
There’s one thing that all of these reasons have in common: they are all about revisitation.
Tabs offer a kind of revisitation that is different from the conventional revisitation mechanisms of the browser (the back button, bookmarks, and the history). The main difference is that when you switch back to a tab, the page is not reloaded.
In this chart, we compare our participants’ tab-based revisitation to their use of the conventional revisitation mechanisms:
The green bars represent tab switches (as a percentage of navigation actions). The dark green are tab switch revisitations–switching back to tab that they’ve already been on. The light green bars represent the first time a tab is switched to.
The red and the pink bars represent conventional revisitation, with back button use in red. The reason we’ve focused on back button use is that previous studies (e.g., Catledge & Pitkow, Cockburn et al., and Obendorf et al.) have consistently shown that the back button is the most frequently used revisitation mechanism, and is the second most frequently used navigation mechanism in the browser, after link clicking.
We can see from this chart that 16 of our 21 participants used tabs for revisitation more often than the back button. So, clearly tabs are often used for revisitation reasons. But also, for frequent tabs users, tab switching is the second most frequent thing they do in the browser.
We asked our participants why they would use tabs for revisitation, instead of using the back button. Here are the top four reasons:
To summarize our results:
Now that we have this data, what can the browser designers take away from it? How can Mozilla (as well as Microsoft, Google, and Apple) use this to improve their browsers?
If you’d like more detail on this study, please take a look at the full paper from CHI 2010. If you have any questions, feel free to send an email to pat at the domain dubroy.com.
Also, if you’re interested in doing research with Firefox, I’d encourage you to take a look at tlogger, the extension I wrote for this study. It’s already been used in two other projects: the TabViz project, and the Multitasking bar paper presented at CHI 2010. I’m happy to answer any questions about the code and how the extension works.
If you’re going to CHI 2010 in Atlanta next week, you should come check out my talk on Tuesday morning at 9. I’ll be presenting a paper on the tabbed browsing study I did last year. Hope to see you there! If anyone wants to meet up, send me an email (pat at [my last name].com) or hit me up on Twitter (@dubroy).
On stage Wednesday at the Yerba Buena Center in San Francisco, Steve Jobs introduced the iPad as “the best browsing experience you’ve ever had. Way better than a laptop, way better than a smart phone.” Quite a claim.
Of course, the iPad browser is Safari. And from what I’ve seen and heard, it’s more like the iPhone than the desktop version. John Gruber reported that “even though the screen offers the same pixel count as what was once the standard size for a laptop display, iPad Safari renders pages like iPhone Safari. The web surfing experience is all about zooming and panning.”
I’ve had an iPhone for a while now, and done my fair share of browsing in Safari. Since I’m interested in web browser user interfaces (c.f. Browser Bits, my blog on web browser UX), I thought it would be interesting to take a closer look at the little details that make for such a great browsing experience on the iPhone. By examining the most widely-used (and arguably best) mobile web browser, we can better understand what it takes to make a great mobile web browsing experience, and how we might do even better. But also, looking at the design choices made under the iPhone’s contraints can help us break free from our assumptions of what a browser interfaces should be.
The first thing I noticed is that despite the restricted screen resolution of the iPhone (320×480) they’ve still managed to fit in most of the browser controls that we’re used to seeing.
Along the top, we’ve got the page title, the URL bar, and the search bar. It’s interesting that they chose to keep the two bars separate, rather than combine them into one as Google Chrome does. They probably made this choice in order to stay consistent with the desktop version of Safari.
In fact, almost everything about iPhone Safari is strongly consistent with the desktop version. Page loading progress is shown in the URL bar, and the refresh and stop buttons appear at the far right of it. The controls along the bottom — the back and forward buttons, the bookmark button, and the “+” button — all work almost exactly the same. One small difference is that the “+” button, instead than just being used to add a bookmark, serves three purposes: “Add Bookmark”, “Add to Home Screen”, and (somewhat oddly) “Mail Link to this Page”. The last function seems a bit out of place on this menu, but without adding a button, I’m not sure where it would make more sense.
One of the clever design choices Apple made is that top control bar is actually positioned inside the scrollable content area, so that when you scroll down, it disappears off the screen. The initial content area is 356px high, and the control bar takes up another 60px, so the result is 17% more vertical pixels for the page contents.
The bottom control bar, on the other hand, is static. I’m curious to know if the choice to put those controls on the bottom was based on any empirical usage data, because it’s not obvious that they would be used more often than the controls on the top [1].
Another thing to note is how small the URL bar actually is. In most desktop browsers, the URL bar is massive, taking up most of the window width. The iPhone browser proves that’s mostly unnecessary — I’ve rarely needed to see more than the ~20 characters it shows by default. However, the URL bar and the search bar are only that size when they don’t have focus. When you tap on one of them, it expands to fill the full width of the screen.
One weakness of the smaller URL bar is that it makes it much harder for the user to detect a phishing attempt: on many sites, the entire domain isn’t visible. Tapping on the URL bar is useless because it shows the end of the URL, and scrolling it to the beginning is complex and slow.
The extra width afforded when typing into the URL and search bars is nice, but I’ve found that it can lead to mode errors. When I want to go to a new page, sometimes I tap on the wrong box, and quickly hit the “X” to clear its contents. Once the box fills the screen, there is little indication which box you are typing in. The differences are very subtle: the search bar has fully semi-circular ends, whereas the URL bar is a rectangle with rounded corners. But what usually happens to me is that I type the first word of a search query, and then freeze when I realize that there’s no space bar on the keyboard.
If you’ve never used on iPhone, that probably requires a bit of explanation. On the iPhone, the on-screen keyboard changes depending on what kind of text field you are typing in. I’ll explain more in the next section.
Text entry on an iPhone is done using a soft keyboard that takes up the lower half of the screen. In most apps, the keyboard is in the configuration shown below left (notice the space bar). Symbols (like the ones you’d need when typing a URL) can be accessed by tapping the button in the bottom left corner. This is the same configuration used when you’re typing in the search bar. But when you’re typing in the URL bar, the keyboard is different, as shown on the right. The space bar is replaced with keys you’re more likely to need when typing a URL.
An additional hidden feature is that you can hold down the “.com” key to access “.net”, “.edu”, and “.org”. Very handy indeed.
One of the coolest things about this feature is that it doesn’t just work this way in the URL bar and the search bar. The keyboard will also reconfigure on any web page that uses one of the new HTML5 input types (like “url” or “email”). In “email” configuration, for example, the space bar is extra small, and buttons are added for “@” and “.com”. (For more information on this, see Mark Pilgrim’s very informative article on forms in HTML5.)
Ok, so most people probably don’t spend that much time thinking about the keyboard. The actual browsing experience in iPhone Safari consists of three main things: tapping on links, scrolling around the page, and zooming in to content.
Before the iPhone, it was hard to use a mobile browser to view a site that wasn’t specially designed for a smaller screen. iPhone Safari made things much better by providing two ways of easily zooming in to view the page contents.
One way of zooming is to use the multi-touch pinch/anti-pinch gestures. Contrary to popular belief, this was not invented by Apple. In fact, Myron Krueger demonstrated this technique around the time that the original Macintosh was first released. That said, the mere existence of pinch zoom is not enough; the devil is in the details, and that’s exactly where Apple excels.
When you put two fingers on the screen and spread them apart, the view zooms in on a point located between your fingers. This is important, because it helps maintain the real-world “stretching” metaphor. An alternative implementation would have been to just zoom in on the middle of the screen. I’ve used interfaces like that — it’s very frustrating.
Another detail that Apple got right is the speed of the zoom. To be absolutely true to the real-world metaphor, the content underneath each finger should remain static relative to the finger. That is, if you put one finger down on the top right corner of an image, and the other on the bottom left corner, and then spread your fingers apart, they would still be over the corners of the image. In iPhone Safari, the zoom is actually slightly slower than this, meaning that you have to move your fingers a little bit further apart to achieve the same level of zoom — but it feels just right.
The second method of zooming in iPhone Safari is to double-tap on any content to make it fill the width of the screen. This is straightforward with an image, but not so much with text. The most common use case is to double-tap on a column of text, and the browser will zoom so that the column fills the width of the screen. In general, this works exceedingly well, but in some cases, it will zoom to the width of a particular element that is smaller that the column. I’ve seen this problem on sites that have threaded comments, such as Hacker News.
On the subject of zooming: many web sites have an iPhone-specific version that uses the the meta “viewport” tag to ensure that the width of the page matches the width of the device. This tag (introduced by Apple for iPhone Safari) has an option named “user-scalable” which allows the developer to control whether the user can zoom in to the page or not. The default is “yes”, but unfortunately many web sites set it to “no”. This can be frustrating on sites like Flickr and DeviantART, where you often do want to zoom in to get a better look at a picture.
The more you zoom in, the more you’ll have to scroll. On the iPhone, you do this using direct manipulation — just swipe in any direction to pull the page in that direction. Note that this is the opposite of how scrolling works on a trackpad. On a MacBook, swiping down with two fingers will move the viewport down, whereas on the iPhone, swiping down moves the viewport up.
When zoomed in, Safari makes it easy to scroll in one direction only. That is, a swipe that is not quite vertical will result in a perfectly vertical scroll with no horizontal movement. This way, when you’re scrolling down a column of text, you don’t have to re-adjust the horizontal position every time you scroll. In my opinion, it’s still a bit too easy to go “off the rails” when attempting to scroll vertically. However, it’s a delicate tradeoff, because you want horizontal scrolling to remain responsive.
When you are in the midst of scrolling, you see bars along on the side of the screen that look very much like a scroll bar in a desktop application. This goes for the entire iPhone, not just Safari. These “scroll bars”, however, are completely read-only. They are just there to help you get a sense of where you are on the page. When you stop scrolling, they disappear. It’s good that they don’t take up any screen real estate, but frustrating when you are on a really long web page and want to scroll all the way down to the bottom. This would be even more of a problem for scrolling back to the top of the page, except that there is hidden feature on the iPhone for this purpose. Tapping the status bar at the top of the screen scrolls the main viewport back to the top. Very useful, but very hard to discover. Well, unless you read the documentation.
When reading said documentation, I actually discovered another hidden feature. To scroll within a frame or a textarea on a web page, you can swipe with two fingers. However, on my iPhone 3GS running OS 3.1.2, this doesn’t actually seem to work with either textareas or iframes.
Just like other iPhone apps, Safari has “kinetic scrolling” — swiping with your finger gives the page momentum, and it will continue to scroll after you’ve lifted your finger off the screen. And when you hit the bottom or the top, it will scroll a bit past it before snapping back. It’s a nice, natural, and cute way of indicating that you’ve hit the end of the page.
For some reason though, web pages just don’t scroll the same as other controls on the iPhone. They seem to have less momentum, or more friction. John Gruber has written about this before, going as far as to say that it’s one of the main ways that iPhone web apps fall short of native apps.
Besides scrolling, clicking on links is the most common thing people do in a web browser [2]. In iPhone Safari, following a link is as simple as tapping on it. It seems pretty straightforward and uninteresting, but given the size of a human fingertip and the small size of web content on the iPhone’s screen, it’s not as easy as it might seem. Given that most web sites are designed for use with a pixel-perfect input device (the mouse), it’s actually pretty amazing that about 95% of the time, I can manage to follow the correct link in iPhone Safari. I’d guess that the hardware is the major factor here, but it still feels like there is some kind of software heuristic at work. It’s easy to take this for granted, but given my experience with various multi-touch interfaces, it’s actually a remarkable achievement that tapping links on an iPhone is as precise as it is.
Another thing to notice is that scrolling and zooming are very rarely mistaken for a link tap. This comes at a cost of responsiveness — there is a slight delay after you tap a link, as it waits to see if another tap is coming. But the delay is subtle enough that it’s hardly noticeable.
iPhone Safari does not support tabs. This isn’t surprising, due to the screen and resource limitations of the iPhone. Instead, it has the concept of “pages”, which are somewhat like multiple windows. When you click a link that would open in a new tab or window (such as in GMail), or open a URL from another iPhone app, it will launch in a new page. When you click on the “pages” button in the lower right, you can flip through all the pages that you have open. From this view, you can close pages, or create a new page.
When a link is opening in a new page, there is a nice little animation that makes it clear that you are going to a new page that is to the right of the current one. Without it, you might become confused and wonder why the back button doesn’t work anymore. This actually happens to me sometimes when using desktop browsers, and I’ve heard other people report having the same problem.
There is a limit of 8 pages that can be open at any time. I suppose this is to conserve resources, and to prevent the list from getting unwieldy. If you have 8 pages open and a link needs to be opened in a new page, you will lose the first one in the list. However, this has never actually happened to me.
Although they are called pages, they behave very much like tabs. They have a close button on the top left, and just like with tabs, when you close a page, the one to its right takes the focus. This is another example of how strongly Apple cares about keeping consistent with the desktop user experience.
The key feature of tabs that is not supported by iPhone Safari’s “pages” is the ability to open a link in the background. You can open a link in a new page by touching and holding the link then selecting “Open in New Page”, but this will immediately switch focus to the new page, rather than loading it in the background.
Well, when you’re trying to put a fully-functional web browser on a mobile device, you can’t do everything. Here are some of the key things iPhone Safari is missing:
One of most discussed (and hotly debated) things about iPhone Safari is its lack of Flash support. It was never entirely clear whether this was a technical decision or a political one; that is, until Wednesday when we learned that the iPad won’t support Flash either. From a user experience point of view, this has its pros and its cons, but I think it’s a good thing for the web.
A small annoyance that bites me from time to time is the lack of find-in-page support. However, there’s a plethora of bookmarklets out there that can add the functionality. (Actually installing bookmarklets is pretty painful, but I digress.)
I mentioned the fact that small size of the URL bar makes it difficult to visually detect a phishing attempt. There’s also no way to see details about the SSL certificate. The only security-related thing I see is the lock icon that appears before the title when you’re using an SSL connection, but what good is that if you can’t even see what site you’re connected to?
Update: In the comments, PCheese points out that iPhone Safari has phishing protection just like the desktop version.
Any file input widgets in HTML forms are just shown as being disabled. Since there’s no user-visible file system on the iPhone, this isn’t too surprising. Still, it would have been nice to provide a simple way of choosing a photo or video to upload.
In the study on tabbed browsing I ran in 2009, I found that many power users habitually open links in new tabs, especially on search result pages and on sites like Digg and Hacker News. I find that the inability to do this on the iPhone seriously changes my browsing habits. And I’m quite surprised that the iPad doesn’t appear to support tabs, though this may of change before it’s released.
The iPhone was a revolutionary computing device, in no small part due to how great iPhone Safari is. In its user experience, Apple did what they do best — put a great amount of energy and effort into perfecting the smallest details of the design. I hope this article has helped you notice more of those details.
The version of Safari on the iPad looks to be highly influenced by iPhone Safari. This is interesting, because its screen resolution is bigger than most web users had in 2005 [3]. If it really is “the best web browsing experience ever”, aspects of its design will no doubt trickle up into desktop browsers.
It’s been a while since I posted a real update here — it’s about time I filled you all in on what I’ve been up to.
In April, I finished my master’s in Compter Science (Human-Computer Interaction) at the University of Toronto. My thesis was a study of how people use tabs in during web browsing, which I’ve written about several times. The best summaries are here and here, or you can check out all my research-related posts. I submitted a paper to CHI 2010, so hopefully I’ll be making a trip down to Atlanta next spring.
So what have I been up to since then?
I didn’t mention anything on this blog, but if you follow me on Twitter, you might have heard that I joined the team at BumpTop back in May. One of the big things I worked on this summer was adding Windows 7 multi-touch support to BumpTop. Last night we announced it publicly, and the reaction has been amazing! It got covered on TechCrunch, Venture Beat, Gizmodo, Lifehacker, and it’s on the front page of TechMeme as I write this.
Oh, and check out the sweet infographics I made — putting my art-school-dropout skills to good use:
If you visited this site in the last day or so, you probably saw a 404. Sorry about that. I had to bring the site down for a bit to upgrade my woefully out-of-date WordPress installation. A few weeks ago, someone actually took advantage of the ancient version of WordPress I had, and filled up some of my posts with dirty words. I think I’ve got everything cleaned up right now, but let me know if you notice anything out of the ordinary.
In the process, I also decided to switch my theme. Many thanks to MidMo Design for their awesome Clean Home theme. Over the next few weeks I’ll be making a few tweaks to make it my own; let me know if you notice any problems or if anything seems wacky.
For the past few months, I’ve been knee-deep in data from the tabbed browsing study that I conducted late last year. Now that I’m finishing up my thesis, I figured it’s about time that I share some of my findings. In this post, I’ll talk about one of the quantitative questions I was trying to answer in my study: how many tabs do people use?
The first thing we need to do is to be a bit more precise with the question. What does it mean to “use” multiple tabs or windows, and how should it be measured? There are few possible answers.
We could simply count the number of tabs that a person creates. One problem with that is that when Firefox starts up, at least one tab is created (and possibly more, if it is restoring a session), so someone who opens and closes their browser frequently would have a high number of tab creation events. To eliminate this effect, we only count tabs that are created after browser startup. We also ignore the first tab that is created in any window.
Another thing that’s interesting to measure is the number of concurrent tabs or windows that a person typically has open. For example, two people might create a similar number of tabs, but one of them might have an email client, an RSS reader, and another tab playing music open at all times. It seems reasonable to say that this person uses tabs more heavily than the other person. We decided to measure the number of windows and tabs that were open whenever a navigation event occurred. For consistency, we ignored all navigation events caused by Firefox’s session restore feature when calculating this measure, although it didn’t have a very significant effect overall.
A third measure of tab usage that might be interesting to look at is the number of tab switches that a person performs, but I’ll address that in a later post. For now, we’ll concentrate on how many tabs.
First, I guess I should mention a little bit about how the study was conducted. Here was the recruitment email we sent out to family, friends, and colleagues:
We are seeking participants (at least 18 years old) for a research study exploring how people use web browsers.
If you use Mozilla Firefox for several hours a day, and often use multiple tabs or windows, then you are a candidate for the study. If you choose to participate, you’ll install a Firefox extension that will log various actions, e.g. clicking on a link, visiting a bookmark, opening a new tab, and clicking the back button. Don’t worry, the names and addresses of the web sites that you visit will NOT be revealed to researchers.
The study will last for two weeks. During this time, you’ll take part in five short interviews (approx. 30 minutes each) that will be arranged at your convenience. In addition, you’ll be asked to record some brief notes several times during the course of each day (again, at your convenience) and to email your log file to the researchers at the end of the day. Participating in the study will take about 4-5 hours in total over the two week period.
We also put posters up with similar wording across the University of Toronto campus. 21 people completed the study, and 1 person started it but dropped out early.
Unlike many other studies like this that have been done, our study participants had a variety of ages, professions, and technical skill levels. One participant was younger than 20, 14 were between the ages of 20 and 29, four were 30-39, and two were 50-59. Only 6 participants came from a computer science or engineering background, while others had studied education, environmental science, business, and psychology. Six of the participants were full-time students (either undergraduate or graduate), and 15 were working in some kind of office environment where they spent most of their time on the computer.
First lets take a look at the tab creation rate. Now, keep in mind that these are only tabs that are created after browser startup, and the first tab in any window is not counted. On the X axis is the “tab creation rate”: the ratio of tabs created to navigation actions. (A navigation action is anything that changes the URL of the page, even if it doesn’t result in a top-level page load.) The height of the bars represents the number of participants with a given tab creation ratio.
Interesting…we’ve got a very clear bi-modal distribution. More than half of our participants (13/21) are clustered around the 0.04 mark. In other words, these people create about 4 tabs for every 100 navigation actions. The rest of the participants are loosely centered around 0.14, meaning that they create about 3 times as many tabs as the other people. In fact, we have two participants who are even higher, creating (respectively) 17 and 22 tabs per 100 navigation actions.
Unsurprisingly, the four highest tab creation rates belong to the four participants with Computer Science and/or programming backgrounds. The other four participants in the high part of the distribution are not your typical “techie” types: one is trained as a civil engineer, and the others have backgrounds in communications, humanities, and marketing.
Another interesting thing to look at is the number of tabs that people tend to have open at any given time. Again, we used navigation actions as our increment of measurement. On the X axis is the number of tabs open when a navigation action occurs, and the height of the bars it the total number of navigation actions that occurred with the that number of tabs open.
Not too surprising — the most common number of tabs to have open is one, with a pretty steady descent down to 9. It flattens out and hits a valley at 13, but then rises slightly again for a second peak at 16. So, again we see a slight bi-modality to the distribution. But if we take the same set of “tab power users” from the first graph, we see that they have roughly the same profile as all the participants put together. In fact, the peak at 16 is almost entirely caused by only two of the power users: participants 14 and 20.
One possible explanation for the bi-modal distribution here is tab bar scrolling. At a typical screen size on a laptop or desktop, the tab bar can fit about 9-13 tabs without scrolling. Many people probably try to avoid having the tab bar scroll, but once you get to the point where it starts scrolling, there’s little additional cost to having more tabs open. So it may be the P14 and P20 are the only ones comfortable with keeping so many tabs open that the tab bar is scrolling.
Take a look at the median and max number of tabs that each participant had open:
(Note that there’s no participant 8…he’s the one that dropped out.)
Participants 14 and 20 definitely stick out — it’s easy to see why they contributed so much to the yellow portions of the previous chart. Participant 14 by far the highest median number of tabs open with 17, while no other participant had a median higher than 6. Participant 20 actually edges out P14 for max tabs though, with 42. (For once it really is the answer.) Forty-two — that’s a lot of tabs! The tab bar would be scrolling two are three times over at this point. He actually commented on this, saying “Now I am opening tabs up from Digg and they are appearing at the end of my massive list. This is truly a bad way to browse.”
P19 is an interesting one…a median of only one tab open, but a max of 27! Participant 2 is similar, but not nearly as extreme: a median of 4 and a max of 20. Even if these two preferred not to have too many tabs open most of the time, they weren’t afraid of opening up lots of tabs when they needed to.
In general though, it looks like most people don’t go far beyond where the tab bar starts to scroll. I’ve marked a gray line on the graph at 13 tabs. At a resolution of 1280×1024, this is the point at which the tab bar starts scrolling (on my computer, at least). We see one person who maxes out at 13, and a few more who max out at 14. In all, there are 9 people whose maximum is between 10 and 14.
There are a few things we can take away from this. First, we saw that people who use tabs heavily can create 2 to 3 times as many tabs as other users. It’s not obvious what the cause for the bi-modality in the distribution is though. From the second and third graphs, we see that having 10 or 11 tabs open is not that uncommon, even for people who aren’t “power users”. And the third chart also show us that even people who don’t have many tabs open on average can sometimes have spikes of a large number of tabs.
Let me know if you have any questions or feedback. Have I explained things clearly? Are there any other numbers you think I should take a look at? Leave me a comment or send me an email. I’ll be posting some other interesting results from my study over the next couple of weeks.
This morning I read an article about CloudKick, a new Y Combinator startup that provides a nice interface for managing cloud computing resources on Amazon EC2 and Slicehost. It turned out to be a good lesson in how paying attention to basic search engine optimization (SEO) techniques can also give you usability benefits.
After reading the article, I was looking for a bit more information on CloudKick. When I Googled searched Google™ for ‘cloudkick’, here’s what I saw:
The CloudKick home page doesn’t appear until #5, and #4 is actually a separate page from their site! Not the best experience for someone searching for more information about the company. It seems odd that their official web site doesn’t appear until #4, almost as if it’s not really the official web site. And why does the contact page appear before the home page?
Notice anything else? The URL on the #4 result has www in front, but the one in #5 doesn’t. I’m by no means an SEO expert, but I know a thing or two — and I could tell that because links to their site weren’t using a canonical URL, Google was treating it as two different sites. This causes two problems: first, Google doesn’t know that the contact page is actually a subpage of the main web site, and second, it dilutes the PageRank, because Google thinks that these two results represent two different sites.
So I sent a quick tweet to CloudKick co-founder Alex Polvi to let him know about the problem and to suggest that they change cloudkick.com to be a 301 redirect to www.cloudkick.com (or vice versa). That lets Google know that both URLs refer to the same site.
And what do you know — they made the change, and just a few hours later, here are the results I get:
Looks a lot better, doesn’t it?
The moral of the story: you may not think you need to bother with search engine optimization, but paying attention to even the basics can give you a big usability win. Make it easier for Google to understand your site, and you make it easier for your potential users as well.
In the next month or two, I’ll be finishing up my master’s degree. After that, I’m looking for full-time, part-time, or contract work doing programming and/or interaction design.
Whenever possible, I like to code in Python. It’s been my language of choice for more than 6 years. Lately I’ve also been writing lots of Javascript, especially for the Firefox extension I wrote for my master’s. (I’m pretty handy with HTML and CSS too.)
I also know my way around Java: I worked on the Eclipse team for a short time, and later worked on IBM’s Java VM for more than four years, where I wrote some heavy-duty C and C++ code. I’m also happy to dive into other languages when I need to: Ruby, PHP, shell scripting, etc.
For the past two years, I’ve been working on my master’s degree in Computer Science (with a focus on human-computer interaction) at the University of Toronto. For my master’s thesis, I did a field study examining how people use tabs in Firefox, which I recently spoke about at the Mozilla Mountain View office. For one of my courses, I also did some interaction work on the One Laptop per Child XO laptop.
And, of course, I’ve been writing about programming and HCI on this blog for more than two years.
For more details about me, check out my about page, or get in touch and I can send you a full CV.
For a full-time job, I’m looking for something where I can do interaction design and also write lots of code. Ideally I’d like to work on a public-facing product or site that I use and love.
For part-time and contract work, I’m up for pretty much anything that’s interesting and challenging.
If you think you’ve got something I might be interested in, send me an email @ pat at dubroy.com.
I ran across an interesting post on programming.reddit called Anecdote Driven Development, or Why I Don’t Do TDD. The article focused on testing, but what I found most interesting was the part about how long a method or function should be:
I recently wrote some code for Class::Sniff which would detect “long methods” and report them as a code smell. [...] Ben Tilly asked an embarrassingly obvious question: how do I know that long methods are a code smell?
I threw out the usual justifications, but he wouldn’t let up. He wanted information and he cited the excellent book Code Complete as a counter-argument. I got down my copy of this book and started reading “How Long Should A Routine Be” (page 175, second edition). The author, Steve McConnell, argues that routines should not be longer than 200 lines. Holy crud! That’s waaaaaay to long. If a routine is longer than about 20 or 30 lines, I reckon it’s time to break it up.
Regrettably, McConnell has the cheek to cite six separate studies, all of which found that longer routines were not only not correlated with a greater defect rate, but were also often cheaper to develop and easier to comprehend.
I’d never heard this before, but this is great, because it verifies what I’ve believed for a long time…
Last year I wrote a post called If this is Object Calisthenics, I think I’ll stay on the couch where I argued (among other things) that making your methods as short as possible is NOT a good idea. My justification was that it just makes the code more complicated: “That which obscures my code is bad.” But this is even better…actual empirical evidence.
I don’t have a copy of Code Complete, so I did a bit more research to see if I could find the actual studies. I found a good summary here (links added by me):
McConnell cites the findings of several studies of the correlation between the size of routines and the cost and/or fault rate of routines. Some findings which favor longer routines are:
- Routine size is inversely correlated with errors, up to 200 lines of code. [Basili and Perricone, 1984]
- Larger routines (65 lines of code or more) are cheaper to develop per line of code. [Card, Church, and Agresti, 1986; Card and Glass, 1990]
- Routines with fewer than 143 source statements (including comments) had 23% more errors per line of code than larger routines. [Selby and Basili, 1991]
- Routines averaging 100 to 150 lines of code need to be changed least. [Lind and Vairavan, 1989]
Hmmm. It looks like the studies are all about 20-25 years old. I wonder if — or how — the results would apply now. I took a quick look at the papers (the ones I could get my hands on), and the programming languages used were: Fortran [Card '86], Pascal and Fortran [Lind '89], and a mix of custom languages (one being PL/1-like) and assembly [Selby '91].
Does anyone know of any more current results? (Greg?) It would be interesting to see if this can be shown with more modern languages. But intuitively, it makes sense. In her book Software Engineering: Theory and Practice, Joanne Atlee summarizes it nicely:
Card and Glass (1990) point out that the design complexity really involves two aspects: the complexity within each component and the complexity of the relationships among components.
By making your methods shorter, you’re just trading one kind of complexity for another.
Update: In the comments, Stephane Vaucher pointed to a much more recent study (from 2002): The Optimal Class Size for Object-Oriented Software. They point out that the conclusion that shorter methods are more error-prone is misleading, at best:
The observed phenomenon of smaller components having a higher fault density is due to an arithmetic artifact. First, note that the above conclusions were drawn based exclusively on examination of the relationship between fault density versus size [...] However, by definition, if we model the relationship between any variable X and 1/X, we will get a negative association as long as the relationship between size and faults is growing at most linearly.
Another way of putting it is: short methods may have more defects per line, but they still have fewer defects overall. There may be a justification for not making methods too short, but these studies do not provide one.
The one sure thing is that the more code you write, the more bugs you will have.
