Google is not enough

May 10, 2007 ⋅ 7 Comments »

I started at U of T this week, and I’m trying to narrow down my thesis topic, so I’m basically doing a swan dive into a pile of papers related to personal information management. Today I’ve read a couple interesting ones about how we use keyword search, and why even the most perfect search engine probably won’t ever replace the use of browsing to find information.

In the first paper, Don’t Take My Folders Away!, the researchers surveyed a small group of people to see how they used folders to organize the files on their computer. When the users were asked why they created folders, the answer was generally “to get back my files”. But when they were asked if they would give up their folders and find their information exclusively using a search engine, the answer was a resounding “no”. The conclusion of the paper was that folders are not just a way to organize information — the folder structure itself actually contains information about a project. For example, a folder structure can indicate subprojects and subtasks of a project.

Compass The second paper, “The Perfect Search Is Not Enough“, investigated how people performed searches both on the web and in their personal information. They found that people rarely searched for the specific item they were looking for; instead, they moved in small, local steps, using context to guide them. For example, rather than directly searching for a professor’s phone number (using a query like “david karger phone number”), they would search for the professor’s web page (maybe even by looking it up in a staff directory) and then try to find the information that way. The authors referred to this concept as “orienteering”:

Orienteering involves using both prior and contextual information to narrow in on the actual information need, often in a series of steps, without specifying the entire information need up front

Together, I think these papers demonstrate that while search is a useful way to find information, most people still need to use some kind of browsing to find the information they are looking for. The orienteering paper suggested that this kind of approach lessens the cognitive burden on the user. I think it really makes sense if you think of a real-life metaphor: you know how to find your way to a certain pub, but you don’t know the address and couldn’t give precise instructions about how to get there. Humans are naturally good at this kind of incremental way-finding, and browsing is a good way to take advantage of that.

(Photo by AlbeJTD on Flickr)


7 Comments:

  1. e - May 10, 2007:

    Regarding orienteering: I haven't read the paper, but is it possible that the authors are overlooking that the search isn't perfect? Keyword search allows us to look for instances of words that we already know. When we're looking for a phone number, we're looking for a word that we don't know. Keyword search isn't designed to do that. It forces the user to search for words that probably appear with the target word, but are not guaranteed to occur with it. The example that you gave is outside of the design parameters of keyword search.

    The problem with phone numbers is that the user is expected to understand what they are through context. When listing a phone numbers on web pages, people usually just put the phone number down. The syntax and context is enough to identify what it is: ###.#### or (###)###-####. It isn't necessary to use semantic waypoints (such as the word "phone") on the page; meaning that a keyword search is almost impossible, unless you already know the phone number.

    We can get around this by making our search engines aware of the syntax of common strings, and index instances of those strings specially. ###.#### or ###-#### is something that is usually a phone number, so store it in a magic index of phone numbers, and allow the user to search for pages that contain an item of that type, without having to know the exact instance that occurs there.

  2. Patrick - May 10, 2007:

    You're right that to some extent people do this because they are used to being disappointed by search results. If a "perfect search" did exist, people might be more inclined to do a direct search. They didn't ignore that in the paper though; I just didn't choose the best example. Here's what the paper has to say about why even the perfect search is not enough:

    However, even when a person knows exactly what they are looking for, the perfect search engine might not be enough. Consider Rachel, ... she attempted to locate a document that she knew existed in her file system. Although she knew exactly what document she was looking for ... she could not describe the document, its contents, or its location in advance:

    I don't know how I could have the directory [the document was in] in mind without knowing its name, but I felt sure which it was

  3. e - May 16, 2007:

    I did get around to reading the paper. The basic gist seems to be "hey! people orienteer!" which I'm find with; but I think part of the reason for some orienteering is due to the crapitude of keyword search.

    There are good reasons to orienteer: finding context, searching for unknown information, or performing a survey of information. However, I would argue that orienteering for known information (ie, something like a phone number or a meeting time that the user has seen in the past, but doesn't know the location of) indicates a failure of the search facilities. That isn't to say that the user shouldn't be given the option of orienteering, and that tools shouldn't be built with orienteering in mind, but that search should be improved so that orienteering isn't necessary.

    Thanks for the article citation. It's nice to dip into academia again. =)

  4. Patrick - May 16, 2007:

    I guess that's a fair criticism. In one paper I read yesterday, the analogy they used for that kind of conclusion was "it's like observing a family listening to the radio, and concluding that they don't want pictures". But still, I think there is a certain cognitive difficulty of specifying exactly what you are looking for, and it's often easier to follow a slower path and orient yourself using the context.

  5. MH - May 22, 2007:

    But what about the idea that you rely on what users do, versus what they tell you they will do? Does that not apply here?

  6. Patrick - May 22, 2007:

    True, this study did rely on the people to report what they did, rather than being directly observed. It's not the most scientific study. But if people were going to "stretch the truth", I would think they would do it towards making their behaviour seem more perfect, e.g. "Oh, I was looking for Jack's phone number, so I searched for `Jack Tripper University of Toronto phone number'". In the paper, many of the people are apologetic or even embarrassed about how they actually performed the search.

  7. Dubroy.com/blog » Greatest Misses: Popular posts from the last year - October 15, 2007:

    [...] Google is not enough [...]