Location Irrelevant for Usability Studies - Really?
We at LeftClick are very interested to read Jakob Nielsens latest Alertbox article where Jakob explains that “as long as you’re testing within a single country, there’s no reason to expend resources traveling to multiple cities and conducting the same usability study again and again. You’ll simply observe the same behaviors repeatedly, and learn nothing new.”
The article draws a clear differentiation between market research, where study location is important and usability studies where its is not. The main reason given is that that “with usability, we test behavior, not opinion.” We find this a little alarming because surely opinion is a critical component that is inextricably linked to the whole “customer experience”.
Herein lies the general problem we see today where market research, customer research and usability testing are treated as completely separate disciplines. The pure “behaviour testing” focus of usability testing that does not explore values, beliefs, opinions etc can and often does lead to misleading results where a website can pass a usability test but still fail to satisfy its customers.
As a case in point, we recently conducted a fairly major e-business study for an online retail company whose customers were regionally dispersed and found significant regional differences. Jakob’s Alertbox article suggests that testing in various locations is not necessary as you’ll “observe the same behaviors repeatedly, and learn nothing new”?. We found the exact opposite. Although there was a great deal of commonality, there were also fundamental differences in online customer behaviour that led to key design decisions being made that otherwise wouldn’t have been.
In fairness to Jakob, we’re not slating the article completely as much of what he says is more product focussed so arguably usability testing a parking meter in one location just may be enough. But we do strongly advise caution for accepting the argument for online business/e-commerce where it is critically important to analyse/optimise the complete user-experience and this can only be achieved by examining aspects beyond usability and behavior alone.
Three comments
I regard ‘context of use’ important - there’s a need to understand factors which affect, distract or influence the user. So for user (field) research I will first try and observe users at their place of work (or usage environment).
We utilise usability testing as a checkpoint to audit our own designs (for prototype software) - so I usually do this from one location… However, if I know the context of use will have a major bearing on the user behaviour then I would be more likely to test on location (or at least factor-in realworld influences).
Hey Jaco and Zef
Thanks for commenting.
Jaco, yes I agree and you kind of reinforce the point that I’m making that, to make say an e-commerce sight really persuasive and effective, then usability testing just isn’t enough, you need to extend the research to inquire about the things that ‘click’ with the customer (care words etc).
And Zef, absolutely, context is very important and don’t get me wrong, usability testing is highly beneficial as a checkpoint to expose problems with ease of use etc. I guess we need to consider though does it really go far enough to move products and websites from good to great?
We’ve learned over the years that a website for example can be technically easy to use (usable) but still fail to convert sufficient numbers of visitors into leads and sales. Consequently, we now take a very different approach to site testing that helps to expose the things that will make people more satisfied. One of the things we do here is to explore opinions during testing and this in itself delivers some real golden value that we act upon directly in redesigns. It’s because of this that I find the comment from Jakob’s comment about not testing opinions as being rather blinkered in many respects.
Leave a comment
Comment Guidelines
- Have no more than 2 links, otherwise your comment will be flagged as spam.
- _text_ to make text italic.
- **text** to make text bold.
Let’s assume that a user’s experience is shaped, or at least influenced, by that user’s culture. We know that cultures can vary quite a bit between different locations in the same country - even in a small country like New Zealand.
Let’s take this train of thought further and say that we can create a more positive user experience, and also a positive perception about the site, if we used Gerry McGovern’s Customer Carewords (http://www.gerrymcgovern.com/mcgovern-carewords.htm).
If we want the Carewords to work, we had better make sure we use the right ones, right? And that they fit the local culture of the visitor.
Does this make sense?