There are dozens of different types of testing and hundreds of different methodologies out there. I’m going to focusing on the big three that we interact with in UX on a regular basis:
User interviews vary greatly depending on whether they are moderated or unmoderated, who wrote the script, and who is conducting the interview. This is a very soft science and a lot of care needs to go into how they are constructed. In general, you will have better results with moderated interviews because you can observe the participants’ body language, correct trains of thought before they derail completely, and ask probing questions when the participants hit on something interesting. So why would you do an unmoderated study, if you’re going to lose all of that insight and control? Unmoderated studies are easier to recruit for and they’re fast; for a lot of companies, it’s that simple. If you’re going to go the unmoderated route, UserTesting.com has a decent platform; but if you don’t have a lot of experience with this type of work, you may find it difficult to get usable feedback. Regardless of the path you take, here are a few things to consider:
When you hear, “Don’t fix what isn’t broken”, you probably think of some iconic luddite from Hollywood; someone who is unwilling to acknowledge or unable to see the fault in the system and has no desire to address it. Unfortunately, sometimes this describes our clients–not every client and not every time–but many of you can probably name at least one project within the last year.
This post is all about why you shouldn’t actually try to fix the things that are broken.
But before I get started, I want to highlight a few exceptions:
In an earlier post, I talked about how UX fits into the development process and ways we could be adapting and improving collaboration. I feel like there is a lot that UX has brought to product development as our field has matured and there is a lot about solid UX principles that we can and should be leveraging in our design process and methodologies.
In this post, I’ll be focusing exclusively on the Planning and Analysis stages of the Software Development Life Cycle (SDLC):
Requirements Gathering (today)
The first phase in most project usually involves gathering requirements. This phase can have a tendency to be very rigid, for example: business analysts conduct stakeholder interviews to get the very detailed answers they need in order to write their business/functional requirements documents. That might be an overly reductionist view of what often happens, but regardless of who begins gathering requirements, what I often see is the following pattern of questions:
The UX field is naturally focused on the user’s experience and that means that we sometimes identify conflicts between user needs and business needs. There was an anecdote floating around not too long ago that one of the Facebook redesigns improved the experience so much that people were able to get what they needed faster and spent less time on the site. Facebook is providing their product for free to its users, thus it makes all of it’s money off of advertising revenue. Therefore, people spending less time on the site translates directly into Facebook making less money. They quickly released another update that was “even better”, but looked a whole lot like the original newsfeed. I’m not a heavy Facebook user, so I couldn’t tell you if this was true from my personal experience and I have not seen any data–though I doubt they’re keen on releasing that.
As someone who cares passionately about the user’s experience, I really dislike when the business gets in the way of good product design. Most often, this happens when there is a specific behavior that the business must see (e.g. session duration or ad exposure for Facebook). The trick is to find the right way to present your value, so that when you do ask for the behavior, users are more than willing to comply.
How do we ask for the behavior without sacrificing the user experience?
The answer to that question is driven by these:
What problem is this user trying to solve with the product?
What behavior does the business need to see in order to make money?
When have we sufficiently proven the product’s value through the experience to justify the ask ?
We will sometimes get hung up on labels, but whether you call them scenarios or journey maps, or user flows; what I’m referring to here is the situation in which someone is engaging with your product.
Kim Goodwin had a great conversation with Jared Spool on scenario-driven design here. It’s only 30 minutes and well worth listening to, but here is the transcript if you prefer the written word.
Why should you care about scenarios?
Moore’s Law is still holding and as technology is rapidly advancing, we cannot accurately predict how tomorrow will solve today’s problems. Therefore, it is unrealistic to expect everything we build today to work for tomorrow’s users.
This is where I feel like scenarios can give us an edge in our design: by increasing the time to obsolescence.
If we design for features first (without considering the scenario), then we’ve really only addressed the symptom of the user’s true problem. That means all someone else has to do is actually solve the problem and their product will be superior, even if we have the better features. Users don’t want features, they want solutions to their problems.