Technology As Fashion

For quite some time now, people have been decorating their notebook computers with stickers and using amazing photographs or graphics for their desktop wallpaper. Apple products have achieved their level of popularity as much for their visual impact as for their technological capabilities, attaining the status of fashion icons. More recently, HP has followed suit, developing its ENVY line of notebook computers, shown in Figure 1, which take some fairly obvious styling inspirations from Apple’s aluminum counterparts.

Figure 1—HP ENVY

Tablets and smartphones have also become fashion statements as much as they are technological tools. The fashion case market has exploded in the past few years, with thousands of artistic designs now available. People are paying hundreds of dollars for fashionable aluminum phone cases like the iPhone case from Element Case that is shown in Figure 2.

Figure 2—Element Case for iPhone

Because fashion is ultimately a matter of personal expression, people value new and innovative products that let them express their own sense of the aesthetic. Custom smartphone and tablet cases and skins let people cover their devices with just about anything they might want. Custom messenger and computer bags like those available through Rickshaw Bags—Figure 3 shows an example—let people choose any fabric or design for the construction of their bag. People can even avail themselves of custom etching services for personal electronics. Figure 4 shows an etching on an iPad. With so many companies offering products and services that provide fashion outlets for technology, the next step may be customizing technology itself for people.

Figure 3—A custom bag from Rickshaw Bags

Figure 4—Etching of a cherry tree on an iPad

Recently, we’ve seen the release of a wide variety of new over-ear headphones, all sporting the same basic technology. The popularity of headphones from companies like Skullcandy, shown in Figure 5, and Urbanears, shown in Figure 6, is as much about their visual style as their sound quality. With so many similar headphones on the market, these companies needed some means of differentiation to avoid being lost in the noise.

Figure 5—Skullcandy headphones

Figure 6—Urbanears headphones

Many manufacturers have aligned with popular artists for help developing and marketing their products—like Dr Dre and Beats headphones or Rza and WESC. Skullcandy has differentiated its products by providing advanced features like the ability to daisy-chain headphones together using detachable connectors. In contrast, Monster gained significant advantage by developing and releasing Beats headphones years ahead of the competition—successfully predicting that high-quality, fashionable headphones would become a significant market trend over time.

With quality over-ear headphones becoming a new fashion trend, other important factors are emerging—such as using fashion as well as features to differentiate products that are fundamentally similar to one another, building customization into the product design and purchasing process, and predicting what future technology might become a fashion trend over the next few years.

Technology Is Social

A major technology trend that we are seeing emerge is that of being social. While social technology has been a developing trend for many years, the trend is re-emerging in new ways, with new devices and connectivity. For example, over the past year or so, music has become social in entirely new ways through Web applications like Turntable.fm. By providing a social aspect to music rather than just access to a library of songs like the popular applications Spotify and Pandora, Turntable.fm has provided a uniquely sticky music experience. The social aspect of Turntable.fm helps people to leverage the social experience of sharing, discovering, and discussing music in a compelling, new way.

For the past few years, Zynga has been leading the charge in social gaming. Recently, one of their most viral games has been the wildly popular Words With Friends. Rather than just allowing people to play a game online in which involving friends is something of an afterthought, Words With Friends is an inherently social, person-to-person game that requires people to play with their friends—and even includes a chat function that enables trash-talking. Through Words With Friends, Zynga has found a way of connecting people directly—sometimes across vast distances—through a competitive and fun activity.

Breathing new life into an old activity by using new technology and leveraging a social experience, Hasbro has updated the game of Lazer Tag for smartphones, releasing Lazer Tag guns that operate with mobile phones. People can shoot their real-life friends with electronic signals that register on their friends’ smartphones. Rather than connecting people across distances through the Web or mobile networks, these new smartphone Lazer Tag guns connect people through technology much closer to home. It’s the kind of game that helps people make new friends.

Each of these technologies establishes new ways of connecting people, whether across great distances or right next door. They do so through established recreational activities that companies have repackaged for electronic delivery. Expect to see new social technologies over the next few years, leveraging the inherently social aspects of newer devices like tablets and connected televisions. Also, think about social activities in which we currently engage that are ripe for an electronic makeover.

Increasing Device Connectivity

Just a few short years ago, the only device users were likely to own was a computer of some sort, likely a desktop computer. However, as the years progressed, desktop computers became notebook computers; then smartphones gained widespread adoption. Right away, transferring data between smartphones and computers became a pain point for users, and easy ways of syncing contacts, calendars, and email were in high demand.

Now, with the introduction of tablets, ultrabooks, connected televisions, game consoles, and cars, we are dealing with more and more data-housing devices and risk our data becoming increasingly fragmented. In response, services like mint.com have sought to consolidate information from multiple sources into a single location and cloud solutions like Dropbox have begun to blur the file-location barrier and provide access from any location or device. However, cloud storage limitations, as well as reliability and connectivity questions are preventing a wholesale migration to the cloud.

Amazon seems to be setting itself up to tackle this problem through their advanced cloud solutions, growing out of Amazon Web Services. Through Whispersync, Amazon provides an excellent model of how to tear down barriers between different devices, having already accomplished it beautifully for ebooks. With Amazon’s forays into music, movies, and now tablets with the Kindle Fire, they seem to be positioning themselves to own content delivery across platforms.

Apple, Microsoft, and Google may have something to say about that, but they are behind when it comes to device connectivity. Apple just recently released iCloud, but their history with connectivity between devices has been mixed—with iTunes providing clunky interactions between smartphones and computers. Microsoft has provided significant device connectivity between computers and TVs using Windows 7 and the Xbox 360, and Windows 8 promises to do even more. Google has made significant moves with their Google Apps line, providing useful quick calendar syncing across platforms, as well as shared access to documents, spreadsheets, and calendar events.

Over the next few years, look for each of these companies to jockey for position in the market by creating a seamless flow between devices and users. Content delivery will be key, especially with the storage landscape changing rapidly, becoming cheaper and cheaper, and connectivity becoming faster and more omnipresent. The ultimate solution will likely be a cross-device platform that is similar to the latest versions of Android—for smartphones and tablets, Android’s Ice Cream Sandwich—and Windows 8 for PCs and tablets. Both are optimized for multiple devices. These large companies will likely be looking to acquire companies that excel at solving data fragmentation issues and establishing the seamless flow of data that they all hope to create.

Conclusion

User experience is about more than just developing new technology and perfecting it. It’s also about paying attention to what users do and want to do with existing technology. Steve Jobs was not a visionary because he created new technology; he excelled at seeing how new technology fit into the lives of consumers.

By understanding users, trends that are emerging in user behavior, and future technology directions, we can set ourselves up to be at the forefront of innovation. By understanding patterns in people’s current and past behavior, we can do more than recognize their current needs; we can anticipate their future needs. Identifying recent trends has led to fashionable products that bring people together and operate seamlessly across devices.

If you are familiar with various research methods and their applications, your organization’s business goals, and product development constraints such as schedule, budget, and the development process, you can create an end-to-end user research plan to cover an entire product development cycle. Having a user research plan helps you to establish and communicate research goals and expectations regarding logistics and set research milestones for a product development effort.

Research Phases

For a User Experience team, the key phases of a product development cycle are Discovery and Design. Discovery consists of defining the market for a product, its potential users, and product requirements. Common user research approaches during the Discovery Phase include survey research, focus groups, expert interviews, and ethnography.

The Design Phase involves bringing a product to life through product or user interface design and refinement. User research during the Design Phase focuses on gaining a deeper understanding of users, confirming research findings from the Discovery Phase, identifying the relative value of various features, and refining a design as it evolves. Common user research methods during the Design Phase include concept testing, usability testing, and competitive benchmarking.

Understanding the different challenges you’ll need to address during these different phases allows you to plan your user research effort throughout a development cycle.

Planning ahead of time lets you prepare for user research studies in advance, reducing the time and effort you must devote to a given research activity. For example, once you’ve identified your target users, you can start building participant lists and networks to facilitate recruiting. When preparing for your first study, you must establish communication with many potential participants and select study participants from that pool of participants for your study. Once you’ve established that participant pool, it becomes easier to find participants for subsequent studies, simplifying your recruiting efforts, reducing the man-hours you must devote to contacting potential participants, and shortening your research schedule. Over the course of multiple studies, this preparation pays enormous dividends.

Discovery Phase

The main goal of the Discovery Phase is to define the product your team is developing. To do that, you must understand the market and the needs of users within that market. Therefore, the first step in planning user research is understanding your team’s information needs. Your organization’s business goals always drive its need for user research. Understanding these goals and needs is a little more difficult during the early part of the Discovery Phase, before you’ve even defined a product concept.

At this early stage, your business goals tend to focus on serving a given market. So, through your user research, you should endeavor to understand that market, define its segments, model the users belonging to them, and learn about the needs of those users. Surveys are an effective way of getting an overview of a market and understanding its characteristics. Through the results of a survey, you can define your market size, identify market segments, and define the demographics of the different segments. However, surveys can tell you very little about user needs and behavior.

Following your survey research, you’ll need to learn more about users and their behavior. Your best option for identifying users’ needs is some form of contextual user research. Performing any kind of needs-finding research that uses self-reported data—including interviews, surveys, and focus groups—is notoriously unreliable. People have a great deal of difficulty being objective about their own lives, and they tend to be blind to the workarounds they employ in meeting their needs.

If you ask people what they need, they’ll tell you a better, faster, and cheaper version of what they already have. Steve Jobs was famous for saying, “People don’t know what they want until you show it to them.” Contextual research approaches such as ethnography or interviews that you conduct within the context of product use, whether at home or in the office, circumvent this difficulty by letting you directly observe users. This allows you to identify needs from their actual behavior. This kind of research can also provide quite a bit of information about a user’s environment and behavior, which can help you to further define a product.

Once you’ve identified a need, the next step is determining the best way to meet that need. At this point, you define the product. The product’s primary features usually target meeting the identified user needs in some way. The nature of the needs often determines the form the product takes. For example, if a need tends to arise when a user is in a coffee shop or walking around outside, the product would likely be a mobile app or a portable device. When defining a product, user research focuses on learning more about current user behavior and verifying identified needs through additional interviews, focus groups, and other forms of generative research. Finally, you must gain as deep an understanding of your users as possible to inform design. This includes identifying additional desirable features and defining the scope of a product’s complete feature set. It is during this phase of research that you might decide to develop personas.

Design Phase

The goal of the Design Phase is to create the design for a product. User research during the Design Phase focuses on gaining an understanding of users and their needs to inform design. As designers develop concepts, you can do concept testing to evaluate whether a design adequately meets users’ needs and creates an effective vehicle for delivering a solution. Activities such as collaborative design sessions and card sorting let you gather ideas about how to design a product to best meet users’ needs. This research also helps you to prioritize a product’s potential features—again to inform design.

As a product’s design evolves, usability testing can help to ensure that its user interface enables users to realize the benefits of the product. Usability testing should be iterative. Include some directed tasks during concept testing to ensure that you discover usability problems early, before the development of a user interface element begins. Doing iterative testing helps you to ensure that any design changes your team has made in response to prior tests have successfully resolved the issues.

Competitive benchmark testing allows you to see how your product stacks up against its competition or against your alternative designs for a user interface. During this period, you should be able to identify competitive differentiators for your product. These are the attributes that make your product different and better from the competition. This research informs marketing as well as design.

Planning Ahead

Generating an end-to-end user research plan can help you to save money by leveraging the findings of and resources from previous studies. As you envision the product development cycle, create a rough user research plan at the very beginning, then refine your plan as additional findings come to light. Start with a general plan that segregates Discovery and Design into separate phases. Then identify what you already know about the market, users, and their needs. If you are familiar with a market through your prior research, you might already have sufficient data about that market, so you can probably skip some of the survey research. Beyond that, build a framework that is flexible, but well structured. Figure 1 shows an example of user research framework for a Web product.

Figure 1—User research during Discovery and Design

Conclusion

When employing an end-to-end user research plan, make sure that you respond to data as it emerges. For example, if important data is not coming to light through interviews, you may want to try another approach to acquire the information you need—for example, journaling or ethnography. It is important to utilize the proper research approaches to satisfy needs as they arise. So fill out your tool chest by making sure that you are familiar with as many research approaches as possible.

Remember, concept tests and usability tests should always be iterative. By planning your user research in advance, you’ll be able to perform this testing relatively quickly by reusing test plans from previous studies and drawing from the same pool of participants when recruiting. With proper planning, you can integrate user research smoothly into the development cycle, including agile development approaches, without disrupting schedules. Good user research helps to mitigate risk and ensure that your product is successful upon its release. Consider doing post-release research that your team can apply as it continues improving a product or plans a follow-up release

In much the same way governments use intelligence in making military or diplomatic decisions, user research provides the information that product designers and stakeholders need to make decisions. If a government’s intelligence is accurate, leaders can make good, well-informed decisions. However, if the intelligence is inaccurate, leaders might make costly decisions that accomplish little or have negative consequences. Think of user research as design intelligence that helps product designers and stakeholders make informed decisions regarding product direction. Inaccurate design intelligence tends to lead to bad decisions, even when the best designers and savviest stakeholders make them. So, what are some of the ways in which user research can go wrong?

The Wrong Users

Perhaps the most serious problem that you might encounter is user research with the wrong participants. When you do user research with participants who are not representative of your target users for a product, any and all of your findings are likely to be incorrect. Information from the wrong participants can lead to designing a product for the wrong population of users. These could be people who are unlikely to adopt the product, do not experience the need that the product attempts to address, or are not qualified to provide accurate feedback on the product.

For example, early in our research careers, we assisted in a research study for which others defined the protocol and performed the recruiting prior to our involvement. We helped only to moderate the sessions. The study focused on how people drive and technologies that could help them drive more effectively and safely. The problem was that several of the participants did not drive on a regular basis, but instead took buses or carpooled. A couple of them didn’t even have their driver’s licenses! Any data that you gather from the wrong participants, as in this example, would not be representative of your actual, intended users, so making decisions based on that data could take you in the wrong direction.

The most obvious cause of doing research with the wrong users is making mistakes in recruiting research participants. To prevent this, make sure that your recruitment screener includes both descriptive and behavioral questions. For example, if you are recruiting Web developers, ask them to describe their work and what programming languages they use. An appropriate participant who is a Web developer should be able to tell you, for example, that he primarily does back-end development and uses JavaScript as his primary programming language. If you’re recruiting users who develop mobile games, be sure to ask what type of phone they own, whether they play games, and what their favorite games are. At the beginning of each research session, start off with similar questions to double check that you’re doing research with the right participant. Even if you find out too late that someone isn’t right for your study and have to scratch a participant, it’s better than getting inaccurate data.

A less obvious cause of doing research with the wrong participants is inaccurate user profiles or personas. If the research that identified users was flawed, it can result in personas that don’t accurately describe users. Factors such as small sample size, idiosyncratic users, too much focus on extremely salient findings, and investigator bias can lead to inaccurate conclusions. If you do not correct such problems through a review process or follow-up research, you can develop personas that don’t accurately reflect your users, and these inaccurate personas can start a domino effect, resulting in inaccurate research throughout the complete development cycle for a product.

The Wrong Questions

Understanding a problem you’re trying to solve is fundamental to meeting user needs through product design. Conversely, failing to sufficiently understand the problem tends to lead to an inability to solve it. The biggest obstacles to gaining this understanding are your own preconceived assumptions regarding users, their needs, and the product. Researchers tend to form their research questions around their assumptions, so our assumptions are the foundation for all the knowledge we gain through user research. If your assumptions are incorrect and you never realize it, everything else that you discover will be flawed.

For example, according to Guy Kawasaki, the prevailing assumption in the early 80s was a “better, faster, and cheaper MS-DOS machine.” If Steve Jobs had never challenged this assumption, the Macintosh would never have revolutionized personal computing with its GUI and desktop metaphor. But he did, and Apple competitors that were committed to MS-DOS soon realized that a market dominated by the Mac, then Windows had left them behind. More than that, they missed out on the chance to take the lead by thinking outside the box and creating their own innovations.

Unchallenged assumptions are dangerous, so it’s always important to challenge your assumptions in your research. When we’ve worked with the military and the FAA—organizations where lives can depend on the accuracy of our research—they’ve required that we document all of our assumptions in our test plans and final reports. Doing this helps with interpreting the findings and judging their reliability. We advise our clients to engage in a similar exercise. We recommend that you organize a collaborative session in which designers, engineers, researchers, and stakeholders try to get all of their collective assumptions on a whiteboard, discuss each one to determine the safety of each assumption, as well as the need to test them and an effective method of testing them. Incorporating assumptions into existing research efforts is relatively easy, only marginally increases cost—if at all—and pays off in big ways.

The Wrong Conclusions

Once you have recruited good research participants and obtained good data by asking good research questions, the next step is interpreting the data to arrive at accurate conclusions that drive product and design decisions. However, interpreting data sometimes leads to inappropriate conclusions, which tends to occur for two primary reasons: investigator bias and being overly aggressive in interpreting data.

Investigator bias can be very difficult to overcome. Biases tend to surface in the form of findings that seem like they should be right or that you expected to see. They become a problem when the data doesn’t support your conclusions. At times, research data may refute your conclusions, but in a way that might be too subtle to easily recognize. As the investigator, you may not be aware of your own biases, which can make it extremely difficult to guard against this problem. The more deeply you’ve worked within a subject area, the more deeply ingrained your biases become.

One effective method of overcoming your biases is to work with a partner who can act as a check against your biases. A good partner is someone who is knowledgeable about user research methods, but isn’t deeply involved in the project or subject area of your research—and thus can maintain objectivity. Everyone has their biases, so it’s important that you serve as a check against your partner’s biases as well.

Biases are closely related to the other source of inappropriate conclusions: interpreting data too aggressively. This can happen when a participant reports a particularly interesting and salient point of data—the kind of data that really gets your attention, usually because it’s something that you haven’t thought about before. However, for this very reason, it’s likely that you haven’t systematically tested the finding across multiple users.

You have to be careful not to rush into turning such a finding into a recommendation without fully vetting it. Our rule of thumb is that once is an outlier, twice is a coincidence, and three times is a trend. You want to look for trends to report rather than outliers.

You might find that someone has a really interesting idea, but upon investigation, most other users hate it. This doesn’t necessarily mean that you have to discard the idea, but it does indicate that it is a much more complex topic than you might originally have perceived. This is something you should be aware of before recommending the idea to stakeholders—or the product team may implement the idea to the detriment of the product and its users.

During your research, if you uncover something interesting with a given participant, try to assess your discovery with subsequent participants by adding it to your research protocol. If this isn’t possible, in your final report, document it as a new research question to explore further and recommend including it in follow-up research.

Conclusions

This month we’ve taken a look at some common reasons for faulty and inaccurate research findings that could end up threatening a product’s success. There are other threats—ways in which research can go wrong—but these are the most common ones and tend to be the most damaging. It is extremely important that the consumers of user research are able to recognize the difference between good and faulty research to maximize their chances of success in product development. The best protection against faulty research is to have a good research team and a good research plan. Next month, we’ll get into planning effective research for an entire product lifecycle and how doing the right research at the right time can help to ensure that you are getting good product intelligence.

If you find yourself in such a position, there are some key principles you should keep in mind that you aren’t likely to discover in any textbook or research manual. Learning any skill involves both knowledge from books and what you learn through some form of personal mentorship in which you can receive feedback on your work. Research is a complex skill, and people who are just getting started need both knowledge and guidance to gain mastery. We’ve tried to distill what we think are the most important concepts that we emphasize when we’re mentoring new researchers. They are principles that we have discovered through our years of experience, and we’ve found that they apply in all instances.

Make Friends

Research is a social activity. Many people have a vision of researchers as stoic intellectuals in white lab coats, sitting and observing people from behind one-way mirrors. The reality is quite different. User research is a process in which you communicate with people so you can learn about their lives and their needs. In the long run, forming an understanding of the people who will be using your product is much more important than just knowing their individual thoughts on a design concept or user interface.

The only way that you’ll be able to gain this understanding is if you can put participants at ease and talk to them as real people rather then just using them as a means of improving your product. The best way to connect with participants is to take your time and get to know them as people before you dive into your research protocol. Ask them questions about their lives, their jobs, and how their day is going. You need to get to a level deeper than just superficial chitchat. You want your communication with research participants to feel like you’re talking to a good friend.

Keep in mind that you have two goals: first, to help a person feel relaxed and comfortable, so he or she will open up and communicate freely; second, to learn as much as you can about their lives. This understanding can provide invaluable insights that can help you to better form your questions, as well as interpret their answers. For example, if a person reports that he doesn’t see any value in a product and you know about his life, you can ask him about potential use cases that he may have overlooked. If someone is being vague in his answers to your questions, you’ll know how to direct him toward making a definitive statement by applying the concept to a specific example in his life.

More specifically, if you were testing a product that lets users find their parked car, but a participant mostly rides his bike and parks his car for days at a time, you might ask whether he sometimes has difficulty remembering where he’s parked his car three or four days ago. The participant might not have immediately thought about applications for this product because he doesn’t drive often, but by suggesting an important use case, you may help the participant look at the product from a different perspective. This data can inform both design and marketing strategies.

Don’t Answer Questions

The purpose of research is to gather information, not to provide it. When people ask you questions, your instinct is to try to answer them, but you must resist this urge, because it will interfere with your ability to get accurate and actionable data.

I’ve seen people doing user research explain a product or user interface to a participant, including all of its features and how it operates. This prevents your having the opportunity to get a participant’s immediate reaction to the product. Instead of explaining a product’s value proposition to a participant, ask the participant What do you think this is? What do you think you would use it for? This lets you get an idea of how clearly a product conveys its concepts. If they are unclear, you can then explain the product and ask participants how you could make the ideas clearer.

When testing user interfaces, present participants with scenarios for tasks that would motivate them to try to figure out how to use the user interface properly on their own. Here’s an example: You’re moving into a new apartment, and you need to sell a couch that you aren’t going to take with you. How would you go about doing that? If a participant asks, Do I go to auctions? Don’t answer the question! Just note his response and tell him to feel free to try things. If a participant notices something in a user interface and asks you, What’s this over here? you should respond, What do you think it is? What would you expect it to be? Try to be like a therapist and always answer a question with a question.

It is more important for you to get a sense of participants’ impressions and reactions to user interface elements than for them to understand every aspect of a user interface. Try to keep in mind that customers won’t have you sitting next to them in the real world. It’s important to try to replicate that reality during research.

Don’t Try to Sell a Product

When you’re doing research, you want to make sure you get an idea of the value that a product provides to users. In fact, conscientious researchers would recommend radically altering or even cancelling a project if they learned that users would not want it or use it. If you point out all of the great things about a product, people will tend to agree with your opinions rather than form their own and tell you about them.

As you are going through your research protocol, make sure you document participants’ natural reactions in the course of their first exposure to a product. As we mentioned earlier, try to see whether they can understand the product on their own, then examine their reaction to the product’s value proposition. You’ll tend to see a variety of reactions, ranging from This is amazing, it’s going to change my life! to I don’t think this would do anything for me at all. Be sure to obtain this information—or you could be in for a significant beating when you release your product to the market. If you identify problems early, you’ll have the opportunity to pivot before devoting any more resources to something that is unlikely to take off.

To do this successfully, you must establish your impartiality. We actually inform people that we didn’t design a product and that we are performing the research to provide an objective opinion. We also inform participants that the goal of a research study is to improve the product rather than to make sure that we are on the right track. This helps to put them into a mindset of providing constructive feedback. As a researcher, you should foster this mindset and try to avoid influencing participants.

Conclusion

In this month’s column, we’ve covered three important guidelines that you’re most likely to learn on the job rather than in a textbook or manual. The higher-order guideline is to avoid influencing participants and keep your opinions to yourself.

Remember, the most important thing to gain through user research is a deep and meaningful understanding of your potential users. This can guide you through all aspects of development, including coming up with marketing and advertising strategies.

Of course, there is much more for researchers to learn, including how to generate actionable recommendations, fit into an agile development cycle, build compelling presentations, and communicate effectively with stakeholders, but those things are often better experienced firsthand. If there are any other essential guidelines that you think should be covered, we invite you to share them with everyone in the comments.

In this month’s column, we’ll discuss both the value of working with a professional recruiter when doing user research and best practices for making this a smooth process.

Why a Reputable Recruiter Is Essential to the Research Process

When conducting research, it is very important that the population you’re examining ties directly to the product or service that’s under development. If you sampled the wrong user population, the results would guide the design and engineering teams down the wrong path, leading to the development of a product that very few people would use.

When you hire a reputable recruiter to assist with participant sourcing, they use a highly structured, well thought out, and detailed screening process to ensure that they pull the right types of people into the study. The recruiting process is a partnership between the researcher and the recruiter. For example, while a researcher might request participants who have limited smartphone experience, the recruiter might refine that request, focusing on people who don’t own or operate a smartphone on a regular basis and, thus, helping to more specifically define the intended participant. As a result, the participants who are in the study will understand the concept of a smartphone, but won’t own one. It’s such tiny details as these that play a role in acquiring the right people and ensuring your study’s success.

Building the Screener

Typically, a recruiter requires you to develop a participant screener that outlines the types of people you want to recruit. The screener includes information such as age, education, income level, and experience with certain technologies. It is also very important to add behavioral questions such as How many hours per day do you watch television? or How many times a week do you go shopping? A good recruiter examines your screener and asks clarifying questions to further refine it before starting the recruitment process. This includes examining your questions to determine whether some are too strict, making it hard to gather together enough participants, or too loose, resulting in too wide an array of participants.

The screener also includes the incentives participants are to receive for helping with your study, and the recruiter will help you to determine the right amount for the incentive to ensure that you’ll get the people you need. If you provide too small an incentive, people might feel it’s not worth their time to participate. However, if you provide too large an incentive, you’ll find some people simply want to collect the cash and don’t care about helping you refine the design of a new product you’re building.

Locating the Participants

Using Craigslist is definitely an option for recruitment, but most reputable recruiters have an extensive database of participants they can contact at a moment’s notice. Their databases indicate what participants respond well during sessions, deceive or lie to researchers, have been no-shows for sessions, or have been involved in too many studies, so are becoming professional participants. Recruiters’ databases also help identify which participants are very communicative and which tend to provide only superficial data. Some of these databases are extremely impressive and powerful, allowing recruiters to gather high-quality participants with surprising speed.

When necessary, a recruiter might use Craigslist to find participants, but they’ll use a rigorous screening process to ensure the participants truly meet the needs of the researcher. Unfortunately, it’s not uncommon for participants to deceive recruiters and researchers just to be part of a study and collect the incentive. However, recruiter databases can help you to avoid this by tracking these kinds of participants and eliminating them from consideration during future recruitment efforts.

Without recruiters and their participant databases, you’ll often be gambling with your recruiting—significantly increasing the likelihood that you may inadvertently recruit a participant who is interested only in collecting the incentive. When research sessions go to waste because of bad participants, you end up wasting a great many resources, including the researcher’s time, the cost of the incentive, and the opportunity cost of not obtaining good data.

Participant Management

A reputable recruitment firm ensures that participant management is part of the recruitment process. Most firms include this as part of their core service offering. Participant management includes calling the participants two or three times in the days leading up to your study to ensure they actually show up, handling cancellations, and dealing with any last-minute issues such as shifting session times. Usually, for a small additional charge, a recruitment firm can also mail out participant incentives, so the researcher does not have to deal with money on the day of the study.

Getting participants to show up for a study is much more difficult than it sounds, which makes participant management a great deal more important than you might expect. People who agree to participate in research usually have good intentions, but when they get stuck at work or have sick kids or car problems, you’ll need a recruiter who quickly notices such issues and can find replacements at the last minute. We’ve found that, with the right recruiter and good participant management, the percentage of participants who actually show up for studies tends to be around 90 percent; without participant management, its closer to 60 percent. This part of the process can make or break your study.

Best Practices When Working with a Recruiter

These guidelines can help you to successfully engage a recruiter and keep the recruitment process moving smoothly.

1. Discuss the types of participants you need with your team before you contact a recruiter.
2. Be ready to quickly build a participant screener when you engage a recruiter.
3. Provide the recruiter with any personas or other documents that describe the product’s intended users.
4. Discuss the screener and your research goals with the recruiter to give him or her a thorough understanding of your needs.
5. Respect the opinions of the recruiter, because they work with researchers and study coordinators all the time.
6. Address issues with individual participants with your recruiter, so they can record them for future reference.
7. Make sure your recruiter provides up-to-date participant schedules, including participant contact information.

Conclusion

Professional recruiters are an extremely valuable part of the research process, so if your budget allows you to use one, we highly recommend that you do. As we said earlier, getting the right participants is just as important as conducting the research. Getting feedback from the wrong participants can have huge negative consequences for your product and may even lead to its failure upon its release. Even if your team discovers problems with a design that is based on faulty research prior to a product’s release, solving them is often very costly and time consuming.

If your budget does not allow you to hire a recruiter, we recommend that you create a very detailed screener for your screening process, and ask people who have a lot of research experience to review it for you. Doing so, helps to ensure that your screener is solid, so you’ll tap the right people to help out with your research.

When you find recruiters who do a good job of finding quality participants for your studies, make sure you hold on to them. They will become your best friend.

We are always looking for ideas and comments on how to improve the recruitment process for user research, so if you have something to share, please let us know.

The Path of Innovation

Everett Rogers introduced his theory of the diffusion of innovations in 1962. [1] Since that time, his theory has provided guidance to businesses and individual innovators when bringing new technologies to market. The normal path to mainstream success begins with innovators and early adopters, as Figure 1 shows. Innovators are typically people who actually develop new technologies. They plug into the newest technologies as a way of feeding their need to innovate. Early adopters are the first consumers in the market and the test bed for other users. Early adopters are typically itching to try new technology, regardless of the quality of its execution. So they usually acquire new products immediately, just to try them out and determine whether something will be a revolutionary success or an innovative misfire. In this respect, they provide a valuable service.

Figure 1—Rogers’s adoption curve

Right behind the early adopters is a group of users called early mainstream. These users typically pay attention to new technologies, but wait for the market to test them. Following early mainstream users are late mainstream users—that is, people who don’t usually pay a great deal of attention to technology innovations, but eventually adopt new technologies and products once they have become firmly established or even have an overwhelming presence in the marketplace. These users usually come to new products passively.

Finally, there are late adopters, who are people who tend to be suspicious of new technology, don’t see the value of its new capabilities, or don’t feel capable of engaging with new products because of their lack of experience with technology. Older users are a classic example of late adopters, but this group also tends to include people who live in rural or isolated areas and adult users who earn less than $55K annually.

Throughout each of these steps in the adoption curve, social influences tend to play a major role. Early mainstream users hear about new products and technologies from early adopters. Late mainstream users get introduced to new technologies by their early mainstream friends. When late adopters do take up new technology, it’s typically pushed on them by a more tech-savvy friend or family member—someone they trust who influences them directly.

The Problem

Trust and direct social influence are powerful forces that can be quite difficult to replicate. In directly targeting late adopters with new products, companies would be skipping ahead in the adoption process, which presents a series of difficult challenges.

One of these challenges is reaching these users in the first place. Many late adopters do not regularly use computers to access the Internet, so common startup marketing strategies such as social media and online advertising do little to build awareness in the target audience. Late adopters typically still have stronger ties to traditional media such as television and print, with their much higher advertising costs, reduced exposure, and fewer measures of advertising effectiveness. Large corporations can tackle this challenge, because they have greater resources at their command, but startups often have difficulty generating awareness of products that target late adopters.

The attitudes of late adopters toward technology present other challenges these companies must overcome, because it’s not uncommon for late adopters to feel suspicious of new technology or think that most new technology is unnecessary.

Substantial experience with technology that has enriched their lives motivates early adopters to try new technology products. However, late adopters tend to lack the behavioral precedent of successfully adopting innovations, so the idea can be somewhat alien and uncomfortable to them. Establishing demand for a new technology among late adopters requires their embracing new behavior, so involves much more than just introducing the technology to the market, making its adoption much more difficult.

The process of designing products for late adopters also presents challenges, because of the perspective gap that exists between the innovators who create such products and late adopters. It can be very difficult for product designers and developers to relate to users who are on the opposite end of the adoption curve. As a result, they could develop something they think is relatively simple, but is still too complex for inexperienced users. They might also employ user-interface design conventions they think are fairly standard, but might be completely alien to a late adopter who lacks similar experience interacting with the Web or consumer electronics. Many of the interaction design standards that we use today are not necessarily truly intuitive; they have just been around for so long that most people accept them. Interaction design concepts as fundamental as the desktop and windows are metaphors for physical items that may not be immediately clear to inexperienced users.

Strategies for Targeting Late Adopters

By understanding the challenges of targeting those users who are late adopters, we can start exploring strategies that can help us to be successful with this demographic. Once we understand the difficulty that late adopters encounter when using complicated technologies and software user interfaces, it becomes fairly obvious that any products targeting this market segment must be very simple to learn and easy to use. However, the challenge is determining whether a product actually is sufficiently simple to use. To ascertain this, you’ll need to conduct usability studies, recruiting participants from this market segment.

Begin with early concept testing of mockups. Don’t rely on demographics when recruiting, or you may accidentally recruit participants who are not truly late adopters. For example, if you merely recruit people who are over the age of 55, you could end up with a 65-year-old retired software engineer who plays World of Warcraft constantly and practically lives on Facebook. Instead, your recruiting criteria should assess behaviors that are consistent with a late adopter such as the following:

• Does not own a computer.
• Does not know how to access email.

Do not rely on friends and family for concept testing, because late adopters can be particularly susceptible to telling you what they think you want to hear rather than the brutal truth.

During testing, show the mockups to participants and ask them to describe what they think they depict and the value such a product could provide to them. Make sure not to provide participants with any additional explanation of a product until they’ve first given you their description, or you will corrupt their answers. Although their answers are very likely to be incorrect, if they are reasonably close, you are probably on the right track. In this case, you might get away with just making a few tweaks or adding some minimal explanatory text to the product before its introduction. However, if they are way off in their descriptions of your product or cannot see its value, you might need to completely rework your product description, value proposition, or even the entire product. It’s not entirely unlikely that you may find the product is just a bad idea, which is unfortunate, but can save you a great deal of money and effort in the long run.

As you develop your product further, you’ll need to evaluate the actual user interface through usability testing. Again, it is essential to recruit the right participants and ask them to complete tasks without any assistance. Keep a close eye on where participants encounter difficulty and document their actual success rates. Late adopters tend to be very forgiving when using a user interface, and they often blame their own inadequacies in using technology rather than providing deserved criticism. Keep this in mind, and don’t rely too heavily on self-reported data. Instead, focus on objective behavioral data.

For innovative new products, it can be helpful to find a way to piggyback on the normal technology adoption process rather than trying to circumvent it. One way of doing this is to keep early adopters in mind during development. This could mean including some advanced features or creating a separate version of a product that provides value specifically to these users.

Ultimately, your goal is to create enough of a following to create a social influence on late adopters. If you intend to employ this approach, include a couple of early adopters in your usability testing and ask them who they think would get the most out of using your product. If they respond by saying something like This would be great for someone like my grandfather, you’ll know that you are on the right track. Apple’s iPad, with its simple interaction model and variety of features, is a great of example of an innovative product that successfully appeals to both early and late adopters. In such a case, trusted brand perception is a major advantage.

However, it’s important to keep in mind that late adopters might interact with these products in a very different way than you would. For example, users who are 55 and older—many of whom would historically be considered late adopters—are currently the fastest growing market for smartphones. [2] But this doesn’t necessarily mean they are using smartphones to play Foursquare and Angry Birds like early adopters would. In our user research, we’ve found that users from this age group tend to use the most basic capabilities like making calls and a few very popular apps such as the eBay app. Keep this in mind if you are developing a mobile application and, when conducting user research, ask about users’ existing behaviors, so you can discover the reasons why late adopters might use some apps, but not others.

Conclusion

Late-adoption populations offer tremendous potential for growth, but they remain untapped for a reason. Breaking new ground with late adopters presents a significant challenge. While, as always, the key to success is understanding your users, this can be especially difficult in this case because of the experience gap between late adopters and the innovators who develop technology products. To bridge this gap, make sure you do adequate user research. Remember, what you think is simple and intuitive might not be for a late adopter. You’ll have to challenge all of your assumptions to create a successful product.

It’s also important to think about how a late adopter might become aware of your product. Common marketing strategies such as social media and online ad campaigns would probably fail to reach this audience. Finally, try to find ways to leverage the adoption curve by creating interest in your product among early adopters and mainstream users. By employing these methods, you are much more likely to produce a product strategy that will find success in the marketplace. We invite our readers to share their thoughts and experiences with developing products for late adopters.

Do Use Focus Groups Early to Identify Important Research Questions

Focus groups can be an effective way to get started on a project. When you are just starting to explore a product idea, getting a group of potential users together to discuss ideas can be extremely helpful. Focus groups can provide you with a cost-effective means of testing your team’s initial assumptions, beginning to identify relevant market segments, exploring product ideas, and acquiring the data you need to create a UX research plan to guide product development.

Don’t Try to Do Too Much with Your Findings from Focus Groups

The biggest mistake people make with focus groups is an overreliance on the findings in informing design decisions. Focus-group findings are subject to a variety of confounds, including the relatively low numbers of participants, participants’ speculation during the session, and the significant influence the opinions of others can have on participants in a focus group. For these reasons, researchers do not typically use focus groups in rigorous scientific research. Focus groups provide high-level feedback that helps you to make strategic decisions. When you are getting to the point where you have to make very specific, tactical design decisions, it is time to move on to more reliable research methods.

Do Recruit Participants from a Variety of Backgrounds

One of your goals in doing early, discovery research is to identify relevant market segments. One way of  doing this is to gather representatives of different market segments into a single focus group to determine how people in different segments respond to particular ideas. You might find that certain ideas resonate strongly with some participants, but don’t interest other participants at all. In this case, exploring the reasons for their different responses can help you to identify different products that might fit into a single product line.

For example, when examining eReaders, you might find that some participants want a very compact product, while others want a larger screen. By exploring further, you might find that participants who are interested in a compact product like to read novels and tend to travel a lot, so the small form factor supports their reading habits. On the other hand, the participants who want a larger screen might like to read newspapers and magazines at home or in their office. These different use cases could lead to different products, each of which is designed to support the reading habits of one of these two very different market segments.

Don’t Recruit Participants Who Are Too Similar or Too Different

It’s important that focus-group participants have something in common for you to get useful information about a topic you’re exploring. For example, if you are considering developing a photography product, you’ll likely want all participants to be interested in photography in some way. On the other hand, it’s important to ensure that participants aren’t too similar for you to acquire a variety of perspectives and foster discussion.

Continuing with the photography example, if you have photojournalists, wedding photographers, and amateur photographers in a session together, you might find that they have very different needs regarding a photography product, but you also might find that there are a couple of themes that are consistent across all groups.

When you uncover such differences, it’s important to explore the reasons why people’s needs are different. For example, while amateur photographers might be extremely interested in advanced photo-sharing capabilities, wedding photographers might not. By exploring further, you might determine that wedding photographers need to keep very tight control of their photos, or work product, so they need more security than amateurs do. You might not discover such needs if you didn’t have these different types of users in the room together, talking with each other about their commonalities and differences.

Do Include More Than Two People in a Focus-Group Session

Good focus groups should include a variety of perspectives and foster communication among people with slightly different backgrounds. In a good focus group, participants comment on what other participants say, either expanding on a topic or expressing a difference of opinion. If you do a focus group with just two people, you’ll tend to end up with something more like two separate interviews that you’ve conducted concurrently, because you’ll have a harder time establishing a discussion with a lively exchange of ideas.

Don’t Include More Than Six People in a Focus-Group Session

Acquiring thoughtful comments on several topics from multiple people can take quite a bit of time. The more participants you add, the more time a focus group takes. We usually find that extending a session beyond two hours causes participants to tire and pull back from the discussion. For this reason, it’s a good idea to limit sessions to only six participants. This also helps you to maintain control of a session as the discussion gathers energy.

Do Prepare a UX Research Plan

A good focus-group session flirts at the edge of chaos. When comments are flying around the room, it can be difficult to maintain a session’s focus and ensure that it covers all of your planned topics. When you’re putting together your UX research plan, begin with the business goals for your research, then identify the research goals that would support those business goals. You should be able to extract some specific research questions from your research goals, and these will be the main topics for your focus group to discuss. Make sure you work all of these topics into each session without disrupting the flow of the discussion.

Don’t Use a Research Plan As a Script

During a focus group, striking the right tone is essential to facilitating open discussion and the flow of ideas. It’s extremely difficult to exercise any control over a discussion’s tone if you are reading from a script rather than responding to comments and ideas as they emerge. Also, focus groups are unpredictable, so you don’t know what new ideas or concepts may surface. It’s important that you be able to respond to new discoveries fully as they arise, and this can be very difficult if you are following a script.

Do Work with a Partner

In our experience, focus groups are the most challenging UX research sessions to moderate, because of the dynamic nature of the sessions, the number of participants you must interact with simultaneously, and the need to strike the right balance between a free-form discussion and a structured research session. Having a partner in the room is incredibly helpful in this kind of setting. Your partner can help out by taking notes, keeping track of time, and helping to enforce structure on the session when it starts to go off the rails. Your partner can also jump in and ask questions when he or she notices that there is a key point that the group isn’t addressing.

Don’t Rely on Your Partner to Help Control a Focus-Group Session

You should introduce your partner as someone who is helping you by taking notes, and you should think of that person as a coach. Only one person can control a focus-group session. If control begins to diffuse across multiple moderators, a session can quickly disintegrate, especially if the two moderators don’t completely agree on where to direct a session.

Do Invite People to Observe Focus-Group Sessions

It’s difficult to distill a complex interchange between multiple people into a report, even when using audio or video clips. For focus groups, it’s common for the dynamic exchange of ideas to be an essential takeaway. The best way to ensure stakeholders get this information is to allow them to observe the sessions. We like to do this by setting up video cameras and piping a feed over a secure network connection to any stakeholders who want to see it. We’ve also found that Microsoft RoundTable is especially well suited to streaming and recording focus groups.

Don’t Allow Observers to Interfere with a Session

As we mentioned earlier, the moderator should be the only person in control of a session. If participants witness others stepping in and directing a session, they may feel that they are also free to influence the direction of the session. As we said, if this happens, a session can quickly disintegrate.

Do Encourage Conversation Between Participants During a Session

Sometimes people find talking in a group to be intimidating, so it can take a little encouragement to get participants to open up. To facilitate everyone’s participation, set the tone for a session by taking the time to chat with each participant before launching a discussion of your planned topics. Introduce yourself and volunteer some information about  your personal experiences to let participants get a feel for who you are.

Do a round of introductions, and really pay attention to what each person says. Follow up each introduction with thoughtful questions that show you really care about who a person is and what he or she has to contribute. If one person seems more comfortable with open communication, begin the session by chatting with that person to set an example of the kind of communication you want, then transition to another participant who might not be quite as talkative. By making this transition, you’ll show more reserved participants that their thoughts and opinions are equally important to you and that they can communicate with you and the group in the same way as more gregarious participants.

Don’t Lose Control of a Focus-Group Session

If a discussion gains too much momentum, it can take on a life of its own and get away from you, so you might spend 20 minutes talking about something that is completely unrelated to your research goals. To avoid this happening, exercise strict control over a session, and don’t be afraid to stop people if they wander too far off topic or take too much time with their responses. But do this by saying something along the lines of I’m really interested in what you’re saying, but I need to get us back to the main topic of our discussion because our time is limited. This way, the participant won’t feel invalidated or shut down. As you get good at this, you’ll find more subtle ways of directing a conversation back on topic by asking questions like So, how does that relate to XYZ?

Also, take the time to identify any alphas in a group. Alpha is our term for people who like to control a conversation and direct its topics of discussion, whether intentionally or unintentionally. Such people tend to have forceful personalities and promote their ideas to a point where they influence the opinions of others. When your focus group includes one or more alphas, try to get their responses toward the end of each topic of discussion, so you can avoid their having undue influence. Be ready to redirect the conversation if they start to pick up too much steam and want to keep talking. If you have two or more alphas in the same group, try to avoid letting them talk one after the other. Insert a buffer between them. If they get going speaking to each other, they can dominate an entire session.

Do Discuss Each of Your Topics and Respond to Ideas As They Arise

As you work through each of your topics and new ideas arise, explore them to see what they mean and whether you can get multiple perspectives about them. Often, when people first introduce an idea during brainstorming, it isn’t yet fully formed, or they may not communicate it effectively. By asking further questions or getting responses from other participants, you can get a much better understanding of an idea. In such cases, a natural sense of curiosity serves you well.

Don’t Dig Too Deeply into Any Given Topic

Remember, your goal in a focus group is to properly frame your questions rather than trying to arrive at definitive answers. You’ll likely find that some of your questions turn out to be less important than you anticipated, while others that you hadn’t considered to be pivotal turn out to be more important than you thought. Don’t feel pressured to leave a focus-group session with all of the data you need to guide your team’s design decisions. Instead, try to leave each session with a firm understanding of your information needs. This can be extremely valuable because it can help you to avoid spending significant resources investigating topics that are unimportant. It can also help you to identify important issues that could seriously hamper your product’s success if you do not address them properly through additional research.

Conclusion

Although UX research and design have begun to migrate away from using focus groups, they do provide significant value when you use them properly. The most important guideline for the effective use of focus groups is avoiding an overreliance on the resulting findings. The findings that result from this method of UX research are not as reliable as those you can gain through more rigorous research methods. However, focus groups can be a cost-effective way of building the understanding that forms an excellent basis for the UX design research you ultimately need to do. The guidelines we’ve presented here can help you get the most out of your focus groups. Of course, it falls to you to exercise your best judgment in determining what research approaches to implement and how to proceed with them. But keep in mind that focus groups are always a tool at your disposal.

Figure 1—Focus group lab with a one-way mirror

Why a One-Way Mirror Is Not Always the Best Option

In the fields of physics, psychology, and biology, there is a phenomenon called the observer effect, which refers to how observation changes the object or person being observed. For example, taking pictures or videos of a tribal culture that has never experienced such technology introduces new concepts into the society and fundamentally changes their behavior, because this is not a natural experience they would normally encounter during their daily lives.

This is a major concern for user research and focus groups that are conducted in labs with one-way mirrors. When you place people in an environment where they sense they are being observed, their behavior changes. These changes in behavior can drastically affect your ability to accurately understand how your target market feels about your product or service. In our experience, people tend to focus on positive responses and have a very difficult time being completely open and honest about their thoughts and feelings.

One of our primary goals when conducting research is to make participants feel extremely comfortable. One of the ways we do this is by taking a little time to get to know participants and enable them feel that they know us a little, too. To minimize the impacts of observation on research, we emphasize the need for honest feedback and also reassure participants that we are not actively watching or evaluating them; rather, we are evaluating the product and taking notes and listening to their comments to enable us to improve the product. What we have found creates the most anxiety for participants is the idea that hidden strangers are staring at them through a window.

The perfect example is when participants try to complete a series of challenging tasks as part of a usability test. It is easy to see their level of frustration rise as they work through the tasks—while repeatedly glancing at the mirror.

User Feedback on the One-Way Mirror

We’ve had the opportunity to work with thousands of participants, and the feedback we tend to get about one-way mirrors is that they are scary, intimidating, nerve racking, and uncomfortable. In one case, when we asked a participant about the one-way mirror, she responded: “I could just feel everyone watching me, and it made me feel dumb for having such a hard time.” This type of feedback is worrisome because users may not be fully expressing their thoughts about the product. They become distracted by the feeling of being watched when they make mistakes rather than remainingve focused on improving the product by providing thoughtful feedback.

In almost every case, participants were in a very nice room with comfortable couches and chairs, and we had provided them with food and beverages. You might think they would walk away from the experience with a feeling of satisfaction and excitement, considering they had just had the opportunity to get a look at a new product or service. Unfortunately, the mere fact that these people feel like they are being watched changes their overall experience.

We took a deeper look into these anecdotal findings and asked a few research participants more detailed questions about their experience with one-way mirrors, and this is what we found out. Once participants walk into a room and see a one-way mirror, they believe there is a certain way they need to act, a certain way they need to respond to questions; and they feel that they need to do things correctly.

The only way we’ve been able to mitigate these feelings is to talk through them and reassure participants that we’re using the room only so we can tape the session, enabling further review. Since we know, in general, what feelings people tend to have when sitting in a lab with a one-way mirror, we try to address them immediately, as best as we can, and are able to put some of them to rest.

Best Practices When Using a One-Way Mirror

By combining our experience with what we’ve learned from discussions with other UX professionals who use labs with one-way mirrors for user research, we can offer some best practices:

• Discuss the elephant in the room and point out the two-way mirror, explain why you are using it, and address any concerns a participant may have about it.
• Talk with participants for a few minutes before a session and help them relax. Ask about their day or their work. It’s best to quickly build friendly feelings with participants to help them relax and feel more comfortable with the situation.
• Wear clothes that are similar to what participants are likely to wear to a session—perhaps jeans and a casual top. If you look like a psychologist or evaluator, that can make participants nervous and less likely to open up. We find that, when you blend in, you build rapport faster and get more honest feedback.
• Don’t let participants sit facing the mirror—have them sit to the side, facing away from the mirror, or if possible, with their back to the mirror. There is nothing more nerve racking than staring at a mirror for an hour when trying out a new product.
• If possible, moderate sessions from within the room with participants. Providing participants with instructions and asking them questions face to face helps them to forget about the mirror and focus on the person in front of them rather than on a disembodied voice coming over the speakers.
• Finally, don’t act like you are formally examining participants. Act casually and treat participants as if you are working with them, not just watching them. For example, when you ask participants questions about their feedback on a product, have someone else take notes, so you can have a normal conversation with them, and they can focus on you the whole time. We find this method very helpful, because it frees up a researcher to ask lots of follow-up questions and focus deeply on what participants are saying in the moment.

Alternatives to the One-Way Mirror

There are a few alternatives to using a lab with a one-way mirror that have proven successful in alleviating the uncomfortable feelings participants endure in such a situation.

• Rather than using a one-way mirror, install an overhead camera and put observers in a room down the hall. Participants tend to forget about cameras, but have a hard time becoming habituated to a large, mirrored wall.
• Bring participants into an office setting and conduct sessions in front of team members working at their desks. This may sound like a stressful environment for participants, but when they see everyone working and just going on with their business, they tend to relax and feel part of the team. This is an excellent opportunity for a team member to take notes, because participants won’t be paying attention to what everyone else around them is doing.
• Visit participants in their own environment—whether their home, business, or a local coffee shop. By doing this, you alleviate the stress of participants’ being outside their usual environment. In this case, you can use Web video services to stream sessions to observers and also tape sessions for future review. More realistic behavior tends to be prevalent when participants are in their own environment.
• Remote sessions are becoming more and more common and involve participants’ using their own computer to connect with a moderator and working through a test scenario or responding to questions like those typically asked in focus groups. This option is less personal, but allows participants to be in a setting that is comfortable for them. Plus, it creates an interpersonal disconnect that typically leads to increased confidence and honesty.

Conclusion

There are many ways in which we can conduct user research—using a one-way mirror is just one option. Although we do not think doing research in a lab with a two-way mirror is always the best option in all situations, we do think it can provide value in certain situations—especially if you follow best practices and give the proper attention to the needs of participants. But, in this column, we wanted to point out that there are alternatives to using a lab with a one-way mirror that have worked well for us over the years. Ultimately, our goal is to ensure that participants are comfortable, understand their role in our design projects, and provide the best product feedback possible.

We would love to hear your thoughts about this. Have participants given you any interesting feedback on how they feel about their experience of being in a lab with a one-way mirror? Let us know!

Users Fixated on Glitches, Bugs, and Errors

When you put a product that has glitches, bugs, or errors in front of users, they usually discover them. When they do, they tend to fixate on them and miss actual usability issues, especially those that are more subtle. It is because of this phenomenon that it is important to do iterative testing to uncover all of a user interface design’s critical usability issues. However, even if you are using an iterative testing approach, participants’ encountering many glitches could extend the number of test iterations you’ll need to do. If you are relying on just one or two iterations of usability testing to get your product ready, you are almost certainly going to miss quite a few issues if participants encounter errors.

When you are doing concept testing, errors can distort participants’ true reactions to a concept. So, rather than responding to the concept’s value proposition, participants instead respond to the execution of its implementation. When discussing barriers to adoption in a previous column, “Barriers to Adoption and How to Uncover Them,” we defined confidence as users’ believing that a product can deliver the value it promises. When you perform concept testing with a build that has obvious errors, participants’ confidence in the product suffers. In turn, their reaction to the concept suffers. Thus, even if participants attempt to compensate for the errors by ignoring them, their reaction is still somewhat tainted, and they can end up overcompensating or imagining a product that differs radically from the intended design. In that situation, your best course of action is to ask participants about the design they imagine. But keep in mind that they are still describing untested and vague design ideas rather than providing true feedback on the concept.

Increased Costs and Extended Schedules

Whenever we test with a buggy build, we know that the testing will take 25–50% longer than it would have with a stable build, because of the troubleshooting that tends to occur during and between test sessions. When a product freezes or crashes during a test, you must stop the session to solve the problem. This can often result in sessions’ extending far beyond their scheduled time. An hour-long session can easily become a 90-minute session, pushing back the start time for the following sessions, during which other participants may experience the same problems. At times, we’ve had to cancel sessions shortly after they started, because the complete failure of the software required that we wait for an engineer to repair or reinstall the build.

Such failures are time consuming and costly because they extend the timeline for acquiring adequate, usable data; require additional engineering support; and can result in the loss of paid participants. The need to replace lost participants means you’ll accrue additional costs for recruiting, participant compensation, and session moderation. In addition, adding more sessions when you replace lost participants can put a research deliverable date in danger of slipping, which could shift the development schedule, which is very costly, or result in design decisions’ being made without research findings, which is risky.

Obviously, it’s not always possible to get a perfectly clean, workable build. When you cannot, the best way to overcome the hurdles we’ve mentioned is to be as prepared as possible before starting usability testing.

We always test a build thoroughly prior to starting a usability study to determine its stability. It’s very helpful to know the ways in which a product can break ahead of time and whether there are incomplete sections you should avoid during testing. It’s also imperative to know whether a build could require a cold boot or even a reinstall. When we know that we are dealing with a buggy build, we can

• compensate by scheduling additional buffer time between sessions—This allows each session to run long as necessary.
• recruit additional participants in case there are cancelled sessions—We can cancel later sessions with some participants if they aren’t necessary. It’s much easier to cancel than to scramble in an attempt to find suitable replacements.
• have a research partner on a study if the budget allows—This allows one person to aggregate data while the other collects data or to troubleshoot while the other person performs a post-session interview or preps the next participant.

Anything that you can do to streamline your process and ensure you meet your deadlines is a sensible enhancement to your test plan.

Testing with Mockups or Prototypes

For usability testing, if you don’t have software to test, fake it. In most cases, testing with mockups or prototypes can provide excellent, actionable data. This holds true whether you are using a simple, clickable Flash demo for usability testing or paper prototypes for concept testing. By limiting the development of a prototype to a product’s front end, you can quickly create a prototype of a user interface that is adequate for usability testing.

If a designer or researcher on your team is familiar with simple Flash development or HTML/CSS prototyping, you can develop a prototype with minimal support from Engineering. There are also some software solutions available from companies like Balsamiq and Napkee that allow just about anyone to produce a clickable HTML prototype like the one in Figure 1.

Figure 1—A prototype made using Balsamiq Mockup

When taking this kind of approach, you should match your test-session design to the fidelity of the mockup or prototype. If you have paper prototypes or fairly simple clickable prototypes, focus primarily on core features, brand messaging, and perceived value. If you have a more complete prototype, you can progress to a more rigorous test of additional features and their added value.

When devising an end-to-end research plan, we typically start with need-finding research such as ethnography, home visits, or interviews, then do concept testing using paper prototypes to assess the value proposition, brand messaging, and feature set. As the design progresses, we test with paper prototypes, incorporating simple test tasks that address a product’s core functionality . Eventually, we’ll transition to testing core functionality, using low-fidelity, clickable mockups. The next step is more robust usability testing with medium-to-high-fidelity prototypes. Finally, we’ll transition to testing a reasonably stable build of the actual product, doing in-depth usability tests or following more advanced testing methods such as competitive benchmarking. We’ve found that this kind of iterative testing schedule is extremely effective in providing actionable design intelligence.

Conclusion

It’s never a great option to test with buggy or unstable builds, because doing so can compromise your data collection, complicate your study’s logistics, and potentially, impact your study’s budget and schedule. You can test mockups or prototypes of various types as alternatives to testing incomplete builds, but it is important to design your study to be compatible with the fidelity of the mockups or prototype you are using.

When a prototype simply won’t do the job and you need to use a build that you know has errors, it’s important to plan for the problems that are likely to arise. Before starting your study, test the build as extensively you can, note the areas in which you encounter difficulties, and plan for troubleshooting. In your test plan, it’s also important to accommodate the possibility of cancelled or extended sessions by recruiting extra participants, including extra buffer time between sessions, and working with a research partner or team of researchers.

As user research professionals, our goal is always to provide accurate, actionable research findings on schedule and on budget. For testing, we recommend to our clients that they provide a build that is ready for primetime. But, if that can’t happen, we rely on the tools we’ve described and a little creativity, so we can anticipate problems, quickly find solutions when we encounter problems, and keep our research objectives on track.

Likewise, many of us remember Hollywood’s early forays into 3D filmmaking. We saw our favorite slasher and monster movies in theaters, wearing goofy red-and-blue tinted glasses. Eventually, polarized lenses in plastic frames and other more sophisticated technologies replaced these. But it wasn’t until recently that 3D film began to fully realize its potential with the creation of newly immersive movie-theater experiences.

This month’s column explores how legacy technologies such as 8-bit video games and 3D movies have helped pave the path for some of today’s largest entertainment companies.

The Evolution of the 8-Bit Gaming Experience

The concept of video-based games has been around since the 1940s, but video gaming did not have worldwide impact until the release of the Atari 2600 in 1977. Atari established the market for home-based gaming consoles and, consequently, launched the billion-dollar gaming industry we see today.  Atari released the console during the holiday season of 1977 and had nine games available for users. These games set the tone for the family gaming experience and inspired many of the games we have all played and loved. It was this initial gaming experience that set the tone for what people could expect and would feel when interacting with a television or computer screen for fun. Atari, and the other gaming systems of the time, established many of the earliest forms of gameplay that are still prevalent in social and mobile gaming.

For example, the popular mobile game Angry Birds uses many gameplay elements of classic artillery-style games such as Smithereens! as you can see in Figures 1 and 2. Developed by Magnavox for the Odyssey 2 and released in 1982, Smithereens! used a simple play mechanic that let players judge the distance to a target and fire a catapult at an opponent. By correctly judging the distance, including the arc of the projectile, a player can accurately find his or her target, much as in Angry Birds.

Figure 1—Smithereens!

Figure 2—The similar gameplay of Angry Birds

Nintendo followed Atari into the gaming market, releasing the Nintendo Entertainment System in 1985 in the middle of a massive video game–industry crash. IGN eventually named the Nintendo Entertainment System the top video-game console in history. Nintendo introduced such classic games as Legend of Zelda and Super Mario Brothers. Their new generation of games enhanced the gaming experience with better graphics and sound, but also established new game mechanics in the areas of strategy, storyline, and character development. The NES gaming platform further solidified the look and feel of home-based gaming and set the stage for that nostalgic feeling we get when playing those types of games today.

This is where companies like Zynga, Playdom, and Playfish come into play. These companies are tapping into the historic user experiences gamers obtained from systems like the Odyssey, Atari, and NES. Their look and feel is reminiscent of the 8-bit, low-fidelity standard of that earlier time, and their simple, but addictive style of gameplay recycles the interaction design models of original favorites like Asteroids and Zelda. As Figures 3 and 4 show, games like Mafia Wars combine the nostalgic feeling of historic games like Dragon Warrior (1986) with the new frontier of social connectivity. This creates a very powerful psychological connection between users and, ultimately, a very large fan base.

Figure 3—Dragon Warrior

Figure 4—The similar gameplay of Mafia Wars

The 3D Movie Experience

In the early 1920s, the introduction of 3D film to a paying audience occurred at the famous Ambassador Hotel Theatre, in Los Angeles, California. The film they showed, The Power of Love, was ahead of its time and not a huge success, in part because of the limitations of the technology available at the time. It was not until the Golden Era of 3D in the early 1950s that 3D films started to take off commercially. By then, stereoscopic color and stereophonic sound had become available and were part of the 3D experience. Large studios such as Disney, Columbia, and Warner Brothers developed 3D films during that era. Films like House of Wax, Man in the Dark, and Melody were a huge success, and 3D film became established as a commercially viable format. Over the next several decades, 3D films became much more common, but the technology was in most common use as a gimmick to enhance sci-fi, monster, and slasher movies rather than for major dramatic films.

Beginning in the mid-1980s, IMAX began releasing 3D documentary films, and studios such as Disney started producing special-purpose films like Captain Eo for their theme parks. With their integration of 3D into amusement-park and educational films, they established a 3D film experience that engendered excitement and joy in the minds of children.

Over time, 3D technology has continued to mature in IMAX 3D films. Much of this progress has occurred under the supervision of filmmaker and innovator James Cameron. This period of rapid development for 3D films culminated in the release of Cameron’s technological masterpiece Avatar in 2010. Avatar was a huge financial and critical success, becoming the highest grossing film to date, as well as the first 3D film nominated for an Academy Award in the Best Picture category. In the wake of Avatar’s success, studios have converted numerous films to 3D prior to their release, ushering in a new era for 3D filmmaking.

While in 2008, there were only three feature-length 3D films that received a wide release, studios have scheduled 39  feature-length 3D films for wide release in 2011. Many of these new 3D films follow in the footsteps of their forefathers and are thrillers and monster movies like Sanctum, Piranha 3D, and Resident Evil: Afterlife. Often these movies employ shots that are similar to those of the thrillers from the 60s and 70s with objects flying at the audience, evoking some powerful feelings of nostalgia.

Using Nostalgia

Using nostalgia is a very effective means of establishing an emotional and psychological connection with an audience. Strategically applying some well-established user interaction models can evoke nostalgia in users and, thus, have an immediate emotional impact on them. The simple block-pushing play mechanic of Boxed In, instantly reminds me of block-pushing puzzles, which, in turn, remind me of The Legend of Zelda: A Link to the Past (1991). Such nostalgic connections tap into long-held feelings of enjoyment, indirectly creating new, positive associations. In addition, these historical references result in intuitive user interfaces, because they use familiar interaction models.

The recent movie Piranha 3D played on many genre conventions to evoke a feeling of nostalgia, including standard 3D camera gags, references to earlier Piranha movies, and even a cameo from Richard Dreyfuss, shown in Figure 6, reprising his classic role from Jaws (1975). This kind of clever use of nostalgia helps to engender a positive emotional reaction in the audience, fostering their close connection with the material.

Figure 6—Richard Dreyfuss in Jaws and Piranha 3D

Thus, you can use nostalgia as a way of developing an intuitive user interface and tapping into your users’ deeply held emotional associations with products of the past. When doing so, make sure you know your source material well. Technologies of the past have influenced all of us, so it’s important to recognize and understand these influences to work them into your designs in a meaningful way.

As an example, to make the most of the historical references in the movie Piranha 3D, you would need to know that Jaws heavily influenced the original Piranha. Thus, these references carry a great deal of meaning. Likewise, when developing products that have nostalgic elements, recognizing your influences helps you to incorporate such references in a way that makes sense to your audience. For example, the Tricorder mobile app taps into the similarity between mobile devices and the influential Star Trek gadget and includes authentic sound effects and visuals. While this kind of reference works great for a mobile device, it would be inappropriate for a desktop application.

Conclusion

As humans, we all have a rich history of interaction with technology and art. We carry this history with us, and it is the lens through which we view every interaction we have with technology. We can often see these threads of influence in the products we encounter. Today, it’s easy to see how the iPhone has influenced the latest HTC handsets. Likewise, it’s easy to see how the early Nintendo console controllers have influenced the design of PlayStation and Xbox controllers. Our ability to recognize these influences opens the door to our incorporating nostalgia into our designs, inspiring positive emotions in the people who use our products and eliciting an immediate understanding of their interaction models.