Dealing with Physical Constraints

There are some well-known constraints we must take into consideration when designing and developing mobile apps—mostly surrounding a device’s form factor and physical user interface. Thus, the type of device on which a mobile app will run is a major design consideration. One nice aspect of designing apps for the iPhone is that the device’s form factor and physical user interface are standardized and well known. Plus, you can market your app and people can buy it using the familiar user interface of the iPhone app store.

However, when designing apps for other brands of mobile devices, you’ll need to deal with significant variability in their screen sizes, form factors, and physical user interfaces. For example, a Blackberry phone may have a small screen with a physical QWERTY keyboard, as on the Blackberry Curve, or it may have a larger touchscreen and a virtual keyboard like the Blackberry Storm. Consequently, the interaction design for each of these devices must be quite different. The Storm requires large buttons to facilitate touchscreen interaction, while the Curve requires smaller navigation elements, so they’ll fit on the smaller screen. For this reason, it’s important to specify the mobile devices on which you intend your app to run when defining requirements.

The Mobile Space Is Not the Web

Many of the assumptions about user interactions that drive Web design do not hold true for the mobile space. It’s essential to recognize that your users will not be sitting at a desk and looking at a big screen for substantial amounts of time, in a relatively peaceful environment. Instead, your users will be mobile—perhaps walking down the street, sitting on a train, or waiting for a latte at a coffee shop—using your app in environments where they will be surrounded by stimuli. So rather than running in a quiet office, library, or home, your app must compete with countless, extremely compelling external stimuli such as the constant movement of people and vehicles around them, as well as interactions with other people. Also, because your app runs on such a small screen and carries less auditory impact, it is not as immersive and is less able to hold users’ attention than a desktop or Web application. Therefore, it is essential that users be able to open your app quickly, accomplish what they hope to accomplish, then exit quickly and return their attention to the outside world. Accomplishing this type of lightning-fast engagement is essential for the success of an application in the mobile space.

One of the easiest ways of achieving the kind of quick engagement the mobile space requires is to streamline your app’s functionality. This means restraining any form of feature proliferation. Single-function or limited-function apps have a definite advantage when it comes to quick engagement. Google provides a great example of this philosophy with their Google Maps mobile app, which is separate from the Google search app. Thus, they can provide various capabilities, while limiting the functionality they incorporate into a single mobile app. This approach offers several advantages. First, it allows users to understand the utility of each app easily—enabling them to more quickly choose the app they need among the collection of apps on their phone, as well as to quickly make a purchase decision when buying a new app. It also makes it easier for users to organize their apps by function, especially now that more smartphones are incorporating file structures as a way of dealing with app proliferation on devices. Finally, single-function apps have simpler user interfaces, which reduces screen clutter and lets users access key functionality quickly and easily, while also minimizing the impact of distractions by reducing the amount of attention using an app requires.

In a real-world environment, a mobile app must overcome competition for a user’s attention, which goes far beyond just overcoming a competing app’s claims on a user’s attention, as on the Web. For example, if you are developing a news delivery app, it’s important to take into consideration why a user would use your app rather than just grab the newspaper sitting next to him at the local coffee shop. Navigation apps have done a great job of competing for users’ attention. Their competition includes traditional maps, printed directions, and dedicated navigation devices. Traditional maps don’t provide position tracking or turn-by-turn directions. Printed directions don’t provide positional information either, so are not helpful if you miss a turn and can’t generate updated directions when you are mobile. Dedicated navigation devices can’t incorporate other functionality like making phone calls, checking your email, or updating your Facebook status, so they add device clutter. A mobile navigation app offers significant advantages over each of the competing methods of completing the same objectives. Your mobile app should offer similar advantages to help ensure its success.

It is also important to take into account the reality that users may be engaging in simultaneous activities that not only require their attention, but may also take up the use of one hand. For instance, a person may be trying to use your mobile app while also trying to carry groceries, walk a dog, or carry a cup of coffee. If users discover that it is difficult to use your mobile app in such situations, they may avoid using it unless they can devote their full attention to it. Thus, single-handed operation is a major consideration for mobile apps.

Mobile Usability Testing

This is a user research column, so we definitely want to address how to do user research for mobile apps. Let’s discuss some factors you should take into account when doing usability testing next.

Considering the Context of Use

As we stated previously, it is extremely important to take into account the different contexts in which people use mobile apps. For this reason, it’s sometimes useful to incorporate environmental distractions into the context in which usability testing takes place. There are few ways of doing this. First, you can perform a field study in which a person attempts to use a device in a real-world environment like a coffee shop or other public place. This kind of approach maximizes external validity, which refers to the extent to which you can generalize the findings of a research study to the real world.

Another testing strategy is to simulate a real-world environment within a controlled area like a usability testing lab. This might include playing music or videos in the background or even inserting additional people into the testing environment to periodically initiate communication with a participant. All of this occurs as the participant simultaneously attempts to perform tasks on a mobile device. Finally, you can introduce distractions in the form of distracter tasks. Introducing a distracter task involves instructing a participant to stop what they are doing, perform a prescribed action, then return to what they were originally doing. An example might be: Whenever you hear the bell ring, stop what you are doing and write down what time it is in this notebook. While distracter tasks do not consist of meaningful actions and need not mimic real-world situations, they are relatively easy to incorporate in a test scenario and give you an idea of how distractions can impact a user’s interaction with your mobile app. If you’re able to do iterative usability testing, you can begin with more traditional usability testing, then introduce distraction elements further along in the testing process.

Capturing Data

Data collection can be a challenge when testing mobile devices. Because of mobile devices’ small size, it can be difficult to get a clear video recording of participants’ interactions with a device. This is particularly true for touchscreen devices, because when a participant interacts with a touchscreen, his or her hand also obscures the user interface. This can make is difficult to see what buttons a participant presses. Also, participants with large hands may obscure the user interface merely by holding a device in their hands. Finally, because mobile devices are usually handheld devices, people tend to move around quite a bit while interacting with them, rendering normal stationary cameras ineffective.

The easiest way to work around these issues is to do initial testing using functional mockups on a desktop or notebook computer. This can provide great initial guidance on your app’s interaction design, but it’s important not to rely entirely on this kind of testing. At some point during development, you should do usability testing on an actual mobile device.

When testing on mobile devices, there are several approaches we employ. To capture user interactions despite participants’ hands sometimes obscuring the screens of mobile devices, we like to use a screen capture utility that outputs a video stream. This lets us see what’s happening on a mobile device’s screen on a connected computer, on which we can save the video stream. To do this, you’ll need to find a screen recording utility for the platform for which you’re designing your app. Here are some screen capture utilities for a few popular platforms:

• Android—The Android SDK makes this possible. For a walkthrough of the process, read “How to Capture Screen on Android Phone.”
• iOS—This a little trickier. Most solutions require a jailbroken device. Veency from Cydia is great, because it allows wireless, live monitoring on your computer. Another option is Screenrecorder, also from Cydia.
• Blackberry—There are a couple of third-party apps that can accomplish screen recording on a Blackberry: BBScreenStream and Mobiola Screen Capture.

Although recording interactions lets us see what user interface elements participants use, it doesn’t tell us whether a participant might have hit a button by accident or show us the participant’s reaction. To capture that information, we like to focus a second camera on the device a participant is using. This helps us capture the way the device fits in the participant’s hands, accidental activations, mistakes in interactions, and any other issues involving the use of the device’s physical user interface.

Because mobile devices tend to move around as participants hold them, we sometimes assign someone to be a dedicated camera operator. In this way, we avoid asking participants to try to hold the device still or in an awkward position where it is visible to a camera, as well as mounting the device in a way that would not represent how participants would actually use the device. However, if having a dedicated camera operator is not a possibility, we ask participants to hold a device in a way that is comfortable for them and set up a stationary camera, zooming in on the device to a distance that ensures the device remains in frame, so long as participants’ movement remains within a reasonable range. Usually, we also set up a second stationary camera to capture participants’ facial expressions as they use an app. By combining these three data sources, we can capture just about any relevant information during a usability test.

Conclusion

The mobile space is an exciting new technology frontier. Eventually, it will probably come to be dominated by massive software companies and require substantial budgets to create apps that have a chance at success. But for the time being, independent developers and a new generation of startups have the opportunity to blaze a new path by creating innovative mobile apps that redefine the way people interact with mobile technology. To meet these ambitious objectives, designers and developers of mobile apps must understand a new set of usability constraints that are unique to the mobile domain, including the variability of the physical user interfaces for devices on a single platform, usage contexts that are rich in distractions and competition for users’ attention, and users’ sometimes engaging in simultaneous activities that require an app to maintain usability during single-handed operation.

Usability testing of mobile devices also presents some new challenges. Traditional, lab-based usability testing doesn’t adequately simulate the actual operational context of a mobile app. Collecting relevant data during a usability test session in which a participant is using a handheld device also requires some new approaches. Fortunately, tools exist to solve this data-capture challenge, and new tools are continually emerging as the smartphone market continues to grow.

This innovative mobile space has the potential to launch the next tech titan. Among the countless app developers who are now working in this space, the companies that rise to the top will necessarily have done much to overcome the new usability challenges mobile devices present.

One caution—When considering doing unmoderated user research, it’s important to keep in mind that unmoderated user research is never as good as moderated user research. You should always avoid attempting to replace necessary moderated user research with unmoderated user research.

One huge fallacy we sometimes encounter is the belief that some user research is always better than none. Unfortunately, this is completely untrue. Improperly conducted user research can lead to bad decisions about product direction that can result in your inaccurately defining a product’s target market, defining the wrong key functionality for a product, or designing poor user interfaces. Each of these issues is enough to doom a product to failure when you release it to the market.

To compound this problem, often decisions that are based on the findings of user research—regardless of its soundness—receive more trust than they deserve, so they are less likely to be challenged and corrected than if you’d conducted no user research. There are many ways in which user research can go wrong, but we’ll save that for another column. For now, we’ll focus on ways of making good use of unmoderated user research tools.

Beyond Usability Testing

Unmoderated user research tools tend to focus on usability testing, but there’s no reason why you can’t use some of these tools for performing unmoderated concept testing or even miniature ethnography studies. For example, you could construct tasks along the lines of Please demonstrate how you would make a purchase from your favorite online store. 

While unmoderated user research does not replace moderated user research, it can be very effective in augmenting moderated user research. For example, generative user research such as ethnography can be extremely costly, but a company can hold down costs by performing ethnographic research with fewer participants, then supplementing their data through unmoderated research sessions. 

When performing user research, we look for trends. It’s very important to distinguish between behavioral trends and idiosyncratic behaviors when determining design recommendations. Distinguishing between trends and idiosyncrasies requires many participants—a major factor affecting schedule and budget. Unmoderated user research can be an effective and low-cost method of obtaining the data that lets you make this distinction. You can use moderated sessions to identify and thoroughly understand the behaviors that are of interest. Then, to verify the trends you’ve observed, look for those same behaviors in unmoderated sessions. It’s best to follow this rule: Do not use the unmoderated sessions to identify additional behavioral trends, because the understanding you can glean from an unmoderated session tends to be superficial. 

Combining limited ethnographic studies with unmoderated user research isn’t as effective as doing a full ethnographic study, but it is a way for cash-strapped startups to get some invaluable consumer insights. This approach of augmenting your moderated user research by involving larger numbers of participants through unmoderated sessions works with nearly any form of user research. 

Longitudinal Usability Studies

Unmoderated user research enables some innovative approaches to usability testing. One of these is longitudinal testing. Using a longitudinal approach to usability studies, we can learn how a person forms a long-term relationship with a product by getting data from people over an extended period of time. 

This approach lets us explore different levels of usability. Most usability studies that evaluate iterative designs focus heavily on discoverability, but a longitudinal study can also acquire data about the learnability and ultimate usability of a product once a user has become fully accustomed to its user interface. 

To get a more complete picture of a product’s user experience, it’s useful to pair your data from longitudinal testing with another longitudinal data source such as diary data. The unmoderated testing sessions provide data that is similar to diary entries describing interactions, while the diary data indicates participants’ goals in using a product, their perceptions of the product, and the nature of their relationship with the product. For example, diary data might indicate that usage is extraordinarily high at first, but then dies out as the product loses its novelty, indicating a need for continuously updated content. Conversely, diary data may indicate low early usage, followed by an explosion in long-term usage, indicating usability issues affecting adoption. 

For longitudinal unmoderated usability testing to be effective, it has some special requirements. First, the automated testing software must allow you to select your own participants, so you can test the same people multiple times. Second, you need to determine a testing schedule that would allow you to observe the factors affecting change. For example, to obtain data about learnability, you must schedule a test session during the critical period when a participant might be having difficulty learning a new user interface. Too early and you would be testing discoverability; too late and the participant would already have learned the user interface. Determining the proper frequency of testing depends on a participant’s frequency of use of the product. Thus, a higher-use product requires more frequent testing. 

It might be helpful to give participants some basic structure for how they’ll use the product during testing—for example, engaging with the product for at least 30 minutes each night. However, the trade-off for this kind of structure is your ability to examine participants’ organic usage of the product. This kind of study can provide some amazing data—of a kind that rarely gets captured at present—but you’d need to tailor your testing method to each product. 

Conclusion

This column has described just a couple of creative, innovative ways in which you can use unmoderated user research tools. We’ve described how by using unmoderated user research tools 

• you can augment the data you’ve acquired through moderated research sessions, reducing the number of participants you need for your moderated studies when identifying trends
• you can perform research that requires repeated measures—for example, longitudinal studies that explore the long-term relationship between a user and a product

However, there are some constraints on the use of unmoderated user research tools that you should be aware of. For one, these tools currently work only for computer-based software. Therefore, they may not be useful to anyone developing software for a mobile platform or for hardware. At some point in the future, these tools may become available for mobile platforms, but we are unaware of any such solutions at present. 

Unmoderated user research sessions are always inferior to moderated sessions—all other factors being equal—so use them wisely. And since it’s untrue that some user research is always better than none, it’s extremely important that you should become a conscientious and discriminating consumer of research. As we mentioned in our column last month, a little education can be extremely valuable in this regard. A thorough education in research methods also enables you to innovate new research methods that can open up new ways of understanding users. 

The two applications of unmoderated user research tools we’ve described in this column illustrate the general concepts behind combining unmoderated sessions with other forms of data to get a more complete picture, but there are likely other effective methods, too. We invite you to share your ideas for other useful methods of using unmoderated user research tools in the comments

Method founder Kevin Farnham presented quite a bit about Method’s history and a few case studies in which they were able to provide great designs for their clients. Kevin also spoke quite a bit about how valuable research is to the design process, at one point going so far to say that Method has been known to perform research on their own when their clients weren’t willing to dedicate the extra budget.

But, it was UX researcher Adam Dole that spoke at length about what it takes to perform quality research. As a researcher, I was very impressed with what Adam had to say. Of course, I’ve known Adam for quite some time, having gotten to know him when we both worked at NASA, but the laboratory-based quantitative research methods employed at NASA can be a bit different from the research that’s needed to support design. We speak with Adam on a fairly regular basis about research that taps into the user experience and I was very happy to hear him discussing these kinds of qualitative approaches to the audience.

Rather than delve into the specifics of research design and methodology, which can vary tremendously from study to study, Adam focused on expounding some general principles that apply to all research. I felt that his approach was correct, learning the various research designs and methods that Adam could teach would take a tremendous amount of time, much more than the hour that we had allocated for the talk. As proof of that, Adam outlined a list of different research approaches that could be used to get a picture of the user experience. These different research approaches varied from ethnography to simple surveys, but it demonstrated the wealth of tools available to researchers. It was much more valuable to the audience to hear the overarching principles that guide every single research endeavor. These principles included thinking outside the box, which amounts to challenging your own assumptions to see if the solution to any given need is something different than what might be assumed. Other principles included putting yourself in the consumer’s shoes and being “relentlessly curious”, about all important factors in performing excellent research. In the end, I was glad to get a chance to see and hear the Method design team discussing the importance of research and getting a chance to get to know the personalities and perspectives behind all of Method’s great work. I was also glad to hear Kevin’s perspective on the value of good research. And finally, I was very glad to hear Adam discussing research and the guiding principles that sometimes get overlooked even by people with deep research backgrounds.

By: Demetrius Madrigal