A Usability Study on Starbucks’ Mobile App

产品设计 2017-09-23 阅读原文

A Usability Study on Starbucks’ Mobile App

As a part of my Advanced Usability Testing class last semester, my team and I conducted a Usability Study on the Starbucks mobile application. This app allows users to pay for purchases, earn rewards, and order drinks in advance before reaching the store.

My Role: Prepared task list. Main observer/note-taker during test sessions. Identified and prioritized usability issues. Complied a video of usability issues. Provided recommendations.

Duration: Spring 2017, INST702 Advanced Usability Testing

Tools & Methods: Usability testing, Pilot testing, Document camera, Note taking, Survey, Qualtrics, Think aloud

The purpose of our study is to :

  1. Identify potential usability and design problems in the current app.
  2. Analyze the issues to determine the causes.
  3. Provide recommendations for improving the usability of the app.

Based on 15 usability testing sessions conducted in April 2017, our team has identified 5 most critical usability issues, provided a corresponding list of recommendations, as well as insights (likes, dislikes, and suggestions) from our participants.


Task List

In order to evaluate many of the app’s vast functions while keeping the sessions short, we created 3 sets of tasks — each with a different focus. These were all programmed in Qualtrics, which automatically assigned participants to one of the three task groups. Below is the finalized list of each task in each group, after 2 rounds of pilot testing to revise any unclear wordings.

List of Tasks for each Task Group.


To keep the conditions of our tests as natural as possible, we conducted and recruited participants at a Starbucks Store.

Approach:Think-aloud method
: Screen recording with document camera, Note-taking

Device: iPhone6 for iOS users, SS Galaxy S7 for Android users

Location: Starbucks store on Cherry Hill Rd., College Park, MD

The process was simple and broken down for each for us to concentrate on a certain aspect of the process. One of us would bring the participant to our table and brief them up on what test we were conducting and how they would have to go about it. We had them verbally approve our consent speech, which said that we would be using the data and recording the phone screen but we won’t be revealing their identity. Then the participant would fill out a background questionnaire. According to their choice, we would provide them an Android or iOS device. The role of each person in the team is as below:

  • Tech support : Start & stop recording the camera, take notes
  • Timer : Stopwatch & notes down time taken for each task, prepares device/resets app
  • Observer : Write down observations, main-note taker
  • Moderator : Runs the session, sits with the tester (introductions, etc.), notes observations


Across two rounds of usability testing, both within the Starbucks store and remotely, 15 participants were randomly assigned to one of the three task groups mentioned above. Due to technical difficulties with the first usability test, demographic information was collected on 14 of the participants. Fourteen participants self-identified as iPhone, and only one participant completed the test on an Android.

Of the five participants who self-reported previously using the Starbucks app, all five participants have been using the app for at least one year, while another two stated they had used the app for three to four years. When discussing whether participants used the app previously or not, three participants spontaneously mentioned only using the app to pay and had never ordered from it.

Participant Characteristics by Task Groups.


Task Completion Success Rate

From the data collected during our in-store sessions with the users, we deduced the task completion success rate. We had 5 users test each of the 3 task groups (15 users in total). The success distribution for the 3 task groups are shown below, ordered by %Incomplete to distinguish tasks with the most difficulties.

Task Success Distribution by Task Group.

Average Time on Task

The graphs below show the average time taken by the 5 participants to complete each task from a particular task set, ordered to distinguish tasks that took the longest.

Average Time Per Task (s) by Task Group

Survey Results

After participants completed their randomly assigned task set, each participant was asked to discuss their likes, dislikes, and recommendations on the app. We identified the most common themes as below.

Common themes for Likes, Dislikes, and Recommendations from participants.

Multiple nested buttons on the same banner make them difficult to distinguish.

Net Promoter Score (NPS)

Participants were also asked the NPS via the following question: “How likely are you to recommend the Starbucks app to a friend, family member or colleague?”. The average NPS was 6.57 out of 10, which is right between a detractor and passive respondent score. The NPS score subtracting the percent of promoters from the percent of detractors (14.29% — 42.86%) was -28.57 . Overall, participants are not likely to recommend the Starbucks app to a family member or friend.

NPS Score by Task Group.

Usability Issues

From the results of our observations and surveys, we consolidated a list of usability issues and then evaluated them based on a severity rating scale of 0 to 5 (0 being not a problem, 5 being a usability crisis). Please refer to this link for a video compilation of the issues we found during the test —


  1. Difficulty in finding the price of an item (Rating 5) : The application only shows the price after a user have gone through to the checkout section. Many users have expected the price to be shown earlier on in the process while they were selecting their items — often times going back to check on the main ordering menu. Additionally, since the checkout process requires a store location to be defined, many users backed out before they reach the point where they see the price.

The user expected the price to be listed in the item description or on the menu.

2. Minimum amount to reload is not clearly visible (Rating 4):After clicking on the drop-down list to select the reload amount, the users are shown options ranging from $25 to $100, with an additional “Other” option. Clicking on “Other” would reveal that the actual minimum amount to reload is $10.

The user added $25 to the card, assuming this is the minimum amount possible.

3. “Add Payment Methods” is not clearly visible (Rating 4):After clicking on the “Manage” in the payment section, users have the option to “Edit Payment Methods” or “Add Starbucks Card”. Since the user wanted to add a credit card, the second option seems very promising.

The user clicked on “Add Starbucks Card”, assuming this would add a new credit card.

4. Misleading nutrition facts after drink customization (Rating 3): Although the application allows users to customize their drink, the nutrition facts listed are not updated accordingly. Users have reported that they did not realize that the calories are no longer valid after their customization. However, the application does update the calories when users change the drink size.

The nutrition info was not updated after adding 2 pumps of Caramel Sauce.

5. Cup size option may lead to misunderstanding (Rating 2):For several drinks (such as an Iced Americano), the users may select the size of the cup to be served in, which is different from the actual drink size. One of our users misunderstood the cup size for the size of the drink.

The user misunderstood Cup Options for the Drink size.


From the usability issues we’ve identified in the previous section, we provide several suggestions for making changes to the application for a better user experience. We rate these recommendations on a scale of 0 to 5 (0 being nice to have, 5 being strongly suggested).

  1. Add each item’s price to the menu (Rating 5) : Many of our participants had difficulty finding the price and expect the the prices would be listed on the menu. Providing this information beforehand avoids the trouble of having to select the store and go through to the checkout process. One of our participants also pointed out that price plays a major factor in her decision-making and having the price right on the menu would make it easier for comparison.
  2. Make the minimum amount to be reloaded clearly visible in the options (Rating 4): For a more logical understanding, the first options shown should start with the minimum amount then incrementally increase with a “More” option for higher values. For example the options should be $10, $15, $20, $25, $30, More.
  3. Add a “Add Payment Methods” option (Rating 4): Instead of requiring the users to go inside the “Edit Payment Methods” section to add a credit card, there should be an additional “Add Payment Methods” option clearly visible at the same menu level with “Add Starbucks Card”.
  4. Update nutrition facts to reflect drink customization (Rating 2): The application can implement additional features to calculate new nutrition facts based on the user’s customization. Alternatively, a note saying that the calories does not account to customization can be added to the top of the nutrition list if this change are not to be implemented.
  5. Change the terminology for “Cup options”(Rating 2): “Size” and “Cup options” can be easily misunderstood, hence the cup size option should use a less ambiguous terminology, such as “Container size”.


While we are aware that several of the identified issues may have been a result of stakeholder’s decisions, our team has evaluated the studies based on actual observations with real users with the goal to report these findings as accurately as possible. Our team have gained an articulate amount of experience in performing hands-on usability testing, recruiting participants in a real environment, and drawing conclusions from our records and observations. We hope this study would bring useful insights for future enhancements.

Lastly, I would like to share the 3 most important lessons I’ve learned from this project:

  1. Finding a balance between the business goals and user goals is crucial to the design of a successful application.
  2. The document camera drew our participants interest and evokes professionalism . This was extremely helpful in our recruiting process, especially with the lack of tangible incentives.
  3. Determine what needs to be done to reset the testing device beforehand. It is extremely important to clear all added cards, settings, and small details before proceeding to the next session.

If you like this article, please feel free to give me some applause *clap clap*.

Please feel free to reach out to me if you want to learn more about our project.

Find me on

, or stop by my
to see my other works!




UI你连这些都不知道 别说你做UI UI你连这些都不知道 别说你做UI 开篇 今天来为大家盘点一下关于UI设计那些基础的知识,相信很多想要从事UI设计的朋友都有自己去网上查阅一些资料吧!那么这边对于UI设计的官方解释我也不多说了,相信大家都自行百度过,那我们说些什么呢?这篇文章主要给大家说清楚以下几个问题! 【解密】30秒...
The ultimate law of mobile site design: Entertain ... Most consumers rely on their smartphones to make purchases and gain knowledge. In 2017, any business that lacks a mobile presence runs a serious ...
iOS 11人机交互指南概览:主题(Themes)... 本文译自Apple官方发布的人机交互指南,希望能够对你有所帮助。 iOS 设计主题(iOS Design Themes) 作为一个app开发者, 您将有机会开发出一个优秀的、可能排在app store榜首的产品。当然,您必须对产品质量和产品功能有更高的要求。 区分iOS和其他平...
iOS 11 设计上的改变 What’s New in iOS 11 – WWDC 20... 从寻路、排版层级、对比度三个方面介绍 iOS 11 设计上的一些变化,从而帮助你设计一个具有丰富的视觉层级、清晰的导航、易于使用和简单的交互的 app。 概述 Wayfinding: 在 iOS 11 中提供更加清晰的导航、贯彻整个系统的寻路; Typographi...
靠着吉祥物,这款闹钟APP成功俘获了用户的心... 产品和用户之间,通过界面和交互来进行沟通,沟通的方式是每个设计师都需要深入思考的事情。不同的界面元素发挥着不同的功能,而其中最特殊的大概就是和品牌有着紧密关联的 吉祥物 了。 在 Tubik Studio 设计和开发闹钟应用 Toonie Alarm 的过程当中, 吉祥...