Heuristic Evaluation: Online Grocery Ordering
Using the Cub.com Website with Instacart Portal
What is a Heuristic Evaluation?
A heuristic evaluation is an evaluative methodology that functions separately from user testing. It uses set of broad criteria to study an existing product. This criteria, called the basic 10 Usability Heuristics, was originally developed by Jakob Nielsen and Rolf Molich.
How is this Typically Done?
It is usually most effective if completed by 3-5 evaluators who specialize in the content area of the app
It is best if evaluators can reference a set of outcomes based on the business requirements.
Ideally the evaluation will focus between 5 and 10 heuristic criteria, culminating in more in-depth findings without overburdening the evaluators.
A severity scale range of 5 states is ideal.
Usability Heuristics Criteria for this Case Study
Visibility of System Status
Match Between Systems and the Real World
User Control and Freedom
Consistency and Standards
Error Prevention
Recognition Rather Than Recall
Flexibility and Efficiency of Use
Aesthetic and Minimalist Design
Recognize, Diagnose, and Recover from Errors
Help and Documentation
In This Case Study
I will be performing an evaluation using the Instacart portal for the Cub.com website, a large grocery store chain offering online ordering. I will limit this study to the website, excluding the corresponding mobile app, unless the information is relevant.
My goal is to present a comprehensive case study including each of the 10 heuristic criteria within the Nielsen Norman Group’s 10 Usability Heuristics for User Interface Design. Since I did not have access to the business requirements for this website, I have done my best to present my own conclusions based on what the business requirements may be.
* Disclaimer: The case study provided here is solely to demonstrate my process of performing a Heuristic Evaluation. It is not meant to be considered an official evaluation for this website. Due credit should be given to the design team who worked on this website and mobile app.
Evaluative Methods
The Nielson 5-point rating scale will be used. I will assign a Pass or Fail for the severity of each item based on this rating scale. The ratio of pass to fail items for the category as a whole will become the “percentage of success.”
Ratio of Pass : Fail = % Success
example: 4 Passes & 6 Fails = 4:6 = 4/6 = 67%
Summary of Findings
Percentage of Success for Each of the Heuristics
Users could still feel fairly comfortable using this site even if they make a mistake. The website’s strongest heuristic segments are Recognize, Diagnose, and Recover From Errors, and User Control and Freedom. Such strengths are good indicators that the user experience has already been considered in developing this product.
On the other hand, it is clear that this website may have deficiencies in the areas of Match Between System and the Real World and Recognition Rather than Recall. This is just one general way to view the data.
It is important to remember that the point of a complete heuristic evaluation is to identify items with higher severity ratings (2-4) in any of the heuristic categories. If designers are able to focus further usability testing and research on those items, the research would be further supported. If budget does not allow for this, at least the items have been identified for improvement.
Detailed Results by Heuristic
#1 Visibility of System Status
The design should always keep users informed about what is going on, through appropriate feedback within a reasonable amount of time.
User is able to tell where they are in a process, they learn the outcome of the steps they have already taken, and they are informed of what the next steps will be.
67% Success
User’s location does not always appear as a section header at the top of the page, and when scrolled is replaced by a sticky top navigation bar. (Severity = 2)
When searching by department, side navigation highlights the current department. (Severity = 0)
Items and CTA’s have hover status although some are very slight. (Severity = 1)
During the checkout process the website displays user progress with a common design pattern using progress dots. During checkout, screens have text explaining what action is needed within that screen. (Severity = 0)
It is very clear when an item has been added. The icon changes color and the text changes also, becoming a numeric quantity. A message appears at the bottom of the screen. (Severity = 0)
The screen for selecting item alternates happens after checkout is complete. It can be quite lengthy, which could be irritating to the user, especially after they have completed checkout. There is an option to skip this, but it may be beneficial to let the customer know this prior to checkout. (Severity = 3)
#2 Match Between System & the Real World
The design should speak the users' language. Use words, phrases, and concepts familiar to the user, rather than internal jargon. Follow real-world conventions, making information appear in a natural and logical order.
Use language and terms familiar to the users. Follow real-world conventions and use natural mapping where possible.
40% Success
User is given virtual ways of choosing products similar to being in the store. Alternate items are automatically suggested for the user’s approval in case their first choice is out of stock. There are additional options to add a note to the shopper and to search for a different backup item not shown as one of the choices. (Severity = 0)
In general, language is clear and there is no jargon. (Severity = 0)
Virtual coupons are not explained or differentiated from sale products. On the coupons page view, all products say “coupon applied” but there is not context or help information for this. (Severity = 3)
Images are clear, but for some products there is no context to reference the product size or scale in real life. In these cases (like produce) the user is unable to tell if an item is miniature or regular size without looking up the details view. (Severity = 3)
For a produce item that is priced by weight, user is able to order by item quantity (i.e. 4 bananas) rather than by weight. However, this doesn’t work for all items (i.e. bulk oranges.) (Severity = 2)
#3 User Control & Freedom
Users often perform actions by mistake. They need a clearly marked "emergency exit" to leave the unwanted action without having to go through an extended process.
Users are able to back out of a process or undo an action. This gives the user more control and an increased sense of freedom and confidence.
89% Success
Very easy to remove or deselect an item and change quantity. (Severity = 0)
Familiar “x” close icon to exit cart and to exit item detail. (Severity = 0)
Back button takes you to previous screen. (Severity = 0)
User is able to back out of an order on the delivery/pickup preferences screen prior to submitting the order. (Severity = 0)
Note to an in-store shopper can be added in the product detail view and in the cart item view at prior to checkout. (Severity = 0)
Once the order is submitted, user is prompted with the options to add/edit items in the cart, or change the order arrival time before the in-store shopper begins. (Severity = 0)
App can also be used during shopping to communicate with the shopper via text or via in-app messaging to replace items or communicate special requests. (Severity = 0)
Departments are listed in the side menu and can be expanded by item type. Search function works well. (Severity = 0)
While a user is re-ordering items off of the “Buy It Again” page and conducts a product search, the website exits this re-order screen. Upon finishing the product search however, the website does not return to the user’s place on the “Buy It Again” screen. This requires extra action on the part of the user to scroll to the spot where they had left off. This may not align with the user’s mental model and add frustration. (Severity = 4)
#4 Consistency & Standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform and industry conventions.
Using consistent industry conventions (external consistency) as well as consistency within a product or product family (internal consistency) keeps the cognitive load low for the user and improves learnability.
60% Success
In the cart view on the desktop website, the icon(s) to edit items are green, while in the mobile app they are red. “Preferences” on desktop lets the user choose a backup item. The same button on mobile is labeled “Instructions.” (Severity = 2)
Desktop Website vs. Mobile App - Cart View
Website menus are at the top and side. Search bar is at the top and remains in place while scrolling. Help, Account, and Cart are all in the upper right. These are conventional uses of menu and button placement and meet user expectations. (Severity = 0)
There is generally good internal consistency regarding the categories of products. The product details are consistent across the mobile app and the website. (Severity = 0)
Items from past orders are included in a “Buy It Again” page on the website, but this is titled a “Your Items” page in the mobile app. (Severity = 2)
After clicking on an item for detail, it is clear what the next steps could be: change quantity/zoom image/add to cart/mark as favorite/cancel. (Severity = 0)
#5 Error Prevention
Good error messages are important, but the best designs carefully prevent problems from occurring in the first place. Either eliminate error-prone conditions, or check for them and present users with a confirmation option before they commit to the action.
There are two types of errors: slips and mistakes. Slips are unconscious errors caused by inattention. Mistakes are conscious errors based on a mismatch between the user’s mental model and the design. Prioritize error prevention first, then provide constraints, good defaults, warnings, and opportunities to undo.
71% Success
Item quantity is very easy to modify, zero out, and re-select in the product selection view. (Severity = 0)
Sale prices are in color and have an icon. They state how much the savings is, so user doesn’t have to do the math. (Severity = 0)
If a user zeroes the quantity out by mistake in the shopping cart, there is no undo button or quick link to find the item again. The user would have to do a new search, look it up by department, or by past order. (Severity = 2)
There is a CTA to confirm the user wants to log-out. (Severity = 0)
If a product is slightly misspelled in a search, website lists the most likely category. (Severity = 0)
Notification settings toggle buttons appear inconsistent between the app and website. They are red when turned on in the mobile app, but green when turned on in desktop view. (Severity = 1)
There is nothing preventing nonsense entries or typos in the Special Instructions box. However, messages like this can be clarified with the in-store shopper via messaging. (Severity = 2)
#6 Recognition Rather Than Recall
Minimize the user's memory load by making elements, actions, and options visible. The user should not have to remember information from one part of the interface to another. Information required to use the design (e.g. field labels or menu items) should be visible or easily retrievable when needed.
Interfaces that promote recognition reduce the amount of cognitive effort required from users. Provide opportunities for users to recognize information to reduce the user’s reliance on short term memory.
43% Success
Allows user to look up entire past orders and add all items or select particular items to add. (Severity = 0)
Smart search populates recently searched for items in order of relevancy. (Severity = 0)
User cannot remove items from the “Buy It Again” section causing unnecessary effort making repeat decisions within a very long list of items. (Severity = 3)
It is not obvious in the list view whether an item is also a favorite. User has to take an extra step to open the product details to mark as a favorite. (Severity = 3)
User has to enter the product details view and scroll a long way (past suggested products) to the bottom in order to enter a note to the in-store shopper. (Severity = 2)
Hint text is located inside the form fields. Hint text is descriptive. (Severity = 0)
Checkout items have images, but they are very small for a desktop; this may make it more difficult for a user to review and check quantities. (Severity = 3)
#7 Flexibility & Efficiency of Use
Shortcuts — hidden from novice users — may speed up the interaction for the expert user such that the design can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Flexible processes can be carried out in different ways, so that people can pick whichever method works for them.
Accelerators: shortcuts and gestures
Personalization: tailored content and functionality
Customization: user control over how the product works
67% Success
Website provides several ways to search for items, and customize items and quantities. (Severity = 0)
Items from user’s past orders are included in a “Buy It Again” page on the website. There are options to repeat an entire order, favorite items, and view specific items from past orders. (Severity = 0)
Users can create shared carts to shop for/with friends and family. (Severity = 0)
If a user returns to the “Buy It Again” page after conducting a product search, the “Buy It Again” page is loaded again from the top, requiring the user to potentially scroll through numerous items to locate their previous place. This creates more work for the user instead of what could be an accelerator. (Severity = 3)
Search results provide a item text and an image. (Severity = 0)
On the “Buy It Again” page, the only filter that seems to function is the “Most Relevant” filter. The re-order items page can be filtered by “Least Recently Bought” (personally, I would be curious to see what usability testing reveals about this filter) “Most Recently Bought”, “Most Frequently Bought” and “Most Relevant”. (Severity =3)
#8 Aesthetic & Minimalist Design
Interfaces should not contain information which is irrelevant or rarely needed. Every extra unit of information in an interface competes with the relevant units of information and diminishes their relative visibility.
Keep the content and design focused on the essentials. Visual elements should support the user’s primary goals.
67% Success
Good use of white space for size of screen. (Severity = 0)
Essential information and images are clear and specific to the product. (Severity = 0)
The website’s cart view is cluttered and difficult to read even on a desktop. The list of suggested items which comes after is fairly extensive and could be considered obtrusive at this point in the user journey. (Severity = 2)
#9 Recognize, Diagnose & Recover From Errors
Error messages should be expressed in plain language (no error codes), precisely indicate the problem, and constructively suggest a solution.
These error messages should also be presented with visual treatments (like bold, red text) that will help users notice and recognize them. Tell users what went wrong in language they can understand. Offer a solution like a shortcut that can solve the error immediately.
80% Success
There is a CTA button to confirm and prevent accidental log-out. (Severity = 0)
If a search is typed in with a simple typing error, the closest match is searched for. Text is provided in language the user can understand to let the user know what they typed in, and an option to begin another search is provided. (Severity = 0)
If a numerical set of numbers is entered in the “special directions for my shopper” section, the system does not initially notify the user that what they entered does not make sense. However, this text is again displayed in the cart view next to the item, for the user to review. (Severity = 2)
If a search is unrecognizable, a custom error message appears with a clear explanation and instructions. Note this text color is grey. (Severity = 1)
If an incorrect item is entered, a custom error message appears with a clear explanation and instructions. Note this text color is red. (Severity = 0)
#10 Help & Documentation
It’s best if the system doesn’t need any additional explanation. However, it may be necessary to provide documentation to help users understand how to complete their tasks.
Help content and information should be easy to search and focused on the user's task. Keep it concise, and list concrete steps that need to be carried out. The help documentation should be easy to search. Help information should be presented in context and with concrete steps to be carried out.
56% Success
Help center populates in a new tab. (Severity = 0)
Get Help CTA on the Help Center page does route the user to a menu to select type of problem. (Severity = 0)
There is a dedicated Senior Support Service phone number for customer service since seniors may prefer this option to resolve order issues. (Severity = 0)
On the website after selecting “Report A Problem” for a recent order, user is prompted to rate their experience. There is a small blue text CTA for reporting an issue with the order. The CTA button for concluding the rating and exiting is given visual prominence. User could easily miss the option for reporting a problem. Note that there are other ways to re-enter this task. (Severity = 2)
Reporting an issue with an order functions well. User can request refunds for damaged, expired or missing items after an order has been fulfilled, resulting in credit toward the next order. This process functions well and choices are clearly laid out, including an “other” category for reason for request. (Severity = 0)
Website has a “how pricing works” help section that is wordy, but does use clear language. (Severity = 0)
Website displays “Coupon Applied” but it does not state how that works. The difference between a sale item and a coupon item is not obvious to the user. There is no hover tip or nearby help tip. (Severity = 2)
If the user clicks on the Get Help button on the main Help Center page, they will not get options for an interaction with customer service, but instead they get a menu to select the type of problem. This button also reads Get Help. (Severity = 3)
No contact links or customer service phone numbers/hours are listed on the main Help Center page for customer support. This is buried within a certain user flow. User must click this path only. (Severity = 3)
Reflections on This Method
Providing an in-depth look at each of the 10 Usability Heuristics casts a broad net in order to explore the capabilities and potential problems. Adding evaluators would provide increased objectivity. Narrowing the scope or the number of heuristics being studied, as is typically done, would benefit the evaluators, helping to eliminate decision fatigue.
The 5 point Nielsen scale used in this case study has clearly defined points and a more broad scale than the PURE system’s 3 point scale. The PURE (Practical Usability Rating by Experts) rating scale is based on the amount of cognitive load for a particular persona and user path. For the purposes of this case study, the Nielsen scale was both easier and more satisfying to use.
Visually plotting the success rate for each heuristic allows for quickly identifying the general strengths and weaknesses. However, there could be a major usability problem within a generally successful category, so looking at the individual severity rating data is necessary.
Adding usability studies and other research methods would help to point out where the more critical problems are when there is budget to do so.
Credits and Citations
Nielsen, J. (1994a). Enhancing the explanatory power of usability heuristics. Proc.
ACM CHI'94 Conf. (Boston, MA, April 24-28), 152-158.
https://www.nngroup.com/articles/ten-usability-heuristics
Ritter, M.(2018, Oct. 7). Heuristic analysis of yuppiechef.com – a UX case study. UXCollective.
https://uxdesign.cc/ux-case-study-heuristic-analysis-of-yuppiechef-com-c92098052ce4
Sauro, J.(2017, Apr 11). Practical Usability Rating by Experts (PURE). MeasuringU.
Image Credits:
Header Logos: Unsplash.com and Cub.com
Screen Captures: Cub.com
Photos: Unsplash.com
Graphics/Illustrations: undraw.co