Conversion Benchmarking Report #1

How elite subscription brands like Netflix, Spotify, and HelloFresh are making UX a competitive advantage

By comparing Spotify, Netflix, HelloFresh, Headspace, and Kiwico, we used our pioneering benchmarking technique to gain valuable critical insights into how these leading brands power their UX.

Access now

A Guide on How To Stop Overwhelming Visitors with Too Much Choice

Egor Driagin

Can you solve a puzzle for me?

Imagine you’re a business selling software that allows busy parents to share photos and videos with their close ones.

When designing your pricing strategy, should you offer your users a single plan? Take it or leave it.

Or should you offer 2 different plans, so that everyone can find something that works for them?

Back in 2010 the right answer would be: you should offer a single plan. When Ash Maurya ran these tests, he found that less people signed up when they were shown 2 plans, and as a result less people became paying customers. One explanation is that people could not decide which plan to choose. So, they left.

This brings us to the long-time problem that marketers have to solve: what number of options should we offer to our customers?

Is large range always better…

…than a small range?

On the one hand, choice plays an important role in our daily lives. More choice can mean customers are more likely to find what matches their taste. In fact, we humans love to make choices. Research shows that the act of choosing activates brain regions associated with motivation and reward.

On the other hand, more choice can lead to indecisiveness, less satisfaction with the made choice and poorer decisions. This phenomenon is known as “choice overload”. Barry Schwartz popularised it in his book, “The Paradox of Choice”.

Since then multiple articles proclaimed choice as a conversion enemy. This even led to some exaggerated statements like the one below:

As always, my job is to separate the wheat from the chaff. As a data-driven marketer you don’t want to follow best practises. You want actual research, so that you can create your own hypotheses and find out what works for your business.

I have done the research for you. Meta-analysis by Benjamin Scheibehenne and his colleagues gave me 50 experiments to work with.

In this guide you will find out:

(If all you want is actionable advice, skip straight to the last 2 sections).

Why is choice harmful for your bottom-line?

Analysis paralysis.

Analysis paralysis is when a person over-thinks the decision to such an extent that he/she does not make it. In marketing context, that means the person won’t buy.

Ryan Engley from Unbounce tested removing the number of options for its webinar sign-up form. After reducing the choice down to 3 options he saw a 16.93% increase in conversions. Perhaps this was because people did not get caught up in thinking what day of the week would work best for them.

The Buyer’s Remorse.

The buyer’s remorse is when a person regrets making a purchase. As the number of choices increases, it is easier to imagine a different choice that may have been better than the one selected. As a result, that kind of post-analysis decreases satisfaction with the purchase.

Iyengar showed this effect in their 2000 series of experiments. They found that when people had to make a choice from a limited number of options (6 vs 30), they expressed greater satisfaction with the made choice. As researchers explain one of the reasons why is because people regretted their decision less. They measured ‘satisfaction’ and ‘regret’ using questionnaires and found that the two were related (r = -.55, p < .0001).

Thinking more long-term, more regret could mean that the customer would not become an advocate of your brand. In the worst case scenario she might cancel or return the product.

Decision fatigue.

Decision fatigue is when we start taking shortcuts as we exhaust our mental energy. Research suggests that humans have a limit on how many active, deliberate decisions they can make in a certain time period. The more we exhaust that reserve, the more likely we are to look for shortcuts.

In one study researchers analyzed more than 1,100 decisions made by court judges. They found that as the day went by (and judges made more decisions), judges were becoming more likely to take shortcuts when deciding which prisoners to release before their official sentence was over.

Result?

Prisoners who appeared at the start of the day received parole about 65% of the time. Those who appeared late in the day received parole less than 10% of the time. Put simply, as judges got tired they used the least-risky option and allowed only a small number of prisoners out.

To translate it to business context, consider onboarding process of eHarmony, an online dating website. To register at eHarmony, you need to answer more than 130 questions about yourself. It’s not as simple as “When you were born?”. Questions range from estimating how warm, clever, dominant you are, etc. down to how often you felt happy, depressed, etc. in the last month.

If you try to have an objective sense of reality, you will likely stress your mental muscles to estimate where you should sit on that scale. After answering 130+ such questions you’ll be exhausted.

By the time you get to the final stage, you might be so tired, there will be no willpower left to figure out why you can’t see anybody’s profiles. The path of least resistance is to abandon the process.

At the same time you might feel so invested into the process, decision fatigue might be in eHarmony’s favour. After all, you just made an active, effortful and uncoerced commitment to meeting a partner online. The alternative shortcut is to pay for a membership.

Decision fatigue will cause customers to take shortcuts in your conversion funnel. The shortcuts might be in your favour, but the opposite might also be true.

Why is choice beneficial for your bottom-line?

Perception of better choice can be a competitive advantages for retailers.

In the review by Benjamin Scheibehenn and his colleagues, they write, “retailers in the marketplace who offer more choice seem to have a competitive advantage over those who offer less”. After all, that’s one of the reasons why we shop at Amazon.

Customers associate higher variety with higher quality.

A study done at Stanford shows that brands that offer higher variety are perceived as having higher quality by the customers.

Adding decoy options can increase average order value.

The Economist offered 3 subscription plans to its customers:

The middle option seems redundant as it is the same price as the last option. So, from “choice overload” perspective we should remove it. But what we’ve missed is that the print & web subscription option seems like a real bargain when compared to the print subscription option.

That’s what Dan Ariely found in his experiment with 200 MIT students:

This extra option more than doubled the number of people who chose the more expensive option.

Choice means freedom. Restricting visitors’ freedom might backfire.

MECLABS Institute tried to reduce friction on a client’s check-out page. They eliminated one of the options in the choice set. Instead of giving users an ability to choose among different subscription lengths (1 month, 6 months, annual), they only offered a monthly plan.

Mike Xiao, research manager at MECLABS Institute, explains that this is what the majority of visitors chose anyways. So, for them, elimination of this step should have only made the checkout process easier.

However, instead of seeing the conversions go up, they saw a 40% decrease.

Mike’s explanation is similar to the Economist experiment. People preferred the monthly plan when it was presented in relation to more expensive 6-month and annual plans.

I want to offer another potential explanation to this result. Visitors might have left the funnel because they did not like that a billing method was forced onto them. Choosing how you pay is an important decision to make, especially when 30% of consumers believe subscriptions are bad value for money.

As I explained in my article on the foot-in-the-door technique, when we feel that someone tries to force us to do something, we often respond in such a way that will re-assert that freedom. In that case, leave the checkout. This process is called psychological reactance and giving customers choice is an effective way to counteract it.

10 factors that drive/mitigate ‘choice overload’

Research by Benjamin Scheibehenn and his colleagues gives us a good idea of what factors make “choice overload” less (or more) likely to happen. We can use that knowledge to design better experiences for users while reaping the benefits of offering more choice.

All 10 factors are based on sound theories, and some of them have already been validated by research. You should remember though that you will only find out if it works by testing it on your own audience.

Factor #1. Users’ preferences towards (and familiarity with) the items in the choice set

If people have strong preference for a particular product, choice won’t overwhelm them. They will simply choose their favourite option.

In 2003 Alexander Chernev, a professor of marketing at the Kellogg School of Management, showed that people with clear prior preferences had no problem choosing from larger assortments. The larger the number of options they could choose from, the more likely they were to actually choose something and to be more satisfied with their choice (study 1 and 2). This is the complete opposite of the research we covered above.

Another study showed that only those people who were unfamiliar with the product category were less likely to be satisfied with their choice.

For example, if I visit SkateHut and know nothing about longboards, I will most certainly be overwhelmed. I don’t have a preference for any brand or type of board. This makes it difficult for me to make a decision.

The solution would be to find out if a large proportion of visitors are actually novices (like me). You could do this by using surveys (see example below), examining live chat data and talking to your sales team.

If the data shows that yes, there is a significant proportion of customers who are novices, the solution would be to educate them. If users become more familiar with different types of boards and brands, it will be easier for them to navigate through a vast array of options.

For example, the following SkateHut page ranks first for a keyword “longboards” in Google:

Instead of sending novices straight to the catalog, SkateHut could offer them educational material first.

In Russia I saw ecommerce websites educating visitors with the use of quizzes. Once you are on a product catalog page, there is a quiz at the top that asks you a set of questions. Based on your answers you will be shown products that match your criteria.

The bot asks me, “Who are you choosing a self-balancing scooter for?”.

I need to tell if it’s for an adult or a child. As I answer these questions, it explains how I should choose my wheels, why some factors are more important than others, etc. As a result of my answers, it shortlists the whole catalog down to 9 options and I even understand why it shows me those 9 options.

At the end I just need to choose the colour I like:

Factor #2. Presence of an obviously dominant option in the choice set

If there is a clearly dominant option, then choice is unlikely to be complicated. Some offers are so inferior relative to one outstanding offer, people would not need to give it much thought.

At a glance, AQF wrist wraps seem way more superior than other offers in the choice set. They have over 700 reviews, most positive and price-wise it’s a real bargain.

The term “dominant” is open to interpretation though. In my view, it means that a user can quickly identify which option matches his use case and it is clear that the other options are inferior. In the example above, all the other options are inferior because they lose on both the number of reviews and price.

Perhaps a better way to think about dominance is through the lenses of elimination. Can a person easily eliminate all the options apart from the one that meets her needs?

In retail review ratings and bestseller tags can act as such filters. We will cover them later.

In SaaS creating plans that match different customer segments can help. Ruben Gamez, the founder of BidSketch, does a great job in simplifying his plans. Users don’t have to make substantial trade-offs when deciding between them.

Solopreneur? Then, Solo is for you. Several members using the software? Then, Team or Business. He makes his plans almost mutually exclusive. At its core plans only vary on the number of users.

Important note: The 2 factors above are necessary pre-conditions for “choice overload” to occur. Put simply, if you can ensure that visitors coming to your website have already developed strong preferences for a certain product and/or the options are presented in such a way that each customer segment will identify one of them as dominant, you won’t have problems with choice overload. In practise, this is difficult to ensure. That’s why we cover 8 other factors that help us beat “choice overload”.

Factor #3. Categorization and Option Arrangement

One study found that an increase in the number of options decreased satisfaction only if the options were not prearranged into categories. Categories make it easier to navigate the choice set and decrease the cognitive burden of making a choice.

HSBC gives its customers only the names of the different credit cards. Unless a person is closely familiar with HSBC credit card offering, those are a poor guide on which credit card is best for you.

FCA carried out research on different motivations of credit card holders. They found that generally there are 4 types of customers:

When either one of these people comes to HSBC website, it won’t be clear what card will help them to achieve their goal.

In contrast to HSBC, Barclays groups its credit cards into categories that fit the motivations above.

This both lowers the number of options that people have to choose from and ensures that people choose only from those options that are relevant to them.

Similarly, HostingAdvice.com categorises its web hosting providers based on use cases.

Not only that, ordered assortments can also ease up the burden of making a decision. For example, WhoIsHostingThis.com presents its web hosting providers in a ranked order. Ranking makes it easier for the user to make a choice.

Factor #4. Difficult Trade-offs

The more difficult the trade-offs that we need to make between different options, the more likely it is that “choice overload” will occur. Research suggests that the trade-offs are particularly tough when options have unique features that are not directly comparable.

To get our heads around it, let’s consider an example.

What credit card would you choose?

Everything is the same apart from a number of differences:

To decide you will need to make some trade-offs. Is a generous rewards scheme more important than balance transfer and purchase rate offers? Is £195 annual fee justified by 80,000 (potential) reward points?

Let’s assume that I am looking for rewards, so purchase rate and balance transfer offers are not important. Unfortunately, HSBC provides no guidance for me to figure out if rewards justify the annual fee.

In contrast, Barclays tells me the exact reward value that I will get based on my monthly spending level:

This lets me to compare the price of the card with the value that I am likely to get out of it. No burden of making mental calculations in my head. Thank you Barclays!

Factor #5. Ease of comparison

To get to the stage of making those trade-offs I needed to compare 2 cards. If the banks make it hard for me to do this, I might drop out of the funnel even before I get to “trading” one feature for another.

HSBC is a bank that certainly does not make it easy.

Continuing with the scenario where I was looking for rewards, I found 2 cards that have them:

Those cards have some common and some unique features. Even for the common features I can’t easily compare them. One gives specific information on how many rewards you will get, the other one does not. One gives specific information about its annual fee, the other one does not. One has “enhanced” travel benefits, but it’s unclear how “enhanced” benefits are better than standard benefits.

If this was the time of the day when my mental energy had been already exhausted, I would have started looking for shortcuts. I could go to MoneySupermarket.com despite originally starting with my bank. Adiós HSBC!

Compare this to what Royal Bank of Scotland is doing. They compare their credit cards in a clear table format:

Not only that, they provide guidance on which card would suit which customer best with their “Decide if it’s right for you” section.

Factor #6. Information Overload

Information overload theory points out that it is not necessarily the number of available products that causes ‘choice overload’, but the number of factors that you need to consider when making a choice in your head.

For example, in SaaS the industry’s standard is to to have between 3 and 4 packages in your pricing grid. It seems like you could have that number, highlight one of the plans as “most popular” and be ok. But your conclusion will be different if you look at the issue through the lenses of information overload theory. What becomes clear is that it’s not the number of plans that matters, but the amount of information that a person has to consider.

Compare the two pricing pages below.

While both offer 4 plans, BidSketch purposefully presents its users with only 1 factor to consider, number of users. In contrast, Surveygizmo used to give its users 15 factors. According to information overload theory, choosing a plan at Surveygizmo will be more overwhelming than choosing a plan at BidSketch.

So, what can we learn from that?

When I interviewed Ruben Gamez, founder of BidSketch, he explained to me that he purposefully omitted many of their features, to simplify the plans. He identified the ‘value’ features, i.e. the features that people considered to be most important. He did this using both data from analytics and Jobs-to-be-Done interviews. Not only that he also did ‘sensitivity testing’, i.e. removing a feature to see if it actually affects conversions.

Factor #7. Time Pressure

Graeme Haynes, psychologist at University of Western Ontario, found evidence for choice overload only if he constrained the decision makers’ time to make a decision.

Thinking of how it applies to the real world, time pressure can be due to seasonal changes (eg. Christmas) or due to time constraints you put on the customer yourself (eg. limiting an offer by a certain number of days).

In these circumstances, you are in an even higher need to simplify the purchase decision for the customer.

As an example, let’s assume I’m looking for perfume for my girlfriend. It’s 15th of December. As someone who hates queues, I search “female fragrance” and come to theperfumeshop.com:

Looking at bestsellers is a not-so-bad shortcut, but obviously ‘bestseller’ does not mean my girlfriend will like the smell. Product descriptions don’t help either.

So, after grilling their customer support for half an hour, here’s how we made progress on what perfume to buy:

Now imagine analyzing hundreds or thousands of such dialogues. We could reverse engineer them into a feature that would help novice customers like myself make similar progress:

In this industry visitors are likely to feel ‘choice overload’ because the products are difficult (if not impossible) to compare. After all, you need to smell them. Time pressure during Christmas escalates that problem.

Instead, the feature above could simplify the choice down to answering a couple of questions. And even if the answers are not perfect, the visitor would still feel like he is making progress. At the end he might end up going to the local The Perfume Shop store or buying a gift certificate. All thanks to not being overwhelmed with choice and closing the website.

Factor #8. Maximizing behavior

Maximisers are those who search for the best option, not just the one that is “good enough”. It is assumed that maximisers would consider more factors, overcomplicate their decision-making and thus be at a higher risk of choice overload.

I believe that maximising behaviors can be identified. For example, imagine a user of Match.com, an online dating website. Instead of contacting a set of matches who have been presented first to him by default, he continues to scroll down, loading all the profiles, analyzing who he should contact first.

We could identify users with such behavior and guide them to take action. Otherwise, they might get caught up in analysis paralysis.

For example, we could show a pop-up box to everyone who loads more than 5 extra pages of profiles. The message will prompt them to start a conversation rather than to keep scrolling through.

(By the way, that research is somewhat true. Read this paper.)

Factor #9. Choice Justification

Scheibehenne found an effect of “too many options” when people knew that they would have to justify their choice later on. Perhaps this effect will most likely occur when a purchase involves several people. For example, you work in an industry where one person will be the user of the product, but above them is a person who approves the budget.

Or it’s a purchase that will be likely discussed with peers (eg. new clothes or a new car).

That means that your role as a marketer is to help customers justify their choices.

3D printing is an industry where multiple stakeholders are often involved in the purchase decision. iMakr, a 3D printing store, not only categorises its printers by the use case…

…it also compares 3D printers in each category, making it easier to justify the choice for a particular 3D printer. For example, B9 Core 530 seems to be the ultimate 3D printer in the jewellery category:

iMakr also gives guidance on who a particular 3D printer is right for, again helping to justify the choice:

It all goes back to helping the user find the ‘dominant’ option.

Factor #10. Simple Decision Heuristics

Many studies have shown that users cope with excessive choice by using mental shortcuts that often guarantee “good enough” decisions.

Some heuristics are useful to know if you’re a marketer:

The elimination-by-aspects strategy

Retailers give customers an ability to screen out some options from the others by providing such cues as review ratings and bestseller tags:

Search filters play an equally important role where users can screen out all the options that don’t match their criteria:

The choice of a default option

Charities use default options on their donation forms to both simplify the decision-making and increase revenue. A 2014 study found that donations closely corresponded to the default amount that was set.

In the same spirit SaaS companies make certain options default. Instead of explicitly asking users how often would they prefer to pay (choose monthly or yearly), they default users to one or the other:

Key Takeaways:

(Think about advice below as ideas to test)

Other articles you might like

Join 5,000 other people who get our newsletter updates