A surprise insight leads to big lift for Sport Chek

Client Sport Chek

Industry Consumer Services

How Canada’s largest sporting goods retailer is getting more value out of its value proposition

Opportunity

Sport Chek is Canada’s largest national retailer of sporting goods, footwear and apparel. Their mission is to inspire every Canadian toward a fit, healthy and active lifestyle. We partnered with Sport Chek just over a year ago and have been working together to optimize their e-commerce experiences, with the goal of increasing conversions in the form of transactions.

A Bit of Context

While Sport Chek’s conversion optimization program is multifaceted, two different tests recently revealed impactful insights about one of the company’s Value Propositions.

Value proposition can be thought of as a cost versus benefits equation that shows your prospects’ motivation. But it’s all about perception: if your perceived benefits outweigh the perceived costs, your prospects will be motivated to act.

Motivation = Perceived Benefits - Perceived Costs

All value propositions have varying degrees of valuableness depending on how they’re interpreted (by the prospect) and how they’re communicated (by the marketer). Your benefits hold different weight for different people―it’s all about finding out which of your benefits are perceived to be most important to your prospects.

Users are unique and tiny changes can have massive effects when your value proposition sits on the fine line between useless and valuable. Often, the same value proposition can have dramatically different effects when framed in slightly different ways.

– Michael St Laurent, Director of Experimentation Strategy, Conversion

The following results demonstrate just how powerful value proposition discoveries can be.

Solution

The Cart Page Test

The Necessity of ‘Free Shipping’

Sport Chek offers free shipping on online orders over a certain dollar amount. Of course, offering some degree of free shipping is basically par for the course in today’s e-commerce world. It’s a Point of Parity―these are the features that are important to your prospects that you also share with your competitors (the basic entry requirements to the game).

The question in this case was: How can Sport Chek communicate this offer in a way that provides more value to their customers? How can they make this Point of Parity look like a Point of Difference―a feature that’s important to the prospect and unique to your business.

Related: For more on Points of Parity, Points of Difference and Points of Irrelevance, check out Chris Goward’s post: Use these 3 points to create an awesome value proposition.

At the time of this test, Sport Chek offered free shipping on all online orders over $25 to their shoppers.

The Hypothesis

On their original cart page, Sport Chek was following a common practice of most e-commerce sites: they were displaying an offer of “Free shipping on all orders of $25 or more”. With an average order value of over $100, it’s safe to assume that the majority of Sport Chek shoppers qualify for this offer.

The Control page featured the phrase “Free shipping on orders of $25 or more.”

It’s a general design rule to “design for probability, rather than possibility”―instead of trying to offer a blanket statement that applies to all customers, focus on optimizing for the most probable occurrences. What if Sport Chek took the same approach to its value proposition?

The hypothesis: Increasing visual prominence of ‘Free Shipping’ and emphasizing the fact that most users have already qualified for ‘Free Shipping’ will result in higher user motivation and more conversions.

Rather than simply stating the fact of ‘Free shipping’ (designed for all possibilities), the goal was to improve Relevance for the majority of Sport Chek users by telling them “Congratulations, you qualify for free shipping!” (optimizing for probability). For the small group that didn’t qualify, Sport Chek took the same approach and did the math for them: “Just $17.45 more to qualify for free shipping”.

As usability expert, Steve Krug, would say, “Don’t make your users think!” By shifting the user’s perspective, Sport Chek turned a statement that required the user to self-calculate into a reason for celebration for those who qualified. It is all a matter of perspective.

Note: This test included two variations. Variation A focused on reducing visual distraction as well as improving clarity. Variation B, described below, was built on Variation A and focused on emphasizing the “Free Shipping” value proposition. The numbers in this case study take into account both variations.

With the winning variation, we celebrated the fact of ‘Free Shipping’!

The results of this test confirmed the hypothesis: Sport Chek users were more motivated to complete their transactions. This winning variation resulted in a 7.3% lift in transactions with 99.5% confidence.

For any company that’s testing, a lift of 7% is nothing to scoff at. But for a company like Sport Chek that receives thousands of online transactions weekly, this result was a big winner.

Result

The Product Page Test (Plus, a Bonus Insight!)

Given the success of the cart page test, both the Sport Chek team and our Strategy team wanted to test emphasizing “Free Shipping” on Sport Chek’s product detail pages. On the original product page, the point about free shipping wasn’t displayed anywhere on the page itself―in fact, it only appeared in the site-wide banner that many users close right away.

Free Shipping’ information was hidden on the Control Product Page.

The findings from the cart test suggested that Sport Chek users may not have been aware that they could receive free shipping—they hoped to further leverage this insight to push users through the funnel.

This experiment had several variations. With variation A, the focus was on diminishing Distraction on the page. Certain elements were removed, others were de-emphasized, and more emphasis was placed on the “Add to cart” call-to-action.

Variation B was the value proposition isolation. Built on variation A, it featured the free shipping information, displayed prominently: “FREE SHIPPING on orders over $50”.

Variation B prominently displayed Sport Chek’s ‘Free Shipping’ offer.

Given the response from the cart experiment, there were high expectations for this variation, but results were somewhat disappointing: a 2.49% lift with only 55% confidence.

When Director of Experimentation Strategy, Michael St Laurent, looked deeper into the data, however, he noticed something odd. Variation B had consistently shown a 6% lift compared to the Control throughout the majority of the experiment. But at the end of the experiment, the steady lift that had accumulated seemed to have rapidly evaporated.

It’s common to see results fluctuate up and down for regular events like sales and weekends, but in these cases all the variations typically respond in unison. After collecting a sample size as large as we had with Sport Chek, it’s very rare to see a graph start to either converge or diverge so dramatically so late in an experiment. This usually means that some factor has started to affect one variation more than another, which is worth investigating.

– Michael St Laurent

In other words, a variable must have been introduced. This converging of variation B and the Control started on July 29; Sport Chek’s optimization team and our Strategists knew that this was the place to start digging.

So what happened on July 29?

As suspected, on July 29 a variable had indeed been introduced: the “Free Shipping” threshold had changed from “orders over $50” to “orders over $75”.

After reexamining the data with this key piece of information, the teams found that variation B was actually responsible for a 6.56% lift in conversions (at 96% confidence) when free shipping was offered on orders over $50.

The graph shows consistent lift for the variation (red) vs. the Control for a full month.

Perhaps more importantly, when free shipping was offered on orders over $75, variation B caused a decrease in transactions of nearly -15% (at 94% confidence) versus the Control.

When the threshold changed to $75, the value proposition of free shipping suddenly became much less valuable, even detrimental.

– Michael St Laurent

This was eye-opening for both the Sport Chek team and our Strategy team : these results highlight the wide degree of elasticity with seemingly similar offers. Now, Sport Chek is armed with a key piece of evidence and can make a more educated decision when dialing in on an optimal “Free Shipping” threshold for their customers.

Takeaways for Sport Chek:

At $50 (and $25), free shipping is valuable for a larger segment of customers and Sport Chek may want to make the offer as visible to shoppers as possible.
At $75, Sport Chek may want to avoid making this top of mind for their shoppers, and may be better off keeping this information out of view until absolutely necessary.

The Results Analysis

In Sport Chek’s case, “Free Shipping” is an extremely elastic value proposition point. At varying “you-qualify-for-free-shipping” price points, there are major swings in user behavior. In the past, the “Free Shipping” offer was an under-utilized value proposition because it wasn’t being emphasized in the right way. Now, this value proposition point is more visible and being communicated with more clarity.

The findings of both experiments comprise a key insight that can help to inform larger business decisions for Sport Chek. There are many variables to take into account before anyone can say “Free shipping on orders over $50” is the best value proposition point for Sport Chek’s business, but this series of tests is a step down the right path to finding out.

Would you like a happy success story like this at your company?

Conversion carefully selects companies that are a fit for our Infinity Optimization Program. Whether you’ve got a team already, aren’t getting the results you’d like, want an outside perspective, or just need to get started, the process can take you to a higher level of expertise.