A surprise insight leads to big lift for Sport Chek

Sport Chek is Canada’s largest national retailer of sporting goods, footwear and apparel. Their mission is to inspire every Canadian toward a fit, healthy and active lifestyle. We partnered with Sport Chek just over a year ago and have been working together to optimize their e­-commerce experiences, with the goal of increasing conversions in the form of transactions.

A Bit of Context

While Sport Chek’s conversion optimization program is multi­faceted, two different tests recently revealed impactful insights about one of the company’s Value Propositions.

Value proposition can be thought of as a cost versus benefits equation that shows your prospects’ motivation. But it’s all about perception: if your perceived benefits outweigh the perceived costs, your prospects will be motivated to act.

All value propositions have varying degrees of valuable­ness depending on how they’re interpreted (by the prospect) and how they’re communicated (by the marketer). Your benefits hold different weight for different people―it’s all about finding out which of your benefits are perceived to be most important to your prospects.

Users are unique and tiny changes can have massive effects when your value proposition sits on the fine line between useless and valuable. Often, the same value proposition can have dramatically different effects when framed in slightly different ways.

– Michael St Laurent, Director of Experimentation Strategy, Conversion

The following results demonstrate just how powerful value proposition discoveries can be.

The Product Page Test (Plus, a Bonus Insight!)

Given the success of the cart page test, both the Sport Chek team and our Strategy team wanted to test emphasizing “Free Shipping” on Sport Chek’s product detail pages. On the original product page, the point about free shipping wasn’t displayed anywhere on the page itself―in fact, it only appeared in the site­-wide banner that many users close right away.

Free Shipping’ information was hidden on the Control Product Page.

The findings from the cart test suggested that Sport Chek users may not have been aware that they could receive free shipping—they hoped to further leverage this insight to push users through the funnel.

This experiment had several variations. With variation A, the focus was on diminishing Distraction on the page. Certain elements were removed, others were de-emphasized, and more emphasis was placed on the “Add to cart” call­-to-­action.

Variation B was the value proposition isolation. Built on variation A, it featured the free shipping information, displayed prominently: “FREE SHIPPING on orders over $50”.

Variation B prominently displayed Sport Chek’s ‘Free Shipping’ offer.

Given the response from the cart experiment, there were high expectations for this variation, but results were somewhat disappointing: a 2.49% lift with only 55% confidence.

When Director of Experimentation Strategy, Michael St Laurent, looked deeper into the data, however, he noticed something odd. Variation B had consistently shown a 6% lift compared to the Control throughout the majority of the experiment. But at the end of the experiment, the steady lift that had accumulated seemed to have rapidly evaporated.

It’s common to see results fluctuate up and down for regular events like sales and weekends, but in these cases all the variations typically respond in unison. After collecting a sample size as large as we had with Sport Chek, it’s very rare to see a graph start to either converge or diverge so dramatically so late in an experiment. This usually means that some factor has started to affect one variation more than another, which is worth investigating.

– Michael St Laurent

In other words, a variable must have been introduced. This converging of variation B and the Control started on July 29; Sport Chek’s optimization team and our Strategists knew that this was the place to start digging.

So what happened on July 29?

As suspected, on July 29 a variable had indeed been introduced: the “Free Shipping” threshold had changed from “orders over $50” ​ to “orders over $75”.​

After reexamining the data with this key piece of information, the teams found that variation B was actually responsible for a 6.56% lift​ in conversions (at 96% confidence) when free shipping was offered on orders over $50.

The graph shows consistent lift for the variation (red) vs. the Control for a full month.

Perhaps more importantly, when free shipping was offered on orders over $75, variation B caused a decrease in transactions of nearly ­-15%​ (at 94% confidence) versus the Control.

When the threshold changed to $75, the value proposition of free shipping suddenly became much less valuable, even detrimental.

– Michael St Laurent

This was eye­-opening for both the Sport Chek team and our Strategy team : these results highlight the wide degree of elasticity with seemingly similar offers. Now, Sport Chek is armed with a key piece of evidence and can make a more educated decision when dialing in on an optimal “Free Shipping” threshold for their customers.

Takeaways for Sport Chek:

• At $50 (and $25), free shipping is valuable for a larger segment of customers and Sport Chek may want to make the offer as visible to shoppers as possible.
• At $75, Sport Chek may want to avoid making this top of mind for their shoppers, and may be better off keeping this information out of view until absolutely necessary.

The Results Analysis

In Sport Chek’s case, “Free Shipping” is an extremely elastic value proposition point. At varying “you­-qualify-­for-­free-­shipping” price points, there are major swings in user behavior. In the past, the “Free Shipping” offer was an under-utilized value proposition because it wasn’t being emphasized in the right way. Now, this value proposition point is more visible and being communicated with more clarity.

The findings of both experiments comprise a key insight that can help to inform larger business decisions for Sport Chek. There are many variables to take into account before anyone can say “Free shipping on orders over $50” is the best value proposition point for Sport Chek’s business, but this series of tests is a step down the right path to finding out.

Would you like a happy success story like this at your company?

Conversion carefully selects companies that are a fit for our Infinity Optimization Program. Whether you’ve got a team already, aren’t getting the results you’d like, want an outside perspective, or just need to get started, the process can take you to a higher level of expertise.