Matt: Hey, James. Thanks for joining us once again. Very excited to walk through another bonus episode here on The Conversion Podcast. Let’s set the scene — in this case, we’re going to walk through an experiment: the background, why we did it, what makes it interesting, and dive in.

James: Thanks, Matt. Happy to be chatting about testing again. If you remember last time, I alluded to the concept of a ‘killer test’ and mentioned an example from a webinar with Kevin Turchin. Let’s start with that one.

James: The idea came from within Whirlpool, specifically KitchenAid, from someone in Kevin’s organization. We had been testing for a while, and the word had started to spread about what experimentation and A/B testing can do. Someone on the product team suggested merchandising their 90-day money-back guarantee on stand mixer product pages.

James: It’s important to note that they already had this offer — it wasn’t new. KitchenAid stand mixers had always been covered by a 90-day money-back guarantee, but it wasn’t merchandised or clearly communicated on product pages or anywhere on the site as a value proposition. The product manager came to Kevin with this “no-brainer” idea: let’s talk about our money-back guarantee.

James: It sounded perfect — it should alleviate customer anxiety when purchasing a stand mixer. But, since he knew about the testing program, he said, “I’d like to test it first to prove how much better it will be for the business.”

James: So we set up a test. The control had no mention of the guarantee. The variation prominently displayed the 90-day guarantee on product detail pages (PDPs) and product listing pages (PLPs).

Matt: And what happened? I’m guessing there’s a surprise here.

James: Yes — a big one. Orders dropped by double digits. This was very surprising to the business, but crucial to discover before rolling the change out.

James: Why did it happen? KitchenAid stand mixers are a category leader. Customers shopping for them — especially on the brand’s own site — rarely doubt the product’s performance or need a guarantee. By surfacing the guarantee, we unintentionally introduced doubt and concern into their minds. Instead of reassuring them, it triggered anxiety.

James: In other contexts, guarantees work well, especially where customers do have concerns. But the key is matching the right message to the right moment in the journey.

James: This became a “killer test” because everyone thought it would win. After seeing the results, people began questioning their own assumptions about other untested ideas. Experimentation activity increased significantly afterward.

Matt: Great example of how context changes everything. We’ve used similar tactics successfully elsewhere, but every situation is different. I have a follow-up — you mentioned PDP and PLP. How did you structure that test?

James: We ran it as a full-funnel experiment targeting all visitors to PLPs and PDPs. This gave us a clean split: one group never saw the guarantee, and one group saw it throughout the shopping journey.

James: In post-analysis, we could segment results by visitor type:

• People who went through the full journey (PLP → PDP)

• People who only saw PLPs

• People who landed directly on a PDP

James: This approach avoids “pollution” where visitors see a message on one page but not another. We recommend integrating analytics platforms with testing tools to unlock deeper post-test insights.

Matt: Very cool. Thanks, James. More to come — we’ll be back next week with another interesting experiment to inspire your own testing.

James: Thanks, Matt.