Spotlight: AB InBev's Alexandra Resnick on Transitioning from an Experimentation role to Product Manager
As the largest diamond retailer in the world, Signet Jewelers is built on a culture of innovation that has seen their sales grow to the tune of over $5.2B across 10 global brands in 2021.
Since investing in Personalization & Experimentation, Signet has achieved significant lifts across each of its business metrics. By increasing both conversion rate and revenue per visit by over 50%, these strategies have contributed to a four-fold increase in eCommerce sales since 2017.
We sat down with Craig Kistler, Signet’s Director of Personalization & Experimentation, to unpack the important–but often overlooked–connection between these two practices, and why one needs to fuel the other.
Your role at Signet is Director of Personalization and Experimentation – a seemingly intentional distinction. What’s the relationship between the two?
They’re two sides of the same coin, but a lot of people talk about personalization and forget that it is absolutely linked with experimentation. I see it happen all the time: people have data to suggest a group of customers typically respond in some way when presented with a specific experience, and so they decide to offer more of that same experience to this customer segment. But unless there’s an experiment tied to it, there’s no way to really know for sure. That’s why at Signet, we experiment with everybody first.
That touches on an area you’ve credited much of the success of your personalization strategy: on leading with experimentation insights. Why did you let experimentation guide your personalization strategy?
At Signet, we had a good idea from our existing data of the primary audiences we wanted to pursue as part of our personalization strategy. However, there was still a strong belief coming from other teams that the audiences may not be right or may not work as expected, which in turn would hurt the overall business. On the other hand, we also had teams coming to us with suggestions of different audience ideas and how we should use them to create a personalized experience on the site. In both cases, we started with a simple experiment to prove or disprove the hypothesis about an audience.
What does that process look like in practice?
The process we follow starts off with an idea or suggestion. For example: “Will visitors in the market for an engagement ring react differently to a message about speaking with a jewelry expert?” From there, my team will look to understand the question being asked and build an experiment around it. When we’re just getting started with an audience, we want to see how the experiment works for everyone (50/50 split of all traffic). We then look to see if the audience performs at a rate that is higher, lower, or the same as the default experience, and we continue to track how each audience performs over time against a number of different KPIs to learn if and how the audiences change.
We use this deep understanding of each audience to show the value of personalization when talking with the variety of teams we support. If we were to jump into personalization first, we wouldn’t understand if the change we made into the experience was valuable because of the personalization or if everyone would have benefitted from the change and we narrowed down to one segment too quickly.
That raises an important question: How do you define your audiences?
We start at a very macro level first and look at what is the broadest way we can break down a group of people into buckets that are differentiated from one another. In my case, I started by breaking down how visitors were shopping for jewelry—engagement rings vs everything else. From there I broke down the ‘everything else’ bucket into fashion and watch shoppers. Each of these three audiences had a unique set of qualities that allowed us to speak to them differently, and were large enough to make an impact. Other examples could be men/women, knitting/crochet/painting/other crafts, home audio/car audio, the list goes on forever. It’s very product specific.
In the past you’ve also spoken about consumer intent. How does that play a role in your audience segmentation?
Consumer intent is the other key lens we consider when defining audiences, grouping visitors into either low, medium, or high intent to buy. Somebody just coming to our site might be trying to decide between buying some jewelry, a new iPhone, or a vacation – their intent to buy is quite low. But then as they continue working through our site, we’re looking for any signals that might warrant moving them into that ‘medium’ bucket so that we can start to treat them a little differently. And then once they’ve shown high intent and are screaming that they’re close to purchase, it’s deciding how we can personalize their experience to encourage them across the finish line.
What about the purchase funnel? Is there a specific part of the funnel where you think personalization is most powerful?
When done correctly, personalization can be very powerful wherever it’s used in the funnel. The placement really depends on what type of audience you’re building. Our macro audiences (broadly focused audiences) are intentionally built to speak directly to different parts of the funnel. We also have audiences that speak to a very specific journey or starting point. Each of these audiences has a very specific goal in what we expect it to do.
Is Signet focused on a specific part of the funnel?
From a personalization standpoint, we focus a lot of effort on the upper part of the funnel – everything before the cart. Again, instead of sending one message to everybody, it’s looking at how we can tailor that message from everybody down to many, and then down to a few – just to make sure the process is easier. A lot of the success we’ve seen in both experimentation and personalization is when we go back to core UX principles. Personalization can make it a lot easier for people to find and make the right purchase. When we get friction points out of the way through personalization is where we’ve seen the biggest impacts.
Do you see a place for personalization in the purchase funnel?
Yes, I believe there is but we’re not quite there yet. There are lots of opportunities around content and messaging and proof points we can be providing to tailor an experience that ensures each customer is confident they’re making the right buying decision. A lot of it goes back to the intent we talked about. For example, how do we make someone with low intent confident in that buying decision? We could look at things like possibly highlighting returns and features like free shipping or the ability to go see it and pick up their purchase in-store. We could talk about the possibilities for hours, but at the end of the day, it is again, something we will only be able to know for sure by experimenting first.
What advice would you give to a brand that’s only just started with personalization?
Crawl, walk, run! Too often when people think about personalization, they associate it with driving 1:1 interactions with each customer–without realizing the incredible level of maturity, sophistication, and time that it takes to get to this point. When starting out, stay away from micro-audiences. These will quickly overwhelm a personalization program and worse, won’t provide enough value. Both issues will sink a personalization strategy.
Instead you need to establish those macro level audiences first, by looking for audiences who have the broadest reach with unique qualities. Doing this will get you bigger wins faster, which will snowball into momentum and learnings you can use to slowly get closer to more granular levels. We’ve been doing this for over 5 years at Signet, and I think we’re still a couple of years away from getting to 1:1 ‘holy grail’ state of personalization.
Craig Kistler is an award-winning strategist with over 20 years of experience spanning product design, user experience, optimization, and personalization for brands that include Signet Jewelers, American Greetings, and Progressive Insurance.
Want to learn more about how leaders like Craig are creating growth through experimentation? Check out Conversion’s latest experimentation study: Maximum impact: How digital experimentation leaders are doing more with less.