Spotlight : Conversion x Best Egg

In this spotlight, we explore the partnership between Conversion and Best Egg, focusing on the experimentation program. We’ll cover how we built the program, the challenges we’ve overcame, and the successes we’ve achieved. Plus, we’ll look at what’s next as we plan for the future. Whether you’re into experimentation or want to improve your own program, this spotlight has insights you won’t want to miss.

Joining us is Meghan Metzner, a seasoned Product Management Diector at Best Egg, a financial technology company dedicated to helping people feel more confident about their finances. Best Egg offers a range of solutions designed to provide straightforward, reliable financial support. With over $21 billion in personal loans funded, Best Egg has earned a reputation for empowering customers to tackle their financial challenges with ease. Meghan’s leadership and the company’s innovative approach have been instrumental in shaping their successful experimentation program.

How Experimentation Inspired My Career

[0.00 – 1.20]

James (Conversion): Thanks for joining me this morning, Meghan. It’s great to see you. I’m excited to chat about everything, experimentation and your experience at Best Egg. That may be the perfect place for us to start. Tell me a little bit about yourself and your current role. 

Meghan (Best Egg): Thanks for having me. I’m excited to be here and speak with you. My career started in public relations and traditional marketing, so I did that for about 2 to 3 years. As I spent time in that role, I started to be more interested in the digital space. It seems like that’s where the world was going and what I wanted to explore. 

I started to shift my focus more into digital channels such as email marketing, paid search, and some paid social advertising, and that’s when I started to stumble into the website optimization space. That’s when I felt like I found my home, and that ultimately led me into product management. 

That’s my role today at Best Egg. I’m a product manager, and I love understanding and learning about customer problems and how we can help solve them to ultimately support the business. So, it’s been all about building new features and optimizing. These experiences have brought me so much fulfillment so far in my career. 

James (Conversion): Awesome. It keeps things interesting. Never a dull day.

Meghan (Best Egg): Sure does.

How Experimentation Supports Best Egg’s Digital Strategy

[1.21 – 2.56]

James (Conversion): Tell me about Best Egg and the digital experience there. What are you working on and trying to optimize in that space?

Meghan (Best Egg): Best Egg is a financial technology company, you might hear the term fintech. The goal in what we do ladders up to we’re always trying to provide flexible solutions to help people with limited savings, feel confident in navigating their everyday financial lives.

We’re supporting customers through a growing suite of products. We have personal loan products, we have flexible rent product and all of these different financial health tools. It’s all in an effort to make people feel more money confident.

In terms of our overarching goals and the things that keep us up at night, creating experiences that really differentiate us from others by offering things like payment flexibility for folks. Products that we feel are truly guiding their progress and building experiences that make it easy for the customer.

We always want to make it easy. We want to guide them. We’re focused right now in terms of when we think about the business; we want to increase our personal loan value. We’re hyper-focused on scaling these new products and then building a cross-sell strategy for our customers now that we have more products to offer them.

James (Conversion): There’s a lot of different places to play there as well, in all different areas to optimize and experiences to improve.

Meghan (Best Egg): It’s opened up our testing program with you all, which has been fun.

How does the Experimentation program fit into the Digital Strategy

[2.56 – 4.09]

James (Conversion): Let’s let’s talk a little bit about the experimentation program and how it fits into this digital strategy that you guys are working towards.

Meghan (Best Egg): I would say a majority of our work is focused with your team on what we call the unauthenticated website experience. That’s what I’m the product manager of. Tthat essentially means with [Conversion], we’re creating testing strategies and optimizations for folks who visit our website before they’ve ever even logged into an account.

We’re also starting to dip our toe into the water on how to optimize experiences for actual logins. Things like creating an online account, username and password reset. If we’re talking about KPIs for tests, the thing we’re hyper-focused on with [Conversion] is application starts and login rates.

James (Conversion): It’s kind of like the new customer and the existing customer.

Meghan (Best Egg): Exactly.

James (Conversion): There are two very different cohorts to work on as well.

Meghan (Best Egg): Two very different ways to attack those cohorts when we’ve been working with you guys and how we think about them.

How Conversion Helped De-Risk Best Egg’s Rebrand

[4.10 – 7.15]

James (Conversion): [Best Egg] have been working with us for a number of years now. We’ve had a lot of fantastic work together. What have we accomplished, and what have we been focused on over the last couple of years in the experimentation program?

Meghan (Best Egg): The first thing that comes to mind is de-risking our rebrand. About two years ago, we went through a total, of more than just the website, company rebrand. As you would expect, touched a lot of different things and [Conversion] were so helpful in instrumenting this idea of ‘do no harm’ testing.

It gave us super valuable insight into understanding what to expect once we push this major shift and look and feel [of the brand] live on Best Egg. Thankfully we didn’t have to, but it gave us the opportunity by gathering these insights if there was going to be major harm done to make some adjustments and optimizations before we just pushed the whole thing live, which was huge.

Now that that’s behind us, we’ve gotten a chance to dig into more of that optimization and performative testing. I would say we’ve moved into this world of thinking about the two things I mentioned earlier, all about increasing loan value and then scaling our new products.

Our bread and butter product is our personal loan product. When we began our relationship with [Conversion], all the testing strategy and optimization was focused solely on, and still is a huge part driving, those personal loan application leads. That’s what we’ve thought about day in and day out.

Now it’s been an interesting shift where, if the primary metric isn’t necessarily this personal loan application lead starts in that top of the funnel when those tests aren’t the primary KPI, there’s still always this looking at ensuring that those tests aren’t harming that metric. Just to illustrate, it’s still a very important metric, and that is our bread and butter.

It’s been an interesting shift, but along the lines of those personal lead application starts. We have, in our work [Conversion] had many tests that have improved that metric by up to 4-5%. What’s so interesting is that things like simply changing the copy of a CTA will give you that lift, which is cool to see.

In terms of scaling these products, we’re making a lot of plans to test and optimize these experiences for things like vehicle equity loans, which are still a relatively new product. I think for us, in terms of our work together with Conversion, our biggest area of opportunity is with any new product we launch just focusing on things like content comprehension.

Some of these products can be complex for the average user, [to understand] what they are, and how they will help them. It’s things like copy testing and content hierarchy testing are great places to start to optimizing those experiences for users.

James (Conversion): I remember there have been some pretty surprising small lifts, but big impact tests in the history there.

Meghan (Best Egg): Absolutely.

James (Conversion): Which is cool to see. The flip side of that, of course, is the big lift and the big change things. Speaking about the redesign, something that I remember quite clearly and admire is the culture at Best Egg. The number of people that are coming to our calls, collaborating, the receptiveness to trying new things and following the data.

Building An Experimentation Culture At Best Egg

[7.16 – 9.31]

James (Conversion): Can you tell me a little bit more about what that culture is like [at Best Egg] and what it takes to establish that culture?

Meghan (Best Egg): We’re a team that embraces that test and learning culture and working with [Conversion] just reinforces that within our culture internally, which is fantastic. It also helps our laser focus and obsession with serving the customer in following that data, has also really contributed to the culture that we’ve created here.

The biggest thing in terms of our work together is, when we have a losing task, we don’t view it as a failure. We view it as a meaningful insight. That just helps guide further optimization and iteration into our product and the testing roadmap on Best Egg.com.

It’s interesting to watch, as we craft our roadmap with you all from our testing agenda; we have that framework in place. Once you’ve read out the results of a test, and maybe it wasn’t considered a winner, but you glean a very interesting insight from that. You just continue iterating and kind of shifting around the roadmap as you identify these opportunities from these, “loosing tests”.

That has been embraced throughout the whole company. Not just from the product organization or our work together. You hear it from every single department, which is fantastic.

James (Conversion): That’s awesome. I’ve got to admit, some of my favorite experiments through my career have come from, originally losing tests because the insights are usually pretty interesting.

It’s also pretty admirable that you guys have managed to build that culture through the organisation. It’s one thing for the organization team to love and admire losing tests. It’s another for the design teams, the product teams, and the executive teams.

Using Mixed Methods To Solve Best Egg’s Friction Points

[9.32 – 13.55]

James (Conversion): We’ve done some cool things recently on combining methodologies, what we like to talk about, mixed methodologies [at Conversion] in that blend of the commonly quantitative work of A/B testing and behavioral data that we use. Then the qualitative arm of attitudinal data and digging a little bit deeper into the why behind behaviors, we’re observing in line through, user interviews or those sorts of things.

We’ve done some cool work, and I think your teams have done some great work integrating those two methods across the organization and extracting so you have more insight there. Can tell me a little about what that’s been like over the last couple of years?

Meghan (Best Egg): In our work with Conversion, that’s been one of the biggest unlocks that you all have unveiled to us. You have shown us we’re pretty good at gathering that quantitative data, but there was a huge opportunity for us to start leveraging our own internal insights team to gather more of that qualitative data.

That was one of the most valuable pieces of insights from an annual business review that we conducted together, really rethinking how we think about our testing lifecycle. We started digging into thism bringing insights in more upfront into the process, we were able to take the understanding of the problem and work with design to come up with these multiple different variations that we hypothesized would solve that problem.

What was happening previously is, we were looking at data, but we were looking at web analytics data and seeing the friction points. Then our teams were kind of just deducing from there, well, maybe this is what’s happening or maybe why, this is a friction point.

Now we have that unlock saying, hey, here’s the data, here’s the friction points, let’s go out and ask consumers and customers why this is a friction point. We can craft design variations now that we feel like we have a better shot at addressing the problem or the opportunity.

What we’ve done is historically it would be, hey, here’s one, maybe two design variations that we passed to [Conversion] that we’re still just deducing would solve the problem. We’ve shifted the way we think about that. Now it’s okay, we have an understanding of the problem. We’re working closely with our design team and all other areas of our pod, our insights team, to focus on those problems and come up with 4 to 5 different design solutions. Then we go back to our internal insights team and say, ‘Hey’ now that we understand the problem from you all, let’s get these design solutions in front of folks and get their preference for which one they think solves this issue or that they’re more inclined to resonate with them.

Once we do that, we’ve whittled down the A/B test variations that we ultimately pass to [Conversion]. By doing this, we’ve been able to give our test variations a better chance of success because, again, it just all goes back to the why. Now we finally have those two arms working more hand in hand, and it has turned into this continuous feedback loop cycle with the team. I can’t stress enough how valuable that is that we’ve started doing that.

James (Conversion): Hearing directly from customers and also having that direct feedback. Any surprises that you can think of when you sort of put forward designs and thought, I’m placing my money on this one, and the user feedback comes back, or the test comes back, and they’re going, No, it wasn’t that one.

Meghan (Best Egg): Yes, there’s always surprises like that. I think things such as color choices, as silly as that sounds, there are things where we’ll put this radical design variation in front of customers, and I’m thinking, there’s no way that that’s going to come out as something we pass over to Conversion. Sure enough, it does.

It just reinforces the thought of, when we get together in our work together, we have meaningful insights to bring to the table and have great ideas, but we aren’t our customers. That just continues to be reinforced as we work like that.

James (Conversion): The humility. You’ve got to still put the ideas out there.

Meghan (Best Egg): Let them tell us exactly.

Navigating Challenges In Best Eggs Digital Experience

[13.55 – 16.27]

James (Conversion): Let’s talk a bit about some of the biggest aha moments, the challenges, the successes of the program. Let’s start with the hard stuff. What have been the challenges of doing this work, building this program, and collaborating with these teams? It’s not always easy.

Meghan (Best Egg): We’ve had some interesting challenges. I would say the first one, I just finished talking about the two arms and how wonderful it is that they’re working together. That is all true, but a challenge can be when qualitative and quantitative data don’t show the same results.

You’re left scratching your head for a minute. I think what that shows us is we are learning that what people say they will do or maybe even think they will do isn’t what they’ll always actually do. It just reinforces the importance, though, of having both of those arms working together. So you’re getting the full picture and more reliable results.

That’s been the first big challenge. The second challenge that’s been a really fun one to tackle is that as Best Egg has evolved, we’ve had to shift how we think about our primary KPIs. At the start of our journey together, we were essentially a single product company and it made discussing our main metric easy. We were looking to impact personal loan application starts. That was it.

Now that we’ve shifted into this multi-product world, the conversations are getting so much more interesting about, identifying what that right primary KPI is, and it’s more challenging. Are we optimizing for personal loan leads or is it for a different product lead? Is it logins? Are we okay if there is an impact on personal loan leads if we see this fantastic result for logins?

It’s become a much more dynamic conversation, which is really interesting but still challenging nonetheless.

James (Conversion): It’s a balancing act, right? As you said at the outset when we were speaking, you’ve got three different audiences all going through the same web experience. Frequently, they’re all experiencing the same touchpoints as they start their journeys. So balancing and creating that basket of metrics, and where is your tolerance for improvement or decreases in some metrics to improve the others.

Building A Successful Experimentation Program

[16.26 – 17.33]

Meghan (Best Egg): The tolerance [for experimenting] has evolved. When we started our work together, there was less tolerance for harming that top-of-funnel personal loan application starts.

We have much more tolerance now if we slightly harm that top-of-funnel personal loan metric. But we see another primary KPI for a specific test, be successful. We’ll look down to the bottom of the funnel now and say, okay, it didn’t impact the funded rate. We’re moving forward with this test,we just didn’t have those conversations, two years ago.

James (Conversion):  It helps that you’ve got representatives from the Analytics Team and the Data Team in the room reviewing and planning these tests; almost getting ahead of what could happen is a big component. When you plan a test, this could go south or impact these metrics. Let’s think about and plan for that in advance, and how we’re going to answer those questions. That’s half the battle.

Unlocking Success By Understanding The User: Strategies For Deep Customer Insights

[17.34 -18.52]

James (Conversion): What about device types and user groups? Is anything weird there that’s happened?

Meghan (Best Egg): Yes, we’ve run quite a few tests now where we’re seeing directionally different results on desktop versus mobile. So we’re still trying to peel back the onion to understand why. It’s an interesting challenge. Going back to that iterative nature of testing and just getting all of these insights and learning some of that will start making its way onto our testing roadmap. What should these experiences look like for a mobile user versus a desktop? It’s clear the behavior is very different on those device types.

James (Conversion): When the industry can finally solve the question of are people just browsing on their phones, or are they shopping on their phones or do they return to another device?

Meghan (Best Egg): That all goes back to, starting to really think about those three cohorts more. Are there relationships, between somebody that’s a brand new visitor? Are they mostly on a mobile device or are they mostly on a desktop device? Because if they’re still in research mode, that may change and evolve the way we design things for that cohort. So it’s super interesting to start digging into that.

Best Egg’s Biggest Experimentation Wins

[18.53 – 23.00]

James (Conversion): How about the successes with that? With hard work, sometimes comes rewards.

Meghan (Best Egg): Just as many successes. So that’s always fun. The biggest one that comes to mind to me is starting my work in website optimization, I was very eager to dig into what are these technically complex overhaul experiences that we test and see what happens.

What I’m finding the longer I’m in this game is sometimes, and a lot of the time, the most simple, low-effort tests just give you the most bang for your buck. So, one that comes to mind that we’ve done is just a simple copy change on A CTA, giving us a 4% lift in application starts. Who doesn’t love that you’re seeing this positive impact? It’s not technically complex. You’re not waiting forever to implement it natively in production. Those have been fun successes that we’ve had.

Another example that I can think of that is, again, working with our insights team, learning that folks, more so this new visitor cohort, they just don’t know a lot about us yet. They’re not sure whether they should trust us. So, in our work with [Conversion], we landed on a testing variation where we simply just added what we call trust badges pretty prominently across a lot of our main pages on our website with the Better Business Bureau and the Trustpilot Badging, not super complex to do that gave us a 3% lift in application starts. Those have been some some big wins with [Conversion].

James (Conversion): I was going to say I remember actually with the trust badging that same ABR [annual business review] you were speaking about earlier. I remember us going through a levers analysis and looking at through the Levers Framework, all the different areas we had tested in where we’d maybe under-explored others.

Trust being a low one and the whole team kind of looked at that and gone there’s probably something there. I think trust would be impactful. Low and behold, not too long later, there’s the Easter egg.

Meghan (Best Egg): We continue to test into that. To your point, because that was a lower explored area, we’ve done things now, adding actual components across the website. We just wrapped up this test where we had a variation that was just our customer testimonials and then another one that was actual Trustpilot testimonials.

That was a winner that’s in our backlog to implement natively. It is taking a look and it’s so great to have the partnership with [Conversion] to expose areas within that whole methodology of, “Hey”, we haven’t explored that. We should tap into that and see what happens.

James (Conversion): I love the small wins as well, you mentioned, right? The small copy and CTA stuff. Hindsight’s always 20/20, but it’s fun to look back on those and think, “How did I not see that before?” It seems so obvious in hindsight.

There’s so many of those little tweaks that you can make as you go through the user experience. They’ve been a certain way for so long, but I feel like the whole team overlooks them. Right, you just take, one final or secondary glance and go, wait a minute. Maybe this copy is holding people back a bit, there’s some anxiety, and there can be some big unlocks. Then, as you mentioned earlier, follow-up tests and new insights that lead to more wins. That’s what keeps this interesting.

Meghan (Best Egg): The other thing I’m starting to learn and see is that, let’s use the CTA as an example. That was a winner, you know that person, right? It is a 4% lift in application starts.

That doesn’t mean just set it and forget it that way for three years. You know, give it its time, give it its space. Take a look at it again because there are always macro-environmental changes happening where maybe a specific piece of copy or whatever it may be that you’re testing just isn’t quite as relevant anymore. So capitalizing on what’s happening in the here and now from that macro space is super important.

Unlocking Success By Understanding The User: Strategies For Deep Customer Insights

[23.01 – 26.15]

James (Conversion): There’s the challenge of being in the product, and the experience you guys have there where, you work in the products day in and day out. It’s intuitive to you and the team, right? There are things that you might overlook with how your customers understand the suite of products, the differences, and the choices they need to make. How has testing helped there?

Meghan (Best Egg): I think that this idea has definitely been around for a while. Again, I can’t stress enough the importance of [Conversion] illuminating how we should leverage our insights team more.

The first concrete example of how we put that into practice was, we’re going to take a look at our home page, the most highly trafficked page on the website. Let’s see how we can understand it. Are there detractors for why people aren’t starting an application? Let’s just learn more about what’s happening there.

There’s been this reasonable hypothesis, that if we removed a certain element from our home page, that’s the unlock. We’re going to open up our top of the funnel. If we were to remove that element, it would be super technically complex. It would have a ton of downstream effects on the way the rest of the business works.

That’s not to say we shouldn’t explore these things. We absolutely should. Let’s be disruptive, especially if we think there’s opportunity there. What’s so important with working with insights is that the data debunks was the primary detractor. It is a detractor, but it’s more of a secondary one.

First, people just want to know more about the products. They just want to hit the site and understand quickly and concisely what we offer and the details behind each offering. That now becomes a different conversation in terms of complexity and level of effort. So we’ve spent a lot of time and a bulk of the roadmap with [Conversion] lately exploring those two areas of opportunity, going back to making sure that the qualitative data and the quantitative data are aligned.

We did test with [Conversion] from an A/B test perspective, removing that component, and we found that it did harm the overall application start. There is an opportunity in different ways of engagement that leads us to want to continue exploring that further. It’s helped validate, in a sense, in terms of everybody’s limited time priorities.

We’re going to start hitting hard on these product descriptions and understanding what the unlock is there. So that’s all happening in real-time with [Conversion], which is exciting to see. That’s a concrete example, of how these insights and your team and just getting these understandings of why upfront are so important.

Website Personalisation Is Crucial For Success

[26.16 – 28.05]

James (Conversion): Lots that we’ve already accomplished and I can hear already, plenty that’s on the near term roadmap. What about long term roadmap? Where does this take us and where does it take Best Egg? What’s does the future look like?

Meghan (Best Egg): I think in a nutshell, mainly talking about the unauthenticated experience at Best Egg, is starting to dig into how we can evolve from kind of using the website as more of an application start mechanism is how I kind of think about it to shifting into creating distinct experiences for those three primary cohorts that I mentioned earlier.

How are we creating a distinct experience for that first-time visitor who knows nothing about us? How are we creating an experience for our existing customers? Then how can we support returning visitors that haven’t yet converted, and how do we ultimately get them to take action?

So in the spirit of all of that kind of vision for the experiences we’re looking at, how we can better leverage data and those specific audiences within our tests with [Conversion] to get more focused learnings for each cohort. Continuing to optimize the website to help serve these limited saving customers, but meeting them exactly where they are in that journey.

James (Conversion): It’s journey optimization and sort of a small step into the world of personalization.

Meghan (Best Egg): Definitely, I would say long term vision, we’ll really start digging into personalization for sure.

James (Conversion): Meghan, t’s been great chatting with you. I appreciate the time and talking about the great work that you and the team are doing and can’t wait to see what’s next.