5x increase in revenue for Shaw Academy with rapid experimentation program

Client Shaw Academy
Industry Education

Shaw Academy had been successfully using Google and Facebook ads to drive traffic to their website, but their conversion rate from visitor to paying customer was too low. We were hired to run a full-scale experimentation program to drive up their site-wide conversion rate. All in all, we were able to double their conversion rate and help increase their revenue by 400%.


Shaw Academy knew the importance of increasing their conversion rate. It was the main factor that was limiting their growth.

They had the ability to scale rapidly. Selling online education courses, there wasn’t much difference in serving 100 customers or 1 million customers. The challenge was acquiring them – profitably.

Traffic was coming from ads on Google and Facebook. Shaw Academy’s website would then motivate visitors to take a free trial, which led into an ongoing paid subscription.

The problem was that their conversion rate from visitor to trial to paying customer was too low. They could only acquire the most qualified traffic – and that was limiting their growth.

So they hired Conversion to deliver a program of rapid experimentation and increase their conversion rate.

Breaking even with the first experiments

On every Conversion project, experimentation starts on day one. During the kick-off meeting, the consultant presents the initial concepts – they then go into production almost immediately.

For Shaw Academy, these first two experiments delivered a +14% increase in sales (both at over 99% significance).

In one of those experiments, we optimized the messaging on the billing page, addressing the user’s primary objections.

In the control, users were unsure about entering their payment information. In the new variation we added copy that clarified when they’d be billed, and reassured them that they could cancel online at any time.

A simple experiment like this is perfect to kick off a new project. By changing UI rather than functionality, we can deploy the experiment rapidly. And because it’s focused on one of the users’ primary objections, it’s more likely to have a significant impact.

Testing bold ideas – with a safety net

The more experiments we ran, the more confidence Shaw Academy had in our methodology. In fact, at the time of writing, the program has a 38% win rate, with all experiments run to 99% statistical significance.

That high win rate meant Shaw Academy were prepared to try more ambitious experiments. After all, the true value of experimentation is not to test what you would have done anyway – it should allow you to take bigger risks, with a safety net, and reap bigger rewards.

In one experiment, we wanted to remind users when their free trial was about to end. That would mean we could reassure new customers during sign-up that we’d tell them before they got charged – but it also risked increasing cancellations.

Rightly, Shaw Academy were concerned about running this experiment. It could increase cancellations and – because free trials lasted 30 days – we’d have to wait over a month before we’d know about it. What’s more, Shaw Academy have a six-figure monthly spend on acquisition, so they’d risk losing a lot of cash as well as a lot of customers if the experiment failed.

So we defined an experiment to limit exposure, introducing a new challenger that reassured the user: “As a reminder, we’ll email you 3 days before”.

Given the importance of the experiment, we wanted to be confident that any successful result wasn’t a false positive. So we calculated the duration based on 99% statistical significance, and launched the experiment.

The impact on free trials was clear early in the experiment: the new challenger showed a significant increase.

But we needed to wait to see what would happen when this cohort of users received the reminder email. As it happens, it had a negligible impact on cancellations – so the positive increase in trials was carried through to sales as well.

Creating a virtuous circle with experimentation and acquisition

The experiments shown above are just two examples taken from the first six months of the project.

In this period, their conversion rate increased 2x – and their sales increased 5x. That’s because Shaw Academy recognised the importance that their conversion rate had on their growth.

When their conversion rate increased 2x, their fixed costs remained fixed. And that meant their profit increased significantly more than 2x. This made all of their acquisition channels much more effective.

They could increase spend profitably – unpausing channels and campaigns that had previously been unprofitable.

So the increase in the conversion rate allowed them to increase traffic – which in turn accelerated our rate of testing. This creates a virtuous circle: the more we test, the higher the conversion rate, the more traffic they can acquire, the more we test…!

Together, this gave Shaw Academy a 5x increase in revenue by February 2020. And as the industry felt the impact of COVID-19 and consumers turned to online learning, Shaw Academy were perfectly positioned to market to an even bigger audience.


Increase in revenue


Increase in conversion rate

Want to work with us?

We work with everyone from small startups to some of the biggest brands in the world.

Get in touch and see how we can help you.

Contact us