Creating a culture of experimentation that celebrates failure and drives performance: An evening with Conversion & Kameleoon
We recently kicked off a workshop series with one of our solution partners, Optimizely, starting proceedings off (quite fittingly) with a session on the topic of ideation.
The workshop session was led by Kyle Hearnshaw, Head of Conversion Strategy at Conversion.com, with support from Stephen Pavlovich, CEO and Nils Van Kleef, Solutions Engineer at Optimizely.
We run ideation sessions all the time, whether between ourselves or in collaboration with our clients. But this had a slightly different twist. We gathered a group of representatives, from multiple different businesses, across multiple different verticals, in a room to experiment with how well different approaches to ideation perform. Every attendee left with an understanding of the different approaches, and insight into which approach they felt they could apply most effectively with their own teams.
For those of you that couldn’t make the session, we want to share some of those ideas so that you too can run ideation sessions that generate impactful ideas.
But first, why does the ideation process matter?
Ideation injects creativity into a data-driven process
Data is incredibly important in driving all of the decisions we make in experimentation. Data is great at telling us where the problems are, but it isn’t good at telling us how to solve these problems. This is where creativity plays a key role in experimentation. Ideation is our opportunity to inject creativity into what we do, to explore new concepts and experiment with potential solutions as we hone in on the optimal solution for each problem.
Our time is limited
For many of us, there’s barely enough hours in the day to get through all our emails, let alone spend hours in unproductive brainstorming. We need to optimize how we spend our time in ideation to get a good return on our time investment. The ideation process needs to reliably generate high quality ideas that can immediately be tested through experimentation.
More people doesn’t always mean better ideas
We’ve been taught that collaboration should generate more and better ideas, but this is only true when people can contribute effectively. When more people are in the room, there is often a tendency for people to become protective of their own ideas, as opposed to sharing, discussing and letting their ideas evolve through the input of the rest of the group. Everyone present in an ideation session should be able to, and also expected to contribute effectively.
Subtle differences in execution can have a big impact on results and build time
The difference between an execution of an experiment that wins versus an execution that loses can be extremely small. Two ideas that seem similar can often perform very differently when exposed to real users. One way of testing an idea might require a lot more effort to create than another. Ideation plays a key role in both defining and refining ideas.
Our top 4 tips for effective ideation, no matter your approach
1. Separate the hypothesis and the execution
Ideas that come up in an ideation session come in all shapes and sizes. Some will come to you fully formed with a well-defined hypothesis and a sketch of the execution of the experiment. Others will be less-formed, and need to be refined into a clear hypothesis first before ideating around the best execution of that idea. When an idea is raised, identify whether it’s a hypothesis or a specific execution. For hypotheses, you can then ideate on solutions. For executions you can step back and ideate around the underlying hypothesis of the idea.
2. Break ideas up into sequences of experiments – starting with the minimum viable experiment
One way of doing this is to think about how the idea might fit into a sequence of experiments. Think about the idea you want to explore. Is it the ‘minimum viable experiment’? Or is it a more formed exploration of a specific solution in this area? Think about how you might take your idea and iterate on it across a range of experiments, continuing to improve, and reap results of the idea you have produced to finally reach that ‘perfect’ version.
3. Is this idea iterative, innovative or disruptive?
When you reach that stage of an ideation session where the ideas start to dry up, a useful exercise is to group your ideas into 3 types. Which are iterative ideas – tweaks, optimizations or designs? Which are innovative ideas – new experiences, journeys or usability? And which are groundbreaking, disruptive ideas that will affect product, pricing, or even the company proposition? Try and make sure you have a good number of each category.
4. Have a plan to weed out your bad ideas
Remember back at school, when you were told to put your hand up and express your ideas, whatever they were? There’s no such thing as a bad idea, right? Wrong.
Contrary to common belief, there is such thing as a bad idea. We would encourage you to have a completely open system for idea creation. Allow people to come up with whatever ideas they like. But, also have a system for critique and review. Some ideas will simply not be good, others will be good, but won’t be possible.
There has to be a good mix of realistic, and ambitious ideas. The last thing anybody wants to do is waste their precious time talking about ideas that are simply ‘dead in the water’. One approach we like for this is using The Disney Method.
The 3 approaches to ideation we tested
We’re going to share with you the three approaches that we explored in our ideation workshop.
Quite often when we talk to people and ask how they ideate, the answer is “Well, we get everyone in a room and talk about some ideas”. This is the more common type of ideation session, and by no means the approach that we would recommend. But, it does provide a useful baseline, and if you apply all of our tips above it can, in the right circumstances, still be effective.
That said, it’s pretty much a free-for-all. Everyone shouts their ideas, there is very little focus, it’s unstructured. For this session, we gave attendees a vague goal of increasing conversion rate and a specific page of a website to improve – to mimic the setup of one of these sessions in a conversion optimization context.
+ Anywhere, anytime
+ Anyone can do it
– Anyone can do it (the input expected from each attendee isn’t clear)
– The ideas generated tend to be unrelated and broad
– Small number of high-quality ideas
In most cases, when we talk about ideation we’re talking about the creation of ideas for AB tests and experiments. What many people fail to remember, is that experiments are just the end-product of a larger strategic process. At Conversion, we build our strategy around our experimentation framework.
Most unstructured ideation sessions tend to be around a loosely defined goal and perhaps a KPI to be improved. However, in order to conduct an effective ideation session we need more structure and focus.
Only after you have defined your Goal and KPIs, then used data to understand and define your Audiences, Areas and Levers, should you start to ideate for experiment concepts. Agree the specific audience, area and lever that you’re going to ideate on and make sure everyone knows this, and has seen any relevant data and research before they attend. The session will then be more focused around solving a specific problem, and structured.
For this session we gave attendees a completed experimentation framework and defined the Goal, KPIs, Audience, Area and Lever we wanted them to ideate on.
+ Impactful concepts
– Needs upfront research
– Takes longer to get started
This is one of our favourite and most enjoyable methods of ideation. It forces every member of the group to produce 8 ideas, in 8 minutes. We’ve adapted the concept from one originally adopted by Google.
Our adaptation of the original Crazy 8s system is to apply it to a structured ideation setup. So again we have defined our Goal, KPIs, Audience, Area and Lever. Then we use the Crazy 8s ideation process on that specific lever to generate a large number of ambitious ideas in a short space of time. Rather than one person generating 8 ideas on one lever, we often rotate the paper so that you have 1 minute to add a new idea on a lever that nobody else has come up with yet. In this way you can cover multiple levers in one session if you’re ambitious.
The purpose of using Crazy 8s is to force everyone in the ideation session to contribute, but also to stretch all of the attendees to contribute more ambitious, creative ideas then they might generate without the added time pressure. It also encourages people to draw ideas to save time, which can bring out new ideas as people get more visual.
+ Large number of ideas
+ Includes all the structured ideation benefits
– Needs an introduction
– Less collaborative
Overall, we thought the workshop was a great success. We had some great feedback from those involved and some brilliant ideas for our attendees to take away.
The key takeaway was that structure is crucial for effective ideation. And by that we don’t mean the minute-by-minute structure of the session itself, we mean the structure, focus and setup of what you’re ideating about. A structured approach not only generates more ideas, but generates crucially more impactful, creative and ambitious ideas. As the host of the session you will walk away with both the quantity and quality of ideas you need to design your experiments.
Ideation is critical to experimentation. In order to create an effective experimentation roadmap, you must engage in effective ideation. Following just a few of these techniques, will have you well on your way. But of course, if you’d like to know more, do get in touch.
If you’d like to attend future events, keep an eye on our events page.