Matt:
Hi everyone. Really excited to bring you another episode of the Conversion Podcast. This is a really cool topic for a couple of reasons.

First, we have a great guest — Colin, who’s the Vice President of North America at Kameleoon — so I’m very excited for that.
Second, today’s topic is prompt-based experimentation, or as Kameleoon calls it, PBX.

I think this is interesting because it feels like an inflection point in experimentation — where it’s going, what’s possible, the efficiency gains, and what this unlocks long-term.

Colin, I’ll let you do a quick intro so everyone knows who you are.

---

Colin:
Yeah, hi everyone, and thanks for having me on the show, Matt.

I lead growth for Kameleoon here in North America. If that means changing the water cooler or jumping on cool podcasts like this to talk about what’s happening in experimentation, count me in.

---

Matt:
Nice.

And just in case people don’t know — and I actually get this question a lot — how do you pronounce Kameleoon? And what do you guys do? Just a quick overview for anyone who isn’t familiar.

---

Colin:
You got it perfectly. Call us Kameleoon, call us Kameleoon — those two O’s. I always want to get it phonetically written on a shirt.

Simply put, Kameleoon works — and we keep it simple. We do two things:

1. We help all teams build experiments any way they want.

2. We help them release features however they want.

That means we’re a leader in all-team experimentation and in experimentation plus feature management in one platform.

---

Matt:
Nice.

Let’s dive right in. What is prompt-based experimentation, or PBX? How would you describe it?

---

Colin:
The easiest way is to take one step back.

Some listeners may already know about vibe coding. There’s a popular product called Lovable, and now there are probably a thousand other vibe-coding tools.

Essentially, you can prompt AI to build a net-new web product — a website, an app, anything. But validating whether what you built actually benefits customers is more important than ever.

PBX is the coupling of vibe coding — building by chatting with AI — and experimentation.

But we took it a step further. Instead of building something net new, we said:

You already have a website or product. You want to optimize or enhance it. You should be able to build variants, tests, experiments, or prototypes directly on your site — and then immediately launch them into a test.

That’s prompt-based experimentation: chatting with AI to build your test.

---

Matt:
How is that different from tools like Lovable or Cursor?

---

Colin:
Great question.

Those tools are a blank-space way of creating something new. PBX is a net-new way of creating experiments — but with humans still in the loop.

Some AI promises that it will understand your audience, identify funnel friction, and suggest variants for you. That’s AI prompting you.

PBX today is you prompting the AI.

That’s a big strategic difference.

---

Matt:
If we think about the traditional experimentation process — research, hypothesis, design, development, launch, measurement, analysis — where does PBX fit?

---

Colin:
At Kameleoon, we think of four phases:

1. Ideation

2. Build

3. Configuration

4. Analysis / activation

The biggest friction point is the build phase — for companies of all sizes.

PBX is designed to remove that friction. It takes the build phase from weeks or days down to minutes.

We like to say it takes you from zero to one in minutes — or zero to 90%, then engineers handle the last 10%.

But it goes further. Soon, LLMs will also help with ideation, targeting, allocation, and goal setup — all through conversation.

This is happening in months, not years.

---

Matt:
It’s exciting — and maybe a little scary — but also incredible.

I love the “baseball innings” analogy. It feels like we’re just entering the second inning.

I also love your “show, don’t tell” approach, so let’s do a demo.

We picked LEGO as the example. We’re filming this on November 27, right before Black Friday, when a huge portion of LEGO’s revenue happens.

---

Colin:
Before we start, I want everyone listening to know: PBX is free to try.

Download our free Chrome extension, create a free account, and you can prompt any website — just like I’m about to do.

We even created the world’s first Prompt-a-thon, where we want people to see how far they can push this.

---

Matt:
Quick fun fact — Colin used to manufacture cocktail cherries.

---

Colin:
Sadly, cherry margins didn’t survive COVID — but I still have a few batches left.

---

Colin (Demo):
I’ve launched Kameleoon on the LEGO homepage.

Matt noticed something interesting: some CTAs say “Add to Cart,” others say “Back Order.” That could be confusing.

So we’ll prompt the AI:

“Change all CTAs that say ‘Back Order’ to blue instead of orange, and add text explaining why someone should back order.”

---

Matt:
And just like that, the AI changed the button color and added explanatory copy.

What I love is that I can immediately see what I like and don’t like — and then refine it.

---

Colin:
Exactly. You can keep prompting, or you can take the generated code and fine-tune it.

This is human-in-the-loop experimentation. You’re correcting, not creating from scratch.

And it works across desktop, tablet, and mobile without breaking.

---

Matt:
Let’s talk limitations. What should people be aware of?

---

Colin:
Prompting maturity varies. Some people are brand new to chatting with AI.

Garbage in, garbage out still applies.

But some of the best experiments we’ve seen came from very open prompts like, “What should I improve on this page?”

The key is balancing specificity with exploration.

---

Matt:
What does adoption look like?

---

Colin:
The original promise of experimentation was that anyone could use it. That promise wasn’t fulfilled — until now.

With PBX, anyone can try it. The barrier is low.

Once you’re in, treat experiments like a portfolio: small, medium, large — plus ideas generated by AI.

Ask yourself: If I could launch any variant today, what would I do?

---

Matt:
I love the correction vs. creation mindset — and the idea that 80% is often good enough.

---

Colin:
Exactly. We’ve seen companies reduce build time by nearly 90%.

An imperfect variant today beats a perfect variant never built.

---

Matt:
Quick reminder — what’s the Prompt-a-thon?

---

Colin:
It’s an open challenge at promptathon.ai.

We don’t know how far this AI can be pushed on live websites — so we want the world to show us.

We’re offering up to $100,000 in prizes. It runs until mid-January, with finalists announced in February.

---

Matt:
Amazing. Thanks so much, Colin.

---

Colin:
Thanks, Matt. Great to be here.