Ep: 4 // Ronny Kohavi, Airbnb // Creating Trust in Experimentation

Ep: 4 // Ronny Kohavi, Airbnb // Creating Trust in Experimentation

Ronny Kohavi is a driving force and leader in experimentation culture for enterprise legends like Amazon, Microsoft, and, most recently, as Airbnb’s VP and Technical Fellow. Among his noteworthy contributions, he’s implemented fundamental changes and achieved wild success for Microsoft by implementing experiments at scale. Ronny has recently published Trustworthy Online Controlled Experiments: A Practical Guide to A/B. And with all that said, would you expect any less from a Stanford graduate who joined their first startup at age 15?

“If you define an intrapreneur as somebody who is within the company that tries to promote some innovation or some change, then certainly, I fit that definition.”

Ronny Kohavi

Running hundreds of experiments a day is not something lots of people have done in their careers. Despite his massive success in the industry, Ronny is also no stranger to experiencing less than stellar results despite having a world-class personalization team. Ronny explains that even the experts end up being humbled by the reality that the users don’t like what we do most of the time.
Of all of his accomplishments, Ronny is most proud of his ability to change experimentation culture. Ronny mentions to Chris, “The main thing that I’m proud of is a cultural change.” Overcoming obstacles like getting executive buy-in, working through organization shifts, changing design and development patterns and building a platform you can trust are topics Chris and Ronny candidly discuss in this episode

Top takeaways include:

  • How to pick a path of least resistance to demonstrate value to management
  • Why selecting stories will help evangelize experimentation across your organization
  • The importance of validating before celebrating

Read transcript