Testing Mind Map Series: How to Think Like a CRO Pro (Part 47)

Interview with David Stepien

David Stepien is the co-founder of Conversion Stash, “the most comprehensive resource directory for CRO practitioners” and he’s also the Site Experience Manager over at pens.com. Throughout his career, he’s meshed the rigorous approach of the scientific method with adventurous exploration, or what he likes to call “playful tinkering.”

A standout among David’s many clever insights is his nuanced perspective on “best practices”. While some see them as the holy grail and others dismiss them outright, David views them as heuristic tools—not definitive answers, but rather guiding principles that offer a starting point. It’s a reminder that best practices may lay the groundwork, but the magic lies in adapting them to fit each unique scenario.

Join us for a delectable chat with David.

David, tell us about yourself. What inspired you to get into testing & optimization?

I began my journey in PPC marketing and soon discovered that the key drivers for enhancing campaign performance were often found on the landing page, rather than within the campaign itself. Noticing that no one was testing these pages, I took the lead and began experimenting with them. The significant results from these initial tests had me hooked, leading me to pivot entirely towards CRO.

How many years have you been optimizing for?

It’s been over 5 years now.

What’s the one resource you recommend to aspiring testers & optimizers?

Probably Conversion Stash 😉  I think it stands out because it acts as one meta-resource, allowing you to leapfrog to hundreds of other valuable tools and information.

Answer in 5 words or less: What is the discipline of optimization to you?

A reproducible success strategy

What are the top 3 things people MUST understand before they start optimizing?

Scientific Method/Statistics:

If I were to start from scratch, I’d prioritize understanding the fundamental principles before delving into the specifics of CRO. By establishing the scientific principles as the trunk of your knowledge tree, you ensure that the leaves (the details) have a sturdy foundation to cling to.

Seeking expert guidance and mentorship is underrated:

An expert is someone who has made all the mistakes there are to make. Soliciting their feedback regularly makes all the difference in your learning curve.

Goodhart’s Law:

When we overly focus on optimization, we risk deteriorating the very thing we aim to improve. Indeed, optimization without the right boundaries can inadvertently lead to negative outcomes, a concern I believe doesn’t receive the attention it deserves.

How do you treat qualitative & quantitative data to minimize bias?

While both qualitative and quantitative data serve as valuable priors for successful experimentation, they aren’t the be-all and end-all. My gripe with research in CRO/experimentation is the tendency to prioritize the sanctity and choice of research methods over the potential value of a solution.

I’ve encountered CRO programs that mandated research data as a prerequisite to be allowed to run an experiment. To me, this approach is misguided. While I understand the intent might be to prevent haphazard “spaghetti-testing,” it feels like an overcorrection.

Certainly, grounding the majority of your experiments in issues identified through research is wise. However, there should also be space for playful tinkering, exploration and probing those intuitive hunches that are hard to articulate. Ultimately, the most effective research strategy is to streamline the process, making experiments as cost-effective and straightforward as possible.

What is the most annoying optimization myth you wish would go away?

The divisive debate on “best practices”. One group views best practices as a low-status method for generating ideas, while the other tends to overly generalize from them. I think this false dichotomy often oversimplifies the realities of the world. Instead of perceiving best practices as the epitome of knowledge or dismissing them entirely, one might consider them as heuristic tools. They are not ultimate answers but guiding principles, offering a starting point, a foundation upon which further inquiry can be built. They are the springboard, not the conclusion.

“You cannot step into the same river twice.” Similarly, in the world of experimentation, no two situations are truly identical. Context is paramount. What works for one website, demographic, or product might not work for another. Herein lies the danger of over-generalizing. They should be approached with a discerning eye, always contextualizing, always probing deeper. I think that by adopting such a dialectical approach, we can harness the best of both worlds: the accumulated wisdom of the past and the adaptive flexibility of the present.

Our thanks go out to David for taking part in this interview! To our lovely readers, we hope you found the insights useful and encourage you to apply them in your own optimization efforts.

Don’t forget to check back twice a month for more enlightening interviews! And if you haven’t already, check out our past interviews with CRO pros Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, Max Bradley, Samuel Hess, Riccardo Vandra, Lukas Petrauskas, Gabriela Florea, Sean Clanchy, Ryan Webb, Tracy Laranjo, Lucia van den Brink, LeAnn Reyes, Lucrezia Platé, Daniel Jones, May Chin, Kyle Hearnshaw, Gerda Vogt-Thomas, Melanie Kyrklund, Sahil Patel, Lucas Vos, David Sanchez del Real, and our latest with Oliver Kenyon.