Testing Mind Map Series: How to Think Like a CRO Pro (Part 34)

Interview with Tracy Laranjo

A powerhouse in the CRO world, Tracy is SplitBase’s Head of Research and the host of the Experiment Nation podcast. With a long list of successful projects under her belt, Tracy is a firm believer in the power of community and the importance of surrounding yourself with other experts to grow in the field.

When it comes to testing, Tracy preaches patience: conduct conversion research before you test to avoid “shots in the dark”! And take a neutral standpoint when interpreting results—that way, you won’t miss any meaningful insights.

Are you ready for some serious insider tips, with a dash of wit? Good, because that’s what Tracy does best.

Let’s dive in…

Tracy, tell us about yourself. What inspired you to get into testing & optimization?

I got into optimization by failing at paid acquisition. Kidding! Kind of.

At Willful, an online estate planning startup, I was curious why new visitors left the site shortly after landing from PPC and social ads. It became an obsession of mine. My COO had a late-night idea to roll me into a dedicated site and web app optimization role under the product team. I’m so glad she did.

Since then I’ve run 7 ROI-positive optimization programs through research-backed experimentation.

I love hunting for clues, developing theories, and putting them to the test.

Today you can find me as Head of Research at SplitBase, a specialist conversion agency for DTC brands. There I set the vision for the talented Research & Data team. We help our clients understand their shoppers, know where the big opportunities are, run meaningful tests, and turn test results into knowledge.

For how many years have you been optimizing?

Three years as a site and landing page optimizer for awesome DTC businesses like Inkbox Tattoos. Five years total if you count email and PPC testing.

What’s the one resource you recommend to aspiring testers & optimizers?

Community. If you want to become a great optimizer, surround yourself with great optimizers. For me, the result was fast upskilling, a mentor, and friends who help me work through new challenges.

Find a group of smart experimenters and ask them a lot of questions. People are often willing to help you grow.

When I started learning CRO, Rommil Santiago reached out and invited me to join his community of testers, Experiment Nation. He later entrusted me with hosting EN’s podcast, which is the best thing that could have happened to a new tester like me.

I ended up learning more from 13 interviews with optimization wizards than through any other resource. How often do you get to ask masters of your craft the questions that keep you up at night? Even better, I was able to share those learnings with an audience of other aspiring testers.

Starting a conversation with even one other tester may open doors you never expected.

Answer in 5 words or less: What is the discipline of optimization to you?

More than just A/B testing.

What are the top 3 things people MUST understand before they start optimizing?

1. Spend time diagnosing problems before prescribing solutions. Junior optimizers love to go straight to A/B testing without doing background research. They try to solve problems that may not even exist, like a doctor writing prescriptions without diagnoses.

I’ve been there and it got me nowhere until I started taking research seriously. Learn how to conduct effective conversion research before you start testing. You’ll focus on the right opportunities in the right way vs. wasting time on shots in the dark.

2. Tests don’t “win” or “lose”. They’re opportunities to learn something. A neutral, stoic detachment from your test variant makes unexpected knowledge possible. It’s that or being sad when your idea doesn’t perform as expected (get used to it). As long as you’ve picked up clues to inform your larger optimization puzzle, you’re on the right path.

3. Take hypotheses seriously. It sounds basic but I see a lot of tests without hypotheses or supporting data. A properly-formulated hypothesis statement sets the stage for what the report will tell you at the end of a test. If you don’t take the time up-front to define this, you will have no idea what to look for in your test results or why you ran the test in the first place.

How do you treat qualitative & quantitative data to remove bias?

When it comes to collecting survey or interview data, I try to keep questions as open-ended as possible. I want to know, in the respondent’s words, what matters to them when deciding whether to act or not.

We don’t know what we don’t know. If we lead the witness by introducing options informed by what we think matters to them, we’re robbed of the opportunity to learn something unexpected. Plus, the responses are great for keyword targeting and copy testing.

What is the most annoying optimization myth you wish would go away?

If your site looks better it will perform better.

Obviously, you don’t want your online store to look like you built it on Neopets in 2003. But make sure you solve basic issues before deciding if a redesign is what you need. For example, are shoppers clear about why you’re better than the 5 other options waiting in their tabs?

One of my clients had their entire site redesigned last year by a hot creative agency. Sure, the site looks cute. But my client’s revenue tanked overnight because it prioritized aesthetics over user experience. It’s so hard to come back from that without extensive CRO surgery.

Fortunately, I love performing CRO surgery.

Thank you for joining us for this exclusive interview with Tracy. We hope you’ve learned some valuable insights from his experiences and advice, and we encourage you to implement them in your optimization efforts.

Check back twice a month for upcoming interviews! And if you haven’t already, check out our past interviews with CRO legends Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, Max Bradley, Samuel Hess, Riccardo Vandra, Lukas Petrauskas, Gabriela Florea, Sean Clanchy, and our latest with Ryan Webb.