Testing Mind Map Series: How to Think Like a CRO Pro (Part 38)

Interview with Daniel Jones

In this eye-opening interview with Daniel Jones, CRO Lead at blubolt, Daniel explains why knowing your company’s strategy and goals inside out is critical to not wasting valuable time on unproductive paths.

Make research your secret weapon, but don’t play favorites. Embrace a holistic approach by incorporating both quantitative and qualitative methods. Avoid the pitfall of relying too heavily on either one, as it can introduce biases into your findings.

And here’s a piece of advice you don’t hear every day: treat your data with skepticism, as if it were inherently flawed. Take the time to scrutinize your tools, cleanse your data, and analyze it meticulously. The goal is to ensure the integrity of the numbers you rely on for decision-making.

Lastly, don’t fall for “quick wins” or “low-hanging fruit.” Daniel urges us to dismiss this hype and instead focus on the long game. That’s where you’ll find success.

Let’s dive in…

Daniel, tell us about yourself. What inspired you to get into testing & optimization?

When I was young, I remember sitting in science class learning about the scientific method of testing. I thought this was fascinating! The idea caught my attention and made me consider how I could achieve a goal through either a sub optimal or optimal method. Fast forward to around 2012/2013, I am knee-deep in UX/UI Design, and a college sends me a link to the Crazy Egg homepage, I am stunned! The content and design were like nothing I’d seen before. It was a beautiful combination of utilitarian design and highly crafted direct-response copyright writing at its best! More akin to the “Internet Marketing” of the decade prior. I later learnt that this was part of an A/B test run by an agency called Conversion Rate Experts. While I was aware of testing, I had never seen anything as extreme. That is when I realized I could utilize a more scientific method of working in my career and it started me on the path to where I am today, as CRO Lead at blubolt, one of the UK’s leading Shopify Plus agencies.

How many years have you been optimizing for?

To date, I have completed 6 ½ years in CRO roles. My advice to anyone starting or who only has a few years of experience, learn from the great optimizers of our generation (the Blank’s, Kohavi’s, Wesseling’s, Sullivan’s, Labay’s). Interact with them on LinkedIn, and go to the events and conferences they speak at. The people who work in CRO are surprisingly supportive and open to dialogue. Perhaps it’s because there is no defined route into the career, and existing optimizers tend to understand the challenges of the work! 

What’s the one resource you recommend to aspiring testers & optimizers?

It has to be the gold standard of CRO credentialism, the CXL Conversion Optimization Mini Degree. At more than 55 hours in length of material, completing it demonstrates total commitment to the discipline of CRO. It sets candidates apart from the crowd in the recruitment process and provides a firm foundation upon which employers can rely.

Answer in 5 words or less: What is the discipline of optimization to you?

Scientific website and business optimization.

What are the top 3 things people MUST understand before they start optimizing?

Great question! When starting optimizing, you must take a step back at the beginning and get clear on the following first.

1. Company strategy and goals. Ensure you are crystal clear on these, as it will stop you from going down the wrong path and wasting time.
2. Conduct quantitative and qualitative research before you start launching experiments! Get a good balance of both when researching the website. Don’t rely too heavily on one and not the other, otherwise, your research will be biased.
3. Get into the habit of asking questions about your data first. Try not to look for data to support your assumptions from the functionality of the front end of the website for example. 

Mastering all three of these will be a challenge, but crack it, and you will be on the right path to success!

How do you treat qualitative & quantitative data?

It might surprise some readers, but I treat all data as if it is wrong! Many businesses have unreliable data from misconfigured tools. Companies regularly make business decisions and spend large sums of money on inaccurate data, which is astonishing!

A common problem in GA3 was that transactions and revenue data could be duplicated by customers revisiting their order confirmation page. GA3 didn’t de-duplicate transactions and revenue, so now you have a serious problem!

I try to question everything until I’ve done some basic checks. Once I have checked the configuration, and ensured that I have access to a good spread of qualitative and quantitative data. Then I follow a cleaning, sorting, filtering, analysis, interpreting and reporting approach. It’s not always linear as I dive in and out of different aspects of work, but that’s a good baseline to follow when handling data like this.

Lastly, I would add that it’s easy to go far and wide with quant and qual data collection, which can produce long reports that bury clients in detail. Yet, this is a red herring as it’s not the length that matters, it is the quality of the insight, the proposed solution, and the speed at which you can move forward, from insight to impact. Ask yourself this question. What is the largest change I can make in the shortest amount of time that will have the biggest possible effect?

This question will keep you on the best path to impacting website performance, helping you to become a more effective optimizer.

What is the most annoying optimization myth you wish would go away?

Despite the plethora of information and data that supports controlled online experimentation, we are still in an era of “quick wins” and “low-hanging fruit” for many SMBs. There are no “silver bullets” in my view. It is process and pressure over time that produces results.

Thanks for tuning in to this exclusive interview with Daniel! We hope you’ve gained some valuable insights from their experiences and advice, and we strongly encourage you to put them into action in your own optimization efforts.

Check back once a month for upcoming interviews! And if you haven’t already, check out our past interviews with CRO pros Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, Max Bradley, Samuel Hess, Riccardo Vandra, Lukas Petrauskas, Gabriela Florea, Sean Clanchy, Ryan Webb, Tracy Laranjo, Lucia van den Brink, LeAnn Reyes, and our latest with Lucrezia Platé.