Testing Mind Map Series: How to Think Like a CRO Pro (Part 4)

Interview with Sina Fak of ConversionAdvocates

A CRO strategy without the right mindset is like a house built on quicksand. It may look sturdy from the outside, but it could fall apart at any moment.

If you’ve invested lots of time and money into an experimentation program but you’re still missing that key component to making it work flawlessly, the Testing Mind Map Series is for you.

The series will help you learn what effective testing looks like from interviews with some of today’s most knowledgeable experts on optimization and experimentation strategies!

In this installment of the series, we’re chatting with Sina Fak, Head of Optimization & Business Intelligence at CRO Agency, ConversionAdvocates.

Sina, tell us about yourself. What inspired you to get into testing & optimization?

My journey into conversion optimization started after launching my first business, a creative agency focused on designing promotional videos for businesses. I loved the art of creating something visually beautiful.

But what I loved more was understanding the impact and results of our work.

Several months after launching, I started following up with clients asking questions about how they were using the assets we created and the results they were getting.

Very quickly I realized my customers had little data about the performance of their campaigns.

That’s how I got into learning analytics. And from analytics, I got into optimization. Because once you have data on how things work, you also have data on how they don’t, and then naturally you want to make them better.

How many years have you been optimizing for? What’s the one resource you recommend to aspiring testers & optimizers?

I’ve been doing CRO professionally for about 8 years.

I would say the number one resource to rely on for aspiring optimizers is their own curiosity. The desire to explore, challenge, and dig into not only uncovering the truth, but understanding what the truth is composed of, is a fundamental skill that needs to be honed and harnessed.

As optimizers, our job is to question everything and work to understand the root of why something is happening. This is what drives our desire for analyzing data, doing customer research, and running experiments.

Because the reality we see at surface level is the sum of many moving parts. The more curious we are about how everything works, the better we will understand why it works that way, and the more opportunities we will find to make it work better.

Answer in 5 words or less: What is the discipline of optimization to you?

Analyze. Test. Learn. Iterate. Repeat.

What are the top 3 things people MUST understand before they start optimizing?

1.   How to research

To be a good optimizer, you need to have solid skills in both quantitative and qualitative research. But being a good researcher isn’t just about asking a few survey questions or looking at Google Analytics.

It’s about

a. asking the right questions that focus your research on the problem you’re solving,
b. using the appropriate research methods to explore the problem, and
c. thinking critically to connect the dots between different sets of data that come from your research to understand the problem.

2.   How to experiment

Running an experiment can be defined as simple as “trying something new”. But as optimizers, we need to understand how to run rigorous experiments.

This involves knowing

a. statistics – how to calculate sample sizes and p-values necessary for experiments to reach significance,
b. persuasion – how to leverage copy, design, psychology to focus on a specific learning or hypothesis within the experiment,
c. process – how to prioritize experiments to maintain velocity and compound learnings from experiments.

3.   How to analyze data

The ability to analyze data is a necessary skill for any optimizer as it’s a critical step both before and after an experiment is run.

And while you don’t need to be a data scientist to be a good optimizer, you do need to be able to

a. navigate different quant and qualitative analytical tools what they are, what data they capture, and how they work
b. understand users and behaviours to segment for in your analysis,
c. how to think critically about the data you’re seeing, what it means and where to explore next in your analysis.

How do you treat qualitative & quantitative data so it can tell you an unbiased story?

The reality is that all data has bias built into it.

Everything from how the data was collected, to the sample data that is used for analysis, to the person reviewing the data and running the analysis – there is an element of bias we cannot fully control.

Data alone won’t give you the whole story. It will only give you a starting point to understand part of the story and draw insights.

The only way to treat data in a way that tells an unbiased story is to put it to the test and run an experiment with it. 

What kind of learning program have you set up for your optimization team? And why did you take this specific approach?

Everyone on our team is an expert in their own field. They have specialized knowledge of unique conversion optimization practices, from data science and analytics, to UX design, to persuasive copywriting, to program management.

As an onboarding learning program, we teach everyone the fundamentals of CRO so that we’re all on the same page when it comes to the process and how each persons’ role impacts other parts of the process. The reason we did this is that CRO is a marriage of multiple disciplines and it’s important for everyone to understand how their contribution is impacting the sum of the parts.

As a continuous learning program, we have implemented regular stand-up calls to review and discuss client experiments and results. This gives everyone a chance to, once again, understand how their work is being used by other people in the organization, the impact it’s made on experiment results, and how it fits in the bigger picture of our CRO process.

What is the most annoying optimization myth you wish would go away?

That we only focus on making simple changes to a website like button colour or headline copy to improve conversions.

Nowadays, it’s easy to get caught up in the CRO rabbit hole and lose focus on what really matters. When you have the right testing mindset though, it truly does make all of those tools and team hours worth their weight!

Hopefully, our interview with Sina will help guide your conversion strategy in the right direction! What advice resonated most with you?

Be sure to stay tuned for our next interview with a CRO expert who takes us through even more advanced strategies! And if you haven’t already, check out our interviews with Gursimran Gujral of OptiPhoenix, Haley Carpenter of Speero, and Rishi Rawat of Frictionless Commerce!