Testing Mind Map Series: How to Think Like a CRO Pro (Part 28)

Interview with Samuel Hess

How does one become an expert CRO practitioner? Ask Samuel Hess and he’ll tell you it’s all about resilience, passion, and a willingness to learn. This is Samuel in a nutshell – an entrepreneur who has faced many challenges throughout his career but never stopped learning or pushing forward.

He went from sleeping in an office for months on end to running 1500 tests for 45 online shops. Samuel is a fervent advocate for the power of data-driven decision-making, and his “less ego more data” motto has helped him become one of the fastest-rising stars in the field.

You’ll find some great advice and an inspiring story in our interview with Samuel below.

Samuel, tell us about yourself. What inspired you to get into testing & optimization?

Originally my co-founder Fabian Gmeindl and I had our own online shops. We started off with dropshipping all sorts of different products – from wooden bracelets to flower pots to jeans.

One thing that we continually ran across was that we had to always optimize our ads, the shop, and so on to run ads profitably.

We learned that iterating fast can be a way to increase performance. We started our dropshipping projects while we were still in high school. Then in 2019, we had a pretty hard fail with one of our own brands but did not want to go to university at all.

So we had to figure out a way to continue making money in order to be able to pay for food ourselves. We had a deal with our parents back then that if we are able to fund our entrepreneurial journey by ourselves we would not need to go to college.

Because we learned so many things about optimizing, we thought, why not do the thing we really enjoy doing and are somehow good at, CRO.

So we came across Johannes Kliesch from SNOCKS and he asked us if we wanted to help SNOCKS to scale their online shop. We said yes, and since Johannes knew that at that time we had no money, he offered us a place to sleep in the office until we could afford to move.

So we spent 3 months sleeping in the SNOCKS office and started optimizing the online shop. That’s how we ended up in the CRO industry.

For how many years have you been optimizing?

Since 2019. Within the past 3 years, we have conducted ~1500 A/B tests for 45 different online shops. But we’re still learning new things every day.

What’s the one resource you recommend to aspiring testers & optimizers?

I definitely recommend the CXL mini degree on CRO to up-and-coming professionals to get a basic understanding of the customer-centric way of working and what options are out there. We started with CXL and did pretty much every course about CRO on the platform.

If you are a little advanced and need guidance on how to structure and build up a CRO course I highly recommend Ruben de Boer from Conversion Ideas. He fundamentally influenced the way I see my work and what CRO and experimentation truly mean at an organizational level.

Answer in 5 words or less: What is the discipline of optimization to you?

Less ego more data. Got that tattooed on my forearm – which means to build products based on evidence and data instead of egoistic and egocentric decisions.

What are the top 3 things people MUST understand before they start optimizing?

Your ABSOLUTE conversion rate is a metric that does not really show how good your CRO program performance is.

Buy-in from the C-Level is one of the most important things. Skills can easily be learned and purchased, but a culture that is non-supportive of experiments is really hard to change.

Experimentation is a marathon and a way of working – not a performance marketing channel.

I like to call it an operating system for companies: “ExperimentationOS”. It is the backbone and not a shiny quick win object. If the backbone is good, amazing things can be built and invented on top of it. But only seeing testing as a quick way to make extra money will not help you to gain a competitive advantage.

Failing and learning what your customers really expect you as an organization to deliver is probably even more important than the ROI of single tests. The ROI of failing and learning can not even be measured because it will help you to come up with new product ideas, etc.

How do you treat qualitative & quantitative data to eliminate bias?

For me, qualitative and quantitative data are there to inspire and evaluate what could influence a user’s behavior.

Qualitative and quantitative data should never replace testing. At DRIP, we use quantitative and qualitative data to prioritize what to test next. The more research we find that backs up a specific issue or idea, the higher it gets prioritized.

However, if we observe that a specific idea or concept won’t work, meaning that it has multiple negative or inconclusive test results, regardless of how much research supports it, we take that into account for the next tests, learn from it, and test other ideas.

What is the most annoying optimization myth you wish would go away?

That optimization can be measured by the absolute conversion rate. Lower your prices by 50% tomorrow and your CR will double or quadruple but your profit will go down.

I hate that people still do not understand that experimentation is a way to innovate and invent but try to see it as a performance channel.

Experimentation is like R&D – just on steroids.

Samuel’s insights into the world of CRO have been nothing short of refreshing and eye-opening. We hope this interview has given you some valuable insights and advice on how to experiment more effectively.

What advice resonated most with you?

Check back twice a month for upcoming interviews! And if you haven’t already, check out our past interviews with CRO legends Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, and our latest with Max Bradley.