Testing Mind Map Series: How to Think Like a CRO Pro (Part 48)

Interview with Maria Luiza (Ziółkowska) de Lange

Maria Luiza de Lange is a self-proclaimed CRO Enthusiast, and a winner of Experimentation Culture Awards 2023 with her team at Tele2.

She held nothing back in our interview, generously sharing tons of advice, her best resources for those new to CRO, and three essential things to remember before diving into optimization. Starting with some very practical advice: Master your statistics, embrace the reality that optimization is a team effort, and never underestimate the critical role of developers (they’re the unsung heroes!). If you’re in the optimization space, Luiza is someone you can’t afford to overlook.

Luiza, tell us about yourself. What inspired you to get into testing & optimization?

Hi Everyone! My name is Luiza. I have been living in Sweden for 12 years now and I consider it my home, however, I was born in Poland and am married to a Dutchman, so I am a blend of these 3 cultures. There is a bit of each in me 😉 Personally, I love traveling and discovering new places. I am a bird photographer and an amateur ornithologist, a hobby that I discovered during COVID times. I am also a die-hard fan of Lewis Hamilton and Formula 1 in general.

I started my career at Electrolux Europe (central headquarters), where I spent 9 years. I come from the business side. Originally, I was a part of the in-store team, but as digital became more and more relevant, I started leading omnichannel projects. After spending half a year in the local market, I moved to the “Digital team”, where I was a content manager, and at the same time I got this side responsibility that was called analytics 😉. At that time, all resources were consultants and I gradually built analytics capabilities for Electrolux in-house, where we managed 90 domains, run experimentation and personalization. That journey took more than 3 years. For most of my career, I have been learning from the fantastic specialists around me, both junior and senior. I did not do much operational work. I was the bridge between the business and the tech and was a line manager for the analysts.

When I was a part of the retail team, it was very hard to measure anything in that environment. We tried a couple of eye-tracking studies, but all of them were more qualitative rather than quantitative results. When I changed the area of responsibility to digital, I was able to measure everything, that was an eye-opener. It seemed like the possibilities were endless. Since then, I love just being able to test a hypothesis and look at the data before making a decision, as opposed to having a discussion about people’s different opinions. I really enjoy being able to directly impact consumer experience online.

How many years have you been optimizing for?

It has been 5 years since I started working with experimentation.

What’s the one resource you recommend to aspiring testers & optimizers?

I really enjoyed reading a book called Trustworthy Online Controlled Experiments by Ronny Kohavi & Co. I wish I got to it earlier, so I recommend it with all my heart!

Though it’s hard to just pick one resource. You need to pick a type that matches your learning style. I am a visual and auditory learner and therefore I love conferences. Just listening to other people’s experiences and hopefully learning from their mistakes. There are many conferences organized by vendors. Some of my favourite annual conferences are Conversion Jam, Conversion Hotel, Measurecamp, Women in Tech Sweden, and Superweek.

There are also a lot of CRO influencers that post fantastic content on LinkedIn, so I recommend following a couple to stay up with the latest.

Finally, there are fantastic communities like Women in Experimentation, CRO Nordic Slack channel and the TLC community that connect testers and optimizers around the globe.

Answer in 5 words or less: What is the discipline of optimization to you?

Being able to reduce risk

What are the top 3 things people MUST understand before they start optimizing?

1. Statistical basics. There’s quite a lot of analysis done before the experiment is launched (MDE and experiment length calculation), and all the metrics set up. During the test phase, there’s continuous monitoring and evaluation for statistical significance. Post-experiment, you can delve deep into the analysis and evaluate results against many metrics.

2. You will never be able to do it all on your own. It’s a company-wide effort. Many smaller companies rely solely on their centralized Growth Teams or growth managers to do experimentation, but such programs are hard to scale. The biggest gains can be unlocked when the whole organization goes through a transformation into a more data-driven company and each function contributes to optimisation.

3. You need developers. As with the point above, there’s only as much one can do without development resources. I have not heard about so many success stories of companies solely running small copy tests unless they are a search engine company. There might be exceptions, but for a standard e-commerce to make a difference, relying on collaboration with development resources and feature flagging significantly increases the odds of success.

How do you treat qualitative & quantitative data to minimize bias?

Different types of data play different functions in the CRO process.

The best results usually come when you are able to combine different data sources supporting the same insight, the so-called trinity approach. It is when you are able to combine the quantitative outcomes from Google Analytics or Adobe Analytics with behavioural data from Contentsquare or Hotjar and then support that with a more qualitative data point from research like feedback from a survey (Qualtrics or GetFeedback) or usability test on your website.

What is the most annoying optimization myth you wish would go away?

That you can do experimentation without developer resources. Many organizations expect results without securing the necessary developer resources. If you expect results, you need to be aware of what are the necessary functions for CRO to work like UX designers, product owners, developers, and analysts.

Our thanks go out to Luiza for taking part in this interview! To our lovely readers, we hope you found the insights useful and encourage you to apply them in your own optimization efforts.

Don’t forget to check back twice a month for more enlightening interviews! And if you haven’t already, check out our past interviews with CRO pros Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, Max Bradley, Samuel Hess, Riccardo Vandra, Lukas Petrauskas, Gabriela Florea, Sean Clanchy, Ryan Webb, Tracy Laranjo, Lucia van den Brink, LeAnn Reyes, Lucrezia Platé, Daniel Jones, May Chin, Kyle Hearnshaw, Gerda Vogt-Thomas, Melanie Kyrklund, Sahil Patel, Lucas Vos, David Sanchez del Real, Oliver Kenyon, and our latest with David Stepien.