Testing Mind Map Series: How to Think Like a CRO Pro (Part 52)

Interview with Lorik Mullaademi

Every couple of weeks, we get up close and personal with some of the brightest minds in the CRO and experimentation community.

We’re on a mission to discover what lies behind their success. Get real answers to your toughest questions. Share hidden gems and unique insights you won’t find in the books. Condense years of real-world experience into actionable tactics and strategies.

This week, we’re chatting with Lorik Mullaademi, CEO & Co-founder of Sogody, a digital experiences company helping organizations establish highly effective engineering solutions, e-commerce and experimentation programmes.

Lorik, tell us about yourself. What inspired you to get into experimentation?

My journey into experimentation was more of an unexpected discovery than a deliberate choice. In 2017, at Sogody, our goal was to deliver exceptional digital experiences at a time when the digital landscape was polarized—either excessively flashy and ineffective or starkly unstructured. Our big break came when DBi, part of Havas Media Group, involved us in conducting A/B tests for leading e-commerce brands like Samsung and Kia, some of which we still work with today. This experience was eye-opening – I saw how minor adjustments in digital customer interactions could significantly boost revenues. This, coupled with our years of experience delivering experimentation programs, led to the creation of our “CRO/Proof” program, aimed at enhancing conversion rates across various industries, including SaaS, by making data-driven improvements to user experiences.

How many years have you been testing for?

I’ve been immersed in experimentation for six years. This period was very revealing that the true nature of experimentation is not just about confirming hypotheses, but more importantly, it’s a journey of learning and tinkering. With the introduction of disruptive technologies like AI, it feels like we’re about to restart our journey of testing, so the excitement continues.

What’s the one resource you recommend to aspiring testers & optimizers?

Rather than pointing to a specific resource, my recommendation for aspiring testers and optimizers is to read about and embrace the KISS (Keep It Simple, Stupid) methodology. This principle advocates for simplicity in design and systems, suggesting that complexity should be minimized to enhance user acceptance and interaction. While often overlooked, KISS is not just applicable in digital design, but also in our everyday lives. It’s a fundamental approach that I diligently follow and consistently encourage within the brands we work with, reinforcing its importance in achieving perfect streamlining.

Answer in 5 words or less: What is the discipline of optimization to you?

Clarity and efficiency through data.

What are the top 3 things people MUST understand before they start optimizing?

Might sound repetitive, but these are important. When working with new brands, I emphasize the need for pre-optimization before even initiating a CRO contract:

Optimize to Understand Your Audience: Gaining deep insights into your audience’s preferences and behaviors is critical in launching relevant tests. This knowledge is fundamental in shaping effective optimization strategies.

Optimize Data Before UX: A structured approach to data is often missing in many brands. Organize and optimize the data to understand user journeys before delving into UX optimization, as this forms the basis of informed decision-making.

Optimize Budgets to Embrace Iterative Process and Long-term View: Optimization is an ongoing process of improvement, not a one-time effort. It’s important for brands to understand the long-term value and ROI of iterative enhancements, rather than being deterred by initial costs and quick results.

Obviously, not everything has to be picture perfect.

How do you treat qualitative & quantitative data to minimize bias?

In our approach to minimizing bias in both qualitative and quantitative data, we employ a mix of methodologies, selecting the best practices for each context.

For quantitative data, statistical analysis is our primary tool, ensuring that our samples are unbiased and representative.

In handling qualitative data, we use structured methods like thematic analysis, which are further enhanced by third-party review processes to reduce personal biases—often involving colleagues not actively working on the accounts. We also involve our industry peers in regular model validation for more serious experimentation programs to maintain reliability, mainly in the case of serving a larger client.

Importantly, we’ve honed our hunch enough to integrate insights from both data types, and previous successful case studies to improve our understanding and our ability to formulate hypotheses that frequently yield positive outcomes.

How (to you) is experimentation different from CRO?

Experimentation is the laboratory where we test our hypotheses—it’s the action. CRO is the philosophy that guides these actions; it’s the strategy behind the myriad of tests we run.

Experimentation can be seen as the individual battles, while CRO is the war we’re strategizing to win, with the end goal being enhanced user experience and conversion rates.

Delving deeper, experimentation leads us to tinker with various methods to achieve our strategic goals. We’re currently leveraging AI to innovate in our approach – experimenting with predictive A/B testing, sequential testing, and even employing generative models to create translations based on language preferences. These experiments allow us to not just react to user behavior but anticipate it, offering a more personalized and effective user experience.

Talk to us about some of the unique experiments you’ve run over the years.

Recently, we undertook a project where we integrated Artificial Intelligence into the sequential testing process of a flagship product launch for one of our esteemed clients in the consumer electronics sector.

As a key part of this initiative, we introduced a sophisticated adaptive search bar powered by AI. This feature brought predictive typing capabilities to the forefront, enabling dynamic refinement of search suggestions in real time. As users entered their queries, the AI-driven system evolved, offering increasingly pertinent recommendations. It stood out not just for its predictive typing feature but also for its capacity to propose customized product bundles based on audience types.

The AI excelled in analyzing user input and preferences on the fly, thereby identifying products that aligned closely with individual user interests. These insights were then leveraged to curate bundle deals. The algorithms took into account user interactions with specific pages and products, as well as the devices used for accessing the platform. This data was instrumental in optimizing the composition of these bundles, particularly the third product in the bundle, ensuring they were highly attractive to targeted audiences. These algorithms were run on real-time data but were trained on a set of previous data during last year’s flagship launch.

Our application of AI in sequential testing enabled us to make quick, data-informed decisions, allowing for agile adaptation of our bundle strategies. The impact was significant, with a remarkable 21.43% increase in conversions in the first week alone, compared to the same period in the previous year. This success not only underscored the vast potential of AI in enhancing sequential testing processes but also highlighted its transformative impact in boosting user engagement and driving substantial revenue growth.

Our thanks go out to Lorik for taking part in this interview! To our lovely readers, we hope you found the insights useful and encourage you to apply them in your own optimization efforts.

Don’t forget to check back twice a month for more enlightening interviews! And if you haven’t already, check out our past interviews with CRO pros Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, Max Bradley, Samuel Hess, Riccardo Vandra, Lukas Petrauskas, Gabriela Florea, Sean Clanchy, Ryan Webb, Tracy Laranjo, Lucia van den Brink, LeAnn Reyes, Lucrezia Platé, Daniel Jones, May Chin, Kyle Hearnshaw, Gerda Vogt-Thomas, Melanie Kyrklund, Sahil Patel, Lucas Vos, David Sanchez del Real, Oliver Kenyon, David Stepien, Maria Luiza de Lange, Callum Dreniw, Shirley Lee, and our latest with Rúben Marinheiro.