Testing Mind Map Series: How to Think Like a CRO Pro (Part 59)

Interview with Eduardo Marconi Pinheiro Lima

Every couple of weeks, we get up close and personal with some of the brightest minds in the CRO and experimentation community.

We’re on a mission to discover what lies behind their success. Get real answers to your toughest questions. Share hidden gems and unique insights you won’t find in the books. Condense years of real-world experience into actionable tactics and strategies.

This week, we’re chatting with Eduardo Marconi Pinheiro Lima, Digital Optimization Director at AIM, Conversion Experts, a consultancy specialized in increasing digital channel conversion through CRO, product personalization, and behavioral research.

Eduardo, tell us about yourself. What inspired you to get into testing & optimization?

I got into the area unintentionally. I had just moved to São Paulo, Brazil, after living 7 years in England and got an operational manager job in an early stage agency specialized in user behavioral studies. The main problem with their business model was that our clients had a lot of barriers to applying our suggestions to their digital platforms and we decided to look into tools that could speed up the process. That was the clue to search for AB testing tools/test-and-learn methodologies to add the solution execution to our operation. I absolutely loved the whole process.

How many years have you been testing for?

13 years. Since 2011.

What’s the one resource you recommend to aspiring testers & optimizers?

Study user behavior. Data means nothing if you can´t connect it with real-world needs and how users want to use your product.

Answer in 5 words or less: What is the discipline of optimization to you?

Making decisions based on data

What are the top 3 things people MUST understand before they start optimizing?

1. Not everything needs to be AB tested. If the level of uncertainty is low, just do it. That will speed up your innovation process.
2. Respect the methodology. Don’t be tempted to rush into decisions just because you want the hypothesis to be a winner.
3. Have a plan. One of the biggest mistakes in CRO operations is to test all kinds of subjects\ pages at the same time and not do follow-ups (testing from learnings). That won’t bring value to the operation as nobody will understand what you are aiming for, especially your stakeholders.

How do you treat qualitative & quantitative data to minimize bias?

Cross-reference information from different sources. If you find negative feedback from a user in a survey about a specific element, before jumping to a conclusion, look for quantitative data that can tell you how many users actually experienced the same problem.

How (to you) is experimentation different from CRO?

Experimentation for me is the evolution of CRO. It’s about applying the discovery, execution, and analysis cycle in a consistent format.

Experimentation is not about doing one AB test, but about continually testing and learning as an independent production line of product innovation.

Talk to us about some of the unique experiments you’ve run over the years.

In my view we can divide an experimentation program into four different strategies:

1. Interface: Improving the platform experience. It can be a simple change in a specific component to a completely new flow.
   
   An experiment that I found very interesting was to analyze changes in behavior for a one-page checkout vs. a checkout (split in stages). With this type of test, we could learn how much time the users spent in each step (address, delivery, payment, and purchase review) and how a phased process affects the speed of completion.

2. Promotional: Improving sales promotion strategies. What are the best promotional formats (discount vs. cashback) or content (highlight full price vs. installments)?
   
   One example that blew my mind was when we tested a marketplace birthday campaign with two different offerings: “up to R$1.000 discount” vs. “up to 50% discount”. As the marketplace had a wide range of products, from R$10 to R$100.000, we saw a huge rise in sales by presenting the “up to 50% discount” just because users that were entering the site to buy products below R$300 didn’t relate to the “R$1.000 in savings” campaign.

3. Product placement: Improving product value in the shoppers’ minds.
   
   We tested the impact of placing a $50 coffee machine next to a $300 coffee machine on a desktop page. For this specific case, we found out that placing a $300 coffee machine next to a $50 coffee machine impacted drastically the $300 product result. In order to improve its conversion we had to place the products in different product lines organized by price ranges (coffee machines in line 1 from $30-100, in line 2 from $100-250, line 3 $250-…).

4. Product Innovation: Improving the client’s experience with a specific product/feature.
   
   We did a test to validate the impact of augmented reality on a furniture shop. The goal was to understand how many people were interested in the feature and the AOV of those who bought a product as a result of engaging with the feature compared to those who didn’t engage with it. Another interesting test was to validate the sales impact of different product recommendation companies on a website.

Our thanks go out to Eduardo for taking part in this interview! To our lovely readers, we hope you found the insights useful and encourage you to apply them in your own optimization efforts.

Don’t forget to check back twice a month for more enlightening interviews! And if you haven’t already, check out our past interviews with CRO pros Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, Max Bradley, Samuel Hess, Riccardo Vandra, Lukas Petrauskas, Gabriela Florea, Sean Clanchy, Ryan Webb, Tracy Laranjo, Lucia van den Brink, LeAnn Reyes, Lucrezia Platé, Daniel Jones, May Chin, Kyle Hearnshaw, Gerda Vogt-Thomas, Melanie Kyrklund, Sahil Patel, Lucas Vos, David Sanchez del Real, Oliver Kenyon, David Stepien, Maria Luiza de Lange, Callum Dreniw, Shirley Lee, Rúben Marinheiro, Lorik Mullaademi, Sergio Simarro Villalba, Georgiana Hunter-Cozens, Asmir Muminovic, Edd Saunders, Marc Uitterhoeve, and our latest with Zander Aycock.