Smith Optics, a popular online retailer selling high-quality eyewear, headgear, apparel, and accessories — was looking to improve its user experience and optimize its conversion rate. To uncover customer friction points and conversion optimization opportunities, Smith engaged surefoot.me.
The Hypothesis:
To uncover the best conversion optimization opportunities, surefoot started by analyzing Smith’s Google Analytics data. They found that Smith experienced a significant drop-off in mobile traffic on the cart page.
In addition to Google Analytics, surefoot also used Hotjar and user testing to gather more insights into Smith’s customers’ behaviors and problems with the current cart. Data revealed that Smith’s mobile users experienced a lot of frustration and confusion during their mobile checkout experience. Recordings showed visitors scrolling up and down the cart page, struggling to figure out next steps. Customers expressed they were overwhelmed by options and needed a clearer path to checkout.
Data revealed the greatest customer drop-off in the mobile checkout process was the cart page, and users reported frustration and confusion with the mobile cart UX/UI.
Laura Stude, co-founder – surefoot.me
As a result of the data collected, the team decided to run a mobile-specific cart redesign test. They hypothesized that doing so would reduce cart drop-off and prompt more users to move forward in their checkout process, thereby leading to more mobile transactions.
The Methodology:
| Spread: 50/50 | Visitors: 6,647 visitors per variant |
| Confidence Score: 95% | Duration: 20 Days |
The Treatment:
Guided by its data-driven hypothesis, surefoot designed and developed a completely revamped mobile cart page to test against the original.
The test variation introduced a redesigned cart page that offered better usability and came with many changes aimed at eliminating the user frustration and friction observed during the research phase.
Among other improvements, it 1) pushed the login option (which originally was accessible only after a lot of scrolling) up to the top of the page, 2) revamped the “edit cart” functionality to make it more mobile-friendly with larger tap targets (to make it easy to toggle and change the quantity of the items and remove them if needed), and 3) added a primary CTA button to check out, thereby making it easy for the visitors to know what to do next.
 |  |
|
Control |
Variation |
Surefoot tested the variation against the control version over a period of 20 days. The test traffic comprised visitors from all channels (paid, organic, social, and referral) and each version saw more than 6,500 visitors.
A fairly lean tech stack powered surefoot’s experiment: Google Analytics, user tests and Hotjar for researching and data analysis and Convert Experiences for running the experiment.
Results:
As the team hypothesized, the variation outperformed the control.
The test resulted in an 8% lift in visits to checkout at 95% statistical significance, 3.4% lift in transactions and an increase of $1.39 in RPV – not a bad win!
Brian Schmitt, co-founder – surefoot.me
In addition, post-test UX analysis revealed that as a result of the improved cart page, Smith’s mobile users experienced a marked reduction in frustration during their checkout experience.
Making the mobile buying experience more intuitive paid off well for this experiment.
Simplification, streamlining and building trust are keys to success, especially deep in the funnel on mobile devices. Invoke the “don’t make me think” principle and you’ll increase your chances of success!
Laura Stude, co-founder – surefoot.me
For this experiment, surefoot monitored the behavior and conversions for multiple segments of Smith’s customers, including pro subscribers who receive access to the products at discounted rates and regular customers who pay full-price. This test successfully influenced the behavior of both segments and made a big impact on conversions from the segment that paid full-price, resulting in a considerable annual revenue bump for the full-price segment alone.
The Big Takeaways For You:
Mobile-first experiments can have a significant impact on your bottom line. Because desktop and mobile site experiences are quite unique, some cases call for mobile-specific treatments. As we saw here, surefoot decided to optimize specifically for mobile. Being device-specific with your tests can help boost conversions, as your customers have different problems, goals and motivations on each device.
You don’t need a million different tools to run high-impact experiments. surefoot executed this experiment using Hotjar, user testing and Google Analytics (for research) and Convert Experiences (for running the experiment). It doesn’t take tons of fancy or expensive tools to plan and test winning experiments – more important are that you move quickly and have a culture of experimentation and learning.
Segment your test data during post-experiment analysis to better understand your customers’ behavior and identify future A/B test and personalization ideas. surefoot didn’t stop at identifying the overall winner once the experiment reached statistical significance. They sliced the data and studied the behavior of multiple segments – including traffic type, new vs. returning, pro deal/full price, and geographic location. This level of analysis helped the team better understand differences in purchasing habits amongst different traffic segments and paved the way for follow-up hypotheses to test.
Surefoot.me is a full-service testing and personalization agency that helps companies improve their conversion rates, lower acquisition costs, and increase revenue. The founders of surefoot cut their teeth working with giants like Keurig, Trulia, Adidas, Weight Watchers, and more and now focus on increasing conversion rate and revenue for SMBs. surefoot’s customer-centric approach allows them to identify the highest impact tests and maximize ROI, regularly netting 4-15x returns for clients.
At Convert, we’re on a mission to create meaningful, sustainable experiences for today’s users and future generations. As the only Climate Neutral Certified A/B testing platform, we’re proud to offer a robust, affordable, and feature-rich platform that’s been an industry favorite for over a decade. But we’re not just about optimization for profit. With a Conscious Business approach, we’re dedicated to making a real difference and driving positive change within the CRO space and the world.