Convert Shopify App: Everything You Need to Know About the Newest Upgrade

Shopify’s infrastructure updates, particularly to its checkout systems, have prompted these improvements. In early 2023, Shopify announced it was deprecating checkout.liquid in favor of Checkout Extensibility, a sandboxed, controlled approach that protects sensitive data, is faster, and ensures checkout customizations don’t break during platform upgrades.

While these changes mean that direct checkout A/B testing isn’t currently possible (a limitation affecting all testing tools), they’ve enabled us to revamp our app with a focus on security, reliability and ease of use.

If you’re using Convert for A/B testing on Shopify, you’ll be glad to know that our new app vastly simplifies your testing. Let’s dive in…

Understanding the Context: Recent Shopify Changes

The major change from the last year impacting A/B testing—specifically A/B testing checkout experiences—is the deprecation of checkout.liquid.

With checkout.liquid, A/B testing tools (and other Shopify third-party apps) could freely manipulate checkout elements by injecting JavaScript, customizing the UI, and modifying functionality directly. While flexible, this introduced security risks and maintenance challenges.

Shopify’s solution was Checkout Extensibility: a sandboxed environment that keeps sensitive checkout info secure. This means standard checkout customizations are available to all Shopify users, but advanced checkout customizations are exclusive to Shopify Plus merchants.

Let’s brush through the key differences between the checkout.liquid era and the changes introduced in the Checkout Extensibility era:

	Checkout.liquid	Checkout Extensibility
Customization	Flexible, because it allowed direct changes to the checkout code.	Changes only possible through Shopify Functions, APIs, and UI extensions.
A/B testing	With JS tracking code, tools could directly modify page elements for experimenting.	Requires an API-driven approach to running and tracking experiments. But checkout A/B testing is not yet possible.
Data collection and reporting	The custom tracking code was enough to pull the required test data.	Depends on Shopify Pixels and order webhooks to track test data.

Working with Shopify as an official Shopify partner, we’ve adapted our app to these changes. The new version of our appuses theme app extensions instead of direct theme editing, making your A/B testing safer as themes remain untouched.

According to Shopify, “Theme app extensions allow merchants to easily add dynamic elements to their themes without having to interact with Liquid templates or code.”

So, instead of freely injecting code, we now work through Shopify’s APIs and backend systems. Our new app integrates seamlessly with Shopify’s new framework to provide you with powerful A/B testing capabilities across your store and is prepared to support checkout testing when such capabilities become available.

But why isn’t it available right now? Here’s why…

How Checkout Extensibility Impacted A/B Testing Since Its Onset

The switch from checkout.liquid to Checkout Extensibility affected A/B testing capabilities in three main ways:

1. Limited JavaScript Access
   
   With Checkout Extensibility, app extensions run in a sandboxed environment, restricting the use of third-party JavaScript on checkout pages. This makes it difficult for A/B testing tools to inject custom tracking scripts as they could with checkout.liquid.

2. Deprecation of Checkout Scripts
   
   The phasing out of checkout.liquid means that apps relying on embedded scripts for tracking had to find alternative methods. This requires adapting to Shopify’s standardized tracking tools, which might not offer the same level of granularity and aren’t available for checkouts yet.

3. Reliance on Shopify’s Tracking Mechanisms
   
   To collect experimentation data, these third-party tools have to rely on Shopify Pixels and Order Webhooks.
   
   Shopify Pixels provide a standardized way to track specific events within the checkout flow. But they don’t offer the same flexibility as custom JS scripts did, limiting customization in tracking.
   
   Order Webhooks, on the other hand, are useful for capturing data about completed orders but don’t provide insights into the events leading up to the purchase. This means a limited understanding of user behavior throughout the checkout funnel — defeating the purpose of checkout testing.

But it isn’t all doom and gloom with the recent updates. In fact, there are pros worth mentioning:

1. Enhanced security and stability: The sandboxed environment ensures that checkout customizations do not interfere with sensitive data or cause instability. This improves the reliability of tracking and reduces the risk of errors from conflicting scripts.
2. Standardized tracking with Shopify Pixels: While Shopify Pixels have limitations, they offer consistency across different apps, leading to more reliable data for analyzing checkout performance.
3. Integration with Shop Pay: When shoppers opt for the faster checkout experience, your seamless integration with Shop Pay would mean you’re getting complete and consistent checkout data.

It makes the prospect of an official Shopify-supported checkout testing framework all the more exciting. Until then, we’ve updated the Convert Shopify app, incorporating all platform changes to fully support your Shopify A/B testing needs.

How the New Convert Shopify App Works

Check out this video that illustrates how easy it is to set up and run experiments on Shopify using the new Convert Shopify App:

Previously, the Convert app integrated directly with the theme files. This allowed for more extensive customizations, including JavaScript injection for immediate, front-end changes.

Tracking codes were added directly to theme files, and A/B testing variations were created by editing those files, giving Convert full control over manipulating elements on your Shopify pages for A/B testing.

That way, the Convert Shopify A/B testing tool could easily modify any element on your Shopify store. Not checkouts, though. Checkout testing has always required special access. Only Shopify Plus merchants could use manual code manipulations to test elements like shipping information displays to discount offers.

However, this came with risks, including potential conflicts, security vulnerabilities, and loss of customization after Shopify updates.

Today, the new app securely pulls experiment data to track goals and events via API and uses theme app extensions to create variants. So you can continue your ecommerce A/B testing with data you can trust right in your Convert account.

Key Features Accessible to You

• Integrate Convert with your Shopify store without modifying theme liquid files — the tracking script is now managed through theme app extensions.
• Use theme extensions to run A/B tests, ensuring your themes are untouched and your configuration remains intact when receiving new updates from the Convert Shopify App — no need to manually re-enable the app anymore.
• Fully compatible with the latest changes to Shopify, such as Checkout Extensibility, keeping you ahead of any deadlines (as long as you’ve upgraded as directed).
• Track goals and events specific to the checkout experience, including cart_viewed, product_add_to_cart, product_removed_from_cart, and other Shopify Standard Events — not just revenue as in our older integration — and get reliable experiment data outside of the checkout environment.
• Real-time tracking, activity tracking, event tracking, segmentation, cohort analysis, and revenue tracking that works universally without relying on the order confirmation page – all powered by Shopify Pixels.
• Access custom dashboards, custom reports, data exports, and remain GDPR compliant in your ecommerce experimentation program.
• And more…

Overall, we’ve adapted the Convert Shopify app to align with Shopify’s vision of a secure, high-performance experience. This means our A/B testers continue to get valuable insights without sacrificing compliance, security, or reliability.

What Are the Next Steps for You?

Here’s what you need to do now:

1. Assess Your Current Shopify Store Setup
   Evaluate your existing checkout customizations and A/B testing implementations and how they align with Checkout Extensibility’s requirements.
   
   You want to understand which parts of your checkout flow use legacy scripts, identify any potential compatibility issues, and prepare to make changes.

2. Upgrade to Checkout Extensibility for Checkout Pages
   Key deadlines for Checkout Extensibility:
   
   • In-checkout pages (Information, Shipping, Payment): August 13, 2024
   • Post-checkout pages (Thank You, Order Status): August 28, 2025
   
   After these deadlines, all customizations will automatically revert to the default Shopify checkout. You’ll lose your customizations if you don’t do anything before then.

3. Prepare for the Deprecation of Post-Checkout Customizations
   Post-checkout customizations using checkout.liquid (such as on the Thank You and Order Status pages) are also set to be deprecated by August 28, 2025.
   
   Begin preparing now by exploring Shopify’s alternatives like Post-Purchase Extensions and using Shopify Pixels to track goals.

4. Check Out Shopify’s Customization Report
   
   Shopify offers a personalized report in Settings > Checkout that provides insights into existing checkout customizations, including those built with checkout.liquid. This report helps you identify what needs to be migrated or replaced.

TL;DR: How Has the Convert Shopify App Adapted to Shopify’s Recent Changes?

Shopify has replaced checkout.liquid with Checkout Extensibility—a secure, upgrade-safe solution that uses APIs, Shopify Functions, and Pixels for checkout customization.

For A/B testing tools like Convert, the transition meant moving from direct JavaScript customizations to an API-driven approach. Our new Shopify app brings major improvements over our previous custom app:

Then vs. Now

• Installation: From complex multi-step setup → One-click install from Shopify store
• Authentication: From multiple API keys → Only Convert API keys needed
• Updates: From manual enable/disable → Automatic updates with no user action
• Revenue Tracking: From order confirmation page dependency → Universal tracking across the store
• Event Tracking: From revenue tracking-only → Full Shopify Standard Events (cart_viewed, add_to_cart, etc.)
• Script Management: From auto-install → Simple toggle in theme editor

While checkout A/B testing is still unavailable (a platform-wide limitation), these improvements make the new Convert Shopify app more user-friendly and powerful for all other store optimization needs.