Step by Step Guide to A/B Testing Shopify Themes (Using Convert Experiences)

As a Shopify business owner, finding the right theme is tough. ​​A lot of time and energy goes into setting up a successful ecommerce store. And it all starts with spending hours picking out the perfect theme to give your online store the right look – which makes it even more frustrating when it underperforms.

So many different themes to choose from, but how do they all compare? Which one should you use for your business to delight visitors and maximize sales?

Well, what if there was a way to pick more than one theme? To make sure the design actually works and is getting you sales before committing to just one?

A/B testing is a great way to measure the effectiveness of your chosen Shopify theme(s). One way to do this is by picking two (or more) themes with different designs (for example, theme A and theme B), then testing their effectiveness against each other!

This tutorial will walk you through everything you need to know about A/B testing Shopify themes. For illustration purposes, I’ll be detailing this process using Convert’s Split URL option.

1. Select the Shopify Themes You Want to Test

First of all, you need to select two Shopify themes to test. (You’ll need to limit yourself to only two themes for the first test. If you’re considering more than two themes, you’ll need to run subsequent tests.)

The main theme will be the one you are showing live to the Shopify store visitors, while the other that you want to test against will be in Preview Mode (not published).

In the Shopify app, go find your live theme in Sales Channels > Online Store, then Themes:

Then, pick the second theme you want to test from the Theme Library:

2. Install the Convert Tracking Code in Both Themes

Now that you’ve chosen your themes from the Shopify platform, you’ll need to install the tracking code. This is an essential part that will ensure visitor behavior is tracked properly and the test will give you the rightful winner!

Simply follow the instructions below to install the Convert Experiences tracking code in BOTH the Shopify themes that you want to split test.

Go to Sales Channels > Online Store > Themes > More Actions > Edit Code.

Then, select the theme.liquid file and paste the Convert tracking code below just before the first “{% if %}” section, as shown in the screenshot. Make sure you replace the project ID number with your own Convert account-project ID.

<!-- begin Convert Experiences code-->
<script type="text/javascript">
var _conv_page_type = "{% if template == 'index' %}homepage{% else template == "collections" %}collection{% else template == "article" %}article{% else template == "product" %}product{% else template == "cart" %}cart{% else %}other{% endif %}";
 var _conv_category_id = "";
 var _conv_category_name = "{{ collection.title }}";
 var _conv_product_sku = "";
 var _conv_product_name = "{{ product.title }}";
 var _conv_product_price = "{{ product.compare_at_price_min | money }}";
 var _conv_customer_id = "";
 var _conv_custom_v1 = "{{ tag }}";
 var _conv_custom_v2 = "{{ order.shipping_address.province }}";
 var _conv_custom_v3 = "{{ cart.item_count }}";
 var _conv_custom_v4 = "{{ cart.total_price | money_without_currency }}";
</script>
<script type="text/javascript" src="//cdn-3.convertexperiments.com/js/<REPLACE THIS WITH PROJECT ID NUMBER>.js"></script>
<!-- end Convert Experiences code -->

Lastly, don’t forget to save your changes!

Like I mentioned above, you’ll need to apply the same steps to the second theme you want to test. Remember, that is the theme you’ll keep in preview mode.

Start by going to Sales Channels > Online Store > Themes > Theme Library > (select the theme you want) > Edit Code.

Then, simply repeat the process for the second theme: Select the theme.liquid file, edit the code using the script above, and save your changes.

It should look like this, depending on which theme you selected:

3. Allow Revenue Tracking

Now for the important part: tracking revenue. How else will you know which Shopify theme worked better for your business?

There are three main ways to track revenue, which I cover in greater detail in this post.

In short, you can decide to either

1. Install the Convert tracking code on the order status Shopify page to be able to track revenue/purchases through Google Analytics,
2. Set up manual revenue tracking code, or
3. Use webhooks.

You can use revenue tracking to know when your customers visit your checkout and how often they make a purchase. It’s also a great way to learn more about purchasing trends in your store, like average order value and total sales per product.

To set it up, go to Settings > Checkout and scroll down to the Additional Scripts section. In the Additional Scripts form field paste the following code:

<script type="text/javascript">
  if (typeof _conv_host == "undefined") {
  window["_conv_prevent_bodyhide"] = true;
  (function() {
    var _conv_track = document.createElement("script");
    _conv_track.src = "//cdn-3.convertexperiments.com/js/XXXXX-XXXXX.js"
    document.getElementsByTagName("head")[0].appendChild(_conv_track
</script>

Replace the text, “XXXXX-XXXXX”, with your Convert account-project ID found in your Project’s Settings.

4. Find the Shopify Preview Theme ID

Next, you’ll need to edit the code of the unpublished theme from within the Shopify website. Remember, you can do so by going to Sales Channels > Online Store > Themes > More Actions > Edit Code.

Then, you can copy the theme ID from the admin URL, /admin/themes/xxxxxxxx.

You will need this ID so that you can append ?preview_theme_id=xxxxxxxx as a variation URL for the split URL test (more on this in the next section).

5. Create a Convert Split URL Test

Now, let’s move to the A/B testing tool, in this case, Convert Experiences, to set up the split URL test.

In the Edit Split URL Variations section, insert this code in the Original URL field :

^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?

Use the following as the Variation URL, where the preview_theme_id variable is what you discovered in the previous step (and don’t forget to replace ‘123456’ with your own Shopify preview theme ID):

$1$3$5?preview_theme_id=123456&$7

The resulting URL will be something like this:

https://www.store.com?preview_theme_id=123456&.

Then, enable Support Regular Expressions and Transfer Original URL variables to the variation URL. Your test variations should look like this (except the theme ID will be different):

6. Specify Which Shopify Pages To Target

Once again, go to the Convert Experiences app. Make sure that the Site Area section includes all the pages in your Shopify store (including all those valuable product pages!).

It could look something like this:

7. Hide the Shopify Preview Bar

If we stop here and just activate the split URL experiment when the preview theme template is shown randomly to website visitors, an annoying popup appears specifying that it is a preview template. This might not be appealing to your visitors and might raise concerns about what other tests are running on your store.

To avoid causing unnecessary friction, add the CSS code below to Shopify to hide the preview banner. Please note that if you want to hide this banner from checkout pages as well, you’ll need a Shopify Plus account, as the code only applies to Shopify Plus clients.

In Shopify, go to Online Store > Preferences > Google Analytics and add the following code in the Additional Google Analytics JavaScript box:

function() {
var css_override = document.createElement("style");
document.getElementsByTagName("head")[0].appendChild(css_override);
})();

var css = 'iframe#preview-bar-iframe { display: none !important; }',
head = document.head || document.getElementsByTagName('head')[0],
style = document.createElement('style');
style.type = 'text/css';
if (style.styleSheet) {
style.styleSheet.cssText = css;
} else {
style.appendChild(document.createTextNode(css));
}
head.appendChild(style);

8. Launch The Split URL Experiment

You are now ready to activate your split URL experiment. To do so, you will need to go to your experiment page in Convert Experiences and set it to Active.

Once the experiment is live, this is what happens when a visitor lands on one of your Shopify store pages:

8. Analyze Your Results and Pick a Winner!

Convert uses Frequentist Inference to determine when and by how much your Shopify themes are better than each other. As a rule of thumb, we recommend waiting two weeks before choosing the winning theme, and the tool itself generally doesn’t tell you a winner earlier than this (unless you have high traffic on your site).

This is also the main reason that our free trial runs for 2 weeks.

In the world of ecommerce, two weeks isn’t even that long. So sticking to at least two weeks is not a bad idea. After all, we are testing entire themes so it won’t hurt to be completely sure about the impact your test has.

Conclusion

So there you have it.

Whether you’re a new Shopify store owner looking to find the perfect theme for your business or an experienced ecommerce business owner who’s looking for a better-performing one, Shopify A/B testing is something that can help you improve user experience, optimize your conversion rates and sales, and make better business decisions.

We’ve outlined all of the steps above so don’t worry if the testing process is unfamiliar territory for you! And if you’ve liked how easy to use Convert Experiences is, click on the banner below to take a free trial so you can start working towards finding the right Shopify theme right away.