The Exact Features You Need to Split Test Two Completely Different Experiences.

A split test or a split URL test is often used interchangeably with an A/B test. This isn’t wholly accurate. While you would still compare two different versions of an element or an asset, in split testing the beauty lies in the simple execution and broad scope. 

Here’s why. 

In a split URL test, in most cases, your A/B testing tool is in charge of splitting traffic between two URLs that already exist on your site. Some examples include: 

• Two different store designs/ themes.  

• Implementing a widget like a loyalty program pop-up that significantly impacts the way people perceive your offering and constitutes you innovating on your existing business model.  

• Different checkout flows for different product categories – a bare bones one-page checkout vs something more elaborate with upgrade nudges and reasons to shop your brand. 

What this means is the A/B testing tool is mostly exempt from manipulating the code and CSS of your website. That part is taken care of by your development team. 

Yes, the variant pitched against the original may not be a high fidelity, fully functional version of the change you wish to see later down the line. But its delivery isn’t restricted to the code editor of your experimentation platform. 

Split testing is thus more strategic. Think of it as a compass, pointing you in the direction of paths and experiences that are worth thinking granularly about and optimizing iteratively. 

Another factor here is the scope of the experiment. If your old site competes against your newly redesigned website, you aren’t interested in minute 1% conversion lifts. You know there is a change. One that is likely to alter traffic response.

This is why you can use a wider net in statistical terms and go with a high MDE (Minimum Detectable Effect). This means, given the same traffic, your split tests will usually conclude faster than a more nuanced A/B test, as the lift will usually be stronger in either direction.

Let’s keep this simple. 

Use split testing when you aren’t interested in barely detectable shifts. When you know you’ve made a significant change to the status quo and you wish to see how the buyer response pendulum will swing. 

Use A/B testing when you want to have a direct causal relationship between the change and the outcome. Many optimizers recommend adjusting only 1 element in an A/B test, so that with an adequate confidence level and the right sample size, you can be quite sure of the fact that your idea (the hypothesis) has indeed moved the needle – either positively or negatively. And the observed effect is not random noise. 

While split testing helps you efficiently discard or adopt approaches, A/B testing is your reliable channel to finesse an approach and take it to the next level, learning about what works for your audience along the way.

Not if you conduct your split tests the right way. 

To better understand the risks, let’s talk about cloaking. 

Cloaking content was mentioned in the early days of search engines (around 1990s). In those days, SEO was all about keyword stuffing. So SEO agencies would display a page full of keywords when a search engine bot visited to crawl / index the page. 

While if a user visited the same page, they displayed the default (normal) version. This strategy of keyword stuffing used to work like wonders so naturally search engines started devising clever ways to detect and penalize such cloaking. But thanks to Google’s PageRank algorithm, keyword stuffing no longer works.

But you still have to be mindful of the following when you redirect your traffic to your variant. 

1. Place the rel=canonical link attribute on all of your alternate links, pointing to your original page. This will help point bots indexing your website to your original page. Experiments involving redirects should be fine as long as they don’t redirect to unexpected or unrelated content.

2. As long as you are not basing your traffic allocation on a search engine vs human distinction (using Googlebot user-agent in Audiences feature), you will not be punished for cloaking. Google doesn’t care if their bot sees one version or another; it just cares that its bot has the same user experience as that of a random visitor.

1. Run the split URL test only as long as it is necessary. You have the right to use the power of experimentation to grow your business. But an experiment, by its very definition, runs for a limited period of time. Respect that.