SPA A/B Testing: A Starter Guide to Single Page Applications & The 10 Best SPA A/B Testing Tools to Use
Single page applications: What are they, how do they work, and, more importantly, how do we run A/B tests on them?
In this guide, we’re going to break this down, as well as look at 10 of the best A/B testing platforms you can use.
We’ll cover their pricing, their features, if they allow trials, and then go into detail on which methods they use to allow you to run A/B testing on single page applications.
This way you can decide exactly which tool is the best fit for you and your testing needs.
(If you already know everything about SPA testing, then feel free to jump to the tools section here).
What Is a Single Page Application (SPA)?
An SPA is a website or web application that’s designed to load super fast and fluidly. Its goal is to replicate the experience of a well-designed mobile app and how it’s built. You click on something and it loads almost immediately.
The name single-page application, however, is a little misleading and confusing for non-tech users. It’s not referring to sites that are only built as a single webpage but instead refers to how it is coded.
Single Page Application Architecture
The best way to understand how an SPA works is to look at how it’s built (the architecture of the site) and compare that to a standard website you might use.
A standard website, otherwise known as a static or a ‘Multi-Page Application’ (MPA), is built by having multiple pages of HTML code that represent each individual page on the site. So you’re going to have one HTML page for the home page, another for the about page, etc.
Each time a new link is clicked, the website makes a server request for the HTML code for that new page to be loaded up, which causes a delay.
You can actually get around this to some extent by preloading these other links on MPA sites to speed up your page load. We talk about it in this guide here, for getting your site ready to meet Core Web Vitals when testing.
SPA (Single-Page Application) vs. MPA (Multi-Page Application)
The main difference between an SPA and an MPA is the SPA has just one single page of HTML code for the entire site which only needs to load a single time, hence the term ‘single page’.
Rather than having to request and load multiple HTML pages, the SPA pre-loads all of the content on the first click, caching the other site content in advance, and then dynamically re-renders and updates the page’s appearance when someone clicks on links instead of loading up new HTML files.
It achieves this by using JavaScript and server-side rendering to load the new content or pages, while the current page is still open. This gives the appearance of a more fluid user experience while also usually loading faster than a traditional MPA as there’s no apparent downtime or blank screens.
The Disadvantages of Single Page Application Sites
As testers, we know that a fast site provides a great user experience, but what are the disadvantages of using an SPA for your site?
1. Slow Initial Page Load
Because the SPA pre-loads the entire site in one go, it can mean that the initial page load can be slower than a standard MPA. However, because of the way it loads, it can still appear to load fast for the user, as the initial viewport usually loads right away giving the appearance of an instant load.
2. JavaScript Needs to Be Installed and Working
If the user’s browser or device doesn’t have JavaScript enabled, the site simply won’t work. This rarely happens and users will be notified to enable and download it, but it can happen.
3. More Effort to Build and Edit (Fat Frontend)
Because of the complexity in building SPA sites, they can sometimes take more time to create or more effort to make changes.
Not only that, but they can also cause possible UX issues if not built correctly.
For example, the user can press the back tab and leave the site instead of going back to the previous ‘page’. Again, very rare when built correctly.
4. Analytics Measurement and Reporting
SPAs can also be difficult to track on-site analytics. Because there is no separate page as such, actions need to be tracked via the dynamic changes occurring instead of URL loads. Here’s a guide on how to do this with Google Analytics 4 and GTM.
5. Possible SEO Issues
This can also affect the SEO for the site as Google’s web spiders struggle to crawl the SPA. (Although it’s currently being improved by Google’s work to track dynamic JavaScript on the page.)
6. Potential A/B Testing Issues
Finally, we can run into some A/B testing issues when testing on single-page applications, mainly due to how the page loads and how the test is delivered, but we’ll cover this in more detail in just a second.
Popular Single Page Application Frameworks. What to Choose: React, Angular, or Vue?
There are several frameworks that developers can use to build single page apps such as:
- Angular
- React
- MeteorJs
- BackboneJs
- EmberJs
- PolymerJs
- KnockoutJs
- Aurelia
The most popular, however, are Angular (with plenty of documentation on GitHub), React (open source, well documented on GitHub), and Vue.js (the newest of the three).
Single Page Application Site Examples
What kind of sites are built on SPA frameworks?
Thanks to its fantastic load speeds, some of the biggest sites out there are built as SPAs:
- Netflix
- Gmail
- Youtube
- Airbnb
The majority of SPAs are built using frameworks like React and Angular.
Here are a few well-known SPA examples:
Netflix
The popular entertainment streaming service makes use of the React framework to allow its users to watch TV shows and movies without experiencing any lag.
When you consider how much data is streamed on Netflix and by how many people, it’s reasonable to conclude that Netflix’s SPA approach makes full use of this method.
Facebook’s news feed is an example of an SPA.
When you scroll through the previous videos or click on other buttons, specific portions of content are loaded onto the page. To endure the experience, you do not need to leave the page.
Gmail
The browser stays on the same web page when you click on your messages in your inbox.
How to Do A/B Testing on Single Page Application (SPA) Sites?
Rather than go into detail about specific A/B testing ideas for SPA sites which we’ve covered before, I want to instead cover how to actually run tests on SPA sites instead.
Why?
Well, running tests on SPA sites is different from a static MPA because of three things:
- The trigger that causes the test to fire,
- The framework that the SPA is built on, and
- The tool that you’re using to test with.
So let me explain the fundamentals…
The Test Trigger
Normally, with most A/B tests, we trigger the experiment based on URL strings. If someone visits the specific URL of the page we want to test and if they meet the criteria we set, then they have a chance for the experiment to run for them.
Because SPAs don’t have a particular URL trigger and use JavaScript to re-render the page elements, we need to instead use other trigger methods such as:
- Polling Triggers,
- Using JavaScript conditions in the site area, and
- Manual Activation.
This way, we can ensure that the test is triggered and runs in the right conditions.
The SPA Framework
It’s not just setting up new triggers though. Because of the way that the 3 popular frameworks run and deliver the SPA site, you may need to adjust your test setup so that it delivers correctly, depending on which framework you’re using.
We have guides for each of these below:
- Running Experiments on an AngularJS Page
- Running Experiments on an VueJS Page
- Running Experiments on an ReactJS Page
The Tool You Use to Test your SPA with
It’s not just the JavaScript frameworks that you have to take into account when setting up tests on SPA sites. You also need to look at how the actual tool you’re using works to run tests on an SPA site because it can affect if you can run the test on your particular framework.
Here’s how we run A/B tests on SPA sites using Convert Experiences:
- Run an A/A test to ensure the tool is set up correctly if it’s your first time using it. (Only needs to be done once.)
- Use the Data Layer to hold the URL parts and other page data (since the URL itself does not change).
- The section of the site in the viewport is an important trigger that can be used for calling variations. This is necessary when the URL isn’t updated with sections scrolled.
- Make sure you use a CDN or pre-fetch content so that the updated site section content is loaded fast & does not disrupt the fluid/native nature of SPAs (one of its biggest advantages).
- Ensure triggered goals are being polled & collected, even if the page is not refreshing.
- You may also go server-side for SPA-based A/B testing.
This is true for any testing platform you might use, so when we break down the different A/B testing tools below, we’ll cover how each of them actually tests on SPAs so you can decide which is best for you.
Top 10 A/B Testing Tools That Have Adapted to Single Page Application (SPA) Sites
Now that we’ve looked at what SPAs are and the common issues you need to keep in mind when testing SPA sites, let’s break down the most popular testing tools that allow you to run tests on them.
- Convert Experiences
- Adobe Target
- VWO
- Kameleoon
- SiteSpect
- Optimizely
- Google Optimize
- ABlyft
- AB Tasty
- Dynamic Yield
#1: Convert Experiences
Pricing: Starting as low as $399 per month, with $299 for every 100k visitors after that.
Plan types:
Do they offer a free trial? Yes, 15 days free trial with no credit card needed.
Do they offer customer support? Yes, right from the start of the trial. What type? Live chat, blog, and knowledge base with more to come.
How does it adapt to handle SPAs? Convert can handle SPAs by providing a Convert Activation function (polling) that can be called at any point after page load. Allows for specific triggering such as goals, JavaScript, Manual activation, and more. Also allows you to reload the Convert settings on every view change and can work on client-side or server-side tracking.
Does it have the ability to trigger experiments through JavaScript? Yes. You can read more about it here.
Does it poll goal triggering without relying on page refreshes? Yes. Through adding some ready to go code on the Project Settings > Global Project Javascript or through direct integration with the SPA framework.
Read more about running experiments on SPAs.
Trust Radius review:
Top customers: Sony, Jabra, Unicef, and others.
Do they care about privacy? We are fully GDPR compliant. We even stopped using tools for our internal work from other companies that don’t meet GDPR, that’s how much it means to us.
Do they care about the world? Yes. It’s in our DNA. We plant trees, run community programs, champion diversity from the initial application, donate to charities, and much more. Heck, we’re 15x carbon negative.
What this A/B testing platform looks like in action:
#2: Adobe Target
Pricing: Only available after booking a call. Starting rate is approximately $10,000 a month.
Do they offer a free trial? No.
Plan types:
Do they offer customer support? Yes, in the form of a knowledge base and video training programs.
How does it adapt to handle SPAs? Adobe Target can handle SPAs through a triggerView() function that can be called within the SPA code. You can read more about this here.
Does it have the ability to trigger experiments through JavaScript? Yes.
Does it poll goal triggering without relying on page refreshes? Yes. However, it needs a direct integration with the SPA Framework.
Trust Radius review:
Top customers: Nvidia, Sprint mobile, Phillips.
Do they care about privacy? Yes, but ironically they require your email and phone number to learn more about this. They are however doing a lot of work in 1st party data for non-intrusive tracking.
Do they care about the world? Very much so. They champion diversity, work towards 100% renewable energy buildings, lowering emissions, run community action programs, and much more.
What this testing tool looks like in action:
#3. VWO
Pricing: Starts at $99/mo. Anything more requires customized pricing according to your needs. And you’ll have to contact the VWO sales team for yours.
Plan types:
Do they offer a free trial? Yes, you can get started with a 30-day free trial.
Do they offer customer support? Yes, you can call them when you need help. Or consult their resource page.
How does it adapt to handle SPAs? VWO can handle SPAs automatically through its standard tracking code, and also provides a function to trigger the page changes.
For SPAs that change URL strings with new content loads, it works with the standard install. For SPAs that don’t change the URL, you need to set up a virtual page using a custom URL.
Does it have the ability to trigger experiments through JavaScript? Yes. Through calling ‘applychanges’.
Does it poll goal triggering without relying on page refreshes? Yes. It will automatically detect URL and browser history changes by only including the VWO tracking code.
Trust Radius review:
Top customers: Ubisoft, AMD, Penn Foster, Optimeria, and IMB Bank.
Do they care about privacy? Yes. They comply with the major privacy laws and regulations but they store some visitor data on their servers.
Do they care about the world? The chairman of Wingify (the brand behind VWO) Paras Chopra, tweeted their support for a couple of COVID relief initiatives including setting up a 10-bed COVID-care facility in Delhi.
What this A/B testing platform looks like in action:
#4. Kameleoon
Pricing: Customized according to your requirements. You get your unique price by contacting the sales team.
Plan types: You need to contact their support team for pricing. Plans are based on your requirements.
Do they offer a free trial? Yes.
Do they offer customer support? Yes. You can even get a dedicated account manager to assist you with complicated projects.
How does it adapt to handle SPAs? Kameleoon has an extensive activation API to be able to activate experiments on their platform. Read more about it here.
Does it have the ability to trigger experiments through JavaScript? Yes. Through the assignVariation() function.
Does it poll goal triggering without relying on page refreshes? It automatically detects URL updates. However, when no URL changes happen, then experiment activation should be called via its API.
Trust Radius review:
Top customers: Lexus, Unity, Providence, Toyota, and Rakuten use Kameleoon.
Do they care about privacy? They are HIPAA, GDPR, and CCPA compliant. They also have a flexible consent management feature to adapt your privacy policy for each A/B test.
Do they care about the world? I couldn’t find evidence of any charity they sponsor or environmental causes they support. Although, it’s interesting to know they have two live chameleons in their Germany and France offices.
What this A/B testing platform looks like in action:
#5: SiteSpect
Pricing: You’ll have to contact SiteSpect for a customized price according to your needs.
Plan types: No information available before a call.
Do they offer a free plan? Yes, they offer a free trial.
Do they offer customer support? Customer support is available via phone call and email for free and paid versions. There are also pre-recorded webinars available in their knowledge base.
How does it adapt to handle SPAs? SitesSpect acts as a proxy where all website elements can be changed and swapped for other versions while it is being downloaded from the servers.
That means that they can modify any element before assembling the page, even before the SPA has been downloaded on the page. However, you will need technical expertise to be able to determine what elements need to be swapped and definitely not as easy as changing a page on the site.
Also, any components that are pre-assembled on the servers can not be reassembled with this solution.
Does it have the ability to trigger experiments through JavaScript? No. It is all done in a proxy, similar to server side.
Does it poll goal triggering without relying on page refreshes? No, for the same reason as before. It’s all done in a proxy, similar to server side.
Trust Radius review:
Top customers: Staples, PetSmart, AmericanGirl, and Urban Outfitters use SiteSpect.
Do they care about privacy? They do. They are PCI DSS 3.2 certified; GDPR, CCPA, and privacy shield compliant, and HIPAA ready.
Do they care about the world? SiteSpect has been known to sponsor some charity projects since 2014.
What this A/B testing tool looks like in action:
#6. Optimizely
Pricing: They’re using a custom pricing model. But Splitbase predicts they cost at least $36,000 per year.
Plan types: I could not find this information without contacting them for a demo.
Do they offer a free trial? No. They stopped the free plan in 2018.
Do they offer customer support? Yes. They have a bank of resources to help users get unstuck and phone numbers to call for help 24/7.
How does it adapt to handle SPAs? Optimizely’s support for SPA can be activated by checking a checkbox. This detects when the URL changes or when the DOM changes which activates the evaluation of the experiments, and the activation of the ones that the visitor has been bucketed on the current page.
Does it have the ability to trigger experiments through JavaScript? Yes.
Does it poll goal triggering without relying on page refreshes? Yes. It polls for URL changes and DOM changes.
Trust Radius review:
Top customers: Microsoft, IBM, HP, eBay, Yamaha, Pizza Hut, and Atlassian use it.
Do they care about privacy? They take into consideration old and new privacy laws and integrate that into their products so that you, the user, don’t have to worry about compliance.
Do they care about the world? Most new hires are sent to volunteer in the community on their second day.
What this A B testing software looks like in action:
#7: Google Optimize/Optimize 360
Pricing: There are 2 versions of this tool. Optimize is free, but for advanced features, you have to opt for Google Optimize 360.
The pricing for Optimize 360 is custom (although rumored to be around $150,000 per year). To get your monthly price, you’ll have to fill a form to contact their sales team.
Plan types:
Do they offer a free trial? Yes.
Do they offer customer support? They have a resource hub with tips, video tutorials, help community, and more.
How does it adapt to handle SPAs? Google Optimize provides SPA support by allowing an automatic activation of experiment changes when the URL or the DOM changes.
It also provides an activation event in which experiment changes can be activated with code.
Does it have the ability to trigger experiments through JavaScript? Yes. Through optimize.activate which they recommend to insert via GTM.
Does it poll goal triggering without relying on page refreshes? Yes. It does when enabling Continuous Activation.
Trust Radius review:
Top customers: Adidas, Spotify, Eurobank and more.
Do they care about privacy? They comply with applicable customer data protection laws in the countries where their products are used.
They also work with third parties and data protection authorities to keep users’ data safe.
Do they care about the world? Google has been carbon neutral since 2007 and plans to be carbon-free by 2030. Apart from that, they’re known for numerous philanthropic gestures with their $1 billion commitment.
What this split testing platform looks like in action:
#8. ABlyft
Pricing: Pricing is available upon request. I did however find an older document where they offered limited tests for FREE for 5k users, then $329 a month after that ($279 if paying annually).
Plan types:
Do they offer a free trial? Yes, you can get started with a 30-day free trial, no credit card needed.
Do they offer customer support? Yes. They have a basic knowledge base and live chat options.
How does it adapt to handle SPAs? ABlyft also counts with automatic and programmatic AB testing. With an option for activation with URL, DOM change and programmatic API.
Does it have the ability to trigger experiments through JavaScript? Yes, via an API function.
Does it poll goal triggering without relying on page refreshes? Yes, when the URL and DOM change.
Trust Radius review:
There were no reviews on Trust Radius for this tool, so I took the review from G2.
Top Customers Elle magazine, Bazaar, and Instyle.
Do they care about privacy? Yes. They comply with the major privacy laws and GDPR.
Do they care about the world? Not that I could find.
What this A/B testing platform looks like in action:
#9. AB Tasty
Pricing: AB Tasty’s plan(s) are gated. You need to fill out a custom form to request a demo, and better understand their pricing model
Plan types: Need to contact their sales team for pricing and plan types.
Do they offer a free trial? No. You can request a demo call to see their new features.
Do they offer customer support? Yes. They have a knowledge base and live chat.
How does it adapt to handle SPAs? AB Tasty allows the activation of experiments when a certain condition is met, which can be checked at page load or periodically until the condition is met. These periodic checked conditions can be a URL value, elements present on the page based on CSS selectors, JS Conditions that return to true, or custom code and also programmatic activation through javascript promises.
Does it have the ability to trigger experiments through JavaScript? Yes. Through Javascript promises or periodically evaluated JS Conditions.
Does it poll goal triggering without relying on page refreshes? Yes. Can evaluate periodically URL Values, elements present on the page through CSS selectors, JS Conditions or JS Promises.
Trust Radius review:
Top customers: Disney, L’Oreal, Kalviyo, and others.
Do they care about privacy? Yes. User IP addresses are used to create an ID code and then immediately deleted before moving into the tool. ID codes are anonymized and then deleted after 13 months.
Do they care about the world? They donate directly to NGOs, they work with social action groups, they recycle and sponsor beehives.
What this A/B testing platform looks like in action:
#10. Dynamic Yield
Pricing: I could not find any information on pricing other than it is 24% higher on average than other tools offering personalization.
Plan types: Again, I couldn’t find any information on this.
Do they offer a free trial? Yes, but you have to request a demo first.
Do they offer customer support? Yes. What type? They offer call and email support, along with a basic knowledge base to help with installation and then an academy to teach testing.
How does it adapt to handle SPAs? Dynamic Yield handles SPAs by leveraging a Web API called MutationObserver, which provides hooks into DOM mutations and enables the tool to track when mutations happen. The use of the API enables the tool to apply changes in real-time as needed..
Does it have the ability to trigger experiments through JavaScript? Yes. They do warn that some older JavaScript libraries might not be supported though.
Does it poll goal triggering without relying on page refreshes? This function needs to be enabled in a campaign’s Advanced Settings, by checking the option to Serve on every SPA event.
Trust Radius review:
Top customers: Urban Outfitters, Ikea, Sephora, and more.
Do they care about privacy? Dynamic Yield is fully GDPR compliant.
Do they care about the world? They actually do something very cool, which is to offer free use of their tool to charities that meet certain criteria. They also do work in the community.
What this A/B testing platform looks like in action:
In a Nutshell: A/B Testing Single Page Applications
So there you have it.
Our entire guide to A/B testing single page applications. What they are, how they work, and the top 10 platforms you can use to optimize SPA sites or web apps. We’ve made the knowledge easy for beginners so even if your skills don’t yet match up with all of these technical terms, we hope you still got something out of it!
Want some more information on each tool? We interviewed a bunch of CRO experts on the top tools they think are best for the next 5 years (including the tools on this list). See what they have to say here.
And stay tuned for part 2 of the A/B Testing on Single Page Applications series, where we take a peek at the backend side of things and share some more technical tips and best practices for running successful tests on single page apps. It’s coming out in two days!
Written By
George Crewe, Daniel Daines Hutt
Edited By
Carmen Apostu
