Best Practices for Running A/B Tests on Single Page Applications (SPAs)

This article is part 2 of the A/B Testing on Single Page Applications series.

In part 1, we’ve introduced the concept of single page application (SPA), showed why it’s growing in popularity, and looked at 10 of the best A/B testing platforms you can use on single page apps.

But running A/B tests on SPAs can be a bit tricky because of the way they function. In this article, we’ll give you hands-on advice about how to run an experiment on an SPA and some best practices to follow.

The Evolution of SPA Frameworks

SPAs are not a new concept.

A patent from 2002 describes a technology that is similar to the contemporary version of SPA.

However, the AJAX technique, which has been in use as a network standard since 2006, made its implementation easier. It’s a set of strategies for creating asynchronous web applications using a variety of client-side technologies. The software can send and receive data from the server in the background thanks to AJAX, without altering the current state of the page or the content display mode, and so without interfering with the user’s experience.

The introduction of AJAX, on the other hand, was not a watershed moment in the adoption of SPA.

The rise in popularity of SPAs is part of a larger trend that has been fueled by the rapid growth of the front end and the growing relevance of UX in recent years.

SPAs were first created using Java applets or Flash programs, as well as the “pure” JavaScript or jQuery libraries.

However, front-end frameworks like Angular.js, React.js, and Vue.js — all of which are relatively new JavaScript frameworks – have aided their development.

What Are the Technologies Behind an SPA Architecture?

An SPA is essentially a JavaScript framework with HTML5 and CSS3 support.

However, using these components, a few distinct libraries and frameworks have emerged and got adopted in the development world:

1. Angular

Developed by Google, Angular is the most mature and the oldest of the 3 frameworks with detailed documentation to use but a steep learning curve.

It is embedded with an original Model View Controller (MVC) architecture but mostly uses an MVVM (Model-View-ViewModel) that allows multiple developers to work separately on the same app section. There can be some migration issues with updating to new versions, but Angular does have the most reliability when scaling up and is great for large, complex projects.

Because of TypeScript, Angular is a fantastic choice for large teams of developers, and some companies already employ this technology in their other products.

On GitHub, it’s an established, mature framework, with a large number of contributors.

Google and its products such as Google Drive and Gmail, as well as Wix, are among the users of Angular for SPAs.

2. React.js

React was developed by Facebook and made open source. It was primarily used for creating user interfaces (Facebook, WhatsApp, and Instagram).

It’s highly popular due to its lightweight code size, and its ability to seamlessly integrate with other frameworks. It also has a simple method for migrating between versions. (Usually a one-click migration).

It is frequently used in conjunction with Redux to manage the state of application components. Uber also makes use of the React.js technology in its products.

React is a good fit for those who are just getting started with JavaScript front-end frameworks, as well as start-ups and agile developers. In addition, this JavaScript library offers valuable integration options with a variety of other frameworks and technologies, which is really useful when working on a project with a large environment. Of all SPA frameworks, React has the most contributions on GitHub, which helps developers troubleshoot potential problems.

It has the easiest learning curve of the 3 frameworks and is best used for projects with a short deadline that require a lot of scalability.

3. Vue.js

Vue.js isn’t owned by a major company such as Facebook or Google, but it’s still picking up pace with new users, especially in Asia. It’s the newest of the above SPA frameworks, having been founded in 2014 by Evan You, a former Google employee. A lot of documentation is usually in Chinese, but there are training sources out there still.

The framework is very light and can be used to simply create high-performance applications.

For most developers, there are no major issues with creating in any of these frameworks, the decision really comes down to the major features of each.

Vue.js is now used by enterprises such as Baidu, GitLab, and Alibaba for their needs.

How Single Page Apps Work

SPAs have a straightforward design. Both client-side and server-side rendering technologies are used.

On a non-SPA website, when you enter a URL into your browser, the browser sends the request to a server and receives an HTML page in response.

On an SPA website, the server only delivers the HTML document on the initial request and JSON data on subsequent requests. This indicates that an SPA will rewrite the content of the current page rather than reloading the entire website.

In a nutshell, an SPA functions like this:

1. The client first establishes a connection with the server and obtains the page content, which consists primarily of HTML code, CSS, and a JavaScript bundle comprising all scripts required to run the application logic.
2. A user’s action activates the execution of relevant JavaScript(s), which then make AJAX requests to the server. The data is usually in a JSON format and does not require a full web page refresh.

Instead of the normal technique of a browser launching a completely new page, a Single Page Application interacts with users by forcefully modifying existing web pages with novel data from the web server. The browser retrieves the HTML, JavaScript, and CSS code necessary, or other appropriate resources, and adds them to pages as needed.

Although HTML5 history API or the location hash can be used to provide the navigability and perception of different logical pages in the web application, the page does not reload at any time during the procedure and does not transfer control to another page.

If you want to see an example of a Single Page Application in action, go to this link and click on the main page, the list of most recent courses, and the top menu. When you do that, you’ll notice that the page doesn’t completely reload, only new data is sent over the wire as the user moves throughout the application.

This is an example of an SPA.

Who Are SPAs most Suitable for?

Because of the benefits of SPAs, they are now the preferred solution in many circumstances.

They excel in projects involving a variety of web technologies, where performance and a pleasing user experience are more important than content.

There are many big players that use SPAs. Facebook, Gmail, Google Maps, Netflix, and Paypal are some of the most prominent examples of this type of technology (see below).

Why Is A/B testing Difficult on SPAs?

All three frameworks–React.js, Angular.js, and Vue.js–are very popular among JavaScript developers because they allow for the development of sophisticated user interfaces that create better user experiences and higher conversion rates.

They’re very desirable to developers because

• Refreshing the page isn’t needed
• Page load speeds are faster
• Interactions are rich and fluid
• Data transfers are reduced
• Reusable pieces allow for faster development
• The developer community is well-established

They’re also the go-to resource for anyone starting a new SPA.

However, there is one major drawback: because of how these frameworks work, client-side A/B testing tools struggle to work properly.

When someone browses a web page on an SPA, the page–including the URL–isn’t refreshed. The status of the page varies with each user interaction (for example, the different elements visible on the screen).

Because A/B testing tools normally only make adjustments once during the initial page load, these subsequent interactions aren’t taken into account.

Here are two common scenarios:

1. If you have pageviews enabled on your SPA, you’ll receive a pageview when someone visits a URL, but not a second one if they browse on a different page where pageviews are enabled. This is because the material is being loaded into the current page rather than a new page.
2. If you try to limit an event to a specific URL, you’ll run into the same issue. The A/B testing tool will “believe” it stays on the same page it was first loaded on, even if the user thinks they are on a different page. Because pieces are moved in and out of the site, there may be difficulty monitoring clicks or other events.

To render experiences, client-side A/B testing software and personalization platforms rely on complete page loads.

Because this concept is absent when using SPA frameworks like React, Angular or Vue, determining whether new content has been added to the page or the status of existing content has changed – and, more significantly, when to inject tailored content – becomes more difficult.

So A/B testing tools must be able to recognize the visitor’s context, such as whether they’ve arrived at a product listing, detail, or cart page, and at what point fresh material should be supplied.

This means the tech team needs to constantly stay on top of changing modularized React SPA or Angular SPA components, as well as updating the experiences influencing those components in real-time.

Each user interaction in React.js, for example, causes one or more of the UI elements to refresh, deleting any changes made by the A/B testing solution.

Client-Side Testing vs. Server-Side Testing on SPAs

When it comes to A/B testing on an SPA, there are two options:

1. Isolating the various “states” of the application you want to test, then setting them up so that when each “state” is displayed, an API is called to do the appropriate alterations. This is called conditional activation and it is done on the client side.

Some examples of “states” include showing a subscription form, loading a list of search results, and so on.

2. Hardcoding the modifications, on the server side, or a custom deployment for each new experience.

Both of these options are influenced by the SPA framework that’s being used. So, as you may have guessed, they require a lot of cooperation between the technical team and the marketers to keep in good working. 

Now, let’s go over both of these options and see what teams must do to be successful.

Conditional Activation on the Client Side

If you still have to perform client-side testing on an SPA application, there is a workaround.

To ensure that testing scripts are only executed when the application reaches the desired state, web developers can use a feature called conditional activation mode.

If you’re unfamiliar with conditional activation, it’s a simple concept to grasp.

Conditional activation, in essence, gives you the ability to define when an experiment should be activated other than through its standard method of doing so on a page refresh.

This can be accomplished by writing a bit of JavaScript. There are two ways to do so.

1. The Polling Function

The polling function is the most basic way to use conditional activation. The function scans the page every 50ms until a specified element appears — this could be a modal, div, or even the entire page.

The experiment begins when the element appears.

The A/B testing platform generates a few lines of JavaScript for you to modify depending on which element you want the polling function to look for.

The experiment is activated once the function evaluates to TRUE.

2. The Callback Function

The callback function’s execution is similar to that of the polling function, but it adds some flexibility.

This function allows you to enter any JavaScript condition you want, and if it evaluates to TRUE, to start the experiment.

While the polling function constantly pings the page to see if a specified element is present, the callback function is more passive and relies on an event handler.

Server-Side Testing

When you perform server-side testing, no changes are made at the browser level. Rather, the experiment variation parameters (“User 1 sees Variation A”) are determined at the server level and hooked directly into the JavaScript application via a service provider making it easier to test more complicated, dynamic content that extends beyond the bounds of a static website’s user interface.

Take, for example, social media apps which are mostly SPAs. While frontend client-side testing may be used to experiment with these, it is much more complicated due to the dynamic content involved, therefore switching to server-side testing makes the process much easier.

Examples of A/B Tests on SPAs

What are some real-world instances of A/B testing on single page applications that you could come across?

Changing an Image on a Landing Page

Let’s say that you need to set an experiment on an SPA to show a dynamic image to 50% of the audience who landed on a specific page. The SPA framework used in this example is React.js which means the page is changing and images are being replaced on a regular basis.

If you display the variant image, flickering issues might occur, meaning a sequence of original image-variant image-original image will be observed.

This issue can be avoided entirely by triggering the polling when the specific landing page loads.

Testing Additional Copy below the CTA

Another experiment on an SPA site built with Angular.js may involve showing an additional line of copy below the main CTA to 30% of the target audience.

In this example, Version A will show the additional line of copy below the main CTA, while Version B will not. We’ll send 30% of our traffic to Version A and 70% to Version B, and then compare the results to see if there is a significant difference in conversion rates.

Since the website is built with Angular.js the elements are changed on a regular basis. The callback event here can aid us in displaying changes to the variant.

Displaying a Gift Voucher

In this example, let’s assume that when a user adds a premium brand’s product to their basket, we want an exclusive gift voucher to be displayed. With the condition that when the user removes the premium product from the basket, the voucher is swiped off the page.

1. What needs to happen here is that when a user performs the required action, a manual activation event should be sent. The experiment will be activated as a result of this event.
2. An activation event should be fired when the user removes the premium product from the basket, deactivating the experiment.
3. The experiment needs to be reactivated to show the premium product if the consumer adds it again.

Showing Tagline below ‘Buy Now’ CTA

If you want to display a unique tagline beneath a ‘Buy Now’ CTA button, you can activate the experiment if you push an activation event on the history change.

This is because the page containing the ‘Buy Now’ CTA button will not load at first.

How Convert Bypassed the Problem of Testing on SPAs

Experiments on SPAs need to be handled differently than other experiments. Because the Convert script cannot read the URL that a website visitor is navigating, it cannot trigger experiments with standard methods.

So here’s how it’s done instead.

Begin by installing the Convert tracking code as described in this article.

Then, apply one of the 3 methods below:

1. Triggering Polling

As mentioned, polling is the process by which the experience conditions are tested to determine if said experience should be triggered.

This includes monitoring the visitor URL, Audience Conditions, or JavaScript conditions to run the test.

Polling is usually triggered by Convert when a new page is loaded. On SPAs, usually, no new pages are loaded on the web app.

In that case, you would need the following code to start the polling:

window._conv_q = _conv_q || [];
window._conv_q.push(["run","true"]);

You should determine what the best event on your SPA would be to trigger the above code. Sample code that should be added on Project Settings > Global Project JavaScript section.

console.log('SPA/Convert Code in Global Project Javascript executed');
if (!window.globalExecutedTs) {
window.globalExecutedTs = true;
var oldPushState = window.history.pushState;
window.history.pushState = function(data) {
try {
setTimeout(function() {
return oldPushState.apply(this, arguments);
console.log('convert activated from pushstate');
window._conv_q = _conv_q || [];
window._conv_q.push(["run", "true"]);
}, 0);
} catch (e) {
console.log(e);
}
};
window.onpopstate = function(event) {
setTimeout(function() {
console.log('convert activated from popstate');
window._conv_q = _conv_q || [];
window._conv_q.push(["run", "true"]);
}, 0);
};
}

2. Use JavaScript Conditions in the Site Area

Because the Convert script cannot read URL changes in an SPA, you should use a JavaScript condition instead of a URL match condition to trigger an experiment.

Find an in-depth explanation of how to do this in the following article.

3. Manually Activate an Experiment

You can trigger experiments manually after you determine that a certain flow has happened. Using this method, the Site Area and Audience Conditions will still be tested after triggering the polling with code. Check this article for more detailed information on manually activating an experiment.

With the 3 methods above, you should be able to trigger experiments at the right moment in an SPA app.

Does Convert Increase the Risk of Flickering on SPAs?

Short answer, no.

All SPA frameworks are compatible with Convert’s anti-flicker technology. Convert leverages the SmartInsert^TM technology behind the scenes to ensure that experiment modifications are applied or reapplied at the appropriate time on your SPA, even during dynamic page reloads.

We recommend reading our whitepaper on the flicker effect to learn more about it and why you should avoid it at all costs.

Common Problems when A/B Testing on Single Page Application Sites

As you can see, there are a few things to keep in mind when running tests on SPA sites. Once you’ve covered the framework used, the tool test method, and the trigger event, then the main issue is usually with the test element not reloading when a new page or view is displayed on the browser.

This can be caused by a few things:

1. Test Changes Appearing on the First-Page View but Not on Subsequent Ones

Sometimes the change you are aiming to display only shows on the first page you load on your browser, but not on subsequent views. This is because the part of the Convert script that evaluates the experiments and activates them does not run when accessing a new view or “page” of the site.

2. Changes Appearing on Visual Editor Do Not Appear When Previewing Them Outside It

It can sometimes happen that the changes made with the Visual Editor don’t show outside the editor preview.

Most SPA frameworks do not use the DOM API and instead use their own methods to deploy the changes on the page. This causes the DOM to get out of sync when updated by the SPA. This is common in React and Angular, causing the CSS selectors found on the Jquery code not to work.

3. Changes Keep Appearing on Subsequent ‘Views’

In this scenario, an experiment change displays in the correct (page) view. But, when visiting new (pages) views, the change does not go away. This happens because on a SPA site, the changes are not removed when reloading a new page.

The Convert script removes the code added when the experiment conditions do not match the subsequent view where the changes were deployed when the polling is called. However, this is not enough as jquery changes will not disappear even when removing the experiment code of the page.

You will need to execute code that reverts the changes brought by the experiment.

Your Turn: Avoid These Mistakes When Optimizing Your SPA

Because of advancements in how information is processed and provided, SPAs may become more commonly accepted, but it’s crucial to note that this new technology has issues when integrating with optimization and experimentation platforms, which normally work with more classic setups.

We hope this article has covered the various components of SPAs as well as some solutions for overcoming those obstacles in your quest to provide visitors with faster, more intuitive, and personalized digital experiences.

However, you will be better off working with a solution provider who can provide seamless support for SPAs, such as the ability to detect changes in UI components and page types for dynamic customization without modifying the source code. Like the one in the box below.